Category: edtech

439. AI Pre-Mortem

Posted in edtech, educational technology, higher education, learning, motivation, teaching
April 1, 2026

Many faculty, administrators, and students often become caught up in the potential benefits of new technology, but do not always consider the long-term consequences. In this episode, Rebecca Winthrop joins us to discuss a report summarizing the long-term benefits and risks associated with student use of generative AI.

Rebecca is a Senior Fellow and Director of the Center for Universal Education at the Brookings Institution, where her research focuses on education globally. Rebecca leads the Brookings Global Task Force on AI in Education and co-leads the Family Engagement in Education Network. In addition to her work with many other global education initiatives, Rebecca has served as the U.N. Secretary General’s Global Education First Initiative’s Technical Advisory Group and served as co-lead for the Learning Metrics Task Force with the UNESCO Institute of Statistics. Rebecca is also a lecturer at Georgetown University and, with Jenny Anderson, the co-author of The Disengaged Teen: Helping Kids Learn Better, Feel Better, and Live Better.

Show Notes

Transcript

John: Many faculty, administrators, and students often become caught up in the potential benefits of new technology, but do not always consider the long-term consequences. In this episode, we discuss a report summarizing the long-term benefits and risks associated with student use of generative AI.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Rebecca WInthrop. She is a Senior Fellow and Director of the Center for Universal Education at the Brookings Institution, where her research focuses on education globally. Rebecca leads the Brookings Global Task Force on AI in Education and co-leads the Family Engagement in Education Network. In addition to her work with many other global education initiatives, Rebecca has served as the U.N. Secretary General’s Global Education First Initiative’s Technical Advisory Group and served as co-lead for the Learning Metrics Task Force with the UNESCO Institute of Statistics. Rebecca is also a lecturer at Georgetown University and, with Jenny Anderson, the co-author of The Disengaged Teen: Helping Kids Learn Better, Feel Better, and Live Better, which we talked about on an earlier episode of the podcast. Welcome back, Rebecca.

Rebecca W.: Thank you. It’s great to be back.

Rebecca: Today’s teas are:… Rebecca, are you drinking tea with us today?

Rebecca W.: I, of course, am drinking tea. I believe this is my third podcast with you, so I have it in hand. It’s Bengal spice herbal tea, and it’s delicious.

Rebecca: That sounds quite nice.

John: it does. I just have an English breakfast tea today.

Rebecca: Just?

John: Yep, the standard Twinings English breakfast tea.

Rebecca W.: Simple, classic.

John: Yes.

Rebecca: You can’t go wrong.

Rebecca W.: Yes.

Rebecca: I have a Scottish afternoon tea, John.

John: One of your favorites.

Rebecca W.: What is Scottish afternoon tea?

Rebecca: It’s a little bolder than an English breakfast.

Rebecca W.: Okay? It’s like, wake you up, keep it going.

Rebecca: It is.

Rebecca W.: All right. Love it. [LAUGHTER]

Rebecca: It’s been a long week…[LAUGHTER]

John: Just today alone.

Rebecca: It’s been a lot of Fridays today.

John: So we’ve invited you here today to discuss your Brookings Report on A New Direction for Students in an AI world: Prosper, Prepare, and Protect. Can you tell us a little bit about the study and why it is described as a pre-mortem, which I thought was a wonderful term for this. [LAUGHTER]

Rebecca W.: So, we at Brookings thought that we should get ahead of generative AI in students learning and development as it is rolling out and collectively, we all know that there is a parallel movie here with social media that did roll out to students into their bedrooms and classrooms [LAUGHTER] without much attention from people who work with and care for kids, be they parents or coaches or teachers or school leaders. And this time, we really wanted to make sure that all of those communities were at the table. And so the idea was, let’s not wait for five [LAUGHTER] or maybe six, seven years until all the evidence comes back from all the randomized controlled trials, and frankly, after social norms have been ossified, to assess how generative AI may or may not impact students’ learning and development. So let’s move, quote, unquote, the autopsy forward to just see where we’re at and ask the question, “Are we heading in the right direction currently in terms of generative AI implementation?” And we looked at the risks and the potential risks from what we know about student learning and development and how AI is rolled out, and we looked at the benefits and the potential benefits, and that was what we called a pre-mortem.

Rebecca: So part of the study was a Delphi panel. For those that are not familiar with this approach, can you describe how a Delphi panel works?

Rebecca W.: You bet. So first, let me tell you what we did. It was a year of research, and we convened a very large global task force with a number of working groups. Hundreds and hundreds of people were involved. We have data from across 50 countries, and we looked at the literature that is emerging fast and furiously, including reviewing over 400 studies, and we spoke to parents, students, educators, education leaders, technologists, who were in schools on the ground about what they were experiencing. And then the last thing we did was, as you said, a Delphi panel, which is a methodology that is very good for forecasting and is useful when a field is new or undeveloped or doesn’t have clear rubrics or guidelines or principles, which is the case we are in now, and it consists of gathering a very diverse mix of experts and running them through a series of very structured questions. So those were the three main things we did to gather data to assess this question, “Are we on the right track in terms of generative AI rollout for primary and secondary students’ learning and development?”

John: And the report provides a lot of discussion of potential benefits as well as potential harms. Let’s start with the more positive thing first. In what ways might generative AI be supportive of student learning and perhaps improve some of the equity that we are not always seeing in our educational system.

Rebecca W.: So we found a lot of potential benefits that were often from very strategic intentional narrow uses of generative AI. In the report, we list our top six alongside the top six risks. And the benefits are things like helping bring kids who are not participating in the teaching and learning process into the teaching and learning process. You mentioned equity, John, so one of my most vivid examples of this is from work to support Afghan girls who, once they reach secondary school, are banned in their country from going to school. And there’s a whole initiative with Afghan diaspora educators who are using AI to create little mini lessons [LAUGHTER] that go by whatsapp on their cell phones to help girls keep going to school, basically, at home, and that is an incredible use case. Now this isn’t AI doing this alone. It is educators who are using this as a tool to help deliver this content that is also adaptive. That’s one example, a very large example that all over the world, people mentioned, especially educators, is saving teachers time. Teachers are overburdened in virtually every country [LAUGHTER] in the world, and there is a lot that generative AI can do to really remove some of the mundane tasks, or at least make them much more efficient, and teachers find that incredibly helpful. It could be admin paperwork. It could be trying to also use AI to improve their lesson planning or materials for students such as, “I have three different grade levels in my eighth grade classroom, can you give me this [LAUGHTER] for the three different reading levels my kids are out so they can all access it and succeed and grow.” So that was really useful. I will put an asterisk next to that one, because many teachers also said, “I hope my managers, be they school leaders, be they more actually, school board [LAUGHTER] leaders, don’t find out how much time I’m saving, because then they’re going to give me double the workload.” So this saving teacher time and freeing them up to do new, more relational, more interesting, more caregiving, more exciting, teaching and learning activities with their time only holds if teacher workload remains constant. There is also a wide range of other ways in which people are experimenting with generative AI, such as new forms of assessment, really being able to see where kids are stuck in their learning journey in a much more granular way, we think, has the potential [LAUGHTER] to be incredibly effective to help educators really help address where kids get stuck, things like helping neurodivergent kids, kids with learning disabilities, multilingual kids. These are the types of benefits that we found in our research.

Rebecca: Are the ways that you’re describing, the ways that AI is actually being used. Is there alignment between them?

Rebecca W.: Yes, Rebecca, you have hit exactly on the issue, which is we found that that is part of the ways in which the education ecosystem is ingesting and deploying AI, but that is not the main way in which young people are interfacing with generative AI. One of our findings was that for young people that live in communities with access to broadband and families with devices, so they may have a phone, they may have a laptop, they can get on the internet, they are interfacing with generative AI anywhere and everywhere. It’s in their Google searches, it’s on their social media platforms, it’s in their gaming platforms, and the line between technology for learning and education, technology for entertainment, and technology for communication have totally broken down. It’s all blurred. One young man told us last year, a high school student in the US, “Well, my school banned ChatGPT, but my friends, we just all go on to social media and get AI to do our homework that way. You can go on to Meta AI or Snapchat AI, and there are AI friends and companions. You can talk to them about how annoying your parents are or get relationship advice. And then you can take a picture of your homework and say, ‘Please write my essay for me.’ And then you can run it through a AI Humanizer that will insert some typos and then you turn it in.” So the thing we worry about most is young people being able to access commercial, often frontier lab AI chatbot models that are not designed for kids and not designed for learning. And I would say the same about AI friends and companions. Currently, by the way, one out of three teens in the US say they prefer talking to an AI friend or companion equally or more than talking to another human being. So this is a lot of kids, and it’s very early days in this technology.

Rebecca: That’s deeply troubling.

John: It is, [LAUGHTER] and one of the reasons why they may prefer that is AI systems tend not to be very judgmental, so if they say something that people might find offensive or troubling, it doesn’t have the same impact if they share that with an AI friend. And it is troubling because we’ve seen some really negative outcomes from some of those where kids were getting some really bad advice, and it’s led to some teens committing suicide.

Rebecca W.: Yes, and Common Sense Media has done a lot of work on this, and they say nobody under the age of 18 should be accessing AI companions just because of that, John. They are designed to be super, super friendly and to keep you talking to them. And everybody, by now, has heard the word sycophantic. [LAUGHTER] So if you’re a teen and you’re out there chatting with your AI friend and your mom makes you do the dishes, and you say, “Oh my gosh, my mom’s so annoying. She’s making me do the dishes, I just find it so unfair.” Your AI friend or companion is likely to say, “I am so sorry. You’re so misunderstood. That’s really terrible, and I’m here for you. Let’s talk about it.” Now, if you were to say the same thing to an actual human peer, they would say, “Dude, my mom makes me do the dishes [LAUGHTER] all the time. What is your problem?” And that right there is the core of the problem. If you repeat that type of social interaction over and over and over and over, that is how young people will develop their social interaction skills, their expectations for how others will treat them, and, frankly, their emotional resilience. Now learning, at the core of it, is about making mistakes and trying new things and taking risks. And if you are a young person who is developing socially and emotionally and your emotional muscles are to be agreed with all the time, we do really worry about what does that do for their ability to take feedback in the classroom, for example, [LAUGHTER] when they need to learn and they need feedback to grow.

John: And, as you note in the report, you’re also learning some social skills by interacting with other real people in the classroom, which is an important part of education, which gets lost when that’s being outsourced to interactions with AIs.

Rebecca W.: Yes, one of the things I keep repeating when talking about this report to people outside of education, in particular, is reminding them that young people are not little mini cognitive processing machines, and that classrooms do not only transmit knowledge from adults to youth. That is an important function, but by no means the only function. A lot of it is, how do you do perspective taking? How do you communicate with other people? How do you read a room? How do you self regulate if you’re used to getting your way all the time? Many things are happening that are really important inside classrooms. We only have to look back a couple of years to COVID when parents were tearing their hair out [LAUGHTER] and saying, please open schools back up and let me send my kids to school, to remind ourselves how important schooling is in kids lives for many reasons. Most kids were getting content at home through online learning, and that was not successful for most kids, again, because they aren’t little mini cognitive processing machines.

Rebecca: So we’ve talked a little bit about some of the harms, including the chat companions and not doing your own homework. [LAUGHTER] Are there any other harms that you’d like to highlight that come out in the report?

Rebecca W.: So we did find that, currently, the risks of generative AI, in terms of how it’s being rolled out, are overshadowing the benefits of generative AI for student learning and development, because the risks are of a very different nature than the benefits. They are undermining young people’s ability to think independently, to relate to other people, to take feedback, as we just said, and to have a trusting relationship with their teacher, which are all foundational to even harnessing and accessing the benefits that generative AI poses. So the risks are things like impairing cognitive development when young people over rely on generative AI, not to assist, but to actually do their thinking for them, they are not learning. We all know that children’s brains develop the way they’re used, and if kids are doing their work through a chatbot, their brains are not doing the work, and it’s like going to a gym. If they go to a gym and bring a machine that lifts the dumbbells for them, and they leave, they’re not getting stronger. It’s the same thing. So it’s not just that they’re skipping steps on their homework and being more efficient. It’s that they’re entirely missing out on developing independent, critical thinking skills. We also, as we discussed already with AI companions, are worried about the social and emotional development. We’re additionally quite worried about the decline in trusting relationships that we heard from many people we interviewed, and it’s across the entire teaching and learning [LAUGHTER] instructional core ecosystem. So educators saying “I’m not entirely sure if I trust that my kids are doing their own work.” 50% of educators say they don’t trust their kids work in the US, and that is very hard. Educators need to know if it’s kids work, because that’s how they know how to teach next and what to do if they don’t know where kids are struggling, they’re flying blind. But students also talk about generative AI undermining their trust of teachers. They don’t use that word, but they say things like, “my teacher doesn’t care about me because they’re using generative AI to give me feedback, and I don’t think my teachers are really doing their job because they’re using generative AI to make a lesson plan.” So even if teachers are using AI in a way that they believe could help kids, it’s not a bad thing to give students opportunity, for example, to get feedback on your written work before you turn it in, because maybe that’s an opportunity to improve. But if you introduce that in an environment where there’s minimal trust, the interpretation on the part of the student is that it’s lack of care and professionalism, and that degrades trust even further. We also heard from educators that parents were doing weird stuff too, like taking their students graded assignments that came home, running them through [LAUGHTER] ChatGPT and getting a different grade and confronting the teacher. There’s also educators saying students are challenging me on my expertise because they came up with an alternative answer in chat GPT and the authority by which they’re imbuing ChatGPT as a veritable source, they seem to equate it sort of equal with the teacher’s perspective. So it is troubling, … that part is very, very troubling. There is, of course, other concerns, which we can go into if you want, but we touched on them very briefly, around children’s safety, whether it’s data privacy, whether it’s emotional manipulation towards self harm, which is not everybody, of course, but there’s been enough cases to show that it’s not just a one off and quite dangerous. Again, those are often commercial, general purpose products, not designed for kids, not designed for learning.

John: And one of the problems that kids are faced with is, in their social media feeds, they’re getting ads nearly constantly about all the things they can do to have AI assist them with their schoolwork, or do the school work for them in ways that are not detectable. And one of the concerns that I’ve heard many students express is that they’re reluctant to not use AI if they think all their classmates are. So there is some adverse peer pressure in the current environment too.

Rebecca W.: Yes, this is a huge issue. I am spending a lot of time now out in communities, disseminating the findings from the AI task force report, talking at conferences, meeting state level leaders and talking at school districts in the US, but also schools. And a lot of time, I talk with students, alongside teachers, and students are really worried, young students, you know, I’ve talked to fourth graders, fifth graders all the way through 12th grade. They have a lot of opinions and perspectives. One of the things I have always been surprised at is how much they’re tracking what this technology is doing. And sure enough, in a classroom of 9 year olds or 10 year olds, multiple kids bring up this question of, “What am I going to do for a job. I’m in school to learn. I’m supposed to learn. But what if robots do everything in the world? What is my role? And how am I going to get money?” And then they bring up things like… this is often more middle school kids like, “We’re supposed to be going to school here to learn and get better. And I don’t understand the point anymore. What does it mean to go to school if a robot chatbot can do everything that I’m being asked to do, what does that mean for me? What am I supposed to do? Now this is very demotivating.” They’re very worried about getting dumber [LAUGHTER]and worried about society getting dumber. [LAUGHTER] One young fifth grader last week, who I was talking to at school assembly with students, was saying, “What if we all get dumber and then I have to go to the doctor’s office or go to the hospital and I get sick, and there’s a doctor who isn’t quite as smart as he or she should be, and is relying on AI, and they accidentally take my heart out…” [LAUGHTER] like only a young child could do. That probably is not going to happen, but the general trajectory [LAUGHTER] is one that we all are worried about. So, yeah, it’s a real concern, and I think the social narrative around AI is problematic. I speak with parents a lot, and one parent told me… I was out in California, I was in Silicon Valley… and she said, “How do I keep motivating my child? She’s in middle school. I drive her to school every day, and we drive past a very large billboard that says, ‘Don’t hire humans, hire our AI agents.’” And I said, “Yes, that’s not a helpful social narrative. I wish that the technology companies would abandon that, first of all as a actual goal, [LAUGHTER]… I think they should abandon that… and as a social narrative, because the truth is, of course, we are here. We live here. We will have many, many things that we can do that are productive. 95% of what makes me me, and what makes John you, you and Rebecca you, you is not documented online, [LAUGHTER] right? Only a sliver of our human experience is really documented online. And there’s great things that AI can do. I wish our social narrative would be about AI directed towards scientific breakthrough, solving climate change, solving the question of food insecurity, figuring out how to cure diseases like that. That’s the promise of AI, not let’s just make humans irrelevant. That’s such a ridiculous social narrative.

Rebecca: And although that’s disturbing, at the same time, I find it good that our young people are concerned about these things. [LAUGHTER] That shows some faith in the future, that we have young people who are concerned about where this might take us if we don’t intervene.

Rebecca W.: They’re very, very aware, and they want to talk about it. And I always urge schools and ed leaders to start thinking about not just AI literacy, but questions of, how does the online world work? We talk about, in our report, it’s holistic AI literacy. People throw around the term AI literacy and mean very different things. We don’t mean prompt engineering and we don’t mean necessarily more time on screens. What we mean is learning about how the online world works. And let’s start by having conversations in person around what is exciting? What are you worried about? And then working through things. There’s all sorts of moral dimensions. Can I break up with my girlfriend by using AI to write a breakup note? Can I write a happy birthday to my parent using AI? [LAUGHTER] Kids want to talk about this stuff, and it really is very interesting if you sort of take a step back and think about what it means for human relationships, thinking, experience, what makes us really human? So those are all important questions that we can start with before you ever introduce a chatbot.

John: Right now, the path of AI use is being determined by the big AI companies with very little oversight or very little direction, but in your report, you do provide recommendations on three pillars that may help redirect this in a more positive way. And the three pillars that you use are prosper, prepare, and protect. Can you give us a few examples of strategies in each of these categories that might be used to shift things so that that balance of benefits and risks are altered in a more positive direction.

Rebecca W.: Yes. Our goal was to say, “Are we on the right track?” And we discovered, “no, we’re not.” And then our goal was, “How do we bend the arc to make sure AI can enrich learning and harness it for benefits and mitigate the risks?” And there were three big pillars, as you said, John. One, under prosper is, “What can we do as educationalists in the system today to shift how our teaching and learning is done, so that AI cannot hack the student experience?” Underneath that, we talk about having co- design hubs with educators. This would be developers partnering with educators and education organizations and education unions who are going to lead the way on what are the use cases that me as a teacher in the classroom, or as a librarian or whatever your role is in a school and in the education system, could use generative AI in a positive way? We talk about making sure students are at the table. I really do think every district in the US, or every school, if districts don’t do it, should have a student AI Council. We had a student author as part of our writing crew, and she very much advocated for this, again, because kids are living it, using it, experiencing it, right alongside educators. And she advocated for young people, saying, “Look, we can review procurement contracts. This is our data, by the way. Hey, we can red team and beta test the AI products before they’re procured by our district. We can help educators say how we might get around an assignment, maybe shift it.” So ultimately, I think what we need as educators to have a very clear mix of pedagogical strategies that are AI aware. And what I mean by that is, if you can shortcut an assignment with an AI chatbot, don’t assign it. Do something else. This can be incredibly difficult and [LAUGHTER] time consuming for educators. It’s really hard at the higher education level if you have large classes, but in the K-12 space, it is doable, for sure. And AI assisted… Where can AI help? Is it neuro divergence? Is it assessment? Is it leveling reading materials. And then AI resistant… when should AI not have a role in educating young people, or rather, when and what types of experiences still need to be human to human? So that’s in the prosper pillar.

Rebecca: So we talked a little bit about how widespread these tools are, and how easy it is for students to get their hands on these tools, commercially or otherwise, it’s kind of hard to just like, turn it off. It’s everywhere. But many elementary and secondary school systems are reintroducing mobile device bans during class hours. Is this something that might be helpful? But then also, I mean, we know that there’s AI glasses and things like this too that are in the mix.

Rebecca W.: So, in our report, we definitely support the large social movement started, frankly, by parents towards digital health [LAUGHTER] and good digital hygiene practices, which includes less screen time for kids. Most kids in the United States spend hours and hours. I think the average is five or seven hours online each day. It might be much higher than that I’m forgetting, but that’s important. We support bell-to-bell cell phone bans in schools if there’s a pedagogical exception, especially for schools that might be rural and not have good science equipment, and they use their cell phone as a gyroscope and stick it in a plastic bag and do an experiment, spin it around, great, but it’s teacher controlled. And I do think we currently know that kids are using screens to access generative AI, but exactly to your point, Rebecca, this will change soon, and so a movement that is about screen free childhood will soon need to pivot to be a pro presence, pro human, pro interaction childhood, because generative AI will be embedded in our clothes and our glasses and our, I don’t know, our writing implements. Who knows? And maybe that’s better and can be used in very specific ways to help augment learning, but it also presents a whole new landscape of difficulty [LAUGHTER] in thinking about how you control that.

John: I am concerned about some of the new technology coming out. Meta glasses, in particular, are a little bit troubling, because right now they’re not being used that often by students, but I’m seeing more and more student posts on various forums asking, “Well, how can I use this?” And one of the challenges is, right now, they’re voice controlled, but they could always put things up on a screen that has answers to test questions or something that may not be easy for instructors to see, even if they are taking tests on paper, for example, and the fact that AI is multimodal makes it a bit more challenging. One of the issues I’ve run into is I’ve been using polling software for many years in my classes, and in the past, students would generally think about it and then pick their own response, but because they’re coming in on their laptops or on their smartphones, they can just take a screenshot of it and just have it automatically answered. Or there are so many browser plugins that if they’re using a laptop, it can automatically provide an answer to them with questions. And that type of instruction used to be really helpful in giving students and me feedback on how students are learning and their progress, and it’s become almost meaningless today because of multimodal AI.

Rebecca W.:yeah, it’s a big challenge, I think, for everyone, but particularly at the higher education level. And I imagine we will eventually evolve for a little period. There’ll be a brief period where we reach some equilibrium, and there’ll be other ways to gather that sort of feedback that you’re used to from the clickers that is no longer relevant, or maybe if classes are small enough, we’ll do just a lot more Socratic dialogs and oral presentations. I know the class I co-teach at Georgetown, we just moved from a final paper to a final oral exam, one hour with each student to present to us their final findings in a PowerPoint that we say you can use AI, because we won’t be able to really control that, but we’re going to grill them [LAUGHTER] and whether they know their stuff, and we can do that because it’s a seminar. Now, if you have 100 students or 60 students, it’s a much bigger challenge, and to me, the alternative seems to be going back to blue books, [LAUGHTER] and nobody is used to writing by hand anymore.

Rebecca: John, you have some experience with this, I believe. [LAUGHTER]

John: In my econometrics class this semester, I had moved away from paper exams during the pandemic, and I really appreciated the fact that having exams that were online or assignments that were online, and projects and so forth that were online gave students much more flexibility in terms of time, for the students who needed a little bit more effort to catch up. They could do that,… for students who process things a little more slowly. But I was finding that that wasn’t happening in the same way anymore, because the responses were all kind of looking like they were AI generated, and the projects that they were creating look very much like AI-generated projects.

Rebecca W.: How can you tell, John? I’m sorry to interrupt you in the middle of your story, how could you tell they looked like AI generated? I’m always curious what people see.

John: The main thing was that students were doing so much better on their written project than when I asked them questions in class. They sounded very professional quite often, in ways that the students generally didn’t in other forms of writing. And they were very polished, and it just showed a dramatic leap in their performance, and I talked to them a bit about it, and they said, a lot of us use AI because we’re concerned that everyone else may be. And I talked to them about all this on the first day of the class, and put the syllabus together, and I was thinking about moving back to blue book exams. An- I talked to them about it, and they liked the idea. And the blue book exams in particular are two stage ones. So they take it first individually, and then I grade them, but then the next class day, they take an equivalent exam in small groups where they’re explaining things to each other, and it’s a very positive experience. I had done this many times before, prior to COVID, before I moved to entirely project-based assessment, the work was dramatically weaker than it had ever been before. And so I talked to them a little bit about that, and they said, we’ve been relying on AI when we got stuck, because it’s just so much easier, and it’s a bit of a challenge.

Rebecca W.: It is, and that’s one of the things that we found also in our risks, such high levels of dependence. Young people are saying, I can’t start anything anymore without AI there to help get me going the minute you sort of are stuck, rather than pausing, thinking, doing the deep reflection, which is needed to deeply engage and embed what you’re learning, [LAUGHTER] You get a quick “Oh, wait, I think, let me just ask AI and it’ll help me get on the right track.” And it feels like it’s okay, because maybe you’re asking a friend, but really it’s stopping young people from doing deep learning.

John: The desirable difficulties that Bjork talked about, they’re bypassing those, and that’s where we get long-term knowledge retention and transferability. And it’s not happening in the same way that it used to.

Rebecca: It seems like we’ve snuck into that second pillar, “prepare,” a little bit here. It’s a nice transition. Just telling students to stop using AI or don’t use it seems like a very different circumstance that won’t work, compared to the conversations like you’ve been having with your students, where they’ve had the opportunity to try to do something that’s not using AI, and they’ve realized that they’re not able to do it, and realizing how much the AI has prevented them from learning the materials. So how do we convince faculty that we need AI literacy? How do we help students recognize that using AI in certain ways is not actually helpful to them. It might be helpful in the moment, but not long term.

Rebecca W.: Yeah. Well, again, as you said, if the first pillar is, let’s shift teaching and learning, that’s something we can control. The second pillar is, let’s make everyone in our education institutions prepared to do teaching and learning in an AI world, which includes, for young people, but also educators, “How does the online world work?” In a K-12 setting, we teach young people about biology so they learn about how the living world works, but we don’t teach every kid about how the online world works, even though they’re spending huge amounts of time there, not because we want them necessarily to all go be a computer scientist. We don’t want every kid to go be a biologist either. But when they know it’s zeros and ones and lives off of our ideas and our data, how it does pattern recognition, what it’s really good at, what it’s not good at, that is very empowering. Knowledge is power, and we know that the more people know about AI actually, the less they use it. So I think we need holistic AI literacy that’s about how the online world works. I think that simple bans, although I empathize with the educators who do them because they are trying to help their kiddos learn. Kids always get around them, I just don’t think they’re feasible. And then, on the flip side, open ended, you can use it no matter what type of a thing is definitely risky, because a lot of the tools are not safe for kids, as I’ve said, and not made for learning. So we really do need to have assignments [LAUGHTER] in a school setting that kids can succeed on and that force them to get knowledge into themselves. You need to get knowledge into yourself [LAUGHTER] in order to be able to use it, to be creative, to apply it. And I do think that everybody needs really some holistic AI literacy. It’s students, it’s families, educators, school leaders, we all sort of need to get ourselves up to speed. And I would say that to the skeptical leader, skeptical educator, that there’s a real urgency to this, because, just like John said, we will miss out an entire generation who will rely on AI too much, even though they might not think they are and actually not be developing their cognitive abilities that they need to thrive.

John: So the last pillar is “Protect,” which involves trying to regulate AI platforms so that they are used more effectively. And I know that is being done to some extent, at least in Europe, we haven’t seen a lot of evidence of that here. In fact, we just saw in late February, the federal government has declared one platform a threat to national security because the federal government wants to allow it to do illegal things without any constraints. In the current environment, at least, is it very likely that we’ll see much regulation of AI platforms at the federal level?

Rebecca W.: It’s very unlikely that we’re going to get significant federal level AI regulation. There does not seem to be hardly any appetite for regulating large tech companies at the federal level from the administration, however, I do think we will probably get regulation at the federal level around children’s safety. This is, to me, the one bipartisan issue that I see, whether you’re a Republican or a Democrat, we all have kids, and we all want our kids to be safe and generative AI is worrisome, AI companions are very worrisome. They should not be available and used by kids, and so there’s a lot of bills, actually, bipartisan bills at the federal level around regulating generative AI for kids, particularly AI companions. But there’s also a great deal of legislative action at the state level because of the gap in federal legislation, states are jumping in and trying to regulate, and that’s annoying for technology companies, because then they have this really complicated landscape of regulation that they’ve got to try to navigate and design products around. It would actually be much better and more efficient if they got together and supported sort of national safeguards and guardrails and appropriately [LAUGHTER] effective regulation at the national level, but they haven’t yet done that. So states are jumping in. So I actually think this is a parent-, student-, teacher-, driven movement, and state houses across the country are very, very concerned about it.

Rebecca: So I’m a parent, I’m an educator, I’m a member of the public. [LAUGHTER] What do I need to do to help change the trajectory of AI in a more positive direction?

Rebecca W.: Yeah, I think you can think of yourself as in concentric circles. As a parent, talk to your kids about it. We are developing a list of tip sheets for families and parents that talk about what families and parents need to know, and also key things to talk with your kids about around generative AI that is derived from our report, little, short, bite size, one pagers and on social media, because we found that this is a huge gap. People are looking at educators and students and leaders and jurisdictions and parents and families are left out of this conversation. So those are available at brookings.edu anybody can download them for free, and we’re beginning and there will be more posted throughout the next couple months. So that’s one. Two, as an educator, obviously you can lean into having conversations about generative AI with your students and really open up and see where they’re at. And if you don’t have an institution that’s leading on it, you could form your own little AI council with your students in your classroom for your class, like, hey, how should I handle this? This is my struggle. You guys want to learn. You’re feeling pressured by students like, nobody’s happy, nobody’s happy about this situation. It’s not like kids are like, this is the best thing in the world, and thank God, especially at the higher ed level, they’re there to learn, and they’re like, oh my gosh, but everyone’s using it, and then I’m going to get worse grades, and I want to go to grad school, so I need to use it too, or I want to get a good job. So I would have conversations at an educator level, and then as a citizen, there’s lots of groups to get engaged, and I would reach out to your local representatives, of course, especially at the state level, possibly the federal level, there’s groups like the EDSAFE alliance that have lots of briefings and opportunities to move these issues forward that you can get engaged at.

John: And we’ve also seen some large educational groups starting to push back. One area in particular is over the use of agentic AI, which can log into a course, complete all the assignments, and then log out all without any human intervention other than giving it the URL, your user ID, and password. And I know, the Modern Language Association and quite a few other groups have been pushing back on the AI companies to limit the use of agentic AI in learning management platforms.

Rebecca W.: I think this is just going to be the story for the next couple years, because the technology is moving so quickly. Again, young people and students are really creative, [LAUGHTER] and I’ve never seen a tech system that fully keeps them out. And so I’m sure there will be, and I’m sure it already is happening, technology companies who are working to make sure agents aren’t logging in for students, there’s real vested interest, especially in anybody doing online distance learning,[LAUGHTER] virtual learning, delivering certified learning courses that it isn’t they’re certifying the work of an AI agent. So I’m sure there’s work arounds. I’m sure there will be long cat and mouse games played for several years to come. And the question is, how much will that undercut our students ability to learn? It won’t undercut the most motivated and engaged kids, highly, highly, highly motivated and engaged kids, which is, as I know from my work with my colleague Jenny on our book, The Disengaged Teen, is a sliver of the K-12 student population. Those kids are going to rock. They are going to use AI to go farther and faster and grow. The issue is all the kids who are in passenger mode or busy or maybe managing work and trying to complete a degree. That’s the issue. How do we make sure those kids have a learning experience where they actually are doing the work they need to grow?

Rebecca: Yeah, I think the fact that many of us are overloaded, even as young kids these days, certainly motivates the desire to reduce workload in whatever way possible, or to remove the tugs on time. I wish there was a solution for that.

Rebecca W.: Seriously, I do think this whole thing of “You’re gonna get to a three-day work week and you’re gonna have so much free time.” It’s never happened before. We’ve had technology evolving. What I’m seeing in my work, at least, is I’m expected to do a lot more in a shorter period of time, and I think that’s what’s going to happen. So we have agency. We all are part of a system. So I do think we can exert agency over our own little ecosystem, and I have a lot of colleagues who I work with on my team who I can try to model, and if I’m not a good model, at least I can manage and tell them, “You should go home. This is too much. You can’t do it all. That’s okay. We’ll tackle it next week.” [LAUGHTER]

John: And that’s one of the possible outcomes of increasing use of AI, that we all become more productive and we decide we don’t need to spend as much time working as a society. Another possible outcome is it could end up lowering the return to education, resulting in increasing income inequality and perhaps more unemployment, with more and more of society’s benefits going to a small number of owners of these tech firms.

Rebecca W.: Yeah, and/or introducing… we talk about in the report, maybe there’s a new dimension of inequality around cognitive stratification, where the highly motivated, highly engaged, and the people who can afford the most expensive and therefore the most accurate and powerful generative AI models combined will be super humans, [LAUGHTER]and the rest of us who are maybe less motivated, less engaged and might not have access to even the top models will just not be able to keep up.

John: It reminds me a little bit of H.G. Wells’ The Time Machine where there was an elite that lived on the surface, but all the work was being done by the masses underneath the ground. And that’s another possible outcome of all this, which I’m hoping we’ll be able to avoid.

Rebecca W.: Yes, let’s all work towards avoiding that. [LAUGHTER]

John: and doing these types of pre-mortems before things become ingrained could really offer some benefits, because, often, as you said, these things become ossified, that we get locked into these directions, and it’s hard to change things once they become the norms.

Rebecca: Yeah, I’d like to keep dystopias to my fiction reading [LAUGHTER] as opposed to my lived experience.

Rebecca W.: That’s right, me too. That’s the goal.

Rebecca: So we always wrap up by asking, what’s next?

Rebecca W.: What’s next for my work is, with my colleagues, we are out talking to people, sharing our findings that came out. It’s about a month and a half ago now, and we are thinking about what is our next phase of research. I’m particularly interested in thinking about how generative AI can enrich learning, but not just learning outcomes, but how could it be used in a way, or not used, to make sure we have trusting, supportive, agentic [LAUGHTER] learning experiences for kids,

Rebecca: We always look forward to seeing what is next for you. Each of your reports is really helpful, and we enjoy talking to you. Thank you.

Rebecca W.: likewise. Thanks for having me on.

John: And thank you for all the work you’re doing in these areas that are so desperately needed.

Rebecca W.: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

435. Insights From The Field

Posted in access, edtech, educational technology, engagement, equity, higher education, inclusion, learning, online learning, pedagogy, personalized learning, social presence, teaching
March 4, 2026

This podcast is a recording of a panel session featuring three online learning leaders, that was presented at the SUNY Online Learning Summit on Wednesday, February 25, 2026. The panelists are Yakut Gazi, Marina Amini, and Van Davis. Yakut is the Vice Provost for Learning Innovation and Digital Education at Duke University. Marina is the Executive Director of the California Virtual Campus, and Van is the Executive Director of WCET and the Vice President of Digital Learning at WICHE.

Show Notes

Transcript

John: Today’s Tea for Teaching podcast is a recording of a panel session, featuring three online learning leaders, that was presented at the SUNY Online Learning Summit on Wednesday, February 25, 2026.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Welcome to: “Insights from the Field,” a special Tea for Teaching podcast on policy, data, and inclusive collaboration. I’m John Kane, an economist and the director of the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: I’m Rebecca Mushtare, a designer and the Deputy Dean of Graduate Studies at SUNY Oswego. We’re the hosts of the Tea for Teaching podcast and the moderators for this session. Our guests today are Yakut Gazi, Marina Amini, and Van Davis. Yakut is the Vice Provost for Learning Innovation and Digital Education at Duke University. Marina is the Executive Director of the California Virtual Campus, and Van is the Executive Director of WCET and the Vice President of Digital Learning at WICHE. Welcome Yakut, Marina, and Van.

John: Today’s teas are: …and we’ll start with Yakut.

Yakut: So I’m Turkish originally, and Turkish tea is the big thing. We always have Turkish tea brewing in our houses. So that’s my tea.

John: Very good. And Marina?

Marina: And I’m from Afghanistan originally, so I’m drinking green tea with cardamom.

John: Excellent. And Van?

Van: Well, I’ve already switched over to water today, but my favorite tea is lapsang souchong. I love a nice smoked tea.

John: I do too. I have to keep it separate from my others so they don’t all become smoked flavored tea, but it’s a very nice flavor. And Rebecca?

Rebecca: I have a Pekoe tea, Edinburgh’s Gin Tea and Roses today.

John: And in honor of this recording time in January in upstate New York, I am drinking a spring cherry green tea, [LAUGHTER]…looking forward to somewhat warmer and less snowy weather. So we’ve invited you here today to discuss the future of online education. We’ve seen rapid proliferation and evolution in generative AI platforms since they were publicly introduced just a little over three years ago, leading to concerns about the integrity of online assessments, especially recently given the development and use of AI agents. How should online learning leaders address the risks that this poses to the perceived value of online courses and degrees?

Rebecca: Yakut?

Yakut: It is true that advances in AI created risk for the way traditional education is delivered, and also assessed. The concern about AI undermining validity isn’t unique to online education, though, it affects all of higher education. If you remember, COVID has created a pedagogical emergency. Generative AI created a new one, but I’m not sure if you’re treating it the way it needs to be treated like we treated in the COVID period as an emergency. Online learners, I think, may be actually better positioned to respond, because we have already been engaging in academic innovation and advocating for active learning and authentic assessment for many years. So this may be an opportunity to implement better approaches in a full-scale manner, where the focus becomes the process of learning rather than the product of learning, because that now can be created through the use of AI, and it’s actually being done in all of our classrooms, whether or not we like it. So we, as the learning leaders, need to push the conversation from preventing AI use to designing learning experiences where the use of AI is transparent, ethical, and skill building. So this means creating policies that allow for AI use, like when and how, rather than ban AI use, and also focusing on authentic assessments, project-based work, reflective practice, collaboration, things that AI can support but cannot replace. But I think the challenge is not necessarily modality, like online as a modality is a challenge. We know how to do these things. The challenge is doing these pedagogically sound things at scale. In my mind, this is a moment that poses a risk for online education at scale, more than online education broadly. But then for us to implement this approach where focus is not on the outcomes but the process, we have to work with employers, we have to be able to articulate our competencies to employers that AI is not a threat to credibility, it’s the part of our formula in higher education and how we how we train people. For example, imagine every course in our institutions have an opportunity for hands-on authentic learning, where at the end there is a skill that’s visible to future employers. There are 8.2 million learners in higher education who wants an internship, but only 3.6 million of those get one. There’s a huge gap already, so hands-on internship opportunities during our course of higher education should not be for those who want it, or it should not be postponed to last year of education. It should be an integrated part of the equation. These are, to me, pedagogically possible, but the challenge is to be able to figure out the logistics and systems to do this at scale.

Marina: So we are talking a little bit about the word academic integrity. You know that really resonated from your description, John, and I’m from the California Virtual Campus, which represents the California Community Colleges. We’re 116 colleges serving about 2.2 million students, and this is an issue that we’ve been grappling with around academic integrity, especially with agentic AI. And just to kind of put it in perspective of what do we mean by agentic AI? Because I think it could have a couple of different definitions. When I’m talking about agentic AI, I’m talking about these autonomous agents that can go and log on into an LMS, a learning management system on behalf of a student, scan for assessments or assignments or discussion posts to complete them and submit them on behalf of the student within seconds. And we already have analytics in our learning management system that indicate that a student can go in, and has gone in, because we have the data and completed a 50-item quiz, short answer quiz, not multiple choice, and submitted it within about four seconds, so we know that the student is not searching for these as prompts on like ChatGPT, because that would actually involve some interactivity with the student. They’re actually having an agent go in and submit it on behalf of themselves. That’s a very big concern that we take very, very seriously. Obviously, the consequence is that our degrees and our programs and our transcripts will lose its value and that they will become meaningless. And so I think it’s similar in the case of being like other crises, like COVID 19, for example, but it’s also uniquely alarming in kind of the destruction that it can have on the value of our grades and our courses and our programs. And the question of, “How do we address the concern?” I think we do handle it in the way that we handle these other risks. We have to be deeply engaging with our faculty. We have to pull together our technology partners. And I think a mistake or a pitfall of this can be if technology says, “Well, the solution lies only in faculty changing the way they assess, adopting project-based learning or authentic assessments.” And if faculty turn around and say, “Well, the problem can only be solved technically if we create some kind of barrier or blocker for these agentic browsers.” And I think those are both equally flawed in terms of its approach, because we’re expecting the answer or the solution to be around the silver bullet, when in actuality, I think the solution lies in a deep partnership where we sit down with each other, with our partners, we look at both programmatic and technical solutions to making sure that the academic integrity of our institutions remains intact.

Rebecca: AI enables more efficient adaptive learning and provides increasingly personalized pathways in support of student success. How do we strike a balance between personalized learning and the formation of human connections essential to creating a sense of belonging? You want to start, Van?

Van: Sure. So this year marks my 30th anniversary of being in higher education, and I started my career as a faculty member, I spent a decade in the classroom as a history professor. And one of the things that I learned very quickly during that time is whether I wanted to or not, you end up teaching to the middle. You run the risk of losing the students that are ahead. You run the risk of losing the students that, for whatever reason, are behind. As much as I wanted to personalize learning, I couldn’t do that, at least not at scale. I think that AI does give us the opportunity eventually to be able to personalize at scale. That does not mean that the human piece loses out. In fact, that human piece, I think, becomes even more critical. We know that relationships matter. We know that, beyond a shadow of a doubt, a student can succeed when they have a relationship with somebody on that campus, ideally, their professor. I’m a big fan of Parker Palmer’s work, and specifically, I’m a really big fan of To Know As We Are Known. And if you’re familiar with it, Palmer talks about relational epistemology, which is really just a fancy way of saying that, at the end of the day, what matters is a faculty member’s ability to create a relation with a student, and for students to create relations with each other, and that is the moment at which learning takes place, not the transmission of knowledge, but real learning takes place in the midst of those relationships, and a skilled faculty member turns the classroom into an opportunity to develop relationships and for people to co-create knowledge together. I don’t think AI changes our ability to do that. I don’t think it takes away our ability to do that. I think what AI may be able to do is to take some of the burden off of faculty when it comes to trying to personalize learning, so that faculty can focus on what a human does best and what faculty do best. And that’s the creation of relationships, and that’s where I think the true learning takes place. And I have to say, when I was a faculty member, that was the piece that excited me the most, and probably the piece that, unfortunately, I got to do the least.

Yakut: If I can jump in here, plus one to everything Van said, I mean, so insightful, and I’m not going to repeat because that’s exactly where I was going. But one thing that, when I was listening to you Van, I was reminded of the statistics. Khan Academy’s, Khanmigo, last year, they published their self-efficacy study results within the school districts that they worked with. If you’re familiar, Khanmigo is basically personalized learning using AI. Within the school districts that they worked with, where the tool was available to all teachers and all students at no cost, only 9% of students, 0, it’s not 19%, it’s not 90%. 9% of students used the tool to the level that Kahn Academy deemed standard use, which is two hours a week. So where I’m going with this is provision of the technology, provision of the platform or the tool, does not necessarily lead to the opportunity to learn. So we think these technologies are going to make these available for everybody, which they will, but then there is still a missing piece that makes the student use the technology to learn, and that’s, I think, the inspiration to learn, and that’s the human thing. So as Van mentioned, it takes a faculty member or a parent or an uncle somewhere to inspire people to do something. So if we relieve people of the busy work that takes place to prepare for teaching, or actual teaching in the classroom, we may give them time to actually do this human element of inspiring to learn. And I think the two are hand in hand.

Rebecca: So Marina was kind of hinting at some of the data that’s available in our LMSs. And one thing that online learning leaders increasingly rely on is this use of data analytics, based on institutional and LMS data, which provides more information on student learning practice. AI tools allow for more efficient analysis of this data. So how can this data be leveraged to improve the online learning ecosystem? This could be faculty workload, course offerings, student engagement, and anything else that you can imagine.

Yakut: There are a few things I can mention. First, predictive analytics for early intervention, identifying student interest much earlier, not just those who are failing, but those disengaging. And there was an example about this earlier, struggling with specific concepts and provide targeted support before they fall behind. We have been pursuing these kinds of systems for a long time. I have 32 years in this profession now, and probably half of it was all about predictive analytics. The challenge has been the siloed nature of our data systems at our institutions that create only a partial picture of student success, unless you invest in data lakes, integrated data solutions and whatnot. And then there’s also the governance issue around data. AI can create a leapfrogging effect here. An ability to do this predictive work can also reduce faculty workload. This way, the human is in the loop and is the one doing the empathetic work and empathetic outreach, asking, rather than finding out who needs this kind of outreach, doing the work of outreach, saying, “Are you okay?” Apparently it’s one of the most powerful things you can reach out to students about, are you okay? The heavy lifting is done elsewhere, and then we become more human. Another example could be curriculum optimization. By analyzing patterns across courses, we can identify where students consistently struggle, what courses are synergetic to take together, for example, what courses are toxic to take together and then provide information to both faculty and students around those. Where are the gaps that exist in our program pathways? So if there are skills that are needed, and then that analysis of curriculum and skills matching can be done by AI, and then this can impact our course redesign and sequencing decisions. And also our institutions can allocate resources a little bit more effectively. Analytics can help us understand the actual demand patterns, optimal cohort sizes. We usually say, “Oh, 30 students,” but then we don’t really base it on I mean, it’s a habit that’s telling us about the cohort size to be. It may be classroom size. We can actually do deeper analyses, especially when we’re facing enrollment pressures and financial pressures. But I have to also emphasize two things here. One, this data should inform our decisions, so it’s data-informed decisions, not even data driven, because sometimes we may have to make decisions that go against data, it can be political, it can be governance reasons. We need governance structures that will take this data and generate insights from it, and then transparent communications with students about how their data is used. And this is crucially important. We need to democratize this data and allow students to have access to it, along with institutional access or faculty access. So transparency, ethical use, shared governance around analytics are very important as we become more excited about AI’s potentials in this analytics stage.

Marina: I love all of those things around predictive analytics, early intervention, and then optimization as well. So I’ll just try to add to that conversation a little bit. I’m really interested in how we’re serving our students in online modalities, and who it is that’s succeeding in those courses, because I think we know that traditional education has served a certain group and demographic and gender over time, and only in recent years are we sort of aware of some of the biases in the design of our curriculum and our courses that serve certain people, and how can we analyze our data more efficiently to address what exactly is it that we’re doing that’s helping some students fail and helping some students succeed? And I’m saying that intentionally, because I think we are agents of that success and failure in many ways, as our institutions are structured, and so I’m very interested in sort of decolonizing online education through the use of data analytics and institutional research is notoriously slow. You have to wait a year to get your IR office to work on some data set. It’s a lot of intensive, hard work, and then a year later, you get this sort of lagging indicator of, like, well, like two years ago, this was the thing, and it’s not even relevant now, or things have changed. So I’m excited that AI and data visualization and optimization can allow us to have this in a more timely manner, like just-in-time data. It can help faculty to realize how their own strategies are impacting certain students, well or not well, and then make those changes immediately. If I have an AI coach in my class and it’s telling me that they have identified, I have six African American students, but one of my strategies is unintentionally impacting them in a negative manner, I want to know that now. I want to know what it is that I’m doing in a private message from my AI assistant, [LAUGHTER] and I want to rectify that and immediately change. So that’s the kind of agility that we currently lack in education and in research that I’m very, very excited about: bringing in AI to help with that sort of private, supportive coaching done in the privacy of my own course from a desire to want to improve and serve all of my students.

John: Now, you’ve talked a little bit about some of the benefits, or some of the cost first, and now some of the potential benefits that we have from using AI. How do you see online learning changing in response to AI? And we’ll start with Marina.

Marina: Yeah, so I just want to say it’s really hard to follow up with all the positive, I think benefits that AI can bring that Van and Yakut have brought up, and I 100% agree with those. So I don’t always want to be the downer [LAUGHTER] in this podcast, but if I had a crystal ball and kind of playing on the previous question that you asked, John, around academic integrity, assuming that that is a problem that we’re not going to be able to immediately solve, either programmatically or technically, one of the concerns that I have is that it will affect a specific kind of online learning, which would be fully asynchronous online courses. The ones where they don’t have any set live meetings, I think people will be most critical of those kind of modalities. And so if I have a crystal ball to look at, I suspect that we’ll be bringing back more hybrid forms of online education to replace more asynchronous forms, because of the concern around academic integrity. That’s my crystal ball. What does that mean? Well, it could mean that hybrid, maybe half the courses meetings will be in person, half online, or it could be that there’ll be more proctored assessment, either online proctoring solutions with some of the main vendors that you may have heard of, or in person. So we would need some kind of proctoring network, or some way to address that concern. So I think as a stop gap measure until we kind of come to terms with the academic integrity piece, I see some of that changing. That being said, I do want to sort of add to the good elements that you know Van and Yakut are bringing up, because I think AI can help humans be more human in some ways. I have not been in the classroom for a number of years, but I did teach online for a long time, and I would love to go into my learning management system, and my AI assistant pops up and says, “Hey, Johnny has not interacted with you or any other students for eight days. You might want to check on him to make sure he’s engaged.” Or my assistant tells me, “Hey, you interacted with these 10 students in the discussions this week. You might want to answer posts from these 10 on the public posts” and things like that that help me to be more human, that help me to be more equitable in terms of how I interface and touch my students online, I think that could be a really amazing use of AI, and I don’t think we’ve really realized the true potential of that.

Van: And if I could jump in, I agree with everything that Marina said. My fear, though, is exactly what her crystal ball says, and that is that asynchronous, online education may become more difficult, or will face additional scrutiny, because we already know that we’re highly scrutinized, but we also know that online, asynchronous education plays a critical role in access for a certain student population, and I would hate to see those students, particularly adult learners, or learners who are working or have caregiving responsibilities, lose those opportunities. I think that the main thing that AI is going to do for online education is, I do think it’s going to force us to rethink our pedagogical practices. And I think that we are probably going to need to think about, how do we do alternative forms of assessment, and how do we move from assessing knowledge to assessing abilities and skills. I said that I started my career as a historian. When I took my PhD history exams back in the 1990s so back in the last century. When I took those exams, I sat in my office with my laptop and had no ability to consult any sort of document. It was purely from memory. It was eight hours of answering questions from memory. The reality is, though, that whenever I got into a classroom to teach, I never created a lecture, I never created an assignment, I never created a test without pulling a textbook down or pulling some sort of reference manual down and looking at it. What I did in terms of my teaching was completely disconnected with the way that my graduate program, which came from an R1 institution, assessed me. And so I do think that one of the things I hope AI is going to push us to do is to really rethink our assessment practices so that they are more in line with the reality of our students after they leave our classrooms. And that’s not unique to online education. I think that that type of shift in pedagogical practices really needs to happen across the board if we want our students to truly be prepared for the world that they’re going into after they leave us.

Yakut: Can I quickly add two things that we’re seeing actually happening in terms of change in online, and that’s in the design process. Those of you who are designing online courses will know that one of the first friction points is when we ask faculty to write learning objectives, because they’re not used to thinking about the end in mind, right? They’re used to thinking about, “This is the content I’m going to teach.” So that friction is reduced by use of AI, because you’re easily creating a set of objectives, reiterating from that. And then the other aspect is video production process. You can now ask a faculty member to say something for three minutes, and then you can make that faculty member say anything in any language. Now it may look fake, but it’s going to get better, right? So the video production process and instructional design process are immediately impacted by the use of AI, and it’s going to impact those professions, so we need to do something about it, but I think it’s making our institutions a little bit more agile in terms of updating courses, but also creating new courses based on the input from the market. I also see in larger, better adoption of open source content, because open OERs are problematic because faculty do not have the time to go and research, curate, adopt OERs. AI is proving useful in that process too, going and researching and bringing you things to review and curate. So I think there are immediate gains there as well.

John: Moving to a somewhat different topic, federal regulations and political shifts, of which we’ve seen an awful lot in the last year or so, can create uncertainty. And we’ve seen a flurry of executive orders designed to dismantle long established higher ed priorities concerning equitable and inclusive education, while also restricting financial support. What advice do you have for leaders trying to stay agile and innovative in this evolving and somewhat seemingly random political environment while maintaining compliance with federal regulations in areas such as accessibility and SARA and any other area that may pop up?

Van: Well, if you don’t mind, I’ll start. Given what we do at WCET, if you’re familiar with WCET, we really function at the intersection of policy and practice of digital learning, and we spend a lot of time monitoring what’s going on, particularly in Washington D.C., at the federal level, at the Department of Education, and increasingly in the executive Oval Office, and translating that to the field, in terms of what that means. It’s not lost on me that before I joined the webinar this afternoon, I was watching several of my colleagues respond in real time to the negotiated rule making that’s taking place in Washington DC right now around things like accountability measures, new accountability measures around student loan limits. So it really is a dynamic environment that we’re in. I think that there are ways that AI can help us navigate this, actually. So, one of the things that I played around with, I spoke to a group of SARA administrators, that’s the State Authorization Reciprocity Agreement. I spoke to a group of SARA administrators this summer at their annual meeting, specifically about AI. And one of the things that I played around with was taking the SARA policy manual, pointing Perplexity to that manual, and then asking it specifically to create a synthesis of the regulations in that manual, and then asking it to go out and pull out the contact information for each state SPE that’s the State portal entity, that’s the primary SARA person in each state, and to provide me with a table of who those individuals were and what their contact information was. This is something that it would take somebody hours, if not days, to do, I got this back in two minutes. So I do think that there are ways that AI can be used to help us navigate the policy landscape more quickly than what we have been able to do in the past, and to democratize that so that everyone has the ability now to understand these regulations, rather than just a few people at each campus. We are going to have to remain vigilant. We are going to have to make sure that we follow what is happening, particularly in Washington D.C. I will tell you that this Department of Education is very bullish on AI. They are particularly interested in finding ways that it can be used in the educational process. Having said that, they have done very little, though, in providing regulatory guidance. So, my advice is to monitor, to lean on organizations like WCET that are monitoring this for you, to not be afraid to experiment with it. And then I would say this, regardless of the technology: start small and fail, and learn from your failures, and don’t be afraid to fail. And I think that that last piece is really critical, it’s critical with any technology, but I think it’s especially critical with AI, because the only way we’re going to be able to figure out how to best leverage this is if we experiment with it, and if we experiment with it, we have to accept the fact that we’re going to fail sometimes, and we can learn from those failures. That’s scary in higher education, we’re taught not to fail in higher education, and I think that we’re going to have to shift our mindset so that we really are embracing a growth mindset, to borrow Carol Dweck’s framework, if we’re going to be able to remain agile, but also be very innovative, and leverage AI to the ability that we are able to.

Yakut: So Van has been doing this for such a long time, I don’t know if I can add anything except for a plus one to everything that he said. Maybe one thing that I can add is agility does not mean reacting to every political signal. Staying grounded in institutional values and your personal values, and then building flexible implementation strategies. So our commitment to accessible, inclusive, high quality education, and student success doesn’t change how we articulate and implement that work may shift because of what’s happening around us, and the importance of being connected to networks like WCET, UPCEA and learning from these state compacts and peer networks is extremely invaluable. There may be institutions out there without enough resources to allow their people to be connected to those networks. So maybe this is the time to urge these professional networks to be a little bit more lenient and allow, for example, I’m just making this up, I just thought of this while I was listening to Van, what would it be like if professional organizations allowed 5% of an institution’s size to be peers like gift memberships? So the institution X, a regional university, may not have the resources to become a member, but it is so important for all of us to be successful at this point in time. Maybe, you know, my institution can gift a 5% of our size to that institution, so like sister institution kind of thing. And very important to document your decision-making processes at your institution, how you’re interpreting regulations, documented, so that you have both accessibility to that information but also accountability. It protects your institution. It demonstrates good faith effort and legal counsel and government relations are your friends in this uncertain political environment, and proactive communication with them is very important.

Rebecca: In states with multiple public colleges such as the SUNY system, each campus is, to some extent, competing with other campuses in offering online degrees. What mix of cooperation and competition is optimal in such environments?

Marina: I can talk a little bit about the California Virtual Campus because the California Community Colleges are very similar to SUNY in terms of just being a big ship to steer. A lot of you know, local decisions, some centralization, and people are always, I think, generally speaking, a little bit threatened by the centralized stuff. When the California Virtual Campus first rolled out, gosh, 10 plus years ago, and we talked about course sharing, there was immediately sort of concerns around, “Are we going to be cannibalizing our enrollment?” We love the word cannibalizing, I don’t know why in education [LAUGHTER], but immediately it’s like, “oh my god, we’re going to lose enrollment. This is going to be terrible for us. The Big Brother is going to take over our curriculum and tell us what to do.” There’s all of these anxieties that come out from administrators, from faculty, from different leaders, and so I think to put that in a different perspective, has been really hard, because you’re fighting against that cultural anxiety that we have around being an independent institution. So when do you hear and when are you independent? What I’ve tried to do with the CBC and our system is to have some very honest conversations around “what exactly is your concern? Is that concern an institution centered concern, or an enrollment concern, or a funding concern, and how does that then relate to your learners?” Really shifting the conversation from, “we’re worried about how we do things to what’s the best-case scenario for our students? Do they have the options that they need? Do they have the access that we need? Are we really leveraging our collective power in a way that benefits the learners in the most compelling and powerful way?” That’s sort of the question that I’ve had to ask, and it took about 10 years [LAUGHTER] for California Community Colleges to get on board with course sharing, and now it’s a big success. We share professional development, we’re sharing financial aid, we’re sharing transcripts, things that people have held on to so closely, of like, “No, this is our school. This is our university.” And I think this idea is very outdated now, thankfully. And the future really is less institutions, especially smaller institutions, rural, under-resourced institutions trying to be everything for their students, trying to give them every class that they need for every program, and meeting every need. That’s exhausting, and what we’re finding is courses are being discontinued, faculty are, in some cases, unfortunately, losing their work, especially part-time faculty and adjunct faculty, programs are being discontinued, and in some cases, entire institutions are being shut down and absorbed by other institutions, or just not being absorbed at all. And so to avoid this, I think, unfortunate, outdated approach, we have to pull together, we have to pool our resources, and we have to put aside our institutional egos and really realize that there’s a lot of strength in our collective power and when we come together. I can give some examples later but one thing that we’re doing in the California Community Colleges, which is really exciting, is several of our rural colleges, which, again are tiny, they don’t have a lot of staff, they don’t have a lot of programs or money. They’re pulling together to create fully online associate degrees. So by themselves, they would never be able to meet all the requirements to offer those degrees online, but collectively they are, so it’s benefiting their programs, they’re able to grow their enrollment, and it’s benefiting a large swath of their students as well. So that’s the kind of thing that I think doesn’t have to be competitive, but they can actually be mutually beneficial for students and institutions.

Van: So you know, the “C” in WCET stands for cooperative. We’re the WICHE Cooperative for Educational Technology. So we take this idea of collaboration and cooperation really seriously. And I think just plus one to everything that Marina said, it’s always hard to follow Marina whenever I’m on a panel. I think that course sharing was a great example. It’s also oftentimes a really scary example for faculty, especially. What I wish that I would hear more about is the way in which institutions begin to share administrative burden. And I think that maybe, if there’s a thin end of the wedge that can maybe help an institution eventually get to that course sharing, It’s to have that conversation about, “Are there administrative functions that can be done better collectively than individually? And what are those?” And maybe the easiest thing to do to start with is an arrangement where you are jointly purchasing services. We know that folks are able to get better deals whenever they go in with multiple institutions than whenever they go in with individual institutions. At an earlier point in my career, I was in charge of online education in the state of Texas, and the legislature asked me to determine what each institution in the state of Texas was paying for its learning management system, and I learned very quickly that A) I didn’t have access to that information because there were non-disclosure agreements, and B) there were non-disclosure agreements because what institutions were paying for the same learning management system were vastly different. And so I do think that one of the things that institutions can immediately do, that isn’t scary, is to create those purchasing co-ops and to look at what administrative functions can you share. And then, by all means, move as fast as you can towards course sharing, because I think that Marina is absolutely right when we think about what is in the best interest of our learners, course sharing is oftentimes in the best interest of our learners. It gives them access to a larger number of classes, it gives them access to a larger number of opportunities to progress in their academic career and move towards that credential that they’re seeking, and it allows institutions to specialize. As a recovering historian, as much as I would love to see every institution offer a history major, the reality is every institution probably doesn’t need to offer an online history major. So think about what is it that your institution can do that is unique to you, that you can double down on and do well, and then begin to share that with other people and with other institutions. And I think that will help us greatly in terms of being able to really move towards what our learners need, rather than just what’s going to be in benefit of us. And the last thing I would say is not only think about, are there programs you can specialize in, but think about, are there student populations that you can specialize in? You know, maybe you have an institution that is particularly good working with English language learners. Not every institution is going to have that level of expertise. And so maybe think as well about, in addition to the programs that you may have some specialization in, are there learner populations that you know that you are doing a good job of meeting their needs, and you can help other institutions do that as well.

John: Institutions vary quite a bit in terms of their size, their funding, how well established online programs are. What strategies can leaders at small or resource-constrained colleges use to develop sustainable online learning practices despite often limited staffing and infrastructure, or in environments where faculty are experiencing burnout or there’s some form of organizational resistance to change?

Yakut: Well let me jump in, and I think there are quite a few parallels between this question and the previous question that was asked. 15 years ago, I was at a regional university in Texas, and in fact, I know Van from those days. In a matter of three years, this was a new institution, we rolled out 13 online programs. It was amazing. There were not a lot of resources, brand new university, faculty resources are scarce. I was a team of one. So fast forward to today. This would not be possible. One, the market is so saturated with degrees, you don’t have the competitive advantage anymore. And two, the resources are even reduced compared to 15 years ago. So in places where resource-constrained or smaller institutions, sustainability of effort comes from focus. So, not trying to do everything like, “Oh, let’s just roll out an online program for every single program that we offer,” but intently looking at the market need and connection to jobs, starting with that community where the institution exists, but a little broader too. So those small number of programs that clearly advance this mission and student access, then invest deeply in quality faculty support, instructional design, quality is by itself, a whole entire conversation, I think. And then faculty burnout. It’s real. And many times when faculty resist online, we take it personally. We don’t like it. But then it’s just exhaustion rather than real opposition. Transparent communication, realistic timelines, recognizing that this is instructional labor, this is on top of everything else that we’re asking from you, and then supporting that instructional labor in a respectful way goes a long way in helping faculty burnout. Sometimes, the most strategic move is to slow down and build that trust and capacity within the institution, because let’s face it, in institutions like that, you’re going to rely heavily on faculty time and faculty effort to design and develop those courses, compared to larger institutions where there are large teams of instructional designers or video producers that are going to do the heavy lifting for your faculty. Again, tying to the previous conversation, shared tech services, vendor collaborations, a lot of these institutions can punch above their weight and without overextending people, so that those kinds of strategies should be part of it. And finally, I want to tie this conversation to the AI conversation. Many universities have responded by signing big enterprise deals with AI companies. And they paid a lot of money for this, and they wanted to achieve equitable access for their own communities, they wanted all their students to have access, all their faculty to have access. But this results in us treating AI as a competitive advantage, where we should start seeing it as a collective challenge. So we should be forming system-wide and cross-institutional partnerships when it comes to AI deals as well. These partnerships can be between universities, between universities and community colleges, sister campuses like sister cities, so that we raise all boats and ensure equitable access for everybody. There is room for all of us to be successful, and our actions should complement that belief. And we need to bring faculty into these decisions, because the moment you announce a deal, AI deal, that will allow everybody to have access to AI, it may be interpreted as a blanket permission for unrestricted AI use in every course, which is not the case at many of our institutions. So faculty’s presence at these tables is very important.

Marina: Yeah, I just want to say amen to Yakut’s point about faculty burnout and kind of respecting the time and the expertise that they bring to the table. Our faculty also have, at many of our institutions, expressed burnout, and so anything we do is really thought around that concern. In terms of AI, if we want to foster leadership amongst the ranks of our faculty, we actually created an AI fellowship, and we selected a couple dozen AI fellows from the system to lead some really important work. And they’re being compensated with reassigned time or stipends in some cases, to do that work, in addition to some staff members, classified staff members as well, because they often, you know, occupy the gray space that we forget about, that support faculty and instruction too. So that’s very important. I think about a parallel to online learning, and actually it’s an important parallel with open educational resources. In my previous role as a dean at one of the community colleges in our state, we wanted to grow open OER, but it was the same thing. Faculty were like, “In order to change my textbook to an OER, it’s gonna take so much work and effort, and I have to change all my tests and all my prompts and all of this.” So we started giving very small stipends. I think at the time, it was like $500 for faculty to adopt OER. And I did an analysis of this over time… I wish I had AI to do that, but I had to do this [LAUGHTER] myself… but essentially, we saved students something like five million dollars within the first couple of years that it was being adopted, and we spent, I think like 200k during that same period of time. So doing the data on the return on investment for supporting faculty, it’s going to be tremendous to kind of ensure that they’re not burnt out, that they have time, and they have the space mentally to do the work that we need them to do. So that would be one key thing. And I think OER, what we’re finding, and what we found in our institution is as well, that there are higher outcomes in terms of success rates and grades in online courses when an OER is used in that course than in the same course with a physical textbook that students had to purchase. So those are kind of connected. And then just to kind of go back to course sharing, I can always count on Van to be my greatest ally with course sharing, but we have colleges that were very reluctant to get involved in course sharing, as I mentioned. When I joined the CVC four years ago, we had something like 2,000 enrollments on course sharing and 2,000 enrollments for a system of our size is not that great. We’d joke that most of it’s our staff trying to see if it works. This year we’re going to have more than 50,000 enrollments. So within four years we went from 2,000 to 50,000. I think we’re going to double and triple that in years to come, because we finally gain momentum and trust and involvement and engagement. Of those 50,000, you look at the numbers, some rural colleges are making out like bandits. They’ve gained 1000s of enrollments through course sharing, by putting their courses on our course sharing platform, and benefiting from it tremendously, and gaining enrollments that they would never be able to get from their local zip code because of their rural environment and so forth. So, it’s a win-win proposition, it doesn’t add any extra work. We let them fill their classes locally with their local registrants, meaning they’re returning students, their athletes, their disabled students, and so forth. And then any courses that are not filled, we pull them into our platform, and they’re open for anybody in the state to take. It’s a very simple formula that, again, has really netted 1000s of enrollments for these small rural colleges, brought in funding, brought in additional resources for their burnt out faculty and their staff and their infrastructure. So I think my suggestion for strategy is find those formulas that are win-win propositions. It doesn’t need to be something where one person loses, another person wins in order to get the involvement, there are ways for everybody to win, and that’s, I think, an exciting opportunity for us.

Rebecca: We always end by asking: what’s next? Start with Van?

Van: Sure. So ironically, I just finished writing a blog post for WC Frontiers on this, this morning. I don’t have a crystal ball. I do have a magic eight ball, though, if you remember those. My predecessor gifted me one to help me navigate the challenging federal policy environment. So I have my trusty magic eight ball. I think there’s six things to look for in 2026. I think the first thing is we need to remember that the connection to workforce education is everyone’s responsibility. It’s not just a few faculty members’ responsibility. This is particularly important with the move to Workforce Pell, but it’s important as we see a workforce increasingly asking for graduates that are AI literate. And I think that AI literacies is everyone’s responsibility. Two is AI literacies. I think that we need to desperately change the narrative from just focusing on academic integrity to beginning to focus on AI literacies and you’ll notice that I use the plural there, because I don’t think it’s black and white. I don’t think it’s you know or not know. I think that it’s a spectrum of issues that faculty and students and staff need to have mastery of. Three, and this really ties to what Marina was just talking about, we need to get creative economically. Like I said, I’m a historian by training. That means that I tend to run towards the pessimistic side, because I know just how bad things can get sometimes, and I am afraid that we’re looking at a very challenging economic environment on the federal level, and that’s going to trickle down to the state level, especially for our public institutions. So I do think that we’re going to have to get creative economically. We’re going to have to do things like think about course sharing, like thinking about purchasing consortiums, changing our business model in higher education and in digital learning. I think that that means that one way that we can do that is to lean into those post-traditional learners. There are 43.1 million Americans who started college and did not get a credential. That is a travesty. We should not have 43.1 million people that stopped out. That tells me that, first of all, we’re not doing right by those folks, but it also means that there are 43.1 million people that we can get back into higher education if we try, and we won’t get all of them back. But we need to think about how do we offer programs that attract those adult learners and keep them. What did we do wrong the first time around to make them stop out? It’s not all an institution’s fault. Adult learners have a number of challenges that they’re juggling, but are there ways that we can attract them back into our classes, and that we can make sure that we provide them with the supports that they need in order to stay and be successful? I don’t think we can lose sight on accessibility in 2026. It is not just the right thing to do ethically, it’s going to be the right thing to do legally, with the Title II regulations that go into effect at the end of April. And the upshot of those five things is that digital learning is going to be more important than it has ever been before. And that means that your job as digital learning leaders, your role on your campus, your role in the SUNY system, is going to be more critical than it has ever been before. And so I think that that’s what we have to look forward to and the upshot of it is that, at the end of the day, what you do as a digital learning leader matters and is going to be more important than it has ever been before, more important than it even probably was during the COVID-19 pandemic. So that’s what the magic eight ball over in my WCET office says.

Yakut: Now, how do I follow this? I mean, you covered everything, Van. I was actually going to start with the same sentiment, online education sits at the center of all the challenges higher education is facing, and the work ahead of us is to leverage its pedagogical power, its ability to expand access to affordability, and its ability to amplify our institutions’ impact. You covered a lot of things. There is one thing that I would like to just append to what you said. I want to point out that we in higher education often forget that we are not just educators. We’re also employers. In many cases, we are one of the largest employers in our communities. And many campuses, many of our institutions still lack a coherent AI upskilling plan for our own workforce. We have tons of projects facing transfer efficiency across our institutions, productivity, but then we don’t really have, in many cases, a plan to help our own people adapt. A Brookings Report from, now, a year and a half ago, estimated 19 million US jobs that are significantly affected by AI. And many of those roles exist right on our campuses. Many of our employees hold jobs at risk of automation, such as legal analysis, paralegal, administrative assistance, accounting, communications, we just talked about instructional design and production. These job categories are, drumroll, they are populated by women. You know, we identify these areas of risk, but we don’t really come up, at least in higher education, strategies to address those. We must understand that there is a differential impact of AI on our workforce. These jobs that I listed are stable middle class professions. They are pathways for upward mobility for a lot of families. So we have to ask this year and beyond, what happens to our societies when women or any subgroup begin to disappear from the workforce as a result of AI, and what are we doing to stop that from happening? Thank you.

Marina: It’s unfair to go after Yakut and Van, but I just want to really say I appreciate that Yakut points to our own employees and our own communities, and starting at home, I think that’s something that’s an important reminder for me myself. If I could just leave with three additional things to summarize some of our conversations from today, I would say, you know, number one is a reckoning when it comes to AI in higher ed. There is a reckoning for specifically, as I mentioned, asynchronous fully online courses, and how we’re going to manage that, and the long game of changing the way we teach and the way we assess our students and the use of outcomes in our courses. So I think this is going to be the next big thing for us to tackle. The second and this is something Van has spoken to about extensively, is a move toward thoughtfully shared resources. In our system, we pay for the common LMS. We pay for and negotiate common ed tech vendor contract tracks as well, course sharing, as I mentioned. So there’s going to be more and more a move toward this, especially in light of federal funding issues that colleges are experiencing. And the final piece of it that’s actually really near and dear to me, is a revisiting of old norms and ways of doing things. The way we handle our transcripts, for example, is… I’ve spoken about this in other venues… highly inequitable. The way we handle our financial aid, that can be very hit or miss depending on what institution a student goes to. The way students have access to courses, our levels of accessibility, the way we gather analytics and research, which I think AI will dramatically transform. Institutional research is going to greatly change in the next couple of years. I think these are going to be really exciting “what’s next” as well for higher ed.

Rebecca: Well, thank you so much for joining us. We appreciated the conversation and the many things to be thinking about as we trudge down 2026.

John: Yes, thank you.

Yakut: Thank you for having us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

431. A Curated AI Framework

Posted in active learning, edtech, educational technology, evidence-based teaching, higher education, inclusion, learning, metacognition, pedagogy
February 4, 2026

Students can use AI tools to support or short-circuit their learning. In this episode, Tim Curry joins us to discuss how he uses AI tools to support students with diverse prior knowledge in a carefully curated framework.

Tim is a lecturer in the Department of Health Sciences at Northern Arizona University. He is developing AI applications for autism research and tribal health service while building Northern Arizona University’s Applied Health Analytics program. He’s also working on the completion of his PhD degree in health equity technology.

Show Notes

Transcript

John: Students can use AI tools to support or short-circuit their learning. In this episode, we examine how one instructor uses AI tools to support students with diverse prior knowledge in a carefully curated framework.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Tim Curry. Tim is a lecturer in the Department of Health Sciences at Northern Arizona University. He is developing AI applications for autism research and tribal health service while building Northern Arizona University’s Applied Health Analytics program. He’s also working on the completion of his PhD degree in health equity technology. Welcome Tim.

Tim: Hey, John, it’s great to be here.

Rebecca: Today’s teas are:… Tim, are you drinking any tea today?

Tim: Yeah, I am, actually. I’ve got one here that is this really nice licorice root spearmint mixture.

Rebecca: That’s an interesting combination. I don’t know that I’ve heard of that.

John: And I have just a plain old oolong tea today.

Rebecca: A nice choice. I’ve swung back to old habits, and I have an English afternoon tea.

John: Must have been a tough day, huh? [LAUGHTER] Today, you’ve been doing some interesting work in which you’re planning to have students work through a learning portfolio constructed for each course module in a health science course in which they’ll complete a structured sequence of interactions with a chatbot tutor that you’ve created. Before we discuss the details of this course, can you tell us a little bit about the course? How large is it? What’s the course level? And who are the students that generally would be taking this course?

Tim: Yeah, absolutely. So this course that we’re focused on today is HS 300 which is our human diseases course. It’s quite an interesting course. A typical class is usually about 40 students, usually juniors, although I do get sophomores and seniors in there. And why I say it’s an interesting class is even though it lives within our department’s program of public health I would say, in the average term, I only have about half the class filled with public health majors. The rest of them are a mixture between students who want to go pre med, so nursing students or med school, DPT, biomedical engineering, and then also a smattering of psychology majors. So it’s quite a diverse group I’ve got in this class, to be very honest, with a range of different backgrounds.

Rebecca: Can you talk a little bit about your general approach to how you’re planning to address the class? A little bit about the portfolio and the structure overall?

Tim: Yeah, do the objectives of this class are for students, broadly speaking, I won’t get into the full curriculum guide here, but broadly speaking, is for them to better understand not only a wide range of human diseases across all of our systems, but also the application of that knowledge to both the clinical side, so think more like Western care and also the public health side. So we have this interesting mixture here. In terms of the portfolios, the reason I chose to start with this class with this concept was, as you can probably imagine, with the varied degrees I have in there, I have students walking into this class with everything from having completed anatomy and physiology I and II and maybe a pathophysiology course, all the way to students who haven’t seen biology since high school, because they just haven’t reached that point yet. That makes it a challenge. It’s a little hard to talk about congestive heart failure if they don’t know the anatomy of the heart. So that is the reason I chose this approach, is because, and I know we’ll get into this, really is focused around meeting the student where they’re at in their current process of learning, which I think is really the best approach, especially in a course like this, where background knowledge is so critical.

John: Could you walk us through the steps that students would take through in a typical module? And I believe each one of these modules is for one week?

Tim: Yes, that is right. So, the way I prefer to build my Canvas shells, and it’s the same in person or online for the general layout, is that I start off each week, beginning in week two, where they have to do the content first before we actually dive into the meat of the week. So the first thing that they have is they are given a portfolio to work from. And the way that these actually works is that they are handed a file, and this file is the thing that I have produced for each week, and it starts off with some core basics about that particular body system, and then we progress into the diseases, and then progress into more application of their work. And they’re going to take that file and not only work to fill out all of the spots that I have included in there, but they’re going to take that file and give it to this AI bot that I have built out, which is going to act as their own personalized Socratic form of a tutor to actually help them work through the entire file. So that’s the general layout with the portfolio. Once they actually go and turn that in, that actually unlocks the rest of the week’s work. So for an in-person class, this means they got to do the portfolio before they show up to class. For an online class, it means they have to do it before they can engage with the more advanced application we would be doing that week.

Rebecca: Can you talk a little bit about why you chose this approach and how this counters how a textbook is designed for a typical student?

Tim: Yeah, absolutely. I know I alluded to this briefly earlier, but when we have a course like this where in, effectively, I’ll call it 14 weeks, we’re going through every body system and covering well north of 100 different ways that we get sick, ill, and die.

Rebecca: Sounds really exciting, [LAUGHTER] all the ways you can die.[LAUGHTER]

Tim: That’s actually the way I kind of go and bring it in, was like, we’re going to go through so many things, and it’s going to convince you that nobody is normal. There’s just no possible way. But students actually really like it, and I hope not too many of them are freaked out by the time they [LAUGHTER] leave my course. So the reasons for this, I found that with a class like this, we really kind of have two options. The first one is, is that it falls into this very, what I’ll say is, traditional, classic style of a lecture class where we show up and we have to effectively spend the first day raising the bar for everybody so that day two, we can actually do some application work where we can challenge their knowledge. That is tough, because with the gambit of not only backgrounds, but also just experience levels in this class, I can tell you that any given week, if I were to do that, I’d be boring, probably about a third to half the class, because they already know about the systems, they’ve been through the courses, they work in the field, whatever it may be. So that is kind of the downside of that approach, and I’ll be honest, the textbook we had been using for this, which I think is a fantastic textbook, if… here’s the big if… if you have some anatomy and physiology under your belt. If you don’t, it’s a lot to swallow, like it is a thick textbook. It’s phenomenal, and it doesn’t adapt. It’s a textbook. So that was really the core reason why I went down this path, was I’ve been working with artificial intelligence chatbots since 2015 which was pre all of what we have now, and back then, it was a lot of work. [LAUGHTER] It was tough work to actually get them to where we wanted them to be. So I’ve been going and doing this for quite some time, and ever since openAI kind of broke the teaching world in November of 2022 it’s been really fun to start to begin to play and see where are these capacities going. And suddenly we’re at this place where we can use this tool that our students can access for free to give them this individualized approach. So my hope is going into this term, that what this will do is truly meet the student where they’re at, if they’re that student who has just one prior biology I course, no anatomy, and physiology, it’s going to spend more time going through the anatomy that they need to know to be able to comprehend the diseases and the processes that we actually care about in this class.

John: Now I have to say this, if the students do not have any anatomy and physiology under their belts, they’re probably a bot already. [LAUGHTER] But moving on, one of the things I noted in looking through some of the modules that you’ve created is that you put in suggested times. And this is an approach that I know Betsy Barre has suggested in terms of a couple of variants of student workload calculators that she’s created. And we’ll include a link to our podcast with her about that. Could you talk a little bit about why you’ve included that and how you came up with the estimates of how long students are likely to take with each of these activities?

Tim: Yeah, absolutely. So I am a big fan of providing estimates or ranges with the students. Personally, whenever I build a course or rebuild a course, I use the Wake Forest workload calculator that they have on their website. I love it. I think it’s fantastic as a way to just get a general scoping. And this also comes from… I have to think here… I believe at this point it’ll change after 5:30pm today, but at this point, I’ve taught 184 sections of courses, and this has been consistent feedback I’ve had from the students I’ve taught, is it’s really helpful. When I look at the list of things to do, if there are like numbers beside it, that even if they’re not 100% right, they give me a way to kind of calibrate my brain. And so I have some idea about how to schedule out my week. I don’t know about you, John, but our students here, just because Flagstaff, if they’re residential, it’s just an expensive place to live. They’re not only full time as a student, they’re working two or three jobs on top of that. So being able to tell, like, where should I spend my time, what should I do first, is really important to them. So the estimates in this I kind of went and derived a couple ways. The first way was I went through and tried to turn my beginner mindset back on. It’s a little bit harder when you spend a lot of time in the field, but I tried to go backwards and work through this and get a feeling for how many questions do I need to ask if I’m just that average student who maybe has done anatomy and physiology I but hasn’t gone further, which is kind of the bar I generally see as an average in this class, and then I will fully admit, I have some unpaid help, called family members, [LAUGHTER] who are more than willing to go and jump in and just like dive into it and go and tell me what worked, what didn’t work. They’re all individuals who don’t have the background, so they represent that one extreme on the far end of their really having to learn everything. So that’s the way I got this rough estimate for what I’m starting with this term. And then one of the things I’ll do is, this term, as they work through the portfolios, I am actually asking for them to report back as they go and turn into each one about approximately how much time did you spend on each one, and so that I can better calibrate this tool as I move forward. And so we’re on a grand adventure.

Rebecca: I had some activities that I did that with students, and had them kind of mark their time and track their time, and I think it was just as interesting for me as it was for them to understand how they spent their time, or where they might spend more time than they thought. I can also see how, having times broken apart, even if they’re not 100% accurate for each individual student, it does give you the idea that you’re going to spend more time on this activity than this activity. This is maybe more important, and therefore I want you to spend more time on it. And it’s a way of conveying some of that prioritization that needs to happen, that students have a difficulty doing if they don’t have expertise in the subject.

Tim: Yeah, and I’ll say as well, I know, because I have these students in class every term, and I love them, but there will be some of them that will spend way too much time on one part, it’s the kid who’s clearly gone into med school, will do great as a physician, or whatever they go and pick, and hopefully this helps them keep on track where they’re not spending five hours in the first bit that should be 15 minutes, and then they panic about the rest of the thing.

John: Yeah, those relative weights are important just in terms of showing how much priority students should place on each.

Tim: Absolutely.

John: We know that active learning approaches, as you’re planning for this course, tend to result in increased learning by students and longer-term knowledge retention and more transfer ability. Yet, when students are asked about how much they feel they’ve learned when they’ve been using active learning, they generally report that they learn more in lecture courses because they feel that they’re not getting as much because they’re struggling with things and any gaps in their knowledge is becoming transparent, while if they’re sitting in a lecture class, they can sit there and nod and smile and everything makes a lot of sense. How are you going to sell active learning to the students?

Tim: That is great. And to be frank, you bring up an item here that is one of my favorite, just bizarre oddities, that I get to explain to my classes. So I have a couple approaches on this one, the first one is on day one, which, as we record, this is a couple hours away, actually, this is one of the first things I actually go and bring up, is to explain, this is not a lecture class. We’re not doing PowerPoint slide decks the whole class. Here’s why. And I explain all these points about the fact that active learning actually does make you learn more and, more importantly, helps you develop these skills that you’ll actually need to make yourself career ready. And that’s really important, and it feels uncomfortable, because you’re going to get it exposed, you’re going to learn what you know you know and what you don’t know that you don’t know [LAUGHTER] and that can be frustrating. It can be anxiety inducing. So I think the first bit is just calling that out, bringing it up, and just acknowledging it from the start. So I think that is part one. Part two is I have built in, and this is an item I actually learned from some fellow faculty at a conference about a year and a half ago. I actually have required work at the end of each week, which is a very short self-reflection that they have to do. And it’s 10 questions, and it includes things around this area, about like, preparation for class, what’d you learn? What you feel like you had a hard time with? …all of that, and I have seen that with just even holding them, of course, the same, that inclusion, the feedback has been phenomenal in that they really enjoy being effectively forced to pause after they’re done with everything and really think about what did they do this week, and then how do I improve for next week? And then I do think the third and final bit, which I realize I might be uncorking a can of worms here on you, John and Rebecca, but I use ungrading for all of the courses I teach, and in fact, for this particular course, I use a contract grading style. And so I do think that that is one major component, is utilizing that style that supports this active learning, where they know that they’re in this effectively safe space to work in, where mistakes are expected, and I’m not as worried about the mistakes. I’m more worried about the mistake I see you make 15 times in a row and not learn. So being able to get that feedback and learn in real time is really the key. And when they feel safe about their grade, I found that they’re much more willing to take risks.

Rebecca: That’s a lot to unpack on that first day.

Tim: Yeah it Is. It’s a big first day. And really, I’ll be honest, it’s the first week. It’s not really the first day, because as much as I would love to dive into how do we diagnose disease on the first day, they haven’t done anything yet.[LAUGHTER] So we can’t do that yet. And so it’s really the first week, I should say.

Rebecca: Can you talk a little bit about the Gemini Gem that you created, that students are interacting with, maybe a little bit more about what their process looks like, how you’re going to coach students around hallucinations, what’s it allowed to do when it’s interacting with students, and what other kind of constraints and guardrails you have in place?

Tim: Absolutely. So Gemini Gems, just for those folks who may not know, this is effectively a, let’s say it’s a tailored version of Gemini that we can instruct to behave in certain ways, or we can give certain knowledge background to that allows us to make a customized chatbot without having to go through the hassle of fully training your own chatbot. So this particular Gem was a bit of a journey over the last approximately three months, where I abstractly played with a range of different models, and for my own work and myself, I work around not only the Gemini bots, but also ChatGPT and Claude, just being like our big three and occasionally Llama, but I don’t do a lot in there. So the creation of the Gem really came about when I went and sat down and said, Okay, if I were able to spend time with every single student, one on one, how would I approach tutoring them? Because that would be our dream. They would learn so much, and there’s just no way, unless I can clone myself a bunch of times. There’s just no way. And I came to kind of three broad, I’ll say, pedagogical points I wanted the tutor to do. The first one being Socratic style. So I don’t want it to just go and lecture. All of the students I’ve taught for years and years now have been very clear. They don’t like that. They want to be involved. So Socratic style. So we’re going to ask a question. I might give you a little bit of a knowledge point, and then we’re going to expand on that, but I’m going to ask you to really use your brain and engage. Point two is that we want to assess prior knowledge and learning. So does Rebecca know about cardiac anatomy and valves, or do we need to kind of back it up and maybe go and cover the basics. So when we dive into a new portfolio, that’s really the first thing it does is to go and ask you a question related to where we’re about to dive into. So again, the heart is my jam. I’ll be very honest. That’s what I do a lot of work in. And so if we’re talking about cardiac disease this week, we’re going to assess, do you actually know about the heart. How much do you know? Is this enough for us to actually talk about disease or not? And if it’s not, let’s back it up and start there, and then go for it. And then the final one is that idea of the zone of proximal development, that idea that ideally we want to be asking a question that’s they’re getting about 80% correct, and so they’re encouraged, but not overconfident, because if they’re getting 100% correct, they think they know this and they really know about half of that. So those were the three bits I began with, and from there, I built out this prompt. The things it is allowed to do, it’s allowed to act in these three ways, and it’s allowed quite a bit of freedom to be in this encouraging form of a tutor that supports work and tells the student when they’re wrong. Because I think all three of us have probably engaged with a bot, and I know that Gemini can be one of the more challenging ones, where it cheers you on, even though it’s blatantly wrong. [LAUGHTER] We don’t want that, so it is encouraged to constructively criticize, I’ll say, when the student is wrong and help them correct that. What it’s not allowed to do, though, is also really important. There’s a whole component to my system prompt that gives it a rule set of what it can’t do. It’s not allowed to just fill out the file for them. It’s not allowed to write their answer for them either. It can help critique their work that they’re going and putting in to kind of help them improve it, but it’s not allowed to just go and do that for them. Those are the guardrails in place, broadly, in this one. And then, for this particular class, I think you also asked about things like hallucinations and background knowledge. I’ll start with the latter. So background knowledge, given the fact that we’ve been studying human disease for quite some years at this point, to go and say it briefly, like at least 100, I found that when I played with this by giving it some RAG files, so some files it can pull from, it actually slowed the bot down and didn’t improve the answers. Because realistically, when we’re talking about human problems, sites like Mayo Clinic, Medscape, all these free sites are already included in the build of the AI, so it already has all that. So what it really needs is more fine tuning to make sure it’s focused on, which, again, is a guardrail, focused on accepted knowledge in this realm from reputable sources, versus trying to give it a bunch of specific files that it then has to go through and search and find what it’s looking for. Now, I will say on the flip side, I have a different course that’s exercise considerations for special populations. So this is a little bit more of my background, exercise physiology wise. How do we do activity with a cardiac patient or a cancer patient or a child who has autism spectrum level three. And in this one, I did provide files because I found that especially with a topic like autism, there’s just so much out there that it can inadvertently get onto the wrong things, that it was better to give it position statements from major fields and bodies to go and pull from to make sure it stays locked in on where I want it to go. So that’s effectively the building of this bot. I have found that I can generally use the baseline system prompt around how this bot behaves and then add in some of the details I want specific to a course to adapt it fairly quickly. Finally, hallucinations. And so I got to ask, what’s like a favorite for you that you’ve experienced around a bot hallucinating a fun thing for you.

Rebecca: I don’t know if it’s fun, but something I’ve seen a lot is hallucinated sources and students claiming that they really do exist. [LAUGHTER]

Tim: My favorite was… now, this was back in 2023, so it has improved, I will say that… But in class, asked it for a weight training plan for a adult who was 85, had never worked out in their life and just wanted to play with their grandkids, which was a example form of a client for that class. And it gave them Olympic lifts on day one. [LAUGHTER] It was like, we’re doing clean and power jerks, like, what are you doing? No, absolutely not. So hallucinations, as either or both of you may have seen, openAI released a white paper at the end of last year where they supported that currently, mathematically, hallucinations are going to happen, period. It’s not a thing that can be fully wiped out, and that’s just the reality we’re in. So what I actually do is they work through the portfolio, then their task two behind that is, by the end of the week, we are going covering that working content, I actually give them my key, and they’re in charge of actually reading through this key and going saying, were there any points where my answers weren’t quite correct? And I actually view that as a valuable skill, because we’re working on going and teaching them that hallucinations exist. Bias exists in these things as of right now. They always will. Hopefully, maybe there’s a way to wipe that out in the future. Right now, it doesn’t seem like there really is. So we’re going to not only use it for learning, but then we’re going to check ourselves against a known source to see where might it have led you astray. And I’ll be honest, that is one of the big unknowns, because, yeah, I’ve done some testing within a small group, but once we roll it out in this semester, it’s going to be north of 100 students who are going to go and do this. We’re going to see what is the actual rate of students having incorrect answers. If it becomes too great, then we have some tuning we need to do on the spot.

John: I hope that you’ve asked the bot to ignore anything coming from the CDC since January of 2025. But going back to the issue of assessing student work. You mentioned you’re using ungrading, but yet you still do have to assign grades, typically at midterm and at the end, but you do have students reflecting each week on what they’ve learned and their progress. So how are you connecting their assessments into those grades? And how often are you doing that?

Tim: Yeah, there’s a couple things I do, so starting with the smaller items in class, let’s pretend like we’re about to walk into week two. So you’ve done the week two portfolio. The moment I go and hand that in, I can immediately get to the key, which is the goal is they can go and get to it right away and see, were there any problems there? And now we’re in class. The first thing we do every week, day one in class is we start, they get into their groups that I pre assign. And to be very honest, until week six, it’s just random. I want them to work with a bunch of people in that class. So they get into these preset groups, and the first thing we do is a tRAT or team readiness assessment test. Now I’ll kind of pause here, because I know it’s a mixture here, John, I’m going seeing a nod and a smile. I feel like you’ve used this. Is that right?

John: I haven’t used it myself, but I do know how it’s used in team-based learning.

Tim: And Rebecca, is that a thing you’re aware of, or would you like me to kind of walk through like, what is a tRAT?

Rebecca: I think for our audience, we need to walk through.

Tim: Awesome. So a tRAT is probably one of my favorite tools for in-person classes, because I effectively get to trick my students into taking a quiz in class that they like taking, which is quite an experience. So this is a multiple choice quiz that you can use these cards that are pre-keyed. I will admit that I’ve actually moved to running a Google script on this. And the idea is basically they work as a small group. So I do groups of three or four, no larger than four in my classes, and what they have to do is, as a group, they go and decide, hey, for question one, we think the right choice is B, and they go and click on it, and if it’s wrong, it will go and turn red and say, try again. So they have to reason their way through and every time I’m wrong. I’m reducing the amount of points I could earn for that one. So because I use four choices in mine, if I get it right on the first thing we choose, it’s worth four points. If I don’t, it goes down to two and then one and then zero, because if you clicked on all four, you truly have no idea what was right or wrong there. Now for my classes, for this term, because I want them focused on their knowledge, and what do they know, and what do they not know? All that they have to do is report what they got, and if they’re there in class and do it, they get marked as all done with that. With that, I know I’ve heard feedback from other faculty being like, “Well, if that grade doesn’t go in the book, don’t they just not try?” And I actually go and say, like, “No, that’s actually the opposite of what I see is, they know how they did, they don’t like not doing well [LAUGHTER] in the class, especially if they spent all this time engaging.” So what they do is, I see from effectively week 2 to week 14, the grades just keep increasing, week over week over week. And what I actually do with these is, instead of just regurgitating what they just learned through the portfolio, we actually take one step more advanced, and I have 15 questions organized, usually around anywhere between four to six major conditions we’re going to cover. And we ask questions that are more advanced and more case application work, where if you don’t understand the background, you’re not going to get this right. And because it’s forcing them to do peer-to-peer engagement and reason through what is right or wrong, it requires them to really assess their own work and knowledge. So that’s a bit of it is, in class that goes and gives me a timestamp, because every single attempt I see live… that goes back to a Google sheet. So I see everything, I can see where we are right on the first attempt, where are we not. Where might we need to refresh on day two, and then the really big bulk of it is around that what happens after the tRAT? And what actually occurs in my class after that is we’ll spend that last hour of that first day and then an hour fifteen on our day two doing some quite advanced application work. So as an example, one week when we talk about pulmonary conditions, they go straight from the tRAT into learning what a SOAP note is. So, for those who don’t know, this is a way that we write a note in more of the clinical medical field. So they learn about this way of writing notes, and then they, as a team, actually work with a bot that I will assign them that acts as a patient, and they have to produce a SOAP note for that that they then have to present to the class on day two. So it’s all, can they pull out the right information? I f they think it’s problem A, they think it’s pneumonia, what other questions do you need to ask about that? And that’s our weekly work. And all of this at the really end of the term leads up to the culminating project in our class, which is three parts. For this class, this is where I’m really assessing them on what do they know? And they can revise and improve. They pick a disease that exists today. So as much as smallpox is fascinating, it doesn’t exist, except for in labs and hopefully never out in our globe again. And so they pick a thing that they find intriguing. The first bit is they have to do research and write me a historical overview of this disease and how it has impacted our health over time, including, how has it affected our clinical past, how has it affected our public health past, our policies, all of that stuff. So it’s this really overview, big scale idea. Number two, they go and take what they just went and wrote there and turn it into fact sheets. Now a fact sheet, I guarantee both of you have seen before, because if you’ve been into a physician, they have those things like: “Get the flu shot, now. Here’s why.” That’s a fact sheet. So they have to write one about their thing that they picked for a general adult, one for a physician, and one for a person who might be in Congress and might be making policy. So it requires them to take this knowledge and turn it and go and say, “What does this general adult need to know? What does maybe a physician need? What does a policy maker need?” And then the last of it is, and we’re ramping up into this, is they’ll actually give a presentation in week 15 of our term, where they have to propose a potential public health program that would help within a certain group of people for this thing and improve their health, which requires them to really thoroughly not only understand the disease, the processes behind but also the populations affected. How might we reach those groups? How do we help them, like, what are they lacking? And then, how would we actually go and reach them? And for this group, that’s usually their first time they’ve ever done this, so that’s a big ask, but they have the knowledge they can do it successfully on that. So really, a lot of it is around the projects, but everything builds up to those projects.

Rebecca: So it sounds like, with all the project-based learning that you have in here, you’ve certainly been thinking a bit about student use of AI and its potentials for cognitive offloading and academic integrity that we know come up in many conversations around AI. Can you talk a little bit more about the reasons why you chose some things and how that might relate to academic integrity issues?

Tim: Absolutely. So I think this is the core of a conversation we could have for hours on here. I have students every term who it’s very clear that a paper and assignment was written by AI. I don’t think that’s going away anytime soon. And [LAUGHTER]] the reality as well is there was, and I would have to go find it again. But there’s been quite a bit of work that’s shown that the majority of employers are expecting them to walk out of a college with some generative AI skills. So we need to be going and teaching this. And part of that is we need exposure. So for myself, I think there’s a couple parts to this question. The cognitive offloading is an item that, again, I do address in week one or week two, just kind of, depending on how week one rolls about just having kind of a very open discussion at the start of a class for about five minutes. About these are the risks if you, for example, try to acquire foundational knowledge and you just offload that to the AI writing the assignment for me. These are the risks that are run, and here’s why I don’t want you to do that. This bot that I am getting you to work with is specifically designed to force you to learn and think and then actively engage in this to help prevent that. So that’s the first bit. The second bit as well is we do actually go and talk about throughout a class like this, we go into a little bit, and I will fold this in maybe the first five minutes, kind of threaded throughout the term, we go and talk about more of this ethical or philosophical component with AI usage, because it goes beyond just things like power and water that is used for going and training these bots. And we do go and talk about that, but we’re beyond that here, and we go and talk about things like, “Well, what is the purpose of learning? When do we use these as an enhancement versus using it in a way that’s going to go and detriment your learning and your skill development? How do we avoid that?” And I’ll be honest, this is not the first term I’ve gone and spoken about these. I’ve done this for a couple terms now. And the feedback I’ve heard, which I suspect both of you will have heard if you have gone and done this before, is that students actually really appreciate it, because a lot of our kids, especially right now, they were in high school during the rapid pandemic lockdown era, and then maybe hybrid, and then maybe [LAUGHTER] back into lockdown. And what I’ve heard from them is they were threatened, and they were told by a lot of the folks in high school, and then moving into being freshman and sophomore, “If you use this, I’ll fail you out of the class, you’ll get expelled.” And so they’re afraid to admit any type of use. And nobody’s talked with them about, how do we develop this as a tool for your toolkit versus replacing your brain. And once we actually get into that, then we can actually explore this region in a much more effective way. So I think we need to have an open talk about this, and then as we build out parts of the class providing the guidance they need. So for example, for the paper that we just spoke about, that historical paper, they’re allowed to use an AI, and I have this built right into that. I’m fine with them if they want to go and use it for brainstorming, for helping them outline the paper, feedback on their own work, feedback on the paper. And I want to hear their voice. I want to hear their writing. So it’s not allowed to write the paper for them. I want them to write it. So I think it’s both the education and then being very clear about what you want them to do or not do, and then why.

John: And given the nature of the assignment, they’re already seeing how AI can be useful in assisting them in various ways that can carry on into their future careers. And the ungrading aspect of it also takes a lot of that pressure of grades off, allowing them to focus on learning. So, those are approaches I’ve recommended to many people trying to do these things. If we can show students how AI can be useful, and if maybe we shift their focus from getting the highest grade to learning the most, it can have an impact.

Tim: Yeah, well, and that, and I’ll go throw out a third one here, building them a box and allowing them the freedom to explore in that box. So I’ve told them what parts they need in the paper, but I’m allowing them to pick a disease that they actually want to research out, and that is a huge help, because when they want to do the work, I’ve had some phenomenal work done where I read it, and I’m like, man, you’re a couple reviews away from being able to publish this as like an undergraduate, like this is phenomenal.

John: That can really tap into intrinsic motivation. I know when I give students freedom in the choice of their projects or topics, they get so much more engaged than if they’re given a list of things they have to do.

Tim: Yeah, totally.

Rebecca: There’s a lot of steps week to week. There’s a lot of things that students need to do. In my experience, people don’t always read all the instructions [LAUGHTER] and maybe don’t follow through in the ways that we’re expecting them to. How are you hoping to prevent some of this? Especially because there’s a lot of steps in “this unlocks that” that can prevent someone from making progress during the week.

Tim: Yeah. So first one is, I have, unsurprisingly here, I’ll say I do have very clear instructions on like, what to do each week. And as you just said, some students just don’t read that. And the first thing that they do is, “oh my goodness, I can’t see the week,” email the professor. So the first bit is, I have learned how much instruction is enough versus how much is too much, because if it’s just pages and pages, they just won’t read it. So how to be succinct there? Part two is, for example, for the first portfolio, we’re going to actually start that in class. So that’s our day. We’re going to actually spend about half the class, and we’re going to sit there and work on it in class. So I’m available right then and there, and I make sure that anytime we have something new happening in a course, and this is the exact same in person or in an online course, is that I, at the start of class, I’m very, very clear both of the days about this is exactly what you need to do. And yes, that will not catch 100%. I still get a student or two out of 40 who are like, “Oh my goodness, it won’t open up. What am I doing wrong?” And I just have to go, remember to turn in your portfolio and that will unlock over the weekend they’re like, “Oh yeah, I recall that now.” So that has been kind of the path I found that works well, is to introduce complex things in class, including the projects, allow some free time where I can help them immediately, and then follow up with regular reminders of what to do.

John: Northern Arizona University, as Oswego does, has a site license for the use of Gemini. Would this work as well if all students didn’t have the same equitable access to a platform?

Tim: Yeah, that’s a big question. And in fact, I’ll say to throw a complexity here, we not only have the Gemini, we’re a Google, and we also have Copilot. I have no idea why, but we do. [LAUGHTER] So, we have lots of fun stuff. Without that, I think that even if you were to use the Gemini free version, that still gives you sufficient usage and space to be able to do a thing like this. The big asterisks I throw on there is making sure that we’re 100% transparent with our students about the fact of, “Hey, we don’t have a license. We’re going to do this. We’re going to do it through the Gemini bot, because the context window… for those of you who maybe that’s a new phrase, that’s basically the memory like in quotes for the bot… is gigantic, and the usage terms for the free tier are quite large enough to do this task. That anything you put into this bot, they could use it to train so don’t put anything that’s personal information into this bot. And I think that is a very good thing that we just need to teach in general, because there are a lot of students I teach who, one, they haven’t been told that they actually have this Gemini license for Ed that we pay for and that it’s all private. They’re going and using the ChatGPT free version, and they have no idea that they’re training on their data, unless they go to, like, flip off like a switch. And so that’s the first one, is we need to go and explain that, because there are students who, once they go and learn that are like, “oh my goodness, I put some very private stuff into here. I had no idea.” And then also thinking about the use in the class, for this class, for this purpose, we’re not asking them to reflect on anything. It’s straight disease and some biology learning. There’s nothing there that should put any personal information at risk. So I think it is 100% doable with the free license if I don’t have it. I think you just have to pick your bot wisely, because I’m a Claude person. I love Claude, and their free tier is very restrictive on the amount of use you get each day, so that’s not a good pick for this.

Rebecca: So we mentioned at the top some of the other ways that you’re using AI in your research. Can you talk a little bit about what that work looks like?

Tim: Yeah, I’ll try to be brief here. So I am a researcher on what’s called the ASD3 project, which is a joint project between the Oregon Health and Science U. and our institution. We have a PI here that I work under, and under that I work on the machine learning and deep learning, I’ll say, arm of the project. This is a multi-million dollar piece of work that looks at better understanding the autism experience for children, specifically around the disparities in the diagnosis and access of care. So a lot of data work. We work on several big code sets, so my work is mainly around using these advanced models to better understand where are these present, and how do we address them? So that’s that work. On the tribal health part of it. This is actually what I am planning on doing for my own doctoral research, is around trying to expand the availability of, specifically, heart care. So the cardiac rehab that I am trained in working on to individuals in a tribal space. The reason for this is that, based on some research we have, not only is heart health a major health burden within this space, but the relative risk of death from a heart-related problem in American Indians is somewhere around 53% higher than the average white adult’s, which is shockingly high. It’s the highest of any group. So, within this space, we also know that rehab is a thing that if we can get people into it, it dramatically drops the rate of a first event being like a heart attack, or if they’ve had their first one, their next event. But the problem is this, this is usually done in a hospital. And if you look at Navajo Nation, this is an area the size of West Virginia, they only have 13 hospitals total in that entire space. It’s very difficult to get to this, especially if you’re working full time, especially if you happen to be a farmer or you ranch or do both. So it can just be non feasible. So my work is focused around developing the foundational knowledge that I’ll take forward after I’m done, into the actual building of a generative AI that can supply this as a service within this space, and that is based on a very engaged approach, where we are working with folks from this space to help drive what does this AI look like? How does it engage? What do we do with cultural adaptation work? Because this is not my lived experience. It’s where I have knowledge, but it’s not my experience. It’s a different form of, a culture, a background, than what I am. So I am working on working with the tribes to be able to better drive what does this AI look like that we eventually build? How does it engage? And so that we’re not trying to walk in and propose a thing that doesn’t actually work because we don’t know what needs to be done yet. And so that’s, briefly speaking, the forms of AI that I do research on.

John: Some very valuable areas of research.

Tim: It’s exciting and frustrating at the same time. [LAUGHTER]

John: So we always end with the question, what’s next?

Tim: Yeah, in two hours, I’m going to go and see how this learning portfolio lands. That’s really what’s next for me is in two hours, we’re gonna dive into this for the first time for the entire class, and it’s no longer a trial. It is on the ground running. And yeah, I would love to do like a follow up, maybe in the summer or the fall, and we’ll actually have some figures and data and student feedback on how did the portfolios go? How did they change the class?

John: That would be great, and we’re looking forward to that conversation, so we’ll keep in touch.

Tim: Yeah, for sure. And I would just like to say, I do appreciate the time to chat, because I’m very excited about what we could do with AI in our space,

Rebecca: And it sounds like you’re finding ways to use it in all your spaces.

Tim: Absolutely.

Rebecca: So thanks for joining us. We appreciate your time and expertise.

Tim: And thank you, both of you.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

428. The Science of Learning Meets AI

Posted in active learning, cognitive science, diversity, edtech, educational technology, engagement, equity, evidence-based practices, feedback, higher education, inclusion, learning, pedagogy, teaching, universal design
January 14, 2026

The widespread adoption of and the rapid evolution of generative AI platforms have created substantial challenges for faculty in how we assess student learning. In this episode, Lew Ludwig and Todd Zakrajsek join us to discuss a new resource they have created that is designed to help faculty use AI to efficiently support teaching practices based on the science of learning.

Lew is a Professor of Mathematics at Denison University, where he served as Director of the Center for Learning and Teaching from 2020 to 2025. Much of his recent work has focused on innovative methods for utilizing generative AI. Todd is an Adjunct Associate Research Professor at the University of North Carolina at Chapel Hill, from which he just retired a few months ago after 17 years. He is also the Director of the International Teaching Learning Cooperative and the Director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published six books (so far) in the past five years.

Show Notes

Transcript

John: The widespread adoption of and the rapid evolution of generative AI platforms have created substantial challenges for faculty in how we assess student learning. In this episode, we explore a new resource designed to help faculty use AI to efficiently support teaching practices based on the science of learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Lew Ludwig and Todd Zakrajsek. Lew is a Professor of Mathematics at Denison University, where he served as Director of the Center for Learning and Teaching from 2020 to 2025. Much of his recent work has focused on innovative methods for utilizing generative AI. Todd is an Adjunct Associate Research Professor at the University of North Carolina at Chapel Hill, from which he just retired a few months ago after 17 years. He is also the Director of the International Teaching Learning Cooperative and the Director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published six books (so far) in the past five years. Welcome, Lew, and welcome back, Todd.

Lew: Thank you.

Todd: Thank you, Rebecca,

John: Our teas today are:… Lew, are you drinking tea?

Lew: I am. I’m usually a coffee drinker, but I have a lovely Cha Atiso. This is a Vietnamese artichoke tea, which a student brought back for me from the summer, and I’ve really, really been enjoying that. So that’s my tea for today.

Rebecca: What does it taste like?

Lew: It’s got a little sweetness to it, which I like. I don’t put anything in it, but I really enjoy it. And it’s supposed to be healthy, no caffeine, all that sort of stuff.

John: And Todd, are you drinking tea?

Todd: Yeah, I went with a roasted dandelion spice tea. It says detox on the package, and I can always use a good detoxing. So I’m doing that and sipping it from my favorite cup. Mmmh.

Rebecca: That’s right, the tea for teaching mug.

Todd: I have my tea for teaching mug here. I never leave home without it. Actually, I never left it home. [LAUGHTER]

John: And Rebecca?

Rebecca: I have a Swiss Breakfast tea, John.

John: Very nice… on a chilly morning here in upstate New York. And I have a ginger peach green tea today.

Todd: Wow. Quite a variety.

Rebecca: Nice, this big variety. So we’ve invited you here today to discuss your newest book project, The Science of Learning Meets AI: A Practical Faculty Guide to Purposeful Integration, Student Engagement, and Ethical Practice. Can you tell us a little bit about the origin of this book?

Lew: Yeah, sure. I can get us started on that. So as you know, Todd and I both have backgrounds in teaching centers of faculty development. And our national conference is something called POD, Professional Organizational Development Network. And in 2024 we were at that conference, and here ChatGPT had already been out for about two years, and we were still kind of struggling with the idea of, how are faculty supposed to use this? A lot of the questions we were getting is, “Hey, how can I use AI in my classroom?” And it wasn’t until this conference was up in Chicago in 2024 where we realized we were asking the wrong question. It’s not really how to use AI in the classroom, but instead, maybe we should be thinking about, why are we using AI?

Todd: Yeah, and actually, I’ll jump in too, because Lew went right to that technical part and everything. Another thing that happened, which is why I love it when people get together, is Lew asked me if I would help out or just talk to him a little bit about some of the science of learning, because he wanted to really work on this project with bringing AI into this, explaining why it’s important to use it. And I said, “Well, that’s interesting, because I just started to outline a book that I want to do on AI, and I was going to check with you about the AI part, because I got the learning part down.” [LAUGHTER] So we just said, “What the heck, we’ll do it.” And Lew didn’t know me very well at the time, so he said, “Yes.” [LAUGHTER] Anyway, we made it through. But the big thing was that we really wanted a book that would teach faculty and help them understand the learning theory behind why the learning happens and everything else, and use AI to augment that. So this isn’t like using AI to do things, it’s using AI to help us do the things that we’ve been doing.

John: You start the book by acknowledging that faculty didn’t ask for this AI disruption or for any of the evolutions of this. Why start with this instead of with excitement about AI’s potential?

Lew: Oh, it’s called learning from one’s mistakes. So again, ChatGPT came out in November of 2022, by August of 23, I remember on my campus, we had a fairly large across the board faculty meeting about AI. It was about a half-day session. So administration was there, IT was there. And as the Teaching Center Director, I gave probably a 90-minute session on, “Hey, here’s this new technology and how we might be able to use it in our classroom.” And what happened was, just by saying, “Hey, here’s how we might be able to use it,” some folks labeled me as AI pro. So I was just pro-AI, hey, you talked to Ludwig, and he’s just going to be all in on AI. And that was almost like my scarlet letter [LAUGHTER]that I was wearing around campus to some folks. So I learned from that. So any of the workshops, any of the things I do since then, I start off with this exact phrasing, just to kind of level the playing field here, I say “You didn’t ask for this. You didn’t request an unregulated, untested, and rapidly evolving technology to suddenly upend education and nearly every other sector of society.” Now, I’ve been delivering that, like I said, quite a few workshops. I did it back in August for a group of high school teachers in Richmond, Virginia. They applauded me. [LAUGHTER] So it’s just kind of getting it out there and acknowledging it that, yes, this is a disruptive tool. We kind of have to acknowledge that grief and then kind of move on from there.

Todd: Yeah, I would say too, is really, I really thought Lew had a good handle on that part of it. I wouldn’t have put that in there, quite frankly, for the book here, and Lew was pretty insistent that it go in there. But as we started talking it through, I realized I always started any COVID workshops this way. When we were doing emergency remote teaching, if you just started talking about emergency remote teaching and didn’t acknowledge just all of the stuff that happened with COVID at the time, on the number of deaths, for instance, you can’t just jump into something and ignore it. So I think that Lew had a really good point here, and then I started pulling together what I did with that, with what he was talking about. And I think the whole concept is we started thinking within the book of, how do we do this? To recognize people are grieving, that they’re struggling, and we say it’s hard that we see you there, the frustration is real. Teaching is changing, just as it did with COVID. We don’t have a choice of that, and AI isn’t going to go away, so let’s recognize what’s going on, and recognize even some of the impact of AI, and then from there, then move on with a sense of acknowledging what’s going on.

Rebecca: It’s almost like you’re modeling some good teaching practices there. [LAUGHTER]

Todd: Kind of. Periodically it happens.

Lew: We try.

Rebecca: This adapt, create, embed, progression, the ACE model, is used throughout the book. Can you talk a little bit about what this looks like in practice for a faculty member who’s never used AI?

Lew: Yeah, so this work kind of, again, developed that starting point back in November of 24 after we had that initial meeting in Chicago and started to realize we needed some early wins for faculty. So we got a lot of faculty out there who are kind of hesitant about this new technology. And if you gave them something like a prompt or something that didn’t work, they’d be just like, “See, I told you, this AI thing, it’s a disruptive thing. So we’ve been workshopping this quite a bit early January onward, and refining the model. Originally, you ready for this? It was, instead of adapt, create, embed, it was improve, create, integrate, which, as an acronym, is icky. [LAUGHTER] We didn’t think that would land very well, so we went with ACE instead. So for example, the adapt, that’s kind of the easiest win. So what you do is, you take an existing assignment, you have that assignment that it’s a big project, but for some reason, it just doesn’t land with students. They always hand in something a little bit different than what you asked for. Well, that assignment’s begging to be TILTed, right? So Mary-Ann Winkelmes has this idea of transparency in learning and teaching, and it’s a technique that you can do for your assignment to make sure that your students understand why they’re actually doing the assignment. And it’s a kind of a three-step process. And AI is great at TILTing your assignment, and faculty members can take an existing assignment that they have. They’re experts on that assignment. AI can help them TILT that with this framework, and then they can see how that might better fit into their language for their course, and help improve what their students are doing. And the TILT framework has been really good. It kind of raises all student boats. But in particular, it does well for underrepresented and first-gen students do exceptionally well with this. So again, just kind of an early win with the adapt model.

Todd: Should mention quickly that we kind of started out and we based a lot of this on Ruben Puentedura’s model of SAMR, which is the substitution, augmentation, modification, redefinition for all of technology. If you have any new technology out there, you started with substituting it. How can we do this instead? You can augment it. And that’s a really interesting model. Didn’t quite work with what we were after. And so Lew had lots of workshops he had been doing. And so we kind of used that as a foundation. But as Lew just pointed out, the adapt, which would be similar to that original substitution, is to get you rolling. And I’m going to go ahead, Lew and take I’ll take the other two real quickly. So next is create. And essentially, it’s looking at how you can do something that you really couldn’t do before. And there’s lots of things that are out there that we could use for this one. We’ll talk a little bit more about those, probably later on, as we’re looking at some specific examples. But when you think about what you can do and couldn’t do before, one quick thing is a daughter, one time, was struggling with something, and she said, “Can you help me?: And I said, “I’m busy right now. I can help you a little bit.” 15 minutes later, I said, “I can help you now.” She said, “Oh, took care of it.” I said, “How’d you do that?” She says, “Oh, I just got on a group, a small online group, and then we solved it together.” And I said, “Who was it?” And this was years ago. And she said, “Oh, there was a student from Germany, one from South America, somebody from Nebraska, and myself.” That was never possible before. And so many things in AI do things like that. They’ll set up things we could never do before. So the “create” is, how can we do things different? And then once you’ve adapted it and created it, the next thing is, just embed it. Just make it part of the regular aspects of your course. How do you do that? And what we like to point out over and over again is that, you do this wherever you’re at. If you’re just getting started, you just adapt some things you’re doing. And that’s where Lew is really good at setting some of those things up. If you’re a little bit nervous, just take something you’ve always done, do it with AI, do a little bit better, faster, and then just get started, but don’t get overwhelmed with it.

Todd: And the other thing about doing with something you’ve already done is then you’re the expert, and you can critique what AI is doing for you. I wouldn’t go off and learn a brand new subject you learn using the AI, because I’d have no idea. But again, coming from that wheelhouse of your own expertise, don’t forget you are the expert with the content as well as your students and context. So keep that in mind whenever you’re dealing with AI.

John: And that example you use of the TILT model is a good one. I’ve given workshops to faculty about it, and a lot of people were reluctant to put in the effort to make all those changes. But when you show them how AI can do that, or better yet, if you share perhaps a prompt, or if you share a GPT or a chatbot tool that they can plug in their assignments to and their learning objectives, and it can do it for them, or at least give them a starting point on it, it makes it so much more likely that people are going to do that. So you talked a little bit about the ACE model, which is used throughout the book. But could you tell us a little bit about the organization of the book?

Lew: Yeah, sure. So the book’s kind of broken up into three parts. The first part, we call it “Creating your own Gen AI roadmap.” And again, chapter one, you didn’t ask for this, but here it is. So just kind of, again, acknowledging that situation that we find ourselves in and how to move forward with it. Then in chapter two, we start talking about what it means to think about shifting our teaching in the world of generative AI, so the idea of being transparent with our students and how we’re using it and setting up good communication with them, and then also the idea of as faculty members, as we’re embracing this, we really have to be curious and flexible as we work with this technology. We’re always pushing our students to take risks, be flexible, be curious. Well, guess what, that shoe is going to now fit on our foot as we try to move forward with this model.

Todd: So that kind of got things up and running, and then we kind of ran right into the learning aspect of it. Just to give you an idea of the book, I guess is probably the easiest way to do this is. Chapter four is on universal design. I’ve been a long time advocate of universal design, and I think AI can be now done to do things that were really challenging before. So we did dedicate an entire chapter to universal design. Then chapter five, we really thought community was important. So chapter five is about building classroom community and trust, looking at belonging, engagement, and integrity, kinds of issues. Chapter six comes into backward design, a huge aspect of teaching and learning, kind of knowing where you’re going as your goal. And then chapter seven is looking more at learning science of cognitive load, attention, spacing, interleaving, retrieval practice. And that’s the “building the foundation section.” And the last section of the book, section three, is for kind of deeper learning. We looked at some things about teaching and learning that just we know works well. So chapter eight is guiding students from novice to expert. We put together in that section zones of proximal development, scaffolding, and the gradual release of responsibility as ways of helping students kind of go into deeper aspects of learning. Chapter nine is on strategic learning. So we looked at transparency, TILT, cognitive apprenticeships, metacognition and those types of issues. And then we wrapped it up with chapter 10 of persistence and motivation. So there we pulled together the concept of self efficacy, growth mindset and self-determination theory. So the chapters have two or three strong learning principles to them, but they’re put together in ways that not only help explain the individual things, but also overall this concept, like persistence and what kinds of things go into that. And then the last thing I’ll say, since I’m on a roll right now, is overall, I think we really thought it to be helpful, repetition is just helpful for learning, right? We know that. And so we did is we built a book that has strong repetition. So chapter after chapter, we basically started with chapter four, and then we go all the way through chapter 10, and it’s set up so that there’s always an opening story or anecdote to set it up, which we’d want to do for just getting the learning focus there. Then from the model, there’s an improve activity, and then we look at the science components, and from there we might be looking at metacognition or Bloom’s taxonomy or something. So that comes into the second two parts of the model. Here we have a create and then embed, those activities are in there. And then we talk about the ACE model, and we have more activities that come into there. Then we look at risks and guardrails. And again, we’re going to acknowledge everything has risks to it. We’re not going to just pretend like it’s all great. And then we have a conclusion, pull it together, and we finished every chapter with discussion questions and reflection questions. So if you’re a faculty developer and you want to use this as a book group, for instance, we’ve got your discussion questions already built in, but for having faculty work through it, there’s reflection questions in there. The last thing I’ll mention is that we talked really extensively about how to cite this stuff, big APA kind of person, but we really wanted this readable, and so we went with a model where we put everything in as end notes, so as you read through it, you don’t get the standard citations with dates all the way through it, it just reads like a novel, almost, and I think it’s really easy to read. So you don’t have to read it cover to cover, although you should, [LAUGHTER] maybe three times. You can just go through and pick out what you like, but if you do go through, we’ve actually built it so that you start with simple kinds of examples and move to more complex ones. And Lew is masterful at putting together how to like, ask questions and do prompts and specifically with a separate font in there, we just have paste this in and add your components, and it spits out the answer. Pretty cool.

Lew: So what Todd’s explaining there is, we have a lot of different techniques. We have things called activities as well as examples. So each chapter has six activities, which are kind of page-long discussions of how you might use AI. And it’ll start off with some prompting, like he said, in that different font, but you put it in your context. We don’t do this: “Oh, a biologist would do this. Now you do it for yourself.” You actually, in the context of this thing, put in your information so you’re directly dealing with it. So we have, like I said, six of those for seven chapters. So 42 of those things. Each one could be a workshop. And then also we have smaller examples for seven chapters, we have 12 smaller examples, which is another 84 prompts. Those aren’t prompts, those are just more ideas. But Todd mentioned this idea of meta prompting. We give you a way to turn those examples into a prompt for yourself. So you’re using AI to create your prompt for you, and I think it’s called meta prompting, so that’s what he was referring to there.

Rebecca: There’s a lot written about the science of learning and Todd, obviously, you’ve written some books on the subject. How did you decide what science of learning concepts to include?

Todd: I just put a whole bunch of stuff in there. And then Lew started saying, “Nope, nope, nope.” [LAUGHTER] No, we got along great on these things. It basically boiled to Lew does tons of workshops, I do workshops, I think I’ve been at 300 campuses so far. So I looked through the different presentations I’ve given, and when things kind of hit and when they didn’t hit. Cognitive load is something a lot of people just don’t quite understand, and is a huge foundation of learning, because within cognitive load, you’ve got automaticity and schema activation and all kinds of different things that are practicing retrieval kinds of things. So I just went back to the warehouse of stuff and picked the things that worked, which is what I still think is really, really valuable about the book, is that these are things that work, and it’ll help explain them as you go. Big things like the metacognition, Bloom’s Taxonomy, cognitive load, automaticity, practice, retrieval, interleaving, the kinds of things we’ve heard. And it was hard, I will say, Rebecca, it was really hard. There’s a lot out there. I mean, obviously I got what is it a 270 page book on learning. So how do you just fold in some of that? And so, it doesn’t have everything, but I think it has the big things.

John: So you mentioned before that AI allows us to do things that we couldn’t easily do before. Could you give us an example and walk us through that in terms of how AI could be used to create learning activities that could not be done as easily in the past.

Lew: Sure, individualization of instruction. I’m at a pretty small school here at Denison, but I could imagine, wow, what if I was teaching a class of 400. So I might end my class here at my small school with, you know, a muddiest point, I get 20 responses. I look those over, what did students struggle with that day? And then next class, I can kind of address that. Imagine if you had 400 students, what could you do with that? Well, now, with AI, you can take this classroom assessment technique of muddiest moment and, on the spot, have students type their prompt into, say, a Google form or something like that, and as maybe the students are off working on some kind of project or discussion together, within two or three minutes, you could use AI to organize those muddiest points and give me the top three things that these 400 students are struggling with, something that was just was not even feasible before. So I think you know, at scale, this thing can be really useful for those types of examples. And with regards to CATs, the classroom assessment techniques, Todd’s got a book of like, 50 of these things, right? So there’s so many things that you can kind of do on the fly that AI can then help you with, kind of the managerial stuff, just the paperwork thing of keeping track of that stuff so you can respond in the moment for what your students are doing.

Todd: And Lew, I appreciate you mentioning that book, it’s interesting, got me thinking real quickly here. So that book is Classroom Assessment Techniques, which not too many years ago was like the standard. Everybody had a copy of this thing, and they did because the first edition was 1988 I believe it was, second edition was 93 though, and in 93 was right about the time we started talking about learning versus teaching, and how would you know if students are learning? And along comes Angelo and C ross, their book was out, I guess, already, and so it explained it. I mentioned this quickly because I then worked with Tom Angelo to do the updated version, the third edition, which is now out 30 years after the second edition. And the second edition was still selling, but we were removing things like “hand out papers to the students, have them respond on the muddiest point, and then take them home, but make sure you don’t ask questions that you don’t have time to respond to. Don’t ask elaborate things because it will take too much time.” Lots of stuff about CATs was about being careful not to overwhelm yourself. And if you think about that in the old days, that’s what you do. Pieces of paper were handed to, you’d read through them and everything. So we took out things like “share your results on the overhead protector,” and those are gone, and then we replace them with things like Lew was saying is, just put your students into a small think-pair-share, have them respond to this thing with a PollEverywhere or something, and then while they’re talking, then come up with the four muddiest things going on, and then teach those in the second half of class. And so, all of those classroom assessment techniques that we’ve been using for 30 years can now be made easier with AI. So I’m really excited about this.

Rebecca: You mentioned earlier about the chapter on Universal Design for Learning, and I know you’ve been a long time advocate, Todd. Can you talk a little bit about how faculty can use AI to design courses that are better for diverse learners and really have kind of access in mind, rather than retrofitting accommodations.

Todd: Yeah. So number one, I think the thing to point out here is, so I teach, this Lilly teaching conferences. They’ve been around for a long time. There’s different types of sessions that I’ll include in the program, and some of the sessions, and I talk to the presenters about this all the time, some of the sessions are just hot topics that you know are just going to go: engaging students in asynchronous classes, it’s going to fill a room. How to do active learning strategies, going to fill a room. They always have. Sometimes you have topics, though, that you put on the program because you have to talk about them. And if you don’t put them on the program, even though they’re not popular, then it doesn’t get picked up. And I remember putting sessions in the late 1990s, early 2000 about diversity that would have four people in a room, and individuals would say “there’s only four people in there.” And I’d say, “We’re still putting it on the program.” UDL was another one, anything on universal design, I can tell you I’d put in the smallest room in the conference, because it would be a great session, but it would draw like seven people. And one of the reasons it didn’t draw many people, and quite frankly, across all of higher education, people might love the concept, it’s really hard to do. If we’re going to do things like multiple means of expression, which is a key factor of universal design, it can’t just say that you can share your results in one way. What if you’re not good at putting text together, but you really have the idea down. So maybe you create a video, or maybe you have a play, or maybe you get in small group and have a discussion. Multiple means of demonstrating the information, so presentation, how do I explain this in a way that people can get it that’s always been hard, it is now being made really easy because we can go to AI with the correct prompts, and we can come up with multiple ways of expression, multiple ways of presentation. Lew was talking about having a class of 400 with the students in there. You could say to the students: “Here’s what I want to do. We’re teaching metacognition. Everybody think of something you love to do now, type into AI. I love to blank. And I want to learn more about metacognition so that I could explain it to my friend.” So I could type in I love long distance bicycling, and I want to learn metacognition. Rebecca might put in that she loves rock climbing. John might put in like he likes baking, and Lew might probably put in there he likes street fighting. [LAUGHTER] So you put those prompts in there, and then what pops out is everybody in the class gets to learn it in their own… I mean, what could be more universal than that? And so I’m just really excited about it. And we did the whole chapter, and Lew and I talked about it, and I think at the beginning I wasn’t even going to do a whole chapter, and he said, “Yeah, we need to do a whole chapter.” And so I think AI will open the gate so that people can finally do universal design in a way that’s meaningful without overwhelming them.

John: Faculty have been overwhelmed, first with COVID and now with AI, and many other things that are happening externally. How does your book address the reality that faculty don’t have enough time or energy to learn a whole new technology on top of everything else they’ve had to learn in the last several years.

Lew: Fair point. I think the thing we need to start with is the idea that we want to be careful and not this chase the technology, because if you do that’s going to make things worse, you’re going to get frustrated. What you want to do is try to use the tool to make things better. So I often talk about there’s 2, t-w-0, too many shiny objects out there. And one of the ideas with the too many shiny objects is there’s all these little prompts that people: try this, try that, try that, and that’s really frustrating for faculty, because it’s just a little bit beyond their skill set of what they want to do. It’s a biology example, and they have to bring it down into what they want to do. The other thing is, with the shiny object, is the idea of treating AI like this big shiny hammer, and then you’re running around trying to find all these curricular nails to try to beat down of how I’m going to use AI to do that, that is just such the wrong way to address AI, letting the technology lead first, you’re just going to make things worse and you’re going to stress yourself out even more. So what we want to do is we want to go back to basics. We want to think about backward design: at the end of this course, what should my students know? What should they understand? Then how am I going to assess whether they understand that? And then finally, how would I go about teaching that? And that’s where we decide whether AI might fit in or not. So again, it’s not the idea of trying to get AI in the classroom. I think as faculty members, we want to be really careful about how do we use it to be a creative assistant for us and kind of level up and improve our existing teaching practices. And the thing is, the book, all these teaching practices that we’re doing, these are well reasoned research techniques that have been laid out in chapters four through 10, backward design, ZPD, TILT, all the fun letters, UDL, all that stuff. But again, it’s trying to use that in a practical way, trying to develop your teaching skills more, and then just using AI as the tool to help you with that guidance along the way.

Todd: I wrote an article years ago. I was like, we just want to be well, that’s all we want. [LAUGHTER] So this concept of what we’re doing, if we think about what we do as faculty members out, as Lew pointed out, we’re chasing things a lot of times, and that’s just a problem. And this goes back to… remember when clickers came out, student response systems. Everybody’s like, “Oh, I want to use a student response system because my colleague’s using it,” and as faculty development directors, we are constantly trying to talk people out of using technology unless they actually needed it. Backward design. Why do you want to use it? Student response systems are always fun, because it’s like, I want to use student response systems and my friends and I would say, “Okay, so why?” “So we can gather student responses” and I’d say, “Okay, you know, you could just have them raise hands. Like, how many of you think this?” They say, “Well, yeah, but I want to know like anonymous.” I say, “Okay, have them close their eyes and raise their hands well.” “But I want to tally them.” “Alright, well have them close their eyes, raise their hands and then count them.” I mean, these things are not necessary. But then when they start to say, “Okay, but I want to know some qualitative aspects, what are they thinking and why?” Now we can use the technology. So I think that’s helpful. And so when you go back and look at what really takes time in teaching, those types of issues do, I can’t, like have everybody talk about muddiest point, because I don’t have time to read through those. Now you do, because AI can skim through those quickly for you. If you’re prep timed, you put in exhausted in there. I was exhausted because when I taught history and systems of psychology the first time, I would be up from 10 o’clock at night till two in the morning putting together a lecture. And I knew the stuff generally, but I had to go look for it. But I spent all my time looking. I was looking for the information. Then I’d find it, and then I would embed it. With AI I’m going to get the same kind of information, I’m just going to get it really faster, and because I can pull that together with the knowledge base I have, then I can put it out there. So prep time is going to drop, grading time is going to drop in terms of figuring out how to read through these types of things. Underpaid, by the way, I think you might have mentioned that one, just so we can’t do a lot about that. So sorry, yeah, for the underpaid part. But I want to finish with one quick thing here. There’s a book that I did actually get accepted for publication that I have never written. The contract was signed in 1994 and the reason I mentioned this is that what I wanted to do was take a stats book, but I thought we’re teaching statistics without kind of framework. So what if we took the old psych studies that were really the cool studies, took the data set, retrofit a data set basically that would give you the same data. Have students analyze the data, come up with what they’ve got, and then talk about what the data means, and then explain to them how that fits into like, that was the Bobo doll experiment, where Bandura had models hit a bobo doll, a blow up doll, and then they give the same doll to kids and see if the kids would do the same thing. You could replicate these studies while learning statistics. And I was told I couldn’t write that because most faculty members teaching statistics don’t know those studies. And so at the time, there was no way to do it. Now, with AI, you actually could, it would be very easy to say, I want to know more about Bandura’s Bobo doll study. And so I think that’s some changes that we’re going to have that are really going to be helpful.

Rebecca: One of the things that you’ve both underscored is leaning on the faculty member’s expertise because they can validate what’s coming through the AI tools. [LAUGHTER] So you’re not worried about the reliability, because you have that expertise, and the things that you’re suggesting people use are relying on that expertise to some extent.

Todd: Yeah, and that’s the thing. They’ve got the expertise. How can we help them with it? And help them by putting together a book that’s very modular. You can just go through and work through these things. Actually, the activities are set up, and Lew did this, I thought it was fantastic, they’re put in boxes. You could pull the whole thing out and make a workshop out of it. Very, very easy to pull this stuff out and do that, but by making these things very consistent, we know that reduces cognitive load, which is going to reduce some of that exhaustion and extra work.

Lew: And the other thing is, the examples were set up in a way, again, that they relate directly to what you’re doing in your classroom, your assignment, your example, not something that, again, the cognitive load that you have to translate from somebody else’s existing example. And again, that’s one of the neat things that AI allows you to do. Because now, instead of me saying in a book, do this example, I can put a little bracket there and say, insert your example from your discipline here, and all of a sudden you have individualized learning, individualized application for the teaching technique that you’re trying to do using the power of AI to do that.

John: And one of the things I’ve seen is that we have a lot of people attend workshops on AI just to learn how they can block it and how they can ignore it. But when the focus is on activities that actually allow them to do their work more productively or more easily or more efficiently, they get a bit more excited about it and then start thinking about how they could be using it to improve their courses and to improve their lives in general. And I think that’s a nice way of introducing people to AI who are initially somewhat resistant to it.

Lew: Yeah, it’s critical. I don’t think you can walk into a classroom and say, okay, students, we’re going to use AI today without understanding it yourself. And you hit the nail on the head there, the best way to do that is to do something that you are the expert in, and then learn to explore this technology on your own. It’s going to take a couple years for us to kind of get used to what’s going on. I think in a few years, we’ll be ready to think about how we might use this with students and things like that. But as a faculty writ large, we’re going to have to have a better understanding ourselves of just how to use those tools. And I think to your point about students misusing it and things, we can now have better conversations with them if we better understand how to use the tools ourselves for our own work.

Todd: And how people learn. [LAUGHTER] So we have tht published in the book too. That’s what it does. It helps you understand how these processes work, and so you learn about things like cognitive load. So then you can not teach cognitive load, but using that, build assignments that have lower cognitive load. That was one of the things that Lew came up… I love that one assignment that you came up with, Lew, is basically it’s you load the assignment in and says, “Where are students going to have high cognitive load in terms of really struggling with this assignment? And then AI says, “Oh, they’re going to stumble here, here and here.” And then you start rewriting it. It’s not a lot of extra work for you, and it helps the students. You don’t have to go through examples and have all those students lined up outside your door in office hours, saying, “I don’t understand why this thing works.” And so yeah, I think it’d fantastic.

Rebecca: I appreciate the idea of meeting people where they’re at, for sure. You mentioned something about a couple of years, maybe we’ll be ready to really integrate it with student learning. In five years, maybe, what does good teaching with AI look like?

Todd: This is kind of a fun one. I actually was asked to do a workshop recently that they were looking for a speaker, and they said they were interviewing different people, and they said, “We just want somebody come in and talk to us about what teaching will look like in 5 to 10 years.” I said, “If somebody tells you that they know what teaching would look like in 5 to 10 years, I would not ask them to give that presentation.” Because I don’t know, we used to talk five years out. Tom Angelo’s book with CATs was good for 20 years and still kind of worked. That stuff’s not going to handle anymore, because just in the last three to four years. Can you imagine, just before AI took off, go back just a few years and predict that it would look like this? So I don’t think we have a clue about what overall it’s going to look like. I do think we can look at some trends, though, and we can look at some directions we’re taking, because we can’t just throw our hands up and say “we don’t know,” but we want to be careful about what it’s going to be and it’s just going to be difficult, but there’s lots of good stuff coming out. And I’m going to actually ask Lew to do this, because earlier today he was explaining to me an article he’s reading. I think this stuff is fascinating that’s coming out.

Lew: Sure, I’ll jump in. So one of the things, not only do I keep an eye on higher ed and what’s going on with AI, but also the K through 12 space, and I follow a lot of folks on substack, it’s interesting. I think a lot of the discourse right now is going on in substack, as far as with AI and teaching and learning things. But anyway, there’s a fellow I’m going to try to pronounce his name, Nick Potkalitsky from Ohio, and he’s working with our Ohio Public Schools and writing about that. So the state of Ohio has mandated that by July 1, 2026 all school systems are supposed to have some kind of AI policy in place. So Nick is working with some of the larger school systems, Cincinnati, Columbus, Cleveland and things like that. And in this article he talked about on substack, he’s like, “Listen folks, this is going to be a three to five year rebuild. This is not going to be a quick policy fix. Rebuild means rethinking instruction, assessment, and curriculum, basically from the ground up. And how might we support teachers to make that happen and things like that. But teachers clearly need to be in that conversation. They need to be in that room.” This is the K through 12 space, and I think that’s only going to translate to higher ed as well. So the sooner faculty can get out there and start engaging in this technology and seeing what it works for them, then they will be able to kind of think down the road of how that’s going to affect their students in their classroom.

Todd: We have an epilogue in the book we did too after chapter 10, there’s a section in terms of what we think is coming along. And I just want to add one more quick thing here, when I think about what good teaching is, throughout my career… so I started teaching in 1984 so I’ve been teaching for a minute… and the concept here is that throughout all of that time, when I think about things I wanted to do, and what frustrated me were things that I couldn’t do because I didn’t have time or I didn’t have a way to do them. Universal design was just one example of that, but lots of things. With 400 students, I would love to ask them an essay question, but there was no way to look at that, even to get a sense of what they were writing. I would like to put students in small groups and looked at multiple versions of what they were talking about. There’s so many things out there. And now, as I start looking at what the future of teaching might look like, it’s kind of like one after another we just remove those barriers. So I don’t know what it will look like, but I know it will look better because it is removing things that we can’t do. And I will just end with my chunk of saying this is that we so quickly, when this came out, went right to cheating. Just immediately, the first articles out were, how are we going to stop students from cheating? How are we going to stop them? Students were cheating before AI came along. Don McCabe did a lot of work on this, and the proportion of students cheating in some way was off the chart. It made it easier, no doubt about that, and it does create a real problem, because if students can just jump in and do AI, then there’re many that will do that. But I don’t think we have to go there. And Lew and I talked a lot about this too, is that we shouldn’t just go there. We should start looking at opportunities. And I just talked to someone the other day who said, like, “How are we going to know when we can fully trust AI when it gives us an answer?” And that’s the kind of stuff that just drives me nuts, because when we talked about hallucinations, and so many faculty immediately say, “Oh, I put that in and it made a mistake.” So do I, sometimes. There are times when I make mistakes. I’ve heard my colleagues say stuff and I say, “That’s not true.” And so it’s no different than humans in that respect. We’re going to be able to trust it when we can look at it and we know it’s right, but it is getting better all the time. And Lew said it earlier in this discussion we’ve had here today too, is we can’t focus on what it was. We have to focus on what it can be. And I think looking forward for teaching, there’s some really cool stuff coming.

Rebecca: I think one of the things that you’re highlighting, both in your book and how you responded, is it remains anchored in the science of learning and just how we’re using the technology, and things might shift, but if we keep focusing on good teaching and learning practices [LAUGHTER] and just get them better, it’s not a surprise.

Lew: Well, it’s interesting, especially at the higher ed. We’re all trained in a very specific discipline, a very specific area. We’re experts in that field. We were never trained to teach, or very few of us maybe had that one course as we were TAs or something like that. And again, what the AI is allowing us to do is to further our expertise and our teaching and to kind of push the boundaries where we just didn’t have that training before. So I think, as Todd pointed out, I think there’s going to be a lot of opportunities just to make things better, hopefully.

Todd: Before we wrap up this question too, I’m just going to kind of put a charge out there, because Lew mentioned a little bit earlier, but in the model we have for adapt and stuff, for those of you who are listening, if you haven’t done it yet, just try dumping one of your assignments in for a topic or an area that you know students struggle with a little bit put into the AI. And the book’s got some examples on how to do this, but you could just say, “Where will the students struggle with this?” When I teach statistics, distribution of sample means, always give them fits. It took me years to figure out how to get around that, but if I was a brand new teacher and never taught before, I could put distributional sample means in there and say, Where will my students struggle? And it will tell you, here’s some places. And then, if you want to set that off to the side and then teach it and see where they struggled and make a comparison. And if it kind of lines up a little bit, you know it’s a gateway to kind of figuring out where the problems are going to be before they emerge. Wouldn’t that be cool?

John: So this is the first time you’ve worked together on a project like this. Could you talk a little bit about the working relationship and how that evolved?

Todd: Yeah, let me jump in there real quickly. Number one is writing a book is hard. I mean, it is really, really hard. And AI actually has helped us out with this too. So I wrote a book, this New Science of Learning first edition took two and a half years to write that thing. This book was nowhere near that, six, seven months, and we put a lot of time and energy into it. But the point is, it’s hard. Working with somebody is even harder because now you got to talk about what you’re doing and look at different points of view and everything else. And I think what was really cool with the way Lew and I kind of set it up is that we were coming from different directions. I had the learning theory. He was coming in from the AI, I would write a bunch of stuff, and he’d say, that’s really cool about all that metacognition. Maybe we can change it from like 3000 words down to, I think, 42, one time. And I said, I’m not explaining metacognition in 42 words. And then he gave me like 150 so we had to compromise and go through that. I would just say, working with him, he gets stuff done fast. He is really, really good at things in terms of understanding AI component. His prompting was amazing. I don’t know he’s a little mathematician-ey for me sometimes, but, you know, it was alright working with him. So all in all it was a positive experience. And I think Lew would say the same thing. So what’s the next question? [LAUGHTER]

Lew: No, it was a lot of fun. I really appreciated, as you’ve already mentioned, Todd is a very prolific author, so he was able to help us navigate all of those rough and challenging waters. So I was so much appreciative. But I think we complemented each other well, as far as what we brought to the table, and it’s exactly you know, what the book needed. And in particular, I remember Todd that GLCA, the Great Lakes College Association talk that you gave, what was the title of that? You remember something about neurodiverse learners?

Todd: It was basically how AI can help neurodiverse learners to be better learners. The title is way better than that.

Lew: Right, and Todd, he graciously used himself as an example, I will say. So a lot of things made sense to me [LAUGHTER] after that presentation Todd, [LAUGHTER] I’ll just leave it at that.

Todd: So, let me just say this. Yeah, I explained a little bit about ADHD and some of the characteristics of people who have ADHD, like time blindness, where you think something’s gonna take 40 minutes, but you don’t have a clue? Might be 10 minutes. Might be 17 hours. Who knows. You set up your task list, but you’ve never, ever finished a to-do list. I have never finished to-do list. And yeah, I went through all of those types of things and a couple of other things that I kind of struggle with. And then Lew when we finished, he says, it’s been great working with you, but I’m not sure if I’d heard that presentation first I would have signed the contract. [LAUGHTER] Well, you did it.

Lew: No, it was fine. It was fine.

Rebecca: So Todd, some of our long-term listeners might be curious what role opera played in this book. Didn’t you listen to a lot of opera last time?

Todd: Yeah, you know what? Let’s just throw this in there real quickly, because this is part of the ADH stuff that I’m learning, to whatever extent this fits in with this whole interview here. So ADHD is one of those things that, much like dealing with gluten intolerance and several other things, people think, if you’ve really got it, you must have known it your whole life. There are people who find out later in life that they have this thing. I always knew that kind of general things with ADHD which always struggled with a little bit. And I knew I had some of it. But, I mean, I was 60 years old when I was diagnosed, and so that concept there is suddenly I started to realize and read bunch of books about things like time blindness and some other things, but I had a terrible time writing. And a person, Tamara Rosier, who wrote a fantastic book, Your Brain’s Not Broken. It’s been reviewed in the New York Times and stuff, she said, “listen to opera, like, crank it up. Don’t do ear damage, but crank it up and do that while you’re writing.” And I said, “that will never work, because I have to have it quiet.” And she said, “Just try it.” And I put opera on and cranked it up, and I churned out work like I’d never turned it out in my life. And she pointed out to me that my brain’s always looking for stuff. Part of it would settle on that, and then it just kind of left this focus for the rest. Still had to have absolute quiet when I read. But for writing, that worked. I got put on some ADHD medicine, and two days after I started the medicine, I cannot listen to anything while I write. [LAUGHTER] To me, that’s fascinating, because what it now means is my brain’s actually kind of slowed down enough that that becomes a distraction. But that to me as a cognitive person, that differentiation is fascinating.

Rebecca: That is fascinating,

John: And that’s a nice preview of our next podcast that will be coming out next week. We’re talking to Karen Costa about ADHD and her experiences with that.

Todd: She’s really good.

Rebecca: So we always wrap up by asking, what’s next?

Lew: So I’m a mathematician by training, and I was just at a recent large gathering of many of our professional organizations, and for years now, they’ve been calling for, especially in like the high school setting, that we need to teach more statistics, data analytics, modeling, in other words, creating mathematical pathways, kind of beyond the calculus curriculum. The rub is, is that the way folks are trained, they’re basically trained for the calculus curriculum and not these other things. So what I’m doing is, I’m actually working with some organizations like Math for America to consider how we might take frameworks to increase the number of teachers that were able to develop these courses in these different areas. So what we’re doing is we’re looking at something I like to call AI-assisted retooling, so teachers are learning new content alongside an AI partner to expand their mathematical pathways. So, that’s kind of a really exciting thing that I’m starting to work with. We got a few case examples that we’ve been doing. I’ve been a case study myself of doing that. So, I’m really excited about that. And of course, you know, I’m continuing to work with teaching centers on workshops and things around the book.

Todd: Nice. For me, so I got another book started, because why not? These are always fun. This particular one’s going to be different, though. So I’ve probably said about as much as I can in terms of some of the teaching tips and kinds of things we do. I really think that what’s going to happen pretty soon is there’s so much stuff coming out with AI, real life stuff is going to become something I think people are going to cherish. So the project I’m working on right now is looking more at how I’ve taught in different aspects of it, but what it really is going to boil down to are things that I experienced in the classroom. They’re really kind of unique, interesting things, and then how it played out and how it could be different. And so I had a student one time have a grand mal seizure in my class. And the quick example here is I quickly ran down to a nursing station. It was just around the corner. I ran over there, got somebody, brought them back, and the nurse came back, and we came into the classroom. And I, right now, would tell you… that was my first year of teaching that happened… I would never leave a classroom when an emergency is going on. I should have said to somebody, “Hey, go to the nursing station.” So what I want to do is talk a little bit about what I’ve done, some of the ways I’ve done it, how I would have done it differently, and then how that ties into learning and different aspects. So I really want to make more of a personal story. I don’t know if anybody will be interested in it, but, you know, we’ll give it a try.

Rebecca: Sounds like an interesting book. I’d read it.

Todd: Good. I just sold one. Oooh.

Rebecca: Yeah. [LAUGHTER]

John: Maybe two. [LAUGHTER]

Rebecca: Yeah, maybe two.

Todd: No worries, but thank you, appreciate that.

Rebecca: People have anxiety about, like, what if X, Y and Z happens in my class, and hearing personal stories about how those sorts of things have been addressed and strategies for addressing them, I think eases some worry for folks, so it could fill a gap.

Todd: Yeah, I appreciate you saying that, and I think one of the big ones is that it’s been about 40 years of teaching, and the 40 years of teaching, I think the framework I’m gonna go with, I’ve made almost every mistake you can make, and I still am seen as kind of a person who’s written these books. I do know a lot about teaching, and I’ve learned a lot about it, but you don’t become a better teacher without making mistakes along the way. I don’t think anybody can. And so I’ve done that. And I probably should point out, too, as Lew said, I run around some campuses. I love meeting people and going to campuses. I’ll still keep doing that.

Lew: So Todd, three years ago, I did a column for the MAA, it’s the Mathematical Association America. It’s called Still Learning From My Misstakes. And mistakes was spelled with two S’s, anyway, but yeah, exactly that, that idea of, I think, showing your vulnerability, especially at the part in the career that he’s at, like, oh, Todd Zakrajsek does everything perfect. I think peeling back the curtain and showing the mistakes that we made for the folks who are coming up, that is so refreshing to see that and what we did well and what we didn’t do well, and then kind of examples of moving that forward. So yeah, maybe three books, you might sell three. [LAUGHTER]

Rebecca: Definitely worth the investment now. [LAUGHTER]

Todd: Lew, that will cover my next tea. [LAUGHTER] But thank you, appreciate that.

Rebecca: Well, thank you both for joining us and for sharing your work.

Lew: Thank you so much. Appreciate it. It was fun.

Todd: Appreciate the opportunity.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

424. Teaching with AI

Posted in critical thinking, edtech, educational technology, engagement, feedback, higher education, inclusion, instructional communication, learning, pedagogy, teaching
December 17, 2025

The rapid evolution of AI tools provides a challenge for educators exploring educational applications. In this episode, José Antonio Bowen joins us to discuss ways in which faculty and institutions can better prepare students for their future lives and careers in a world in which AI tools are ubiquitous.

José has a joint PhD in musicology and humanities, served for several years as President of Goucher College, and is the author and co-author of several books, including: Teaching Naked: How Moving Technology Out of Your College Classroom Will Improve Student Learning; Teaching Naked Techniques: A Practical Guide to Designing Better Classes; Teaching Change: How to Develop Independent Thinkers Using Relationships, Resilience, and Reflection, and a co-author, with C. Edward Watson, of the first and second editions of Teaching with AI: A Practical Guide to a New Era of Human Learning.

Show Notes

  • Bowen, J. A. (2012). Teaching naked: How moving technology out of your college classroom will improve student learning. John Wiley & Sons.
  • Bowen, J. A., & Watson, C. E. (2017). Teaching naked techniques: A practical guide to designing better classes. John Wiley & Sons.
  • Bowen, J. A. (2021). Teaching change: How to develop independent thinkers using relationships, resilience, and reflection. JHU Press.
  • Bowen, J. A., & Watson, C. E. (2024). Teaching with AI: A practical guide to a new era of human learning. JHU Press.
  • Bowen, J. A., & Watson, C. E. (2026). Teaching with AI: A practical guide to a new era of human learning. 2nd ed. JHU Press.
  • ChatGPT
  • Gemini
  • CoPilot
  • Claude
  • Hume AI
  • Sesame AI
  • Latimer
  • Mike Caufield (2025). “Deep Background GPT Released.” June 16 substack post.
  • Apertus – new Swiss AI platform
  • Storm (from Stanford)
  • weteachwithai.com (website accompanying Teaching with AI.)
  • Undermind
  • FutureHouse
  • AAC&U Institutes
  • Nano Banana AI
  • BoodleBox
  • Poe AI
  • Comet AI browser

Transcript

John: The rapid evolution of AI tools provides a challenge for educators exploring educational applications. In this episode, we discuss a resource that can support faculty in this process.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is José Antonio Bowen. He has a joint PhD in musicology and humanities, served for several years as President of Goucher College, and is the author and co-author of several books, including: Teaching Naked: How Moving Technology Out of Your College Classroom Will Improve Student Learning; Teaching Naked Techniques: A Practical Guide to Designing Better Classes; Teaching Change: How to Develop Independent Thinkers Using Relationships, Resilience, and Reflection, and a co-author, with C. Edward Watson, of the first and second editions of Teaching with AI: A Practical Guide to a New Era of Human Learning. Welcome Jose.

José: Thanks. Good to be here.

John: We’re looking forward to this conversation. Today’s teas are:… Are you drinking tea by any chance?

José: Coffee. Sorry.

John: That’s not uncommon, that and water are probably the most common teas.

José: I had tea this morning already, though, so…

John: Okay.

Rebecca: Alright, so you’re not anti tea. Alright. [LAUGHTER]

José: I lived in England for five years. How could I be anti tea? [LAUGHTER]

Rebecca: Right? [LAUGHTER] I have a Constant Comment today.

John: And I have a ginger peach green tea today.

Rebecca: So we’ve invited you here today to discuss the second edition of Teaching with AI. We ran a very successful faculty reading group on the first edition of this book in fall of 2024,
and since the first release of Teaching with AI in late April 2024 we’ve experienced extremely rapid change in the development of generative AI platforms and tools. What were some of the changes that prompted this relatively rapid release of a second edition of the book?

José: Wow. Yes. I mean, first of all, the technology has continued its march, and so we are seeing pretty steady improvement and diversification of kinds of tools and platforms. So we initially started a website with a list of tools. There was the big three and a few others, and now they’re probably 18 to 20 foundation models that are all over the world. When we wrote the first edition, there were not all of these cultural models. There were not Arabic and Chinese models and Spanish models, and there’s a new Swiss model this week, which speaks 1000 languages, we didn’t have Latimer, which is trained on black and brown sources. So there are more models, there’s a lot more tools, and every week seems like we get more of them. But it’s also the case that people’s attitudes have changed, student use has changed, faculty use and resistance have changed. But a lot of it was also that we just spent a lot of time visiting campuses. So I think each of us visited 100 campuses this year alone. So that’s a lot of ideas for assignments, things we hadn’t thought of, that “Oh, I’m using it to do this. I’m trying it to do that.” So we always intended this as a practical guide. There are lots of assignments, lots of assessments, and there’s a lot more of that now. There’s a whole new chapter on chatbots, because that wasn’t really an easy thing to do when the first edition came out, and now there’s hundreds of platforms that do that. And there’s also an extra chapter on literacy and ethics. We were initially a little scared to write a chapter on literacy. I mean, partly, we didn’t have time. I mean, it was just we were like, “We got to get this out.” And also, I think we didn’t know what it meant at the time. It was brand new. And now we have literacy, we have fluency. So this time, there’s enough, I think, research and evidence out there for us to say, “Okay, here’s how people are thinking about this.” And so there’s a whole new chapter on literacy, and there’s an extended chapter on sort of role play and feedback. That’s less about tools, although the voice tools have really… we didn’t have that in the first edition, the ability for voice and also emotional voice tools like Hume and Sesame that really do understand the tension in your voice and respond in an appropriate way that feels and sounds human. So role playing is a bigger thing. I could go on and on and on, but so much has changed. I thought we should do it as an annual, [LAUGHTER] but I don’t think I can do that. I think that’s too much work.

John: And it will probably stabilize to some extent, as most other tools have, at some point in their evolution.

José: We thought so, but the first three years, it certainly hasn’t yet. I’m with you. I sure hope so, because it’s impossible to keep up.

John: In your earlier Teaching Naked books, you focus on the importance of maximizing human connections in the classroom and using appropriate technology for student work mostly outside of the classroom. Do you still advocate a teaching naked approach in the current environment, with the ubiquitous availability of AI tools?

José: Yes, mostly, I think there are going to be places in the classroom where you have to bring AI in, the same way that you can’t teach digital literacy if you don’t have the tools in the classroom. I do think there are a whole class of assignments where I want students to do things in class and talk to each other. An example that I’ve borrowed from Mike Caulfield’s great work, or sort of altered, but used his tool, is that I have students come up with a list of controversial statements. What are some things that might be true, or does the Earth have two moons? And then the students now write a prompt, an 800-word prompt, to test this group of ideas. If I said, “Write an essay. write a prompt,” sounds a little bit more meaningful. I’d let them go do that at home. Go write your prompt, and then they come back. And so now everybody has a prompt, and now I say “So now get up with a partner and pick a controversy and test it with your two prompts. See who wrote the better prompt that actually gets underneath and does this thing well.” Then I show them Mike Caulfield’s Deep Background, and I said, “so his is better.” So I don’t show them the prompt. I just say, “Can you alter your prompt to get it to do this, to make a table, to divide the question into sub questions, to trace back the origins. What are the other things that you didn’t think of? Do you need a chart that with a scale of “Well, some statements might be true on a scale of one to five? This is definitely true, definitely not true. But what about the maybes?” So there’s a lot of active learning in class, but the tool is still with us in class, in the same way that I don’t think we should ban PowerPoint from the classroom, but I do think we should ban PowerPoint slides that have 400 words on them. Put a picture, that’s good, and six words that’s, I think six words still works as a maximum, and then write that. You use that as the way to kind of focus the conversation. “Oh, that’s what we’re talking about. We’ve kind of wandered off, but there’s the picture and the six words.” So I think there’s a place for some use of technology in the classroom, like I always did. But I still think that if you’re going to have students drive to campus, get parking and take the extra investment to be face to face, rather than be online, you have to give them something different, something they cannot get online, and that’s the smell and the feel and the touch and just the presence of other people and the ability to have those kinds of conversations and look how people are responding. I still think we should be using class for that. I do think that, as with the internet and now with AI, we might not need to meet three times a week for 50 minutes, especially if you’ve got over 50 or 60 students. If you’ve got hundreds of students, you do not need to meet together with 600 people. My suggestion is, let those 600 sessions… let that happen online, tape your videos, your lectures, whatever, but then meet with students in small groups of 15 or 20. So maybe that’s only once a week. Maybe I have my 600 students and I have my four TAs. So maybe what we do is we only meet in groups of 12 face to face once a week. We could radically rethink what we do with that face time, and that’s kind of what I proposed in Teaching Naked and so I still think we could do that. And now, instead of those lectures, maybe it’s an AI tutor that does the out-of-class work, but I certainly don’t need the AI tutor in class.

Rebecca: One of the main concerns that faculty raise about generative AI is the concern over academic integrity issues, so faculty sometimes opt or wish to ban the use of generative AI or focus their efforts on trying to detect its use. Why is this not a very useful strategy, and what should faculty maybe be thinking about instead?

José: There are two things. So the first is faculty should be concerned firstly, with learning, not with academic integrity, right? Learning. So if students are not learning, the problem is that they’re not learning, and I need to rethink how they should be learning. The second problem, though, is that the phrase “academic integrity” implies that we have a special integrity. It’s not like your integrity, our integrity, right? And the minute you say that, it’s like, “Well, then in four years, I can forget about it, right?” It’s only academic. So why not just talk about integrity? Why not talk about authorship and the importance of human contributions? And by the way, what we call cheating, business calls progress. So the first thing to realize is that if the product your human students produce is no better than AI. You’re not doing them any favors because they won’t get a job. Yes, there are things… we have to learn how to add, even though the calculator will do that, but there are other reasons for that, right? We’re really learning number sense. But in fact, most accountants are going to use a calculator. They’re not going to add up long columns of numbers. The learning to add is part of the process to get us to a kind of thinking that moves us along. So if we’re clear about, oh, this is the reason you’re learning to add when the student says, “ Well, no accountant is not going to use Excel spreadsheets or computer or calculator, why do I have to learn how to add?” Well, the answer when I was a kid was, so you get the right change when you buy a candy bar at the drugstore, [LAUGHTER] and now they don’t even make pennies. Or they used to say, “Well, the patient will die. If you can’t do long division, you won’t be able to give the right kind of meds.” No doctor anywhere does long division in the hallway on a piece of paper. They all use an app. Now it’s so you don’t put as many zeros on your taxes, right? You want to be able to estimate so that, oh, I put the numbers in wrong. This doesn’t look right. This is too many milligrams. So the ability to have number sense still matters, maybe matters more. The process of learning to do those things matters. Is that really an integrity issue, or is that a learning issue? I’d much rather frame this as a learning issue and demonstrate to students that this learning will be useful. At the same time, I want to raise standards in the same way that a spell checker raised standards. It did not make me a better speller, but it did make my spelling better, and that was sufficient. So now one in three fortune 500 companies require Grammarly for every communication in and outside of the company. So the requirement that you don’t have spelling mistakes is now pretty common, but I don’t grade spelling like I used to. So those 10 points that I used to give for spelling, I now put on the thesis and the thinking, I put it someplace else. So AI is going to make us do the same thing. We have to rethink what is essential for students to do, and how will I encourage them to do it and motivate students to do it? And how will I raise standards so in fact the end result is something that neither AI nor student alone could do, in the same way that my accountant could not do my 700 pages of taxes without a computer program. She simply couldn’t do it, but she does know how to add, I think. [LAUGHTER]

John: But she doesn’t need to.

José: We do talk numbers all the time, and she’s estimating things. That kind of thing happens all the time. I’m glad.

John: A related concern, though, is that perhaps students are using AI too much at the early levels of skill development, so that they’re not developing critical thinking skills and disciplinary expertise to be able to evaluate the output of generative AI. What are some strategies that we can use to help students develop these skills while working on learning how to use AI tools effectively?

José: The first part, is yes, they absolutely are overusing it. But back to your cheating question, instead of calling that cheating, maybe we should call that overreliance or misuse. Is it really fraud? Is it really intentional deception? Cheating is a very loaded term, and so a student might not necessarily know when they’re overusing it or how they’re undermining their own learning by over reliance or misuse. On a practical level, there are two things that you need to use AI well. One is expertise does matter, that being able to say, “Huh? I don’t think that’s right.” I often say that AI literacy is fundamentally two things. One is asking better questions, and the other is evaluating answers. The two things that we all do, the core of a liberal education has been those two things for 1000s of years, so we all already teach AI literacy, but how do I get expertise if I have this tool that pretends to have expertise? It’s kind of the same question as the calculator problem. And I know the calculator is not a perfect example, but it’s a good example of this. I still need to know how to add to know when the calculator might have made a mistake, and so even though the calculator can do the work, I need to have that experience. So the calculator challenged us in two ways. One is, do I still need to teach long division and addition of long, long lists of numbers? The answer to both of those is not much. I should probably do a little bit, but I don’t have to spend six months like I did just adding up long series of columns. So I have to change what I teach, but I also have to think about the sequence. When do I introduce the calculator? So the question for us is not, do I introduce AI? The question is, when and how and what are the skills that I still need to preserve? Spelling is another one of those. I’m able to write books with a spell checker, and if I had to look up words in a dictionary still like it was 1970, I would be much slower… if I was still using index cards for my bibliography… So, I no longer have to teach the card catalog. Are there skills that I can not teach in order to add the other one? Academics, we love addition, we hate subtraction. [LAUGHTER] We don’t like taking stuff away, but we’re going to have to take something away. But that’s a big question. How do I do that? But I do think we’ve been teaching digital literacy for a while, and we should add AI to that. So the question is, is this true? Although, again, Mike Caulfield’s version of this is, “is this what you think it is?” is a great better question. Is this what you think it is, especially now that you can’t tell because it has six fingers. There’s no more “Oh, it has six fingers. I’m going to show you how to identify AI pictures.” Well, that doesn’t work anymore. So understanding what’s the source? What are the kinds of claims? Do other people say this too? So that’s probably a separate skill from expertise, just finding out what’s true. And again, all of us are doing it on a daily basis, because the news is full of stuff that it’s like, “Is that really true?” And “What’s the source for that?” and “is that picture really real?” So we’re all having to do that. So that should be easy, but the expertise is the harder part. How do I get you to be an expert if the junior level jobs and internships are going away. And so I think that that is a primary question that we should be thinking about. And my rudimentary answer is we need a lot more talk about motivation, a lot more talk about process that “yes, you will have a calculator, but the process of learning to add and to multiply is valuable, and here is why.” And so here’s what we’re going to do. And then I promise you, we will introduce the calculator. It is coming, but you will not be as good a user of that if you don’t first do this. And my analogy to that, so I teach music, and so people wanted to learn how to play jazz pian, and so you would give them these timed tests. I’d say, “okay, E flat minor, what key are we in?” And they’d have to write that. And so those were, like, those old multiplication, you know, 60 seconds, and you’ve got to, like, scribble as fast as you can. “I want to play the piano, why am I…” I said because, if you have to think about what key E flat minor is in, and how many flats that is, the band is already… that time has to be immediately in your fingers. And so in order to do that, I’m going to give you what feels like really rote memorization. Five flats, by the way, is the answer. And unless you see E flat minor and immediately go three, two, and you have that hand ready to go, you can’t converse with me in jazz. So I had to explain to students why I was making them do these, what appeared to be really dumb, like rote third grade math tests, because that will allow you to speak with me. When you see a piece of paper and we’re playing in real time and it’s going, click, click, click, click, E flat minor, F minor, you can change key. You know how to do that because it’s automatic, and so I have to make it automatic for you. So notice how much time I spent convincing you that you had to know how many flats. So that explanation is not something we’re always good at. So that’s not a great answer, but I think it’s the right question.

Rebecca: What you’re talking about is reminding me of a conversation I had with my graduate student a couple weeks ago about changing disciplines and fields and learning the language of a discipline, and how much that actually allowed him to communicate about what he wanted to communicate about this whole time. And so he’s worked on a really great presentation about this [LAUGHTER] exact topic that kind of becoming fluent in the language of the discipline allows you to communicate things that you couldn’t communicate before.

José: And that you can do with AI. There’s a great tool from Stanford called Storm that actually has a little button that allows you to change the disciplinary focus. And so one of my assignments is,”Okay, so pick a topic, pick a question, and go to Storm and then look at it through three disciplinary lenses, and then write an essay about what is the difference? What did you learn? What are the differences in how anthropologists and historians think about this question? So I do think that’s an important piece. And again, it’s often not explicit. We should make it more explicit. In the same way we should make the critical thinking discussion more explicit. For years, we’ve just said, “Oh, they’re learning that. They’re learning critical thinking.” It’s like, “ah?” We did a great study four years ago, and the students were like, “Critical thinking, what’s that?” You just spent four years doing it. It’s like, “but you never use those words.” And so adding the words, “This is critical thinking,” turned out to really help students. So we’ve got to be more explicit about that disciplinary knowledge and why it’s useful and how we’re learning it. And I remember the light bulb in graduate school. I remember sitting in that talk when it was a guest speaker, and all my professors were there trying to outdo each other, and it’s so “Civilization and culture.” And the other guy said “Apollonian and Dionysian.” And it’s like, “Oh, they just want to use their own terms.” It’s like, “Oh, my God, they’re not really arguing. They’re just like, trying to see who’s top dog. And so I remember [LAUGHTER] that light bulb going, “Ah, please.” [LAUGHTER]

Rebecca: Can students use of AI tools help them reduce the equity gaps that students have faced in their prior learning experiences. This kind of goes back to John’s earlier question a little bit.

José: Yeah, so I think the answer is yes, but we have to be careful. So the first is that AI has bias. We have to make sure that everybody knows this, but it’s also the case that humans have bias, and if we were ignoring human bias, we weren’t doing anybody a favor either. So it turns out that AI bias is a little easier to fix if you know what you’re looking for. If I know that AI tells me I should be earning less money because I’m a woman, I need to tell it to respond to my resume like a man. [LAUGHTER] How much should a man make in this job? And then that’s a useful tool. So understanding the bias is even more important than it used to be, because now I can say, “Oh, well, how could I correct this?” My example is, if I said to you, “so rank the candidates for the new position of history professor that you’re searching for,” can you ignore the prestige of where they got their PhD? Uhhh, that’d be “no,” because you’re a human. It’s like, you know, don’t think about the cookies. You can’t not think about something I just mentioned. But in AI, I could say, just rank the candidates based solely on the quality of their teaching, review the faculty based solely on the quality of student feedback. I can do things as a check on equity. I can also do things like so I can make you a bot. So you didn’t take calculus that’s required for this course, or most people have had it. So I’ve created this bot that will help you figure out what you’re missing, or create a bot that will help me, I didn’t take biology before chemistry, and I could do individualized tools. And AI is pretty good at learning, “Well, you tend to make this mistake a lot. This is a grammatical mistake you always make, and so you made this in the last paper,” and one that people don’t think of, which is that most of us, well, all of us, because I do it too, and everybody does it. All of us provide students with way too much feedback at once. [LAUGHTER] Here are 37 things to fix about your paper. And the truth is, you can only do one, one at a time. After you fix that. I’ll give you something else. This hit me when I was trying to relearn tennis a few summers ago, and it was like, move your feet, hold the hand, the grip. It’s like, “Shut up. Just give me some more balls. One thing at a time, let me just put my feet right, and then… So I use AI to take my feedback and what is the most important one thing the students should fix for next time, and then save the rest, because I overdo it. And so I can tell the AI not to give the student too much feedback at once, and it can do it. And I have a… all of us, I think… have a really hard time doing that. So the potential for AI to close equity gaps, to help us see bias, to make more ethical decisions, is there, but it was not going to happen automatically. We have to be very, very careful and very aware of the kinds of things AI gets wrong and the kinds of biases that are built into it, in order to make sure that doesn’t happen. But as a check on human bias, it’s really a pretty amazing new tool that we didn’t used to have.

John: In terms of that issue of feedback, when students are given 20 or 30 different things to correct or to improve in a paper and assignment, they will often choose whatever is the easiest, which is not always what is the most important. And that idea of focusing on the most critical or most important type of feedback makes a lot of sense.

José: Well, I can give an example. So this is a real example from a PhD student who’s like, comes back to me and says, “Oh my god, I handed to my dissertation to my three readers, and I got three different types of feedback. My God, they don’t agree. Is this normal for you people?” Uh, yes, [LAUGHTER] you got contradictory feedback. [LAUGHTER] Congratulations. Welcome to the club. And she’s like, “Well, what do I do? One of them wants me to cut chapter two. The other one wants more. What do I do?” I said, “So, you put the whole thing into AI with the three letters, and you say, ‘so, make a table list all of the things they want me to fix, from easiest to fix to hardest to fix, and then list: Does everybody agree that should be fixed?’ Check, and then another column, if they disagree. What’s the easiest way to make them both a little happy?” Graduation. So easiest to fix, the hardest to fix. So fix the easy stuff first. That’s fine, but an AI here could keep you on track to make sure you actually prioritize. So yeah, here are the three easy things you should fix, because that spelling mistake was terrible. But here is the real issue, the one real thing you should focus on. And I would say students leave a conference with me, and I didn’t know this until AI, but now they go to the AI bot and they say, “Okay, what’s the one thing I really need to fix?” Oh, my God, I didn’t make that clear. And I know that because I can see them talking to my bot about the conversation [LAUGHTER] they had with me.

John: With all the changes that are occurring in terms of the development of AI, we’re beginning to see some shifts in labor markets that are not that well defined yet, but it’s very likely that many of the jobs that students were preparing for in the past may not be there or may not be there in the same way going forward, and jobs are very likely to be changing in the future. I think you’ve already partly addressed this, but how can we prepare students to thrive in a world where the nature of the tasks that they’ll be doing might be changing fairly dramatically in response to more widespread use of AI in many occupations.

José: So, you know, I’ve been saying this for a while, but the future has been unknown for a long time. And so first you said, “Look, your major has never mattered less. I don’t know what the jobs are going to be in five years.” In fact, when I wrote Teaching Naked, my daughter was still in college, and then she became a director of social media, which was not a joke I put in the book because it hadn’t happened yet, but there’s like, oh, wait a second, you could be a director… this didn’t exist five years ago, and now you can direct it. So this isn’t new, that new jobs are changing and being created all the time. The internet destroyed jobs and created lots of IT jobs, but it’s impossible to guess what those are going to be. So in some ways, that’s great news for those of us in the humanities and liberal arts, because it means that the thinking is going to matter. Good use of AI is creative thinking, asking better questions, evaluating answers. Computer Science, it’s hard to know which exactly to do. But that was also true 20 years ago, your degree in the iPhone I was not going to be useful forever, unless you could figure out they’re going to be new iPhones. And so adaptability, flexibility, has always mattered. It’s just it was slower, and now we’re seeing it’s happening more quickly. It is very hard to know what the job market is going to look like, whether it’s going to be we all work three day weeks, whether certain classes of jobs go away. Is AI really taking jobs? It’s clearly going to take some jobs, but the research says that every job is going to change. There’s at least one task. And so for faculty, my first thing is find out what you no longer need to do, whether it’s making a makeup test, grading, drafts of accreditation reports, assessment of student for accreditation, assessing student learning. There’s got to be something that you hate to do that AI could already do better now. My doctor no longer sits at the computer and types, my doctor now gives me eye contact, which is a little creepy, [LAUGHTER] but the evidence says that doctors who use AI transcribers to do the nurse’s notes, the insurance notes, whatever, that they have more cognitive load to focus on patients, and they’re happier, there’s less burnout. So this is a good thing, right? Because no doctor went to insurance school, they went to medical school, so insurance notes are not part of the job I love. So could AI help with that? And my doctor says the AI is now better, the first six months, he checked it and checked it, but it turned out he was looking stuff up, because the AI is better at it. So what could faculty do that AI is already better at? The department schedule, chairs? Because that’s what’s going to happen, or the bibliography, the literature review, keeping track of what are the latest trends. So that should give you time to do something else. And for faculty that will learn a bit about AI, because you’ve got a few hours back, but I think for students, the question is, how can you be constantly thinking about what else you could do, how your job is changing. I’ll give you one example. So in the old days, if I had a new product or I needed to know how to price my product, I would send a group of interns out to various cities to do focus groups. Go find 12 people at the supermarket and give them a taste, see if they would pay $4.99 for 12 ounces of this. Now I can do that with AI. You’re a 35 year old suburban soccer mom and you live in Oklahoma, and it can give you the synthetic personas to do marketing. So on the one hand you say, “Oh, I’m going to lose my job.” Well, no, because you’re going to go back to your boss and say, Okay, boss, so instead of doing 10 focus groups. I did 1000. I did them in every state, and I did them by year. I did them for every year between 30 and 40, and I did them for rural. And so that’s actually happening. So somebody is setting up synthetic personas to do marketing focus groups for their products, and they’re now able to do work of much higher quality than they were. So there’s still a job, but the job is now different than it was, and so I would classify that as creative thinking that’s raising standards but also how could I do this, not just more efficiently, how could I do this better? What do I want? I want to know how much to charge for my product. What could I do to get better data? And the answer is, “Oh, I could create 1000 synthetic AI personas to do this market research and go back to my boss with way more data than she was expecting.” Those are the assignments I like, too, right? So I might give students an assignment and say, “Okay, so you need to figure out a way to use AI to do something you couldn’t do by yourself. How do you make this better? What else could you now do that you couldn’t do before?” And that’s thinking, that’s the critical thinking, part. And so the process becomes important. It’s not just did you produce this thing, but how did you think about it? How did you make sure it was right? How did you validate? Those are going to be really important questions in the future. Did you validate? Did you make it better? We should be able to do that. We love that stuff.

Rebecca: One of the things that we’ve talked a bit about today is the rapid amount of change, change in what jobs look like, changes in tools. It’s hard to keep up…

José: Impossible.

Rebecca: Right. So what are some low effort and high impact AI practices that faculty might explore if they want to get started, but haven’t quite gotten there yet.

José: Well, the first thing I think you need to do is to try it in your own life. I actually think that just try an AI browser, try shopping, trying to see if AI could help you buy a new subwoofer, buy appropriate gifts for your kids or whatever. I have a really hard relative, Uncle Larry, plays the ukulele, and he likes whiskey, but I got him that last year. Give me 10 ideas for new gifts. Link me to the website. Shopping is really changing, so just finding a way to use it. So we have this website: weteachwithai.com. We have lots of models, lots of tools. Try a literature search with Undermind. That will change your life. Try FutureHouse, if you’ve not tried that, if you’re a STEM person. Try some of these tools. And start to think, “Aha.” Because if you try a disciplinary tool, right? We talked about disciplines earlier, you’re think, “Aha, this is going to change how experiments are run. This is going to change how literature search is done. This is going to change how verification is done.” And now you have the basis for rethinking your assignments. But the easiest thing for faculty to do is to take your syllabus, and if you’re nervous about putting your whole syllabus in, then start with your academic integrity policy, take your cheating policy, put it into AI and say, evaluate this as if you were… and then put the description of the students in your class, right, 18-year olds at Nebraska City College, whatever, put in the description of your students and then say what might be insensitive or unclear. How could I make this policy more motivating or more meaningful to and then describe your students. That works for every assignment, it works for your syllabus. In fact, one thing you can do is build a course profile. So I build a course profile. So I have 64 students. I have three who have this accommodation. I have this student who didn’t take calculus. I put in the details. Then maybe even I asked them what they like. I’ll come back to that. And then I say, so is there anything about this reading list or this assignment that is not going to feel inclusive to my students. I’m teaching engineering, and I didn’t used to have a lot of women. Now I have more women, but I’m using the same syllabus from 1975, could I change some of the examples so they might be more inclusive of a more international or gender diverse group of students? So get feedback from your students. It’s a focus group, essentially, right? Because you don’t know… you’ve been here too long. You like school so much, you’re still here. You’re weird, all of those things. So if I want to know how an 18-year old would respond to my assignment, I need to find an 18-year old. So those of us who had kids go through college found that very useful. It’s like, hey, what do you think of this? But once they graduate, it’s like, “Oh, my focus group is gone.” So now I could use AI to get feedback. So a course profile is a great way to do that, and that’s low-hanging fruit.

John: So you’ve suggested a few ways in which faculty can use AI to improve their teaching. Are there other things they can do to update their courses, to make them more inclusive, etc?

José: Yeah, I have a prompt on the weteachwithAI website that says. “So here’s my syllabus. I want to update my syllabus. I’m really skeptical about AI, but I know students are using AI. So what are some things I need to know? How do I update my policies? You give me 10 ideas for how to create new assignments that might prepare students for this new world. And by the way, you’re an expert.” Learning to prompt, right, recognizing that you have to tell it it’s an expert, is useful. So I think that’s a pretty easy thing to do. Another thing you could do is customize your assignments. Ask AI, “I need an idea for a 10-minute activity to break up this lecture. I need an idea for new assignments, or I have this assignment, but I have 50 students. I know they have different interests. Not all of them are majors, some of them are pre-med. Could you create different versions of this, one version of the problem set for pre-meds, one for this, one for that?” Because nobody really cares about Train A leaving the station and going 20 miles an hour faster than train…. Right? Nobody cares. So why not? The Republicans are registering 20 voters an hour and the Democrats are registering… It’s the same math, but the research says the students were going to do that problem more because it feels like a poli sci problem, or the wide receiver you’re covering runs 30 miles an hour and you’re only…, that feels more like a football problem. And so it turns out that matters. So you could even go to the extent of customizing every assignment for every student. Do a survey before class or the first day of class. So give me an alias. I don’t have to put your name in the AI, but give me an alias and then tell me what motivates you to do your best work. What do you want to be when you grow up? What are your favorite colors? What’s your favorite sports team? What are your favorite hobbies? And then I feed my generic assignment with that and the spreadsheet, I say, now create an individualized assignment with a unique link that you will then email to all of the students individually, that’s a customized version of this assignment. You can also do this in class. So an example: “So write a five page paper on Hamlet.” Okay, so students are like, “Oh, really?” So first what I want you to do is, I want you to tell the AI a bit about what really motivates you. Tell it about your passions for baseball or dancing or whatever it is, and then ask it, how could you customize this assignment so that you could write a paper about Hamlet that overlaps with what really motivates you? Could you write a paper on Hamlet and baseball or Hamlet, using analogies for whatever, and so AI is actually really good at that creative back and forth, because it hallucinates. It makes stuff up. It’s creative. So that’s the kind of way where I can get students to increase their motivation, more customization of their assignment, because they’re using an AI to create and again, I couldn’t do that at scale with 600 students, but with AI, I can customize 600 assignments just as fast as I can customize three.

Rebecca: You’re starting to hint at this, but if faculty implement some of the strategies you talked about today and are in your book, what types of changes would you expect to see in your classroom and in the student learning outcomes?

José: Well, I think that we have two problems. One is that we charge a lot of money for something that’s now out of date. If you’re still delivering content in a lecture, there are cheaper ways to do that. If you’re giving a credential, there are cheaper ways to get the credential. But if what we’re teaching is an experience of critical thinking, of being with other people, of learning about diversity, we should double down on that part, and we can outsource again, not unlike I talked about in Teaching Naked and the internet, but it didn’t quite happen with the internet, right? But we could outsource some of the content acquisition. They could do things outside of class to get first contact with content. And so I do think that classrooms will probably physically change. We do not need 600-seat lecture halls anymore. We need smaller classrooms. We may not need to meet as often with students, but we do want to meet them in smaller groups. So I think that’s going to change, but I think more face-to-face human reactions. If you have an asynchronous course that was mostly online or asynchronous, most of those students are within a few 100 miles. They are fairly local. I was a consultant for one state, and they were having this problem because the students, they’re spread out. And I said, “Well, why are you moving the students around? Move the faculty around. You have seven campuses in your state. The faculty could go to a different campus a day and meet with students, and then the rest of it is online already.” So we have to think differently about what our classrooms look like, what we do in those classrooms. But I think that’s going to be the harder change. Certainly, AI could do grading of papers. There’s a whole lot of caveats to that, but AI could do content delivery. Instead of having to read the chapter, work with my bot that’s interactive, an interactive textbook instead of a midterm. Maybe I have you talk to Einstein, and when Einstein thinks you know enough about thermodynamics, the chat bot puts a grade in the grade book. So those things I think could be automated if we do them carefully. The bit we can’t automate is the “I believe you can do it” piece. “I believe in you, and I’m a human. I believe in you. You can do this.” That bit we can’t outsource. And so the question is, how do we make sure we have time for that. My example is advising. So we think we’re advising. The student comes into our office for a 20-minute meeting, and we go, you don’t have a prerequisite, but you want to study abroad next spring, so you have to take this course now. Oh, you don’t like vampires, and you want to switch? Well, let’s find another sec, right? And we spend 20 minutes adjusting their schedule, and it’s like, “Oh, bye.” [LAUGHTER] And we never got to ask them, “How are ya? How do you feel? Do you have friends?” That’s the mentoring piece. And what we do often is we do administration stuff. Is your schedule right? Do you have the right prerequisite? I have to sign this. That’s administration. So if we separate administration from real mentoring, perhaps I now have a 20-minute mentoring conversation with you and say, “You know so AI is going to make sure you have the right prerequisites, and it’s going to adjust your schedule. And so I’m not going to do that. How are you?” Not everybody can do that, by the way. We have some faculty who are not good at relationships. You used to be able to skate by with, “I’m a content expert. I got my three Nobel Prizes. I’m just going to lecture.” I think that’s going to be harder. So I think that we’re going to see changes to everything about the campus, how it looks physically, how we divide up our time, what people do with students. And I do finally think the stand and deliver lecture is probably going to finally, maybe, go away. Maybe.

John: Turning just a little bit to institutional policies. Are there any sort of policies you’d recommend for institutions in terms of institutional policies with AI. Many campuses have introduced syllabus requirements, that there be an AI statement or something to that effect. Some campuses have gone a little bit further. Are there any guidelines that you’d suggest for campuses that would involve best practices, currently, in terms of AI use?

José: Yeah. So the good news is, the AAC&U has been doing this institute. This is, I think, the third year, and so we have 180 campuses, so we’re seeing a lot of what people are doing. So the first is, last year, everybody was worried about policy. So I would worry less about policy, because policy is changing. It’s not a bad idea to make sure everybody has a syllabus statement, but lots of people will just put “No AI,” and then they think they’re done. So I actually think the first place to start is with control. So I call this CLEAR. So C is for controls. Do I have a safe AI? Do I know which AI i can use? Can I put student data in? Can I not? Do I have to use CoPilot? Or if I’m not going to use CoPilot, what can I use? And what data can I put in there? Because people are confused and they’re told, just use CoPilot… But I want to use Nano Banana. Well, that’s okay, as long as you’re not putting student data in. So understanding what the controls are. L, is for learning. You’ve got to give them places to play. E is for examples. How are you using it in your course? And then A is for adoption. So we need some mechanism for Okay, let’s run a pilot. Let’s do that. Let’s try that. And then R is for review. Did this help or did it hurt? Oh, that was bad. But,there’s going to be failure. This is not all going to work, but in order to figure out what does work, just like anybody who builds… I think it’s Adidas who has a failure funeral when shoes don’t work, they acknowledge that not every pair of shoes is going to be a best seller, not everything we do is going to work. So we have to be able to experiment and try and forgive each other, and be able to have a climate where we know what’s safe. We have some examples, we’re all learning, and then we review to say, “Oh, we tried it twice, because you can’t try it once.” I mean, some things you can, but a lot of the times where we think, Oh, I tried this new thing in class and it didn’t work. So yeah, that’s because the first time you did it, you were crap at it. You’ve been lecturing for 20 years. And so you tried this new active learning thing, and it didn’t work because you didn’t introduce it right, you didn’t set it up right? You aren’t good at it. Try it again. [LAUGHTER] You’ll be better at it next time. And so I think that for teaching, we often think that teaching is just content. And I discovered when I started doing corporate work and I had to do the same 3-hour presentation 35 times in a row, [LAUGHTER] that it got better by the 34th or the 35th time, that actually practicing… I wasn’t changing the content. I was just getting better at the delivery and the management of the client, all those sorts of things. And that rarely happens in higher ed, where we do the same thing over and over again. We get feedback, and we try to make the activity and the learning better, and not just, “Oh, well, I’ll do that next year and try some new thing.” So I would worry a little less about policy and a little more about building the right culture where people are having the right conversations or experimenting. I do think we’re starting to have the curriculum conversation, which is, where does it go? I think people now recognize, okay, I can’t graduate students without any AI. That’s got to be someplace. So the first place to start is the senior seminar, because they’re about to go look for jobs, and so they should know how to do a practice mock interview with an AI before they go: give me a 30-minute mock interview in the format that Microsoft uses for my interview next Tuesday. Here’s the job description. That’s an easy thing to do with an AI and have students practice. But it’s also, read between the lines of this job description to tell me what they’re really looking for, so I can write a better cover letter. AI can read between the lines. So that’s cool. And also, how is the discipline changing? Think about it, you just spent four years as an accounting major, and then Excel spreadsheet gets introduced when you’re a junior, but the curriculum can’t catch up because, like, well, we don’t teach that, because it didn’t exist when you started. Four years from now, maybe. So the seniors’ capstone is the place to say, well, you know what, this new thing just happened. It’s called a spreadsheet, and I’m going to introduce you to the spreadsheet, because you’re an accountant. So that’s a good place to say, Okay, I still need to learn how to do all those other skills by hand, but before you graduate, and that means you’ve got at least one person in every department who is having to keep up with what is going to happen when our students graduate, and what are they no longer going to be asked to do, and what new thing will they be asked to do that we might need to prepare them for. So, senior seminar is a great thing to say, “Let’s put some AI there.” Probably at the freshman year too, about… again, I would redefine cheating. I would think about it as what’s misuse and what’s really fraud. Why does integrity matter? Why do credits matter? It’s the same thing with the bibliography. That’s not about academic integrity. You go to the movies and what do they do at the end? They show you the credits. Is that academic integrity? No, that’s the movies, right? But telling people who gets credit for this matters, and students get that, right? They really care about that. If you steal somebody else’s TikTok, that’s not academic integrity, but it matters. So there are probably other places you need to do this, but I don’t think you can have one policy for everybody. I am noticing that some universities are now having a policy that you can no longer ban the use of AI for specific things. In other words, you can ban AI, but you can’t ban it for feedback, or you can’t ban it for brainstorming. So they’re limiting, because students were going from all AI to no AI, and it’s like, “I’m confused.” So nobody can ban AI for everything, in the same way that you can’t ban a spell checker, right? We don’t do that anymore. We used to, we used to ban pencils and erasers too. We didn’t like the typewriter either. But we’ve had a long history of this. It’s going to change everything. Yep, and it did.

Rebecca: So going back to your idea of controls, rather than really thinking about policies, but giving some boundaries on things, you mentioned, identifying specific tools that maybe people are allowed to use, what kind of data they can put into such tools. Paid services often have more services or more capabilities, and so in the interest of equity, should institutions adopt some sort of standardized tool that everyone then has access to?

José: Ah… it’s a lot of money. So I do think that we need to think carefully about those issues. It may be the case. The problem is that Microsoft CoPilot is expensive and you buy one tool. So I think the first thing is for people to understand what happens to data when you put it in if you’re already using Microsoft on your computer, and you wrote your dissertation on Microsoft and you use iCloud, or you used Microsoft 365, OneDrive, or you used Google Docs to write your dissertation, it’s already in the AI. You have much bigger privacy concerns when you talk in your car. There are other things that we need to talk about for privacy. So I think the first step is helping people understand what does happen to your data when you put it in an AI and which AI. And by the way, there are these controls. If you open your settings, you can turn some things off… not sure I trust them… but you need to know that the default in Claude is off, but the default in ChatGPT is on, and that will probably change tomorrow. So you need to be checking your settings, and you need to be thinking about what kind of data is it ethical and legal to put in here? And do I want to do that? If I do that, then there are a lot of free tools I could use. So I think step one is not saying, “Do whatever you want, but only do it in this tool,” because then everybody’s an idiot. Much better to educate people, that’s what we do, and to say so let me educate you as to what the tools do and how they do it, and what are the parameters. And then you can make informed decisions about whether or not you want to use free Gemini twice a day, which is all [LAUGHTER] you can use in the best model, but at least you have that. And then you can switch to ChatGPT for your three times a day or et cetera. And we’re going to have this model, and you can always use this model for this kind of thing. But I think if you don’t also educate people as to how this works, then having one tool for everybody is just a waste of money, because not everybody’s going to use it. People are going to use their favorite tool anyway, and they’re going to use it poorly and with risk. So the first question, the way to reduce your liability is to educate, not to restrict. And so I would start with that, I like BoodleBox because I think the pay structure is reasonable. It’s a used-based model. It gives you access to lots of different models, it’s not just only ChatGPT or Gemini, because one of the things we now know is that Claude is better at Excel than CoPilot, and that it used to be the best writer, but now ChatGPT is a better writer, and Claude is a little verbose, and Gemini does other more…. So they are distinguishing themselves. They’re doing different things. And of course, we have all the millions of APIs that really do different things, and so I don’t think, for the moment, anybody can get by with just using one model. I mean, yeah, you should try making a picture in Gemini and writing text. That’s good. But we’re in the AskJeeves, AltaVista phase. [LAUGHTER] I don’t know where we’re going to end up. And so for the moment, it’s useful to try a couple of things if it doesn’t work in one try another, but to understand how data and privacy work.

John: And that reminds me of Dogpile, I think that was the name of it, where if you put a query in there, it would search all of the major tools, [LAUGHTER] which sounds like perhaps the best way of approaching this now.

José: Yeah, and there are tools that do that. POE will do that, but I also think that it’s not just about getting in it because you have to follow up. Iteration is a part of good usage. And so picking the right tool is important, but mostly following up. I was doing some shopping the other day, and I had a really odd problem. I have this old set of speakers and an amp from the 1990s so I needed a new piece of equipment. So I tried Gemini, ChatGPT. I was using Comet. I tried different things, and I didn’t know what the answer should be, but I got pretty good advice. But let me just try this over here. Let me try changing the prompt a little bit, so I think that sort of skill is going to be necessary for everybody to manage, in the same way that digital literacy, not everything you see on the internet is true.

Rebecca: What?

José: I know, I know. It’s hard.

John: Well, we always end with the question: “What’s next?” …which is an especially important question in addressing AI.

José: Oh gosh, yeah, nap would be the right thing. Well, here’s what I think. So I think what’s next is that we’re about to see lots of new competition from places you didn’t expect. This one. I do know, we’re going to see a $6,000 eMBA from an accredited university rolled out in the next couple of weeks, the whole degree. So it’s the professors have been turned into AI avatars, so it’s scalable. So it’s cheaper than your MBA [LAUGHTER] or even your eMBA, and they’re hoping to scale it. So we’re going to see degrees that are totally AI, that are going to be cheap and scalable. And this is something new, because traditionally in strategy, you can either be customized and expensive or low cost. Can’t be both. But AI allows you to be low cost and customized. So your new competition is an entirely new class of things. On the other end, we’re going to see boutique institutions. You’re going to see Harvard. They don’t have to change because they’re Harvard. So we have in-person classes. We do this, but they’re expensive. So the question is, for most of us, where in that spectrum are you going to be? Most of us are not Harvard, so we can’t charge what they charge, and then have boutique. So are we organic vegan boutique, or are we, you know, mass? So most of us are going to be neither extreme, but those two things are coming at us, and so the market is about to differentiate or spread. And so for most places, thinking about what kind of institution should we be? Where are we going to plant our flag? And it’s not whether or not we’re going to have AI for most of us, it’s how much and where. Harvard could ban AI and get away with it and give you a four-year degree with no AI because they’re selling a credential, but most of us cannot do that.

Rebecca: Well, thank you so much. This has been a great conversation. We appreciate you joining us today.

José: I just make stuff up as I go along. [LAUGHTER]

Rebecca: So does AI. [LAUGHTER]

José: And that’s why we’re both creative people [LAUGHTER], or creative entities, I guess.

Rebecca: Beings?

José: I don’t know. [LAUGHTER]

Rebecca: I don’t know. [LAUGHTER]

John: And by the time we release this, the second edition of your book will be out. In fact, my copy is arriving in my mail tomorrow. I’m looking forward to the second edition.

José: Yeah, December 2 is the drop date, so Amazon should switch over everything, it should switch to the second edition tomorrow, and there’s a lot more. We didn’t think it was going to be 40% new, but it is. We thought, oh, we’ll make some updates and we’ll fix this. It’s like, “Oh God, scrap that, whole new chapter.” So it’s a thicker book with a lot more than we thought it was going to be.

John: As I think has happened with AI in general, it’s a lot more than many people thought it was going to be when it first appeared.

José: There you go.

John: Thank you.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Ryan Metzler.

[MUSIC]

423. Using AI for Project-Based Learning

Posted in active learning, authentic learning, community based learning, edtech, educational technology, engagement, higher education, learning, motivation, teaching
December 10, 2025

Students are motivated to learn when they have autonomy and see the purpose in what they are learning. In this episode, Tara Chklovski joins us to discuss a curriculum in which students use AI tools to solve challenging problems in their communities. Tara is the founder and CEO of Technovation, a nonprofit developer of a curriculum used in over 160 countries and reaching over 400,000 students, to prepare young women for careers in technology.

Transcript

John: Students are motivated to learn when they have autonomy and see the purpose in what they are learning. In this episode, we examine a curriculum in which students use AI tools to solve challenging problems in their communities.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Tara Chklovski, the founder and CEO of Technovation, a nonprofit developer of a curriculum used in over 160 countries and reaching over 400,000 students, to prepare young women for careers in technology. The current focus of Technovation is on teaching students to use AI to solve real-world problems in their communities. Welcome, Tara.

Tara: Thank you, Rebecca, thank you for having me.

John: Our teas today are… Tara, are you drinking tea?

Tara: Yes, I am. [LAUGHTER]

John: What kind of tea are you drinking?

Tara: It’s Costco’s green tea. I love it.

John: Very good.

Rebecca: Excellent. I have a Twinings English breakfast that I actually got when I was in London.

John: I have Twinings English breakfast that I got delivered to my house by Amazon. [LAUGHTER]

Rebecca: Actually, I made a whole pot this time, John, like it’s the good stuff. So we’ve invited you here today to discuss Technovation. But before we start with that, can you tell us a little bit about your journey to this position.

Tara: Yeah, I grew up in India, and I always wanted to be an aerospace engineer, and I came to the U.S. to do my Masters, and then my PhD. And it was just interesting to me that I did an internship in India at Hindustan Aeronautics Limited, and I was one of the only women on the research and development team, and then when I came to the US, and I was doing an internship at a drone company, I was also one of the only women on the research and development team. And it was just very surprising to me that the most powerful country in the world had a similar kind of talent makeup and distribution as a very low-income country. India was much lower income than it is now, and it made me kind of think about it, and it was just surprising to me to run into people in the U.S. where, especially women, thought it was okay to say, “I’m just so bad at math,” and that is almost like such a shameful thing to say in India, because it’s kind of saying like, “Oh, I’m not intelligent or I’m not a good person.” And it was just kind of interesting to me to see the culture of even people in a wealthy country just assuming that certain paths are not open to certain people based on their gender or socio-economic status, and in the richest, most innovative country in the world, that’s also the most entrepreneurial, that was just such a shock to me that that is okay for us to say. And so I stepped out of my PhD program and actually started a nonprofit 20 years ago, and the goal was to bring cutting-edge technologies and an innovator’s mindset to groups that are not typically thought of as innovators, and that was the sort of path. And we were funded by the U.S. Navy in the early years, and we were very, very data driven, trying to figure out what type of model makes the most impact on students and their mindset as a innovator. And one model really rose to the front that had a durable impact on students’ identity as a problem solver, and that was a three-month long experience where you work in a team, find a problem in your community, and actually build a technology solution and present it to your community. And that experience is unforgettable. And 10 years after that experience, students would choose STEM careers because of that very, very formative, collaborative learning experience. And when we saw that, it was like, “Okay, this is it, and we need to scale this up.” And we brought it to 120 countries, and it has run primarily as an after-school program, and now we have adapted a new version of the same curriculum, but making it easier for educators to use in the classroom. And that’s the AI in Action curriculum.

John: Can you tell us about this program?

Tara: Yeah, as I was sort of mapping out the journey, it is really our Technovation curriculum and our Technovation model of supporting students to go out into their community, talk to nonprofit organizations, actually any organization, find out what are some big problems that are impacting people, and then developing a technology solution and testing it with those users. So we started doing AI nine years ago, before anybody was talking about AI. Nvidia and Google and the Patrick J McGowan foundation were the three sort of leaders who recognized what was coming and wanted to make sure that there was a lot of education support built in when AI really became mainstream. And guess what? It took about nine years for it to become what it is today. And so we are entering the space with a lot of evidence-based curriculum, a lot of incredible stories of students actually using and building AI solutions for good. And so, as I had mentioned, this curriculum has run primarily in after school, and so we adapted it to be a shorter version that educators can adapt for in school, and you go through the same sequence of how do you find a problem that’s a meaningful problem, impacts a lot of people, and you need to have no prior technical background. How do you actually build an AI prototype? How do you build your own data set? How do you train a model? How do you make sure that it is not going to hurt anyone? And then actually test it with users, iterate, and then present it to the broader community on kind of like a demo day.

Rebecca: You mentioned that it was after school, now you’ve shortened it so it can be embedded into curriculum at schools. How do educators come about the program and actually bring it to their classrooms?

Tara: Yeah, so it’s completely free. Just go to technovation.org, and you can go onto the curriculum. And so there are two tracks. Basically, one is a competition track, and that is primarily for girls, and so that has typically been what we’ve been running after school. And then we have the AI in Action track, which is for educators, for use in the classroom, again, completely free, and you can access it from our website. You can also sign up for our educator guide that will provide you. Again, you know, you need no prior AI background, and the exciting thing is, we want you to become part of our global educator community and get connected to other communities of practice, learn from one another, and especially for students, its super inspiring to see what other students around the world are using and building to improve the world.

Rebecca: Can you talk a little bit about the competition track, why the focus has been historically on young women?

Tara: Yeah, we don’t have, like, I started like, the problem statement. I think every one of the 190 countries legally discriminates against women. They have a very big ladder to climb. And I just think that a country really loses out on innovation when you just have one type of person that is in the leadership role or in the sort of innovator’s seat. And, I think, if a country is really to make progress, you want everybody to have these kind of future-ready skills and these future-ready mindsets. And so I don’t see this as from an equity lens, I really see it as this is what a country needs to innovate and to stay thriving. So that, especially in the age of AI, where so many of your traditional white-collar prestigious jobs are being automated, there’s a real question of like, what are the big problems that are left for humans to solve? And these are these very complex human-related problems where so many people don’t have access to healthcare, education, there’s hunger across the US. And so who’s solving these problems? These are very, very difficult problems, and we are not empowering students to tackle these very complex problems. And so I think that this kind of future-ready skilling and mindset is really important to bring, especially to groups that have not been part of that.

John: Can you tell us a little bit more about the competition track? How is it structured?

Tara: Yeah, absolutely. So there’s just so much evidence to show that having a deadline motivates humans to try doing big things in a short amount of time. And so, yeah, it is a challenge and the challenge is find a problem in your community with a team. And so we invite girls to form teams sometime around now, anyone can go onto our website and register and start basically going through all the steps. As an educator, you can think about if you want to invite or do a club, or an after school club, or bring some students together, and then have the students self select into teams… anywhere between one to five… and then the most exciting part is for them to find a problem that they all care about, that they want to solve. And this is the highest learning opportunity, because they have to sort of think about their own value systems, who they are as people, what do they care about, and then they have to compromise as a team. And so the collaboration that happens there is very, very deep, and the conflicts that happen and the negotiation that happens. Once you register on the Technovation platform, we try to connect every team with mentors from industry, depending on your zip code or whether you’re virtual, and so educators are always connected to more help, so that the teams are not on their own. And then the team of girls and mentors walk through the 12-week curriculum. There are three subtracts divided by age. So beginners, eight to 12, junior is 13 to 15, senior is 16 to 18. And now we have an advanced accelerator track for 19 to 24. And so the curriculum is appropriately scaffolded for these age groups. And then, yeah, you spend 12 weeks actually prototyping, testing your product with users, and then doing that final demo day. And depending on how effective your solution is and how big your problem is, you may make it to the different rankings, so quarter final, final, and then we have the big world summit, where we apply the finalists from around the world. Last year we had it at Nvidia, and that experience is absolutely life changing for these girls, because they make friends with other innovators from around the world, and they stay in touch with them, and that’s their cohort of fellow inventors. So yeah, that’s the whole journey. This past year, we had about 33,000 girls compete.

Rebecca: That’s wonderful. I’m thinking, as an educator, it sounds really enticing and really exciting, but also maybe a little overwhelming to like, take a group of students on a journey like this. You mentioned that maybe the educators don’t necessarily have to have a background in AI, but what are some things that they would need to be prepared to do to lead a project? And is there training for the educators?

Tara: Absolutely, there’s a training for educators, and you can sign up, and now is the time, and you get connected with my incredible team of colleagues who absolutely love to support educators on this professional development journey and this journey of sort of bringing a sense of purpose to their students. It’s so, so fulfilling, because it’s not like your average class project, because you’re actually going out into the community. You have students listen to people and hear about their problems and then frame a solution. And students are very self motivated, because when you sort of promise somebody else that I’m going to make a solution for you, the students have an increased sense of accountability and responsibility, which grades can never do. And so you really unlock this deep sense of purpose and a problem solver’s identity in students, and that’s very fulfilling for educators. So Rebecca, you’re absolutely right. It feels overwhelming, but I think the thing to remember is like, there’s no right or wrong, because even a very lightweight experience for students is very powerful, and like with FIRST Robotics or any kind of competition framework that lasts for a long period of time, every year you do it, you get more students, you understand things a bit better, things become familiar, and you just get better at facilitating. But you’re not alone, and I think if you keep in mind that you don’t need to be the expert, but you can basically use AI to solve so many of these questions that you may have, it’s not going to be that overwhelming, because if there’s something that you don’t know, you just say, “I don’t know, let’s go find out.” And that part is just fun. I find that it’s so exhilarating to work with young people, because they’re so unafraid, like I’m afraid to try new things and break things, they just go bang at the keys, break things, and nothing happens. And it’s very inspiring to me, and that kind of reverse mentoring is something I’m like, “Yeah, I’m a facilitator, but I’m learning so much here too.”

John: You mentioned that the AI in Action curriculum was free. Who bears the costs of this? I know you’re a nonprofit organization.

Tara: Yeah, that’s a great question, John. So technology companies and our corporate partners who are really interested in building a talent pipeline, a talent network, because if we are to increase resilience in the United States, there is a real question of, where are the people coming from? And so the industry is very, very interested in addressing the skilling problem. AI skilling is a very important part of the full AI supply chain, and that’s what they have been supporting, the development the build out of the curriculum, as well as engaging their employees as mentors.

Rebecca: I want to pick up on something you were just talking about, which is there’s a lot of opportunity for educators to learn from students about lack of fear. A lot of college students, for example, are more comfortable with AI than maybe some of the faculty. Is this also true in elementary and secondary education, and if we’re going to venture into an AI space that we might be unfamiliar with, what are some basic things to get started?

Tara: I honestly think the Technovation curriculum and experience is a very powerful one, because it actually unlocks and opens up the black box of how a large language model works. I’ll give you one example of what a student builds, because it’ll help you immediately understand what it means to build a dedicated AI solution for a problem. So a girl was looking to clean a local lake. This was in India, and bringing volunteers together, and she wanted to sort of give some data back to the volunteers about how the lake was getting better, and she didn’t have access to a water sensor and her idea was she would train an AI model to detect the sounds of birds, and more birds would come as the water became cleaner. And so when she was looking online for an AI model that would automatically detect bird song, she only came across North American models. And so she built her own data set of Indian birds so that it could actually recognize the Indian bird songs. And these are examples of students around the world building new data sets that don’t exist currently for problems that are not being addressed by the AI models and solutions out there. And so educators can really be on the forefront of that if they were to partner with young people and students. And that’s where the innovation lies, and the only thing holding us back is fear. And I think we’re all super busy, and so the other thing holding us back is that we probably feel too overwhelmed and uncomfortable to make time and to prioritize this. But here’s the thing like, this is unlike previous technologies. This is unlike the internet. This is unlike mobile. This is unlike a lot of the other kind of technology waves, because it’s never threatened what humans did so closely. And the kinds of conversations we are having today are very different from what we used to have before, because sort of what it means to be human is being questioned when an AI, sort of chatbot responds sympathetically to you, and what if you didn’t know it was an AI chatbot, like, what if your friend was not a true friend and responded in a similar sort of robotic way? It still makes us feel the same way, right? And so that is a very scary piece, because that means that I don’t know what it means to be uniquely human. And so I think that over the next five years, people, especially our age, adults, if we are not fully grasping our hands around this crazy, scary thing, we’ll just get more and more buried by our own fears. And so the most we can do is to really embrace it and experiment and young people do it the best. So I think the Technovation model of adults co-learning with young people is such a powerful and a fun way. Otherwise, online learning is so boring. [LAUGHTER]

Rebecca: One thing that you’re highlighting is having a goal, [LAUGHTER] or like a problem that you’re interested in and connected to and that you want to solve. And I think that’s always a motivating way to pick up some new skills.

Tara: Absolutely, and I think grounding it in your community, whatever your community would be, it could be physical, geographical, online.

John: Now, I know at the college level, students are often racing a bit ahead of their professors in terms of adopting AI tools, and in terms of that willingness, as you said, students are much more willing to try things and not be too worried about the risks of things breaking, but I know a lot of college professors are a little reluctant to try it. Are you seeing the same sort of things in K-12 education? The response of many school districts initially was to try to ban the use of any AI, but that broke down pretty quickly, but are you still seeing that same sort of resistance to introducing AI tools in K-12 education?

Tara: Absolutely, you can see it everywhere. It’s the same kind of percentage of adoption, fear, hesitation, almost in every workplace, and especially so in the education space, because it takes so many years to build curriculum, build evaluation methodologies, set those systems in place, align teacher… like all the incentives and performance around that. And so I think we’re going to have this period of intense chaos where the evaluation methods have not caught up with the tools that students are using. And so I think the place is really to have open conversations with your students, because this is such an interesting time. You can bury your head in the sand and say, “This is above my pay grade,” or “This is not my job,” or you can really engage your students in some fascinating questions about how they are using AI tools to do their homework at home, and how do they see their brains changing as they are using AI for a lot of their assignments. And I think that will be very, very interesting. I don’t know if schools can quickly adapt and ask these questions, of like, what are the skills that are important to teach? Because I think we’re all grappling with that question. I think when people talk about empathy, being uniquely human, I was just kind of saying, I don’t know what that means when a chatbot can mimic empathy much better than even a human could. So I think that this is a time for parents and educators to really open their minds and actually ask young people and their children, no matter how young, because guess what? They’re all using AI. What do you think is happening in your brain? I have two daughters, and my older one was saying that she can feel her brain going soft in some places, but being exercised in different ways. And so that’s very interesting. Like, I totally feel my brain going soft in some ways and stretching in other ways. And so, yeah, what is happening here?

John: People who already have become experts or have developed some expertise in an area are able to critically evaluate the output of AI systems. But there’s a lot of concerns that beginners might do some cognitive offloading and not develop those critical thinking skills needed to critically evaluate the output. And we know those hallucinations do occur because of the nature of these models. What can teachers do, at all levels, to help develop the critical thinking skills needed to be able to critically evaluate the output?

Tara: I think that’s such a great question, John. And I’ll come back to really encouraging students to tackle the big problems, because complex systems thinking is not a linear problem, like where you follow a bunch of steps. I’ll give you an example. A very common problem that students try to tackle is hunger, especially in the United States. And the problem is so complex, because many people in the middle class families see the amount of food being wasted in restaurants, at your home, and so you immediately think, “Oh, what if it was a better food distribution method that would bring this food that’s being wasted to the people that are hungry?” And many people will just stop at that solution, and that is not a solution. The hard part is, really, this is perishable, these are perishable items. They’re dependent on preferences. It’s dependent on where you are, and the cost of logistics and transportations are enormous. And how would you do that at scale? Everything can be done for one little thing, but you’ll break down when you do it at scale. And so that’s a complex system, and that kind of mental mapping and problem solving is completely different from the little project that you would outline to a student. And so those are the kinds of skills that are really the ones that are what we need. And I think teachers have… there is time in the classroom. And if you align a lot of, I think, reading, literacy, math standards, into a rich project like this, I think you are preparing students very well for the future. And then they’re completely understanding they’re using AI at every step of the game, at every step of that way, and they’re seeing exactly what you were mentioning because they’re so invested in the end outcome that they’re checking and they’re like, “Wait, that doesn’t line up.” And so they’re motivated, they’re invested in their learning journey. I see students short cutting when they don’t care. We’ve traditionally sidestepped that question when students ask, “why should I care about this? Why does this matter to me?” And we typically say, “This is important for your brain, this is important for your future.” And I think we can’t hide behind that for much longer.

Rebecca: You’ve mentioned that the competition track at least is 12 weeks and I know the one for the AI in Action is a little bit shorter, but there’s a structure to it, because it goes week by week. Can you talk a little bit about what that structure looks like?

Tara: Yeah, absolutely. And so the very first part is problem identification, kind of like what we were talking about, and not very broadly, like, “Oh, I’m going to solve hunger,” but really narrow it down into what exactly is the problem, and then how can AI technologies actually address it? And so if you were to go back to our hunger example, it could literally be, how do we use AI to predict supply and demand? And how could we make the distribution of unused food as efficient as possible with minimal use of gas and energy and things like that? And so that problem identification, a meaningful problem, takes a few weeks, and we provide a lot of exercises, worksheets, for students to be able to do this, and for educators to facilitate that. The next phase would be prototyping, where you actually prototype on paper, and then you go out into the community and get feedback from actual people on would this actually help you? Where are you getting stuck? And then kind of building out the user interface for that. Third stage would be to actually start to code it, and this is where they’re learning whether they’re going to build it as a mobile app, or have it as a website or whatever. And then you start to think about, once it’s actually working, you test it again with people, and then you actually think about, how can you bring it to a lot of people? And so there’s a marketing element to it. There’s a business element to it. You look at competitors, you look at how much money is needed to actually scale it. And so that’s where that entrepreneurial learning happens a lot. And then the last phase is again, after iterating your solution a bunch of times, you actually develop a pitch video or a pitch where you tell your story and you practice explaining your solution to a larger audience so that more people adopt it and give you feedback. So that’s the full spectrum, and it’s a spectrum of starting a startup. I think for educators, and Rebecca, you were talking about like it’s a very real thing, like when you’re busy on your day to day, it feels overwhelming to try something new, to learn something new, and to restructure your life to make room for that. And I think that, as I was saying, that this is a very unusual time, and you may think that, “Oh, I’ll be fine,” but the option is, there’s a very real risk of jobs being completely eradicated. I think educators are not going anywhere, because we learn best from humans, but you can really be a superstar if you learn how to leverage these tools to support you, rather than have that be pushed down on you when you’re not ready. And so I think taking ownership of your own learning journey and your career and your professional development, I think it’s a very exciting sort of juncture for adults today where you can choose to really be a world class superstar because you have the world class superstar tools at your disposal, and the only thing holding you back is your mindset.

Rebecca: So we always wrap up by asking, what’s next?

Tara: Yeah, I think for us, it’s really about trying to support and get educators to help their students get ready to solve these big problems. Because I sort of see an extreme case where corporations are making so much money with AI automation, maybe, like, I don’t know whether universal basic income is a real thing, but what if it is? Then the big problems that corporations will not be solving are the ones that still cause human suffering, healthcare for all, high quality education for all, no hunger, and those are not incentivized by industry. And so whose job is it to solve those? It’s our job. And I think that really encouraging students to have that sense of empathy and empathize with another human’s suffering and to try to do something about it. I think that’s the core of what makes us human. And so that’s why we want to bring this and work with educators to bring this to many, many more students.

John: So if people would like to find out more about the program, we’ll include links to your website so that people can learn about the AI in Action Program and the other programs that you run. And thank you for all of your work on this.

Tara: Absolutely. Thank you for having me.

Rebecca: Yeah, thank you for sharing this exciting work. And it’s even better that it’s free.

Tara: Yes, absolutely!

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

422. Negative Examples

Posted in active learning, clickers, edtech, educational technology, evidence-based practices, evidence-based teaching, higher education, learning, pedagogy, teaching
December 3, 2025

To understand what a concept is, we must also understand what it is not. In this episode, Bill Goffe joins us to discuss his study of the effect of the use of negative examples on student learning outcomes.

Bill is a Teaching Professor in the Economics Department at Penn State.  He had previously been one of our colleagues here at SUNY Oswego. Bill is very well known in the profession for his Resources for Economists on the Internet, which was one of the very first internet guides available for economists, and it’s now hosted and sponsored by the American Economic Association. He has served as a member of the American Economic Association’s Committee on Economic Education, the Secretary-Treasurer for the Society of Computational Economics, an Associate Editor for Computational Economics and the online section of the Journal of Economic Education. Bill is currently co-chair of the Liberal Arts Teaching Group, a faculty learning community at Penn State.

Show Notes

Transcript

John: To understand what a concept is, we must also understand what it is not. In this episode, we discuss a study of the effect of the use of negative examples on student learning outcomes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Bill Goffe. Bill is a Teaching Professor in the Economics Department at Penn State. He had previously been one of our colleagues here at SUNY Oswego. Bill is very well known in the profession for his Resources for Economists on the Internet, which was one of the very first internet guides available for economists, and it’s now hosted and sponsored by the American Economic Association. He has served as a member of the American Economic Association’s Committee on Economic Education, the Secretary-Treasurer for the Society of Computational Economics, an Associate Editor for Computational Economics and the online section of the Journal of Economic Education. Bill is currently co-chair of the Liberal Arts Teaching Group, a faculty learning community at Penn State. You can also find Bill on many listservs and forums devoted to teaching and learning. Welcome back, Bill.

Bill: Thanks, John. Appreciate that.

Rebecca: Today’s teas are:… Bill, are you drinking any tea with us today?

Bill: I am. I’m drinking Twinings green tea, decaf.

Rebecca: Nice, good choice. How about you, John?

John: I am drinking a peppermint tea from the same vendor.

Rebecca: Nice. I have hot cinnamon spice today because my daughter requested a pot of it. So we’re sharing a pot of hot cinnamon spice this afternoon.

John: Very nice as we approach the holiday season here when we’re recording. So we’ve invited you here today to discuss your recent working paper with Austin Boyle entitled “Negative Examples in Lecture Improve Student Learning. First, can you tell us what you mean by negative examples?

Bill: Sure. So you’re describing a concept to someone, let’s say a young child, and describing what a dog is, and you talk about different types of dogs: Dalmatians, Pomeranians, whatever. And unless you describe how a cat is not a dog, they’re probably not going to have a great idea of what a dog actually is. And this might be the case, say, for a second language learner, where you might have different terms for different types of mammals, also related to something called contrasting cases, where experts are very good at delineating different types of phenomena or different types of facts, if you will, say a physician or variation theory used in math. So these are all ways of getting students a rich understanding of something.

Rebecca: Can you talk a little bit about what prompted your interest in negative examples?

Bill: Sure. So I’ve used clickers to teach for a long, long time. You currently use iClicker Cloud. Love it. And I tried to write a paper on just in general, using clickers to teach economics, and the paper didn’t really work very well. There’s reasons it’s never been written, and the reason was I use clickers in many different ways, so a little bit more focused approach to having a paper. And a few years ago, I came across the book How to Teach Economics by Yousuf Hamid. He was a high school teacher in Great Britain, and I had to read that. Well, to be honest, I read it during graduation. You have to go to graduation every year, but, you know, it always kind of work. And he talked a lot about negative examples. And I thought, “Oh, that’s pretty interesting.” And it also followed work by Kirschner, Sweller, and Clark on the idea of direct instruction lowercase “d” and “i”, and I thought, “Well, that’d be an interesting to try, and it’s fairly focused,” and that’s the genesis of the paper, and pulled Austin into it as well.

John: We’ll include citations to those resources in the show notes. So, many people, when they’re teaching, tend to use primarily positive examples. What’s the problem with just using positive examples of a concept? I think you’ve already addressed that a little bit.

Bill: One example, the original work on this, at least formal original work… the idea goes back a long, long time… it was by Engelmann and Carnine, a book on direct instruction. And they give an example of a glem, a G- L-E,-M. Could you define that just with one example? Well, you could not. And they give an example of a pencil. Well, is it wood, or is the pencil horizontal, whatever? But it helps you focus in better on an idea when you understand what it’s not. For example, in economics, we talk about capital quite a bit. Unless you tell students, it’s not stock, they probably won’t pick up on that. Or you give examples of money, how money is very specific to economics. To economists, they’re probably going to pick up on it’s not some general sense of wealth.

John: And I can affirm that the concept of capital is one tht students have always had a lot of trouble about because the way we use it in economics differs from how it’s used in everyday use. And just for our listeners, could you explain the difference between the everyday use of capital and the way we use it in economics?

Rebecca: I’m an everyday listener.

Bill: Great. We’ll see if this works. So, to an economist, capital is something owned by business, manufactured and used to make things, so, a commercial mowing machine, a commercial oven, a jet liner, construction equipment. In everyday usage, it might also include financial capital, stocks, for example, might be considered capital. I would add one thing economists really haven’t done, the Bureau of Economic Analysis actually redefined capital recently, includes intellectual capital, and I just don’t teach that. It would be confusing, but there’s actually about as much intellectual capital these days as physical capital in the United States. But maybe in an intermediate course. I would do that.

Rebecca: Seems like an easy thing to get confused. Can you talk a little bit about how you pick which concepts to use negative examples for? I can imagine that some concepts are more ripe for this strategy than others.

Bill: Yes, we wanted something fairly limited, not a long concept of how to manipulate a graph or something, that could be a further work, but just a definitional sort of thing, but the students have misconceptions about this, as we described there with capital. And so we picked three different ones, money, capital, and technology. Technology in everyday use, our phones, for economists, it’s basically efficiency in the production process. You know, it is rather unfortunate, there’s what, 50,000 words in the English language, and we use words that are used in other ways just to make it more complicated. And I joke to students, I wish I could use different words, but that would hinder their further education, I fear..

John: And this has been a problem for economists for well over a century in those very same issues, and it would be nice if we didn’t have to do that, because it introduces that confusion right when students are just starting to learn the vocabulary of economics, and the way I talk to them about I say, “Hey, part of this is you’re just learning the language of economics, which may look similar to how we use these words in everyday use, but it’s a bit different, and you need to focus on these things because of the ways in which we’re using the term.”

Bill: I would add that I’ve seen some STEM education research papers. They teach concepts first and jargon second, and they find that actually improves learning because you’re trying to remember the terms and they’re using the terms, it overloads your working memory. And there you teach jargon later, they find improvements.

Rebecca: We do that sometimes in design, where we separate the technical skill from the concept, because learning them all at the same time can be complicated. I was thinking of a term that I run into all the time where there’s confusion, where this strategy might work well, which is digital accessibility, where people confuse the idea of meeting these standards to help people with disabilities gain access to content and material with the idea of “just available,” like it’s on the internet.

Bill: I learned that distinction just recently myself. I would add on choosing these terms, we found even after instruction, students got it right between 60 and 71% of the time. So we seem to have picked a nice set of terms and definitions to use.

John: Which is actually much better than they would tend to if you had only used positive examples. And we’ll talk about those results a bit in just a minute.

Rebecca: Can you tell us a little bit about your study? How you had it set up? How many students were involved?

Bill: Yes, we had about 1200 students. I teach large sections, about three sections every semester, about 340 students that did this over two semesters, basically just repeated things, get more power for our tests, and did a so-called crossover study. Crossover study, have your participants that are both controls and the intervention side of things. We did a 3×3 one, which is a little bit hard to describe on a podcast, where you can’t visualize things, but there’s a fairly famous one for ultra processed diets. You might have heard of by Hall and company in 2019. They put people into hospitals, they control directly what they ate, and one group got ultra processed foods for two weeks, the other group got more wholesome foods, I guess you could say, and then they switched who got the wholesome foods, who got the ultra processed foods. So thus you could deal with the fact that people might have responded differently, that different foods have different effects on people. And we did the same thing here in our study, except we had three concepts and three sections and three teaching methods.

John: So what were the three teaching methods that you used in the study?

Bill: So we just use a positive example, where I would list a half dozen examples of capital or money, or not quite of money, because there are quite six examples of that. Then with lecture, we had both positive and negative examples, where these things are money, this is not. This is capital. This is not. And a third type of teaching method was with a cliicker question, where I gave a student’s list of things. Some are positive, some are negative examples, and they just choosed which was which.

Rebecca: So what did you find? [LAUGHTER]

Bill: We found that including negative examples improved learning in the following class by about 21 percentage points. Let’s say the class got 50% correct just with positive examples, the class that got positive and negative examples got a 71% correct. So it is very statistically significant, it was robust with lot different statistical methods we used. So we’re pretty pleased with that, it was a clean result. Sample size helps, and I think probably doing it just with precise definitions, not long drawn out concepts or something like that.

John: Based on these results, would you suggest that perhaps we should all focus a bit more on using negative examples as well as positive examples in our teaching?

Bill: Yes, most certainly it’s an easy intervention. For example, you might know Jim Lang’s book Small Teaching, where he talks about easy interventions you can do, and this one’s easy, as long as you remember to do it. But you’re making up your notes, you think of things that aren’t whatever you’re talking about. You’ve probably seen those on exams, overall. Interesting result we had too that the clicker questions, the active learning, was no different than the lecture. So we’d like to do further work on that at some point, most active learning studies have been semester long, active learning not very well defined, and we didn’t find any difference here, so presumably, at some point, active learning make a difference be interesting to find out what that difference might actually be.

John: That’s an interesting result, and that’s probably worth some follow up.

Bill: I think partly on if you do see similar results with the flipped class and active learning and such, people don’t really have a framework on what they actually ought to be doing before class and what active learning should look like. I’ve got another paper on this, I mentioned, that uses so-called deliberate practice, That’s the 10,000 hour rule. It’s really not 10,000 hours, but there’s some key parts of active learning people don’t do that much. One is to focus on sub skills, for us in economics, things like change in demand versus change in quantity demanded. We generally don’t teach that specifically, is my thinking. And then also, you want to make connections between things. How are supply curves and demand curves separate? How are they similar? Because we remember things in terms of frameworks or schemas. The negative examples help develop schemas. You understand what things are not, kind of triangulating, if you will. And good active learning does the same thing, but if they’re just filling in a worksheet with numbers, they plug into the formula, that’s active learning, I suppose, but not much learning there. So we ought to focus more on exactly what active learning is doing for people.

Rebecca: Following your study, what are some different ways that you’ve incorporated this practice into your own teaching practice?

Bill: So I definitely try to do the negative examples, capturing, oftentimes what I see on these exams and when I’m thinking of a question, I always try to have it connected to something else. Oftentimes one concept’s related to another one, and we make that connection explicitly in clicker questions so that, “Oh, if you remember this, this is a variation of that.” For example, how is consumer price index similar and different than the GDP deflator would be one thing we do, and why are they fairly similar? To understand better how those things work. Oftentimes, I explain things based upon what they’ve done before. Another thing I do is we have very nice technology at Penn State, I have a main screen in the classroom. I have a side screen. Side screens, I put up an outline includes both the outline for the day and our bigger topics. So I’m constantly referring to that outline saying “what we’re doing here is a variation of this, is different than that,” and so forth. And if I do surveys as students, they find that very useful. A better study would be to not have the outline there someday and see how learning goes. I just now thought of that as an idea to do a study on.

John: We know that students come in with a weaker set of connections than faculty, and sometimes we assume that they’re able to make these connections and see the relationships, but they haven’t yet developed that skill, and sometimes it’s generally believed that that type of scaffolding, by putting up an outline of the main concepts you’re trying to get across, can help. But I don’t know if I’ve ever seen any tests of that, certainly not in economics, but I’m not sure if I’ve seen it in other areas either.

Bill: I have not. I can’t say I’ve looked very hard. I would add, to do my literature searches. Now I oftentimes start with ChatGPT, but that’s another whole podcast, [LAUGHTER] I suspect.

John: Yes, and there’s some other good tools out there, too. There’s research rabbit and other things that will actually, even in some cases, will provide more accurate results. But ChatGPT is getting a lot better all the time for these things.

Rebecca: Other than in the test results, have students responded? Or have you asked students about using the negative questions and how that impacts their learning?

Bill: I have not actually, that’s a good point. That’d be another thing to add. It’d be interesting see if they actually can pick up on that. Oftentimes, they’re not that great on understanding their own learning. Something we’ve all seen as instructors, someone thought they did much better on the exam than they actually did, which is something you want to avoid. That’s a tough place to be in and you have to be sympathetic as instructor when that occurs.

John: Now, one thing to keep Rebecca happy, we should also ask a little bit about the methodology that you used. In the study, you used both pooled data as well as panel data, and you also used ordinary least squares and binary dependent variable models. So could you just talk a little bit about why you used multiple approaches and how consistent were the results across those.

Bill: Yeah, the results are very consistent. So sometimes you don’t quite know what econometric approach you take, you can argue this is better, that’s better. And another approach, and it’s pretty common these days, I think it’s by far the most common, is you take multiple approaches and see if the results are robust, to see if you get similar results. Indeed, we did. So it’s a very happy occurrence when that occurs.

John: So were you surprised at the magnitude and strength of these results? I know when I read the paper, I was surprised at just how large that effect was, because it’s very rare to see effect sizes that large in a study of alternative teaching approaches.

Bill: I don’t think I had many preconceptions here. That’s odd. I certainly was pleased, but I can’t say I was quite surprised. It certainly makes me want to take this approach again with where we do the crossover studies, you can do many things with that. For example, we’re told we should teach about things students find interesting. Well, is that really true? I did an example of that this semester where Penn State students, they do a large fundraising project called Dance Marathon, or THON, and last year it raised $17 million, about half the student body is involved. And so I do a question about real prices, and the question was, did THON actually raise more money when you correct for inflation, you know, something they’re invested in a lot, and informally looking at the data, that didn’t seem to have any impact on that segment, on calculating real prices, I need to do more work on that, but I hope more people take this methodological approach, the crossover studies, where you could study a lot of things, the outlines, is it interesting or not, and so forth?

John: Because with the crossover study, you’re ruling out the possibility that there was just some systematic difference between the sections or between the other conditions, right?

Bill: Yes, exactly. We actually found there were differences between the sections. Some did much better on the questions than others, but by effectively pooling these different interventions and controls together, we can adjust for that. We can correct for that. I don’t see it that much in economics. John, have you seen it very much, the crossover studies?

John: I haven’t seen very many, but one that stands out, and I think it was done by economists, was back when the initial studies came out that suggested that when people took notes by hand, they remembered more than when they used a computer. I do remember a study, and I’ll include a citation to this in the show notes. What they did is they had all the students use mobile devices for note taking for a portion of the semester, they broke up the group into two halves, and half of them, for the first half of the semester, took all the notes using mobile devices, and the other half took notes by hand. And in the second half of the semester, they switched, and what they found was there was no difference between how students did when they took notes by hand and how those same students did when they took notes on the computer. However, those students who regularly chose to take notes on a computer, on average, did less well regardless of whether they were taking notes by hand or on a computer, which suggests that the students who choose to take notes by hand may be perhaps a little bit more focused on the material no matter how they’re taking the notes, and those students who may bring in mobile devices to use, perhaps may do it because they prefer having the availability of distractions. So it’s not quite the same thing, but it’s similar in that you’re controlling for the individual effects and separating it out so that you are getting a better comparison, correcting for any self selection in the people who are involved, which is really, I think, what crossover studies are trying to remove, the effect of any systematic selection between the groups.

Bill: I would add, I’ve surveyed my students, and generally my students aren’t performing quite as wel do choose to use laptops, and higher performing students do not, so consistent with your more serious study, for sure.

John: And another thing I’ve been noticing a lot is that for quite a while, I’ve been walking around quite a bit, but I’ve also noted that the students in the front third or so of the class tend to be very actively engaged in answering questions, and they tend to do really well, and they tend to ask more questions. But when I’m in the back of the room, I see a lot of laptops where there perhaps may be a video playing, there may be some online shopping going on, or there may be some sort of video game that they’re working on on the mobile device and their laptop or one or the other. And I think we often do see that sort of sorting in terms of the way in which students choose to work. I’m not sure how I could get rid of that effect, other than moving them all up to the front.

Rebecca: it does seem like at least the multi taskers, in quotes, are at least removing themselves, and because they’re probably distracting to other people, they’re just moving themselves to the back so they’re not a distraction to the people who are trying to focus.

John: Except they’re surrounded by other people, and there may be a bit of a contagion effect going on back there.

Bill: I’ve seen papers on the contagion effect, yes. There was a Carl Wieman paper. They did look at seating in large classrooms and found a very definite effect. They even switched seats halfway through the semester to bring the people in the back up to the front, and they still did worse.

John: Some of our students are just much more interested in the material and much more engaged than others, and there’s just so much we can do with that.

Bill: It’s tough, but I would be a little bit sympathetic to students. It’s been a long time since I’ve had to sit through a typical undergraduate today, and you know, you’re three, maybe four courses a day, a lot of sitting. That would be tough, a challenge. There’s a book called My Freshman Year, and there’s a field anthropologist who went to college for a year to understand her students better, and she found it was harder than she thought.

John: And she found some dramatic differences, too, in the way in which students connected with each other from when she had been a student, that they had much tighter networks with the people they knew before they came to college, because it’s less costly to connect with them over social media and so forth.

Bill: Yeah, friendships have changed with the rise of social media.

John: It used to be costly. I remember having to bring all these quarters to put in pay phones back when I was in school.

Rebecca: You mean, in those 1900s phones that my daughter refers to, John?

John: Yes, I think that would be the case.

Bill: What’s a quarter?

Rebecca: Yeah. [LAUGHTER] Well, I think we still know what quarters are. We just don’t know what pennies are anymore. [LAUGHTER]

Bill: Well, that could be, yes. [LAUGHTER]

John: Are there any other things you’d like to share about this study?

Bill: Well, I think that’s pretty much it. I think I would emphasize this little bit more about, again, developing student schemas. But we talked about that earlier today, this broad network, and it’s through connections, and this helps them make connections more focused on whatever the topic might be at hand. So I think negative examples could be used for teaching most anything.

Rebecca: Including our colleagues when we’re doing professional development and all kinds of other contexts.

Bill: Yes. [LAUGHTER]

John: And it’s easy for us to lose focus, because we already can distinguish these things more easily, and students have not yet developed that capability.

Bill: The curse of knowledge really reigns here, where you know it well. It’s hard to understand how other people don’t. I was doing a think aloud exercise the end of last semester where I interviewed students half the semester in how they’d interpret different things. And I spent a lot of time on aggregate supply, aggregate demand model in my course, standard staple things course, asked about certain phenomena, no one actually used the model. I was only interviewing four students, so, I mean, it’s not much power there, but it was kind of crushing, but it gave me insights on how they think about things.

John: In an earlier podcast with Doug McKee, he talked about having students come in and just work through the problems verbally, so he could see what sort of process they were using, and by seeing that, it was easier to help guide them in a more productive direction. Because if we don’t really understand how students are processing things, it’s hard to help them understand things more deeply.

Rebecca: Yeah, we use that talk-aloud method and design a lot too around troubleshooting, because troubleshooting strategies require, especially around technical skills, you need to think about problems in particular kinds of ways to be able to solve them correctly, or to be able to solve them at all, [LAUGHTER] and doing that kind of talk aloud strategy works really well for figuring out where something went awry in the troubleshooting.

Bill: And experts think a lot differently than students, I would suspect. You would never think someone thought that way.

John: And fMRI studies confirm that different areas of the brain are activated for experts and for novices when trying to solve a problem. So it’s easier when you become an expert, but we have to help students develop the skills to get there. While we’ve got you here, could you also tell us a little bit about how you teach classes so large with so many students, every semester.

Bill: I use Perusall as a way to have them read ahead. They come into class, they have some sense of things and then a lot of clicker questions, typically, as long as it’s not something new. For example, I was doing long-run growth. I do that a fair amount, and the book doesn’t really talk much about the institutions and the role of markets in that. So that was a day or two of me talking. I’m going to be doing the business cycle starting tomorrow, and that’s a fair amount of me talking. But other than that, it’s them pretty much asking questions, or me asking questions. Probably on Thursday, we’ll get to calculate employment rates like read about that in Perusall. I’ll ask them, “Who in this group of people is unemployed?” Then I’ll show them some data, what we compute from that data. So it’s a lot of using ideas. I do a lot of peer instruction where they first do votes on their own, and then I say, “Okay, there’s a lot of uncertainty here. Please convince your neighbors.” And I use an app by iClicker, so I can see their votes on my phone, and then they do a revote, and they usually improve 20, maybe 30 points, then I provide further instruction on the topic at hand. And there’s a couple papers I’ve seen. Let’s see Michelle Smith at Cornell, a computer science paper. They found that if you have students revote, that helps them; if you have a further explanation from the instructor that helps them; if you both, it helps them even more, and it’s probably enriching their schemas, they’re seeing several explanations that reinforce the connections between things. Also I do a series of quizzes, rather than midterms. I do seven quizzes. We’re very fortunate at Penn State, we have a testing center, so I send all my students there for testing. So I run the quizzes for two days, after the quiz they do a second stage of the quiz. In class they start, then they answer at home using any resource, and finally I release the answers, and then they reflect on and describe what they found most difficult on that quiz. So we do a two-week cycle. There’s a lot of feedback, even though the course is quite large, with about 1000 students a semester.

John: So having that testing center gives you a bit more security in the assessments than many of us have when we give quizzes online. Many of the assessments I use are done online, and I used to feel a whole lot better about them before we had ChatGPT and other AI tools appear, especially with some of the browser extensions, which will directly answer things, so you’re able to avoid some of that, which is really nice.

Bill: I really feel very fortunate, yes. I would add, it’s a good thing that the AI tools came after Covid, rather than before.

Rebecca: So one of the things that you mentioned when you’re describing your class was Perusall. Can you describe what that is for people that don’t know?

Bill: Oh, yes. An e-text goes inside perusal, and then students mark it up with comments, questions, and responses to other people’s questions. So it has them thinking bit about the text. It grades them automatically. They need to start early so there’s a little bit of spacing there. They need to make sure they document across the assignment, and they’re also rewarded for opening up multiple times, not just the evening before the exam. And it’s fairly straightforward. Most students get full credit on each assignment. So it’s a bit of encouragement there to read ahead. I used to use so called JITs for just-in-time teaching assignments where I’d ask some questions, including what they found confusing, but I like Perusall better. It’s less grading work, and also, I think it covers a little bit more topics or not just looking at the answers to my JIT questions.

John: Now, do you break them up into groups when they’re using Perusall?

Bill: Yes, Perusall actually does that. When I look at the marked up text from 1000 students, the page does not load basically, or has difficulty loading. And you can also download, I’ve not done this, you could download all their comments, and you can run through a chatbot or whatever to get a sense of what they have trouble with, or what they find confusing. I think textbook authors would find that really, really useful. You could say your textbook was designed with the input from 1000s of students, and so your students can read this more carefully. I’ve not found an author willing to do that, but I think that would be a big win for someone.

John: It could be, and from a marketing standpoint, that could be really helpful. When iClicker announced that they had AI tools being built in, I was really hoping that they would integrate that into the text responses, because right now, it’s just generating questions, which is not bad, and it doesn’t do a really bad job of that, but it would be so nice when you got maybe 1000 or more responses coming in, or a few 1000 responses, if it could very quickly generate a summary of the responses to free-response questions. That seems like something that wouldn’t be all that hard to do, and it would be a nice extension.

Bill: I agree. I think there’s some polling technologies that do that. For some reason iClicker doesn’t. I wonder if there’s a legal issue or some other constraint. I would love to see that as well so I’m not just asking five different possibilities.

Rebecca: So we always wrap up by asking: what’s next?

Bill: So I want to do more work on active learning, the ideas from deliberate practice. Does it make a difference if students have good sub skills. And does it make a difference if you teach by making connections between different topics? There’s many ways active learning are implemented. And it would be cool to do a study where you look at active learning in many different implementations, but some get those interventions, some do not, in a crossover study, of course. So that’s what I would like to work on next a bit.

John: That sounds like a very productive area of research that could really be helpful for all disciplines.

Bill: Yes, well thank you. That’s the idea, I think, a bit about marketing of papers, you know, who would find it interesting, and so forth.

Rebecca: Well, thank you so much for joining us, and we look forward to your future research.

Bill: Thank you so much. It was great fun.

John: It’s great talking to you, Bill, always. I’ve learned a lot from you over the years, and it’s great to still be able to see your work out there in various places.

Bill: Great. Okay. Thanks so much, John. It’s great to connect up again.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Ryan Metzler.

[MUSIC]

416. Teaching More Effectively with ChatGPT

Posted in active learning, artificial intelligence, assessment, edtech, educational technology, feedback, higher education, learning, pedagogy, teaching
October 22, 2025

The rapid evolution of generative AI tools has introduced an expanding set of educational applications. In this episode, Dan Levy and Angela Perez Albertos join us to discuss how these changes are affecting faculty and classrooms.

Dan is an economist and a senior lecturer in Public Policy at Harvard University where he teaches courses in quantitative methods, policy analysis, and program evaluation. Angela is a graduate of the MPA program in International Development at the Harvard Kennedy School, and is the U.S. Head of Strategy at Innovamat. Dan and Angela are the authors of the first, and now the second, editions of Teaching Effectively with ChatGPT.

Show Notes

Transcript

John: The rapid evolution of generative AI tools has introduced an expanding set of educational applications. In this episode, we discuss how these changes are affecting faculty and classrooms.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Dan Levy and Angela Perez Albertos. Dan is an economist and a senior lecturer in Public Policy at Harvard University where he teaches courses in quantitative methods, policy analysis, and program evaluation. Angela is a graduate of the MPA program in International Development at the Harvard Kennedy School, and is the U.S. Head of Strategy at Innovamat. Dan and Angela are the authors of the first, and now the second, editions of Teaching Effectively with ChatGPT. Welcome back, Dan and Angela.

Angela: Thank you so much for having us.

Dan: Thank you very much.

Rebecca: Today’s teas are:… Dan. Are you drinking tea with us today?

Dan: Yes, Moroccan tea. My family is originally from Morocco, so I tend to drink this one.

Rebecca: Always a good flavor, for sure. How about you, Angela?

Angela: Yeah, this morning I’m drinking chai tea latte. This is my usual morning drink.

Rebecca: How about you, John?

John: I have a tea which actually has a French name, which I’m not going to try to pronounce, but the English translation is chimps’ tea. It’s a black tea with fig and honey and a few other things in it, and it’s really delicious. It was sent to us a while back by one of our listeners in France, which was really nice.

Rebecca: It’s always nice to be able to try new things. I have an Assam tea this morning from Pekoe Tea in Edinburgh.

John: Impressive. She got it there herself, mine came through the mail. [LAUGHTER] We’ve invited you here today to discuss the second edition of Teaching Effectively with ChatGPT. Since some of our listeners may not have heard our earlier discussion about the first edition of this book, can you provide an overview of the purpose of the book and who is the target audience.

Angela: Thank you so much, John. The main purpose of the book is to provide a guide for educators to understand how ChatGPT and other AI tools can be useful for more effective teaching and more effective learning. Now what our main approach is to provide plenty of examples that are practical, to really demystify what AI looks like and how it can look like in and outside the classroom for learning.

Rebecca: In Chapter Two, you describe three guiding pedagogical principles that you use throughout the book, and these principles we certainly talked about on the podcast before and are great advice to anyone involved in course design, whether or not they’re using generative AI. Can you tell us a little bit about these principles and why they’re so important?

Dan: Sure. So the three key pedagogic principles were meant to give the reader of the book an overview of what are the underpinnings of the pedagogic advice of the book. For those of you who listened to this podcast, you might not be surprised in hearing the first one is to be student centered. That sounds very obvious, but yet we spend a lot more time thinking about what we do in the classroom and less time about what our students are doing in the classroom. We spend more time thinking about what we will cover in a course, rather than what our students will uncover. And so being student centered just means paying very much attention, and the focus of the teaching enterprise is stimulating the learning of our students. So that’s kind of that principle. The second principle is to plan for active learning. I think the two of you in this show have said, many times, there’s ample research suggesting that people don’t learn by just listening to someone telling them stories they learn when they actively engage. And the third one is to begin with the end in mind, and this is a little bit of a different way of saying we use backward design. So the idea we start with, what do we want the students to be able to do or master at the end of every learning experience, and then we design backwards from there. And so the book, very much, has these three principles embedded. A lot of the activities that we propose for educators to do have to do with how they can create more active learning in the classroom, and so the three principles embody a lot of what the book is about.

John: Could you each give us an example or two of ways in which Chat GPT can be used to design new classes or redesign existing courses?

Angela: Absolutely. So I can get started with a very cool use that, Dan and I actually did together when we were redesigning one of the classes on the Bayes’ rule in a statistic class, and there was a specific question or activity that was based on COVID and a positive and negative test and the possibility of a false positive and false negative, and we felt that was already an outdated activity by the time that we were working together, and so we wanted to update it to new examples that felt more appropriate for the times. And we could not pin down what could an idea of a different theme or contextualized activity could be. And we asked ChatGPT to just share what could be like 10 potential topics or things that we could explore that would help the students understand how Bayes’ rules can be valuable in a real-world context. And it gave us an infinite amount of ideas that we would have never guessed just by brainstorming him and I together, and just to give like two specific examples that we liked. One was about art forgery and being able to identify fake art pieces, and the other one that we ended up using was about being able to detect AI plagiarism in student work. And so it’s incredibly helpful for anything that has to do with brainstorming contextualized tests and activities to context that are valuable for students.

Dan: Yeah, so similar to Angela, I can think of several examples in terms of designing course, but just to give a recent one, this is more about designing an assessment, similar to what Angela just mentioned. I created a project for a course… and I’m sure we’re going to talk in a little bit about projects… the syllabus has a lot of things, and there was a question that we had created last year that we wanted to change. And so I uploaded the question to ChatGPT, and I said, “Can you tell me what you perceive to be the learning goals of this question that I just gave you?” And I think it did an amazing job, better than I could have stated them myself. This was very helpful. I think part of it is because it had access to the goals of the course, but I think it was very helpful in helping me be better at interacting with ChatGPT in the refinement of this learning activity. So that’s just like one example. I realized both examples that Angela and I gave are in the realm of assessment, but it turns out to be a very important realm in terms of designing sort of a new session or new course, the book goes over to what I think, still today, is my favorite and maybe Angela’s favorite activity. This was kind of maybe the spark of the book. We were trying to think about an activity surrounding the Cuban Missile Crisis. And over two weeks, basically had a conversation with ChatGPT along the lines of, “help me design a better class session on this topic that I don’t know too much about.” After I saw that, I thought this has a potential to be an incredibly useful tool for teachers and learners, as Angela said.

Rebecca: Your first example seems like a really good reminder that we can use tools like this to help implement things like TILT (or transparency in learning and teaching), having it help us be more clear about what the course objectives or assignment objectives are, so that we can then convey that information to students.

Dan: Right. And you might have the reaction, but how can you not know? It’s not that you don’t know, it’s that maybe some of them are subconscious to you. I mean, one of the most useful things that a teacher can have is someone, or in this case, it’s not a person, but put a mirror to what you’re trying to do in the classroom. So that’s why having a colleague come and see you teach is so valuable, because you might do things and you might not realize why you’re doing them, and maybe having someone ask you or sort of give you ideas for why you’re doing them, it’s helpful. The book is based on the premise that AI can be an incredible assistant, but also an incredible thought partner. And I think both of those examples have those two roles being played quite intensively.

Rebecca: One of the other things that you certainly have examples of in the book is the way that ChatGPT can be used during class sessions. Can you give some examples of this?

Dan: Yes, we can give several examples. So, one of them, it’s an example that is described in the book. But our colleague, Mitch Weiss, who teaches at the business school, has a wonderful activity that is about discovering the capabilities of AI, and students use that in the classroom in a case called Storrowed. And so probably not the best use of time here to describe the full activity, but suffice it to say that students are put into two groups, and they’re kind of competing to be able to see who comes up with the best ideas, but they’re able to use AI for proposing some of their ideas. So that seemed like a very interesting one. The book also highlights two examples of using AI to summarize student input in the classroom, and I think that’s an incredibly powerful tool. I think a lot of educators have heard about the one minute paper, the way that it typically worked, at least until not too long ago, was you ask people to give you what was the main idea that they learned, or maybe the muddiest point, or whatever it is, at the end of class. This is incredibly valuable, but normally you would just get out of the classroom and then have to read and sort of think about how it works. And now you can do it in the classroom, live and understand what are the main themes that people are picking up. You can also do it with questions. And then a very exciting use that I experimented with a month ago is with group work. I don’t know how much you have been in situations where you assign students to work in groups, and then 10 minutes later, you’re trying to debrief. But debriefing is a little bit hard because you don’t really know what happened in the groups. You might circulate, but if you circulate, typically, the moment you approach a student group, the conversation changes, so you’re not really listening to what they’re describing. And so what I did is to build a simple custom GPT that did the following. So the students were put into groups, and they were asked to input their answers to the activity in a Google slide. So this was typical use, especially during COVID. You did breakout rooms, everyone fills in a slide. As soon as they’re done doing that, what custom GPT does is it produces four slides for the instructor. So this is in a matter of a minute. And so the first slide has the main themes. So of all the groups you had in the classroom here, the main themes that were emerging. The second slide has particularly novel or interesting ideas that emerge. But not only they tell you the idea, but it also tells you it was group three that came up with this idea. So you can use that in the debrief, “Hey, group three, I noticed you came up with this idea. Can you expand on this?” The third slide tells you about points of contrast or tension between what the groups did, and then the fourth slide gives you some ideas for how you could debrief. And so the fact that all of this is available within one minute of students finishing the group work, I found it to be incredibly empowering to lead a much better debrief discussion. So those are the kinds of uses that I think advance learning in the classroom in a way that it would be very hard to do without a technology like this.

John: You have a chapter discussing how ChatGPT can be used to either design or grade assessments, and there’s been a lot of discussion on this in various social media forums and also in some editorials and articles and so forth, in terms of whether generative AI should be used to create assessments and to grade student work, with some students being concerned that they’re not getting the direct feedback from their professors. Could you talk a little bit about what ways might be appropriate and how this may be helpful.

Dan: So, I would like to separate the question into two parts. One is about designing assessment, and the other one is about giving feedback in terms of assessment. I am convinced that for designing assessment, AI can be incredibly helpful without sacrificing the voice of the instructor or the learning objectives. And the two examples that we gave at the beginning, I think, are suggestive of how can they be used. It can convert you in a much more creative teacher, because you have a thought partner with which to bounce ideas. It can give you feedback on the assessment that you’re doing. My colleague from the business school, Mitch Weiss, again, he has this custom GPT that is like eliminate ambiguity in my exam questions. And so he gives the exam question to the AI tool and essentially “write it in this way so that there’s less ambiguity for students.” So you could imagine ways in which the design of assessments can be improved with the help of AI. I’m not saying we should farm out the design. It’s like, “okay, I’m teaching this class, and now you go, AI, create the assessment, and that’s it.” But I do think, in my mind, whatever drawbacks there are, weighing everything, I think the impact is positive. For providing feedback on assessment, I think you have decided this is a big topic, a lot of controversy about student expectations on assessment and so on. I am not going to tell you that I’m convinced about what the right answer is, but I would say, as we describe in the book, there’’re both benefits and risks associated with doing this. But what I would like to suggest is that what we know from the science of learning is that students benefit from both practice and timely feedback on that practice. So that we know. And so the question to me is, are our students getting enough opportunities to practice and timely feedback? And my sense is the majority of us educators feel that the answer to that question is, no matter how hard we work, it’s very hard to provide timely feedback to students of the sort and in the quantity that they need to really, really make a difference in terms of their skill development. And so what we end up is with assessment strategies that are the most feasible given all the constraints that we face. But I think we’re leaving some value on the table in terms of our students being able to practice. So I’m not suggesting that AI should do all the assessment work for us, but if it could work as a complement to the work that we already do, I think this could potentially be a useful thing. Just to be super concrete, maybe if you had five ways of assessing students in the semester, but you’re only able to grade two because of time constraints or whatever it is, maybe you can use, in a very transparent way, here’s AI to help you provide feedback on the assessment. Now, of course, this has to be done with a lot of care and thought, but we can’t do it by hiding from the students that we’re doing it. And also, I think we need to explain to the students why we’re doing it, and if we’re just saying to ChatGPT, “just give them feedback without our input,” then our students, very justifiably will say, “Well, what are you doing? So I can do that.” So I think there is thought that needs to go into how AI is used to be able to do it. Just to close, we have here the Kennedy School, a course taught by Shard Goel, Teddy Svoronos, and myself, and we have our Teaching Fellow, Calvin Isley. He basically helped us design a system by which the assignments would have a preliminary assessment done by AI, and then our teaching team would take into account that and use it as input for grading the assignments. Students were able to opt in to this pilot. And what I think happened is the students who opted in got more detailed feedback than they would have gotten in a system where we’re just counting on human labor. So, in sum, I’m not saying AI should do everything in providing feedback, but I do think there’s a role.

John: And humans get tired. And if we’re providing feedback on many students, the quality of that feedback may very much vary over time in a way that it wouldn’t if you’re working with an AI tool for assistance and at least providing a starting point.

Dan: For sure. I mean, one of the things we say in the book is that AI has like, two very important virtues: it’s available 24/7, and it has infinite patience, and I don’t know many human beings for whom we can say that.

Rebecca: One of the things that you just highlighted was student practice. And you have a number of chapters in your book about how ChatGPT can help students learn. Can you give us some examples in which ChatGPT can support student learning?

Angela: Absolutely, these were one of the chapters that I was more excited about writing, because when we find many of the books that are there is that they focus a lot on the teaching side of things. But there’s so much that as educators, we can do to just teach students to use ChatGPT in a way that is beneficial for learning. And just to give a couple of examples, the first one I’ll give is in continuation with this idea of practice, we have an example in the book of one student that was studying medicine, and she was learning and preparing for an exam, and as she was finalizing her preparation, she wanted more activities to practice, because she had already completed all of the activities that her teacher provided. And so she just asked ChatGPT, “Can you create more questions? These are some of the questions that I received. Can you generate more questions and then don’t give me the answer? I will give you the answer, and then you will let me know if it’s correct or not.” So it’s just an extension of this practice that, again, it’s one of the things that for educators, it’s hard to provide enough practice opportunities for students because it means generating more questions and more questions that cannot be used for formal assessments. So that’s one possible example. The other possible example, it’s the personalization of learning, and this, again, it’s one of the big moments that for Dan and I, it just sparked our interest even more for what AI could do for learning. And the specific moment was related to time when in one of Dan’s classes, he encouraged his students to learn about risk management strategies before the class itself, and then he asked. The students to share what the conversation with ChatGPT had been like. In addition to that, he nudges students to ask ChatGPT about risk management in a certain way. So for example, sharing what context they had about risk management strategies, etc. And we basically saw very different conversations depending on the student. You had, students that were much more familiar with risk management strategies and the kind of explanations that they received were much more technical, much more detailed, less intuitive for somebody that would be just exposed to this concept. Well, for some students, it was the complete opposite. It was a much more intuitive, just like high level conceptual explanation with lots of examples that could be relatable. And this is something that would be very hard for an educator to replicate, a specific one-on-one explanation that would fit the needs of every single student. So many multiple uses for students that can really help improve learning.

John: In the spring of 2025 we had a reading group on campus that read your book. And just as a little bit of an aside, one of the things that happened is, right at the very beginning, people suggested that people would experiment with something, try it, and then bring it back and report at the next reading group meeting. And that worked really well. It’s a great book for a reading group. But one of the things that surprised people a little bit who had experimented with ChatGPT was the ability to customize it to meet your individual needs. Now some people weren’t able to do that very well if they were using it in a wide variety of applications, but now, with the new edition, you’re able to talk about projects in ChatGPT. Could you talk a little bit about how you can customize ChatGPT in general, and also how you can use projects for a similar sort of purpose?

Angela: Absolutely. So to put it simple, because sometimes I feel it can be hard to get lost in all of these, like features and new releases of ChatGPT. The idea here is that you can customize ChatGPT so that it’s more responsive to your needs or your specific context. And to put this into an example, my specific context, for example, may be I teach an economics course to first grade university students. But the reality is that I don’t use ChatGPT only for my classes. I may use ChatGPT also for other uses. And so what openAI has rolled out in this past year is the possibility to customize the responses of ChatGPT at different levels. The first level is to have instructions that apply to any chat that you open in ChatGPT, and that’s what system instructions do in chat GPT. For example, you can keep this to a very general level, like, “I’m a woman, I’m 30 years old. I do this as a profession. I’m from this country, etc, etc” things that may be relevant. For cases where you may use ChatGPT frequently, but it doesn’t apply to any use of ChatGPT that you do, you may want to choose projects instead, where you may have a project for a specific course that you teach. You may have a project for something that you do frequently with ChatGPT. Perhaps you were talking about travel planning before, like, perhaps you travel plan a lot, and you want to have a project specifically where you can provide instructions around the kind of travel that you like to do and the kind of places that you want to go to and the kind of things that you want to do when you go on traveling, and so that’s what projects allows you to do. It allows you to have instructions that you don’t need to be repeating every time you open up a new chat, but then don’t apply to all of your chats within ChatGPT.

Dan: The only thing I would add to what Angela just said is that not only can you add instructions which are very helpful in customizing, but you can add files, and those files can be incredibly helpful. So, for example, for each of my courses right now, I have not only instructions about teaching philosophy, goals of the course and so on, but also the syllabus and relevant material that I think has helped, as Angela said, the advice that you get from ChatGPT be much more customizable. Back in the old days, and old days here means 2023, when you were writing one chat and then you would begin another chat, it was like two completely independent conversations, and now I think AI is much better at understanding through not just the customizations that we just talked about now, but through the use of memory and so on, it’s much better understanding who you are as a user. I think one aspect of the progress in LLMs, ChatGPT in particular, but I think it’s true for all of them that we have picked up in the second edition of the book, is increased personalization and customization that is now available to a user to be able to get more customized and specific advice from their LLM. And the last thing I want to say is that everything that Angela said is very, very, very, very easy to do. You don’t need to code. You don’t need anything. If you know how to prompt ChatGPT, you can create a project, and it’s as simple as you put in instructions, you put in files, and then you create the chats within that project, that’s all you need to do.

Rebecca: You’ve hinted at some ways of taking advantage of projects. Can you give one or two specific examples of where this could benefit student learning?

Dan: So I think the easiest way Angela mentioned is you can create a project for each of your courses. Once you do that, then a lot of the help that you asked ChatGPT would be useful for creating activities for students, helping you design assessments and so on. So that’s one use of projects. If you want to create something that students consume directly, then you would move from projects to custom GPTs. Custom GPTs can be shared with students and they can interact with it. In the first iteration of custom GPTs that were most popular were the tutor bots, so where you might ask students to interact with a tutor bot, for example, to learn some skills in whatever subject that it is that you teach, but where the instructor was playing kind of a design role, was adding materials and so on. I think those tutor bots continue to be helpful. One of the things I think, I suspect many of us have discovered is that if the tutor bot is not part of the natural workflow of students, then it becomes a little bit harder for them to benefit from it. So if students are using ChatGPT as their AI tool or any other AI tool, it’s going to be hard to move them to the custom GPT or to the tutor bot, unless you sort of have very directed task at them doing that. But there are many, many other ways in which you can use custom GPTs with students. And before we were talking about giving students an opportunity to practice, but imagine you were teaching negotiation, for example, you could imagine, and there are some available out there, bots that allow students to practice negotiations or public presentations or so many other skills. So if you teach any skill that you think benefits from repeated practice and feedback on that practice, I think the potential of AI to help you and your students in that task is immense, and custom GPTs are a way to make this happen. And again, the book is called Teaching Effectively with ChatGPT, so we put our stake in the ground with selecting one LLM, partly because it was most popular sort of tool. But if you’re using some other LLM, like Claude or Gemini, the equivalence of everything we have talked about exists. So Claude has projects. You can build custom GPTs in Gemini, through what they’re calling GEMs and so on. So I think a lot of these companies are all building and competing with each other in this space.

John: And in addition to providing those tutor bots, you can also design them to help you with course design, things that will generate things in the TILT framework, for example, or make suggestions for adding more active learning activities. Pretty much any of the things that you talk about in your book, if it’s going to be done repeatedly, could be turned into a custom GPT. And your book was really good at providing them, that was a really popular chapter with the reading group.

Dan: Thank you. I just want to say one thing. So since the first edition of the book, projects became a thing, and I would say, I think building custom GPTs is relatively easy, but if you’re only going to use that tool for yourself, then building projects is an even easier way to accomplish the goal. Obviously, custom GPTs have the advantage that they’re shareable, so you can share not just with students, but with colleagues and so on. But if you’re getting started and are feeling somewhat intimidated with how do I do all this stuff, projects is a very, very natural step up from your ad hoc use of ChatGPT.

Rebecca: There’s still a lot of faculty who maybe would prefer to ban student use of AI or focus their efforts on trying to detect student AI use. Are there any ways of reliably detecting AI use?

Dan: So I’m not a computer scientist, but the ones I speak with give me pause that we will ever be able to really do this at a level of reliability that suggests that we can take action on this. My general sense is that this is a lost battle in the battle of an instructor against students. I think students always win, is my sense. If this is the battle that we’re fielding, we’re not going to win. I think we have to do differently. But just to give you a sense of for why I think it’s so hard, is that now AI is so embedded in everything that we do, that I think it’s going to be very hard. Back in the old days, again, 2023, you could ask students, okay, go to ChatGPT and then have a conversation and then give me the transcript. But now it’s embedded in the things that we do. It’s auto complete things in Word and so on. And more importantly is that you could, as a student or as a human being, produce an essay by yourself. You created all these ideas, and then you give it to ChatGPT and say, “Please improve the clarity of my writing.” That’s all you’re asking it. It will add a few em-dashes and a few words, and then everyone will say, “this was written by ChatGPT.” And you might decide, as an instructor, particularly if you teach writing, that even the use that I just described should not be allowed. But I’m going to suggest that at least for some subjects, that use is fine. And if that use is fine, an AI detector is going to say that this was created by AI. I think we’re in trouble.

John: Since the first edition of your book, openAI has released ChatGPT versions 4 and 5 more recently. You mentioned projects, but what are some of the other changes that have been added to ChatGPT since then?

Angela: Great question. So projects, as you very well mentioned, is one of the main features that was released. There’s another very interesting one, which has to do with the models that are available within ChatGPT. So before ChatGPT 5, there were multiple models available, as most people in the audience would know, where there was a basic default model, which was valuable for most of the tasks. And then there was a deep research model that is valuable for in-depth, multi-step, more complex tasks that require time to think and process before being able to generate an output. So this can be useful, for example, to summarize academic research, to create classes or provide feedback on longer documents, anything that is more complex than the average task. Now, what used to happen is that experienced users knew about this model, and they would change the model that they wanted depending on the task that they were wanting to do, but non-experienced users would just default to the basic model and would not be using the most out of the capabilities of ChatGPT and so with the release of ChatGPT 5, one of the key developments was that ChatGPT 5 automatically selects the model that best fits the task at hand. And so this is a great improvement in terms of being able to really put at the service of users all of the capabilities that ChatGPT has. One of the important things is that when it was first released, there was a big controversy, because some people claimed or complained about the fact that ChatGPT was not able to always optimize for the best model, and they wanted to retain that agency. And since then, openAI has backtracked and still provides everybody the ability to be able to choose the specific model that they want to use.

Dan: We’ve already alluded to this, but to me, a big development in the last year or so is the ability to customize and personalize your experience, not just through projects, through memory. and other things that we go into in the book. In fact, the chapter that we used to call Creating Custom GPTs, we’ve now broadened the title of that chapter to include many other ways in which you could customize and personalize your experience and that of your students through the use of this feature. So that, to me, feels like obviously ChatGPT 5 is better than ChatGPT 4, but back then, now there are these reasoning models that are automatically selected and so on, is useful. But the thing that you will notice more as a user, I think, is the potential for customization. And obviously the reasoning models can be very, very helpful for more complex tasks, deep research, which Angela briefly mentioned, is another important advancement for much more complex tasks.

Rebecca: As we’ve noted in this conversation and also in your book, generative AI platforms change quickly. [LAUGHTER] And when we talked to you in the first edition, you mentioned having a companion website that you would continue to update. Are these updates something that are going to continue with this edition?

Angela: Absolutely, that’s one of the main advantages of the companion side, the ability to be able to update it. As we know this is a fast evolving space, and that we felt that was a good format to be able to digest information. But one of the main inconveniences is that it’s not life, and so users and readers and listeners can always go back to the companion side for more up-to-date information.

Dan: And the companion side has all the prompts that were used in the book, including, for those educators that shared with us their prompts behind their custom GPTs, they’re also there. The site is freely available, and then it also has a lot of the examples from educators.

Rebecca: And we’ll make sure we have that link in our show notes.

John: And people found it really valuable when we’re doing the reading group to have all those examples there so that they could modify them for their own needs. We always end by asking: “What’s next?”

Dan: Well, I want to say something that’s next that is, thanks to Angela. So I think we are maybe one or two weeks away. Hopefully by the time you publish this show, it will be out. We have a edition of the book in Spanish that Angela gets all the credit for making it happen. So we hope that means that more readers around the world will be able to use some of the insights from the book. If you’re a listener and want the book to be available in some other language, let us know, and we’ll try to see if we can maybe collaborate with you and make it happen.

John: And Angela, what’s next for you?

Angela: Getting on a plane, flying back home for the weekend, and getting that book ready.

John: Well, thank you. It’s been great talking to both of you.

Angela: Thank you so much for having us. It’s always a pleasure to come and have tea with both of you and to talk a bit about AI developments in teaching and learning. So always a pleasure. And thank you so much. Hopefully we’ll see you again sometime soon.

Rebecca: Safe travels.

Angela: Thank you so much.

Dan: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

413. Faculty Perspectives on AI

Posted in artificial intelligence, edtech, educational technology, higher education, learning, pedagogy, teaching, Uncategorized
October 1, 2025

Faculty adoption and use of AI in higher education varies widely. In this episode, three colleagues from the University of Mississippi: Josh Eyler, Emily Pitts Donahoe, and Marc Watkins, provide their perspectives on AI use in higher education. Josh is the Senior Director of Center for Excellence in Teaching and Learning and Assistant Professor of Teacher Education, Emily is the Associate Director of Instructional Support in the Center for Excellence in Teaching and Learning and Lecturer of Writing and Rhetoric, and Marc is a Lecturer in Composition and Rhetoric and Assistant Director of Academic Innovation.

Show Notes

Transcript

John: Faculty adoption and use of AI in higher education varies widely. In this episode three colleagues at one institution explore different perspectives on AI.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Josh Eyler, Emily Pitts Donahoe, and Marc Watkins. Josh is the Senior Director of Center for Excellence in Teaching and Learning and Assistant Professor of Teacher Education, Emily is the Associate Director of Instructional Support in the Center for Excellence in Teaching and Learning and Lecturer of Writing and Rhetoric, and Marc is a Lecturer in Composition and Rhetoric and Assistant Director of Academic Innovation. Welcome back, Josh, Emily, and Marc.

Marc: Thank you. Thank you for having us.

Emily: Thank you.

Josh: Thanks, great to be here.

Rebecca: Today’s teas are:… Josh, are you drinking tea?

Josh: Well, you know that I never drink tea, but I have a delightful water here.

Rebecca: I will continue to say that that’s the foundation of tea.

Josh: It is.

Rebecca: How about you, Emily?

Emily: I’m appalled that so many people come on this podcast and don’t drink tea. I am drinking a Scottish morn tea, one of my favorites.

Rebecca: Very good choice. It aligns with mine this morning. And how about you, Marc?

Marc: I am drinking a tea called Red Dragon that my wife got for me in Portland, and it’s quite good. I really recommend it. It came from Willow Tea House.

Emily: That’s what I like to hear.

Rebecca: Very nice. That sounds like legit tea.

John: And I am drinking a black raspberry green tea.

Rebecca: That’s a good choice. There’s no cherries involved.

John: I almost did.

Rebecca: Oh yeah, I know. And I have a Scottish morning tea.

John: We’ve invited you here today to discuss the impact of generative AI on teaching and learning in higher ed. We see a pretty broad spectrum of opinion from faculty. Some faculty oppose all student use of AI because of concerns it will impede student skill development. Other faculty have fully embraced student use of AI and argue that we need to prepare students for their futures in a world in which generative AI tools are ubiquitous. Most faculty, though, are somewhere in between those extremes. To start this discussion, could each of you provide an overview of your general position on AI use in higher ed by students?

Marc: Oh, boy, that’s an interesting conversation to begin with. I don’t really know about my own personal stance, if that’s even something that I have much control over, because so much of this isn’t being directed by faculty members, it’s being directed by students bringing these products into the actual conversation. I think that changed a little bit last year when the new dimension came where we had a lot of this technology starting to be embedded in existing university systems. So I try to provide my students guidance. I don’t advocate for them to use AI in assignments. I don’t make them use AI in assignments, but I do make it clear that if they are going to use them, they have certain responsibilities, and they also should have certain expectations from me as a teacher to be a little bit more informed about this, and also from their peers, if their peers are using AI with them in some way, shape or form, too. So really, it’s about just kind of guiding them in certain ways. What this is going on depending on what style class I teach. If it’s a small low-enrollment class too, I’ll have a values-based statement when I talk about… I invite them to discuss how they’ll use or not use AI, versus a large class that might be a writing intensive course too, like our first year writing classes, where I use a stop light approach, where Emily has actually helped develop this too, about red light, yellow light, and green lights for students to do so.

Josh: I thought a lot about what I was going to say today, because I want to not seem like I’m the physical manifestation of that Simpsons’ meme of “old man shakes fist at clouds.” So what I’d like to lead with is that, as a faculty member who believes very strongly in shared governance and academic freedom, I completely support any of my colleagues who want to experiment with it, build it into their courses, support students, all the things that we’ll hear about in some shape or form today, but as an individual faculty member for my own courses, I will never build intentional use, and I want to clarify what that means in a second, intentional use of AI into my courses, and I don’t want students to be using it either. And what I mean by intentional, because I know Marc very rightly says people are using AI even if they’re not aware that they’re using AI, right? If I’m using Google and I forget to phrase it correctly, AI is giving me things like that. So when I say intentional use of AI, what I mean is me as the instructor intentionally building it into courses or assignments or activities, and students intentionally going to AI to do the work of the course or engage cognitively with it. So there are labels for these things. I don’t put a label on myself, “refuser,” “resistor,” things like that, but that is my own position pedagogically.

Emily: Yeah, I think I agree with a lot of what Josh and Marc have already said. I think on a curricular level, students should be exposed to generative AI at some point during their college careers, because they’re going to encounter it as students and kind of beyond school in their professional and personal lives. So I think it’s important for them to know what it is, how it works, and to develop I think what some people have called critical AI literacy, understanding the kind of implications ethically of AI use,. understanding who benefits from AI and who benefits from the narratives that we have around AI. And so I think all of that is important. There are some classes where it’s really appropriate to incorporate AI and think about how it might be used in a specific field, and some classes where I think it’s totally inappropriate. In my own classes, we do talk about AI a lot, and I give students the choice of whether or not they want to engage with it. And I think it’s interesting that Josh brought up the word intentional, because I was going to say what I really want for my students is for them to use AI intentionally if they use it, or to refuse to use AI intentionally, if they want to refuse it. And by that, I mean they should know kind of where it is appearing in their lives. And they should be able to say, “I want to use AI and I want to use it for this specific purpose, and here’s why,” or “I don’t want to use AI for any of these purposes, and here’s why.” So one of the things that I think I want to work on most in my own classes is developing that sense of intentional use, instead of just like, “uh, I guess I’ll use it because I’m late on this assignment, or I guess I’ll use it because everybody else is using it. Or I guess I’ll use it in this way just ‘cause.”

Rebecca: I think all of you have kind of mentioned this idea of intentionality around student use. Can you talk a little bit more about why the frame of intentionality… I mean, I think we can kind of make some guesses… in relationship to maybe some of your general concerns of students using AI.

Marc: Sure. I think it’s deeply important that everyone in the process is intentional about AI, whether you’re a teacher bringing it up in the class, setting up a course policy too, or thinking about designing certain assignments where students might be navigating the technology or might be using tools to bring in assessments that you would think you don’t want them using it for too. So you have to look at this from the basic framework of every single person that’s going to be involved in that sort of process: student, teacher, in some cases the actual librarians were actually helping them navigate resource materials that are available to them, maybe even administration. We want everyone within this process to be intentional about the technology. Just the past month alone, we’ve had actual reports from police officers using it to write a crime report, lawyers using for criminal and civil cases, all undisclosed. We even had a report of a judge using it. We want to really be thoughtful about how this technology is playing out in broader society, too, and start thinking about our roles as educators, and how we can start talking with students about what is ethical, what’s appropriate, and what is actually a good, helpful use case of this tool. I’m very concerned that we’ve been looking at usage in terms of augmentation, collaboration, automation. I think we can handle the first two in some ways. They all have their own ethical issues with that, too. But I’m really getting more and more concerned with the automation aspect. We are starting to see students also do the entire assignments. We’re also seeing these new tools, like AI agents with promises of being able to completely automate the entire coursework for students and also for faculty too, and actually grade and assess student writing and other types of student assessments.

Josh: Yeah. I mean, for me, intentionality means a couple of things, but I think awareness is an important pillar of intentionality, knowing all of the different questions that one must consider before engaging in intentional use of AI. So what are the ethical dilemmas? What are some of the key questions? And for myself, the ethics are divided into two categories. There’s the outside of the classroom ethical debate and the inside the classroom ethical dilemmas. So how are people aware of the choices that they could potentially make, and then a critical thinking process for working through the actual decision to use AI.

Emily: I think for me, the reason that I’m concerned a lot about intentionality is because I’m really concerned about student agency. And my biggest concern about AI is the way that it can take away our agency. Agency is something I really highly value in my teaching… I think, a guiding principle for my classes, as well as one of my most desired outcomes for students. I want students to leave my class with more agency in the world and the skills and knowledge to make better, more informed decisions for their lives. And so I have a lot of concerns about how AI, but also just our general technological developments in this moment are making it really easy for people to give up their agency. So I’m thinking of things like in the book filter world, Kyle Chayka, and I’m not sure I’m pronouncing that name correctly, but they talk about how algorithms control our tastes and our consumption and our lives. It kind of used to be the case that we would go out and seek out the media that we wanted to consume, and now a lot of that is kind of determined for us, and so there’s this, I think, huge loss of agency. And you could say the same about generative AI. When you allow AI to tell you what something means, or what you think about something, or what to say about something, you’re giving up some of your agency, and you know a little bit less about what you think, or what you want to say, because you haven’t really kind of wrestled with that. So I think giving up your agency in that way can be really seductive for students who are under a lot of pressures and when the tasks that we’re asking them to do are very difficult cognitively, but it’s also really important and rewarding to go through those processes and develop that sense of agency, and I don’t want students to give that up just because they think they need an A in the class, or because they don’t have the time to finish an assignment. So intentionality is so important for me because it’s a huge part of agency.

John: So there’s a lot of concerns that you’ve all expressed about students using AI to reduce their agency and to develop their skills, but are there some ways in which student use of AI might perhaps support student learning, or might also reduce some of the equity gaps that we observe because our students come in with very different backgrounds, and it’s hard to design courses that will address all the needs of all the students in classes.

Marc: Yeah, I think we’re seeing some really interesting experimental research designs coming out of California, but they have something they’re called PAIRR, which is peer review from a human being, followed by AI, followed by review by instructor, and then actually the reflection and review all put together. So it’s an augmentation type of process in that way too, where you get a standardized peer review and a writing sort of session too, and you can add AI on top of it. But what they’re finding is that the students that use the AI can actually go through multiple rounds of peer review, then the actual teacher can review both the peer’s comments, the AI’s interactions with them, and ask the student to reflect on the entire process holistically. So it’s able to do a lot more in terms of giving feedback and rounds of feedback than it is just from one human being or a series of human beings. We’re also seeing lots of other different examples come out here about AI being used in research and actually looking to use different types of breakthroughs, but also being used to support students and different student learning outcomes by going through a process of coming up and attaching different research points to student learning processes throughout this. Sometimes this can be as simple as a static sort of like summary that you give to students. Sometimes this could be as wild as an AI voice, which I have some problems with, talking with the student too, and interacting with them. So from every sort of response, there’s a lot of caveats about what we’re seeing from the research studies and databases popping up here with AI in them. But it does look like that if it is intentionally designed, there are some benefits for students and student learning.

Emily: I feel like I should jump in here to build on what Marc is saying, because I suspect that Josh is going to have a totally different take. So in my view, I think that the question about, are there ways that students can use AI to increase learning is really less about the kind of particular usages and more about student attitudes toward AI, or the kind of orientation they take to using AI. So I’ll give an example to illustrate kind of what I mean. A couple of years ago, I surveyed students at the end of this semester in my writing class about their uses of AI, and I had one student say I used AI to correct my grammar, and so I learned a lot from that. And then I had another student say I only used AI to correct my grammar, so I didn’t learn very much. And so the difference is not about the usage. They were both using AI in the same ways, but it was about how they were approaching that use and what they were trying to get out of it. One student was approaching it with the idea that AI could help them learn something here, and the other one was approaching it with the orientation of AI can kind of correct a product that I’ve created. So I think that the key is that when students approach AI, they need to be approaching it with a learning orientation, or they’re not gonna get much out of it. I mean, this is just like anything. And I think the students you interviewed on this podcast recently, Kaija, was a great example of that, and one of the things that she said is that she really wants to figure out things for herself, and she uses AI in ways that support her in figuring things out for herself. I think the trick here is that students don’t always know where the line is, and have trouble determining when am I actually learning, because we don’t often ask them to think about that. So I think one of the things that we need to be doing right now is working with students to develop their metacognitive abilities. If they’re going to use AI, they need to know what’s going on in their brain when they’re using it, [LAUGHTER] what specifically they’re getting out of it. So, what uses of AI can be productive? I think lots of them as long as students are able to understand and articulate what they are getting from their use of AI. I think the question about equity gaps is maybe a little different, and I feel like I need to think more about this, but I’m a little suspicious, to be honest, of a lot of discussion about using AI to reduce equity gaps, because I have a kind of suspicion that AI might just Be a band aid here on a larger problem, and we should think about how to fix the larger problem, like if students are not educationally prepared, or if they are going into a workforce situation where they’re being judged unfairly by, for example, their proficiency in English, then I think we need to change the thing that creates the equity gap in the first place, rather than trying to paper over that with AI. I’m not saying that AI can’t be used productively to reduce equity gaps. But I’m a little suspicious of some of the ways that have been proposed for that.

Josh: Right, that’s a really great point. So I’m deeply skeptical that either of those two things that you said, John, will ever be true. I think the major issue right now is that, as Marc was indicating, the research is so nascent, it’s just developing. So for every study that someone holds up in support of AI and learning, I can find one that says the opposite. We’re just at the early stage where it’s more like a tit-for-tat kind of process than it is revealing anything that I think we can draw on to make truly evidence-based claims about learning. So I think at this moment, we are embedded in true academic argumentation. We’re bringing to bear research that we know from other fields and other areas and other aspects of the cognitive processes of learning, and applying it to the AI question right now. Now that will change over the next few years. We’ll get more research. We’ll be able to, I think, draw together claims that rest on a more solid foundation, but I would need someone to be showing me evidence that using AI is more effective for learning than other strategies that we have. It’s really as simple as that. I think Emily has a great point about the equity gaps here. So I think we learned a lot during the pandemic about access equity, specifically with technology. In Mississippi, especially, we were learning a lot about what access students had to technology and how that was unequally distributed. And so that’s one aspect of it. Another aspect is, when we talk about equity gaps, we’re talking about systemic problems and systemic solutions. So what systemic problem is AI supposed to be addressing? And how is it a systemic solution? And so I think that it’s hard for me to see how AI is a better answer, or a more productive answer than other things that we have been talking about and looking at that themselves have been incomplete answers to the equity gap problem, but that are drawn from a wider body of research at this moment.

Emily: Just to respond to what you said, Josh, you’re making me think that in the same way that I was arguing AI can paper over or be a band aid for pre- existing equity gaps. I think it can also be a band aid for problems in the learning space, in that a lot of people are turning to student uses of AI, or even instructor uses of AI, because it has become unmanageable to teach so many students at once. for example. Students need AI feedback because the instructor can’t provide the volume of feedback that’s needed. And so, I mean, I do think that insofar as that’s happening, we should be looking at what are the problems in the learning space that we can and should fix. Can we get smaller class sizes? [LAUGHTER] Can we get more support and time for instructors in that kind of thing, instead of turning to AI? And we see a lot of universities making these huge investments in AI tools or access to certain kinds of AI platforms, when they could be using that money to support the kinds of learning interventions and teaching support et cetera that we already know can improve student learning. So I do take your point about that, and I agree that there are a lot of problems. [LAUGHTER] I think ideally AI can be used by students in learning focused ways. But the trick is that it’s actually hard to do in practice.

John: In terms of equity gaps and fixing the underlying problems, that ultimately is what I think we should be doing. But those problems start back in our elementary school system and go all the way through with inequitable support and funding and quality of education, and we do have students who are the product of that system, and we do want to do the best that we can to address the students we have. And one of the things we do know is an effective way of doing this is small classes, one-on-one interaction, and providing more tutoring support and so forth. But we also know that first-gen students and students who perhaps face some degree of imposter syndrome are less likely to be going in for extra help, less likely to visit office hours. And if we could break that down, then we may not need as much support. But one nice thing about having Socratic tutor AI systems, potentially, is that they’re available 24 hours a day, seven days a week, and they’re available to everyone without any fear of being embarrassed by admitting to someone that you don’t understand something. My personal view is, I think there may be a role for some of those things.

Rebecca: John’s in the debate now.

John: Well, I just built a Socratic tutor [LAUGHTER] for my class of 270 students.

Josh: So here I am John, shaking my fist at the cloud, because when I hear you say that, my mind immediately goes to Justin Shaffer’s high structure course design. He was on campus presenting a lot of different workshops, which is a pedagogical strategy to handle exactly what you were just talking about in a way that doesn’t involve AI. I mean, there are analogs to this question, Emily and I think a lot about equity gaps related to alternative grading practices. And as soon as you begin to peel one layer of the onion, what you see is that every aspect of our educational systems in America are wrapped up in the question of the problem of grades, and I won’t go on that train, we’ve been there before, but I’m saying that there are analogs to the question of AI and equity gaps, and that Emily’s metaphor here of band aids is, I think, really an instructive one for thinking through the ability of AI to address those gaps.

Rebecca: We’ll put a link to that podcast that discusses those learning assessments in the show notes.

John: As well as a discussion of Justin Shaffer’s work. And I do use some very similar techniques, but even then, students, when they’re working outside of class, sometimes get stuck on things. And it might be nice to have some sort of tutor, if it’s designed in a way that doesn’t directly just provide answers, and that’s hard to do, because most systems do break down when they’re repetitively queried.

Emily: And I’ll add, to carry the metaphor of band aids, sometimes a band aid is all you got, right? And if a band aid is what you have, like, systemically, we need something else; on an individual level, if what you have as a band aid, and the band aid is helping like, I don’t judge anybody for that, right? We all have to use band aids sometimes.

Marc: Yeah, I think that we should be thoughtful about this in terms of not just students and interacting with them, but John did mention K through 12 and how this is actually becoming a major issue within their landscape. In fact, the Walton Family Foundation report that was through Gallup came out about a month and a half ago. It indicates that teachers are using it to address the material conditions of their labor in K through 12, saving between six to seven hours of labor a week, translated about six weeks of labor a year. That’s going to have a huge effect on our world coming into higher education too, because students are going to have certain expectations about when their assignments are going to be returned. And if teachers are using AI in K through 12 to grade student assignments, they’re going to be expecting us to hand that back to them very quickly. And that deeply concerns me. I was just talking with some of my students here for the first week of class, one of them said that her mother is a 10th grade English teacher and uses it to grade essays. We need to start really thinking about this in terms of how we are approaching equity within our landscape of teachers here at the university level, and how this is going to be addressed on campuses. As Josh mentioned, academic freedom is the sort of law of the land in terms of technology usage here. That door swings both ways, in terms of someone who wants to not use this tool, but then someone who wants to go overboard and start just automating everything. We need very clear guidance, and we really need to start talking as communities about where we think this technology belongs, in terms of that space for our own labor.

Rebecca: I think Marc just opened up the door to a policy discussion around AI use to some extent, because institutions are making some decisions about the availability of tools, putting guardrails in some places and not others, and the entire university system includes a mix of faculty, staff, and students who may or may not be using AI. So can you talk a little bit about the role faculty might want to play in policy development around this topic, or even what it might look like at the individual level in a class when you’re trying to negotiate the absolute use that some students are using AI, and maybe the learning outcomes that you have or the policies that you have in place. Like Josh, I’m thinking about if you’re not allowing the use of AI, but obviously some students are going to use AI, how do you navigate those situations?

Josh: Right, and a lot of that’s tied to my grading strategies of collaborative grading, and so removing some of the incentives to use AI in the first place is a part of that. And Marc has done a lot more at our university in thinking through kind of institutional policy. I’ve been on a committee. Mark, do you want to talk a little bit about this? I mean, my own view is that faculty should absolutely be deeply involved in creating a policy at an institution. But Marc, do you want to say some things?

Marc: We are doing a pretty good job of involving faculty in the actual conversation. It does feel like we are kind of in this endless loop, though, that the main guidance that we’ve established, and also, I think other universities have established too, is it all goes back down to faculty freedom. We’re giving individual faculty members wide leeway in deciding what is appropriate and what’s not appropriate inside their classroom, what’s appropriate in their discipline. We don’t really know how certain faculty are going to be experiencing using these tools to completely automate certain things that I have troubles with because of my values, like, I would never want to automate completely feedback. I would never want to automate grading, or emails even, or letters of recommendation. Those are all things that I value, being a human being with my students and the relationship that we have. It is becoming apparent, though, from some different research that we’ve had too and some conversations with people that’s other than faculty, think very differently about that, and I think we’re gonna have some problems with this. I don’t really know. I know, John, your background is in the economy. I don’t really know many economic examples where some members of the workplace choose to automate the majority of their labor while others don’t, and how that works out long term. So I do think we need some very firm guidance about automation that keeps the role of AI into words of collaboration and augmentation. And really does have that to be a little bit of a hard line about using that to be incorporated so heavily that it starts impacting a person’s day-to-day job, their day-to-day life.

Emily: The only thing I’ll add is that I think students should be a part of this conversation as well, like both on an institutional level and on the individual classroom level. I try to involve students as much as possible in creating my AI policies and letting them choose kind of the pathways that they want to take, especially when they’ve got all these different instructors doing different things in their different classrooms, they don’t always know what’s going on across the university, but students do? Students have a good eye about what’s happening in their class on this side of campus versus the class on the other side of campus, and I think can give us some really good insights. So I want to advocate for involving students in policy discussions as well.

Josh: Yeah, completely agree. Great point, Emily. I was having a discussion with my students on our classroom compact yesterday, and this was one of the discussion questions technology and AI use in education more broadly, but in our class, specifically.

John: From faculty’s perspective, one of the things we hear most often is concern about assessing student work when it may not necessarily be their work. What are some strategies we might use to encourage students to behave ethically, or perhaps to design assignments that are less subject to inappropriate or unethical AI use.

Rebecca: And I’ll add interventions that might need to occur when unethical behavior has occurred, because I think that’s one of the things that faculty are deeply struggling with.

Emily: So one of the things I keep repeating about AI use these days is that we don’t actually have an AI problem in higher ed. We have a crisis of purpose that AI has exacerbated, and AI has made this whole [LAUGHTER] kind of higher education project untenable given the crisis of purpose that we have. So I mean, I think that school is increasingly viewed by students as a transaction. I think a lot of faculty have noticed this, and they’re doing work to get grades, to get a degree, to get a job, and of course, that’s not students’ fault, because that’s what we’ve told them college is for since they were kids. But I think maybe we’re just starting to wake up to all the ways in which we as teachers have inadvertently encouraged that transactional mindset. So I think a good response to AI would be thinking about how we get that out of our classrooms. And so grading systems, I think, are the main way that transactional approach shows up. So if you take grades away, you take away a really important token in that transactional economy, and then what’s left, right? If you take the grades away, well, like hopefully learning is left, and it takes students a little students a little while to adjust to that [LAUGHTER] and to understand what the new purpose is when there aren’t grades. So that’s what I’m always trying to do in my classes. And we could talk forever and ever about alternative grading and different approaches to grading systems that help students move out of that transactional mindset, but if they’re not in that mindset, if the point is not just to create a product to get a grade, if the point is to go through a learning process, and if we incentivize going through a learning process as opposed to just generating a product. In my classes, I’ve had good success seeing less AI use or misuse than one might expect. But the other thing, I think, is just talking to students explicitly about purpose. Why are we here? [LAUGHTER] What are we doing? And one of the things that I really like to cite when I talk about this, Josh and Marc will have heard me talk about this, and maybe tired of it already, but there’s a study that came out about 10 years ago by David Yeager and a whole host of co-authors called boring but important, a self transcendent purpose for learning fosters academic self regulation. And one of the things that they found in this study, or that they show in this piece, is that giving students what they call a self-transcendent purpose for learning, so that’s a purpose that’s about doing something good in the world or helping someone besides yourself, giving students that kind of purpose can help them better persist through tedious or challenging tasks. So I think one of the mistakes we’ve made when we’re talking to students about purpose is to focus exclusively on their self interest. How is this going to help you pass the next test or get into the next class, or succeed in the next class, or succeed in your job or whatever? And I really think we should lean into kind of bigger questions about the purpose of our discipline. What kind of good does the work that we do here do in the world, and how can you use the knowledge and skills that you are developing here to do good things, to help other people? Because I think, you know, none of us are here in purely self-interested motives, right? We don’t get paid enough for that. And I think the same is true of students. We all want to do good things in the world, and so one of the things that I’m really trying to lean into as a way to motivate students and kind of mitigate the temptation of misuse AI, is to talk to them a lot more about purpose and help them buy into that, create structures that incentivize their engagement with the process.

John: We’ll put in a plug for David Yeager book 10 to 25 because it contains a summary of that and much other research done on student motivation. And it’s a superb book to address those issues.

Rebecca: I believe we have a podcast episode on that that we can link in the show notes.

JOSH I would just add two things to what Emily said, I completely agree about the motivation structure that grades bring to courses, we’ve incentivized the use of things like AI when we put all the emphasis on extrinsic motivators. So there was a piece Inside Higher Ed today that was basically a recap of survey results, and there the two top motivations for students to commit acts of academic misconduct or cheating. One was the pressure to get good grades, that was the top, and the second one was short on time. And so there are pedagogical innovations to address both of those things. Emily addressed grades. I would say that there are some alternative grading practices that address the second one as well, the time. But there are other things, like flexible deadlines. There are ways to address the time issue as well that take those top two reasons that students might turn to AI off the table. So that’s one thing. The second thing is, we have a lot of great work, Cate Denial, Peter Felton, etc. many people who are pointing to relationship building, trust building with students. Now, there’s this great facet of human nature that when we feel like we are trusted, we want to be more trustworthy, and so creating classroom environments that really build those kinds of connections and relationships, I continue to feel, will mitigate some of this use as well.

Rebecca: I think one of the things that we see faculty doing, I think, out of desperation, really is turning to AI checkers and things to evaluate or find a way to identify whether or not it’s the student’s real work. And obviously a strategy is to try to mitigate the desire to cheat in the first place, but some people will still do it. Rather than going to an AI checker or something, what are some other ways to maybe address the issues that arise when students do cheat?

Marc: Well, I think cheating is a very loaded term here, too. This is what becomes very complicated, because some faculty have policies that are banning AI in every aspect of learning, which I’ve told them is not really appropriate or acceptable, because you can’t possibly enforce this. And we’re starting to see a lot of students using this on their own time outside of class to work through problems, to research. So we have to really define what is academic misconduct using AI in different terms. We also have to be aware of what the actual technology can do, too. And now multi-modal AI is here, it’s free, the voice in ChatGPT, the actual turn on the camera, it can actually start scanning things around you. That adds more dimension to this than just text, and so we have to be really aware of that, and also the fact that AI agents can now automate quite a few different things in the research process, are all adding to this sort of process. So to me, it’s always going to come back down to a conversation with a student that goes back to openly disclosing how an AI tool or process was used, and bringing that up. And that works really well in a small classroom where I’m supported and have time to do this. It does not work well when I’m teaching a large lecture class, too. If I have 80 or 100 students, that’s where this completely falls apart, and people turn to AI detection services or tools. And there are a lot of them out there right now. In fact, I think Turnitin just released a humanizer detector, because a lot of people have been using AI to humanize the writing, to make it sound more human. And now Turnitin now has a system that supposedly is going to detect when you’ve humanized it, and their other system that they just released too is called Clarity, which is a process tracker that allows you to view students writing long term for days, even weeks at a time. And I’m really against surveillance in this sort of sense, because, again, it’s not really a pro teaching sort of strategy. It also doesn’t really work very well. At the same point in time, I’m very aware that a lot of faculty are sort of struggling with this. And what I tell them, if you do turn to AI detection and you’re starting to bring your students up on academic misconduct charges, you need to be aware of the fact too that there’s very little procedural fairness involved in this. And I do flip that around on them. I said, Look, if you’re going in for promotion and your dossier got ran through an AI detector and somehow came up as positive, what defense are you able to offer it? And I’ll give you a hint, there really isn’t one, unless you’re keeping track of all your documents five or six years ago and how you wrote them. So we need to be really cautious about how we’re dealing with this. And I think the best thing to do is to obviously bring them in and do this. University of Texas has a really great restorative justice program for academic misconduct. Penn has been piloting one for 10 years now, in fact, that’s the second most likely outcome that they have for an academic misconduct is they go through restorative practice to see if they can’t mend the trust that was broken in the classroom before taking a student up on charges.

Emily: Yeah, I appreciate a lot of what Marc said, and I appreciate the question Rebecca, because I feel like sometimes, especially when Josh and I start talking about this, people think that we’re a little like out of touch or something. [LAUGHTER] Because when we’re talking about student motivation and engagement et cetera, some people are like, “Yeah, yeah, yeah. Like, that’s fine, but that won’t solve the problem.” But obviously, as you say, there’s no kind of perfect solutions to this. I think that we should lean into that student motivation and engagement piece, because really, I haven’t seen any kind of viable solutions for mitigating AI use outside of that that I’m comfortable with, but I do think there are room for some kind of assessment security measures, or kind of, however you want to phrase that, as long as we’re varying the kinds of ways we assess students, occasional in class, worker assessments. I think oral assessments are tricky in a number of ways, but may have a place. And I think in terms of what to do when students have misused AI when you’re following up, I think Marc is absolutely right that conversations are the way to go when you can do that. I’m also like Marc, pretty uncomfortable with a lot of process-tracking measures. I tried to do this on my own. I recorded myself writing a blog post, and that totally changed the way I wrote. And I did not have a good time when I was writing, knowing that somebody would see that later. And so I have a lot of mixed feelings about it. One of the things that I do is I do ask students to write in Google Docs. I do tell them about version history and show them how version history works. We talk about this. We set these guidelines together at the beginning of the semester. If I have concerns about your work based on my own reading of it, I might go into the version history to like, look at how your piece came together, and I will ask you questions about why did you write this? What was your process like? Can you tell me what you meant when you said whatever? And again, that’s something you can do with smaller classes. It’s a lot harder in larger classes. So there’s no perfect answer here. And I have a lot of mixed feelings about all of it. [LAUGHTER]

Josh: That’s for sure. There is no perfect answer. And when I hear Marc talk about all the twisting and turning that ed tech companies are doing to try and pile band aid on top of band aid, I guess I just immediately think, if only we use this much time and money to solve other problems in our world of education, that’s ultimately what I think. Continuing my theme for the day, when I was a graduate student and first started teaching writing, it was before Google launched, and so we were taught how to handle plagiarism, was to have a conversation with students, was to ask them about their ideas, to say, “Oh, this is an interesting word. Could you help me understand why you used it in this context?” And that part of what would emerge from those conversations would be either an admission, “Okay, you’re right. I didn’t write this” or it would be a really interesting intellectual conversation about their work, “I used it because of this, and here’s why I…” you know. So I still think that, rather than using machines that may or may not have various degrees of fallibility in this, I think those conversations are ultimately a productive, pedagogical way to handle it. Marc’s right. It doesn’t scale very well, though, so large classes present a problem. In my own classes, I use collaborative grading, which means that I don’t give any grades. Students do a lot of self assessment. I give a lot of feedback. They propose their final grade at the end. I know Emily does this as well. I take grades out of the equation. I take deadline pressures out of the equation by having best buy dates that they can petition to change for themselves if they need to for some reason. If ultimately… I realize I’m an outlier here… if ultimately, even within that context, a student is going to use AI that’s going to be their decision. If that’s what they want out of it, then I’m gonna teach the way I feel like I need to teach, and I’m gonna work with the students in the way that I think will be as meaningful as possible. And people, ultimately, Rebecca, to your point, are going to continue to be people.

Rebecca: I just wanna just note that if there’s a tool that’s detecting humanization, I wanna know like, is the opposite then the dehumanization like, it’s an interesting word choice. I just want to put that out there. But I was just mostly sitting here thinking that’s really like an interesting word choice. [LAUGHTER]

Marc: It is. I do think something for both Emily and Josh that might be helpful to talk about… Grammarly just launched a new product about AI agents that supposedly can actually grade a teacher based off of a rubric that the student supplies from the teacher, and also includes sort of a scan of the sort of public persona of who that teacher is, and gives it a best guess. I have some very personal feelings about this that are not very positive, too, but I wanted to get you guys take on this about how we’re seeing that sort of transactional framework being taken up by other companies as well, too.

Emily: Yeah, I’m not a fan. [LAUGHTER] I mean, there’s so many reasons. The main thing for me is that it keeps students tethered to that grade motivation, right? Like, if you’ve got these Grammarly tools that are gonna guarantee for you, I can help you write an A paper, and I can grade it for you and tell you that it’s gonna be an A, then students are writing to conform to whatever Grammarly tells them an A is. And of course, I think the idea that it would scan the Internet to determine what your instructor might think based on their like Rate My Professor comments or whatever, is incredibly disturbing. And the only good thing that might come out of it is that instructors might get a sense of what it is like to have AI reading you in a way that might make them more cautious about putting their student work into an AI.

Josh: Yeah, I wrote a little bit about this on LinkedIn, and I just want to thank Grammarly for helping Emily and I to make the case that grades are truly subjective and kind of pointless. If the machine can gather some of these random bits of information and suggest what might possibly be a grade that I myself would give, I think that that just explodes the notion that a grade is something that meaningfully measures learning. It’s absurd to me, but I think that that has been a through line here, of a lot of my comments, but that especially, I think, takes it to a new level.

Emily: I guarantee Grammarly does not grade the way that I would. [LAUGHTER] My priorities are not Grammarly priorities.

Marc: I think the big picture we’re seeing, too, is that there’s the performance versus the reality, and that’s what we’re kind of getting back into the overall concept here, where we’re talking about. Teachers are human beings. We’re still flesh and bone for now, hopefully that’s going to remain for this future too. Yet there’s still a huge desire to have that performance. I’m sure Grammarly too thought about this. I’m sure they beta tested it too, because it’s what students want. I want to know what my teacher thinks about this essay at midnight that I’m writing this. So I think we need to start thinking about how we can communicate our value and what we provide to students too, in a way that is better than what a company or a product or just AI in general can offer. And that’s going to be incredibly challenging, because right now at higher education, we are on the receiving end of a fire hose of these different products.

John: We are reaching the end of the time we’ve scheduled. So we always end with the question: What’s next? …which, in anything dealing with AI is something we’re all asking on a daily basis, it seems.

Emily: Well, this semester, we are doing at the Center for Excellence in Teaching and Learning a reading group on the book The Opposite of Cheating: Teaching for integrity in the age of AI, and Marc is also doing a number of AI workshops with our academic innovations group. And to go back to the grading conversation, I think what’s next is I’m continuing to work on a book on collaborative grading, and Josh and I are working on another book on alternative grading, and so we’re not going to explicitly talk about AI too much in those books, but I think they have a real bearing on some of the AI stuff that we’re seeing right now.

Josh: I think an important message to share what’s next is keep doing the work. We’re going to keep doing the work of understanding, learning, and working with our students. I think certainly we will continue to have conversations about AI, but a message from my what’s next, the message that I am continually promoting to administrators here on campus is that AI is one thing among many things that are still at issue in the world of teaching and learning, and so we need to still be addressing those as well.

Marc: Yeah, I think that’s a really great point to end on, too. For me, I’m focusing much more about how multimodal AI is affecting our mental health, in some ways, especially teen mental health. We’re seeing a lot of use cases for therapy and companionship, for these tools that you can talk with. I think that’s going to be something that explodes as a topic over this next year. I also think it is going to be imperative for us to have conversations with our students about how they’re using these tools, because very few people are really talking to them about it and asking them if you are using it, are you thinking you’re talking to a person or to in it, a machine or private company giving all your private information to it? So I think more or less caution is where I’m headed for this with AI, I think we’ve got far too many different products and different use cases out on the market to really keep track of, so I would hope we’d be going to slow down, but I don’t think that’s going to happen anytime soon.

Rebecca: Well, thanks for a really great conversation. We always enjoy talking to all of you, and it was great to have a diversity of perspectives and a more nuanced look at this topic today.

John: Yes, thank you. It’s great talking to each of you, and we’re looking forward to many more future conversations. And I’d also like to put in another plug for the Opposite of Cheating. We’re going to be doing a reading group on that here as well this semester.

Emily: Thanks, John and Rebecca,

Josh: Thanks everyone.

Marc: Thank you guys.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Madison Lee.

[MUSIC]

409. Vibe Coding

Posted in edtech, educational technology, higher education, learning, teaching
September 3, 2025

Imagine having the ability to create personalized web applications that address your individualized needs. In this episode, Dave Ghidiu joins us to describe how you and your students can create such tools using a vibe coding process using free generative AI platforms.

Transcript

John: Imagine having the ability to create personalized web applications that address your individualized needs. In this episode, we explore how you and students can create such tools using a vibe coding process with free generative AI platforms.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Dave Ghidiu. Dave is an Associate Professor of Computing Sciences at Finger Lakes Community College. Prior to his time at FLCC, he spent a few years as a Senior Instructional Designer at Open SUNY, where he was a lead designer for the OSCAR rubric software. Welcome back, Dave.

Dave: Thank you so much. It’s so great to be back.

Rebecca: Today’s teas are… Dave, are you drinking tea today?

Dave: I am drinking tea. As you know, I get my tea from Saratoga Tea and Honey, and I’m drinking the Broadway Berry today.

Rebecca: It’s got a nice name.

Dave: It is. It’s the first time I’ve ever had it.

John: They have some wonderful teas there. I haven’t been back there in about a year and a half or so.

Dave: Oh, it’s worth a trip.

John: I’ll be heading back there soon. So, I’ll be picking up some new ones.

Dave: Excellent.

John: And I have a wild blueberry black tea from the Republic of Tea. I don’t have any teas from Saratoga Tea and Honey this time.

Rebecca: It’s disappointing. I just have a Lady Grey today.

John: So, we’ve invited you here today to discuss how you’ve used generative AI to teach vibe coding. To set the stage for this discussion, can you first define what vibe coding is?

Dave: Yes. So earlier this year, there’s a famous computer scientist, Andre Karpathy, and he was one of the founders of OpenAI. He was the director of AI at Tesla, and he just made this offhanded tweet about vibe coding, and it’s essentially using natural language to make software applications. And he’s using it in a much grander way than I am and then I’m advocating for. And I actually don’t even like the word vibe coding. His whole tweet goes into like feeling the vibe of the software and the programming, and getting all involved. I think it’s really more of a dialogical coding experience. And I stole that term from Curt, whom I work with. He’s in the humanities department, and it’s really just having a conversation with AI to create software. And so you can imagine, if you want to create a simple ToDo app, instead of writing the code yourself, you just describe the app to an AI like Claude or ChatGPT and it would generate the code for you. Then you could refine the app by having a dialog with the AI and asking it to add features, change the design. And I think we should do some level setting here, because I just want to make it clear that this is entry level. This is for everybody. If you can’t code, that’s kind of the vibe that I’m going for. I want to help you take your ideas and turn it into code. We’re not making enterprise software, we’re just making small web apps. And you can think of it as like H5P which some of you might be familiar with, or Quizlet, where you can get these little interactives that go into your course in your learning management system. And I’m not using Python or Java, we’re just using HTML and other tools of web design.

John: So what you’re saying then is that really no knowledge of programming in pretty much any language is needed to engage in a vibe coding project?

Dave: Absolutely and again, at the high end, yes, there are people out there who are paying lots of money to have the really expensive tools: Cursor, Windsurf, Claude Code, but that’s not what we’re talking about. You can do what we’ll be talking about with the free tier of any of these AI services.

Rebecca: Is vibe coding a process that students and faculty should consider, and in what context?

Dave: I think the process can and should be abstracted. So at a high altitude, vibe coding is really just prompt engineering. It’s AI literacy, and we’re SUNY schools, so you know that AI literacy is part of our general education now. And it’s really just learning how to vibe code is not much different than using AI to generate an image or a video or help you with any text that you’re writing. The challenge is you have to imagine it in your head, and then you wrangle the AI to make it come out. So you can imagine that you have content in your learning management system, and we use Brightspace here at SUNY, wouldn’t it be neat to embed some slick interactives in the middle of the reading to check for understanding? So maybe the learners are scrolling through reading it, and they see a little Wordle that is confined to only the vocabulary from the current unit. Or maybe there’s a deck of cards, not like playing cards, but just cards with words on them, like facts, and they can swipe left or right to kind of test their knowledge to see if it’s true or false. And actually, I was listening to your podcast, the recent one, where you had the student who was presenting their honors paper, and I got to think, I was like, “Oh, that would be really cool. So if a learners doing research, and let’s say it’s on water quality in a conservation class, and we have big conservation program here at FLCC, maybe they could, instead of presenting at a conference and having charts and graphs from Excel, what if they could have a fully interactive data model, and you can pinch and zoom and you spread out all this data and really interact with it?” And that’s possible now. And in fact, a few years ago at FLCC, there was a learner that I was working with. She was in the creative writing program, and she came to me, she’s like, “Dave, I have this idea. I want to make a campus-wide Mad Libs, and I need to have a website.” And she was going to crowdsource all these different things. And so the users would go to the website and it would have a prompt, and they’d type it in, and then she aggregated all and made like this website out of it, and it was like, this creative story, that kind of captured the vibe of FLCC. And she designed it, and we worked with it, but I did most of the coding. It really wasn’t effective for me to sit down and teach her how to code for this thing. But now if someone came to me today, and was like, “Hey, Dave, I have this idea,” I’d be like, “Okay, give me 30 minutes of your time. I’ll get you started, and then you can come back to me if you get stuck.” But that’s the reality now.

John: It’s an interesting project where someone is capturing the vibes of the institution using vibe coding. So we’ve got that meta level going on there. [LAUGHTER]

Dave: It’s very meta.

John: So basically what you’re saying is just using one of the generative AI systems, students can generate the code and then have it created in the form of JavaScripts that could then be inserted into a website.

Dave: Yeah, so I may have lied a little bit earlier when I said you don’t need to have any experience, but it’s very little. You don’t have to understand HTML or JavaScript or CSS, and for those of you listening who care, HTML is the content, the words, CSS is the design of it, the colors, the size of the font, and JavaScript is the interactivity. So there’s usually, like, three different components to a website, and the AI systems, they manage all that. The challenge is, once you design it, and it’s in your AI system, where you can download to your computer, you might say, yourself, “Self, well, how do I get this into my learning management system.” So I suggest, there’s your tech stack… if you want to sound super cool and super nerdy… the tech stack is, I use AI to create it, and I store the code in GitHub Pages. GitHub Pages is like GitHub, except it can render the pages, so you can actually store websites up there. And then in Brightspace, or in the LMS, I embed the code that lives in GitHub. And I know that sounds complicated, and I hope your listeners aren’t tuning out right now, because I do want to just say the very first time you do it, it might be a little hard, it might be a little confusing, but it is super easy, and once you get started, you’re golden, and you will have so much fun vibe coding, making like all these different apps.

John: And you can do that as a free account on GitHub, or do you have to pay for an account?

Dave: Free account. In this whole endeavor, I want it to cost $0, to have minimal knowledge of technology and so this tech stack is that. It celebrates you don’t need to know anything, you don’t need to pay for an account. Now, I do pay for an account at Claude, and I think Claude is the absolute best for code generation, I find Gemini 2.5 is the second best. And up until [LAUGHTER] today, I would have said ChatGPT is the third best, but ChatGPT 5 dropped, and it hasn’t dropped into my account yet, so I can’t test it, but ChatGPT 5 is supposed to have some pretty slick coding in there as well.

Rebecca: Has vibe coding been something that you’ve been able to use with students to demystify programming?

Dave: It has. So it certainly has the potential, and I haven’t applied it into my classes yet, but this is something I intend to do in the fall. So we’re at this inflection point in computer science and in programming courses and even how we teach it. And I can imagine a JavaScript class that I teach where on the very first day, we start coding a web app, and we do it using Claude, or pick your favorite AI, and then we put it on the projector and like, “Okay,” and for this conversation, let’s say it’s a to do list. So we come up with the features, and we come up with this prompt, and we put it up there, and it generates the prompt and, recognize that at this point, some of the learners might know some HTML and maybe have heard of JavaScript before, but that might be the extent of it. But that’s fine, because now you can look at the code, and you can look at it side by side, because all these AI platforms now have that canvas on the side, and then you can start making features. So I’d say, “Hey, what feature do you want to use?” and a learner might say, “Oh, I’d like to change the colors of the check mark”, or have the user change the check mark to an X or a smiley face, or whatever in this to-do list. And so then we put that into the prompt, and then you can see when the code changes. It’s eerie. It like backspaces all the code. It doesn’t just wipe it out, it backspaces it character by character. So you can actually see it and use it as a learning experience. And you can go a little bit further than that too. So if I were in this aspirational example, I might have the learners say, “Oh, well, I didn’t understand what happened.” And so we put that into the prompt and we say, “Well, can you explain what you just did?” and then it can give you a natural language response to it. So it’s like having a software developer that is at your beck and call all day, every day.

Rebecca: And who’s willing to speak English.

Dave: Yes, [LAUGHTER] or whatever language you want.

Rebecca: True! [LAUGHTER]

John: Taking that just a little step further, do you think this might be something that might get more people interested in learning coding, once they have this additional tool to help them get started with this? Because a lot of students may be intimidated taking courses in computer science, because it’s like a foreign language to them.

Dave: Yeah, and I’m gonna get a lot of hate mail for this, but if we think about the current state of computer science, and even in this conversation, we talked about the breakneck pace of AI like ChatGPT5 dropped today. Claude dropped yesterday. OpenAI just gave every employee a $1.5 million raise. All that happened this week. Google just dropped the study mode, Google just dropped the children’s book. All these things happened in the past week. So there’s no avoiding it. You can’t escape the velocity of AI. And let’s talk about non-majors for a second. A lot of non-majors are taking Word, Excel, and PowerPoint as their computer science course. So imagine the institution next door, and they don’t have that as a computer science course. That’s like the table stakes. Instead, you come into this vibe coordinating class, maybe it’s a 15-week course, even if you know nothing about coding, by the end of it, you’re going to be weaponized and making web apps, and you might understand the lingo a little bit more, and it doesn’t matter if you do or not, but you can make these things to help you. And I gave the example of the conservation learner earlier, but imagine there’s someone who’s going into some type of program like mindfulness. You can make the breathing apps that have a little needle that says, inhale, exhale. So all of a sudden, these learners are start thinking in this like fourth dimension, where it’s tangential to their domain, and it’s no different than creating artwork with AI or creating text or video.

Rebecca: What about things like accessibility when we’re thinking about developing these web apps?

Dave: I knew you were gonna ask that, Rebecca. [LAUGHTER]

Rebecca: I appreciate that you knew I was gonna ask you. [LAUGHTER]

Dave: Yeah, and you’re the accessibility guru. So I have a generic prompt that I give, and this is the secret sauce. So everyone out there take notes. There’s a few different things that are kind of what I call non-negotiables when you’re making a prompt for the software. So my secret prompt, my secret sauce, whenever I start prompting, there’s three or four things that need to get in there, and the first is to curb the default behavior of these AI systems. Most of them try and use React, which is a nerdy framework that us computer nerds use. So I always say “I’d like to make a web app. I’d like it to use vanilla JavaScript, I’d like it to be responsive, and I’d like it to be accessible. So that’s how I start the prompt, just with those four parameters and 95 out of 100 times what comes out is exactly that. One of the things that I think is important, Rebecca too, is like, you need to test this and this is true for all AI everywhere. You are the supervisor, like you co-create, you don’t abdicate. So it’s important to test that accessibility and you kind of have to know what you’re looking for. And in general, when we’re talking about web apps, you want to make sure if you have images, there’s alt tags. You want to make sure that there’s the textual hierarchy with Heading1, Heading2, and all those things are almost always present anyhow. And then you also have to test it for responsiveness. So test it on a mobile device, because you never know how many of your learners are out there on mobile devices.

Rebecca: I would add in probably check it with a keyboard too.

Dave: I’m glad we’re having this dialog because we’re vibe podcasting right now because that’s exactly what you should be doing, is let’s say that you generate this software and for this case, let’s just say it’s a interactive data thing. You might want to say, “Hey, let me test this with a keyboard,” and if it doesn’t work, then you say to your AI of choice, “Oh, this doesn’t work with a keyboard. Here’s the functionality I’d like to see.” So I think that’s really important. In fact, one of the best skills I’ve acquired, and I’ve been programming for many, many years now, is debugging, because all the load of coding is done by the AI. I could spend way more time testing it and testing those edge cases and thinking about how it’s gonna be used.

John: We saw a presentation that you put together at the SUNY Conference on Instruction and Technology earlier in the summer, and you talked about developing a course on vibe coding. How has that process been going?

Dave: It is going great. Thank you for asking. One thing that I’m excited about is at Finger Lakes Community College, we’ve had some phenomenal support in the AI realm from the President and the board, so they’ve given us carte blanche, top down support. And so what we’re seeing is pockets emerging from the grassroots, like how we’re going to be using AI. And one of the things that I’m most proud of is we’ve launched the FLX AI hub. We launched that on June 11th, and one of our charters is to support educators in implementing AI. So we are dropping one course every month, and you can think of this as one-hour course to maybe three-hour but on average, about two hours a course that’s going to drop, and it’s going to be relevant for educators. And we have at least one vibe coding course dropping this academic year, possibly two, because there is this whole second level where you can use Google Sheets as a database for some of your apps. And I have some examples of that if you want to see them. I just want to mention you can sign up for these courses, or at least get information on them at flcc.edu/ai.

Rebecca: Well, that one should be easy to remember.

Dave: [LAUGHTER] I hope so.

John: And we will include a link for that in the show notes as well.

Dave: Thank you.

Rebecca: Can you talk a little bit about how you see the vibe approach preparing students for a future where AI coding is ubiquitous, if we are saying that it’s not already. [LAUGHTER]

Dave: I think it almost is ubiquitous. I think that vibe coding, if you abstract it, it’s no different than prompt engineering, than AI integration specialist, like that you are wrangling AI. One of the ways it frees learners is by giving them the space to think about program design and computational thinking. And again, I’m gonna get a lot of hate mail for this too, Software engineering isn’t learning where to put the semicolons or learning the difference between Python and your indents or JavaScript. It’s great if you know those languages, but really it’s about understanding what the problem you’re trying to solve is and designing a flow for that. So already, there’s been conversation in this field for probably the last decade about low-code environments, and we have environments where they have like blocks that you can plug in and make the software go, but this is on a whole ‘nother level. It’s just so freeing. I was born in the 70s, survived the 80s, but I grew up on an Apple IIe and for a long time we were limited by the technology, even if we had grand ideas. And then there was this era that ended probably just four or five years ago, where we were limited by our imagination, so the hardware was no longer an issue, but we couldn’t think of what to do. But I think we’re at this inflection point where now we’re not limited by imagination or by hardware, because when you’re using AI, and you know this with regular conversations too, that aren’t vibe coding, when you’re done with a conversation, it might say, “Oh, did you think about doing A, B or C?” Like, “Would you like to potentiate your work a little bit more?” So I really love where this is going, and I think that if it’s not ubiquitous, it will be six months, a year from now, at least at this level. And I’m not saying that we don’t need Python programmers and Java programmers, because we do. But what I am saying is the entry level is a lot lower now, and maybe someone starts vibe coding and they absolutely love it, and they’re like, “Oh, I do want to go into software engineering. I want to get into the nitty gritty.”

John: Now, in terms of teaching computer science classes, how will you be able to assess students’ knowledge of basic programming skills when they have all these AI tools available?

Dave: When AI first came out in November ‘22 I said to my learners, “absolutely no AI”. And then over winter break, I was talking to one of my buddies, and he programs Python for a living, and there’s a Python class I was teaching, and he does a lot of work with some big firms up here in Rochester. And Malcolm was like, “Oh, I don’t care how people get their software done, as long as it works.” I was like, “Okay.” So that semester, I was like, “Hey, folks, all bets are off. Use AI as much as you want.” And I think that was a good experiment for me and there was one lab that was unintentionally designed to be AI proof, and it was like two thirds of the way through the semester, and I got so many emails and people were lost, and I was like, “Oh, you just have to do a loop.” And they’re like, “What’s a loop?” And I realized what had derailed from this process. So now, for using AI for the first two weeks, not allowed to use AI in a capacity. Then for the next two or three weeks, depending on where we are, they can use it as a study guide. And actually, I took your chatbot course at CIT this past summer, so I will be rolling that out so I can kind of confine the knowledge in there, and then after that, once we get through the fundamentals, once learners understand input, output, variables, if statements, and loops, then they can start potentiating with AI. And one of the things that I think is important in this contractual agreement with the learners is, “Hey, if AI spits something out and you don’t get it, just ask, ask it. It will totally explain it to you.” So I’m hoping that this can be like a Khan Academy type tutor bot type experience, self governed by the learner.

Rebecca: Do you have students in your classes that are building software explain how they built it as a way of assessing that, or is it really just the final product?

Dave: So to be clear, I haven’t rolled out this AI vibe coding mentality 100% yet, this is a work in progress, but one of the things that’s really important in software engineering, and you can ask anyone who works at any company who programs is commenting, and comments are lines of code that can be in plain English that the computer doesn’t process. And it’s very good for if you have a to-do list like, “Oh, don’t forget, I have to do this, this and this feature,” or if you have code that you don’t want to execute, you can comment it out. So the comments are kind of like the user’s guide. And I’m leaning more and more towards saying, “Hey, you really need to do great comments in here.” Part of the challenge is AI does comments. In fact, that’s one of the ways you can tell if a learner’s using AI and just copy and pasting, because if the code has comments, you’re like, “Hmm, that might have been generated by AI,” but I want to go to that next level, those really deep, rich comments that explain what’s going on. And I was working with some faculty at a K-12 school up here, and they said, “Well, what about essays?” and his is an English teacher, and the teacher’s like, “What if, for their submission, they can do AI generated, but I wanna see the 20 or 30 or 40 prompts they use.” Like, “I wanna see the thought process.” He’s like, “Would that be okay?” And I was like, “You’re the teacher, but yeah, that sounds like a great way to get people to learn to use AI, and you can see the thinking and the interrogation of what’s going on” and I don’t know that I hate that idea for programming either.

John: That’s the topic that’s come up in a lot of past discussions of AI on the podcast, where people are shifting from evaluating the final product, which could very often be easily created by AI, to an analysis of the process that students use, which would include all the prompts they use, and a description of in what ways they used AI and so forth. We need to prepare students for a world in which they’re going to be using AI tools in some form, and it’s hard to imagine what those forms will look like in the distant future, say, next month, but at least if we focus on getting them to use the tools effectively that might help prepare them for their future careers more effectively than assuming that they’re not going to be using the tools that are out there that make their work easier and perhaps more efficient.

Rebecca: Or enjoyable.

Dave: Yeah, I was just gonna say, like the self-determination theory, you want that autonomy, and you want that relevance and the competency, and you can have all of that with vibe coding. So just as an example of after that CIT conference, I was talking to Bridget and Nicole, who were two librarians at Brockport. They had actually presented on this software they made. They were vibe coding. They didn’t even really know it. They didn’t know anything about programming, but they were like, “We want to make this interactive version of Bloom’s taxonomy.” And they did and they presented on it. And then Ed Beck emailed me a week later and he’s like, “Listen, I made this interactive website. So you go there, and as an educator, you can click on: I want this assignment to have no AI, some AI, and then you pick your poison for your recipe, the guardrails for each assignment, and then it generates, like an image that you can stick into your document. And I did recommend to him that the image also have the alt tag with it, or you can make it in HTML so it’s embedded. So he made kind of like an assignment picker, and he did that using AI. And I can imagine a world… like right now my Python learners, their final projects are usually relatively all the same. Like, you have a certain amount of skills that you can use, and so you’re a little bit limited by that. But I love the idea of saying, “Hey, the sky’s the limit.” Like, “Don’t think small. Don’t do a to do list, because that checks off the box of arrays or lists and for loops. Think something big, make a game, make something that’s relevant to what you want to do.” And so I think you get that relevance and that excitement and that energy, and it’s no harder than it would be to do in a regular project but now they’re engaged in the process.

Rebecca: Circling back to what you were mentioning before about sharing the prompts and the reprompts and the reprompts and the dialog that’s happening, [LAUGHTER] I think encouraging students to submit those sorts of things or to document those sorts of things, really helps them recognize this is how they’re using the tool, and they’re disclosing that they’ve used a particular tool, and really enforces the idea of maybe some ethical behavior.

Dave: I hadn’t even really thought about that, but there’s that, like, metacognitive aspect of it, but also there is a bit of an awakening, because the way you just described that to me, I hadn’t even thought about that, but you can look at it and you can see the evolution of their confidence, like what threads they pulled, so you really get a more in depth understanding of their process. Oh, that’s great. You should teach vibe coding.

Rebecca: Well, I do teach coding, so…

Dave: All right, well you’re 90% there. [LAUGHTER]

John: Is this something perhaps that many faculty could introduce into their own use as well as their own instruction. I could imagine, for example, people who are teaching courses in statistics, econometrics, and other areas, using vibe coding to do some work in terms of working with databases, in terms of extracting data, reformatting data, and getting it in a form ready for use with statistical packages, in ways that may be much more efficient than the ways in which students are doing that manually right now.

Dave: Yeah, it really is the great potentiator. It doesn’t really matter what your discipline is, what domain you’re doing. We all know that data is creeping into every single domain, and it’s not a secret that technology is too, so why not just latch onto that and really just celebrate it? So you’re right. There’s all sorts of database work you can do and insights you can get from databases, but there’s nothing cooler than being some college learner who all of a sudden is making web apps that help them, or making software, it doesn’t have to be a web app, that helps them really internalize and they can share it and be proud of it. The sky is the limit. One other thing that is really interesting is there’s been an awful lot of talk about fear of failure, and failure to launch because of the fear of failure, and I really think that this is an opportunity to teach iteration, because it’s so private, there’s like, this micro failure, you do a prompt and it falls flat, it doesn’t do what you want. There’s no one looking at you. You just do it again and again and again, and then all of a sudden you get energized, like, “Oh, I can do this.” And then you customize it. So I think this is a really good on-ramp towards some type of iterative thinking process that might hopefully build confidence in other venues as well.

John: I know one of the problems that a lot of computer science professors have mentioned in the past is that they have students coming into their intro courses with a pretty wide variety of backgrounds. Some have never done any programming. Others have already built their own games and such things. And I would think that aspect that you just described, in terms of being able to work on things independently and experiment, might reduce that sort of psychological barrier where those people who come in with a little less background might feel much more comfortable interacting with the computer on their own time, without having to go to an instructor and admitting that they don’t know some things, where they see other students already able to do those things. So I think it may break down some of those barriers and equalize the starting point a bit for students.

Dave: Yeah, that’s a really astute observation and MIT created the language Scratch specifically for that purpose, because they were getting people from all different levels, and they’re like, “No, we’re all on the same playing field right now.” And, again like, maybe you should teach vibe coding. I didn’t even really consider that, but it really does level the playing field.

Rebecca: So let’s say folks are super stoked about vibe coding. [LAUGHTER] What are some examples of how to maybe get started?

Dave: I think if you want just that real easy entrance way into it and start to understand the environment that you can have it make a QR code generator, and usually it can do that in one prompt. That’s not a hard problem to solve. But then you might say, “Oh, can you give me a color picker so I can change the color of the QR code,” and you can make it download as a PNG instead of a JPEG or whatever, or you could do….and this is actually a problem that is a very pervasive problem in online teaching… in discussion forums in the beginning of the semester, I’m always like, “Take a picture of your work area, post it in this discussion forum, and let’s talk about what you like about it, what you don’t like.” And more than half the learners now have iPhones. And guess what? That takes pictures in HEIC, which is not web accessible. So one thing you might want to try is create an HEIC converter to PNG. That’s an easy win, it’s low-hanging fruit, and you can actually use that in your course. You can embed that and then use it.

John: Which is awfully convenient when students send you files in a format that is not a web standard or is not an accessible standard. So having that handy would be very useful.

Dave: Sure.

Rebecca: I just like your approach and using it to solve your own problems, starting with things like, “Oh, this is a barrier I face. Can I build something to remove that barrier for myself?”

Dave: Yeah, andI just started lifting at a gym, and I was like, “Oh, I need an app that does like, 60 seconds on 20 seconds off,” but then sometimes I want to be 60 seconds on 30, whatever it is. And so I just made an app that did that. I probably could have found one in the play store, but I’d have to download something. And now it’s customizable. And as I started growing this app, I was like, I want it preset. So on Mondays, it does this. So now I’ve built it, and no one else cares about that app, but I do, so I can control it exactly how I want it.

Rebecca: For a long time, we’ve talked a lot about user experiences that are personalized and individualized, and what you’re describing is completely individualized, because now you’re creating your own tools that are completely for whatever it is that you personally need or want.

Dave: And can you imagine the budget you would have needed, like, 10 years ago, or even 5 years ago if you hired someone to make an app for you that did, like a breathing time or something. It would be thousands of dollars. Now you can do it in 10 minutes for free.

John: This is not in any way related to the topic, but you did mention that workshop that Judie Littlejohn, and I gave at the CIT conference where we had people build chatbots. Could you tell us a little bit about the chatbot that you built while you were there during that session?

Dave: I would say, if you’re looking for an easy win for making your first chatbot, I believe the one I did was a Gen Alpha translator. So you could copy and paste paragraphs from your, say syllabus, and then it would spit it out in Gen Alpha speak. It was fun. It was maybe useful, we’ll find out, but it was just a big win. So it was nice to do that, it was my first chatbot.

John: And it was one of the most interesting ones that we saw there. We always end with the question, “what’s next?” which, in anything dealing with AI, is a question we’re always asking.

Dave: So, there are two things that we’re really proud of at the college. The first is this FLX AI hub, and the four pillars are AI across the institutions. That’s how we’re supporting educators and learners and the business side of things. The second pillar is supporting business and government for implementing AI into their organizations. The third is AI for the community. So we’re doing some courses for the community education about how to use AI, but the one I’m really excited about is the AI for educators, and that’s going to have the courses that drop every month. And you can go to flcc.edu/ai, for that. But the other thing, and again, because I’m such a fan boy of this podcast, I was listening to one two or three weeks ago, and Rebecca, you’d remind us about the 10-day accessibility challenge. We built a 10 day AI-challenge. And you can see it aichallenges.org and this is free for any institution to roll out. It starts very simple. We identified an appetite for AI, but sometimes people are embarrassed and they don’t wanna sign up for an account. So you can sign up, or you can just go along, every day, we’ll release a new one, or you’ll release one for your institution. So the way it worked at FLCC is we picked these challenges and we emailed them out once a week, and it was very simple, and there was a sharing component, and the sharing component, you could go to this forum and type in like, say you were doing images this week, you could type in your prompt and upload the image, and then you could share it with everyone else at the institution. So we have about 15 different challenges, and we’ve been working with other organizations. If they want to roll it out, we’ll give them their own private in quotation marks “sandbox,” where each organization can have their own and they can roll out whichever challenges they want, they can replace the YouTube videos with my ugly mug with their own, if they so choose. And it’s really just an attempt to get everyone, just to have them open their eyes to what AI can do and how it’s so wide and so personalized.

Rebecca: Sounds like a great resource.

Dave: Love to roll it out to Oswego, if you want it.

John: We’ll certainly share a link to it to everyone. [LAUGHTER]

Dave: And they’re at that flcc.edu/ai site, you’ll be able to see there’ll be a checkbox for “Hey, I want information on the AI challenge. I wanna roll it out in my institution.”

John: Great. Well, thank you. It’s always great talking to you, and we look forward to future conversations.

Dave: Yeah, this has been wonderful. Thank you so much for giving vibe coding the space that it so deserves. And I love the ideas. I’m so glad I came today because I learned two or three things that I’m totally going to put in my courses now, after talking to you.

Rebecca: That’s why we like doing interviews too.

Dave: That’s awesome. Thank you. Thanks for joining us.

Rebecca: Thanks for joining us. [LAUGHTER]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]