Category: assessment

Show Notes

• Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
• Miller, M. D. (2022). Remembering and forgetting in the age of technology: Teaching, learning, and the science of memory in a wired world. West Virginia University Press.
• Miller, Michelle (2024). A Teacher’s Guide to Learning Students’ Names: Why You Should, Why It’s Hard, How You Can. University of Oklahoma Press.
• Diane Halpern
• Gerlich, Michael. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies, 15(1), 6.

Transcript

Rebecca: Throughout human history, we have relied on technology to make our work easier. In this episode, we discuss how to foster students’ critical thinking skills in the age of AI.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Michelle Miller. Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University. She is the author of Minds Online: Teaching Effectively with Technology, Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World and A Teacher’s Guide to Learning Students’ Names: Why You Should, Why It’s Hard, How You Can. That list keeps growing since we first interviewed you. And Michelle is also a frequent contributor of articles on teaching and learning in higher education to a variety of publications including The Chronicle of Higher Ed. Welcome back, Michelle.

Michelle: Oh, thank you. And yes, it’s really satisfying to see these books making their way out into the world. And if it weren’t for engaged faculty audiences, that wouldn’t happen. So I guess I’ll put that gratitude out to the audience as well.

Rebecca: Today’s teas are:… Michelle, are you drinking tea?

Michelle: I sure am, I’ve got a nice green tea from Hong Kong.

Rebecca: Nice. How about you, John?

John: I am drinking a pure peppermint tea today.

Rebecca: So far, we’ve got a good variety, and I’ll follow up with some black tea. [LAUGHTER] I have a Lady Grey today.

John: Very good. So we’ve invited you here to discuss the development of critical thinking skills. And let’s begin by talking about what critical thinking is and any misconceptions or missed opportunities that you’ve noted in your work around this issue.

Michelle: So this is something that I’ve really started engaging with myself back when I was writing the book Minds Online. And those who are familiar with its structure know that I try to use a really, really simple little heuristic for divvying up the cognitive demands and some of the relevant cognitive research and processes that are involved in the type of learning that we address in higher education, and one of those is thinking, and even within that is this topic of critical thinking skills. And so I had just kind of caught myself, year after year, monitoring what was going on in the development of research around this, of course, thinking about it in my own teaching, but also finding myself being really reticent when I was working with faculty audiences to say, well, this is a really kind of a, if not totally unique, a pretty specialized subset within thinking skills more generally, and then the bigger, broader umbrella of cognition in general. So it’s almost like the first missed opportunity is my opportunity to engage with faculty around this. So I love it, it’s this really deep topic. But that’s one reason why it’s hard to just say, oh, and in five minutes, sandwiched in between five other things, we’re gonna address this. And so that’s what I’ve kind of been reexamining myself. So when we talk about what is critical thinking, that is actually the first really big challenge and, or sometimes misconception around the topic. And no surprise, there’s differences of opinion in defining things, and of course, really getting into that challenge of how do we define a thing and pin it down, and let’s all kind of debate the definition. But critical thinking here really is a special case in that there have been quite a few attempts to make these very general sort of top level, kind of elaborations of what it is. Think about words like evaluate or analyze, reflect and so on. But I don’t think we have converged on one of these as really the best across disciplines, and I kind of fall in with some other thinkers in the sphere who’ve said maybe a discipline independent, context agnostic definition doesn’t exist, [LAUGHTER] because maybe the phenomenon itself doesn’t exist. So I’ve, myself, really been drawn towards the more, I guess you’d call them, situated or contextualized definitions of what it is. If you drill down a little bit further, you get, for example, Diane Halpern, psychologist, this towering figure in our field. So naturally, I would gravitate towards what she says, but she says things like evaluating and analyzing evidence in the service of making decisions, so thought processes that lead to a good outcome. So that’s a little bit more practical, but it still doesn’t really make it so a given faculty member teaching in a given course can say this is what it looks like when students came in without a critical thinking skill and they leave with it. [LAUGHTER] And as with anything, if we don’t have that, then no surprise that we’re maybe not going to get there, or not get there as completely as we had hoped to. And so that’s one kind of set of, if not misconceptions, challenges or things that people often don’t realize when they wade into this issue of teaching critical thinking. And it follows from that, unfortunately, that we can have the misconception, that we can describe it to students and they’ll get it. I know, for me, I would be embarrassed to say how many years I taught, for example, Introduction to Psychology, which we pride ourselves on being like this is the critical thinking class. And even if you never take another psychology course, you’re going to come away with these great reasoning skills. And gosh darn it, I had a whole half of one class period that we discussed it, maybe, and it was on the study guide somewhere. So we think, “Oh, okay, I’ll just tell students this is important.” Say, “think critically.” And of course, naturally, they don’t know what it is either. And I can really see, in my mind, at least, how students could come away with the idea that critical thinking means just sort of criticizing everything indiscriminately. It means nobody knows anything, a total relativism, that no, that’s not what we want, or in its worst forms, it can read too as just like, agree with me. Like I can go and say, Look, “I’m a psychologist, and I’m pretty critical of the idea of horoscopes.” And if you think that your horoscope is going to predict your future, you’re not engaging in critical thinking, you’re wrong. So that’s what we want to get away from. And where I think that it can be this really great fertile ground for faculty if they step back and say, “Okay, what does this really mean at that more actionable level,” and then start to align instruction to that, because it’s gonna take a lot more than half a day, I’ll tell you that much.

Rebecca: Seems like you’re suggesting that it would be helpful for students to be able to identify specific behaviors or specific practices that we would identify as critical thinking. But if they know what the beginning, middle and end of an action is, [LAUGHTER] they have a better idea of what that looks like and what that feels like.

Michelle: Yeah, absolutely. And that’s also a pretty psychological way of coming at it as well. Like, okay, of course, this is something that happens in your mind, out of view, between your ears, but I want to see what happens on the page or the decisions that you do make in the world that align with what I would consider to be evaluating and analyzing evidence in a powerful way. And yeah, when we get into a little bit more of the challenges, I mean what I’ve laid out for faculty, and again, really drawing on a lot of research that’s been out there for a while, even beyond that defining issue, which, yeah, that’s going to challenge faculty. It’s going to get in the way of designing the activities. It’s going to challenge students. They don’t know what they’re trying to aim for to meet this thing that’s on the syllabus. It is also what I would almost call a cueing problem, or someone’s just kind of off the cuff, called a “when to” versus a “how to” problem with a lot of the skills that we are helping students develop. It’s “Okay, here is how you break this down. Here is how you conduct the statistical test. Here is how you read an APA style research article.” But this is something that’s like, “Okay, here’s how to find, for example, issues with correlation versus causation in the interpretation of a study.” Great, we can do that. But are you going to know to do that when you’re writing your paper? And even a more far transfer of that as you’re scrolling through whatever social media feed of choice is, and you get the like, well, I drank a cup of lemon juice a day, and now I’m healthy as a horse. [LAUGHTER] It’s sort of the same problem. Am I going to have that sort of inclination or disposition to say, “Oh, right, this is when I need to do this.” So there’s that. A lot of folks have talked about effort as well, part of this running theme of cognitive miserliness, that our brains and our minds really seem to run in high-efficiency mode, where we save time and save energy over getting to a more idealized solution. And also, as a lot of folks have talked about, a motivation to maintain beliefs, whether it’s a very profound thing, like, “I don’t want to critique this belief because it’s really foundational for intellectually or emotionally or spiritually,” or even just dumb things like, “Oh wow, I really want to buy this thing I just saw an ad for. And maybe I just won’t take a look at all those negative reviews,” or “I’ll assume that the reviews that all have these wonderful five stars are perfectly reputable.” So all of those things form quite a steep set of challenges for students and for faculty alike, right?

John: Very much so. And what makes it more complex, and I think you alluded to this, is that each discipline has its own definition of what critical thinking skills are. I remember I spent about 10 years on faculty assembly one semester where they were coming up with a new Gen Ed curriculum, and one of the main issues was, how are we going to build critical thinking into our requirements? And ultimately, it was left up to the individual disciplines because there could not be a consensus formed. Each individual department had a very good definition of what they believed critical thinking was, but it differed quite a bit from what other departments thought were important skills. So it makes it somewhat challenging for students when they’re faced with this wide variety of definition of critical thinking in terms of being able to develop those skills.

Michelle: What you’re describing, what a perfect example. I mean, it sounds everybody went in good faith, had this really strong set of ideas going in. But I don’t see this as just like, “Oh, there go the academics again… consensus… they can’t reach it.” I think that they were probably running into a real issue. And again, there’s a couple things at work here. I think that there’s also this sort of flawed metaphor that I think it’s flawed, and I think it runs through a lot of cognitive aspects of learning, of the muscle metaphor, of like, “Oh, there’s a critical thinking muscle in your brain, and if you just build it up, then it’s gonna be strong for a really long time.” And most things in the mind actually don’t totally work like that. They again, are more situated or contextualized. But I always say to faculty, I’m like, if we go in, we can sort of say, “That’s okay, that’s all right.” And hopefully more institutions will do what it sounds like may have happened more or less by default in this particular example of like, well, okay, we’re gonna go back to our colleges social sciences and so on, or we’re gonna go back to our individual disciplines and try to operationalize it there. So I look at it, and I see that this is a very rich spectrum. It’s like in business, I’ve been trying to compile all these different examples, and if I’m a business major identifying pattern that suggests some kind of financial mismanagement, don’t take the company’s word for it. What does it look like? One hears that’s critical thinking there. If I’m in healthcare, being able to look at relevant research findings, and try to translate that into selecting the right kind of interventions. Engineering has its applied systems thinking and so on. And so if we come at it the right way, that can be a strength, not a weakness, I think, and I may be a little unusual in that, but that’s what I come away with.

Rebecca: One of the things that you mentioned earlier about knowing like when to do a thing really makes me think a lot about a move in some disciplines, and I know this was true in design and art a bit is to focus a bit more on process rather than always on outcomes, so that you can see some of the thinking and really build in when to do things and when to implement particular techniques to work towards a particular goal, and slowing things down a bit so that we could emphasize some of those things.

Michelle: I think that that is also a really important and strong approach. And here, too, I’ve heard across lots of different disciplines as well, sitting with faculty and kind of grappling with lots of, to me, interesting teaching issues. They’ll say like, “Okay, yeah, I taught that how of how to work their problem,” or, like, “if I’m teaching students how to do statistical analyzes in their research projects,” like, “Okay, I can go through and conduct the steps for a t-test or an ANOVA and so on.” And I’m not dismissing that, that’s important stuff that is important to be able to do, but I’ve heard so many variations on “But they can’t set up the problem. Once it’s set up, we’re good to go.” And that’s where, instead of kind of saying, “Ah, our students can’t,” well, what are their opportunities to do that? So let’s build in more of those really messy this is more about watching you do it and having you explore different approaches then you have to check all the boxes exactly right. There’s a time for checking all the boxes exactly right. Again, not knocking that, but yeah, of putting our students more systematically in those situations, we might see some real growth there.

John: One of the challenges, I think, that we’re facing is that we’re seeing so much misinformation being shared on social media and AI tools provide increasingly realistic, deep-fake multimedia files with audio and video. And there’s a lot of politicians out there who seem to have little regard for centuries of science, for example, where they are questioning things like whether or not vaccines are useful, despite over a century of evidence on this. How can we help students develop their critical thinking skills they need to navigate this environment where there’s so many opportunities to be misled by all this information.

Michelle: Oh my gosh, yeah. You’ve really brought this together, and it’s such a powerful way. I mean that really says it, right? And as you’re describing this, absolutely. Talk about a perfect storm. Talk about an absolute perfect storm. So here we have something that runs off of low effort, and I don’t buy into it, like, “Oh, people these days are cognitively lazy.” I think that’s just an overriding feature [LAUGHTER] of human cognition that I feel like, if there have been changes, that’s been a drop in the bucket, but I do have access to something that is incredibly easy to use, click it open, start scrolling. And so it really biases us towards low effort. It biases us towards speed too. I mean, Instagram has never kind of been my thing, but those who really like it, it is amazing how that that’s what is sort of designed into that system, the ability to just whiz through and visual processing is so fast, and that’s great and all, very entertaining. But here I am in that fast, quick, low-effort mode, as you said, it’s now multimedia. That is also what makes it so compelling, is the visuals and the audio and “Oh, guess what just came along?” Really great ways to fake that. And is anybody else doing it? Is there that? Can I see anybody else who’s cueing me and saying, You know what? Maybe we should authenticate this video before we post it 48 million times? No one is. And last of all, you’ve thrown motivation to maintain our beliefs into the mix. Wow. And so if I am inclined, if I’ve always had a bad emotional feeling about vaccines. And I’ll just say that, as a psychologist, I kind of think that that’s where some of it comes from. Vaccines are weird. They’re not pleasant [LAUGHTER]. They’re okay to think about, and I haven’t liked it. I don’t want to get one. Last year’s flu vaccine made me sick and made my arm hurt. I mean, it does not take much to knock me straight off into. To okay, I’m posting it, I’m believing it, and that’s where we are. So that’s what I see when I kind of spin that scenario out. So there’s lots of things that we can look at as far, and we’ll probably delve into a little bit further, as far as kind of the instructional strategies for critical thinking. But in this one instance, let’s take that perspective of instead of starting from 50,000 feet with critical thinking of, let’s have a general definition and then try to figure out a way to have it filtered down. Let’s do bottom up. Let’s say, “You know what? If critical thinking looks like stopping the feed and before I hit post, if it means at least some proportion of the time I ask about authentication, if I’m applying what I should know about the potential for AI deep fakes. If that’s what it looks like, then that’s the practice that students will need.” And I would throw it out to the ever creative and always innovative faculty across disciplines to say, “All right, if this is something that is relevant to thinking in your discipline, or something that you just want to reinforce in your students, how can we have students exercising that and build that from the bottom up?” And then maybe that 50,000 foot view of critical thinking and that bottom up, “Okay, I learned to at least process my feed in a more critical way. Now I’m looking at how I read things. Now I’m really looking at media literacy and so on, and so maybe those things will meet in the middle.” This is an audio only podcast. I’m making all these gestures here, [LAUGHTER] my little hands getting closer and closer together, but that’s what I see when I look at that issue.

Rebecca: I hear a lot of scaffolding and backwards design in what you’re suggesting. Can you talk a little bit more about some of these practical strategies that we can use in the classroom to address critical thinking?

Michelle: Yeah, and it’s another really tough one to communicate with faculty maybe why I’ve been so like, squeamish in some of my work with them, just because you kind of have to start with a negative message. And that’s why I share of like, “Oh yeah, me and my whole one half of a whole day on critical thinking,” because that’s the thing. you first have to know, this is a big, long journey, and I’ll throw out a metaphor. Some of you know that I spent a lot of time here at the Grand Canyon in Arizona, and I’m always on forums discussing all the big hikes and so on and what we always see is like, “Oh my gosh, I’m gonna go to the river and back. I’m gonna hike from the north rim all the way to the south rim. What am I in for?” It’s like you need to know that this is gonna be no walk in the park. [LAUGHTER] Really wrap your brain around what’s involved with this. Worth it? Totally, absolutely, an incredible accomplishment if you get there. So hopefully I can infuse a little bit of that spirit in it. So first of all, go in forewarned, don’t walk to the bottom in 110 degrees with one bottle of water and figure it’s going to be okay. So with that, yeah. And then reflecting, we always want to do this with thinking skills. But again, backing up and saying, “all right, what does this look like?” Free yourself as much as you can from “Oh no, it has to look like somebody else’s definition.” Or is this general or all encompassing enough? Just really get concrete about that, and then focusing on the why. And that’s also been a running theme here. When you talked about process, that to the extent that things do work, and there are no silver bullets in this arena. We don’t have anything, [LAUGHTER] unfortunately, like that. But to the extent that there are things that we can say, yeah, that was really effective, it’s where the focus is on the why and not the answer. Whether it’s just a multiple-choice question of, “Hey, if you got a question wrong, come and explain it or discuss it and maybe recover some of your points,” or something very, very big, like we’re having a big debate in class about two approaches to an issue, and one of the big things I’m going to want to look at is your reasoning. And sometimes this is talked about in cognitive psychology as focusing on the deep structure of a problem. So all those kind of things that make similar problems link them together, like all those correlation versus causation problems that I can see as an expert, but the novice is just like, these are all totally unrelated [LAUGHTER] instances, but when you start to say, “Okay, but here’s the sort of working parts underneath that, the mechanism,” that can move the needle a little bit. And direct practice. Students almost always need more practice over time. And the research that’s out there that I’ve seen is showing like more like a couple of months of practice [LAUGHTER] instead of a couple of hours or a week or two. So if that’s what we mean. So practice, practice. And lastly, that motivation to maintain beliefs. That’s a really tricky one. Most of the folks in the field seem to say you got to take careful and indirect approach. If I come in again and be like, “How could you think that horoscopes [LAUGHTER] could possibly…?”, you know, if somebody asked me my sign on a first date, that would be the last date. [LAUGHTER] You come in with that kind of attitude, well, naturally, people shut down. So going with respect and sometimes asking the questions of, “why?” Okay, so explain to me how this works. That can be a better thing than just mowing it down. So be direct in the practice, and be indirect [LAUGHTER] and use a light touch when there’s a possibility of those kind of things that are maintained by motivations.

John: One of the things that’s been happening is that with the increasing use of ChatGPT and other AI tools, there’s a lot of concern that students are using it to offload some of the cognitive processing that they might normally do while they’re reading texts or analyzing materials. And there was a study by Michael Gerlich at this Swiss Business School earlier in January that found that students who used AI tools more, tended to do a bit more cognitive offloading of these tasks, and that there seems to be at least some correlation with the decline in students’ critical thinking skills when they rely, to a greater extent on AI. What types of learning activities can help students develop critical thinking skills so that they can more critically evaluate the types of output they’re receiving from generative AI tools?,

Michelle: Yeah and here too, that could further decompose into different multiple issues. There’s critiquing output you get. And I’ve heard some really clever assignment ideas where that’s part of it is like, “Okay, go ahead and generate this essay,” like I heard a professor, there was a very cliched topic [LAUGHTER] that her students were always coming back to, and instead of just forbidding it, she’s like, “Okay, have ChatGPT write that essay, and then you critique it.” And then there’s the more process oriented: “Here’s the process that I want you to engage in,” for example, reading and evaluating, say, research studies in psychology, the way we do a number of my classes. And the alternative is stick it and say, “ask ChatGPT to give you the [LAUGHTER] talking points from the article. So part of it is going to depend on what exactly do we mean, but I did want to really address too, or come back to, the study with the correlation between use of AI tools and cognitive offloading, in particular, and then critical thinking skills. This is a research area that absolutely I hope that we kind of build out on. But there’s some things that I have curiosity about and want to kind of at least have people pump the brakes a little bit, because this is one of these dynamics that can happen, maybe a little bit of motivated reasoning here. If I like, “Ah, AI, is this issue in my classes? I’m really wrestling with it. We’re wrestling with it on campus. Oh, AI, kills critical thinking.” And this is the dynamic that I feel like we’ve seen a little bit before, for example, with the flap over laptops and whether those degrade learning. And boy, people ran away with that in ways that were not supported by the original study, and then even with what the author [LAUGHTER] said about the study. So a couple of things that we should keep in mind with that research, which, again, I’m really glad they conducted it, and I hope there’s more research like that. This is not a laboratory study where we’re assigning people to groups, of course, and saying, “Here’s what happened to critical thinking abilities,” and this was a multi age study. Was also we want to be cognizant this wasn’t like in a class or educational setting, for what that’s worth. There’s a lot of self report here. I always have an issue with that in lots of research on technology. So if you ask people, how often do you use AI to, I think, it was answer questions or solve problems, which also a couple different things are covered there in the original study. If you just ask people that, I don’t know that people can track that, that’s different than actually measuring that. It’s a start. But know what you’re kind of getting there. And I mean cognitive offloading too. This is the thing I’ve really noticed this tilt towards interpreting cognitive offloading as a bad thing, and that I don’t think is in the spirit of how it was originally intended. It can sometimes be a good thing. It’s a neutral thing. And everything from using Post-It notes to GPS on your phone to using an abacus or counting on your fingers are all instances of cognitive offloading. So this is not something [LAUGHTER] that just started. So there’s a couple of things there. So does it predict a decrease in critical thinking skills? And again, here, this is self report using some good measures adapted from decent established instruments. But here, again, we weren’t watching to see okay, here’s students who are actually grappling with academic problems or making decisions in real life. So I would absolutely encourage folks get out there and read the study itself and make your own decision, also it’s very much in the spirit of what we’re talking about here. So there is that. But I do think that a lot of experts who are looking at the development of AI in higher education, they’re all saying some variations of the same things, like, you can’t send students home with something and say, here’s a really difficult effortful thing that I know is important for you to engage in. Maybe you know that, maybe you don’t. Here’s a trap door [LAUGHTER] that you can evade the entire thing with AI. You cannot assume that students are going to get what they need to get out of that assignment anymore. So that is a real question that we had. So let’s not run away with the idea that, “Oh, AI is there, and it’s going to destroy critical thinking by its very existence.” But let’s really be thinking through what we’re assigning students to do. And I’m hoping we’ll hear more from faculty, sort of in the field, say, “You know what? I was really transparent with students, and here’s how I got them on board with ‘Yeah, we’re actually going to read these articles, even though ChatGPT can summarize them for you.’“ Or here’s how I just closed that trap door up and actually made it work that way. So that’s a complicated take on a complicated issue.

Rebecca: The onset of AI in teaching and many other things have led to a lot of faculty overwhelm around a wide variety of topics, and AI is certainly just one of them. Have you seen this? And are there things that might help faculty as they navigate new technology, new policies, many new things in their work?

Michelle: Yeah, and the issue of overwhelm, I mean, this is one that just kept coming up into my mind again and again, as I’ve done at least my first little modest set of things that I’ve written about and shared with folks. A couple of things, and some of these are going to, no doubt, rub some people the wrong way, but I’m going to put them out there anyway, because I think they’re a really practical way to go. The first thing is, I think that a really important starting point is always going to be our own professional practice, and again, discipline independence, just like critical thinking, it’s really hard to say, “Okay, make sure you evaluate evidence and make that somehow stick across a curriculum.” I’ve really pushed against the like, “Well, there’s an AI future and AI workplace,” and I don’t know what that is, but I can prepare students for careers that are related to psychology and future lives that are very well informed by all the wonderful things that psychological science has revealed for us. That I can do, and I do that every day, and so really kind of narrowing, in a good way, the focus and the scope here. I hope our leaders are getting this message as well. We’ll put out fewer of these like, here’s “AI for everybody,” and more like, “Okay, I want to know how you’re going to be preparing students in social science for uses of AI that are appropriate to them in the future.” And part of that is getting in there and using the tools yourself. And I will tell you one really important thing, even more so, believe it or not, than all the other ed tech I’ve seen come and go [LAUGHTER] over the years, the level of hype and over claiming that seems to just get by is astonishing, and it falls apart very rapidly when you sit down with the tool yourself. And I think that’s where we faculty will actually really, really shine. I won’t name too many names right now, but like, “Oh, wow, AI can do that for me.” And that’s terrible. I’m not gonna recommend this to my students. And other ones where I am like,”Oh, okay,” and only a disciplinary expert knows this. And to give like, an example, I was working with a faculty member in a workshop. She was in pharmacology. Wow, talk about something that’s way out of my field, and so I’m a little odd. And she talks about “And I made this assignment, and part of it is to critique some AI output.” I’m like, “Oh, I know where this is going.” She’s gonna tell me that “Oh, AI is gonna tell me bizarre things about medications.” And now she’s like, “Oh no, no, no. I use it to generate some scenarios.” It was something involving, like rare toxins, a really cool assignment. She’s like,”Oh no, no. For this particular tool, it taps into this particular database. I happen to know that that’s really, really strong, and I haven’t seen a problem with it.” So that’s where we can really get in there and start to craft things. So you can start it at that level. And another really, probably unpopular piece of advice, but one that I think is important at this juncture, is, okay, we have these big questions of the ethics of AI use: How did this stuff get built? Who’s going to be regulating it? I think that’s incredibly important. And I would imagine that pretty much everybody listening to this podcast would agree with that, however, going and saying like, “I have to figure that out myself [LAUGHTER] before I start writing an AI policy for my syllabus” or trying some of the tools, so that I can talk intelligently to students about the pros and cons. That’s an issue. So I would say those kind of have to go on parallel tracks if we’re going to make some immediate progress. And I feel like, and I felt of myself initially, of like, “oh my gosh, all these big issues.” Well, that’s not my job to solve. It’s great if I can contribute to that and support solutions and approaches that I think are ethical, but I’m not the one who’s going to figure that out all by myself. So here’s some positive things I can do with my students in the meantime.

John: One of the things that Michael Gerlich suggested in that article is that in case there is some issue with a decline in critical thinking skills, we probably should assume that students are going to be experimenting with AI tools, and he suggests that maybe we should focus a bit more on training students to apply critical thinking to the output of AI, because if they’re going to use it anyway, we might as well use it as a tool to help develop these skills that we’d like to improve. Does that seem like a reasonable strategy at this point, with a lot of experimentation going on by a lot of faculty right now in trying to integrate this into their practices?

Michelle: Yeah, and I think higher education faculty are in a great position to do that, because from industry, we get this kind of kind of cutesy pooh, “Oh, it’s hallucinating.” It’s like, no, it’s wrong. It’s inaccurate. If this was a book, you would roast it in an Amazon review if it was just wrong. So we can get away from that. We can also not fall into the “Well, use it, but don’t use it, trust it, but maybe not so much.” And that starts to remind me of like all the terms of service. Like, well, read the terms of service when you click into something, but we know you do didn’t really. I have no patience for that. So really being able to nail down the limitations, and particularly, again, situated in these disciplines where we are incredibly skilled experts at knowing where it is likely to trip up and what sorts of mistakes are a problem and which are less of a problem and guiding students into that. So if we can design those ideas, and yeah, sometimes we can take it in another direction and use the power of the tools for our own purposes. And that’s also where I get a little nervous about the like, blanket, “Oh, AI will kill your critical thinking.” Well, I wouldn’t want a faculty member to say, “Well, then I’m not going to touch it, because if there’s any way that it’s sort of contaminated my assignment, that’s bad.” I don’t think we’d consciously think that, but it can start to imply that. So I’m looking back when I used to teach research methods, one of my assignments I was very proud of was their final exam was I wrote beginning to end a fake study with fake data that was full of all kinds of problems that spanned everything from study ethics to degrees of freedom in the statistics module. [LAUGHTER] So I had maybe two practices, and then, for their final example, they went in and spotted the problems. They didn’t have to spot all of them, but they had to justify and say what’s wrong with it. And students would go wild over this, even though they know that I made this thing up. They would go crazy with all these issues. And it was perfect, if not perfect, I loved it, because that’s what we want students to come away with, but the labor that went into that, and I find myself like reusing the same scenarios across semesters. Okay, we’re gonna have this one be practice, and it was just so hard. And today, every student in the class could have a new one every day, and so it could do that work. I come in and here we go. Here’s these rich scenarios with students grappling with the messy problems, focusing on the why, but in the middle of it has to be a faculty expert to make all that work.

Rebecca: There’s just some great points and great fun use of AI for sure. I know that I’ve used it for scenarios and other things in my teaching and in other contexts, as well as we work with leadership on our campuses, thinking about policies and other ways to approach AI and making sure that we’re not losing sight of critical thinking and any kind of deep thinking [LAUGHTER] what kinds of recommendations should we be making?

Michelle: Ah, for leadership, and I love it, the receptivity to new ideas that’s definitely going to be key? Well, I’m gonna probably rehash a few of those points of be very careful about convening, like, say, a big, expensive task force to find the one consensus, to converge on the one solution. There are probably situations where that is absolutely what needs to happen. But one of the things that I think we’ve seen, too, you don’t really like to talk about it that much, I don’t think, but higher education is spectacularly unprepared to deal with just the pace of change here. And yeah, partly, industry is sort of driving the bus and the pace of change, and we don’t always have to accept [LAUGHTER] their timelines. But the fact of the matter is, is that you go to sleep, you wake up, and there’s a new version of something, [LAUGHTER] and so I was really surprised. I shouldn’t have been, but I was pretty surprised by, like, how it unfolded in my local context, and just among what I could see of, like, “Okay, here’s a little PowerPoint that I can take around,” and then two weeks later, I’m like, “I can throw that away?” I’m not used to that. [LAUGHTER] So, yeah, higher education, our deliberative nature is not a bad thing, necessarily. That’s good. It’s kind of like that’s what you pay us for, is to take the long view that’s grounded in a lot of evidence. But in this o ne instance, it makes it really, really hard for us not to feel like we’re flat footed. So beware of those sorts of things, and it’s demoralizing for faculty who well meaning, they’re interested in this, they’re excited about it, they want to contribute, and they spend four months doing something they can’t see how it played out. So my gut tells me that going for that college level is going to be important. So maybe I don’t want to chase down every single department or every single course to say, “How is AI going to be involved here, especially around critical thinking issues,” but saying, “Okay, the School of Engineering, or the School of Earth Sciences, the School of Social Sciences, Literature and Humanities, however, it’s sort of organized.” I think that that’s a pretty powerful sort of organizational approach there. And that’s something where too, it’s enough to where we can get some multiple voices in there and get some buy in, but it’s specific enough to where I’m not trying to align what I’m doing in psychology with something that people are doing in art history, with what forestry is doing, which, again, I think is really gonna neutralize us if we try to take that approach. And also dividing it up. The last thing that makes this such a hard thing for us is that it’s just so inherently multifaceted, much more so than, for example, even when learning management systems came on the scene and we’re like, “Okay, do we require everybody use these? Which one do we want? How do we pick? How do we train people?” At the very bare minimum, we got to kind of separate out academic dishonesty is over here, and it’s got its own little set of policies, and let’s set a team working on that, and positive uses and preparing students for that workplace of the future, and so on. These are separate issues over there. So yeah, be prepared to go in and just say, “Okay, two separate things, we’re gonna deal with them two separate ways.” I can’t imagine trying to sit at home on a Saturday and craft some long email that is somehow going to bring clarity to all the different aspects of AI. And I’m wondering if there’s any sort of resonance or spark of recognition there, as leaders, that you might see.

John: Do you have any other suggestions on how we can help students develop their critical thinking skills.

Michelle: One little theme that pops up from time to time in all the great stuff that’s been written about critical thinking, and one that I think might be worth its own special mention, is an element of what you could call intellectual humility, or even, I guess, the flip side of that too, is curiosity. Think of me with kind of cutting down some student over horoscopes in my class, even though I find the basis for that in psychological sciences. If I do not care at all about why a student believes that, or what it might add to their lives to believe in, I mean, maybe it’s just something fun that is like a little Rorschach test that I wake up with every morning. If I have no interest in that, and then I’m also not demonstrating critical reflection, nor am I setting a good stage for a discussion that foregrounds critical thinking. So yeah, and that is our big challenge to faculty with our incredibly developed intellectual abilities to balance that. Yeah, evidence is important, and some facts can be right or wrong. There’s different ways of interpreting them, but here’s some areas where they have to support the analysis, balancing that with all right and I could be wrong. I could have a cognitive bias of my own that I’m not as aware of right now. I kind of have to be on the lookout for always checking myself as I’m scrolling through that feed or going through those reviews for the thing that I really do want to buy, no matter how many one star reviews that the company suppressed. I need myself to get in the habit of doing that, and then I can model that for my students, and again, set the stage for more productive discussion. So I’m hoping that more research also gets into that too. What is that wonderful, magical balance of really valuing evidence and reasoning and then realizing and I could be wrong?

Rebecca: Well, we always appreciate, Michelle, conversations with you. I know I always leave thinking a little more deeply about whatever we’ve talked about, but we always wrap up by asking, what’s next?

Michelle: Oh, thank you, and you all as well. Even just kind of looking forward to this conversation has been such a catalyst to revisit topics and say, “Yeah, this is something that I really want to be doing more in.” And so that’s going to be a bit of a what’s next. I have a sabbatical year right now, and I’m kind of trying to do what one ideally does, which is say, “Okay, what do I refocus on?” And so I’m really at that very early stage of continuing to design ways of studying, interventions that I could then share with faculty, and just getting an ever needed sense of the lay of the land in this area of critical thinking. So you can definitely look for more on that. And in fact, for folks who might be considering attending an in-person event, I am going to be speaking at a couple of upcoming events where I know that I’m going to be talking about this particular thing, again, Rocky Mountain Psychological Association for the psychologists out there, my people. If you want to come out, I think it’s in Denver this year, I have a featured session. So that’s coming up. I’ll also be speaking at the Teaching Professor Conference, which is in, I believe, Washington, DC, over the summer. Some of your listeners may be regulars, or maybe thinking about that. So that’s a big “what’s next” item? I continue to work on this big writing project on attention. It’s still at this kind of embryonic, or if not embryonic, partially formed stage, where I’m still working on it. But folks can watch this space for more on that, and that is actually related. I will tease a little bit, that attention, what do you know, gets involved in critical thinking, because it is involved in pretty much all other interesting cognitive processes that take place in the mind. So that’s my big what’s next?

John: Well, thank you. It’s always great talking to you, and again, as Rebecca said, it always spurs some new ideas and some new thoughts on these topics. So, thank you.

Michelle: Thank you. You as well.

Rebecca: I look forward to your upcoming work.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

Rebecca: Throughout human history, we have relied on technology to make our work easier. In this episode, we discuss how to foster students’ critical thinking skills in the age of AI.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Michelle Miller. Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University. She is the author of Minds Online: Teaching Effectively with Technology, Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World and A Teacher’s Guide to Learning Students’ Names: Why You Should, Why It’s Hard, How You Can. That list keeps growing since we first interviewed you. And Michelle is also a frequent contributor of articles on teaching and learning in higher education to a variety of publications including The Chronicle of Higher Ed. Welcome back, Michelle.

Michelle: Oh, thank you. And yes, it’s really satisfying to see these books making their way out into the world. And if it weren’t for engaged faculty audiences, that wouldn’t happen. So I guess I’ll put that gratitude out to the audience as well.

Rebecca: Today’s teas are:… Michelle, are you drinking tea?

Michelle: I sure am, I’ve got a nice green tea from Hong Kong.

Rebecca: Nice. How about you, John?

John: I am drinking a pure peppermint tea today.

Rebecca: So far, we’ve got a good variety, and I’ll follow up with some black tea. [LAUGHTER] I have a Lady Grey today.

John: Very good. So we’ve invited you here to discuss the development of critical thinking skills. And let’s begin by talking about what critical thinking is and any misconceptions or missed opportunities that you’ve noted in your work around this issue.

Michelle: So this is something that I’ve really started engaging with myself back when I was writing the book Minds Online. And those who are familiar with its structure know that I try to use a really, really simple little heuristic for divvying up the cognitive demands and some of the relevant cognitive research and processes that are involved in the type of learning that we address in higher education, and one of those is thinking, and even within that is this topic of critical thinking skills. And so I had just kind of caught myself, year after year, monitoring what was going on in the development of research around this, of course, thinking about it in my own teaching, but also finding myself being really reticent when I was working with faculty audiences to say, well, this is a really kind of a, if not totally unique, a pretty specialized subset within thinking skills more generally, and then the bigger, broader umbrella of cognition in general. So it’s almost like the first missed opportunity is my opportunity to engage with faculty around this. So I love it, it’s this really deep topic. But that’s one reason why it’s hard to just say, oh, and in five minutes, sandwiched in between five other things, we’re gonna address this. And so that’s what I’ve kind of been reexamining myself. So when we talk about what is critical thinking, that is actually the first really big challenge and, or sometimes misconception around the topic. And no surprise, there’s differences of opinion in defining things, and of course, really getting into that challenge of how do we define a thing and pin it down, and let’s all kind of debate the definition. But critical thinking here really is a special case in that there have been quite a few attempts to make these very general sort of top level, kind of elaborations of what it is. Think about words like evaluate or analyze, reflect and so on. But I don’t think we have converged on one of these as really the best across disciplines, and I kind of fall in with some other thinkers in the sphere who’ve said maybe a discipline independent, context agnostic definition doesn’t exist, [LAUGHTER] because maybe the phenomenon itself doesn’t exist. So I’ve, myself, really been drawn towards the more, I guess you’d call them, situated or contextualized definitions of what it is. If you drill down a little bit further, you get, for example, Diane Halpern, psychologist, this towering figure in our field. So naturally, I would gravitate towards what she says, but she says things like evaluating and analyzing evidence in the service of making decisions, so thought processes that lead to a good outcome. So that’s a little bit more practical, but it still doesn’t really make it so a given faculty member teaching in a given course can say this is what it looks like when students came in without a critical thinking skill and they leave with it. [LAUGHTER] And as with anything, if we don’t have that, then no surprise that we’re maybe not going to get there, or not get there as completely as we had hoped to. And so that’s one kind of set of, if not misconceptions, challenges or things that people often don’t realize when they wade into this issue of teaching critical thinking. And it follows from that, unfortunately, that we can have the misconception, that we can describe it to students and they’ll get it. I know, for me, I would be embarrassed to say how many years I taught, for example, Introduction to Psychology, which we pride ourselves on being like this is the critical thinking class. And even if you never take another psychology course, you’re going to come away with these great reasoning skills. And gosh darn it, I had a whole half of one class period that we discussed it, maybe, and it was on the study guide somewhere. So we think, “Oh, okay, I’ll just tell students this is important.” Say, “think critically.” And of course, naturally, they don’t know what it is either. And I can really see, in my mind, at least, how students could come away with the idea that critical thinking means just sort of criticizing everything indiscriminately. It means nobody knows anything, a total relativism, that no, that’s not what we want, or in its worst forms, it can read too as just like, agree with me. Like I can go and say, Look, “I’m a psychologist, and I’m pretty critical of the idea of horoscopes.” And if you think that your horoscope is going to predict your future, you’re not engaging in critical thinking, you’re wrong. So that’s what we want to get away from. And where I think that it can be this really great fertile ground for faculty if they step back and say, “Okay, what does this really mean at that more actionable level,” and then start to align instruction to that, because it’s gonna take a lot more than half a day, I’ll tell you that much.

Rebecca: Seems like you’re suggesting that it would be helpful for students to be able to identify specific behaviors or specific practices that we would identify as critical thinking. But if they know what the beginning, middle and end of an action is, [LAUGHTER] they have a better idea of what that looks like and what that feels like.

Michelle: Yeah, absolutely. And that’s also a pretty psychological way of coming at it as well. Like, okay, of course, this is something that happens in your mind, out of view, between your ears, but I want to see what happens on the page or the decisions that you do make in the world that align with what I would consider to be evaluating and analyzing evidence in a powerful way. And yeah, when we get into a little bit more of the challenges, I mean what I’ve laid out for faculty, and again, really drawing on a lot of research that’s been out there for a while, even beyond that defining issue, which, yeah, that’s going to challenge faculty. It’s going to get in the way of designing the activities. It’s going to challenge students. They don’t know what they’re trying to aim for to meet this thing that’s on the syllabus. It is also what I would almost call a cueing problem, or someone’s just kind of off the cuff, called a “when to” versus a “how to” problem with a lot of the skills that we are helping students develop. It’s “Okay, here is how you break this down. Here is how you conduct the statistical test. Here is how you read an APA style research article.” But this is something that’s like, “Okay, here’s how to find, for example, issues with correlation versus causation in the interpretation of a study.” Great, we can do that. But are you going to know to do that when you’re writing your paper? And even a more far transfer of that as you’re scrolling through whatever social media feed of choice is, and you get the like, well, I drank a cup of lemon juice a day, and now I’m healthy as a horse. [LAUGHTER] It’s sort of the same problem. Am I going to have that sort of inclination or disposition to say, “Oh, right, this is when I need to do this.” So there’s that. A lot of folks have talked about effort as well, part of this running theme of cognitive miserliness, that our brains and our minds really seem to run in high-efficiency mode, where we save time and save energy over getting to a more idealized solution. And also, as a lot of folks have talked about, a motivation to maintain beliefs, whether it’s a very profound thing, like, “I don’t want to critique this belief because it’s really foundational for intellectually or emotionally or spiritually,” or even just dumb things like, “Oh wow, I really want to buy this thing I just saw an ad for. And maybe I just won’t take a look at all those negative reviews,” or “I’ll assume that the reviews that all have these wonderful five stars are perfectly reputable.” So all of those things form quite a steep set of challenges for students and for faculty alike, right?

John: Very much so. And what makes it more complex, and I think you alluded to this, is that each discipline has its own definition of what critical thinking skills are. I remember I spent about 10 years on faculty assembly one semester where they were coming up with a new Gen Ed curriculum, and one of the main issues was, how are we going to build critical thinking into our requirements? And ultimately, it was left up to the individual disciplines because there could not be a consensus formed. Each individual department had a very good definition of what they believed critical thinking was, but it differed quite a bit from what other departments thought were important skills. So it makes it somewhat challenging for students when they’re faced with this wide variety of definition of critical thinking in terms of being able to develop those skills.

Michelle: What you’re describing, what a perfect example. I mean, it sounds everybody went in good faith, had this really strong set of ideas going in. But I don’t see this as just like, “Oh, there go the academics again… consensus… they can’t reach it.” I think that they were probably running into a real issue. And again, there’s a couple things at work here. I think that there’s also this sort of flawed metaphor that I think it’s flawed, and I think it runs through a lot of cognitive aspects of learning, of the muscle metaphor, of like, “Oh, there’s a critical thinking muscle in your brain, and if you just build it up, then it’s gonna be strong for a really long time.” And most things in the mind actually don’t totally work like that. They again, are more situated or contextualized. But I always say to faculty, I’m like, if we go in, we can sort of say, “That’s okay, that’s all right.” And hopefully more institutions will do what it sounds like may have happened more or less by default in this particular example of like, well, okay, we’re gonna go back to our colleges social sciences and so on, or we’re gonna go back to our individual disciplines and try to operationalize it there. So I look at it, and I see that this is a very rich spectrum. It’s like in business, I’ve been trying to compile all these different examples, and if I’m a business major identifying pattern that suggests some kind of financial mismanagement, don’t take the company’s word for it. What does it look like? One hears that’s critical thinking there. If I’m in healthcare, being able to look at relevant research findings, and try to translate that into selecting the right kind of interventions. Engineering has its applied systems thinking and so on. And so if we come at it the right way, that can be a strength, not a weakness, I think, and I may be a little unusual in that, but that’s what I come away with.

Rebecca: One of the things that you mentioned earlier about knowing like when to do a thing really makes me think a lot about a move in some disciplines, and I know this was true in design and art a bit is to focus a bit more on process rather than always on outcomes, so that you can see some of the thinking and really build in when to do things and when to implement particular techniques to work towards a particular goal, and slowing things down a bit so that we could emphasize some of those things.

Michelle: I think that that is also a really important and strong approach. And here, too, I’ve heard across lots of different disciplines as well, sitting with faculty and kind of grappling with lots of, to me, interesting teaching issues. They’ll say like, “Okay, yeah, I taught that how of how to work their problem,” or, like, “if I’m teaching students how to do statistical analyzes in their research projects,” like, “Okay, I can go through and conduct the steps for a t-test or an ANOVA and so on.” And I’m not dismissing that, that’s important stuff that is important to be able to do, but I’ve heard so many variations on “But they can’t set up the problem. Once it’s set up, we’re good to go.” And that’s where, instead of kind of saying, “Ah, our students can’t,” well, what are their opportunities to do that? So let’s build in more of those really messy this is more about watching you do it and having you explore different approaches then you have to check all the boxes exactly right. There’s a time for checking all the boxes exactly right. Again, not knocking that, but yeah, of putting our students more systematically in those situations, we might see some real growth there.

John: One of the challenges, I think, that we’re facing is that we’re seeing so much misinformation being shared on social media and AI tools provide increasingly realistic, deep-fake multimedia files with audio and video. And there’s a lot of politicians out there who seem to have little regard for centuries of science, for example, where they are questioning things like whether or not vaccines are useful, despite over a century of evidence on this. How can we help students develop their critical thinking skills they need to navigate this environment where there’s so many opportunities to be misled by all this information.

Michelle: Oh my gosh, yeah. You’ve really brought this together, and it’s such a powerful way. I mean that really says it, right? And as you’re describing this, absolutely. Talk about a perfect storm. Talk about an absolute perfect storm. So here we have something that runs off of low effort, and I don’t buy into it, like, “Oh, people these days are cognitively lazy.” I think that’s just an overriding feature [LAUGHTER] of human cognition that I feel like, if there have been changes, that’s been a drop in the bucket, but I do have access to something that is incredibly easy to use, click it open, start scrolling. And so it really biases us towards low effort. It biases us towards speed too. I mean, Instagram has never kind of been my thing, but those who really like it, it is amazing how that that’s what is sort of designed into that system, the ability to just whiz through and visual processing is so fast, and that’s great and all, very entertaining. But here I am in that fast, quick, low-effort mode, as you said, it’s now multimedia. That is also what makes it so compelling, is the visuals and the audio and “Oh, guess what just came along?” Really great ways to fake that. And is anybody else doing it? Is there that? Can I see anybody else who’s cueing me and saying, You know what? Maybe we should authenticate this video before we post it 48 million times? No one is. And last of all, you’ve thrown motivation to maintain our beliefs into the mix. Wow. And so if I am inclined, if I’ve always had a bad emotional feeling about vaccines. And I’ll just say that, as a psychologist, I kind of think that that’s where some of it comes from. Vaccines are weird. They’re not pleasant [LAUGHTER]. They’re okay to think about, and I haven’t liked it. I don’t want to get one. Last year’s flu vaccine made me sick and made my arm hurt. I mean, it does not take much to knock me straight off into. To okay, I’m posting it, I’m believing it, and that’s where we are. So that’s what I see when I kind of spin that scenario out. So there’s lots of things that we can look at as far, and we’ll probably delve into a little bit further, as far as kind of the instructional strategies for critical thinking. But in this one instance, let’s take that perspective of instead of starting from 50,000 feet with critical thinking of, let’s have a general definition and then try to figure out a way to have it filtered down. Let’s do bottom up. Let’s say, “You know what? If critical thinking looks like stopping the feed and before I hit post, if it means at least some proportion of the time I ask about authentication, if I’m applying what I should know about the potential for AI deep fakes. If that’s what it looks like, then that’s the practice that students will need.” And I would throw it out to the ever creative and always innovative faculty across disciplines to say, “All right, if this is something that is relevant to thinking in your discipline, or something that you just want to reinforce in your students, how can we have students exercising that and build that from the bottom up?” And then maybe that 50,000 foot view of critical thinking and that bottom up, “Okay, I learned to at least process my feed in a more critical way. Now I’m looking at how I read things. Now I’m really looking at media literacy and so on, and so maybe those things will meet in the middle.” This is an audio only podcast. I’m making all these gestures here, [LAUGHTER] my little hands getting closer and closer together, but that’s what I see when I look at that issue.

Rebecca: I hear a lot of scaffolding and backwards design in what you’re suggesting. Can you talk a little bit more about some of these practical strategies that we can use in the classroom to address critical thinking?

Michelle: Yeah, and it’s another really tough one to communicate with faculty maybe why I’ve been so like, squeamish in some of my work with them, just because you kind of have to start with a negative message. And that’s why I share of like, “Oh yeah, me and my whole one half of a whole day on critical thinking,” because that’s the thing. you first have to know, this is a big, long journey, and I’ll throw out a metaphor. Some of you know that I spent a lot of time here at the Grand Canyon in Arizona, and I’m always on forums discussing all the big hikes and so on and what we always see is like, “Oh my gosh, I’m gonna go to the river and back. I’m gonna hike from the north rim all the way to the south rim. What am I in for?” It’s like you need to know that this is gonna be no walk in the park. [LAUGHTER] Really wrap your brain around what’s involved with this. Worth it? Totally, absolutely, an incredible accomplishment if you get there. So hopefully I can infuse a little bit of that spirit in it. So first of all, go in forewarned, don’t walk to the bottom in 110 degrees with one bottle of water and figure it’s going to be okay. So with that, yeah. And then reflecting, we always want to do this with thinking skills. But again, backing up and saying, “all right, what does this look like?” Free yourself as much as you can from “Oh no, it has to look like somebody else’s definition.” Or is this general or all encompassing enough? Just really get concrete about that, and then focusing on the why. And that’s also been a running theme here. When you talked about process, that to the extent that things do work, and there are no silver bullets in this arena. We don’t have anything, [LAUGHTER] unfortunately, like that. But to the extent that there are things that we can say, yeah, that was really effective, it’s where the focus is on the why and not the answer. Whether it’s just a multiple-choice question of, “Hey, if you got a question wrong, come and explain it or discuss it and maybe recover some of your points,” or something very, very big, like we’re having a big debate in class about two approaches to an issue, and one of the big things I’m going to want to look at is your reasoning. And sometimes this is talked about in cognitive psychology as focusing on the deep structure of a problem. So all those kind of things that make similar problems link them together, like all those correlation versus causation problems that I can see as an expert, but the novice is just like, these are all totally unrelated [LAUGHTER] instances, but when you start to say, “Okay, but here’s the sort of working parts underneath that, the mechanism,” that can move the needle a little bit. And direct practice. Students almost always need more practice over time. And the research that’s out there that I’ve seen is showing like more like a couple of months of practice [LAUGHTER] instead of a couple of hours or a week or two. So if that’s what we mean. So practice, practice. And lastly, that motivation to maintain beliefs. That’s a really tricky one. Most of the folks in the field seem to say you got to take careful and indirect approach. If I come in again and be like, “How could you think that horoscopes [LAUGHTER] could possibly…?”, you know, if somebody asked me my sign on a first date, that would be the last date. [LAUGHTER] You come in with that kind of attitude, well, naturally, people shut down. So going with respect and sometimes asking the questions of, “why?” Okay, so explain to me how this works. That can be a better thing than just mowing it down. So be direct in the practice, and be indirect [LAUGHTER] and use a light touch when there’s a possibility of those kind of things that are maintained by motivations.

John: One of the things that’s been happening is that with the increasing use of ChatGPT and other AI tools, there’s a lot of concern that students are using it to offload some of the cognitive processing that they might normally do while they’re reading texts or analyzing materials. And there was a study by Michael Gerlich at this Swiss Business School earlier in January that found that students who used AI tools more, tended to do a bit more cognitive offloading of these tasks, and that there seems to be at least some correlation with the decline in students’ critical thinking skills when they rely, to a greater extent on AI. What types of learning activities can help students develop critical thinking skills so that they can more critically evaluate the types of output they’re receiving from generative AI tools?,

Michelle: Yeah and here too, that could further decompose into different multiple issues. There’s critiquing output you get. And I’ve heard some really clever assignment ideas where that’s part of it is like, “Okay, go ahead and generate this essay,” like I heard a professor, there was a very cliched topic [LAUGHTER] that her students were always coming back to, and instead of just forbidding it, she’s like, “Okay, have ChatGPT write that essay, and then you critique it.” And then there’s the more process oriented: “Here’s the process that I want you to engage in,” for example, reading and evaluating, say, research studies in psychology, the way we do a number of my classes. And the alternative is stick it and say, “ask ChatGPT to give you the [LAUGHTER] talking points from the article. So part of it is going to depend on what exactly do we mean, but I did want to really address too, or come back to, the study with the correlation between use of AI tools and cognitive offloading, in particular, and then critical thinking skills. This is a research area that absolutely I hope that we kind of build out on. But there’s some things that I have curiosity about and want to kind of at least have people pump the brakes a little bit, because this is one of these dynamics that can happen, maybe a little bit of motivated reasoning here. If I like, “Ah, AI, is this issue in my classes? I’m really wrestling with it. We’re wrestling with it on campus. Oh, AI, kills critical thinking.” And this is the dynamic that I feel like we’ve seen a little bit before, for example, with the flap over laptops and whether those degrade learning. And boy, people ran away with that in ways that were not supported by the original study, and then even with what the author [LAUGHTER] said about the study. So a couple of things that we should keep in mind with that research, which, again, I’m really glad they conducted it, and I hope there’s more research like that. This is not a laboratory study where we’re assigning people to groups, of course, and saying, “Here’s what happened to critical thinking abilities,” and this was a multi age study. Was also we want to be cognizant this wasn’t like in a class or educational setting, for what that’s worth. There’s a lot of self report here. I always have an issue with that in lots of research on technology. So if you ask people, how often do you use AI to, I think, it was answer questions or solve problems, which also a couple different things are covered there in the original study. If you just ask people that, I don’t know that people can track that, that’s different than actually measuring that. It’s a start. But know what you’re kind of getting there. And I mean cognitive offloading too. This is the thing I’ve really noticed this tilt towards interpreting cognitive offloading as a bad thing, and that I don’t think is in the spirit of how it was originally intended. It can sometimes be a good thing. It’s a neutral thing. And everything from using Post-It notes to GPS on your phone to using an abacus or counting on your fingers are all instances of cognitive offloading. So this is not something [LAUGHTER] that just started. So there’s a couple of things there. So does it predict a decrease in critical thinking skills? And again, here, this is self report using some good measures adapted from decent established instruments. But here, again, we weren’t watching to see okay, here’s students who are actually grappling with academic problems or making decisions in real life. So I would absolutely encourage folks get out there and read the study itself and make your own decision, also it’s very much in the spirit of what we’re talking about here. So there is that. But I do think that a lot of experts who are looking at the development of AI in higher education, they’re all saying some variations of the same things, like, you can’t send students home with something and say, here’s a really difficult effortful thing that I know is important for you to engage in. Maybe you know that, maybe you don’t. Here’s a trap door [LAUGHTER] that you can evade the entire thing with AI. You cannot assume that students are going to get what they need to get out of that assignment anymore. So that is a real question that we had. So let’s not run away with the idea that, “Oh, AI is there, and it’s going to destroy critical thinking by its very existence.” But let’s really be thinking through what we’re assigning students to do. And I’m hoping we’ll hear more from faculty, sort of in the field, say, “You know what? I was really transparent with students, and here’s how I got them on board with ‘Yeah, we’re actually going to read these articles, even though ChatGPT can summarize them for you.’“ Or here’s how I just closed that trap door up and actually made it work that way. So that’s a complicated take on a complicated issue.

Rebecca: The onset of AI in teaching and many other things have led to a lot of faculty overwhelm around a wide variety of topics, and AI is certainly just one of them. Have you seen this? And are there things that might help faculty as they navigate new technology, new policies, many new things in their work?

Michelle: Yeah, and the issue of overwhelm, I mean, this is one that just kept coming up into my mind again and again, as I’ve done at least my first little modest set of things that I’ve written about and shared with folks. A couple of things, and some of these are going to, no doubt, rub some people the wrong way, but I’m going to put them out there anyway, because I think they’re a really practical way to go. The first thing is, I think that a really important starting point is always going to be our own professional practice, and again, discipline independence, just like critical thinking, it’s really hard to say, “Okay, make sure you evaluate evidence and make that somehow stick across a curriculum.” I’ve really pushed against the like, “Well, there’s an AI future and AI workplace,” and I don’t know what that is, but I can prepare students for careers that are related to psychology and future lives that are very well informed by all the wonderful things that psychological science has revealed for us. That I can do, and I do that every day, and so really kind of narrowing, in a good way, the focus and the scope here. I hope our leaders are getting this message as well. We’ll put out fewer of these like, here’s “AI for everybody,” and more like, “Okay, I want to know how you’re going to be preparing students in social science for uses of AI that are appropriate to them in the future.” And part of that is getting in there and using the tools yourself. And I will tell you one really important thing, even more so, believe it or not, than all the other ed tech I’ve seen come and go [LAUGHTER] over the years, the level of hype and over claiming that seems to just get by is astonishing, and it falls apart very rapidly when you sit down with the tool yourself. And I think that’s where we faculty will actually really, really shine. I won’t name too many names right now, but like, “Oh, wow, AI can do that for me.” And that’s terrible. I’m not gonna recommend this to my students. And other ones where I am like,”Oh, okay,” and only a disciplinary expert knows this. And to give like, an example, I was working with a faculty member in a workshop. She was in pharmacology. Wow, talk about something that’s way out of my field, and so I’m a little odd. And she talks about “And I made this assignment, and part of it is to critique some AI output.” I’m like, “Oh, I know where this is going.” She’s gonna tell me that “Oh, AI is gonna tell me bizarre things about medications.” And now she’s like, “Oh no, no, no. I use it to generate some scenarios.” It was something involving, like rare toxins, a really cool assignment. She’s like,”Oh no, no. For this particular tool, it taps into this particular database. I happen to know that that’s really, really strong, and I haven’t seen a problem with it.” So that’s where we can really get in there and start to craft things. So you can start it at that level. And another really, probably unpopular piece of advice, but one that I think is important at this juncture, is, okay, we have these big questions of the ethics of AI use: How did this stuff get built? Who’s going to be regulating it? I think that’s incredibly important. And I would imagine that pretty much everybody listening to this podcast would agree with that, however, going and saying like, “I have to figure that out myself [LAUGHTER] before I start writing an AI policy for my syllabus” or trying some of the tools, so that I can talk intelligently to students about the pros and cons. That’s an issue. So I would say those kind of have to go on parallel tracks if we’re going to make some immediate progress. And I feel like, and I felt of myself initially, of like, “oh my gosh, all these big issues.” Well, that’s not my job to solve. It’s great if I can contribute to that and support solutions and approaches that I think are ethical, but I’m not the one who’s going to figure that out all by myself. So here’s some positive things I can do with my students in the meantime.

John: One of the things that Michael Gerlich suggested in that article is that in case there is some issue with a decline in critical thinking skills, we probably should assume that students are going to be experimenting with AI tools, and he suggests that maybe we should focus a bit more on training students to apply critical thinking to the output of AI, because if they’re going to use it anyway, we might as well use it as a tool to help develop these skills that we’d like to improve. Does that seem like a reasonable strategy at this point, with a lot of experimentation going on by a lot of faculty right now in trying to integrate this into their practices?

Michelle: Yeah, and I think higher education faculty are in a great position to do that, because from industry, we get this kind of kind of cutesy pooh, “Oh, it’s hallucinating.” It’s like, no, it’s wrong. It’s inaccurate. If this was a book, you would roast it in an Amazon review if it was just wrong. So we can get away from that. We can also not fall into the “Well, use it, but don’t use it, trust it, but maybe not so much.” And that starts to remind me of like all the terms of service. Like, well, read the terms of service when you click into something, but we know you do didn’t really. I have no patience for that. So really being able to nail down the limitations, and particularly, again, situated in these disciplines where we are incredibly skilled experts at knowing where it is likely to trip up and what sorts of mistakes are a problem and which are less of a problem and guiding students into that. So if we can design those ideas, and yeah, sometimes we can take it in another direction and use the power of the tools for our own purposes. And that’s also where I get a little nervous about the like, blanket, “Oh, AI will kill your critical thinking.” Well, I wouldn’t want a faculty member to say, “Well, then I’m not going to touch it, because if there’s any way that it’s sort of contaminated my assignment, that’s bad.” I don’t think we’d consciously think that, but it can start to imply that. So I’m looking back when I used to teach research methods, one of my assignments I was very proud of was their final exam was I wrote beginning to end a fake study with fake data that was full of all kinds of problems that spanned everything from study ethics to degrees of freedom in the statistics module. [LAUGHTER] So I had maybe two practices, and then, for their final example, they went in and spotted the problems. They didn’t have to spot all of them, but they had to justify and say what’s wrong with it. And students would go wild over this, even though they know that I made this thing up. They would go crazy with all these issues. And it was perfect, if not perfect, I loved it, because that’s what we want students to come away with, but the labor that went into that, and I find myself like reusing the same scenarios across semesters. Okay, we’re gonna have this one be practice, and it was just so hard. And today, every student in the class could have a new one every day, and so it could do that work. I come in and here we go. Here’s these rich scenarios with students grappling with the messy problems, focusing on the why, but in the middle of it has to be a faculty expert to make all that work.

Rebecca: There’s just some great points and great fun use of AI for sure. I know that I’ve used it for scenarios and other things in my teaching and in other contexts, as well as we work with leadership on our campuses, thinking about policies and other ways to approach AI and making sure that we’re not losing sight of critical thinking and any kind of deep thinking [LAUGHTER] what kinds of recommendations should we be making?

Michelle: Ah, for leadership, and I love it, the receptivity to new ideas that’s definitely going to be key? Well, I’m gonna probably rehash a few of those points of be very careful about convening, like, say, a big, expensive task force to find the one consensus, to converge on the one solution. There are probably situations where that is absolutely what needs to happen. But one of the things that I think we’ve seen, too, you don’t really like to talk about it that much, I don’t think, but higher education is spectacularly unprepared to deal with just the pace of change here. And yeah, partly, industry is sort of driving the bus and the pace of change, and we don’t always have to accept [LAUGHTER] their timelines. But the fact of the matter is, is that you go to sleep, you wake up, and there’s a new version of something, [LAUGHTER] and so I was really surprised. I shouldn’t have been, but I was pretty surprised by, like, how it unfolded in my local context, and just among what I could see of, like, “Okay, here’s a little PowerPoint that I can take around,” and then two weeks later, I’m like, “I can throw that away?” I’m not used to that. [LAUGHTER] So, yeah, higher education, our deliberative nature is not a bad thing, necessarily. That’s good. It’s kind of like that’s what you pay us for, is to take the long view that’s grounded in a lot of evidence. But in this o ne instance, it makes it really, really hard for us not to feel like we’re flat footed. So beware of those sorts of things, and it’s demoralizing for faculty who well meaning, they’re interested in this, they’re excited about it, they want to contribute, and they spend four months doing something they can’t see how it played out. So my gut tells me that going for that college level is going to be important. So maybe I don’t want to chase down every single department or every single course to say, “How is AI going to be involved here, especially around critical thinking issues,” but saying, “Okay, the School of Engineering, or the School of Earth Sciences, the School of Social Sciences, Literature and Humanities, however, it’s sort of organized.” I think that that’s a pretty powerful sort of organizational approach there. And that’s something where too, it’s enough to where we can get some multiple voices in there and get some buy in, but it’s specific enough to where I’m not trying to align what I’m doing in psychology with something that people are doing in art history, with what forestry is doing, which, again, I think is really gonna neutralize us if we try to take that approach. And also dividing it up. The last thing that makes this such a hard thing for us is that it’s just so inherently multifaceted, much more so than, for example, even when learning management systems came on the scene and we’re like, “Okay, do we require everybody use these? Which one do we want? How do we pick? How do we train people?” At the very bare minimum, we got to kind of separate out academic dishonesty is over here, and it’s got its own little set of policies, and let’s set a team working on that, and positive uses and preparing students for that workplace of the future, and so on. These are separate issues over there. So yeah, be prepared to go in and just say, “Okay, two separate things, we’re gonna deal with them two separate ways.” I can’t imagine trying to sit at home on a Saturday and craft some long email that is somehow going to bring clarity to all the different aspects of AI. And I’m wondering if there’s any sort of resonance or spark of recognition there, as leaders, that you might see.

John: Do you have any other suggestions on how we can help students develop their critical thinking skills.

Michelle: One little theme that pops up from time to time in all the great stuff that’s been written about critical thinking, and one that I think might be worth its own special mention, is an element of what you could call intellectual humility, or even, I guess, the flip side of that too, is curiosity. Think of me with kind of cutting down some student over horoscopes in my class, even though I find the basis for that in psychological sciences. If I do not care at all about why a student believes that, or what it might add to their lives to believe in, I mean, maybe it’s just something fun that is like a little Rorschach test that I wake up with every morning. If I have no interest in that, and then I’m also not demonstrating critical reflection, nor am I setting a good stage for a discussion that foregrounds critical thinking. So yeah, and that is our big challenge to faculty with our incredibly developed intellectual abilities to balance that. Yeah, evidence is important, and some facts can be right or wrong. There’s different ways of interpreting them, but here’s some areas where they have to support the analysis, balancing that with all right and I could be wrong. I could have a cognitive bias of my own that I’m not as aware of right now. I kind of have to be on the lookout for always checking myself as I’m scrolling through that feed or going through those reviews for the thing that I really do want to buy, no matter how many one star reviews that the company suppressed. I need myself to get in the habit of doing that, and then I can model that for my students, and again, set the stage for more productive discussion. So I’m hoping that more research also gets into that too. What is that wonderful, magical balance of really valuing evidence and reasoning and then realizing and I could be wrong?

Rebecca: Well, we always appreciate, Michelle, conversations with you. I know I always leave thinking a little more deeply about whatever we’ve talked about, but we always wrap up by asking, what’s next?

Michelle: Oh, thank you, and you all as well. Even just kind of looking forward to this conversation has been such a catalyst to revisit topics and say, “Yeah, this is something that I really want to be doing more in.” And so that’s going to be a bit of a what’s next. I have a sabbatical year right now, and I’m kind of trying to do what one ideally does, which is say, “Okay, what do I refocus on?” And so I’m really at that very early stage of continuing to design ways of studying, interventions that I could then share with faculty, and just getting an ever needed sense of the lay of the land in this area of critical thinking. So you can definitely look for more on that. And in fact, for folks who might be considering attending an in-person event, I am going to be speaking at a couple of upcoming events where I know that I’m going to be talking about this particular thing, again, Rocky Mountain Psychological Association for the psychologists out there, my people. If you want to come out, I think it’s in Denver this year, I have a featured session. So that’s coming up. I’ll also be speaking at the Teaching Professor Conference, which is in, I believe, Washington, DC, over the summer. Some of your listeners may be regulars, or maybe thinking about that. So that’s a big “what’s next” item? I continue to work on this big writing project on attention. It’s still at this kind of embryonic, or if not embryonic, partially formed stage, where I’m still working on it. But folks can watch this space for more on that, and that is actually related. I will tease a little bit, that attention, what do you know, gets involved in critical thinking, because it is involved in pretty much all other interesting cognitive processes that take place in the mind. So that’s my big what’s next?

John: Well, thank you. It’s always great talking to you, and again, as Rebecca said, it always spurs some new ideas and some new thoughts on these topics. So, thank you.

Michelle: Thank you. You as well.

Rebecca: I look forward to your upcoming work.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]