Category: randomized controlled trial

379. Mentor Mindset

Posted in emotion, engagement, growth mindset, higher education, inclusion, instructional communication, learning, motivation, pedagogy, randomized controlled trial, teaching
February 5, 2025

Many attempts at influencing adolescent behavior fail. In this episode, David S. Yeager joins us to the use of a mentor mindset by faculty members can  increase student motivation and academic success.

David is the Raymond Dixon Centennial Professor of Psychology at the University of Texas at Austin, and a co-founder of the Texas Behavioral Science and Policy Institute. He is best known for his research with Carol Dweck, Angela Duckworth and Greg Walton on interventions that influence adolescent behaviors. David has served as a consultant for Google, Microsoft, Disney and the World Bank, and is the recipient of over 15 awards for his work in social, developmental, and educational psychology. He has published extensively in scholarly publications, and his research has been featured in The New York Times Magazine, The New York Times, The Wall Street Journal, Scientific American, CNN, Fox News, The Guardian, The Atlantic, and many, many other places. David and Carol Dweck and others have also created a MasterClass on The Power of Mindset. His most recent book is 10 to 25: The Science of Motivating Young People.

Show Notes

Transcript

John: Many attempts at influencing adolescent behavior fail. In this episode, we examine how a mentor mindset may be used to increase student motivation and academic success.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is David S. Yeager. David is the Raymond Dixon Centennial professor of psychology at the University of Texas at Austin, and a co-founder of the Texas Behavioral Science and Policy Institute. He is best known for his research with Carol Dweck, Angela Duckworth and Greg Walton on interventions that influence adolescent behaviors. David has served as a consultant for Google, Microsoft, Disney and the World Bank, and is the recipient of over 15 awards for his work in social, developmental, and educational psychology. He has published extensively in scholarly publications, and his research has been featured in The New York Times Magazine, The New York Times, The Wall Street Journal, Scientific American, CNN, Fox News, The Guardian, The Atlantic, and many, many other places. David and Carol Dweck and others have also created a MasterClass on The Power of Mindset. His most recent book is 10 to 25: The Science of Motivating Young People. Welcome, David.

David: Thanks for having me.

Rebecca: Today’s teas are:… David? Are you drinking any tea with us today?

David: Uh…. No, I am not. I read a short story about tea recently, so that’s the closest tea story that I have for you.

Rebecca: Alright, we’ll count it, we’ll count it, we’ll count it. [LAUGHTER] I know you have water with you, which is, as we know, the foundation of tea.

David:It’s pre tea.

Rebecca: It’s pre tea. [LAUGHTER] How about you, John,

John: I am drinking a spring cherry green tea, continuing the trend of cherry-based teas. [LAUGHTER]

Rebecca: Yes, the theme continues. I’ve reverted back to the blue sapphire tea.

John: And I thought the spring cherry tea was particularly good with a couple feet of snow that we’ve had in the last few days here.

Rebecca: And it is currently snowing very hard out my window, John.

John: I don’t have windows here, so it’s perfectly nice here until I leave. So we’ve invited you here today to discuss 10 to 25 which brings together a large volume of research findings concerning student motivation in a very accessible manner. Educators at all levels have been raising growing concerns about student motivation. That’s always been there, but it seems to be a little bit more serious in the last few years, while many students report that educators seem to be out of touch with their lives and their experiences, so this book is extremely well timed. How did this book project come about?

David: Yeah, it’s a great question. It’s kind of a convergence of a few different things. So one was just my observation over years that as a society we are so bad at changing and influencing adolescents. One program after another seems to not work, and probably the impetus for that was I was asked to write a paper for the special issue of something called the future of children, like 2017, it’s on social and emotional learning, and it’s an awesome group of scholars and led by Stephanie Jones, who I admire very deeply, and I was asked to write the chapter on adolescents. So what works for social and emotional learning for adolescents? And tell you what, like I kept looking in the literature, and nothing really worked. And then when you look at the meta analyses, it seemed like there was this negative trend where the older the kids in the study, the less effective the program. And the programs that did seem to work weren’t social and emotional learning programs, they were something else that later somebody labeled social and emotional learning. It was like Facing History and Ourselves, which is like a history curriculum. And the side effect of good history education was some better social and emotional outcome. But it wasn’t like they were teaching you square breathing or meditation or something. And I just wanted to get to the bottom of it and I was like, why is it the case that we’ve got all these things that seem to work with younger kids? Because SEL programs did have pretty good results, third, fourth, fifth grade, maybe even younger. So like, what’s going on, seventh grade to 12th grade? And what I discovered was that in general, we kind of have the wrong view of young people, and therefore we keep defaulting to the same kinds of wrong programs, and then we’re shocked that they don’t work, but it’s because they come out of a particular mental model. So everything from DARE to Just Say No to anti-bullying programs, and the list goes on, and I went on sabbatical at Stanford about 10 years ago to try to get to the bottom of this and figure it out. And this book involved a lot more than that, but that’s what started it. And so it’s really been kind of 10 years of work to expand on that idea.

Rebecca: So you just mentioned DARE, and you talk about this and many other educational interventions in your introductory chapter. And similarly, college faculty who give students detailed feedback on their writing often see little feedback being implemented in follow-up submissions. Can you talk a little bit about why many interventions that are designed to improve the lives of adolescents are just so unsuccessful?

David: The first kind of passive insight was that there’s two things: One is that what the programs are trying to change, like the basic hypothesis they’re going after wasn’t the right thing to be changing. And the second is that even if they were targeting the right thing, they didn’t say it in the right way. So just take anti-bullying. The most common way to reduce bullying is to teach kids social skills. It’s like, “Well, the reason why you’re bullying is you don’t have a way of dealing with social conflict that is productive, that you have an unproductive habit.” And the research ended up suggesting that actually, for young kids like third, fourth, fifth, sixth grade, bullying is associated with poor social skills. It’s kids like some of my kids, where they’re ADHD and they just kind of get a rise out of bothering people and saying things repetitively, or getting a rise out of somebody. But for middle school and high school, often bullying is associated with better social skills, because the kind of bullying you’re doing is strategically destroying someone else’s reputation while maintaining plausible deniability, so the adults can’t get you in trouble, but then all of your friends see it, so that way you gain status and you look powerful, and that’s actually very sophisticated to do that. Real world bullying is more sophisticated than what you see in Mean Girls. And it’s not the case that kids need more social skills, because, in fact, the more social skills they had, maybe the better they would be at bullying. So what you often saw actually was an increase in bullying as a result of middle school or high school anti-bullying programs. So that was one big issue. And the second big issue is that even if you were working on the right stuff, and the right stuff, by the way, would be something about trying to maintain your reputation and gain status in a productive way, rather than by destroying others, would be an alternative, for example. But the way the programs are often framed is super lame. I mean, it’s like, DARE is like someone with a neon fanny pack throwing out headbands, like wearing socks and sandals and like, 0% of kids are like, “Oh yeah, you’re right. Next time someone I have a crush on offers me a cigarette, I’m gonna be like, ‘no, because I dare to resist drugs.’” No one’s gonna say that. And in fact, like, if you think back on DARE, they teach skills through scenarios, but like, what are the scenarios? It’s like, you do a skit in front of the class, and like, one kid stands up and they offer the other kid drugs, like, “Hey, man, you want some cocaine?” And then the other kid’s supposed to be like, “No way, dude.” And in every case, the kid offering the drugs looks way cooler than the nerd who says no, and all you’ve done is prove in front of everyone that awesome people offer everyone drugs. And so again, like drug use, on average, goes up in response to DARE and similar programs. So the way to summarize this is that you have to be attuned to the adolescent desire for status and respect, their desire to maintain a good reputation in the eyes of people whose opinions they care about, and if you don’t, then you might just push them away, or you might even make the bad behavior more enticing. And so that’s a launching off point for this book, where I try to suggest better approaches.

John: One of the other things you talk about and you’ve written about is how to provide feedback effectively to students. And one thing that many people have been taught for a very long time and still are is to use that compliment sandwich where you put some critique in between two positive comments. And it turns out that that’s not very effective. One of the things you suggest is that a wise feedback strategy may be more effective. Could you talk a little bit about why the compliment sandwich doesn’t work and why wise feedback may be a more effective way of providing feedback that students will respond to?

David: Yeah, so I’m starting from this premise that young people are like exquisitely sophisticated in they’re thinking about their reputations and how others are perceiving them socially, and given that, when they’re critiqued by a teacher or a coach or a manager or anyone who has power over them, then they’re not just processing it as information to include in their mental model of perfecting a skill, there’s a social implication: “Who is this person who’s implying that I’m not good enough yet? And why do they feel entitled to very publicly say that I’m inadequate? And furthermore, do they think I’ll ever be not inadequate? Like, is there any possibility that I could grow and improve?” So they’re asking these big, existential questions that have their roots in concerns about their status and reputation. What is the adult thinking? Well, they’re thinking, I don’t have to give you any feedback at all, like you’re lucky that I’m even spending any time on you, and don’t complain, stop whining, and the least I’ll do will be to try to soften the blow by being a little nice, but if you don’t like that, then you’re entitled, and you’re a wimp. That’s kind of a lot of adults’ approach. And so what occurs to them is to be like, Alright, well, I don’t want you to cry, so I’m gonna do the compliment sandwich. And the compliment sandwich is something like, “Look, I love your enthusiasm.” It’s a nice thing. And then it’s a negative thing, like,”All of your work is terrible and has to change.” And then there’s the nice thing, like, “Thanks for turning it in on time.” And we do our mental math. It’s like we’re balancing a stoichiometric equation. And then we say “Net positive. So you should thank me for this feedback.” But again, you haven’t addressed the core question in the young person’s mind about their status and their worthiness in our eyes. And so the alternative is to directly address the concern that young people have no value, that they have no potential, they have no future, et cetera, by using something that Geoff Cohen, a Stanford professor, one of my mentors, has called wise feedback. And wise feedback is very simply saying, I’m giving you these comments because I have very high standards, and I know that you can meet them. So it’s first an appeal to a high standard, and second an offer of support so that way they can meet it. And it turns out that that comes across as a very respectful thing to say, and therefore it builds trust, and it puts the young person in a place in which they’re open to criticism, because you’ve addressed their main fear, which is that we view them as incompetent.

Rebecca: So in 2014 you were the first author on an experiment involving wise feedback. Can you talk a little bit about this study?

David: Yeah, so in one experiment that, again, was led initially by Geoff Cohen, my collaborator, and other great people like Valerie Purdie-Vaughns at Columbia, we partnered with a school in the East Coast, and we had teachers assign their seventh grade students an essay to write. So kids write a standard five-paragraph essay, and then the teachers cover them in comments so they do whatever the six plus one writing tactics are, and they kind of do their worst, and then the kids have a chance to revise the essay or not. And the question is, when do kids get their essays back, covered in red ink and say, “This is great. I’m really happy for all of these ways in which I can improve. So thanks for pointing out my flaws, and I’m excited to fix them. So here we go.” When did they say that? And when do they say, “My teacher is such a jerk that he spent his whole weekend trashing the essay in the ego of a seventh grader.” And we found that in the default control group, where we didn’t do any wise feedback, only about 40% of kids even turned in their essays. But we also had teachers append handwritten notes on the essays that said, “I’m giving you these comments because they have high standards, and I know that you can meet them.” And then they sealed the essays in an envelope and handed them back, so that way kids didn’t know which got that note or which got a control group note that didn’t convey any new information. And then a week later, kids had the chance to revise, and what we found is that 80% of kids revised their essay in the wise feedback group. And for me as a parent, you know, I have four kids, and so if twice as many of my kids listened to the things I told them to do, then at least two of my four kids would have pants on by the time it’s time to go to school, and my life would be better. And I was a middle school teacher, and I remember spending all weekend grading essays and thinking, “Oh man, they’re going to be so grateful for all this feedback. They’re gonna carry me on their shoulders like Robin Williams at the end of Dead Poets Society and be a hero.” And instead, they were just like, “Why is this guy such a jerk?” And my life would have been a lot better if I knew about this research at the time. But I will say it’s been interesting since we published that study. There’s a tendency to over claim how important it is. Obviously this is just one assignment one time. I’m not claiming this is a magic bullet. The author, Dan Coyle, in his book The Culture Code, called it magic feedback. And I don’t like that. I don’t think it’s magic. To be honest, what I think the magic is is the feeling of dignity and respect when you’re in a vulnerable position and someone is looking down on you, and they instead assure you that you are a person of worth and value, that, I think is magical. It might happen to the post it note might not. There’s a lot of other ways you can show that, but I think when you do that, then you start tapping into what’s going to motivate adolescents to change. And the more we do that in schools, I think the better off they would be.

John: Is this something that cuts off once you pass adolescence? Could this also refer to the feedback from reviewer two [LAUGHTER] on that paper?

David: It’s funny, I’ve grown less patient as a reviewer over my time as a reviewer. I’m only mean to a specific kind of author. I lose my patience if you’re like, at the University of Chicago, and you have every resource at your fingertips and you still wrote a lazy paper, then I’m like, “I don’t have time for you.” Anybody else, I’m wise feedbacking away. But anyway, I guess I would say that I’m not arguing there’s a magical cutoff of 10 years old, all of a sudden you need wise feedback at 25 you need it, in 25.1 day you don’t it’s really more that the window of 10 to 25 is like the onset of puberty until the adoption of an adult-like role. And that just tends to be a period where your status is very precarious and ambiguous and it’s uncertain to you if you’re going to be taken seriously, if you’re going to be afforded the rights and privileges that you think you deserve. Like a good example is this wonderful study I love from 1998 nobody cites by Ruck and colleagues, and it’s on self-determination needs. And it’s very simple idea. They ask kids at what age do you think you should have a certain right or privilege? And then they ask adults, at what age should kids have it? So an example is being able to write a letter that’s critical of the principal and publish it in the school paper. So seventh graders are like, “Yeah, I should totally be able to do that.” And adults are like, “Not till 11th grade.” So you’ve got this gap. [LAUGHTER] So you got, like, four years or more, depending on the measure, there’s variation in that outcome. But like, you’ve got 1, 2, 3, 4 years where a young person is ready for some right and privilege and society’s saying, “No way.” That’s what I call the adolescent predicament, that your status and respect is in question because of that gap and mismatch, and I think you could experience that later in life. This one interesting guy read my book and emailed me. He’s a general counsel for a large like Fortune 100 company, and he’s like, “Man, I really have to worry about giving feedback to my junior associates, like when they write a brief.” So he’s thinking of himself as the mentor. And then it came out that he’s actually doing a master’s in education because he’s about to retire from being a lawyer, and he’s going to go be a classroom teacher for the sunset of his career. And I was like, as a 60 year old, but first- year teacher, if there’s a principal in the back of the room sitting there with a clipboard making notes on every single thing you’re doing in the classroom, are you going to feel nervous and worried? He’s like, “Oh, absolutely.” So he’s gonna feel back again like a 13-year old getting his essay criticized. So I really think that the operational definition of adolescents, at some level, is this adolescent predicament, this mismatch between the status and respect you think you’re ready for, that you desire, and then what you’re afforded by your context.

John: Beginning early in your book, and through much of it, you talk about the mentor’s dilemma. Could you talk a little bit about what that is and how we can address that?

David: Yeah, the mentor’s dilemma is, again, something coined by Geoff Cohen. It’s the simple idea that it’s very hard to simultaneously criticize someone and motivate them. And the reason why it’s a dilemma is because it feels like you have two bad choices, like one choice is “I’m going to be really honest and tough and impossible to please and tell you all your flaws, but sacrifice your motivation and crush your spirit.” Or the alternative is “I’m going to be nice and friendly and supportive, but I’m not going to tell you the truth. I’m going to withhold feedback.” And neither of those really feels satisfying. It feels like they’re not what you’re supposed to be doing. And what we’ve found, and what we’ve proposed, is that the solution to the mentor’s dilemma is something we call the mentor mindset, and that’s kind of being the walking, talking embodiment of the wise feedback note, where in all your interactions with young people, you’re neither too harsh nor too much of a pushover. You are very tough, you have high standards, but your supports are as high as your demands. So that way young people can meet that demanding standard.

Rebecca: Can you give us some examples of how this mindset might be implemented in a day-to-day student interaction?

David: Yeah, so mentor mindset is something that comes up a lot in let’s just say, how you deal with a student mistake. So in our work, we have this fellowship program for math teachers. Carol Dweck and I and several others developed a new program called FUSE, Fellowship Using the Science of Engagement. And the idea is that the culture teachers create in the classroom can influence whether students benefit from our instruction. So I get to spend all day giving you illustrative math or Eureka math, and correcting all your math misconceptions. But if, fundamentally, the kid thinks that the reason why they made a mistake is because they’re a lazy slacker that the teacher is looking down on, because the teacher explained it clearly and you didn’t do it, and therefore you weren’t paying attention, if that’s your default, then you’re not going to engage with this conceptually demanding, cognitively sophisticated curriculum. You’d be like “No way, because if it’s not clear to me right away, then it’s a threat.” So as a teacher, you need to have a culture in which mistakes are your friend. Otherwise you’re stuck doing a curriculum in which students only ever master things at 100% and they’re never confused, they’re never lost. Therefore, in order to create that culture, you have to have good routines for troubleshooting mistakes with students, because you can’t just blame and shame them or yell at them for making mistakes, nor can you avoid the mistakes altogether. And so I spent some time observing great mentor mindset teachers, teachers who had very high standards and were very supportive, and they make students own their thinking, and they ask tons of open-ended questions, and the students squirm sometimes. And a normal novice teacher, like I would have done, they feel uncomfortable. They think the student’s head’s gonna explode, that they’re sitting there not knowing the answer, but their wait time is unbelievable for these great teachers. And there’s a routine that I call collaborative troubleshooting, and there’s three steps. One is validating what the student has done, so like finding something right about the student’s answer. Next is asking questions to understand: so where did this come from? What was your reasoning? And that ends up being super important, because a lot of teachers immediately jump in and presume they understand the student’s mistake. Oh, you forgot to carry the two. So next time, carry the two. But sometimes we’re wrong, and what teenagers hate is when adults make assumptions about them. And furthermore, when you make assumptions and you’re not right, you end up telling them solutions that were already obvious to them, and then that’s also insulting, like imagine you were lost and you pulled over to ask someone for advice on where to go, and then they’re like, “Did you think about checking Google Maps?” You’re like, “Yes, I thought about checking Google Maps.” It’s a very insulting thing to say, like it’s the most obvious thing. So a lot of times when students come to teachers, that’s how they react, is they tell them to do something obvious that the students already tried. So great mentor mindset teachers don’t do that. They ask questions to understand before they tell and then, last, they kind of ask leading questions to build a bridge to a better understanding, and they wait for the student to piece it together in their minds. Novice teachers don’t do that because they feel anxious that the student is stressed, and then they rush in to tell the answer. This goes back to classic research from Mark Lepper on expert tutors in the mid 90s. Mark Lepper, a Stanford psychologist who coined the term intrinsic motivation initially. He spent years just watching great tutors and found that over 90% of what they say is a question. They’re not sitting there telling you what to think. They’re like, “Huh? Where did that answer come from?” or “Is that right?” And then the student has to have a coach in the head to try to solve the problems. There’s a lot of other examples, but I love that one, both because it’s intuitive once you get it. It’s also not obvious when you’re a novice teacher,

Rebecca: I feel like those same three steps would be great in the healthcare industry.

David: Any industry. You don’t think people like doctors talking down to them for making such idiotic health choices?

Rebecca: I was thinking that might be a great place for teachers to get some empathy. Like, “Where have you been talked down to recently?” Or tech, or any of these places.

David: Well, the interesting thing is, we have this fellowship program for teachers, and their empathy is not hard to get. Just ask them about a time they talk to an instructional coach. Instructional coaches, ideally, are these endlessly empathetic listeners. But that’s not often how it works. It’s like they race in, they tell you what to do. You’re doing everything wrong, and I’m going to check your test scores next week. And I have compassion for the coaches too, because a lot of districts, especially poor districts, use instructional coaches like backup admins, so they’re pushing paper three fourths of the day. They hardly ever get to actually coach. So without blaming any individuals in the system, most teachers almost immediately have access to this idea of everyone making an assumption about your limitations and then telling you obvious things and then making you feel like an idiot. So even though they know that, that’s still often teachers’ default when they talk to kids.

John: As part of this whole mentor mindset, the goal is to treat students with respect and to let them know that you think they’re capable. But a lot of students come in with stereotype threat, a lot of students come in with perhaps some imposter syndrome that they don’t feel they’re going to be successful to begin with. How can we help build that growth mindset in students who may have some doubts about their own ability to thrive?

David: Well, I’d just say that stereotype threat often is more of a situational predicament. It’s more like they’re aware that other people could judge them, even if they don’t personally believe the stereotypes. But I think that for students who are used to adults using something I call an enforcer mindset, which is kind of all standards, no support, or as someone I interview in the book calls it: “Yell, tell, blame and shame.” If that’s what you’re expecting, then it’s very hard for students to see the teacher’s growth mindset and be like, “Yeah, that’s great. I love it,” because all they see is the standard, and they think, “Oh, here we go again, another grown up is going to yell at me for not being good enough.” So I spent a long time interviewing this great teacher who’s kind of like the best growth mindset teacher in Texas. His name is Sergio Estrada. He’s at Riverside High School in El Paso Texas, and Carol Dweck and I interviewed him a lot, like every Friday for months. And every once in a while, some kid would wander in his class during his off period, when we’re talking to him. And I would be like, “Hey, who’s the opposite of Mr. Estrada?” And, I tell you what, I’ve heard some stories. And one kid, Emiliano, told me a story about his English teacher, and his English teacher assigned some essay where they had to write the pro and con or side one/side two, and then he could only come up with one of the two. So he went to the teacher after class to ask for help. He’s like, “I have the assignment. I can only figure out half the assignment.” And then the teacher said, “Tú no lo entendiste porque no quisiste,” which is you didn’t understand it because you didn’t want to. And he was crushed by this, and came back to me and was like, “Dr, Yeager, it’s so unfair.” And I was like, “What do you mean? It’s unfair?” He’s like, “Well, I have ADHD, I only understand half of what anybody tells me,” like, “this is how I get the other half. I can’t imagine wanting the other half anymore than coming after school to come talk to this teacher to try to get it and yet she’s assuming that I was screwing around in class, that I don’t care, I don’t care about my education, don’t care about learning, and then I’m trying to, like, ruin her lesson plan by making everybody be behind and asking questions.” And like, Sergio would never do that, but he also knows when students come in his class that that’s what they’re used to. So he has learned he has to give a speech at the beginning of class explaining that this is a new culture. Say, for example, this idea of questioning, Sergio, like a great tutor asks questions way more than he tells. So if a student is doing a problem set and they’re like, “Mr. Estrada, is this right?” He’ll say, “Well, I don’t know. Do you think it’s right?” So there’s a way of doing that that does not go over well, it goes over like a lead balloon. There’s a way of doing it where you look like the jerk law school professor cold calling people and forcing them to explain their thinking, and you’re trying to humiliate them. And adults do often ask questions of kids in public to humiliate them. I mean, as a parent like, I do that sometimes, even though I don’t want to, I’m like, “what were you thinking?” Or like, “Is that the right way to put your chair away?” Or like, “Should you be grabbing that cookie?” Like, it’s not an authentic question. It’s a question meant to shame and humiliate. So that’s what kids are used to. And so Sergio, before he does his old questioning routine and his collaborative troubleshooting, he’s like, “Alright, guys, I care deeply that you are good at physics after this class. And in fact, everyone who passes this class eventually deeply understands physics, and I don’t want to deprive you of the chance to know that you can do it, so I’m not going to tell you answers, but it’s not because I’m being unhelpful. It’s because I want you to own the thinking. And every year, students eventually get it, and every year students don’t like it at the beginning. So I just want you to know that I’m not trying to shame you, not trying to embarrass you. I’m trying to honor and respect you and make you know that I think you can think, and I want you to build that muscle” or something like that. And then he’ll print it out and he’s got it on the wall. So the next time students are mad or offended and they’re thinking, “Oh, here we go, another enforcer mindset, he’s going to yell at me.” It’s like on the wall, it’s crystal clear. And then by the end of the year, these are students who say, “Mr. Estrada changed my life. He believed in me.” They’re like the first in their family to go to college. They’re getting internships at NASA. I’m not exaggerating, like every summer, students get internships at NASA, but it takes that transparency. You can’t just show up and do mentor mindset if students have that enforcer mindset baggage that they’ve experienced.

Rebecca: So I can imagine that some folks might be thinking adopting a mentor mindset might increase their workload somehow. Is that true?

David: Yeah, it’s interesting. It’s one of the first things I worried about writing this book. I was like, “Oh, am I writing an impossible to accomplish self-help book?” I mean, it would be like, grabbing a diet book where like, “The best way to lose weight is to never eat anything that you enjoy.” Well like, “Yeah, I knew that. The hard part is to eat what I enjoy and still not gain weight.” So I’m not saying the best way to be a teacher is to do it 120 hours a week. And there are things that Sergio does where he’s like, “I don’t recommend anyone do this. It’s like a lot of work.” But I will say two things. One, every great mentor mindset teacher I saw had procedures and policies that they set up early on in the term, and that often involved getting peers to look at each other’s work and having a good culture of peer feedback. It’s kind of a “ask two before me” situation, and they’re not doing that to dismiss kids like gnats. They’re doing it to give them a sense of agency and teach a skill of collaboration. But that, by the way, saves them tons of time, because 80, 90% of errors, like some really high percentage of errors, are things where the minute the kid tries to explain it to their peer, they’re like, “Oh my god, I see it now.” So they actually didn’t need the teacher, like, you just need another human being when they’re explaining. So one thing is, you can still do a lot of this stuff and save time if you just have good procedures for peer collaboration and feedback, it saves you tons of time. The second is, there’s often a short-term, long-term trade off. It does take a little more time to set up procedures and create a relationship of respect, but then the whole second semester, you’ve got independent, autonomous students that are working hard. You don’t even have to like mess with them. So I remembered we interviewed Sergio during COVID, and it was like, April of 2020, and everyone’s on lockdown or whatever. And I was like, Sergio, how’s it going? He’s like, “I feel really bad.” Like, “why do you feel bad?” He’s like, “I’m bored and I feel guilty.” Like, “Why are you bored? Why do you feel guilty?” He’s like, “because I don’t do anything.” “What do you mean, you don’t do anything?” He’s like, “I post the problem sets, then students meet in groups, and they talk about physics all day. Maybe they ask me a question, but otherwise they just work diligently and then learn physics.” And he had the highest rates of passing college level physics that year in his whole district during COVID. And he like was on Tiktok all day, not because he was goofing off, but because students were so self sufficient, and I think that’s the payoff, is you use a mentor mindset, and then you get people where their work is like kind of area under the curve, leading to greater growth and learning, because you’ve created independent, agentic young people.

John: So, it sounds like giving students more agency, perhaps an autonomy-supportive instructional approach and using peers for feedback could be a really effective way of helping improve motivation and maybe some authentic assessments where students see that the work they’re doing has some intrinsic value beyond just a series of hoops they have to jump through, that the instructor is kind of imposing on them to get away from that enforcer mindset and giving students more control over the learning process, it sounds like would be really helpful.

David: Yeah. I mean, I think that there are curricular changes and pedagogical changes that are well known in the literature to be good for student learning and make them agentic or whatever. And I’m not pretending like I’m inventing constructivism or anything like that. It’s more this idea that there’s an unspoken conversation between a student and their past that is present anytime a teacher is pushing them, challenging them, using this maybe more progressive curriculum. And if you’re not attuned to that conversation, and if you’re not able to help a student see that this is different, then they’re going to be wearing a set of lenses that ’s going to cause them to interpret what we’re doing in the worst possible light. And that’s not because Gen Z is too wimpy or too woke or too entitled or whatever it is. It’s because they’re legitimately responding to prior experiences that they’ve had, and they’ve made sense of those, and they’ve allowed that sense making to influence them in the future. That’s just how human thinking works. We form little lay theories of the world based on our experience. So we kind of need to be agents of change, of giving them a new theory that when a teacher is pushing you and challenging you, it’s because they believe in you, and they’re often going to offer you support, and if they don’t, then the support is what you need to find in order to meet that standard.

Rebecca: Transparency can be really transformative and it sounds like that’s something that you’re really advocating for, is being really transparent about [LAUGHTER] doing things that’s respectful.

David: Yeah, like I’ve been saying, you can’t just presume that they will interpret your actions in the best possible light. So a good example that helps, it’s kind of an analogy to teaching, is a study led by Kyle Dobson, who’s a University of Virginia professor, and previously was my postdoc, and we did an experiment where we looked at the context of police officers approaching civilians on the street, and typically, what happens if a police officer comes up and asks you a bunch of questions, is people feel threatened, and we know why. It’s because there are all kinds of stories of arrests and police brutality and things like that. But police officers are baffled by this, because they’re like, “Well, my badge says serve and protect, and why would you not think I’m here to serve and protect? Of course, it’s what I’m doing.” And so there’s this mismatch between the intentions of the officer usually, and then the responses of the civilians. And so we did a study where we had people sitting in desks all around this downtown Austin, and then we had real on-duty police officers, wearing guns and badges, go approach people and ask them a bunch of questions, and they either launched into the questions in a control group, and there, people felt super stressed about it. They had short, kind of curt replies. Their stress physiology didn’t look good when we measured it, implicitly using a PPG monitor on their wrist, and in general, there was very poor rapport. But when the officer started the interaction with a simple transparency statement, when they just said, “Hey, I’m walking around trying to get to know the community, would you mind if I ask you some questions?” When that happened, then people were really open to the conversation. They were kind of into it, and they formed a rapport. And what that suggests is that even in a context where the institutional authority should, in their minds, be viewed as positive, but in civilians’ minds, there’s this history of distrust. And we can look, we see trust going down every single year on major metrics for police and other institutions, you can kind of create a bubble where trust can be restored if you begin with a transparency statement. Now I did a workshop one time for a bunch of principals and superintendents on transparency statements, and I gave the example of, “Hey, imagine you’re a school leader, and there’s a teacher who’s brand new, and you went and set in the back of their class and you took a bunch of notes, and you were giving them feedback. How could you use the transparency statement so that way they didn’t view your feedback in the worst possible light.” And one superintendent was like, “Oh, very easy. I would go up to him and say, ‘hey, look, just want you to know the reason why I’m observing your class is because I think you’re the worst teacher and you need to be monitored.’” And I was like, “You should not be transparent about that. Keep that to yourself, like because that is the main worry that was in their mind already, so do not say that.” So I’m not saying you should be transparent in that way. What I’m saying is [LAUGHTER] be transparent in dispelling the most negative possible interpretations and instead replacing them with better ones. I appreciate the sound of your forehead being slapped [LAUGHTER] as I told that story. That’s a real story. And they were very earnest. They’re like, this is the first person to raise their hand, like, “Oh, this is easy.” I was like, “Really, in a workshop of 100 school leaders, that’s your answer?”

Rebecca: We all know that person.

David: Yeah, they were very confident.

John: In terms of knowing that person… You talked a little bit about the importance of helping students develop that growth mindset and so forth, but there was that study by Elizabeth Canning and others that talk about the importance of faculty having a growth mindset. Both Rebecca and I at various times in various ways do work with professional development. How can we help faculty recognize that maybe having an expectation that students could grow and could become more successful in learning? How can we spread this, other than encouraging people to read your book and other resources, how can we get faculty to recognize that students can become more proficient as they work and develop skills?

David: Yeah. I mean, I think that the beautiful work by Mary Murphy on fixed mindset cultures and Elizabeth Canning is a collaborator, has suggested that what faculty believe can dramatically influence the decisions that they make and what makes sense to them from their perspective. The question of how to change faculty is a little different. So I don’t think that you can just give K-12 teachers or college professors the student growth mindset intervention, like, “Hey, here’s this stuff about the brain.” And a big reason why, we’ve learned, and I write about this in our PNAS article last year, led by my postdoc, Cameron Hecht, growth mindset is really good news to students. It’s like, “Oh, awesome. I don’t have to feel dumb anymore when I make a mistake.” It’s not good news to teachers sometimes, because they’re like, “Wait a second, if mistakes are learning opportunities, then it sounds like I have to re-teach you stuff after you make a mistake, and then I have to regrade it after you try it again, and that sounds like more work.” And as I’ve said, that’s not always the case, but that’s like people’s first interpretation. And so you can’t use that same argument. What we found is you need to use something called a values-aligned argument. And this is a framework developed by Chris Bryan and me. Chris is a business school professor at UT, but he’s a social psychologist by training, and values alignment is very simple the idea that if you want to change someone’s beliefs or attitudes or behavior is easier to align the targeted attitude, behavior, belief, with what they already care about than it is to try to convince them to care about something new. And in the case of teaching, for college professors like we’re very rarely held accountable for what we do, and very few people are ever told even how to backwards plan, let alone teach. And for K-12, they do obviously care a lot about instruction. And obviously college professors do care, but like, in my experience, often the best college professors know about as much about teaching as like, the middle of the road K-12 teacher, because we’re just not given much pedagogical instruction. So for professors, like it’s very hard for your culture of learning and creating growth mindset culture to get to the front of the list. For K-12, it’s also hard because you’re given an endless number of things you’re supposed to be changing and fixing at all times, and you’re going to three PDs a year and there’s three new binders you’re going to put on yourself. It’s going to make you feel guilty that you never implemented them. And so the way we think to cut in line, to get to the front of the line of your motivational priorities, so that a growth mindset culture, or what I call a mentor mindset, is what you want to prioritize, is to align it with what people already care about. And what people already care about often is having the kind of classroom where, when you walk in, students are tracking what you’re doing. They’re motivated, they’re engaged, they’re curious. They ask you questions about your content area. They care more about the learning than the grade. You don’t have this constantly threaten them or blame them or console them to do the minimum. It’s really motivating to have that kind of better motivated classroom culture. And so we say, “Look, if you believe that no one can grow and learn and that it’s all or nothing, that your first grade will be your last grade, then students will pick up on that as disrespectful, because you’re basically telling them that their efforts are wasted and that there’s no point in trying, because if you got an A, well then you’re going to get an A at the end. If you got a C, you’re going to get a C in the end. So what’s the point of doing anything?” That’s a disrespectful way to run your class. And if within 10 to 25 young people are sensitive to disrespect, then they’re going to be disengaged, if that’s how they feel. But if you want to engage them, they need to feel respected. And the way to respect them is to make them feel like they can grow and learn even when they struggle. So we frame growth mindset not as like you should do this, like eating your broccoli kind of thing. It’s more like, maybe there’s some broccoli lovers out there. I don’t there’s an alternative. I don’t want to offend any broccoli eaters. So brussels sprouts…

Rebecca: I like both of those.

David: …specifically brussels sprouts covered in bacon as an appetizer, is mine… But anyway, let’s just pick a food that people kind of have to hold their nose and eat for any objecting broccoli or Brussels sprout eaters and like, that’s not gonna work. But what will work is saying this is a way to be the kind of person you always want it to be, and that, I think, is motivating and exciting for people. And so mentor mindset, framed as a way of respecting young people, and therefore as a way of engaging them, is, I think, more effective.

Rebecca: I think that’s a good pro tip [LAUGHTER] to wrap up on. So I want to be cognizant of our time, so we always wrap up by asking: “what’s next?”

David: Our big initiative right now is basically taking the mentor mindset ideas from the book, and spreading them to educators. So we have a fellowship called the Fellowship Using the Science of Engagement or FUSE that Carol Dweck and I and our collaborators, we personally wrote all the materials and are very excited about them and the results look really good. So in a large randomized trial that’ll be coming out this year later, we find that teachers, if they change their mindsets, they change their behavior. Students, then, when they get the mentor mindset classrooms, they view the classroom as more respectful and more motivating. They feel comfortable raising their hand and asking questions, making mistakes, and then they learn more. And the effects, well, I don’t know if they’ll always be this big, but they look pretty big right now, of .4 standard deviations, which is pretty large in education. And then teacher attrition is down by half, and so is burnout. So the mentor mindset stuff ends up being the only educational intervention that we know of that, in a rigorous randomized trial, changes teachers, changes students’ perceptions, changes students’ grades, and improve teacher well being, and it does so for like $23 a kid, so it’s like 10 times more effective than individualized tutoring at scale for like, 1/100 the cost. So that’s coming, and we’re thinking about how to expand that, scale it and make it better. I will say we hired Sergio Estrada as one of the facilitators for FUSE, and so it’s a real treat for them.

Rebecca: …some really exciting stuff.

John: It sounds like a wonderful program, and we encourage everyone to consider picking up a copy of your book. We’re going to be using it as a reading group on our campus, so we’re really looking forward to it, and we will include links to all the studies that you’ve referenced in the show notes.

David: Great.

John: So thank you, although I do have to say one other thing, that putting bacon around brussels sprouts sounds a lot like a compliment sandwich [LAUGHTER] in some way.

David: I mainly just listened to the Jim Gaffigan bit on bacon and so that was fresh in my mind.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

205. Tutoring

Posted in access, adaptive learning, behavior economics, edtech, equity, higher education, learning, mastery learning, pedagogy, randomized controlled trial, STEM, teaching, tutoring
September 15, 2021

Equity gaps in educational outcomes play a major role in perpetuating economic inequality. In this episode, Philip Oreopoulis  joins us to discuss his research examining how tutoring and computer-aided instruction can be used to reduce disparities in educational outcomes. Philip is a Distinguished Professor of Economics and Public Policy at the University of Toronto, the Education co-chair of MIT’s Abdul Latif Jameel Poverty Action Lab, and an award-winning researcher who has conducted a wide variety of studies relating to education and educational policy.

Shownotes

Transcript

John: Equity gaps in educational outcomes play a major role in perpetuating economic inequality. In this episode, we discuss research examining how tutoring and computer-aided instruction can be used to reduce disparities in educational outcomes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Philip Oreopoulis. Philip is a Distinguished Professor of Economics and Public Policy at the University of Toronto, the Education co-chair of MIT’s Abdul Latif Jameel Poverty Action Lab, and an award-winning researcher who has conducted a wide variety of studies relating to education and educational policy. Welcome, Philip.

Philip: Thanks so much for having me.

Rebecca: Today’s teas are:…Philip, are you drinking tea?

Philip: My tea is coffee. I love coffee. I once looked for a reason not to drink coffee, I couldn’t find one. I love my black coffee.

Rebecca: A true researcher at heart. [LAUGHTER]

John: And I am drinking a bing cherry black tea, a custom Tea Republic tea made for Harry & David.

Rebecca: And I have Irish breakfast tea. I really need to get some new tea [LAUGHTER]. I’m going to a tea store this weekend, so I’m looking forward to getting some new options.

John: And we have lots of tea in the office, some of which may not be as fresh as it was a year and a half ago. But this one still is good. It was purchased right before the shutdown.

Philip: You guys are inspiring me. I think I’m gonna have some tea sometime today.

Rebecca: All right, good, good.

John: In a November 2020 Scientific American article, you describe a meta analysis that you worked on with some colleagues that found that tutoring results in significant improvements in student learning. Could you describe this meta analysis a bit and what you found?

Philip: To backtrack a little bit, how it got started: my colleagues at J-PAL, Vincent Quan, Andre Nickow, and I, had heard about the potential of tutoring to be an effective form for increasing test score learning performance. For example, there’s Benjamin Bloom’s seminal article in the 80s, where he had two very small studies done by his students that both found off the charts improvement from offering tutoring in randomized control trials. In fact, that’s why he called it the ‘“2 sigma problem” that he found estimated impact from these two small studies were raising learning performances by enough to potentially solve most of our problems that we would be having in education policy. There were a number of recent studies as well, a randomized control trial coming out from the University of Chicago’s Ed lab, also finding very promising results from an RCT looking at providing in-class tutoring to grade nine students. And so we wanted to explore whether there was some consistency in these results, so we decided to try to take a more systematic look, and we gathered up all the RCTs, randomized control trials, in the last 40 years for about 96 studies, and we took a look and we found that consensus was quite remarkable. About 80% of those studies found significant effects larger than .2 of a standard deviation, and the average effect size was .38 of a standard deviation, which is like the equivalent of almost an entire extra year of school, from receiving these programs. And not only were the impacts really quite meaningful, as about as large as you get from education interventions, but they were consistent across the board. I think that this is about as much consistency as you’re ever going to get in an education policy intervention. So we were quite excited about that. We found that the effects were pretty consistent no matter which type of program that you looked at. They were larger for things like in-school delivery, three days a week, one-to-one delivery, full time tutors, but even in cases where that wasn’t the case, usually there were still significant effects.

Rebecca: Can you talk about what age the students were, what grades they were in?

Philip: It was for K-12.

John: I think it’s probably safe to assume, though, that the same effect would hold in the college environment as well. Those are some pretty dramatic effects.

Philip: Of course, to some extent, maybe it’s not that surprising. Giving instruction one-to-one leads to higher learning gains, and the biggest challenge, of course, is cost. We can’t all have our own teacher when we go to school. And so the biggest challenge, which gets back to Bloom’s point calling this the 2 sigma problem, is I think we have a powerful intervention to help education, it’s just that it costs too much to implement it on a larger scale. So the fundamental problem is to figure out a way to scale this in a way that can complement the classroom instruction.

John: And so that’s one of the things I think you’re looking at now, how this can be scaled up in a more cost effective manner. Could you tell us a little bit about your current research in terms of computer-assisted learning?

Philip: Sure. So computer-assisted learning or computer-assisted instruction is a type of educational software designed to help students progress through topics at their own pace. It has a lot of similar features as what you might receive when you’re receiving tutoring. So a typical example might be Khan Academy, MATHia, there’s lots of other types of software designed to help with different topics, math and reading, but they all have these sort of common features that allow students to progress through topics at their own pace. You receive immediate feedback from trying to work through your own problems and a chance to understand where you went wrong. If you do make a mistake, there’s data that’s generated from going through it that someone like a teacher might be able to follow and respond to. And so computer-assisted learning can, in some ways, simulate the tutoring experience, but of course, at a much lower cost. The challenge is you don’t have a real person guiding you through it. So even though a platform like Khan Academy is easily accessible, your willingness or motivation to go through it on your own is probably not as great as if you had a real person guiding you through the same material. So there has been some experimental evidence on computer-assisted learning, not as much as theories on tutoring, but of the 15 or 20 randomized control trials that have been done in this area, they have also been showing quite promising results. In cases where computer-assisted learning is provided, especially during a school setting, those receiving it also seemed to be performing at significantly higher rates than those in the comparison group. So there does seem to be some promise at using computer-assisted learning to generate the gains that we see from tutoring. But the way to introduce it, the instructions that teachers need to learn how to use it effectively, are not yet maybe as developed as we’d like them to be. So getting to, I guess jump into what I’m working on, I think that there’s a lot of potential for leveraging existing resources to combine with computer-assisted learning in a way that might come close to the tutoring experience. And so what I’m thinking of is in the classroom, that the kind of facilitated practice that might go on, say, in a math subject might be much better through a tool like Khan Academy than paper and pencil that we often give students. And so the question I’m investigating is around reshuffling the classroom in a way where the teacher is trained how to use computer-assisted learning more effectively in the classroom to generate that type of experience. So in the context of the program that I’m looking at now, which tries to integrate Khan Academy more into math classes, the teacher is still instructing and presenting topics, but now emphasizing the students following an individualized roadmap that allows the students to progress at their own pace, rather than having to keep up even if they’re missing on topics and not understanding. So the program which we’re calling “Coaching with Khan Academy,” or CWK, has students receive a roadmap of incremental topics and videos to follow at the start of school that roughly proceed in the same order that the teacher is going through. Now, the teacher has the students to try to work on this roadmap for at least an hour, an hour and a half a week, and tries to facilitate that time during the class and encourage more done at home, and the students then have the ability to hopefully get into a routine of watching a video and taking the exercises, and if they don’t score high enough on the exercises they’re asked to try to understand why they made the mistake using the hints and tips and guidance that Khan provides or gets help from the teacher, and then repeat it so that they don’t move on to the next topic until they’ve mastered that. So the students are not proceeding all in the same pace, but it is just a much better way to learn math such that the students don’t go on to the next topic until they’ve established a strong enough foundation on the first one.

John: During the global pandemic, most high schools moved to emergency remote instruction for an extended period, and there’s quite a bit of evidence that that led to a decline in overall learning, but also some growing achievement gaps which are tied to household wealth and the wealth of the school districts in which the students reside. What types of policies could be implemented at the K-12 level so that students are more equally prepared for entry into college

Philip: On COVID, we’ve all been exposed to online learning now, and most research suggests that it’s not a great substitute for in-person but there are certain benefits from being able to speak with a real person over a computer in regards to tutoring. So the biggest one is convenience, both for the tutor and the tutee. It’s nice to be able to jump in on a call and spend just 30 minutes on that or an hour, and not have to drive to the person’s location or do this after school. The opportunity to facilitate more tutoring, I think, is increased by having this online access. So I think there’s a lot of interesting promises from that. This one particularly interesting study that was done during COVID last summer, where a group of Italian faculty organized a volunteer tutoring experiment where they got the Deans of their respective universities to invite university students to volunteer their time, three to six hours a week to reach out and connect with students who have been struggling in the high schools and lower grades. And on the flip side, they got the school districts of several locations in Italy to ask teachers to identify students that they thought could benefit from having this one-on-one instruction. And then the response was great in both ways, there were a lot of people willing to volunteer their time for this effort, and there was also a lot of perceived need for students that needed this. And so from this large set-up, they randomized who they were able to give this offer of assistance to. And it was done all online, sometimes over the phone, but more often through Zoom, or Skype, or whatever was most convenient for the match to take place. The tutors met with tutees, for three hours a week, over six weeks. The topics were either math, Italian, or English, and then at the end, the researchers collected the survey and found similar gains to what we were finding in the online overall. Not only that, but they also collected data on mental health and found improvements in feelings of connection, more positive outlook on life. And what’s also interesting as they seem to show improvements and positive outcomes for the tutors themselves, as well. So it stands the potential for a win-win, and this was all done online. So it’s like the only online study I know, but it seems to show the potential that it might be done there. One other example I should mention is Khan Academy has also initiated another organization that facilitates free volunteer online tutoring. It’s called ‘schoolhouse.world’ and it’s been interesting to watch that trying to get up and running. Their system allows anyone in the world to volunteer their time as a tutor, and then they try to connect anyone in the world wanting to receive that tutoring. And you get some sense of some of the challenges from doing that. How do you screen for quality? And also, how do you screen for safety? So they’ve had to go away from a one to one model to more of a group model. They’ve had to have systems in place to check the quality of the tutoring, what’s being discussed. They’ve had to switch to allowing only high school students to receive the tutoring and a few other challenges. And so there’s challenges but also a lot of potential in this that wasn’t available from always having to meet your tutor in school or after school or face-to-face. So the potential scalability is enormous, and that’s where the intriguing possibilities are with that tool.

Rebecca: So if we’re looking to reduce achievement gaps, we’ve talked a little bit about COVID and the mix of instruction that students might’ve had during COVID, the quality of instruction, access to technology, to even have interactions with teachers in some cases, and historically even, differences in ability when students arrive in higher ed. What are some of the things that the higher ed community might be thinking about in terms of this research? Should we be advocating for certain kinds of policies or programs in K-12? Should we be trying to institute some of these things in higher ed? What are your thoughts on that?

Philip: So just in terms of advocacy and thinking about facilitating more equality, there’s no question that tutoring has, in general, been an unequal program. There’s the whole private sector of tutoring where a lot of households for more affluent families seem to receive it than those from less affluent households. And so one thing we can do as policy-makers is to try to facilitate more tutoring to happen in schools, especially at schools for more disadvantaged backgrounds. We can also focus on providing tutoring to those who need it most. I think that there is a growing awareness of the potential for tutoring to make a real difference in helping address the learning loss that may have occurred with the pandemic and just helping address education inequalities in general. And so a lot of resources have started going towards trying to increase the amount of tutoring happening in schools. I think that the more we understand how to implement it successfully, the more guidance that we can provide the K-12 sector in trying to introduce that. I think that there is a lot of optimism now around its potential. I think tutoring is one of the most effective programs that we can offer to make a meaningful difference at scale, such that we can get more students arriving into post-secondary ready to handle it and succeed well there. So that’s on that end. I think that there’s no reason why we also can’t consider tutoring at the post-secondary level as well, and the potential benefits that might come from that. Even if we just look at first-year calculus, or other subjects in math, computer-assisted learning is well developed even at that level, the need for tutoring at that level is there as well. And so it really does go from that importance of establishing a foundation that one might benefit from tutoring at earlier ages. But even at the post-secondary level, regardless of what level the student is, we can all benefit from one-on-one instruction compared to being in a calculus class of 500, right? I think there has been less research that’s been done in that area, but the evidence certainly points to the direction that tutoring at the post-secondary level would be also effective and important to consider.

John: And you mentioned that Italian experiment where college students were providing tutoring, and you mentioned that that was a very positive experience for the college students as well. That might be an interesting model where college students could improve their own skills and develop a bit more automaticity and more practice in basic concepts, while helping bring students up to a higher level in secondary schools. That’s a program that I think offers a lot of potential.

Philip: So I would agree, absolutely, the expression is you don’t really understand something until you teach it. I think that there’s something to be said for that. I think that there’s also a lot of skills and experience that is gained from trying to help others, from trying to connect with perhaps younger individuals that have not had the same background as you. I think that the experience is also attractive to employers looking at who to hire. I think there’s huge gains from all the things that you might volunteer or use your time for in college, spending some time to volunteer to do something like tutoring could be a very rewarding thing as well. So I’m also excited about that model. I think that there are ways to try to facilitate that kind of model at scale and more research needs to be done to explore how to do that.

Rebecca: One of the things that I heard you mentioned early on in the conversation is the idea that, historically, folks who had access to tutoring are more affluent. So the students who most need the tutoring are the ones that aren’t always getting it, because they can’t afford it. So I love the idea of having it in schools or it’s a part of our programs. But also I think sometimes tutoring has a negative connotation to it. It’s like a deficit model. Especially I’ve seen this in higher ed, students don’t want to go to a tutor because it makes them feel like they’re dumb or something.

Philip: My first reaction to that is that tutoring can be beneficial at any level. For example, in the Khoaching with Khan project that I’m looking at, the potential is to help all students in the class regardless of their level, because every student can be given their own individual roadmap. And that not only includes those that are behind grade level that benefit from establishing a stronger foundation in that earlier material so that they can catch up, it also includes those at a higher level that don’t have to be held back or wait for the instructor to cover new material can use a platform like Khan Academy or a tutor to work on more challenging material that interest them. And so how to remove that stigma that exists in general, I agree the usual perception is when someone asks, “Do you need a tutor?” it’s because you’re struggling. It doesn’t need to be that way, but at the same time, I think the more we become aware of the benefits from the tutoring, the more we realize that it’s a great resource to take advantage of. Getting back at the college level, I don’t know about your own experiences, but it always amazes me how few students take advantage of all the free tutoring that’s being offered by the universities through, like, office hours. The opportunity for receiving one-on-one discussion is often there, and yet so few students seem to take advantage of it, perhaps because of that stigma or perhaps they’re too busy. Some of us, when we went through college, were pleasantly surprised by how much you can get with office hours of graduate students and extra tutoring and how much you can learn from that process.

John: As in a lot of classes, students are treated as if one-size-fits-all education and students come in, especially in subjects such as math where there is a very rigid structure, if you don’t have a solid foundation and concepts, learning new topics is not going to be very productive, because you don’t have that foundation to connect to. And I see that in my own classes, and it’s a bit of a challenge to try to do that. Because of issues of scale I often teach large classes, I try to rely on peer instruction as much as possible with small group activities. Could small group peer interactions in working through problems and problem sets achieve something similar to the one on one attention?

Philip: In the literature, it’s called peer-to-peer, we did not look at peer-to-peer in our meta analysis on tutoring, but there is some literature and there’s some effort to consider that. It’s a little bit of a different model, because you’re relying on slightly older students or similar students to help assist other students. I think more research needs to be done on how to make that happen effectively. On one hand, the potential is there to make this a scalable, effective program that doesn’t cost very much. On the other hand, monitoring quality and the potential to train to be a tutor and to do a good job with it may not be there as much as with the regular type of tutoring program.

John: In particular, I was thinking of activities in class where students work on problems in groups, and they try to argue out solutions. They work together and they can explain to each other things they don’t understand, but the key aspect of that is they get feedback on whether they’re correct or not, some constructive feedback on where they went astray. But I was just thinking that those types of small group interactions could provide some of the benefits without that stigma of needing to go to tutoring and perhaps at a higher scale than tutoring might work.

Philip: The advice that I often give my students is to study until you feel you can explain it to someone else. And so there’s a similar, perhaps, mechanism at play when we’re thinking about that. When you try to write down a concept or explain it, even to yourself, out loud or to someone else, you quickly realize what you understand and what you don’t. There does seem to be a lot of potential there.

Rebecca: Sounds like one of the keys to reducing stigma around all of this is making the coaching or this tutoring model just something that’s normalized. Maybe it’s normalized in class, it’s normalized through the school day, and then people might be more apt to take advantage of it because they have access to it. But also, it becomes a standard way of being, that’s what other people around them are also doing.

Philip: Absolutely! I think if we can reframe tutoring as just individualized instruction or personalized instruction, then we can all understand the potential benefits of receiving more personal help than in a classroom setting, and that goes for pretty much anyone.

Rebecca: It really also matches up well with a lot of universal design for learning principles of flexibility as well, and allowing students to go at their own pace and finding ways of teaching and learning that match well for students and where they’re at.

Philip: And of course, the issue is scale. Getting children to learn in a classroom of 25 to 30 students, when these students vary enormously in academic levels, is just really difficult. And trying to figure out a way to provide that individual attention is the challenge that all teachers face and have been facing for many, many years. And if we can find a way to scale adding on or providing more and more individualized attention, it has the potential, I think, to make a real difference in education. Of all the potential policies that we can be looking at, I do think that, at the school level, leaning towards more individualized instruction is where we should be looking at, for a solution.

Rebecca: It’s so interesting to me that we’re having this conversation early on in our semester, because after teaching online for a year, which I hadn’t done previously, I’ve really worked to make my classes more flexible and actually offer some of those kinds of models that you’re describing where students are going more at their own pace, and that they can get some individualized instruction when they need it and that they need to do this mastery learning so that they build on things over time. It looks to me like maybe I need to look more into tutoring and coaching models that have worked really well to see if I can’t implement some of that more during class time.

Philip: There may be different ways to do it. Some may be more effective than others, but I do think, getting back at what John was saying, it’s harder to provide that individual support or help to students arriving in college without that foundation. I have done some other work at the college level, trying to facilitate more personal attention to students arriving, trying to help them out and encourage them to get into better habits, and it has proved quite difficult to change behavior, and so I have found myself reacting to that by focusing more on earlier grades to see if there might be more promise on trying to foster better study habits, better learning habits, earlier on with the hope that students arrive in college more prepared.

John: I think that’s one of the things a lot of behavioral economic studies have found. Interventions that result in long-term changes of behavior are challenging in general.

Philip: Absolutely.

John: And I think you’ve done some research on that.

Philip: Absolutely. So if we have to change one-time actions, like helping students through applying for college, applying for financial aid, those types of interventions are much more promising at affecting one-time goals than to change habits or routines that involve much more continuous behavior. So helping someone study more effectively, spend more time studying, these are much harder problems to solve. And maybe low-cost nudges that we’ve been looking at in the literature may not be as effective. I think that does tie back into how my perspective has changed over time. It’s hard to have significant influence without personal connection. It’s a lot more expensive, but there’s only so far you can go with sending an email or a text message or a one-time meeting in trying to change someone’s learning trajectory or life trajectory. And the more you sort of look at education policies that have been successful, the more you notice that they often come with this personal connection that’s been important for making that meaningful change.

Rebecca: It seems like we should all be really advocating then for these much more early interventions. It’s much more cost effective if we get those habits in place really early [LAUGHTER].

Philip: I will say there’s surprisingly not enough research on the long-term effects of tutoring. I’ve seen one study that has found that the benefits of receiving that tutoring continued one year past the program ended; the effects faded, but not by that much, and that’s the only study I’m aware of that actually does a long-term study. So on the question of whether we can have these life-changing impacts from targeting earlier ages, certainly, there’s a literature for the very young… like, almost helping at the household, but at the school, I think that more work could be done.

John: And that could be a really productive research area. Before we started recording, we were talking a little bit about, with the pandemic, creating our own videos. Could you talk a little bit about how you try to implement what you’ve learned in your own classes at the college level?

Philip: Yeah, I think that using the situation last year to put my lectures online has freed up space in the actual lectures to be more interactive. So I think it was a benefit both ways. The videos of the lectures themselves became more streamlined, I got a chance to break them up into smaller parts, sort of like Khan Academy videos, where instead of one video that’s two hours long, that goes all over the place, and you’re staring at me and the Blackboard, I created five- to ten-minute videos of vignettes that I could focus on with slides and have a series of these videos that students could watch at their own pace. I could edit them and make sure that the video is as succinct as possible and gets across what I really want to say. So that was good on the video side, and then on the actual lecture side, we spent that time going through problem sets and answering questions and it was much more interactive, closer to the spirit of more personalized instruction. So there was more opportunity for questions, more opportunities for the students to get more involved, and I think it did lead to more satisfaction of that approach. Obviously, the big question is, ‘Do they really watch the videos when they’re asked to do it on their own?’ I think there are ways to try to incentivize that, but just like any class, the students really perk up when they’re working on a problem that was, say, a previous exam question.

John: I’ve used a very similar approach. I’ve used videos for like 20 some years in my classes, but one thing I started doing last year is I embedded questions in the middle of the videos, and that’s a pretty effective incentive structure. It does get them all watching the videos, and at least thinking about it and trying to make some connections while they do it, and that’s worked pretty well.

Philip: Not only that, but you can make them mandatory for class participation. So you stick those questions in and they have to watch the video to find the questions when they pop up, there’s software that can do that. And then you can make it as a way to encourage them to have to watch the video.

John: Do you think that more use of computer-aided instruction is going to be helpful in allowing more students to be successful?

Philip: I’m very optimistic on this potential of leveraging computers with teachers and parents working together on trying to facilitate high-dosage practice. We’ve been talking mostly in math, but it could also be language as well, and maybe other topics. But I think this really is a good way to learn, as long as the practice time is long enough, and the student’s not stuck. I think that it takes a while to get into the habit, getting used to the software, getting used to the routine, both for the teacher providing this and for the student doing it, and so that, for me, right now, is the biggest challenge. I am optimistic that if we can facilitate a way to help teachers and students get to that higher-dose practice using computers, then very good things will happen. I think that the evidence is highly suggestive that the high dosage is a worthwhile thing to get done. I’m hoping that we can generate evidence that that’s the case, but we are finding that there are challenges because there’s a learning curve, it is changing the way that the classroom is done and changing the way the student usually learns, but I’m optimistic that if we can get past that, the students and the teachers will come to like this approach, and that we can do more of it at scale.

John: And I think a lot of people began experimenting with some sort of a flipped approach where they created videos and then use the classroom for more interactive activities, ast least at the college level, I don’t think that’s happened quite as much at the secondary school level. But I think that has helped provide at least some professional development for faculty. But it is an adjustment that students are not adjusting to perhaps as easily as I would like, I know I always have trouble getting across to students that there is some benefit of working through problems in class and watching videos and learning some of the basic concepts outside of class. Students would rather be lectured to, there was that big study that was done at Harvard not too long ago, where students were asked about active learning classes versus lecture classes, and the research certainly showed that active learning in the classroom led to significant learning gains, but students perceived a higher learning gain from lecture classes, and that’s where I think that issue of students’ adjustment is a challenge, and until we get to see a large amount of this occurring, it’s going to be a while convincing students of this, because it’s really easy to sit there in a lecture and nod and smile and have it all make sense and it seems to fit together very logically, but then when you try to apply it, there’s a bit of a problem, and then the questions are somehow unfair. But when students are faced with problems and interactive work in class, they’re confronted by not knowing things as well as perhaps they thought they did, and it’s not as pleasant of an experience. And I think that’s the source of that metacognition, that students perceive that lectures are more effective, because it’s easy to sit there and listen in, and it all seems reasonable. But the problem is when they try to work through problems and realize they don’t quite have those connections fully there yet.

Philip: The lecture seems to make so much sense until you sit down when you get home and try to go over it again, but I do think there’s the potential for this middle ground that even in the experiment we’re looking at, we’re not entirely flipping the class, in fact, we want to work with the teacher to understand what their own preferences are, while still trying to hit this high dosage of practice, which may occur in class, but also could occur at home as well. And I think that there is something to be said by having a lecture of a new topic being done in class, in person, with the real person. It gets back to that importance of personal connection that the computer is not able to provide. And so maybe there is a sweet spot around providing real instruction, real empathy, but also enough time to be working through these problems at your own pace. My vision for the Khan project is that students say, in grade four, getting 90 minutes of math a day, maybe half an hour of that would be the teacher’s own instruction of a new topic, but then a lot of the other time would be students working on their own devices, while the teacher takes the time… instead of just sitting up at their desk… walks around and spends a lot of time looking over the student’s shoulder, using the data that they’re seeing to understand who’s struggling and where, and spends a lot of time working individually while the student is using the computer. So there’s still that interaction going on and taking advantage of the personalization. I think they too can go really well together.

Rebecca: That’s definitely something I’ve been experimenting with. I went all the way flipped before, and right now I think I’m right in the middle. There’s some flipped, there’s some demos that are live so that people can interact and ask questions, and then there’s lots of practice with individualized attention. And it does take a little time to get everyone on board, to get everyone trained to do things in a new way. So in a 15-week semester, it might take two full weeks to develop new habits and workflows for everyone, but really after we get over that two- week hurdle at the beginning of the semester, my classes tend to settle into a routine that seems really productive and that students have been pretty positive about.

Philip: A key feature of the coaching with Khan program, is that every teacher gets their own coach that we spell with a “kh,” and our coaches meet with the teacher prior to school to go over our suggested recipe to follow, but then they don’t just leave it at that, they keep working with the teachers to check in and try to troubleshoot or brainstorm or reassure and remind the teacher until things are going smoothly. But it can take longer than two weeks to figure out how things are going, and then on the student side, it can take a while for them to adapt and understand that there’s some independence on their own for wanting to do it. The hope is that the students start to gain confidence when they see their own progress, when they see that maybe they didn’t consider themselves a strong math student, but if you start them at the right spot on this roadmap, and then they proceed incrementally, and they can see that they are advancing, then they start to understand the potential benefits and internalize the desire to keep going on their own.

Rebecca: Yeah, that autonomy and that empowerment, I think, is really key to the whole puzzle. And I think something that probably tutoring historically helps students achieve is that they can do this. They might have a little extra guidance initially, but then they achieve it and can do it, and that’s really empowering.

John: That’s our hope

Rebecca: We always wrap up by asking: “What’s next?”

Philip: What’s next? I think I made some notes on that. [LAUGHTER] So I think the issue around tutoring and individualized learning is all about, now, scale. I don’t think we need another study to demonstrate that one-on-one instruction, or one-on-two is an effective additional tool for learning, that more should be done if it were possible. A lot of resources are now going into trying to provide individualized instruction. I think a lot of policymakers and governments are looking to tutoring as a way to address some of the learning loss that may have gone on during the pandemic, and I think, in that space, there’s some optimism by researchers and policymakers to try to understand what types of scale up are better than others in a way that we can make a meaningful difference at the aggregate level.

Rebecca: Well, thanks so much. I’m really excited to hear more as your research develops and more information becomes available!

Philip: It was a pleasure to get a chance to chat with you guys. It’s a topic I’ve been spending a lot of time on and losing a bit of sleep on trying to get things to work. The experiment that we have going on, this is going on in Texas, and one of the challenges of doing a field experiment is that so many things go wrong while you’re trying to deal with real people, real students, and provide evidence that this is a good idea. And it’s always a bit frustrating to face these challenges, like just account issues, students have trouble getting on to Khan Academy and the teachers getting frustrated, and it would be a shame to have those issues that can be worked out actually create this wedge from the program going smoothly and making the difference between having these great impacts or not. So it is stressful, but I think it’s worth it to try to keep at it, and I hope to be able to do so. With funding and policy support we’ll just keep trying. I think there’s a lot of interest in it, I think that it hasn’t been difficult to motivate these ideas and wanting to do more on it. So thanks a lot for giving me the chance to share these thoughts.

John: Your work is incredibly important. And so much income inequality is associated with differences in educational attainment, that understanding these achievement gaps and what we can do to narrow them can have a really dramatic impact on society.

Philip: Fingers crossed!

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle.

[MUSIC]