In episode 12 of this podcast, Doug McKee joined us to discuss the Active Learning Initiative at Cornell. In this episode, Doug returns to give us an update on this initiative and some initial findings on how this initiative has affected student learning. Doug is a Senior Lecturer in the Department of Economics and an Active Learning Initiative Project Lead at Cornell University.
Shownotes
- Doug McKee (2018). The Active Learning Initiative at Cornell. Episode 12. Tea for Teaching podcast. January 17.
- University-wide Active Learning Initiative – Cornell Provost
- Carl Wieman Science Education Initiative.
- Teach Better podcast
- Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410-8415.
- Miro
- Jamboard
Transcript
John: In episode 12 of this podcast, we discussed the Active Learning Initiative at Cornell. In this episode, we get an update on this initiative and some initial findings on how this initiative has affected student learning.
[MUSIC]
John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…
John: …and Rebecca Mushtare, a graphic designer.
Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.
[MUSIC]
Rebecca: Our guest today is Doug McKee. Doug is a Senior Lecturer in the Department of Economics and an Active Learning Initiative project lead at Cornell University. Welcome back, Doug.
Doug: Thank you, Rebecca. So glad to be here.
John: We’re happy to have you back. It’s been a while. I was looking back to see when our last episode was and it was in 2018. It seemed like it was just yesterday in the before times, or at least in that sense of how time seems to have been compressed the last year or so.
Doug: It also seems like it’s been like 20 years.
John: It does.
Rebecca: Yeah, at least those couple decades for sure.
Doug: But I’m glad to be back.
John: Our teas today are… the last time we asked you I think you said “I hate tea.”
Doug: I did. I remember that.
Rebecca: What kind of comment do you have about tea today?
Doug: Well, I can’t say I’ve acquired a taste for tea. I don’t mind Lipton tea out of a can. But I’m not drinking that today. Today, it’s ice water.
Rebecca: That’s a good healthy choice.
Doug: Yeah.
Rebecca: And the basis of tea.
Doug: Right. I’ve been listening to this other podcast. And they always start with “What are you drinking?” And it’s four hosts. And three of them are always talking about what kind of craft beer or liquor they’re drinking. And it’s the healthy one that drinks the tea.
Rebecca: See, this is a very healthy episode,
Doug: Right. It is.
Rebecca: I’ve got my English afternoon today, John.
John: And I have ginger peach green tea.
Doug: Ooh, nice. It sounds nice. Are you really drinking hot tea in the summer? Is that really a thing?
John: I do. All day I’ve been drinking iced tea, though. And so I switched over for the podcast, just because it’s what we do.
Doug: Got it.
John: When you will last on the podcast, we talked about the Active Learning Initiative at Cornell. And we thought this would be a good time to get an update on how things are going. And for those people who are not listening to our podcast back in 2018, could you just provide some information on that project and how it came about?
Doug: Sure, I think even the people that were listening back in 2018 might need a refresher. The Active Learning Initiative is a university wide project at Cornell, it was inspired by a similar project that Carl Wieman started at University of Colorado, and then brought to the University of British Columbia called the Science Education Initiative. And the idea was that if you want to improve teaching, instead of trying and getting individuals to improve their teaching, and instead of trying to do it at the university level, the right level was the department. And so departments would write proposals and compete for money, and they would promise to incorporate active learning methods, different evidence-based methods to improve courses. And they would use the money to hire outside people, usually postdocs in that discipline that really knew the material, but also knew about modern pedagogy to do the heavy lifting. My experience is when you tell faculty to make changes and use active learning, you get three kinds of pushback. You get, “I don’t know how to do it.” Well, this postdoc is the person that knows how to do it, and they can help with that. You get, “I don’t have time.” Well the postdoc is going to do the heavy lifting. And you get “It won’t work in my classroom. It only works in other people’s classrooms.” And a big part of the Active Learning Initiative, certainly as we’ve implemented it in the economics department is concurrent evaluation. So at the end of the process, you really have hard measures of the impact of the program. Now the program started, I think, in 2017, at Cornell with three departments, then in 19, they added six departments, and that was when I joined Cornell, and so one of those was economics. And then more recently, they added nine more departments. And they’re getting proposals together for phase four right now. Now, in economics, we said we’re going to transform our eight core classes, the details are always a little messier than the theory. But we’ve transformed six classes so far, and we just started working on the last two. The big stated goal is to change how we teach these big classes. And seven of these eight classes are required for the major. So we’re really completely changing the experience that students have going through the program. The second goal, which is not as explicit, but really important, was to train faculty. And we’ve worked with eight different instructors during the process so far. And over the next two years, we’re going to add five more. The two classes that we still have to work on have three different people teaching one and two teaching the other one. And that’s been, I think, a pretty good success. So the faculty we worked with have been pretty happy and learned a lot from the process and continue to use those methods. The third goal, that was fairly unstated, is changing the culture. So what we want is a much more teaching focus and quality of teaching focus in the departments. And I think we’ve been doing a pretty good job on that and instilling this kind of continuous improvement idea. And then the fourth major impact that we’ve had, I think, was not something we really saw coming. And that is that, in the process of doing all this work, we’ve developed these assessments of learning and gathered all this data about what’s happening in the classroom and who these students are that are coming in. We’ve collected a huge amount of data. At the same time, we’ve spent a lot of time reading the literature and learning all about how much we don’t know. And it’s managed to enable all this research. And so I was talking to Rebecca, before we started recording, saying that a big part of my job is now actually like doing research using this data. And it’s been super exciting and fulfilling. And John, you asked me earlier, again, before we started recording about the Teach Better podcast, which is a podcast that Edward O’Neill and I hosted for a long time, and we haven’t had any new episodes posted for a couple of years. And it’s not because the band is broken up. But we’ve just been busy with other things. And I would say that the time I would spend writing blog posts and recording podcasts is now writing papers, and presenting research at conferences. So that’s been pretty exciting.
Rebecca: One of the things that you mentioned, right at the start of talking about this program, are the different push backs we get from faculty, and so you addressed two in how the postdoc helps with the two. The last was, “I can’t do this in my class, you can do it in other classes.”
Doug: You show them research from physics, and they say “This is economics.” You show them research from economics, and they say, “That’s from a very different kind of an institution.” You show them research from economics in a very similar institution, and they say, “Well, I teach my class very differently.” And that’s hard.
John: There’s always some explanation that people can come up on why their class is somehow different than everyone else’s. What have you done to try to reach some of those faculty to help keep the initiative spreading?
Doug: I have the answer that I find really compelling that never work. And that is, there’s this really wonderful meta analysis by Freeman and a whole bunch of other co-authors about the impact of active learning in STEM. They go and they collect lots of really serious research, high- quality empirical methods, that shows our big learning gains from transitioning from a pure lecture environment, to an environment where students are active in the classroom. And they find that there are large, statistically significant effects in eight different disciplines. And when they test the hypothesis that the effect is exactly the same in all eight disciplines, they can’t reject the null hypothesis. In other words, the size of the effect in physics and chemistry and biology and math and engineering… they’re all pretty much the same. And so if they’re the same in all those disciplines, shouldn’t they be the same in economics, too, because surely economics is closer to some of these than physics is to biology. And I find that really compelling, but no one else does.
Rebecca: Alright, so what does work then?
Doug: Well, what does work is being really nice and saying, “Can we just try it? And surely there are things that you’ve wanted to change in your class for a while and haven’t had the resources. And my job is to help you improve your class. And I’ve got some ideas too, and I’m going to lay them out, and we’re going to try.” And they’re like, “Okay.” I mean, so much of it is just plain old social skills. And like, “trust me” and building that level of trust. And they go and they talk to their colleagues that have worked with me in the Active Learning Initiative. And “Yeah, he’s not so bad. Like “he seems at first like an arrogant jerk, [LAUGHTER] but he’s okay, he’s pretty reasonable to work with.”
John: And the chance to make some of those changes they might have been considering, with some support to help move that along, could be a useful incentive, too.
Rebecca: You never know when that option is going to come along again, right? [LAUGHTER]
Doug: Well, that’s true too. So if Carl Wieman was here (he’s kind of the godfather of teaching in physics), what he would say is “Don’t even try to convince them that these methods work better for student learning, because a lot of the faculty, they say they care, but they don’t actually care that much. Tell them it’s fun, and that it’s much more fulfilling to have students doing things and interacting with them during the class than just standing up there and giving the same old lecture every year.” And he says that’s really effective. I have not had great success with that. [LAUGHTER]
John: One thing that actually worked better than anything we’ve ever tried here in the teaching center was this pandemic.
Doug: Right. That’s totally true.
John: People learned they needed to try doing things in different ways and we gave them lots of workshops and talked about the importance of active learning and gave them lots of things that work in a variety of modalities that they would never have considered otherwise because they were comfortable with what they were doing. But when people didn’t have a choice in terms of restructuring their classes in some way, they were much more open to trying evidence-based approaches than we’ve ever seen before and we had many more people attend a workshop during a pandemic than we ever did and many of them have said, “I’m never going back to the way I was teaching this class before.” Did some of that happen at Cornell?
Doug: It for sure happened. And I’ve talked to colleagues all over the world that have said exactly the same thing. That is absolutely 100% true. Just the other day I was talking with an instructor, we’re going to be doing a bunch of things with them to their class, and the conversation started with, “I tried these three new things in my online class, and I really want to bring them to my in-person class.” But you two, in the teaching center, interact with lots of faculty every day, for a brief period of time. But with me, I interact with a very small number of faculty for a very long time. We did not suspend the Active Learning Initiative for the last year, but we were originally a five-year project. And we decided to stretch it to six years and go into like a lower energy mode, while we focused on staying above water
Rebecca: …as was necessary for just about everybody.
Doug: Yeah.
Rebecca: You have all these new departments joining the initiative. Do these departments interact and support one another? Are these like completely separate pockets?
Doug: Yeah, so that’s interesting. The postdocs for all the departments meet regularly and do training together. There’s not a huge amount of interaction between the instructors across departments, or even the instructors that run the department initiatives. That’s something I think fell by the wayside during the pandemic. But we’ll probably pick up again, as things hopefully normalize, because there’s huge benefits from it. We’ve certainly run into lots of things that these newer departments are running into now and they can benefit from our experience. And then they come in with really fresh ideas that we haven’t thought of.
Rebecca: The other thing you mentioned, too, is all this data you’ve collected. Can you talk a little bit about what that data has started to tell you?
Doug: Yeah, so the process that we follow when we transform a course starts with documenting the learning goals for the course. Learning goals are not topic. How we define learning goals… because I understand in the ed world, people get very militant about their definitions and differentiations between learning goals and learning objectives and other kinds of things. But what we call learning goals are things that we want students to be able to do by the end of the semester, and we can break it down such that it’s probably two to three pages long. And then we develop an assessment of those learning goals. We also write down, in the same way, these are the skills we expect students to have going into the class. Now when we have a sequence, the skills that they have at the end of intro are prerequisite skills for the intermediate level course a lot of the time. And so we’ve developed, I think, 10 assessments so far. We have two math assessments, one at the intro level, and one at the intermediate level, because we add calculus. We have our intro micro assessment, we have stats, we have a whole bunch. So we have test scores on these things at the beginning of the semester, and at the end of the semester. So we have all these assessments. At the same time, we ask students about attitudes at the beginning of the semester and at the end of the semester. So do you think about economics every day? Do you feel like you belong in this economics course? We have like four pretty standard questions that we asked at the beginning and the end of the semester. Often when we give exams, we have exam wrappers. So how much did you study for this exam? How much of the material have you seen before? That sort of thing. And then at the end of the semester, we have our standard course evaluations, then we also ask students there: Did you feel like you were part of a community? Do you feel comfortable asking questions? Do you like the other students in the class? …those sorts of things. And so with the assessments at the end of the semester, we can tell how the distribution of learning outcomes changes. And we always have a control semester where we don’t do anything we just measure. And then we make all the changes after that, in a different semester, so we can compare. And what we found is that it often takes more than one semester for you to see learning gains. So you do all this stuff and you look at the results. And sometimes there aren’t very strong results. But you note what worked and what didn’t work. And you can look at outcomes by learning goal. So you can say like, “Oh, this learning goal we did a lot better on but not this one.” And so you can refine and then that refinement process, we’ve been able to actually see improvements in learning. We can use those same tests to see if there are gaps between kind of mainstream students and students from underrepresented groups. So we compare outcomes of males and females, we compare outcomes by race. And what we found is those gaps declined as we incorporate more active learning methods. We found that… this is a kind of a big one that I did not see coming when we started this project… so, we knew clickers were going to be a big part of this and like having activities that students work on in class, that’s like the bread and butter, and we have all different kinds of activities we do and I could talk for hours about that, but the thing didn’t see coming was how useful assigning students to small groups at the beginning of the term and having them work together in those groups inside and outside the classroom. And it actually was even more helpful during the pandemic with the online classes. So that’s been pretty cool. And just that treatment has radically increased the sense of community among our students. So we’ve been really happy about that. So those, I think, are the big results.
John: So, have all those core classes switched to the use of persistent groups.
Doug: Yes, but the extent to which the groups are used differs across courses. In my courses, it’s incredibly intensive. I have them sit in those groups in the classroom, I encourage them to work together on problem sets outside class, we do two-stage exams where they take the exam individually and then they take it again, in their same small groups. And they have a semester long project that they work on in those same groups. Probably the least intensive, we have is our big intro micro class, students are assigned to groups, they work together in those groups in discussion section, but not in the lecture. Because with 200 students, and a 350-seat lecture hall, it’s not too bad. But if you’ve got 440 students in a 450 seat lecture hall, it’s a little harder to manage that.
John: Yes, it is. I often teach classes that size. And I’ve generally just done ad hoc groups rather than persistent groups, but it is something I’ve considered switching to.
Doug: Well, in the pandemic, I had lots of colleagues that tried the ad hoc groups, ‘cause in Zoom, you can just press a button, and boom, everybody’s in a randomly assigned group. And you don’t build any social capital with other students in the class, you’re like suddenly in a room, and everyone’s got their camera off. But if you know everybody in your group, then you turn your cameras on, and you’re like “Yay, another person, Hello.” It’s pretty different. And in the classroom, I think there’s a similar element. There’s this other piece, which is in the classroom, people tend to sit in the same place. There’s a lot of inertia there. But I also think the random assignment gets people talking to people they wouldn’t otherwise talk to. And it assures that nobody’s kind of left out. So I think that works pretty well.
Rebecca: That was one of the things that I wanted to follow up on with the persistent groups is that sometimes you have the one student who feels left out or doesn’t feel included or doesn’t participate or doesn’t contribute. And so how do you handle that if that same team is doing everything together from tests to projects.
Doug: So in my classes, when they do everything together, in the two-stage exams, there’s a pretty big payoff to getting everybody’s input. And so I’ve never seen students as engaged in a classroom as they are, when we do these two-stage exams, it’s about as high stakes as you can get. On the projects, there’s a peer rating component, where at three points in the semester, they have to rate their teammates and say, on a one to three scale, are they contributing a lot? Are they not contributing very much? Are they contributing nothing? …and they rate them on effort level and dependability. And then the first one is just informative. You learn what the average rating you got from your team is, and you can do something about it. And then the next two, it’s actually a pretty big part of your grade. And what happens is, the students either improve, they get the message that they have to contribute, which is great, or they don’t, but then the other students are like, “Well, at least they’re getting what they deserve.” And so they’re pretty happy, because so much of it is about fairness and justice, that they’re like I’m willing to do the work, I just think it’s not fair if they get the same grade as me because they didn’t do the work. I mean, every once in a while you get a group that just doesn’t work very well. And these are always different. Like, it’ll be like, you’ll just somehow have like the three guys that happen to know each other. And they’re like, let’s all just give each other great ratings and not do any work. And then there’s the fourth person who ends up doing all the work. And then you talk to the person, you’re like, “Look, don’t worry, I’ll take that into account when I make your grade for the class.” Now, we actually had an instructor this semester, who had a really creative solution to this, which is at the halfway point in the semester, they surveyed the class and allowed people to bail out of their group and go into a new pool and get reassigned to a new group. And I think like 20% of the class decided to do this. And it was pretty interesting. I haven’t looked closely to see if our community measures were higher in that class. But I’m intrigued. I might try that in my class.
John: I had something similar in my econometrics class this spring, I tried using persistent group, some were working really productively, and others just were not working. And I was getting some pretty regular complaints from people. I talked to them and said, “Well, often you’ll be working at some point in your life with people where you need to forge a good productive relationship,” but eventually it got to the point where I said “Okay, anyone who wants to switch out of their group is welcome to and we’ll just re-form some new groups after this and anyone who wants to stay could.” And the shifting helped the people who were really productive and were frustrated that some of their colleagues were not quite as productive were really happy to start working with other people who were in a similar position, and I think there was some really positive complementarities from those people. The remaining people in those groups may have sunk down a little bit in terms of their performance in the class, but they had the option of becoming a bit more active and working harder, and they just chose not to take it.
Doug: And some students will do that. And look, as important as I believe econometrics is, some students, they’re like, you know, I have other ideas about what’s important in college, and maybe it’s different classes. Maybe it’s the social aspect, and like, “Who am I to judge?” But, don’t expect an “A” if the social part of college is more important to you.
Rebecca: There’s also always the group too that when you start talking to all of them, they all think they’re doing something different. It’s really hard to get a read on who’s doing what, and this person thinks this person is not doing anything, and this other person thinks they’re not. Those conversations happened together, but a little intervention and clarity and communication can go a long way. And I’ve had some really dysfunctional teams become functional, just with a little intervention early on, when you realize that they’re all talking past each other and are all quite unhappy.
Doug: Do you have them identify their role and write down like, when they submit a group project, say who did what? And would you recommend it? Because I don’t do that. I’ve always wondered if I should do that.
Rebecca: I’ve gone back and forth. Sometimes I have projects where they have defined their roles. I do it in sprints, or break it into pieces. And so they might switch roles in each round or whatever.
Doug: Which they should.
Rebecca: Yeah, I do also, I tend to ask a question on the team evaluations, which I do multiple times throughout the semester that ask who contributed the most to the group? And why or what did you learn from each colleague, and those tend to be really informative, and reflection papers.
Doug: One more thing I want to add to this part of the discussion is, this is really important, but we always make sure not to isolate women or underrepresented minorities in groups, because when you do that, they, in general have a pretty bad experience. And we’re not going for that /we’re going for good.
John: In Episode 182, we had Olga Stoddard on the podcast, who talked about a study where she and some colleagues had done in an MBA program where they had persistent groups. And the groups that had only a single female ended up evaluating the contributions of a female participant much lower. And in fact, they rated themselves lower as well. And there was some really substantial gains in their perceived participation, when they represented a larger share of the group, when it was a majority female group. They’ll be following it up with some research on individuals from other underrepresented groups.
Doug: So when I first started doing these peer ratings, I had three categories. So in addition to effort and dependability, I had collegiality, because I was trying to capture like, is this person… they’re putting in effort to show up on time… but they’re just a jerk, and like you should get penalized for that. And what I found is that the male students all gave the female students low collegiality ratings, and the female students all gave the male students low collegiality ratings, so it was just like incorporating sexism into the grade, and I didn’t want to do that. So, I just removed it altogether. It didn’t help.
John: One of the things that I’ve been doing based on, actually something our Dean suggested when she gave a workshop on group activities, was having groups put together some type of a group agreement on how they would resolve conflicts and so forth at the very start of the semester. And that seems to have really significantly reduced many of these issues, and providing some time during the semester for them to talk over any group processing issues they may be having. So that if they are facing challenges with people contributing, they have a chance to work that out amongst themselves and just talk about that a little bit before the problem becomes more ingrained. And it seems to have been helpful.
Doug: That’s super interesting, I do something not as good, which is at the beginning of the semester, I do this exercise in class where they have to write down what they think are like the five behaviors that are consistent with a well functioning group. And then I’ve got mine. And then we do the big word cloud. And we talk about it like listening to what everyone in the group has to say, being nice, giving people the benefit of the doubt, things like that. I guess it’s more proactive rather than reactive. But I think there’s probably a role for both.
Rebecca: I’ve mentioned before in the podcast, too, that I’ve been doing an activity called a retrospective at regular intervals throughout the semester, which is like a group reflection. I’ve been using virtual whiteboards and sticky notes, to do it with some basic questions like “What should the group keep doing? What should the group stop doing?”
Doug: Oh, interesting.
Rebecca: “And what should the group start to do?” And that was, I found, really productive and the amount of team issues that I’ve had since I started doing that has been greatly reduced.
Doug: I’m writing this down, sot I can do it too. I like that idea a lot.
Rebecca: It’s worked surprisingly well. I’ve been using Miro, which is a virtual whiteboard. And they actually have a number of templates for different retrospectives that you could use as a starting point.
Doug: You like that better than Jamboard?
Rebecca: Yeah, it has way more features but I also teach design, so we need fancier things, and it needs to look nice and you can make it nice in there. [LAUGHTER]
Doug: I used Jamboard last semester and it was great. The breakout rooms actually were able to communicate a lot better with each other when they could all write on a Jamboard.
Rebecca: Yeah, my use of whiteboards virtually has greatly improved collaboration.
Doug: Right. I’m not sure what to do in the classroom now.
Rebecca: I know. [LAUGHTER]
John: I used Jamboard in all my classes and I was thinking, should I ask students to do this while they’re in a big classroom? And I’m kind of thinking, yes.
Rebecca: Yes.
Doug: Maybe, yeah, maybe… because the alternative is like one person writing on a piece of paper, and everyone else leaning over their shoulder. And it’s not as good.
Rebecca: Oor wasting a lot of paper by using a lot of sticky notes.
Doug: Right.
Rebecca: Because you can do the physical version of it, but…
Doug: But it’s got to be big enough. In a class where you’ve got 200 students, you don’t really have enough whiteboard.
John: Could you tell us a little bit more about the impact of these changes on student learning.
Doug: We see improvements in student learning, but it often takes more than one semester of refinement to get there. And I think that’s a really important lesson: that you have to be patient when you make these changes.
Rebecca: Hey, we’re new at something and you have to have practice. Is that what you’re saying, Doug?
Doug: Right, but how many times have you run into someone who said, “I tried clickers and it didn’t work in my class?” Like, of course, it didn’t work. When you got on the bike, did it work the first time you tried to ride it? No, of course not. We’ve managed to reduce achievement gaps in some classes and we’ve really increased the sense of community among people. I think, actually, those two are pretty strongly related. We’ve moved from a real laser focus on increasing average learning to a wider range of outcomes: more attitudes, more gaps, we spent a lot of time thinking about how to lift the bottom of the distribution, like, “Who are these people that are scoring in the bottom quarter of the class? And why?” So yeah, I would say those are the main results we’ve had.
John: And by building an environment which creates more of a sense of community, it’s more likely I think that persistence towards graduation and degree will increase. Have you looked at any data on that yet, in terms of student persistence as a result of some of these changes?
Doug: Well, at Cornell, most people end up getting their degree. So that’s a way in which I think like an Ivy League college is pretty different from a lot of other colleges. Does active learning work in the classroom? Yes, and that’s true no matter where you are. But issues of retention, we actually spend a lot of time thinking about retention. But when we say retention, what we’re talking about is how much do you remember of what you learned in the class after the class is over? How much do you retain in terms of the knowledge, and we’ve actually been analyzing a fair amount of data and collecting a lot of data on this. And in some cases, students forget a lot. In other cases, there are for sure cases where they retain a lot. And so we do a lot of following up students after the course is over, and then giving them the test, the same end of semester assessment again, and we’re in the midst of an NSF funded research project right now, to see how much active learning matters. So we’re following up with students in our kind of baseline courses, like one, two, and three years later, and following up with the students who’ve taken the same course but with active learning, and we’re seeing if the retention is different, I’m really excited about that. What we’ve seen so far, with different work, is that if you apply what you’ve learned in another course, or in an internship, your retention is far higher. So that’s a really great result, and we’re really happy about that and it’s intuitive. We have another result, which is that if you take a class where you learn a different set of methods for answering the same questions. So we see this, if you take a stats class, and then you take a machine learning class, where you learn a different set of methods for doing similar kinds of things, you actually perform much worse. Those methods that you learned, they just get substituted out of your brain and then new methods are plugged in. And it’s not good. There’s evidence for this in psychology, and even a name for it. It’s called adaptive forgetting. And I think there’s like real, pretty serious implications here for our teaching, that we need to do a better job kind of connecting these things together, so that this doesn’t happen. And then the third thing that we’re looking at is, are there systematic differences by gender and by race in retention. And the data is too preliminary now, but it looks like there might be. And that’s pretty scary and we don’t know why. And we really need to get to the bottom of that. But the first step is to identify the problem. And then the next step is to kind of hypothesize ways to fix the problem. I think we were identifying a pretty serious problem there.
John: And you do have some control measures there in the more traditional classes that you can compare that to to see what types of interventions may be best in reducing some of those gaps.
Doug: Yeah, that’s exactly right. That’s exactly right.
John: We always end with a question, what’s next? And it sounds like you’ve got lots of plans for that.
Doug: So I feel like the Active Learning Initiative is chugging along and we have a process and we’re applying it and we’ve saved our macroeconomics courses for the end because there’s less consensus about what macro is then the other courses, and so that’s fine, but I’m not a macroeconomist, but it’s important and it needs to be done. So I’m pretty excited about collecting and analyzing this retention data that I was just talking about. I’ll give one more project that I’m excited about that we have going right now. We came up with this math assessment and we learned that the math assessment, if you give it at the beginning of the semester, in both our intro classes and our intermediate classes, it’s strongly predictive of how well the students do in the class. And so in particular, if you do badly on the math assessment, you’re much more likely to perform poorly in the class itself. And so some students, they just come in, and they’re behind on day one. And this semester, we are working jointly with colleagues at George Washington University and at the Copenhagen Business School. And in all three sites, we are trying something different to try to help these students, we’re all going to give the same math assessment at the beginning of the term. And we’re going to try different things to actually help these students succeed. And it’s going to be a little bit of a let… well, I was gonna say, let a 1000 flowers bloom, but it’s really like three flowers bloom… We’re gonna hope we can actually find a solution to a problem we identified. So that’s what’s next. That’s what’s on the agenda for the fall.
Rebecca: Maybe you’ll even find more than one solution.
Doug: That would be great. They’re pretty different in terms of how intensive they are. They go from: here are some online resources that you can work with, all the way to, why don’t you hold off on taking your intro class and take this full semester math for economics course, and then take intro, and then the Cornell treatment is actually an intermediate level treatment, the expensive treatment should work. But we’re gonna formally show that, but these other two… we don’t know. And so we’re looking forward to finding out. And we’re looking forward to helping a whole ton of students that would otherwise have a really bad experience in an economics course, and then never take another one.
John: And that was actually one of the things I was thinking about in terms of retention, that the retention isn’t only at the institution, it’s also persistence to that degree in that major, and we lose a lot of people in economics, and a lot of it is related to math ability.
Doug: Yeah, that’s right. I think it’s obscured by the fact that we get a lot of students that fall out of the hard sciences. So it seems like we’re getting a lot of students that are continuing, but they’re not really. we’re losing a whole bunch of we’re gaining some.
John: Thanks, Doug. It’s great talking to you. And I’m looking forward to seeing you at some economic conferences as they start to pick up more in person again.
Doug: I know, wouldn’t that be nice? I’m looking forward to it. The virtual conferences have been really good, but they for sure missed out on the social part.
Rebecca: And the travel part.
Doug: I don’t miss the travel that much.
John: I’ve been able to attend many more conferences, but it just doesn’t feel the same.
Doug: No.
Rebecca: There is something to be said about moving yourself to a different space, so that you can focus on what’s at hand, whether or not the travel and all the logistics associated with that are pleasant or not. There’s something about just moving yourself from the normal everyday to some other way that can be helpful in focusing.
Doug: …where the cats and the dogs and the kids can’t walk in on your session.
Rebecca: Indeed. Well, thanks so much, Doug. It’s always great to hear how things are going. And I hope we’ll get a future update as the project continues to move along.
Doug: Oh, yeah, I would love to.
John: Well, thank you. And I hope we do see a return of the band of the Teach Better podcast once things settle down.
Doug: Well, the crazy thing is, you always hear about these bands and their last tracks? And I’ve never really understood like, why didn’t they just put it on the album? What is this with the lost tracks that get released much later? We have two episodes that have never been published, and they’re really good.
John: That’s the easy way to bring it back.
Doug: So someday, we’ll release like the… [LAUGHTER] like the hidden tapes.
John: …the basement tapes. This been super fun you guys. It’s so nice to catch up.
[MUSIC]
John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.
Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.
[MUSIC]