182. Gender and Groups

When we sort students into cooperative learning groups, we often attempt to create balanced groups that reflect the diversity of the students in our classes. In this episode Olga Stoddard joins us to discuss her recent research that suggests that this approach can be harmful for female students in classes in which a majority of the students are male.

Olga is an Assistant Professor of Economics at Brigham Young University, a Research Fellow at IZA (the Institute of Labor Economics), and the Research Director at the Science of Diversity and Inclusion Initiative, and the Co-Director of the Gender and Civic Engagement lab at BYU.

Show Notes

  • Stoddard, Olga B.; Karpowitz, Christopher F.; Preece, Jessica (2020) Strength in Numbers: Field Experiment in Gender, Influence, and Group Dynamics, IZA Discussion Papers, No. 13741, Institute of Labor Economics (IZA), Bonn
  • Zölitz, Ulf and Jan Feld (2018), “The effect of peer gender on major choice.” University of Zurich, Department of Economics, Working Paper.
  • Sandberg, S. (2013). Lean in: Women, work, and the will to lead. Random House.

Transcript

John: When we sort students into cooperative learning groups, we often attempt to create balanced groups that reflect the diversity of the students in our classes. In this episode we discuss recent research that suggests that this approach can be harmful for female students in classes in which a majority of the students are male.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Olga Stoddard. Olga is an Assistant Professor of Economics at Brigham Young University, a Research Fellow at IZA (the Institute of Labor Economics), and the Research Director at the Science of Diversity and Inclusion Initiative, and the Co-Director of the Gender and Civic Engagement lab at BYU. Welcome, Olga.

Olga: Thanks so much for having me. It’s great to be here.

John: …really pleased to have you.

Rebecca: Today’s teas are:

Olga: I was going to be prepared and I have my mug, but unfortunately, it’s only filled with water because I ran out of time to heat it. [LAUGHTER] So, water for me today.

John: Tea is mostly water. We’re recording this in mid February when there’s a bit of a nationwide snow covering. And I’m drinking spring cherry green tea to set a better mood for the future.

Rebecca: I think that seems like a good plan. And for a change, I’m drinking Chai.

John: Wow. Okay, I don’t think I’ve seen you drink that on here before.

Rebecca: It’s not a common one for me. But it’s nice to mix it up occasionally. Of course my Chai doesn’t have dairy in it. So it’s just the tea part of the Chai

Olga: Is it flavored Rebecca?

Rebecca: Yeah, it’s nicely spiced.

Olga: Nice.

John: We do normally in our office have a variety of flavored Chai teas, but they’re safely locked up in our building. We haven’t visited in a long while. We’ve invited you here today to discuss your research with Chris Karpowitz and Jessica Preece concerning how the gender composition of teams affects women’s participation and role in team activities. Could you tell us a little bit about the study?

Olga: Yeah, absolutely. So this study was a collaboration with a top 10 accounting program in the US. We partnered with them to randomly assign different gender compositions of teams in this program. So, like many programs in the US, especially business programs, like MBA programs, this particular program relies on a pedagogical group-based approach in which students are assigned into teams, in this case, teams of five. And they work together quite intensively throughout the semester. So throughout the four months that’s their first semester in the program, they work on assignments together, they meet socially outside of the classroom, they even do some of the exams as a group. And so there’s a lot of interaction between those students within those seats. for that period of four months. Normally, because this program has a really small percentage of women, so about 25% of the students in the program are women, the way that these groups had been formed in the past is to assign one woman per group, so as to sort of dilute the women, to have men have experience in an academic and professional setting interacting with women. There is some prior research in the laboratory that has shown that this really is detrimental for women’s ability to be influential, for their willingness to participate, to be engaged. And so what we wanted to do is we wanted to test whether that laboratory evidence plays out in a similar way in the, so to speak, real world setting, more naturally occurring kind of environment. So we partnered with the program and randomly assigned some women to be in the condition where they were the only woman in the group. So the status quo, this is how things have been done. One woman and four men in a five person group, and then other women were randomly assigned to be in a condition where they were in the majority. So there were three women and two men in a five person group. We then tracked these students for the following two years. We had them complete monthly surveys and peer evaluations of their group members. We had them come into the laboratory twice a semester, where we had them work on a team-building exercise, and we watched who’s participating. These exercises were recorded, so we could see who’s speaking, who is interrupting, with speaking for how long. so that we could precisely measure women’s participation, but also measure their level of influence. Because on these tasks, the way that we designed them, women could exert more or less influence depending on certain decisions they make. So we had different ways to measure their level of influence, their participation, and whether others perceive them to be influential, and sort of more like leaders in the group. And so we did that for the following two years. We had two cohorts of students participate in the study. And what we found is that women who were randomly assigned to be the lone woman in the group were perceived to be significantly less influential, and were actually exerting a lot less influence in the group than the equally qualified women who had been assigned to be in the majority in their group. And so we saw really striking differences across those two conditions. Again, these are equally qualified, very well prepared academically women. This program is very competitive. They have prior leadership experience. And yet we find these huge differences across the two conditions in our case, depending on whether women were in the minority or the majority, they were seen significantly less influential by their peers.

Rebecca: Was the perception of the women in the groups different from the start of the study or the beginning of the group formation versus the end of the group formation? Or was it kind of consistent?

Olga: Yeah, that’s a good question. So one advantage of our study is that we can track these students over a relatively long period of time. Most laboratory studies up to date have relied on sort of these one-shot types of interactions, where strangers meet for a period of an hour or so and never interact again. One thing we wanted to know is do these patterns that had been observed in the lab to date, exacerbate over time, or do things get better as team members get to know each other, they get to experience women’s authority or their expertise. And what we found is that it’s mixed evidence on this. So in these surveys, these are monthly surveys that students have to fill out about each other… we call them peer evaluations… and in these peer evaluations, we ask them “Who is the most influential member of your group?” And they state who is the most but also who’s the least influential. What we found that over time, over the course of those four months that these students work together as a group, there is an improvement for the lone women, that their peers perceive them to be more influential over time. For the women in the majority, there seems to be no change. And so we do see the gap closing by the end of the semester, relative to the large gap in the beginning of the semester, but only in the survey data. Once we actually look at the data from the lab, where we observe students interacting in teams, where we can measure who is exerting influence on a task, we see no difference over time. So it seems that there is some improvement for the lone women in these sort of general assessments of influence in these monthly “Who was the most influential member of your group over the course of the month?” But when you actually get down to the specific tasks, we don’t see any improvement for women over time.

John: I know you were looking at this in a very broad context, in terms of teams and organizations and firms and so forth. But in terms of classroom groups of the sort that you were actually experimenting with, a growing number of classes in pretty much all disciplines now rely on group activities. What does the study suggest about how we form these groups in terms of the gender composition of groups, so that everyone can have an active role in the group?

Olga: Like you said, both in the workplace and in many academic settings, group work is crucial. And many faculty members rely on group- based activities. Understandably, they prefer collaborative thinking and develop the skills that students will need as they go on in workplaces where increasingly there’s reliance on group work. And so certainly the implications from our study are that assigning groups in which women are the lone woman or in the minority is going to have costs for women, costs in terms of participation, in terms of influence, in terms of whether they’re seen as authoritative, as leaders in the group. Those are the types of questions we ask and things that we can measure. And so certainly, if at all possible, groups that are gender balanced, or groups in which women are in the majority, are going to be significantly better for women in terms of these types of outcomes. Now, I would add a couple of caveats here. One is that in our study, we can track the grades. We can see what students actually get at the end of the semester. And we find no penalty for women, as far as grades can tell, when they’re in the minority. The women who are in the minority receive about the same grades as women who are in the majority. However, the grades in this program are largely group based. So it may not be surprising, because so much of the grade is based on the group work that we’re not finding those differences. Moreover, we don’t know how women get to those grades. It’s possible that because of these influence gaps, they’re having to work extra hard to get the same grade, or to be seen as sufficiently expert in that particular class. And so those are the two caveats that, even though we don’t observe differences in grades in our study, that doesn’t mean that there aren’t underlying differences in how hard students are having to work or how much effort they’re having to exert. I would also note that, regardless of the gender composition, there were no differences in man’s perception. So the man, whether they were in the minority, or in the majority, saw no deficit in influence, they were equally likely to be seen as a leader, they were seen equally influential. And so, if one thinks well, putting men In the minority is going to all of a sudden hurt the men in the group. That’s not what we’re finding. And there is in fact quite a bit of literature now confirming that. There are laboratory studies and studies in different settings, like nursing school, where men are in the minority, and in fact are not incurring any kind of deficit as far as influence or participation or authority that the women are incurring in these kinds of settings in which they’re a minority. I would also mention one study, it’s a working paper by a PhD student at University of Zurich, and it’s a really great working paper. She’s looking at a setting in which women are a minority… economics… a setting we’re familiar with. And in that setting, she’s using some data from, I believe it’s University of Zurich, it might be another university in Europe, but at that university, they also created different study groups, just like in our study, except these are larger study groups. These are sections of about 50 to 60 students, and they also randomly assigned gender compositions of these study groups. And what she shows is that, over time, the women that are assigned to be in a group in which they’re a minority, are much more likely to drop out of the study group altogether; that they not only incur these potential influence deficits, which we document in our study, but there are, in fact, very serious consequences to their ability to thrive in that class, or to thrive in that environment in which there are a minority. So that’s closely related, of course, to our study, and confirms really similar patterns.

John: We’ll share a link to both studies in the show notes. You mentioned that in disciplines like economics, and more generally, in STEM fields, women are often underrepresented as students, but they’re also underrepresented in faculty. It’s likely that these types of issues will carry over into group meetings and team meetings and department meetings and so forth on campuses. What can women and departments do to address this problem?

Olga: Yeah, that’s a very good question. Certainly, the setting in which we study these topics is student groups. But we are more than confident that these kinds of patterns replicate in a variety of settings, including professional settings, whether you’re a faculty or a student, being in the minority as a woman entails these costs to your level of influence, to your ability to exert influence, to your ability to be heard and taken seriously. And certainly there are other studies that have found very similar patterns in other kinds of settings. So I would not be surprised that if we ran this study in a professional setting or a workplace, we would find very similar patterns among women at all levels, including leadership. Certainly, some studies have confirmed similar patterns among the board of directors, female directors. The question of what can women do to sort of fix this is a really complicated one. And I say that because, what we find in our particular study, for example, is that women can’t just overcome that deficit by working extra hard. One thing that we observe is their levels of participation, how much time they put into coursework, and things like that. And we find that to be the same, regardless of the condition in which they’re in. They’re working extra hard already. Another thing that we observe is their talk time. In this laboratory setting, we can measure how long each person talks. And so you might say, well, maybe women, they’re just not leaning in, maybe they’re not participating enough in these group discussions, and so of course, they’re not seem as influential. Well, we find that’s not the case. These women are in fact leaning in. They’re speaking just as much regardless of the condition in which they’re in. The women and the minorities are going out of their way to try to get their opinions heard. They’re speaking just as much, as far as we can tell, based on the speaking turns and speaking time that we can observe. And so the failure to lean in can’t explain this gap in influence. So the common sort of Sheryl Sandberg “lean in” approach is that women just need to participate more and become equal participants in the process. That doesn’t seem to be supported by our research. Even when they try to do that, that doesn’t help them overcome this gap in influence. And so that’s kind of a depressing thing to discuss, that there isn’t much women actually can do to change those kinds of outcomes when they find themselves in these settings where they’re underrepresented. That it’s really men’s attitudes and men’s behavior that seems to be changing when women are in the minority versus women in the majority. So in our study, it’s men that are evaluating women as more influential when they are randomly assigned to be in a group with more women relative to when they’re in the group with just one woman. But of course, these underlying causes are really structural. So if you were to ask me, you know, what can organizations do to avoid those kinds of consequences for them, and I would say, “Well, number one is they need to hire more women.” Creating an environment in which women are no longer in the minority is certainly the direct implication of our research. However, that might be the more longer term goal. If organizations, say a tech firm, says over time, “We’re trying to hire more women, but we just don’t even have enough qualified women in the pipeline. What can we do now? How can we fix this given that women are still going to be in the minority for a while…” Then thinking about the structures of the teams and how they’re assigned, but also the norms within those teams? So for example, my co-author Chris Karpowitz has done some research in the past about the norms of deliberation and whether teams make decisions by majority rule, or whether teams make decision unanimously. That seems to be really important to women’s ability to contribute in environments in which they’re underrepresented. So maybe restructuring some of the team norms so that decisions have to be made unanimously, such that women’s voices are heard and they’re able to contribute even when they’re in the minority.

John: One thing I’ve been thinking when I read your paper and during our discussion is that there’s a similar cultural issue that affects teaching evaluations. And there’s at least some research that suggests that the negative bias that students may have in evaluating female professors can be overcome somewhat when students are made aware of the existence of this. And one nice thing about studies like yours is that it is making people more aware of this. But it would be interesting to see if students were given information about this at the start of their group formation, if that may affect the way in which group behavior is formed.

Olga: I am aware of those studies and I like them very much, because they show us one way, an easy nudge, which can change behavior, in this case, in the context of student evaluations of teaching. So in our study, of course, we try to keep the framing about students’ participation in this research, very neutral. We didn’t want them to be primed that this was a study about gender dynamics in groups and things like that. But I can envision future work thinking about the next step, which is what can be done to reduce this gender gap, what can be done to improve outcomes for women when they do find themselves in the minority, and one of those could be making students aware then making these patterns a lot more salient. Because honestly, if you probably ask a lot of the students whether they think that women in these groups are incurring any kind of penalty, they would probably say, “No.” The majority of the male students would probably not think that these things are happening. They’re happening in a subconscious basis, not through explicit discriminatory practices. It’s certainly possible that some male students are explicitly discriminating. But one measure that we have of that is how satisfied students are with their groups. And what we find is actually, regardless of the condition in which women are in, they report very high levels of satisfaction with their group. So even when they’re in the minority, and we can see that they’re incurring this really strong cost or deficit of influence, they still report being equally satisfied with their groups, and as happy with the group interactions as the women in the majority. So it seems that even the women themselves are not often recognizing that these deficits are occurring, let alone the male students in the group.

Rebecca: They’ve experienced it forever, it doesn’t seem different, right? [LAUGHTER]

Olga: That’s right, and this is not the first setting in which they’re experienced in this. There’s research showing that these kinds of patterns exist as early as school levels, where difference in competition is found as early as kindergarten, basically. And so the socialization that takes place even prior to college is probably conditioning women to feel that that is a normal kind of environment.

Rebecca: Your study reminds me a lot of conversations around all girls schools in K-12 and some of the benefits of that for women and also thinking about compositions of committees and things that might exist in professional environments where they’re trying to diversify, and they diversify by having token representation. And we often see that that can be problematic, but this is demonstrating other ways in which it can be problematic, which I think is a lot of interesting food for thought.

Olga: Yeah, absolutely. That was one of the biggest motivation… thinking about this is when you look at these policies, both private and public initiatives that are aiming to diversify these settings, like school boards or corporate boards, political assemblies, often, like you said, the solution is let’s just add one or two token women or minorities to the setting to help us be more diverse, and certainly we wanted to know what impact is that having on the women that are added, the women that become those token or lone members of the group and it’s not looking great. [LAUGHTER]

John: It’s a cultural issue and cultural changes tend to be slow. And as you said before, the only real solution is to have more balanced representation in all groups.

Olga: Absolutely. Yeah. And often, of course, what you hear, especially in the private sector is, “Well, it’s a meritocracy. Everybody can apply for these jobs, and we’re just not finding enough qualified women. And you know that certainly could be a valid concern in some stages of application process, but it is an important hurdle to overcome and think about how do we get more women into the funnel? How do we make sure that our women persist through the application process and actually make it into these jobs, because there are barriers at different levels, at different stages of that process that lead to these gender disparities in the share of women that go into these occupations, it’s not all choice. Choices are made, not in a vacuum, they’re made based on the constraints and information that people have. And so making these environments more appealing, more welcoming to women, should be an important objective of any organization that is struggling to increase diversity, gender diversity, in their rank and file.

Rebecca: As someone who teaches when an area of design that is also not balanced, [LAUGHTER] I teach in a more tech heavy side, it’s much more male dominated, because there’s more code and stuff involved and so historically, there’s less women, I’m thinking about all the group work that I do in my own classes, in the context of your research, and thinking about how productive and exciting it’s been to see some groups of all women, and what that looks like and what that feels like. But also having that little voice in the back of your mind saying maybe we need diverse teams that represent different kinds of people, because we’re designing for different kinds of people. And that, for the benefit of males in the class of interacting with women, maybe it benefits them, but they already have a benefit. And so that’s a really interesting consideration that I don’t think we often think about… not in a systematic way… or thinking about groups. I thought about majors and all kinds of things when I was formulating my groups, but I didn’t necessarily think about this.

Olga: Yeah, and I think that’s very common, especially in environments where there are serious binding constraints, you only have a few women. So I’m at BYU, and we have our share of women in the majors only about 20%. So any faculty trying to form group is going to be faced with these really serious constraints. One thing I would say is, in addition to this quantitative evidence that has been generated over the years showing how harmful it may be for women to be in the minority, there’s also, in our study, some qualitative evidence that we find. And since we’ve presented this study in different places, it’s been such an interesting experience, because you get these women just nodding their heads and saying, “I know exactly how this feels having been in the minority, and having compared my experience as a woman in the majority, just how much more heard and influential I feel in those kinds of settings.” So I think compiling qualitative evidence, pointing to the fact that it is significantly more difficult for women in the minority in these group settings to exert their influence and to get their voices heard.

John: Are you thinking of extending this research to other areas in terms of say race or other categories in which there may be similar effects?

Olga: Yeah, absolutely. So the original study certainly can only speak to gender, we have very few non-white students in the sample and can’t say very much since they weren’t randomly assigned across the group composition. But our goal long term is to look at whether these patterns extend beyond the gender domain. My guess is that we’re going to find very similar patterns for racial minorities, for example, who find themselves being underrepresented in many kinds of similar settings. They may even be exacerbated relative to the gaps that we find for women. And so we’re very interested, we’re in conversations with one firm and another institution trying to design a study that might work but this is a work in progress. And I hope it happens, because certainly we want to know whether other kinds of minorities find themselves in similar predicaments when they are underrepresented.

Rebecca: It also seems like it would be interesting to know whether or not, if you have multiple people from different underrepresented groups, if that somehow starts treating that more as a majority of underrepresented people, or if it’s just specific to a particular group at any given time.

Olga: Yeah, that’s a good point. One thing that we are doing is we do have a study in the field that is sort of following up from their original study, which includes the groups in which women are still in the minority, but they’re not the only woman. In our original setting there’s either one woman and four men, or three women and two men. So it’s not a symmetric kind of setting. And that’s by design, because there’s so few women in that program that if we created two women groups we wouldn’t have enough sample size to confidently say whether these results are statistically significant. But in the follow up study that we have been doing in the field, actually, for the last year and a half, we do have groups with two women and groups with two men. So we can compare sort of more symmetric, does it help to have another woman in a team? Or does it not make a difference, because you’re still in the minority. Some preliminary findings that we have, are that, unfortunately, it’s not tipping the scale… that unless women are in the majority, they’re still going to incur those deficits in terms of influence. And that’s supported by some of the prior laboratory research. But this is still ongoing… so, unfortunately, not the full findings yet. Another interesting extension of this work that we have started implementing, sort of by accident, or by necessity, rather, when COVID head and a lot of the group interactions have moved online… our entire lives have moved on to virtual settings… we wondered whether these same patterns would be exacerbated in virtual settings. There’s some anecdotal evidence that it’s even harder for him to get their voices heard in these kinds of settings. And so the study that we had been running in person has been turned into a study using Zoom as a platform. So we can now, at the end of this semester and next semester, say something about whether these patterns are different in online settings versus actual face-to-face settings, and what kinds of additional burdens may fall on the women when they’re having to influence outcomes or participate in the deliberative process in an online setting.

Rebecca: …sounds fascinating.

John: It’s a great natural experiment. …let me rephrase that… [LAUGHTER] we should probably not refer to the pandemic as a great experiment…

Olga: I know.

John: …but it does provide an interesting source of data on that issue, and virtual work is likely to become much more common in the future anyway.

Olga: Absolutely. It doesn’t seem like it’s going to go away. Even if the pandemic ended today, people are getting used to these kinds of interactions. There are advantages to them in terms of flexibility and the kinds of geographical constraints that no longer seem to apply. But they may also have these unintended hidden costs that I think are important to be able to quantify, particularly as it relates to these gender and racial disparities that already exist in a lot of these settings and workplaces.

Rebecca: So we always wrap up by asking: “What’s next?”

Olga: So this study has really led us to think carefully about these gender disparities, and to try to understand what kinds of interventions can help improve the outcomes for women. So the next step is certainly for us to try to test and evaluate the effectiveness of some of these interventions. So for example, I mentioned we’re doing a study in the field using Zoom as a platform for team meetings, we’re playing around and designing different kinds of changes in group norms, which operates through Zoom on, for example, who gets to start the conversation, or timing each participant in the group, so they know how long they’ve been speaking for… things that have been possible through technology, and trying to see whether those kinds of interventions will help improve the outcomes for women when they’re in the minority. So that’s one direction in which we are continuing this research agenda. And then another one, of course, is looking at other kinds of minority status. So particularly looking at race, we’re very interested in collaborating either with firms or other institutions that have ethnic or racial minorities, and are interested to know what implications do these settings have on their minority employees or students?

Rebecca: Looks like a lot of great work coming down the pike. I’m excited to hear what you find.

Olga: Thanks, Rebecca, thank you so much.

John: You’re doing some wonderful work, and I’m looking forward to seeing more of it in the future.

Olga: I really appreciate it. Thank you guys. It’s been a pleasure. Thanks so much.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

159. Nurturing a Growth Mindset

Emotions and past experiences can lead us to develop fixed mindsets in particular aspects of our lives and learning. In this episode, Kelly Theisen joins us to discuss ways to help foster growth mindsets within a course from the beginning to the end of the semester. Kelly is an Assistant Professor of Biochemistry at the State University of New York at Plattsburgh.

Show Notes

  • Brown, P. C., Roediger III, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Bjork, R.A. (1994). “Institutional Impediments to Effective Training”. Learning, remembering, believing: Enhancing human performance.
  • Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.
  • Yue, C. L., Bjork, E. L., & Bjork, R. A. (2013). Reducing verbal redundancy in multimedia learning: An undesired desirable difficulty?. Journal of Educational Psychology, 105(2), 266.

Transcript

John: Emotions and past experiences can lead us to develop fixed mindsets in particular aspects of our lives and learning. In this episode, we discuss ways to help foster growth mindsets within a course from the beginning to the end of the semester.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Kelly Theisen. Kelly is an Assistant Professor of Biochemistry at the State University of New York at Plattsburgh. Welcome, Kelly

Kelly: Thanks for having me.

John: Our teas today are:

Kelly: I have Earl Grey today.

Rebecca: And John, get this. I have a different kind, Gingersnap tea.

Kelly: Oh, that sounds good.

John: Where did you get that?

Rebecca: It’s a Tea Forte, you’d be happy to know.

John: Oh, I haven’t seen that one.

I have a summer berry green tea that I picked up in Epcot last year. My supply is dwindling, though.

Rebecca: I know, and you’re very disappointed you’re not going to be there this fall.

John: I know. I had planned to, but I will be online at that conference.

We’ve invited you here, Kelly, to talk about how you’ve been working to help students reframe their academic anxiety by helping them to cultivate a growth mindset. Before we discuss how you’ve been doing that, could you tell us a little bit about the courses that you teach?

Kelly: Yeah, sure. So I teach primarily my general biochemistry course for non major students, and I teach that every semester. And then in the spring, I also teach physical biochemistry, which is a much smaller class for biochemistry majors only.

Rebecca: Can you talk a little bit about why it’s important that students develop a growth mindset?

Kelly: Yeah, so I think it’s especially important for my 371, the general biochem students to have the growth mindset, because they usually come into my class terrified, absolutely terrified of biochemistry, they’ve heard it’s like the worst class ever. And they think it has math, and they’re just so scared. So, I think that it’s important for them to have the growth mindset so that they feel like they can actually succeed in the class, which a lot of them again, they come in not thinking that they can. So, developing a growth mindset, reminding them that it’s hard, like the class is not gonna be easy, but that they can do it, they can get better with trying, is really key for helping them to keep going if they start to struggle. What I don’t want is for them to get to the first set of content that’s difficult, and then just give up. I want them to keep working at it, because I know that practice is going to make it better for them. For me, that’s why it’s so important, it’s because I want all of my students to be able to succeed, not just the ones who are already super motivated, and everything. I want everybody to get through the class and do well.

John: Why do you think so many people come into our classes with this fixed mindset?

Kelly: So, it can have lots of different origins. And I think some are internal and some are external. So, for example, a student could have somebody in a previous class, any previous STEM class who has told them, you’re not good at STEM, or you’re bad at math, or things like that. And so all it takes is maybe one teacher in school, a professor, when they get to college, who tells them that,for them to decide that this is just not for me, and I just have to take these classes for my major, but I’m just going to get through them because I’m not going to be good at it. And so that’s one way they can develop a fixed mindset about it. Also, it’s possible to have a fixed mindset in one area of your life and not in others. So student athletes is where I think of the most for this, where they have a growth mindset in terms of their athletics. They know that going to practice, working on whatever technique it is, is going to help them improve and do better during the game. But, they don’t always think to apply that to their academics as well. And so they might say, “I did badly on this test. That’s it, I’m done. There’s no point in me trying any more in this class.” Things like that can lead to the fixed mindset in classes, even if they don’t have one in other areas. So, it can be like I said, internal from them, or external with other people telling them, “You’re bad at this,” or whatever. And that happened to me actually a lot growing up, and in my career. Lots of people told me it shouldn’t or couldn’t be in chemistry, lots of very stupid reasons for that. But still, it happened enough that if I hadn’t had a growth mindset myself, and knowing that just because this person tells me I’m bad at something doesn’t mean I really am. Or just because I had to ask for help doesn’t mean I will never get this or that I’m bad at it, then I don’t think I would be here having this conversation with you, frankly.

Rebecca: Sometimes I find that the students that you might least expect to have a fixed mindset do, they might be the students that you think of as good students who have done well or have succeed[ed] previously in other classes they’ve had with you or in the discipline, but they come across a hurdle, maybe for the first time, and they just don’t know what to do, because things have come easy for them, or they haven’t had to work so hard.

Kelly: Right? Or they got by on just memorizing in high school, and then they get to college, and it doesn’t work anymore. And so then they can say, “This is just not working. I’m just going to give up. And I don’t know what else to do besides memorize, and if that’s not cutting it, then what am I doing here?” So, yeah, definitely. And biochemistry is a difficult class, and so not everybody is going to get 100% on every exam. [LAUGHTER] And so that can be challenging for some students, where they really want that hundred percent.

Rebecca: Yeah, and especially in the sciences, or any place where you need to explore or experiment, taking a risk can be really challenging if you have a fixed mindset.

Kelly: Yeah, exactly. And I want the students to think critically about what they’re learning. I don’t want them to just memorize the information and then spit it back out on the test. That’s not what science is really about. It’s about exploring and trying to figure out why things work the way that they do. And so, that risk taking, I usually make them do that every day in class. We do active learning, and they have to say, “Here, try to answer this problem before I’ve explained it to you. And so you’re gonna get it wrong, it’s fine.” And so that process, again, it’s sort of encouraging the growth mindset, but it’s difficult for them at first. They want to know the right answer ahead of time a lot of the time. So, you have to remind them and reassure them: “It’s okay, I’m not gonna grade you badly if you get it wrong. You’re just supposed to try and do the best you can.”

John: While anyone can have a fixed mindset coming into your class, some of students who expect to do well, some others based on their prior experience who might not expect to do very well. But, are there some patterns, perhaps, where first-gen students or students from underrepresented groups might be more subject to this, particularly in the STEM fields,

Kelly: I think that they’re just more likely to have been told that they’re not good at this or that they shouldn’t be in X, Y, or Z discipline. And as I said, that happened to me a lot. I was a first-gen student and female man going into chemistry, which is still pretty heavily male dominated. And then I went into computational chemistry, which is even more heavily male dominated. And so, yeah, I think that just because of that background, they might be more likely to have heard things like that before. And so actually, one of the things I do on the first day of class is, I say, “How many of you were told this or something similar to this?” I don’t usually get a lot of hands. I don’t think a lot of people want to disclose that necessarily in front of the rest of the class, which is fine. But the point of asking is, so that I can tell them, this happened to me all the time. And I made it through and I’m now a professor, and I’m doing these things that they told me I wasn’t any good at, but I actually am. So, that means that you can too, basically.

Rebecca: It’s funny how those early comments from teachers can have a really big impact for a long time.

Kelly: Yeah.

Rebecca: I had some similar experiences. I remember very distinctly in eighth grade, like math teacher telling me I couldn’t do math. I remember a seventh grade art teacher telling me I couldn’t do art. And now I do art that has math in it.

Kelly: Right. Exactly. My teacher in high school told me I wasn’t good at computers, because I couldn’t type both quickly and accurately. Turns out, I’m just dyslexic, and so I just hadn’t practiced typing enough at that point. And I’m now a computational chemist, and I work with Linux and programming, and so it’s fine. It just took me a little longer to get up to that level that they were expecting, then it did some other people. But again, it’s just about focusing on “You can do it, you just have to keep practicing.” And knowing that where you start is only where you start, and that you’re the one who gets to decide where you end up.

Rebecca: You suggested that a growth mindset is a scientific mindset. Can you elaborate on that?

Kelly: Yeah, I think anybody who has done research knows that you have to have a growth mindset to go into research… to enjoy doing research, at least… because you’re going to fail all the time. You’re going to start an experiment, and it’s not going to go right, or it’s not going to happen at all, [LAUGHTER] and you have to figure out why, you have to understand, “Okay, something went wrong. Did I do something wrong? Is it the experiment is actually showing us negative data? What is happening? And that really does take a growth mindset. You have to be willing to fail to go into research. You have to understand that it’s not you necessarily failing, the process of science requires a lot of failure. So, that’s one of the other things I try to tell the students is like, you’re actually living like a scientist, right now. [LAUGHTER] And this is what we do all the time, we set something up, and who knows? We just have to see what happens and then go from there. And one of the things that I really like the most about growth mindset is it sort of freeing, it gives you the ability to just try and know that your first effort is not going to be your best. And I really love that because it frees you from perfectionism, or wanting things to be exactly a certain way, the very first time you do it. You just know, whatever I do, it’s only where I started, and it’s not going to be perfect. And that’s okay, I’m always going to get better from that point. And so I feel like that, at least, has helped me to approach research and teaching and knowing that I might not be great at everything to begin with. And it helped me to try things that I might not have been comfortable with necessarily. But again, it was “Okay, I know, I’m not gonna be great at this right now, but I’m still gonna do it. And I’m going to try my best and that I know, I’m going to get better from there.”

John: What are some of the approaches you use to help nurture a growth mindset in your students?

Kelly: So, there’s actually quite a few that I like, and I use them in all my classes, but again, especially in the general biochemistry for non majors class. So one of them is the frequent low-stakes assignments, it helps the students to build confidence in the material and it gives them sort of a grade cushion, for the exams. And the low-stakes assignments, it helps with inclusion and equity as well. So, one of the other things I do when we have face-to-face exams… I’m not doing it this semester, because this semester everything’s online. But, when I head face-to-face exams, I was doing exam corrections. So, basically, the students could earn back up to half the points that they missed on the exam. And basically, I think that this helps students continue to engage with the material, instead of you learn it for the exam, and then you forget it immediately afterwards. They would have to go back and look up what they had missed, and try to understand why they got it wrong, which helps them to keep engaging with that content. But it also helps them to stick with learning things that are difficult, right? Even the exam is kind of not the end, I guess. I always thank students for asking questions, or for volunteering answers in class, even if they get it wrong, because that kind of thing helps everybody learn. So, I always tell them that I appreciate that, that that’s good. I guess just reassuring students that even if they failed at something, that that’s a step in the right direction to helping them succeed eventually. This semester for the exams, I have them look up a research paper ahead of time. And a lot of students were apologizing because they hadn’t gotten it right the first time or even the second time. And so I had to remind them just because you tried it, that’s the point. Even if you failed, you’re getting better at it just by trying it. And then I think the last thing that I do, and the students really seem to like this, because it comes up on my evaluations quite frequently, is the learning objectives. And I actually think that this promotes growth mindset, because having sort of almost a checklist that they can go through and say here is everything I’m responsible for for the exam. It kind of gives them a way to say “Okay, here’s the things I already know, and here’s the things they still need to work on.” So, it almost forces them to have a growth mindset by going through and checking everything off. So, I really like to do that as well.

John: You mentioned having a list of learning objectives, is that something that you include, say in the course module in your learning management system?

Kelly: It’s on the slides every day. So, when they walk into class, the first couple slides will have the learning objectives. And then I show them again at the end. When we were doing face-to-face classes, there would actually be on their daily worksheet, they’d have a question at the very last thing, which said, “Is there anything you’re still confused about from today? Are there any learning objectives you feel were not met?” And then that gave them a spot to write in, if they had any questions remaining. And that way, I could kind of check in with them as well. It’s harder to do that online I found. And so I kind of missed that this semester. But the students really do seem to love putting questions in the chat and things like that. So. I think we’re still managing to do that okay.

Rebecca: One of the things that, along those same lines, that I like to have conversations with students about, is the more mistakes and things they make they end up learning more. [LAUGHTER] So, it’s like, “Well, if you got that all right the first time, you wouldn’t have had this whole learning adventure that you didn’t plan for.” And I find that framing it like that tends to put a positive spin on something that they might seem as being a very big negative.

Kelly: I don’t know if anybody else is as big of a nerd as I am and watches Disney movies, but in Meet the Robinsons, there’s the part where he’s trying to fix something and he completely fails, and it just like bursts apart. And they tell him that “Well, you learn from failure, you don’t learn as much from success.” And so it’s kind of the same idea.

John: And earlier, you mentioned too that you share some of your own struggles and some of the challenges you were faced with. And I think that probably helps build a growth mindset in your students too, by setting that example and normalizing struggles and failure as part of the growth experience.

Kelly: Yeah, exactly. And I think one of the things that instructors can definitely do is to model a growth mindset for their students, to tell them that things are not always easy for me, even. I’m an expert, but I still come across research problems where I’m like, “I don’t know what’s wrong.” I did that over the summer, I started a new project, and it took us, gosh, it was like months to figure out what was going wrong with our experiment. So, you’re right, normalizing it is great. Saying that just struggling with something is good, in fact. It’s how you learn.

Rebecca: You also mentioned setting up problems that they solve in class that they’re not quite ready for. Can you talk about how that nurtures growth mindset as well?

Kelly: I got this idea from a book called Make it Stick, where it’s called desirable difficulty. And I just love it. We do this all the time in my other classes. Well, that’s like the main principle that I use for my smaller majors class is having them try it before it’s explained to them. And again, it’s trying to get them to let go of that idea that I have to know the right answer before I can try anything, or I have to have had this explained before I can even try, which I think it’s kind of burned into them in the K through 12 system sometimes. Where you’re given the information, memorize it, spit it back on the test, that’s it, you’re done. And that’s what learning is to them. So getting them out of that habit is what I’m trying to do with giving them these assignments that they’re not quite sure about. And I actually tell them that that’s what this is called, especially this semester, I’ve really leaned into just like, here’s my teaching method, it’s called desirable difficulty. And here’s what it’s going to do for you, it’s actually going to help you understand it better when I explain it to you if you’ve tried it on your own first and gotten it wrong. And so we’ll do things like I will ask them, How does the entropy increase when the hydrophobic effect occurs? And they’re like, “Well, I don’t know.” And I was like, “I know you don’t know. Think about it. Here’s the system, here’s what’s going to happen. And what do you think and then don’t look it up?” And actually, they seem to not look it up this year, which was good. I was worried they might because we were on Zoom. But, they actually seemed to refrain from googling it. Because I did see a lot of wrong answers. I think it went pretty well. And then just kind of over the semester, they get better and better and less fearful about putting wrong things down on the worksheets, because they know, first of all, that they’re going to get the grade no matter what, as long as they put something down. But, also because I think that they’re learning to try more and to think critically. And so that’s what I hope for them at least.

Rebecca: Do you have them discuss their solutions in small groups? Or is it an individual activity?

Kelly: Yeah, so in class, I would typically do like a think-pair-share where they would get with another person, and then I could talk to them as the class. This semester, we’re now doing breakout rooms, because I have 50 students, and it’s a little hard to get them in pairs, and then in a bigger group, so I’ve just assigned them groups for the semester at this time. And then they will go and work with their group and discuss. And usually there’s a few minutes delay anyways, in terms of getting the breakout rooms ready to go and everybody into them. So, they have some time to think on their own as well, just because of that.

John: How large are the groups that you’ve been using in the breakout rooms?

Kelly: About five usually for each group. And that’s because, first of all, I wasn’t sure how engaged everybody was going to be in each group. So, I kind of wanted it to be big enough that if a few people went AWOL, that they still had a group. But also it’s nice just that people can work better with other people, and so then they have a couple options in terms of partners, if they wanted to work with just a few people.

Rebecca: Kelly, you’ve talked a lot about ways to foster the growth mindset throughout the semester. But, how do you set the stage maybe on the first day of class,

Kelly: This year, I actually did what’s called first-day fears, a brand new activity I’d thought of, but I had emails and some survey responses over the summer that said a lot of students were terrified of online learning, things like that. And everything was changing, and we were scared. And so it was just this whole mess. And so I basically said, “Here’s your first activity for the day, go into the breakout rooms with your group, write down everything you’re anxious about for the semester, and then I’m going to talk to you and we’re going to try to work through everything.” And so they did that. And I had a lot of “I’m scared of online learning. I don’t know how I’m going to do this.” And we talked about how you can approach some of those anxieties with growth mindset, in terms of try something for the online learning. If it’s not working right away, then change it. Talk to me, see what else we can do that can help you manage your time better, whatever it is. And so that was how I framed it this semester was, “Yes, you’re anxious about things, here’s how you can address some of those with growth mindset.” Unfortunately, all the growth mindset in the world will not make COVID go away, it won’t give us more money if we need it, that kind of thing. So it’s not perfect, obviously. But in terms of the things that we can control, that’s what I love growth mindset for. So, helping them to understand that I’m a resource, that their group is a resource, and that we’re going to keep doing check ins throughout the semester. The other thing that I did is part of the activities, I told them what I was anxious about. So, I told them that I’m anxious about online learning, too. I’ve never done it before. I spent the summer learning about it, but it’s still the first time I’m doing it. And so I told them, here’s some of the things that you can do to help me, which is, if your camera’s not on, I can’t tell if you’re confused. So, you have to tell me that you’re confused. Say, “Hey, wait, stop, I need that explained again,” or put your questions in the chat. And here’s the ways you can do that. Yeah, so we basically just talked about things all of us were anxious about and trying to show them that, first of all, they’re not alone. Look, how many of your classmates have the same worries, this is why you’re in a group, so that you have people to talk to and then you can talk to me as well, then just trying to clear the air a little bit before we could get started, I guess, with learning this semester.

John: So you’ve talked a bit about how you try to help your students build a growth mindset. Do you explicitly talk to them about the differences between growth mindset and a fixed mindset.

Kelly: I do, actually. So, there’s a slide from the first day where I put some students’ examples of what a fixed mindset could look like, and what a growth mindset could look like. So, for example, from the student perspective, if you did badly on an exam or in a class, it might lead you to think “I’m stupid, I’m not gonna get this, why am I trying?” Or “Why am I participating in this class, everybody else is better than me,” it might lead you to think things like that, which are more fixed mindset. And then on the other side of the slide, then I had what a growth mindset would look like for that, which is to say, if you’re struggling, you should ask for help, you can learn more with practice. So, you should go and get more resources, ask for it to be explained, again, things like that. That you should still contribute to class because your responses are unique like you are. And so my favorite example, the one that I shared with them, I had a student my first semester at SUNY Plattsburgh, who was a great student, wonderful in class, and they completely bombed the first exam. And I felt so bad, because I knew they were trying, but anyways, so they kind of wasn’t sure how they were gonna bounce back from that, or if they were, and then we a little bit later, maybe a week or two, after the exam, we were having an activity about allostery, and nobody was getting it. Everybody was complaining: “This is so hard.” “I don’t understand what we’re doing.” “What is going on?” And there was just complaining. [LAUGHTER] And so I was walking around the class trying to help. And I got to the student’s desk. And they were like, “Well, I don’t know. But what about this?” …and they had got it, clearly they had understood what I was actually asking. And so I asked them to share with the class. And they did, and it was like a light bulb went on for everybody else. And so I was just reminding the students like this person failed the first exam, but they were the only person in the class who actually got the next thing. So, it doesn’t matter if you failed, you still have these valuable contributions to make, you’re still a part of the classroom, you’re still supposed to be here. So that’s, as I said, one of the ways that I tried to improve inclusion is just to say, “You’re always supposed to be here, this is where you are, we want you and you’re supposed to be in this class.”

Rebecca: I like how you’re framing things related to the anxiety and emotions that can be big blockers, to moving forward and addressing those emotions and normalizing those emotions and verifying that yes, indeed, we might be frustrated or confused or scared, but if we can acknowledge that and know that that’s what we’re working with, we can move forward and continue to grow and learn. And I think that students don’t always recognize that those emotions can get in the way.

Kelly: Yeah. And it’s about also recognizing what is in your control and what’s not. So, there’s some things that are not in your control, right? COVID is not under anybody’s control right now. Not any of us anyways. So, you can’t tell the students well, you should growth mindset yourself so that this doesn’t affect you anymore. No, that’s not how this works. The growth mindset is to say, “Okay, I’m trying in my classes, something’s not going right. Is there something I can change to maybe make it go better?” Or even just to recognize, “I’m doing the best I can. This is the most that I can do right now. And that’s okay.”

Rebecca: Yeah. Especially with the balancing act and the extra stress of COVID-19, and what have you. Recognizing that like, it might take longer.

Kelly: Yeah.

Rebecca: It might not be an A, it might not be a B.

Kelly: Right.

Rebecca: But, like, you got something. [LAUGHTER]

Kelly: Yes, exactly. And if you decide to try it again, later, you’re gonna do better because of it.

John: And having students share their anxieties makes them feel perhaps a bit less isolated, and recognizing that some of these challenges are ones that are shared by everyone, which I would think would help to build a better community or more productive community within the class.

Kelly: Yeah, that’s what I was hoping for. And again, that’s why I assigned them to the groups as well. With 50 students, I knew they weren’t going to meet everybody online. They barely meet everybody if they’re in person. So, I wanted them to have kind of a core smaller community that they knew, “Oh, yeah, we did this on the first day. We were all nervous about the same thing.” Yeah.

Rebecca: I’ve had those consistent smaller groups in my classes this semester, too, and it’s helped a lot… have it that tight community to express anxiety or share frustrations with. [LAUGHTER]

Kelly: Yeah, and I’ve called them their growth mindset groups, which is hokey, but I couldn’t think of another name. But, yeah, we’ve done a couple check ins so far. There was a question on the first exam about a situation that they had to face, like an academic challenge and did you approach it with a growth or fixed mindset… and then “How could you maybe change what you did?” or something like that, and then we did another… right after the first exam, we did a learning reflection, which is… same idea. I told them, you know, check your current grade on Moodle, because a lot of times they don’t always realize that that’s up there, and then come up with a growth mindset plan for going forward. You know, if you’re not happy with your grade, okay, what can we try that might help you do better?

John: Yeah, that sort of metacognitive reflection, I think, can be really helpful and helping students recognize how much they’re learning, and to see that they can change the pattern.

Kelly: Yes, and to look at the overall grade instead of just the exam grade, because a lot of them saw the exam grade and panicked. [LAUGHTER] And they didn’t realize that the exams are only 40% of the total grade. So, [LAUGHTER] you’re probably still okay,

Rebecca: We always wrap up by asking: what’s next?

Kelly: So, what I would like to do next… a couple things. First, the SUNY Plattsburgh orientations have started incorporating a growth mindset aspect to it. And so what I’m hoping is to see more and more students coming in who already know about growth mindset, then have started to develop it in their earlier classes too, which would be fantastic. I already lean in pretty hard to growth mindset in my classes, but maybe lean even further into it in terms of assessing it, and trying to see if my class structure actually helps students develop a growth mindset as they go. I’ve had a few students put that on their evaluations at the end, that the idea of growth mindset helped them to succeed in the class. And it was one of the first times they’d heard of it or things like that, where they’ve said that that plus the active learning helped them to be successful. But, in terms of actually assessing overall, even if they don’t tell me that up front, you know, can I determine if it’s actually helped them to develop a growth mindset is one of the things I’d like to do.

Rebecca: Sounds like good research project.

Kelly: Yes, yes, I do education research on top of computational research because I’m a crazy person, and I want to study everything. [LAUGHTER]

John: Well, thank you. It’s been great talking to you.

Kelly: Thanks for having me.

Rebecca: Yeah. Thank you very much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

127. Gender and Grade Changes

Grade change requests in college are relatively rare, but when they do occur, evidence suggests that male students make the request more often than female students. In this episode, Dr. Cher Li joins us to discuss these gender differences in grade change requests in college and why they might occur.

Cher is an assistant professor of economics at Colorado State University. Her research focuses on how public policies and social institutions affect the decisions of, and outcomes for, women. She is also a co-author of a January 2020 National Bureau of Economic Research working paper that investigates gender differences in grade changes.

Show Notes

Transcript

John: Grade change requests in college are relatively rare, but when they do occur, evidence suggests that male students make the request more often than female students. In this episode, we discuss these gender differences in grade change requests in college and why they might occur.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Cher Li. Cher is an assistant professor of economics at Colorado State University. Her research focuses on how public policies and social institutions affect the decisions of, and outcomes for, women. She is also a co-author of a January 2020 National Bureau of Economic Research working paper that investigates gender differences in grade changes. Welcome, Cher.

Cher: Thank you. It’s great to be here.

Rebecca: Today’s teas are:

Cher: I have Oolong tea from the Ali Mountain area in Taiwan. It is one of the best tea regions in Taiwan.

Rebecca: Sounds good.

John: Nice. And I have Cranberry Blood Orange black tea.

Rebecca: And I have a mango black tea today. John’s looking at me a little strangely because it’s very unusual for me. [LAUGHTER]

John: We’ve invited you here today to discuss your recent working paper with Basit Zafar on regrading requests and outcomes. Could you tell us a bit about the study?

Cher: Yes, this study is about gender differences in grade changes in college, and we look into the differences and why are the causes for the gender differences?

John: What prompted you to look into this?

Cher: So this research is actually motivated by my personal experience. In the last six years, I had many male students, but only one female student asking me to bump up their grade at the end of the semester. And I know from previous studies that men are more likely than women to negotiate for their salaries and promotions, and that made me wonder if these kinds of differences actually started earlier when they’re still in college. So, that is the motivation for this study.

Rebecca: Can you talk a little bit about how you conducted the study?

Cher: Sure. We tackled this research using three very different sources of data, including administrative records from a very large four-year university and we also use surveys and laboratory experiments. So, the first data source is a very unique administrative data from Colorado State University between 2010 and 2016. We have over 1.3 million observations of grade changes across all the departments, and this data set is very unique because it recorded not just the final grade, but it also recorded any changes associated with it. So we focus on the changes made by instructors, and our analysis showed that, although grade changes are pretty rare events, 95% of the time they will change to a better grade, and among those positive changes, men are about 18.6% more likely than women to get a better grade. And the difference in grade changes cannot be explained by the characteristics of students, instructors, and classes. However, the administrative record has several limitations. First of all, it doesn’t provide information on the cases when any regrade request was rejected by instructors, and it also doesn’t tell us about the grade changes during the midterm or any assignment throughout the semester. And finally, it doesn’t tell us “What is the scenario that caused the gender difference in grade changes?” So we can think about at least three different scenarios that all lead to the same data pattern. For instance, the first one is male students are simply more likely to ask, although instructors treat men and women equally… so, that is a possibility, given the previous literature evidence. And the second potential scenario is that male and female students are equally likely to ask, but somehow instructors are treating men more favorably, and if it is this scenario, we’re going to have something to worry about about discrimination. And the third scenario we thought about is, there may be a difference in terms of the timing of asking. So if men procrastinate while women are more organized throughout the semester in they ask their instructor to make changes while they find any errors during the semester, then they have no need to ask at the end of the semester. So, to examine the scenarios, we conducted two surveys, one on instructors and the other one on students, and we checked whether the reports are very similar across different surveys. And surprisingly to us, both surveys show pretty consistent patterns: that male students are indeed more likely to ask, and we find no evidence that instructors are treating males students more favourably. And we also didn’t find any support that there is a timing difference in terms of their asking.

John: That’s certainly consistent with what I’ve observed myself, that males have been more likely to ask both during the semester and after.

Rebecca: I don’t think I’ve gotten that many requests, generally.

John: Well, I haven’t either, I may get one a year or two a year out of like seven or eight hundred students, but…

Rebecca: I can’t think of a single one. I know that I have but I can’t think of a particular…

John: Well, your classes are smaller, too. So, that could be a factor.

Cher: Yeah. So I also typically only got like one to two students per semester at making that request. So it jumps out at me when there is only like one female student throughout the whole six years of my experience, that really makes a difference to me when I compare the frequency in terms of they approach me at the end of the semester.

John: One of the nice things I liked about it was your incentivized controlled experiment that looks at why there may be differences. Could you tell us a little bit about that aspect of the study?

Cher: Definitely. So focusing on the gender difference in asking, we did this lab experiment to answer that question, in particular in the lab setting. The participants basically take a quiz of 20 questions, and their payoff is dependent on the grade in the quiz. So we tell them we’re going to give them the grade. However, we also tell them the grade is noisy. So, we randomly picked three out from the 20 questions to randomly grade them. The grade we tell them could be their true grade, but it could also be higher or lower than the true grade. And then they have to decide under 10 different cost scenarios, ranging from paying up to $3.50 to being paid to $1, whether they are willing to pay the cost in each of the scenarios to get a regrade. If a participant chooses to pay the cost to ask for a regrade, then the true grade is revealed and their payoff is adjusted and we subtract the cost they have to pay, and if a participant chooses not to ask for a regrade, then the initial grade becomes final and they will be paid accordingly. And we find that, as long as the cost is positive, so basically the participants have to make a payment, men are always more likely than women to pay the cost to ask for regrades, and the gender difference in asking cannot be explained by their difference in risk aversion. We find that other majors, including their difference in confidence level, in their uncertainty about their performance, and personality traits explain nearly half of the gender differences in asking.

John: How accurate were the students’ metacognitions in terms of how appropriately they made a decision to request regrades? Did they gain by that on average, or did they lose?

Cher: So in general, they have pretty good guesses about their expectations in terms of how well they do in the quiz. However, their decision here really depends on their attitude in terms of pursuing more aggressively in the regrade or not. And we find that, in general, most students who decided to pursue the regrade were likely to get a better outcome, and we find that depends on the cost, basically. So when the cost is low, the more likely a student will get a better outcome, because they are facing a very low cost to offset the gain they got from a better grade. And the reason that they are more likely to get a better outcome is due to our lab experiment design, we make it asymmetric, basically. So, we are trying to mimic the real world scenario. So from the administrative data, we find that 95% of the time, students got their grade change to a better grade. Only 5% of the time it got changed to a worse grade. We intentionally designed our lab experiment to allow that asymmetry. So basically, when a student asks, they will have a higher chance to get a better grade than to get a worse grade.

John: How much more common was it for males and females at low, and then again, at high prices?

Cher: When there is a cost that really they have to pay out of their own pocket, around like half of male students are willing to pay the cost, while only about one third of women are willing to do so. However, when the cost is zero or they got paid to do it, it’s very similar. I think their percentage of requests is very high. It’s like nearly 80% when they were getting paid $1 to do so.

John: What happened when the cost was high? I believe you found some interesting effects for males at that.

Cher: So when the cost is high, we find that the gain from asking actually disappeared, because now the high cost offset the gain made by men. So although men are more likely to ask, they are not always made better off, so they are only gaining relative to women when the cost is relatively low. When the cost is above like $1.50, we see that men are also more likely to see their payoff decrease compared to women.

Rebecca: Can you talk a little bit about the gender difference in the regrade requests and what might explain those differences in general after looking at those three different approaches that you’ve taken to study it?

Cher: So we find that, it seems to be, there are some subjective beliefs that are different to begin with. So first of all, there are some evidence from other studies that men and women are different in terms of confidence level, and men tend to be more overconfident than women in a lot of contexts. So, we explore that too in our study, and we find that indeed, when we look into each of the questions they answer, we also ask them how confident they are on each of the questions. And we find that when they got the answer wrong, men and women are similarly over optimistic about their accuracy. However, when their answers are actually correct, women are much less assertive about it, so they are less confident about their correct answer. So that is part of the reason that is driving the gender difference in asking, and the other factor that we found are important in terms of the gender difference in asking is about their perception about the potential outcome, so basically the uncertainty they face when they make that decision. So we asked them about like, “What is the number of questions you think you got right?” And they will give us their best guess. And then we ask them then for that scenario, “What is the likelihood you think that’s gonna be happening?” So they will tell us like, oh, if they are very confident about “I’m getting 15 questions right out”, maybe assigning 100% as the probability for that outcome, if I’m super confident that is the outcome. But then if I’m not very confident about my guess I maybe say “Okay, 50% probably,” then I’m not really certain about that outcome. So we find that when men and women are giving us those guesses, women are much less certain about their guess. So they are likely to think it could be different outcomes, maybe it’s 14 or 13 questions they got it right, so they would also assign the probability of other outcomes that they anticipated. And that also explained a very large proportion of the gender gap, and we also find there are some differences in their personality traits. For instance, women are more agreeable than men, so they may be more agreeable to the grade they received and less likely to challenge it in terms of the context of asking.

Rebecca: When we start thinking about the impact this might have on GPAs or their academic standing, how does this really impact men and women differently?

Cher: So the overall impact on GPA depends on how we measure it. For those people who ask, we did a back of an envelope calculation and we find that for the female students, the accumulative effect from asking is up to .43 points in their overall GPA. But for male students, the number is a bit higher, they could receive up to .47 points as an improvement in their GPA, conditional on asking.

John: Which is a non-trivial difference in terms of that, especially if you were also to extrapolate and think about the possibility that this is also occurring during the class.

Cher: Yeah, we take that into account in extrapolating, and that is a cumulative kind of effect we anticipate. If a student reports a fraction of courses they asked for a regrade, that would be kind of cumulative effect overall.

Rebecca: Do you have any idea of what the not asking is doing, maybe to women, if they’re less likely to ask in the first place, how that might be negatively impacting their GPAs?

Cher: So in terms of not asking, you are comparing to those who ask, so that raw difference could be low depending on who we are comparing to, or we can compare it to those who actually are successful throughout their time, then that difference will be up to like .4 to .5 points in terms of the GPA.

John: At the start of this, you talked about how there’s a lot of evidence out there that women are less likely to bargain for higher wages for promotions and so forth. From those studies, is there a lot of evidence of similar types of causation?

Cher: So most of the studies we saw before look into their different tendency to ask, and one of the particular research I think is very interesting is in terms of the context. In the study made by other economists, including John List in University of Chicago. In their study, they did a field experiment, and they experiment the job applicants’ attitude in terms of negotiation for their salary. In their experiment, there are two settings, one is the salary is ambiguous if there is no hint or any information about it. And then they found under that context, men still negotiated for their salary, but women in that context tend not to negotiate and instead, they signal their willingness to accept a lower offer. In the separate setting, when they make it clear in their advertisement that the job salary is negotiable, then there is no gender difference in terms of asking; men and women are similarly likely to negotiate their salary when that kind of information is clear, that the social cue is you are able to negotiate. In that case, there is no gender disparity, and women are similarly likely to ask. So we find it relevant in our context as well because a lot of time professors do not put a clear policy about regrading, so students may have to take that into their own guess, whether this is an appropriate behavior to make and approach their instructors about it. So I found that kind of ambiguity is relevant in terms of the context when the grade change policy is ambiguous.

John: I’d be really hesitant to put in a procedure explicitly for negotiating grade change requests, but I could see how that could make a difference when it’s normalized as an appropriate policy. But are there any ways that you can see, based on your results, where we could try to compensate for this, where we could try to offset this difference in grade change requests between men and women?

Cher: So in addition to making a transparent regarding policy, I’ll be arguing, first of all, we are not trying to push women to over asking, basically. And a lot of time, we also find in some other literature, they find when you pressure women to ask, the results are not always favorable. So that is truly not our intention. The point we make about transparency in terms of regrading is to make that kind of policy more formal, and sometimes maybe also imposing some cost in terms of the regrading in terms of like, the professor may be able to regrade the entire exam or the assignment, typically, would be, like, not enforced, when students are just bringing your attention to one particular question they think they should get more points and argue for it. But if they have to come through a particular procedure, for instance, making those requests through like documentation, or they have to make some arguments for the points they think they should earn, it would make it clear to students in terms of what they have to jump through, and then other policy that we think may be helpful is in terms of providing better signals. In terms of the performance, for instance, “How is the grade assigned? What is the metric? What is the rubric that a professor uses to assign a higher grade when the student has performed well?” and just really to show them like these are the guidelines why a student would get an A instead of getting a B. What are the distinctions between the different grades, for instance, to make it clear to students why they’re getting their grade, especially when the grading is based on something that’s more ambiguous, like essays or something.

John: So if you grade essays, make sure you provide a rubric that provides a clear delineation and makes a grading process more transparent to everybody.

Cher: Exactly. So when the women are getting a grade and they see, for instance, “I’m getting a B, but I’m qualified for getting an A based on the rubric,” then I will be more confident that I am eligible to ask the professor in terms of whether my grade is accurate or not.

Rebecca: I think it’d be interesting then, the next step would be to see if it’s not just gender differences in asking, but then also, if it’s students that grew up in lower income families or other kinds of situations that might be less likely to ask in the first place.

Cher: Yeah, I think that would be interesting, but we do not have that much information about student income, so we are unable to really look into it. But we do look into some of the different demographics, including their race composition, and we do not find much differences when we account for those differences. And it doesn’t seem to be the main reason that’s driving those gender differences, but there might be some other interesting questions that could be asked in addition to the gender disparity.

Rebecca: One of the things that always comes to mind is if you start putting in policies or things that make it more transparent, it still means that a student would have to take action in some way, so they would have to feel empowered to be able to take those actions. So it’s always curious to kind of think about those power structures beyond just gender.

Cher: Right, I agree.

John: With providing something in a syllabus for challenging grades would probably help reduce those based on what you said.

Rebecca: Especially because it establishes that behavior as something that would be appropriate in that context.

Cher: Yeah, exactly. Because when we ask students in our survey, “Why don’t you ask?” because we start asking them whether they ever consider asking their instructor to reconsider their grade at some point, and if they do, we follow up in asking for the courses they consider to make that request and the reasons they decided not to pursue it, and we find that a non-trivial fraction of students actually just never thought of it. [LAUGHTER] So they don’t think that is appropriate or that is available, so that is one of the reasons we believe it’s good to have a transparent policy in place.

Rebecca: So we usually wrap up our conversations by asking “What’s next?”

Cher: Okay, so the future, so right now our current research is going through the review process, and we have thought about a few directions that we can follow up on the study. One direction is to look into the gender interaction between students and instructors, and we got this question a lot when we presented our work. And a lot of people are curious whether female instructors are getting more requests, and what are the results when you consider those gender interactions? And I think that could be one of a very interesting to look into. And one very interesting thing, I think, from our current study is, when we design our lab experiment, we intentionally take those mechanisms out. So, in our lab design, we shut down a lot of potential mechanisms that could be very important in real life. For instance, the fear of backlash when women ask, which I have found some evidence in current literature, that is totally shut down in our lab experiment. So that is not the reason that’s explaining the gender differences in terms of asking, based on our lab experiment. And we also shut down the gender interactions because the participants in our experiment do not have to interact with a real person. So they just have to make that decision whether they want a regrade or not, and their wish will be granted, basically. And that also shut down any potential gender differences in terms of negotiation skills, if men have been negotiating all along, since they were little, they may become better at it, and that may be contributing to the gender gap in that case.

Rebecca: Those sound like some really interesting follow-up studies.

Cher: Yeah. And we were also thinking about, one very cool thing that could be done, is to follow up with the students into their labor market experience and see how their negotiation behavior may actually lead to some differences in their labor outcomes. Are people who tend to negotiate their grade in school are getting better outcomes when they are working for others, for instance.

John: And I know many alumni programs collect data on first salaries out of college, and that might be an interesting place to do that linkage since you have the administrative data there.

Cher: Exactly.

John: Excellent. I really enjoyed your study, and I like the way you designed the experiment to elicit some detail on why this is happening, because the studies I’ve seen, mostly… just in terms of the labor market… just report that this is occurring, but really haven’t done as much, other than the study you mentioned with List and others, haven’t done that much to try to figure out why it’s happening. So I really enjoyed it.

Cher: Thank you so much. We enjoy the research ourselves, too.

Rebecca: Well, thank you so much for joining us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Brittany Jones and Savannah Norton.

[MUSIC]