Grade change requests in college are relatively rare, but when they do occur, evidence suggests that male students make the request more often than female students. In this episode, Dr. Cher Li joins us to discuss these gender differences in grade change requests in college and why they might occur.
Cher is an assistant professor of economics at Colorado State University. Her research focuses on how public policies and social institutions affect the decisions of, and outcomes for, women. She is also a co-author of a January 2020 National Bureau of Economic Research working paper that investigates gender differences in grade changes.
Show Notes
- Li, C. H., & Zafar, B. (2020). Ask and You Shall Receive? Gender Differences in Regrades in College (No. w26703). National Bureau of Economic Research.
- Colorado State University
- Leibbrandt, A., & List, J. A. (2015). Do women avoid salary negotiations? Evidence from a large-scale natural field experiment. Management Science, 61(9), 2016-2024.
- John List, the Kenneth C. Griffin Distinguished Service Professor in Economics and the College at the University of Chicago.
Transcript
John: Grade change requests in college are relatively rare, but when they do occur, evidence suggests that male students make the request more often than female students. In this episode, we discuss these gender differences in grade change requests in college and why they might occur.
[MUSIC]
John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…
John: …and Rebecca Mushtare, a graphic designer.
Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.
[MUSIC]
John: Our guest today is Dr. Cher Li. Cher is an assistant professor of economics at Colorado State University. Her research focuses on how public policies and social institutions affect the decisions of, and outcomes for, women. She is also a co-author of a January 2020 National Bureau of Economic Research working paper that investigates gender differences in grade changes. Welcome, Cher.
Cher: Thank you. It’s great to be here.
Rebecca: Today’s teas are:
Cher: I have Oolong tea from the Ali Mountain area in Taiwan. It is one of the best tea regions in Taiwan.
Rebecca: Sounds good.
John: Nice. And I have Cranberry Blood Orange black tea.
Rebecca: And I have a mango black tea today. John’s looking at me a little strangely because it’s very unusual for me. [LAUGHTER]
John: We’ve invited you here today to discuss your recent working paper with Basit Zafar on regrading requests and outcomes. Could you tell us a bit about the study?
Cher: Yes, this study is about gender differences in grade changes in college, and we look into the differences and why are the causes for the gender differences?
John: What prompted you to look into this?
Cher: So this research is actually motivated by my personal experience. In the last six years, I had many male students, but only one female student asking me to bump up their grade at the end of the semester. And I know from previous studies that men are more likely than women to negotiate for their salaries and promotions, and that made me wonder if these kinds of differences actually started earlier when they’re still in college. So, that is the motivation for this study.
Rebecca: Can you talk a little bit about how you conducted the study?
Cher: Sure. We tackled this research using three very different sources of data, including administrative records from a very large four-year university and we also use surveys and laboratory experiments. So, the first data source is a very unique administrative data from Colorado State University between 2010 and 2016. We have over 1.3 million observations of grade changes across all the departments, and this data set is very unique because it recorded not just the final grade, but it also recorded any changes associated with it. So we focus on the changes made by instructors, and our analysis showed that, although grade changes are pretty rare events, 95% of the time they will change to a better grade, and among those positive changes, men are about 18.6% more likely than women to get a better grade. And the difference in grade changes cannot be explained by the characteristics of students, instructors, and classes. However, the administrative record has several limitations. First of all, it doesn’t provide information on the cases when any regrade request was rejected by instructors, and it also doesn’t tell us about the grade changes during the midterm or any assignment throughout the semester. And finally, it doesn’t tell us “What is the scenario that caused the gender difference in grade changes?” So we can think about at least three different scenarios that all lead to the same data pattern. For instance, the first one is male students are simply more likely to ask, although instructors treat men and women equally… so, that is a possibility, given the previous literature evidence. And the second potential scenario is that male and female students are equally likely to ask, but somehow instructors are treating men more favorably, and if it is this scenario, we’re going to have something to worry about about discrimination. And the third scenario we thought about is, there may be a difference in terms of the timing of asking. So if men procrastinate while women are more organized throughout the semester in they ask their instructor to make changes while they find any errors during the semester, then they have no need to ask at the end of the semester. So, to examine the scenarios, we conducted two surveys, one on instructors and the other one on students, and we checked whether the reports are very similar across different surveys. And surprisingly to us, both surveys show pretty consistent patterns: that male students are indeed more likely to ask, and we find no evidence that instructors are treating males students more favourably. And we also didn’t find any support that there is a timing difference in terms of their asking.
John: That’s certainly consistent with what I’ve observed myself, that males have been more likely to ask both during the semester and after.
Rebecca: I don’t think I’ve gotten that many requests, generally.
John: Well, I haven’t either, I may get one a year or two a year out of like seven or eight hundred students, but…
Rebecca: I can’t think of a single one. I know that I have but I can’t think of a particular…
John: Well, your classes are smaller, too. So, that could be a factor.
Cher: Yeah. So I also typically only got like one to two students per semester at making that request. So it jumps out at me when there is only like one female student throughout the whole six years of my experience, that really makes a difference to me when I compare the frequency in terms of they approach me at the end of the semester.
John: One of the nice things I liked about it was your incentivized controlled experiment that looks at why there may be differences. Could you tell us a little bit about that aspect of the study?
Cher: Definitely. So focusing on the gender difference in asking, we did this lab experiment to answer that question, in particular in the lab setting. The participants basically take a quiz of 20 questions, and their payoff is dependent on the grade in the quiz. So we tell them we’re going to give them the grade. However, we also tell them the grade is noisy. So, we randomly picked three out from the 20 questions to randomly grade them. The grade we tell them could be their true grade, but it could also be higher or lower than the true grade. And then they have to decide under 10 different cost scenarios, ranging from paying up to $3.50 to being paid to $1, whether they are willing to pay the cost in each of the scenarios to get a regrade. If a participant chooses to pay the cost to ask for a regrade, then the true grade is revealed and their payoff is adjusted and we subtract the cost they have to pay, and if a participant chooses not to ask for a regrade, then the initial grade becomes final and they will be paid accordingly. And we find that, as long as the cost is positive, so basically the participants have to make a payment, men are always more likely than women to pay the cost to ask for regrades, and the gender difference in asking cannot be explained by their difference in risk aversion. We find that other majors, including their difference in confidence level, in their uncertainty about their performance, and personality traits explain nearly half of the gender differences in asking.
John: How accurate were the students’ metacognitions in terms of how appropriately they made a decision to request regrades? Did they gain by that on average, or did they lose?
Cher: So in general, they have pretty good guesses about their expectations in terms of how well they do in the quiz. However, their decision here really depends on their attitude in terms of pursuing more aggressively in the regrade or not. And we find that, in general, most students who decided to pursue the regrade were likely to get a better outcome, and we find that depends on the cost, basically. So when the cost is low, the more likely a student will get a better outcome, because they are facing a very low cost to offset the gain they got from a better grade. And the reason that they are more likely to get a better outcome is due to our lab experiment design, we make it asymmetric, basically. So, we are trying to mimic the real world scenario. So from the administrative data, we find that 95% of the time, students got their grade change to a better grade. Only 5% of the time it got changed to a worse grade. We intentionally designed our lab experiment to allow that asymmetry. So basically, when a student asks, they will have a higher chance to get a better grade than to get a worse grade.
John: How much more common was it for males and females at low, and then again, at high prices?
Cher: When there is a cost that really they have to pay out of their own pocket, around like half of male students are willing to pay the cost, while only about one third of women are willing to do so. However, when the cost is zero or they got paid to do it, it’s very similar. I think their percentage of requests is very high. It’s like nearly 80% when they were getting paid $1 to do so.
John: What happened when the cost was high? I believe you found some interesting effects for males at that.
Cher: So when the cost is high, we find that the gain from asking actually disappeared, because now the high cost offset the gain made by men. So although men are more likely to ask, they are not always made better off, so they are only gaining relative to women when the cost is relatively low. When the cost is above like $1.50, we see that men are also more likely to see their payoff decrease compared to women.
Rebecca: Can you talk a little bit about the gender difference in the regrade requests and what might explain those differences in general after looking at those three different approaches that you’ve taken to study it?
Cher: So we find that, it seems to be, there are some subjective beliefs that are different to begin with. So first of all, there are some evidence from other studies that men and women are different in terms of confidence level, and men tend to be more overconfident than women in a lot of contexts. So, we explore that too in our study, and we find that indeed, when we look into each of the questions they answer, we also ask them how confident they are on each of the questions. And we find that when they got the answer wrong, men and women are similarly over optimistic about their accuracy. However, when their answers are actually correct, women are much less assertive about it, so they are less confident about their correct answer. So that is part of the reason that is driving the gender difference in asking, and the other factor that we found are important in terms of the gender difference in asking is about their perception about the potential outcome, so basically the uncertainty they face when they make that decision. So we asked them about like, “What is the number of questions you think you got right?” And they will give us their best guess. And then we ask them then for that scenario, “What is the likelihood you think that’s gonna be happening?” So they will tell us like, oh, if they are very confident about “I’m getting 15 questions right out”, maybe assigning 100% as the probability for that outcome, if I’m super confident that is the outcome. But then if I’m not very confident about my guess I maybe say “Okay, 50% probably,” then I’m not really certain about that outcome. So we find that when men and women are giving us those guesses, women are much less certain about their guess. So they are likely to think it could be different outcomes, maybe it’s 14 or 13 questions they got it right, so they would also assign the probability of other outcomes that they anticipated. And that also explained a very large proportion of the gender gap, and we also find there are some differences in their personality traits. For instance, women are more agreeable than men, so they may be more agreeable to the grade they received and less likely to challenge it in terms of the context of asking.
Rebecca: When we start thinking about the impact this might have on GPAs or their academic standing, how does this really impact men and women differently?
Cher: So the overall impact on GPA depends on how we measure it. For those people who ask, we did a back of an envelope calculation and we find that for the female students, the accumulative effect from asking is up to .43 points in their overall GPA. But for male students, the number is a bit higher, they could receive up to .47 points as an improvement in their GPA, conditional on asking.
John: Which is a non-trivial difference in terms of that, especially if you were also to extrapolate and think about the possibility that this is also occurring during the class.
Cher: Yeah, we take that into account in extrapolating, and that is a cumulative kind of effect we anticipate. If a student reports a fraction of courses they asked for a regrade, that would be kind of cumulative effect overall.
Rebecca: Do you have any idea of what the not asking is doing, maybe to women, if they’re less likely to ask in the first place, how that might be negatively impacting their GPAs?
Cher: So in terms of not asking, you are comparing to those who ask, so that raw difference could be low depending on who we are comparing to, or we can compare it to those who actually are successful throughout their time, then that difference will be up to like .4 to .5 points in terms of the GPA.
John: At the start of this, you talked about how there’s a lot of evidence out there that women are less likely to bargain for higher wages for promotions and so forth. From those studies, is there a lot of evidence of similar types of causation?
Cher: So most of the studies we saw before look into their different tendency to ask, and one of the particular research I think is very interesting is in terms of the context. In the study made by other economists, including John List in University of Chicago. In their study, they did a field experiment, and they experiment the job applicants’ attitude in terms of negotiation for their salary. In their experiment, there are two settings, one is the salary is ambiguous if there is no hint or any information about it. And then they found under that context, men still negotiated for their salary, but women in that context tend not to negotiate and instead, they signal their willingness to accept a lower offer. In the separate setting, when they make it clear in their advertisement that the job salary is negotiable, then there is no gender difference in terms of asking; men and women are similarly likely to negotiate their salary when that kind of information is clear, that the social cue is you are able to negotiate. In that case, there is no gender disparity, and women are similarly likely to ask. So we find it relevant in our context as well because a lot of time professors do not put a clear policy about regrading, so students may have to take that into their own guess, whether this is an appropriate behavior to make and approach their instructors about it. So I found that kind of ambiguity is relevant in terms of the context when the grade change policy is ambiguous.
John: I’d be really hesitant to put in a procedure explicitly for negotiating grade change requests, but I could see how that could make a difference when it’s normalized as an appropriate policy. But are there any ways that you can see, based on your results, where we could try to compensate for this, where we could try to offset this difference in grade change requests between men and women?
Cher: So in addition to making a transparent regarding policy, I’ll be arguing, first of all, we are not trying to push women to over asking, basically. And a lot of time, we also find in some other literature, they find when you pressure women to ask, the results are not always favorable. So that is truly not our intention. The point we make about transparency in terms of regrading is to make that kind of policy more formal, and sometimes maybe also imposing some cost in terms of the regrading in terms of like, the professor may be able to regrade the entire exam or the assignment, typically, would be, like, not enforced, when students are just bringing your attention to one particular question they think they should get more points and argue for it. But if they have to come through a particular procedure, for instance, making those requests through like documentation, or they have to make some arguments for the points they think they should earn, it would make it clear to students in terms of what they have to jump through, and then other policy that we think may be helpful is in terms of providing better signals. In terms of the performance, for instance, “How is the grade assigned? What is the metric? What is the rubric that a professor uses to assign a higher grade when the student has performed well?” and just really to show them like these are the guidelines why a student would get an A instead of getting a B. What are the distinctions between the different grades, for instance, to make it clear to students why they’re getting their grade, especially when the grading is based on something that’s more ambiguous, like essays or something.
John: So if you grade essays, make sure you provide a rubric that provides a clear delineation and makes a grading process more transparent to everybody.
Cher: Exactly. So when the women are getting a grade and they see, for instance, “I’m getting a B, but I’m qualified for getting an A based on the rubric,” then I will be more confident that I am eligible to ask the professor in terms of whether my grade is accurate or not.
Rebecca: I think it’d be interesting then, the next step would be to see if it’s not just gender differences in asking, but then also, if it’s students that grew up in lower income families or other kinds of situations that might be less likely to ask in the first place.
Cher: Yeah, I think that would be interesting, but we do not have that much information about student income, so we are unable to really look into it. But we do look into some of the different demographics, including their race composition, and we do not find much differences when we account for those differences. And it doesn’t seem to be the main reason that’s driving those gender differences, but there might be some other interesting questions that could be asked in addition to the gender disparity.
Rebecca: One of the things that always comes to mind is if you start putting in policies or things that make it more transparent, it still means that a student would have to take action in some way, so they would have to feel empowered to be able to take those actions. So it’s always curious to kind of think about those power structures beyond just gender.
Cher: Right, I agree.
John: With providing something in a syllabus for challenging grades would probably help reduce those based on what you said.
Rebecca: Especially because it establishes that behavior as something that would be appropriate in that context.
Cher: Yeah, exactly. Because when we ask students in our survey, “Why don’t you ask?” because we start asking them whether they ever consider asking their instructor to reconsider their grade at some point, and if they do, we follow up in asking for the courses they consider to make that request and the reasons they decided not to pursue it, and we find that a non-trivial fraction of students actually just never thought of it. [LAUGHTER] So they don’t think that is appropriate or that is available, so that is one of the reasons we believe it’s good to have a transparent policy in place.
Rebecca: So we usually wrap up our conversations by asking “What’s next?”
Cher: Okay, so the future, so right now our current research is going through the review process, and we have thought about a few directions that we can follow up on the study. One direction is to look into the gender interaction between students and instructors, and we got this question a lot when we presented our work. And a lot of people are curious whether female instructors are getting more requests, and what are the results when you consider those gender interactions? And I think that could be one of a very interesting to look into. And one very interesting thing, I think, from our current study is, when we design our lab experiment, we intentionally take those mechanisms out. So, in our lab design, we shut down a lot of potential mechanisms that could be very important in real life. For instance, the fear of backlash when women ask, which I have found some evidence in current literature, that is totally shut down in our lab experiment. So that is not the reason that’s explaining the gender differences in terms of asking, based on our lab experiment. And we also shut down the gender interactions because the participants in our experiment do not have to interact with a real person. So they just have to make that decision whether they want a regrade or not, and their wish will be granted, basically. And that also shut down any potential gender differences in terms of negotiation skills, if men have been negotiating all along, since they were little, they may become better at it, and that may be contributing to the gender gap in that case.
Rebecca: Those sound like some really interesting follow-up studies.
Cher: Yeah. And we were also thinking about, one very cool thing that could be done, is to follow up with the students into their labor market experience and see how their negotiation behavior may actually lead to some differences in their labor outcomes. Are people who tend to negotiate their grade in school are getting better outcomes when they are working for others, for instance.
John: And I know many alumni programs collect data on first salaries out of college, and that might be an interesting place to do that linkage since you have the administrative data there.
Cher: Exactly.
John: Excellent. I really enjoyed your study, and I like the way you designed the experiment to elicit some detail on why this is happening, because the studies I’ve seen, mostly… just in terms of the labor market… just report that this is occurring, but really haven’t done as much, other than the study you mentioned with List and others, haven’t done that much to try to figure out why it’s happening. So I really enjoyed it.
Cher: Thank you so much. We enjoy the research ourselves, too.
Rebecca: Well, thank you so much for joining us.
[MUSIC]
John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.
Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Brittany Jones and Savannah Norton.
[MUSIC]