The “One Big, Beautiful Bill” contains a provision eliminating federal loan availability for programs in which past graduates had early career earnings that fall below a designated threshold. In this episode, Debbie Furlong joins us to discuss the statistical measures used to determine federal loan eligibility.
Debbie has served for over 30 years doing applied higher education policy analysis at public comprehensive universities, most recently as the Director of Institutional Research at SUNY Oswego. Prior to her arrival at Oswego, she worked and taught at the University of Wisconsin at Green Bay, St. Norbert College in Wisconsin, and the American University in Washington, DC, where she had completed her PhD degree in international relations.
Show Notes
- One Big Beautiful Bill
- College Scorecard
- Carnegie Classification
- American Council on Education (ACE)
- National Student Clearinghouse
- Mincer, Jacob (1974). Schooling, experience, and earnings. Columbia University Press.
- Hause, J. C. (1980). The fine structure of earnings and the on-the-job training hypothesis. Econometrica: Journal of the Econometric Society, 1013-1029.
- IPEDS
- National Survey of Student Engagement (NSSE)
Transcript
John: The “One Big, Beautiful Bill” contains a provision eliminating federal loan availability for programs in which past graduates had early career earnings that fall below a designated threshold. In this episode, we explore the statistical measures used to determine federal loan eligibility.
[MUSIC]
John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…
John: …and Rebecca Mushtare, a graphic designer…
Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.
[MUSIC]
Rebecca: Our guest today is Debbie Furlong. She has served for over 30 years doing applied higher education policy analysis at public comprehensive universities, most recently as the Director of Institutional Research at SUNY Oswego. Prior to her arrival at Oswego, she worked and taught at the University of Wisconsin at Green Bay, St. Norbert College in Wisconsin, and the American University in Washington, DC, where she had completed her PhD degree in international relations. Welcome, Debbie.
Debbie: Hi.
John: Thanks for joining us. Today’s teas are:… Debbie, are you drinking any tea today?
Debbie: I am waiting for my tea to cool off. It’s a decaffeinated green tea, because it’s too late in the day for me to have any caffeine.
Rebecca: Green tea is a nice choice at this time of day. I have Blue Sapphire today.
John: Back to an old standby, and I have ginger peach green tea today.
Debbie: I’ve had a ginger peach tea. It’s some of my favorite flavors.
John: It’s one of mine too. This is a Republic of Tea variant. Rebecca particularly likes flavored teas.
Rebecca: It always smells good. [LAUGHTER]
Debbie: Yes, it does.
Rebecca: We’ve invited you here today to discuss the growing focus by the federal government on using earnings data on students who evaluate the success of college programs. The One Big Beautiful Bill Act prohibits universities from awarding federal loans to students in programs where its former students have earned low wages. Can you tell us a bit about how we got to this point?
Debbie: Yes, it’s been a long trip down. Some might say the government has been providing post-enrollment earnings data for several years now through a tool called a scorecard. Assessing the quality and value of a college degree in terms of students’ earnings has been growing in popularity for a long time. Increasingly, we’ve all talked in higher ed about people viewing their degree in terms of what it’s done for me as a private good, as opposed to focusing more on the public good of education. We can see data where this trend has been growing for 50 years. I think that trend was really exacerbated when public universities, when states started having to trim public funding, and parents and families were bearing more and more of the cost, and when the price, the stated prices were going up, up, up, this trend became even more predominant. Universities really have not helped at this, because, like, our marketing areas for recruiting have really leaned into the job placement as something that needs to be touted as an important thing, so we’re all sort of involved in telling students they should pay attention to how much they’re going to earn. I think the original interest of the federal government – I’m just speculating… based on timing and the administration that was in play at the time we first got the scorecard… I think the original interest was sort of a consumer protection kind of position. We had had an explosion of for-profit schools that really had some very serious problems. Students were not completing the degrees, and even when they were, they were unable to earn enough to pay back their loans, and the loan default rates were going up for these schools. So it was like, “Well, let’s protect the consumers,” so about 10 years ago, when Obama was actually still in, the Department of Ed launched the College Scorecard, and the College Scorecard was really the first time that families and students could look at a university’s profile and see: this is the graduation rate, this is how much it cost, this is what percent of students get loans, and here’s the average income a certain number of years after graduation. And here’s the percent of students that are paying back loans, so, bringing it all together. Where we’re at right now, the new state that we’re in right now is the Department of Ed looking and saying, “Gosh, we have all this data. Let’s begin using it to stop supporting programs where students are not earning enough money. Since we have all the data, we should protect not just the consumer, but protect the taxpayer. The taxpayers shouldn’t be asked to pay through Pell grants and through loans for programs where students aren’t even able to earn as much as they could earn if they went and got a high school degree.” That’s how we got where we are, and that’s where we’re at. I hope you can hear some of the passion of my voice about this stuff. [LAUGHTER] I really think faculty and staff need to understand where the data come from and understand how we’re all in this together, and to feel confident to go and explore and learn on their own.
John: So, this started off as something designed to protect consumers, but it’s moved on to something a little bit bigger, and because of that, it’s become increasingly important to understand the data, and you spent a fair amount of time researching the data and how it’s measured, and you’ve raised some concerns with it. Could you talk a little bit about the primary sources of data first?,
Debbie: Yeah, there’s two really big national data sets. One is from the Department of Education, and one is provided by a group called the Carnegie Classification, which is a branch of the Carnegie Foundation, which is a nonprofit. The Carnegie Foundation has teamed up with the American Council on Education, ACE, which is a big nonprofit player in the higher ed sphere. Both of these organizations are providing data that is publicly accessible and downloadable, and they both have a website search engine that sits on top that enables you to look up one school or to compare a few schools, but the data that’s running those engines is fully accessible. So this winter I was bored, and I got in, and I started playing. It’s like I just really want to learn how they’re going to do all this, and what does this stuff all mean. So the College Scorecard is the most important resource. Like I said earlier, the Obama administration really launched the scorecard, and this was the first time that consumers could see, for one institution, all the stuff about debt and debt repayment and post-enrollment earnings, and they could see those metrics right next to the data about the tuition and the costs. Department of Ed could not have done this without technology advancing to the point that we could really wrap our minds around really truly big data sets. This downloadable data set I just referred to has over 3500 variables, so it’s not for the faint at heart. In order to measure graduates’ earnings, or the earnings of formerly enrolled students, because both groups are looked at. In order to do that, you got to have two things: you have to have tax records that indicate was the person employed, were they employed full time, and what was their income, their earnings, and you have to have something with a matching field that says what school did this student go to, did they graduate, and if they graduated, what field? No single university has this. So, the way this works is the Department of Ed has the student identity information from its loan and grant recipients, and all the earnings data is only measured for students who have taken out federal financial aid, either through a loan or by getting a Pell Grant. So, the Department of Ed has these files that have individual identification and information about whether the student graduated and what field. They send these files over to the Treasury Department. The Treasury Department does a merge and match with their records and sends the information back to the Department of Ed. As a little sidebar, we’re going to be talking in more and more depth today about this, and the fact that we’re limited to just looking at loan and grant recipients is a significant limitation. The sidebar I’ll drop in right now is I am surprised the Department of Education has not yet asked all of us to provide full identification records for all of our enrolled students. The reason I’m surprised they haven’t yet done this, there’s two things, [LAUGHTER] the first one is the Department of Ed did ask for student identified records earlier this winter when they started asking for information about access, information so that they could make sure that students who are black or low income were not being admitted when they had lower test scores and high school grades. The data file that the Department of Ed requested this winter is student identified, so they’ve gotten over their concern about privacy issues. [LAUGHTER] The second reason I’m surprised is they’ve been asking states to provide voter record identification, so I think it’s only a matter of time. The other reason, there is another organization out there that’s not really related to this topic we’re talking about, the National Student Loan Clearinghouse, or National Student Clearinghouse. The NSC has student-identified data, and I don’t see how the Department of Ed is going to feel too happy knowing somebody else has it and they don’t have it. So, I think it’s only a matter of time before that little limitation goes away. So, the Department of Ed sends these files over to Treasury, they get back the earnings data. The second data set out there has used the Scorecard file, and this other data set is the . I’m going to be referring to this as access and earnings, or more just Carnegie, the Carnegie framework. The Carnegie framework about access and earnings was released about a year ago, but it was in development for quite a bit longer. Carnegie’s real contribution that’s relevant for what we’re talking about today is they began trying to deal with the fact that in the scorecard the only earnings that people were getting compared to were national, and more recently, statewide data, and Carnegie’s framework argues that earnings for a school’s students should be compared to the earnings of high school graduates in the geographic areas where those students come from, not necessarily to the whole nation, and not necessarily even just to the state where the school is located. This issue about adjusting the earnings to reflect regional differences in incomes is definitely confounding the Department of Ed right now. The OBBBA is in negotiation. There is a negotiated rule making going on right now, and one of the last things that the Department of Ed is trying to figure out how to deal with before they roll out this framework, where if your program’s earnings are too low, you can’t get loans. The last issue they are mucking with is the regional differences in earnings, and I don’t know how they’re going to do it. The Carnegie folks did it one way and sort of got the ball rolling. Carnegie’s real contribution here, I think, is showing how incredibly difficult this is going to be. We’ll have to see, we’ll have to see how they do it, and it may take a few years before they really figure out how to most appropriately set a benchmark for earnings for every program in every school.
John: So, are you saying that the earnings of people in rural Alabama might be different than Manhattan? [LAUGHTER] Who would have thought?
Debbie: They are. Yes.
John: Among the other problems there that you’ve noted is that there’s a fair amount of sample selection in the sample that they’re working with, and you’ve already referred to one aspect of that. Could you talk a little bit more about that, and how that might affect the validity of the earnings measures?
Debbie: Yeah, yeah, so what John just referred to is the fact that only students who get federal financial aid are currently included, and for many of us that doesn’t seem like very much of a limitation, because so many of our students have to get loans and a huge percent of students get Pell grants, but there are a bunch of schools out there where those percentages are really, really small, and I went and I looked up Harvard’s data, just because everybody knows Harvard, you know, Harvard, Harvard. And I went and looked at Harvard because I anticipated Harvard students would not need to rely a lot on federal aid, not because I think Harvard students are rich or poor, I don’t particularly know, but Harvard is very, very wealthy, and they’re able to provide a lot of institutional aid, which keeps those students out of this federal tracking, tying together with your Treasury data. So I looked up Harvard. In a recent year, 20% of their students got Pell grants, and 4% of their students took out loans. So by the time that cohort gets through the system, only a quarter of the students would have their wages reflected in these median wages. I don’t really know what this is going to mean. Is this good? Is it bad? It makes me feel uncomfortable about the validity of the data. Another thing that I know with this coverage, I have been watching that the percent of students in the United States agreeing to take out loans fell after COVID and did not rebound. I think this past year they have started rebounding a little bit. What this means is that the coverage overall, nationally, school after school after school, this coverage rate is going to be going down for the next several years. So, this is another reason why I am expecting the Department of Ed to be knocking on everyone’s doors very soon, saying: “The gig is up. We want a student-identified file. We want to know everyone who’s enrolled every semester. Here’s the portal. Drop the data.” Another issue that kind of deals with coverage that gets very confusing has to do with the timing, and this is kind of devastating to me, is a validity issue. And what I mean by timing is, how long do you wait after a student has left the institution, or after a student has graduated. How long do you wait before you go and grab their earnings data? Okay, so timing issues can be very confusing, and what I would really encourage anybody who’s paying attention to this stuff, don’t try to sink into your mind exactly how these things are measured. When you go to one of the websites, look for the information notes, they will tell you exactly what they’re measuring. Carnegie’s Access and Earnings framework originally relied on a measure… they looked at students’ earnings eight years after the student initially enrolled, and there are reasons why they made that choice. One of the things that higher ed has been deeply criticized about is the poor retention of students after the first year, and there’s a huge amount of tax money that goes to students who are at the institution for one year, and then they never really do anything with that education. So looking at students based on where they started is not stupid, but it is kind of misleading. I think when people say, “Oh, here’s Cornell’s earnings data,” I think people assume that’s the earnings data of students who graduated from Cornell [LAUGHTER] or from whatever school, and the current version of the scorecard has flipped over and is relying almost exclusively on earnings data measured four years after the student graduated. So the cohorts are now measuring students who actually completed the degree, which is probably more valid for what people actually think they’re looking at, and they give students four years, so now it’s like, okay, four years… is four years long enough to get a valid measure of a student’s earnings potential? I think this is a huge limitation. Many of the highest earning fields, and I’m thinking like medicine and law, those are going to be either really hard or downright impossible for a student to complete the whole thing and be out and earning. So a lot of fields where students go on to complete masters and doctorates, those students are ultimately going to be the highest earners, but they’re not going to be earning that stuff four years out, so I was thinking about, like, where I was four years after my undergraduate degree. I had gotten my first full-time job, but it was just a job to pay the bills, because I was working on a PhD. So, was I earning more than a high school student? Maybe. I don’t know. I was mostly a graduate assistant, getting paid a graduate fellowship. Even fields where students don’t go on for advanced degrees, I think that there’s a lot of research that indicates people who complete a baccalaureate degree will ultimately have higher lifetime earnings, even if they don’t get an advanced degree. We know this lifetime earning data exists, so is four years long enough to predict the lifetime earnings of a particular person in a particular field of study from a particular university? I think it’s a big validity question. The last thing that I wanted to mention is this timing, all this stuff about the validity of the timing of when the data are grabbed, it’s grabbing data right smack at an important time for young families that are deciding to have kids. I looked up before coming today, because I knew I wanted to make this point, because I think it’s important, the average age for women in the United States who have had children at this time, the average age of when they had their first baby was 26 that’s just right in this four years… you graduate when you’re 21 or 22 So I mentioned this because it’s very common for young families, for either the husband or the wife, the mom or the dad, or one parent, to back off a little bit from working full time or working at the high pressure job. So there’s a lot of problems with the data.
John: Going back to what’s referred to as the Mincer equation, which was first estimated back in the early 1970s. economists have been estimating the overtaking age, the age at which the earnings of college graduates overtake the earnings of high school graduates with equivalent demographic and ability characteristics, and back then was about seven years after graduation. More recent studies suggest that it varies a fair amount by discipline and by occupation, but it’s generally between six and eight years after graduation from college, which means on average the fact that people are earning more than high school graduates puts them way above where you would expect them to be had they not gone to college, or people with similar characteristics would receive had they not gone to college. And so it’s really troubling that they came up with a metric that’s so evidence-free, basically.
Debbie: Yes, [LAUGHTER] problematic.
John: And also, you mentioned law school graduates, so they might be a year out of law school, but some of the people who come out of law school are going to be working with large firms for ridiculously low pay, and those are the people who are going to have the highest rate of growth of earnings. Other people will start their own practice and will have higher starting salaries than those who go for those large law firms, at least in terms of hourly wages, they’ll be paid a lot less, but they will end up with a slower rate of growth of earnings, and there’s another classic economic study back during this period on on-the-job training and the fine structure of earnings, which notes that holding other things constant, people in the same occupation with lower starting salaries tend to have higher rates of earning growth over time, because part of what they’re picking up on the job are some skills that will improve their productivity, and they’re essentially accepting lower wages during those times in return for the increase in training or human capital development they get on the job, which is certainly true in medical school, because then they’re starting their residencies, where they’re paid again ridiculously low amounts, but they end up with much higher earnings just a few years later.
Rebecca: I knew John would be passionate. [LAUGHTER]
Debbie: Yes. And everything he said is true today, the research is still valid and important.
Rebecca: One of the other things that you mentioned is those regional variations that Carnegie was developing, or kind of initiated. Can you talk a little bit about these regional variations, and like, what does their framework mean? How did they do it? How maybe should they do it? [LAUGHTER]
Debbie: Yeah, so like I said, when they first put the scorecard out, my perception is that was kind of a consumer protection thing, like here’s the tuition, here’s the percent of students who have loans, here’s the percent of students who have paid back their loans, and here’s the average amount that their graduates earn, without judging whether or not that average was adequate or met a benchmark. Well, those days are behind us. The OBBBA wants to establish this benchmark of acceptable earnings, and the negotiation is moving forward. It seems very clear that they’re going to be using this high school degree population as the benchmark, but obviously people earn different wages based on different regions and different wages based on whether or not you’re in an urban area or a rural area. So the Carnegie framework is the first place I’ve seen where they tried to take this on. So let me tell you how they did it. A little segue, when Carnegie adjusted for regional variation, they have also adjusted for variances based on race and ethnicity. Carnegie’s argument was, if a student goes to a certain university, when they get done, that student should earn as much as high school graduates matched by the racial and ethnic distribution of the service area of that university and weighted to reflect the earnings differences of different racial groups, so they did this racial weighting. I’m not going to talk too much about that, because the OBBBA, the language I have seen, specifically says that the earnings benchmark cannot be adjusted for race. They want a race-neutral approach, just like they want race-neutral admissions. This is part of what we’re going through and working on with the Department of Ed right now. So let me talk about how they do the regional stuff. What Carnegie looked at, they said, “Okay, let’s look at where the entering class of a university is coming from.” The Department of Ed, for 20 or 30 years, has collected this data. It’s called a migration file. In the migration file every second fall, you’re not required to submit migration data every year… I don’t know why not… but in the file, the institutional researchers at the university have to go in and look and say, “Okay, how many people are in my freshman class, and where did they come from? How many of them came from New York State. How many of them came from Pennsylvania?” And the file goes through all of the states, so that data exists in IPEDS. IPEDS is the federal government’s collection of data about colleges and universities, and it’s a massive data set. So Carnegie said, “Well, gosh, since IPEDS has this migration data, let’s use that to figure out this concept of a service area.” What is the service region of this university? So, if a university has 90% of its students are coming from New York and 10% are coming from Pennsylvania, its earnings should be similar to the high school graduates of New York, weighted a little bit to reflect the fact that people from Pennsylvania have lower earnings. Now, so a 90%, 10%. In the Carnegie framework, it takes the top three states, so it says, okay, school X. If you had 50% of your students from one state and 40% from a second state and 5% from a third state, your earnings benchmark will be the weighted average of those elements, and then the remainder, the last little bit, we’ll just use the national data. This sounds kind of reasonable, I guess. Carnegie was very careful developing this. They got feedback, they got pushback from the two year schools saying, “Well, wait a minute, everyone at our university comes from one state, but they don’t come from all over the state, they all come from my county, or these two or three counties, because we have a lot of two-year schools out in rural areas.” This is a really serious issue, because the state earnings levels are very influenced by what’s happening in the big cities, and rural incomes tend to be much lower than urban incomes. So, Carnegie said, “Well, this is a reasonable complaint, so we’re going to back off, and we’re not going to use your state, where you have most people from. We’re going to use this core-based statistical area to gather your data.” There are a couple of other problems before we really get into the CBSA, and what that is. There are a couple of other things about relying just on the migration data that concerned me. My professional background was all in public comprehensive universities, and every place I ever worked had a huge pool of students who transferred into my universities, so I know transfer students may or may not be accurately reflected in this migration data in the schools where I’ve worked. In the systems where I’ve worked transfer students had a very different geographic distribution than new freshmen, and I think this is true for a lot of public comprehensive universities that the freshman class tends to come from all over the state much more, but the transfer students tend to be students who started at a nearby community college, and now they are your own community students, so that migration data that we’re using may not really be a great fit. The other thing that’s problematic is we are just assuming that the geographic distribution of graduates is the same as the geographic distribution of new entering freshmen. It may be, it may not be, and it may or may not matter, but these kinds of assumptions that are baked into the method, I think, cause people to feel a sense of discomfort about really feeling confident that the data really are measuring what we think they’re measuring,
Rebecca: So Debbie, I have another question. It seems like most of the data that you’re talking about is really focused on the states, so not necessarily international students and entirely undergraduates.
Debbie: Yes, and no, all of the earnings data is for US citizens only, and I will remind you that in order to have a student’s earnings get studied, they have to have gotten federal financial aid. International students are not eligible for Pell grants and for federal loans. So even if a former international student got OPT training and landed a job and became naturalized, we would have no way of knowing… the Department of Ed would have no way of knowing to tell the Treasury Department, “send me that person’s salary and earning data.” So universities that have huge international student bodies, especially if they’re doing really well, and supplying really fabulous students into high-need areas here in the United States. All of that is just kind of a loss. Now, your other question, like “Debbie, all you’re doing is talking about undergrad, undergrad, undergrad. What about graduate students?” The research that I really focused on this winter was about this Carnegie Access and Earnings framework, which is exclusively about the undergraduate experience. The scorecard data does include earnings data for graduate students, and the OBB BA work going on right now is also going to be relevant for graduate students. Graduate students are eligible to get federal loans in certain circumstances, and those loans will be cut off if a student gets a graduate degree from a school where their program is not earning on average as much as baccalaureate earners in whatever state or region the OBBA decides to look at, so it’s the same kind of framework, notched up one level.
Rebecca: The other thing that I was curious about is the data that you were talking about, is where students are coming from initially, which may not be where they are working.
Debbie: Yeah.
Rebecca: Am I right about that? [LAUGHTER]
Debbie: Oh, yeah. [LAUGHTER] And I’ve got a bunch of notes about this thing, and I definitely wanted to hit this. So you’re jumping into something that I already also agree is really a problem. Two issues: do students go back to where they came from? No. The other thing that’s really problematic, I think, is that the earnings data are all aggregated on the basis of what the student studied, not what they’re actually doing. So, I was thinking in my head, like examples, I know your field is art and design, design arts. Someone who studies design arts and graduates with an art degree and goes and does marketing in New York City is going to have a very different earnings experience than someone who completes an art degree at the same school and stays in that local area and runs art programs for the YMCA. These are both wonderful jobs, both jobs that would require an art degree to be really effective, but they’re incredibly different fields. I mean, one is marketing and the other one is social services, and that’s not accounted for. So, what a person’s actually doing with their degree is not part of the mix.
John: You talked a little bit about these core-based regions that are used for measuring geographic effects. What are they, and could you tell us a bit more about how they’re defined, and whether there’s some problems with the use of those.
Debbie: Yeah, the two year schools kind of pushed back and said statewide data doesn’t make any sense for us. So the Carnegie Foundation said, “Okay, the regional adjustment for two-year schools, we should use the data from the closest geographic region to their service area.” The Census Bureau has these geographic entities called core-based statistical areas. A CBSA is a geographic area that contains a large urban center, and then the surrounding areas that are kind of integrated economically with that urban center. Like, I live in Oswego, New York, which is about 30 miles north of Syracuse. I’m in the Syracuse CBSA, which makes perfect sense. There’s so much stuff that we depend on Syracuse for, and people go back and forth there all the time. So, Carnegie said, “Okay, let’s use core-based statistical areas for the two-year schools, and let’s also use them for these public regional universities.” There’s this category of universities called a regional public university, or an RPU, which is most public universities that are not focused on research. So, in New York, Stony Brook, and Albany and SUNY Buffalo, those are all research-focused institutions, they’re not RPUs. Almost all of the other four-year public universities in New York are considered RPUs. So, Carnegie said, “Well, they’re called regional, they must be serving their region. Let’s do all of our benchmarking for them based on the core-based statistical area.” Well, I really dug into this with my research, because the truth of the matter is most of these regional public universities are not recruiting heavily from our own regions. At most, RPUs would have maybe 20 to 30% of the students from their region. We’re all recruiting heavily in the big urban areas, because that’s where the students are. This is the whole piece of this enrollment cliff. Not only is certain states and regions losing students, but the students who are left are in the cities, and if we want to maintain universities out in the smaller communities, we’ve got to move the students out into these smaller communities. So I did some actual sensitivity analysis to see, does it matter? Does it impact the outcomes for these RPUs in the Carnegie world if you change from looking at them through a core-based statistical area lens, or using just their regular state lens?” And there’s a pretty big impact. It really can make kind of a big difference. One of the other things I looked at, just to sort of confirm for myself that this is kind of questionable, is I was thinking all these RPUs have… not all of them, but many of them… have huge dormitories, [LAUGHTER] and we have the ability to house huge percentages of our student bodies. So, while most two-year schools don’t have dorms, and the students have to come from right there in that community, that is just not true for the RPUs. I did a lot of work looking at the impact of this flip between CBSA and state data on the metrics measuring diversity. I didn’t do as much looking at the income piece, but it’s definitely a problem. Most RPUs are not getting the bulk of their students from their CBSA, and yet their benchmark salary is being determined, in very large matter, by the incomes in that CBSA. Is this good? Is it bad? I don’t know. It is a validity question, because we’re measuring something, thinking it means one thing when it really might not.
Rebecca: One of the other metrics that you were talking about is how the Department of Ed is comparing to median salaries of high school graduates. Can you talk a little bit about how they’re doing that, and what concerns you might have around that, in terms of like how they’re benchmarking which programs get loans.
Debbie: I’ve been speaking mostly about how the Carnegie made adjustments in order to accommodate regional variation. I want to make it clear at this point that the Carnegie framework did not consider field of study, which is where I think you were kind of starting to head, Rebecca, because it’s the field of study that the Department of Ed wants to use to cut loans off. They are not suggesting cutting off loans and grants to entire universities, although that is a possibility if a university has a really large proportion of its programs failing to meet the benchmark, then they could talk. I don’t know. But it really is this program level data. So, my understanding is that the Department of Ed, right now, is trying to figure out how to do some regional salary adjustments that may or may not look like how Carnegie did it, but the OBBBA data set is going to be doing these regional adjustments on salaries aggregated to reflect program of study, so you can go in right now to the scorecard, and you can go into fields of study, and for fields of study that have an adequate number of graduates, you can see what is the median earning for those graduates from that program four years later. On the same page, it has a little bar that says here is the average earnings of high school students in the United States at this point in time, and then there’s a bar right beneath that that said, and here’s the earnings for this school’s state. So right now OBBBA would be leaning on a data set, or the scorecard is saying here’s the field of study, but everybody gets matched up to the same national high school graduate earning data, or state high school earning data, and what I mean by high school earning data, it’s earnings of people who are employed full time in that year whose highest degree is a high school diploma.
Rebecca: So basically the numbers are comparing someone who studied this particular topic with anyone who’s in any job with a high school degree [LAUGHTER] in any field. [LAUGHTER]
Debbie: Yes, yes, and you know something that I haven’t really thought too much about, but it’s one of those things that I think I think about it in the middle of the night. The federal government right now is spending a lot of time and effort to make more federal funding available for short- term academic programs, which is like technical training. I feel like the federal government is working hard to improve the salary and earnings [LAUGHTER] of people with a high school degree, you go and get a certificate in a particular field, which is great, and I completely support this, but these high school earnings figures, I think, are going to be coming up. If the federal government is successful, the benchmark that they’re going to be comparing the four-year schools to is going to be going up more. I’m not really sure that’s going to happen, because I think earnings are really more a reflection of the labor market than they are a reflection, particularly of field of study and variances between different programs. Another problem, though, that I really want to talk about, and I hope this doesn’t give you heebie jeebies, Rebecca, because I know you’re more of a qualitative person and less of a quantitative person, but all of this benchmarking is going to be made based on comparing median earnings of a group of people to median earnings of a larger group, the high school comparison, so we really have to ask the question, we hot to dig back and think, okay, what is a median, or even what is an average, and what does it mean to enact a policy where we treat an entire group of people in a particular way based on the average of a group that that person happens to be a member of. This is a bigger problem, really, for Carnegie, because they were putting people in groups left and right, but this is a problem. This fact that the median is not necessarily particularly representative, because there are people who have much higher salaries and people who have much lower salaries. So, how fair is it to treat everybody in a program one way because of a median value, there’s two real issues here. The first is the issue of the cutoff value. I’m trying to think of a really good example to illustrate the cutoff value problem. In higher education, it’s very common for universities to say “everybody with a high school GPA of 3.5 and above is automatically admitted, directly admitted.” The cutoff problem is that there’s going to be a lot of people who have a 3.49 who have very, very good reasons for arguing that their 3.49 justifies their admission to this university to much greater extent than some other people that have a 3.5. Maybe that person went to a very exclusive, difficult high school, maybe the person began high school speaking a foreign language, maybe the person had trauma in their last year, all that stuff, and we all know colleges have these vast mechanisms for making exceptions that are required because of the cutoff value problem. Whenever you’re going to take a quantitative element and put people in groups, you’re going to have a cutoff problem. It’s like for the OBBBA, the cutoff problem is there’s going to be programs where the median high school salary is only $1 or two or three, it’s very, very close to the median for this group of students, for whatever reason. And that median could be low for a lot of reasons, maybe it’s low because that program has a bunch of students in grad school, maybe it’s low because those people all decided to take a mission year and they’re just now starting their careers. Whatever it is, you’re going to have people really close to the cutoff value. The second numeric problem, you have to think back when you were in stats class and they made you learn all about the standard deviation and the variance around the midpoint, this is a huge, huge problem. So, just to illustrate this, if you think about a couple of universities, and you think about business school graduates, you could have one university where the average business school grad earns 50,000 and another one where the average earns 40,000 so there’s a $10,000 difference between those two. Well, I can guarantee you, if you looked at either one of those, at the 40,000 you would find a lot of students earning over 50 and even over 60,000 and you would find lots of students earning 30,000 and below. The definition of a median is that half the people are above and half the people are below. There’s been a lot of research in higher education, specifically the research I’m most familiar with was done by the National Survey of Student Engagement, the NSSE. About 30 years ago, the NSSE started collecting student experience data, and very quickly they started putting out data showing that the variance of student experiences within any single school is vastly greater than the variance of student experiences from one school to another. This is just by definition, this has to be true in this earnings data. S o we’re very likely going to be eliminating programs that could have served students where we would have predicted that at least half of those students would have done just fine in the earnings metric. Now the scorecard has a ton of data about this variance measures. For every single program, they show the standard deviation, they provide a mean, they provide a median, and they provide the quartile values at the third quartile and the first, so that you can measure and see how wide is the gap within which 50% of students fall. The fact that they’re providing all this variance data tells me the people at the Department of Ed deeply understand the importance of variance, but deeply understanding it mathematically doesn’t explain to me how they’re going to accommodate for the variance when they start deciding which programs get cut off, because they have not talked about that at all. So there’s a lot of stuff really coming out here, [LAUGHTER] a lot of questions still on the table.
John: And you mentioned the cut off value, and how it’s somewhat arbitrary, and it’s very precise, though, they come up with a specific value, and yet so many of the benefits of education are not measured in salary. People with college degrees have lower unemployment rates, they have longer lifespans, they have higher quality of life, as measured by all sorts of happiness surveys. They have lower probabilities of divorce, they’re less likely to commit suicide, they’re less likely to commit crime, and so there are many aspects of jobs and the returns to education that are just completely being ignored by this focus on one specific value, and that’s kind of troubling.
Debbie: It is troubling, and it’s very, very sad. I hope that we can eventually, as a country, return to a time where we value learning and personal growth. I think our labor market is kind of not really supporting all of our people particularly well right now. I would like to live in a world where if someone is called to ministry, that they don’t have to worry [LAUGHTER] that they can’t go and get a degree that they can afford to pay for, because being a minister is not going to pay very much, or if a person is called to be a pianist, concert pianists don’t always earn a lot of money. Does that mean we taxpayers, with our taxpayer support, that we shouldn’t support that person’s education? I don’t want to live in a world that doesn’t have concert pianists who have been supported all the way through getting a Doctor of Music Arts, because the people who have those degrees are amazing in the way they connect our world to our history through music, and I’m picking on fields that are often sort of the punching boys for “don’t study theater, you’ll never make an income.”
Rebecca: But these are the fields that often contribute a lot to a community, and these measures don’t actually have anything to do with community benefit or the public good.
Debbie: Yep, the public good.
Rebecca: So, on that note, [LAUGHTER] we always end by asking, what’s next?
Debbie: I’m so glad that you are ending with that, because I think there’s no question we will be… we in higher education… will be controlled by these student earnings data for a while. This final OBBBA ruling is supposed to come out of its last session of a negotiated rule making at the end of May, we’re sitting here, May of 2026, we are speaking about this as it’s happening. If you’re starting to pay attention to this, or you’re looking for information about it, I want to enable you to learn more and be empowered. So, really, what’s your favorite publication for higher ed stuff? Is it the Chronicle of Higher Ed, is it Inside HigherEd or even The New York Times and will be covering this information. What you’re looking for is, they might speak about this ruling as financial value transparency work. The old language, which some of us are still kind of stuck in, we talked about this as gainful employment regulations. I have just started seeing reference to a new acronym, STATS, which stands for Student Tuition and Transparency System. I’m not sure, but I think that stats may be how the Department of Ed plans to rebrand the Scorecard, so that when they bring it out as the basic thing undergirding this new regulation of limiting students’ access to loans and Pell grants, that STATS is what we’re going to be looking at. So those are some keywords to look for and listen for, and the last thing is wherever you are, figure out who your institutional researchers are. Make sure that they are pulling up these data files. If you Google Scorecard and higher education, if you Google access and earnings, or if you look for Carnegie classification, all of these will take you to these websites where you can look for the data and download the data and learn about its structure. So don’t wait, be empowered. The federal government has a lot of power in this conversation right now, because they control the data, but we can at least begin to educate ourselves, which is why I wanted to come and talk about this, and I really appreciate that you guys gave me the opportunity, and you ask me great questions, and you listen to my answers, and I hope you’ve learned some about how the data are put together and what we’re going to be looking for in the next couple of months.
John: So one implication of all this is that, in terms of admissions decisions, would this whole incentive system encourage, perhaps, colleges to only accept white males from higher income households because they end up with higher salaries on average.
Debbie: I don’t think we’re going to go there in higher ed. What I think is more likely is an acceleration of what we’re already seeing. I have a daughter in school in Indiana, and she keeps me apprised of what they’re doing out there, so I pay attention. I think many states have used the OBBBA work as a reason to go through and cull their programs, not particularly on the basis of earnings, but to get rid of all the small programs, and these two things tend to go together. They just do. So, I know Indiana has chopped a ton of programs. Universities are eliminating small programs left and right under an argument that they’re saving money, that they don’t have enough resources to maintain these programs. Any university with an IR person worth their salary is making sure that universities do not eliminate small programs that have very high earnings. I worked at a university that had a really small engineering program. They’re not going to eliminate that program, [LAUGHTER] even though it’s really hard and expensive to run, but those students do earn really high salaries. I think, getting to your point, John, universities are so desperate for tuition revenue, I do not think that they are going to filter for the potential future earnings of white men. Some might, but I don’t know. I’ll do that research four years from now, and I’ll come back and let you know how it’s going. [LAUGHTER]
John: It’s just one of the worst possible outcomes of the incentive system that they’ve devised.
Rebecca: I think it’s really helpful for faculty to be more apprised of this information, because it’s kind of zooming past us as policy and regulations are changing so swiftly, and so having this explainer is very helpful. [LAUGHTER]
John: Yes, thank you. This was really good and it’s things that I think most faculty are not that aware of right now.
Debbie: I think people are viewing it as something that is happening to us, and as I close, saying yes, that’s kind of it, it is happening to us. There may eventually come a time when we can speak back and push to recreate a more equitable, more reasonable understanding that incorporates more of the community values and the public values that we get from higher ed.
John: And we are lucky to be in a state where there is much more support…
Debbie: Oh, yeah…
John: …for the positive aspects of education.
Rebecca: Thank you so much.
[MUSIC]
John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.
Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.
John: Editing Assistance provided by Madison Lee.
[MUSIC]