Category: regulations

449. Student Access and Earnings

Posted in access, diversity, equity, first-generation students, higher education, inclusion, regulations
June 10, 2026

The “One Big, Beautiful Bill” contains a provision eliminating federal loan availability for programs in which past graduates had early career earnings that fall below a designated threshold. In this episode, Debbie Furlong joins us to discuss the statistical measures used to determine federal loan eligibility.

Debbie has served for over 30 years doing applied higher education policy analysis at public comprehensive universities, most recently as the Director of Institutional Research at SUNY Oswego. Prior to her arrival at Oswego, she worked and taught at the University of Wisconsin at Green Bay, St. Norbert College in Wisconsin, and the American University in Washington, DC, where she had completed her PhD degree in international relations.

Show Notes

Transcript

John: The “One Big, Beautiful Bill” contains a provision eliminating federal loan availability for programs in which past graduates had early career earnings that fall below a designated threshold. In this episode, we explore the statistical measures used to determine federal loan eligibility.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Debbie Furlong. She has served for over 30 years doing applied higher education policy analysis at public comprehensive universities, most recently as the Director of Institutional Research at SUNY Oswego. Prior to her arrival at Oswego, she worked and taught at the University of Wisconsin at Green Bay, St. Norbert College in Wisconsin, and the American University in Washington, DC, where she had completed her PhD degree in international relations. Welcome, Debbie.

Debbie: Hi.

John: Thanks for joining us. Today’s teas are:… Debbie, are you drinking any tea today?

Debbie: I am waiting for my tea to cool off. It’s a decaffeinated green tea, because it’s too late in the day for me to have any caffeine.

Rebecca: Green tea is a nice choice at this time of day. I have Blue Sapphire today.

John: Back to an old standby, and I have ginger peach green tea today.

Debbie: I’ve had a ginger peach tea. It’s some of my favorite flavors.

John: It’s one of mine too. This is a Republic of Tea variant. Rebecca particularly likes flavored teas.

Rebecca: It always smells good. [LAUGHTER]

Debbie: Yes, it does.

Rebecca: We’ve invited you here today to discuss the growing focus by the federal government on using earnings data on students who evaluate the success of college programs. The One Big Beautiful Bill Act prohibits universities from awarding federal loans to students in programs where its former students have earned low wages. Can you tell us a bit about how we got to this point?

Debbie: Yes, it’s been a long trip down. Some might say the government has been providing post-enrollment earnings data for several years now through a tool called a scorecard. Assessing the quality and value of a college degree in terms of students’ earnings has been growing in popularity for a long time. Increasingly, we’ve all talked in higher ed about people viewing their degree in terms of what it’s done for me as a private good, as opposed to focusing more on the public good of education. We can see data where this trend has been growing for 50 years. I think that trend was really exacerbated when public universities, when states started having to trim public funding, and parents and families were bearing more and more of the cost, and when the price, the stated prices were going up, up, up, this trend became even more predominant. Universities really have not helped at this, because, like, our marketing areas for recruiting have really leaned into the job placement as something that needs to be touted as an important thing, so we’re all sort of involved in telling students they should pay attention to how much they’re going to earn. I think the original interest of the federal government – I’m just speculating… based on timing and the administration that was in play at the time we first got the scorecard… I think the original interest was sort of a consumer protection kind of position. We had had an explosion of for-profit schools that really had some very serious problems. Students were not completing the degrees, and even when they were, they were unable to earn enough to pay back their loans, and the loan default rates were going up for these schools. So it was like, “Well, let’s protect the consumers,” so about 10 years ago, when Obama was actually still in, the Department of Ed launched the College Scorecard, and the College Scorecard was really the first time that families and students could look at a university’s profile and see: this is the graduation rate, this is how much it cost, this is what percent of students get loans, and here’s the average income a certain number of years after graduation. And here’s the percent of students that are paying back loans, so, bringing it all together. Where we’re at right now, the new state that we’re in right now is the Department of Ed looking and saying, “Gosh, we have all this data. Let’s begin using it to stop supporting programs where students are not earning enough money. Since we have all the data, we should protect not just the consumer, but protect the taxpayer. The taxpayers shouldn’t be asked to pay through Pell grants and through loans for programs where students aren’t even able to earn as much as they could earn if they went and got a high school degree.” That’s how we got where we are, and that’s where we’re at. I hope you can hear some of the passion of my voice about this stuff. [LAUGHTER] I really think faculty and staff need to understand where the data come from and understand how we’re all in this together, and to feel confident to go and explore and learn on their own.

John: So, this started off as something designed to protect consumers, but it’s moved on to something a little bit bigger, and because of that, it’s become increasingly important to understand the data, and you spent a fair amount of time researching the data and how it’s measured, and you’ve raised some concerns with it. Could you talk a little bit about the primary sources of data first?,

Debbie: Yeah, there’s two really big national data sets. One is from the Department of Education, and one is provided by a group called the Carnegie Classification, which is a branch of the Carnegie Foundation, which is a nonprofit. The Carnegie Foundation has teamed up with the American Council on Education, ACE, which is a big nonprofit player in the higher ed sphere. Both of these organizations are providing data that is publicly accessible and downloadable, and they both have a website search engine that sits on top that enables you to look up one school or to compare a few schools, but the data that’s running those engines is fully accessible. So this winter I was bored, and I got in, and I started playing. It’s like I just really want to learn how they’re going to do all this, and what does this stuff all mean. So the College Scorecard is the most important resource. Like I said earlier, the Obama administration really launched the scorecard, and this was the first time that consumers could see, for one institution, all the stuff about debt and debt repayment and post-enrollment earnings, and they could see those metrics right next to the data about the tuition and the costs. Department of Ed could not have done this without technology advancing to the point that we could really wrap our minds around really truly big data sets. This downloadable data set I just referred to has over 3500 variables, so it’s not for the faint at heart. In order to measure graduates’ earnings, or the earnings of formerly enrolled students, because both groups are looked at. In order to do that, you got to have two things: you have to have tax records that indicate was the person employed, were they employed full time, and what was their income, their earnings, and you have to have something with a matching field that says what school did this student go to, did they graduate, and if they graduated, what field? No single university has this. So, the way this works is the Department of Ed has the student identity information from its loan and grant recipients, and all the earnings data is only measured for students who have taken out federal financial aid, either through a loan or by getting a Pell Grant. So, the Department of Ed has these files that have individual identification and information about whether the student graduated and what field. They send these files over to the Treasury Department. The Treasury Department does a merge and match with their records and sends the information back to the Department of Ed. As a little sidebar, we’re going to be talking in more and more depth today about this, and the fact that we’re limited to just looking at loan and grant recipients is a significant limitation. The sidebar I’ll drop in right now is I am surprised the Department of Education has not yet asked all of us to provide full identification records for all of our enrolled students. The reason I’m surprised they haven’t yet done this, there’s two things, [LAUGHTER] the first one is the Department of Ed did ask for student identified records earlier this winter when they started asking for information about access, information so that they could make sure that students who are black or low income were not being admitted when they had lower test scores and high school grades. The data file that the Department of Ed requested this winter is student identified, so they’ve gotten over their concern about privacy issues. [LAUGHTER] The second reason I’m surprised is they’ve been asking states to provide voter record identification, so I think it’s only a matter of time. The other reason, there is another organization out there that’s not really related to this topic we’re talking about, the National Student Loan Clearinghouse, or National Student Clearinghouse. The NSC has student-identified data, and I don’t see how the Department of Ed is going to feel too happy knowing somebody else has it and they don’t have it. So, I think it’s only a matter of time before that little limitation goes away. So, the Department of Ed sends these files over to Treasury, they get back the earnings data. The second data set out there has used the Scorecard file, and this other data set is the . I’m going to be referring to this as access and earnings, or more just Carnegie, the Carnegie framework. The Carnegie framework about access and earnings was released about a year ago, but it was in development for quite a bit longer. Carnegie’s real contribution that’s relevant for what we’re talking about today is they began trying to deal with the fact that in the scorecard the only earnings that people were getting compared to were national, and more recently, statewide data, and Carnegie’s framework argues that earnings for a school’s students should be compared to the earnings of high school graduates in the geographic areas where those students come from, not necessarily to the whole nation, and not necessarily even just to the state where the school is located. This issue about adjusting the earnings to reflect regional differences in incomes is definitely confounding the Department of Ed right now. The OBBBA is in negotiation. There is a negotiated rule making going on right now, and one of the last things that the Department of Ed is trying to figure out how to deal with before they roll out this framework, where if your program’s earnings are too low, you can’t get loans. The last issue they are mucking with is the regional differences in earnings, and I don’t know how they’re going to do it. The Carnegie folks did it one way and sort of got the ball rolling. Carnegie’s real contribution here, I think, is showing how incredibly difficult this is going to be. We’ll have to see, we’ll have to see how they do it, and it may take a few years before they really figure out how to most appropriately set a benchmark for earnings for every program in every school.

John: So, are you saying that the earnings of people in rural Alabama might be different than Manhattan? [LAUGHTER] Who would have thought?

Debbie: They are. Yes.

John: Among the other problems there that you’ve noted is that there’s a fair amount of sample selection in the sample that they’re working with, and you’ve already referred to one aspect of that. Could you talk a little bit more about that, and how that might affect the validity of the earnings measures?

Debbie: Yeah, yeah, so what John just referred to is the fact that only students who get federal financial aid are currently included, and for many of us that doesn’t seem like very much of a limitation, because so many of our students have to get loans and a huge percent of students get Pell grants, but there are a bunch of schools out there where those percentages are really, really small, and I went and I looked up Harvard’s data, just because everybody knows Harvard, you know, Harvard, Harvard. And I went and looked at Harvard because I anticipated Harvard students would not need to rely a lot on federal aid, not because I think Harvard students are rich or poor, I don’t particularly know, but Harvard is very, very wealthy, and they’re able to provide a lot of institutional aid, which keeps those students out of this federal tracking, tying together with your Treasury data. So I looked up Harvard. In a recent year, 20% of their students got Pell grants, and 4% of their students took out loans. So by the time that cohort gets through the system, only a quarter of the students would have their wages reflected in these median wages. I don’t really know what this is going to mean. Is this good? Is it bad? It makes me feel uncomfortable about the validity of the data. Another thing that I know with this coverage, I have been watching that the percent of students in the United States agreeing to take out loans fell after COVID and did not rebound. I think this past year they have started rebounding a little bit. What this means is that the coverage overall, nationally, school after school after school, this coverage rate is going to be going down for the next several years. So, this is another reason why I am expecting the Department of Ed to be knocking on everyone’s doors very soon, saying: “The gig is up. We want a student-identified file. We want to know everyone who’s enrolled every semester. Here’s the portal. Drop the data.” Another issue that kind of deals with coverage that gets very confusing has to do with the timing, and this is kind of devastating to me, is a validity issue. And what I mean by timing is, how long do you wait after a student has left the institution, or after a student has graduated. How long do you wait before you go and grab their earnings data? Okay, so timing issues can be very confusing, and what I would really encourage anybody who’s paying attention to this stuff, don’t try to sink into your mind exactly how these things are measured. When you go to one of the websites, look for the information notes, they will tell you exactly what they’re measuring. Carnegie’s Access and Earnings framework originally relied on a measure… they looked at students’ earnings eight years after the student initially enrolled, and there are reasons why they made that choice. One of the things that higher ed has been deeply criticized about is the poor retention of students after the first year, and there’s a huge amount of tax money that goes to students who are at the institution for one year, and then they never really do anything with that education. So looking at students based on where they started is not stupid, but it is kind of misleading. I think when people say, “Oh, here’s Cornell’s earnings data,” I think people assume that’s the earnings data of students who graduated from Cornell [LAUGHTER] or from whatever school, and the current version of the scorecard has flipped over and is relying almost exclusively on earnings data measured four years after the student graduated. So the cohorts are now measuring students who actually completed the degree, which is probably more valid for what people actually think they’re looking at, and they give students four years, so now it’s like, okay, four years… is four years long enough to get a valid measure of a student’s earnings potential? I think this is a huge limitation. Many of the highest earning fields, and I’m thinking like medicine and law, those are going to be either really hard or downright impossible for a student to complete the whole thing and be out and earning. So a lot of fields where students go on to complete masters and doctorates, those students are ultimately going to be the highest earners, but they’re not going to be earning that stuff four years out, so I was thinking about, like, where I was four years after my undergraduate degree. I had gotten my first full-time job, but it was just a job to pay the bills, because I was working on a PhD. So, was I earning more than a high school student? Maybe. I don’t know. I was mostly a graduate assistant, getting paid a graduate fellowship. Even fields where students don’t go on for advanced degrees, I think that there’s a lot of research that indicates people who complete a baccalaureate degree will ultimately have higher lifetime earnings, even if they don’t get an advanced degree. We know this lifetime earning data exists, so is four years long enough to predict the lifetime earnings of a particular person in a particular field of study from a particular university? I think it’s a big validity question. The last thing that I wanted to mention is this timing, all this stuff about the validity of the timing of when the data are grabbed, it’s grabbing data right smack at an important time for young families that are deciding to have kids. I looked up before coming today, because I knew I wanted to make this point, because I think it’s important, the average age for women in the United States who have had children at this time, the average age of when they had their first baby was 26 that’s just right in this four years… you graduate when you’re 21 or 22 So I mentioned this because it’s very common for young families, for either the husband or the wife, the mom or the dad, or one parent, to back off a little bit from working full time or working at the high pressure job. So there’s a lot of problems with the data.

John: Going back to what’s referred to as the Mincer equation, which was first estimated back in the early 1970s. economists have been estimating the overtaking age, the age at which the earnings of college graduates overtake the earnings of high school graduates with equivalent demographic and ability characteristics, and back then was about seven years after graduation. More recent studies suggest that it varies a fair amount by discipline and by occupation, but it’s generally between six and eight years after graduation from college, which means on average the fact that people are earning more than high school graduates puts them way above where you would expect them to be had they not gone to college, or people with similar characteristics would receive had they not gone to college. And so it’s really troubling that they came up with a metric that’s so evidence-free, basically.

Debbie: Yes, [LAUGHTER] problematic.

John: And also, you mentioned law school graduates, so they might be a year out of law school, but some of the people who come out of law school are going to be working with large firms for ridiculously low pay, and those are the people who are going to have the highest rate of growth of earnings. Other people will start their own practice and will have higher starting salaries than those who go for those large law firms, at least in terms of hourly wages, they’ll be paid a lot less, but they will end up with a slower rate of growth of earnings, and there’s another classic economic study back during this period on on-the-job training and the fine structure of earnings, which notes that holding other things constant, people in the same occupation with lower starting salaries tend to have higher rates of earning growth over time, because part of what they’re picking up on the job are some skills that will improve their productivity, and they’re essentially accepting lower wages during those times in return for the increase in training or human capital development they get on the job, which is certainly true in medical school, because then they’re starting their residencies, where they’re paid again ridiculously low amounts, but they end up with much higher earnings just a few years later.

Rebecca: I knew John would be passionate. [LAUGHTER]

Debbie: Yes. And everything he said is true today, the research is still valid and important.

Rebecca: One of the other things that you mentioned is those regional variations that Carnegie was developing, or kind of initiated. Can you talk a little bit about these regional variations, and like, what does their framework mean? How did they do it? How maybe should they do it? [LAUGHTER]

Debbie: Yeah, so like I said, when they first put the scorecard out, my perception is that was kind of a consumer protection thing, like here’s the tuition, here’s the percent of students who have loans, here’s the percent of students who have paid back their loans, and here’s the average amount that their graduates earn, without judging whether or not that average was adequate or met a benchmark. Well, those days are behind us. The OBBBA wants to establish this benchmark of acceptable earnings, and the negotiation is moving forward. It seems very clear that they’re going to be using this high school degree population as the benchmark, but obviously people earn different wages based on different regions and different wages based on whether or not you’re in an urban area or a rural area. So the Carnegie framework is the first place I’ve seen where they tried to take this on. So let me tell you how they did it. A little segue, when Carnegie adjusted for regional variation, they have also adjusted for variances based on race and ethnicity. Carnegie’s argument was, if a student goes to a certain university, when they get done, that student should earn as much as high school graduates matched by the racial and ethnic distribution of the service area of that university and weighted to reflect the earnings differences of different racial groups, so they did this racial weighting. I’m not going to talk too much about that, because the OBBBA, the language I have seen, specifically says that the earnings benchmark cannot be adjusted for race. They want a race-neutral approach, just like they want race-neutral admissions. This is part of what we’re going through and working on with the Department of Ed right now. So let me talk about how they do the regional stuff. What Carnegie looked at, they said, “Okay, let’s look at where the entering class of a university is coming from.” The Department of Ed, for 20 or 30 years, has collected this data. It’s called a migration file. In the migration file every second fall, you’re not required to submit migration data every year… I don’t know why not… but in the file, the institutional researchers at the university have to go in and look and say, “Okay, how many people are in my freshman class, and where did they come from? How many of them came from New York State. How many of them came from Pennsylvania?” And the file goes through all of the states, so that data exists in IPEDS. IPEDS is the federal government’s collection of data about colleges and universities, and it’s a massive data set. So Carnegie said, “Well, gosh, since IPEDS has this migration data, let’s use that to figure out this concept of a service area.” What is the service region of this university? So, if a university has 90% of its students are coming from New York and 10% are coming from Pennsylvania, its earnings should be similar to the high school graduates of New York, weighted a little bit to reflect the fact that people from Pennsylvania have lower earnings. Now, so a 90%, 10%. In the Carnegie framework, it takes the top three states, so it says, okay, school X. If you had 50% of your students from one state and 40% from a second state and 5% from a third state, your earnings benchmark will be the weighted average of those elements, and then the remainder, the last little bit, we’ll just use the national data. This sounds kind of reasonable, I guess. Carnegie was very careful developing this. They got feedback, they got pushback from the two year schools saying, “Well, wait a minute, everyone at our university comes from one state, but they don’t come from all over the state, they all come from my county, or these two or three counties, because we have a lot of two-year schools out in rural areas.” This is a really serious issue, because the state earnings levels are very influenced by what’s happening in the big cities, and rural incomes tend to be much lower than urban incomes. So, Carnegie said, “Well, this is a reasonable complaint, so we’re going to back off, and we’re not going to use your state, where you have most people from. We’re going to use this core-based statistical area to gather your data.” There are a couple of other problems before we really get into the CBSA, and what that is. There are a couple of other things about relying just on the migration data that concerned me. My professional background was all in public comprehensive universities, and every place I ever worked had a huge pool of students who transferred into my universities, so I know transfer students may or may not be accurately reflected in this migration data in the schools where I’ve worked. In the systems where I’ve worked transfer students had a very different geographic distribution than new freshmen, and I think this is true for a lot of public comprehensive universities that the freshman class tends to come from all over the state much more, but the transfer students tend to be students who started at a nearby community college, and now they are your own community students, so that migration data that we’re using may not really be a great fit. The other thing that’s problematic is we are just assuming that the geographic distribution of graduates is the same as the geographic distribution of new entering freshmen. It may be, it may not be, and it may or may not matter, but these kinds of assumptions that are baked into the method, I think, cause people to feel a sense of discomfort about really feeling confident that the data really are measuring what we think they’re measuring,

Rebecca: So Debbie, I have another question. It seems like most of the data that you’re talking about is really focused on the states, so not necessarily international students and entirely undergraduates.

Debbie: Yes, and no, all of the earnings data is for US citizens only, and I will remind you that in order to have a student’s earnings get studied, they have to have gotten federal financial aid. International students are not eligible for Pell grants and for federal loans. So even if a former international student got OPT training and landed a job and became naturalized, we would have no way of knowing… the Department of Ed would have no way of knowing to tell the Treasury Department, “send me that person’s salary and earning data.” So universities that have huge international student bodies, especially if they’re doing really well, and supplying really fabulous students into high-need areas here in the United States. All of that is just kind of a loss. Now, your other question, like “Debbie, all you’re doing is talking about undergrad, undergrad, undergrad. What about graduate students?” The research that I really focused on this winter was about this Carnegie Access and Earnings framework, which is exclusively about the undergraduate experience. The scorecard data does include earnings data for graduate students, and the OBB BA work going on right now is also going to be relevant for graduate students. Graduate students are eligible to get federal loans in certain circumstances, and those loans will be cut off if a student gets a graduate degree from a school where their program is not earning on average as much as baccalaureate earners in whatever state or region the OBBA decides to look at, so it’s the same kind of framework, notched up one level.

Rebecca: The other thing that I was curious about is the data that you were talking about, is where students are coming from initially, which may not be where they are working.

Debbie: Yeah.

Rebecca: Am I right about that? [LAUGHTER]

Debbie: Oh, yeah. [LAUGHTER] And I’ve got a bunch of notes about this thing, and I definitely wanted to hit this. So you’re jumping into something that I already also agree is really a problem. Two issues: do students go back to where they came from? No. The other thing that’s really problematic, I think, is that the earnings data are all aggregated on the basis of what the student studied, not what they’re actually doing. So, I was thinking in my head, like examples, I know your field is art and design, design arts. Someone who studies design arts and graduates with an art degree and goes and does marketing in New York City is going to have a very different earnings experience than someone who completes an art degree at the same school and stays in that local area and runs art programs for the YMCA. These are both wonderful jobs, both jobs that would require an art degree to be really effective, but they’re incredibly different fields. I mean, one is marketing and the other one is social services, and that’s not accounted for. So, what a person’s actually doing with their degree is not part of the mix.

John: You talked a little bit about these core-based regions that are used for measuring geographic effects. What are they, and could you tell us a bit more about how they’re defined, and whether there’s some problems with the use of those.

Debbie: Yeah, the two year schools kind of pushed back and said statewide data doesn’t make any sense for us. So the Carnegie Foundation said, “Okay, the regional adjustment for two-year schools, we should use the data from the closest geographic region to their service area.” The Census Bureau has these geographic entities called core-based statistical areas. A CBSA is a geographic area that contains a large urban center, and then the surrounding areas that are kind of integrated economically with that urban center. Like, I live in Oswego, New York, which is about 30 miles north of Syracuse. I’m in the Syracuse CBSA, which makes perfect sense. There’s so much stuff that we depend on Syracuse for, and people go back and forth there all the time. So, Carnegie said, “Okay, let’s use core-based statistical areas for the two-year schools, and let’s also use them for these public regional universities.” There’s this category of universities called a regional public university, or an RPU, which is most public universities that are not focused on research. So, in New York, Stony Brook, and Albany and SUNY Buffalo, those are all research-focused institutions, they’re not RPUs. Almost all of the other four-year public universities in New York are considered RPUs. So, Carnegie said, “Well, they’re called regional, they must be serving their region. Let’s do all of our benchmarking for them based on the core-based statistical area.” Well, I really dug into this with my research, because the truth of the matter is most of these regional public universities are not recruiting heavily from our own regions. At most, RPUs would have maybe 20 to 30% of the students from their region. We’re all recruiting heavily in the big urban areas, because that’s where the students are. This is the whole piece of this enrollment cliff. Not only is certain states and regions losing students, but the students who are left are in the cities, and if we want to maintain universities out in the smaller communities, we’ve got to move the students out into these smaller communities. So I did some actual sensitivity analysis to see, does it matter? Does it impact the outcomes for these RPUs in the Carnegie world if you change from looking at them through a core-based statistical area lens, or using just their regular state lens?” And there’s a pretty big impact. It really can make kind of a big difference. One of the other things I looked at, just to sort of confirm for myself that this is kind of questionable, is I was thinking all these RPUs have… not all of them, but many of them… have huge dormitories, [LAUGHTER] and we have the ability to house huge percentages of our student bodies. So, while most two-year schools don’t have dorms, and the students have to come from right there in that community, that is just not true for the RPUs. I did a lot of work looking at the impact of this flip between CBSA and state data on the metrics measuring diversity. I didn’t do as much looking at the income piece, but it’s definitely a problem. Most RPUs are not getting the bulk of their students from their CBSA, and yet their benchmark salary is being determined, in very large matter, by the incomes in that CBSA. Is this good? Is it bad? I don’t know. It is a validity question, because we’re measuring something, thinking it means one thing when it really might not.

Rebecca: One of the other metrics that you were talking about is how the Department of Ed is comparing to median salaries of high school graduates. Can you talk a little bit about how they’re doing that, and what concerns you might have around that, in terms of like how they’re benchmarking which programs get loans.

Debbie: I’ve been speaking mostly about how the Carnegie made adjustments in order to accommodate regional variation. I want to make it clear at this point that the Carnegie framework did not consider field of study, which is where I think you were kind of starting to head, Rebecca, because it’s the field of study that the Department of Ed wants to use to cut loans off. They are not suggesting cutting off loans and grants to entire universities, although that is a possibility if a university has a really large proportion of its programs failing to meet the benchmark, then they could talk. I don’t know. But it really is this program level data. So, my understanding is that the Department of Ed, right now, is trying to figure out how to do some regional salary adjustments that may or may not look like how Carnegie did it, but the OBBBA data set is going to be doing these regional adjustments on salaries aggregated to reflect program of study, so you can go in right now to the scorecard, and you can go into fields of study, and for fields of study that have an adequate number of graduates, you can see what is the median earning for those graduates from that program four years later. On the same page, it has a little bar that says here is the average earnings of high school students in the United States at this point in time, and then there’s a bar right beneath that that said, and here’s the earnings for this school’s state. So right now OBBBA would be leaning on a data set, or the scorecard is saying here’s the field of study, but everybody gets matched up to the same national high school graduate earning data, or state high school earning data, and what I mean by high school earning data, it’s earnings of people who are employed full time in that year whose highest degree is a high school diploma.

Rebecca: So basically the numbers are comparing someone who studied this particular topic with anyone who’s in any job with a high school degree [LAUGHTER] in any field. [LAUGHTER]

Debbie: Yes, yes, and you know something that I haven’t really thought too much about, but it’s one of those things that I think I think about it in the middle of the night. The federal government right now is spending a lot of time and effort to make more federal funding available for short- term academic programs, which is like technical training. I feel like the federal government is working hard to improve the salary and earnings [LAUGHTER] of people with a high school degree, you go and get a certificate in a particular field, which is great, and I completely support this, but these high school earnings figures, I think, are going to be coming up. If the federal government is successful, the benchmark that they’re going to be comparing the four-year schools to is going to be going up more. I’m not really sure that’s going to happen, because I think earnings are really more a reflection of the labor market than they are a reflection, particularly of field of study and variances between different programs. Another problem, though, that I really want to talk about, and I hope this doesn’t give you heebie jeebies, Rebecca, because I know you’re more of a qualitative person and less of a quantitative person, but all of this benchmarking is going to be made based on comparing median earnings of a group of people to median earnings of a larger group, the high school comparison, so we really have to ask the question, we hot to dig back and think, okay, what is a median, or even what is an average, and what does it mean to enact a policy where we treat an entire group of people in a particular way based on the average of a group that that person happens to be a member of. This is a bigger problem, really, for Carnegie, because they were putting people in groups left and right, but this is a problem. This fact that the median is not necessarily particularly representative, because there are people who have much higher salaries and people who have much lower salaries. So, how fair is it to treat everybody in a program one way because of a median value, there’s two real issues here. The first is the issue of the cutoff value. I’m trying to think of a really good example to illustrate the cutoff value problem. In higher education, it’s very common for universities to say “everybody with a high school GPA of 3.5 and above is automatically admitted, directly admitted.” The cutoff problem is that there’s going to be a lot of people who have a 3.49 who have very, very good reasons for arguing that their 3.49 justifies their admission to this university to much greater extent than some other people that have a 3.5. Maybe that person went to a very exclusive, difficult high school, maybe the person began high school speaking a foreign language, maybe the person had trauma in their last year, all that stuff, and we all know colleges have these vast mechanisms for making exceptions that are required because of the cutoff value problem. Whenever you’re going to take a quantitative element and put people in groups, you’re going to have a cutoff problem. It’s like for the OBBBA, the cutoff problem is there’s going to be programs where the median high school salary is only $1 or two or three, it’s very, very close to the median for this group of students, for whatever reason. And that median could be low for a lot of reasons, maybe it’s low because that program has a bunch of students in grad school, maybe it’s low because those people all decided to take a mission year and they’re just now starting their careers. Whatever it is, you’re going to have people really close to the cutoff value. The second numeric problem, you have to think back when you were in stats class and they made you learn all about the standard deviation and the variance around the midpoint, this is a huge, huge problem. So, just to illustrate this, if you think about a couple of universities, and you think about business school graduates, you could have one university where the average business school grad earns 50,000 and another one where the average earns 40,000 so there’s a $10,000 difference between those two. Well, I can guarantee you, if you looked at either one of those, at the 40,000 you would find a lot of students earning over 50 and even over 60,000 and you would find lots of students earning 30,000 and below. The definition of a median is that half the people are above and half the people are below. There’s been a lot of research in higher education, specifically the research I’m most familiar with was done by the National Survey of Student Engagement, the NSSE. About 30 years ago, the NSSE started collecting student experience data, and very quickly they started putting out data showing that the variance of student experiences within any single school is vastly greater than the variance of student experiences from one school to another. This is just by definition, this has to be true in this earnings data. S o we’re very likely going to be eliminating programs that could have served students where we would have predicted that at least half of those students would have done just fine in the earnings metric. Now the scorecard has a ton of data about this variance measures. For every single program, they show the standard deviation, they provide a mean, they provide a median, and they provide the quartile values at the third quartile and the first, so that you can measure and see how wide is the gap within which 50% of students fall. The fact that they’re providing all this variance data tells me the people at the Department of Ed deeply understand the importance of variance, but deeply understanding it mathematically doesn’t explain to me how they’re going to accommodate for the variance when they start deciding which programs get cut off, because they have not talked about that at all. So there’s a lot of stuff really coming out here, [LAUGHTER] a lot of questions still on the table.

John: And you mentioned the cut off value, and how it’s somewhat arbitrary, and it’s very precise, though, they come up with a specific value, and yet so many of the benefits of education are not measured in salary. People with college degrees have lower unemployment rates, they have longer lifespans, they have higher quality of life, as measured by all sorts of happiness surveys. They have lower probabilities of divorce, they’re less likely to commit suicide, they’re less likely to commit crime, and so there are many aspects of jobs and the returns to education that are just completely being ignored by this focus on one specific value, and that’s kind of troubling.

Debbie: It is troubling, and it’s very, very sad. I hope that we can eventually, as a country, return to a time where we value learning and personal growth. I think our labor market is kind of not really supporting all of our people particularly well right now. I would like to live in a world where if someone is called to ministry, that they don’t have to worry [LAUGHTER] that they can’t go and get a degree that they can afford to pay for, because being a minister is not going to pay very much, or if a person is called to be a pianist, concert pianists don’t always earn a lot of money. Does that mean we taxpayers, with our taxpayer support, that we shouldn’t support that person’s education? I don’t want to live in a world that doesn’t have concert pianists who have been supported all the way through getting a Doctor of Music Arts, because the people who have those degrees are amazing in the way they connect our world to our history through music, and I’m picking on fields that are often sort of the punching boys for “don’t study theater, you’ll never make an income.”

Rebecca: But these are the fields that often contribute a lot to a community, and these measures don’t actually have anything to do with community benefit or the public good.

Debbie: Yep, the public good.

Rebecca: So, on that note, [LAUGHTER] we always end by asking, what’s next?

Debbie: I’m so glad that you are ending with that, because I think there’s no question we will be… we in higher education… will be controlled by these student earnings data for a while. This final OBBBA ruling is supposed to come out of its last session of a negotiated rule making at the end of May, we’re sitting here, May of 2026, we are speaking about this as it’s happening. If you’re starting to pay attention to this, or you’re looking for information about it, I want to enable you to learn more and be empowered. So, really, what’s your favorite publication for higher ed stuff? Is it the Chronicle of Higher Ed, is it Inside HigherEd or even The New York Times and will be covering this information. What you’re looking for is, they might speak about this ruling as financial value transparency work. The old language, which some of us are still kind of stuck in, we talked about this as gainful employment regulations. I have just started seeing reference to a new acronym, STATS, which stands for Student Tuition and Transparency System. I’m not sure, but I think that stats may be how the Department of Ed plans to rebrand the Scorecard, so that when they bring it out as the basic thing undergirding this new regulation of limiting students’ access to loans and Pell grants, that STATS is what we’re going to be looking at. So those are some keywords to look for and listen for, and the last thing is wherever you are, figure out who your institutional researchers are. Make sure that they are pulling up these data files. If you Google Scorecard and higher education, if you Google access and earnings, or if you look for Carnegie classification, all of these will take you to these websites where you can look for the data and download the data and learn about its structure. So don’t wait, be empowered. The federal government has a lot of power in this conversation right now, because they control the data, but we can at least begin to educate ourselves, which is why I wanted to come and talk about this, and I really appreciate that you guys gave me the opportunity, and you ask me great questions, and you listen to my answers, and I hope you’ve learned some about how the data are put together and what we’re going to be looking for in the next couple of months.

John: So one implication of all this is that, in terms of admissions decisions, would this whole incentive system encourage, perhaps, colleges to only accept white males from higher income households because they end up with higher salaries on average.

Debbie: I don’t think we’re going to go there in higher ed. What I think is more likely is an acceleration of what we’re already seeing. I have a daughter in school in Indiana, and she keeps me apprised of what they’re doing out there, so I pay attention. I think many states have used the OBBBA work as a reason to go through and cull their programs, not particularly on the basis of earnings, but to get rid of all the small programs, and these two things tend to go together. They just do. So, I know Indiana has chopped a ton of programs. Universities are eliminating small programs left and right under an argument that they’re saving money, that they don’t have enough resources to maintain these programs. Any university with an IR person worth their salary is making sure that universities do not eliminate small programs that have very high earnings. I worked at a university that had a really small engineering program. They’re not going to eliminate that program, [LAUGHTER] even though it’s really hard and expensive to run, but those students do earn really high salaries. I think, getting to your point, John, universities are so desperate for tuition revenue, I do not think that they are going to filter for the potential future earnings of white men. Some might, but I don’t know. I’ll do that research four years from now, and I’ll come back and let you know how it’s going. [LAUGHTER]

John: It’s just one of the worst possible outcomes of the incentive system that they’ve devised.

Rebecca: I think it’s really helpful for faculty to be more apprised of this information, because it’s kind of zooming past us as policy and regulations are changing so swiftly, and so having this explainer is very helpful. [LAUGHTER]

John: Yes, thank you. This was really good and it’s things that I think most faculty are not that aware of right now.

Debbie: I think people are viewing it as something that is happening to us, and as I close, saying yes, that’s kind of it, it is happening to us. There may eventually come a time when we can speak back and push to recreate a more equitable, more reasonable understanding that incorporates more of the community values and the public values that we get from higher ed.

John: And we are lucky to be in a state where there is much more support…

Debbie: Oh, yeah…

John: …for the positive aspects of education.

Rebecca: Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

206. U.S. Regulations for Online Classes

Posted in authentic learning, higher education, instructional communication, learning, online learning, pedagogy, regulations, social presence, teaching
September 22, 2021

To be eligible for U.S. federal financial aid funding, colleges and universities offering distance learning programs must satisfy new federal regulations that went into effect in July 2020 and July 2021.  In this episode, Russell Poulin joins us to discuss how these requirements have changed and what these changes mean for faculty and institutions offering online classes.

Russ is the Executive Director of the WICHE Cooperative for Educational Technologies (WCET), and the Vice President for Technology Enhanced Education at the Western Interstate Commission for Higher Education.

Shownotes

Transcript

John: To be eligible for U.S. federal financial aid funding, colleges and universities offering distance learning programs must satisfy new federal regulations that went into effect in July 2020 and July 2021. In this episode, we examine how these requirements have changed and what these changes mean for faculty and institutions offering online classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners..

[MUSIC]

John: Judie Littlejohn is joining us as a guest host for this episode. Judie is the instructional designer for Genesee Community College, and has been a guest on several of our past episodes.

Judie: And our guest today is Russ Poulin. Russ is the Executive Director of the WICHE Cooperative for Educational Technologies (WCET), and the Vice President for Technology Enhanced Education at the Western Interstate Commission for Higher Education, which, of course is WICHE. Welcome, Russ.

Russell: Oh, it’s so great to be here. Thanks for asking me.

John: Our teas today are:… Russ, are you drinking tea?

Russell: Oh, yes, I’m drinking tea and I live in Longmont, Colorado, which is near Boulder and we have Celestial Seasonings here, and so, I’m drinking Earl Grey that has probiotics in it. That’s a new product for them.

Judie: So you have Earl Grey, but I’ve got Lady Grey black tea from Twinings.

Russell: Oh, very nice!

John: And I have pineapple ginger green tea from the Republic of Tea. We’ve invited you here to discuss the new federal regulations concerning regular and substantive interaction in distance-learning courses. These regulations went into effect in July 2021. We also would like to talk a little bit about the requirements for identity verification that went into effect a year earlier, but before we discuss this, could you tell us a little bit about WCET?

Russell: Oh, I’d love to, and thank you for asking. And so WCET is part of WICHE, the Western Interstate Commission for Higher Education, which is a regional higher education compact focused on the west, but back in 1989, they started WCET to focus on educational technologies, distance ed, online learning – which wasn’t a thing yet then – and, even from the start, we had other states that came in, and now we’ve grown and now we have, members through all 50 states and Canada and even Australia. And our members, are institutions and organizations and corporations interested in the use of distance ed.

Judie: How did WCET become the go-to source for information on regular and substantive interaction?

Russell: Well, we were hearing from members that there was some confusion about exactly what they were supposed to do with this, and that there wasn’t complete clear guidance from the US Department of Education and so Van Davis, who worked at Blackboard at that time, and now works for us here at WCET. And what we did was that we went through all of the guidance, there were some guidance that had been given, and also the findings against several institutions and then tried to put together what is it that they were looking for? What is it that is expected? And then we put together a blog post of our findings, and put that out there, and for like, four or five years, no matter what else we published, that ended up being the top blog post for the year because people were seeing that that was the only place where somebody had compiled this all together and knew what to do. And then in 2019, I was named to the Department of Education’s negotiated rulemaking subcommittee that worked on new rules, and there are people who actually get together and write these rules – you can imagine how exciting that is – and I was part of that and worked on it. And one of the issues that we worked on was this distance education issue and updating this regular and substantive interaction part of it and so you could say I was in the room where it happened! [LAUGHTER]

John: So basically, there was a gap out there that needed to be filled, and there was a lot of concern because of the sanctions that were placed against Western Governors University?

Russell: Yeah, it’d been unclear, and then also yeah, there was these findings against Western Governors University that their courses were all correspondence courses, and that they were expected to pay back – really, when you got to the end of it – it was a billion dollars which they didn’t have in their pockets at the time, and so it really caught the attention of a lot of people and this was a rule that was first written in 1992, had incomplete guidance. It was being administered in different ways and so it was time for a real update and more details about what is it exactly that they’re expecting? And so we worked on that, and we also were able to put in that there’s greater reliance in terms of the relationship between the accrediting agency and the institution, because there were times where it seemed like the auditors were sort of overruling that, and that doesn’t make sense because if you’re meeting what the accreditors want, why is this out of balance?

Judie: So what is needed, then, to differentiate a correspondence course from a credit-bearing online course?

Russell: So, it’s really good to go back and make that distinction and remind people that this is all about federal financial aid and what qualifies for federal financial aid. And they always want to get into, “Well… interaction, the academics, and the pedagogy of it.” Well, okay, put that aside [LAUGHTER] for the moment, we’re talking about, for federal purposes, what qualifies for federal aid and what does not? And the idea was that distance education is something that qualifies for full federal aid, whereas correspondence response education… that you might not get full aid for those courses or if you have enough correspondence students or enough correspondence courses… if over 50% of your students or courses are labeled as correspondence… then your institution is now ineligible for aid, and that’s what happened with Western Governors University. And so, it’s good to be clear, especially as we’re seeing more and more distance ed, hyflex, hybrid sorts of courses about what fits and what does not, and what activities work. We need to know what happens, so that we know that we’re in line, so that we keep our federal financial aid for students.

John: The technology for offering online courses has changed quite a bit since the initial regulations were developed. Under the current revisions, what is meant by interaction in that definition of regular and substantive interaction?

Russell: Yeah, that’s a good point that this rule was written in 1992, prior to online learning [LAUGHTER]. There’s been a lot of changes. But you really do need to parse out the parts of it, and so thanks for asking about the interaction part of it because back in the old definition, prior to July 1, that there’s this really odd thing that, the only thing that counted for interaction was something that was done by the instructor. So, providing a lecture, providing a video, providing an assessment or providing an assignment, any of those sorts of things. Those are the only things that counted, so our interaction here… “Yeah, you said something and I’m the instructor that didn’t count what you said” That’s weird. So, now, there are parts of it (freshly hidden in the regular part of the definition) that if you go down there, that you’ll see that there’s the expectation that you respond to students and student requests. So, at least we’ve made that move that the student being part of the interaction is now a part of what counts, and that’s the great move forward.

Judie: And what is expected for substantive interaction, like how do you define the substantive part?

Russell: So going back again to the old one, that it was all just about content, that if you’re teaching a history course, as one might, that you’re talking about what happened in a particular battle, or what happened in 1792, or those sorts of things, particular things about the course, and interactions about basketball or other extraneous things didn’t count. But under the new rule, we’ve gone to an activity-based notion of it, and I’m going to read these, so stay with me on this, and they’re very short. So one, one of the activities is providing direct instruction. Two, is assessing or providing feedback on coursework. Three is providing information or responding to questions about the content or competencies. Four is facilitating a group discussion. Five is other instructional activities, as approved by the accreditor, that’s a really wide open one. But for these that, if you have any type of online course, I sure hope that you’re assessing what the students are doing, I sure hope that you’re responding to them when they ask questions, and so, you’ve already got two of those right there. The one that’s really in question is the one about direct instruction, and…everyplace else they talk about synchronous or asynchronous, but I know through some things that we learned about the Department of Ed’s thinking of that, and we asked this question directly of them during a webinar about direct instruction. At that time, they said that it was only synchronous education, and sometimes people freak out when I say that, and we are asking for clarification on that in writing, but remember that it’s two of the five, so even if they define it as just synchronous instruction, you’re probably meeting at least three of those five already if you have any type of quality course.

John: One of the issues, I think, that you’re also seeking to resolve is whether synchronous online office hours would count. Has there been any feedback on that yet?

Russell: Yeah, it seemed to us quite clear about the office hours, that that is something that would count and it’s something that did not count in the past. And so, it’s an interesting change for that and one of the reasons that they did that was one of the groups I was representing was competency-based education, and so, we’re going to get into notion of regular here in a moment, you know, how do you define regular for something that, by definition, is irregular in terms of competency based? And it’s based upon student pacing over the faculty pacing, and so, there’s a nod to that in terms of if you have regular office hours and have that posted on the syllabus or somewhere that that would count. And we’ve had some pushback from our financial aid friends, because that’s new to them. They said, “Well, that’s never counted before.” And so again, this is something where we’ve asked the question of the department and hope to get that in writing yet again so that that’s reaffirmed and everybody’s under the same understanding for that one.

Judie: You started to talk about regular, so how do you define “regular” interaction?

Russell: So regular? Yeah, as I was alluding to, that was the hardest one to do, because remember, we’re writing these for, depending on how you count, four to six thousand institutions. Some have short courses, some have competency-based, you think of every variation that you have out there. So saying “meeting once a week,” or doing something once a week just didn’t work because it didn’t fit all those different ones, and once a week would not be enough in a five- or six-week course, that’d be too little. So, we tried a formula and that was a disaster. So anyway, so we have these words, and then there’s a lot more, again, back to the relationship with the accreditor on this and what works. So, these are a little bit vague, but you need to work with your accreditor and how they’re defining these. So there’s really two parts to this. The first is going to be predictable and scheduled, and so this is something where either you have it in your syllabus, and then you have the syllabus at the start and here’s when things are going to happen, and then with that, that you may actually have a two- or three-week break where an instructor is not putting things out or you don’t have interactions, because it makes academic sense that you have the people out doing a paper or group work for two to three weeks and doing that. So that once a week would actually not work there, right? Or the predictable part of it was that maybe it’s not exactly in the syllabus, but what you do is you say, “Every Wednesday, we’re gonna meet or have office hours, or we’ll do something at webinar times.” So something predictable. So, that’s the first part of it. The second part of it was about monitoring the students’ academic engagement. And that was something we really brought in with competency-based education in mind, where what you’re doing is that you are actively following the student and making sure that they’re not out there on their own, and that you’re making sure that the student’s not floating and that you’re seeing that, “Okay, do they need interaction? Do they need some intervention?” Or the second part of that is what I alluded to before, or that the student says, “Okay, I need help with this,” or “I’m ready to move on to the next part of my competencies.” So that’s the other part of it is the bringing that together in terms of monitoring the engagement, so something that’s predictable, scheduled, and then also, outside of that, that you’re actually monitoring and interacting with the student.

John: Are there any requirements concerning the extent to which there should be interactions with individual students as compared to interactions with the class as a whole?

Russell: Well, both of them count. So if you’re giving a lecture, or if you’re doing a group discussion and doing that, that that’s a group thing, and so that counts. And also, remember, the second part of the regular was that you are responding to student requests, and so that’s on an individual basis. And when there is a federal financial aid review, or ”audit” as it’s called, of your institution, what they’ll do is they’ll go and take a sample of classes, and that they’ll look to see what happened in that course? Were there group interactions? Were there individual interactions? That they’ll look to see what happened and then they’ll look to see, does it meet the regulations? And have you developed faculty? Have you let them know that these are the expectations of them? That they’re looking for those sorts of things, and did it actually have an effect in the courses?

Judie: So what would be some examples of regular and substantive interaction that we could build into a course?

Russell: Yeah, that’s a really good question. You’re an instructional designer, you plan the whole course out, right? And you’re going to get more points on the regular side for the predictable and scheduled if you have a syllabus, that… it doesn’t have to be detailed down to every last thing that you’re going to do… but at least you’re showing that what are the expectations along the way? That, when are you going to have assessments? That, if you’re – let’s say that for some courses – that you’ll send out a video with a different lesson every Monday or every Friday, what you’re doing is that you’re hitting the marks on the regular, another is that you are showing that you have some expectations, in terms of the feedback and the feedback loop. And sometimes institutions have this as a policy, sometimes they leave it to the faculty member. But the more classes that you have where you show that the faculty person… you don’t have to respond immediately… but they’re showing that they will respond in 72 hours, not counting a weekend… that they will respond to the students. And there’s that expectation that they will do that because I had a question from one, that they had a faculty member that they put the discussion out there at the start of semester and then graded it at the end of the semester [LAUGHTER]. And first of all, that’s terrible teaching practice, let’s start with that, nevermind the regulations, but that faculty person thought it was good. And the other is, that there’s this expectation that, as you go along that you’re working with the students. So those are some examples of things that you’d want to do there.

Judie: So just a couple of different things that I like to try here is, I really like to try to encourage faculty to give feedback prior to the next due date. That just makes sense, so that if a student is making an error, they’re not going to repeat the same error because they haven’t gotten their feedback yet. So I just think, pedagogically, it makes sense to give the feedback as soon as possible. But I also like to just have faculty create a communication plan when they’re developing their course. I think faculty plan in their heads, you know, “I’m going to send this announcement, I’ll do this feedback, I’ll do X, Y, and Z.” But when they sit down and really map it out in a communication plan for their course, I think that really helps get into that regular schedule. And whether it’s date driven, or day-of-the-week driven or at specific points throughout the course when students reach different milestones, I think that really helps them. My understanding is that those types of things would help people meet the regular and substantive interaction definition. What do you think of that?

Russell: I love both of those ideas, and actually that, really, if you work with the instructional designers, look at what works in terms of good pedagogy, that these are the things that you’re talking about is that having a plan ahead of time and being open with the students so that they know, and then getting back to the students in between assessments so they have the feedback in terms of, they know where they might be falling behind a little bit in some areas and so that they know, “Oh, I don’t quite get that concept. And then, I have a math background and that was so key in mathematics that, if you don’t get this one, you’re not going to do any better on the next test right? And that’s probably true in so many different fields as well. And so, I love both of those ideas in terms of doing things and where you’re informing the student, and then keeping them engaged and then constantly moving them forward.

Judie: It’s great to hear that kind of feedback from you, thanks! [LAUGHTER] I hear a lot from faculty now, especially during COVID, when many are teaching in Zoom. And so they’ll record a live lecture with their students and, assuming that FERPA rules are followed, and there’s no students caught in the video or audio, they want to just show that recording again in the next semester, and want to know if they’re meeting regular and substantive interaction that way. And I tell them that, when they’re giving their lecture, that is regular and substantive interaction when they’re engaged with their live students. But I say that once you make a recording, and put it in the course, it becomes course content, because it’s no longer a unique experience with those individuals talking about their understanding of the course content. And I see like a real fine line there and I wonder what you think of that? Or how that might be interpreted? What do you think of using old recordings versus always expecting some sort of fresh and unique interaction with the students?

Russell: Yeah, I think if that’s all that you did, I think that you’d have a hard time in terms of meeting the regular and substantive interaction, and this is the one where we get back to the direct instruction question on that one, and we did gather that question plus several others and pose them to the Department of Education, because we felt that even since they released the rule that we were hearing different things from them than from the accreditors. And so I’m a little hesitant to give you a yes, that works or not, under the new rules or not until we get a better answer from the Department. My feeling was that, under the old rules that I felt a little bit better about that that probably was problematic. It might still be, but I’m kind of curious to see what they say about the synchronous versus asynchronous going forward. I think that, if that’s all that you relied on, I think that that’d be problematic that you would need to have other sorts of interactions that might make that work.

Judie: Thanks.

John: But videos that were custom created for that week’s activities, or that provide feedback for the class would count, right?

Russell: Yeah, the ones that are custom created.

Judie: I encourage them to make like small targeted videos for clarification. Like, to address a specific topic that they know that students struggle with, as opposed to just making an hour video of you standing at the front of the classroom, talking to people that future students don’t even know.

Russell: Yeah, and I think if you just use the video over and over again, and I certainly saw this in some engineering courses where they’re using the same ones that, what happens is that you have to update your materials every once in a while, too [LAUGHTER].

Judie: Oh, sure

Russell: You need to be doing that. And so I remember witnessing a course where they were falling behind on some facts or raised a lot of questions about advances that had happened after the video had happened, or were quite clearly dated, because they were talking about things that were going on in space as a future thing, instead of a past thing. And so I think, if nothing else, that you’re going beyond whether you meet these rules or not, that you’re diminishing confidence of students in terms of the value of what they’re receiving.

Judie: Sure, that’s a good point.

Russell: One other issue that we didn’t touch on so far had to do with the definition of an instructor, and that was a difficult one for us. And that was another one that Western Governors University got hit on this one. It was a finding against them. And that it seemed like some of the definitions meant like an instructor, that there was one person that was in front, and that a lot of institutions have gone to team teaching or bundled instruction, or using GAs or TAs or there’s several people in the course and with the unbundled instruction that WGU did that they had one content expert providing the content of the course and another one doing the assessment and maybe somebody else doing some of the advising. So you broke it up and that they weren’t counting that even though it was approved by their accrediting agency. So that is one where we have worked on “instructor” and we’re very clear in this, that it is what is approved by the accrediting agency that that is what counts. T hat was sort of alluded to before, now it’s very clear. And so if you have a non-traditional sort of model for your instructors or faculties, you may want to talk to your accrediting agency about how they view that and get something in writing about that.

John: I think that is a pretty common issue where there’s often a master course developed by the content expert and then again, there are instructional teams that work on the whole course, but the division there can vary quite a bit. And I know there’s a lot of interest in institutions in trying to scale online education to make it more efficient, and this is an area that certainly needs to be addressed with the accrediting agencies.

Russell: Definitely, definitely, yeah.

Judie: So these rules that we’ve been talking about also addressed student identity verification. So is the student identity verification related to the regular and substantive interaction? Or is this another area that requires a more precise definition?

Russell: This is actually a whole other area that was in a different part of the regulations, and this is one that actually went into effect in July of 2020, and it’s part of what the accrediting rules are, and there’s a whole list of things that the accrediting agencies are supposed to be looking for when they’re doing your accrediting reviews. One of the things that they’re supposed to be doing is making sure that the institution has, really, policies and processes to make sure that the student who enrolls in the course is the same one who’s completing and submitting the assignments in there. So it really is about academic cheating, and that this is only in the distance-ed world that they have to do this. I have to tell you, in a subcommittee, we tried to expand it and got beat back. So, sorry we lost that fight for you. But it’s still in distance ed where the accreditors are expected to check for that to make sure. And then the big change that happened in that is that, previously, there were some, what were considered ‘“examples” in there. And one was that you had some sort of ID, some sort of login ID for that or that you did proctoring, and that those were meant as examples. And those were taken out, because all too often, what would happen is that an institution would say, “Well, we have an ID!” and they would do nothing else. And so that was clearly insufficient in terms of doing it, and so, they’re raising the bar. The intent is that you have a plan, and that you’re executing the plan, and that you’ve worked that out with your accreditor, and that when that financial aid review happens, that you will be able to demonstrate what you’re doing and if it’s effective. Another part of it, there’s a second section to it, that also talks about that if you have additional costs, and so, let’s say that you’re using a proctoring software, and that that costs so much per student, that you have to notify the student at time of registration, that there’s an extra cost for that. And this is something that a lot of institutions have fallen short on, because what they’ll do is that they’ll notify the student in the syllabus, and so the first day that the student starts they see the syllabus, and all of a sudden they have to pay more money. And the idea is that the student should be able to have a choice at the time they’re picking between which course that they might take or know that they’re going to have an additional cost for participating in that course. And that rule is out there, and it’s a good one, because you’re being clearer to the consumer about what’s going on.

John: So authentication with a password to a course management system is not sufficient, and some type of proctoring software is, but there’s a lot of concerns raised with proctoring. Are there any other ways to authenticate students that meet the requirements without moving to software proctoring solutions?

Russell: Yeah, and I think that over the last year that we’ve seen, the concerns about proctoring software have risen to the fore, and there’s some good ones and that they do some good things, and so you should not throw them all out. But yet, you should pay attention to the concerns about that. But there are ways that we have worked on this in terms of different ways that faculty can work in terms of their assessments. One of the things we talked about is face-to-face proctoring, As distance ed grows, though, that gets to be harder and harder to find enough proctoring sites and the ability to do that, but that is an option. Some of the other things that have been proposed have to do with how you do assessments, and that having more frequent assessments and doing things where it’s easier to take the course than it is to cheat. That if you do like one or two big assessments per term, that it’s a lot easier to get someone to do those for you or the big papers and all that, so that’s one. There’s others where getting people involved in terms of group coursework, or other sorts of authentic assessment type of things, where you get involved in different sorts of things, where you have to stay engaged more and more often through the course, and it’s harder to get somebody else to do that for you. There are people who will take the whole course for you… that’s a problem. But the more barriers that you can put up, and we really do love our instructional designers, but the more that we can do to help faculty with thinking about assessment strategies and effective assessment strategies, the better off that will be with all of this. Aso there are other areas like… oh shoot, I’m blanking on… ICAI, they have a lot of strategies as well out there, and for some reason, it’s early in the morning here and I’m blanking on their name, but there’s an institute for academic integrity that has a lot of good resources on this issue.

Judie: Yeah, I think it’s great to encourage all faculty to work with their instructional designers on authentic assessment.

Russell: Yes, yes, yes! That success will be more sure if you work with your instructional designer.

Judie: So, do you think that this authentication concern is only for assessments? Or is it for day-to-day coursework and interactions, too?

Russell: Yeah, it’s really about any type of quiz or paper or anything that you’re going to be evaluating the student on. That’s really what it’s looking at, because there’s the opportunity for cheating or something bad to happen there. So there are other things that you do in terms of papers, you know, with Turnitin or other sorts of activities that people do or trying to create papers that are more authentic or real: “Write something about your hometown or work on a project in your hometown where there’s not a lot of papers.” [LAUGHTER] So, you can do those sorts of things where it’s harder to plagiarize.

John: It’s really nice to hear that open pedagogy projects, videos that students create, where they’re actively engaged in it, group projects, and all those things can serve the same role without moving to the extreme of proctoring.

Russell: Yes.

John: It is good to note that any courses that require proctoring must list that up front so that students are aware of the cost. And the other issue with that is, as colleges enroll more first-gen students and more students from the lower income quintile, many students won’t have computers or networks that will necessarily support proctoring software. If students are working on their course through their smartphones, most proctoring solutions don’t work with smartphones. But it’s pretty easy for the student to take a video of themselves talking about something, so allowing faculty to have more options for authentication is something that allows for a more inclusive learning environment.

Russell: That was a huge lesson from the move to remote learning due to the pandemic, that you had so many students who did not plan to be in a remote course that uses online tools… that they were using, as you said, cell phones or different types of tablets that were not compatible with some of these proctoring software solutions, or that they didn’t have the adequate bandwidth for taking the test in that way with a full video… that that was a real problem, privacy issues with it. So there were all sorts of things that were problematic with that. And so, being upfront with the students is very good and we do often cite that there wasn’t a federal finding, but it was a student in Nevada, who was in a course and they were not notified that they were going to be using proctoring software. The student was pre-law, decided to flex his pre-law muscles, and got a whole bunch of students behind him and took it to the institution and ultimately went to the Board of Regents there, and all of those students had all of their fees repaid. It wasn’t a federal finding, but what was shown was that they were out of compliance with the federal rule, and the Board of Regents decided to remedy that.

Judie: So looking at regulations like this, we can see that online classes are held to a higher standard than face-to-face classes. Do you think similar requirements should also be implemented for face-to-face classes?

Russell: Well, I think what’s happening is that the vast majority of courses are now digital courses. And whether it is fully online, whether it’s hybrid, hyflex, blended, or just the old term that the OLC used to use of “web enhanced,” that you use the web a lot, even though that you meet face to face Monday-Wednesday-Friday, that we’re seeing all these digital tools going throughout. And what happened with the pandemic? It really took off, right? And so there’s even more of that is gonna happen. And some surveys that one of our organizations, Every Learner Everywhere did, that there was more interest and uptake from faculty in terms of, “Well, I’ve done this now I should do it again.” And so I think what’s happened is that we’re gonna see more and more use of digital technologies through, if not every course, the vast majority of courses. Well, the thing that happens is that these same sorts of problems are in all of these. And you and I know, keep this as our secret between us [LAUGHTER] but, all this stuff was happening even without technologies, right?

Judie: Right.

Russell: A lot of the cheating scandals in some of the service academies in the last few years had very little to do with technologies other than sharing some information. So we know that’s going on. So it’s gonna be interesting. This is one of the things that we’re looking at that I bet we’re going to see a lot of new guidance coming out of this Department of Education that recognizes that and may expand this out quite a bit more, because there are people in the department who have huge concerns about consumer protection issues and the use of online or digital learning regardless of where it’s used, and that they’re seeing that some of these rules need to be applied more broadly. So I wouldn’t be surprised to see that we get guidance that says exactly that coming out and that we have James Kvaal was just finally approved as the Undersecretary there. I think that that staff has been working on these sorts of things, and have been waiting for him to be approved. I would not be surprised that in the next three to six months that we don’t start seeing some new guidance coming out or answers to our questions. I don’t think they wanted to answer our questions until James Kvaal was in. And so I think that we’ll see clarifications and guidance about some of these things where we’ve had questions before, and how do they apply in a hyflex setting? How do they apply in a blended setting?

Judie: That’s good news.

John: And whether the rules are expanded or not, they’re just good practice… that regular and substantive interaction is good pedagogy.

RUSSEL: Yes! [LAUGHTER] Yeah, you really nailed it with that. And that was one of the things that surely the people that run the subcommittee and then the main committee were trying to look at: “What do we do that makes sense in terms of best serving the students?” And we have to remember at the end that these are consumer-protection practices that have to do that, there are also federal-financial-aid protections that aid is going to worthy activities, and so we need to remember that in all this.

John: We always end with the question: “What’s next?”

Russell: Well, I kind of previewed thata bit, that I really do believe that we’re likely to see several more clarifications coming out, or maybe some surprises coming out in the next few months. And so I think it would be good for people to pay attention to what’s going on, and we certainly write about whatever comes out in our WCET Frontiers blogs. So, be watching for that. And on something completely different, that we’re getting together some folks to work on the issue about veterans and their housing allowance. And just quickly on that, that veterans who take all their courses online, get about half or a little bit more of the housing allowance of veterans who take just one course face to face… they could have it all online, but just one course face to face. And it really is antiquated thinking, and it’s something that we need to get fixed. Because, I could be the same student in one term, take all my courses online, and the next term take just one course face to face. I have the same housing cost, I have the same family [LAUGHTER] I still need to eat, but somehow my aid is less. And so we’re working on that one.

John: And you shared many resources with us that we’ll include in the show notes, so those will be available on the website. Well, thank you for joining us, this was really helpful, and I think it’s going to benefit a lot of institutions and a lot of faculty and instructional designers as they plan for future semesters.

Russell: Well, it was a great pleasure being here with you today, and having a little bit of tea in the morning is always good. And so thank you, Judie, thank you, John for inviting me and for having me here.

Judie: Yes, thank you. This was fun. Take care.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

79. Self-Learning vs. Online Instruction

Posted in higher education, learning, pedagogy, regulations, teaching
May 1, 2019

Research shows that online classes are most effective when there is substantial interaction among the students and between the students and the instructor. In this episode, Dr. Spiros Protopsaltis and Dr. Sandy Baum join us to discuss the possible adverse effects of proposed changes in federal regulations that may reduce the extent of this interaction.

Dr. Protopsaltis is an Associate Professor and Director of the Center for Education Policy and Evaluation at George Mason University, and he was a Deputy Assistant Secretary for Higher Education and Student Financial Aid at the U.S. Education Department during the Obama administration. Dr. Baum is a Fellow in the Center on Education Data and Policy at the Urban Institute, and a professor emeritus of economics at Skidmore College.

Show Notes

Transcript

John: Research shows that online classes are most effective when there is substantial interaction among the students and between the students and the instructor. In this episode, we discuss the possible adverse effects of proposed changes in federal regulations that may reduce the extent of this interaction.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guests today are Dr. Spiros Protopsaltis and Dr. Sandy Baum. Dr. Protopsaltis is an Associate Professor and Director of the Center for Education Policy and Evaluation at George Mason University, and he was a Deputy Assistant Secretary for Higher Education and Student Financial Aid at the U.S. Education Department during the Obama administration. Dr. Baum is a Fellow in the Center on Education Data and Policy at the Urban Institute, and a professor emeritus of economics at Skidmore College. Welcome.

Spiros: Thank you for having me.

Sandy: Glad to be here.

John: We’re glad to have you here.

Rebecca: Today’s teas are…

Spiros: Coffee for me.

Sandy: Water for me.

[LAUGHTER]

John: I’m drinking Ginger Peach Black tea.

Rebecca: And I have English Afternoon.

John: One of your usuals.

Rebecca: Yeah. We invited you here today to discuss your January 2019 paper on online education entitled “Does Online Education Live Up to Its Promise? A Look at Evidence and Implications for Federal Policy.” In this paper, you raise some significant concerns about the outcomes of online education. What prompted your interest in this topic?

Spiros: The conversation in higher education around the role of technology in tackling a lot of the major challenges in the sector has been going on for a long time. In recent years, we’ve seen the conversation focus on MOOCs, most recently on competency-based education, but before all that, the big focus was on online education. And there was a big debate in Washington that unfolded over a decade or so about whether online education should gain access to federal student aid, which it did in 2006. When I was working in the Senate for Senator Harkin, who was chairman of the Health Education, Labor, and Pensions Committee, we conducted very thorough and brought investigation of the for-profit college sector in light of increasing concerns regarding student debt, student outcomes, and overall predatory practices in the sector. And as we were doing that investigation, one thing that jumped out at me is that a lot of these actors—a lot of these schools that we were investigating—had a huge online presence and had experienced exponential growth in just a very short period of time. Obviously, that was the period of the Great Recession. So, as we know, enrollment in higher education is counter-cyclical to the economy, so part of it was explained by that. But this huge explosion in online education was something that I wanted to investigate a little bit more in depth. At the same time, there were these increasing conversations happening around online education and regulation. And a lot of folks were saying whether we should deregulate online education, whether there’s possibility for more innovation in the space, et cetera et cetera. So I partnered with Sandy to write this paper because it was an opportunity to do exactly that: to look back and see, what does the evidence tell us? Did online education live up to its promise as it’s advocates had stated when they successfully had the law changed in 2006 and gained access to federal student aid? What exactly is this potential and has it been realized? So it was a combination of many years working on higher education issues and constantly listening to the same argument of advocates on the innovation side of the equation, saying that technology holds the key for transforming higher education, and tackling it’s big challenges. And that if we deregulate higher education, and if we remove “barriers to innovation,” we’re going to unleash this huge potential of technology and we’re going to be able to reduce costs, get more students in college, and improve student outcomes. This was a great opportunity for us to take the time and to do a deep dive and to see what exactly is the evidence in this area?

Sandy: One of the things that is really notable about the conversation about increasing access to higher education and reducing the price of higher education in recent years is that the discussion so rarely talks about quality and about what students learn. So the idea sort of seems to be if we can get more students to spend less time getting more credentials, we’ll be in good shape, and that’s clearly not enough. So we really wanted to look at, if students do engage in this different method of learning, will they learn? Will they succeed? What does it mean for them?

John: What’s been happening to the share of students in online education?

Spiros: I’ll talk a little bit on the enrollment trends and then I’m going to pass it on to Sandy, who specifically focused on a lot of the review of the research literature on outcomes. We had seen an explosive growth in online education, almost one-in-three college students, over 6 million students, take any online courses. Half of these students—three million students—are enrolled in exclusively online programs, what we would call fully online programs, while the other three million take at least one, but not all of their courses online. So one-third of students today are involved in online education, one-sixth of students are enrolled in exclusively fully online education. Now, when you look at for-profit colleges, and just so we get a sense of the concentration—or rather the disproportionate share in the sector—for-profit colleges enroll six percent of all students, thirteen percent of students taking at least a course online, but a full one-quarter of fully online students. Eighty percent of students at four-year for-profit colleges take courses online. That’s more than two and a half times the rate at public institutions and triple the rate at private nonprofit higher education institutions. Today, seventy percent of four-year for-profit college students are fully online. So in other words, when you look at the entire four-year for-profit college sector, seven out of ten of their students never have in-person contact with peers or instructors. They are fully online. That is seven times the rate at public institutions, and a three and a half times the rate at nonprofit institutions. So big picture, online education enrollment is booming, it has experienced an explosive growth, and it has been disproportionately significant in the for-profit college, and how it coincided with a change in the law, I think you can reasonably say that, yes, there is a relationship there for sure, and that was a significant driver of the enrollment growth. But I would like Sandy to weigh in on these points as well, because she’s focused on that.

Sandy: So just a couple of clarifications about this enrollment growth. One is that although the growth has been much more rapid in the for-profit sector than in other sectors, and the share of for-profit students enrolled online is higher—much higher than the share in other sectors—large public state universities are seeing big efforts in the online space. I mean, you could interpret this as trying to provide opportunities for different groups of students, or as looking for a revenue source. So the revenue is also an issue for these public institutions. And there are a number of private nonprofit large institutions—just a very small number—that enroll thousands and thousands of online students. Another issue that is really important to understand is that much of the research that we looked at focuses on courses that are taught fully online. So there’s a distinction between your whole program being online and never seeing a human being, and just taking one course online. But there’s also a distinction between taking a course that is fully online and of course, that is what we call a hybrid course, that adds technology to some classroom experiences. And the evidence is that these hybrid methods are actually quite good for students and that the outcomes are at least as good as they are in traditional classroom courses. It’s the fully online coursework that we’re talking about, and about which there is a lot of evidence of questionable outcomes.

John: Earlier you noted that one of the arguments for increased use of online instruction is to lower costs and increase access. Have costs been lowered, either to students or to institutions by expanding online programs?

Sandy: By and large, no. There is some evidence that at institutions that do a lot of online education there may be some lower tuition prices, but the fact is that tuition is frequently higher. So it may be possible in the future for online coursework to save money. We don’t want to say that we know that that can’t be the case—because it’s logical that it could be the case—but right now, it hasn’t happened, and some of that has to do with individual institutions reinventing the wheel of course, and big investment overall in online education could in the future solve this problem. But today, no, it’s not cheaper.

Spiros: One point I wanted to add to what Sandy said is that it’s also an issue of incentives here. We obviously have University of Maryland, University College that has been one of the big players in the space for a long time and Arizona State University has always been a player, they’re announcing even more of an expansion. And as Sandy mentioned, one of the reasons is revenue. In other words, a lot of institutions see online education as a way to get more revenue. Then the question is, even if they become successful in reducing the costs of production—which as Sandy correctly pointed out, raises significant issues around quality—what would be the incentive to pass on the savings to students? So in other words, in many ways online education—even if it has the potential to reduce costs of production—the big question remains, is that going to translate into reducing the costs of attendance for students? And I’m not sure that the incentives are there for that to happen. And one of the things that we have seen is that not only do they charge the same tuition as they do for their regular programs, so if the university let’s say has a Bachelor’s in psychology, which is a brick-and-mortar program and then it offers a Bachelor’s in psychology online, and we see that they charge the same tuition, then they add on top of that a variety of technology fee s, and other things just for online, which ends up making it more expensive. And so there was one conclusion that we reached on the paper is that no, online education has not realized its potential of reducing costs in higher education today.

Rebecca: I’d like to focus a little bit on the quality issue that you’ve raised multiple times now. There’s a wide variety of courses that are online, you mentioned, certainly recording a video and putting it online and allowing many people to see it is very different than interactions that you might have with students. Can you talk a little bit about the differences in quality and what some of your findings were in that area?

Sandy: Well unfortunately, a lot of the studies that compare outcomes among students in fully online versus other courses don’t actually go into much detail about how those courses were taught, so the careful studies of this are rather limited. What is clear from all of the evidence is that interaction, personal interaction, makes a difference. That when you look at students who have no interaction with their classmates or with instructors, they are less likely to get good grades, they are less likely to complete their courses, and they voice this concern. In particular, it is vulnerable groups of students who suffer from this lack of interaction. So you see that students who are well prepared, who already know how to study, who are motivated, know how to learn, have good GPAs going into the courses, are most likely to do just as well in a fully online course as they would in a classroom course. But it’s those with weaker preparation, it’s students with low GPAs, it’s black and Hispanic students, students from low-income backgrounds. These students do very poorly in these courses, so that the socioeconomic gap in outcomes is greater in online courses than it is in classroom courses. This is a huge problem because people actually are hoping that online courses will make it easier for adult students, students with other responsibilities, work responsibilities, children, that for them—where the travel to a physical location is more difficult—this online learning could be the saving grace and that they would learn more from this. And those are the very students who are struggling most from it. And there is just a lot of evidence that the problem is learning—and we know this from the psychological literature as well— learning is a social process. I mean, we can all read something, but it’s not just a matter of pouring information into people’s heads, it’s learning to solve problems, it’s engaging in discussions, and for people who don’t have a lot of experience with that, the interaction is particularly important and the fully online learning environment is particularly risky.

Rebecca: You’re kind of emphasizing the idea that having an expert in the field to help guide the way students are learning how to think in the discipline is particularly important?

Spiros: Yeah, substantive expertise is key. The instructor has to be a subject matter expert, and has to interact with students. That is the essence of education. The educational process is intrinsically an interactive process. When if you go and look up in Oxford the definition of education, it’s gonna say, it’s between a teacher and a student interacting. I mean, you can’t have an education without the interaction. What is happening though now is that we’re seeing a glorification of self-learning. We’re seeing a glorification where you have students interacting with software programs, students interacting with technology, and that might be fine, but that’s not education. That’s self-learning. And the big distinction in the federal law between correspondence education—which is self-learning—and distance education is that in distance education—online education—students have to interact with an instructor on a regular basis and in a substantive manner. That regular and substantive interaction is legally, statutorily (in law), a regular education. And what was surprising to us is that in looking at the evidence we uncovered a large number of studies making that exact point. How from a pedagogical perspective, interaction between students and instructors is absolutely critical. And we found evidence that that is what increases student learning, that is what increases student satisfaction, that is what improve student outcomes. We reviewed those studies and the verdict was in, you got to have strong interaction. And so the current efforts by this administration to water down those requirements is very concerning because it’s going to blur the lines between self-learning and education. And interestingly, our paper coincided with a beginning of the negotiated rulemaking process at the Department of Education on a host of issues, including this one, under the general guise of promoting innovation and flexibility in higher education.

John: Actually, I think at least one of the studies that found this was a study of online instruction at the State University of New York by Peter Shea who found that classes that had more active instructor engagement and more interaction were more successful. But adaptive learning, on the other hand, has been shown to be effective in face-to-face classes, and I believe online as well. But I don’t think the ideas are mutually incompatible – that having students work individually with computer mediated instruction can be useful as long as they’re also interacting actively with the instructor and the class community and getting feedback on their work. So I don’t think we have to rule out adaptive learning tools to say that having active instructor engagement in the class is really important to learning.

Sandy: Adaptive learning is a whole other thing from what we’re talking about. And so we’re not saying that technology can’t help—of course, technology can help—and adaptive learning can be terrific. It’s not cheap, but it can be terrific. We’re not saying that there are no good fully online courses, either. I mean, that’s not the question. It’s about how most of this is done, and how it’s done, if it’s not carefully designed adaptive learning, and if it doesn’t involve personal interaction with subject matter experts.

Rebecca: In addition to the personal interaction, were there other things that you discovered would be really useful to help those populations that are particularly at risk that you mentioned earlier?

Spiros: I’m gonna ask Sandy to weigh in as well. But I can tell you that what we know from the literature about student outcomes in higher education in general, beyond what we examine in online education, is that academically underprepared and at-risk populations—including students from lower-income families, first-generation students, and the students with remedial needs, I mean, what we would call in general at-risk populations in higher education—they need significant academic and student support services to help them succeed. That requires a significant investment from the institution. It requires a purposeful and meaningful strategy that specifically is designed to address the unique needs of these populations. Of course, it’s difficult to cover all of the research and the literature in my comment, but we know that it takes a whole lot to support these students and to ensure their success. Everybody talks about the CUNY ASAP program, which has become the new poster child of what it takes to have tremendous success with these populations. That’s a very expensive program. It’s a great program, but it’s also a very expensive program. It again confirms what we know, which is: we know what needs to be done, but you have to have the investments and you have to have a purposeful strategy. But Sandy, I would like for you to weigh in on this as well.

Sandy: Yeah, I think it’s really important to understand that all of that evidence suggests that for at-risk students, for students with weak academic backgrounds, and with many factors that could interfere with their academic success, these students need really strong support systems, it’s not just about money. And it’s not just about time, it’s about the need for support systems. So it is certainly true that for many of these students, if they can take some courses online, this could actually help them to graduate because you can take more courses online, because you can do them in the evening and on flexible schedules. And even if you fail a lot of them still, you might end up accumulating more courses. But taking some courses online is not the same thing as thinking that you can just sit at home with your computer and get your whole education and succeed that way. Because being part of a community, part of an intellectual community, part of an academic support community, and part of a social support community is what these students need. Understanding that should make it possible for us to develop systems that take advantage of the things that online learning can contribute without assuming that it’s going to be the silver bullet, and that it can replace the important components of traditional learning. I think that’s one of the problems we always have is somebody sees something that adds an important component to the education system where they say, “Okay, let’s junk everything else and just do that. And that our evidence clearly suggests is not going to be a good strategy.

John: How do employers view online education relative to face-to-face education?

Sandy: There are surveys of employers, of faculty members, of academic administrators, of the public, uniformly they say that the perception is that online education does not produce the same outcomes as in-person education. Now, perceptions are not always a reflection of reality, but particularly for employers you would expect that perceptions would pretty quickly catch up to reality because those perceptions are, to a great extent, based on the performance of people they hire.

Rebecca: So you shared a lot of things that I think faculty or administrators who are developing online courses should certainly be thinking about. We’ve also talked a little bit about some of the legal issues that we might want to be concerned with. Are there things that faculty and institutions should be doing to make sure that quality is ensured, and that as institutions of higher education, and the value of education is held moving forward?

Spiros: Every industry needs to listen to its customers. Higher education needs to listen to students. And students surveys consistently emphasize the desire for greater interaction with their instructors and their peers, and so that is something that higher education needs to be responsive to. So even if they dismiss our paper, they dismissed what I think, what Sandy thinks, let’s just boil it down to the basics which is they need to listen to the students that they serve. I don’t like talking about students as customers or consumers. But just for the purposes of this conversation, they should listen to the consumers of their services, and so they definitely need to put a focus on that. They definitely need to understand that we have to ask if students need additional services. And they need to listen to their faculty. And we’ve seen many examples for faculty involvement in online programming hasn’t been that strong. Institutions are in a hurry to set up programs for revenue purposes and others, and they do not take the thoughtful approach that includes feedback and input from the faculty. So I think faculty involvement is critical. At the end of the day, faculty are the experts, faculty know pedagogy, faculty are the ones who work and teach the students, so I think that that’s another advice that that I would have. And again, I would be making the point that quality matters. It’s not just about just building the platform and going out there and offering online, you have to pay attention to making sure that it’s a quality education.

John: Speaking of Quality Matters, there is a Quality Matters rubric out there that’s used in many institutions to evaluate online education. And SUNY—along with the Online Learning Consortium—has also developed the OSCQR rubric. Will those types of things perhaps help ensure a higher quality of online instruction?

Spiros: Frankly, I’m not familiar with either of them. I’m not a traditional academic, I came out after eight years in government. So to be honest with you, I’m not particularly familiar with those. I don’t know if Sandy is, but I would say that all tools that are geared towards ensuring quality definitely are welcome into contributing that effort.

Sandy: So, what we do know is that the people who have worked on quality for online education agree that an important component of quality is making sure that there is substantive interaction. So these efforts seem to all be headed in a similar direction, which is, let’s make sure that we think about how students learn and that we incorporate that knowledge into how we design our courses. The problem that they’re going to run into is that it’s more expensive to do that. So it’s really important for the people who are focused on quality, who have evidence about quality, who care about quality, to stand up to the people on campus who are just saying, “Well, the goal here was to make more money, so let’s make it cheaper.” So yes, I mean, all of these efforts are really moving in the right direction and certainly compiling evidence is the best way to make sure that we improve outcomes at the same time that we broaden opportunities and promote innovation.

Rebecca: So you’ve mentioned that, on campuses, people who are emphasizing the quality are all moving in the right direction. But you also mentioned earlier that the legislation is moving in the other direction. So can you talk a little bit about that disconnect, and maybe some of the actions that we could try to take to, to help align those?

Sandy: Let me just say that—and Spiros is more of an expert on the details of the legislation—but there’s a clear movement under the current administration to reduce regulation, regardless, and there are obviously regulations in higher education and elsewhere, that are counterproductive that raise costs without improving outcomes, that burden providers and so on. And so looking at regulations is certainly worth doing and I think past administrations have done the same thing. But in this case, there is just a wholesale effort to remove the regulations that have been developed in order to protect students… to protect consumers. And so we just need to look at that and say, “Let’s look each time there was a proposal to reduce regulation and ask why that regulation was imposed, if there’s any evidence that it’s helping, and what the results of removing it would likely be,” and in higher education, it’s really frightening to watch and see how everything that in any way interferes, particularly with the providers of for-profit education, is assumed to be a bad thing. And we need innovation, we need to allow things to change and develop. We need new ideas. But we have a big history of the changes in the federal financial aid system, opening up to for-profit providers, leading to generating a huge growth in this sector that obviously included—well, it includes some quality institutions included—lots of organizations that were not interested in really educating students. So that could easily happen again, if we just say, “Oh, you have a good idea, you say it works. Go ahead, do it. We’ll pay for it.” That’s so obviously going to lead to big problems.

Spiros: Yeah, everything Sandy said was correct. And the one point I would make is people use the word innovation and I think it’s important to remember there’s good innovation and there’s bad innovation. Not all innovation is a good thing. Innovation means doing things differently, you can do things differently and have a better outcome and you can do things differently and have a worse outcome. So I think a lot of these changes that are being advanced by the administration, and here’s the disturbing part: Oftentimes, with the support of the higher education institutions, and the higher education sector overall—and I don’t know of any industry that loves regulation, higher education is no exception.—and it’s under this framing of “if we get rid of all these laws and all these regulations, et cetera, we’re going to have innovation.” And if we’ve learned anything from history—if we study history of regulation in higher education—is that every time we do that, there was a huge period of a lot of fraud, waste, and abuse, because the incentive structure in higher education promotes those behaviors. And we need to understand that when we change our regulation, we’re not just changing it for the good schools out there, we’re changing it for all the schools. So when you lower these barriers, when you remove these barriers—all of whom, by the way, are there for a reason—and the good actors might continue acting in a responsible manner, but there are a lot of bad actors who have an incentive to take advantage of that removal of the “barrier.” So when we’re talking about one hundred fifty billion dollars a year—that’s according to the College Board, when you throw all the aid in and Sandy’s the author of that report so she knows that well—but when we’re talking about one hundred fifty billion a year with tax expenditures and everything, that’s a pretty significant investment we need to safeguard and we need to understand that all of these changes actually have significant implications, most importantly for students, but also for taxpayers. So the stakes are high, essentially, is what I’m trying to say and we need to look behind the curtain of innovation and behind the veil of flexibility, and see exactly what are these changes meant to do and who is going to benefit from them. So just to give a specific example, a year and a half ago the OIG—the Office of the Inspector General—in the Department of Education did an audit of Western Governors University and found that they hadn’t really complied with regular and substantive interaction. And WGU is what I would call—based on the outcomes it has—decent student outcomes, affordable school, nonprofit, and does not have a track record of any sort of abuses of predatory behavior or anything like that. So it’s a pretty solid school that has a very strong reputation among policymakers. So everybody jumped to the defense of WGU saying, “Oh, no, no, no, you know, let’s be careful here, let’s not penalize them, because maybe they were not in compliance, let’s change the law to make sure that the law complies with WGU, rather than WGU complying with the law.” The point here is though, that while WGU might have been the specific example, changing the law is not only going to help WGU, it’s going to impact everybody. So while WGU might benefit, a lot of bad actors are also going to benefit from such changes. And the results from such a change with bad actors is probably not going to be something that we’re proud of or that we should be proud of. And so in politics and in policy it’s important to look not only at who is the obvious beneficiary of something but also who is the potential beneficiary behind the scenes, and we talk about that in the paper towards the end.

John: What recommendations would you have for faculty teaching online classes or developing new courses or programs in creating high quality online programs?

Sandy: Many faculty members who have a bias against online learning have found when they engage in it that actually it goes very well and their students learn, but those same people are very likely to say that they put more work into the online course then they did into their regular courses. So I would say that faculty should be open to the opportunity but they should not expect it to be just the easy way out, they should be prepared to learn how to teach well in this environment, to be prepared to put their resources and their time into getting to know the students, and into helping the students learn. And it’s quite likely to be a big investment at first, a bigger investment than just continuing your courses. Now we know that faculty members don’t all do a good job in the classroom either. So the idea is not that every classroom course is better than every online course. But faculty members need to understand that this is a big investment and they need to learn how to do it. They need to accept support from people who have experience at doing it well. And they have to urge their institutions to have the goal of doing this right, putting resources into it, integrating—again—personal interaction with the technology, and making sure that the primary goal is not just to save the institution money, but to really create positive learning experiences for students and I think if faculty and the administration share those goals, that there is potential for this to go a lot better. But the there’s a real risk of everybody just looking for an easy way out. I think many people agree on what is important, but they don’t know quite how to do it. And so learning how to do it… doing a lot of studying, examining the successful efforts. And then again, putting the students first is the best advice for people who are developing this kind of learning opportunity.

Spiros: I would only add, as well, for faculty to pay attention to what’s happening right now in negotiated rulemaking, that these changes that are being considered are going to have an impact on education, on faculty. And there are a lot of folks in higher education, who really want to downgrade the role of faculty in the educational process and want to minimize this importance of interaction between students and faculty, and would rather have students interacting with software and calling that an education. So I know that AAUP and others are paying attention to this, but I think it’s important for all faculty to be following these conversations because they have wide ranging implications that really, really impact our daily lives and how we engage in this practice,

Sandy: I think we really want people to understand that what we’re talking about is learning, education, and quality. We’re not talking about sort of making a blanket condemnation, either of technology and online learning or of the for-profit sector, but we are saying that these problems are very evident there and that we have to work this out, acknowledging where they are concentrated.

Spiros: The one thing I would I would say is this. From a policy perspective—from like the thirty-thousand foot level—the argument is that higher education has not been serving non-traditional populations well and that technology has the potential to provide access to opportunity for a lot of these populations. Based on what we found, and the bad outcomes that online education has been producing for these exact populations, I am very, very worried that if we don’t fix the issues identified in the paper, we are moving in a situation where instead of engines of mobility and equality, we are making online education engines of inequality. And instead of helping and addressing the huge achievement gaps and attainment gaps in this country, we’re actually going to be making them even more severe and augmenting them. So from a very big sort of high level, the concern here is that if online education is the tool that folks believe can help close the achievement gaps, is actually increasing those achievement gaps, then it’s urgent that we fix these issues. Otherwise, we are really making things worse instead of better.

Sandy: Yeah, I think that is really our most important point. And if the institutions that have ample resources put those resources into doing a really good job of adaptive learning and hybrid learning and so on, then the gap is going to grow even greater… that students who have had the preparation and the resources to enroll in institutions that do this well will have even better opportunities whereas students who are at risk and are going to the under-resourced institutions that are struggling to really provide the support that they need are going to end up pushed online into courses that are not carefully constructed in programs that don’t serve them well. And the socioeconomic gaps in educational outcomes and attainment are going to grow and that’s what we really want to avoid.

Rebecca: I think that the research that you shared should be really compelling to any faculty whether or not they teach online, to push their institutions to do the right thing to push their colleagues to do the right thing… to push their representatives do the right thing in the regulatory sphere.

John: We’ve already seen decades of worsening economic inequality and reduced intergenerational mobility and your arguments suggest that that’s just going to get worse if online education, which is a growing share of education, continues to widen the performance gap between first-generation and continuing-generation college students.

Sandy: That’s right.

Rebecca: So we always wrap up by asking, what next?

Spiros: What next? I mean, you know, as with everything, once you you write a paper on something, you do research on something, and it’s not like you switch off the light. We continue being engaged. We recently published a notepad with Sandy in the Chronicle of Higher Education and we summarize a lot of the arguments that we have in the paper. We’re participating in podcasts like yours and events and panels, and we continue to write about the issue and following closely what’s happening in D.C. on this. So I think we’re just continuing to spread and share what we learned in the paper with others, and hopefully, it’s going to have an impact.

Sandy: You know our main interest, I mean online education is one component and one symptom of the issues that we’re interested in. So thinking about access and success in higher education and increasing opportunities is the key. And certainly we expect our research to continue in that direction, but understanding the relationships. I mean, I think that there are a lot of cases where people think that a solution is a progressive solution that will really increase opportunities and it turns out not to be. And how can people understand the difference between something that sounds like it increases opportunity and something that really does increase opportunities is very difficult and very important.

Rebecca: I look forward to following your research. That’s a really interesting and important topic. So thank you so much for joining us and sharing your work with us.

John: Yes, thank you.

Sandy: Thank you.

Spiros: Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.