166. Course Villain

A number of online services exist that facilitate academic dishonesty. In this episode, Zachary Dixon and Kelly George join us to discuss Course Villain, a platform they created to detect crowd-sources plagiarism. Zachary is an Assistant Professor of English at Embry-Riddle Aeronautical College, and Kelly is an Associate Professor of Economics, also at Embry-Riddle Aeronautical College.

Show Notes

Transcript

John: A number of online services exist that facilitate academic dishonesty. In this episode, we discuss one institution’s project designed to automate the detection of academic integrity violations.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Zachary Dixon and Kelly George. Zack is an Assistant Professor of English at Embry-Riddle Aeronautical College. And Kelly is an Associate Professor of Economics, also at Embry-Riddle Aeronautical College. They are co-creators of the Course Villain tool, which is designed to track crowd-sourced plagiarism. Welcome.

Zachary: Thank you.

Kelly: Thank you.

John: Our teas today are:

Zachary: Sparkling water for me.

Kelly:I have hot lemon water for me.

John: Oh, that’s close.

Rebecca: That’s close enough to tea. I have Scottish breakfast.

John: And I have ginger peach black tea. We’ve invited you here to talk about Course Villain. I saw your presentation on this at the OLC Accelerate conference recently, and we invited you here so you can share that with our listeners. Could you give us a brief overview of how this program came about?

Zachary: Sure. Actually, we were at a college faculty meeting for all of the College of Arts and Sciences and plagiarism was kind of up for discussion a little bit. And I’m on the younger side of the faculty, or I was at the time, at least, and Kelly has children of her own to keep her more in touch, I guess, with the digital scene. And so Kelly and I were like, “Well, does anybody else know what’s happening on this place called CourseHero. And so many of our other fellow faculty did not know they’re kind of like, “No, what’s that?” …and we pulled it up on a screen on a laptop and really started to blow some minds. And so Kelly and I started talking about that and just got to the idea that, you know, this doesn’t seem like it has to be an impossible problem. This is more an interesting problem. Let’s see if we can game this a little bit. And we started coming up with some strategies and ideas on how to track this kind of behavior and see if we could use old Google Alerts, it was our first kind of ambitious attempt. We tried to manipulate Google Alerts into telling us when things happened on Course Hero , and it was a terrible failure. It didn’t really show us anything at all. And so that kind of prompted us like,”Wow, we need to do better.” And so we started calling the computer science department and trying to find some researchers, and it kind of just took off from there. We saw this one little issue and we thought maybe we could game it and play around with it, and it’s developed into something fun from that.

John: Could you tell us a little more about how the program you developed functions? What does it do?

Kelly: The program itself is a software function that it’s been built in a number of iterations with computer engineering students, and they use a web crawler and a web scraper. So you would put in your query. John, being an econ major, you would say, “Well, I wonder how much of econ 101 John Kane is on CourseHero.” And so you would put in the query the search features. And then what this program does is it goes into CourseHero, and it then generates hits, which are artifacts. And depending on the time that we have it running, it will generate a report to us, the user, of the 20 most recently uploaded artifacts that meet your query, so you can determine for your class what’s going on. So before it, old school, if you were a faculty member, you’d create a syllabus, and then you want to see what’s out there in the world that would make your course compromised. And at one point in time, we had it scanning once every 24 hours. You can set the duration, we can change it to 12 hours to 24 hours. The report comes back to us, and then we can look at the artifacts. Sometimes it’s a very top level screening by the instructor. We can say, “Yeah, that’s the final exam that shouldn’t be uploaded,” We had an option in the second part of the program is it auto generates that takedown feature that is so difficult to overcome with CourseHero. If you did it manually, if you have 40,000 documents, I think someone at our conference said last year, they generated something to the tune of like 60 or 70,000 takedown requests for CourseHero. And if you’re doing that by hand, that’s more than just one person doing that takedown request because of their requirements. So that was the bigger challenge, I think, in this program was to generate the takedown request. So what we envisioned was this program just kind of constantly working. So it’s constantly scraping and generating reports, and we were generating takedown requests. And Zack was the one who said it’s kind of the old Disney YouTube dynamic and phenomenon. Way back early in YouTube, back in the early 2000s, you can find every Disney movie across YouTube. And then basically Disney’s teams of lawyers and copyright content owners got involved. And they made it so difficult that now YouTube, they have algorithms and they have firewalls where you can’t upload Snow White on YouTube. And so that was kind of like in our perfect world, Where would you want to get to be where CourseHero says “Ah, that’s from Embry-Riddle. We don’t want it. It’s too difficult.”

Zachary: In terms of the functionality of the program, it’s deceptively simple, where really we just kind of built a very carefully tuned search engine, just like Yahoo or Google, where we’re essentially just leveraging the API metadata, that application program interface that the big dogs use. We kind of just built our own smaller version that does what we needed to, and that’s about it.

John: Have you tried using this with anything other than CourseHero, for some of the other sites out there for Chegg and and all the many sites that have been appearing in the last few years?

Kelly: No, not yet.

Zachary: No, we haven’t.

Kelly: It’s on that map of what you want to do. It’s all resources. You’ve got to get students to maintain the program and keep working it and to alter it a little bit. That would be the ultimate goal, to put them on the other big crowdsourced study aid platforms.

Zachary: And following up, it connects back to that leveraging that API metadata and that interface does restrict our ability to move between platforms, because we would have to re-tailor it, so to speak, kind of recut and sew it. And one of the challenges over the lifetime of the project now for coming up on three years has been exactly that, maintaining that interface. CourseHero is also always updating and revising it, trying to get better profiles on the search engines and maybe assumably trying to keep people like us out. So the maintenance keeps this one step working, and then that’s also kind of deterred us from trying other platforms.

John: So you’re searching primarily on the instructor’s name and the course? Is that the criteria?

Zachary: Well, you can define any criteria in Course Villain right now. The only kind of hard tuning we have is that we have manipulated Course Villain to only search for Embry-Riddle content right now. So we’ve restricted the search engine’s functions by our university, and that has allowed us to have better accuracy in our reporting, because before we are getting anything with English 123 would pop up and really cloud the results, so we had to kind of throttle back the scope.

John: One thing I’ve noticed recently with CourseHero in particular, is that my material now is showing up with me at nearby institutions. So students may have tuned into this because I have Eco 101 materials and Eco 350 materials at Cornell, at Syracuse University, and a few other places now, including, I think, one in North Carolina, which is my content that I created, and it has my name there, but students are getting a little bit… not terribly…but a little bit more savvy on trying to avoid detection by instructors. So, that may be something you want to look at.

Kelly: But, I think also one of the explanations for that is, not only our students getting savvy, Lord knows, they can re-engineer anything fairly quickly, instead of just doing the work. But if your material is being recycled, so primary, it’s yours at your institution, and then someone who’s just looking to do a paper or to do something is just plunking around out in the web and steal some of your things and brings it in and then when they recycle your material and upload it, because CourseHero has got that pay for play, you got to upload something. So, I can just upload this material, and then you move into the second and third generation of your material.

John: Interesting. I hadn’t thought about that possibility. Because this was material from last year, some of it was exams from last year. And I just assumed it was my own students who posted it somewhere else to share it with other students, letting them know where to look so that they could evade detection. But the possibility of it being recycled is certainly a real one.

Zachary: Seeing your own work not labeled appropriately. I also saw the same thing. I think there’s also like a level of student error involved where students will like create department names and college names that don’t exist at the university, and maybe they just don’t know. Something else, and Kelly observed this really well in our data analysis, was the issue of student agency is really hot here. And it’s probably like, three steps ahead of where we’re thinking, because students are gaming us, they’re gaming the university, and they’re gaming CourseHero too. They will not just post reasonable material. It could be John Kane’s exam, and it’s just like two lines of nothing text. CourseHero doesn’t appear to have a lot of oversight over their own collection and so the students are really steering the ship in a crazy way.

Kelly: The one thing that made me look at that was when we were doing an analysis on one of the first reports and these four documents for like a physics course kept popping up. And they were like almost management case studies or something like… notes on Boeing or something… they kept popping up. And then I drilled down and what I realized was one student must have been doing the pay for play and the student just took six or seven documents that were readily available, not necessarily their documents, like their paper, where someone could actually go in and if you didn’t remove your name or anything like that you can see. My students examples are always Joe Bagadonuts. So, you know, it’s Joe Bagadonuts’ English paper or economics paper. So this individual was just uploading these PDFs, and these documents, just to get their ticker up so that they can download documents to compromise.

Rebecca: Do students know about Course Villain at your institution?

Kelly: They do now. [LAUGHTER] Marketing finally got ahold of us. We’ve had a little bit of localized publication. And then actually, from that, a fellow colleague of mine, who didn’t know we were working on this, came back and said, “Here’s an idea,” he actually appealed to his students, and said why you wouldn’t want your work up on CourseHero. And through that, in his small class, he was actually pretty successful in getting them to take down the work. And because I asked him, I was like, “How did you do that?” And he said his line was “Well, I asked them politely,” but then he gave an explanation of how he appealed to students. And it got me thinking that, in our next iteration, instead of just trying to prevent or catch the plagiarism after the fact, that that would be a good preemptive strike, to create some type of a PR campaign with our students in our university to say “Here, you can help us out here. This is why you don’t want your material up on CourseHero.” So if we came at the problem from both ends, it just hadn’t slipped our mind. That was eventually our end goal. My end goal was always that as a university, our honor code system would expand so much that our students had so much pride in the honor code system, that they would work to elevate that. And that would be one idea of how to start out that prong of the research.

Zachary: One interesting student dynamic has been our student researchers, our programmers. So we’re pretty proud of the fact that we have funded all of Course Villains so far with internal grants through the university, and we’ve only employed student programmers to do the work. Neither Kelly nor myself are capable programmers. I can maybe throw together some HTML if I tried, but we really have been leaning heavy on our student programmers. And they’ve had a really positive response to this experience. They’re aware of CourseHero when they come into this and the kind of that ilk, and I think they appreciate the hacker mentality that we kind of foster. We kind of have a cool proverbial David and Goliath situation here, this kind of huge onslaught of material and this kind of corporate juggernauts. And compared to just a couple faculty members and a small budget, it does create a cool atmosphere. And I will say, with no judgment on anyone using or not using CourseHero, though, when we do get our student programmers involved and we kind of sit down and talk about it, I definitely get the sense that diligent students appreciate this work in that it is combating something that they understand undermines the quality of their education experience.

JOHNr: How have your colleagues responded to the reports you’ve generated on the frequency of materials from their courses appearing on CourseHero?

Kelly: Shocked. It would shock any instructor. Disheartened, and then engaged. They want to use our software right away. And, because not only do they want to pre-empt, they want to get their own material taken down. They want to catch the students that are flagrantly violating academic integrity, and they want to use it. So, they’ve sprinted way, way ahead of us. So, the motivation is there. I think it’s a matter of trying to figure out where in the grand piece of the puzzle of our institution that this product can work.

Rebecca: How many takedown notices has Course Villain created or generated?

Kelly: I was not prepared for that question.

Rebecca: Oh. We can cut it out.

John: I hadn’t thought of it either., but it’s a good question.

Rebecca: But I am curious. [LAUGHTER]

Kelly: No, I wouldn’t take it out. I just think, wow… That’s a good question. Actually. We were too busy the last two years. The first year we did was “Could we create this mousetrap?” The second year was dedicated to “How well does this mousetrap work?” And then this next year, this year that we’re in, was really a question of “How can we incorporate the use of this? How effective is it? And how do we incorporate it into our instructional design process and get it into our workflow to where it helps us rather than becomes just an obstacle you had to come around?” So we don’t have the data on the takedown requests. I’m gonna get it though. [LAUGHTER]

Zachary: The early experiments that I can definitely recall and speak to on the takedown requests have been pretty largely unsuccessful insofar as CourseHero is looking for specific amounts and specific kinds of verbiage in that takedown request. And since we’ve been more focused on, like Kelly said, program development and stabilisation, we haven’t been able to focus in on “What is that verbiage that we can achieve from a programmatic perspective? Like, “Can I make Course Villain say the right thing that will work every time?” or do I really have to work with an instructor level basis, like “You’re gonna have to supply this part of it,” because we can auto populate the whole form and send it on. But, so far, the couple dozen I’ve tried (focused on trying to make the thing work) have all gotten bounced back to me, you know, “More information is needed.” Essentially, you need a better argument. And the next step is largely experimental. Like, what is that? And it’s a lot to do with institutional workflow, really clarifying, like, whose intellectual property is what and why? What are we able to defend and argue? Some of these are questions for the development team, for Kelly and I, some of these questions for administration at our university, to help us chisel out here, to have that ability to kind of go back to CourseHero. You get to that legal end of things and things get weird, I think.

Kelly: Within our college, I just taught a master’s level class and now all the major assignments that are submitted in this research levels class does have the copyright at the bottom. Remember, Zack, about a year ago, we were talking about that? Well, if all Embry-Riddle submitted assignments had this statement at the bottom, like in the footer of the assignment, then it would be protected. That was one of the ways to combat the changing language and requirements of CourseHero. So if you can say “No, this is copyrighted, I’m a representative of Embry-Riddle. So, you do have to take this information down.” But I did notice that, in the last class that I taught, the research methods, that that was a fundamental change.

Zachary: There is also a pretty murky or wicked issue in terms of disciplinarity. And what exactly students are posting in there. And so there is kind of a level of judgment that has to be made where, “is this appropriate to be taken down? Is this truly our/my/the university’s intellectual property or just does that belong to the student?” And as I study academic integrity, and plagiarism, in particular, that line is really down to communities of practice, professional organizations, disciplines, courses, and instructors themselves. So that line has to be teased out too. That’s kind of worth mentioning.

Rebecca: So in addition to tools like this, what are some other strategies you’ve seen faculty in your department or at your college start implementing to help reduce the use of sites like this? You mentioned one faculty of doing an intervention. [LAUGHTER] Have you heard of other things that faculty are doing perhaps as a result of you’ve been hearing about Course Villain, and CourseHero?

Zachary: Well, I know one large-scale institutional change is we’ve actually revised the Student Code of Conduct to include these kinds of sites and the behavior of sharing work. It is defined as plagiarism, as a violation of academic integrity, to even be involved. Still, that kind of leads to weird and murkyish paths, where “Can we track down someone based on a name on a document and an avatar on an open site?” I don’t know. That kind of these legal things that go beyond my simple English professor life and knowledge. But changing the Student Code of Conduct was a big one. And I think in STEM fields, physics, math, I know we talk a lot with a faculty about micro changes to assignments, where you don’t have to do a lot to revise a mathematics or a physics equation-based assignment: change decimals, change units, change little things here and there that can really fundamentally alter the structure of an assignment. And that makes it more resilient in terms of how it’s shared and used. And from an English Composition perspective, my best work against plagiarism has always been to encourage students to follow their own passions and ideas, to write about what they want to, to let students guide topics and selections as much as they can. And then to complement that with long-term scaffolding where you’re gonna pick something and we’re gonna work on that something all term. And so if you’re committed to plagiarizing in that context, then that’s a whole lot of dedication I’m not prepared to counter at the moment, but otherwise, it’s like, “Well, if I get to choose my adventure here, maybe I don’t have to go steal it.”

Kelly: Yeah, I was gonna piggyback off of Zack with “It’s the authentic assignments.” Having taught this last half year to a year I’ve been teaching a lot of the upper-level research methods class, where these are the students they’re rolling into their thesis, we call them capstones. So they start with the problem statement. Each problem statement is unique. Getting the data… the data has got to be the most recent. I did have one student that I think he must have had some material. And about halfway through the class, he realized that the authenticness was going to be removed fairly quickly, and he was going to be found out, because he changed pretty quick about halfway through the semester he said, “You know, I’ve got a better idea” and did all that. So it’s just building a better learning experience for the student. Unfortunately, it’s a better learning experience for the student that’s born out of this malicious behavio.r

Zachary: And otherwise, at Embry-Riddle, we also do operate with TurnItin software, there are investments being made in lockdown browsers, and two or more factor identification to try and get at some of the more complicated forms of contract cheating that are… I don’t think they’re pervasive… but they are very hard to deal with.

John: Have you requested any funding from your university in developing this package?

Kelly: Early on with the university, we filled out our university intellectual property, and you meet with the intellectual property office. And the whole idea is: “Is there some technology that you can eventually move into some type of a commercial project out of the academic world into the commercial world?” And I think maybe we weren’t just expressing ourselves. I almost feel like it was Shark Tank. We had this idea, but it wasn’t a business yet. It was a great idea. We had some kind of functionality, but it wasn’t a business yet. It just wasn’t developed enough yet for them to be able to take it out and market it around. Because now there is a team in our IT department {we’re on the committee) where they put out a request for proposal. And it was for software that helps combat plagiarism and things of this nature. So we haven’t seen the proposals that have come back in. But it seemed to us this was a tool that you would think that one of these educational learning management systems would be interested in taking and incorporating into… we all use Canvas… if that was integrated into Canvas, where not only just a turnitin report came up, CourseHero or one of these other crowdsourced platforms would come up as well. So the request for proposal’s out, we haven’t seen anything in there. But it would be interesting if we could re-engage the Intellectual Property Office at our university to see if there’s really, as Jerry Seinfeld would say, “Is this something?”

John: Along those lines, I know Lumen Learning, who produced the Waymaker personalized learning platform, have put together an API that automatically checks the web for postings of their questions and content. And they do use a lot of algorithmic questions so that each student gets a different version of the question. And I believe some publishers have started to do this. I haven’t checked that myself. But it would be nice if the arms race could escalate a little bit more on the side of deterring academic integrity violations. One of the things that’s happened, though, on the other side is many departments had picked up departmental subscriptions to places like CourseHero, because one thing that many of us are troubled by, myself included, is having to pay $15 or $16 a month for a subscription to a service primarily engaged in facilitating cheating. And many departments have created departmental accounts. And Chegg, for example, now, is only allowing you to use a given account on specific computers. So, you’re limited to the number of computers that you can connect to. I had my own account for this for a while now, because I teach a lot of classes where these things come up. But, I know some departments had created departmental accounts where many people were using the same account, cheating a little bit on the instructor side, to try to shut some of this down at a lower cost and without subsidizing them quite as actively. But more and more places are starting to move their stuff behind a paywall, which may make it a little bit harder to detect some of these violations.

Zachary: Well, John, I’ll follow that up. Actually, the College of Arts and Sciences here at Embry-Riddle Worldwide, we’ve also purchased a kind of college subscription for the same access. And Course Villain isn’t exempt from that kind of dependence either, because one of the really annoying issues we stumbled into early, was CourseHero’s own document preview limits, where you could only view 10 documents… their cover page, so to speak… before they would ice you out and give you kind of a grade. This is only for your eyes only kind of situation. And if you don’t have that paid access… that limit. So paying for it does kind of peel back that one layer and at least give us the option to view the artifacts and help correlate them. But that’s tough. But I also want to come back to the kind of arms race idea. If I had just my closing stump speech for Course Villain, it’s that it doesn’t have to be a very difficult arms race. It can be kind of an arms walk, where I’m really excited and pleased with our development of Course Villain and we’ve accomplished that with about $8,000 of internal funding. $8,000, two faculty members, and we’ve hired two research assistants… two student programmers… over the course of these couple years. And we’ve developed what we have. And I think our model is replicable. If we’re not going to create a product that the university wants to sell to other people. I really encourage other universities, other faculty, other interested researchers to start coming up with your own low-cost mousetraps, because that’s all it takes is kind of just chipping away at it, and before you know it, we will find that corner, that inroad, that kind of loophole that allows us to make really good work. It’s not out there. We don’t have to wait for publishers and juggernauts to do it. We can handle this. I really am confident in that.

Kelly: And we actually had this conversation after the last conference, should we have one more conversation with our university with the Intellectual Property Office of our university? Or do we just turn around and say, here’s the open source, go forth and provide it to other universities, and do our own crowdsource of how to combat it and see where it can go from that?

John: That could be an interesting project where other institutions may contribute work on other platforms, where people could get a menu where they could choose what to look for, and perhaps some people could specialize on generating those takedown requests. So that it wouldn’t require the work of just a few people there, it could be distributed more widely. That would be an interesting possibility.

Kelly: I bet there’s a grant in there somewhere, Zach.

Zachary: I hope so. [LAUGHTER]

John: Are there any other things you’d like to share with our listeners?

Zachary: If there’s one thing I would add that I don’t think I got a chance to really frame in the strongest sense that I wanted to at the conference, is the idea that what’s also at stake in this kind of scary moment, as we talked earlier about how people are finding themselves forced into this digital environment. And it’s easy to have your breath taken away at the scope and scale of this problem of people sharing coursework in kind of an illicit fashion. The other opportunity there is that we’re suddenly getting more data than we’d ever dreamed of. For my research from looking back at the histories of plagiarism of academic integrity violations. When you try and find the hard numbers of frequency or prevalence of this behavior, it doesn’t look like the Internet has really changed the nature of academic dishonesty, it just brought it into a new venue. And that makes it look scary, because now it’s like, “Oh, it’s happening fast and on really big scales.” But this kind of fundamental issue appears to remain largely the same. But what’s changed is our ability to understand and track that problem, because now it’s not between hands, under desks, or in dorm rooms or in the hallways where it could never, or very rarely, be tracked down. I think of like the kind of investigatory efforts invested in like the Naval Academy plagiarism scandals, where you have to track down students and interrogate them, and like bring them against charges, to get them to rat each other out, and then you expose this network of plagiarism when it was really just some people sharing something in a problematic way. But now that’s rendered in this digital space, and that’s a really cool opportunity to re approach a lot of our old ideas about how, why, these things happen, and put us in touch with a really cool new data set about this behavior that I think can help us understand how and why people commit academic dishonesty.

Kelly: I do know of one graduating student that I’ve advised along the way, and he’s going into the cyber security world. So here’s a huge world. He’s going through lie detector tests and background checks and things like that. And one of the things that he thought of was, they ask like, “Have you cheated in your coursework?” And he had to think, “Okay, with CourseHeroe, is that considered cheating?” It’s certainly unethical. Well, I guess that’s a whole ‘nother podcast in of itself. But we think it’s unethical, it’d be interesting to get someone on the other side, arguing that it wouldn’t be. I don’t know, if I was going for any kind of security clearance or anything like that, I don’t want to be defending that choice to the investigator that I may or may not work for at some point in time.

John: I think one takeaway, though, from your discussion is that we can design our courses in such a way so that it’s not so easy just to copy and paste solutions into questions. And we can come up with better ways of assessing learning, which many people have been working online have been doing for many years now. But with the sudden transition of so many faculty online, who are trying to, as in the early days of online instruction, replicate what they were doing in the classroom in an online environment, it just doesn’t work in the same way. And we need to work with things that work better in this medium.

Kelly: Yeah.

Rebecca: This has been a really interesting conversation, really providing some new perspectives on things that I think will help faculty think through these things a little bit more. We usually wrap up by asking: “what’s next?”

Zachary: I think what’s next from a research perspective is Kelly and I are on track to track brand new course material from its online birth, so to speak, and to track it long term, kind of one course or one assignment. What does that course look like? What is its kind of online life look like… almost like tracking a single butterfly, like a monarch, as it goes from point A to point B. I’d like to use Course Villain to track one really specific thing and try and flesh out that narrative. So, just like I mentioned earlier, to get at that “What’s happening here beyond the obvious?” And I think that’s a cool way this tool facilitates that.

Kelly: I would like to also go into strengthening the student portion of this, try and to get them engaged and come at it from that point. I’d like to do some work in there and thinking about working with my colleagues. Then also I’ll work with the workflow. How can we put it in our workflow? We’re are all trying to be so efficient, so you can’t come up with a unique thing every time a course is run. So I’d like to see where in the workflow that will go, and get more students involved. There’s one group of students that we really gave free rein to, because we weren’t paying them and it was during COVID. Also, we just said, “Here’s some source code, here’s the problem, what would you do with it?” …and we kind of threw it at them. And they went on to “Oh, we’re going to have it learn and go…” And one student was really big into AI, so he was going to work with them. And so that was kind of fun. I’d like to reengage them and see what they would do with it. They’re the students. They’re the ones with the better ideas.

Zachary: And I’ll throw a shout out there to Embry-Riddle Prescott campus… that’s their cybersecurity club. We basically gave them a clone of the tool and we’re like, “Hey, play with it. See what you can develop.” They’ve been really hip on the idea of machine learning. And how can we get Course Villain to figure out its own prerogatives and do its own tracking work without so much supervision?

Kelly: They each had, like a specialized talent. And one, I really think it would work more efficiently if we hosted it using this language instead of that language. And I said, “Sure, go ahead”. He was a little taken aback like “You’re not bought into that.” And I was like, “No, you guys are the creators. Go for it.

Rebecca: So, it’s pretty exciting to see how it might have a life of its own.

Kelly: Yeah.

John: Especially if it becomes an AI bot, somehow. [LAUGHTER]

Zachary: Yeah, absolutely.

Zachary: I love your idea, John, of a Linux model. I’d like to see some version of this program or some version of a program like this. Getting back to that kind of David and Goliath narrative, if we can all chip in some small piece to a project like this, we could develop something that’s really cool, that I think could give these kind of sites and their prerogative a kind of a tough run for their money.

John: You could always create a GitHub site or a SourceForge site, and then people could work on it from there.

Zachary: That’s a good idea. Thank you.

Kelly: A GoFundMe page. [LAUGHTER]

Rebecca: You might get a lot of takers on that.

Zachary: I just hope we don’t find the code itself on CourseHero. [LAUGHTER]

Kelly: Yes.

Zachary: That would be very disheartening.

Kelly: Imagine that takedown request.

Zachary: That’s right.

John: Well, thank you. This has been fascinating, and I think this is an area that many people are concerned about, and this was very informative. Thank you.

Kelly: Oh, thank you.

Zachary: Rebecca, John, thank you so much.

Rebecca: Thank you. I hope it sparked a lot of ideas.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

111. The Business of Academic Dishonesty

There are a number of websites that market themselves as study tools and tutoring services that are used by students as tools for cheating. In this episode, Dr. Liz Schmitt joins us to discuss how these sites work and the steps faculty can take to protect their intellectual property and the academic integrity of their courses. Liz is an economics professor and Acting Chair in the Department of Economics at SUNY Oswego.

Show Notes

Rebecca: There are a number of websites that market themselves as study tools and tutoring services that are used by students as tools for cheating. In this episode, we’ll discuss how these sites work and the steps faculty can take to protect their intellectual property and the academic integrity of their courses.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Liz Schmitt. Liz is an economics professor and Acting Department Chair in the Department of Economics at SUNY Oswego. Welcome, Liz.

Liz: Thank you.

Rebecca: Today’s teas are:

Liz: Cocoa and coconut.

Rebecca: Interesting.

Liz: It is. It’s my favorite. And I actually nagged John until he bought me more.

Rebecca: Excellent.

John: And I have ginger peach black tea.

Rebecca: I have Lady Grey today.

John: We’ve invited you here to talk a little bit about the growing problem that we’ve seen with the growth of online services that seemed to be designed primarily to facilitate academic dishonesty. I know you’ve had some issues with that in your courses recently. So, we thought this would be a good time to talk about that.

Liz: No, excellent. My post-traumatic stress hasn’t been maximized by the issue. So, let’s talk about it some more. But seriously, actually, it’s a really important issue. And I think our faculty are just not as well informed of this as they need to be. I think in the coming year, it’s going to become my mission to talk about this a lot more.

Rebecca: Can you talk a little bit about the incident that just recently happened to you to give some context and to help faculty understand how these kinds of things play out?

Liz: Certainly. So, really for years, even with the growth of the internet, if you’re using a publisher product… and it used to be sort of paper test banks and end-of-chapter questions… they show up on the internet. Some faculty just aren’t careful and were just posting them on non-password, open web pages. And then sites begin to grow where publisher content was widely published, by students often, with answers and answer keys to sort of pay it forward to future students who might be reusing it. I think what struck me this fall was the extent at which original content (because of this problem I had given up on publisher content)… but my original content in terms of my questions were being uploaded, essentially in real time. And within a few days, custom answers were being made by some of these sites. And this is really a new step in it, because it used to be the easiest way to get around this was to be writing your own content and guarding that content. And we find unfortunately, in September, that that was not enough.

John: The problem isn’t entirely new, though. For decades, maybe for centuries, fraternities and sororities and so forth, have kept files of old exams. But, the internet allows that to scale much more extensively and creates profit opportunities for people who facilitate those services. So could you tell us a little bit about some of the sites that do this?

Liz: Well, let me say this takes a step from the fraternity/sorority test file with this obvious broadening of the audience online, and no longer in geographic proximity. So, social affiliation and institution is no longer an indicator of what’s available to you. It’s searchable. So, it’s fraternity test file on steroids. It’s really one of the issues. But the other issue is these online proliferation of sites correspond to publishers emphasis on their digital content. As publishers really deal with essentially trying to get out of a used textbook market and kind of adapting for new ages of information, they really push their online content… which means faculty are relying more on that online content to operationalize many best practices such as retrieval practice for their students. So, it’s also more useful for students to have access to this. But, I think what really happened this fall is their ability to customize in short frames of time really eliminates the ability for me to just change up assignments in order to control the problem. So, really for low monthly fees, Chegg.com is what I ran into trouble with and for the low, low price of $15 a month they can upload questions. There’s a limit on how many questions, I think, they get a month but students potentially being strategic about it can upload questions and then get customized answers. Caveat emptor, some of the answers were great, some not so great and downright strange, which is really how I gleaned on to the problem to begin with. [LAUGHTER] And there are other sites. Chegg has some competitors I would say Chegg’s probably the market leader in this, but there are some competitors: Course Hero…

John: Course Hero has been out there for quite a while. My first exposure to this was about four or five years ago, when I had a student post in their online class that they were a member of Course Hero and they encouraged other people to join, mentioning that they could get their membership just by contributing a certain amount of graded work from the course to the site in return for that membership, and asking them to use her code so she could also get some credit for them joining. I had a little chat with that student about academic integrity and reminded her that if she posts anything on that site it would have to be taken down and she was going to be reported for this. This type of thing seems to be increasingly more common.

Liz: …much more common. And then you even have an issue like Quizlet. And one of the issues like Quizlet is students actually use it to create flashcards to study. So, I actually think there’s some honest intent for students that often come on to Quizlet. By making quizlets out of your questions and things you ask though, they actually create something for students that come after them. But, often Quizlet is really set up for students to actually create flashcards for different kinds of courses and study. But that’s really the leftover intent. And then, of course, there are online paper mills sites that actually offer existing papers… and again, narrowing topics can get around that and just not say “Write a paper on any broad topic…” that’s kind of really asking for trouble. But again, the customization that has come along makes that even more problematic as well.

Rebecca: And it’s somewhat affordable. It’s not incredibly expensive to be a member of these sites or to get a paper written for for you.

Liz: Yes, Chegg’s membership is $15 a month. So, that’s incredibly affordable. There have been sites… they come and go… kind of on a dark webish sketchy servers… but there are also sites offering to just take your online course. Those tend to be $1200 dollars, and up… Those sites get to be more pricier. But frankly, if you were a faculty member serving an institution with a more affluent student body, you should be very concerned about that.

John: Many of these sites also provide custom paper writing services. So, even if you’ve specified a narrow range of paper topics, students could still order a custom written paper.

Liz: Exactly. I would be fair to Chegg.com… that Chegg com responds within 24 hours in my experience to DMCA requests. So, they’re actually very responsive. And they take them down immediately. And they send warnings to the account that you posted things that you didn’t have permission to post. Chegg.com also cooperated with an honor code request, basically giving me upload dates and the emails of the members that uploaded content as well. So, they will cooperate, like I said, a pretty rapid fashion. I think it really, in the area of legality, it’s not clear if you correct yourself but become this conduit for things to happen, then, I don’t know what kind of legal responsibility is going to come there. It’s sort of beyond my expertise. But, I suspect it would take big players it takes like publisher lawyers and things like that, to really come up against this and demand that you really set up a situation that facilitates the repeated stealing of our content. Because it can be an intellectual property whack-a-mole game, because they take it down, but then another student post It from entirely different account and Chegg and these sites are kind of saying they’re not responsible for that. Legally, I’m not sure if you facilitate that, how responsible you are. There’s also paraphrasing tools. Paraphrasing tools… that’s very caveat emptor or buyer beware type of situation… and I actually have an example to show you what can happen when plagiarism tools go bad. Friends don’t let friends use plagiarism tools or this is going to happen. So, let me show you the answer from Chegg expert which was a very nice answer… is as follows: “Expectation of rise in inflation will lead to a drop in the prices of Treasury bonds. This is because the required yield by investors will increase in order to generate higher returns after beating the inflationary pressure.” Great answer, run it through a paraphraser and this is what you’re going to get: “Desire for ascend in the expansion will prompt drop in the cost of the Treasury bond. This is on the grounds that the required yield by the financial specialist will increment so as to produce higher returns subsequent to beating the inflationary wait.” [LAUGHTER] So, that’s not a thing… that is not a thing. I think the odd choice of vocabulary… Here’s words that college students almost never used. And here’s 10 of them right now, in the same paragraph. Obviously, that’s the warning sign. And the idea is, of course, context is important. Certain words that are used by convention or tradition in financial markets. Something that’s really a synonym, technically, in the English language is not in the context of financial markets.

John: A traditional way of catching students who were doing this often was that they were providing you with something that seemed a lot stronger than their other work.

Liz: Yes.

John: And this sort of reduces that down to where, perhaps, it may not be as obvious in all cases that someone is doing that because the quality of the work will no longer look exceptional in the same way.

Liz: Yeah, the exceptional quality of work, you’re right… because the first answer actually does set up my radar because it uses a complexity in sentence structure that I might not expect in this answer. But, I think the other one was just immediately obvious because the word choice just doesn’t fit. So, even if you were actually looking around on Investopedia, or some other website to borrow language, it’s not language that would be used by someone writing for that.

Rebecca: Even as a non-expert in the field, I could tell you that that is not language of the field.

Liz: “Desire for ascend in the expansion?” Yeah.

Rebecca: No, no.

Liz: I think pretty much called that one out.

John: But if you have some foreign students in your class, that sort of thing might sometimes happen. I’ve certainly seen examples where students have used synonyms inappropriately. So, some of this may get by.

Rebecca: In some cases, though, if it was a student where English wasn’t the first language, you would probably have some assignments and know already the kinds of mistakes that that student would make.

John: Yes.

Rebecca: So, you would have an idea of whether or not that would be consistent for that student or not.

John: We touched on the issue of hiring other people to do a student’s work, but could you talk a bit more about how the gig economy may play a role in academic integrity issues?

Liz: Well, the gig economy is basically the idea of you give me your specialized topic and I will write a paper custom for you: words, APA or MLA, the formatting etc., ready to go… Course completion, I will take your online course for you… paper writing services, again, is sort of that issue. And again, in ome of these paper writing services, it’s sent as a Word document, and they don’t actually strip the properties of who created that document.

John: And one of the things that has helped facilitate this, though, is the international reach of the internet. In countries such as India, where incomes and wages are a lot lower, you can pay people relatively small amounts to get reasonably high quality papers written in very strong, solid English

Liz: Right, exactly.

Rebecca: So, what can faculty do? This sounds very dismal.

Liz: It does sound dismal and there’s no solution that doesn’t involve some time on task here. And I think that’s a big problem. Because most of my faculty colleagues, I don’t really know anyone who sit around doing nothing… who has extra time to deal with this, and that’s got to give. Scaffolding is a common way to deal with this. So, it’s a recipe for disaster to assign a paper, be very unspecific about the topic: “Do any topic in post-World War II US history out…” That’s going to be bad. You want to focus the topic, then you actually scaffold. When do you want a thesis statement? When do you want an outline? an annotated bibliography? So, trying to have the pieces turned in really prevents their ability from coming out to get a paper or it’s really the red flag when a student several weeks into the process suddenly wants to change topic, having had struggled earlier with the topics and then they tell you when they want to change, you need to get ready for what that paper is going to look like. And you can scaffold in other ways, even on homeworks: you reference. I reference “Using the model from chapter seven,” “using the model we talked about in class in week six” or “using the discussion issues that were brought up in discussion three.” And so you can’t easily post those and get answers because then the students would have to provide the context, which defeats the purpose as well, of the questions. Algorithmic questions, and algorithmic questions can be posted and solved, but algorithmic questions are really after the fact, it makes it a lot easier to snag people. Because, if you just have a general problem…. So, I had a problem on the internal rate of return, which is a common time value of money concept in the field of finance. And while I think students really got their answers from Chegg, unless they were kind of lazy enough to exactly copy the wording of the answers, I couldn’t really get them because there really was a way that you solved this in the spreadsheet. So, the solution should look the same. So, you get around that by algorithmic questions where the numbers are unique, and then you know, exactly who uploaded that question. And in a learning management system, it’s easy enough to regenerate numbers every semester or year that you’re teaching the course. So, there’s a new set of unique numbers. So, that’s more about enforcement, I think. Questions that work from very specific data, maybe a data set, or quotations. So you can start with: “Mitt Romney, in a debate in 2012, called China a currency manipulator. What does that mean?” And so that quotation again, makes it harder to find general answers to that question. Questions that reference their own experiences, where they have to call up a specific experience themselves, and expound on it, and apply it in class. And current events. So, if you do actually have a very current event, then you can actually prevent going back in time and trying to find older questions, because it’s a current event as well. And so you’re constantly changing. So, you might be testing on the same topic, but you’re constantly changing the context. And your question has to be written so that you forced them to address the context as well.

John: I’ll often ask students to find an example of something in the last six months that illustrates some concept that we’ve just discussed in class. Because if I only offer the class once a year, they’re not going to be able to go back and find earlier examples from the class.

Liz: Right. And then I guess my least compromised questions are ones where they actually create a graph using the Federal Reserve Economic Database based on macro data or financial data. And again, that’s right, because it has to be current. And from year to year you change the time period, or what you want them to look at, or different measures of inflation, things like that. And then you can really grab that, and I have yet to see a FRED question appear on Chegg.

John:(…and FRED is an acronym for Federal Reserve Economic Data.).

Rebecca: What is your process for monitoring because clearly, you’re doing some monitoring.

Liz: Right. Well, when I found the extent of this problem in September, I basically went forward several weeks, and just copied and pasted questions into the internet search engines. I also ended up at least getting a subscription to Chegg so I could look, because the other issue is if someone screenshots an image of a question from their learning management system that might not show up in a Google search, but it’s going to show up on Chegg because Chegg actually has the text… kind of an alt text… that comes up, and that will show up as well. And so I started doing that. I opened a Google doc. And then every time I found one, I knew I had to change the questions in my graded activities. And then I would put it in the Google document. And I ended up with a 20-page DMCA document to Chegg about all of the questions I wanted to take down that take me through about week 10 my course. So… to be continued… as to how many I will find. But really, the monitoring really comes up at first when you get just a very unusual answer, or you start getting all the same answers. So, in a class of about 37, there’s about 10 students that were trying to tell me the same thing, especially if they illustrate a concept… I don’t ask for a numerical example… But they give me one. When you see the first person do that, you’re like “That’s great.” And then you think, “Well, why would the second, third and fourth person choose the exact same numbers to illustrate this concept?” There’s only one reason and it’s not a good one.

John: You mentioned the paraphrasing tools making it a little harder to find when people were trying to copy materials. There’s other tools out there too, though, which are online plagiarism detection sites where students can get a paper that they think fits the requirements of the assignment, upload it to that, see if it’s found, and if it is, just make minor changes and resubmit it until it ends up with a relatively low plagiarism score. And complicating this a bit, is that some of the major paper mills advertise that they use a TurnItIn service to check the papers that they sell for plagiarized content. Because one of the issues for the companies that are selling papers to students is that sometimes the people they hire to write papers simply plagiarize existing material. So, this makes it a little bit harder perhaps to detect that sort of work for hire. What might you do to detect papers that have already been run through the TurnItIn or similar systems and modified to come up with low plagiarism scores?

Liz: Well, I think you have to go backwards from that and recognize the limitations of those tools. I was never a big fan of TurnItIn. SafeAssign is just kind of a starting point. SafeAssign often gives a lot of false positives as well. Because if people are citing sometimes similar sources, which you would expect, then you’re going to see that pop up in SafeAssign. So, I think you got to go back and say, are you choosing the topic that is sort of paper mill proof, or things like that? Downloading when you have electronic submissions, which I do, and downloading them can also pick up issues of extra characters designed to trip it up. So, people put characters and they change the color to white to try and trip up that detector and that easily shows up when you highlight them. Those are some of the things that I would use. wWe haven’t had TurnItIn on our campus. We had SafeAssign, which I already knew the limitations, but the tell is you read it, and you’re like I’ve met five college students that write like that… in 23 years. So, I’m now deeply suspicious of that. Often the language doesn’t match, which is why scaffolding also becomes important… because the language doesn’t match what’s been done earlier in terms of: if you don’t think the student wrote the paper and you can’t find the source, you invite them in. And you ask them about where they found the source. Or I really love what you said about the impact of inflation on average households in the 1970s. Could you explain how you developed that? …and they won’t be able to and as a result, they’ll often say, “Well, I wrote this paper and I immediately forgot everything.” And faculty members get caught up into this. “Well, I have to prove a negative or I have to prove that that’s not possible.” And you don’t, I think you’d say: “On your face, that’s a really ridiculous argument. Either you need an MRI right now, because something wrong has happened. You’ve had a stroke and you need help, or you’re not being honest about where that paper came from. And you’re allowed a chance to defend yourself from these charges. But that defense isn’t reasonable. And I’m going to move it on.”

John: Pretty much every time I’ve run across that and brought the students in, they almost always confessed pretty quickly when they realize that they can’t explain what they had written.

Liz: Yeah, I think turn it in has these separate issues too. Is it ethical? We’re complaining about our intellectual property being posted. But, students that do write original papers, that’s their intellectual property. And we’re forcing them by virtue of getting their degree and meeting requirements of a course, to release some of that somewhat. So, I think there are issues of privacy and property, ethical issues that I know a lot of other faculty out there have just really pushed back against for years with turn it in. Others argue it’s about us policing students by requiring this prevention method, we’re almost taking the assumption that something’s going to go wrong, that it can create a hostility and an adversarial relationship that we don’t want in students. I think there’s some truth to that as well, which is why it becomes so important to think about the design of your course and your learning activities, because it’s so much better to prevent this and to make it very difficult than to deal with it afterwards. As I tell my students in my annual first day of class, don’t test me on this because I fail people at least once a year for this kind of thing. I actually say this is the worst part of my job, and I hate it, but I do it. So, that should tell you how important it is to me.

Rebecca: I think that it’s sometimes it’s a little more obvious to faculty how to prevent some of these things from happening in an face-to-face class, because you can be doing in class assignments, you can have them working on things in class and see their progress. But that doesn’t play out maybe as easily online because you don’t necessarily know what they’re doing. So, do you have some strategies of how you scaffold or do things a little bit differently online?

Liz: I would back up again, because some people say, “Well, this is why I don’t teach online, because I’m worried about ringers and things like that.” But, often people that tell me that do assign papers, and I say “Basically, anything that’s done outside of class is susceptible to these sites…anything.” And so really, again, there’s a lot of face-to-face classes, particularly in STEM that are using homework packages that students complete on their own. And then there are plenty of classes that assign papers that are done outside of class. The idea is, is that the in-class possibility of a face-t- face class does provide this check. But in an online class, you really compel them to actually talk about things in a discussion. And so you can use your online discussions to lay the groundwork of what they want to talk about, requiring them to reference discussions within the course as well. And so that’s one way to try and mimic that check that in-class does. And there’s proctoring software, which again, is potentially foolable as well, but there’s also design. So in an online class, the idea is no one thing should be worth a lot. Any one thing should be worth less than 20% in my opinion, because the idea here is if you want a ringer, you don’t want that ringer, by just taking a single exam, to move your letter grade significantly. You’d need a ringer for the whole course or a really, really good friend to do this for you. So, that’s another thing and in designing them exams, you can time them, not necessarily just time them but there’s one question at a time, no backtracking options. So, you want to think about a very structured way to require students to demonstrate understanding that just make it a lot more difficult to outsource.

22:08 redo

John: You can do the same type of scaffolding online as you do in a face-to-face class. You just have different stages, as you said. In the first stage, you could have them submit a thesis statement. In the next stage, you could ask the students to submit a bibliography, followed by an annotated bibliography, and then a rough draft and a final draft of the project . And that’s not really much different than it would be in a face-to-face class.

Liz: And all in the same place are all of these writing samples because that’s how you’re communicating with students: via emails, course messaging, discussion forums, and then other graded work. So, in some sense, there’s a large body of written work to form a basis for your suspicions or concerns.

Rebecca: I’ll also add that we focused on lot on written work, test questions, and things like this. But, the same kind of plagiarism can happen with images, it can happen with code. I’ve had those same experiences in my classes as well…. sharing of digital files to make a particular design and boy does that look kind of similar to something I’ve seen before or an image that’s being claimed as their own or not documented where it came from. So those same things happen. They don’t play out in the same way in Chegg and some of these other sites, but those same practices happen through like a gig economy or just sharing amongst other students and when their digital files are a lot easier to share them when they were physical things.

Liz: Also in creative fields, there’s something worse in the way that’s sort of accepted, because if you look at fashion design, the ripping off of top designer early designs to then the knockoffs is astonishing. There’s a photo that has a Manolo Blahnik sandal, frankly next to an Ivanka Trump sandal, and they’re the identical red sandal. And it’s really just you slapped a different name on it and use different materials, perhaps. And obviously, they’re about six months apart in terms of product cycle. So, I think people in creative fields see that recycling, even in the music field, whether they get permission, you see music, that’s a remix of other older musicians, and you need permission for that, but you don’t necessarily know that when you’re listening to it and enjoying it. So, I think you actually see in some of the real-world situations how visual borrowing and frankly, stealing, is kind of accepted in some of these fields by some very successful people. Whereas, in the written world, when authors get found with plagiarism, it’s considered a big deal. And it’s kind of very embarrassing for them to get caught. Books can be recalled… things like that. It seems like written word… there seems to be more of a consequence, for now, when that happens, whereas in creative fields, that’s not always true.

Rebecca: There’s a lot of visual and audio copyright cases, though. And that’s where that tends to play out. The fashion example that you’re giving… those are considered functional things and not considered creative. So therefore, they’re not protected by the law in the same way.

Liz: Exactly. But, I think for what students observe, and behavior that they might emulate, when they look at professionals and their choices, that distinction isn’t going to come up.

Rebecca: One thing that I always argue is that when we talk about intellectual integrity, that copyright comes into the discussion in particular fields, because it is sometimes a common practice. And it’s there for a reason. we iterate on our culture, our culture creates new culture. It’s not a crazy concept, but you need to know where it comes from. And you need to provide attribution and in an appropriate way, which is no different than following MLA or APA or some other thing. So, I think that’s always something that people should be thinking of, and that when you’re having written papers, and there’s images and things in it, that you’re also thinking about that part of the content as well and not just the written word.

John: One other thing that I’d like to bring up is an earlier podcast that we did with Judith Boettcher, who talked about one way of avoiding this issue by having group projects that provide students with a lot more autonomy, but in a very structured fashion. And that’s perhaps a way of getting around this where students can take more ownership of the project and they create more of the project as a group, which would make it a little harder to engage in academic dishonesty. And we’ve also, in previous podcasts, talked about some open pedagogy projects, where the work that students do is posted publicly. And if they know that when their work is posted publicly, and they’re copying something from anyone else, it’s much more likely that they would be found and they’d get in some trouble later. So, those are two other things, perhaps, that might be ways of reducing the incentives for academic dishonesty.

Rebecca: Community based learning is another one… or service learning where you’re doing projects with the local community, because all of that context would be unique every time you’re doing something. So, that’s another opportunity for grading those assignments that really aren’t reproducible, and would be really, really hard to get an answer for… unless the person lived in the same community.

Liz: I agree. I think the biggest challenge is in some of these courses that are tool courses. In some courses you’re acquiring the tools that you would learn for projects and to consult, etc. And so when you teach these tool courses, it’s not always appropriate to have these kind of finished product things, because they’re in progress of assembling that toolbox that they’re going to use. And this is where reliance on sites like Chegg become a big problem.

Rebecca: At the beginning, we started with the idea that faculty aren’t always aware of these tools, or even the ways that students try to manipulate the system. Can you talk about ways that we can help increase this awareness with our colleagues?

Liz: I was just talking about this this morning with another colleague and we were bemoaning our Chegg purgatory this semester, and she says, “I just don’t think other faculty realizes… or how can we be the only ones that care about this.” So, I think some of them are honestly unaware because faculty aren’t always in sort of the student space and understand what those crazy kids are doing these days. So, I think, in some sense, faculty that have been more tuned in to creating learning community or kind of developing a relationship with their students are more likely to get ideas of what’s going on as well. But I also think there’s some willful ignorance here… this whole “Well, I didn’t know.” But I took steps to make sure I didn’t know because once you do know, it obligates you to do something, if you “don’t know,” and I’m using quotes, which is not helpful on a podcast, air quotes, but if you really don’t know that you’re MyLab component of the course is completely compromised, that doesn’t obligate you to think about changing up your course, and the weighting of activities and what activities. Once you know, it really obligates you to act as a faculty member. And so some people say, “Well, I wonder if my stuff is up there?” and like, “Oh, it is.” I don’t even have to look, I can say “Yes, it is.” And then academic dishonesty takes time. I’m really at 40 hours and counting with academic dishonesty documentation, DMCA documentation, and then reworking items in the course to deal with this issue so it wasn’t a lost semester. And I’m a full professor, think about an assistant professor not only trying to balance and develop their research agenda in conjunction with this, but also not wanting to rock the boat with unhappy students. I’m going to be getting a nice bottle of something sparkly, when I read my faculty evaluations in January. They’re going to be lit. Perhaps we’ll do a dramatic reading and have a tea party with tea and maybe something stronger. But, you know what? I can weather that. I can take that. And that’s going to be an issue. And frankly, I’m really proud of this institution, about how administrators really back faculty, enforcing the integrity of coursework and the degrees. And I know that doesn’t happen at other institutions, frankly other institutions that are maybe more tuition dependent and driven… that are unwilling to make steps that make students leave with financial implications. So, in some sense, this is really one of the best environments here at Oswego to actually try to enforce these policies as well. So, I think that’s one of the reasons that faculty don’t act, because there absolutely is some blowback to that.

John: And faculty might also see that each year they teach the same course, their students are doing better and they might be very content just to see that improvement in the scores so that all the work being submitted looks more like the best work from the year before. And it’s really easy to passively accept that.

Liz: Well, I would actually note that what’s more astonishing, is not the extent that Chegg has corrupted these issues, but there is a significant contingent of students that do not use it. Maybe they are unaware of it. Maybe they decide they don’t want to use it. But, when someone says, “Well, I can’t believe how many students cheat,” I would say, “Well, I can’t believe how many don’t, given the incentive structure.” And so that’s somewhat encouraging. But, also I had to decide if I was going to email the entire class and say, “Look, I’ve seen this happen. And you have to know that this is not allowed, that I’ve already made it clear in the syllabus. This is not allowed, and this is what will happen.” And it was a real-life decision about the Streisand effect, because you have to wonder if 20% of the class is saying “Wait, there’s a Chegg and we can get answers there? Alright!” You didn’t know if you wanted to clue them in, but I figured I had to act under full information.

Rebecca: I think it’s also really interesting when you start seeing students wanting to apply for jobs at those kinds of organizations, and then what kinds of conversations you’re going to have with those students, when they say, “Hey, I found this really awesome job that I want to apply for at Chegg.” It’s like, “Well, think about that. What does that mean? And what are those implications? And where are the ethics behind that?”

Liz: Well John, and I were talking earlier, there are Faculty Fellows at Course Hero, and some of them have made a name about teaching in the profession. John was probably more polite about that. One of them’s at a conference and I think I would stand up and say, “How do you reconcile partnering with a site that facilitates academic dishonesty and intellectual property theft every single day?” I would be curious as to what the answer is.

Rebecca: The pay is really, really good? [LAUGHTER] I don’t know.

Liz: I guess. The one I’m thinking of has a pretty sweet gig right now, so I’m not really sure.

John: this presentation was right after one that was riddled with references to learning styles, so I saved my powder for that one.

Rebecca: So we always wrap up by asking what’s next?

Liz: First, I’m already kind of brought this up in faculty assembly. And I’ve brought it up in academic meetings, because I’m acting chair this year. So I brought it up in leadership meetings. And I hope we can actually do a workshop through our Center for Excellence in Learning and Teaching. So, maybe a breakout workshop talking about our experience, and try and really broaden the understanding of the issues here. I’m also working with our faculty union, the United University Professors is also very interested in compiling violations of intellectual property rights, and trying to deal with that and push back against those sites. And I’m actually sharing my DMCA documents. So I made an editable form for some of the big sites, so you can easily go in and change them. I made a template for what you would ask the Associate Dean to fill out in order to ask for upload information from these sites as well. So trying to minimize the work involved for people who want to do this and take action. And then, finally, I’m just looking ahead about how I’m going to really redesign this course that I’ll teach a year from now, and to motivate and enforce original work.

John: And I should note, we’ve also been offering workshops for at least seven or eight years now.

Liz: You’ve been offering workshops on these sites and things like that….

John: …and attendance has been generally limited. We’re lucky if we get 15 people.

Liz: But you don’t have me. I’m a draw. I’m a draw… a star. [LAUGHTER]

Rebecca: …especially if we could do the dramatic reading of the examples.

Liz: Exactly. There will be a dramatic reading, and there will be sufficient supplies of snark. And I think it’ll work. But, I actually think maybe a case study. So, we come and we talk about these sites, but I can sit down with other faculty who’ve had this problem. And this is what I found… this is how I found out about it. And this is what I did about it, and this is what you should be doing about it.

Rebecca: This has been super informative. Thank you very much, Liz.

John: Thank you, Liz.

Liz: Well, thank you for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Brittany Jones and Kiara Montero.