Category: academic integrity

388. The Opposite of Cheating

Posted in academic integrity, assessment, cheating, engagement, higher education, instructional communication, learning, motivation, teaching
April 9, 2025

Student use of generative AI tools as a substitute for learning has led to increased concerns about academic dishonesty. In this episode, Tricia Bertram Gallant and David A. Rettinger join us to discuss why students might use these tools and strategies instructors can use to encourage academic integrity.

Tricia is the Director of the Academic Integrity Office at UC San Diego and Board Emeritus for the International Center for Academic Integrity. David is an Applied Professor and Undergraduate Program Director in the Psychology Department at the University of Tulsa. He is a Professor Emeritus at the University of Mary Washington, where he directed Academic Integrity Programs and the Center for Honor, Leadership, and Service. David is also President Emeritus of the International Center for Academic Integrity. Tricia and David are the authors or co-authors of numerous articles, books, and book chapters on academic integrity. Their most recent book, The Opposite of Cheating: Teaching for Integrity in the Age of AI, was recently released as the 4th volume in the Teaching, Engaging, and Thriving in Higher Ed series at the University of Oklahoma Press, edited by James Lang and Michelle Miller.

Show Notes

Transcript

John: Student use of generative AI tools as a substitute for learning has led to increased concerns about academic dishonesty. In this episode, we discuss why students might use these tools and strategies instructors can use to encourage academic integrity.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Tricia Bertram Gallant and David A. Rettinger. Tricia is the Director of the Academic Integrity Office at UC San Diego and Board Emeritus for the International Center for Academic Integrity. David is an Applied Professor and Undergraduate Program Director in the Psychology Department at the University of Tulsa. He is a Professor Emeritus at the University of Mary Washington, where he directed Academic Integrity Programs and the Center for Honor, Leadership, and Service. David is also President Emeritus of the International Center for Academic Integrity. Tricia and David are the authors or co-authors of numerous articles, books, and book chapters on academic integrity. Their most recent book, The Opposite of Cheating: Teaching for Integrity in the Age of AI, was recently released as the 4th volume in the Teaching, Engaging, and Thriving in Higher Ed series at the University of Oklahoma Press, edited by Jim Lang and Michelle Miller. Welcome, Tricia and David.

Tricia: Thank you.

David: Thank you.

John: Our teas today are;… Tricia, are your drinking tea?

Tricia: I am, my standard tea of peppermint after lunch.

John: Very good. It’s one of my favorites. And David?

David: I’m a coffee drinker, I have to admit. But it’s a really good Sumatran.

John: Excellent.

Rebecca: Good choice.

John: And Rebecca?

Rebecca: I have Lady Grey today, John.

John: And I have an Irish Breakfast tea today.

Rebecca: Nice. So, we invited you here today to discuss the Opposite of Cheating. Could you tell us a little bit about the origin story of the book?

David: Sure. Well, Trish and I have been doing this work for a long time, both separately and together, and so as AI has really started to become a big issue, but even before that, when the pandemic hit, a lot of our colleagues have been asking us, what should we do? Oh, my goodness, our students are cheating, and Trish and I both sort of smiled separately and said, “Yeah, they’ve kind of already been cheating. You’re just kind of deciding to worry about it now.” This has been an ongoing and persistent problem for probably as far back as Plato’s Academy, if not further, because people are people and it’s a fundamental problem. So when Jim and Michelle came to us and said, “Would you consider writing a book for faculty that’s practical, that’s positive, and that really takes a keen eye on academic integrity?” we said “We’d be happy to.” So we wrote this book. We were kind of proud of it, and it was ready to go to the publisher pretty much exactly the same week the ChatGPT came out.

John: Nicely timed.

Tricia: Yeah.

David: Yeah, couldn’t have done it better if we tried. So the result of which being, we sent it to the publishers and to the editors, and instead of a two-week turnaround, there was about a six-week turnaround. And Jim broke the news to us and said, “You know, you’re probably going to have to rewrite the book,” which is, I’m sure, his favorite sentence to tell an author. But we did. We wrote the book, and as we did, AI grew faster and better and more accessible, and we learned more and more about it, and now I think we’ve got a book that can really be useful in an evergreen sort of way, because the principles we’re applying are not based on the tools that you see today. They’re based on the people who are using those tools, and the people really haven’t changed at all. And so we really hope this book will be evergreen.

John: And as you noted, students have always cheated to some extent, but there was a bit of an increase in that during the pandemic based on self-reported results from students in terms of academic integrity issues. Why did it increase so much during the pandemic?

Tricia: A simple reason. As David said, students are humans, and so if you think about a couple of things that were going on during the pandemic, if you were a student. One is, like everybody else, you were operating at Maslow’s bottom hierarchy of needs: survival, safety, security, that’s all that mattered at that time. Like integrity would be way, way, way at the top of that pyramid, so that was the first thing that was going on. The second thing was increase in temptations and opportunities. All the assessments went remote, and where before, there were proctors walking around the room, maybe checking your ID, maybe checking the authorized age you brought in, seeing if you brought something else in, all of a sudden, there was none of that. If you take our traditional age college student of 17- to 21-year old, putting them in that kind of situation, or putting any of the younger kids in that kind of situation, of course they’re going to cheat. It’s a lot to ask a student to resist those temptations, especially under times of great stress and pressure. So two things created a perfect storm.

Rebecca: Can you talk a little bit about, you’ve hinted at this already, but why students cheat in the first place, or maybe why they act without the integrity that we’re hoping they would act with?

David: Well, it’s a how long have you got [LAUGHTER] sort of an answer? You can break it down into two parts, though, and Tricia, pretty much already did. It’s the situation that they find themselves in, and the individual differences are the nature of who we are as people, and those are going to interact to create the situation where students cheat. Well, I think to take a step backward. The first thing to say is that almost all students, just like almost all people, if you put them in the right circumstances, will make a decision that they’re not proud of later. So it’s not as if we can just weed out the few bad apples and solve the cheating problem. Having said that, there are a few bad apples, probably about eight or 10% who use what we would call cheating as a dominant strategy throughout their higher education and probably throughout their life. Those are students, yeah, we really do want to try to weed those out. But for the other 80%-ish, some percent, I’m going to guess 20%, would never do something dishonest knowingly. Some of them still will cheat because bringing us to reason one, which is they don’t really understand what we expect of them, and that’s a situation that’s growing more and more as higher education becomes more accessible to people who don’t have the same background, values, cultures and understanding. This is a good thing, that higher education is more plural and more diverse, but it does change what our strategies need to be in order to help everybody be successful. And then you have the other situational reasons, things like it’s a lot easier to cheat in a high-tech environment in a lot of cases, so students are more able to do it. They feel like they’re less likely to be caught in those situations. And then you get into the interpersonal ones that I think are the most interesting, but I’m a psychologist, so of course I do, these are things like number one on the hit parade is peer pressure. Students are very sensitive to peer pressure and to peer motivators. If students think that cheating is acceptable, they’ll be more likely to do it. If they think it’s unacceptable to their peers, they’ll be less likely to do it. If they think their peers are using it as a strategy, students don’t want to be left behind, and so they’ll be more likely to try to cheat to keep up with the Joneses. So peer pressure, peer motivation is a big deal. Then you can add in things like, how wrong do they think it is? And different kinds of behavior are quite varied. This is where AI comes into play, because we’re all in a new frontier, and none of us really understand what AI is doing. I shouldn’t say none of us, but most students don’t really understand how AI works, and so they don’t really understand whether using it is inappropriate, and almost none of our students understand why we’re asking them to do the assignments. If they knew at a deep and intuitive level why they are doing what they’re doing in the way that we do when we create the assignments, then they could make a better evaluation about whether the tools they’re using are appropriate or not, but because they don’t know, we haven’t told them, and not all students are coming from a context where it’s automatically expected that they know. We’re left in a situation where students sometimes don’t think what they’re doing is actually wrong, although as often as not they do and they’re doing it because of things like time pressure, stress, desire to be successful. And there is more, but that’s a lot.

Tricia: In other words, I’ll sum up what David just said. They cheat ‘cause they’re human. [LAUGHTER]

John: And there’s multiple reasons. And one of the things you stated in the book is that there isn’t one solution to addressing these academic integrity issues, but there’s a right approach. Could you provide just an overview of an approach to try to address this collection of challenges that may lead to academic integrity issues,

Tricia: I think we would call it a teaching and learning approach, rather than a crime and punishment approach. So a teaching and learning approach means that we’re centering the faculty and the students in the conversation, and we are centering learning and what we can do to best facilitate, which is one job of the educator, and assess learning, which is the second job of the educator. And we often forget that second part. We like to focus on helping students learn, and we forget that assessing learning is a huge reason why students come to us. They need that degree. That degree is supposed to be a culmination of the assessments of learning that represents what they now know and can do. And so the approach is: focus on teaching, focus on learning, and part of that is focusing on good assessment design. So that’s the right approach. It does a couple of things. One, it’s less emotionally draining for faculty when they feel like they have to be police officers to catch cheating and to report it. That doesn’t sound like fun, and it doesn’t sound like what they signed themselves up for. But if you have them focus on learning, that’s like, “Oh, well, yeah, that’s what I signed up for. “So it feels better for the faculty, and it’s also, of course, better for the students, because they do want to learn. Yes, there are some, as our colleagues in Australia call them, enrolled persons in our school that do not want to learn. Put them aside. Like David said for a second, the rest of the students do want to learn. We’re humans. We love learning. We loved learning as kids before we got into school. And it could be that school maybe dampened that [LAUGHTER] love of learning a little bit, but people love to learn, and so having that focus says we’re in this together. And you know what, I am going to closely observe you to make sure that you actually learned what I think you learned or what you were supposed to learn. But it’s not to catch and punish you, but it’s to validate your learning and to ensure that you are making progress that you want to make.

Rebecca: You hinted at this already a bit about a misalignment between faculty understanding of what acting with integrity might be versus what a student might think. There’s a discrepancy there. Can you talk a little bit about why there might be this discrepancy, and then also how we might address it?

David: So we’ve already talked a little bit about the discrepancy in knowledge between faculty and students, and that we are not teaching mini me’s. We are an unusual subset of university students who go on to be professors, and we sometimes forget that they are not us. It’s also the case that the student population has changed in the time that a lot of us have been doing this. So we shouldn’t teach the students we were nor the students we had. We need to teach the students who are in front of us, and sometimes that means matching our expectations to theirs. It doesn’t mean changing our expectations, but communicating what ours are in a way that they can understand. So some of it is knowledge, but some of it is also goals, as Tricia alluded to. They are there, and I’ve asked them, to get a degree, to get a diploma, to get a good job, to make money, to have a place in society, and learning is a path to do that. And so we need to leverage their desire to do these things in an authentic and meaningful way….and yes, they do have that desire, most of them… by communicating the value to them of the learning that we’re asking them to do and the assessments that we’re asking them to do. None of this involves watering them down or dumbing them down or simplifying nor justifying your existence to your students. It involves helping them bridge the gap between their experience and yours, which is, of course, learning, such that they’re doing these things in a way that will prepare them best to be successful in the future, whether as professionals or as humans or as family members or as voters, whatever. And if we can persuade them that what we’re doing is worth doing, we’re going to at least give them the opportunity to act in a way that’s in accordance with our expectations.

Rebecca: Sounds very aligned with the TILT or the transparency and learning and teaching approach.

David: We’re not inventing anything new with respect to pedagogy or teaching. And in fact, I think, going back to the origin story question, from my perspective, our first concern when we wrote this book is, is there any audience for it? Because we’re not writing anything new. But then it kind of occurred to me, and I think to Tricia as well, we’re writing about teaching and learning for academic integrity folks, and we’re writing about academic integrity for teaching and learning folks. And so we hope to introduce some of the core concepts in these fields that we bridge to folks who maybe haven’t had the opportunity to bridge it in the same way yet.

John: It sounds as if one of the strategies you’re recommending is to help students see that education is valuable because of the skills they’re developing, and that it’s more than just getting that certificate at the end that matters, that if they don’t acquire skills, they’re not going to really be all that more valuable after they graduate, and just helping to align their values with the values that we hold concerning education. Would that be part of the message we’re trying to get across to students?

Tricia: Yeah, I think so, as I’ve been telling students of late, look, no employer is going to want to hire you if the extent of your skills is you pushed a button on ChatGPT and it gave you output and you handed it in, they don’t need you for that. These things are becoming agentic, and they can operate without any human intervention. So you have to bring more value to the employer than that, and so that helps to explain to them, one the importance of developing skills, but also of why we don’t allow them to use AI for every single assignment or every single activity. We still teach students basic arithmetic, even though there’s calculators, because there’s a belief or knowledge that learning that is helpful in order to better understand more complicated math you’re doing later, or just [LAUGHTER] to understand math. And so we have to figure out the same thing for students. And then second, it’s related to one of the causes of cheating that we haven’t mentioned yet, which is the extrinsic motivation, right? If I’m only motivated by the grades and the degree, I’m more likely to cheat, and if I don’t see the point of the assignment in helping me get there. And so raising students’ intrinsic motivations means we’ve got to really seriously rethink what we do in college and university. Are these learning objectives still relevant in the age of AI, and if they’re not, what needs to be updated? Are my assessments still relevant? So many courses require students to demonstrate their knowledge in writing. Why? Is writing really something that every single class needs to use as an objective measure of a student’s knowledge and abilities? Probably not. And so we really need to rethink some of those things to make everything more authentic and meaningful to students’ lives today, which will increase their intrinsic motivation and help reduce the likelihood that they might cheat.

David: We want to make it clear that learning objectives are where things should begin. It’s that classic backwards-design approach, and so Tricia says we don’t want anyone to think that Tricia is saying that writing is not important. The question is not writing is not important, the question is, is writing important to what you do? And that’s what for instructors to determine. Sometimes it’s critically important. Sometimes it’s just really useful. But at any rate, if you design your courses from the objectives back, then you can be really thoughtful about what you’re asking your students to do in a way that can incorporate AI when it’s appropriate, ban it when it’s appropriate, and be agnostic when it’s appropriate. But to focus on the learning… It doesn’t matter what you’re doing, focusing on those learning goals and updating what you do is maybe the other key theme. It’s not all about student motivation. You can’t design cheating out of a class, as our friend Kath Ellis likes to say, you can only design it in, so that’s one, and very important component, is motivation. Another is assessment, as Tricia said, and communication.

Rebecca: This emphasis on learning is certainly aligned with one of the strategies that you suggest around growth mindset. Can you talk a little bit about how to help students develop that growth mindset in this kind of a context?

Tricia: Well, we mentioned that in the book because it’s one of the pieces, as David said, we’re not introducing anything new. And so our book really isn’t about so much of how do you create growth mindset is thinking about, if we have a growth mindset about our students, that will change the ways in which we tackle the problem of cheating, and if we have a growth mindset not just of students in my content that I’m trying to teach, but in their ethical decision making. So a lot of faculty may start off thinking, well, these students are already adults. They’re 18 years old. They should know better. My job is not to educate people on ethics and morals, and so they think of them as having fixed mindsets. If they cheat, that must mean they’re a cheater, and therefore they need to be punished. And our point about the growth mindset is, in both of those areas, consider that your student still has a lot of room to grow in making ethical decisions, especially when under stress and pressure, especially when given temptations and opportunities, and consider that they still have room to grow in skills and knowledge that is related to your classroom as well. So it’s not so much that we present strategies to help them do that as here’s another way of framing your approach to the cheating problem that you might find helpful and enlightening and empowering rather than depressing, I suppose.

Rebecca: Sometimes that seems tied to this idea of an assumption that students need to come in with certain skill sets, rather than meeting students where they’re at and recognizing where they might be at around this particular idea of academic integrity.

David: Growth mindset really is bound up with another really important psychological concept, which is self efficacy. Students are much less likely to cheat if they think they can do the assignment to their standard in an honest way. If they think it’s impossible or too time consuming or unlikely, then they’re more likely to take a risky or a dishonest approach. And of course, if they come into a class thinking they can’t do it, and they can’t ever do it, then you have low self efficacy and a fixed mindset. And those are students who are either going to fail or cheat because they don’t see any other choice.

Tricia: Fail, cheat, or drop out. We’re going to lose them one way or another.

David: So the question is, are we educators, or are we cops? And if we’re educators or are we gatekeepers? And if we’re educators, then our job is to figure out how to get them to at least engage with the material enough to have one success experience and hopefully let that build to a second success. So one might argue, yeah, our job is not to make our students feel good about themselves, and I would actually agree with that, both halves of it, but it is our job to give students tasks that they have a shot at completing, at least to start out. Because we’re not gatekeepers, at least not mostly, we’re educators,

Tricia: Because our book is focused on instructors, and when we talk about it, can sometimes feel like we’re piling on instructors, that we’re giving them all sorts of advice, and there’s so much on their shoulders. And I want to point out that in our book, we also make sure we say, “Hey, institutional leaders, if we’re going to expect faculty to do all this, to make these changes, to adapt their courses and assessments to the age of AI, they need to be given the time, training, and support to do so.” We were not taught in our PhDs how to teach. Like K through 12 teachers were taught how to teach. We were not taught how to design valid assessments. And yes, a lot of us will work at some place where there’ll be a teaching and learning center that will have maybe two instructional designers or maybe one person, and they’re often focused on helping faculty create online courses, as if teaching in person is easier and everybody just knows how to do it. So as we’re giving all this advice to instructors, we’re saying here’s a menu of things we want you to think about. Choose what gives you agency, what empowers you now to tackle this problem? Choose something that you might do next year after you do some more research and institutional leaders, you need to give your faculty, time, training, and support to deliver on this new teaching and learning imperative for the age of AI.

John: We’ve done several podcasts with Elizabeth Canning, and she’s done a fair amount of work on the importance of both instructors having a growth mindset and in building that growth mindset for students, not only does that tend to benefit all students, it tends to also reduce some of those equity gaps. So we’ll include a link to those podcasts and some of the resources there in the show notes as well. With the introduction of AI, a lot of faculty panicked, but I don’t think, from what I’ve seen of surveys, student cheating continued, but it didn’t seem to really increase. The surveys that I’ve seen have indicated that was mostly a shift in the type of cheating. Instead of subscribing to Chegg or other services, students were perhaps a little bit more likely to rely on AI to do some of that, and faculty, though, seemed to be much more concerned about this type of academic integrity issue than with previous ones. What can faculty do to address the issues with AI? You mentioned before that one issue is, some things can be done by AI. Some things can’t. What can we do to help faculty, perhaps, discourage inappropriate use of AI?

David: I think it’s all the things we’ve already talked about. Start by making sure you understand as a faculty member why you’re banning AI or what the rules of engagement are. Is it a knee-jerk reaction. Is it fear and ignorance, or is it a principled approach based on the learning goals of your class? Then, once you’ve done that, and if you’re confident that your approach to AI is based in the learning objectives of your course, make it clear… to start within the syllabus, but I would say with each assignment that you write create some sort of meaningful engagement with AI for your students. I don’t mean have them use it, but tell them what the rules of engagement are for every assessment and every activity in the course, and why it is what it is. And I don’t mean a defensive “why,” because we ban AI when we’re asking students to do fundamental skill-building tasks that will ultimately benefit them in the future, as they try to do more complex tasks, as a general rule, and we allow AI when that AI is enabling them to demonstrate the knowledge that they have in other ways more clearly. And so I think figuring out why you’re doing what you’re doing, telling students when it’s appropriate and when it’s not and when it is appropriate, I think it’s incumbent upon us to learn how to use these tools appropriately and teach our students how to use them appropriately as well. There’s a number of really useful research tools out there, for example, that I don’t think anyone would question the ethicality of using if you use a Google search, you would probably should also be okay with using an AI-aided Google search as an example. So we should start with thinking about it and then talking about it and then doing it, I would say.

Tricia: We encourage, really in the book, communicating with your students. So that means asking them, what tools are you using? How are you using them? How are you finding them helpful? How are you not finding them helpful? Hey, you know how faculty always complain that students don’t read the syllabus right? And they complain that they don’t look ahead, they don’t plan ahead for all the assessments in their class. So let’s look at the syllabus together. Here’s all the assessments or activities we’ve got coming up. When do you think it would be ethical to use AI or in what ways? And when would you think it not be ethical? When students hear each other talk about this, whether it’s asynchronously on a Google Doc or in class, again, pure norms… they start to say, “Oh, so I’ve got my peers saying I shouldn’t write my first draft of my paper with AI, interesting.” That lands different than it does when the professor just says, “Don’t do it.” And then the second thing is, if you’re going to say, don’t do it, you better secure that assessment. So if you are on a diet and you’re trying to not eat any sugar, you do not put sugar in your kitchen. You secure, you lock that baby down, right? And you say, like, that cupboard has no sugar in it. And yet again, we tell students not to use something. And then we say, now go take your exam unproctored at home on your computer [LAUGHTER], and it’s a dereliction of duty. And so we really do have to decide. If we’re going to ban it, it needs to be a secure assessment so we can actually enforce that, and so students don’t feel like you told us not to use it, but everybody’s doing it anyway. So this is not fair. So those are two things I would add to what David said.

Rebecca: One of the things that we often find students focusing on is grades and maybe not always learning. And obviously this conversation is focused a lot on the learning aspect, and even with the idea of assessments, kind of talking about some of the values of those assessments. Are there other approaches, like alternative grading, or other things that you might recommend that might help shift that perspective?

David: No… just kidding. I’ve tried them all. I haven’t tried ungrading, but I’ve done all of them, and I love some parts of all of the different alternative grading schemes, and really don’t like some other parts. So I’m loathe to recommend anything to anybody else beyond stay open-minded and then ask what grades are really for in your context and institution. At elite institutions, I think that they really do play a different role than they do at open admissions, broadly serving institutions. And so I’m not going to sit here in my office and tell anybody how to do anything in their classes, particularly grading, but I will suggest that you rethink how important grades are in your courses, and how important you communicate about grades in your courses. When students are pitted against each other for points, when faculty talk about the points and the grades and the numbers instead of the learning and the outcomes and the processes, students internalize that, they absolutely do. I for one, have shifted away from rigidity in my courses to flexibility with all sorts of things with respect to grades. So a simple example of this is the redo. I’d call it a mulligan, but I don’t think any of my students would really understand what that means. So in one of my project-based classes, they get two redos, plus they can earn one by actually reading the syllabus and responding to an Easter egg in it. So they get three to start. And then if I need to give extra credit, I can give an opportunity to earn more redos, rather than just giving away points. Those redos do a million different things, but one of them is they build in the possibility to fail. They allow students to try something they’re not sure they can do in a way that completely crashes and burns. And I tell them, please crash and burn at least once. It means you got out of your comfort zone, and it’s free, it costs you nothing. And that gives those students who maybe are concerned about engaging and concerned that they can’t be perfect to their own standard, a chance to give it a shot. And of course, more often than not, they’re better than they think they are, and they often don’t even need those redos by the end, but they took the chance. And you can also use a redo in my class, by the way, to get an auto extension so you don’t even need to ask for it. You just can use a redo for that. And then the flexibility means that you have that student who might be failing a class after the first assignment suddenly is still in a position to get an A as long as they don’t do it again, and it just opens up a world of learning opportunities for your students, and at the same time, reduces their desperation levels. And desperation levels are completely correlated cheating. Those kinds of flexibility, whether it be in the structure of assignments like redos, or whether it be something like specs grading, where you can take a look at bundles of points and give students the opportunity to demonstrate their learning in ways that make sense to them. Ungrading has its challenges as well as a wholesale change. And I say, go for it if you have the time and the motivation to do it. But whatever you do, think about flexibility and think about ways to lower that student desperation.

Tricia: And David’s answer there, just reminds me of something that might be important to say in terms of our book. This is what our book is. Our book presents you with multiple options. So as David said, we don’t tell you you must do ungrading and that will solve the cheating problem. We say, “Think about grading. Here’s all the options.” And we use a lot of storytelling from both of our experiences and the experiences of others in the book to illustrate, to provide that anecdotal data. And that’s what this whole teaching and engaging and thriving series in higher ed is about, is these practical books full of research-backed, theory backed-strategies, along with personal stories. And it really is just a menu of things. I think we’d be really firm on a couple of things, like you have to secure assessments if you’re going to ban AI and you really need to communicate with your students. I’ve talked to hundreds of students now about Gen AI misuse, and a lot of them just said, “I just wish our professors would be more clear on what’s allowed and what isn’t. I just wish they’d be more clear.” So those might be the two things that we’re really only very firm about that you really should do everything else is a menu of options.

John: You mentioned securing assessments. The share of students taking classes remotely has grown pretty rapidly since the pandemic, which raises a lot more concerns with this. And there’s also concerns about proctoring services in terms of Fourth Amendment issues and legal issues associated with that. What sort of advice would you provide for people teaching online classes in addressing the use of AI?

Tricia: Yeah, I’ve said a few times now on LinkedIn and social media, online learning can continue. I don’t know that online assessments can continue, at least not if they have to be secure. There’s those proctoring companies that have their own concerns. We’ve got deep fake. We used to say, “Oh, just have Avixa, have an oral meeting with the student where they tell you what they know” …they could deep fake it. Or Derek Newton just showed us at the International Center for Academic Integrity conference in his keynote that there’s just even software to make it look like I’m looking at you taking my test, but I’m really looking up here at my notes. And so even doing those kinds of checks on knowledge are going to be super difficult. One thing that we’re doing here in the University of California system is we’re looking at computer-based testing facilities. Now this is not our idea. University of Illinois, Urbana Champaign started this a while ago. Faculty created an assessment platform called Prairie Learn. So the great thing about Prairie Learn is it does allow mastery-based assessment because it helps you individualize the assessments, and so students can take them over and over again until they master the learning objectives to the level they hope or that we want them to. But also, in a computer-based testing facility, we take tests or assessment administration out of the hands of faculty and TAs who, frankly, their time could be better well spent engaging with students, teaching students, coaching students, tutoring students. And the students come in, we make sure it’s the student who is enrolled in the class is the person actually taking the test, and then they have no access to anything they’re not supposed to have access to. And we’re looking at building those out across the system, and then having an agreement with the Cal State system and the community college system so that any of our, at least California, students who are taking online classes could easily still accessibly transport themselves to a local in-person assessment. Now, the other thing we wanted to make sure when we talk about this is something that another Australian colleague, Phil Dawson, talks about, is probably not every assessment in every course needs to be secure. And we got to start maybe thinking at a programmatic basis, what do students need to know and be able to do by the time they finish this program? At what points do we need secure assessments to make sure they’re hitting those benchmarks? And then, if they’re not hitting those benchmarks, how can we help them catch up? And so I think we just need to rethink how we deliver [LAUGHTER] curriculum and how we assess whether students have met those learning objectives. Another Australian colleague…. we keep always citing our Australian colleagues because they’re brilliant… but Kane Murdoch says that’s like putting a bandaid on an amputated limb. It’s just not going to work if we just try and tinker around the edges.

David: To cite another one of our Australian colleagues, you want to take a Swiss cheese approach to academic integrity, because no one thing will work. So start by building a basis of integrity, by communicating integrity, building dynamic, interesting, and relevant courses for your students, just like we’ve been talking about. But then some assessments that you give them will need to be secured, as Trish said, not all of them do. You can think of it as sort of a division between formative and summative, but it doesn’t quite necessarily have to break down exactly that way. But when you’re getting to those big summative, high-stakes assessments where you’re making sure people can actually be effective doctors or engineers or even psychologists or teachers, it doesn’t really matter what it is, everybody should be able to demonstrate those skills and knowledge. That’s when you have to start using series of other options. So that might mean a technological option, especially in online courses, technology is going to have to play some sort of role, but today, you can probably secure an online assessment at the margins, but still, anybody really motivated can probably work around it. In five years, most elementary school students will be able to work around it. So yeah, I think even if we can secure online assessments today, and that’s an if, we will not be able to secure them long term, and not even the new, clever assessments that the technology allows for. Eventually, we’ll get to a point where AI is testing AI, and people won’t be in there at all, unless we choose to make it so that we’re in there. And that means more human time for us as instructors and assessment creators, and more human time for the students as well.

Tricia: And when we think about it, what do students need us for? They need us for human-to-human interaction. They need us to develop skills and knowledge experientially in solving problems. They do not need us to deliver information. And when you think about it, online learning was really a way to deliver knowledge to people. My master’s was in rural extension studies. How do we deliver knowledge out to everybody, whether they can come to campus or not, and we’re just not in the business of delivering knowledge anymore. We’re in the business of creating knowledge. We’re in the business of co-creating knowledge. We’re in the business of human-to-human interaction. So if we want students to give us money for something other than a piece of paper they get in four years or three years, then we need to provide them with something that AI can’t and what is that one thing? It’s interaction with other humans. It’s learning with other humans. And I thought maybe this is a good time. My tea is Yogi Tea. It comes with quotes, and I just think it’s so fitting that this is the one tea bag I just pulled today. And the quote is, “we can always start again.” And really, at a higher education college, we need to start again. And that sounds very painful, but we have to start again in thinking from scratch of what are we offering, what’s our value to students and to society in the age of AI.

Rebecca: Part of that seems like we all start again as instructors as well, and make sure that we’re familiar with how AI can help us and help our students, instead of constantly fighting it, like better understand what some of the possibilities are so that we can do that redesign work.

David: I don’t think anyone said yet in this podcast, rearranging the deck chairs on the Titanic, but I think it might be time to say that. You should buy our book and you should use the techniques that we suggest, but unless post-secondary education makes some fundamental changes philosophically and practically, that’s what we’re doing. We need to recenter humans in what we do, and we need to offload the parts that we can offload. AI is going to be an incredibly powerful tool for us as teachers, just as it’s becoming for assessment takers and hopefully learners at some point as well. So we need to figure out how to deliver the value that Trish is talking about.

John: When I give workshops to faculty, I’ll often remind them that we need to teach students for their future, not for our past, things that worked for us in the past in the world before, AI may not work so well, and there are a lot of people out there trying to find ways to integrate AI into this, because students will be working with AI in the future, and they have to be able to add some value to that. And if we can somehow assess your ability to go beyond just AI, perhaps that might be a direction that’s worth exploring. And there’s a lot of people trying to work on that right now. Is that something that might be a way around some of these issues?

David: Yeah, I think it’s exactly what we’re advocating for, which is to build assessments that allow AI to be used in a way that is fair and equitable and clear and that allows us to see what students are bringing to the material over and above those same fairly used tools that everyone else. Having said that, there’s also going to be those assessments and those skills that you really need to be able to use without AI, because those are the bedrock, foundational skills, and also the things that you simply don’t want to trust AI for. Computers don’t, and probably won’t for a while, have the ability to do intentionality. They don’t understand that there’s a physical, real world out there. They are living in an electronic world, and so that means you can’t ultimately trust them with ethical choices. So that’s on us to make those ethical choices and to use machines in ways that are ethical, and that’s going to always be, at least for my lifetime, a big part of what we’re teaching our students to do.

Tricia: And I also think that when faculties play with the tools, they have to remember that they are experts in the discipline, probably in what they’re asking the AI tool to help them. And so the other thing I’ve heard, some mistaken conclusion is, “Oh well, I found it very useful for helping me write, so I’m going to let students use it to help them write.” They’re novices. They’re novice writers compared to faculty, they’re novice in your discipline, they’re novice in the content. They’re novice at a lot of skills that you may have been working on, and so how they should be using it, or how you need to scaffold or coach their use in it, is very different than how you would use it. It’s important for us to keep that in mind, that we have to think again, like you said, the future, right, for their future? Well, what does the future look like? Because right now, we only know how AI is beneficial to people with expertise who didn’t have it and developed all these knowledge and skills without it. How do we prepare them for that? If we just allow them to offload everything to AI, then will they be able to use it in any way that’s meaningful?

Rebecca: That seems like a really good note to get to our last question, which is we always end by asking: “What’s next?” [LAUGHTER]

Tricia: That’s a big question.

David: What’s next is change, that I can promise you. We’re at an inflection point, and this change has been coming for a while. COVID, we thought, was going to be the impetus for that change, and maybe it would have been without Gen AI, but there’s no doubt that Gen AI is going to be the impetus for a huge amount of change in almost any part of our lives. But I think that education is going to be the pointy end of the spear with respect to that, and you can think of it from just one little example, which is the intellectual power of writing. Writing, until now, or rhetoric, maybe more broadly, has been the way that we’ve honed our ideas as humans. It’s the way that the late, great Frank Yates, who taught me undergraduate psychology, said writing is nature’s way of telling you how unclear your ideas are. And that’s true of all rhetoric, and it’s actually a fundamental fact about our lives as intellectuals up until now, but suddenly it’s possible to create a coherently and sometimes even well written document without understanding what you’re writing. That’s what Gen AI does. Is this a good thing or a bad thing? Well, hard to say, but it’s a new thing, and it is what’s next. And so the question is, what is the next generation of intellectuals look like when they don’t have to write to think? I don’t know what it’s going to be, but it’s going to be a bumpy ride getting there.

Rebecca: It’s a good thing. You said you were leaning into flexibility, huh? [LAUGHTER]

David: It’s really the only choice. It’s amazing what you can do when you have no choice. [LAUGHTER]

Tricia: I have nothing to add there. That’s a great answer, and I think I’m just gonna upvote what David said. [LAUGHTER]

John: Well, thank you for joining us. We strongly recommend that people take a look at your book, because everyone in higher ed is addressing these issues, and it’s a really nice resource that’s very timely. And thank you for joining us.

Rebecca: Yeah, thank you so much.

David: You’re welcome. Thanks for having us.

Tricia: Yeah, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

166. Course Villain

Posted in academic dishonesty, academic integrity, higher education, learning, pedagogy, plagiarism, teaching
December 16, 2020

A number of online services exist that facilitate academic dishonesty. In this episode, Zachary Dixon and Kelly George join us to discuss Course Villain, a platform they created to detect crowd-sources plagiarism. Zachary is an Assistant Professor of English at Embry-Riddle Aeronautical College, and Kelly is an Associate Professor of Economics, also at Embry-Riddle Aeronautical College.

Show Notes

Transcript

John: A number of online services exist that facilitate academic dishonesty. In this episode, we discuss one institution’s project designed to automate the detection of academic integrity violations.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Zachary Dixon and Kelly George. Zack is an Assistant Professor of English at Embry-Riddle Aeronautical College. And Kelly is an Associate Professor of Economics, also at Embry-Riddle Aeronautical College. They are co-creators of the Course Villain tool, which is designed to track crowd-sourced plagiarism. Welcome.

Zachary: Thank you.

Kelly: Thank you.

John: Our teas today are:

Zachary: Sparkling water for me.

Kelly:I have hot lemon water for me.

John: Oh, that’s close.

Rebecca: That’s close enough to tea. I have Scottish breakfast.

John: And I have ginger peach black tea. We’ve invited you here to talk about Course Villain. I saw your presentation on this at the OLC Accelerate conference recently, and we invited you here so you can share that with our listeners. Could you give us a brief overview of how this program came about?

Zachary: Sure. Actually, we were at a college faculty meeting for all of the College of Arts and Sciences and plagiarism was kind of up for discussion a little bit. And I’m on the younger side of the faculty, or I was at the time, at least, and Kelly has children of her own to keep her more in touch, I guess, with the digital scene. And so Kelly and I were like, “Well, does anybody else know what’s happening on this place called CourseHero. And so many of our other fellow faculty did not know they’re kind of like, “No, what’s that?” …and we pulled it up on a screen on a laptop and really started to blow some minds. And so Kelly and I started talking about that and just got to the idea that, you know, this doesn’t seem like it has to be an impossible problem. This is more an interesting problem. Let’s see if we can game this a little bit. And we started coming up with some strategies and ideas on how to track this kind of behavior and see if we could use old Google Alerts, it was our first kind of ambitious attempt. We tried to manipulate Google Alerts into telling us when things happened on Course Hero , and it was a terrible failure. It didn’t really show us anything at all. And so that kind of prompted us like,”Wow, we need to do better.” And so we started calling the computer science department and trying to find some researchers, and it kind of just took off from there. We saw this one little issue and we thought maybe we could game it and play around with it, and it’s developed into something fun from that.

John: Could you tell us a little more about how the program you developed functions? What does it do?

Kelly: The program itself is a software function that it’s been built in a number of iterations with computer engineering students, and they use a web crawler and a web scraper. So you would put in your query. John, being an econ major, you would say, “Well, I wonder how much of econ 101 John Kane is on CourseHero.” And so you would put in the query the search features. And then what this program does is it goes into CourseHero, and it then generates hits, which are artifacts. And depending on the time that we have it running, it will generate a report to us, the user, of the 20 most recently uploaded artifacts that meet your query, so you can determine for your class what’s going on. So before it, old school, if you were a faculty member, you’d create a syllabus, and then you want to see what’s out there in the world that would make your course compromised. And at one point in time, we had it scanning once every 24 hours. You can set the duration, we can change it to 12 hours to 24 hours. The report comes back to us, and then we can look at the artifacts. Sometimes it’s a very top level screening by the instructor. We can say, “Yeah, that’s the final exam that shouldn’t be uploaded,” We had an option in the second part of the program is it auto generates that takedown feature that is so difficult to overcome with CourseHero. If you did it manually, if you have 40,000 documents, I think someone at our conference said last year, they generated something to the tune of like 60 or 70,000 takedown requests for CourseHero. And if you’re doing that by hand, that’s more than just one person doing that takedown request because of their requirements. So that was the bigger challenge, I think, in this program was to generate the takedown request. So what we envisioned was this program just kind of constantly working. So it’s constantly scraping and generating reports, and we were generating takedown requests. And Zack was the one who said it’s kind of the old Disney YouTube dynamic and phenomenon. Way back early in YouTube, back in the early 2000s, you can find every Disney movie across YouTube. And then basically Disney’s teams of lawyers and copyright content owners got involved. And they made it so difficult that now YouTube, they have algorithms and they have firewalls where you can’t upload Snow White on YouTube. And so that was kind of like in our perfect world, Where would you want to get to be where CourseHero says “Ah, that’s from Embry-Riddle. We don’t want it. It’s too difficult.”

Zachary: In terms of the functionality of the program, it’s deceptively simple, where really we just kind of built a very carefully tuned search engine, just like Yahoo or Google, where we’re essentially just leveraging the API metadata, that application program interface that the big dogs use. We kind of just built our own smaller version that does what we needed to, and that’s about it.

John: Have you tried using this with anything other than CourseHero, for some of the other sites out there for Chegg and and all the many sites that have been appearing in the last few years?

Kelly: No, not yet.

Zachary: No, we haven’t.

Kelly: It’s on that map of what you want to do. It’s all resources. You’ve got to get students to maintain the program and keep working it and to alter it a little bit. That would be the ultimate goal, to put them on the other big crowdsourced study aid platforms.

Zachary: And following up, it connects back to that leveraging that API metadata and that interface does restrict our ability to move between platforms, because we would have to re-tailor it, so to speak, kind of recut and sew it. And one of the challenges over the lifetime of the project now for coming up on three years has been exactly that, maintaining that interface. CourseHero is also always updating and revising it, trying to get better profiles on the search engines and maybe assumably trying to keep people like us out. So the maintenance keeps this one step working, and then that’s also kind of deterred us from trying other platforms.

John: So you’re searching primarily on the instructor’s name and the course? Is that the criteria?

Zachary: Well, you can define any criteria in Course Villain right now. The only kind of hard tuning we have is that we have manipulated Course Villain to only search for Embry-Riddle content right now. So we’ve restricted the search engine’s functions by our university, and that has allowed us to have better accuracy in our reporting, because before we are getting anything with English 123 would pop up and really cloud the results, so we had to kind of throttle back the scope.

John: One thing I’ve noticed recently with CourseHero in particular, is that my material now is showing up with me at nearby institutions. So students may have tuned into this because I have Eco 101 materials and Eco 350 materials at Cornell, at Syracuse University, and a few other places now, including, I think, one in North Carolina, which is my content that I created, and it has my name there, but students are getting a little bit… not terribly…but a little bit more savvy on trying to avoid detection by instructors. So, that may be something you want to look at.

Kelly: But, I think also one of the explanations for that is, not only our students getting savvy, Lord knows, they can re-engineer anything fairly quickly, instead of just doing the work. But if your material is being recycled, so primary, it’s yours at your institution, and then someone who’s just looking to do a paper or to do something is just plunking around out in the web and steal some of your things and brings it in and then when they recycle your material and upload it, because CourseHero has got that pay for play, you got to upload something. So, I can just upload this material, and then you move into the second and third generation of your material.

John: Interesting. I hadn’t thought about that possibility. Because this was material from last year, some of it was exams from last year. And I just assumed it was my own students who posted it somewhere else to share it with other students, letting them know where to look so that they could evade detection. But the possibility of it being recycled is certainly a real one.

Zachary: Seeing your own work not labeled appropriately. I also saw the same thing. I think there’s also like a level of student error involved where students will like create department names and college names that don’t exist at the university, and maybe they just don’t know. Something else, and Kelly observed this really well in our data analysis, was the issue of student agency is really hot here. And it’s probably like, three steps ahead of where we’re thinking, because students are gaming us, they’re gaming the university, and they’re gaming CourseHero too. They will not just post reasonable material. It could be John Kane’s exam, and it’s just like two lines of nothing text. CourseHero doesn’t appear to have a lot of oversight over their own collection and so the students are really steering the ship in a crazy way.

Kelly: The one thing that made me look at that was when we were doing an analysis on one of the first reports and these four documents for like a physics course kept popping up. And they were like almost management case studies or something like… notes on Boeing or something… they kept popping up. And then I drilled down and what I realized was one student must have been doing the pay for play and the student just took six or seven documents that were readily available, not necessarily their documents, like their paper, where someone could actually go in and if you didn’t remove your name or anything like that you can see. My students examples are always Joe Bagadonuts. So, you know, it’s Joe Bagadonuts’ English paper or economics paper. So this individual was just uploading these PDFs, and these documents, just to get their ticker up so that they can download documents to compromise.

Rebecca: Do students know about Course Villain at your institution?

Kelly: They do now. [LAUGHTER] Marketing finally got ahold of us. We’ve had a little bit of localized publication. And then actually, from that, a fellow colleague of mine, who didn’t know we were working on this, came back and said, “Here’s an idea,” he actually appealed to his students, and said why you wouldn’t want your work up on CourseHero. And through that, in his small class, he was actually pretty successful in getting them to take down the work. And because I asked him, I was like, “How did you do that?” And he said his line was “Well, I asked them politely,” but then he gave an explanation of how he appealed to students. And it got me thinking that, in our next iteration, instead of just trying to prevent or catch the plagiarism after the fact, that that would be a good preemptive strike, to create some type of a PR campaign with our students in our university to say “Here, you can help us out here. This is why you don’t want your material up on CourseHero.” So if we came at the problem from both ends, it just hadn’t slipped our mind. That was eventually our end goal. My end goal was always that as a university, our honor code system would expand so much that our students had so much pride in the honor code system, that they would work to elevate that. And that would be one idea of how to start out that prong of the research.

Zachary: One interesting student dynamic has been our student researchers, our programmers. So we’re pretty proud of the fact that we have funded all of Course Villains so far with internal grants through the university, and we’ve only employed student programmers to do the work. Neither Kelly nor myself are capable programmers. I can maybe throw together some HTML if I tried, but we really have been leaning heavy on our student programmers. And they’ve had a really positive response to this experience. They’re aware of CourseHero when they come into this and the kind of that ilk, and I think they appreciate the hacker mentality that we kind of foster. We kind of have a cool proverbial David and Goliath situation here, this kind of huge onslaught of material and this kind of corporate juggernauts. And compared to just a couple faculty members and a small budget, it does create a cool atmosphere. And I will say, with no judgment on anyone using or not using CourseHero, though, when we do get our student programmers involved and we kind of sit down and talk about it, I definitely get the sense that diligent students appreciate this work in that it is combating something that they understand undermines the quality of their education experience.

JOHNr: How have your colleagues responded to the reports you’ve generated on the frequency of materials from their courses appearing on CourseHero?

Kelly: Shocked. It would shock any instructor. Disheartened, and then engaged. They want to use our software right away. And, because not only do they want to pre-empt, they want to get their own material taken down. They want to catch the students that are flagrantly violating academic integrity, and they want to use it. So, they’ve sprinted way, way ahead of us. So, the motivation is there. I think it’s a matter of trying to figure out where in the grand piece of the puzzle of our institution that this product can work.

Rebecca: How many takedown notices has Course Villain created or generated?

Kelly: I was not prepared for that question.

Rebecca: Oh. We can cut it out.

John: I hadn’t thought of it either., but it’s a good question.

Rebecca: But I am curious. [LAUGHTER]

Kelly: No, I wouldn’t take it out. I just think, wow… That’s a good question. Actually. We were too busy the last two years. The first year we did was “Could we create this mousetrap?” The second year was dedicated to “How well does this mousetrap work?” And then this next year, this year that we’re in, was really a question of “How can we incorporate the use of this? How effective is it? And how do we incorporate it into our instructional design process and get it into our workflow to where it helps us rather than becomes just an obstacle you had to come around?” So we don’t have the data on the takedown requests. I’m gonna get it though. [LAUGHTER]

Zachary: The early experiments that I can definitely recall and speak to on the takedown requests have been pretty largely unsuccessful insofar as CourseHero is looking for specific amounts and specific kinds of verbiage in that takedown request. And since we’ve been more focused on, like Kelly said, program development and stabilisation, we haven’t been able to focus in on “What is that verbiage that we can achieve from a programmatic perspective? Like, “Can I make Course Villain say the right thing that will work every time?” or do I really have to work with an instructor level basis, like “You’re gonna have to supply this part of it,” because we can auto populate the whole form and send it on. But, so far, the couple dozen I’ve tried (focused on trying to make the thing work) have all gotten bounced back to me, you know, “More information is needed.” Essentially, you need a better argument. And the next step is largely experimental. Like, what is that? And it’s a lot to do with institutional workflow, really clarifying, like, whose intellectual property is what and why? What are we able to defend and argue? Some of these are questions for the development team, for Kelly and I, some of these questions for administration at our university, to help us chisel out here, to have that ability to kind of go back to CourseHero. You get to that legal end of things and things get weird, I think.

Kelly: Within our college, I just taught a master’s level class and now all the major assignments that are submitted in this research levels class does have the copyright at the bottom. Remember, Zack, about a year ago, we were talking about that? Well, if all Embry-Riddle submitted assignments had this statement at the bottom, like in the footer of the assignment, then it would be protected. That was one of the ways to combat the changing language and requirements of CourseHero. So if you can say “No, this is copyrighted, I’m a representative of Embry-Riddle. So, you do have to take this information down.” But I did notice that, in the last class that I taught, the research methods, that that was a fundamental change.

Zachary: There is also a pretty murky or wicked issue in terms of disciplinarity. And what exactly students are posting in there. And so there is kind of a level of judgment that has to be made where, “is this appropriate to be taken down? Is this truly our/my/the university’s intellectual property or just does that belong to the student?” And as I study academic integrity, and plagiarism, in particular, that line is really down to communities of practice, professional organizations, disciplines, courses, and instructors themselves. So that line has to be teased out too. That’s kind of worth mentioning.

Rebecca: So in addition to tools like this, what are some other strategies you’ve seen faculty in your department or at your college start implementing to help reduce the use of sites like this? You mentioned one faculty of doing an intervention. [LAUGHTER] Have you heard of other things that faculty are doing perhaps as a result of you’ve been hearing about Course Villain, and CourseHero?

Zachary: Well, I know one large-scale institutional change is we’ve actually revised the Student Code of Conduct to include these kinds of sites and the behavior of sharing work. It is defined as plagiarism, as a violation of academic integrity, to even be involved. Still, that kind of leads to weird and murkyish paths, where “Can we track down someone based on a name on a document and an avatar on an open site?” I don’t know. That kind of these legal things that go beyond my simple English professor life and knowledge. But changing the Student Code of Conduct was a big one. And I think in STEM fields, physics, math, I know we talk a lot with a faculty about micro changes to assignments, where you don’t have to do a lot to revise a mathematics or a physics equation-based assignment: change decimals, change units, change little things here and there that can really fundamentally alter the structure of an assignment. And that makes it more resilient in terms of how it’s shared and used. And from an English Composition perspective, my best work against plagiarism has always been to encourage students to follow their own passions and ideas, to write about what they want to, to let students guide topics and selections as much as they can. And then to complement that with long-term scaffolding where you’re gonna pick something and we’re gonna work on that something all term. And so if you’re committed to plagiarizing in that context, then that’s a whole lot of dedication I’m not prepared to counter at the moment, but otherwise, it’s like, “Well, if I get to choose my adventure here, maybe I don’t have to go steal it.”

Kelly: Yeah, I was gonna piggyback off of Zack with “It’s the authentic assignments.” Having taught this last half year to a year I’ve been teaching a lot of the upper-level research methods class, where these are the students they’re rolling into their thesis, we call them capstones. So they start with the problem statement. Each problem statement is unique. Getting the data… the data has got to be the most recent. I did have one student that I think he must have had some material. And about halfway through the class, he realized that the authenticness was going to be removed fairly quickly, and he was going to be found out, because he changed pretty quick about halfway through the semester he said, “You know, I’ve got a better idea” and did all that. So it’s just building a better learning experience for the student. Unfortunately, it’s a better learning experience for the student that’s born out of this malicious behavio.r

Zachary: And otherwise, at Embry-Riddle, we also do operate with TurnItin software, there are investments being made in lockdown browsers, and two or more factor identification to try and get at some of the more complicated forms of contract cheating that are… I don’t think they’re pervasive… but they are very hard to deal with.

John: Have you requested any funding from your university in developing this package?

Kelly: Early on with the university, we filled out our university intellectual property, and you meet with the intellectual property office. And the whole idea is: “Is there some technology that you can eventually move into some type of a commercial project out of the academic world into the commercial world?” And I think maybe we weren’t just expressing ourselves. I almost feel like it was Shark Tank. We had this idea, but it wasn’t a business yet. It was a great idea. We had some kind of functionality, but it wasn’t a business yet. It just wasn’t developed enough yet for them to be able to take it out and market it around. Because now there is a team in our IT department {we’re on the committee) where they put out a request for proposal. And it was for software that helps combat plagiarism and things of this nature. So we haven’t seen the proposals that have come back in. But it seemed to us this was a tool that you would think that one of these educational learning management systems would be interested in taking and incorporating into… we all use Canvas… if that was integrated into Canvas, where not only just a turnitin report came up, CourseHero or one of these other crowdsourced platforms would come up as well. So the request for proposal’s out, we haven’t seen anything in there. But it would be interesting if we could re-engage the Intellectual Property Office at our university to see if there’s really, as Jerry Seinfeld would say, “Is this something?”

John: Along those lines, I know Lumen Learning, who produced the Waymaker personalized learning platform, have put together an API that automatically checks the web for postings of their questions and content. And they do use a lot of algorithmic questions so that each student gets a different version of the question. And I believe some publishers have started to do this. I haven’t checked that myself. But it would be nice if the arms race could escalate a little bit more on the side of deterring academic integrity violations. One of the things that’s happened, though, on the other side is many departments had picked up departmental subscriptions to places like CourseHero, because one thing that many of us are troubled by, myself included, is having to pay $15 or $16 a month for a subscription to a service primarily engaged in facilitating cheating. And many departments have created departmental accounts. And Chegg, for example, now, is only allowing you to use a given account on specific computers. So, you’re limited to the number of computers that you can connect to. I had my own account for this for a while now, because I teach a lot of classes where these things come up. But, I know some departments had created departmental accounts where many people were using the same account, cheating a little bit on the instructor side, to try to shut some of this down at a lower cost and without subsidizing them quite as actively. But more and more places are starting to move their stuff behind a paywall, which may make it a little bit harder to detect some of these violations.

Zachary: Well, John, I’ll follow that up. Actually, the College of Arts and Sciences here at Embry-Riddle Worldwide, we’ve also purchased a kind of college subscription for the same access. And Course Villain isn’t exempt from that kind of dependence either, because one of the really annoying issues we stumbled into early, was CourseHero’s own document preview limits, where you could only view 10 documents… their cover page, so to speak… before they would ice you out and give you kind of a grade. This is only for your eyes only kind of situation. And if you don’t have that paid access… that limit. So paying for it does kind of peel back that one layer and at least give us the option to view the artifacts and help correlate them. But that’s tough. But I also want to come back to the kind of arms race idea. If I had just my closing stump speech for Course Villain, it’s that it doesn’t have to be a very difficult arms race. It can be kind of an arms walk, where I’m really excited and pleased with our development of Course Villain and we’ve accomplished that with about $8,000 of internal funding. $8,000, two faculty members, and we’ve hired two research assistants… two student programmers… over the course of these couple years. And we’ve developed what we have. And I think our model is replicable. If we’re not going to create a product that the university wants to sell to other people. I really encourage other universities, other faculty, other interested researchers to start coming up with your own low-cost mousetraps, because that’s all it takes is kind of just chipping away at it, and before you know it, we will find that corner, that inroad, that kind of loophole that allows us to make really good work. It’s not out there. We don’t have to wait for publishers and juggernauts to do it. We can handle this. I really am confident in that.

Kelly: And we actually had this conversation after the last conference, should we have one more conversation with our university with the Intellectual Property Office of our university? Or do we just turn around and say, here’s the open source, go forth and provide it to other universities, and do our own crowdsource of how to combat it and see where it can go from that?

John: That could be an interesting project where other institutions may contribute work on other platforms, where people could get a menu where they could choose what to look for, and perhaps some people could specialize on generating those takedown requests. So that it wouldn’t require the work of just a few people there, it could be distributed more widely. That would be an interesting possibility.

Kelly: I bet there’s a grant in there somewhere, Zach.

Zachary: I hope so. [LAUGHTER]

John: Are there any other things you’d like to share with our listeners?

Zachary: If there’s one thing I would add that I don’t think I got a chance to really frame in the strongest sense that I wanted to at the conference, is the idea that what’s also at stake in this kind of scary moment, as we talked earlier about how people are finding themselves forced into this digital environment. And it’s easy to have your breath taken away at the scope and scale of this problem of people sharing coursework in kind of an illicit fashion. The other opportunity there is that we’re suddenly getting more data than we’d ever dreamed of. For my research from looking back at the histories of plagiarism of academic integrity violations. When you try and find the hard numbers of frequency or prevalence of this behavior, it doesn’t look like the Internet has really changed the nature of academic dishonesty, it just brought it into a new venue. And that makes it look scary, because now it’s like, “Oh, it’s happening fast and on really big scales.” But this kind of fundamental issue appears to remain largely the same. But what’s changed is our ability to understand and track that problem, because now it’s not between hands, under desks, or in dorm rooms or in the hallways where it could never, or very rarely, be tracked down. I think of like the kind of investigatory efforts invested in like the Naval Academy plagiarism scandals, where you have to track down students and interrogate them, and like bring them against charges, to get them to rat each other out, and then you expose this network of plagiarism when it was really just some people sharing something in a problematic way. But now that’s rendered in this digital space, and that’s a really cool opportunity to re approach a lot of our old ideas about how, why, these things happen, and put us in touch with a really cool new data set about this behavior that I think can help us understand how and why people commit academic dishonesty.

Kelly: I do know of one graduating student that I’ve advised along the way, and he’s going into the cyber security world. So here’s a huge world. He’s going through lie detector tests and background checks and things like that. And one of the things that he thought of was, they ask like, “Have you cheated in your coursework?” And he had to think, “Okay, with CourseHeroe, is that considered cheating?” It’s certainly unethical. Well, I guess that’s a whole ‘nother podcast in of itself. But we think it’s unethical, it’d be interesting to get someone on the other side, arguing that it wouldn’t be. I don’t know, if I was going for any kind of security clearance or anything like that, I don’t want to be defending that choice to the investigator that I may or may not work for at some point in time.

John: I think one takeaway, though, from your discussion is that we can design our courses in such a way so that it’s not so easy just to copy and paste solutions into questions. And we can come up with better ways of assessing learning, which many people have been working online have been doing for many years now. But with the sudden transition of so many faculty online, who are trying to, as in the early days of online instruction, replicate what they were doing in the classroom in an online environment, it just doesn’t work in the same way. And we need to work with things that work better in this medium.

Kelly: Yeah.

Rebecca: This has been a really interesting conversation, really providing some new perspectives on things that I think will help faculty think through these things a little bit more. We usually wrap up by asking: “what’s next?”

Zachary: I think what’s next from a research perspective is Kelly and I are on track to track brand new course material from its online birth, so to speak, and to track it long term, kind of one course or one assignment. What does that course look like? What is its kind of online life look like… almost like tracking a single butterfly, like a monarch, as it goes from point A to point B. I’d like to use Course Villain to track one really specific thing and try and flesh out that narrative. So, just like I mentioned earlier, to get at that “What’s happening here beyond the obvious?” And I think that’s a cool way this tool facilitates that.

Kelly: I would like to also go into strengthening the student portion of this, try and to get them engaged and come at it from that point. I’d like to do some work in there and thinking about working with my colleagues. Then also I’ll work with the workflow. How can we put it in our workflow? We’re are all trying to be so efficient, so you can’t come up with a unique thing every time a course is run. So I’d like to see where in the workflow that will go, and get more students involved. There’s one group of students that we really gave free rein to, because we weren’t paying them and it was during COVID. Also, we just said, “Here’s some source code, here’s the problem, what would you do with it?” …and we kind of threw it at them. And they went on to “Oh, we’re going to have it learn and go…” And one student was really big into AI, so he was going to work with them. And so that was kind of fun. I’d like to reengage them and see what they would do with it. They’re the students. They’re the ones with the better ideas.

Zachary: And I’ll throw a shout out there to Embry-Riddle Prescott campus… that’s their cybersecurity club. We basically gave them a clone of the tool and we’re like, “Hey, play with it. See what you can develop.” They’ve been really hip on the idea of machine learning. And how can we get Course Villain to figure out its own prerogatives and do its own tracking work without so much supervision?

Kelly: They each had, like a specialized talent. And one, I really think it would work more efficiently if we hosted it using this language instead of that language. And I said, “Sure, go ahead”. He was a little taken aback like “You’re not bought into that.” And I was like, “No, you guys are the creators. Go for it.

Rebecca: So, it’s pretty exciting to see how it might have a life of its own.

Kelly: Yeah.

John: Especially if it becomes an AI bot, somehow. [LAUGHTER]

Zachary: Yeah, absolutely.

Zachary: I love your idea, John, of a Linux model. I’d like to see some version of this program or some version of a program like this. Getting back to that kind of David and Goliath narrative, if we can all chip in some small piece to a project like this, we could develop something that’s really cool, that I think could give these kind of sites and their prerogative a kind of a tough run for their money.

John: You could always create a GitHub site or a SourceForge site, and then people could work on it from there.

Zachary: That’s a good idea. Thank you.

Kelly: A GoFundMe page. [LAUGHTER]

Rebecca: You might get a lot of takers on that.

Zachary: I just hope we don’t find the code itself on CourseHero. [LAUGHTER]

Kelly: Yes.

Zachary: That would be very disheartening.

Kelly: Imagine that takedown request.

Zachary: That’s right.

John: Well, thank you. This has been fascinating, and I think this is an area that many people are concerned about, and this was very informative. Thank you.

Kelly: Oh, thank you.

Zachary: Rebecca, John, thank you so much.

Rebecca: Thank you. I hope it sparked a lot of ideas.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

111. The Business of Academic Dishonesty

Posted in academic integrity, cheating, grading, higher education, intellectual property, teaching
December 11, 2019

There are a number of websites that market themselves as study tools and tutoring services that are used by students as tools for cheating. In this episode, Dr. Liz Schmitt joins us to discuss how these sites work and the steps faculty can take to protect their intellectual property and the academic integrity of their courses. Liz is an economics professor and Acting Chair in the Department of Economics at SUNY Oswego.

Show Notes

Rebecca: There are a number of websites that market themselves as study tools and tutoring services that are used by students as tools for cheating. In this episode, we’ll discuss how these sites work and the steps faculty can take to protect their intellectual property and the academic integrity of their courses.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Liz Schmitt. Liz is an economics professor and Acting Department Chair in the Department of Economics at SUNY Oswego. Welcome, Liz.

Liz: Thank you.

Rebecca: Today’s teas are:

Liz: Cocoa and coconut.

Rebecca: Interesting.

Liz: It is. It’s my favorite. And I actually nagged John until he bought me more.

Rebecca: Excellent.

John: And I have ginger peach black tea.

Rebecca: I have Lady Grey today.

John: We’ve invited you here to talk a little bit about the growing problem that we’ve seen with the growth of online services that seemed to be designed primarily to facilitate academic dishonesty. I know you’ve had some issues with that in your courses recently. So, we thought this would be a good time to talk about that.

Liz: No, excellent. My post-traumatic stress hasn’t been maximized by the issue. So, let’s talk about it some more. But seriously, actually, it’s a really important issue. And I think our faculty are just not as well informed of this as they need to be. I think in the coming year, it’s going to become my mission to talk about this a lot more.

Rebecca: Can you talk a little bit about the incident that just recently happened to you to give some context and to help faculty understand how these kinds of things play out?

Liz: Certainly. So, really for years, even with the growth of the internet, if you’re using a publisher product… and it used to be sort of paper test banks and end-of-chapter questions… they show up on the internet. Some faculty just aren’t careful and were just posting them on non-password, open web pages. And then sites begin to grow where publisher content was widely published, by students often, with answers and answer keys to sort of pay it forward to future students who might be reusing it. I think what struck me this fall was the extent at which original content (because of this problem I had given up on publisher content)… but my original content in terms of my questions were being uploaded, essentially in real time. And within a few days, custom answers were being made by some of these sites. And this is really a new step in it, because it used to be the easiest way to get around this was to be writing your own content and guarding that content. And we find unfortunately, in September, that that was not enough.

John: The problem isn’t entirely new, though. For decades, maybe for centuries, fraternities and sororities and so forth, have kept files of old exams. But, the internet allows that to scale much more extensively and creates profit opportunities for people who facilitate those services. So could you tell us a little bit about some of the sites that do this?

Liz: Well, let me say this takes a step from the fraternity/sorority test file with this obvious broadening of the audience online, and no longer in geographic proximity. So, social affiliation and institution is no longer an indicator of what’s available to you. It’s searchable. So, it’s fraternity test file on steroids. It’s really one of the issues. But the other issue is these online proliferation of sites correspond to publishers emphasis on their digital content. As publishers really deal with essentially trying to get out of a used textbook market and kind of adapting for new ages of information, they really push their online content… which means faculty are relying more on that online content to operationalize many best practices such as retrieval practice for their students. So, it’s also more useful for students to have access to this. But, I think what really happened this fall is their ability to customize in short frames of time really eliminates the ability for me to just change up assignments in order to control the problem. So, really for low monthly fees, Chegg.com is what I ran into trouble with and for the low, low price of $15 a month they can upload questions. There’s a limit on how many questions, I think, they get a month but students potentially being strategic about it can upload questions and then get customized answers. Caveat emptor, some of the answers were great, some not so great and downright strange, which is really how I gleaned on to the problem to begin with. [LAUGHTER] And there are other sites. Chegg has some competitors I would say Chegg’s probably the market leader in this, but there are some competitors: Course Hero…

John: Course Hero has been out there for quite a while. My first exposure to this was about four or five years ago, when I had a student post in their online class that they were a member of Course Hero and they encouraged other people to join, mentioning that they could get their membership just by contributing a certain amount of graded work from the course to the site in return for that membership, and asking them to use her code so she could also get some credit for them joining. I had a little chat with that student about academic integrity and reminded her that if she posts anything on that site it would have to be taken down and she was going to be reported for this. This type of thing seems to be increasingly more common.

Liz: …much more common. And then you even have an issue like Quizlet. And one of the issues like Quizlet is students actually use it to create flashcards to study. So, I actually think there’s some honest intent for students that often come on to Quizlet. By making quizlets out of your questions and things you ask though, they actually create something for students that come after them. But, often Quizlet is really set up for students to actually create flashcards for different kinds of courses and study. But that’s really the leftover intent. And then, of course, there are online paper mills sites that actually offer existing papers… and again, narrowing topics can get around that and just not say “Write a paper on any broad topic…” that’s kind of really asking for trouble. But again, the customization that has come along makes that even more problematic as well.

Rebecca: And it’s somewhat affordable. It’s not incredibly expensive to be a member of these sites or to get a paper written for for you.

Liz: Yes, Chegg’s membership is $15 a month. So, that’s incredibly affordable. There have been sites… they come and go… kind of on a dark webish sketchy servers… but there are also sites offering to just take your online course. Those tend to be $1200 dollars, and up… Those sites get to be more pricier. But frankly, if you were a faculty member serving an institution with a more affluent student body, you should be very concerned about that.

John: Many of these sites also provide custom paper writing services. So, even if you’ve specified a narrow range of paper topics, students could still order a custom written paper.

Liz: Exactly. I would be fair to Chegg.com… that Chegg com responds within 24 hours in my experience to DMCA requests. So, they’re actually very responsive. And they take them down immediately. And they send warnings to the account that you posted things that you didn’t have permission to post. Chegg.com also cooperated with an honor code request, basically giving me upload dates and the emails of the members that uploaded content as well. So, they will cooperate, like I said, a pretty rapid fashion. I think it really, in the area of legality, it’s not clear if you correct yourself but become this conduit for things to happen, then, I don’t know what kind of legal responsibility is going to come there. It’s sort of beyond my expertise. But, I suspect it would take big players it takes like publisher lawyers and things like that, to really come up against this and demand that you really set up a situation that facilitates the repeated stealing of our content. Because it can be an intellectual property whack-a-mole game, because they take it down, but then another student post It from entirely different account and Chegg and these sites are kind of saying they’re not responsible for that. Legally, I’m not sure if you facilitate that, how responsible you are. There’s also paraphrasing tools. Paraphrasing tools… that’s very caveat emptor or buyer beware type of situation… and I actually have an example to show you what can happen when plagiarism tools go bad. Friends don’t let friends use plagiarism tools or this is going to happen. So, let me show you the answer from Chegg expert which was a very nice answer… is as follows: “Expectation of rise in inflation will lead to a drop in the prices of Treasury bonds. This is because the required yield by investors will increase in order to generate higher returns after beating the inflationary pressure.” Great answer, run it through a paraphraser and this is what you’re going to get: “Desire for ascend in the expansion will prompt drop in the cost of the Treasury bond. This is on the grounds that the required yield by the financial specialist will increment so as to produce higher returns subsequent to beating the inflationary wait.” [LAUGHTER] So, that’s not a thing… that is not a thing. I think the odd choice of vocabulary… Here’s words that college students almost never used. And here’s 10 of them right now, in the same paragraph. Obviously, that’s the warning sign. And the idea is, of course, context is important. Certain words that are used by convention or tradition in financial markets. Something that’s really a synonym, technically, in the English language is not in the context of financial markets.

John: A traditional way of catching students who were doing this often was that they were providing you with something that seemed a lot stronger than their other work.

Liz: Yes.

John: And this sort of reduces that down to where, perhaps, it may not be as obvious in all cases that someone is doing that because the quality of the work will no longer look exceptional in the same way.

Liz: Yeah, the exceptional quality of work, you’re right… because the first answer actually does set up my radar because it uses a complexity in sentence structure that I might not expect in this answer. But, I think the other one was just immediately obvious because the word choice just doesn’t fit. So, even if you were actually looking around on Investopedia, or some other website to borrow language, it’s not language that would be used by someone writing for that.

Rebecca: Even as a non-expert in the field, I could tell you that that is not language of the field.

Liz: “Desire for ascend in the expansion?” Yeah.

Rebecca: No, no.

Liz: I think pretty much called that one out.

John: But if you have some foreign students in your class, that sort of thing might sometimes happen. I’ve certainly seen examples where students have used synonyms inappropriately. So, some of this may get by.

Rebecca: In some cases, though, if it was a student where English wasn’t the first language, you would probably have some assignments and know already the kinds of mistakes that that student would make.

John: Yes.

Rebecca: So, you would have an idea of whether or not that would be consistent for that student or not.

John: We touched on the issue of hiring other people to do a student’s work, but could you talk a bit more about how the gig economy may play a role in academic integrity issues?

Liz: Well, the gig economy is basically the idea of you give me your specialized topic and I will write a paper custom for you: words, APA or MLA, the formatting etc., ready to go… Course completion, I will take your online course for you… paper writing services, again, is sort of that issue. And again, in ome of these paper writing services, it’s sent as a Word document, and they don’t actually strip the properties of who created that document.

John: And one of the things that has helped facilitate this, though, is the international reach of the internet. In countries such as India, where incomes and wages are a lot lower, you can pay people relatively small amounts to get reasonably high quality papers written in very strong, solid English

Liz: Right, exactly.

Rebecca: So, what can faculty do? This sounds very dismal.

Liz: It does sound dismal and there’s no solution that doesn’t involve some time on task here. And I think that’s a big problem. Because most of my faculty colleagues, I don’t really know anyone who sit around doing nothing… who has extra time to deal with this, and that’s got to give. Scaffolding is a common way to deal with this. So, it’s a recipe for disaster to assign a paper, be very unspecific about the topic: “Do any topic in post-World War II US history out…” That’s going to be bad. You want to focus the topic, then you actually scaffold. When do you want a thesis statement? When do you want an outline? an annotated bibliography? So, trying to have the pieces turned in really prevents their ability from coming out to get a paper or it’s really the red flag when a student several weeks into the process suddenly wants to change topic, having had struggled earlier with the topics and then they tell you when they want to change, you need to get ready for what that paper is going to look like. And you can scaffold in other ways, even on homeworks: you reference. I reference “Using the model from chapter seven,” “using the model we talked about in class in week six” or “using the discussion issues that were brought up in discussion three.” And so you can’t easily post those and get answers because then the students would have to provide the context, which defeats the purpose as well, of the questions. Algorithmic questions, and algorithmic questions can be posted and solved, but algorithmic questions are really after the fact, it makes it a lot easier to snag people. Because, if you just have a general problem…. So, I had a problem on the internal rate of return, which is a common time value of money concept in the field of finance. And while I think students really got their answers from Chegg, unless they were kind of lazy enough to exactly copy the wording of the answers, I couldn’t really get them because there really was a way that you solved this in the spreadsheet. So, the solution should look the same. So, you get around that by algorithmic questions where the numbers are unique, and then you know, exactly who uploaded that question. And in a learning management system, it’s easy enough to regenerate numbers every semester or year that you’re teaching the course. So, there’s a new set of unique numbers. So, that’s more about enforcement, I think. Questions that work from very specific data, maybe a data set, or quotations. So you can start with: “Mitt Romney, in a debate in 2012, called China a currency manipulator. What does that mean?” And so that quotation again, makes it harder to find general answers to that question. Questions that reference their own experiences, where they have to call up a specific experience themselves, and expound on it, and apply it in class. And current events. So, if you do actually have a very current event, then you can actually prevent going back in time and trying to find older questions, because it’s a current event as well. And so you’re constantly changing. So, you might be testing on the same topic, but you’re constantly changing the context. And your question has to be written so that you forced them to address the context as well.

John: I’ll often ask students to find an example of something in the last six months that illustrates some concept that we’ve just discussed in class. Because if I only offer the class once a year, they’re not going to be able to go back and find earlier examples from the class.

Liz: Right. And then I guess my least compromised questions are ones where they actually create a graph using the Federal Reserve Economic Database based on macro data or financial data. And again, that’s right, because it has to be current. And from year to year you change the time period, or what you want them to look at, or different measures of inflation, things like that. And then you can really grab that, and I have yet to see a FRED question appear on Chegg.

John:(…and FRED is an acronym for Federal Reserve Economic Data.).

Rebecca: What is your process for monitoring because clearly, you’re doing some monitoring.

Liz: Right. Well, when I found the extent of this problem in September, I basically went forward several weeks, and just copied and pasted questions into the internet search engines. I also ended up at least getting a subscription to Chegg so I could look, because the other issue is if someone screenshots an image of a question from their learning management system that might not show up in a Google search, but it’s going to show up on Chegg because Chegg actually has the text… kind of an alt text… that comes up, and that will show up as well. And so I started doing that. I opened a Google doc. And then every time I found one, I knew I had to change the questions in my graded activities. And then I would put it in the Google document. And I ended up with a 20-page DMCA document to Chegg about all of the questions I wanted to take down that take me through about week 10 my course. So… to be continued… as to how many I will find. But really, the monitoring really comes up at first when you get just a very unusual answer, or you start getting all the same answers. So, in a class of about 37, there’s about 10 students that were trying to tell me the same thing, especially if they illustrate a concept… I don’t ask for a numerical example… But they give me one. When you see the first person do that, you’re like “That’s great.” And then you think, “Well, why would the second, third and fourth person choose the exact same numbers to illustrate this concept?” There’s only one reason and it’s not a good one.

John: You mentioned the paraphrasing tools making it a little harder to find when people were trying to copy materials. There’s other tools out there too, though, which are online plagiarism detection sites where students can get a paper that they think fits the requirements of the assignment, upload it to that, see if it’s found, and if it is, just make minor changes and resubmit it until it ends up with a relatively low plagiarism score. And complicating this a bit, is that some of the major paper mills advertise that they use a TurnItIn service to check the papers that they sell for plagiarized content. Because one of the issues for the companies that are selling papers to students is that sometimes the people they hire to write papers simply plagiarize existing material. So, this makes it a little bit harder perhaps to detect that sort of work for hire. What might you do to detect papers that have already been run through the TurnItIn or similar systems and modified to come up with low plagiarism scores?

Liz: Well, I think you have to go backwards from that and recognize the limitations of those tools. I was never a big fan of TurnItIn. SafeAssign is just kind of a starting point. SafeAssign often gives a lot of false positives as well. Because if people are citing sometimes similar sources, which you would expect, then you’re going to see that pop up in SafeAssign. So, I think you got to go back and say, are you choosing the topic that is sort of paper mill proof, or things like that? Downloading when you have electronic submissions, which I do, and downloading them can also pick up issues of extra characters designed to trip it up. So, people put characters and they change the color to white to try and trip up that detector and that easily shows up when you highlight them. Those are some of the things that I would use. wWe haven’t had TurnItIn on our campus. We had SafeAssign, which I already knew the limitations, but the tell is you read it, and you’re like I’ve met five college students that write like that… in 23 years. So, I’m now deeply suspicious of that. Often the language doesn’t match, which is why scaffolding also becomes important… because the language doesn’t match what’s been done earlier in terms of: if you don’t think the student wrote the paper and you can’t find the source, you invite them in. And you ask them about where they found the source. Or I really love what you said about the impact of inflation on average households in the 1970s. Could you explain how you developed that? …and they won’t be able to and as a result, they’ll often say, “Well, I wrote this paper and I immediately forgot everything.” And faculty members get caught up into this. “Well, I have to prove a negative or I have to prove that that’s not possible.” And you don’t, I think you’d say: “On your face, that’s a really ridiculous argument. Either you need an MRI right now, because something wrong has happened. You’ve had a stroke and you need help, or you’re not being honest about where that paper came from. And you’re allowed a chance to defend yourself from these charges. But that defense isn’t reasonable. And I’m going to move it on.”

John: Pretty much every time I’ve run across that and brought the students in, they almost always confessed pretty quickly when they realize that they can’t explain what they had written.

Liz: Yeah, I think turn it in has these separate issues too. Is it ethical? We’re complaining about our intellectual property being posted. But, students that do write original papers, that’s their intellectual property. And we’re forcing them by virtue of getting their degree and meeting requirements of a course, to release some of that somewhat. So, I think there are issues of privacy and property, ethical issues that I know a lot of other faculty out there have just really pushed back against for years with turn it in. Others argue it’s about us policing students by requiring this prevention method, we’re almost taking the assumption that something’s going to go wrong, that it can create a hostility and an adversarial relationship that we don’t want in students. I think there’s some truth to that as well, which is why it becomes so important to think about the design of your course and your learning activities, because it’s so much better to prevent this and to make it very difficult than to deal with it afterwards. As I tell my students in my annual first day of class, don’t test me on this because I fail people at least once a year for this kind of thing. I actually say this is the worst part of my job, and I hate it, but I do it. So, that should tell you how important it is to me.

Rebecca: I think that it’s sometimes it’s a little more obvious to faculty how to prevent some of these things from happening in an face-to-face class, because you can be doing in class assignments, you can have them working on things in class and see their progress. But that doesn’t play out maybe as easily online because you don’t necessarily know what they’re doing. So, do you have some strategies of how you scaffold or do things a little bit differently online?

Liz: I would back up again, because some people say, “Well, this is why I don’t teach online, because I’m worried about ringers and things like that.” But, often people that tell me that do assign papers, and I say “Basically, anything that’s done outside of class is susceptible to these sites…anything.” And so really, again, there’s a lot of face-to-face classes, particularly in STEM that are using homework packages that students complete on their own. And then there are plenty of classes that assign papers that are done outside of class. The idea is, is that the in-class possibility of a face-t- face class does provide this check. But in an online class, you really compel them to actually talk about things in a discussion. And so you can use your online discussions to lay the groundwork of what they want to talk about, requiring them to reference discussions within the course as well. And so that’s one way to try and mimic that check that in-class does. And there’s proctoring software, which again, is potentially foolable as well, but there’s also design. So in an online class, the idea is no one thing should be worth a lot. Any one thing should be worth less than 20% in my opinion, because the idea here is if you want a ringer, you don’t want that ringer, by just taking a single exam, to move your letter grade significantly. You’d need a ringer for the whole course or a really, really good friend to do this for you. So, that’s another thing and in designing them exams, you can time them, not necessarily just time them but there’s one question at a time, no backtracking options. So, you want to think about a very structured way to require students to demonstrate understanding that just make it a lot more difficult to outsource.

22:08 redo

John: You can do the same type of scaffolding online as you do in a face-to-face class. You just have different stages, as you said. In the first stage, you could have them submit a thesis statement. In the next stage, you could ask the students to submit a bibliography, followed by an annotated bibliography, and then a rough draft and a final draft of the project . And that’s not really much different than it would be in a face-to-face class.

Liz: And all in the same place are all of these writing samples because that’s how you’re communicating with students: via emails, course messaging, discussion forums, and then other graded work. So, in some sense, there’s a large body of written work to form a basis for your suspicions or concerns.

Rebecca: I’ll also add that we focused on lot on written work, test questions, and things like this. But, the same kind of plagiarism can happen with images, it can happen with code. I’ve had those same experiences in my classes as well…. sharing of digital files to make a particular design and boy does that look kind of similar to something I’ve seen before or an image that’s being claimed as their own or not documented where it came from. So those same things happen. They don’t play out in the same way in Chegg and some of these other sites, but those same practices happen through like a gig economy or just sharing amongst other students and when their digital files are a lot easier to share them when they were physical things.

Liz: Also in creative fields, there’s something worse in the way that’s sort of accepted, because if you look at fashion design, the ripping off of top designer early designs to then the knockoffs is astonishing. There’s a photo that has a Manolo Blahnik sandal, frankly next to an Ivanka Trump sandal, and they’re the identical red sandal. And it’s really just you slapped a different name on it and use different materials, perhaps. And obviously, they’re about six months apart in terms of product cycle. So, I think people in creative fields see that recycling, even in the music field, whether they get permission, you see music, that’s a remix of other older musicians, and you need permission for that, but you don’t necessarily know that when you’re listening to it and enjoying it. So, I think you actually see in some of the real-world situations how visual borrowing and frankly, stealing, is kind of accepted in some of these fields by some very successful people. Whereas, in the written world, when authors get found with plagiarism, it’s considered a big deal. And it’s kind of very embarrassing for them to get caught. Books can be recalled… things like that. It seems like written word… there seems to be more of a consequence, for now, when that happens, whereas in creative fields, that’s not always true.

Rebecca: There’s a lot of visual and audio copyright cases, though. And that’s where that tends to play out. The fashion example that you’re giving… those are considered functional things and not considered creative. So therefore, they’re not protected by the law in the same way.

Liz: Exactly. But, I think for what students observe, and behavior that they might emulate, when they look at professionals and their choices, that distinction isn’t going to come up.

Rebecca: One thing that I always argue is that when we talk about intellectual integrity, that copyright comes into the discussion in particular fields, because it is sometimes a common practice. And it’s there for a reason. we iterate on our culture, our culture creates new culture. It’s not a crazy concept, but you need to know where it comes from. And you need to provide attribution and in an appropriate way, which is no different than following MLA or APA or some other thing. So, I think that’s always something that people should be thinking of, and that when you’re having written papers, and there’s images and things in it, that you’re also thinking about that part of the content as well and not just the written word.

John: One other thing that I’d like to bring up is an earlier podcast that we did with Judith Boettcher, who talked about one way of avoiding this issue by having group projects that provide students with a lot more autonomy, but in a very structured fashion. And that’s perhaps a way of getting around this where students can take more ownership of the project and they create more of the project as a group, which would make it a little harder to engage in academic dishonesty. And we’ve also, in previous podcasts, talked about some open pedagogy projects, where the work that students do is posted publicly. And if they know that when their work is posted publicly, and they’re copying something from anyone else, it’s much more likely that they would be found and they’d get in some trouble later. So, those are two other things, perhaps, that might be ways of reducing the incentives for academic dishonesty.

Rebecca: Community based learning is another one… or service learning where you’re doing projects with the local community, because all of that context would be unique every time you’re doing something. So, that’s another opportunity for grading those assignments that really aren’t reproducible, and would be really, really hard to get an answer for… unless the person lived in the same community.

Liz: I agree. I think the biggest challenge is in some of these courses that are tool courses. In some courses you’re acquiring the tools that you would learn for projects and to consult, etc. And so when you teach these tool courses, it’s not always appropriate to have these kind of finished product things, because they’re in progress of assembling that toolbox that they’re going to use. And this is where reliance on sites like Chegg become a big problem.

Rebecca: At the beginning, we started with the idea that faculty aren’t always aware of these tools, or even the ways that students try to manipulate the system. Can you talk about ways that we can help increase this awareness with our colleagues?

Liz: I was just talking about this this morning with another colleague and we were bemoaning our Chegg purgatory this semester, and she says, “I just don’t think other faculty realizes… or how can we be the only ones that care about this.” So, I think some of them are honestly unaware because faculty aren’t always in sort of the student space and understand what those crazy kids are doing these days. So, I think, in some sense, faculty that have been more tuned in to creating learning community or kind of developing a relationship with their students are more likely to get ideas of what’s going on as well. But I also think there’s some willful ignorance here… this whole “Well, I didn’t know.” But I took steps to make sure I didn’t know because once you do know, it obligates you to do something, if you “don’t know,” and I’m using quotes, which is not helpful on a podcast, air quotes, but if you really don’t know that you’re MyLab component of the course is completely compromised, that doesn’t obligate you to think about changing up your course, and the weighting of activities and what activities. Once you know, it really obligates you to act as a faculty member. And so some people say, “Well, I wonder if my stuff is up there?” and like, “Oh, it is.” I don’t even have to look, I can say “Yes, it is.” And then academic dishonesty takes time. I’m really at 40 hours and counting with academic dishonesty documentation, DMCA documentation, and then reworking items in the course to deal with this issue so it wasn’t a lost semester. And I’m a full professor, think about an assistant professor not only trying to balance and develop their research agenda in conjunction with this, but also not wanting to rock the boat with unhappy students. I’m going to be getting a nice bottle of something sparkly, when I read my faculty evaluations in January. They’re going to be lit. Perhaps we’ll do a dramatic reading and have a tea party with tea and maybe something stronger. But, you know what? I can weather that. I can take that. And that’s going to be an issue. And frankly, I’m really proud of this institution, about how administrators really back faculty, enforcing the integrity of coursework and the degrees. And I know that doesn’t happen at other institutions, frankly other institutions that are maybe more tuition dependent and driven… that are unwilling to make steps that make students leave with financial implications. So, in some sense, this is really one of the best environments here at Oswego to actually try to enforce these policies as well. So, I think that’s one of the reasons that faculty don’t act, because there absolutely is some blowback to that.

John: And faculty might also see that each year they teach the same course, their students are doing better and they might be very content just to see that improvement in the scores so that all the work being submitted looks more like the best work from the year before. And it’s really easy to passively accept that.

Liz: Well, I would actually note that what’s more astonishing, is not the extent that Chegg has corrupted these issues, but there is a significant contingent of students that do not use it. Maybe they are unaware of it. Maybe they decide they don’t want to use it. But, when someone says, “Well, I can’t believe how many students cheat,” I would say, “Well, I can’t believe how many don’t, given the incentive structure.” And so that’s somewhat encouraging. But, also I had to decide if I was going to email the entire class and say, “Look, I’ve seen this happen. And you have to know that this is not allowed, that I’ve already made it clear in the syllabus. This is not allowed, and this is what will happen.” And it was a real-life decision about the Streisand effect, because you have to wonder if 20% of the class is saying “Wait, there’s a Chegg and we can get answers there? Alright!” You didn’t know if you wanted to clue them in, but I figured I had to act under full information.

Rebecca: I think it’s also really interesting when you start seeing students wanting to apply for jobs at those kinds of organizations, and then what kinds of conversations you’re going to have with those students, when they say, “Hey, I found this really awesome job that I want to apply for at Chegg.” It’s like, “Well, think about that. What does that mean? And what are those implications? And where are the ethics behind that?”

Liz: Well John, and I were talking earlier, there are Faculty Fellows at Course Hero, and some of them have made a name about teaching in the profession. John was probably more polite about that. One of them’s at a conference and I think I would stand up and say, “How do you reconcile partnering with a site that facilitates academic dishonesty and intellectual property theft every single day?” I would be curious as to what the answer is.

Rebecca: The pay is really, really good? [LAUGHTER] I don’t know.

Liz: I guess. The one I’m thinking of has a pretty sweet gig right now, so I’m not really sure.

John: this presentation was right after one that was riddled with references to learning styles, so I saved my powder for that one.

Rebecca: So we always wrap up by asking what’s next?

Liz: First, I’m already kind of brought this up in faculty assembly. And I’ve brought it up in academic meetings, because I’m acting chair this year. So I brought it up in leadership meetings. And I hope we can actually do a workshop through our Center for Excellence in Learning and Teaching. So, maybe a breakout workshop talking about our experience, and try and really broaden the understanding of the issues here. I’m also working with our faculty union, the United University Professors is also very interested in compiling violations of intellectual property rights, and trying to deal with that and push back against those sites. And I’m actually sharing my DMCA documents. So I made an editable form for some of the big sites, so you can easily go in and change them. I made a template for what you would ask the Associate Dean to fill out in order to ask for upload information from these sites as well. So trying to minimize the work involved for people who want to do this and take action. And then, finally, I’m just looking ahead about how I’m going to really redesign this course that I’ll teach a year from now, and to motivate and enforce original work.

John: And I should note, we’ve also been offering workshops for at least seven or eight years now.

Liz: You’ve been offering workshops on these sites and things like that….

John: …and attendance has been generally limited. We’re lucky if we get 15 people.

Liz: But you don’t have me. I’m a draw. I’m a draw… a star. [LAUGHTER]

Rebecca: …especially if we could do the dramatic reading of the examples.

Liz: Exactly. There will be a dramatic reading, and there will be sufficient supplies of snark. And I think it’ll work. But, I actually think maybe a case study. So, we come and we talk about these sites, but I can sit down with other faculty who’ve had this problem. And this is what I found… this is how I found out about it. And this is what I did about it, and this is what you should be doing about it.

Rebecca: This has been super informative. Thank you very much, Liz.

John: Thank you, Liz.

Liz: Well, thank you for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Brittany Jones and Kiara Montero.