457. Intellectual Property

Faculty create large quantities of content, but aren’t always aware of their intellectual property rights, or lack thereof. In this episode, Kent Kauffman joins us to discuss legal issues associated with copyright law. Kent is a Professor of Business Law and MBA Programs Faculty Liaison in the Doermer School of Business at Purdue University Fort Wayne. He is also the author of Navigating Choppy Waters: Key Legal Issues College Faculty Need to Know.

Show Notes

Transcript

Rebecca: Faculty create large quantities of content, but aren’t always aware of their intellectual property rights, or lack thereof. In this episode, we explore legal issues associated with copyright law.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Kent Kauffman, a Professor of Business Law and MBA Programs Faculty Liaison in the Doermer School of Business at Purdue University Fort Wayne. He is also the author of Navigating Choppy Waters: Key Legal Issues College Faculty Need to Know. Welcome back, Kent.

Kent: Thank you very much for having me. I’m always glad to talk with the two of you, and I appreciate the invitation again to be on your awesome podcast.

Rebecca: Today’s teas are:… Kent, are you drinking tea with us today?

Kent: I am drinking tea with you today. Due to the heat here in Indiana, I don’t know if I’m pronouncing it correctly, but I’ve been drinking the Luzianne iced tea, where the tea bags are turned into an iced tea. So that’s my tea of choice as of late.

Rebecca: Something nice and cool seems like a good option in the heat.

Kent: Absolutely.

John: It’s a hot summer pretty much everywhere. And I am drinking a blueberry green tea.

Rebecca: Green must be the theme in here because I also have green tea today, John. I have a jasmine green.

John: Very good.

Rebecca: Looking for something lighter for the summer. [LAUGHTER]

John: So we’ve invited you here today to discuss intellectual property issues for faculty. One issue that comes up fairly often for faculty is the question of ownership of course materials that are created by faculty. Who owns the copyright to course materials that is stored in a learning management system.

Kent: Well, I think that’s a really needful and urgent question, and to me, it sort of parallels the urgency of academic freedom issues that faculty face right now. Because both of those issues involve finding out well, what what are my rights, whether it’s my right to teach and research the way my discipline respects, or my rights with respect to the work products I create that are attendant to, in this case, my teaching and also my scholarship as well. So if I can just do a quick tutorial, and I’ll try not to take too much time on this. I think of intellectual property like three siblings and a cousin that’s like invited to Thanksgiving dinner, but isn’t really a part of the family, and the siblings are trademarks and patents and copyrights, and that distant cousin is the trade secret. And the difference I see is this: intellectual property traditionally gets its value from being publicly displayed. That’s the point. What is the value of a song if no one’s ever heard you play it or sing it? What’s the value of a script if no one’s ever said it somewhere? And so intellectual property law, federal law, by statute, protects those particular aspects of our creative property. Trade secret law actually is protected by you, and so there’s a completely different way that we get value out of hiding something that we think is valuable, and that’s trade secret law. For us and copyright, for which faculty more than any other intellectual property, notwithstanding some faculty might deal in patent creation, but that’s a whole ‘nother discussion. But as it concerns copyrights, I think the first question to ask is, “What are we creating that is actually worthy of copyright?” And that’s not to be insulting. That’s just to say, by way of background, copyright does not protect our ideas. Don’t tell someone your great idea. Keep it to yourself. Copyright law protects the expression of your ideas. What the statute calls when your ideas are “put into a tangible medium of expression,” or what I might say, a fixed medium of expression. Your podcast is an example. By talking into this recording device, there’s copyright created automatically because this is a recording of the expression of our ideas. So once we understand that copyright protects our work automatically, we don’t need to seek a copyright registration to get that protection. The beautiful thing about it is it protects it automatically once it’s put into a fixed or tangible medium of expression, but the next question is, and this is the perhaps insulting part, is “What are we doing that’s worthy of copyright protection?” Because that requires creativity, and there is a Supreme Court case involving the phone book to which many younger listeners might say, “What’s that?” [LAUGHTER] But there was a lawsuit over the phone book because a competing company was using the white pages out of a phone book, and that went to the Supreme Court of the United States, and they had to answer the question, “What here is worthy of copyright?” And just to cut to the chase, the yellow pages are protected by copyright, but the white pages are not because there’s no creativity in assembling people’s names and addresses. And one feature the court mentioned is that copyright requires that your work have a spark, that was their word or a modicum, their word, of creativity. So if we think about that, the question then becomes: “Am I doing something that is put into a tangible medium of expression that is worthy of copyright that is instructional materials?” The foundational document probably is the syllabus. I think of the syllabus as the starter yeast of academia because it began somewhere else and it keeps getting handed over and over again as new things are added to it. And so, where I work, my syllabi is longer than it’s ever been – not by design, but because new parts must be added to it. Well, I don’t own a copyright in the works of others, so we jump out of that for a second. If I’m creating PowerPoints or tests or any other materials that come from the publisher of the textbook I might use, then I have no copyright in those materials whatsoever. So then we say, okay, is my syllabus copyrightable if I have enough of my own original content in it. That’s a worthy question.

If you think that you have enough originality, creativity, that your compilation of facts, which can qualify for copyright, are original enough, then you might be able to say it belongs to me. There can be a problem with that because one might argue that the syllabus is the department’s document as opposed to your document, since even if you’ve created the class, it’s still under the aegis of your department and your university as well. But in terms of these instructional materials, whether it’s a syllabus or PowerPoints, tests, really interesting essay assignments or projects that students do, I would argue that a lot of those can be worthy of copyright protection because they are first put in a fixed medium of expression because they’re written down digitally or on paper, and I would say that really, for faculty that care about what they’re doing and trying to make a vibrant life out of their classes, absolutely they can be protected by copyright because they meet those modicum standards, those spark standards of creativity. So yes, I would say this, from an initial standpoint, if you believe that your work meets that expectation of creativity, then you could make the argument that your instructional materials could be copyrightable and belong to you, but there’s this however, and the however might lead to the next topic, which is instructional or institutional policy.

Rebecca: I can imagine that if folks were looking at copyright law at face value, they might assume that this would fall under works for hire because it’s part of your job, maybe, to create instructional materials. So, can you talk a little bit more about where this role of institutional policy kind of dovetails with the law here in determining who has the copyright to these materials?

Kent: Yeah, there’s two issues there to think about, and one is policy, and the other is works for hire. And if I can just swing around to the policy issue, and then get to works for hire next. The beauty from an employment standpoint is that universities, and I’m saying this not as an employer, but from their standpoint, the beauty is that they can create institutional policy that also covers intellectual property. And one way that a lot of universities cover our instructional materials is that they will define categories of work we create at a university, and those are usually three types: traditionally administrative, and then scholarly, and then instructional. Now, someone like me who’s really a big believer of scholarly teaching… One could argue that teaching materials or instructional materials could fall into the category of being scholarly, but that’s again another discussion. As it concerns administrative materials. If you at your university are doing a teaching evaluation of a tenure-track faculty member, no matter how well you write that letter, it’s likely not to belong to you because universities categorize those documents as simply being university-owned property. As it concerns scholarly works, peer-reviewed papers, poster presentations, books we write. What I find interesting there is that: 1. universities tend to exempt themselves from ownership of those to start with, which I think is almost a moot point because I have to transfer copyright to those works anyways to the publisher generally to get them published. And there is open-access publications, I realize that, but still we live in a world where lots of our publications are conditioned on our releasing our copyright. So my university can’t claim ownership to my books, for instance, because I don’t have any ownership [LAUGHTER] to my books as well. So then the next thing is the instructional materials, and here we get a situation where universities do something that I think I’m not comfortable with it. But what happens is they often will say you own your instructional materials, but you have granted to us, by condition of employment, a non-exclusive license to use your materials, and the key there is, if you look at your institutional policy, if it has that, is it will often say non-exclusive because an exclusive license, similar to a transfer of copyright, requires both signatures. I must sign, and so must the proper agent of the institution sign as well. But a non-exclusive license doesn’t need signatures. And so, for instance, Purdue’s policy, the university-wide policy, not just where I work at Fort Wayne, but across all of Purdue institutions, is a non-exclusive, royalty-free, perpetual license. To me, that means that as it concerns my instructional materials that are in… and Purdue uses what’s called Brightspace as its LMS… all of that is, in my view, effectively co-owned by Purdue, not owned because they granted me the right to keep my copyright, which is an interesting language phenomena to say we allow you to keep something that you already own as the author of the copyrighted material, but a non-exclusive, perpetual, royalty-free license does mean that they can use my materials, everything in Brightspace or anyone’s LMS for as long as they want, no matter if I work there or not. Some institutions limit that, they might provide a royalty structure if they’re using your materials in a class you don’t teach. Other institutions might say we will cease using your materials if and when you cease working for us. And those are nice, gracious ways of sort of managing that. But to faculty that put their instructional materials on an LMS, as almost all of us do nowadays, I would say yes, you still would own what is worthy of being owned for copyright purposes, but you sort of now have a co-owner, effectively by your employer, who has taken a non-exclusive or maybe even an exclusive license in them. But the work for hire situation is a different angle to this. There’s two ways to create a work for hire, which would mean that the author is not even the copyright holder. They’ve transferred the copyright altogether. The first is the easy one, and that’s where I transfer my work as an independent contractor to the hiring party, and that requires two signatures there.

A couple years ago, I wrote scripts for some educational cartoons for a business law course that was online, and I created the subjects for what they were looking for. They approved them. Then I gave them ideas, and they approved them. Then I wrote scripts for educational cartoons, and they approved those as well. Well, none of those I own because that contract, being a work-for-hire independent contract, meant that they own all my work. The other way is what is known as what creative works that we produce in the scope or in the course of our employment, and that’s another part of the copyright code. And the scope of employment doctrine is an agency law doctrine that says that if I’m hired to do certain types of things that might involve creating intellectual property, my employer gets them automatically, not by needing a signature because I’m creating them in the scope or course of my employment. For faculty, an outsider might say, “Well, everything you do is in the scope of employment, and therefore, wouldn’t your university own them all? Well, yes and no. And to the no part, I’m not a greeting card writer at Hallmark, so I can’t disclaim some of my really pithy statements and say, “Well, I own this one because I’m hired to do that.” But as agency law looks at what faculty do, as it concerns instructional materials, it’s a confusing world. Is that in the scope of my employment? There’s no consensus on that. Some courts will say it is in the scope of your employment, particularly as it concerns curricular materials. There’s a case, I find it kind of sad, where a woman… she was a department chair at a two-year college tasked with creating a whole new program at her college within her discipline… and eventually she was let go and she brought an age discrimination claim and along the way she filed copyright registrations for the curricular documents that she created. One doesn’t have to do that to own them, but one needs to do that if they’re going to sue someone for infringing their copyright. Well, she lost on both counts and, as concerned the copyright issue, the court said part of her job as a department chair included doing things like creating new curricular materials, and those are clearly the work of her employer. But for regular faculty who are creating courses, not new courses by way of a grant, so if you’re hired to create a new course by way of a grant, that’s a work for hire in the first category. You’re going to sign a document and give up your rights to that material. But for the work we create that’s regular instructional work, there isn’t consensus, in my opinion, legally on whether that really is work for hire, and therefore we don’t own it, anyways. But if there’s an institutional policy that actually carves out the whole work-for-hire concept. So for instance, I know that my instruction materials are not work for hire at Purdue because the institutional intellectual property policy already says I own them and grant them a license. Well, therefore, it’s not work for hire because they said I own them; in a work-for-hire situation, I wouldn’t own them to start with.

John: I know in the SUNY system, any work that’s created for online classes, as part of an agreement with the union, copyright is maintained by the faculty members, however, at least a number of institutions have worked with adjuncts to develop courses as works for hire as a way around that. So that we have a mix of both of those in SUNY. I think that Oswego, we allow the creators to own the copyright unless there’s a work-for-hire arrangement that overrides those conditions. So most of the issues that I’ve seen have been in terms of online content that’s created. But what about lectures that are delivered in person? Do you own the rights to those?

Kent: There’s the question. I enjoy face-to-face classes. I certainly am happy to have online classes, and I take them very seriously. I want to make them vibrant, but I still really enjoy being in a room with people and having a dialog. Here’s an interesting case. So there was a lady who called herself the Banana Lady. You can look her up online. She dresses in a banana costume, and she does like corporate events. But what she really does is she tends to sue people who take pictures of her at her events and claim that these are copyright infringements. And in the federal court system here in Indiana, the Seventh Circuit Court of Appeals not only ruled against her, but one thing that they said was she was trying to sort of gin up litigation. But they said that people taking photos of her aren’t infringing on her copyright because she wasn’t recording herself doing her banana woman sort of dance routine stuff. So that’s the starting point. If you’re not recording yourself in class, then you’re not creating a tangible medium expression of your teaching, no matter how creative it might be. So, A) are you recording it? B) is who’s recording it, because if your university is recording it either automatically or that you use university technology to record it, I would regrettably argue that that’s not your property. It’s probably your employer’s property if you’re using its technology as the conduit for creating a tangible medium of expression. If you’re recording it yourself and your university is recording it, I think you could claim you have ownership over what you yourself are recording. But if you’re not recording it, there’s no copyright, and that goes to everything. If you put notes on a blackboard, you’re putting it in a tangible medium of expression. But if you’re erasing it, it’s gone anyways. And so the question becomes: “Well, then, what do students own if they’re taking notes from what I’m doing in class?” If they’re taking notes from what you’re saying and you’re not recording it, they have a valid claim that they own their notes. Again, something can be not worthy of copyright, but still be your property. Copyright is a type of property, so I could own something that’s not worthy of copyright. That’s what trade secrets are all about. So that a business who has a client list they believe is a trade secret property that’s not copyrightable, but it’s still their property. So I’m not saying a student owns the copyright in their class notes because that’s a copyright creativity question. But they could argue they own those notes if they’re taking notes off of what I’m saying. If they’re taking notes off of my powerpoints, and my powerpoints belong to me. I would argue that they’re actually just simply transcribing what is already mine, and they don’t own that. There are some cases, very few, but they’re out there, where faculty have sued publishing companies because in some really large universities, publishing companies might sell you notes for the classes that you’re taking that are highly common large enrollment classes, and faculty have sued saying that’s actually my property, but students are selling those notes to publishing companies, who then create study guides out of them. That’s a really thorny question because if they’re simply transcribing what is your property, you’re not transferring your property to them. So that would be a copyright infringement. One area is what’s called transformative use, which is a part of what’s called the fair-use doctrine, which allows a person to raise a defense to copyright infringement by arguing a statute in the code called the fair-use doctrine. And fair use is about limited usage of some of those copyrighted work for reasons that not are allowable, but are part of the variables for determining if you’re committing a non-violation of a use of some else’s material. Transformative use says if I use your material in a way that’s completely different than how you used it, I have transformed your purpose into a new purpose.

So you could perhaps find a situation where a very creative student is creating a graphic novel out of their class notes that might qualify as transformative use, and they might say that’s a brand new thing, and it might belong to me separately. So just to sum that up, if you’re not recording your work, then someone could claim ownership in it, and that could include notes. If you’re in a situation where students are taking down as notes what you’ve already created in a tangible form, generally speaking, that’s not their property. That’s your property.

Rebecca: So, on the theme of recording, we’ve talked a bit about posting materials on the learning management system and maybe sharing through the learning management system. But many faculty also use third-party sites like YouTube or Vimeo that have licenses very similar to what you described at Purdue. [LAUGHTER]

Kent: Yes, right.

Rebecca: So, how does copyright work for those situations or ownership in those contexts?

Kent: That’s a really interesting and strategically powerful question, and I show my students when it comes to the section in business law and intellectual property, I show them Instagram and Meta’s terms of service for example because what they may not realize is that they’re actually granting co-ownership of their property to those sites, and the language is quite broad. It’s hard to find, but it’s quite broad. So you’re raising an interesting question that if I’m putting instructional materials that I’ve created on a third-party site, as opposed to my own hosted site, I’m actually transferring rights to that site. That is a really scary question. I would say the answer is yes, just because vacation pictures aren’t the only thing that social media companies care about claiming as their property as well. But as a general question to conflict of university ownership, you could make an argument that the university can only license to themselves what’s in their system, and therefore there’s no way for them to use what’s in a third-party system because they have no access to it. So if you provide students links to external sites, which may include third-party sites, I think you can get around that licensing issue because it faces like an efficacy problem where it’s just simply not feasible for the institution to get access to it. It wouldn’t take very long for their lawyers to draft new policy that would say you can’t use third-party sites for our classes because even though I don’t think FERPA would have any bearing on this, you could just simply call it a FERPA risk issue. You could call it anything that a lawyer would want to call it. So they could marshal those back into the institutional LMS. They could also say that you must provide them copies of what you create for your courses out of a courtesy of making sure the curriculum is consistent. So they could get around that, and then you’re back to where you perhaps have started. So that could be an issue down the road. But I think initially, certainly on your own created site, you would maintain all ownership over those materials. And if you give students access to those materials, again, you’re not transferring copyright to them. In a way, you’re almost giving them a limited license to have access to your materials, not really sharing ownership of it with them.

John: On a related note, suppose that you’re writing a book and you share drafts of it with your students in an LMS. Are you giving up some of your property rights to that material if the course was created as a work for hire?

Kent: That is a super scary question, and I would say that generous heart of saying “let me share some materials with you” might inadvertently cause you to sort of step on your own rake head there and find out that you got slapped with the realization that you’re losing ownership rights over that material. If it’s protected by copyright… I have put drafts of books that I’ve written, chapters or parts of chapters, into an LMS to share with students as well as an alternative to maybe the book I choose for the course and say here’s an alternative way of looking at the material because that’s already owned by the publishing company. I’m not granting them any rights because I don’t have rights to own it as well. But unpublished work that you maintain the copyright in because you own it as soon as you write it, putting it in an LMS in a way at least grants a license to that, unless you figure out a way to scrub it from your LMS at the end of the semester and remove it as an archived piece of information.

Rebecca: AI has been a very popular [LAUGHTER] topic on this podcast and just in general, and folks are certainly utilizing generative AI platforms to edit materials, enhance their work, use it as a dialog for feedback, and those kinds of things. So, does the user own any rights to images, text, and other content that they’re using generative AI to create?

Kent: Boy, here we go. Who can take another AI discussion, right? It’s like when the zeitgeist has become full tilt, and we just can’t take it anymore. But this is relevant, and it really is a needful question for faculty to think about. So, if I can just do a background on AI and give you a lawsuit involving an Indonesian monkey. So a British photographer named David Slater became friendly with some monkeys in Indonesia as a wildlife photographer. He gave them names. One was named Naruto. He gave them his 35 millimeter camera, and then they started taking pictures of themselves. He then published a book of monkey selfies, which is charming in a variety of different ways. And in came people for the ethical treatment of animals, PETA, who on behalf of this monkey named Naruto sued the author of the selfie book in American court, alleging he violated the copyright of the monkey who took the pictures. I wish I could make that up. I lack that creativity. The upshot of that lawsuit, as it relates to AI, and it does, is this: unlike Solomon, everybody lost. PETA was thrown out of court in effect for saying you have no standing to bring a lawsuit on behalfof a monkey in Indonesia. The photographer lost because he didn’t take the photographs, so he didn’t have the copyright in them. Now you might say, “Well, didn’t the monkeys take them for him?” No, they’re not his employees, so it’s not a work-for-hire monkey selfie book he was working on. Poor Naruto lost as well, ultimately because of this. Copyright law is clear. To be an author, that doesn’t mean just for written word, that means for anything that’s copyrightable. to be an author, you must be a human being. Now, by way of contrast, corporations are persons under the Constitution, but the word “person” is not the word “human being.” So, to be a copyright author, you must be a human being. That moves us a few years later to AI. Nothing that comes out of large language model AI is copyrightable. Now, if we get to this, it’s to me copyright infringement, but it’s not copyrightable because a computer is not a human being. So no matter what it is, if it comes out of generative LLM AI, it is not copyrightable. We know of this for two reasons. One, there’s an interesting case where a super smart guy created his own AI system and had it create some sort of impressionist style painting that he gave a title to, I think it’s called “A Recent Entrance to Paradise,” and he sought a copyright registration for it. It was denied. He went through the court system as far as you can go, and he lost every time because AI is not an author and it can’t be. He tried a second bite at the apple by saying that the computer system he generated was a work for hire, and he owned the work itself. And the court’s like, you know what? Again, work for hire start with a copyright author, that has to be a human being. Now that means we know that from case law, and the Copyright Office in 2023 issued a very long pamphlet statement, which clearly states this: there is no copyright in AI-generated work at all. Here comes the however, and there is one, and they did say this, and there’s a case on it. They said there could be, on a case-by-case basis, a copyright granted to what the copyright office called AI-assisted work, not AI- generated work. I would just say, by a quick analogy as an example, if I ask AI to write an essay for me, or let’s just make it worse, let’s make it a paper, let’s go there. I ask AI to write me a paper I want to get published, and then I take what they give me and then I edit it myself. Not only is that unethical and should be unpublishable, it’s not copyrightable because that came out of AI.

But suppose I write an essay that I’d like to get published on today’s educational issues going on in America, and the requirement is that it must be 2500 words, and I just can’t get down from 2700. If I put that into the AI system and ask it to get me to 2500, I would argue that I still own the copyright in that because I used AI to assist me in a relatively minor editing method, as opposed to, in effect, co-generate with me. So case-by-case basis for AI-assisted work, but AI-generated work there is no copyright whatsoever.

Rebecca: I’m so glad you shared some of these stories. These are some of the cases I like to share with my art and design students. [LAUGHTER]

Kent: Well, can I do an art design for you? [LAUGHTER] Because the case I’m thinking of is… and I’m terrible with names. The name has slipped me, but a person used AI to write a graphic novel. I’m old enough to call them comic books, but I guess that shows I’m too old to not know they’re graphic novels. But a really interesting looking graphic novel in which AI was used to generate all the art, and when that author of the graphic novel saw the copyright in it, it was denied because it was AI-generated art, and then the author sought a rehearing, you might say, of that, and they got a Solomon-like outcome in which the copyright office said, “Okay, here’s the thing: you own the words that are in the bubbles in your graphic novel, and you own the placement of them with the AI-generated art, but you do not own the AI-generated art.” So that means if I wanted to use any visual from that graphic novel, I am free and clear to do so because nobody owns it. But I can’t use the cells because the cells are copyrightable, and certainly the language inside those thought bubbles are copyrightable as well.

John: What if you write something and then you run it through AI and it makes some suggestions for change and you adopt those suggestions? Does it depend on the extent of AI assistance in terms of that case-by-case analysis, or would it be determined on where the thoughts originated, whether they were human originated thoughts that were then cleaned up and edited by AI or rewritten or restructured. It seems like there’s no clear dividing line between the work of the human and the work of the AI.

Kent: That’s why, and this sounds like a cop out, and I apologize, but that’s why case-by-case basis. While I’m saying I don’t think the copyright office was wrong in to say that, it creates a line that you’ll never see the end of, of finding out have I done enough for that to work. Like in corporate law, shareholders can be personally liable for the debts of the corporation if the corporation is undercapitalized, but that’s not based upon a mathematical basis of examining debt to equity. It’s done on a case-by-case basis. So, in your scenario that you gave as a hypothetical, I would say to me that sounds like that would belong to the human being who started with the work. But how much of that goes from doing fine tuning to actually creating really what is a new thing with the use of AI? When it becomes a new thing, as I read that copyright advisory document, it’s really not protected by copyright. And the scary thing to think about is this, and I know that some AI companies will tell you they’re still not doing this. You believe them at your own peril. But once you put your own work into an AI system, in my view, you’re contributing [LAUGHTER] voluntarily to the copyright infringement that is the reason for the existence of AI in the first place, and so the web scraping that goes on to create an AI system, which is the subject of a variety of large copyright infringement lawsuits that are still ongoing, you’re at least volunteering your work in an AI system that may allow your work to then be used for training and other purposes that maybe you don’t like as well. So there may be systems that claim that you can eliminate that, or prohibit that, or stop that from happening. Again, the next question asked is how would you take that to court? Because we live in a world that’s controlled by arbitration agreements, and if you have a membership or a subscription to a paid AI system, they’re not going to want you to sue them. So just like with your cell phone agreement and your credit card agreements, you’re going to have to arbitrate that, which would be a nightmare anyways. And so in a way, it’s almost like I would say when they say don’t lend money to family, treat it like you’re giving it away. I would presume to say the same thing, if you’re going to put your work in an AI system, presume you’re giving it away for purposes you can’t control. But if the purpose why you need to use it in an AI system overwhelms that, then just go ahead and do that.

Rebecca: So I want to like pull a thread that you’ve hinted at in a few different places already, which is some of the work that we may have created that are our own intellectual property has been scraped and is part of these systems. And this is certainly something that comes up a lot in my land of art and design, where particular images of artworks or whatever may be scraped and now are part of these large language models that are now used to create these images that are not copyrightable, [LAUGHTER] right? And so faculty may be using AI to generate images and things to illustrate their ideas or other things, but it’s pulling on a database of things that don’t necessarily belong to them. Can you talk just a little bit about that angle?

Kent: Yeah, the question. might be, if I’m hearing this correctly, is it possible that a faculty member is infringing on the copyright of somebody else when they’re using AI materials that they’re pulling out, even if they’re strategically asking for what they want to ask, because what has gone into AI is, to me, an infringement of someone else’s copyright. And I think that’s a big question because even though there’s no copyrighting what comes out of it, what went into it, in my opinion, has violated the copyright of the author. And so I would say that faculty aren’t infringing on the copyright of an AI system by using AI for whatever they’re using it for. The lawsuits that are ongoing are still ongoing, and there have been some settlements. I’m an author of a textbook that’s owned by Cengage so I know I’m in a settlement with the AI system Anthropic, and that settlement is in, I think, the billions of dollars. But what it gives is to each copyright holder a couple thousand dollars. And I think it’s kind of unfortunate, but that is the way that it’s going, is that the capital that those AI companies have is so huge that they can settle a class action lawsuit. And some authors have said, “I’m not going to be a part of this. I’m going to sue individually.” To them, I say, “Godspeed.” But those settlements allow them to just simply wipe that as an expense and then continuing what they’re doing, but the ethical issue is: Am I doing something I really should be doing? Am I using AI to generate ideas for myself or quick answers I can’t find, or am I using it to actually create work that I want to claim as my own if I massage it with my own assistance when in fact it came originally from another author or even an academic author who sort of lost their rights to it when it was web scraped into an AI system. It’s a deep question. It’s not a copyright infringement to the AI system, but it could be sort of an ethical infringement to the original person who didn’t choose to lose their work into an AI system.

John: Now let’s take that just a little bit further. Suppose you used it as a faculty member to generate some imagery that goes into a book that you sold to a publisher, and then the imagery that was created by the AI platform involves a copyright violation of the original copyright holder.

Kent: Wow.

John: Who would be legally responsible for that violation? Would it be the AI platform that generated that, or would it be you as the author who is using essentially slightly modified copyrighted material?

Kent: Yeah, that is not only novel and really brilliant. It is, to my knowledge and my research, unanswered as of now, so I’m only projecting what I think is the case, and I do see a world where perhaps an original owner, let’s say, of art that ended up in an AI system that ended up again in someone else’s work, could sue that end user, saying you infringed on my copyright, and you could find a situation where a faculty member might have to ask themselves, “Do I have an insurance policy that would cover the legal costs of defending myself in an infringement lawsuit?” Because let’s say that suppose that happened to me, perish the thought, is that a situation where Purdue is duty bound to defend me? But if, like you said a book, what we’ve said is these books tend to be not owned by institutions because their copyright is exchanged for some fee structure. But if you read your publishing contracts carefully, you owe the publisher a hold harmless clause to make them whole if they violate copyright because of what you put into a book they published, and so that’s a very scary financial question. And again, that’s something to think long and hard about before using AI-generated art. And again, it doesn’t matter if you have that background knowledge like Rebecca does. It matters if you’re simply using work that really is sourced to somebody else, because the middleman, in this case AI, is not liable for it, at least not yet, and maybe not ever. But you, the end user, could be.

Rebecca: We haven’t talked a lot about the role that students play. Students are often creating work; they’re submitting it to you, maybe as a faculty member to review. Now you have a copy of it in your possession, perhaps. Do students own their own classwork?

Kent: There is a lot to say about students. It’s not just the cheating part, although that is worthy of discussion if you want to talk about cheating and AI detectors and things. But students own their work for the same reason that faculty might own their work as well, and I think students should be treated respectfully with respect to their work. And so, if we say that the student work passes the modicum of or spark of creativity test we talked about, that does belong to them. There can be some exceptions. Sometimes institutions have policies for graduate students if they’re creating work that’s part of a project that is sponsored by the institution. Sometimes there might be policies that say students don’t own their work if it has been hired out by an outside party. For instance, suppose a marketing class has an instructor who’s connected to a business that wants a project done that the students can free help the business, right? Well, who owns that work is a huge question. Policies might carve exceptions out there, but as a general basis, students own their work. So I have, in instances, particularly in online classes, said to students, “You know, some of your work here is so good. Is it okay if I use it anonymously as a template or a model of what ideal student work would look like?” I shouldn’t take their work without their permission. If I take their permission, I seek an email that I can archive and say they granted me permission. There is a case, there are more than them, but there is a case out of a university in Detroit where an art student was asked by her instructor to have students create artwork that would be used by the Mercedes-Benz exhibit at the Detroit Auto Show, and she knew what she was giving up, but she didn’t realize that Mercedes, with the knowledge of her instructor, used more of her work that she created than she had acknowledged to give over to Mercedes. And she sued Mercedes-Benz for copyright infringement, and she won. And it didn’t matter that she didn’t have a market value in her work because there’s another part of copyright law that allows you to get statutory penalties for simply infringing on someone’s copyright. And then she sued her professor for what is commonly known as a breach of a fiduciary duty to her as a student. I don’t know where that lawsuit ended up, but the court allowed that lawsuit to go forward. So I would say that faculty should be mindful that what students create in their coursework, including artwork, musical compositions, and just essays and projects, really do belong to the students, absent some external independent contractor work for higher situation like the Mercedes Benz was for three of the four artwork pieces that ended up at the exhibit. So I do think faculty should be mindful of that. And there’s a separate issue, which is a faculty member could say, “Yeah, but there’s no fair market value in this student’s work. So what does it matter?” Well, that question alone shows why it does matter, because faculty should be respectful of their students as individuals and as owners of their own work. So whether there’s a market value for it to me is not the question. It is, “Does it belong to them?” And if it does, and it does, then I should be respectful and just ask to use it. And if they say yes, get it in writing somewhere, and you’re good to go generally,

John: Following that up a little bit, if we upload student work into a plagiarism or AI detection software package, or perhaps just to an AI platform to get some feedback on student work to assist with grading, would that be a violation of students’ rights? I was thinking in particular of how if there was some artwork in that student’s work that was uploaded that was used for training it, it would be in some ways a similar sort of violation.

Kent: I totally agree with that. Often AI and students, from an HR standpoint at work, is discussed with respect to FERPA. Well, I mean, you’ve got to be an idiot to use AI systems to create education records that would violate FERPA. But in terms of what you’re saying here is, am I really transferring without their even knowledge, perhaps, their ownership into an AI system by using it for evaluation purposes? That is a serious thing to consider. Now, would anybody know that I did that? They could argue the only way that my work ended up in an AI system is I turned it into you, and you stuck it in there. So it wouldn’t be terribly hard to figure that out. But that’s an ethical question and a legal question as well. One issue to think about with respect to AI detection, if I can just mention that, is when you’re using it, you really then transferring copyright because the original plagiarism detector that we all knew… I never used it… in fact, I didn’t pronounce it correctly unless someone corrected me on it is Turnitin, but students from a high school class sued Turnitin for copyright infringement, and I say “you go, good for you,” and they said you took my papers and violated my copyright by sticking in to Turnitin, and the students lost because the court said this is a transformative use of someone else’s work for plagiarism detection, and there is no infringement on their ability to publish their work by being archived for plagiarism detection purposes. That’s not, however, what’s going on now. Because even Turnitin’s AI system has been turned off by some universities. So I would say this: here’s some free legal advice, which means it’s worth nothing. Don’t use AI detectors. Simply don’t do it. It’s not worth it for a variety of reasons, and the most important reason is to me it’s error prone, and the last thing a faculty member should do from a legal protection standpoint is accuse a student of cheating based upon an AI detection system because they have been known to be error prone, they’ve been proven error prone, and even if they’re improving through retraining, they are error prone. If I haven’t said that yet, and if you’re saying I’m giving up my sort of autonomy as a judgment maker to an AI system, which is based upon predicting the next logical word, that’s why they fail. That’s why they make mistakes because they’re based upon perplexity, “What is the next most logical word coming up that we think you want?” So if I say I’m dropping my children off at blank, logically it’s going to be perhaps school or the playground or the pool. That’s the lower perplexity. Well, you’re giving up your autonomy, your judgment decision when you’re saying to AI, check if this has been cheating. Now, if you can reverse engineer that by saying I put your work in an AI system and it showed that it came out of already published work. That’s a traditional plagiarism detector. But AI systems also flag often non-native English speakers as cheating because of the way they write sentences with low perplexity and what’s called burstiness, which is an interesting word. Burstiness is the variation we use in the sentences we write: short, long, those kinds of things. So, non-native English speakers and sometimes autistic students are flagged because of the low burstiness in their sentence structures as being cheating. So, to me, just don’t use AI detectors because AI detectors also sell humanizers to students as well. AI humanizers are like police officers selling radar detectors on weekends. So if you want to avoid being caught by us by this product, when you’re dealing in a world where AI humanizers are already being used by students to bypass your using AI detection, then the question is, what are you doing for your job? Is that your job to be the hall monitor of all hall monitors? And the cheating is rampant. Just recently, I think a Brown University award-winning economics professor realized that allowing his students to do a take-home exam basically the AI cheating was rampant, and he was just horrified by that. So we know that it’s going on, and I’m not saying that we shouldn’t care about it.

What I’m saying is, out of all the time you have in your life, of which a lot of it is work, how much of it can you give to hunting down the possible AI cheaters in your classes? I don’t have that kind of time, and if I’m going to be wrong, I’m really wasting my time as well. So I think there’s a legal issue with using AI detectors. MIT, who you can, if you want to, you can source MIT as the first creator of an AI system, says don’t use them. So if MIT’s teaching and scholarship learning center says don’t use them, why do I need to seek a second opinion? I’m not going to use them, and so I’m not saying you’re wrong to use them. I’m saying you use them at your own peril because of the issues involving the last thing I want to be accused of is committing an act against a student that is known as arbitrary or capricious. Sometimes it’s called arbitrary and capricious. One of the few areas where a faculty member faces a legal risk is when they act arbitrarily towards a student, and I believe using an AI detector as your reason for accusing someone as cheating is getting you into that arbitrary world that I don’t think is worth risking.

Rebecca: On the flip side, students often submit tests, assignments, and other materials created by the instructor into generative AI platforms [LAUGHTER] without the consent of the instructor. How should faculty address this issue?

Kent: One, and I know you don’t need me to say this, so I apologize for preaching to the choir here. But have a syllabus policy first of all, because this may seem too smart by half, but a student has a good point by saying I didn’t do anything wrong because you didn’t say I couldn’t. And so if we presume they’ll know they shouldn’t do something, then we’re wrong as being in charge of our classroom. So, have a syllabus and specifically delineate what you don’t want to be done. You, again, can’t control what they’re doing on their own time, and unless you get back to that Paleolithic world where we had everything done in class, either on paper or in front of people, it’s hard to catch those kinds of things. People sometimes have students sign honesty pledges attached to syllabi. Those are all well-meaning. They have no legal value. So, other than policies that allow you to go to the dean of students or have some sort of institutional disciplinary process for that if you catch it, outside of a clearly delineated policy and examples and encouragement. And again, this may sound hokey, but like I tell my students, you’re in a business school. A business school is a professional school. What does it mean to be professional? Like, what does it actually mean? It’s not about making money. What does it mean to be a professional in a professional school? How does ethics fall into that? And again, that may seem like I’m wasting their time if they don’t care, but at least they know that I’ve thought about it ahead of time, and that I tell them it’s not worth getting caught, and you’re not going to feel better down the road because you cheated here, and so I just try to be that good angel on the one side of their shoulder, reminding them that it’s just not worth it. Again, that may be a waste of my time to tell them that, but I can’t hunt down all the possible cheating unless I bring everything back into the classroom, which some faculty are again trying to do, and we’re now living in a like a record player world again, where we’re saying, you know, something, let’s go back to the record player and let’s bring the classes not just back into the classroom, but bring the assessments down to more of that low technology world, which cuts down the chance of cheating.

John: We always end with the question. What’s next?

Kent: Well, I’m thinking about or trying to work on creating a podcast for myself, not to compete with you at all, but on legal issues for faculty, and I’m trying to work out how that might come together, and hopefully that will come to fruition sometime in the near future. And I’m working on some conference presentations to interested faculty members and book groups where we get together and discuss issues from this book that I’ve written on legal issues for faculty, and then the fall semester is around the corner, so that’s on its way as well.

Rebecca: Yeah, summer never fails to slip by.

Kent: No, I finished my summer semester about a week ago, and now I’m thinking about the fall semester already.

John: And it’s always good to see more podcasts related to higher ed, including legal issues in higher ed. We’ve lost a few good ones in the last year, so it’s nice to see new ones coming online.

Kent: Well, if I can overcome the technology difficulties I had prior to this recording, that may be something to figure out. Can I actually make this happen? So, thank you for the encouragement on that.

John: If you need any help, just let us know. [LAUGHTER]

Kent: Well, that’s recorded now, [LAUGHTER] so you can’t walk that back. Rebecca didn’t say it; you said it. So I have one person to go to, and I can pinpoint when you specifically said it. So thank you for that invite. [LAUGHTER] Il I appreciate it more than you might care to know.

John: Well, thank you. It’s great talking to you. And these legal issues are ones that many faculty are thinking about. So this was really helpful.

Rebecca: And I always enjoy these conversations. So thanks for joining us.

Kent: Thank you very much for letting me be here. I really have a wonderful time talking with the two of you. So thank you again for allowing me to be on your podcast.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

425. Authentic Voice in the Age of AI

Student use of AI tools presents challenges for faculty teaching writing. In this episode, Anna Mills joins us to discuss when and how AI tools can be used to help students develop their writing skills.

Anna has been a leader in exploring effective strategies for integrating AI into higher education in a manner that fosters the development of student critical literacy. Anna serves on the MLA Task Force on Writing and AI and as a lead advisor on the instructional design for MyEssayFeedback.ai. She also has served as the only educational specialist recruited by Open AI to test GPT-4 pre-release. Anna is also an OER advocate who has released numerous OER resources including two OER textbooks, one on How Arguments Work: A Guide to Writing and Analyzing Texts in College, and the other on AI in College Writing: An Orientation. She is also one of the developers of the PAIRR process in which students develop writing skills through feedback from peers, AI, and individual reflection.

Show Notes

Transcript

John: Student use of AI tools presents challenges for faculty teaching writing. In this episode, we examine when and how AI tools can be used to help students develop their writing skills.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Anna Mills. Anna has been teaching college writing courses for 18 years and currently teaches writing at the College of Marin. She has been a leader in exploring effective strategies for integrating AI into higher education in a manner that fosters the development of student critical literacy. Anna serves on the MLA Task Force on Writing and AI and as a lead advisor on the instructional design for MyEssayFeedback.ai. She also has served as the only educational specialist recruited by Open AI to test GPT-4 pre-release. Anna is also an OER advocate who has released numerous OER resources including two OER textbooks, one on How Arguments Work: A Guide to Writing and Analyzing Texts in College, and the other on AI in College Writing: An Orientation. She is also one of the developers of the PAIRR process in which students develop writing skills through feedback from peers, AI, and individual reflection. Welcome Anna.

Anna: Thank you. I’m excited to be here.

John: We’re very happy to talk to you.

Rebecca: Today’s teas are:… Anna, are you drinking any tea today?

Anna: Oh, yes, I am drinking Yorkshire Gold. It’s my standby.

Rebecca: A classic.

John: That’s long been a favorite of some of our British colleagues, and I am drinking ginger peach green tea.

Rebecca: Nice. I have switched on to Jasmine green tea.

John: Very good. So we’ve invited you to here today to discuss issues that you’ve been writing about and talking about quite a bit related to fostering academic integrity while helping students develop AI literacy skills. First, though, can you tell us a bit about how you became interested in the possibilities for using AI in education?

Anna: Sure, so I actually grew up in Silicon Valley, and my dad was a software engineer working with the dinosaur age AI of the time, and so when I kind of stumbled onto today’s large language models, before ChatGPT, I was playing around with how well it would do generating essays that fit my assignment criteria, and I realized that it had made a point that I hadn’t thought of in critiquing a sample essay that I had written. And I just thought, this can complement my thinking. This can stimulate my own thinking, this can stimulate and complement what I already do. And if it can do that for me, then it might also stimulate my students, so it might support their learning, depending on how it’s used. But I think I’ve always been interested, growing up in Silicon Valley, in the divergences and convergences between human experience, human thinking, and mechanical processes’ logic and early computing. So I’m always interested in those kinds of tensions and questions, and I just was electrified when I realized that systems that can simulate thought are here. The Turing test has been passed, and that’s the strange age we’re in. [LAUGHTER] So, I think it’s an exciting moment.

Rebecca: Not all faculty lean in [LAUGHTER] necessarily in the same ways that you do, given your past history and your curiosities. One of the major concerns that faculty have about student use of generative AI platforms is that students might use it to substitute learning, rather than complement it, like you had just kind of noted. Many faculty have reacted to the availability of generative AI by simply banning its use. Is that a productive strategy?

Anna: I actually think it’s a pretty reasonable strategy. I would just say it can’t be the end, right? It’s a beginning, it’s not the full answer. And certainly, if we’ve designed a learning activity for the student to do, then it makes sense to ask them to do it, right? and not to have a machine auto-complete it for them. So as a first step, I think banning it for a learning activity that’s not designed for it makes some sense. I just think it’s sort of similar to the dawn of the internet, in that we might ban it for some things, but we also will start to build AI literacy curriculum to guide students as to how to understand this phenomenon, how to think about it, how to consider whether to engage with it or not, and we start to develop some assessments that might involve it as we figure out how it actually could support learning rather than getting in the way. So not everything has to involve it, and certainly we need students to do something, right? So we have to ban it in some contexts and in some ways, even if we want to work with it. So I don’t think instructors who ban it should be demonized at all. I think they just need a little more support to think about those next steps, as far as, how can I continue to guide students in this evolving landscape? How should I start to consider applications, limited applications and pedagogy. I don’t want to demonize anybody for banning it. I think it’s a very reasonable first step.

John: Would limiting or prohibiting its use on lower-level tasks be appropriate so that students could develop some basic skills in the discipline that they could then use to evaluate AI output? Because AI can be really productive, as you’ve noted, but it does also hallucinate, and it does produce output sometimes, which is not very reliable, but in order for students to be able to evaluate that, they need to develop some skill in order to be able to distinguish between what is quality output and what is not. So might bans be more appropriate as students are beginning to develop the skills and then perhaps progressively allowing more use of AI?

Anna: Absolutely, yeah, I’m completely with you on that, and I think there are parallels to calculators and computer use. And I’m very concerned about setting some guardrails, some guidelines, so that students still have these formative experiences, so they can develop the kinds of skills that they need in order to do their own thinking, make their own decisions, to interact with AI productively, to be skeptical of what it gives them, to improve on what it gives them. So absolutely, I want to do that, and that’s kind of why I’ve gravitated toward not inviting my students to use it in hybrid ways in their writing, I want them to have those experiences in college of developing an essay from start to finish, with their own words, their own ideas, their own organization. They’re making the decisions and AI is used in my pedagogy mainly for feedback, that they then reflect on critically. So, absolutely and I think, unlike the calculator, I think you have to wait longer before you’re really ready to be learning with AI in these hybrid ways. I think we have to protect space for students to develop their own skills, making their own choices.

Rebecca: We talked a little bit about disciplinary skills, but we’ve also mentioned AI literacy. Can you define what that is? I think we like vaguely have an idea of what that is, and we’ve been using that term over a couple of episodes, but haven’t really defined it.

Anna: Yeah, I think it’s hotly contested. Sometimes people use it to mean sort of technical knowledge about AI systems, and that’s not usually what I mean. But I think that you have to know enough to be skeptical, as John said, to know that there are errors and hallucinations, that these are fundamentally sort of pattern-matching systems. And beyond that, I think a lot of the things that help you interact well with AI are not specific to AI. So there can be general information literacy skills, reading, writing, and critical thinking skills, expertise in a subject matter, but I think they also involve knowing what current systems are capable of and some of the ways that people use them strategically, so some familiarity with this landscape of what people are doing with these systems and how well they perform. And that’s, of course, a moving target, but I think it is important for educators and students to be keeping up, to some extent, with what’s possible, because that will be part of workplaces, certainly. And at the same time, it’s not about the technical details. It’s not about learning Python and being able to build these systems yourself. It’s not even about prompt, quote, unquote, engineering. What we’re hearing is that, with the current systems, as they improve, you really don’t need these specific technical approaches. You need expertise and thinking and reading and writing skills. So I think any look at AI literacy should really emphasize those. But I think that there is also a certain familiarity that we need to support teachers and students to develop with what can you do now with these systems? What are people doing with them, and what are the pitfalls? What are the dangers as you use them?

John: One of the things that’s happened over quite a period of time is that more and more assessments in classes have been shifted outside of the classroom. That is especially true with online work, and if you are trying to restrict student use of AI tools while they’re developing those basic skills, one of the things that many faculty have tried to do is rely on AI detectors. How reliable are these? When is it appropriate to use them? And in what ways might it be appropriate to use them?

Anna: So they’re definitely not reliable, even the companies will acknowledge that those detection systems sometimes label AI text as human written and sometimes they label human written text as likely AI. So any use of them has to take that into account and not rely on them. That said, I do think they have a limited place in my practice, I have a lot of colleagues who strongly disagree with me, and I recognize some of the concerns that we don’t have rigorous testing of these systems for things like bias against English language learners, and I would like to see more testing and regulation around them, but that said, we do need ways to deter misuse, to make it less tempting to students, and I think AI detection does quite a lot in that respect when you use it to complement other strategies that reduce misuse, and when you reassure students that you know it makes mistakes and that you will take that into account, that you will meet with them, you will use other approaches to determine whether they’ve used AI. In my case, I’m not using it punitively. I am offering students another chance to rewrite. I’m giving them the benefit of the doubt where that’s necessary. If I meet with them and they can explain the essay pretty well, I’m not going to penalize them, even if the writing was flagged as AI. So I think there is a judicious way to use them, but I would also like to see a lot of guardrails around their use.

John: And we have seen a lot of cases where faculty just universally will accuse students of AI use when the detectors flag a high probability. But as you noted, that probably is not appropriate without some additional evidence, is that the approach you’d take?

Anna: Definitely, so I’m using detection, but I let students know that if their work is flagged, that means that we’re going to meet and talk about the essay. And when I meet with them, I let them know that I’m not trusting it, but that I am concerned. I want them to be learning, to be developing their confidence, their voice, and we can work together on that. If there is something that was more AI than they intended or they should have done, then that means they haven’t done the learning yet, and there’s still space for that in the class. And then I’m using it in association with a lot of other things. First and foremost, I’m emphasizing what can I do so that they feel engaged with the class and intrinsically motivated. I’m giving them a lot of choices about essay assignments. We’re talking about how writing helps us develop our thinking skills and how that’s applicable to many other things in life. And I’m checking in with them a lot about their engagement with the assignment. I’m offering a lot of support for their process, a lot of formative assignments along the way that are credit/no credit, a lot of tutoring support, scaffolding for the skills to help them build that confidence in their own voice. And I’m also offering ways to use AI that support their learning, that are not cheating and so kind of trying to guide them into that other path, where it stimulates their thinking rather than replacing it. So I think it all goes together. There’s not one right approach and one wrong approach. There’s a lot of things that can help. And you can get around detection, also. There are tools that will, quote, unquote humanize AI text to get around detectors. You can get around process tracking. You can simulate typing into a Google Doc and make it look like you’ve spent an hour on that Google Doc. So these approaches, they’re not adequate gotchas, and they shouldn’t be used in that spirit. But that said, we need accountability in most areas of human life, even if we mean well, sometimes we need help sticking to that, and we need deterrence for those actions that deviate from the path. And I think AI detection can be that, it can help with that. And as you said, we’ve got quite a situation with online learning, and we may need some proctored assessment to complement it as well going forward. But I think this is a little bit like cybersecurity, so it’s never going to be perfect, and yet you can’t stop working on it. And I think some technical mechanisms to discourage misuse can support educators, can be an ethical tool. So I think we should keep asking for ethical tools that support academic integrity.

Rebecca: Both you and John have talked about online learning and the challenge of assessments [LAUGHTER] in the age of AI, so other than oral exams or in class blue book exams, proctored exams, what else can we do in the age of AI to make sure that we have good assessments for students when these other ones might be a bit vulnerable?

Anna: Well, I think that there’s a great literature on humanizing online learning. People like Michelle pacansky Brock have really shared a whole bunch of wonderful practices that are good pedagogy anyway, so things like collaborative annotation, where students are commenting on the readings for class in a social way and responding to each other. Video and audio assignments, using platforms like VoiceThread or Canvas Studio, so building courses so that they are social in a variety of ways, including discussions, but beyond discussions, I think those things can help. Online learning doesn’t have to be sort of one direction, content delivery and then performance. There’s a lot more possibilities than that. I’m interested in things like lockdown browsers for a little bit more of a guardrail around some online assessment. But I also recognize that’s not secure either. Even with a lockdown browser, you can have another device and have aI going on that device. So I do think we might need some complimentary sort of proctored assessments. There’s this two-lane approach coming out of Australia that says we can have formative assessments where students may or may not use AI, but then we need to validate that there has been learning with some form of secured assessment. And I think we need to keep looking at what that looks like for online courses, balancing things like validity with student access. We got to keep going with online learning, because so many students would not be able to access their education otherwise, and we have to find ways to validate that, and that might mean various forms of observing student learning, close observation, whether that’s proctored assessment in person or some form of online process observation.

John: One of the things that I found somewhat troubling recently is the number of browser extensions that have been developed to automatically detect questions on the screen and to offer to answer those questions. In fact, Chrome provided one for a short period before taking it down. Would a lockdown browser be a strategy to prevent that, or would they not recognize the fact that there is this AI agent hiding in there that is doing the data input, rather than the students themselves?

Anna: Yeah, I have been wanting to test that. I’m not sure if lockdown browsers can block agentic AI browsers like Perplexity Comet or ChatGPT Atlas that will just move around and click and fill out forms for you in the learning management system. I think they might help. But long term, I don’t think teachers should have the burden of trying to design around this. I really think we need the learning management companies and the AI companies to help, to say to these systems, don’t complete work for students. It’s a very simple step that the big AI companies could take to drastically reduce this kind of threat to learning. I think we’ve got to pressure them to do that. And I think there can be mounting public pressure from parents who want their kids to learn, and they want their kids learning to count for something and not be judged against somebody else’s cheating. So I think it’s sort of a common sense societal interest that we’ve really got to push. We’ve got to make space for learning online, and that means don’t let the AI systems jump in and take over. Make sure that it’s the students. And there’s not a whole lot that I think I can do as a teacher to block AI agents. I might be wrong. There are educators experimenting with this. Tim Mousel just released a video where he shows how he built this little system to try to detect if it’s an AI agent and not a student. And maybe that can inspire the learning management companies to do something similar. But I think for most educators, we’re not running the Canvas or the Blackboard, and there’s not too much we can do. Multimodal assignments might slow down an agentic browser, like it’s probably not going to be able to generate a video with my face too easily without a little help. So yes, those things might help a bit, but I don’t think it makes sense to ask teachers to redesign our entire pedagogy when we really don’t have a lot we can even do to block this, and the big companies do have things they can do to block it.

John: We’ll include a link to that video in the show notes. I saw it this morning on LinkedIn myself and I. He was able to block two of the three things he tested with his work. But it’s an escalating battle. We have better things to do with our time as educators than to fight that.

Anna: And why should we throw out pedagogy that works and that is research based? Why should we deprive students of those kinds of learning experiences that we know how to provide? So I really want to protect things that we know are valuable. I want my 14-year old son to have those learning experiences, and teachers shouldn’t have to throw them away. That doesn’t make any sense.

Rebecca: You talked a little bit earlier about ways to preserve academic integrity, but also to motivate students. And you hinted at a few things like process, scaffolding, and things like that. Can you dive a little bit more into what that looks like, maybe in one of your classes?

Anna: Sure. So in writing pedagogy, we’ve been teaching writing processes and strategies for years. So in my class, that starts with collaborative annotation of readings for the class, so students are speaking back to the text in the margins and talking to each other about it, and that starts to lay the foundation for what they want to say in their essays and for making space for their voices. Then there are a lot of low-stakes assignments where they’re brainstorming or they’re writing drafts where they have permission for them to be terrible. We’re reading about writing strategies that help us get past those difficult moments. We’re acknowledging that thinking is hard and sometimes painful or frustrating, and that’s okay, and there are lots of things we can do to seek support. I’m assigning tutor visits as homework. I’m giving lots of chances to review each other’s drafts, to revise, to reflect on AI feedback before there’s a final draft due, and even then, I’m saying, “You can rewrite to raise the grade.” So this is an ongoing process where there’s support all along the way. We’re going to get stuck, but there are a million moves we can make to feel better and to ultimately push our own thinking further in the writing process. And you get credit for all of those things that you do. So I’m hoping that that’s really an affirming and a hopeful orientation to writing that carries a lot of empathy for the moments [LAUGHTER] when it’s hard and onerous, but that ultimately offers a lot of options for every step.

John: Many of the guests we’ve had on the podcast in the past have advocated alternative grading systems as a way of shifting the focus from grades to learning. Have you experimented with that? Or have you talked to many people who have tried this, and has it had an impact on improving student motivation in shifting the focus from just trying to maximize grades to maximize learning?

Anna: Yeah, I think it definitely does help. A proportion of my class is sort of ungraded credit/no credit for process assignments, and then I do have a portion that’s graded. I think it can help. I think you still have to have a certain minimum standard for passing the class, but I think it’s something I’d like to explore further, because I think it does remove a lot of that anxiety for students. The sense of inadequacy is huge in first-year composition classes and the sense of imposter syndrome, and “maybe I don’t belong here, and I should be using big, fancy words.” So, there’s so much sense of inadequacy and fear and imposter syndrome in first-year composition in community colleges where I teach. So you can say to students, “I want to hear your voice and you don’t have to use fancy vocabulary and all of that,” but it’s hard for that to really sink in. I think that ungrading can be part of just affirming whatever it is that students bring to class. But I think also sharing examples with students of student voice, and celebrating that, can be powerful as well, so that it really becomes the students’ space and more on their own terms, and I think ungrading can be part of that.

Rebecca: One of the ways that you’ve talked a little bit about doing this is through the PAIRR process. Can you talk a little bit about this process for our listeners?

Anna: Sure. So PAIRR stands for peer and AI review plus reflection. This was actually developed first at University of California – Davis by Carl Whithaus, Lisa Sperber, and Marit MacArthur, principally, but I was doing something very similar on a parallel track as a volunteer advisor for MyEssayFeedback, which is this not-for-profit app, and then I joined forces with the UC-Davis PAIRR researchers, and we’re now working together because it’s an approach essentially to bringing AI into what we already know about good writing pedagogy and building AI literacy in the process. So basically, you keep what’s good, including peer review and instructor feedback and students drafting, putting together their own draft, and then you bring AI in for feedback that stimulates further revision, and you encourage students to question that feedback and compare it to what they got from their peers, and think about how they feel about that, and what was their actual purpose with the writing. Did the AI feedback get it right, or did it misinterpret what they want to do? So you’re building this habit of skepticism of AI and also building some prompting skills, helping students get the most out of that feedback by pushing back in various ways and asking for more from the system. So that’s the core of the approach. But we also start out with some AI literacy readings, so that students have some basic concepts about what these systems are and AI literacy, including questions around the ethics of AI and how those ethics should influence our decisions about AI use. That’s a big piece of it as well, but really the goal is to support students’ sense of agency around their own words, their own thinking, and their choices around using AI. So it’s a really simple method, and I think, very complementary to existing pedagogical practices. So it’s kind of something that people who are against or afraid of AI can use, and that people who are excited about AI can use. I think of it as kind of a meeting place, this approach.

John: How have students responded to this approach, or the approach that you’ve talked about with MyEssayFeedback?

Anna: So the UC Davis researchers, they have a piece out in Computers and Composition. They found that students overwhelmingly preferred having both peer and AI feedback. They found them the most useful in combination. They didn’t want one or the other. Often the AI feedback reinforced the peer feedback. Sometimes it provided a complementary perspective. They also noted flaws in it, that it wasn’t always right, and so that’s overwhelmingly what I’ve seen as well in student response to MyEssayFeedback. I’ve used it for I think this is the fifth semester, and that’s what we’ve seen in other teachers’ observations and student testimonials on MyEssayFeedback, that it’s sort of like, “Oh, this is a way that AI can support my writing and not interfere and not get in the way. I’m still in a driver’s seat, but I’m getting help.” So I think it’s a really powerful paradigm for students that they appreciate. There’s the concrete help, but there’s also that paradigm shift around what AI can be for them. And we hear that again and again in student comments, and we’re still doing further research, so we’re still looking to improve the process, to improve the prompts that we use, the ways we request AI feedback. We’ve done a lot of testing on those prompts, and we shared them publicly as well. So I think we can keep improving that model and sharing it and inviting others to improve on it as well.

Rebecca: One of the things that I can imagine as a student that would be interesting is when you have peer feedback and you also have AI feedback, there’s sometimes conflicts in that feedback, but there’s also places where that feedback converges, and that happens when you have feedback from multiple people anyways, it’s not always the same, so sometimes that helps, maybe, prioritize what to work on or to focus on.

Anna: Yeah, I think sometimes students don’t know whether to trust the peer feedback or the AI feedback, so it is helpful to see where they overlap with each other. We also designed one of our prompts so that it would try to nudge students to speculate on what the human readers will feel or think or want, to kind of remind them that the point is the human readers’ response, ultimately, not so much does it meet the criteria or the grade, although that’s also part of the feedback, but what are your human readers going to want out of this? And that the AI can help them think about that and remind them of that framework, so it really can complement this human-oriented writing process.

John: We have a lot of faculty, though, who are still a bit reluctant to try anything related to AI, and they just like to bury their heads in the sand, or who are just giving up on a role for faculty in affecting the evolution of how we do this in education, because they feel that it’s being dictated to us from the big tech companies out there, and we don’t have any sort of say. How would you respond to those faculty who are either just ignoring AI or who are perhaps giving up any hope of being involved in how we use these tools?

Anna: Yeah, I would say “I hear you.” That makes sense. It’s a lot of cognitive load and labor to engage with it, to learn about it, to try out new strategies, to understand how people are using it. Those of us who are on social media a lot learning about AI and education often liken it to a fire hose coming at us. There’s constant change, and this is labor. I think it is possible, though, to support faculty, to compensate the labor to develop programs that make space for faculty to learn and to shape and reshape our pedagogy, our AI policies, our own practice in an emergent way, in a collaborative way, that is not dictated by the AI companies or by for profit ed tech. And there have been some great examples of this that I’ve been able to be part of. One program at College of the Siskiyous that has participants get a stipend. They attend meetings, they learn certain things, but they’re developing their own project that they’re later going to share and showcase that builds on what they’ve learned, but that is very much their own. And that’s the kind of community of practice model that has some learning, some outside instruction, but also a lot of faculty making decisions based on their own experience and practice. And I think that’s the kind of model that shows respect for faculty, time, energy, and labor that I feel good about saying, if we do this, then we really can ask faculty to engage, and that will feel very different from the AI company saying, “Look at this cool thing we built. You should be using it in X, Y and Z ways. So do a whole bunch more work.” It’s understandable that people would want to say no to that kind of ask, but I think it is possible to support faculty to engage on our own terms, and that’s what we need to do. I just think faculty empowerment around AI is really something I want to see more of. Those of us who are afraid of or hate AI, those of us who are excited about it, all of the faculty voices are needed to shape what is our society going to look like? What is learning going to look like? We should be the ones shaping that. So I want to see more involvement, more people listening to us, in terms of how these systems are shaped, how the learning management systems are shaped, where the money goes. And I think we don’t need to be intimidated. These systems are very straightforward to interact with. We have the expertise to work well with AI. We have the expertise to give guidance on how it should and shouldn’t be used, and what the systems should do and not do. So really, they need us. Society needs us. Let’s step up to that. Let’s claim that, a more powerful place in shaping our future with AI.

Rebecca: So we always wrap up by asking, what’s next?

Anna: Well, I want to improve on this AI orientation textbook that I’ve done and really share it, and I want to start making more student facing videos. I think one space where a lot of learning happens that both faculty and students haven’t yet tapped into as much as we could is social media, and so I do a lot of engagement with other faculty learning on social media, and I want to reach students on there as well, and be in dialog with students on platforms like YouTube and TikTok, because I think that this is a moment of great vulnerability, risk, possibility, overwhelm, change, and the only way to feel good [LAUGHTER] in those kinds of moments is to sort of turn toward each other, to support each other, and to keep talking through it, and to sort of make space for new practices to emerge. So that’s really where I get my energy and hope. And I think that my wish would be for more faculty and students to get to experience that with each other and find more spaces for that with each other.

John: Excellent. I’m not sure I’m ready to do any of the TikTok dances, but that would be a place to go to reach students. [LAUGHTER]

Anna: Right. And I’m all about like that vulnerable message to students of “I don’t have all the answers, and even I question sometimes whether I should use AI at all, given the ethical concerns, I don’t have the perfect prescription for ethical AI use, but, here’s what I’m learning, here’s what I’m doing now, and why and how you might critique that, but why I’m still doing it.” I think we can be honest and vulnerable with students, and it feels better to me when I do that. I think ultimately, as an educator, it is exciting that this is all changing so fast, and students want to talk about it, they want to learn about it, they want to develop their own opinions about it and responses to it. And that’s where we are, too, as teachers. So that’s the moment where we can really meet students in a really genuine way and be energized together. To be honest about this, I think that’s a good aspect about the moment. There are a lot of challenges, but that is a good thing about where we are today as teachers.

Rebecca: I really appreciate that idea of bringing people together to talk about the issues that are really bothering them or exciting them, or whatever emotion they’re [LAUGHTER] having about it, and that experts and the novices can be together in the same space and learn from each other.

Anna: Yes, AI keeps hijacking our department meetings in the English department. [LAUGHTER] Everybody wants to talk about it at every meeting, whether it’s on the agenda or not. So we need each other. We need those conversations. And there’s so much energy around it. We really can support each other and grow together given that great need.

John: And it’s hijacking pretty much every educational developer conference that I’ve been to in the last three years. [LAUGHTER]

Rebecca: Or podcasts for that matter, John.

Anna: Yeah. [LAUGHTER]

John: Well, thank you. It was great talking to you, and we’re looking forward to more conversations in the future.

Anna: Thank you for your wonderful questions and a wonderful conversation. I appreciate it.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

398. Multicampus AI Initiative

Faculty are faced with the need to adjust instructional strategies in response to AI. In this episode,  Racheal Fest and Stephanie Pritchard join us to discuss a professional development initiative for faculty involving six campuses.

Racheal is a Pedagogy Specialist at the Faculty Center for Teaching, Learning, and Scholarship at the State University of New York at Oneonta. She also teaches writing courses in the English Department. Stephanie is the Coordinator of the Writing Center, the Coordinator of Writing and Ethical Practice, and an instructor for classes in poetry and English composition here at SUNY Oswego. Racheal is the Principal Investigator and Stephanie is one of the campus coordinators on a SUNY multi-campus grant focused on faculty development related to AI.

Show Notes

Transcript

John: Faculty are faced with the need to adjust instructional strategies in response to AI. In this episode, we discuss a professional development initiative for faculty involving six campuses.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Racheal Fest and Stephanie Pritchard. Racheal is a Pedagogy Specialist at the Faculty Center for Teaching, Learning, and Scholarship at the State University of New York at Oneonta. She also teaches writing courses in the English Department. Stephanie is the Coordinator of the Writing Center, the Coordinator of Writing and Ethical Practice, and an instructor for classes in poetry and English composition here at SUNY Oswego. Racheal is the Principal Investigator and Stephanie is one of the campus coordinators on a SUNY multi-campus grant focused on faculty development related to AI. Welcome Racheal and welcome back Stephanie. It’s been a while.

Racheal: Thanks for having us.

Stephanie: We’re very happy to be here.

Rebecca: Today’s teas are:… Racheal, are you drinking any tea today?

Racheal: Yes, I actually came prepared with two teas. I’m a big tea drinker, and I’m starting with an organic loose leaf orange pekoe, and then that will be my allotment of caffeine for the day. And then I’ll be drinking an herbal tea, which is a honey turmeric chai.

Rebecca: Both of those sound delightful, but maybe you’re slightly an overachiever in the tea category. How about you, Stephanie?

Stephanie: I am drinking my go to Earl Grey tea, and then perhaps this afternoon, will indulge in some blueberry green tea.

Rebecca: I knew that Earl Grey was gonna be involved.

John: We do have, by the way, I think, four different varieties of Earl Grey tea at the teaching center, and I think Stephanie has tried each of those at one point or another.

Rebecca: How about you, John?

John: I have a spring cherry green tea. Now that it really is spring here, we haven’t had snow for over a week.

Rebecca: At least today, it feels like spring. We’ll see what the dice rolls tomorrow. I have Chai today.

John: We’ve invited you here to discuss this multi-campus grant funded AI professional development program that you’ve been working on all year. Can you provide an overview of the program?

Racheal: So, this program is called Teaching with AI: A Cross-Campus Community of Practice, and essentially we brought together faculty from six SUNY campuses, including regional comprehensives, technology schools, and community colleges, and we asked them to design learning activities that integrate AI into the classroom in ways that are critical, active, and inclusive.

Rebecca: So you mentioned that there’s six campuses involved. Can you talk about how many faculty are participating?

Racheal: We have about 66 faculty participants at this time, and the majority of them are funded by our IITG funding. But we were able to get some additional funding at Oneonta and Oswego as well. And Oswego, as Stephanie and John, you can attest, had a really big and enthusiastic cohort.

Rebecca: For those that aren’t familiar, can you describe what an IITG grant is?

Racheal: The IITG grants are innovative instruction technology grants. They’re awarded across the SUNY system, and they’re peer reviewed through a competitive process that is used to distribute funds, especially focused on educational technology, and often OER (open educational resources).

Stephanie: We are participating with six other SUNY campuses, and those campuses are Alfred, Morrisville, Oneonta, Orange, Oswego, which is my campus, and Schenectady.

John: Could you talk a little bit about what the participants in this program have been asked to do as part of their work in the program.

Stephanie: Our project began in January, and we started with a webinar from Anna Mills, who is an advocate for AI literacy and also for OER or open educational resources. She teaches writing at the College of Marin, and we made her webinar available to all of the participants in our grant. But we also had some interest from SUNY, so we advertised the webinar through SUNY’s Center for Professional Development, and we had a very large group of participants attend Anna Mills’s talk, which really focused on thinking about AI, how it’s affecting the different assignments that we’re giving in our courses in all sorts of different ways in higher education. She also spoke about some strategies that teachers should consider as they are really working to either adapt AI into their courses or sort of try to manage how AI is affecting the classes that they teach.

Racheal: And I’ll just add that one thing I really love about Anna is she is focused on critical AI literacy, and that was a big piece of our project, as we’ll talk about as we develop the details, but she has been a voice that is kind of guiding conversations about evaluative engagement with AI and not only ways to integrate and use it in the classroom, but also ways to think about how to protect learning objectives now that these tools are widely available.

Stephanie: So we started our project by listening to Anna Mills, like I mentioned. It was very well attended, and that was in early January. After that, each of our campuses had a day-long kickoff event where participants in each of our cohorts gathered in person on the various campuses, and we led them through a series of structured activities and introduced them to the grant objectives and deliverables. We also shared with them the Brightspace shell that we had created specifically to help guide our cohorts through this process.

John: And I’ll add that in the initial meetings, one of the things that was really helpful in getting everyone there, besides requiring people to be there as part of the process, there was also lunch available, which made it a little bit nicer, and it created a very nice environment for people to begin their work together.

Stephanie: It was a very nice way to pull everyone together on our different campuses, introduce them to one another, and also sort of see what level of knowledge or engagement our different participants had at the very beginning of the project. We had participants on our campus who were doing all sorts of different things. Some had already been integrating AI into their courses in various ways. Others were experimenting with ChatGPT for the very first time during that day-long kickoff event. So it was really interesting and kind of energetic and wonderful to see all of these different people in a room learn what they were doing and also learn about what they wanted to do and what goals they wanted to achieve with the project.

John: And I’ll also note that some of the faculty were there mostly to learn how to keep students from using it as an alternative to learning. Once they started seeing, over the course of that day, ways in which they could use it and perhaps have their students use it productively, t here was a pretty significant shift in their attitude towards AI use over just the course of that very first day, which has continued over the project.

Racheal: That’s a great point, John, we saw that here at Oneonta as well, and I think that we have a number of faculty on our campus who are quite enthusiastic about AI and have been adopting it, and some of the other diversity that we saw in the room that I thought was really interesting. There were folks who were more skeptical about AI or more invested in kind of protecting learning objectives, and I did want to make sure that the experience helped them to think about those problems and questions, because those are so pervasive. And then there were faculty who also have been adopting really enthusiastically in different ways, so creating a space for both of those groups to kind of learn together and hash out different positions, even while maintaining those different positions, because I think AI’s integration, it really varies across disciplines, and I think that that richness and diversity was a really valuable and important part of this project.

Rebecca: So beyond that first kickoff day, what were some of the other things that faculty have been required to do as part of the project?

Racheal: So after the kickoff, we moved on to a series of monthly meetings that we had planned, and those meetings were designed to continue the conversation and also to offer pedagogy support around designing learning activities. So in addition to thinking about AI integration, we were also offering that pedagogy support of thinking about backward design, thinking about how to begin with course objectives, how to design an activity that is active and inclusive for students, and then how to think about feedback and assessment. So we’ll talk a little bit about the rubric that we use. I mean, we did use it to assess faculty deliverables, but it was more of a tool to guide the conversations about what a strong learning activity could look like in a range of contexts. So that rubric kind of guided our monthly meetings. We were focusing on having an AI policy in your syllabus, developing at least one course objective related to AI. And then we were focusing on designing the learning activity and finally offering feedback and assessment on the learning activity. And we’re wrapping up those monthly meetings in April. Some campuses have already hosted them. Mine is actually this Friday coming up, and at that April meeting, we were giving folks a chance to share and give each other feedback on drafts of their assignments and learning activities before they submit those final drafts.

Stephanie: One of the things that we tried really hard to do in those monthly meetings was to model effective ways of teaching to everyone who was participating in our program. So in addition to the things that Racheal mentioned, like backwards design, we also spend a good chunk of time focusing on the TILT approach. For example, they had an option, they could do an in class learning activity with AI or they could have their students submit something in an out of class way. So we spent some time talking about what those assignments could look like, and why that level of transparency in your expectations as a teacher matters, and why that’s really important. We also spent time, as Racheal said, talking about feedback and all of the different ways that we give feedback to our students, whether it just be a grade or written feedback or verbal feedback, using a rubric or not, using peer feedback as a successful strategy. So we really tried, during these monthly meetings and through our Brightspace shell, to really model what effective teaching looks like, in hopes that our participants would take that and then apply it to the projects that they were developing.

Racheal: And another important piece that pairs with that, I think, Stephanie, we also tried to be really responsive to the different needs and contexts that our faculty were bringing. If you were planning an in class learning activity, for example, you’re not going to grade that or provide written feedback, and we certainly weren’t expecting that. We wanted to allow for a range of approaches, so we talked about strategies for giving feedback during class or through conversation. We tried to ensure that a range of different approaches to learning activities and assignments were represented in the conversation.

John: And just following up a little bit on that, one of the reasons we focused on the TILT framework is because, in the current environment, students may choose to use AI as an alternative to learning material, and we’d like them to be aware of why we’re asking them to do specific things, because if they understand that they’re developing skills or tools that are going to be useful for them later, it’s more likely that they’ll actually engage in it actively themselves, rather than trying to skip the actual learning by using AI. So we’ve encouraged faculty not only to do it in these activities, but also to integrate the TILT approach into all their teaching and learning activities.

Rebecca: For those that aren’t familiar, TILT stands for Transparency in Learning and Teaching. We do have a previous episode on that, that we’ll put in the show notes.

John: And we’ll share a link both to that and to Mary-Ann Winklemes’s website, which describes this as well. And just as one more aside related to this, if you’re not sure of how to create a TILT assignment, you could always feed an assignment into AI and ask it to put it into a TILT framework. And that works especially well if you also share your course learning objectives with the tool as well.

Rebecca: One of the things that I’ve heard all three of you mention is this Brightspace course that you set up for faculty. Can you talk a little bit about this?

Stephanie: So we spent some time as a leadership team developing the Brightspace shell. We knew that we wanted a common place for all of these materials to live, especially the support materials, as our participants were working their way through our different months, or our different modules. So we worked with members of the leadership team to develop this Brightspace shell that could kind of serve as a common place for us all to come together. In that shell, in addition to uploading our monthly materials that would focus on a different piece as we moved through that rubric that Racheal was talking about earlier, we also established clear guidelines for all of our participants so that they could understand what their requirements were as they moved through the program, which we’ll talk more about in a little bit. But that Brightspace shell also offered opportunities for our participants to talk to one another. One of the things that we required our participants to do was engage with one another in what we called our community of practice discussions. So our participants were required to engage with a small group of their colleagues from different campuses. This was a little bit interesting, because Oswego and Oneonta had larger cohorts than some of our other participating campuses. At Oswego, we were lucky enough to receive some additional funding and support from our Chief Technology Officer, Sean Moriarty, which we really appreciated. And I know that Oneonta had some extra funding as well. Is that right, Rachael?

Racheal: That’s right. We had initially requested funding for 10 faculty to receive stipends of $600 each, but our Faculty Center for Teaching, Learning, and Scholarship was able to provide additional stipends for, I think for us, it was six more members to participate in the program. And I think for you, you were able to double that number.

Stephanie: So, Oswego ended up with 24 participants total, which was about 14 more than what we had initially planned for. So in the community of practice discussions, it was a little bit heavier with membership from Oswego and Oneonta, but we were still able to design these discussions based off of what our participants requested. So some wanted the opportunity to speak with people from other campuses who were in similar disciplines as themselves, and some wanted to be in groups with people who were from totally different disciplines to sort of learn what other people were doing.

Racheal: I think we’re bringing out here the two tracks that ran parallel throughout the course of this experience. On the one hand, we had our in-person cohort meetings on each of the six campuses, and that allowed people to connect across departments from a range of disciplines with folks on their own campus. And then we had this second element that Stephanie has been talking about, that was the cross-campus community of practice element, and those were virtual meetings with three or four faculty members per group. And we asked those cross-campus groups to meet for a minimum of two times and then to post their takeaways to the discussion area in Brightspace. That way we could track and see what our faculty were talking about. We could take a look and answer questions that might have come up. And we also, as a leadership team, responded to those discussions on Brightspace, and we really saw the groups talking about a range of topics. One of the reasons that we wanted to connect faculty across campuses and not just run these cohorts on individual campuses, is that we’re seeing faculty bring a lot of different approaches and attitudes to AI, and a lot of that is place specific. A lot of it is related to institutional cultures or resources, and so the ability for faculty to meet from across the system and to share what their institutions are doing to support professional development related to AI, as well as to share what different departments or what conversations around AI might be happening in those different spaces. We wanted to create space for people to come together and have those conversations.

Stephanie: It was also really interesting, because participants in the cross-campus groups, in addition to having these other conversations about AI and pedagogy, began to really work well together as a unit as they worked on the deliverables for this project. So they worked together to share their ideas for their learning activity, their learning objective, their syllabus statement. And then as we got deeper into the project, they started to share drafts with one another and to ask each other, is this meeting the criteria for the grant? Are we making a learning activity that’s critical and active and inclusive? Do you have feedback for me? So as we talked about earlier, we had the meetings on each of our individual campuses with our campus groups, but it was really nice that the cross-campus groups were engaging with one another beyond what we had initially imagined, which I thought was really a nice bonus from this.

Racheal: I completely agree, and to try to facilitate some of that conversation as a leadership team, we didn’t want to impose structure on these meetings, but we also wanted to provide some resources for getting those conversations started. So we created on Brightspace question banks that faculty could choose to draw from to guide those conversations. And so with those suggested structures, I think we were able to leave room for that creative exchange. And each group was able to kind of determine how they wanted to use that cross-campus group time.

Stephanie: We also had, in addition to the cross-campus group discussion boards, we also made additional discussion boards that were optional, so participants could choose whether or not they wanted to participate in a board called community conversation, which was very informal, just a way for people to share their thoughts or ask questions to one another. And again, this was not required, but we did have participants posting there, engaging with one another in that way. Then we also had another discussion board that we called news and tools, which was basically the opportunity for participants to share new tools that they had stumbled across, or post things related to things that were happening as new AI tools were developed and came out. As we know they’re changing so rapidly. Things are coming out every single day, so participants had the opportunity to share that information with each other as well, and I think it was nice to give them the opportunity to participate in more than just one forum, because, as Racheal said, we had several members who were very engaged in the project, and I think valued other spaces to collaborate.

John: What are some of the examples of learning activities under development by faculty?

Stephanie: So, materials for this project are not due for another two weeks or so, but we have had several participants submit their materials already, getting things all squared away before the madness of final exams week here. So we did have a few faculty who were thinking about engaging with AI tools as part of the brainstorming process. So for example, one of our participants here gave an assignment where students were asked to use Perplexity, which is a free AI tool that also can access the web and provide suggested sources. He’s asking his students to use Perplexity to help them continue brainstorming for an assignment that they’re working on in the class. So students already have some general ideas of what they want to do, and he’s asking them to engage with Perplexity to deepen their brainstorming. So for example, ask Perplexity to provide some suggested sources to the topic that they’re considering, and then evaluate the sources that Perplexity is suggesting. He also asks them to prompt Perplexity, to give them a counter argument to what they’re exploring. So those are some examples. T hat would be an example of a learning activity. It’s something that he’s going to have them do in class. And the nice thing about his prompt is that he gave his students four different sample prompts that they could take and then put into the tool, but they would have to fill out the details of their particular brainstorming. So that’s one example from our campus. Racheal, would you like to give one example from yours, and then I can do another one?

Racheal: Absolutely, that sounds great. I’ll share an example from Oneonta, while saying as well that the leadership team will be taking a look at the examples across campus once folks are uploading them, as Stephanie has said, but we haven’t done that yet, so we only have the knowledge of folks we’ve been working with on our own campuses. So one faculty member in sociology here has been developing a project that I think is really interesting and exciting. He’s going to be teaching a course in the fall that has a unit focused on conspiracy theories and thinking specifically about how to debunk or counter conspiracy theories. And so he has identified an AI chatbot called DebunkBot, which has been developed, I think, out of Cornell. And he’s going to ask students to identify a conspiracy theory of their choice, engage with DebunkBot to see how the AI counters that conspiracy theory with factual information, and then they’re going to evaluate those AI outputs. And I think within the conversations he’s having with students, too, is the question of, do facts work against conspiracy theories? There are various opinions on that, so not only evaluating DebunkBot’s outputs, but also thinking about it as a tool to counter conspiracy theories. What are its strengths and weaknesses? And what I really like about this project is, I think that it exemplifies that critical element that we’ve been emphasizing through this project. By critical, we really mean evaluative. We are asking that all of these learning activities and assignments take up that AI literacy task of helping students evaluate AI outputs and evaluate AI as software.

Stephanie: Tagging on to the evaluative piece, one of our colleagues here is working on an assignment that asks students to sort of reverse write a paper. So she’s giving her students a prompt and asking an AI tool of their choice, probably ChatGPT I imagine… one of the components of our grant that we really emphasized was trying to make these assignments or learning activities as inclusive as possible. So we’re thinking about access, who has access to this technology. And since ChatGPT doesn’t require users to make an account, there is a free version, a lot of faculty are leaning toward those sorts of tools for this. So anyway, one of our faculty here is asking her students to have ChatGPT generate an essay on one of the books that they’re reading for the course, and students are then going to look at the essay and evaluate the output that ChatGPT has created. So they’re going to look at things like whether or not the material is accurate, if there are hallucinations, if it is in fact doing what it was asked to do. So I think they’re being asked to analyze, “So how is the quality of ChatGPT analysis?” And this faculty member also has a series of follow up prompting questions to have the students continue to engage with this essay that was generated and see if they can make it better, or see if that’s it, if it’s as good as it’s going to get, and if there are still holes in the analysis or with the argument. As a writing teacher, I thought that that was a really cool way to look at what this technology can do, really consider its limitations as well, and think about how are we going to do writing, teach writing as we keep going, as more students rely on these sorts of tools.

Racheal: I love that you were emphasizing the inclusivity piece there, Stephanie, which has come up in our conversation where you were mentioning especially how you emphasize TILT on Oswego’s campus. On Oneonta’s campus, we talked a little bit about UDL, universal design for learning, but those were frameworks that we wanted to keep in the mix as part of the conversation, we didn’t necessarily have time in our progression to really dive deeper focus. So we set some basic guidelines for inclusive activities that we encouraged everyone to meet. And those were, as you said, making sure that whatever AI tools you’re asking students to use they have equal access to. So that would look different on different campuses as well. Here at Oneonta, we’re a Microsoft campus, and that means that we have Co-Pilot for all students over 18. And if you’re thinking about, when you’re introducing students to AI tools, not only making sure that everyone can access it, but also making sure that you’re thinking and talking about who might have access to paid versions, because students are sometimes subscribing to ChatGPT at that higher level, and just making those conversations about what tools you’re using part of the conversation. The other inclusive piece that we really emphasized was just that providing instructions to make sure that you’re aware that people… students, specifically… are bringing different levels of knowledge of technology to the classroom. Some students are using AI all the time, others have never used it and don’t want to. I think we’ve had those conversations in our group. So just being aware of the diversity of student capabilities as part of that inclusivity and accessibility piece was really important to us too.

Stephanie: Just to kind of tag on, I thought it was really interesting that, as we talk about access and who has access to a paid tool versus a free tool, I’m sure you have seen, I know John and I have spoken about this a little bit that, right now ChatGPT has a free trial of their paid version that they’re marketing specifically to students. So it’s ongoing from now until the end of May, which I thought was really interesting. And I know that they’re not the only tool that has done that. Is that, right, John?

John: Gemini and Claude have also done it, and one of them I forgot, which one actually is extending it until next year. So there’s a little bit of competition going on to get more students using their particular models, because student use does make up a remarkably large share of the use of all the AI tools now, for reasons that perhaps are not always as positive as we’d like.

Stephanie: I think it will be really interesting to see how that continues to develop as we move forward. They’re doing this two month free trial. Now. What does that mean for the beginning of the fall semester and beyond that?

John: One thing we should also note is that this podcast is coming out in June. We’re recording this in late April, which is why snow was as recent as last week, and why many of the references are towards the last stages of the semester. This is a rare occasion for us to be this far ahead in our recording. We often record things a week or two before their released, but we’ve happened to be able to get a lot of podcasts scheduled within a fairly short period of time. Along the lines of the assignment that Racheal mentioned in evaluating sources, Mike Caulfield has been developing a prompt for SIFT, which is his approach for analyzing the veracity of online claims, and as of last night, he’s developed an expanded version of that prompt, and we’ll share a link to that in the show notes as well, because we can use AI tools to try, not only to create false information, we can also use it to try to verify claims.

Rebecca: One of the things I’ve heard you all talk about quite a bit, and I’d like to hear a little bit more about, is the rubric that you were using to evaluate the assignments and things that faculty were creating.

Racheal: Yeah, the rubric that we developed, as I mentioned earlier, we really thought of this more as a tool to guide our conversations, rather than as an assessment piece. But we’ve run at SUNY Oneonta, similar cohorts devoted to different learning activities and aims in the past, and we knew from those experiences that being really clear about what the deliverables are and what elements we’re privileging and would like to see, that’s going to help faculty design learning activities that then can move as we’d like to move them, to our public facing repository. So we focused on three categories of evaluation on the rubric. First, we were asking folks to develop an AI policy for their syllabus. That was a big part of this project, too. As we are thinking about helping faculty and students navigate the emergence of AI, students are coming into a range of classes that all have different expectations for them. So some faculty might be saying, “if you use AI at all, you are cheating.” Others might say, “I want you to use AI. You have to use it. And here are some of the ways that I’ll be evaluating your use of it.” So just thinking about how we as a leadership team could support students by encouraging AI policies through this experience was important to us, and those policies would capture a wide variety of approaches to AI in the classroom. Some policies that we’ve seen have given guidelines assignment to assignment, where some assignments require AI use, some ask students not to use it. We’ve seen policies that outline the specific ways students might use AI throughout the semester, and we’ve also seen policies that said you can’t use AI except in these very specific ways that I ask you to on this learning activity. So we’ve really seen a range there. And the rubric begins with that AI policy element and with the course objective. So one of the elements that we’re collecting as part of the deliverable with the learning activity is the syllabus with an AI policy and a learning objective.

Stephanie: And I think it’s important to emphasize as well that regardless of how faculty feel about AI, like whether they’re embracing it or they’re saying, “I really would prefer you not use AI in this course.” It’s really important for us to be transparent with our students about what the rules are. As Racheal said, our students are generally taking five different courses or more with five different faculty who have a whole bunch of different feelings about this. So some people might think I don’t want them to use AI in my course, so I’m just going to not put anything on my syllabus. It’s so important for our students that we are transparent, regardless of how we feel about it.

Racheal: Yes, and that was one of our aims with including that syllabus element. So the second rubric element focused on the learning activity itself. And again, that had the critical, active, and inclusive piece. I think we’ve done a good job talking about how we define critical as evaluative, and how we define inclusive as tool access and instructions. And I guess I can just briefly define active as students are using the tools themselves. So we’re not just asking you to demo AI in a lecture. We’re saying, “Get students in there evaluating outputs for that AI literacy piece.” And I do think it’s important to mention here that we had many conversations about students who want to opt out of AI tool use, students who might have environmental reasons for not wanting to use the tools or other ethical reasons, and we did talk about strategies for that, including alternatives that would allow students to maybe engage with an output that the instructor created from AI or alternative assignments that aren’t actually asking them to engage with AI, but might be asking them to evaluate AI in a broader conceptual way. So those alternatives did come up for us. And Stephanie, maybe you could talk about that last rubric piece, the assessment and feedback element.

Stephanie: Sure. So we thought that it was important for our faculty, as they are designing this assignment, to really think about how this assignment or learning activity would be assessed. How are they going to give feedback to their students as they experiment with these tools, as they evaluate these tools, and what does that assessment piece mean? So we have some participants who are just doing one assignment or one learning activity, so they have to just think about that one piece. But we’ve had several participants in this project who have become inspired by the one part and have decided to make lots of changes to their assignments or their learning activities, and then think about what assessment means for them and for their students. So as we talked about earlier, the feedback piece is very important, as our students are critically and actively engaging with these tools, because we have to guide them through what they are learning. If we are really focusing on AI literacy and helping our students understand or begin to understand how these tools might impact, not only the work they’re doing in the classroom, but their future jobs. Every career is going to look a little bit different. Some are really going to be embracing these tools and expecting our students to understand how to use them, where others perhaps not so much. An example, I teach creative writing courses. Right now. In those courses, I am not emphasizing AI use because, for many of my students, they’re looking to submit their work to different journals, and most creative journals at this time are not accepting AI-generated work. So it’s important to acknowledge that, and it’s also important to focus on why we’re going to continue to do things the way that we’ve done them a little bit for the past, right? But other courses, if I’m teaching English composition, and I’m teaching them different ways that they can outline or draft or brainstorm, how they can use AI to get feedback on their own work and their own writing. Those are important skills that will come in handy for them later on, as we talk about things like efficiency and making more polished drafts and getting feedback before you turn something in. Those skills will come in handy more so for those students than for some of the creative writers.

John: We have a student advisory board for our teaching center, and they met last week, and one of the things that came up was the topic of AI, and two concerns that students discussed pretty actively was that a lot of faculty have not made clear yet what their expectations are about whether AI use is allowed, when it is allowed, and so forth, and they noted that it was more rare than the norm for people to actually have made those policies. We’ve obviously been encouraging faculty to do that for a couple of years now, but it hasn’t made it out to all of our classes yet. But our institution will be requiring an AI policy statement in the syllabi beginning next fall. So this transparency issue and being clear about your expectations is really important. The other thing they were concerned about is being falsely accused of AI use, and this is something that is very concerning for students, and we have a lot of cases where some faculty are just completely ignoring AI, and others are assuming that AI is being used by all students all the time, and accusing students of using it without any real proof in many cases, and it’s a somewhat troubling environment.

Racheal: Yeah, I think that’s such a good point, John, about the range of approaches that students are taking to AI, as well as the different assumptions that faculty members are bringing. And I think that there’s sort of a spectrum of assumptions, where, on the one hand, you have the idea that AI should be completely left out of the classroom and any use of it is cheating. On the other end of the spectrum you have we should embrace AI and use it for everything in the classroom, and if we’re not teaching that, we’re being irresponsible. So we are seeing very polarized views on our campuses, and I think that a professional development opportunity like the one we’ve been running needs to respect and honor and speak to both of those positions, while also making the conversation a little bit more nuanced. And I think one of the ways that we’ve done that is to show that bringing AI into the classroom actually addresses the problem of students using it for cheating and plagiarism, because students are: A) aware that the faculty member knows about and is actively engaged with those tools, so they might be less likely to just copy and paste, which is the use that most people do regard as cheating or plagiarism. And in addition, it not only shows that faculty members are engaged with and knowledgeable about these tools, but in my experience, students are really hungry for guidance from faculty on how to engage with them in an ethical and informed way. So we’re helping them build those AI literacy skills by confronting and talking about these tools in our classes. And to me, that might lead to them using the tools more in specific ways that we’re guiding them to, but it can also lead to them using them less in situations where we want to protect those learning objectives. Like Stephanie, I teach in the writing classroom, and I have been devoting this semester to engaging with and thinking about AI, but that has meant we have plenty of writing and activities where I’m asking them not to use it, and then we have activities where I’m asking them to use it in specific, guided, critical ways. And so I think that that more nuanced approach than we need to totally keep it out or we need to totally embrace it, that’s the area that a lot of our faculty are working in, and I think it serves our students really well.

John: And Stephanie and I have also, in a series of workshops here, encouraged faculty to co-develop their AI policy statements with students, so that students are involved in that conversation, and if they help develop the guidelines for the class, they’re much more likely to abide by them and to buy into them.

Stephanie: To piggyback off of that as well, in our cohort, especially, I mentioned earlier that we had a range of experience levels with AI tools. And what’s been really interesting is that for participants who are trying these tools for the first time, they’re having some important realizations about what these tools are or are not capable of, and I think that’s really important. If we are teaching and we’re saying, here is my AI policy, and here’s why, if we are familiar with what the AI tools can do, I personally feel that our students are going to be more receptive to that. If we are making assumptions about what the AI tools can do, or how well they can complete a task, but we don’t really know, then how are we going to recognize an AI’s work when we come across it? So for example, this is a dated example, but I think it works still really well, in a writing classroom, for a little while after ChatGPT came out, the solution to avoiding AI-generated work was to ask students to write reflectively, because they’ll be writing from their own personal experience, and an AI tool can’t replicate that. And I remember leading a workshop, and I showed an example of a reflective essay that was generated by AI, and this was like two years ago. All I had to do was prompt it a little bit. I asked it in a revision of the reflection to include a sad story about how the tool had a childhood dog that they really loved, and that relationship with their dog is the reason that they wanted to pursue veterinary studies. And it generated this beautiful reflective essay that you would have never known from anything else. As I continued to prompt it, I asked it to include volunteer activity, and it made up a name of an animal hospital… at the time that didn’t exist, but now it would, because AI tools have gotten so much better. So it’s just really interesting as we think of what the limitations are, and how there are not as many limitations in so many different fields. That is, of course, not the case for all fields, but the tools continue to evolve so quickly. So I think it’s important for us to recognize what those limitations and capabilities are, and then if we know, I feel that our students are more likely to trust our judgment, and then believe us when we say, “Yes, perhaps you could use AI for this, but let’s not because I want you to develop this skill that will help you as you continue taking more classes here, or as you graduate and start looking for a job.”

Rebecca: We’ve talked a bit throughout the conversation about how faculty have engaged with the program over the course of the year so far, in creating assignments and in the discussion boards, et cetera. Is there any additional things that you want to share about how faculty have responded to participating in this program.

Racheal: I think that some of the feedback we’ve received is that it’s been really nice to have institutional support to help faculty do work that they felt the need to do in their classes. Faculty want to respond to AI’s emergence, and oftentimes we know faculty are overworked. There’s so much to do, and so being able to get that institutional recognition through a stipend to say “this is important, and we’re offering support for developing these activities.” We’ve heard faculty report that they’re happy that the opportunity is available to do things that they’ve already wanted to be doing, but here they can see the institution supporting that.

Stephanie: I agree, and it’s also been very helpful as we have these conversations with our participants to think about effective practices on the whole with AI. So for example, there has been conversation everywhere about using AI tools to grade student work and to assess student work. So we’ve had opportunities to have those sorts of conversations about why, perhaps, it’s not in our best interest as teachers to use AI to give feedback to student writing, for example. And it’s been nice to hear what other people are doing, like Racheal said, with that support aspect, but also the conversations allow us to sort of come across those topics that might just be neglected otherwise.

Racheal: And I’ll just jump in, Stephanie, because your comments about using AI to assess and grade has also been an important part of our conversations at Oneonta. And again, we’re seeing really polarized views, where some people are saying, “I would never do that.” Some people are saying, “I’m already doing that in these specific ways.” And we’ve talked there, for example, about the Student AI Bill of Rights, which circulates online. And one of the elements of that AI bill of rights for students is that students should have the right to know when faculty are using AI for feedback or other forms of engagement, and so I agree that this has been a really great place to have those conversations which might be taking place informally or in departments, but in this interdisciplinary setting, people are able to talk about those issues and share their views.

John: So we always end by asking, and this is something particularly appropriate with AI, what’s next?

Racheal: As we’re wrapping up this project, we are looking to move the learning activities that our faculty will be submitting to Brightspace into an open access repository that my colleague Ed Beck here at SUNY Oneonta is helping to develop, not only for our grant, but for another IITG-funded project out of Albany that is generating some K-12 resources. So we are creating that repository space via SUNY Create and after we collect our faculty learning activities on Brightspace, we’ll be guiding them to move revised versions of those activities into the repository. So we also have our presentation at CIT, which is the Conference on Instructional Technology that SUNY convenes every year across the system, we will be presenting at CIT on this project, and by that time, we will have more information on the website and how to access it, so that faculty, not only across SUNY, but across the internet, will be able to view the learning activities that we’ve been talking about today.

John: And we should mention that SUNY Create is basically an instance of a system-wide license for Reclaim Hosting’s domain of your own.

Racheal: And after CIT, we’re hoping to extend this work, maybe in new ways. We’re thinking about ways to bring campuses together for communities of practice. And one thing that’s been really important to me about this project has been the value of working with colleagues in educational development, whether we’re talking about faculty or staff at all of our six campuses, all of whom have brought different skills, knowledge, expertise, passion to this project. We’ve had leadership team members who work in the social sciences and have brought that qualitative research and assessment knowledge to the project. We have created pre- and post-assessment surveys that are helping us collect data on the success of the project, and we couldn’t have done that without Deepa Deshpande, who’s at SUNY Alfred. We also have leadership team members who are bringing expertise in the classroom, that includes Stephanie, John, and Laura Pierie, who is with us from Morrisville, and having that faculty insight from a range of disciplines has really grounded the project in the needs of our participants. We’ve also had Dana Salkowski and David Wolf join us from community colleges, and having their input as directors and leaders in teaching and learning has been really crucial to helping us think about how our project could serve faculty at all of our different campuses that we’ve been working with. So I think that it’s been a really amazing group effort, and I’m so grateful for the leadership work of that entire team. I think together, we’ve been able to really support faculty in some of this work, and I’m excited to think about the collaborations that we might be able to pursue going into next year.

Rebecca: Well, thank you so much for joining us. It’s been a great conversation, and it’s a great project to share with others that they might want to implement something similar on their campuses.

John: Thank you. It’s been great working with both of you and I’m looking forward to future collaborations.

Stephanie: It’s been wonderful chatting with you all.

Racheal: Thank you for having us, and it’s been wonderful talking with you today.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

385. More Than Words

Many students use generative AI tools to complete writing assignments. In this episode, John Warner joins us to discuss what may be lost when they do so. John has twenty years of experience teaching college writing at five different institutions and is the author of 8 books encompassing a wide variety of topics including political humor, short stories, and a novel, including Why They Can’t Write: Killing the Five-Paragraph Essay and Other Necessities. He writes a weekly column on books for the Chicago Tribune and an associated newsletter, The Biblioracle Recommends. John is also a contributing writer to Inside Higher Ed. His most recent book is More than Words: How to Think About Writing in the Age of AI.

Show Notes

Transcript

John: Many students use generative AI tools to complete writing assignments. In this episode, we explore what may be lost when they do so.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is John Warner. He has twenty years of experience teaching college writing at five different institutions and is the author of 8 books encompassing a wide variety of topics including political humor, short stories, and a novel, including Why They Can’t Write: Killing the Five-Paragraph Essay and Other Necessities. John writes a weekly column on books for the Chicago Tribune and an associated newsletter, The Biblioracle Recommends. John is also a contributing writer to Inside Higher Ed. His most recent book is More than Words: How to Think About Writing in the Age of AI. Welcome, John.

John W.: Very glad to be here. I’m a longtime listener and fan.

John: Well, we’ve been a long time reader of your columns and your other work. So we’re very pleased to have you here.

John W.: That’s great.

John: Today’s teas are… are you drinking tea by any chance?

John W.: Not at the moment I was earlier, I have sort of a post-lunch tea to keep me going. And post lunch, it is green tea, breakfast, I’m a black tea guy, and I’m into tea because this is a strange true fact about me. I’ve never had a cup of coffee in my entire life. So I’m only a tea drinker as far as warm caffeinated beverages go.

Rebecca: There we go. A true fan, [LAUGHTER] a true fan, unlike all those pretend tea drinkers [LAUGHTER] who drink coffee all the time on our podcast.

John: …and Diet Coke. [LAUGHTER]

Rebecca: Yeah, and Diet Coke, right.

Rebecca: I have English Afternoon today, although I was drinking a Christmas blend earlier to match our winter blizzard that we’re recording in.

John: Yes, it’s mid February here, but it’s been snowing about a foot or so or more of snow every day for the last week, it seems. So, that is very fitting. And I have an Earl Grey today.

Rebecca: Classic.

John: Yes, from Tea Forte, so it’s in a nice little pyramid.

John W.: Oh, wow.

Rebecca: So John, we’ve invited you here today to discuss your most recent book, which focuses on writing in the age of AI. You begin the book by describing initial reactions to the potential impact of the introduction of ChatGPT on writing. What was your initial response to the release of ChatGPT?

John W.: This may sound strange coming from somebody who teaches writing and has taught writing for a long time. I was concerned, but I was simultaneously excited. I saw it as an opportunity rather than a pure threat, because for many years, and people can go back to my archives at Inside Higher Ed, I’ve been lamenting the way writing is taught, not just in college, but prior to college, as a sort of formulaic exercise in filling in prescribed templates. And my experience as a teacher mostly of first-year college writing was that my students came in not with deficient skills around writing, but cramped attitudes about writing, where it’s not that writing was hard, it’s more that it was boring. They’d done the same thing over and over again, and couldn’t find any sort of intellectual interest or stimulation in writing. And I saw writing as the opposite of this. And so the fact that ChatGPT can churn out a five-paragraph essay in seconds, and it’s highly proficient at these things, to me, was an opportunity for us to examine our priorities around the kind of writing we ask students to do in school, and maybe even more importantly, than the kind of writing they do, the way we assess that writing, the values we bring to how we assess that writing. Has that been borne out in the two plus years since? …not as much as one would like, but that’s one of the reasons why I wrote More than Words. And there actually are some encouraging signs around these conversations, that we recognize both the limits of asking students to write like algorithms, which is what a five-paragraph essay format does, and the limits of the syntax generated by a large language model like ChatGPT, that there is space for humans to still write. I’ll believe this until they drag me off the scene, [LAUGHTER] but I think more people are seeing this. They maybe don’t have the kind of background or experience that allows them to really think deeply about it, but they sense that there’s a problem associated with outsourcing all this stuff to ChatGPT.

John: One of the things you write about in the beginning of your book is how Daniel Herman, in an Atlantic article, argued that ChatGPT would be the end of high school English. And as you’ve already indicated, that’s something that you welcome. Could you talk a little bit more about what’s wrong with writing in the K-12 system, this sort of mechanical system that’s become really standard for the last few decades, I think?

John W.: Yeah, it’s not that I want to not have high school English class. I think we should have more of those. But it would be great if high school English got back to helping students learn to think and relate to written texts, including literature, including whole texts, like books, as opposed to extracts that can be tested against quite easily. And really this is the sort of core of the problem. The standardization of instruction around something like a five-paragraph essay is driven by the standardization of curriculum around demands related to accountability measures that have been a long-term aspect of the so-called school reform movement. In my previous book, Why They Can’t Write, I sort of date it to the A Nation at Risk Report, which came out during the Reagan Administration when I was in middle school, where they were worried about a rising tide of mediocrity in the nation, of which apparently I was a member, because I was in eighth grade, and they were certain that Japan was going to eat our lunch because of the Sony Walkman, basically. We’re not making the Walkman, they are, and so we need to get on with this. This is the Sputnik moment of my generation, and it sort of began what is not ill meaning, right, like the idea that we should be able to measure what students are learning and be able to know where additional interventions should go in terms of schools or districts or students, is all very sound thinking, but it became, over time, an exercise in a concept known as Campbell’s Law, which is essentially a social science precept where once the measurement becomes the goal, it’s no longer a good measurement. And that has happened writ large in education in all kinds of different domains, and writing is one of the sort of most obvious manifestations of this, where we allow the creation of a five-paragraph essay to stand in for writing ability and writing experiences, and it just isn’t. Writing a five-paragraph essay is not the same process as writing something for an audience inside a full rhetorical situation. And so we now have a couple of generations of students who have really, in some cases, only experienced this, by their own testimony. They’ve written these forms over and over. And then they would arrive in my first-year writing class in college, and I would pull the rug out from under them and say, “none of this is useful.” They had been told explicitly, you have to do this so you can do it in college. And then I come in and say, “That’s all wrong.” These disconnects were what motivated Why They Can’t Write, and More than Words is motivated by my concern that now that we have this tool to which these forms can be outsourced, we’re going to forget to actually teach students to write.

Rebecca: So as you’re talking about the evaluation of writing, and earlier, you mentioned the values in writing and how they have changed, maybe need to change [LAUGHTER] again. Can you talk a little bit about what we should be valuing in student writing?

John W.: Yeah, I have long believed, long before ChatGPT showed up, that students should write in genuine rhetorical situations of message, audience and purpose. That audience should not be the teacher unless the teacher is an authentic audience for the thing they’re writing, which could be the case. We could be communicating something to a teacher as an audience, rather than teacher as assessor or grader. I think this is a good idea for a lot of reasons, one of which is students are much more engaged by these challenges than they are when writing for agrade, they’re much more authentic to the work they will do in other classes or outside of school once they graduate. The goal, as I put it in Why They Can’t Write and another book called The Writer’s Practice, is to develop their writing practices, the skills, knowledge, attitudes and habits of mind of writers. These writing practices, they’re then transportable to new and novel writing problems that they’re going to need to solve. It’s just much more flexible. It’s much more useful. And really, what I realized over the course of my teaching career that I wish somebody had told me before I started teaching, was that I had been lucky enough to develop this writing practice, sort of on an ad hoc basis, through the course of my education as an undergraduate, as a rhetoric major, my graduate studies in creative writing and English literature, and then a job at a market research firm where I had to learn how to write things like focus group reports and questionnaires, forms that I had no prior knowledge of, but which I realized were amenable to the same process I used to also write a form that I was not familiar with when I was in graduate school, which was a 20-page explication of a single poem. I didn’t know how to do this, but I could fall back on my writing practice that allowed me to do it, and I realized I had been given this great gift of a writing practice without anybody knowing it. And so my mission became to try to help students see that they could develop a writing practice. So those values are really just around the things writing is: the act of thinking, the act of feeling, the act of communicating. This is going to sound like I’m simultaneously tooting my own horn and of being a little coy about myself, but I did an interview recently where the interviewer called my work groundbreaking, and I laughed. And he asked why I laughed, and I said, there’s no groundbreaking. I’m doing what I was taught in grade school. It’s just that we have gotten away from these values around writing and so that it feels groundbreaking is really ironic to me, given that I think they’re fundamental, but we have to agree at the values level that these things are fundamental, otherwise we will be stuck in the same rut we’ve had, only now we’re letting ChatGPT solve these problems for us, rather than making students write these sorts of formulaic essays.

John: I also went through a school system where there had been no discussion of a five-paragraph essay. I didn’t run into that until some colleagues started talking about how students were not very good at writing these five-paragraph essays. And I wondered where that whole concept came from, because I remember when I was moving through, a little bit earlier than either of you were, but we had a wide variety of writing assignments, and we also did an awful lot of reading. And one thing that I think has been lost along with a decline in writing or with a shorter form more formulaic writing, is that students are not asked to read as much, and I think that’s also been affecting their ability to write, because engaging with multiple styles of text in reading can be really helpful in developing as a writer yourself.

John W.: It’s absolutely necessary. Reading and writing are inextricably related to each other, like the skill of interpreting reading across all kinds of different genres is the internalizing of the message. Like this is how this is delivered, and then we’re going to do this in turn. We are of a certain age. Those of us who are of a certain age did not encounter the five paragraph essay. My first encounter was in high school when I was taking AP classes, and I was coached to use this form to pass the AP assessment. And my first thought was, “Well, this is dumb, like this is the easiest thing I’m going to do all year is write these AP essays.” And indeed, they were. I dedicate Why They Can’t Write: Killing the Five-Paragraph Essay and Other Necessities to my actual grade school teachers, because in hindsight, I realized how fortunate I was by the accident of my birth and my parents to grow up in a Chicago suburb and go to my neighborhood grade school at a time where the teachers were given the freedom to engage their students, and I still have my fifth grade writing portfolio that my mom saved. If this was a visual podcast I would show it, but it’s called 12 stories, and it’s laminated construction paper cover tied together with hole punches, tied together with yarn binding. And if you look at those artifacts, none of them are a five-paragraph essay. They are all in some way asking me to solve a writing problem in a rhetorical situation. There’s the classic Thanksgiving assignment where you draw a picture of a turkey by tracing your hand and decorate it, and then for each letter of Thanksgiving, like T, H, A, et cetera, you have to give something you’re thankful about. There’s a limerick. There’s a haiku. There is a number of different genres of fiction, science fiction, historical fiction, which required us to read those forms, to understand them, and then produce them in the image of the original form. This one totally cracks me up. There is a classified ad that you would read in a newspaper for my skateboard. And so it’s like “Used skateboard, blue works good. $2 or best offer.” [LAUGHTER] Actually, it doesn’t say “or best offer.” It says “OBO,” which is the shorthand you would use in a classified ad at the time, because you paid by the word. That is rhetorical problem solving, that is internalizing how to think about audience as you write. This was fifth grade, and sometimes I can’t believe what we’ve done to students in the name of helping them try to prepare them for quote, unquote, college and career. We’ve really, in a lot of cases, done the opposite. And I know it doesn’t need to be this way, because I lived through a different time. I know something else used to be done. When I wrote Why They Can’t Write and went back and did all kinds of research, and I found articles dating back to when I was in grade school and middle school about some of the things I would later do that don’t help you learn to write, things like diagramming sentences, which is a valuable exercise in critical thinking, but doesn’t help you learn how to write or research papers. There’s a report from 1984 by this guy, Richard Larson, who wrote about how asking students to do research papers in high school is a fake genre, because you can’t actually ask high schoolers to do a research paper. Now, if the goal is to help us learn to use the library, or the card catalog at the time, that kind of stuff, that’s one thing. But if you think they’re doing research, you’re kidding yourself. And these are the things we need to know so we know what are students actually experiencing when we ask them to write in our classes. We have to think through that. It doesn’t mean that we should abandon whole genres or not have them do things ChatGPT can do, but if we’re going to ask them to do it, it has to be rooted in something meaningful. And to your point about reading, my superpower, which I don’t think of as a superpower, because it’s just been something I’ve been able to do since I was quite young, is reading texts for extended periods of time. It helps me with concentration, helps with my mood, [LAUGHTER] it helps me learn things. And the fact that we have allowed school to get away from that clearly important underlying skill, which is also a practice, which also has habits, which also has attitude, also has knowledge involved, is sort of tragic, and it’s a real heavy lift to get away from it. But it is, I think, like the thing honestly, to tackle if we’re going to help students become the kind of people they wish to be. Like, honestly, I think if we still read whole books, the distractions of cell phones and these things we argue about would be significantly less. Like, don’t get me wrong, I can be distracted by my cell phone like anybody, but I also know how to put my cell phone away and get lost in a writing or reading task for hours at a time, because I have that embedded in my practice. If students have never had that in their practice, and we’re asking them to suddenly engage in this behavior, we’re looking at trouble

Rebecca: Reflecting a little bit on your portfolio, John, and I’m thinking about the one that I also have from fifth grade, except at that time, I think I was also really exploring my design skills, because it’s hard bound…

John W.: Wow.

Rebecca: …and I made a full cover and bound the edges with thread, and all the things,.

John W.: That’s awesome, I mean my fifth grade portfolio is shameful because I had horrible handwriting. I always got “needs improvement on my handwriting” in my report cards and my artistic abilities are non existent, so that part is awful. But to your point, the opportunity to engage in even that relatively small instance of self expression in the context of something you did in school just strikes me as vanishingly rare in today’s culture of schooling, and not for the better in any way.

Rebecca: I was also reflecting on your mention of reading to change your mood, and I’m realizing how much I’ve really read since the beginning of this year, probably for the same reason. You argue in your book that ChatGPT just does text generation and not writing, because writing involves thinking and feelings. How do we convey this to our students and maybe to our colleagues?

John W.: Yeah, the only way to do this is to experience it for yourself. And my hope is somebody teaching in higher education, regardless of discipline, has had the experience of writing as thinking, which I like to talk about as the simultaneous expression of an idea. You’re trying to capture something that’s in your head and put on the page, but it is also the exploration of an idea. So as we are writing, we have this notion that we’re trying to capture, and often that notion shifts as we are writing. That’s the exploration of the idea, and that is by itself, the best and clearest manifestation of critical thinking that I can imagine. In fact, as the way I assessed writing evolved, I would have assignments where that was the only criteria. It was have a thought [LAUGHTER] while you’re writing. So the thing students were writing, it wasn’t that it was unimportant, but what they were actually assessed on was the later reflection about the thought they had while they were writing. Could they articulate to me the experience of thinking they had while they were writing this particular artifact, because I knew once that had happened and they had had that experience, it was something they could return to again and again and again, recognizing that there’s a difference between I’m just sort of going through the motions to get words on a page to satisfy a grade, and writing as an act of thinking and development and not only capturing our thoughts, but developing those thoughts as we go. So that’s the only way it can be done. But if we’re going to do it in school, we have to prioritize it, and in terms of feeling, it’s kind of the same thing. And I write about some of the stuff in my book, where I write about my own instances of being like, as weird as this sounds, emotionally moved by my own writing, because I’m conjuring something for myself and my memories or my life that is meaningful to me, but it can even be something small that upon reflection, we realized this writing was meaningful. I think I have this anecdote in the book; if not, I should have put it in there. But it was during the pandemic, I was grocery shopping and on the floor, and we’re supposed to keep our distance and only go one way in the aisles, it was that period, and on the floor, I saw a yellow post it note that clearly had somebody’s shopping list on it. And I picked it up. It had a bunch of items, but one of them was “cherry Pop Tarts” in all caps with three exclamation points, underlined, and I just had this sort of moment to myself. I said, “those cherry pop tarts are very, very, very important.” And reflecting on it later, I thought of the person who’d sort of put that notion in the list writer’s mind. But I also thought of the list writer, sort of, particularly at that time… we can remember how fraught everything felt… going to the grocery store knowing that the central importance of that mission was to come home with these Cherry Pop Tarts to help soothe the spirit of the person in their household who needed them. Writing a grocery list can be an emotional experience if we attend to it, if we’re aware of it. Now, it’s not like we can go around 24/7 having emotions over everything we write. That would be disturbing and inefficient, but everything we write has that potential, and I think we should be cognizant of it, and every so often we should pause and reflect on it. And to the extent that school does not allow room for that, again, is to wall students off from some important aspect of their intellectual, emotional, social, let’s say even economic development, all these things matter, I think, long term, and so we should give students a chance to experience them in their education. Because why not? What else are we doing with our time? If not that?

John: Along those lines, you’d suggest that pretty much everything that ChatGPT does is essentially like hallucinations. There’s a lot of discussion of hallucinations in getting things wrong, but you suggest that essentially everything it does is just predictive text which is devoid of meaning to the system itself, that it’s missing the human context. One of the things I was thinking about is you also had that example where you mentioned cinnamon rolls, and that ChatGPT could write about cinnamon rolls, but it wouldn’t bring up the same sort of connotations that people would have with that. What are students missing when they rely on generative AI to generate writing or to summarize readings that they’ve been asked to do, which is an increasingly common use of AI tools.

John W.: Yeah, the reading, quote, unquote, I’m going to put air quotes around reading when you talk about ChatGPT or large language models “reading,” but that is a burgeoning and worrisome application of this technology. Now it’s not like we should never ask a large language model to summarize a source for us. That may be useful to us, but we have to be very, very cognizant of the difference between what a large language model does and what humans do. When humans read and humans write, we’re having an experience, and that experience is encoded in memory, it involves our senses, it involves sort of a metacognitive process. Large language models do none of that. They are syntax generators. They can’t think, they can’t feel, they can’t communicate with intention. So if we’re going to outsource that to a syntax generator operating on probabilities, we have to be aware of what is being compromised in the exchange. Now, in some cases, that’s fine. If I’m going to go to Wikipedia for something, because I need some kind of background about a source or an idea or a concept, maybe I’m going to go to ChatGPT instead, although the risk of hallucination can make that fraught, because Wikipedia, while it is the product of flawed human beings, has been checked and edited in ways that make it significantly more reliable than a large language model. You guys know this, reading is fodder for writing. So if we’re going to need to write, ultimately write about something, failing to read it leaves me ill equipped to write about it, because the reading itself is where my ideas, my thoughts are generated. It’s sort of ironic that we call this stuff generative AI, because it really is like reading is truly generative, like it creates the conditions under which our thoughts can occur. And so I’m disturbed by some of it. Like, the deep research release that can supposedly put together a PhD level dissertation is BS. I mean, it really is, the only way you believe that is if you ignore what it means to write a dissertation. No offense, I’ve never done a dissertation. I don’t have a PhD. No offense to the dissertation producers out there, we don’t write dissertations solely to be read. In fact, if somebody wants to publish their dissertation outside of an academic artifact, they have to turn it into a book. Dissertations are fundamentally used to demonstrate an extended engagement with a type of thinking and knowledge creation. And so when I see somebody on social media saying, use this to generate your dissertation’s literature review, my top of my head lifts off and my brain orbits around the sun and comes back, because I cannot believe that this is something anybody would tolerate for a knowledge producing exercise and industry, and yet, I have seen real academics recommend these behaviors and the imperatives of speed and efficiency and productivity override our good sense around what is the meaning behind what we’re trying to actually do. And this is my concern, that speed, efficiency, productivity, these are not educational values, yet, they are the things that are often valued inside of higher education institutional frameworks. So all this stuff needs to be examined if we’re going to make truly productive use of this. If it isn’t just going to be like I can now churn out six articles are necessary. I’m going to turn out 600 now, because I’ve got my large language model cranking this up. I just ask folks, “What is the point of all of that extra stuff?” It’s just slop. We don’t need more slop. We need good, reliable, thoughtful human products. And I said this at the top, this is the gift of this technology to help us understand what humans should be doing, but only if we take advantage of the gift. We could squander it. And in a lot of ways, I think we are squandering it, unfortunately.

Rebecca: So along the lines of productivity and efficiency, you do note in your book about automating parts of the writing process, like moving from cursive writing to typing and word processing are beneficial, including other things in different processes, like calculators, GPS systems, et cetera. There’s ways that technology has certainly made our lives more efficient. Can you talk a little bit about how ChatGPT is essentially, fundamentally different than those other kinds of automations?

John W.: I have to admit from the outset that even those quote, unquote good automations have trade offs. I’ve hugely benefited from these automations in my life and career. Talking earlier about my lousy handwriting, I really could not begin to write until I learned to type on a manual typewriter in quote, unquote, summer school in fifth grade, when my parents needed me out of the house and do something and I took a typing class at the junior high, typing island photography in fifth grade. It was great. I loved it, but I really loved typing because that I could do, and I could type so much faster and with more clarity than I could hand write that I now had a way to capture my thoughts roughly at the speed with which they occurred. So it was a great boon, and the noise the manual typewriter made was just a bonus, because it was super fun [LAUGHTER] to hammer away on that thing and then throw the carriage return across. Nobody’s done it. They should. Go find a manual typewriter somewhere and spend some time doing that. It is a good time. That said, like, I can’t handwrite. I can’t write cursive anymore. I recently had to write a paper check for the first time in months, and had to rewrite it three times in order to get it legible so somebody could actually cash my check. So automation is not unalloyed good without trade offs. That said something like automating the creation of human writing on a page from cursive to typewriting, doesn’t really alter the underlying labor of what’s happening. Now we do know that the typewriter and the word processor have changed the nature of writing. The existence of cut and paste as a technology that didn’t involve actually cutting paper and pasting paper changed the type of writing that we produce. So all this stuff is pretty fascinating, if you start to look at it. The analogy that people have drawn between the calculator and ChatGPT, it’s not that it’s entirely unlike that, it’s different from it in some very important ways. The most important difference is that when a calculator does its mechanical calculations versus a human doing the calculations, that underlying act is identical. Now a calculator does it differently. It uses sort of these binary calculation tools to do it while humans are using our brain, but the end result is identical through those different mechanisms, and the calculator has the benefit of ,unless we wrongly input a number, of being more accurate. Now, it doesn’t mean we shouldn’t still teach addition, subtraction and long division and this kind of stuff. We still have to be able to think about numbers in a way that makes them sensible to us. The difference though, is that a calculator is a labor-saving device for something that we would do not as well through our own calculation. ChatGPT, as a syntax generator, is not doing the same thing as human writing. Human writing is thinking, feeling, communicating. So when we outsource the production of text to a syntax generator, we are doing something fundamentally different in the production of that text, and that difference sands away the most interesting features of writing, which is the demonstration of a sort of spiky human unique intelligence, which is the thing that ultimately gets us interested in reading, particular writers write their unique intelligence. The misfortune of school is that we forgot this in how we assign and assess student writing, we’ve asked them to take away their unique intelligence, what they present to us, so the large language model output looks a lot like what gets good grades in school or has gotten good grades in school. But this is just the opportunity of resetting those values and keeping in mind these differences in the underlying labor is important, because again, if we want students to learn certain things, like writing, like critical thinking, like active reading, we have to have them do it. If they don’t do it, they can’t learn it. We learn through experiences, and if we outsource some critical part of that experience to a large language model, we are denying students the experiences to learn it. One of the early applications of large language models, people enthused about it, so I’m going to use it to do a first draft, and then I will edit the draft. This is ass backwards. [LAUGHTER] The first draft is where all the thinking happens. That’s the stuff. And so you can’t edit something using your thoughts that was not produced through thinking. We need to keep all this stuff in mind as we do it. Otherwise, we’re really going to spin our wheels around how this technology can be genuinely useful, rather than simply as a substitute for the stuff that we should be doing for ourselves, particularly in learning contexts. I can’t say this enough, like people write me like, “I’ll use this for the marketing copy I write for my business, and it’s fine.” I’m like, “Great, great. That’s fine, and that’s a great labor-saving device, but you have to remember all of the stuff you, the adult employed human, had to experience and learn, the skills, knowledge, attitudes and habits of mind in order to look at that output and say, ‘This is good,’ that can only be done through writing that cannot be done through becoming a prompt engineer with large language models.” I get heated up about this stuff, I’m sorry. [LAUGHTER]

Rebecca: You’re talking about the sound of the typewriter. Just recently, last week, my mom sent me a little letter that I had written to Santa when I was little, and I had asked for a typewriter. [LAUGHTER]

John W.: We’re gonna sound like a bunch of fogies here, but my work set up at my home office, I have a laptop and I have a large monitor, and because I like two screens, and I actually ordered a special Bluetooth keyboard for my laptop that was designed to purposely make noise, more like an electric than a manual, because a manual that really would be something, it was just a noisy mechanical keyboard. It was very satisfying to use it. I actually broke it because I worked it too hard, and the company went out of business. Maybe it wasn’t a great product, but I loved it. And loved it, and I had it while I was writing the book, and my wife would know when I had a good day by the sounds that had been coming from my office, [LAUGHTER] because it was like, clack, clack, clack, clack, clack and know if I had a bad day if it was mostly silent. I mean, this is real, like the experiences of our lives are our lives, and to the extent that we’ve sanded those experiences out of learning in education contexts is just kind of tragic and unnecessary, has no relationship to how much students learn or the value of their learning. In fact, it’s the opposite. And so I’m a true believer in this stuff.

John: Going back along the same line, I also, throughout my elementary school, got comments about my handwriting, and in fact, that continued into the next generation, because when my son’s teacher, and I think it was fourth or fifth grade, was complaining about his handwriting being illegible, apparently, he brought in something that he found that I had written at some point, and he said, “See, it’s genetic…” [LAUGHTER] but I benefit a lot, too, from those early automations. Getting a typewriter was a tremendous improvement in the ability to express myself more clearly. But then when word processing came out, it changed the way in which I wrote. Before that, I used to have very elaborate outlines and organized things very carefully, because there’s just so much you can do with Wite-Out and with correction tape. And my writing was much more efficient when I had the ability to move things around and to rephrase things and edit it before it finally was produced as a more final version. And I was at Stony Brook at the time, where dissertations had to be done on a typewriter, and I wasn’t about to do this was like 180 pages or so with all these math symbols. So I had a typewriter with a daisy wheel and an RS 232 interface from my computer, and I connected that to my computer to print that on a typewriter. It took 12 hours because I had to keep swapping the daisy wheel every time there were Greek symbols, but I certainly wasn’t going to go through and having to type it page by page, and the ability to edit using Word Processing made it so much more efficient. But as you note, students are not getting the same sort of efficiencies when they rely on ChatGPT and effective writing takes a lot of time and effort and thought and critical thinking, and we’re living in a time where there’s so much fake news out there, or distortions of reality, and so many conspiracy theories and critical thinking skills, I think, are perhaps more important than they have been in the more recent past. How can we help students develop those critical thinking skills when they do rely to such a large extent on generative AI to get summaries of content.

John W.: It’s a challenge. It’s an interesting challenge, and I’m gonna turn listeners on to some fascinating experiments being done by a guy named Mike Caulfield, who is training large language models to become essentially fact checkers, particularly around images, because they’re very good at this, because it can compare image features in ways that humans can’t. But the reason Mike can do this is because he has a career of thinking about the problem of fact checking, of how we think about information and how we check these things. I really recommend people checking out his newsletter, because his experiments are fascinating. Still, the only way to become critical thinkers is to require students to do a lot of thinking. The absence of thinking from large language model outputs should be sufficient warning for us to be wary of letting students use them to bypass parts of the process that may require thinking. Ferreting out misinformation is a practice. In fact, Mike Caulfield has a previous book called Verified, where he talks about a SIFT method for helping assess the veracity of online information. Once you get this method, you can employ it to help you understand the world, and this is sort of my pitch to students around writing, is that if they can develop the writing practices, if they can develop their powers of observation, drawing inferences from observations, and then drawing analytical conclusions from those inferences, they have a significantly increased ability to just go through the world, seeing, reading encountering things, and having that detector that allows them to say this thing doesn’t make sense. But it is something that has to be practiced and experienced to get good at. And writing is a kind of unique way of being able to do this, because it should require students to turn all of those senses on, but again, we have to make them do it when the assignment comes. And then we have to assess the assignment according to criteria that values those experiences. More and more as I wrote the book and as I think about the problems, it’s not so much the assignments themselves, although there are some assignments that are just sort of like we’re gonna have to put those away. Those are too vulnerable to the technology, and they’re too uninteresting to students to ask them to do these things, and so we have to do something else. But if the assignment is intrinsically interesting and engaging and gets students thinking deeply about these things, we then have to use assessment criteria that values that, as opposed to surface level features that so much of student writing is attended to. And some of this also involves convincing students that this is what we’re going to do. The first part of my first-year writing courses was always sort of fraught, because students would want to give me what they’d done before, and they were very conditioned to producing what I called pseudo-academic BS, which was a sort of high falutin fancy-languaged version of what they thought a smart student who had sort of read the assignment would sound like on paper, and they had been conditioned to this performance of good, smart studentness. And one of my rubric criteria was absence of pseudo-academic BS. And as soon as I saw it, I would just turn it back to the student and say, this is pseudo-academic BS, you need to revise, and they hated me for it, absolutely hated me for it, at least at the start. Some at the end also, but at the end, some would come around. But this is a reorienting around the values that we attach to learning, and if students have not been exposed to those values, or if those things had not been valued in their school context before, we have to help them reorient to them and not get overly frustrated, as I sometimes would when they’re not there already. When they enter my class and they’re not doing the stuff I need them to do, I have to remember that in a lot of cases, this was like new, it was unfamiliar. I think I used this analogy in my book, I don’t know if I did or not, but it was like the beagles that had only been raised in a lab and had only experienced like a cage, and then they took them to a park, and they opened the cage doors and say, “Run, be free, beagles,” and the beagles are afraid of the grass because it’s foreign. Maybe I’m going to regret reducing students to beagles. It’s an analogy, people. But the idea that we know the grass is the great thing, and that the beagles who are in the cages are going to love this grass as soon as they just get to experience it, but they might be afraid of it because it’s unfamiliar, and because the system within which they’ve been working has been rewarding these non-grass-like behaviors, they’ve been rewarding staying in the cage. They’ve been rewarding hitting the prescriptions. And so part of my work was always that reorientation, like “It’s okay, come out of the cage. There’s good stuff out here. You might get muddy, you might step in dog crap. There might be something that doesn’t go well, but we’re going to be okay, because ultimately on the other side of this is a genuine experience of learning in life, and you will appreciate this.” Again, no teacher bats 1000 with their students in the course of the semester. And I think, like a lot of instructors, I can remember every horrible end of semester comment a student ever wrote about me. But overall, the batting average was pretty good because students are thirsty with us. They’ve been really wishing for it, and as soon as they’re invited to do it, most students will embrace it pretty thoroughly.

Rebecca: We spent most of our time today talking a lot about students and learning, related to ChatGPT, the ethics around that, but you’ve also discussed several other ethical concerns related to the training and operation of generative AI, including copyright, exploitation of precarious workers, environmental issues, etc. Can you talk a little bit more about these concerns?

John W.: Yeah, it’s important to recall that these models are trained significantly on stolen merchandise, including things I’m willing to bet all of us in this virtual podcast room have had our work sucked into these models and is being regurgitated back to us without notice, without permission, without compensation. It is not impossible to create a system where the origins of these models is compensated. It’s not impossible. It’s incredibly unlikely to happen, though. So it’s not that I think we should sit around being sort of “Ah, whatever that ship has sailed.” I think we actually should keep it in mind every time we use it, that this stuff basically exists because of the use of material without permission and without compensation. Now our judicial system is going to work through this in terms of our actual copyright law, and I think it’s probably going to turn out in favor of tech companies. But I think this is primarily because our copyright law was written at a time where the thought of this sort of technology was not even in science fiction. So we can’t confuse the strict outlines of copyright law with justice and what is right. The environmental impacts of this technology is unknown. It could be incredibly severe. We already know that it’s using a lot of extra energy and a lot of extra water in the data centers that are online in order to train and deploy this technology. There’s a lot of arguments subsequent to me writing the book around like, well, it’s not as much as like watching a single series on Netflix or something like that. These comparisons I find interesting. But again, anytime we’re going to do this sort of stuff, we have to think, what is the cost to this benefit? If ChatGPT really is just kind of a novelty, then should we be using energy and draining aquifers in order to power these servers? And the exploited labor, the labor in third world countries that is used to train this is again, it’s the story of our Westerners’ use of tech over and over again. It’s nothing new, but it’s an intensified version of it, and it’s something we should be mindful of. We should be thinking about the ethical concerns of all technology we use. Like I have an iPhone. It has minerals that were mined under unconscionable conditions. I have Netflix that is using power that is cooling a server that allows that sort of stuff to happen. I still do these things. I also know that one of the biggest contributions to global warming is the beef industry. I still eat. I have to be able to recognize these harms even while I do these things. I don’t require anyone to like put on the hairshirt and repent for existing in modern society. We are where we are, but I think we do have an obligation to understand the truth behind these potential harms around our actions, just to be reasonable human beings in the world, like I’m about to go get on a plane to do some various events, to talk at institutions and promote the book. None of that is necessary. I could do everything over Zoom. I’m going to do it anyway, because I think it actually has a little better impact than it does over Zoom. And it’s an enjoyable part of my life. Being able to visit an institution and talk about these things, or work with faculty around how we adjust to this technology is fulfilling. And so I have to be aware of the balance between the life I want to live and the trade-offs and the costs to society. So it’s really just a matter of being thoughtful about in the same way we have to be thoughtful around student learning when we’re deploying this technology, we have to be thoughtful about these impacts. Otherwise, I think, as the technology develops, as greater demands are made around this technology, both funding, the money these tech companies want to develop something like artificial general intelligence or to deploy this technology in our institutions, often without our permission, without asking. We should be aware of these things so that we can use them as frames for thinking about how we want to use this technology in our lives, as individuals and as educators, and as educational institutions. We should never forget them.

John: So we’ve talked a lot about the challenges and the potential concerns associated with the use of generative AI, are there some uses that may actually benefit students’ learning and maybe even benefit students’ writing, perhaps doing some proofreading using AI tools, or perhaps evaluating a text that the students created to find some weaknesses in their arguments. Might there be some benefit for those types of activities?

John W.: Anything is possible, and there’s a lot of people doing some very interesting experimenting around the intersection of learning using these tools and writing, and I encourage all of that experimenting, even as I am not the person who is going to do it, because I don’t like to think of myself as a fundamentalist, but I do believe that this technology is primarily useful to people with well developed professional practices and to introduce it at times before those things are there, I’m dubious. That said, I’m encouraging of class contexts that are transparent and open… allow students to experiment with the technology as part of developing their practices. I think that’s all good, as long as it’s done mindfully and with important end goals around learning in mind. This is not my practice. I found this stuff increasingly less useful as I worked on the book. I set out to use it as I was working on it as a sort of demonstration to myself and to see what would happen. Over time, it was less and less interesting, because I realized that the predictive outputs of a large language model were not generative to me. I needed that unique intelligence in order to help me do my work. I do have some what I think are hard and fast lines around the use of this technology. If a student wants to sort of curiously and voluntarily engage with this technology for their writing, I think that’s great. I would never personally have a student write a piece for my course that was only going to be assessed and evaluated by a large language model, and that’s a betrayal of the relationship we should be establishing with our students. I don’t know why I would do it. If I’m teaching writing, the most important input I have for my teaching is my students’ writing. So the idea that I’m going to let a large language model grade that instead of me is sort of nonsensical. But even to ask students to use it to get feedback on their writing by my own urging, I don’t know. I mean, this is a probability machine giving feedback on writing. If I’m going to do that, I need to prove to myself that that output is more useful to learning, not to the outcome, not to the grade, but to learning, than, say, peer review or asking a student to read the work out loud to themselves, or do something that I call a reverse outline, which is just a technique for looking at something you’ve written, write a question based outline for each paragraph. What audience question is being answered by this paragraph, and does that reverse outline make sense? I’ve got a million things that students could and should do with their writing that are not interacting with a large language model in order to improve their writing. So personally, I would have to exhaust those before I would turn to them. But I’m not against experimentation. I know folks who are doing that. It’s not for me, as long as the experiment is predicated on learning, not just improving a student’s output for the purpose of a surface-level criteria grade, I’m for it, and I’m eager to see them, and people share them with me, and sometimes I’m taken aback, and I’ll have to eat some of my words and think, “Okay, maybe I still wouldn’t do this, but I get why this person is doing it.” But again, that’s just a mindful teaching practice; that is using technology that’s available to enhance the experiences of their students around learning. That’s the work. So I can’t criticize or gainsay that. When it is a substitute for what teachers should be doing, when it allows for an end run around our own labor, I think it’s dangerous. And I would just say, since instructional faculty are overwhelmingly your audience, I assume, we should be very, very, very, very, very, 12 more verys in there, careful about outsourcing our instructional labor to automation, because it is a encouragement for those who would like to automate faculty labor to do so, and for those of us who have worked at less than fully resourced institutions, we know what this sort of pressure is like. It’s constant, and the decisions that are made often have nothing to do with the quality of learning or instruction. And so if we’re going to allow these things to, like write our syllabi or grade our assignments or give peer student feedback or something like that, I would just say, be careful, because what you may be demonstrating is, for the purposes of the institution, to collect tuition and convey credits and credentials, you may be proving you’re not necessary. Now there’s no way that that process is the same as a good teacher, human interacting with other students, but that’s not a requirement for an institution to give grades and confer credentials, and so we have to be aware of these things. I’m not saying we have to be Luddites ready to destroy the looms owned by the factory, but we should understand the lessons of the Luddites and respect the actions of the Luddites, which were not just to protect their labor, but protect the quality of the goods that were produced for the public. The mass-produced cloths were not as good as what they did. The public ultimately decided mass production was more desirable than employing hand weavers, but we’re talking about education here, and when is mass produced education proved superior to individual, bespoke relationships?

Rebecca: Seems like a good note to move to our last question, [LAUGHTER] which is we always end by asking, what’s next?

John W.: That’s a good question. So whenever I write a book, I promise my wife to not write another book for a year, because, as much as I love writing a book, and I love writing books, because of what we’ve been talking about, like the depth of immersion that I have to do to write a book is just steeply, deeply pleasurable, but it also makes me checked out of my life in other ways. Now, fortunately, this book was written quite quickly relative to my other books, so that was good. But to answer your question, I’ve become increasingly interested in the idea of apprenticeship and how we learn to do what we do. I’m gonna guess both you guys either had formal apprenticeship programs to l earn to teach you had, like, literal mentors, or I, unfortunately didn’t have that. But what I did have was a lot of other people who are teaching around me who I would go bug and say, “This isn’t working. What do you do? [LAUGHTER] What can I do differently? Because here’s what’s happening, and it is very, very unsatisfactory for everybody involved in the equation, what else can we do?” And so I cobbled together a sort of apprenticeship model to help me learn how to teach and reading. I read a bunch of books. I’ll still never forget reading Ken Bain’s book What the Best College Teachers Do, where it was literally like the scales falling from my eyes. I’m like, “Oh my God, there’s people who care about this stuff out there, and I can read about it, and I’m going to start thinking about it too.” It was literally life changing. That book was a form of apprenticeship to better learn how to do the work of teaching. And the reality is, huge swaths of our society and work are based on an apprenticeship model without us recognizing it. And my concern over this technology is we’re going to lose it, either because we decide to outsource the work of the apprentice to the technology in the interests of increased efficiency and lowered cost, or when we ask apprentice workers to do the work, we will not be requiring them to have the kinds of experiences that allow them to become the mentor. If an associate lawyer is writing a brief using large language models, they will be missing out on the kinds of thinking that will make them the mentor lawyer one day. My older brother’s a lawyer, and he’s concerned about this stuff, like “How do I make me?” And what makes him is all the thinking he’s done over the course of his career to do this. So I’m fascinated by that. So I’m thinking of some kind of project around that, maybe not a book, maybe a podcast, so I can tell my wife I’m not writing a book, I’m doing a podcast.[LAUGHTER] But that’s what’s interesting me right now. That’s sort of, how do we learn to be what we are, and what does that entail, and how does this technology threaten those things, and what will happen if we allow it to dissolve those relationships in our workplace?

John: Well, this conversation and your book has provoked an awful lot of thought, and I’ve really enjoyed both listening to you and reading your book, and I’d strongly encourage anyone who is thinking about using generative AI tools to read through your book.

John W.: Oh, it’s my pleasure. I have been a long time listener, and so getting on Tea for Teaching is a career honor that I’ll always be glad for.

Rebecca: Thank you so much.

John W.: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

384. Writing Together

Writing is often perceived as a solitary activity, but this may lead to a sense of isolation. In this episode, Rachael Cayley, Fiona Coll, and Dan Newman join us to discuss the benefits of writing in community.

Rachael is an Associate Professor in the Graduate Centre for Academic Communication at the University of Toronto. Before joining the University of Toronto, she worked as an editor at Oxford University Press. Fiona is an Assistant Professor at the Institute for Studies in Transdisciplinary Engineering Education & Practice and at the Graduate Centre for Academic Communication. Fiona had earlier been one of our colleagues at SUNY-Oswego. Dan is an Assistant Professor in the Department of English and the Director of Graduate Writing Support in the Faculty of Arts and Sciences, also at the University of Toronto. Rachael, Fiona, and Dan  are the editors of Writing Together: Building Social Writing Opportunities for Graduate Students, which was recently released by the University of Michigan Press.



Show Notes

Transcript

John: Writing is often perceived as a solitary activity, but this may lead to a sense of isolation. In this episode, we discuss the benefits of writing in community.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Rachael Cayley, Fiona Coll, and Dan Newman. Rachael is an Associate Professor in the Graduate Centre for Academic Communication at the University of Toronto. Before joining the University of Toronto, she worked as an editor at Oxford University Press. Fiona is an Assistant Professor at the Institute for Studies in Transdisciplinary Engineering Education & Practice and at the Graduate Centre for Academic Communication. Fiona had earlier been one of our colleagues at SUNY-Oswego. Dan is an Assistant Professor in the Department of English and the Director of Graduate Writing Support in the Faculty of Arts and Sciences, also at the University of Toronto. Rachael, Fiona, and Dan are the editors of Writing Together: Building Social Writing Opportunities for Graduate Students, which was recently released by the University of Michigan Press. Welcome back, Fiona and welcome, Rachael and Dan.

Fiona: Thank you for having us.

John: Our teas today are:… Rachael, are you drinking tea?

Rachel: I am. I’m drinking something called Christmas in Paris, which is a mix of chocolate, peppermint, lavender and vanilla.

John: That sounds wonderful. And Fiona?

Fiona: I am drinking a classic. I am drinking Murchie’s Cream of Earl Grey.

John: Nice. And Dan?

Danl: I am not drinking tea. I am drinking water in a mug. It’s weak tea. [LAUGHTER]

Rebecca: There’s one in every bunch. Dan.

John: …at least one. [LAUGHTER] … very weak tea.

Rebecca: Very weak. [LAUGHTER] In honor of talking to Fiona, and it’s been a while, I have gone to my very old favorite, which is English afternoon tea.

John: Very good. And I’m drinking a new tea. It’s a three mint tea, which is peppermint, spearmint, and field mint. And I don’t quite know what field mint is, but it tastes okay. [LAUGHTER]

Rebecca: We’ll have to look into that, John.

John: Yes.

Fiona: You and your adventurous taste buds, John.

Rebecca: Stay tuned for more on field mint in the future. [LAUGHTER] We’ve invited you here today to discuss Writing Together. Can you talk a little bit about the origins of the book?

Danl: Sure. I’ll get us started. We’ve been working together, Rachael, Fiona, and I, in different capacities for a long time, and we’ve all ended up working in graduate writing support and teaching. And so we’ve been chatting about this, and we started off with some anecdotes, and the way I’ve been reconstructing the history of this project is starting with a conversation when we were talking about noticeably skewed gender ratios in a lot of our writing support activities, particularly those that were kind of group based, so like writing boot camps or feedback groups or peer review groups, and these groups, anecdotally, seemed to us to be very tilted towards more women than men participating, which wasn’t true for some of the more instructional things that we were doing. And we started discussing this as something interesting and neat and potentially something to look into. And then we decided it might be more interesting to shift the conversation away from gender per se towards the question of sociality. What is it about social writing that makes it more or less interesting to different people? And that got us more interested in the pedagogical implications of this and how we can foster sociality to help graduate students become better writers, become happier writers, move through their programs, but also just generally, learn the ropes of research and writing in general.

John: In your introduction to this work, you argue that writing together is useful, both as a form of graduate student pedagogy, but also for professional development purposes. Could you talk a little bit about this argument?

Danl: Yeah, fundamentally, I think the argument is that good graduate pedagogy should also be good professional development, and that encouraging social writing is an important way of realizing that ideal. And a lot of grad programs have professional development activities. And I think grad programs, in general, from what I can see, are getting better at this, but generally, these tend to be framed as separate from writing, especially separate from dissertation writing, which for a lot of grad students is kind of the main task. But dissertation writing is not just a way to complete your program, it’s also kind of a student’s, in a lot of cases, the first major entry into professional research writing. And so the idea that we could sort of approach both holistically is kind of what we’re getting at with that idea of good pedagogy is also good professional development. When I started working with grad students closely, I was struck by a disconnect between how they view writing and how my faculty peers and I write, and I’m simplifying horribly here and generalizing. But anyway, students would talk about withdrawing to their apartments for months trying to perfect a thesis chapter before showing it to anyone. And meanwhile, my colleagues and academic friends and I are sending each other emails saying, like, “Does this paragraph make sense? Here’s a really rough draft. What do you think of this?” Basically asking for feedback constantly. And again, not everyone does this, but this is my experience. Why are so many of our students not taking advantage of their communities, I guess is one of the questions behind this, too, and I think part of the answer, at least, is that they’re not taught to. And I see this as an opportunity for graduate pedagogy and programs and maybe writing together more like their profs do, could help them professionalize even as it helps them navigate their grad school experience.

Fiona: Another angle, maybe, to dig into a tiny bit is just the idea that writing is something that can be taught. I think one of the trickiest parts about that transition to grad school is that strange set of assumptions by which grad students are made to feel as though they should already know how to write things they’ve never written before. So grad students arrive and are sort of faced with these quite major writing deliverables that are different in scale and in scope than anything they’ve written before. And they somehow absorb this sense that if they don’t know how to do it, or struggle in any way, that there’s something wrong with them, that they don’t belong, or there’s some sort of grievous lack or problem. And so to treat writing as something that is a lifelong pursuit in many ways, but more specifically, that there are a set of skills that are unique or uniquely developed in graduate school around research writing, that there should be no shame in needing to learn these and that that learning process at the graduate level becomes richer and more sustained when it’s done in, and as part of, the development of a community, a disciplinary community. Rachael Cayley has a wonderful line that I steal as often as I can when talking with graduate students, which is when graduate students write a dissertation, they’re actually producing two things. Yes, they’re writing the dissertation itself, but they’re also developing themselves as researchers, so they change over the course of writing that giant piece of work, and there are skills and strategies, there are things to learn all along the way.

Rebecca: The focus of this collection of essays is on experiences in which sociality is an integral component of the process. What are the primary benefits of building this social component into graduate writing processes?

Rachel: I think the benefits of sociality in writing are similar to the benefits of sociality more generally. We are social beings who struggle with too much isolation, who naturally learn well in community with peers and have trouble keeping intentions when we’re not making them with other people. And I think you can hear in those three things that they would be particularly acute for writing, which is often institutionally organized and coded as a solitary activity, as Dan said. Even though many faculty have realized that that’s not the best model, students certainly think that that’s how they ought to be doing it. But we know that graduate students do much better when they’re not alone, when a day of writing is a day that has engagement in it. We know also that peer learning around not just writing, but also writing productivity and processes can be really, really helpful. By working with others, they can learn that everyone feels isolated, that everyone thinks their writing is uniquely terrible, that everyone thinks they’re probably not cut out for this, based on their writing difficulties. So to learn that their range of productivity challenges are within the sort of normal range is really helpful. And we know that setting public accountability goals is intensely useful, especially during the period of thesis writing that requires a degree of consistent self discipline that I think exceeds the bounds of what most people naturally have. And so given that we are social in that way, and writing particularly brings out the worst of us in our capacities to go looking for the sociality that we naturally need. We thought it was so important to make it central.

John: So by building more sociality into the process, does this provide a more inclusive environment, so that people won’t feel as isolated, and perhaps we won’t lose as many people in graduate studies, as we often do in many departments?

Fiona: I think the answer is absolutely, there’s an inclusivity piece to this puzzle. There’s an equity piece to this puzzle in many ways. As you’ve noted, there’s those feelings of isolation or those feelings of alienation are not evenly distributed across all demographics who arrive at grad school. And so there’s just a fundamental gain by creating places or spaces where that sense of loneliness or alienation can be dissolved away. There’s also a hidden curriculum piece here, again, as with all kinds of hidden curricula, it’s not an intentional obfuscation of how to learn to write, what it means to write in a disciplinary way, but these are all things that become invisible so quickly to the members of those disciplines that [LAUGHTER] it’s hard to remember what it felt like to not know the things you know. And so these writing groups are a fantastic place to explore and reveal and make explicit those expectations. There’s also some fascinating research that’s been done on the link between confidence in academic writing skills and that sense of disciplinary belonging. That connection is especially important for students who belong to historically underrepresented demographics in grad school, and indeed, several of the programs that we’ve included in this book collection feature writing groups that are organized not necessarily around disciplines or around levels of development, but are organized around identity or interest groups. And indeed, the University of Toronto, to give one example, has a writing accountability group that is co-led with our accessibility services organization. And so there are ways that these groups can be really specifically attuned to the needs of particular students, particular groups, in positive and inclusive and equity raising ways.

Rebecca: There’s a number of contributors to this volume. Can you talk a little bit about how you selected your contributors?

Rachel: Yeah, the most noteworthy part of the selection process was just the significant enthusiasm for this project. And we ended up with 14 chapters, but we started with more than 50 proposals, and we thought that enthusiasm probably stemmed from two things. First of all, just excitement about social writing opportunities, like we were really happy to learn that so many people were eager to talk about these. I think there was also some interest generated by the unusual nature of this call, which derives from the unusual nature of this book series. So the book series of which this is the first volume is a partnership between the Consortium on Graduate Communication and the University of Michigan Press. And this series, overall, seeks to provide practitioners with ideas that bridge the gap between theory and practice, so really meet practitioners where they are in terms of how they’re actually doing the work. As we said in our call for proposals, we were looking for “how we” accounts so not looking to tell people how to do things, but really asking people how they did them so others could learn from them. And I think there is a real hunger for people to talk about our pedagogical work in ways that are outside the bounds of traditional scholarly research publishing. And I think this book gave people a really easy, direct opportunity to talk about what they’re doing in the classroom, the things that they really love doing. So I think that led to some of the enthusiasm as well. So our actual selection process was enjoyable because we had so many great submissions, and also, of course, difficult for the same reason, ultimately, we chose these 14 because they offered us a wide spectrum of different types of initiatives across North American institutions, so we largely chose them so that we had different models that people could read about when deciding how they wanted to learn from this book.

Fiona: I will just add that one of the things I really appreciated about the 14 chapters that we did include is that they point towards just how collaborative the construction of these programs is. So often, these programs emerged out of an obvious student need in some way, and students sort of have driven the development of these programs in many ways, in ways that I find inspiring and really, really encouraging. So no matter the exact origin of the program or the way that the program is run, this sort of combination of writing experts or teachers of some kind, the grad students who are looking for that support, and then everybody else who’s involved in bringing these things to life, it’s people working together to write together.

Danl: And in some cases, it’s actually graduate students who are taking the initiative without support from the institution, which is encouraging to see. And that’s the kind of thing that I’m hoping other programs can generate, as well, models that students can use to do these things on their own.

John: And in this volume, you have quite a wide variety of diverse examples of writing experiences. Could you each share just a few examples of the experiences that are discussed in this volume?

Danl: Sure, I’m going to start. I’m going to talk about peer reviewing, which is something that a lot of the contributors talk about as part of their programs, but it’s also one of my favorite activities. It’s not revolutionary or anything. I think people have been doing peer reviews for a long time. I do them a lot, partly because it is a great way for students to get feedback and to give feedback to,o which I’ll talk about in a second. If framed correctly, it’s a good way to have students practice showing their writing to other people in a way that can be less intimidating than, say, sending it to your professor, your supervisor, advisor. So I do use it as practice for that which makes social writing easier to do in the future, ideally. I’m particularly interested, though, in the way that giving and getting feedback teaches people to become better writers, particularly giving feedback. Obviously, getting feedback is a great way to learn how to write better, but I found from working these groups where we do a lot of feedback, that the students who are most effective in giving really relevant feedback, it feeds back on their own writing in ways that are interesting. I feel like that’s something I would love to study more somehow.

Fiona: I will offer another example of a social writing initiative that I run here at the University of Toronto, and it’s called Mondays We Write!, and it is another example of just how rich and sort of widely ranging the sorts of conversations we have about writing can be in these social circumstances. So in many ways, Mondays We Write! is a very straightforward writing accountability exercise. So from nine until 12 every morning, I hop on Zoom and students are encouraged to come and drop in essentially any Monday they want, for as long as they want. The structure of the morning is such that I often start with some small piece of teaching, let’s say, some tip or some concept or some principle that I encourage students to work into their writing for the day if it fits with where they’re at, then students give a sense of what they’d like to work on that morning, and we embark on a series of sort of quiet writing intervals, where we’re all on Zoom together, but working on individual things, enforced breaks, which are good and healthy for all kinds of reasons. In those breaks, we have quick conversations if needed, we do some troubleshooting, and the morning continues. And what happens is that the students who attend regularly start to gain more and more confidence in their own ability to navigate: A. their own writing processes, but also B. their ability to give helpful feedback or ideas to their peers and to watch that sort of community building is what it is, that sort of building of trust, sharing of knowledge and experience and resources, often in disciplinary-specific ways, is tremendous. The other kind of cool aspect of this Mondays We Write! is that, because it’s on Zoom, if there is a student who’s running into a particular issue that’s a little more specific to their own situation than it is relevant to the group as a whole, that student and I can pop out into a breakout room and have a one-on-one conversation in the actual moment when they’re feeling that writing struggle. And so it’s this lovely model of sort of how sociability can enable a sort of writing practice to develop. But then our structure allows for these sort of dipping in and out of individual challenges as they come.

Danl: Something Fiona said really struck me, and it’s very related to what I just said before about peer review, but also to the question about pedagogy and professional development. This kind of feedback that you’re observing, how students become better at giving feedback, that clearly is part of a professional development for an academic who will eventually be giving feedback to their students, and is just such a great example, I think, of what it looks like in practice for good pedagogy to also be good professionalization.

Fiona: And I think it holds true for those who go on to professions other than academia. Most people are going to be in a position of needing to give feedback, often on communication related deliverables of some kind, especially in my world, which is heavily skewed towards engineering these days. And so you know that feedback process is tricky. Despite our sophisticated ways of being in the world, figuring out how to be constructive is much harder than it sounds. And so to offer students an opportunity to practice, to learn how to do this in a genuinely productive, constructive way is pretty fantastic.

Rachel: Then you’ll see that we’ve all chosen to talk about initiatives we’re doing ourselves, rather than choose favorites from the book, because we love all the chapters in the book and all the different things that they’re doing. I wanted to talk briefly about a sort of minimalist program that I run called the Doctoral Completion Cohort. It’s for people in their final year from any discipline, and it’s really simple. All I’m asking them to do is to come for half an hour on Monday mornings and check in. And so we start each week together. These are people who are really pressed to finish, like this is their core activity. And I usually give a short presentation on some emergent topic to do with productivity or revision, the things they’re thinking about most, and then the focus is with them finding co-writing times with each other, not running through me. So it’s designed to build that skill of reaching out to people and saying, I like to write on Tuesday mornings. Is anyone else free? Some of them are doing them in person, at different places. Some of them are running them on Zoom. Some people are in different time zones, so they’re using WhatsApp and from different countries, so they take on that part of the responsibility. And I’m just there to provide a Quercus or Canvas site that provides a lot of resources and a venue for them to make those connections. But it’s creating some of that sense that part of what you have to do if you want to have social writing opportunities is reach out to people and ask them to write with you.

Rebecca: Can you talk a little bit about the kinds of assessment that have been involved in these kinds of social writing opportunities and the kind of feedback that students have provided about this kind of work?

Fiona: For sure, we asked all the contributors to this volume to provide us with three important things in their very short, very readable chapters. The first, of course, was details about exactly what motivated the program that they described. We asked them to explain, of course, the model of facilitation that they used. And the third thing was, we asked them to explain how they evaluated or assessed each of their programs, because we know how important it is to be able to evaluate in some way or assess what’s working or what’s not working in any program, but especially one that deals with skills of such complexity as are involved in writing, but also as a kind of argument, an argument in favor of the value of this sort of work. And so one of the fascinating things that emerged out of this book collection was a sense that there were two distinct assessment modes or directions that didn’t always point in an aligned fashion, let’s say, and one of those threads of assessment was from a kind of pedagogical perspective. So of course, everyone running these programs was interested in how it was working for students, how students could themselves understand the developments or changes that were happening. And everyone is interested in iterating and improving these, improving the reach, improving the effectiveness of these programs. But a second line of assessment or evaluation was essentially proving the worth of these slightly odd in some way shaped programs to administrators or the sources of funding, and often those two are not exactly aligned, as I said, in the sense that to explain what happens in these complex exchanges, these really rich, productive exchanges that contribute not only to the writing skill development of a student, but to the sort of flourishing of that student in their disciplinary identities, that growing sense of belonging to a discipline, that increasing sense of confidence, that ability to sort of think on the page in more and more successful ways for the student, that is not an easy thing to convey In a spreadsheet. And so several of our chapters described that tension. Now what students have reported really consistently across the board, across these 14 chapters, is inevitably a sense of surprise at how powerful these communal or community-based social writing experiences were. So students just really describe feeling kind of relief, feeling a sense of camaraderie that they didn’t know could exist around the writing experience, just feeling that their learning was richer and they felt more motivated. And so we know that many students respond very positively to this sort of structured sense of community building. And one thing I’ll add is that one of the ways of evaluating or assessing these social writing initiatives is, of course, to have students reflect in qualitative ways back on their writing experience as they go. Many of our programs in the book do so at various points in the program. So there’s a gradual over time awareness of how things are developing for students, and in fact, enabling students themselves to recognize what’s changing about their relationship to writing is an incredibly powerful kind of assessment to be able to make.

John: In most graduate programs, individuals tend to write on their own. I know in economics, when I first started, quite a while back, co-authored papers were downgraded a bit in rankings. Today, most research in most disciplines tend to have two or more authors. And so I think this idea of having people share writing experiences together puts people in a much better preparation for the types of careers that they’re likely to be having later on. In this book, you’re focusing primarily on graduate programs in writing. But is this something that, perhaps, should be done in all graduate programs to some extent?

Danl: Should, is a tricky word. I think there are undeniable differences in how even something as capacious as writing together might look in economics versus sociology versus philosophy or math. I mean, some fields really do tend to work around more of a single author mode, and I think that’s true of the fields that Rachael, Fiona, and I grew up in in the humanities. I know people in math typically write their papers alone, whereas social scientists and a lot of STEM fields co-author together. And so I think one of the things that has come out of this book is trying to develop a broader understanding of what sociality means, as opposed to say collaborative writing, which is a type of sociality, or co-authorship, which is a type of sociality, but we’re trying to think of ways that are adaptable to different disciplinary practices. So writing in the same room as someone is a kind of diffuse type of sociality, and it can be deployed intentionally in ways that can be helpful. Even writing on Zoom with a group, which Fiona does and I do as well, is sociality, even if we’re each working on our own thing and not talking. Just for myself, I get a huge amount of momentum just from the false impression that all these students are looking at me when I’m writing. I better not look at the news, because they’ll judge me. But I think thinking about the ways in which writing together obtains in different contexts is something that we should think about when you’re developing programs, even when, for example, if you’re a faculty advisor for students, what will writing together look like for us or for the student in my lab, or whatever? I will say just as I’m wrapping up, that the introduction for the book, Writing Together, was the first time I’ve actually written together with other writers, and I’ve published two and three author papers before, but we separated the task of writing. In this case, we really wrote together, and it was great. It’s very unusual for our fields, but I don’t think it could be the norm, or certainly I don’t think it should be the norm.

Rachel: And certainly most graduate students who are writing with their supervisors are still writing alone. They’re being sent off to write the first draft often, and then they’re being corrected in some way, but it isn’t particularly social, often, and so we’re still trying to encourage even the ones who are very much in co-author situations that they might wish to find social writing opportunities that don’t necessarily even have to be with those co-authors. Although, as Dan said, it’s lovely when you can have both, but even when you can’t, you can still find a time to write with other people, your own writing, and develop a lot of richness from that.

Fiona: And I would argue that one of the real benefits of these social writing programs that we describe in the book is how they open up conversations about the writing process. So they help to share a language about the writing process and its stages and its iterations. They help to show what the process looks like. There’s a sort of writing-process literacy that I think is worth opening up and sharing much more widely across programs, especially across graduate programs, and that kind of fluency with ideas, or greater familiarity that with different stages of the writing process can absolutely inform the sorts of literal co-writing or co-authorship scenarios that, John, you point to that are becoming, I think, more and more common, even outside those traditional STEM fields where co-authorship is as prevalent as it is. So I will take a stand [LAUGHTER] and say, I do you think that most graduate programs would be doing a beautiful service to their students to think about opening up and sort of making this writing based awareness a valued and valuable part of that training experience.

Danl: I think there’s also very real ways in which, if students are getting support from their communities, it can change the nature of the support they get from their advisors, who might not have to deal with some of the really rough first stuff. And also when we’re talking about the peer review, I think one of the most striking benefits I’ve seen, which I think relates to what Fiona was just saying, is students realize that the kind of writing that they do that looks so different from the writing that they read when they read an article or a book, they realize that other people’s writing also looks like this. So I think that’s part of the writing process literacy that you’re talking about is really “Okay. I’m not just really bad at this. No one’s good at this.” And maybe talking about good and bad in terms of drafts is not the right language.

Rebecca: Sounds like there’s a little bit of an added benefit too, in both the examples that you’re pointing to, for those that are facilitating to also have some writing time, [LAUGHTER] some accountability for the facilitator too. [LAUGHTER]

Danl: It’s the only time I write. [LAUGHTER]

Rebecca: Now we all know. [LAUGHTER]

John: I know when I was in grad school, I spent an awful lot of time digging up sources, working with the data, and would write only when I had to, when I had to get a draft in. It was something I tended to avoid. This type of process where there’s this sociality, it’s essentially a form of commitment device. You’re making a commitment to work with other people at these times, and it’s much more likely that people will devote more time to writing when they have that sort of an experience, I think.

Fiona: I think you’re absolutely right. It is 100% a commitment device, these sort of regular accountability groups, for sure. And I often joke that one of the superpowers that graduate students often arrive at grad school with is the ability to write to an immediate deadline [LAUGHTER] in the full adrenaline charged way that that has happened and that that remains a useful skill to have access to, but it is also worth expanding that writing toolkit to include other kinds of writing and to even just learn to even feel what it feels like to write without a deadline is sort of revelatory to students. They say things like, “I can’t believe it. I can’t believe I was able to write even though nothing is due.” [LAUGHTER] But it goes to your point exactly, right? These are learnable skills and writing is a practice. It’s a practice that you need to build stamina around. It’s a practice that improves and changes and evolves depending on what else you’re doing, what kind of writing you’re working on, but that it is worth developing a sort of sustainable relationship with it, a sustainable and sustained relationship with your own writing process, because you need all sorts of versions of that writing process to be with you as you move through a program.

Rebecca: …just sitting here thinking like, “oh, writing isn’t always an emergency?” [LAUGHTER]

John: We should try that sometime. Yeah. [LAUGHTER]

Rebecca: One of the tools that you shared in the book is the writing initiative planning prompts table. Can you talk a little bit about what this resource is and how it works?

Rachel: Sure, as I said earlier, we didn’t want the book to be a “how to”, but more of a “how we.” But after we finished it, and so we had these 14 chapters and our own intro, we wanted to think of some way that we could actually leverage all that to help people think about what it is that they were going to do. And so that’s why we meant by having a chart full of prompts. So as Fiona said a moment ago, we asked each group of authors to organize their chapters around rationale, organization, and evaluation. So we wanted each chapter to have roughly that layout, so we’re finding out from each group: Why did you do this? How did you do it, and what did people think about it? How did you evaluate it afterwards? And so we put together this chart of prompts that asks questions of people. So it doesn’t say, this is how you ought to do it, it says: Why are you doing this? So questions like, will your group be formed by convenience or by theme, or by discipline or by stage of study? We saw examples of all of those things in the book, and it’s just an example of a question that people can think, is this just going to be convenience, right? It’s just going to be the people who can come at that time. Or maybe you want it to be more disciplinary and then have some more writing feedback involved in that, or you’re going to actually put out a call for people with a particular identity, so it’s more of an affinity group style to writing groups. So just by asking these series of questions, how much space do you have? Who’s going to run these groups? How do you feel about food? Those kinds of questions that are really important. One of the things that comes up in a lot of the chapters is hospitality matters when thinking about writing groups. And so our goal with this chart, which is available to anybody on this book site, and we’ll put all these links in your show notes, but people can just go and download that PDF and look at all those different questions and use it to either think about creating a writing initiative that they might not have already in their institution, or just think about how they’re doing it by going through this series of questions you might refine or slightly alter whatever it is that people are already doing.

John: In this collection, you have quite a wide variety of different ways in which people write together. Would you suggest that we should build more ways of having people work together on their writing, however, that’s done?

Danl: Just in the same way that I guess we’ve been trying to expand what we mean by writing together, the definition of sociality, or how we understand that. I think something I find myself saying a lot to graduate students is that a lot of things that they don’t think of as writing is writing. And so one of the slogans I find myself repeating is “it’s all writing.” So meeting one of your friends from your program and talking about what you’re working on can become part of the writing process, and it’s also part of writing together.

Rebecca: So we talked a bit about the number of chapters and contributors. Can you talk a little bit about the variety of examples that are available, just so people get a sense of the scope of what’s covered?

Fiona: Sure, these chapters cover writing groups with these highly social dimensions to them that are small, some as small as four or five students per group, some that are enormous. One of the chapters describes a writing initiative that, in fact, has grown to be a part of the entire institutional culture. We have groups that are disciplinarily organized, sometimes by the students in those disciplines. So we have a chapter, for example, by a group of students from a single discipline who were interested in developing their own cross-institutional connections, and made a writing program part of that sort of inter-institution set of connections. We have one chapter that is about a writing program that focuses on the experience of multilingual students. We have a chapter that describes a completely accidental development of a social writing phenomenon. So we really did choose these to be as idea inducing, as sparky as possible by showing just how different the shapes can take, just how different the sizes can be, just how different the facilitation models can be, and just how easily or ambitiously these can take shape and form in whatever writing context a person is in.

Rebecca: So, we always wrap up by asking: “What’s next?”

Rachel: Well, when we thought about this question, we thought we would answer it by talking about what we are doing together next, and our next joint project is to work on the issue of supervisory writing feedback for graduate students. So no matter how much writing facilitation or writing instruction we provide, our graduate students are very focused on this complex dyad between them and their supervisor. So the feedback they’re getting, the feedback they’re not getting, what their supervisor thinks of their writing, how the process is going, what the logistics are of that process. All of those things are always very present for graduate students and engaging in the sophisticated task of writing this high-stakes document that is by definition, in a new genre for them is incredibly demanding, and many supervisors are not able to meet that moment with the kind of writing feedback that those students need. And so in addition to the work we do with students to help them understand the dynamic and help them be better at getting feedback, we also thought it would be interesting to try to work a little more upstream with supervisors, to help them with some of the information and perspective that they might need to be able to provide better writing feedback. Many supervisors approach this task with a great deal of trepidation, because it is so time consuming and because novice academic writers are difficult to work with when you don’t have a lot of skill in that area. So we started with a faculty-facing workshop here at U of T through our Healthy Research Teams Initiative, and we’re interested in now building on that workshop to see if there’s other ways that we can try to support this relationship from the supervisor side.

Rebecca: Well, thanks so much for joining us. This has been a really great conversation and lots of things for folks to be thinking about in terms of bringing their writers together.

John: And it’s a great resource for anyone who’s considering building more sociality into their writing program.

Fiona: Thank you so, so much for having us. It’s an absolute delight to converse with you, as always.

Rachel: Thank you.

Danl: Thanks so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

343. Writers’ Groups

Faculty writing groups can help motivate writing, provide peer feedback, and lead to higher quality writing products. In this episode, James Lang, Sarah Rose Cavanagh, and Mike Land join us to discuss their highly productive long-term writing group.

Jim is a Professor of Practice at the Kaneb Center for Teaching Excellence at the University of Notre Dame, the author of 6 superb books on teaching and learning and is the author of a regular column in the Chronicle of Higher Education. He was the founding editor of the West Virginia University Press series on teaching and learning, and is now a co-editor of a new series at Oklahoma University Press. Jim also was the founder and long-time Director of the teaching center at Assumption College.

Sarah Rose Cavanagh is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and is also a regular contributor to The Chronicle and many other publications. Jim and Sarah are regular keynote speakers and have both provided keynote addresses at SUNY-Oswego.

Mike Land’s early writing and editing experiences included 15 years of newspaper journalism, a masters and doctorate in Creative Writing at the University of Missouri-Columbia, and 23 years of teaching journalistic and creative nonfiction at Assumption, working for many years in the office next door to Jim Lang’s and a short walk from Sarah Cavanagh’s. He’s an Associate Professor of English and Director of the Community Service-Learning Program at Assumption University.

Show Notes

  • Cavanagh, S. R. (2016). The spark of learning: Energizing the college classroom with the science of emotion. West Virginia University Press.
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Lang, J. M. (2021). Small teaching: Everyday lessons from the science of learning. 2nd ed. John Wiley & Sons.
  • Jensen, J. (2020). Write no matter what: Advice for academics. University of Chicago Press.
  • Scrivener
  • A General Education – Jim Lang’s substack account
  • Cavanagh, S. R. (2023). Mind over monsters: Supporting youth mental health with compassionate challenge. Beacon Press.
  • Reading groups at Oswego (in the last three we were joined by Jessamyn Neuhaus and colleagues from SUNY Plattsburgh):
    • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning.
    • Lang, J. M. (2020). Distracted: Why students can’t focus and what you can do about it. Hachette UK.
    • Lang, J. M. (2021). Small teaching: Everyday lessons from the science of learning. 2nd ed.
    • Cavanagh, S. R. (2023). Mind over monsters: Supporting youth mental health with compassionate challenge.

Transcript

John: Faculty writing groups can help motivate writing, provide peer feedback, and lead to higher quality writing products. In this episode, we explore a highly productive long-term writing group.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are James Lang, Sarah Rose Cavanagh, and Mike Land. Jim is a Professor of Practice at the Kaneb Center for Teaching Excellence at the University of Notre Dame, the author of 6 superb books on teaching and learning and is the author of a regular column in the Chronicle of Higher Education. He was the founding editor of the West Virginia University Press series on teaching and learning, and is now a co-editor of a new series at Oklahoma University Press. Jim also was the founder and long-time Director of the teaching center at Assumption College. Sarah Rose Cavanagh is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and is also a regular contributor to The Chronicle and many other publications. Jim and Sarah are regular keynote speakers and have both provided keynote addresses at SUNY-Oswego. Mike Land’s early writing and editing experiences included 15 years of newspaper journalism, a masters and doctorate in Creative Writing at the University of Missouri-Columbia, and 23 years of teaching journalistic and creative nonfiction at Assumption, working for many years in the office next door to Jim Lang’s and a short walk from Sarah Cavanagh’s. He’s an Associate Professor of English and Director of the Community Service-Learning Program at Assumption University. Welcome back, Jim and Sarah and welcome, Mike.

Sarah: Thank you.

Jim: Thank you.

Mike: Thank you.

Rebecca: Today’s teas are: …Sarah, are you drinking tea today?

Sarah: Never. You know me. I’m drinking seltzer. It’s too late in the day for coffee. [LAUGHTER]

Rebecca: How about you, Jim?

Jim: I had several cups of tea already today, it’s orange pekoe? And then my last one was English Breakfast actually at about one o’clock.

Rebecca: Nice. How about you Mike?

Mike: I am sampling some Bengal spice which the box tells me is brimming cinnamon, ginger, cardamom, and cloves and it’s an adventurous bland.

Rebecca: Does it taste adventurous?

Mike: It does. Everything about this is an adventure to me. [LAUGHTER]

John: And I have a blueberry green tea today.

Rebecca: And I have not my favorite tea. [LAUGHTER]

John: And what do you mean by that?

Rebecca: It’s not a good tea. It’s the first time I’ve ever had it. It’s supposed to be gingerbread. And I think it’s kind of disgusting. [LAUGHTER]

John: Now you know why we never get any sponsorships from tea companies.

Rebecca: I didn’t say where it came from. [LAUGHTER]

John: That’s true, but I’m sure there’s probably not too many gingerbread teas out there.

Rebecca: There shouldn’t be. [LAUGHTER]

John: In past podcast discussions with Jim and Sarah, we had heard about the writing group that you’ve all participated in. Can you tell us how this group got started?

Sarah: I guess I can take that one. I think it was about 2014…2015. Mike, do you know the year? He’s giving me a big nod. 2014. And Jim had recently invited me to write the Spark of Learning and his first series at WVUP and Mike was working on a book…had just come off sabbatical doing a cross country trip. And Jim was working on the first edition of Small Teaching. Since all three of us were working on book projects, Jim said, “Hey, why don’t we form this writers’ group.” And we started meeting. And it was really such a beautiful process to write my first book with my editor [LAUGHTER] reading every chapter as it came out, was a very developmental process. And as well as forming some really great friendships.

Rebecca: Sounds like a great start to a long endeavor. Can you talk a little bit about how regularly this group meets and if you still continue to meet?

Jim: We do continue to meet, but not regularly, as you might expect, [LAUGHTER] especially just because we were initially together at Assumption and were all working in the same area, and I just mean sort of geographical, and now Sarah’s in Boston, I’m going back and forth between Worcester and South Bend. So it’s a little bit harder to get us together in the same place at the same time. So we kind of meet on a more ad hoc basis at this point. if one of us has a project that we want feedback on will sort of send an email around saying “I need help, [LAUGHTER] and so let’s get together and can we have a writers group coming up?” So we’re still probably doing that maybe six to eight times a year, I would say. Maybe we were trying to do it every month but now it’s kind of morphed into that kind of more occasional kind of schedule.

Sarah: So we usually squeeze in a Christmas one.

John: That’s true.”

Sarah: So usually a mid December writers’ group. Some of them are in Zoom just because of how we’re spread around but the Christmas one is always in person. It’s really nice.

Jim: Yes, we have holiday themed groups. [LAUGHTER]

Mike: …with teas.

Sarah: Well, for some of us.

Mike: …or beers, but yeah, we’ve tried to have a couple of year that are also just social and in person. We’ve got more used to Zoom,as everyone has, and since we’re in three different…well, Jim and I live two doors from each other, but it’s harder to get us all together physically in the same space, but we’ve try to do that two or three times a year.

John:. When you were meeting regularly at Assumption University, what happened during the meetings when you got together? How long were the meetings? And what did you do in the meetings?

Jim: That would also depend upon where everyone was, in terms of their writing projects. A typical meeting would last maybe 90 minutes. Mike’s especially a social guy, [LAUGHTER] he would sort of want us to start with some social time, we sort of built that in. And so we’d have a little social time initially, you might be around a meal, for example, for the first 15 to 30 minutes, just catching up on our lives. And then we kind of just go through the project, essentially. So we essentially will just pick one off the pile, and say, “Okay, so here’s my essay this time,” like, for example, I might be doing a Chronicle essay or a chapter of a book. You have to read it in advance. So that’s an important thing to note, that we don’t read out loud, which sometimes writers’ groups will do that. You read in advance, we send the stuff out at least 48 hours in advance. You’re expected to read that material, come into the group having already written comments, either online or actually some of us still use the paper versions and bring those in as well. So then, essentially, the person whose material that we’re critiquing is supposed to just sit and ask questions, and the people that have read will kind of go through, essentially page by page, there might be like an initial comment, “overall, this is what I think of the piece,” but then we kind of go through page by page. That doesn’t mean every page is commented on, because it depends on where the issues are. But say, each person’s piece could take 15 or 20 minutes or so to walk through it. And then we move to the next one. And so we move around the circle that way. That’s the basic process, but what am I leaving out?

Sarah: No, I think that’s most of it… Kind of the flow.

Rebecca: Can you talk a little bit about the benefits of having formed the writing group and what other people might think about in forming a group?

Sarah: It is so invaluable, I think, in so many different ways to have first readers who can read something. It’s so vulnerable writing and putting your thoughts out into the world, and to have a safe place where people can give you feedback on very early drafts, I think is so valuable. And it’s also a really good test of are you expressing your ideas well, and are your references sound. I’m always telling Jim, people who don’t know sports aren’t going to understand this reference, and he does the same for my sci-fi references, and so that kind of tells you what a broader audience might think of your ideas. And it’s just a place to bounce your ideas around as well, like sometimes we do a lot of line-by-line feedback, and Mike is awesome at that from his journalism background. But we also sometimes just ask advice and just say, Is this even working? Should I draft this project and go to something else? Kind of really broad ideas as well. How about you, Mike, what do you see as the benefits?

Mike: The things you said. I think it’s a lonely pursuit, and if you don’t have an audience of people who are intelligent readers to let you know whether you’re totally going in the wrong direction or not, you’ve been staring at what you’ve been working on for so long, you convince yourself that yeah, this makes perfect sense, or this was a good move, or they’ll laugh at this, whatever it is. And so at some point, you got to try it out on audience members you trust and who also understand the writing process, because they themselves are writing and have written a lot. So they differentiate some things, maybe what not to do or to do. There’s a temptation in fiction writing, when I was in the fiction writing program, we were warned about trying not to turn the short story of the person being critiqued just into the story you had felt like writing instead. [LAUGHTER] Well, I figured out what the person is trying to do, and help them do that, and stay locked in on that, to some extent. So yeah, there was a lot there. And it is an extra deadline and something to get you moving.

Jim: That’s an important thing to know, too, that it was oftentimes a way of saying, “Okay, I have this book coming out, so I want to make sure I go chapter by chapter, and so I would make the writing group deadlines, my internal deadlines for getting those chapters done.” That’s a really, really helpful structural enabler, when you’re trying to especially do like a long project. The other thing I mean has come up in actually both these comments from Mike and Sarah, about the audience. And that’s true in terms of your audience. Think about your ideal audience who you’re trying to really write for. It’s also really true for your editors and potential publishers. And we often talk about that, like, “Okay, who would really be the right place to submit this to?” And we might be able to share examples of the publishers that we’ve worked with or strategies for reaching out to an editor, maybe we have a connection, and we know somebody. So we can have also that as well. So especially if you have books in your writers group who have published other things, you can start building networks around a writer’s group and I think that’s really important, too. Again, I think what Mike and Sarah are saying about audience is a real big benefit of you’re sort of thinking about the readers that you’re trying to reach, and you can often refine those. The example that Sarah gave about references, like if I’m writing to a community of people who love science fiction, I don’t have to worry about those references. But if I am going to expand beyond that audience then I have to either explain them or make more universal ones, and I think that process of like hearing at least two other readers, helps you expand a little bit in the number of people that you can reach with your writing.

John: And I would think having that relatively timely feedback would be much better than writing a big chunk of a book, sending it off, having it go out to reviewers and then getting reviews back when it’s perhaps a little bit late to make some of those major changes, that getting that more immediate feedback, I would think, would be helpful in keeping the writing project going in the direction you intended.

Jim: Absolutely. For example, I had like a little mini readers group with my Chronicle editor. Now I’d submit my work to her, and she would really sort of hack it to bits, [LAUGHTER] in a nice way, she’s a great person. But over time, I’ve sort of recognized, this is what she wants, this is what she looks for, this is what she changes. And I kind of learned to sort of mold myself into that kind of form that she wants. And the same thing is true for your writers’ group, you can sort of start to anticipate the things that they’re going to point out to you, which are often little habits that we have, which are not always great ones. [LAUGHTER] I always remember that Sarah would always point out that I used the word “so” all the time as like, “and so this,” and I never would have noticed that unless she started to circle all my “so’s” and then “look how often you do this,” that was like a really helpful thing for me to know. So now I’m aware of it, I still use them, but I’m just more aware of it now. So it’s very helpful to see other people’s perspectives and then you can start to anticipate that.Sarah sometimes uses phrase, “I had Jim in my head when I was reading this sentence,” or “I had Mike in my head…

Sarah: Yep. [LAUGHTER]

Jim: …telling this anecdote” or something like that. So that’s a really helpful thing to have as well.

Mike: Yeah. And it’s interesting, you’ll learn those things about each other, and then we can decide when to accept or not accept that, right? [LAUGHTER] In my case, and I think it’s true of Sarah, we can anticipate that if we throw in one too many anecdotes, we know Jim’s going to be there waiting to pounce on our extraneous anecdote that we love. And I’m somewhere in the middle, from the South, so somewhat more elliptical in my storytelling and less linear. So I’m favor of the extra anecdote. But there are times when you can tell, when you looked at it, and you have that voice and you go, “Okay, this really is extraneous, get it out of the way, don’t even bother other people with it.” And that’s that kind of thing and that’s kind of why we assign undergraduate students to do peer editing of each other is to learn to be editors, and to hear those voices too. So it’s really useful that way.

Jim: Mike, you know about other readiness groups and this question, John asked us about, like, guidance for other groups, can you give some examples, Mike, if you know of other writers’ groups that have sort of different ways of working?

Mike: There have been some groups that seem to exist just for people to pat each other on the back. And they don’t really do much criticism at all. A friend of mine who’s dyslexic said that he was in a group where you could never correct anything ever, grammatically or stylistically, which would drive me crazy, because I would want people to note every time I slipped up on something. So those are a couple of examples, and I’ve been trying to come up with some more, but it seems like there are a lot of variations in the rules in how they do it.

Jim: Sarah, you have also one that’s more like keeping you accountable, right?

Sarah: So I have another writers’ group with some friends at Tufts University, and we call it “writing and hijinks.” And so we gather together in a coffee shop or somewhere and we just write quietly, although one of them, like Mike, is very social and has trouble with the quiet part. [LAUGHTER] And so we just write, it’s just dedicated writing time for a couple hours, and then we go out and have fun. Although we’ve been sliding more and more towards just the fun and not doing the writing [LAUGHTER] over the last few years. This is motivating me to say to them, maybe we should bring back the writing part.

Rebecca: We have a writing group at Oswego that functions like yours, Sarah, that you’re talking about, where it’s just kind of some dedicated time, accountable time, this is the time we’ve set aside to write…

Sarah: Right.

Rebecca: …which can be really helpful if you have a designated time and space to do that, if you aren’t good at scheduling it on your calendar otherwise. Julie

Jim: Joli Jensen has a book about writing from the University of Chicago Press, and she actually makes the argument that writing groups for academics should not be content critique groups, which is what we have, actually. And she makes the argument that it’s not helpful, for example, English professors to be working with a psychologist because they won’t get your references and all that kind of stuff, and it won’t be very helpful to you. But actually I would argue that what we’re doing here, because most of us are running for, like broader audiences, the content critique part is really important. If Mike and Sarah don’t understand what I’m saying, well, then I’m not gonna be able to reach a broader audience instead of just writing specific to my little expertise audience. So that’s another way to think about distinguishing content, whether you focus on the content or just the writing, the sort of grammar and the stylistic part of it. So that’s another way to think about what are you actually trying to do in your writers’ group? Are you trying to expand beyond your discipline and getting different kinds of feedback? Or are you just trying to focus on, as Mike said, maybe patting each other on the back or accountability or style and grammar? So there’s a lot of different ways you can go.

John: I think that would depend a lot on who your intended audience is. So that if you’re all psychologists, or all economists, or all mathematicians, if you’re going to be submitting things to journals, it might not be a bad audience. But for a general audience, having someone who is not necessarily an expert in the field, seeing it from a different perspective would lead to a lot of really productive comments in terms of what some of the hidden assumptions are, the implicit assumptions in the writing.

Jim: Absolutely, you can actually even think about it as a teaching thing too, I mean teaching to like a gen ed class, their first-year students, that’s almost like a content critique group in the sense you can’t take too much granted about background knowledge in discipline, like a really focused content critique group, all from a discipline, you’re essentially teaching a graduate seminar. And so you’re looking for different things from those experiences, and you’re being required to explain different levels of things to those two audiences.

Rebecca: So I’m curious what participating in this particular writing group with the three of you has done for you as writers? How has it shaped your writing? And why are the three of you a good group for each other?

Sarah: I think that one of the things that has really shaped my writing process, this is beyond all of the wonderful things that we have said already about audience and things like that, is being in a writers’ group with two English professors is really valuable. I tested out of intro writing in college, so I actually never took a writing course of any sort, and then decided I wanted to be a writer. And my writing is very organic and intuitive. And working with these two really helped me like the things that they would correct. Like I didn’t know periods went inside the quotes, until Mike corrected me. [LAUGHTER] But the first year, we were writing together, and you would think I would just absorb that from reading, but I just never noticed that. And so really tiny things like that, but then also just thinking about the structure of your writing and the organization of your writing, because I had never really done that for this kind of general audience writing. And so having this writers’ group with writing professors is really amazing, as Jim would sometimes send me to the whiteboard, when we would meet [LAUGHTER] at Assumption, because it’d be like, you’d have a lot of really interesting ideas here in the writers’ group, but they’re just in a big pile,[LAUGHTER] and he’d have me map out, do a literal map of my work. And that was very, very helpful.

Mike: That’s one of the fascinating things about being in a group is working with two people that have such different composing processes. And Jim, you’re fairly linear…

Sarah: fairly…

Mike: kind of have it in order [LAUGHTER] and smoothly.

Sarah: …the computer. [LAUGHTER]

Mike: So what was the computer program that you’re using that divided the screen into quadrants?

Sarah: Yes, Scrivener, you can put your bits of writing on literal digital index cards, and just move them all around.

Mike: Yeah, which is something I’ve seen poets do. I’ve seen poets tear their poems apart, line by line and move the lines around in the coffee shops. So it’s just two equally effective ways of doing that. And I guess I’m somewhat in the middle, but with the road trip narratives, obviously, there was someone’s this town and that town, but it’s fascinating to watch how people’s brains work differently.

Jim: Yeah, I think for me, Sarah and Mike are different kinds of writers. But as a result, I’m kind of benefiting from both the sort of storytelling advice that comes from Mike and from Sarah, again, more of the research, making sure that I’m supporting my points well, and thinking about other perspectives on the topics I’m addressing. And it’s helpful to get both those things, because I’m getting different things from others, like editors, for example. I like the idea of starting a piece of writing with a story or something like that, my editor at The Chronicle always wants me to lead with the thesis, essentially. And so I’m trying to put some of Mike’s ideas advices into that kind of storytelling opening, but also trying to satisfy my editor. So anyways, it kind of helps you develop new strategies and see how strategies might land with different people and Sarah’s especially good at ways that I should include more readers, more thinkers from different backgrounds, and essentially making my writing more inclusive. And I think that’s really valuable as well, and making sure that I engage with the research that I might not be aware of, in the teaching and learning fields. So that’s really important, too. So I can look at my own writing and see how it’s benefited from both their perspectives.

Rebecca: I think one thing that I’ve benefited from when I’ve participated in writing group opportunities is actually seeing early stages of other people’s writing, and just what that looks like and feels like and sometimes provides strategies to future me [LAUGHTER] when I’m stuck, [LAUGHTER] like, “oh, wait, this other person would have attempted it like this.” And when I felt stuck on something, sometimes I use the strategies that I’ve observed other people using, which can be really beneficial, that things that I wasn’t taught or hadn’t been exposed to otherwise.

Sarah: That’s great.

John: You mentioned future me. And one of the issues that we know from behavioral economics and psychology is that when given a choice between pursuing your long range objectives and immediate gratification, immediate gratification sometimes wins out, [LAUGHTER] because it’s always easier to start these things tomorrow or next week. And one of the things you mentioned earlier, Jim, was that having the regular meetings of the writing group provided a bit of a commitment device that you want to get to the meetings with some new material you want to share that material before it. Do you think that has helped increase the pace in which your work has taken place?

Jim: For me absolutely, yeah, having that structure there. I think structure is always a really helpful thing in teaching and writing. If you sign a book contract, the deadline might be two years away. So you know that two years, an editor might check in after a year, but like, you need to have more structure in that to kind of get yourself moving along. And I’m a pretty structured person, but I can also fritter time away. I’m pretty good at that, too. [LAUGHTER] But I’m self aware enough to know I need a little help in terms of structure. And so it definitely has pushed me to become more productive.

Sarah: Absolutely. I agree. This whole conversation is making me want us to do this more regularly, [LAUGHTER] instead of the as needed, because I think about especially the beginning of the writers group. And when I was writing Spark of Learning, my daughter was a toddler. And she was only in daycare three days a week, and I was on the tenure track and just doing a billion different things. And I do not think I would have written that book if it weren’t for writers’ group monthly meetings. I think at that point, we were really regularly meeting in person monthly. And just finding the time at four in the morning [LAUGHTER] or terrible times often, but I don’t know that I would have gotten it done if it were just like, “oh, I have a contract and I must do this giant thing.”

Mike: That definitely helped this begin…the structure. I don’t have a long-term project I’m working on currently so hard for me sometimes to come back around and really have anything that’s worth them looking at. But yeah, you mentioned those long period of the book contract, and there’s a long period of only doing shorter things, whereas so there’s not that continuity from chapter one to chapter two to chapter three, and you’re eager to write those next two pages that are the bridge to the next thing, so group can help with that as well. It seems like a lot, but what happens in writing group is when someone is working on a Chronicle column that needs to be turned in in 48 hours or letters and things like that. And I was wondering if you can give examples of that kind of stuff when the group comes into play. That’s not the big primary thing we’re working on, but it’s kind of the emergency stuff.

Jim: We’ve done a little bit of that, but it’s kind of hard because especially for an upcoming deadline, I think typically we’ve handled that when someone just needed something very quickly, we just handled that by email. But the writers’ group is available in those emergency situations, because we already have a quick sense of like, if Sarah sends me something, and it was for a quick turnaround, it’s for The Chronicle, I can be able to spot very quickly a couple of things that you could maybe do in that short window. And likewise for me. So it’s challenging, though, because you want to have time for people to have opportunity to read and think about that as well. But it’s available for those situations. So it’s good for that.

Rebecca: I can imagine that you’re strong relationship with one another, having written with each other for so long allows for that quick turnaround by email, because you can sense the tone someone’s providing some feedback in [LAUGHTER] and it…

Sarah: Help!

Rebecca: …has a different kind of context. [LAUGHTER] Yeah.

Jim: Absolutely.

Sarah: And it’s such a pleasure, like it’s hard to ask for emergency work from people, even people you’re close with, and maybe even especially people you’re close with, because you know, they have so much else going on. But it’s such a pleasure to read these gentlemen’s words, that it’s not a burden at all, even the emergency ones because it’s like, “Oh, I get to relax, not with a cup of tea, [LAUGHTER] and read some words from Jim or Mike.

Jim: That’s true. That part of it is good, because you know that you can help them in the emergency because they’ve helped you in the past, that same situation. Right now, you know, I’m sort of developing a substack account. And I haven’t really used the writers’ group for that. And as a result, I need to have my wife proofread for me,[LAUGHTER] because substack doesn’t have an editor, and I’m really annoying her by this process. So I need to start using my substack columns for writers group stuff as well.

Mike: Yeah, it is really helpful and we know that there’s just a lot less explaining to do amongst the three of us. If I were going to send something cold to another friend, I think I would have to explain a lot more about what was going on for them to get it. So that’s incredibly useful that way. But it’s funny how different it is from like growing up in a newspaper family, and I would interview someone in the morning and I would revise it four or five times in a period of like four hours and hand it to my editor who would then tell me to change three things. And the whole thing from beginning to end was less than a day. But there was no deep thought, there was no deep reflection or big structural things, and then my time sort of expanded magically as I became an academic and suddenly you can get trapped in a lot of whirlpools of uncertainty about which way to go or that way to go. Whereas in journalism, there wasn’t much choice a lot of the time.

Sarah: Writers note, I love that “whirlpools of uncertainty.” If I were in writers’ group, I would say underline it, put a little exclamation point next to it. [LAUGHTER]

Mike: Well, I was a really terrible student in an undergraduate poetry workshop at the same time my sports editor was trying to teach me to write in a linear way. And my teacher said at one point, Mike what you’re writing, as poetry is prose with line breaks in random places. And he said, “Let the language go where it wants to go.” And I was having a hard time imagining telling my sports editor that on [LAUGHTER] Friday night during high school football season, but it seriously took a long time for me to learn I could do both. I could loosen up and be this one kind of writer, and then be this very mechanical quick writer. And that took years as an undergraduate to figure out how can I do both of these things at the same time.

Rebecca: One thing I heard Sarah say is that currently you’re meeting as needed, but then I also heard in the conversation, that maybe we need these things, and we just don’t know it. So having it regularly on the calendar could be helpful. [LAUGHTER]

Sarah: Mm hmm.

Jim: Yes, we’re convinced now this conversation is very helpful to us. [LAUGHTER]

Mike: Yeah, and plus this almost the end of the semester, so suddenly, we’re gonna have a lot of time on our calendars.

Sarah: …well, those of us still teaching. [LAUGHTER]

Mike: Yeah.

Sarah: I’m at 12-month contract now.

MIKEL Oh, you are?

Sarah: Yeah.

Mike: For you, year round.

Rebecca: Me too, Sarah. [LAUGHTER] Maybe we need a separate writers’ group. [LAUGHTER]

Sarah: Or therapy. [LAUGHTER] …support group.

John: What sort of advice would you give to other people who are thinking about forming a writers’ group?

Jim: Actually, there have been been other people in the writers’ group over the years, I’ve been in other writers groups too. For me, the thing that I look for is people who I thought were sort of ambitious and were writing, because you don’t want to be the person who is trying to gather people together, when they’re just not writing, they don’t want to write, you don’t want to be like the taskmaster in a writers’ group. So find people that are writing and are interested in writing, and find people that you find interesting, you want to hear more of their thoughts. So that has always been a driver for me. And sometimes, if I’m looking for people that I want to hear their thoughts about my own writing, and I want to talk to them about their own writing, I’m gonna be more interested in going to that writers’ group and participating. And so those are sort of two general points I would make: look for people that are writing and interested in writing, and then people who are interesting, you want to see it develop and getting access to their ideas as they’re developing. And I would just say, most people, I think, are flattered, you would like to hear and read their work. So if you identify someone that and you might say, “Well, oh, you know, they’re probably too busy, or they probably already have other people to work with,” that might not be true. And so extend the invitation and see if they want to join you in pushing writing forward.

Sarah: And I think I would add, it’d be hard to figure this out, [LAUGHTER] but one thing that I appreciate about this group, and I would want in any writers’ group is having a similar level of like sensitivity and openness to feedback and criticism. So I think that all of us are really great at taking feedback and taking constructive criticism, without either taking it personally, or I think Mike referenced this, but sometimes the Jim in my head or the Jim on my page, I’m like, “Yeah, I’m not listening to you.” [LAUGHTER] But then a lot of times you do. And so t o people who are at that level of like, open to hearing criticism, but also confident enough to stick with their gut if they feel it’s right. And yeah, and so that’s a delicate thing. And it’s hard to figure out about someone without actually engaging in writers’ group. But I think if they were in your department meetings or on committees with you, kind of keep an eye and get a sense for how open they are to other people’s thoughts and how willing they are to give them.

Mike: Yeah, and along the line with open is something I always warn undergraduates about, but in grad school it would come up with separating the characters on the page, even the narrator, that that person I’m writing the first person about, it’s Mike, the narrator on that page, but it’s not necessarily all of what I am. So, ‘cause when you’re talking to someone, then you can point out stuff that you think is maybe sexist, or is whatever, but be able to talk about it in a separation where you’re not personally attacking the writer, but that you are holding that in a certain place. I think we’re really good about those sorts of things, we know the level each of us are trying to get to, we’re just trying to help each other get there and again, smooth out anything that the audience might misinterpret. That’s a mental discipline that takes a while to be learned. And I’ve had friends who have been in groups where things have devolved to name calling, and to really unfortunate situations where the writing group had to be broken up, and so I’ve even heard that happen in academic settings, like in actual seminars. Fortunately, I was not part of any of those, but where people went stomping out of the room. And so yeah, having the maturity in just the way you carry yourself in life, I guess, to know how to treat other people and work with them constructively, a really important thing. But otherwise, in terms of advice, let’s say that whatever kind of writing group you want to have, you want to sit and just right beside each other and then go have drinks that what this one’s gonna be, are you all aiming to publish and you’re trying to help each other get published? That can be a very different mentality, depending on what you’re trying to do. That’s my two cents.

John: Do any of you have any other thoughts about writing groups that you’d like to share with our listeners?

Jim: I will say just as an editor, you know, editing the book series, I can also sometimes tell people who had writers’ groups when they turn their manuscripts in who had prior readers. So just from my perspective as an editor, I absolutely recommend this, forming a writers’ group, especially if you’ve signed a contract, and you’re intimidated by it. The fact that you have this deadline, you have this big project coming up, A writers’ group can not only put some structure on to the process but also forestall a lot of the work that I will have to do as an editor. So you can make my life easier when you’ve had some prior readers who kind of work with you to iron out a lot of the kinks that have developed, as any writing project will have in the drafting.

Sarah: Yeah, my editor was very grateful for writers’ group. We met monthly, my editor for Mind Over Monsters and she was wonderful. But writers’ group was like a third character in the room. [LAUGHTER] Because we would meet monthly, I’d be like, “Alright, well, here’s the chapter, but this is what writers’ group says, and they think that I should reframe this.” So it was really like the three of us, or I guess, the four of us, melding you two into one being in the conversation and in the room. And that was an interesting process.

Mike: I think another thing that’s been interesting to me is that because Jim and Sarah have books out a lot of the time, and there’s a good bit of talk about the publishing aspect of it, there are times when say, Jim might know someone that one of us should hit up, or contact, or things like that, or how to interpret an editor’s reaction. And there’s been a good bit of conversation about those things where I kind of felt like I know, these editors I’ve never met. [LAUGHTER] I’m sure that’s a big stopping point for a lot of people is they’ve written this thing, and now they don’t know what to do. They don’t know what the path is, and how to get there, how to get someone’s attention. And so there’s a lot of talking back and forth about things like that.

Rebecca: Do you have wishes for your group moving forward?

Jim: I guess, more regular meetings.

Mike: …more regular meetings, yes. [LAUGHTER]

Sarah: Sounds like a way to save Jim’s marriage.[LAUGHTER]

Mike: I guess, I should sell them another road trip so I could write another road trip manuscript.

Jim: Yeah.

Sarah: There you go. I would read that.

Mike: As we sneak toward possible retirement…a travel book…that could be the plan.

Rebecca: So we always wrap up by asking: “what’s next?”

Sarah: Well, more regular writers’ groups, I guess. But I think as Mike just referenced, he’s retiring. So he’s going to have a lot more time to write coming up. And Jim and I are both, I think at the phase of thinking about next book projects. And so maybe it’s going to be a new day for writers’ group.

Jim: That’s true. I’m actually doing the final, final, final revisions of my current book manuscript, which are due on April 22nd. So that will be coming up shortly. And so I’m in that phase where like, I’m kind of finishing everything up, and also the editing and all that stuff, and copy editing, but still at least starting to open up in the sort of creative part of your brain about like, what might be next? And so that’s maybe one final thing I might say about writers’ group is sometimes we will come in with just like a one pager of ideas. And Sarah has done that several times. And we kind of give her feedback like, this is the one that seems like it might be the best path to pursue, for example. Anyways, I’m in that space right now. So I hope that for me going forward writers’ group might help me land on the right project next.

Mike: I should mention too that just the venting of frustration is good. And it just crossed my mind, one of Jim’s funnier Facebook posts of all time was when you were finishing up a book and what you posted about how you felt about the book. I don’t know if you recall this or not?

Jim: Yeah, absolutely.

Mike: So tell them. [LAUGHTER]

Jim: So I just remember talking about the fact that whenever you finish a book, I’m just so sick of it. At that point, I never want to think about it again. [LAUGHTER] And then of course, a year later it comes out and you have to think about it all the time. [LAUGHTER] I’m a person that has multiple interests, and they sort of change on a regular basis and like, by the time I finished a book, I’m like, “Okay, I’m done with this now, what’s the next thing?” But then, you know, hopefully a year goes by when it’s coming out, and then you get interested in it again during that time, hopefully.

Mike: Well, the specific line I remember was, “I hate, hate, hate, hate, hate you, stupid book.” [LAUGHTER]. It was that quote.

Jim: Yeah.

John: Well, I can see it’s probably good that you focus on the writing and not the marketing of these books. [LAUGHTER]

Jim: Yeah.

Mike: It wasn’t this book, it was a different book.

Rebecca: Well, thank you so much for joining us. It’s always a pleasure to talk to you about your work. And we’ve been hearing about this writers’ group [LAUGHTER] for a while. So we were glad to have the opportunity to dive in a little more deeply.

John: And we’re also grateful that we’ve been able to benefit from the writers’ group. I was just thinking back, four of our reading groups over the last few years were products of this writing group.

Jim: Wow.

John: So we appreciate the work that you’ve been doing.

Jim: That’s very cool.

Sarah: Thank you. It’s an honor.

Mike: Yeah, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

324. Unmaking the Grade

A growing number of faculty have been experimenting with ungrading. In this episode, Emily Pitts Donahoe joins us to discuss her ungrading approach and the documentation of this process on her blog. Emily is the Associate Director of Instructional Support at the Center for Excellence in Teaching and Learning and Lecturer in the Department of Writing and Rhetoric at the University of Mississippi.

Show Notes

Transcript

John: A growing number of faculty have been experimenting with ungrading. In this episode, we discuss one instructor’s ungrading approach and her documentation of the process.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Emily Pitts Donahoe. Emily is the Associate Director of Instructional Support at the Center for Excellence in Teaching and Learning and Lecturer in the Department of Writing and Rhetoric at the University of Mississippi. She experimented with ungrading and chronicled her experiences in her Unmaking the Grade blog. Welcome, Emily.

Emily: Thanks. I’m really excited to be here.

John: We’re very pleased to be talking to you. We talked recently at the POD conference, and I’m looking forward to this additional conversation.

Emily: Yes, where I got this lovely tea for teaching mug from John, which I’m so excited about and drinking from right now.

John: And since this is only audio, Emily was holding the mug [LAUGHTER]…

Emily: I’m showing it off.

John: So our teas today are:… Emily, are you drinking tea in that mug?

Emily: I am. [LAUGHTER] I am drinking tea out of my tea for teaching mug. I’m a big tea drinker. And so today I’m drinking my favorite tea, which is a strong black tea called Scottish morn. And I got this tea from a tea shop called Apothica in Niles, Michigan, which I used to go to when I lived in South Bend, so highly recommended if you’re in that area.

Rebecca: Sounds wonderful. I was ready for you to say that you had coffee or something in the tea for teaching mug so it’d be completely blasphemous… [LAUGHTER]

Emily: Never.

Rebecca: …because coffee is one of the most frequent flavors. [LAUGHTER] Emily, you’re doing it right. [LAUGHTER] I have blue sapphire tea today.

John: And I have Irish Breakfast tea today.

Rebecca: So, we invited you here today, Emily, to discuss your experiences with ungrading. Your blog is based on your spring 2023 course experiences but your ungrading experience predates this course. Can you tell us a little bit about your initial experience with ungrading?

Emily: Sure. So I have a background in Writing and Rhetoric and English literature, and so a lot of what we do in Writing and Rhetoric, I think, is already pretty aligned with some ungrading goals and practices. So in my courses, students have always had the opportunity to revise their work based on feedback, and include it in a final portfolio, and most of their grade is based on their revised work. So it’s not just that they get graded and then that’s the end of it, they always have a chance to improve based on feedback. The first time that I stopped putting letter grades and percentages on student work was in spring of 2022, when I was teaching a general education literature course at Notre Dame, and so the course was on pre-modern and early modern literature, but with a focus on how the texts that we were studying might help us examine or wrangle with some of the questions that we’re preoccupied with as a culture and society today, and thinking about how those texts might relate to student lives. So it was a course for kind of all levels and all majors, which I think made it a good course to experiment with. So I had first-year students all the way up through senior-level students from engineering and business and English and psychology and all kinds of different majors. So like all teaching experiments, I think there were definitely some kinks to work out after that first course; they never go exactly right the first time. [LAUGHTER] But I would say that, overall, it was a huge success. And that’s not because I did it perfectly, or even because I did it particularly well, but because I think ungrading really helped some of my students move beyond this idea of the school as a points game, to help find their interest and their motivations to study the material that we were studying for their own purposes, and to focus on developing the skills and knowledge that they wanted to develop rather than on attaining a specific grade.

John: And the second time you did this, you created a blog describing the process. What prompted you to create the blog?

Emily: So, I was inspired by a post written by Robert Talbert, who’s the co-author of a recent book, Grading for Growth, which I really highly recommend on alternative grading, and he and David Clark, his co author, also had a blog and substack newsletter on Grading for Growth. And so in December of 2022, Robert posted a stop, start, continue for the ungrading community. And if you’re not familiar with stop, start, continue, it’s an evaluation exercise used for mid-semester reflection, very often in classrooms where the instructor or another facilitator will ask students: “What kinds of teaching practices or classroom practices would you like to stop, start, or continue in the class?” And so Robert’s post was a stop, start, continue for the ungrading community. And one of the things that he recommended that the ungrading community start doing was getting into the weeds and writing in detail about the daily experiences and specifics of upgrading, so: what we’re doing, what kind of successes we’re having, what challenges we’re encountering, how we’re adjusting in real time, and he recommended keeping this as the kind of blog or like a captain’s log of weekly reflections. So when I read his post, I thought, “Well, I could do that. It didn’t sound that hard, and it sounded like a lot of fun.” So that was what prompted me to start writing weekly reflections to share some of the methods that I was using, successes and challenges that I had, and even some of my doubts or misgivings about some of the things that I was doing in the class.

John: And just as an aside, we were so impressed by Grading for Growth, that we’ll be using it in the spring reading group here, both at SUNY Oswego and at Plattsburgh with Jessmyn Neuhaus. So we’re very much looking forward to that. And Robert and David will be giving a keynote address at the start of our workshop series in just a few weeks in early January. So we’re very interested in doing more of this on our campus as well.

Rebecca: Can you talk a little bit about your decision to delay posting the blog until after the semester ended?

Emily: Sure. So the selfish reason is that it’s supposed to be an unfiltered look at the ungraded classroom. And part of me thought, what if I messed this up so badly that I’ll be embarrassed to talk about it. So it was partly that, but I would say the more important reason is that I thought it might be weird for students if I was talking in public about what was happening in the classroom in real time. And I was concerned that it would damage the relationship of trust that I wanted to build with my students if they knew that I was reporting on our conversations to a larger audience about the class. So I offered to hold things for that reason. But the other thing that I wanted to do was make it clear to students that I was writing these reflections that might later be published, and then to get their input on them. So some of what I shared on the blog, I didn’t really want to share without getting students consent to do that. But I also wanted to get their input on some of the things that I was talking about. So I believe really strongly in taking a students as partners approach to higher education and learning experiences. And I try to employ that in my educational development work in the classroom. So that means bringing students into conversations about teaching and learning and asking for their experiences and their expertise as students really and then using that to inform our practice. So delaying the posting allowed me also to get some input from students and to be able to share some of their thoughts and opinions on the blog whenever I could.

John: In reading through your blog, your spring 2023 class seemed really interesting. Could you describe that for our listeners?

Emily: Yeah. So this is the second course in our sequence of first-year writing courses here at the University of Mississippi, but it’s a little bit different. So for the second course in the sequence, students have the opportunity to take either Writing 102 or Liberal Arts 102, and I was teaching Liberal Arts 102. And so the goals of each course are really the same. But Liberal 102, as we call it, is conducted within the context of a research area in a specific discipline. So that means that it can have a very specific theme. So some of my really excellent colleagues in the department and in other departments have taught courses like writing about true crime, or the rhetoric of sports, or I believe there’s a course on fashion. So because one of my areas of expertise is teaching and learning, the course that I taught was called Examining Higher Ed Teaching and Learning in a College Classroom. And so I think this is a great course for first-year students as they’re entering this next phase of their education to reflect on where they’ve been and where they want to go. And so what we did was looked at questions like: What’s the purpose of a college education? Why are we all here? What kind of benefits does it provide to individuals or to society? What kind of collective benefits does it provide? And then how are those benefits enacted or engendered in the classroom. And so we explored a lot of debates around higher ed in the US and students have an opportunity to reflect on and draw on their own experiences as students and their expertise as students, and then integrate that with larger areas of research on education or current events in education. And then they communicated their ideas about education to audiences outside of our classroom. And so it was really, I think, an ideal course for ungrading because we could talk about grades, not only as a matter of course policy, but also as a core subject matter. So in the beginning of the semester, we read Alfie Kohn’s piece “The Case Against Grades” and talked about it both as a way to introduce the course grading system and as a kind of larger prompt for reflection about grades as an issue of concern in higher education right now.

John: And it sounds like a great opportunity to have students reflect on what you’re doing as you’re doing it. How did students respond to this?

Emily: So I think every time I’ve ungraded a class now, and I’m currently on my third ungraded class, students have responded a little bit differently every time and of course, every individual student is also different. I think there are three broad categories of response that I see. One is just enthusiasm; some students are really excited about ungrading, and usually that’s students who feel that, for one reason or another, their grades in the past haven’t been representative of their learning or who feel that some of their creativity or their risk taking has been stifled by their desire to get a good grade. So I wouldn’t say this is a lot of students, this is probably a smaller group of students, but some students are really excited about it. I would say another group of students are really hesitant because ungrading is a big unknown for them, especially for students who are dead set on getting that “A” grade. It can be really nerve racking not to know kind of where you’re at in the middle of the semester. When I get negative feedback on ungrading, it’s almost always students who say,”This is really interesting, but I never know where I’m at.” And so they’re really concerned about how they’re going to measure their progress. But they’re not thinking about their progress in learning, they’re thinking about their progress toward a specific grade, which is really understandable because grades are important. And so that’s also a smaller group of students. But I would say, by far, the biggest reaction I get from students is something like cautious optimism. So I always start ungraded courses with a conversation about grades and learning, and I ask students to share with me their experiences of grades and share with each other. And very often, they have a lot of negative experiences to share, and sometimes positive experiences as well. But we talk about the relationship between grades and motivation and grades and learning and they have a chance to reflect on that. And so I use these conversations as a jumping off point to explain why I use the system that I do and how I think it will benefit them as learners. And I think students find it really helpful to be able to talk about their experiences with grading openly and to be heard by a teacher. And I think that that alone makes them more willing to buy into the system. So once it’s explained fully, and once students start to see the potential benefits, I would say that most of them are cautiously optimistic.

Rebecca: I think that largely aligns with my experiences and explorations when ungrading as well. In your class, you included five different assessments and opportunities for revision. Can you talk about and describe these assessments?

Emily: Sure. So the first thing that students had the opportunity to do were weekly writing practice assignments. So these are what you might think of as formative assessments, they’re preparation for class discussion and major assignments that mostly involve reading or short writing prompts. And these students weren’t able to revise, they either kind of did it or they didn’t. But then students had an opportunity to do more longer major assignments, which are more typical assessments for the writing class, a series of papers and a multimodal kind of project or two, with an imagined audience kind of outside our classroom. And so these students could revise up to three times if they wanted to do that. And so I had to, for this class, adjust a little bit and do slightly fewer assignments than I might have done in a traditionally graded class, because the expectation is that students would be doing more intense work on each assignment in their revision. I also asked students to do self assessments periodically throughout the semester. So students would answer a series of questions about their progress toward the learning goals, about their goals for the remainder of the semester. And they would also propose a current grade for themselves based on evidence that they provided. And this is a similar activity to what we did at the end of the semester to determine their course grades. They were also assessed on their final portfolio, which had revised versions of their major assignments. So their work… basically as good as they could make it… their best version of their major assignments, plus another final self assessment of their work in the class. And then they were also assessed on class engagement, so their class attendance, their preparation for and participation in class discussion, the timeliness of their work, their support for the learning community and fellow students and things like that.

Rebecca: Can you talk a little bit about how you provided your feedback on those assessments?

Emily: Sure. So for the writing practice assignments, usually I just wrote a sentence or two in response to students work to let them know if they were on the right track, or they may be needed to adjust slightly as they prepare for their major assignments. For the major assignments, I did do a lot of longer written feedback, and a lot of this looked like feedback that I would give in a traditionally graded class. But I think it was a little bit more oriented toward future growth, rather than reflecting on past work. It was giving them a lot of direction on how they might want to revise to make their work better in the future, not just for future papers, but for this particular paper. And with that feedback, I also gave them a rubric, where I marked kind of where I thought they were at along the specific assessment categories and criteria that we were using for that assignment. So for example, we might have a category for audience and purpose. Students could see whether I think their work on that category was still developing, whether it was at the level of proficient or whether it reached the level of excellent, so I had these different categories and these different kind of progress metrics, where I would indicate this is a skill that I think you’re still working on, or this is a skill that you’re doing really well in. And I also provided feedback to students through our individual conferences. So we met once at midterm, I met with each individual student, and then once at the end of the semester, to have a face-to-face conversation about how students were doing and if they needed to make any adjustments to reach their goals for the rest of the semester. And I didn’t give any feedback through letter grades or points. Those are not really great forms of feedback. They don’t actually give us a lot of information about our progress. So most of the feedback that I provided was kind of qualitative feedback.

Rebecca: You just talked a little bit about rubrics as part of a feedback that we were providing students, can you talk a little bit about how those rubrics were developed?

Emily: Yeah, so I’ve tried several different methods of creating rubrics. And one of the things that I like to do is co-create them with students or get students input on rubrics. And I attempted to co-create rubrics with students in two different classes and have done it a few times. And sometimes it’s a huge success, and sometimes it really does not work very well at all. And in my spring class, it didn’t work very well at all. [LAUGHTER] So basically, what I’ve done previously is ask students what they think makes a good argument or a good piece of public writing, or whatever it is that we’re working on. Or I ask them what kinds of things they believe their writing should be assessed on, and how we would determine whether or not their writing is successful. And then we have a conversation about that. I make notes on the board as we’re talking. And then I go away and distill those notes down into a rubric with about five categories in which student work is assessed. And then there’s the three metrics, so developing, proficient, and excellent. And sometimes this works really well. But in my spring class, I think the students were really at a loss when I asked them about assessment, and even when I asked them what good writing looks like. And I think that’s because that no one has ever asked them to think about that before. They’re used to being told what their work should look like by someone else, and then trying to conform their work to those expectations, or to someone else’s standards. And obviously, we need to have standards and expectations for student work. But one of the things that that does is get the students only thinking about what the teacher wants and not what they want. They’re not thinking about what they want their work to look like, or what their standards are or what they’re trying to accomplish. And so I think, ultimately, if we’re preparing students for the real world, whatever that is, this is what we want them to be able to do, not just to blindly follow somebody else’s standards, but to create and work independently in a self motivated way, and then assess their work independently. So I think we have to kind of start with baby steps in that process. So in the spring course, the rubric activity really turned into a conversation about how students have been assessed in the past and why they’ve been assessed that way. So when I ask them about their experiences, how their writing had been assessed, they would say things like, we were graded on the word count, or we were graded on whether or not we included two sources in every paragraph. And then we use that as a jumping off point to talk about the principle behind those what they viewed as arbitrary rules. And so if your teacher is concerned about word count, it’s because they want you to make a substantive argument. And what does a substantive argument look like? If your teacher is concerned about including sources, it’s because they want you to make an argument based on strong evidence. And so what would strong evidence look like? And so we use that conversation to think about the assessment criteria for their college level work, and how that might be similar or different from what they experienced in the past. So in that way, I do think that conversation was helpful. And I do think based on my few experiences, that it helps to have students extrapolate criteria from some examples, rather than from thin air, kind of as I was asking them to do, but either way, having a transparent conversation about what students are being assessed on and why and a purpose behind the criteria is key to their success.

John: In one of your blog posts, you describe some of the writing assignments that you used in your class, and they seem quite interesting. Could you describe some of the writing prompts that you gave to students?

Emily: Yes, I was really excited about some of the assignments that I developed for this class. And I think I’m going to keep using them or versions of them in future classes. So for this class, I had a menu of assignment options. So the students could, for each major assignment, choose the kinds of projects that they found most compelling, or they would really draw on your strengths. And I thought this was important, partially because we were working some with generative AI in the class. And I’m sure everyone is aware by now some very serious data privacy concerns, ethical concerns around the use of generative AI. So I didn’t want to mandate that students work with AI if they didn’t want to, and some really didn’t want to. So I tried to offer them several options. And I was really happy about some of the options that I came up with. So obviously, one of the things that I was thinking about going into this semester was how to deal with generative AI and I have a lot of thoughts about that. But one big way to think about how to discourage inappropriate use of AI or encourage appropriate use of it is to think really carefully about assignment design. So one of the things that I did to help students navigate AI was lean into some multimodal work, think about argumentation in media other than writing or that kind of worked alongside writing. So we didn’t abandon writing entirely, but I did include a photo essay as one possible assignment in a menu of options. And for this course, I asked students to take a look at a book of photographs by Cassandra Horii and Martin Springborg that’s called What Teaching Looks Like. It’s a really, really cool book with candid photos of college teaching. And so we use this book to talk about visual rhetoric and also to use the photos as a launchpad for this discussion about college teaching practices and students’ experiences in the classroom. And so the assignment that students did based on their work with this book was to create their own photo essay called “What Learning Looks Like.” So instead of what teaching looks like, what learning looks like, with a specific audience and purpose in mind. So that was one of the assignments that I liked. Another one that I thought was really promising was something called “share your story,” which asks students to tell a personal narrative about their educational experience, and then connect that to a larger issue of concern or a body of research in higher education. So of course, this is not totally AI proof. ChatGPT can make up a story, but it can’t tell my students’ own stories. And I think most of us really like to tell our stories to other people. So I think that provided a little more motivation for students to do their own work. And then the last assignment that I really liked was an assignment that actually asked students to work with ChatGPT. So they created a prompt based on a template that I provided for an argumentative piece that they then fed to ChatGPT, so they gave ChatGBT the prompt. And then they took the ChatGPT output and critiqued it. So they annotated it, noting what pieces they thought were strong, whether or not the piece had weaknesses, and what those weaknesses were, where it might need revision or overhaul. And then they had to totally rewrite the piece that ChatGPT produced and make it their own. So they had to do a substantial revision. And then they had to annotate their own work and tell me why they made the revision decisions that they did. And so I want to clarify that I do think it’s important that students learn to generate first drafts on their own, because drafting is an essential part of the writing process, and it’s where a lot of the thinking happens. But I like this assignment as just one assignment in the sequence, because it does help students learn about generative AI and develop some AI literacy. And it also helps students get over that terror of the blank page. So a lot of students procrastinate because they don’t know how to get started. They’ll open their laptop to begin a paper and stare at the blank screen. And I find it really difficult to get over that first hump of starting work, and just close their laptop and go away and try it again later, usually the night before it’s due. So I think starting in this way, with a ChatGPT prompt and an essay gives them a jumping off point, and it’s an easy way to help them start building the confidence they need to do their own first draft.

John: We should mention that we did talk to the authors of What Teaching Looks Like, and we’ll include a link to that discussion in the show notes. One of the things you described in your blog is that having this ungraded environment encourages students to be perhaps a little more open and honest with their instructors, but that could lead to some challenges in terms of additional emotional labor. Could you describe the challenges that you faced in this class with that?

Emily: I think one of the big themes in the spring class in particular was that I asked students to share with the class and for audiences beyond the class about their previous experiences in school, or their current experiences, and I think this also happens in a lot of ungraded classes. One common method of introducing an unfamiliar grading system is getting students to think about their previous experiences with grades. And so one thing that happened in this particular class is that students’ related a lot of past educational trauma to me, and usually that involves bad experiences with previous teachers. And I’m really glad that they were able to speak honestly about that sort of thing. And I think it helped them to have a teacher take them seriously when they related those stories. At the same time, it was pretty difficult to navigate those conversations, because I was managing their emotions, my emotions, and also doing that when sometimes I only knew one side of the story. And so that was a little bit difficult. Another thing is that students in this class didn’t seem to feel that their grade depended on telling me what I wanted to hear. So I think they were a lot more honest about their views than in other courses that I’ve taught. So in my traditionally graded classes, or I think, in any class where students are discussing hot button issues, they tend to think that they’ll be graded more harshly if they express views that the instructor disagrees with. So very often, I think, they try to say what they think you want to hear rather than what they really think. And that didn’t seem to happen as much in my spring course. I had several students endorse viewpoints that I definitely disagreed with whether they knew that or not. And I think that’s good, because students have to sort through their own views and values. But it also required me to do a lot of thinking about how I would approach and address students not just whose views I disagreed with, but whose expression of those views might be damaging to others or to themselves. And so, we need to do some work around community building and relationship building in the class. And then of course, when you build relationships with trust with students, they’re more likely to tell you about the personal problems that they’re facing, whether that’s their mental or physical health or their personal relationships or family emergencies or grieving. And it’s really just a lot. And I do sometimes lay awake at night worried about my students. And there’s a lot of care work happens, I think, in all classes and also especially ungraded classes. And so there’s a lot of work in referring students to other resources and helping them navigate campus resources, and also just a lot of kind of management of your own emotional state [LAUGHTER] that has to happen. So I did want to be honest on the blog about some of the emotional toll of the work of teaching in general and of ungrading too.

Rebecca: You’ve described the care work, you’ve described individual conferences with students, you’ve described students’ anxiety over not knowing where they are sometimes and allowing substantial revision. Can you talk a little bit about how all of those things play into workload and how you’ve managed things?

Emily: Yes, this is a good question. And I feel like I have a very complicated answer to whether or not ungrading has increased my workload, because I get asked this question a lot. In my traditionally graded classes, giving all that feedback felt like a waste of time for a few reasons. First, because I had the sense that students weren’t really reading the feedback. So we know from research that when students receive a letter grade and also feedback on their work, they tend to see the grade and then ignore the feedback, or at least receive the feedback as a justification for a grade, rather than something that’s going to help them improve their work in the future. So I also spend a lot of time in traditionally graded classes worrying about how students would receive my feedback if they got a low grade on their paper. So how could I write feedback that would be appropriately honest, but also appropriately encouraging, so that students didn’t just see a C-minus or a D on their paper, and then give up and throw it in the trash. So I don’t really worry about those things since I’ve adopted ungrading. I provide feedback honestly, and with the mindset of a coach rather than a judge. And I provide it with at least some confidence that students will read it and use it. So they have plenty of opportunity to revise. And in fact, their ultimate achievement of the course is measured by their growth in specific areas and their demonstration of learning that arises from taking a piece of writing from not so good to much better. And so the expectation is that what they turn in the first time is not their best work and they’ll only get to their best work after they incorporate feedback. So I do spend a lot of time responding to student work now that I’m ungrading. But the process is more efficient because it accomplishes my goals rather than wasting my time. And it’s more enjoyable, because it causes me less angst. So I guess it is more work to provide feedback, but it’s also more efficient and enjoyable work. I think I feel kind of the same way about individual conferences with students, that it does take quite a bit of time to do those and it would be, I want to acknowledge, so much more difficult, it may be impossible to do if I was teaching more classes and more students. I’m very lucky that I teach small class sizes. And because I work in educational developments and work in a teaching center for most of the time, I only teach one course at a time. So I think there are ways to do this with larger courses, but I’m very fortunate in my course to be able to conference individually with each student twice in the semester. And those conferences are incredibly time consuming, and they can be really draining, but they are also really joyful. And I think it’s really important that students are able to have those one-on-one conversations with me. And they are much more, I think, effective in accomplishing a lot of the goals that I have for student learning than just simply doing written feedback or peer review or things like that. Having that face-to-face time to give students some individual attention is a really both enjoyable and effective part of the learning process.

John: A question that often comes up from people who have not tried ungrading is how well do students’ perceptions of their learning align with your perceptions of the learning when you do have to assign those midterm or final grades in the class?

Emily: Yeah, this is a really good question. And I would say I’ve had to do a little bit of work on my process to make sure that our expectations are aligning well. Sometimes, there are cases where students don’t automatically start off knowing what I expect from their work and what good work looks like, even when I thought that I was clear about that. So that is an issue that I’m working on. I would say for the majority of students, they do understand what good work looks like. And when they’re asked to provide evidence for their course grade, most of the time they know what good evidence looks like and are able to demonstrate to me in really, sometimes ways that I hadn’t anticipated, that they really had learned in the class and progressed and improved their work. For those students who struggle, and I think it’s more frequent for first-year students to struggle with this than more advanced students, for students who do struggle, I think it is important to be able to show them models of student work early in the semester so that they can get a sense of what a successful assignment looks like or what’s kind of level of expectations we have for what student work counts as really excellent work. So that’s been really helpful. And I’ve also made some changes to my class this semester to help clarify for students a little bit what kind of evidence might be good evidence for specific grade proposals in the course. So if they are really shooting for an A or B in the course, what kinds of actions or behaviors or demonstrations of quality work do they need to be able to show in order to attain that grade?

Rebecca: I know one of the things that I struggled with in some of the classes where I’ve done ungrading is that sometimes the midterm conference feels like it comes too late. So I experimented, I think once with like a one-third, two-thirds, three thirds approach, but then that was so much more conferencing.

Emily: Yeah.

Rebecca: So I’m curious about your timing and how the midterm time works in the classes that you’ve been teaching.

Emily: Yeah, I think that’s where the self-assessment assignments come in. So I do think midterm is quite late for students to be getting the first level of feedback. And so I have students do a self-assessment form, I guess, a quarter way into the semester. So we start that process really early of having them look back at the work that they’ve done, and propose a grade for themselves based on that work. And when they do those self assessments, I don’t conference with students every time they do a self assessment, but I do look at where their grade proposals are. And I’m able to say if our expectations are very, very different, or assessments of that students work is very, very different, I’m able to reach out and say, “Hey, I don’t think we’re aligned here, and here’s why.” And being able to intervene early on that I think is really important.

John: We’re recording this near the end of the fall semester of 2023. And you’re currently teaching another ungraded class, could you tell us a little bit about what types of changes you made from the spring class to the fall class?

Emily: Sure, there are a few changes that I’ve made, because there were some things that I think did not go very well in the spring and I wanted to try to improve that. So what didn’t go well was that I had a real problem with attendance, there were quite a few students who struggled with their attendance, especially in the latter half of the semester, and quite a few students who struggled to submit their work on time kind of throughout the semester. And that made it difficult for me, but I think more importantly, it made it really difficult for the students who once they missed an assignment, they found it really difficult to get back on track. So my challenge is really to figure out how to motivate students to attend class regularly and submit work on time without penalizing them for absences or late work or without a kind of point system to encourage them to show up to class and to submit their work on time. So that was one thing I wanted to address. And then the other thing is that students’ anxiety about not knowing where they stand in the midst of the semester at any given point. So what I developed to address all of those challenges was a course progress tracker. And so this is a document, a pretty comprehensive document, inspired by David Clark’s Grading for Growth post about grade trackers that I read during the summer while I was designing this course. And so the document is really a series of tracking worksheets in three different categories. So the first category is readings and assignments, students can see each week at a glance what work they have to complete, and then they can check off boxes as they complete that work, and then note, if there are assignments they’re submitting late, they can record those late submissions. There’s a category for attendance and engagement where students can check off the classes that they attend, and then make notes about their in or out of class engagement during each week of the semester. And then there’s a section… the most important section… for learning and growth where students can remind themselves of what the course goals are, and then track their progress along those goals, so they can see and make notes about where they’re still developing, where they’re doing excellent work, and how they’re improving their writing as the semester goes along. So, so far, it’s been going really well. And I’ve had fewer challenges with attendance and late work this semester than last semester, though, I can’t say to what extent that’s just a result of a different population of students or the fact that it’s fall instead of spring. So I will add that caveat, but I have surveyed students about their use of the progress tracker last week, and I’m really looking forward to diving into that next week. So the last thing that I think the progress tracker does, which I didn’t totally intend, but which has been really excellent, is that I think it helps students a lot with their self-assessment work, which they also struggled with, to some extent last semester, or I should say, maybe I struggled to teach really well. So I had a realization at the end of last semester, that when I sat down to think about a student’s work over the course of this semester, I was really thinking about three things. And so the first thing is quantity. How much work did students do? How much labor did they put in? How many assignments did they submit at a satisfactory level? How many class days did they attend? How much time did they spend on their major assignments? So that was one category. The second was quality. So how good was the work that they were doing according to the standards that we laid out? Was it still kind of developing work? Or was it excellent work? And then the third thing was growth. So how and how much did student work improve over the course of this semester? And can they demonstrate that they’ve gained knowledge and skills that they didn’t have before? Or can they demonstrate that they’re better off from having taken the course. And so while we’ve had conversations about those things in the spring, I never articulated to myself or to my students, that particular model; I never kind of said it in quite this way. So the last section of the progress tracker includes a guide to determining final letter grades for students. And it gives them space to think about quantity, quality and growth, and how that might contribute to the grade that they propose for themselves, if they’re interested in the specific letter grade. And it gives them a sense of what kinds of evidence they could provide if they want to show good evidence for a specific letter grade. So you can provide good evidence for an A if your work rises to the level of excellent on multiple assessment categories. Or if you attended class every time that you were able and submitted the vast majority of your writing practice assignments. So all of these things are good pieces of evidence if you want to propose an A grade for yourself. So there’s some flexibility there, I don’t prescribe exactly what students have to do for a specific grade, but I do make suggestions and say, here’s a guideline for you. If you’re confused about what kinds of work you need to be doing, or what you need to do to demonstrate that you’ve attained a certain letter grade, you can take a look at this guide, and learn a little bit more about what the expectations are. Currently, it’s just something for the students to use for themselves. I have provided time in class for students to fill out sections of it, because I think that if I did not ask them to do it in class, they might struggle to keep up with it outside of class. But I think maybe in the future, I will consider asking them to keep up with it. And then periodically checking in on those progress trackers throughout the semester so that I can intervene early, if there are any issues and maybe leave comments on the documents to let students know if I think they’re struggling in a specific area, or if they’re doing really well in a specific area. I am playing around with that idea for my next version of the class.

John: So we always end with the question: “What’s next?”

Emily: So this spring, I won’t be teaching an undergraduate course, but I will be working with graduate students who are preparing to teach their own Writing and Rhetoric courses. And I’m really looking forward to talking with them about teaching. And I’ll also be continuing to write about ungrading on the blog. So specifically, I’m hoping to share a bit more about what I learned from my experiences this semester. And I’ve been collecting data from my current students about their use of the progress tracker, about their use of AI in their writing this semester, and about their feelings and impressions of ungrading. And what I’m planning to do throughout the spring is share some of those student thoughts with the readers of the blog, and I’m really excited to be able to share students’ ideas about ungrading and other topics as well.

Rebecca: Sounds great. We’ll look forward to reading that for sure.

John: Thank you, Emily. It’s great talking to you and we look forward to future conversations.

Emily: Thanks, this has been fantastic.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

311. Upskilling in AI

With so many demands on faculty time, it can be difficult to prioritize professional development in the area of AI. In this episode, Marc Watkins joins is to discuss a program that incentivizes faculty development in the AI space. Marc is an Academic Innovation Fellow at the University of Mississippi, where he helped found and currently directs the AI Institute for Teachers.

Show Notes

Transcript

John: With so many demands on faculty time, it can be difficult to prioritize professional development in the area of AI. In this episode, we examine a program that incentivizes faculty development in the AI space.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Marc Watkins. Marc is an Academic Innovation Fellow at the University of Mississippi, where he helped found and currently directs the AI Institute for Teachers. Welcome back, Marc.

Marc: Thank you, John. Thank you, Rebecca. It’s great to be back.

Rebecca: We’re glad to have you. Today’s teas are:… Marc, are you drinking tea?

Marc: I am. I have a Cold Brew Hibiscus, which is really great. It’s still very warm down here in Mississippi. So it’s nice to have something that’s a little bit cool. That’d be refreshing.

Rebecca: That sounds yummy. How about you, John?

John: I am drinking a peppermint spearmint tarragon blend today. And it’s not so warm here. In fact, my furnace came on for the first time yesterday.

Rebecca: Yeah, transitions. And, I have English tea time today.

Marc: Well, that’s great.

John: So we have invited you here to discuss your ongoing work related to ChatGPT and other AI tools. Could you first describe what the AI Institute for Teachers is and its origins?

Marc: Sure, I think that when I was last a guest here in January of this year on your show. And it seems like 1000 years ago [LAUGHTER], but during that spring semester, I really took a much deeper dive than the original pilot with a lot of the generative AI tools in the fall. And we started noticing that the pace that big tech was deploying these tools and integrating these with existing software from Microsoft and Google was only accelerating. So in about April or May, I went to my chair, Stephen Monroe, and said, “I think we need to start training some people to get them prepared for the fall,” because we kind of thought that fall was going to be what it is right now, which is a chaotic just sort of mash up of sort of everything you can imagine that some people dive in deeply, some people tried to ban it, some people are trying to do some critical approaches with it too. So we actually worked with the Institute of Data Science here at the University of Mississippi, and we got some money. And we were able to pay 23 faculty members $1,000 apiece to train them for a day and a half about everything we knew about Generative AI, about AI literacy, ethics, what tools were working in the classroom, which wasn’t. And their whole goal was to go back to their home departments over the summer and serve as ambassadors and help prepare them for the fall semester. And we started that, we’ve had funding for one Institute, and now we’re doing workshops, and searching, as we all will, for more funding for doing,

Rebecca: How did faculty respond to (A) the incentive, but (B) also [LAUGHTER] the training that went with it?

Marc: Well, not surprisingly, they responded really well to the incentives, where you can pay people for their time, they generally do show up and do so as well. We had quite a few people wanting to take the training both internally from the University of Mississippi and then people started finding out about it, because I was posting it out on Twitter, and writing about it on my substack. So when we had interest from graduate students in Rome, interest from other SEC schools wanting to attend, and even interest from a community college in Hawaii. Definitely seen a lot of interest within our community, both locally and more broadly, nationally.

Rebecca: Did you find that faculty were already somewhat familiar with AI tools? I had an interesting conversation with some first-year students just the other day, and we were talking about AI and copyright. And I was just asking, “Hey, how many of you have used AI?” And I and another faculty member indicated that we had used AI to make it safe to indicate. And many of them really kind of shook their heads like “No, they hadn’t,” and they were unsure. And then I started pointing to places where we see snippets of it, in email and in texting and other places where there’s auto-finishing of sentences and that kind of thing. And then they’re like, “Oh, yeah, I have seen that. I have engaged with that. I have used that.” What did you find faculty’s knowledge?

Marc: Extremely limited. They thought of AI as ChatGPT. And one of the things we did with the session was basically frame it out as “Look, this was not just going to remain as a single interface anymore.” One of the things that actually happened during the institute that was completely wild to me was the last day, I woke up that morning. And I’d signed up through Google Labs, and you can do it as well, to turn on the features within the Google suite of tools, including in search and Google Docs, and Sheets and everything else. And they gave me access that last day, right before we began. And so I literally just plugged in my laptop and said, “This is what it’s going to look like in Google docs when you have generative AI activate in Google Docs. it pops up and immediately greets you with a wand with a phrase “Help me write.” And what I tried to explain to them and explained to faculty ever since then, is that it makes having a policy against AI very difficult when it shows up at an existing application with no indication whatsoever that this is in fact Generative AI. It’s just another feature that’s in the application that you have grown up with, from many of our students’ perspectives their entire lives. So yeah, we need to really work on training faculty, not just in the actual systems itself, but also getting them outside of that mindset that AI that we’re talking about is just ChatGPT. It’s a lot more than that.

John: Yeah, in general, when we’ve done workshops, we haven’t had a lot of faculty attendance partly because we haven’t paid people to participate [LAUGHTER], but what’s been surprising to me is how few faculty have actually explored the use of AI. My experience with first-year students was a little different than Rebecca, about half of the students in my large intro class had said that they had explored ChatGPT, or some other AI tool. And they seem pretty comfortable with it. But faculty, at least in our local experience, have generally been a bit avoidant of the whole issue. I think they’ve taken the approach that this is something we don’t want to know about, because it may disrupt how we teach in the future. How do you address that issue, and getting faculty to recognize that this is going to be a disruptive technology in terms of how we assess student learning and in terms of how students are going to be demonstrating their learning, and also using these tools for the rest of their lives in some way?

Marc: That’s a great question. We trained 23 people, I’ve also been holding workshops for faculty too, and again, the enthusiasm was a little bit different in those contexts, too. And I agree that faculty, I feel like they feel overwhelmed and maybe some of them want to ignore this and don’t actually want to deal with it, but it is here and it is being integrated at phenomenal rates in everything around us too. But if faculty don’t come to terms with us, and start thinking about engagement with their technology, both for themselves and for their students, then it is going to create incredible disruption that’s going to be lasting, it’s not going to go away. We’re also not going to have things like AI detection, like it is with plagiarism detection to come in and save the day for them too. And those are all things we’ve been trying to very carefully explain to faculty and get them on board. Some of them though, just aren’t there yet, I understand that. I empathize, too. This is a huge amount of time that you spend on these things to think about and talk about as well. And we’re just coming out of the pandemic, people are exhausted, they don’t want to deal with another, quote unquote, crisis, which is another thing that we’re seeing too. So there’s a lot of factors that are at play here that make faculty engagement, less than what I’d like to see.

Rebecca: We had a chairs’ workshop over the summer, and I was somewhat surprised based on our experience with other interactions with faculty, how many chairs had used AI. The number was actually a significant number. And most of them were familiar. And that to me was encouraging [LAUGHTER], it was like, “Okay, good, the leaders of the ship are aware. That’s good, that’s exciting.” But it’s also interesting to me that there are so many folks who are not that familiar, who haven’t experimented, but seem to have really strong policies around AI use or this idea of banning it or wanting to use detectors, and not really being familiar with what they can and cannot do.

Marc: Yeah, that’s very much what we’re seeing across the board too, is that the first detectors that I’m aware of that really came online, I think, for everyone was basically GPTZero, there are a few others that existed beforehand to IBM had one called the Giant Language Testing Lab. But those were all based on GPT-2, you’re going back in time to 2019. I know how ridiculous is it to go back four years in technology terms and think about this… that was a long time ago. And we really started adopting that through education or seem to be adopted in education based off of that panic. The problem is in incidents of education putting a system like that in place, it’s not necessarily very reliable. TurnItIn also adopted their own AI detector as well too. A lot of different universities began to explore and play around with it, I believe, and I don’t want to be misquoted here or misrepresent TurnItIn. I think what they initially came out with it, they were saying there was only 1% false positive rate for detecting AI. They’ve since raised that to 5%. And that has some really deep implications for teaching and learning. Most recently, Vanderbilt Center for Excellence in Teaching and Learning made the decision to not turn on the AI detection feature in TurnItIn. Their reasoning was that they had, I think, in 2022 some 75,000 student papers submitted. If they had the detector on during then that would give them a false positive grade about 3000 papers. And they just can’t deal with that sort of situation through a university level..No one can. You’d have to go through it investigating each one. You would also have to get students a hearing because that is part of the due process. It’s just too much. And that’s one of the main concerns that I have about the tools that it’s just not reliable in education.

John: And it’s not reliable both in terms of false positives and false negatives. So some of us are kind of troubled that we have allowed the Turnitin tool to be active and have urged that our campus shut it down for those very reasons, and I think a number of campuses, Vanderbilt was one of the biggest ones, I think to do that, but I think quite a few campuses are moving in that direction.

Marc: Yes, the University of Pittsburgh also made the decision to turn it off. I think several others did as well, too.

Rebecca: It’s interesting, if we don’t have a tool to measure, a tool to catch if you will, then you can’t really have a strong policy saying you can’t use it at all. [LAUGHTER] There’s no way to follow up on that or take action on that.

Marc: Where we’re at, I think, that for education, that’s a sort of conundrum. We’re trying to explain this to faculty. I think much more broadly, in society, though, if you can’t have a tool that works when you’re talking about Twitter, I’m sorry, X now, and understanding if the material is actually real or fake, that becomes a societal problem, too, and that’s what they’re trying to work on with watermarking. And I believe the big tech companies have agreed to watermark audio outputs, video outputs, and image outputs, but they’ve not agreed to do text outputs, because text is a little bit too fungible, you can go in and you can copy it, you can kind of change it around a little bit too much. So, definitely it’s gonna be a problem, too when state governments start to look at this, and they start wondering that the police officer taking your police report is writing this with their own words, the tax official using this as well, too. So it’s gonna be a problem well outside of education.

Rebecca: And if we’re not really preparing our students for that world in which they will likely be using AI in their professional fields, then we’re not necessarily doing our jobs and education and preparing our society for the future.

Marc: Yeah, I think training is the best way to go forward too and again, going back to the idea of intentional engagement with the technology and giving the students these situations where they can use it and where you, hopefully if you’re a faculty member, you actually have the knowledge and the actual resources to begin to integrate these tools and talk about the ethical use case, understanding what the limitations are and the fact that it is going to hallucinate and make things up, and to think about what sort of parameters you want to put on your own usage too.

John: One of the things that came out within the last week or so, I believe,… we’re recording this in late September… was the introduction of AI tools into Blackboard Ultra. Could you talk a little bit about that?

Marc: Oh boy, yes indeed, they announced last week that the tools were available to us in Blackboard Ultra. They turned it on for us here at the University of Mississippi, and I’ve been playing around with it, and it is a little bit problematic, because for right now, what you can do is with a single click, it will scan your existing materials in your Ultra course and it will create learning modules. It will create quiz questions based off that material, it will create rubrics, and will also generate images. Now compared to what we’ve been dealing with ChatGPT and all these other capabilities, this is almost a little milquetoast by comparison. But it’s also an inflection event for us in education, because it’s now here, it’s directly in our learning management system, it’s going to be something we’re going to have to contend with every single time we open up to create an assignment, or to do an assessment. And I’ve played around with it. It’s an older version of GPT. The image version I think is based on Dall-E, so you would ask for a picture of college students and you get some people with 14 fingers and weird artifacts all over their face, which may not be the one that would actually be helpful for your students. And while the other learning modules there are not my thinking necessarily, it’s just what the algorithm is predicting based off the content that exists in my course. We have that discussion with our faculty, we have them cross that Rubicon on and saying, “Okay, I’m worried about my students using this, what happens to me and my teaching, my labor, if I start adopting these tools. There could be some help, definitely, this could really streamline the process, of course creation and actually making it aligned with the learning outcomes my department wants for this particular class.” But it also gets us in a situation where automation is now part of our teaching. And we really haven’t thought about that. We haven’t really gotten to that sort of conversation yet.

Rebecca: It does certainly raise questions about, obviously, many ethical questions and really about disclosing to students what has been produced by us as instructors and what has been produced by AI and authorship of what’s there. Especially if we’re expecting students to [LAUGHTER] do the same thing.

Marc: It is mind boggling, the cognitive dissonance, with having a policy and saying “No AI in my class,” then all of a sudden, it’s there in my Blackboard course, and I could click on something. And, at least at this integration of Blackboard, they may very well change this, but once you do this, there’s no way to natively indicate that this was generated by AI. You have to manually go in there and say this was created. And I value my relationship with my students, it’s based off of mutual trust. I think almost everyone in education does. If we want our students to act ethically, and use this technology openly, we should expect ourselves to do the same. And if we get into a situation where I’m generating content for my students and then telling [LAUGHTER] them that they can’t do the same with their own essays, it is just going to be kind of a big mess.

John: So given the existence of AI tools, what should we do in terms of assessing student learning? How can we assess the work reasonably given the tools that are available to them?

Rebecca: Do you mean we can just use that auto-generated rubric right, that we just learned about? [LAUGHTER]

Marc: You could, you can use the auto-generated rubric separately from Blackboard. One of the tools I’m piloting right now is the feedback assistant, it was developed by Eric Kean and Anna Mills. I consulted with them on this, too. She’s very big on the AI space for composition. It’s called MyEssayFeedback. And I’ve been piloting this with my students. They know it’s an AI, they understand this. I did get IRB approval to do so. But I’ve just got the second round of generated feedback, and it’s thorough, it’s quick, it’s to the point. And it’s literally making me say, “How am I going to compete with that?” And maybe the way is that maybe I shouldn’t be competing with that, maybe it’s I’m not going to be providing that feedback. But then maybe then I should be providing my time in different ways. Maybe I should be meeting with them one on one to talk about their experiences, maybe that way. But I think you raise an interesting question. I don’t want to be alarmist, I want to be as level-headed as I can. But from my perspective, all the pieces are now there to automate learning to some degree. They haven’t been all hooked up yet and put together a cohesive package. But they’re all there in different areas. And we need to be paying attention to this.Our hackles need to be raised just slightly at this point to see what this can do. Because I think that is where we are headed with integrating these tools into our daily practice.

Rebecca: AI generally has raised questions about intellectual property rights. And if our learning management systems are using our content in ways that we aren’t expecting, how is that violating our rights or the rights that the institution has over the content that’s already there.

Marc: A lot of perspectives of the people that I speak with too, their course content, their syllabi, from their perspective is their own intellectual property in some ways. We get debates about that, about the actual university owns some of the material. But we have had instances where lectures were copyrighted before in the past. And if you’re allowing the system to scan your lecture, you are exposing that to Generative AI. And that gets at one aspect of this. The other aspect, which I think Rebecca is referring to is the issue with training this material for these large language models itself could indicate that it was stolen or not properly sourced from internet and you’re using it and then you’re trying to teach your students [LAUGHTER] to cite material correctly too, so it’s just a gigantic conundrum of just legal and ethical challenges. The one silver lining in all this, and this has been across the board with everyone in my department. This has been wonderful material to talk about with your students, they are actually actively engaged with it, they want to know about this, they want to talk about it. They are shocked and surprised about all the depths that have gone into the training of these models, and the different ethical situations with data and all of it too. And so if you want to just engage your students by talking to them about AI too, that’s a great first step in developing their AI literacy. And it doesn’t matter what you’re teaching, it could be a history course, it could be a course in biology, this tool will have an impact in some way shape or form in your students’ lives they want to talk about, I think maybe something to talk about is there are a lot of tools outside of ChatGPT, and a lot of different interfaces as well, too. I don’t know if I talked about this before in the spring, the one tool that’s really been effective for a lot of students were the reading assistant tools, one that we’ve been employing is called ExplainPaper. They upload a PDF to it, it calls upon generative AI to scan the paper and you can actually select it to whatever reading level you want, then translate that into your reading level. The one problem is that students don’t realize that they might be giving up some close reading, critical reading skills to it as well too, just like we do with any sort of relationship with generative AI. There is kind of that handoff and offloading of that thinking, but for the most part, they have loved that and that’s helped them engage with some really critical art texts that normally would not be at their reading level that I would usually not assign to certain students. So those are helpful. There are plenty of new tools coming out too. One of them is called Claude 2 to be precise by Anthropic. That just came out, I think, in July for public release, it is as powerful as GPT-4. It is free right now, if you want to sign up for it as well too. The reason why I mentioned Claude is that the context window, what you can actually upload to it is so much bigger than ChatGPTs. I believe their context window is 75,000 words. So you can actually upload four or five documents at a time, synthesize those documents. One of the things I was using it for as a use case was that I collected tons of reflections for my students this past year about the use of AI. It’s all in a messy Word document. It’s 51 pages single spaced. It’s all anonymized so there’s new data that identifies them. But it’s so much of a time suck on my time, just go through to code those reflections. And I’ve just been uploading to Claude and having it use a sentiment analysis to point out what reflections are positive from these students, in what way, and it does it within a few seconds. It’s amazing.

John: One other nice thing about Claude is that has a training database that ends in early 2023. So it has much more current information, which actually, in some ways is a little concerning for those faculty who were trying to ask more recent questions, particularly in online asynchronous courses, so that ChatGPT could not address those. But with Claude’s expanded training database, that’s no longer quite the case.

Marc: That’s absolutely correct. And to add to this rather early discussion about AI detection, none of the AI detectors that I’m aware of had time to actually train on Claude, so if you generated essay… and you guys are free to do this on your own, your listeners are too… if you generated and essay with Claude, and you try to upload that to one of the AI detectors, very likely you’re going to get zero detection or a very low detection rate for it too, because it’s again, a different system. It’s new, the existing AI detectors have not had time. So the way to translate this is don’t tell your students about it right now, or in this case, be very careful about how you introduce this technology to your students, which we should do anyway. But this is one of those tools that is massively popular, a lot of people just haven’t known about it because, again, ChatGPT just takes up all the oxygen in the room when we talk about Generative AI

John: What are some activities where we can have students productively use AI to assist their learning or as part of their educational process?

Marc: That’s a great question. We actually started developing very specific activities for them to look at different pain points for writing classes. One of them was getting them to actually integrate the technology that way. So we built a very careful assignment, which called on very specific moves for them to make both in terms of their writing, and their integration of the technology for that. We also looked at bringing some research question, building assignments that way. We have assignments from my Digital Media Studies students right now about how they can use it to create infographics. Using the paid for version of ChatGPT Plus, they can have access to plugins, and those plugins then give them access to Canva and Wikipedia. So they can actually use Canva to create full on presentations based off of their own natural language and use actual real sources by using those two plugins in conjunction with each other. I just make them then go through it, edit it with their own words, their own language too, and reflect on what this has done to their process. So lots of different examples, too, I mean, it really is limited only to your imagination in this time, which is exciting, but it’s also kind of the problem that we’re dealing with, there’s so much to think about.

Rebecca: From your experience in training faculty, what are some getting started moves that faculty can take to get familiar enough to take this step of integrating AI by the spring?

Marc: Well, I think the one thing that they could do is, there are a few really fast courses. I think it’s Ethan Mollick from even from the Wharton School of Business put out a very effective training course that was all through YouTube, I think it’s like four or five videos, very simple to take, to get used to understanding how ChatGPT works, how Microsoft’s Bing works as well too, and what sort of activities students can use it for, what sort of activities faculty could. Microsoft has also put out a very fast course, I think takes 53 minutes to complete about using generative AI technologies in education. And those are all very fast ways of basically coming up to speed with the actual technology.

John: And Coursera has a MOOC through Vanderbilt University, on Prompt Engineering for ChatGPT, which can also help familiarize faculty with the capabilities of at least ChatGPT. We’ll include links to these in the show notes.

Marc: I really, really hope Microsoft, Google and the rest of them calm down, because this has gotten a little bit out of control. And integration of these tools are often without use cases, they’re often waiting to see how we’re going to come up and use them too. And that is concerning. Google has announced that they are committed to releasing their own model that’s going to be in competition with GPT4, I think it’s called Gemini by late November. So it looks like they’re just going to keep on heating up this arms race and you get bigger models, more capable and I think we do need to ask ourselves more broadly what our capacity is just to keep up with this. My capacity is about negative zero at this point… going down further.

John: Yeah, we’re seeing new AI tools coming out almost every week or so now in one form or another. And it is getting difficult to keep up with. I believe Apple is also planning to release an AI product.

Marc: They are. They also have a car they’re planning to release, which is the weirdest thing in the world to me, that there could be your iPhone charged in your Apple Car.

John: GM has announced that they are not going to be supporting either Android or Apple CarPlay for their electric vehicles. So perhaps this is Apple’s way of getting back at them for that. And we always end with the question, what [LAUGHTER] is next, which is perhaps a little redundant, but we do always end with that.

Marc: Yeah, I think what’s next is trying to critically engage the technology and explore it not out of fear, but out of a sense of wonder. I hope we can continue to do that. I do think we are seeing a lot of people starting to dig in. And they’re digging in real deep. So I’m trying to be as empathetic as I can be for those that don’t want to deal with the technology. But it is here and you are going to have to sit down and spend some time with it for sure.

John: One thing I’ve noticed that in working with faculty, they’re very concerned about the impact of AI tools on their students and student work. But they’re really excited about all the possibilities that opens up for them in terms of simplifying their workflows. So that, I think, is a positive sign.

Rebecca: They could channel that to help understand how to work with students.

Marc: I hope they find that out, there’s a positive pathway forward with that too.

John: Well, thank you. It’s great talking to you and you’ve given us lots more to think about.

Marc: Thank you guys so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

304. ChatGPT Inspired Course Redesign

AI tools such as ChatGPT have the potential to significantly disrupt how we work and how we learn. In this episode, Don Donelson joins us to discuss a course redesign strategy that could help prepare students for a world in which AI tools will be ubiquitous. Don is a senior lecturer in the Miami Herbert Business School at the University of Miami. He is a recipient of the Spring 2016 University of Miami Excellence in Teaching Award and the Dean’s Excellence in Teaching Award from the Miami Herbert Business School.

Show Notes

Transcript/h3>

John: AI tools such as ChatGPT have the potential to significantly disrupt how we work and how we learn. In this episode, we discuss a course redesign strategy that could help prepare students for a world in which AI tools will be ubiquitous.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Don Donelson. Don is a senior lecturer in the Miami Herbert Business School at the University of Miami. He is a recipient of the Spring 2016 University of Miami Excellence in Teaching Award and the Dean’s Excellence in Teaching Award from the Miami Herbert Business School. Welcome back, Don.

Don: Glad to be here.

John: Today’s teas are: …Don, are you drinking tea?

Don: I am. It’s the same tea that you’re drinking, black currant, and it’s great.

Rebecca: It’s a John favorite for sure. I have a Tazo Awake tea today.

John: Does that mean you’re woke? That may be an issue down in Florida.

Don: …not in a private school.

Rebecca: It means that I couldn’t make a pot of tea. I didn’t have time. So I had to use a single tea bag. [LAUGHTER] That’s what it means.

John: And I am still using the mug from Australia that Clare McNally gave me with kangaroos all over it.

Rebecca: I like that mug.

John: I do too.

Rebecca: I look forward to seeing it in person.

John: Soon.

Rebecca: Yeah, you’ll be back soon, right? A couple weeks.. I’ve got my grad studies mug. We’ve invited you here today to discuss your plans to revise the graduate and undergraduate core courses in critical thinking and business communication at Miami Herbert. Can you tell us a little bit about this course?

Don: So this course started at the grad level, MBAs in particular, in 2008. That’s what I was hired to teach. And it grew with the program, expanding into specialized master’s programs. And then it went out into the undergrad program. And it’s a core course required for all full-time business students, undergrad and graduate. This past year, we had 46 sections of undergrad courses and 21 sections of graduate courses, about 900 students or so in total.

Rebecca: So a really small situation going on here.

Don: Oh, yeah, very small, [LAUGHTER] no problems with scaling or anything like that.

John: What was the typical focus of this course in the past before this revision that you’re working on?

Don: So the course was called “Critical Thinking and Effective Written and Oral Communication,” and it lived up to its name. It was about those three things. At the time that we started in 2008, we called them soft skills. We don’t use that phrase anymore. We like to call them fundamentals, something of the sort. We think that soft skills sends a bad message. But it’s been overhauled three times, this will be the third overhaul since. And the things that we would do in the courses, from the very beginning, the main evaluations would be based on writing memos and giving presentations

Rebecca: Which should be about the kind of communication you’d be doing in business. [LAUGHTER]

Don: Yeah, it’d be based on hypothetical cases, some non-hypothetical cases, the standard Harvard Business publishing 10, 20 page case on “How did Netflix beat Blockbuster?” or something of the sort.

Rebecca: What prompted this big overhaul?

Don: Well, the accreditation body, AACSB, required program evaluation. And it’s sometimes an annoying task that people do just to go through the motions. But what we found, the first time I went through it, was that we actually learned a lot from going through those motions. And so in my department, at least, we institutionalize curriculum audits on a semester basis. And so, in between the fall and spring semesters, we have a shorter meeting, where we kind of look at what happened in all the instructors, faculty teaching in that space in the fall, what happened and what worked, what didn’t work, and we might make some minor revisions. And then at the end of the spring semesters, we’d have a little bit of a longer meeting. And the last few of those had turned out some opportunities for change.

Rebecca: Can you talk a little bit about what some of those opportunities were?

Don: Yes. So we found that this is a challenging course to get buy-in from the students. And so we still haven’t figured out 10% of what we can figure out about teaching in this space, I’m sure. But one of the things that we’ve seen is that it is somewhat of an innovative curriculum, and one of the challenges with being innovative is students haven’t had material like that before. It’s a core course. And so students have to take it. And there’s always challenges with that. But this is a bit of a different challenge. And so I was talking with John the other day, he has some core required economics courses for business students, and some of the challenges that come with that. But this is a bit different in that those students know that economics is a field of study. And they know that people take economics courses, and there’s a textbook and critical thinking and communication. They’ve been taught kind of as separate. They’ve been add ons in other courses, not as a discrete course itself. And so we think there’s some challenges with that. And really, the challenge that we’ve seen, in addition to that, is that from the faculty view, critical thinking and communication are not separate things, they are one thing. And so critical thinking, I would call it problem solving, is really what we’re teaching. But communication is a component of that. And from the student view, we’ve had a hard time getting them to see those as integrated. And so when they do a memo, that’s an evaluation metric, they see it as: “Well, that’s just looking at the writing and not critical thinking.”

Rebecca: That’s interesting. Some of the things that we’ve done in our design courses around critical thinking and writing across the curriculum, my department, which is art and design, is doing some of those same things. We would do projects and embedded in those projects would be things like memos and other ways of communicating as a way to critically think about the decisions that our designers were making on things. But we would run up sometimes against the same kinds of challenges, like how do you really make that feel practical, that’s relevant, and then also keep it interesting. And it helped, I think, in those cases, because it was tied to a project. So is that a challenge that you face in this particular class is because there are these kind of standalone case studies, and it’s hard for students to buy in or get them into a business space?

Don: That’s one of the things actually I think that’s going to be changed is more of an arc to the course. And one of the things that I’m looking at is more integration of assignments. And so things building more towards the other assignments, and so we have skills building on top of each other. But, ideally, the assignments that they’re doing all build towards one culmination assignment, capstone type project.

Rebecca: Where does this course fit into their other required courses? Is it something that happens in the beginning? or in the middle? towards the end?

Don: So that’s partially an administrative question that is dependent on staffing. We see some students wait until the very last semester to take it, particularly the students who don’t have English as a first language, but they can start taking it as early as their sophomore year. But usually, it’s junior.

John: What’s the difference between the undergraduate and the graduate versions of the course?

Don: So the graduate versions are taken on a quarterly basis, and the undergraduates on a semester basis and so there’s more contact time in the undergraduate version of the course. They use different materials, and they’re more in depth. And so much like you would see with undergraduate economics classes, the graduate version of the economics classes might have similar titles, but go far more in depth into the material.

John: So one of the main issues is that students don’t see the critical thinking aspect of it as being important in their writing. How are you going to change your course to focus a bit more on the development of those critical thinking skills?

Don: Well, this is where I need to go back and add more to what Rebecca asked before about what prompted this because of course, ChatGPT prompted a lot of the revisions as well. And so ChatGPT, AI in general, while it’s kind of an independent axis of revision, we were thinking about some of these other problems well, before ChatGPT even became a thing that people were aware of. But they go hand in hand, really. A lot of the problem that I’ve seen with the writing assignments, and why students don’t necessarily view them as critical thinking and focus on the writing, is because there’s writing for aesthetic, and then there’s writing for substance. And if you’re teaching anything about writing, you kind of have to be teaching both. But when you’re teaching both together, the students tend to focus more on the aesthetic. And they connect it back to English composition classes that they might have taken in ninth grade or 10th grade. And those classes are certainly very important. But they’re a bit different than what we’re doing in these classes. And so I think it primes them to approach the course in a way that is not really conducive to getting what we want out of it. And so with AI, well, it remains to be seen, but it looks to me like you don’t need to be teaching the aesthetics of writing so much anymore in a class like this. And so I’m going to experiment with just not. [LAUGHTER]

Rebecca: It’s interesting, because in design classes where we were doing some similar kinds of things, aesthetics obviously always come about, because if we’re doing visual design, aesthetics are a part of that conversation. But we would have the same thing. It was like, “Well, that looks nice. That reads nice. It just doesn’t say anything.” [LAUGHTER]

Don: Right. Yeah. [LAUGHTER]

Rebecca: So it’s interesting that we bump up against these same kinds of challenges across a wide variety of disciplines. And that ChatGPT does offer some opportunity to focus on some different things.

Don: Absolutely.

Rebecca: I’m curious what exactly you’re going to focus on and how you’re gonna leverage ChatGPT in the context of this class.

Don: So I think ChatGPT is an insane, wild, amazing tool. And it’s going to only be more wild, more insane, more amazing next month, [LAUGHTER] or six months from now. But I see it really changing the way that I teach, the way I prepare for teaching. So kind of on the end of creating lesson plans and what it does for me as a teacher and the kinds of things that I can do in a class that I wouldn’t be able to do in a class before without a lot more hours in the day. And then also, from the student side, changing the way that they do assignments. They’re not going to be writing memos outside the classroom in the way that they have in business for decades and centuries. They’re going to be using ChatGPT. And so if, in this course, which is meant to be a practical course, we can’t make it practical if we’re not allowing them to use the tools that they’re not only going to be encouraged to use, they’re going to probably be required. And so if they don’t use ChatGPT in the future, they’re going to have bosses saying, “Why are you spending X amount of hours on this client memo instead of doing something else.” And so we really need to prepare them for that world. There’s been some early research and I’ll get John the citations, but we looked at research over the break between fall and spring, this past year, some preliminary research about the kinds of jobs in the way the labor market is going to be affected in the future by AI and ChatGPT and the jobs that were predicted to be the hardest hit in terms of reduced wages, and just reduced demand are jobs that involve writing and the jobs that were predicted to be the most insulated from Ai were jobs that involve problem solving and critical thinking. And so really, when you look at that research, it doesn’t even give us a choice. Even if we weren’t thinking about making some kind of revisions before, we’d probably need to just on that alone.

John: So is the focus now shifting more to the critical thinking skills and a little bit less on the basic structure of writing?

Don: Yes. And that’s really where even though the impetus for the revisions were independent, in practice, they’re not going to be that independent. And so it really dovetails nicely. And so I’ll give you an example, if a student is writing a memo, where a business is making a decision between two or three different courses of action, and one of the main criteria is the profitability of those courses of action, the structure is kind of guided by the math of profitability. So if you’re not talking about revenues, independently of talking about costs, you’re not proving profitability. And so when we talk about structure in this course, that’s really what we mean. But students very often, because of some of the things I’ve talked about previously, they’re looking at it as far as like the five-paragraph structure. And that’s not really what we mean. And so by being able to focus less on the aesthetics of writing, and more on the substance, I think we’ll be able to undo some of that priming,

Rebecca: …almost like this shift to articulating the decision making…

Don: Yes.

Rebecca: …rather than talking about writing, because articulating, it could be verbal, it could be in written language, it could be in a lot of different formats. But the point is that you thought critically about the issue, and how you made the decision. [LAUGHTER]

Don: Yes, exactly. In presentations, I’ve never had as much of the same problems as we have in the memos. Part of that, I think, it’s because of contemporaneous feedback. My students early on learned that this comment is kind of a trolling comment. And it’s not really meant as a attaboy or attagirl. But, sometimes a student will give a speech, and when they’re done, I’ll say, “I’m very impressed with your public speaking skills.” And they think, at first in the early parts of the class, that that’s a compliment. But they realize that that’s actually not a compliment. What I really mean is “No one would be buying what you’re selling, no one would be buying this stock, no one would be making a decision based on this, but you have very impressive charisma and confidence.” And that’s not really what we’re about, maybe in politics, but that’s a different question.

Rebecca: I’m curious about integrating ChatGPT as part of the process. Are you thinking about requiring students to reveal and discuss how and why they use ChatGPT in particular instances, and how they leverage the tool.

Don: So I think part of it is going to be first showing them how ChatGPT is not a critical thinking tool. And so I think it’ll be kind of walking on the escalator backwards for a bit just so that we can walk forward. It’s not going to be ChatGPT’s here, so you should use it. Go. ChatGPT is like a personal assistant, who is extremely capable and competent, but will do precisely what you tell it to do, and nothing else. The input you give it determines the quality of the output. And so if you go to ChatGPT, and you say, “I’m writing a letter of recommendation for Rebecca, and she was a great student, and she’s applying to law school period,” it’s going to give you about what you would imagine…it’s going to make up some stuff about Rebecca, it might even not get what program you’re in right. It’s not going to use the last name because I didn’t give it one. And it’s going to give you a very fluffy, perhaps disingenuous response. Now if I give ChatGPT a really robust stream of consciousness almost about Rebecca Mushtare was a student in the spring of whatever and she got this grade and she did phenomenal in these areas. And this assignment she really stood out most because of this, this, and this, it might give me a much more usable response that I can then play with. And so I think that’s going to be the first to instruct students on: what it does not do, which is critical thinking. And from there, I think they’ll have to use it however they feel comfortable. We’re still going to have some writing assignments that are scored. But what I’m hoping for is that these changes will make it so that they’re focused much more on the critical thinking parts of it. And so for some students that might look like writing a fairly complete draft on their own, and then putting it into ChatGPT and telling it to edit this for brevity and clean up grammar mistakes or do something of the sort. For some students, it might be much more of a back and forth kind of a conversation with ChatGPT, which I think a lot of students will be surprised to learn that it functions in that way. And when I find myself using it, it’s mostly as a conversation. Like, I didn’t like what you did here, cut that part out and do this again.

Rebecca: It’s funny that we don’t always think about it as a chat tool, despite the fact that chat is in its name.

Don: Yeah, exactly.

John: Before making this major change in your curriculum, have you experimented with any changes in this course recently to put more focus on critical thinking skills before introducing ChatGPT?

Don: Yes, so in some of the sections, especially at the graduate level, since we have so many different master’s programs, when I first started, it was MBA, and pretty much that’s it. Now with where the business world is going, there’s a lot more demand for specialized skill sets. And so we have, in addition to MBAs, we have a Masters of Science in Finance, a Masters of Science in Sustainable Business, so on and so forth. And each of those sections afford some opportunity to take things in a different direction, really, not even just an opportunity, but we kind of want to, to be more responsive to those fields. And so in the graduate sections, we’ve had some isolated ability to experiment with more problem-based learning, which I think ChatGPT goes really, really well with on the faculty end as far as creating problem-based learning curriculum. But we haven’t experimented with the AI component of it yet, really, because it’s so new, and it doesn’t feel like it right now…it kind of feels like it’s 20 years old, but yet haven’t used it. But it’s very new. And so I don’t know about every other institution, but we don’t move at the pace of jets when it comes to curriculum revision at the University of Miami. We move, I think, faster than probably most but still, it takes time. And so we haven’t had the opportunity to do anything with the AI yet. But we’ve revised in the past couple years to focus more on some of the problem solving in some of the graduate sections.

Rebecca: The faculty member in me heard I can use ChatGPT to help me with problem-based learning classes, and I want to know more about that.

Don: Oh, yeah. [LAUGHTER] So if you type into ChatGPT, you have to give it really, really good direction: the who, what, when, where, why…that you are a professor teaching a negotiation class. And it is a upper-level, undergraduate course. And you are going to create simulation practice for negotiation in which you play one role and the student plays the other, and you will create a scenario and interact with the student, but wait for the student’s response after each of your responses. And then at the conclusion, give the student feedback based on what you know about the science of negotiation from a management sciences perspective, as well as a legal perspective. And then you hit go, you will be blown away with what ChatGPT starts to create. And so it will give you a little blurb. A couple of weeks ago, I did something of the sort, and it said, Sally is the owner of a handmade furniture manufacturing company in North Carolina and has been contacted by so and so that owns a furniture retail store. And so and so has been impressed with Sally’s furniture and wants to arrange a distribution agreement. The meeting begins over the phone and so and so ask Sally what her goals are in this arrangement. And then that’s where I would type in and I said my goals are to reach this level of profitability and to have a productive long-term relationship with the other party, and it responded back. So it can create an entire dialogue that you can then ask afterwards, once you tweak it and say, “Well, I liked this part of it. I didn’t like this part of it, write the Python code for this, and it will write the whole Python code and allow you to turn it into a web-based interactive program. It’s really quite wild.

John: So basically, it gives every faculty member the ability to create interactive simulations for their classes, which could be done for pretty much any topic I would think.

Don: Absolutely. In the past that kind of thing was in some courses, probably a bit aspirational. It’s the kind of thing that would probably require some kind of course leave to develop it. And for faculty who become really comfortable with it, it will get to a point where it’s doable within a day or two of a lesson. And so you can on kind of miniature scale, you can do these on a daily basis, really high quality ones.

Rebecca: It sounds like something that we can use in a lot of contexts in higher ed, including if we want to do simulations for interviews for new positions or other things as well, if you’re trying to better understand how someone might approach a problem.

Don: Absolutely. I think that’s a very good application, in fact.

John: One of the things though, that I think has generated some panic for a lot of faculty is the effect that this may have on how we assess student learning. So how can faculty address issues related to the ethical use of artificial intelligence?

Don: Well, I’ve never known any faculty to ever panic over a technological innovation…sarcasm ended. So I think faculty have to assess this on their own, but also part of the community. One thing that I think’s going to be an early problem are faculty doing things in a different way, I think that’s probably unavoidable. And so I say all that as kind of a disclaimer that my approach and what I think our approach is going to be in my department, and even if the disclaimer applies to that, I don’t even know that for sure, is perhaps going to be different than others’ approach. And so since this course is supposed to be so much of a practical course, and the writing is on the wall, no pun intended, well, it’s in the AI software, I view that we really have no choice. And so there’s been a lot of commentary in The Atlantic magazine, a lot of commentary in the higher education journals. And most of that I have seen focused on this question, but using as an assumption that it’s wrong to use ChatGPT. And so the easiest way to make it not a question of cheating is to allow it to be used, and then it’s not cheating. And so that’s the direction that I’m leaning in. And I think, ultimately, for the practical tools, for the practical courses, that’s going to be the direction it goes. But again, I can’t even speak for my own department on that, because we’re so new in this.

John: And that will be an issue, I think, everywhere, as it has been in the past with things like calculators, or smartphones, or even Apple watches, I remember getting all these memos coming in from various places at one point to make sure your students are not using a smartwatch while they were taking an exam, because somehow the answers are going to miraculously appear on that tiny little screen for the test that you’re giving them.

Don: Right. And I think you can’t really separate the assessment design and the student response to the assessment in this. There are going to be some courses, I can imagine, in different disciplines, that they’re focused on more fundamental foundational skills, that it’s going to be more of a challenge for them. Well, I’m not saying that students don’t need to know and learn about the aesthetics of writing; that has to keep happening, but not in this course. And so I don’t know how the faculty in those spaces and really the 9th and 10th grade composition teachers that I talked about before, I don’t know how they deal with it, probably in person assessments, that sort of thing. But for this practical application course, I would view it really as kind of training track runners to hop on one foot. And so that wouldn’t be very practical. And so if you have a cheating or plagiarism or honor code policy that requires them to only use one foot, then it would be plagiarism for them to run on both feet. But that wouldn’t be very helpful. And so I really viewed ChatGPT as the same thing in a practical sense, if you’re plagiarism or honor, code policy defines ChatGPT as out of bounds, you’re training them to run on one foot.

Rebecca: So we’ve talked a lot about the writing component, and really building in stronger structures to focus on critical thinking. One of the other issues that you identified was that students don’t necessarily see the intrinsic value of the course or like get the buy-in. Can you talk a little bit about the ways that you’re redesigning to help with that piece of it as well?

Don: Yeah, and so certainly a lot of students do get that. But it really depends on how intrinsically motivated they are. And I think it requires faculty to kind of sell it somewhat. And so in my courses, I’ve found success with selling it in that way, which I really don’t like to do. And it’s something that a lot of faculty probably think is kind of an icky thing to do. But for instance, I will repeatedly tell students, I’m not here to make myself feel good. I’m not here to make myself feel smart by putting you all down. I’m here to help you all get jobs and to get promotions at those jobs and do well in your careers. And so I will focus a lot on kind of pointing things out as criticism that I also tell them these are not affecting your grade, however, X, Y and Z. And so little things like in a presentation if they go…you know, we have to have time limits for presentations because it’s basic math, we have X number of students, 75 minutes in a class session, we have to have time. And so when it comes to a student has five minutes and they go over, what do you do? I’m not going to take off on grades for that. But I am going to point out for a student that, in some settings, if you’re given a time limit, that’s because the CEO has another meeting five minutes after you start and you will be cut off, not because they don’t like what you’re saying not because you haven’t followed the directions, but because they’ve got somewhere else they need to go. And so a lot of the problem I think, is just students are so focused on grades, to the shock of everybody, [LAUGHTER] that when the things that you’re grading and are affecting their grades are these kind of…and aesthetic isn’t the right word, but they would view grading something like that as a bit ticky tack. And when you’re scoring things like that, it’s much harder to have a serious conversation about the nitty gritty substance, and how, if you’re trying to prove that this course of action is more profitable than the others, and you didn’t provide any support for the change in costs, you really can’t have accomplished your goal. You don’t get the same attention from the students in the same response, if you’re also talking about things like, well, “you went five and a half minutes when you only had five minutes,” or “You didn’t use 10 point font when you were told to use 10 point font and that ChatGPT, with that second example with that 10 point font, if the instructions said 10 point font and the students input the instructions it produce it in the appropriate formatting.

John: And I know in the past, when I’ve graded student papers, I, as many other people do, spend far too much time correcting grammatical errors, reminding them that there’s a difference between singular and plural or the difference between all the various homonyms out there. Might be easier for us to evaluate student work when we can actually focus on the arguments they’re making, and their ability to engage in critical thinking, rather than getting ourselves so tied up in all this minutia, which I always try to avoid doing, but when I see so many errors in student work, it’s hard not to at least correct some of it so that they could become more proficient. In the future, they may not need to have that type of correction.

Don: Yeah, John, I think you really hit the nail on the head there. You really feel like you have an obligation to correct those. And when communication or writing is one of the titular topics of the course, even more so. But I have always felt that you get diminishing returns on the things that you focus on. And every time you’re talking about grammar, you are not talking about the critical thinking, and the grammar does matter. I can tell you that I have lots of conversations with CEOs with HR directors, etc, in the ongoing effort to make sure that my curriculum is responsive to what’s happening in the market. And one thing that I consistently hear is grammatical errors, spelling errors on slides or in cover letters are catastrophic. And it’s not because they’re nitpicky, it’s because the markets are so competitive, that they get a window that’s maybe 5% or less of what someone’s actual quality as a candidate may be. And that’s just something that there’s going to be some other candidate that is just as qualified and equal in every other way that didn’t make grammatical mistakes in their PowerPoints, and so on and so forth. And so it is important, but it doesn’t matter how good your grammar is, how compelling your vocabulary is, if you are missing some of the logical components of the argument, you cannot be correct.

John: And when students get feedback, where they see dozens of comments on it, the easiest strategy is to focus on correcting those small grammatical errors that are riddled through it. You might also have told the student that they don’t have a very substantive argument. But if they’re going to make a lot of corrections, it’s easier for them to focus on correcting the grammar and ignoring the more fundamental problem.

Don: Right. The very first writing assignment I ever did as a graduate student was a 50-page memo and I got back no comments anywhere except for on the front page: “Do you talk like this?” I think probably somewhere [LAUGHTER] in between those two is ideal. But you’re exactly right, that the more that we focus on things like grammar and tense and such things, the less we can focus on the meat and the critical thinking.

Rebecca: It’s funny how that often is the level of polish would be something that goes from someone that’s got like a really high grade to like an excellent grade.

Don: Right, exactly.

Rebecca: Something that’s foundational, that’s often not how our feedback structures work. And even if we keep form and function feedback separate and even weight them very differently. It’s really easy to address the form issues, because it’s almost like a series of checkboxes and it doesn’t require a lot of thought because the critical thinking part’s the hard part. And so it’s funny that even if they’re weighted differently, and to keep the comments separately, students will always flock towards the thing that’s kind of easy to fix. I mean, who wouldn’t? Then it becomes a checklist.

Don: And that’s exactly really kind of where this boils down to me, it’s not to say that those things aren’t important. They’re still very important. But in the world in which ChatGPT is a real thing, which it now is, and will continue to be and only be more powerful than it is, the juice that we get out of spending time in class or in feedback, in office hours, whatever it may be, talking about those sorts of things, is getting much less of a return than it did before ChatGPT. I am not a walking detector of 100% perfect polish by any means. But it seems to me that the product that ChatGPT can produce, in terms of those things that you were speaking about, Rebecca, is pretty dang good and hard to distinguish for me from highly polished products. But again, where it is easy to distinguish is this is a load of crap [LAUGHTER] that is fluff and has no substance to it, but a very polished load of crap, but nonetheless…

Rebecca: It’s pretty crap. [LAUGHTER]

Don: Exactly. It’s very pretty crappy with a nice bow,

John: …which reminds me of some work that I graded just the other night, where spelling and grammatical issues have mostly disappeared in student essay responses since the advent of ChatGPT, but the substance is not always there. And there were many responses that I provided feedback on which said, “this is a really nice response, but not to the question that you were asked to address.”

Rebecca: Yeah, or you spent two paragraphs and you haven’t actually said anything yet.

John: So teaching students how to use ChatGPT or other AI tools more effectively might allow them to be more productive in their learning as well as beyond their college experience.

Don: And might allow us to make for more productive learning environments as well.

Rebecca: So we’ve talked a lot about course content, and what to maybe focus on and not focus on. One of the most important things a course has is its syllabus or course outline. Can you talk a little bit about course policies and the way that you might make change in that realm?

Don: Yeah, so I think you’re gonna have to be more detailed than you probably are used to being in terms of putting language and syllabi, very specific and upfront. And so some of the policies that I’ve seen that I’ve liked elements of and are going to end up including in the syllabi the explicit weaknesses of ChatGPT. It is not a critical thinking device, it will produce responses only as deep or as shallow as you instruct it to. You are still responsible for the critical thinking, essentially, and very explicit in terms of what’s allowed, what’s not allowed. And I think also, it would probably be a good idea for faculty to be putting in explicit language that what is allowed in this course, is not necessarily the same as what will be allowed in other courses, and it is incumbent on students to navigate those differences themselves.

Rebecca: And part of the reason why things might be different across courses is because the focus of those courses is different and really helping students understand that there’s reasons why policies might be different in other classes. It’s not necessarily arbitrary.

Don: Right, exactly.

Rebecca: So we always wrap up by asking: “What’s next?”

Don: Well, what’s next is I figure out how to do all this stuff, [LAUGHTER] and not just to talk about it.

Rebecca: …and you’re gonna send us a memo, right with that in it. [LAUGHTER]

Don: Oh yeah. Yeah, I’m happy. [LAUGHTER]

John: …or at least have ChatGPT generate a memo explaining….

Don: Exactly. So yeah, what’s next is to put this stuff into action. Of course, as I mentioned, some of the things here have already been experimented with, the non-ChatGPT parts of it at least, but really kind of integrating them and seeing if what I am imagining is what comes to fruition in terms of do these things dovetail as well as I think. I really think that they do. …that kind of pre-existing urge to go more towards the critical thinking element, and really, I think, does dovetail well with the AI, but putting it into practice, it will be over the course of probably all of next year. And so there’s going to be some experimental sections, most of the sections are probably not going to look very different than they did in the spring. And I think that’s probably a very good plan. But there’s going to be some experimenting in some of the sections at the undergraduate level, and part of a faculty learning community on problem-based learning. This course is going to be participating in that in the fall. And so a lot is going to come out of that, I think, as well.

John: Do you think there’ll be much buy in from other people teaching the course?

Don: So, students, by and large, do not like writing. Faculty, by and large, do not like grading writing. And so I don’t think this is one of those political monsters of how are we going to get this through? How are we going to make these changes work? I think there’s probably a lot of people who have nervousness about how you would make these changes. But with those two facts that I don’t think you’d get much disagreement from, I think even across disciplines, I don’t think it should be that difficult for this to be implemented

Rebecca: Well I hope you’ll join us after you’ve implemented some of the things to share some of your reflections and let us know how it went.

Don: I’m happy to.

John: Well, thank you, Don.

Don: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]