Category: grading

122. Differential Grading Policies

Posted in assessment, equity, grading, higher education, learning, pedagogy, STEM
February 26, 2020

Students generally receive lower grades in STEM classes than they receive in other disciplines. In this episode, Dr. Peter Arcidiacono joins us to discuss how these differences in grading policies across departments can help to explain the relatively low proportion of female students majoring in many STEM disciplines. Peter is a Professor of Economics at Duke University.

Show Notes

Transcript

John: Students generally receive lower grades in STEM classes than they receive in other disciplines. In this episode, we investigate how these differences in grading policies across departments can help to explain the relatively low proportion of female students majoring in many STEM disciplines.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Peter Arcidiacono. Peter is a Professor of Economics at Duke University. Welcome, Peter.

Peter: Thank you.

John: Welcome.

Peter: Thanks for having me.

John: Our teas today are…

Peter: Green tea. I got it specifically for this conversation.

Rebecca: Excellent. I have English afternoon.

John: I’m drinking Spring Cherry Green tea.

Rebecca: Oh, that’s a little switch for you.

John: I’m looking forward to the spring now that it’s 10 degrees outside here in Oswego in January when we’re recording this.

We’ve invited you here to talk about your December 2019 working paper co-authored with Thomas Ahn, Amy Hopson, and James Thomas, on Equilibrium Grade Inflation with Implications to Female Interest in STEM Majors. In this paper, you know that disciplines with harsher grading policies result in higher earnings than the earnings received by those majoring in disciplines with easier grading policies. Can you first describe what you mean by harsher grading policies?

Peter: Well, harsher grading policies… you look at average grades in STEM classes and economics, they tend to be a lot lower than in the other social sciences and in the humanities. In the school we look at, which is University of Kentucky, it’s a third of a grade lower. And that holds true across many other colleges I’ve looked at, be it the Duke University or Berea College, it just seems to be a general pattern that the STEM classes and economics tend to have lower grades. And in fact, there was a very nice book written about this with Duke data called Grade Inflation: a Crisis in Higher Education or something like that written by Valen Johnson a few years back, and it was making that exact point about the big grading differences across disciplines.

John: How does your model explain the grading differential across departments?

Peter: Yes, so that to me is where the big contribution of the paper lies, though. I don’t think our model’s particularly satisfactory in this regard. I think lots of people have noticed in the past these differences in grading standards. Where I think our contribution is, is thinking about how those grading standards impact different groups of individuals, but also in what you just asked about: What are the sources of this? And I think a portion of it is certainly demand for courses. So, at the school we look at, the STEM classes are twice as big as the non-STEM classes. An analogy from my own experience… when I first got to Duke, no one signed up for my grad labor class. And that was a bit of a disaster, because that would mean I’d have to do a new prep for some other course. And what I did is I bribed two students to take the class. And the way I did that was I said, “The class is going to be: we’re going to write a paper together.” And it was a great experience for them. They did get a paper, they had a revise and resubmit when they went out in the job market. And what do you know, they both got A’s in the class, but now I’m a little more popular. Not everyone gets A’s and I’m not co-authoring papers with everyone in the class. So I think part of it is these differences in demand for the classes affect things. You hear about weed-out classes, they’re weed-out classes because there’s plenty of students who are interested in taking that class, so they can afford to do that. If you’re in a discipline that has low demand, you’re not going to have a weed-out class because you’re not gonna have any students. So, I think that’s a portion of it. A portion of it can also be things like differences in preferences, maybe it is the case that some disciplines are just meaner than other disciplines. And a third component may be that it’s just easier to grade those classes, it might be easier to grade a math problem than an essay. I’m a little skeptical of the last one, in part because when I talked to high school students, it doesn’t appear to be the case that the math classes are that much easier than the humanities classes, for example, or grade that much harder than. And I think part of that is in high school, you have to take those humanities classes and when you get to college, you have a lot more choice and that influences things.

John: This also seems to relate to the academic labor markets. Because there’s so much demand for classes, there’s often more demand for faculty. So, faculty are not under as much risk of losing their job if they don’t fill their classes. While in markets in those disciplines where grades tend to be lower, labor markets tend to have a little more slack in them. And faculty perhaps may, using the analogy you used before, have to be a little more generous to make sure their classes fill so they can keep their jobs, particularly when there’s a large component of adjunct faculty and contingent faculty.

Peter: I think that’s exactly right. I’ve heard some economics professors brag about their low teaching evaluations. I just thought they can get away with that in other disciplines.

John: And we see that in many STEM fields as well, at least anecdotally… certainly not on our campus…[LAUGHTER] But certainly many STEM disciplines do have that reputation of bragging about keeping their grades low and having high standards and such things.

Peter: That’s right. You know, in my view, it’s when you change what you grade around, that doesn’t necessarily mean that the standards… how much work is required in the class may stay the same. And that’s where, I think what’s interesting about men and women, is that women actually study much more than men. So, if you change the workloads, that’s more of a deterrent for men than it is for women. So, you can think about different mixes of either giving high grades but with high workloads, or lower grades, with lower workloads and student life, on average being different between those things, but they won’t be indifferent between men and women. At least on average. Obviously, this is just an on average statement. There are some women who are not interested in studying hard, and some men who are. On average, that appears to be the case.

John: And you do know, though, that the STEM disciplines not only assigned lower grades, but they also require larger workloads based on the survey data you were looking at, I believe.

Peter: That’s right, substantially higher workloads. And that was my own experience… that where I went to college, I guess economics hadn’t quite crossed over to STEM at the time, but I started as a chemistry major, and there the formulas are a lot more complicated and you had to memorize them. In economics, I didn’t have to memorize the formulas and they were much simpler than those in the chemistry major and I had to work a lot harder in those chemistry classes.

Rebecca: Can you talk a little bit about how the difference in grading policies impacts underrepresented groups in these different disciplines? So in STEM, for example, where you’re saying there’s these stricter grading policies, there’s less women, there’s less underrepresented groups in general.

Peter: That’s right.

Rebecca: How do those things connect? Can you talk us through that a little bit?

Peter: Yeah, actually I had caused quite a stir here at Duke back in 2011, talking about underrepresented minorities, because you do see, for example, African Americans at Duke… They came in just as interested in the STEM classes in economics as white students, but they left at a much higher rate. And so the negative headline was something like, you know, “Potentially racist study says Black students are taking the easy way out,” and that was not the case. It was that they’re coming in with different levels of preparation, due to pre-college inequalities. We have serious differences in pre-college education. And that’s resulted in this coming in and then finding that it’s very hard to succeed in the STEM disciplines. For women, the story is different. And there, because women come in just as prepared as men and are interested in studying a lot more. For them, it’s not the workloads that are affecting this, but more the lower grades themselves, given that they’re working so hard, and then to see these low grades seems to have a bigger effect on women than men. I think one of my very interesting result from some of the work that Stinebrickner and Stinebrickner have done is actually asked men and women, “What grades do you expect you’re going to get?” And then they’ve showed them their grades later and said, “Now, that didn’t work out quite as you expected.” They got lower grades. Why do you think that was, and men were much more likely to say it was bad luck. And you could see that they actually believed it was bad luck, because they continue to take the same classes where they were getting low grades, and they didn’t change their study habits. Whereas women were much more likely to say that it was something about them, either that they needed to study more, or they weren’t as prepared as they had hoped to be for the class. And then you can see that they ended up being more likely to switch to different types of classes and to change their study habits.

John: I think you also found that women tended to not just equally prepare, but I think women came in with somewhat higher GPAs, if I remember correctly.

Peter: They do come in with higher high school GPAs, for sure, and they get better grades in college than the men, both in STEM classes and in non-STEM classes.

John: So the difference seems to be that women are more sensitive to the lower grades or they tend to blame themselves for the lower grades, while the men tend to be less cognizant of the fact that they may be the cause of the lower grades.

Rebecca: I feel like I’ve been socialized to blame myself for not being adequate.

Peter: Yeah, it’s been remarkable, you know I have five boys. And they’re not socialized that way, despite my best efforts to get them to really care about their grades and to think that studying even matters. Here I am, a researcher who knows that studying matters, and yet some of my boys are convinced that if they do poorly, it’s bad luck. It has nothing to do with their studying.

John: One of the things I assigned in my labor class is a reading from, I think it’s a sociologist, who noted that women seem to be more concerned with what she termed “local status,” while men tend to be more concerned with “global status.” Basically, the argument is that men tend to be more concerned with their income expectations and the impact that the courses in the majors have on their grades, while women tend to be much more concerned with the relative grades and seeing grades as a perception of their worth or their value in some sense, which makes them much more sensitive to that grading differential between majors. While men tend not to be as concerned about their local standing, their relative status, while they tend to be more concerned with how the degree will impact their future earnings.

Peter: And I think it’s compounded by the fact that I don’t think people know when they go to college how different the grades are. I think that they figure it out rather quickly, but at first they just don’t know that they’re graded so differently. I mean by the time they figure it out, they’ve already made the switches, which is too bad.

Rebecca: So what do we do so that there’s more representation in these disciplines?

Peter: Well I think at a minimum, that universities should at least have an obligation to tell people, “Here’s what the average grades are in these different classes,” so that students can make informed choices. I personally am in favor of doing something where, like they did at Wellesley, where they put in something that either capped the average GPA in the class, or having all the classes that are above a certain size give the same median grade so that people are not having their decisions distorted by the fact that one department is giving lower grades than another department. It’s not as though they’re learning less in these STEM classes, just because they’re getting these lower grades. They’re actually working very hard in these classes, so, it would be nice if that was somewhat remedied.

Rebecca: I wonder if even knowing the information across institutions and knowing that people get lower grades in these fields, no matter what school they’re at, would actually be useful to those groups.

Peter: Oh, I think it’d be very helpful. Yeah.

Rebecca: Not just within your own institution.

Peter: That’s right. Yeah, I think that’d be great. Because then you know what you’re getting into. And maybe when you get that B in the economics class, you’re not feeling like you failed, that that is a much more typical grade than what you expected.

John: I think a lot of people might object to the notion that there should be the same GPA on the grounds that you’re forcing students into this curve where if you happen to have a class that does really well you might be lowering their grades, which doesn’t always happen in the STEM field, but I think there might be some resistance to that. But the notion of transparency and making people more aware of that early on might help reduce that differential effect by gender.

Peter: That’s right. And I think also making departments aware… the implications that the level of their grading has on things like the gender composition, should matter. I don’t want the way I grade my class to be a deterrent for women taking economics classes. I think that that’s bad. And I’ve changed the way I grade a bit in response to this. If other disciplines are going to give much higher grades, and I want to have a good gender representation in my class, I’m still assigning the same amount of workload. So I’m not making the classes easier, but I am changing the level that I’m curving around, and I typically teach a class that has over 100 students in it.

John: And this is especially important because a large share of the gender differential in earnings is tied to major choice, because the fields that women are more likely to enter tend to offer much lower wages on average, while the STEM fields tend to offer much higher wages. So the differential impact on major choice has quite a bit of an effect on the gender wage gap.

Peter: That’s right. And this is potentially a very low- cost way, at least from a pure financial standpoint, to changing that. It’s not going to solve all of the gender issues in economics or in STEM fields. It’s not going to come close to solving it, but it can put a dent in it, where it’s often very costly to change something like that. Changing a climate is expensive and difficult.

Rebecca: I think I would have been one of those students that would have moved. I’m a person who does a lot of coding and things and could easily have been in computer science. But I’m not, I’m in graphic design, and it’s a field that is much more populated by women. I got much higher grades in the things that I did, and I was very sensitive to grades. So as a student, if it was something that I didn’t do well in, then I thought I couldn’t do well in. And I had a very fixed mindset when I was earlier on in my career. Obviously, it’s much different now. I do much different things and explore other things. But at that time, I think I was incredibly sensitive to that. If you came in as an A student and then it’s like, all of a sudden you’re not getting A’s, it’s like “Woah…”

Peter: Yeah.

Rebecca: “…maybe college isn’t for me, even.”

Peter: Yeah. And that’s just really unfortunate that decisions will get based on nominal differences in these grades, that don’t really have content outside of something that is relative, relative to whatever the professor’s curving around. Another advantage to doing something like this, too, is potentially getting spillovers. Because once you change the gender composition a little bit, then that makes the environment more comfortable. And then that can have multiplying effects.

John: So, why don’t departments want to do this to improve the gender composition of their classes and to get more students in their classes and persisting to graduation in their majors?

Peter: Well, this is sort of where the equilibrium aspect comes in. Because even if you want to do this, everybody ends up ratcheting up their grades, because some departments really need more students. And in order to attract students, you’re going to have to do something if the students don’t have an innate demand for your field. And people have been talking about this for some time, that demand for the humanities has been falling and what to do about it. So, I think that the pressure’s sort of on both sides, even if the STEM departments decided, “Look, we want to have our grades be a bit higher for this reason,” that’s going to lead to response by non-STEM professors as well. So I think it’s hard to get rid of the grading differences across fields, absent doing something that’s pretty draconian.

John: By imposing a grading standard that’s equal across departments.

Peter: Exactly.

Rebecca: I wonder if initiating conversations between the humanities and STEM fields about the interdependence there actually are on those two fields and that people in the humanities should learn things about STEM and people in STEM should learn things from humanities and that there could be more of a relationship there that would actually help start to equalize some of those needs, because there could be some referrals to certain classes or other things that might help boost some of those relationships. I don’t know.

Peter: That’s right. So right now we’re playing sort of a non-cooperative game and move it more towards a cooperative one. Yeah.

Rebecca: Yeah, it’s competitive rather than thinking about “Why are we at a liberal arts institution? And what does an environment like that or general education offer?”

John: Well think about what would happen, though, if grading standards were relaxed in the STEM disciplines. More students would major in them, enrollments would tend to fall in the non-STEM field, and that would tend to lead to fewer jobs for non-STEM faculty. And maybe that’s not something that departments would react well to, and they face the same incentive to just inflate grades.

Rebecca: I don’t think my suggestion was necessarily to do something about the grades directly, but rather to start helping to increase some demand for humanities and helping out some of those…

John: You mean, more interdisciplinary programs?

Rebecca: Yeah, I’ve been thinking a lot about, even for my students, the benefits of them exploring a lot of different disciplines to be more effective in their own, and understanding how other people problem solve, and think through problems, and explore the world.

Peter: I really like that too because if you end up providing these skills that are then rewarded in the marketplace through taking some of the STEM stuff, but combining it with the humanities, that will increase demand. And then that will naturally even things out a bit more.

John: And you could move more from STEM to STEAM.

Rebecca: Hey, I didn’t say it, but I’m totally a STEAM gal. [LAUGHTER]

John: Well I was expecting that at any moment. And just in case some of our listeners don’t know what STEAM is, would you like to…?

Rebecca: Yeah, included the A …it’s for the arts, [LAUGHTER] which I am a part of.

Peter: Yeah, they’re doing that at my kids’ high school, you know, having a STEAM program to try to get that integration happening a bit more.

Rebecca: There’s some research happening about the need for these creative outlets to help with innovation and things in STEM fields. So, there’s certainly some research suggesting this interdisciplinary approach to all of our disciplines could actually improve the work that we do.

Peter: Makes sense.

John: And many STEM students tend to specialize as much as they can in their field. So, that broader training could be useful in the future in making them more creative perhaps.

So, we always end with the question: “What are you doing next?”

Peter: Well, it’s been interesting, nothing quite related to this. I got derailed because I’m actually an expert witness in the Asian American discrimination case against Harvard, and that is taking up all of my time. We released a couple of papers on legacy and athlete preferences at Harvard, and we’re going to be writing some more out of my reports for that case. That’s what’s keeping me busy.

John: Very good.

Rebecca: Well, thank you so much for joining us, this was really interesting, and I hope that it launches more conversations across disciplines.

Peter: Thanks for having me, I really appreciate it.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Savannah Norton.

[MUSIC]

111. The Business of Academic Dishonesty

Posted in academic integrity, cheating, grading, higher education, intellectual property, teaching
December 11, 2019

There are a number of websites that market themselves as study tools and tutoring services that are used by students as tools for cheating. In this episode, Dr. Liz Schmitt joins us to discuss how these sites work and the steps faculty can take to protect their intellectual property and the academic integrity of their courses. Liz is an economics professor and Acting Chair in the Department of Economics at SUNY Oswego.

Show Notes

Rebecca: There are a number of websites that market themselves as study tools and tutoring services that are used by students as tools for cheating. In this episode, we’ll discuss how these sites work and the steps faculty can take to protect their intellectual property and the academic integrity of their courses.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Liz Schmitt. Liz is an economics professor and Acting Department Chair in the Department of Economics at SUNY Oswego. Welcome, Liz.

Liz: Thank you.

Rebecca: Today’s teas are:

Liz: Cocoa and coconut.

Rebecca: Interesting.

Liz: It is. It’s my favorite. And I actually nagged John until he bought me more.

Rebecca: Excellent.

John: And I have ginger peach black tea.

Rebecca: I have Lady Grey today.

John: We’ve invited you here to talk a little bit about the growing problem that we’ve seen with the growth of online services that seemed to be designed primarily to facilitate academic dishonesty. I know you’ve had some issues with that in your courses recently. So, we thought this would be a good time to talk about that.

Liz: No, excellent. My post-traumatic stress hasn’t been maximized by the issue. So, let’s talk about it some more. But seriously, actually, it’s a really important issue. And I think our faculty are just not as well informed of this as they need to be. I think in the coming year, it’s going to become my mission to talk about this a lot more.

Rebecca: Can you talk a little bit about the incident that just recently happened to you to give some context and to help faculty understand how these kinds of things play out?

Liz: Certainly. So, really for years, even with the growth of the internet, if you’re using a publisher product… and it used to be sort of paper test banks and end-of-chapter questions… they show up on the internet. Some faculty just aren’t careful and were just posting them on non-password, open web pages. And then sites begin to grow where publisher content was widely published, by students often, with answers and answer keys to sort of pay it forward to future students who might be reusing it. I think what struck me this fall was the extent at which original content (because of this problem I had given up on publisher content)… but my original content in terms of my questions were being uploaded, essentially in real time. And within a few days, custom answers were being made by some of these sites. And this is really a new step in it, because it used to be the easiest way to get around this was to be writing your own content and guarding that content. And we find unfortunately, in September, that that was not enough.

John: The problem isn’t entirely new, though. For decades, maybe for centuries, fraternities and sororities and so forth, have kept files of old exams. But, the internet allows that to scale much more extensively and creates profit opportunities for people who facilitate those services. So could you tell us a little bit about some of the sites that do this?

Liz: Well, let me say this takes a step from the fraternity/sorority test file with this obvious broadening of the audience online, and no longer in geographic proximity. So, social affiliation and institution is no longer an indicator of what’s available to you. It’s searchable. So, it’s fraternity test file on steroids. It’s really one of the issues. But the other issue is these online proliferation of sites correspond to publishers emphasis on their digital content. As publishers really deal with essentially trying to get out of a used textbook market and kind of adapting for new ages of information, they really push their online content… which means faculty are relying more on that online content to operationalize many best practices such as retrieval practice for their students. So, it’s also more useful for students to have access to this. But, I think what really happened this fall is their ability to customize in short frames of time really eliminates the ability for me to just change up assignments in order to control the problem. So, really for low monthly fees, Chegg.com is what I ran into trouble with and for the low, low price of $15 a month they can upload questions. There’s a limit on how many questions, I think, they get a month but students potentially being strategic about it can upload questions and then get customized answers. Caveat emptor, some of the answers were great, some not so great and downright strange, which is really how I gleaned on to the problem to begin with. [LAUGHTER] And there are other sites. Chegg has some competitors I would say Chegg’s probably the market leader in this, but there are some competitors: Course Hero…

John: Course Hero has been out there for quite a while. My first exposure to this was about four or five years ago, when I had a student post in their online class that they were a member of Course Hero and they encouraged other people to join, mentioning that they could get their membership just by contributing a certain amount of graded work from the course to the site in return for that membership, and asking them to use her code so she could also get some credit for them joining. I had a little chat with that student about academic integrity and reminded her that if she posts anything on that site it would have to be taken down and she was going to be reported for this. This type of thing seems to be increasingly more common.

Liz: …much more common. And then you even have an issue like Quizlet. And one of the issues like Quizlet is students actually use it to create flashcards to study. So, I actually think there’s some honest intent for students that often come on to Quizlet. By making quizlets out of your questions and things you ask though, they actually create something for students that come after them. But, often Quizlet is really set up for students to actually create flashcards for different kinds of courses and study. But that’s really the leftover intent. And then, of course, there are online paper mills sites that actually offer existing papers… and again, narrowing topics can get around that and just not say “Write a paper on any broad topic…” that’s kind of really asking for trouble. But again, the customization that has come along makes that even more problematic as well.

Rebecca: And it’s somewhat affordable. It’s not incredibly expensive to be a member of these sites or to get a paper written for for you.

Liz: Yes, Chegg’s membership is $15 a month. So, that’s incredibly affordable. There have been sites… they come and go… kind of on a dark webish sketchy servers… but there are also sites offering to just take your online course. Those tend to be $1200 dollars, and up… Those sites get to be more pricier. But frankly, if you were a faculty member serving an institution with a more affluent student body, you should be very concerned about that.

John: Many of these sites also provide custom paper writing services. So, even if you’ve specified a narrow range of paper topics, students could still order a custom written paper.

Liz: Exactly. I would be fair to Chegg.com… that Chegg com responds within 24 hours in my experience to DMCA requests. So, they’re actually very responsive. And they take them down immediately. And they send warnings to the account that you posted things that you didn’t have permission to post. Chegg.com also cooperated with an honor code request, basically giving me upload dates and the emails of the members that uploaded content as well. So, they will cooperate, like I said, a pretty rapid fashion. I think it really, in the area of legality, it’s not clear if you correct yourself but become this conduit for things to happen, then, I don’t know what kind of legal responsibility is going to come there. It’s sort of beyond my expertise. But, I suspect it would take big players it takes like publisher lawyers and things like that, to really come up against this and demand that you really set up a situation that facilitates the repeated stealing of our content. Because it can be an intellectual property whack-a-mole game, because they take it down, but then another student post It from entirely different account and Chegg and these sites are kind of saying they’re not responsible for that. Legally, I’m not sure if you facilitate that, how responsible you are. There’s also paraphrasing tools. Paraphrasing tools… that’s very caveat emptor or buyer beware type of situation… and I actually have an example to show you what can happen when plagiarism tools go bad. Friends don’t let friends use plagiarism tools or this is going to happen. So, let me show you the answer from Chegg expert which was a very nice answer… is as follows: “Expectation of rise in inflation will lead to a drop in the prices of Treasury bonds. This is because the required yield by investors will increase in order to generate higher returns after beating the inflationary pressure.” Great answer, run it through a paraphraser and this is what you’re going to get: “Desire for ascend in the expansion will prompt drop in the cost of the Treasury bond. This is on the grounds that the required yield by the financial specialist will increment so as to produce higher returns subsequent to beating the inflationary wait.” [LAUGHTER] So, that’s not a thing… that is not a thing. I think the odd choice of vocabulary… Here’s words that college students almost never used. And here’s 10 of them right now, in the same paragraph. Obviously, that’s the warning sign. And the idea is, of course, context is important. Certain words that are used by convention or tradition in financial markets. Something that’s really a synonym, technically, in the English language is not in the context of financial markets.

John: A traditional way of catching students who were doing this often was that they were providing you with something that seemed a lot stronger than their other work.

Liz: Yes.

John: And this sort of reduces that down to where, perhaps, it may not be as obvious in all cases that someone is doing that because the quality of the work will no longer look exceptional in the same way.

Liz: Yeah, the exceptional quality of work, you’re right… because the first answer actually does set up my radar because it uses a complexity in sentence structure that I might not expect in this answer. But, I think the other one was just immediately obvious because the word choice just doesn’t fit. So, even if you were actually looking around on Investopedia, or some other website to borrow language, it’s not language that would be used by someone writing for that.

Rebecca: Even as a non-expert in the field, I could tell you that that is not language of the field.

Liz: “Desire for ascend in the expansion?” Yeah.

Rebecca: No, no.

Liz: I think pretty much called that one out.

John: But if you have some foreign students in your class, that sort of thing might sometimes happen. I’ve certainly seen examples where students have used synonyms inappropriately. So, some of this may get by.

Rebecca: In some cases, though, if it was a student where English wasn’t the first language, you would probably have some assignments and know already the kinds of mistakes that that student would make.

John: Yes.

Rebecca: So, you would have an idea of whether or not that would be consistent for that student or not.

John: We touched on the issue of hiring other people to do a student’s work, but could you talk a bit more about how the gig economy may play a role in academic integrity issues?

Liz: Well, the gig economy is basically the idea of you give me your specialized topic and I will write a paper custom for you: words, APA or MLA, the formatting etc., ready to go… Course completion, I will take your online course for you… paper writing services, again, is sort of that issue. And again, in ome of these paper writing services, it’s sent as a Word document, and they don’t actually strip the properties of who created that document.

John: And one of the things that has helped facilitate this, though, is the international reach of the internet. In countries such as India, where incomes and wages are a lot lower, you can pay people relatively small amounts to get reasonably high quality papers written in very strong, solid English

Liz: Right, exactly.

Rebecca: So, what can faculty do? This sounds very dismal.

Liz: It does sound dismal and there’s no solution that doesn’t involve some time on task here. And I think that’s a big problem. Because most of my faculty colleagues, I don’t really know anyone who sit around doing nothing… who has extra time to deal with this, and that’s got to give. Scaffolding is a common way to deal with this. So, it’s a recipe for disaster to assign a paper, be very unspecific about the topic: “Do any topic in post-World War II US history out…” That’s going to be bad. You want to focus the topic, then you actually scaffold. When do you want a thesis statement? When do you want an outline? an annotated bibliography? So, trying to have the pieces turned in really prevents their ability from coming out to get a paper or it’s really the red flag when a student several weeks into the process suddenly wants to change topic, having had struggled earlier with the topics and then they tell you when they want to change, you need to get ready for what that paper is going to look like. And you can scaffold in other ways, even on homeworks: you reference. I reference “Using the model from chapter seven,” “using the model we talked about in class in week six” or “using the discussion issues that were brought up in discussion three.” And so you can’t easily post those and get answers because then the students would have to provide the context, which defeats the purpose as well, of the questions. Algorithmic questions, and algorithmic questions can be posted and solved, but algorithmic questions are really after the fact, it makes it a lot easier to snag people. Because, if you just have a general problem…. So, I had a problem on the internal rate of return, which is a common time value of money concept in the field of finance. And while I think students really got their answers from Chegg, unless they were kind of lazy enough to exactly copy the wording of the answers, I couldn’t really get them because there really was a way that you solved this in the spreadsheet. So, the solution should look the same. So, you get around that by algorithmic questions where the numbers are unique, and then you know, exactly who uploaded that question. And in a learning management system, it’s easy enough to regenerate numbers every semester or year that you’re teaching the course. So, there’s a new set of unique numbers. So, that’s more about enforcement, I think. Questions that work from very specific data, maybe a data set, or quotations. So you can start with: “Mitt Romney, in a debate in 2012, called China a currency manipulator. What does that mean?” And so that quotation again, makes it harder to find general answers to that question. Questions that reference their own experiences, where they have to call up a specific experience themselves, and expound on it, and apply it in class. And current events. So, if you do actually have a very current event, then you can actually prevent going back in time and trying to find older questions, because it’s a current event as well. And so you’re constantly changing. So, you might be testing on the same topic, but you’re constantly changing the context. And your question has to be written so that you forced them to address the context as well.

John: I’ll often ask students to find an example of something in the last six months that illustrates some concept that we’ve just discussed in class. Because if I only offer the class once a year, they’re not going to be able to go back and find earlier examples from the class.

Liz: Right. And then I guess my least compromised questions are ones where they actually create a graph using the Federal Reserve Economic Database based on macro data or financial data. And again, that’s right, because it has to be current. And from year to year you change the time period, or what you want them to look at, or different measures of inflation, things like that. And then you can really grab that, and I have yet to see a FRED question appear on Chegg.

John:(…and FRED is an acronym for Federal Reserve Economic Data.).

Rebecca: What is your process for monitoring because clearly, you’re doing some monitoring.

Liz: Right. Well, when I found the extent of this problem in September, I basically went forward several weeks, and just copied and pasted questions into the internet search engines. I also ended up at least getting a subscription to Chegg so I could look, because the other issue is if someone screenshots an image of a question from their learning management system that might not show up in a Google search, but it’s going to show up on Chegg because Chegg actually has the text… kind of an alt text… that comes up, and that will show up as well. And so I started doing that. I opened a Google doc. And then every time I found one, I knew I had to change the questions in my graded activities. And then I would put it in the Google document. And I ended up with a 20-page DMCA document to Chegg about all of the questions I wanted to take down that take me through about week 10 my course. So… to be continued… as to how many I will find. But really, the monitoring really comes up at first when you get just a very unusual answer, or you start getting all the same answers. So, in a class of about 37, there’s about 10 students that were trying to tell me the same thing, especially if they illustrate a concept… I don’t ask for a numerical example… But they give me one. When you see the first person do that, you’re like “That’s great.” And then you think, “Well, why would the second, third and fourth person choose the exact same numbers to illustrate this concept?” There’s only one reason and it’s not a good one.

John: You mentioned the paraphrasing tools making it a little harder to find when people were trying to copy materials. There’s other tools out there too, though, which are online plagiarism detection sites where students can get a paper that they think fits the requirements of the assignment, upload it to that, see if it’s found, and if it is, just make minor changes and resubmit it until it ends up with a relatively low plagiarism score. And complicating this a bit, is that some of the major paper mills advertise that they use a TurnItIn service to check the papers that they sell for plagiarized content. Because one of the issues for the companies that are selling papers to students is that sometimes the people they hire to write papers simply plagiarize existing material. So, this makes it a little bit harder perhaps to detect that sort of work for hire. What might you do to detect papers that have already been run through the TurnItIn or similar systems and modified to come up with low plagiarism scores?

Liz: Well, I think you have to go backwards from that and recognize the limitations of those tools. I was never a big fan of TurnItIn. SafeAssign is just kind of a starting point. SafeAssign often gives a lot of false positives as well. Because if people are citing sometimes similar sources, which you would expect, then you’re going to see that pop up in SafeAssign. So, I think you got to go back and say, are you choosing the topic that is sort of paper mill proof, or things like that? Downloading when you have electronic submissions, which I do, and downloading them can also pick up issues of extra characters designed to trip it up. So, people put characters and they change the color to white to try and trip up that detector and that easily shows up when you highlight them. Those are some of the things that I would use. wWe haven’t had TurnItIn on our campus. We had SafeAssign, which I already knew the limitations, but the tell is you read it, and you’re like I’ve met five college students that write like that… in 23 years. So, I’m now deeply suspicious of that. Often the language doesn’t match, which is why scaffolding also becomes important… because the language doesn’t match what’s been done earlier in terms of: if you don’t think the student wrote the paper and you can’t find the source, you invite them in. And you ask them about where they found the source. Or I really love what you said about the impact of inflation on average households in the 1970s. Could you explain how you developed that? …and they won’t be able to and as a result, they’ll often say, “Well, I wrote this paper and I immediately forgot everything.” And faculty members get caught up into this. “Well, I have to prove a negative or I have to prove that that’s not possible.” And you don’t, I think you’d say: “On your face, that’s a really ridiculous argument. Either you need an MRI right now, because something wrong has happened. You’ve had a stroke and you need help, or you’re not being honest about where that paper came from. And you’re allowed a chance to defend yourself from these charges. But that defense isn’t reasonable. And I’m going to move it on.”

John: Pretty much every time I’ve run across that and brought the students in, they almost always confessed pretty quickly when they realize that they can’t explain what they had written.

Liz: Yeah, I think turn it in has these separate issues too. Is it ethical? We’re complaining about our intellectual property being posted. But, students that do write original papers, that’s their intellectual property. And we’re forcing them by virtue of getting their degree and meeting requirements of a course, to release some of that somewhat. So, I think there are issues of privacy and property, ethical issues that I know a lot of other faculty out there have just really pushed back against for years with turn it in. Others argue it’s about us policing students by requiring this prevention method, we’re almost taking the assumption that something’s going to go wrong, that it can create a hostility and an adversarial relationship that we don’t want in students. I think there’s some truth to that as well, which is why it becomes so important to think about the design of your course and your learning activities, because it’s so much better to prevent this and to make it very difficult than to deal with it afterwards. As I tell my students in my annual first day of class, don’t test me on this because I fail people at least once a year for this kind of thing. I actually say this is the worst part of my job, and I hate it, but I do it. So, that should tell you how important it is to me.

Rebecca: I think that it’s sometimes it’s a little more obvious to faculty how to prevent some of these things from happening in an face-to-face class, because you can be doing in class assignments, you can have them working on things in class and see their progress. But that doesn’t play out maybe as easily online because you don’t necessarily know what they’re doing. So, do you have some strategies of how you scaffold or do things a little bit differently online?

Liz: I would back up again, because some people say, “Well, this is why I don’t teach online, because I’m worried about ringers and things like that.” But, often people that tell me that do assign papers, and I say “Basically, anything that’s done outside of class is susceptible to these sites…anything.” And so really, again, there’s a lot of face-to-face classes, particularly in STEM that are using homework packages that students complete on their own. And then there are plenty of classes that assign papers that are done outside of class. The idea is, is that the in-class possibility of a face-t- face class does provide this check. But in an online class, you really compel them to actually talk about things in a discussion. And so you can use your online discussions to lay the groundwork of what they want to talk about, requiring them to reference discussions within the course as well. And so that’s one way to try and mimic that check that in-class does. And there’s proctoring software, which again, is potentially foolable as well, but there’s also design. So in an online class, the idea is no one thing should be worth a lot. Any one thing should be worth less than 20% in my opinion, because the idea here is if you want a ringer, you don’t want that ringer, by just taking a single exam, to move your letter grade significantly. You’d need a ringer for the whole course or a really, really good friend to do this for you. So, that’s another thing and in designing them exams, you can time them, not necessarily just time them but there’s one question at a time, no backtracking options. So, you want to think about a very structured way to require students to demonstrate understanding that just make it a lot more difficult to outsource.

22:08 redo

John: You can do the same type of scaffolding online as you do in a face-to-face class. You just have different stages, as you said. In the first stage, you could have them submit a thesis statement. In the next stage, you could ask the students to submit a bibliography, followed by an annotated bibliography, and then a rough draft and a final draft of the project . And that’s not really much different than it would be in a face-to-face class.

Liz: And all in the same place are all of these writing samples because that’s how you’re communicating with students: via emails, course messaging, discussion forums, and then other graded work. So, in some sense, there’s a large body of written work to form a basis for your suspicions or concerns.

Rebecca: I’ll also add that we focused on lot on written work, test questions, and things like this. But, the same kind of plagiarism can happen with images, it can happen with code. I’ve had those same experiences in my classes as well…. sharing of digital files to make a particular design and boy does that look kind of similar to something I’ve seen before or an image that’s being claimed as their own or not documented where it came from. So those same things happen. They don’t play out in the same way in Chegg and some of these other sites, but those same practices happen through like a gig economy or just sharing amongst other students and when their digital files are a lot easier to share them when they were physical things.

Liz: Also in creative fields, there’s something worse in the way that’s sort of accepted, because if you look at fashion design, the ripping off of top designer early designs to then the knockoffs is astonishing. There’s a photo that has a Manolo Blahnik sandal, frankly next to an Ivanka Trump sandal, and they’re the identical red sandal. And it’s really just you slapped a different name on it and use different materials, perhaps. And obviously, they’re about six months apart in terms of product cycle. So, I think people in creative fields see that recycling, even in the music field, whether they get permission, you see music, that’s a remix of other older musicians, and you need permission for that, but you don’t necessarily know that when you’re listening to it and enjoying it. So, I think you actually see in some of the real-world situations how visual borrowing and frankly, stealing, is kind of accepted in some of these fields by some very successful people. Whereas, in the written world, when authors get found with plagiarism, it’s considered a big deal. And it’s kind of very embarrassing for them to get caught. Books can be recalled… things like that. It seems like written word… there seems to be more of a consequence, for now, when that happens, whereas in creative fields, that’s not always true.

Rebecca: There’s a lot of visual and audio copyright cases, though. And that’s where that tends to play out. The fashion example that you’re giving… those are considered functional things and not considered creative. So therefore, they’re not protected by the law in the same way.

Liz: Exactly. But, I think for what students observe, and behavior that they might emulate, when they look at professionals and their choices, that distinction isn’t going to come up.

Rebecca: One thing that I always argue is that when we talk about intellectual integrity, that copyright comes into the discussion in particular fields, because it is sometimes a common practice. And it’s there for a reason. we iterate on our culture, our culture creates new culture. It’s not a crazy concept, but you need to know where it comes from. And you need to provide attribution and in an appropriate way, which is no different than following MLA or APA or some other thing. So, I think that’s always something that people should be thinking of, and that when you’re having written papers, and there’s images and things in it, that you’re also thinking about that part of the content as well and not just the written word.

John: One other thing that I’d like to bring up is an earlier podcast that we did with Judith Boettcher, who talked about one way of avoiding this issue by having group projects that provide students with a lot more autonomy, but in a very structured fashion. And that’s perhaps a way of getting around this where students can take more ownership of the project and they create more of the project as a group, which would make it a little harder to engage in academic dishonesty. And we’ve also, in previous podcasts, talked about some open pedagogy projects, where the work that students do is posted publicly. And if they know that when their work is posted publicly, and they’re copying something from anyone else, it’s much more likely that they would be found and they’d get in some trouble later. So, those are two other things, perhaps, that might be ways of reducing the incentives for academic dishonesty.

Rebecca: Community based learning is another one… or service learning where you’re doing projects with the local community, because all of that context would be unique every time you’re doing something. So, that’s another opportunity for grading those assignments that really aren’t reproducible, and would be really, really hard to get an answer for… unless the person lived in the same community.

Liz: I agree. I think the biggest challenge is in some of these courses that are tool courses. In some courses you’re acquiring the tools that you would learn for projects and to consult, etc. And so when you teach these tool courses, it’s not always appropriate to have these kind of finished product things, because they’re in progress of assembling that toolbox that they’re going to use. And this is where reliance on sites like Chegg become a big problem.

Rebecca: At the beginning, we started with the idea that faculty aren’t always aware of these tools, or even the ways that students try to manipulate the system. Can you talk about ways that we can help increase this awareness with our colleagues?

Liz: I was just talking about this this morning with another colleague and we were bemoaning our Chegg purgatory this semester, and she says, “I just don’t think other faculty realizes… or how can we be the only ones that care about this.” So, I think some of them are honestly unaware because faculty aren’t always in sort of the student space and understand what those crazy kids are doing these days. So, I think, in some sense, faculty that have been more tuned in to creating learning community or kind of developing a relationship with their students are more likely to get ideas of what’s going on as well. But I also think there’s some willful ignorance here… this whole “Well, I didn’t know.” But I took steps to make sure I didn’t know because once you do know, it obligates you to do something, if you “don’t know,” and I’m using quotes, which is not helpful on a podcast, air quotes, but if you really don’t know that you’re MyLab component of the course is completely compromised, that doesn’t obligate you to think about changing up your course, and the weighting of activities and what activities. Once you know, it really obligates you to act as a faculty member. And so some people say, “Well, I wonder if my stuff is up there?” and like, “Oh, it is.” I don’t even have to look, I can say “Yes, it is.” And then academic dishonesty takes time. I’m really at 40 hours and counting with academic dishonesty documentation, DMCA documentation, and then reworking items in the course to deal with this issue so it wasn’t a lost semester. And I’m a full professor, think about an assistant professor not only trying to balance and develop their research agenda in conjunction with this, but also not wanting to rock the boat with unhappy students. I’m going to be getting a nice bottle of something sparkly, when I read my faculty evaluations in January. They’re going to be lit. Perhaps we’ll do a dramatic reading and have a tea party with tea and maybe something stronger. But, you know what? I can weather that. I can take that. And that’s going to be an issue. And frankly, I’m really proud of this institution, about how administrators really back faculty, enforcing the integrity of coursework and the degrees. And I know that doesn’t happen at other institutions, frankly other institutions that are maybe more tuition dependent and driven… that are unwilling to make steps that make students leave with financial implications. So, in some sense, this is really one of the best environments here at Oswego to actually try to enforce these policies as well. So, I think that’s one of the reasons that faculty don’t act, because there absolutely is some blowback to that.

John: And faculty might also see that each year they teach the same course, their students are doing better and they might be very content just to see that improvement in the scores so that all the work being submitted looks more like the best work from the year before. And it’s really easy to passively accept that.

Liz: Well, I would actually note that what’s more astonishing, is not the extent that Chegg has corrupted these issues, but there is a significant contingent of students that do not use it. Maybe they are unaware of it. Maybe they decide they don’t want to use it. But, when someone says, “Well, I can’t believe how many students cheat,” I would say, “Well, I can’t believe how many don’t, given the incentive structure.” And so that’s somewhat encouraging. But, also I had to decide if I was going to email the entire class and say, “Look, I’ve seen this happen. And you have to know that this is not allowed, that I’ve already made it clear in the syllabus. This is not allowed, and this is what will happen.” And it was a real-life decision about the Streisand effect, because you have to wonder if 20% of the class is saying “Wait, there’s a Chegg and we can get answers there? Alright!” You didn’t know if you wanted to clue them in, but I figured I had to act under full information.

Rebecca: I think it’s also really interesting when you start seeing students wanting to apply for jobs at those kinds of organizations, and then what kinds of conversations you’re going to have with those students, when they say, “Hey, I found this really awesome job that I want to apply for at Chegg.” It’s like, “Well, think about that. What does that mean? And what are those implications? And where are the ethics behind that?”

Liz: Well John, and I were talking earlier, there are Faculty Fellows at Course Hero, and some of them have made a name about teaching in the profession. John was probably more polite about that. One of them’s at a conference and I think I would stand up and say, “How do you reconcile partnering with a site that facilitates academic dishonesty and intellectual property theft every single day?” I would be curious as to what the answer is.

Rebecca: The pay is really, really good? [LAUGHTER] I don’t know.

Liz: I guess. The one I’m thinking of has a pretty sweet gig right now, so I’m not really sure.

John: this presentation was right after one that was riddled with references to learning styles, so I saved my powder for that one.

Rebecca: So we always wrap up by asking what’s next?

Liz: First, I’m already kind of brought this up in faculty assembly. And I’ve brought it up in academic meetings, because I’m acting chair this year. So I brought it up in leadership meetings. And I hope we can actually do a workshop through our Center for Excellence in Learning and Teaching. So, maybe a breakout workshop talking about our experience, and try and really broaden the understanding of the issues here. I’m also working with our faculty union, the United University Professors is also very interested in compiling violations of intellectual property rights, and trying to deal with that and push back against those sites. And I’m actually sharing my DMCA documents. So I made an editable form for some of the big sites, so you can easily go in and change them. I made a template for what you would ask the Associate Dean to fill out in order to ask for upload information from these sites as well. So trying to minimize the work involved for people who want to do this and take action. And then, finally, I’m just looking ahead about how I’m going to really redesign this course that I’ll teach a year from now, and to motivate and enforce original work.

John: And I should note, we’ve also been offering workshops for at least seven or eight years now.

Liz: You’ve been offering workshops on these sites and things like that….

John: …and attendance has been generally limited. We’re lucky if we get 15 people.

Liz: But you don’t have me. I’m a draw. I’m a draw… a star. [LAUGHTER]

Rebecca: …especially if we could do the dramatic reading of the examples.

Liz: Exactly. There will be a dramatic reading, and there will be sufficient supplies of snark. And I think it’ll work. But, I actually think maybe a case study. So, we come and we talk about these sites, but I can sit down with other faculty who’ve had this problem. And this is what I found… this is how I found out about it. And this is what I did about it, and this is what you should be doing about it.

Rebecca: This has been super informative. Thank you very much, Liz.

John: Thank you, Liz.

Liz: Well, thank you for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Brittany Jones and Kiara Montero.

107. Project NExT

Posted in assessment, evidence-based practices, faculty learning community, grading, higher education, learning, math ed, metacognition, pedagogy, professional development, STEM, teaching
November 13, 2019

Faculty beginning their teaching careers often rely on the teaching methods that were inflicted on them when they were students. These practices are not always consistent with evidence on how we learn. In this episode, for Assistant Professors from the Math Department at SUNY-Oswego join us to discuss how our math department is transforming its instructional practices through the use of professional development opportunities provided by the Mathematical Association of America.

Show Notes

Transcript

John: Faculty beginning their teaching careers often rely on the teaching methods that were inflicted on them when they were students. These practices are not always consistent with evidence on how we learn. In this episode, we examine how one department is transforming its instructional practices through the use of professional development opportunities provided by its national professional organization.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today we are joined by four assistant professors from the Department of Mathematics at SUNY Oswego. Our guests are:

Sarah: Sarah Hanusch.

Rasika: Rasika Churchill.

Jessalyn: Jessalyn Bolkema.

Zoe: And I’m Zoe Misiewicz.

Rebecca: Welcome everyone!

John: Our teas today are:

Rasika: I’m having Earl Grey.

Jessalyn: I just poured myself a cup of lemon ginger.

Sarah: I’m not having any tea today. I’m not much of a tea drinker.

Zoe: I’m not having any tea today either. I just haven’t unpacked to that point yet.

Rebecca: And I have… English afternoon.

John: And I have Bing Cherry Black Tea. So, we invited you here to talk about Project NExT, which is something that people in our math department have been involved with. Could you tell us what Project NExT is?

Sarah: So, Project NExT stands for New Experiences in Teaching. It’s a program that is sponsored by the Math Association of America that brings new mathematics faculty…so you have to be in your first or second year of a full-time job…but they bring these new mathematicians in from all over the country to teach them about active learning.

Rebecca: How did your involvement, or the department’s involvement, with Project NExT get started?

Sarah: I learned about it as a graduate student, and was highly encouraged by a lot of people to apply. And so I kind of brought it into the department by saying, “Dear Department Chair, will you pay for this?” And since then, in part because of my starting it, we’ve encouraged everyone we’ve hired to apply. And as a result, there’s now five members of the department that have either completed or are still in Project NExT.

Jessalyn: Yeah, I will echo that experience. It was something that I was aware of as a graduate student, in part because some of my mentors had gone through Project NExT…it’s now 25 years old…just celebrated 25 years. And so for me, it was something that I knew I was interested in. And in fact, when I visited Oswego for a campus interview, and the department said “Oh, yeah, we have Project NExT fellows on the faculty, and we would be happy to support you in that,” that was a really exciting and encouraging thing about the department.

Rasika: For me, actually, I didn’t heard about that before. But, when I got the job offer, it came with that. I said “Yeah, sure.”

Zoe: I was just hired this past year and so I’m doing Project NExT, but I think I can already see the effects that it has had. It was a program I already knew about, I really wanted to participate in. So, as I was going through the hiring process, one of the first things I would ask the chair at a place was “Would you support an application for Project NExT?” …because it does require a bit of funding. And so seeing that there were already multiple Project NExT fellows in this department was also a good sign for the department as a whole when I was thinking of what sort of department I’d want to be at. And so I think it’s just showing that it’s already been recruiting people who are interested in it already, at this point.

Sarah: I was just going to clarify a little bit about how the funding for it works. There’s actually no fee to participate in Project NExT. The way it’s organized is that you attend special sessions at three of the national conferences in mathematics. So, you attend two math fests in the summer, and then the joint math meetings, which is in January. And so these are big nationwide meetings in mathematics. And so the idea is that you’re going for some special sessions during the meetings. And then your first year, you go for a couple days pre-conference for the really heavy duty workshop. So, the financial commitment from a department is just the funding to go to those three conferences.

Rebecca: You mentioned active learning. Can you talk a little bit more about how those workshops and things are structured?

Zoe: There were a lot of workshops about active learning and just using evidence-based pedagogy, so saying not only active learning is good, but we have evidence to support it and here are some of the things that you could do in terms of active learning. And all the sessions obviously are structured with that in mind. So, we’re not just sitting there listening passively to someone tell us about active learning, but they really make sure you’re doing something, whether it’s a fun little game like building a marshmallow tower, or some other interactive activity in each session. The sessions aren’t only about active learning, there’s a lot about inclusivity and diversifying the profession. So, a lot of sessions on that, or maybe I just chose sessions on that. But, there’s also a whole professional development stream. So, there’s stuff about how to get started in your career in terms of grants and so on. It’s really a lot of everything in there.

Rasika: It’s categorized like if you interest on the tactile learning, so are you interest on the group work, are you interest on some other…you know, inquiry based and mastery grading and so forth. So, depending on your interest, actually, they give more opportunity to listen, go talk with people and have a conversation: what they had, what they tried and what failed and what succeed. Which is like a really nice thing for us, as a beginner, to see what people have gone through and what I should expect, and so forth. Actually, I was interested about the whole program.

Sarah: So, they do some three-hour breakout workshops where you get to go based on what your interests are. So, I did one that was focused on teaching future educators because that’s my background, but I doubt any of these other ladies chose that same session because that’s not their expertise and not what their job is going to be about fundamentally.

Jessalyn: I will add, I attended two workshops that stand out to me in retrospect. One on making active learning intentionally inclusive. That was all about inclusive pedagogies and ways to incorporate group work in the classroom in a way that benefits all students and allows all students to participate fully. I also did a longer breakout workshop that was building a toolkit for student-centered assessment, that was all about learning objectives and exam structures from a more experienced instructor. And then there are also facets of Project NExT that extend well beyond the physically meeting in person. So, as Rasika mentioned, there are lots of ways that you can navigate the workshop according to themes that are of particular interest to you. So, if tactile learning or kinetic activities are of interest, or you’re really focused on educating future teachers or whatever that might be, you might be encouraged to declare a goal for yourself in your first year related to one of those areas of interest. And then we’ve got little email exchanges that go on for people who’ve declared interest in one of those goals like “this email list is all about mastery based grading, check in when you’ve tried something. check in with your questions.” So, there’s a little bit of accountability built into that structure that these people know what you’re trying to do, and they’re going to check in with you on it. But, then just the larger structure of email lists is that you have this cohort of other new instructors who will fire off questions like, “Oh, I’m teaching this class next semester I’ve never taught before, what textbook might I use?” or “I had this really strange interaction in my classroom, and I’m not sure how to handle it” or “I think this part of my syllabus is just crashing and burning. Help! Has anyone been here before?” And so you have this sort of communal resource and the community experience of brainstorming and problem solving together.

Sarah: …and included in that they assign each of us a mentor. So, a more experienced instructor that’s a mentor is assigned to each person in the program currently, and it’s always someone that is outside of your department. In fact, they will not allow anyone to be a mentor who has a fellow in their department. So, as long as we keep having fellows, we won’t have any mentors here. But, what’s nice is when you do send emails out on that list of “I’m trying this and it’s not going well, help!” you do get responses from your peers. You also see responses from all of the mentors for that cohort, which I think is also valuable because sometimes they have a little more experience than your actual cohort.

Rasika: We have a group that people who are interested on the inquiry based or tactile work, they have their own little Zoom conversation whenever they have time together. You get to know all different schools, what they’re doing and, you know, share your experience.

Rebecca: Would you all like to talk a little bit about how Project NExT has influenced your own teaching?

Rasika: For me actually, I was really interest on the tactile experience from this Project NExT. So, I decided to do some activities this semester starting as a beginner and also some group work. And also something that… not exactly what I’m getting from the Project NExT, but it’s like I will say, part of the SUNY Oswego Reading Group, that I was so interested on the book that we are reading. And I decided to give a couple of pages for the students every week to read, and I assigned them 5% for the final grade that they have to read and write half a page to one-page report for me and tell me what they think. Do they think like it’s feasible for them to change and try and do the things in that nature? So far, it’s really going well, and I have good comments from students saying that “you are opening up different ways of thinking…that we were stuck and never complaining about everything. But, we are now having, you know, in a broader way of looking at the things about growth mindset and so forth.” So, I was speaking here and there like chapters from some interesting books. So, that’s what my experience so far this semester, as a beginner.

Sarah: I think for me, it just gave me a lot more lesson plans and ideas to draw from. I already had a pretty active approach to my teaching, but it just opened a broader view of what kinds of things could work well. Especially in some of the more tactile things available that can be helpful for helping students to learn.

Jessalyn: Within my own teaching, I think it’s been really easy or natural to draw on resources from Project NExT in setting up my class or setting up lessons. When I taught Calc I, on day one, we made zip lines out of ribbon and key chains and measured average velocities and it was fun and it was memorable and it got students working in groups and they reported at the end of the semester. “Hey, remember when we did zipline? That was fun!” and I 100% would not have pushed myself to do something that involved or non standard, I’ll say, without thecontext of Project NExT saying “Oh, just try one new thing each semester.” I completely overhauled the Calc II class to be entirely mastery based grading in response to some of my own frustrations with how I had been setting up my class. And Project NExT supplied a whole lot of resources, a whole lot of people, a whole lot of information and motivation to try something like that, which I think was helpful. As far as department culture goes, I think the fact that we’ve had this many Project NExT fellows and continue to have Project NExT fellows gives us a shared language to talk about teaching. Some shared frame of reference on “Oh, yeah, you know, this person who tried this technique,” or “Have you heard anything about…. “”…Oh, hey, this came through on my Project NExT list.” That I think has encouraged just our conversations about teaching and being intentional in how we’re structuring our classes, or how we’re handling things.

Sarah: I’m experimenting with mastery based grading this semester because of the information you and John got, from your experience in Project NExT. And so your experiments with it last year has led me to experiment with it this year. So, it definitely has changed just how we even hear about new things to try.

Jessalyn: That’s delightful. I appreciate that it’s trickling around.

Sarah: It is trickling for sure.

Zoe: So, I’d say it’s still obviously fairly early. We’re only one month into my first semester after going through the first part of Project NExT. But, I’d say a lot of it has been both an affirmation of things that I have been doing and also it’s sort of given me the confidence to do the things that I was doing even more fully and to advocate for these approaches, even though I am brand new in this department. So, I’m not afraid to send to the whole department email list like “we need to be more positive toward our students and not say that it’s all their fault if they’re struggling. we need to take responsibility for that.” Or just to try things that may or may not work well. For example, I’m doing mastery-based grading just of the homework in my general education math course. And I’m using an online system that,it turns out, is not that great for mastery-based grading of that course, even though I’ve used it for other courses. Students, I think, still benefit from it, but it’s not quite as effective as I might have hoped. But, I’m just willing to try these things and willing to speak up about things, so those are the main impacts in my courses.

John: Could you tell us a little bit about how you’ve implemented mastery learning technique?

Sarah: I think we’ve all done it a little different. Why don’t you start, Zoe, since you were just talking about it.

Zoe: I’ve done it only in the homework, so not in their exams. So, the homework is done online, it’s 15% of their grade. And so for each little subtopic, they have to do a little quiz. It’s five questions: three medium, one easy, one hard, and they need to get at least 90% on it. And they can try as many times as they want, but they do have to keep trying. And so, in courses like college algebra…is the one that’s most similar to where I’ve done it before…the material all builds on itself and it divides nicely into little component and there, I’d say it’s going well. The students complain about it at the beginning, but already after I asked them to reflect on their first test performance, a lot of people said, “Oh, it’s actually really helpful that I had to go back and keep learning these things until I fully understood them.” Whereas in the first couple of weeks, there’s always a bit of pushback about “Why do we need to get 90% on this. It’s too hard to get 90%…couldn’t it be lower?” And then once the results come in, they see it’s worthwhile. The other course I’m doing is similar, the gen ed math course…it’s also their online homework…15% of their grade, but that textbook just doesn’t break down the material into as nice sections and the questions are longer and the grading of the online system is pickier. So, that one has some issues, but the same basic idea.

John: Are you using publisher provided questions then, and tools?

Zoe: Yeah, publisher provided questions and tools.

John: Are you allowing unlimited attempts or is a limit on the number of attempts?

Zoe: Yeah, unlimited attempts, and flexible deadlines too. So, I do say they need to achieve a certain amount before each of the test. But, the idea is that if you haven’t yet mastered something, you can still go back and do it several weeks later. As you keep practicing the material, we keep building on it. So, it’s not that you have just one chance and you’re done. The goal is to get them all to understand it fully by the end of the semester.

Jessalyn: My approach to mastery based grading in my first implementation was to go totally off the deep end, and just structure the whole class with a mastery-based grading scheme. So, what this meant was that I did away with midterm exams, everything was broken down into learning objectives roughly correlated to the sections that we were intending to cover in the textbook. And the primary mode of assessment was quizzes. So, my students had quizzes that they could retake as many times as they needed to. And each quiz had three questions and I wrote problem banks of many many questions for each quiz. And in order to earn an A at the end of the semester, the expectation was something like, “Oh, you need 18 of your quizzes to be three out of three and the rest of you two out of three.” So, it was not a points accumulation scheme, it was just quizzes and repeated quizzes. They also had online homework through web work and that was unlimited attempts. There were deadlines, and they just needed to… there was sort of a threshold percentage associated to an A or a B, or a C. And then I had a few more other activities and elements going on. But, primarily, the structure involves these mastery quizzes. And I owe a great deal in the structure of this class to Laura Taalman from James Madison University, who shared a lot about how she structured her class that way and so I sort of borrowed and adapted from her setup for my experiment.

Sarah: So, my class is pretty similar to Jess’s. The main difference is I’m doing it in a proof-based course, so it’s fewer questions. She had three questions per objective. I have one, because they’re a little bit longer questions. The only exam in my class this semester is the final and that’s only because I’m required to have some common questions on a final exam. So, I had to have a final exam…instead I’m doing weekly quizzes. Each week, we add one to two new objectives. There’s about 20 for the entire semester. So, our first week we had two questions on the quiz. The second week, we had four questions on the quiz, but questions one and two were the same objective as one and two from the first quiz. So, the questions are just going to grow cumulatively…so our last quiz will have about 20 questions on it. Although I did tell them once everyone has mastered a question, it’s just going to say mastered, it’s going to be no new question writing and at some point, I’m going to recycle some of the early ones.

John: Your building in some interleaved practice and spaced practice as well.

Sarah: But, the idea is that once they have mastered a question, they no longer have to do it again. They’ll have the questions for practicing and for getting ready for the final. In addition to these mastery quizzes, I’m having them write a portfolio, which is going to have a little bit more of that interleaving practice and making sure that at the end of the semester, they still remember how to write some of these early proofs and it’s also to focus on the writing aspect. So, to help make sure they’re really using the language precisely. Sometimes with a quiz when it’s timed, you’re a little more flexible, but I want to make sure that they have that precision of language down by the end of the semester. So, I’m sort of balancing those two aspects of it that way. They have “unlimited attempts” in air quotes…restricted by what? …there’s 12 times I could quiz during the semester…13 for something…So, restricted to…they need to do number one all semester long. They can have all semester to do it, but we are eventually going to run out of time.

Rasika: So, for me, I haven’t tried to mastery based grading yet. Maybe in the future.

John: Are there any other new techniques any of you have used in your classes?

Sarah: I’ve done a lot of experiments with this idea of embodied cognition, where you actually have students sort of using their bodies to experience things mathematically. One way that we did this with my pre-service elementary school teachers, I give them a bunch of clothesline, and I have them make a circle. So, you may think, “Okay, no big deal.” But, what happens is, it’s not good enough until it’s a perfect circle. Part of this is to elicit the definition of a circle, because to non-mathematicians, I’m going to pick on you for just a moment, Rebecca, how would you define a circle?

Rebecca: One continuous line that’s in a loop.

Sarah: So, a lot of times they come up with something like that. Well, how does that distinguish, though, a circle from an oval. So, it’s not really a precise definition of a circle, right? With the precise definition is being it’s all of the points that are a fixed distance from the center. But, what happens is, by forcing them to make their circle better and better and better and better, they actually all know that’s the definition of the circle. Maybe they don’t remember it, but they know that there’s this radius thing involved. And so by not allowing them to sort of quit until they actually are in a perfect circle, the only way to do that is you have someone stand in the center, and you take another piece of clothesline to measure your radius, and you move everyone in and out as appropriate. So, that activity of physically making the circle and by having to have that person in the center, and that radius gets them to say the definition of the circle properly, first of all, but they get to experience it in a way that they don’t get to otherwise. And that’s an activity that I never would have thought of without going to Tensia Soto’s session at my first Project NExT meetings.

John: It is certainly safer than giving them all compasses with sharp points where they can stab each other, which was how people used to do it.

Sarah: We still do compass and straightedge constructions in geometry, but again, that doesn’t actually help you really understand what the definition is. I think doing this physically actually helps them understand why a compass works. I know that sounds silly, but it really helps make those kinds of connections. I have another activity where we take clothesline and I make a triangle on the ground, and I make them walk the interior angles of the triangle and you spin 180 degrees and it, again, helps them experience that the sum of the internal angles of a triangle is 180 degrees. And again, that’s something that, the first time I did it, it was baffling because first of all, it’s hard to turn the interior angles. Your instinct is to turn the exterior ones, but you end up backwards. From a geometry standpoint, it makes sense, but somehow that physical aspect just really changes things.

John: It makes for a much more memorable experience, where they’re seeing things from a different perspective. And I think that’s really useful.

Sarah: I agree. That’s why I do it.

Rebecca: Does anyone else want to chime in about how having so many fellows from Project NExT has influenced the larger department? Because you’re not just five people in your department, you’re how many?

Sarah: There’s 14 of us tenure/tenure-track now. I do think it’s changing the way some things are done. It’s slow going. I think everyone would concur with that. Jessalyns’s smirk is definitely confirming that. It’s slow going, some of us would like change to happen faster. But, I do think change is happening. I think there’s a lot of respect from our colleagues that we are trying new things. I think a lot of them have a “You can do what you want, but don’t make me change yet.” But, I think we’re starting to get them a little bit, too.

John: If your students are more successful, that often convinces people and sometimes when students say, “I did this in this other class and was really helpful,” that’s often really persuasive to other faculty. But, it’s convenient that you had so many people all come in at once, because that’s not typical in most departments that have such a large cohort, in a short period of time.

Sarah: We have had a lot of retirements, one right back on top of each other. So, we have had an influx of young faculty in our department, which…that alone…to have so many in this program as well. Definitely.

Rebecca: I think it really helps to have models of ways that you can do things because if you didn’t learn using these methods or you didn’t have exposure to that as a student, you have no way of knowing how those really play out unless you have examples. So, it sounds like Project NExT played that role for you, but then you are playing that role for other faculty in your department.

Jessalyn: Thinking about department culture more broadly, not just among discussions and relationship among faculty, but in terms of the student experience, and this engagement that we’re having from our majors and the sort of activities that we’re involving them in. I think there has been a Project NExT influence there as well. Sarah, you and John started the Putnam Competition before I came even and a lot of other conversations and gatherings have come out of that, like we’re getting together with our majors and talking about preparing them for graduate school if that’s something they want to do. The math club or other organizations have taken on a different role in the department and I think a lot of that comes out of some of the ideas in Project NExT, like hearing about how another department celebrates their students participating in something like the Putnam Competition. But, it also comes out of the relationships you build in an active learning classroom and the way that we connect with students when we are trying new things. And we’re being honest with them and saying, “Hey, I’m trying something new. And I’m going to want your feedback.” The community that you build in a classroom flows into the community that we support and foster as a department.

Zoe: So, it’s a bit hard for me to talk about departmental culture change in the one month that I’ve been there not having seen it before I did Project NExT. But, I can certainly talk about how the department seems different from other departments, just in the willingness to embrace new ideas. And there’s also a sense that these ideas are just supported. Even if we haven’t had an explicit conversation, I know that there will be support for trying something new that was suggested in Project NExT. And it seems, when it comes time to make policies, that we have almost a majority just of Project NExT people. Obviously, we need a couple more people, but there are other people who haven’t participated in the program who would still support these sorts of initiatives. Knowing that that base of similar views is there, makes a big difference in what sorts of ideas we would even suggest or consider.

Sarah: I think a lot of our Project NExT fellows have also been very active with doing undergraduate research with students.

Rasika: I think even like talking to colleagues. For me, like I have a personal experience, because my husband is also a mathematician and teach at SUNY Oswego. If I learn something new, I share with him of course, he’s not a Project NExT fellow, but…

Rebecca: So, it sounds like the program’s working really well. You’re all really excited about it. It sounds like it’s engaging all of you. So, glad that you’re able to share it with us.

Sarah: The MAA has definitely done a lot to support improving teaching in mathematics and I do think it is a program that other disciplines could look at and possibly model. I will say they have put a lot of money and a lot of investment into making this a success. It is well run and has been well funded, which is a testament to how important professional organization views it.

John: We always wrap up by asking, what’s next?

Sarah: Well one thing that’s next is we’re trying to get one of our other new faculty, his application was rejected last year. We’re also hiring two people, hopefully this year…So, possibly trying to send them next year as well.

Jessalyn: Another immediate thing that’s next is that our two current NExT fellows will be attending the joint math meetings in January and maybe organizing some Project NExT sessions or at least attending some sessions.

Zoe: I’ll be helping to organize a session on getting started in math education research, which I was made part of because I said it was something I wanted to do, but it’s not something I have any background in. So, I’m finding it a bit of a challenge to assist in this organizational process. But, I also, possibly for Math Fest next summer, helping organize a session on reducing math anxiety, which is something that a previous NExT fellow who I follow on Twitter help organize this session. So, having attended NExT, I think, gave me the confidence to respond on Twitter to this senior mathematician and say, “Oh, yes, I’m interested in this topic.” And so that will come later. And that’s something I actually feel like you could contribute to in a meaningful way, unlike math education.

Rebecca: Well, thanks so much for joining us. This has been really interesting.

Sarah: It’s our pleasure.

Jessalyn: Yeah, thanks for having us.

Zoe: Thank you very much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Brittany Jones and Kiara Montero.

95. Specifications Grading

Posted in evidence-based practices, feedback, grading, higher education, learning, metacognition, pedagogy, self-regulated learning, specifications grading, teaching
August 21, 2019

Faculty often find that grading student work is a stressful and time-consuming activity. Students sometimes see grades as a subject of negotiation rather than as an assessment of their learning. In this episode, Dr. Linda Nilson joins us to explore how specifications grading can save faculty time while motivating students to achieve the course learning outcomes.

Linda is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of Specifications Grading: Restoring rigor, motivating students, and saving faculty time as well as many other superb books, book chapters, and articles on teaching and learning.

Show Notes

  • Linda Nilson – Director Emeritus of the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University
  • Nilson, L. (2015). Specifications grading: Restoring rigor, motivating students, and saving faculty time. Stylus Publishing, LLC.
  • Nilson, L. “80. Self-Regulated Learning.” Tea for Teaching podcast. May 8, 2019.

Transcript

John: Faculty often find that grading student work is a stressful and time-consuming activity. Students sometimes see grades as a subject of negotiation rather than as an assessment of their learning. In this episode, we explore how the use of specifications grading can save faculty time while motivating students to achieve the course learning outcomes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Linda Nilson. She is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of many superb books, book chapters, and articles on teaching and learning.

Rebecca: Welcome, Linda.

John: Welcome.

Linda: Thank you very much. It’s great to be here.

Rebecca: I think it’s more of a welcome back, right?

John: Yes. Welcome back.

Linda: Welcome back. Yes, yes, yes. Good to be here again.

Rebecca: Are you drinking any tea today?

Linda: As a matter of fact, I am. Yes, yes, yes. Of course I had coffee this morning as well. But I am drinking tea. I am drinking a berry tea, but it also has black tea in it. So it’s still a bit of a stimulant anyway, but it tastes really good.

Rebecca: Sounds good.

John: I have ginger peach green tea.

Rebecca: And I have English breakfast.

John: In an earlier podcast we talked with you about your work with self-regulated learning and one of the topics that came up with that was specifications grading.

Linda: Yes.

John: So we’d like you to tell us a little bit about your book, Specifications Grading: Restoring rigor, motivating students, and saving faculty time… especially that last part. [LAUGHTER]

Linda: Yes, saving faculty time. Yeah, I figured that would help sell the book. But it’s true. It does save faculty time. And one of the things that inspired this book was just my hearing so many complaints from so many faculty over the years, about grading and the aftermath of grading and returning grading material to students. And the constant steady stream of students trying to get another half a point, just arguing… just conflict… constant conflict… students being stressed… faculty being stressed… faculty getting larger and larger classes with less and less help… fewer and fewer TAs, if there ever any TAs. And I got tired of it. But there was a part of this that I did not invent. I heard it from a faculty member in the School of Management. And she was doing that pass-fail grading. It was saving her tons of time. She had huge classes, online classes. And she just invented this and she was also sick and tired… getting complaints from students and students not paying attention to her feedback, which of course, took her hours and hours and hours to write and return. So I took some ideas from her. But I also wanted to tie grades somehow to outcomes. And this is where another aspect of specs grading comes in. And that is with respect to bundling assignments, or turning them into modules or whatever. But I prefer the term bundles because it’s much more universal. So anyway, this was a solution to a problem. And that’s what a lot of my work has had to do with, making the faculty members job easier and more rewarding.

Rebecca: I think that you’ve been spying on me for the last many years. [LAUGHTER]

Linda: Yes I have. Yes I have. [LAUGHTER] I’ll admit it.

Rebecca: I knew it. I knew it. So I think John and I are both really interested in the idea of saving time, as are many faculty. But you also talked about, in your book, the history of grading… how its evolved… the 4.0 system. How is this different? And how does this relate to the history of grading?

Linda: Well, let’s look at the history of grading first. Grade started, well everywhere, in 1783. It was Yale’s idea. And what they started doing in that year was an achievement-based student classification system. They were not using As and Bs, what they were using were Latin designations of like optimi outstanding and pejores for failing, like as in pejorative, right? Anyway, then in 1800, Yale dropped the Latin designations and started using numbers 0 to 4. Sound familiar? But that was Yale doing that now. In 1850, the University of Michigan initiated grades, but for them, it was strictly pass/fail, and it only took 50% to pass. So we talk about grade inflation now… look backwards. [LAUGHTER] Mount Holyoke, though, just a few years later set passing at 75%. Harvard, also a little bit later, invented the A to F system, but passing was only 26%. So if you were wealthy enough, you had to know less, right? Okay. But anyway, that’s where ultimately grades came from before 1783. This was in Europe. And this started hundreds of years before our notion of grades. There was something… it was like a Jeopardy game, where graduates… or graduates to be… the hopeful graduates… were answering questions in a tournament style, but the stakes were really high. And so yes, if you are winning throughout, you were really showered with honors. But if you were at the bottom, you lived years in shame. It was terrible. We talk about high stakes. Oh my. So anyway, grades were invented after universities. Socrates didn’t talk about grades, right? It’s a relatively new invention for sorting students.

Rebecca: How does the specification grading relate to this letter system or this 4.0 system?

Linda: Well, it kind of takes a break away. Because first of all, all this grades that I was talking about was with respect to courses, there have been pass/fail courses. Sometimes they worked well, sometimes they didn’t work so well. They work pretty well in medical school ‘cause we’re dealing with a highly motivated students who really understand the need to know. But other than that, most students would do the absolute minimum to get their C-… whatever, it didn’t work very well in terms of like motivating students to learn, they learned the absolute minimum. With specs grading, the pass/fail is within the course, the assignments and tests in the course. But, you don’t pass with a C or C-. You set the passing level at what you would regard as a B level. And this is what restores the rigor. I think we have been sold a poor bill of goods, when we don’t set our students to high standards. We say, “Well, maybe they can’t do it, and then they’ll get…” They can do it. Come on. They’re just not doing it because of our partial credit system and our point system ‘cause they can always get another half a point, right? …just by wearing you down. [LAUGHTER]

Linda: Really, they do and they know it works, it worked in high school, so it works now. Now it may have worked the previous semester, these students aren’t stupid. And they can do what we ask them to do for “B” work… certainly at the undergraduate level, or actually at the graduate level as well. They could do it. But they choose not to, because they can survive otherwise. They could do well enough to pass or get their “B” or even get their “A.” So why should they sweat it, they can always get the partial credit. And students get partial credit. It’s almost like going back to the University of Michigan 50% passing for assignments and tests. I mean, we pretty much lowered the bar, because we want so many people to get over it. Well, people are getting over it without preparing… without doing a decent job. And they’ll spend 10 minutes… 20 minutes… the night before on an assignment knowing that no matter what they do, they will get partial credit and they will pass. So again, they’re not stupid, it’s just that, for them, college is a game that doesn’t have much to do with learning. That’s not where the focus is, and at least with specs grading, there is a lot more focus on learning, because you are tying the assignments and tests to student learning outcomes. And that’s a really nice part of the system in that those grades, the ABC, mean something in terms of outcomes achievement, and these, in turn, might be tied to program outcomes as well. So all of a sudden, you’ve gotten rid of an entire step at the departmental level of having to measure program outcomes. And why do you have to do this? Because accrediting agencies know that our grades don’t mean much of anything, as far as learning is concerned, as far as what they’re interested in, which is outcomes achievement. And outcomes achievement, it’s an up-or-out sort of thing. You can either do something or you can’t, and as long as there are standards set for that, and certainly the accrediting agencies want standards set for that. So let’s say “Okay, so your goal is a student can write a good quality, maybe not great quality, but a good quality business proposal.” Okay, fine. So if that is your outcome, what really does that entail? What exactly are you looking for in terms of a very good business plan? And that should be incorporated in your directions to students. But we don’t articulate that, do we? We really don’t. We speak very vaguely, “Well a business plan should have this and here’s your rubric.” Well, first of all, students often don’t understand the language, but is simply not detailed enough such that all students can understand the directions and actually follow them. We need to put more detail. Usually what we assigned undergraduates is some sort of a template or a formula to do this. And we’re not talking about that. We’re not sharing this template or formula with our students. And that’s what we need to do… not to say that you can’t allow for creative work. Matter of fact, with this system, you can and you don’t have to worry about tearing your hair out in different ways. “Well, I don’t know how to grade a movie.” Well, no, but you can talk about certain qualities of that movie, as in just simple things like the length, or perhaps the number of scenes that you want to see. And certainly they’ll be a learning goal and a communication goal connected to it. And that’s pretty much all you have to do. For most of our assignments, they are formulaic, but students don’t know the formula. And we need to tell them, and those are what you would call our specs for an assignment. So that’s really what we need to do. So all our work is up front and laying out what those specs are. It’s like a one-level rubric. But we don’t have to worry about different levels of the rubric, where there are four-level rubrics and five-level rubrics. We don’t have to worry about that. All we need is one. So we can afford the time to actually specify what we’re looking for… what that template involves.

John: And then the focus is helping students reach the standard, rather than negotiating with them over the grades for what they’ve received.

Linda: Exactly, exactly. Yeah, forget negotiation. There’s some things that ought to be non-negotiable. I mean, we’re not sloppy when we grade, I know, we get sick and tired of it. But we are so diligent in our grading. And then we have a line of students outside our office saying, “I deserve another two points on this because Susie said the same thing. She got 12 points, I only got 10.” …like write an essay about it… justifying it.

John: One thing I really like about it, though, is that right now we have these two levels of assessing students’ learning, we have these complex assessment plans for each department. And then we have a grading system, which often bears little resemblance to the assessment. And this is making the assessment transparent and obvious to students. And it’s forcing both faculty and students to focus on the learning objectives for the course.

Linda: Yes, as we should be, you think about it, what does it mean, when a student gets an A? Does that mean that that student has achieved all the objectives? all the outcomes at the level we want to see? Well, maybe for some of the students who get an A. What does a “B” mean? Now we’re starting to get into really ambiguous territory. Does that mean that the students sort of achieved or barely achieved all the outcomes or maybe achieved some really well and then didn’t achieve others? And a “C,” forget it. You can’t tell what’s going on there, especially given the way we give out “Cs” these days. So maybe the “C” student achieved one outcome well, but which one? Yeah, I mean, no wonder accrediting agencies pay no attention to our grades. They don’t even want to know them. And I can understand why, if they’re focused on learning outcomes. And so yeah, we’ve got this whole extra level of work that we have to do and the department chairs have to do and the Provosts have to do. I mean, who needs it?

Rebecca: Can you talk a little bit about the relationship between specs grading and contract grading, because it seems like they have some things in common?

Linda: Yes, they do have something in common. With contract grading, students individually work it out with a faculty member as to exactly what they’re going to do. And if they do all those things that they will get there “A” because I’ve never heard of a student contracting or a ”B,” it just doesn’t happen. Now, contract rating goes back to the 60s, and the first half of the 70s. And what was happening was since there was this individual relationship, first of all, faculty didn’t specify enough about what they wanted… what constituted “A” work… same problem we have had up to now… and faculty weren’t all that much different then, in fact, they probably gave less guidance. But in any case, there were the specifications. And then faculty and students would develop this relationship, this individual relationship. And it became particularly difficult for the faculty member to give anything but an “A” to the student. And the student would have thought, “Well, why didn’t you tell me to do this to get my ‘A,’ because that’s what I contracted for.” So contract rating was: 1. sloppy, but 2. highly individualized. With our class sizes, we can’t even talk about contract grading. And because it was something that was mutually established, between the faculty and the students, and specs grading is nothing like that. This is all in the faculty members’ hands. Well, at least, setting out what is required to get the “As” and “Bs” and “Cs” and “Ds” . But students choose what grade they’re going to go for. And according to what grade they believe that they need, but also according to what kind of a workload they want to shoulder for this particular course. And you know, maybe all they need is a “C.” And you know what, that’s okay. And we don’t look down on that student if the student says: “Well, all I need is a ‘C.’ We do otherwise. We say “Wow, this is a lazy student.” That’s what we’re thinking in the back of our heads. We have negative thoughts about students who settle for low grades. But with specs grading, you don’t have to feel that way. This student chose a seat for whatever reason, I don’t care. My course is not in that student’s major. So I’m not going to take it personally. So, that’s nice. But the fact that that’s making that choice, gets rid of all kinds of grading complaints, or that thing at the end of the semester: “What can I do to bring up my grade?” Well, it’s very clear what you could have been doing. If you went for a “C” now you could have done these additional assignments, or taking this additional test to get a “B.” But you didn’t do that. Now, you’ve got a week left, maybe you want to try to do that. But don’t ask me, I laid out the contract, I laid out the terms, I laid out the specs at the beginning of the semester, and you make your choices. This whole thing makes students feel a lot more responsible for their grades. They made the choice. And we respect that.

Rebecca: So I think a question that would come up for many faculty is you’ve laid out that certain assignments need to be accomplished to get even a “C…”

Linda: Yes.

Rebecca: …and what the specifications are. What happens when a student struggles to meet those specs?

Linda: Oh gee. Well, now when you say struggle, you can struggle before the assignment, or you can be disappointed and angry after the assignment. So if you’re talking about struggling before and say, “Professor, I don’t understand these two specs. I don’t even know if I can do it. Could you give me some more guidance?” No problem, and then because the stakes are higher, you get credit or no credit whatsoever… doesn’t count towards your bundle… doesn’t count towards whatever grade you’re doing. So you’d better figure this out. Now, afterwards, don’t meet the specs. Okay. And obviously, you didn’t come to see me in advance, did you or otherwise, we would have hashed this out. Anyway… so you didn’t pass? So you missed these two specs. All right, well gee, that’s most unfortunate. But there’s this system called tokens… or I call them tokens, but you can call them… I don’t know, you can call them pigs in a blanket for all I care, it doesn’t matter. They’re like get out of jail free cards. I’ve seen them called hail Mary cards. In a geography course, they were called globes, but anyway, they are opportunities to either redo an assignment that didn’t meet the specs, or to get a 24-hour extension. And there might be some other things that you make up along the way if absence counts against you in your course, you can get out of absence. But let’s say it’s for redoing this assignment, which is really important early in the semester, because students are not going to believe that you’re going to grade them pass/fail, because this is all new for them. And they go, “Well, how can this be…” and “There must be some sort of partial credit or something…” They won’t believe you. So there will be a number of students who will need this token or get out of jail free card to redo the assignment. It’s up to you, as a faculty member, as to how many get out of jail free cards or tokens that they might have. Three is a nice number. I like it, but some people give five. Some people give two. But I think it behooves you to have some reasonable number of tokens with specs grading, because that takes the pressure off of students; it allows them to screw up at least a few times. So this is something that students can get second chances out of this because the stakes are higher than typical. But the thing is, when you’re grading these assignments, all you’ve got to do is check the specs that weren’t met, and say, “This is why your assignment failed.” Now, if you want to give additional feedback, by all means, I’m not going to stop you. But I want you to have a life. That’s the only thing. And you can give positive feedback as well. And then you know what, you’ve passed something, then you get positive feedback from your instructor that you figure, “Wow, the instructor gave me this positive feedback out of the kindness of her heart, because she really cares about me and my success… told me how I could maybe do better in the future.” First of all, I’m going to read this, because this is meaningful feedback. This is not justification for having taken off points, because that’s what most of our feedback is about: why you didn’t get full credit. And now this is actual substantive learning feedback. So, it decouples feedback from evaluation in a way, but students get the chance to do it over and now that you have their attention, that let’s say they didn’t pass a certain assignment, that they can come to you and say, “Okay, how should I do this better?” Oh, okay. Isn’t this lovely? So the worst that will happen is you’ll have a conversation with a student about how to improve his or her work, that’s the worst that will happen. Isn’t that wonderful? [LAUGHTER] And sometimes students know, they just got lazy.

Rebecca: How would you suggest structuring specification grading for a class that’s more project based or even collaborative work?

Linda: Oh, sure. Whether it’s projects, or papers or whatever, again, it’s a matter of specs. Oftentimes, with our projects, that we might allow our projects different media, what we have to do is lay out specs for each media. Or say, if you can think of a different media, you can do a film, you can do a play, you can write pamphlets, you can write a paper if you want to, or you can do any of, let’s say, a half a dozen things. And let’s say if you want to do something else that’s not on the list, come talk to me, and we’ll work it out, we’ll work out some specs for it, too. So what you have to do is you have to lay out specs, but oftentimes the specs that you have to lay out, and students love creative assignments, is you have to lay out, let’s say, length, and so you might say, “Okay, if you’re going to go with a film or video, I want 20-mintues of that. Going to go with pamphlets? I want at least four pamphlets, I want each pamphlet to have at least 250-words in it. If you’re going to go with writing a play and performing a play for your peers, I mean, wow. But I do want it to be a half an hour long. And I want it to involve everybody in the group, everybody’s acting. And if you’re going to write a paper, I want it to be at least, let’s say, 2000-words long… or maybe that’s too long, whatever… and you lay it out, and I want it to follow this kind of an outline.” You can lay out specs for different kinds of media, and let students run with the project. Now a design project, what you might allow students to choose is exactly, let’s say, in architecture, what kind of a building they’re going to design. Let’s say, “Okay, you can design a residence, single family, a separate dwelling, but I want it to be at least four bedrooms, I want to be two stories, this and that. And you could do an office building too. But I want this to be at least two stories, and I want it to be so many thousands of square feet.” And then you let them run with it. And you know what, we know from our history, and from a lot of publications, that students run with this sort of thing. So I’ll give a little guidance and off they go, it’s a beautiful thing. Now in terms of projects having to do with group work, I don’t know that that specs grading will make group work smoother. [LAUGHTER] And I’ve talking about out-of-class group work. If you want to play marriage counselor, you just go right on ahead. But you don’t have to. I mean, there are some people who say, “Hey, you guys work it out on your own, I don’t want to hear about it and you can fire freeloaders. And if you are doing all the work, you can resign from your group and any other group will be happy to pick you up. But all this has to be done by the end of the fifth week,” whatever. So I don’t know that we’re going to get rid of those problems. [LAUGHTER]

REBBECCA: I think my question was more about the high-stakes nature or the long-term nature of projects.

Linda: Yes.

Rebecca: So would you recommend doing scaffolded specs or something like that to help make sure they’re on track?

Linda: Well, yes, what you need to do, as with any major project, you need to divide it up into smaller tasks that students can be at accountable for along the way, and maybe even get feedback for. Now, you can give feedback… your peers… people in other groups… can give feedback. Because after all, we’re doing criteria referenced grading, we’re not grading on the curve. So there are different ways for students to get feedback. But in any case, this divides a massive task into little pieces. So maybe starting out with, “Okay, by the end of the third week, we want your group to have a literature review put together.” And we put specs for the literature review. “And we want so many references. And we want so many of them in the last five-years,” And what have you. So we can even put the specs, I even specify specs for the format that we want: APA format, or ASA format, or Chicago, whatever it is that we want. That’s something that we can put in the specs if we want to, and that way students will actually proofread. Now I think we ought to be somewhat tolerant with respect to format anyway, we ought to allow three little errors in format. Because you know what, when we submit a paper to a journal, they’re tolerant of us, right? [LAUGHTER] I mean, you can’t be sloppy, but if we put a period instead of a comma, hey, we are forgiven. And we’ll catch it later on the proofs. So we need to allow some errors on something like that. But anyway, yeah, we divide up a big task into pieces. And then students get feedback on the pieces. And that’s also a good way to find out early about who are your freeloaders, who’s not doing the work early, so they can get rid of the freeloaders if they need to. Here’s another thing that you can do with specs grading, when you do bundles, you know, the project, the big project, you can make that big project required for an A, period. And then you will only have groups with students who are going for As. So it depends on the size of your class, of course. You might wind up with only, let’s say, three groups who are going for As. Or you know, the number of students that will fall into three groups. Well, guess what, you’re going to get some really fine projects, you’re not going to have to worry about it, and they might not be freeloaders. But anyway, in the examples in my book that a lot of faculty members and I recommended as well, that group projects be reserved for getting an A, for the A bundle. And other assignments be in the B bundle and C and the D bundle for that matter, which might be very minimal. But hey, if you want to a D, far be it for me to stop you. So, anyway, that’s another way to solve the problem, you just put those projects in just for the students are going for an A.

John: You’ve mentioned bundling a few times in terms of grades, could you talk a little bit about that and how you can go from the specs grading to the course grades that are assigned in the course.

Linda: Okay, there are two ways and the first way I’m going to just mention and not recommend. But you can keep your point system, and you might have some haggling over grades. But in any case, you can keep the point system and say, “Okay, my course is 100 points and if you get 91 points or more, you’ll get an A.” And okay, fine, but the bundling system allows students to choose their grade. So what you do is, you set up clusters of assignments and tests that students have to pass at that B level to get credit for their bundle. So you set those up. And the easiest way to do this, so you got a D bundle, is pretty minimal. And you can tell students, “Look for your D, here are the only outcomes that you will be able to achieve if you go for the D.” Some students say, “Hey, I don’t care, not in my major.” Fine. So it’s minimal work, maybe it involved just passing tests at let’s say, a 70% level, and maybe doing a couple of little written assignments along the way. But, they all have to meet whatever the specs might be. Whatever specs you set out for those assignments… And C bundle, “Okay, you got to do everything that you do in that D bundle, plus there are extra assignments as well.” And so maybe what you are having students do, and I’m just making this up as an example, is the students have to turn in 15 out of the 18 reading assignments that you have, they have to turn in a typed up outline on a particular reading. This way, at least your C students are going to do the readings, right? Because at least most of the readings, almost all the readings, and you’d give your teeth for almost all your students to do almost all the readings, right? But that’s all they’ve got to do. Maybe for a B you have to do all the requirements for a C, maybe even more of those reading notes, or something different. Perhaps keep learning journal, keep a journal on how you are learning, what you’re having trouble learning, the different strategies you are trying to learn the material, and it’s going to be collected. But you really have to specify what questions you want students to answer. So that will be collected maybe four times during the semester, and you might collect some every week or so and look at a few of them. But again, you’re just looking for the answers to the questions. That’s what you have to do for a B. You’re going to get a good handle on those readings and you’re going to find out how to best learn this material. For the A, you got to do everything for the B, but you have to do some sort of a group project where you’re going to learn, let’s say application, analysis, evaluation, what have you, whatever higher thinking levels that you have in there. At the C level, that’s pretty low. But good lord, at least they’re understanding the readings, at least they’re reading the readings and presumably understanding it. And you know, you can spot check some of them to see that they are understanding and if they’re not, you can help them with that. And as soon as they’re learning how to learn for that B, but students are learning how to analyze the material, even higher-level learning outcomes for that A. Now if your student learning outcomes are dictated by a professional accrediting agency, then you’ve got to put all of those outcomes in what’s required for a C, or maybe your institution considers D passing, some institutions don’t. Whatever you consider passing, that’s where all of those outcomes have to go. B students and the A students, they will achieve even more outcomes. If you’ve got an outside accrediting agency telling you what your students have to be able to do, then guess what? All the students that pass the course have to be able to do these things. And you’ve got to set out the specs accordingly. There are some accrediting agencies that layout good outcomes with active verbs. And other accrediting agencies, they give you crummy outcomes where you’ve got to essentially rewrite them so you can assess them. But I’m not going to get into that. [LAUGHTER] Did I answer your question, I hope?

Rebecca: Yeah, I think a lot of your examples focus on a more traditional class and not necessarily a lab or a studio kind of class where there’s a lot of class work that happens inside of class. What might a bundle look like in a situation like that? Where a lot of the learning activities are happening in person rather than for homework. So, for example, I teach studio classes that meet for six hours a week, and they presumably do about three hours outside of class. Versus the reverse, which many people have and labs are similar.

Linda: What area are you in?

Rebecca: I teach graphic design.

Linda: Okay.

Rebecca: But it would be similar to something that would happen in the sciences too and having to do lab.

Linda: Yeah, to an extent, it depends on whether it’s a decent lab or not. [LAUGHTER] Some labs are a waste of everybody’s time. But anyway, what you have to do there is, you’ve got projects, right? …throughout the semester, different things that students are doing. but how many projects do you have in a semester?

Rebecca: Usually, it’s like four big projects, or three.

Linda: Okay, well, now this sounds very, very radical. But what if for a D, you only had to do one project? …and I mean, at that B-level that you set to pass. What if for a C, as a student, I only have to do two. Now, for the first two, because I presume this is somewhat cumulative. For a B, I only have to do three projects. But an A, of course, I have to do all four, if there are four projects. Now, what does this mean? Oh, by the way, do you have tests?

Rebecca: No.

Linda: Just curious. Okay, it means that assuming your D student does passable work, that first project, you can say bye bye after that first project is done. If the person wants to sit in… you can’t, you know… it’s a free country, right? Well, not really. But they can sit in if you want them to, that’s fine. For a C student again, you can say bye bye after the second project, assuming they do it at that passable level. And by the end of the semester, you have only you the most motivated and committed students that you have to be concerned with. And you can give them some very challenging work. Now, I don’t know how you feel about that. But that’s the way you could do that. Now, you could also bundle it differently. You can expand the number of projects that you have as well and simply make them shorter if that’s what you would be required. But I don’t know how you react to that.

Rebecca:I think my accrediting agency would react to that in that they have to put a certain number of hours in. [LAUGHTER]

Linda: Okay, okay, so your accrediting agency is time focused.

Rebecca: It’s part of it. Because presumably with more practice, you get better.

Linda: Well, presumably. [LAIUGHTER] …practice with feedback, you get better. But yeah, if we are committed to this hours business, that makes things very unruly, everybody’s got to put in the same amount of time…

Rebecca: In class anyway.

Linda: Oh, in class, okay. That’s very unruly, with respects to specs grading because specs grading isn’t about time. I mean, you can recommend that such and such assignment should take you at least three hours for it to be passable. But other than that, that’s your recommendation for a certain given assignment. But yeah, that makes it very unruly when your accrediting agency says, “Okay, they’ve got to spend so many hours in class and not just looking at their phone.”

Rebecca: Yeah, I mean, we do critiques and things like that. We provide a lot of feedback. But yeah.

Linda: Yeah. Okay. You do critiques?

Rebecca: Yeah lots of critiques.

Linda: Yeah, like art classes and things like that?

Rebecca: Yeah.

Linda: Okay. Well, that’s nice. But now, you could hold your B and C students to doing critiques on all the assignments, just so they can put in their hours. But they don’t necessarily have to do the assignments, at least they can see models. But again, I don’t know if your accrediting agency would be happy with that either.

John: This may not exactly be specs grading, but could you give, say, different levels of activity in each of the assignments? If you have four-projects, you could have one bundle in that project that would give a C, another that would give a B, and another that would give an A, changing the scope of the projects and it would be specs grading within that scope.

Linda: Yes, that has been done and I’m not sure about what kind of projects you’re doing. But yes, you could do that. And you could set different specs for each level. So D students, lets say, have to do a D-level project, which is not nearly as time consuming and not nearly as high a level cognitively, it doesn’t demand that much, doesn’t demand much time, it doesn’t demand that much thinking. And so you could do that and that way students would put in their time, and as long as you can set out the specs for those levels. Yes, absolutely you could do it that way.

Rebecca: Yeah.

Linda: But yeah, accrediting agencies can really get in the way of flexibility. Rebecca, would your accrediting agency go along with that?

Rebecca: With scope, probably. Yeah.

Linda: Yeah. Okay, good. I want to tell you about a course that does this, it’s in computer science, they have students writing programs, and there are six projects that students have to do, they do these individually. If you are going for a C, you get some pretty easy projects to do, pretty easy problems to solve. If you’re going for a B, well you get sort of intermediate-level problem, you don’t do the C one. Because if can do the B ones, you can do the C ones. Now for an A you get some much more sophisticated ones. But still, for every level, you only get six. So there is that, that as long as you can designate level of difficulty, or for that matter, breadth of knowledge, if you can designate that… of course, usually people can do that… that you can make levels out that you can make different bundles out of that. This is another little interesting take, in this particular course, if you are late handing in your projects, that you’ve got to do two more projects at that level. So in other words, you are penalized for lateness by having to do more work. Now that really hits students where it hurts. And if you’re like super late, you got to do yet two more. So any way that keeps students on their toes. [LAUGHTER] Anyway, so in this particular course students are rarely late. So yes, there are ways that you can do that just in terms of level of complexity or level of cognitive operations demanded.

John: How do you explain this to students, if you’re going to introduce a specs grading system?

Linda: First of all, you’ve got to sell specs grading, because they’ll look at you like, “Huh?,” which is fine. And you should explain this, but you can tell them, “You know, I’m going to hold you to higher expectations.” And we know that holding students to higher expectations feeds into student success, student learning, but also student success in general. You could explain to them the concept of andragogy. In other words, that pedagogy really has to do with little kids. Andragogy has to do with adults. Saying, “I’m going to treat you like an adult.” And students like that kind of thing, you might be choosing the kind of project you’re doing. It’s easy to build choices into this. “This will be a safe but a challenging environment because you know exactly what you’re supposed to do. I’m going to give you the descriptions of what you have to do for the assignment to pass in advance and in detail. So you almost can’t screw this up unless you don’t read the directions or pay attention to the directions.” So you want to emphasize the choice and control over their grade that they’re going to have. You want to tell them about tokens, we’ve got some wiggle room, if you will. Now these tokens, by the way, are not physical, you make them physical students will develop a black market, [LAUGHTER] believe me, but virtual and you keep track, or you can ask for an extension, but you don’t ask me for an extension. You’re late, I just take a token away. That’s all. We don’t have to talk about it. I don’t care why you’re late, this is just the way it’s going to be. Just like your boss won’t care why you are late or why you are absent, it counts as like a holiday. Now we’re going to tie what your grades is to what you are learning. If they look at the syllabus at all, everybody’s going to know what you have learned in this course. So that’s how you sell it to students. Now students will need to be reminded about how the system works, because this will strike them. “This is so weird.” Even if they initially like the idea, they still need to be reminded a few times because it is really strange. According to my research, students way prefer this system to traditional grading. And one of the reasons why they do is because we give better direction, we tell them what we want, we give them the formula, we give them the template. or we give them tons and tons of freedom to meet the specs. Especially with students when we talk about length or whatever, length means depth to the students. It doesn’t mean that necessarily to us. Students really like this because they feel way more secure in it. And they do like the element of choices that are built in, that’s motivating for them. And they do learn more. And if they are more motivated, they’re likely to do more of the work at a higher level, they are more likely to excel. And the A students are going to do A work anyway, because they don’t know any better. All they know is A work. And there’s a sense in which they want us to love them. I mean, really, so they’ve got this strong sense of loyalty towards us and they really want us to respect them. And they’re used to feeding off of that they’ve been doing it all their lives. Don’t worry about them getting sloppy, they’ll never get sloppy on you. [LAUGHTER]

John: If someone wanted to transition to specs grading, how should they get started?

Linda: What you might want to do if you’re switching, transitioning to specs grading, you want to look at your assignments, and for that matter, look at your tests, like what you have in your tests. You want to look to see what can be transferred into pass/fail. You also want to look at your tests, if your tests are very objective, and you’re relying on a test bank, remember, it’s so much easier for you to grade with the specs that you can start assigning more written work or design work on tests, what have you. But definitely higher-order thinking types of questions and you will have the time to grade them, because you’re going to lay out the specs and without giving away the farm, you will tell your students the specs of your essay questions in advance, and they will study accordingly. But anyway, you want to look at your outcomes and then you want to identify the cognitive level of your assignments and your test questions which again, you might want to change. You want to be able to group your assignments and tests by that cognitive level so you can develop bundles. It’s kind of a radical way to do it, but it makes your life so much easier once you have those bundles. Because all you’ve got to do is say, “Okay, let’s see. So I had four assignments in this bundle, this student passed them all, hey B. It’s so much easier, you just have to be sure that you set out the deadlines for the different assignments in the bundles. You don’t want everything being submitted to you in the last week or two. So you still have to have your deadlines along the way. Just warning you about that. I want to refer people to my book for examples of courses, and not just my courses either, that have been specs graded. And they cover a range of over a dozen different disciplines. So they can see all kinds of examples.

John: That’s a great resource for those who are considering moving to specs grading.

Linda: Yes.

Rebecca: And it also sounded to me like a couple of cautionary tales that you have about transitioning are about time and making sure that you spread things out…

Linda: Yes.

Rebecca: …but also being very clear in what the specs are, and then following through with whatever you said.

Linda: Yep, yep. And if you didn’t lay out enough detail, well, there’s always next semester. [LAUGHTER] But you can’t change those specs in midstream. “Oh, I wish I thought of this and then start grading students.” No, no, no, no, no. But again, we live and learn, right? We’ll get better the second time.

John: We always end with the question. What are you doing next?

Linda: Oh, what am I doing next? Well, what I’ve been doing is I’ve been doing a lot of traveling, doing keynotes and workshops… oftentimes on self-regulated learning, sometimes on specs grading, and sometimes on any number of different topics having to do with either teaching or academic writing, or something like that. So I’m going to be taking a little bit of a break from that until the end of, or late, July anyway and then things start up again. But that’s kind of nice. I don’t mind that because this keeps me semi-retired and that’s the way I want to be. A semi-retirement is Nirvana, just letting y’all know. [LAUGHTER] So, that’s what’s next for me, that and to finish my tea.

Rebecca: Sounds like the key to semi-retirement is the semi not retired part. [LAUGHTER]

Linda: Yes, it is, it is. Your brain is still working and you still got your thumb in the pie so to speak. And it feels good because you’re not under that same 60 hour a week pressure that you otherwise have with a regular job and then doing these other extra things on the side and it doesn’t work real well.

Rebecca: Thank you so much for joining us.

Linda: Well, thank you for this opportunity. I normally say… autograph books and things like that… happy teaching, but I want to wish everybody happy grading. [LAUGHTER] An odd phrase right? Happy grading. [LAUGHTER] What an odd phrase, right? Happy grading!

John: Thank you.

Rebecca: Thanks, Linda.

Linda: Bye bye.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.