173. Pseudoscience

In an era of conspiracy theories and fake news, our students come into our classes with misconceptions and misunderstandings about our disciplines. In this episode, Kristin Croyle and Paul Tomascak join us to discuss how a first-year science seminar class confronts pseudoscience. Kristin is a Psychologist and Paul is a Geochemist. Kristin is the Dean and Paul is the Associate Dean in the College of Liberal Arts and Sciences at SUNY-Oswego.

Show Notes

  • Shermer, M. (2014). Why People Believe Weird Things. Naturalist.
  • Zener cards – American Psychological Association
  • Huff, D. (1993). How to lie with statistics. WW Norton & Company.
  • Van Der Kroon, C. (1996). The Golden Fountain: The Complete Guide to Urine Therapy. Wishland Incorporated.

Transcript

John: In an era of conspiracy theories and fake news, our students come into our classes with misconceptions and misunderstandings about our disciplines. In this episode, we discuss how a first-year science seminar class confronts pseudoscience.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guests today are Kristin Croyle and Paul Tomascak. Kristin is a Psychologist and Paul is a Geochemist. Kristin is the Dean and Paul is the Associate Dean in the College of Liberal Arts and Sciences at SUNY-Oswego. Paul also had been the Associate Director here at our teaching center at SUNY Oswego before he entered the Dean’s office and Rebecca joined us as Associate Director.

Kristin: Thank you.

Paul: Hi, John. Hi, Rebecca.

Kristin: We’re happy to be here.

Rebecca: Today’s teas are:

Paul: I have a special tea for you. I have a tea that has a best buy date of March 2000. A special tea.

Kristin: Does it have flavor still?

Paul: In a way… Yeah, It’s got a special flavor. [LAUGHTER]

John: A vintage tea…

Paul: Yeah.

John: …a good year.

Kristin: And I have coffee in a Christmas mug because the Christmas mugs are still out.

Rebecca: Mine are out year round.

John: And I have Prince of Wales tea.

Rebecca: And I have Big Red Sun.

John: …for a change.

Rebecca: Ah, it’s a little switch up. It seems sciency… It’s what I had open.

John: We’ve invited here today to discuss the first- year seminar course you both offered on “How to Think about Weird Things: science confronts pseudoscience.” First, could you remind our listeners a little bit about what the first-year seminar courses are here. We’ve done some past podcasts on them, but it’s been a while since we talked about that program.

Kristin: The first-year seminar course at SUNY Oswego is a relatively new initiative started just before I came here in 2018. But that’s before I came to SUNY Oswego, so I’m allowed to be wrong on dates before I started. It was initiated by our Provost, Scott Furlong. And the first-year seminar courses, the way that we envisioned them, is partially as passion topic courses for faculty, but also as a transitional experience for new freshmen so that they can have an experience in which they have both some social bonding, some interesting and challenging and really fascinating materials to talk about in course, but also some built-in experiences to help them connect to their new university and transition into kind of the college student way of functioning and being in a supportive atmosphere. So both academic challenge and excitement along with kind of the adjustment to the new university culture… Oh, and those are all taught in classes of 19 or less, so that there can be a strong peer-to-peer experience. And they also have writing intensive experiences involved.

John: What are some examples of pseudoscience that you address in your classes?

Paul: I’ve been teaching this course prior to the first-year seminar series for some years in a variety of different places: as an upper-level Gen Ed course for non majors, as a honors course, because the topic just transcends level, and it’s something that everyone can get something out of. And every time I’ve taught it, I’ve ended up emphasizing different things. And that persists. At one time, I was adamantly avoiding talking about conspiracy theories, because conspiracy theories are just bollocks. It’s a zero-sum proposition, there’s really no way out of it. There’s no good dealing with the topic. But given the fact that conspiracy theory is something that we all really need to be talking about nowadays, it’s something that I’ve brought in little by little, but it’s still dicey. You can talk about creationism, and have some strong things that you can bring up as, this is why this really is not tenable in there, lots of things you can talk about in terms of cryptozoology or psychical ability, or persistence of life after death, consciousness after death. And there are scientific things that you can point to with these. But with conspiracy theories, it’s always going to be “Oh, well…” there is always an “Oh, well” out of it. And so that’s a hard one to grapple with in any real constructive way.

Kristin: Well, one of the things that attracted me to the course…. Actually, let me tell you about how I got into it. As Dean, I wanted to get a stronger connection to the students. It’s good to have the experience in the classroom, especially at a new university for me, because I can see what faculty were going through in terms of: setup your course shell… What are the policies that you have to include? What are the students like in the classroom? How do you submit your grades? …all those kind of technical aspects also that Deans know. I wouldn’t have necessarily chosen Fall 2020 if I had perfect foresight about what that would have been like, but still… not necessarily as my first experience teaching at Oswego. But I still think it was valuable. But one thing that attracted me to the courses when I was thinking about what courses to teach, intro psych was actually my first choice because I enjoy hanging out with freshmen. It was my field. But then I thought… these freshman seminar courses, and I got a chance to talk with Paul on a regular basis in previous years, he was teaching a bit about all the interesting things we were talking about. And I think that course is fascinating, but as a psychologist, some of the things that really attracted me are pseudoscientific beliefs, particularly about interventions and treatments and the way people are scammed the way that having an understanding of how the brain and body actually work, and what evidence for treatment looks like versus people who are charlatans who are taking advantage of people who are in vulnerable positions. That’s the part that really hooks me into pseudoscience and why it’s so important to teach students about it. But with that, as a hook, you’ve got all kinds of possibilities, because it’s many of the same thinking errors and misunderstandings that open you up to paying thousands and thousands of dollars for getting your future read repeatedly. It’s the same kind of thinking errors that opening you up to those and some other things that are not necessarily mainstream.

Rebecca: So how do you overcome some of those thinking errors, or help students overcome their thinking errors?

Paul: I’m going to say “um” a lot and I’m going to pause a lot, because I know that it’s something that John enjoys editing out.

Kristin: But you should totally leave that…

Rebecca: Um….what do we think about that? [LAUGHTER]

Paul: When I teach this class, there are a number of things that I emphasize. But I emphasize that we are on some level, all scientists, we are all critical thinkers. And in order to get through life successfully, you have to be able to do these things. And I like to draw the horizontal line on the board on the first day and say, on this end is complete gullibility, complete credulousness, you’ll accept anything as truth. And on the other side is complete dismissiveness, complete cynicism, and you won’t accept anything, regardless of how well it’s shown to be acceptable or true. And that it’s important that you understand that there is a spectrum. And that being skeptical doesn’t mean being dismissive. It means that you ask questions, it means that you don’t accept things at face value, especially if they don’t really smell right. And if something has the taint of, “Well, this is too good to be true” …it probably is. And you’d be doing yourself a favor by looking more closely at things, getting some more information. So I try to disabuse students of preconceptions by asking questions and by forcing them to ask questions. And even with things that seem to be “Well, that makes sense, so yeah, I’m going to buy into it.” Well, why does that make sense? What’s the physical reality that underlies that, that makes you think that that is the way it should be, the way it might be? And where do you get your information? And that is a very productive line of inquiry, where you start to break down the “Well, I heard it from this person…” Well, what does this person know? “Well, I heard it from this website.” Well, let’s go to that website and look and see if there’s anything that we can connect to. And is this someone who’s just manufacturing information? Or do they have links to somewhere where you can say, “Wes, this is verifiable on some level.” So it’s good regardless of whether you’re talking about something that’s way out there or something that’s not so way out there. It’s good, basic, critical thinking.

Kristin: And one of the things that I think is very helpful is repetition. I went through a lot of topics, but in each case, there is this harking back to what kind of thinking errors might be present, what kind of scientific errors might be present. And as they start to do that over and over, they get better. For example, one of the early topics that I talked about was alien abduction. When we talked about alien abduction, we talked about how does memory formation work, we talked about sleep, the sleep cycle, hypnagogic and hypnopompic hallucinations and sleep paralysis. We talked about false memories, and how false memories are formed, and that they are experienced in the same way as real memories. If you have a false memory, it’s not like a different thing for your experience. We talked about all of those kinds of normal processes, as well as, unfortunately, the role of hypnosis in creation of false memories, which has a lot to do with beliefs and induction. I say, unfortunately, as a psychologist, it’s horribly embarrassing for the field. it really is a terrible thing. So we talk about all of the scientific contributions, and then we talk about “Okay, now the experience of alien abduction.” How does hypnosis fit in there? How do sleep paralysis, and hypnagogic and hypnopompic hallucinations, fit in here? Those are hallucinations as you’re falling asleep or waking up…it feels very real, but are actually more like a dreamlike state. How do all of this fit in? And then we look at an account of alien abduction and say, “Okay, what do you see here?” And then they can identify some of the thinking errors, like “Okay, here’s this part… looks like a false memory.” But sure, they’re really upset because it feels real. This part here, extraordinary claims require extraordinary evidence. There’s no extraordinary evidence, so they can start to identify both how do we separate the science from the non-science and then where can we start to identify thinking errors. And as we do that topic after topic, they get better and better and better at it.

John: In all of our classes, following up what Paul said, students come in with models of the world and those models aren’t always accurate… or we often have better models that we’d like to share with our students. But it’s important to break them down. And you’ve talked a little bit about how you can provide them with evidence to help them perhaps modify their models of how the world works. But, what do you do with those students who are really resistant, who really deeply believe in some of those pseudo science principles?

Paul: Yeah, this is something that Michael Shermer talks about in one of the books that I’ve used as a quasi textbook has been Shermer’s Why People Believe Weird Things. And in the later editions of the book, he has a specific chapter, that is “Why Smart People Believe Weird Things.” Because, again, early on in the class, there’s something of an inclination to think of, “Well, I don’t think crazy things like that, and it’s only the gap-toothed yokels that believe in alien abductions or that believe in whatever it is.” But it’s important to understand that this is not something that’s limited to people who aren’t smart. There are plenty of people who are genius-level smarties who believe, not just weird things, but things that are patently out there. And so getting students to accept that, “Okay, we can talk about this as a group, because we’re not just pointing out that you’re a dummy, these are things that lots of people believe, and there are reasons why they believe them other than just being morons.” So the idea that preconceived notions are things that aren’t necessarily rooted in ignorance, or rooted in stupidity, but they’re rooted in misinformation, they’re rooted in being told something by someone you trust at some point, and not questioning it. So I think creating an atmosphere that people can feel good about talking about these things, and not just sitting there going, “Oh, I hope he doesn’t talk to me about this, because I actually believe in ghosts,” is useful. And I’ve had students in class who are ghost hunters. And we’ve gone through an entire lesson on why some of the classical ghost hunting techniques really don’t make sense when you analyze them. And I’ve had a student say, at that point, “Well, we don’t really do that, what we do is this,” and everyone in the class looks nervously at one another, that “Oh, that’s interesting. I didn’t realize that they were among us.” But, they are among us, because we are them. They are us, we all have an equal opportunity for believing weird things.

Kristin: One of the things that I also talk about is different ways of knowing. And that when you say science proves X, Y, Z, it has to meet a scientific standard. But if you say, for example, my faith tells me X, Y, Z, that’s a different way of knowing. And it’s not subject to the same kinds of proofs, it’s subject to different proofs. An example that we explicitly talked about is angelic visitations: are angels real? If you say science proves that angels are real, it has to stand up to scientific scrutiny. And in many religions, that would not only be a weird thing to say, it would be antithetical to their religious perspective. As soon as you start saying science proves my religion is correct, it becomes in some ways, a non-religious argument, and that it’s perfectly fine to have different ways of knowing different aspects about the world. But if you say science says this, this is the way the world works, because scientists have proved it, then you can subject it to scientific scrutiny. Another example is intuition and personal experience, that there are aspects of intuition and personal experience that may tell you certain truths about yourself or your relationships with others or whatever. And you don’t have to have the kind of scientific scrutiny in order to believe that you understand the way that your relationships work. But that’s a different way of knowing, it’s a different aspect of the world, and we do talk about that explicitly. And it’s fine with me if students choose to hold two ideas in their mind at the same time, they say, “Well, perhaps this idea that I have doesn’t actually make any scientific sense. I still believe it right now.” But I have some faith that if they continue this process to continue to analyze different ideas using the same skill sets: How does this make sense? What are their thinking errors? Is there an underlying explanation that makes some scientific sense that fits with the way that we know the world works. If they continue to do this, that eventually some of those closely held beliefs, which are scientifically disprovable, that they will start to kind of chip away at the edges there.

Rebecca: I know both of you are big advocates of active learning. Can you talk a little bit about some of the activities or exercises or things that you have students do as part of this course.

Paul: One of the classics, when we talk about psychical ability is pairing students up and having them basically test each other and their clairvoyant skills. So you give them the set of five Zener cards with the star and the squiggly lines and the square and you have them run through a series of “Okay, I’m projecting an image to you, you write down what it is.” And that’s good from a couple of standpoints. One is that it’s active and people are taking part in it, two is that people can understand: “Okay, if I really wanted to do something to show that there is something viable here, what would I have to do differently? Why is this test flawed?” And we talk about the development of good scientific tests. And that’s very productive, because there’s a lot of situations where you can say, “Well, you know, you’re still not controlling for this…” Okay, and the series of sort of nested tests that you have to go through in order to get to something that everyone would say, “Okay, I will accept the results of this” gets to be pretty complex. The other thing that’s good about this on a basic level is that it regresses to the mean. And regardless of the number of students, the number of tests, occasionally students will cheat and you can talk about that. But aside from cheating, you end up with a bunch of people that score exactly what statistics would say you should get and you can talk about one of the big things that I like to emphasize is not to let people use numbers to try to prove something to you that isn’t accurate, basically lying with statistics. A former student in the class sent me a book at some point, this little book called How to Lie with Statistics. And it’s a great medium to talk to students about things that are mathematical in a world where people are fearful of math, and they hate math. And this is a good application of mathematics, sort of basic mathematics to show something that is easy to wrap your head around. And it’s something as well in Shermer’s book, he talks about going to Edgar Cayce’s Institute, and doing this sort of mental ability test or psychical ability test. And he does the same thing. And he tries to convince people that “Well, just because you got 5 right out of 25 doesn’t mean that you’ve got some exceptional ability,” and he draws a bell curve, and they talk about it. And in the end, the person still doesn’t accept it. But it’s a good experiment to run, it gets people thinking about something that is not necessarily easy to think about otherwise.

Kristin: I’ll start by saying that I have huge sympathy for all the new faculty that started in Fall 2020 and were trying to build new courses while coming up with different teaching techniques. I was challenged this semester, this last semester, to build the course while trying to adapt to what was an unfamiliar form of teaching for me. Paul was very gracious in sharing materials. But, you know, when you teach the course yourself has to be rebuilt because it’s your own thinking, and your own style. Just for disclosure, though, I had intended the course to be a hybrid course in which we met with our faces, at least, three times a week, sometimes in the classroom altogether, and sometimes all online together. But as the semester went on, it did not work that way. I ended up having some students that always want to come face to face (a small number), and some that always ended up being online. So it was not the course I anticipated. But that’s okay. I know that we all experienced that. What my students responded to the most enthusiastically ended up being analysis of web comments. So I would often bring in slightly adapted web comments, I would correct for grammar and, you know, readability …say here is this diatribe this person and removing their identity and things because it’s about analysis of argument and they would go to town on it. Here’s this diatribe about astrology, it runs from how scientists are paid to debunk astrology all the way down to how you should stop being sheep and see the truth in front of you and everything in between, with all kinds of false analogies that don’t make any sense in the middle, all that good stuff. They loved that. And I loved it too. We all loved it, because that’s what I really want them to be able to walk out doing, to be able to see kind of something that looks like a well argued and well written diatribe against the world who doesn’t understand and to be able to look at it and say, “Oh, wrong, wrong. wrong, thinking errors, misstatement, false analogy, ripples in a pan have nothing to do with how stars move, and all kinds of different things. [LAUGHTER] So we ended up doing a lot of those kinds of similar things. I think one of the last things I did in the last homework that we worked on together was on a manifestation website service, you sign up for $1,000, you get these courses, and you can manifest wealth in your life and their analysis there was really excellent. It was excellent about why this might appeal to people. What is wrong with all of these arguments? It doesn’t matter how many incredibly well done video anecdotes you get from individuals who have manifested wealth in their life, that that’s not gonna transfer to other people. So lots of analysis of web comments.

John: With social media, there’s a very rich source of data that could be used for this.

Kristin: Exactly.

John: Could you tell us a little bit more about the course structure and what you’re doing in these classes?

Kristin: I have avoided student presentations in class for 10 years, because I usually find them to not be a good use of course time, let’s just say that. But Paul was using student presentations, and I put them in for this course and they were awesome. So, I have completely changed my opinion. But part of it is also that I was teaching larger classes in the past. So figuring out how to integrate student presentations in a way that is a useful use of everyone’s time, but the student presentations in this class were fantastic. They were typically on a specific pseudoscience topic that we wouldn’t have spent a lot of time in class on. But it gave them an opportunity to again, have this kind of repeated, “Here’s a thing that you think is really different.” Like. maybe… maybe not… Chromotherapy, you know, does exposing yourself to different colors of light effect different organ functions beyond jaundice, and beyond seasonal affective disorder where there’s clear evidence… if you look at blue light, or red light, or whatever. People go “Hmmm, I’ve seen videos on this on TikTok… well, wait a minute, doesn’t make any sense.” And here are the arguments, a little scaffolding from a student presenter, here are the arguments about why this doesn’t make any sense, then students popping up with other arguments. And having that experience repeatedly, of student presentation after student presentation, I have worked them like you know, three or four weeks, it gave them more experienced practicing. And honestly, some of those topics are fabulous to talk about in class. Although I allowed students to select their topic out of a menu so that they didn’t have to know what was pseudoscience right at the beginning of classes. No one selected urine therapy, though, I was hoping given how much success Paul has had in his classes with that.

Paul: Urine therapy is number one.

John: Could you elaborate on that a little bit, Paul?

Paul: The student response to the class has been really good historically. And I will occasionally, and sometimes out of the blue, receive a book in the mail from a student. This person that I had never heard from after the class, student says: “I was in a bookstore, I saw this and I thought of our class, and I thought you might like it.” So that’s always really nice. But it’s especially nice when the person sends you the definitive book on urine therapy, because my library was not inclusive enough of that topic. So now I have something that when a student chooses, or pulls the short straw, on urine therapy, I have something I can give them as a resource for this topic.

Rebecca: A whole book….

Paul: A whole book. I think it’s called the Golden Fountain. I’m not kidding. When I do the course and I have students do some sort of presentation, I will, so that I don’t run into the problem of a student doing something that they already know a ton about, I’ll have them draw them at random. And from the start, I’ve got the little hat with pieces of paper in it, and I’m telling them: “Who’s going to draw urine therapy?” …and it’s hotly contested. And it’s great when the student comes in to give their presentation that day, and starts out with a long pause and says, “This really makes me sick.” [LAUGHTER]

John: I’m not sure if I should ask, but what is urine therapy?

Paul: Well, I’m surprised being a man of the world that you are not well aware of this, John, but by consuming your own urine, you’re able to tap into a great deal of vitality and essential nutrients, etc, perhaps some reparations to your chakras as well, through consuming your urine. There are people out there who will attempt to get you to pay them money to teach you how you should be doing this. But it comes down to drinking your own urine and having that basically cure any disease. And you can take it purely internally, you can rub it on your skin to produce a healthy skin tone, you can use it in your hair. There are certainly people out there who will claim that it is a cure for cancer. And that’s sort of the bar for all pseudo-medicine is when are we going to get to the end, this cures cancer. And sure enough, there are people out there. It’s usually a sad case where the person had cancer, they went through a number of different treatments, nothing was working, and they hit on this and suddenly they’re cancer free. And it’s a good place to talk about correlation and causation. It’s a good place to talk about how we design clinical tests for medications, vaccinations, whatever. When an agency says “Yes, this is demonstrated efficacious or this is demonstrated safe…” what does that actually mean? Well, it has to go through a certain process, which is not some random process that someone hands over some money and “Okay, yeah, you’re good to go,” that these are real things. So that, I think, is another area in which I’ve significantly improved over. I think I started teaching this in 2006. I talk more about anti-vax. I talk more about clinical trials. I talk about the placebo effect, and Kristin has actually helped me a lot with that. Because she knows about things that I didn’t know about when it came to placebo effects. So there’s a lot of good stuff there that, again, it’s science, but it’s not something that you need to have a degree in something to understand and to be able to then apply in your own life.

John: In terms of the placebo effect, there’s two things that just really struck me in terms of fairly recent research. One is that the strength of the placebo effect seems to be growing over time. And secondly, that the placebo effect still seems to exist, even when people know they’re taking a placebo. Any explanations of why that’s happening?

Kristin: Isn’t that fascinating? I just think that’s amazing. No, no explanations. I have great admiration for the power of the mind.

John: Mystical powers? [LAUGHTER]

Kristin: Well, for example, there is excellent research that says that people who have even late-stage cancer will survive longer, if they have social support. That’s not placebo. That’s because your mind and body are constantly one system and that we survive in a social environment… just one reason the pandemic has been so difficult… and that people survive and thrive better when they’re in a supportive social environment. Totally not placebo. But it is, in some ways, our traditional Western medical approach would see that as a psychological or mental intervention. It’s amazing. Although the early psychoanalysts, they did some strange stuff, and claimed some strange things, Freud and his students, some of that early work, it really does demonstrate if you believe that something is going to be very different. Hysterical pregnancy is a great example. People who believe that they are pregnant strongly believe that they are pregnant who are not actually pregnant, show many physical signs of pregnancy, including abdominal distension and ending of periods. Sso there’s a lot of different things that the mind can do. Unfortunately, only that only takes you so far. But that is definitely something that I talk about in class, as well as the waxing and waning nature of many illnesses, and how that opens people up for charlatans to take advantage of them. Multiple Sclerosis is a great example, where there’s unpredictable often waxing and waning symptoms. And people with MS have been targeted for many, many, many, many years for completely wacky, expensive, invasive, painful treatments because of the waxing and waning nature. And if their experience is that it has healed them, it’s hard to say that’s not your experience. But it is easy to say there isn’t any scientific evidence that this would help anybody else. They’re taking your money, unfortunately. And I also talk about how parents with children with significant developmental disability are often also at a point of desperation, where they’re sometimes ripe for this kind of thing too. One of the students in my class presented on hyperbaric oxygen chamber treatment, which of course is a great treatment if you have the bends after scuba diving, but is not effective for autism, though there is a market to sell people, these chambers for $20,000 to have a chamber in their homes so that they can put their child who has autism in the chamber on a daily basis, which for one thing is expensive and not effective in any way. But it’s also potentially also really scary for a child who doesn’t understand what is going on being shut up in the chamber every day. So, beyond the improved understanding of how the world works, there is, also real harm being done by some of these things. And we’re talking with students about the importance of a control group. Why does having a control group make all the difference? And talking about that repeatedly as these other examples come up, I really believe will help them to understand the world better, and become better consumers and self advocates.

John: One of the things you just mentioned is the importance of a strong social network and of human connections. How did you nurture that in this somewhat challenging circumstance of fall 2020 during the pandemic?

Kristin: That was really hard, because it’s something that I have never struggled with in class before. And it was a real struggle this semester. I don’t know if that was the case for you too, Paul, or Rebecca. But this is something that I consider to be an easy and normal thing in my teaching. But this semester, it was really a challenge to have students make peer-to-peer connections. I feel fairly comfortable that they felt a connection with me. And I certainly felt a connection with them. But getting them to connect peer to peer was a challenge. And I attribute that to first, not ever having done it this way before. I think if I had another chance I could do it better. Just like any kind of teaching, the second time around is usually better than the first. But part of it was that I was so responsive to students who felt like they needed the face-to-face interaction that I continued to meet face to face every day with them with a chunk of students on Zoom. And it would have been, given my teaching style, it would have been a better experience, I think, for all of us if we’d stayed in one together format more often, if that makes sense.

John: I think this is a problem we all faced, that student peer-to-peer connections were challenging, both because of the modality and because of the circumstances in which we’re all living right now. Paul?

Paul: This past fall, I taught a different course. And it was an upper-level honors course. So these are students who… they’re high achieving, they had figured college out. And it was, for me the easiest of all scenarios, because they were on task, and not that they weren’t necessarily happy with the way that the world was going, but from an academic standpoint, it was a fairly easy scenario to adapt to.

Rebecca: I wanted to circle back for a minute about the diversity of topics that you addressed in class, and what you’re using as hooks, and the value of the different kinds of topics as hooks for students. So there’s some that I think fit in the category of very outlandish, which are probably really easy for some students to really get into… find fun… and then there’s also some of the medical things that you’re talking about that I think students might relate to more directly, and they can see how it fits into their lives. Can you talk a little bit about how you chose the topics and how your students may be related to those topics?

Paul: Certainly, when you’re just talking about science, it is harder with a mixed audience of students who aren’t necessarily buying in from the start. In previous incarnations of this class, it was nominally a natural science course, but realistically, it was being taken by everybody. When I taught it as a first-year seminar course, there was a fair number of psych majors. But really, it was a complete mixture. So, I felt obligated to present a certain amount of science. Here’s a big idea in science, why do we think this? What’s the evidence for this? Why is this important? Why should you care? So I was able to get to things like creationism through the door of “Well, how is it that we know that the earth is as old as it is? And why is it that this is not just something that was handed to us, and we believe it, but it’s something that’s objectively demonstratable?” And beyond that, when you start talking about biological evolution? And okay, why is it that we believe that this is at least a reasonable description of what’s going on in nature? Okay, here’s some stuff that’s a little bit dry. But the end goal is being able to say, “Yeah, I can accept this beyond just having it handed to me.” Evolution is a good one, in that it integrates a lot of different things. So you can bring in the purely biological, you can bring in populational, you can bring in geological and physics, and you don’t have to dwell in any one particular spot to try to make the point. But nevertheless, there are portions of the class that are somewhat more pure sciency, and I try to front load those in the course to keep the carrot out there of “Oh, we’re going to be talking about psychical abilities soon, and we’re going to be talking about UFOs soon,” because that’s fun stuff and ghost hunting and all that. But yeah, the science is a critical underpinning for the course and trying to get it so that it’s not just: “Here’s the scientific method, memorize this,” …to have it be science is a process that we all are invested in, and when you stop investing in it, then there’s trouble. And I think that the past year has really underscored the fact that that’s something that everyone should be… certainly every college educated person… but really everyone, should be understanding of the fact that science is a critical tool. And it’s not just the sacred tablets that have been handed down from the clouds, it is something that has objectivity, and there are processes… and what makes a scientific paper. We keep talking about, “Well, this vaccine test was done, and it was published in The Lancet, or it was published in the Journal of the American Medical Association. Why do we care? Is it just we paid more to get our article in this journal that people quote? No, it’s that these journals actually have a high bar for what they accept as publishable. And if it’s published in there, it means something. It doesn’t necessarily mean that it’s going to be true a week from now. I think in dealing with science, it’s good to emphasize that it’s not just something that is dusty books sitting on shelves. But by the same token, there’s an inherent danger when you expose the fact that we don’t know anything for certain. And it’s nice and comfortable to think that when you drop the apple, it’s going to fall at a certain rate. And when you get up tomorrow morning, the sun is going to be rising in the east. But when it comes to it, the more contentious the scientific question comes, perhaps, the bigger the scientific question becomes, the greater the likelihood that we’re going to continue to develop our understanding of things and rooting out the question of “Well, that’s just a theory.” Well, it’s not just a theory. If it’s a theory in science, it means something. It doesn’t mean that it’s a hunch. It means that this is something that we’ve put an awful lot of effort into, and awful lot of thought into. A lot of people have had their eyes on this. It’s not just one really smart person saying, “Okay, this is the deal.” …just the process by which we have to go in order to get to the point of saying, “Yes, we accept this as the way things work, whether it’s biological evolution, or whether it’s the verifiability of vaccine, or whether it’s anything.”

Kristin: And one of the things that you’re touching on there, I think, is also an important theme that comes out: that science is a continuing investigation, that it’s very comfortable for students, especially in K through 12, to think about scientists having answers instead of being an ongoing investigation. And typically the things that are taught in K-12 are the things science has answers for, not the things that are continually being investigated. So it can be scary for students who have that background to be confronted with news that our understanding of a virus is changing over time, because that’s the way understanding works. It changes over time as we learn more and more. This theme keeps coming up throughout the semester as well saying, “Hey, this is what we understand now. The state of our knowledge is this. The door isn’t closed to the state of our knowledge to be different in the future. It also gives us a good opportunity to bring in the importance of diverse voices as scientists. So one of the things that I talk about in my class is the roots of psychological assessment and intelligence testing, and how some of those roots have explicitly racist foundations among people who were explicitly racist and some probably unintentionally racist, but having racist impacts. And some of that is clearly because there were only white men doing work at that time in that area. And when you have only one perspective, it leads to one group of answers, that if you have a more diverse group of scientists who are studying a question, they expand the definition of the question, they expand the definition of what is possible evidence, the answers that they come up with are different and better answers because of the nature of scientific investigation. That it’s not just we have a question, and here’s the answer. It’s we have this question about the world, what does the question mean? Is that the right question? Is there a bigger question? How can we investigate it? Let’s look at different evidence, let’s expand our understanding. As part of that, we also talked about the foundations of photography, and what happens when you have only white people creating photographic film and processing. And what happens when you expand that into a more diverse group of people on a more diverse group of images, the same kind of idea. Although I have to say the horoscope and astrology stuff was the stuff that got the most excited,

Paul: Ah ha, the fallacy of personal validation. [LAUGHTER]

John: But I think we can also generalize what you were just talking about in that all of our disciplines involve in ongoing investigation, and that students come into our classes, thinking of them as these defined bodies of knowledge that they just have to memorize. And it is a bit of a shock and adjustment to students to see that there are many things we don’t know. And that takes a while to get them comfortable with that idea and accepting that idea.

Kristin: And that it’s not a flaw in the scientific process or the state of knowledge, the fact that it’s changing. That’s not a flaw, that’s actually a feature. Yeah, that’s a tough one.

Paul: And one of the things that I specifically talk about in the whole science, you know, what is science? What is pseudoscience? …is where things go wrong. And we talk about fraud. There are a number of times during the course where we’ll talk about “Well, this was published in this journal, and it was wrong.” And let’s see what happened later. And we talked about retraction and things like that. So the self- policing nature of science, when it’s working, right, it’s the best way to get to the point of feeling good about an explanation for something. It doesn’t necessarily mean that something is proved or something is fact. But we have this process in place, and as long as it’s a topic that people feel is important enough to have lots of eyes on it… well, there’s going to be no way of hiding that one set of results that doesn’t seem to agree with everybody else’s. And those things get found out, they get basically debunked, and the science moves on. So the idea that science is fallible, the idea that science isn’t perfect, it’s something that has to be embedded in that. But by the same token, because of the nature of the process, we can say that science is about as good as we can do when it comes to understanding and this was Carl Sagan… all that.

Rebecca: We always wrap up by asking what’s next?

Kristin: What’s next? What’s next… I’m looking forward to spring semester. I’m looking forward even more to the next fall semester. I think we all are in that position. I really do appreciate the experience that I have with my students and I’ll teach again next year, but since the universe is paying me to be Dean, I have to do that work as well this spring.

Paul: Well, my life has been leading up to this podcast. So really after this, there’s not a heck of a lot left for me. Now, it’s nice to know that CELT wasn’t destroyed by my being part of it once upon a time, and it actually seems to have improved since then. That’s a nice job.

John: Thank you. I think this is a fascinating course. And teaching students to more critically analyze what they read and hear in social media and in their social network is a really valuable skill. So I’m glad you’re working on that

Rebecca: It really does seem like what college is all about.

Kristin: Well, thank you. It was a lot of fun. And throughout the whole semester, I was grateful to Paul for the scaffolding that he gave me. He was able to answer all kinds of questions and gave me interesting materials to work off of. So thank you, Paul.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

169. Statistical Simulations

Abstract concepts can be really difficult for students to grasp. In this episode, Matt Anderson joins us to discuss how simulations can be used to make statistical concepts more tangible. Matt is a lecturer in the psychological sciences department at Northern Arizona University. He was a recipient of the 2020 College of Social and Behavioral Sciences’ Teacher of the Year award at NAU.

Show Notes

Additional simulation resources:

Transcript

John: Abstract concepts can be really difficult for students to grasp. In this episode, we look at how simulations can be used to make statistical concepts more tangible.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Matt Anderson. Matt is a lecturer in the psychological sciences department at Northern Arizona University. He was a recipient of the 2020 College of Social and Behavioral Sciences’ Teacher of the Year award at NAU. Welcome, Matt.

Matt: Thanks very much, John. It’s a pleasure to be here. And I’m really excited to be talking about the use of simulations in introductory statistics classes.

Rebecca: Today’s teas are:

Matt: Today’s tea is lemon ginger.

Rebecca: Well, that sounds good.

Matt: Yeah, that’s a fave.

Rebecca: How ‘bout you, John?

John: I have Christmas tea…

Rebecca: Aha.

John: …with a cinnamon stick.

Rebecca: That’s only because I was drinking it last time.

John: You inspired me to buy some Christmas tea, which I’ve been drinking for the last couple of weeks.

Rebecca: I almost sent you some. And I have a Scottish afternoon tea today.

John: We’ve invited you here today to talk about how you’ve been using simulations in your courses. But first, could you tell us a little bit about what courses you normally teach?

Matt: Absolutely. My main mission is to teach the labs for the undergraduate statistics courses, the introductory statistics courses that are taught within the Department of Psychological Sciences. And so I teach about six to eight of those a semester. And I think it might be useful to just contextualize the labs. They’re one-credit labs, and they’re embedded in four-credit, introductory statistics classes. And all of our faculty use the same schedule, and we use the same textbook, and so there’s a lot of coherence among the sections that we have. So, the thing I love about the lab the most is that, because I’m getting all the psychological sciences majors, I have a chance to meet almost all of them. And I have a chance to do that early in their academic trajectories. And that provides an opportunity to get things on their radar for those who are going on to graduate education such as the graduate record exam and things like that. We have a little bit of a focus on SPSS in our lab, in addition to the normal course content that we would see aligned with the textbook. In addition to the course faculty teaching the lecture portions, and I’m teaching the labs, we’ve got some wonderful tutors, and those tutors come from our Academic Success Center. And we’ve also got a tutor who’s a second year graduate student in our Master of Arts in Psychological Sciences program who adds a lot to the course delivery. So it’s a really wonderful place for me to be teaching. I feel very well supported, and I love the mission, even though I understand that statistics may be, to be contemporary, not everybody’s cup of tea. [LAUGHTER]

John: So how many students are there in these classes? And what level are they? You mentioned they’re fairly early in their career, are they mostly sophomores?

Matt: Well, it’s a mix. Yeah, I think that the tendency is that we’ve got sophomores and juniors, we do have first-year students, and we also have seniors, but most students are sophomores, or juniors. And in each section, I’ve got maybe 30 students. So you can see when I’m teaching eight of those, that’s a lot of folks. And that’s complicated a little bit by the online delivery that we’re using right now as well. I also teach a thing called an undergraduate teaching apprentice class. And in this very small class, I’ve got students who are interested in learning more about the science of teaching and learning. And we focus on statistics, and they have applied assignments where they might help me with our learning management system. I think I’ve been inspired by all of the great simulations out there, and I’m going to add an assignment related to those as well. And then I also teach a fully online statistics lab for undergraduate students who are transfer students, who might not have had a lab experience when they took their lecture. And so this uses the same materials that we use in the face-to-face labs or the labs that we use for our basic introductory classes.

Rebecca: So, you participated in a redesign of your introductory statistics classes, can you talk a little bit about this redesign, maybe where it started, and now where it’s ended up?

Matt: The redesign that I was involved with had to do with the lab portion of the class. And it started around 2013, when I was hired to teach the labs for these courses, and also for our research methods in psychology classes. And up to that time, the labs were taught quite capably by graduate students, but there was variability in content and delivery and things like that. And so it was in the interest of the department to consolidate those into a single uniform experience. And that’s what I had the pleasure of putting together. And so what I started doing was building these what I called lab modules, and they would be used in each of the classes. And when I first started doing those, the version I used on Thursday was much different than the version I used on Tuesday. And there’s a lot of evolution that took place. And teaching eight of these labs a week, it was nice to have some development take place that was meaningful over the course of a single semester. And right now that lab manual is still being used. It’s fortunately not something I have to stand at the copier and print, but it comes in a bound book through our NAU bookstore. And it’s got modules that are aligned with each of the chapters in the textbook that we use, as well as very specific freestanding modules related to things like SPSS assignments and power analysis, and a little bit of Excel that’s built in there as well. And the fun thing about putting this thing together, I just loved the creative process. And I benefited enormously from the input from instructional designers at NAU. We’ve got some just phenomenal folks there who had some really important insights to provide that we put into the lab manual. So, it’s got QR codes, for example, in it. And so if a person gets to a particular part in an SPSS assignment and can’t remember how to do this, they can just use the QR code to see a very short little tutorial on how to do that. And I think being able to build those kinds of resources in something like this, make it interactive, I think is useful for the students and for me. It’s a really fun part of the creative process.

John: When you started working with constructing these labs, did you start using simulations right away? Or was that something you’ve gradually been adding since then?

Matt: Well, I started adding them soon after I started building them. But it wasn’t until maybe a year later that I started embedding simulations in as assignments. I was one of those students who really struggled, I probably shouldn’t say this out loud. My first stats class, it was very abstract. It was early in the morning, which complicated things for me. But, what I found was that I really benefited from seeing things to help marry these abstract concepts to real data. And about the time that I started teaching, there was a series of videos that were put out one was called “The dance of the P values.” It was by Dr. Geoff Cumming and it had a beautiful simulation attached to it. And so I was just starting to learn R at the time. And so I started seeing if I could replicate his findings using R and was able to, and that gave me a little bit of encouragement about building them. And at the same time, in 2014, our mathematics and statistics department helped host a International Conference on the Teaching of Statistics here in Flagstaff. It’s a huge international event. And it brought people together who were just marvelous at explaining and had these beautiful simulations. And they also talked about how to teach courses using R. And I just found that whole thing inspirational, in addition to having the pleasure of meeting some of my colleagues in the math department that I might not have met otherwise. And so that opened a whole new window into what simulations are out there, created by these really incredibly bright and capable and devoted teachers to the introduction of statistics and psychology,

John: I have to ask, what does “The Dancing P values” do as a simulation?

Matt: Well, they don’t actually dance. They do move. This is very similar to the way that I dance I suppose. But what it shows was that with small sample sizes, the P values just really were not consistent. And that was a message that was really central to what he was trying to put across. And the way that it was articulated and illustrated, I thought, was really compelling.

Rebecca: And who can argue with that title. That’s the hook. You have a good stimulation with a good hook, you got your attention.

Matt: It is a hook, right? Yeah, even if you didn’t have an interest in statistics, there might be dancing involved. Yeah.

John: And p-values is a concept that students often have trouble with. So, having that practical application, I would think would be helpful.

Rebecca: For those that aren’t familiar. Can you describe what R is?

Matt: Yes, R is a statistical software package that was built from the ground up to do stats and represents statistical graphics. It’s incredibly powerful. It’s free. And it’s also open source in the respect that people build these things called packages for them, which extend their capabilities quite a bit. And so if you can think of almost any esoteric statistical procedure that you would like to implement in your own lab, for example, there’s probably a package out there to do that. And the thing that I liked about it was that it was able to be paired with a thing called Rstudio, which I thought was a nice integrated development environment, which has some additions that allows you to take some of the things that you do on your local machine and put them on the web. So it was really a nice match between what I wanted to do in the lab and what I wanted to put out on the web for people to be able to see.

John: How do your simulations use R.

Matt: That’s a great question. They basically are simulations that I’ve built in R in this add on called Shiny. And so the students don’t see any R code at all. That said, in the labs themselves, I do think it’s useful for them to be able to interpret statistical output from different software packages. So I do give them some R output and ask them to make meaning out of it. But I don’t ask them to do any coding at all.

Rebecca: Can you talk about the difference in students experiencing simulations versus different kinds of exercises you might have had them complete prior to introducing the simulations into your course.

Matt: Maybe right now, I should just define what I think a simulation is. And so this is a very wide net. And I think it’s basically any visualization that allows you to “What if?” questions, to explore and demonstrate connections between abstract concepts and real data. Some of the simulations that I’ll talk about allow you to do statistical inferences as well… so, incredibly powerful. So I think these simulations and the use of the simulations exist across a continuum. I think there are some environments, such as the one in which I am operating, where we use simulations to try to reinforce critical points. So, Central Limit Theorem comes to mind. But there are also some courses where they build the entire semester around the use of simulations, they start them very, very early in the course, leveraging people’s natural inquisitiveness and their desire to see patterns and use that over the course of the semester to develop this deep understanding, not only of the details regarding statistics, but the big picture, how these things are all wired together.

John: Going back to Rebecca’s question, how have students responded to the use of the simulations compared to what you’ve done earlier in some of these lab assignments.

Matt: One of the simulations is one that’s done by the Rice virtual statistics lab. And it’s one that has to do with sampling distributions, which for me, when I was learning it and teaching it and for students still, a difficult concept. Imagine, if you will, just this normal population at the top of your screen, and then three boxes below, and the box immediate below, you have the mean of a sample that’s drawn. And then below that, it gets pushed into what emerges as a sampling distribution based on that sample size. And then the fourth box, the one at the bottom, would allow you to do a different sampling distribution. So you can do two at once, if you will. But this is visually really appealing, because it allows you to see the random sample being taken and where that winds up being put and how those samples aggregate to develop the sampling distribution. So that’s one that I built an entire assignment around, because you can predict some of the values of the sample distributions based on the math. And so it was nice to be able to put all those things together, I think, and the added beauty of this is that you can take that normal parent population, and you can make it one that’s non-normal. And you’ll see when you rerun the sampling distribution that you wind up with, in most cases, a very normal looking sampling distribution that allows you to run those inferential statistics. So it helps connect some of the dots that might not be connected otherwise. And so, while I don’t have any p-values myself to evidence how successful this has been, I have heard a lot of “a-has” when I’m talking to students about this, which to me is the Holy Grail. And they seem to get it with these simulations. As I researched simulations in preparation for this particular conversation, that was something that was echoed in all of the presentations was just the students really getting it and being able to leverage previous knowledge and being able to put all these things together so they can anticipate what happens in the future, when they do other simulations.

Rebecca: There’s something really powerful about being able to observe something and make that rule for yourself rather than just being told the rule that you have to follow. Otherwise, it seems really arbitrary.

Matt: Rebecca, that’s absolutely true. And it’s kind of fun to see these things played out with real world data that is much more compelling to students.

John: Inferential learning about inferential statistics.

Matt: [LAUGHTER] Absolutely, yeah.

John: But those are things, again, that students do have trouble with. They have trouble understanding that the estimators themselves have distributions. And this should make it a whole lot easier for them to see it. I’m getting a lot of ideas here, because I’m teaching an econometrics class this spring, and many of the things you’ve mentioned are things that my students have trouble with.

Rebecca: As someone who just learned some statistics this January, I’m thinking this could have been really helpful. [LAUGHTER]

Matt: Yeah, and so when I look at what I’m doing, I’m really happy to be using simulations. But as I look at the universe of simulations that are out there, I can see that there’s more that I can do, and that I’m really motivated to do after seeing some of these wonderful things. I shouldn’t get too far without talking about some of the wonderful things that are being done on a grand scale with simulations in introductory statistics courses. There are actually textbooks out there, which are built around these. And what they’ll do is they’ll start off early in a semester using simulations, and without giving names to things like sampling distributions and confidence intervals and P values. But they’ll take some real world data. And then they’ll say, what’s the model that you would use to best describe these data and then run some randomization samples to collect data. And then ask, “How likely is it that the original data were from that distribution?” And so that’s a powerful thing because a person doesn’t need to know any statistics coming into that class and being able to make meaning out of a lot of those things. There are multiple textbooks that use this simulation-based inference testing process to great effect and in the links that are going to be associated with this podcast, you’ll be able to go and find those, and just see the rich resources that they have supporting those texts, which actually can be used independently as well for reinforcing specific points that a person might have about their own statistics class or econometrics class. Another thing that I think is useful to point out is the fact that there’s a document that helps guide all of this. And the American Statistical Association has got the Guidelines for Assessment and Instruction in Statistics Education. This is called the GAISE guidelines. And they were revised in 2016. And they provide some very implementable recommendations for improving introductory statistics classes. And some of them are very consistent with what we’re talking about today, increasing the use of technology and the use of simulations, and decreasing that distance between abstract concepts and these students’ real worlds.

Rebecca: Can you talk a little bit about how to get started in implementing these sorts of things into your classes? So if you’ve never used simulations before, how do you start?

Matt: Well, I think, for me, the best thing to do would be to reach out to colleagues who might be doing that. So, for example, here at NAU, colleagues in the mathematics and statistics department are using these. So I would go to them. And I would say, “What have you found most useful, and how might I implement that in the class, given the context I have?” But, you can also do some wonderful internet searches. And I think I’ve curated a few really good starting places for you, in the resources attached to this podcast, I would recommend, for example, just seeing what’s out there by looking at these lists of applets that exists to teach this and to teach this and to teach this. And if you are thinking about something that’s on a grander scale, listen to the video by Nathan Tintall and Beth Chance, about how they implement this simulation-based inference testing in their classes and the rewards they have from doing that. I think getting a real broad sense of what’s available early can be really helpful in figuring out how you might want to do this. But I think that the nice thing is that you don’t have to do it on a grand scale, to start. You can use a single applet to reinforce a point that you might find your students struggling with. There are some other resources. There are journals on teaching statistics that are very, very useful. And I think this cross-pollenization between mathematics and the psych stats classes, is really useful. So I think it’s helpful to get this strong situational awareness of what others are doing to help inform how you might do what you’re doing better.

Rebecca: I think this idea of “one small step” is always a great approach to trying something new, and it seems very manageable. So I can try one simulation and see how it goes, and then feel confident to implement more and more. But that kind of iterative approach seems really helpful. Of course, not everyone has eight sections that they can iterate through all at once. [LAUGHTER]

Matt: Yeah, even if you’re doing it once. I mean, think about the environment that you have. You’re explaining these findings that are very visual in nature, and how all these things are wired together. And I think one of the most important things the faculty contribute to students’ education is not just the facts, but how these things are all linked, and being able to hear from a seasoned faculty member can help develop student’s ability to think these things through in a more expert way, rather than just memorizing simple facts. So I think that not only do we get some sense of accomplishment in putting those things out there for students to use, but the students do as well, because they want to get this too, and they’re much more enthusiastic about content when they think they’re really getting it, or they know they’re really getting it.

John: I think anyone who has been teaching for any length of time knows where some of the pinch points are, the things that students always have trouble grasping. And those would be a good place to start, not just in statistics, but more broadly, in any discipline, where there’s some concepts that students don’t always make the connection between theory and practice or practical application. So those would be the places I would think where people should get started… thinking back on where students are having trouble making connections, because it’s generally the same areas year after year after year. And that information could be used to help us improve our instruction by using tools that make it easier for students to see those connections.

Matt: Those are all good points. And you know, one of the things that was evident when I was looking into this more deeply was the frequency of which these simulations are being used in AP statistics and earlier. And so it’s much more likely now that we’re gonna see students coming into our classes who are somewhat familiar with this way of presenting information. And so they’re going to get it pretty quickly. And so a nice way to make them feel more at home might be to put these things in and, again, to leverage their learning, give them this feeling of self efficacy, that’s going to be really helpful to them as they get into more difficult concepts.

Rebecca: How have you adapted your instructional approach during COVID-19 and teaching remotely?

Matt: That is a great question. And one of the things that helped was that this online stats lab that we’ve put together over the years really made it so that these labs were kind of ready to go. So, in that respect, the materials had been developed as had many that people had put together at the end of the spring 2020 semester. Those were just in the bank and ready to be used in the fall and are even stronger now. There are lots of models being used throughout the country, for dealing with COVID-19 and instruction. So maybe I’ll just drag the one that we’re using so that listeners can get a better sense of how it all fits together. We’re using a modification of the HyFlex system, which is called NAUFlex. And we started using that really in the fall of 2020. As is the case, I think, for many, after spring break of 2020, people went into mostly completely online mode. And so the NAUFlex system starts off with teaching being done completely online. And that allows students to get on campus and be tested and all of the things that build that strong safe infrastructure. And then somewhat later, students are able to opt in if they choose to participate in in-person classes. Now, the online classes that are held are synchronous, so there’s an expectation that students will be there for those. And then we also have COVID adjusted room capacities. And so what that means for some classes, is that they have two groups or three groups of students who can come in, so that we can maintain that distancing. My experience has been that most students have opted to stay online, which means that they show up for the lectures or the labs in either Zoom, or what I use is BB Collaborate, which is built into our learning management system BB Learn. And so that’s how it works for us. And kind of the unsung heroes in this whole evolution have been the instructional designers who helped make this work, and also the folks from Information Technology Services who found hardware that works, that allows us to both interact with our students in the classroom and push it out there to students who may be in places that have varying degrees of connectivity. What I’ve done to modify my instruction, somewhat based on feedback I’ve gotten from students from the spring and fall semesters: what they liked, what they didn’t, how it worked for them, and try to really bring to the lab, this sense of organization and consistency and safety. One of the things, as educators, we’re used to doing is walking into a classroom and being able to gauge energy levels and look around the room and be able to tell who’s got that faraway look, and maybe we need to go back and regroup and cover some material. And some of those cues aren’t there anymore. And so what I’ve found is that I’m much more elaborate in my explanations so that I don’t leave anybody behind. And I tried to foster an environment which makes everybody feel comfortable asking questions when they want. And it’s very rewarding for me when they do. And I know that, in a class of 30, if a student asks a question, there are several others who probably have the same one. So the other thing that I think, and I know that this is something shared by your listeners too, is just that the notion of teaching with compassion, these students are really out of their academic element, if you will, of the in-person classes and going from one place to another. That sequencing is no longer there. And so it’s a much different world in which they need to learn. And some of them learn very well in an online environment and some don’t, but they’re forced into that anyway. And so I tried to have lots of compassion for what the students are going through and try to extend that in my syllabi for late assignments and things like that. And I think I’m a little more careful with humor, because I don’t know the backgrounds of all the students. I have not had that experience with them in the classroom. And so I’m really careful about how I express things so that everybody can get it, and it’ll be something that everybody can accept and understand. The nice thing about the labs is that we have all of these resources that we can use. And so it’s designed to be delivered in an online environment completely. And so students have interactive tutorials they can go to that help them master the content, complete the assignments. As an aside, one of the things that I found helpful in this communication, is that I wear a clear face mask when I’m teaching. Part of it is because I appreciate that myself. I’ve got some hearing loss. And so I’m a little bit reliant on reading lips. And in the classroom, students need cues that “this is important,” or “I’m trying to be funny here.” And I think that it helps the students understand the content and my commitment to their success when they can see my face better. That’s a little thing, but I thought I’d put it out there. I’ve gotten feedback from students that they found that helpful. And the other thing is that having taught this lab so many times, you mentioned pinch points before, I’ve got an idea of where those are going to occur. And so when we’re coming up on one of those, I can be more explanatory, give them a much better foundation for getting past those. So those are the things that I’ve changed.

John: One nice thing that may come out of this whole difficult teaching experience this past fall, is that I think all faculty have learned to be much more inclusive in their teaching approaches for all the reasons that you mentioned. And I’m hoping that that’s something that will continue as we move past the pandemic.

Matt: I agree with that completely. I think that there are really important initiatives to promote that in every classroom. But I do think that the situation which we find ourselves in now does encourage us or motivate us to do a better job with that. And so that’s one of the things that I would throw out as well is this whole idea of universal design for learning, something that I think is really important, and simulations play into that nicely, don’t they? …in that they provide this other way of representing information, content that students can get, particularly those that the students can work on themselves time after time after time until they feel like they really get it. And so I think that this universal design for learning thing is something that we should probably keep in the forefront as well. And some systems really do a nice job with making that easy for us. So for example, there’s an add on to BB learn that takes PDF files, for example, and creates those in alternative formats.

John: Ally.

Matt: And so you’re familiar with that. And for some students, it’s the only way to get that content. For others, it’s a convenient way to listen to content while they might be on the bus or in the car. And so I just think with these simulations, it just feeds nicely into what I think is a mandate to try to make things available in as many ways as possible, so they can really resonate with students.

Rebecca: Do you have any other tips related to simulations that you want to share with folks who might be teaching similar kinds of courses?

Matt: Well, that’s a great question. And while I’ve talked about simulations, one of the things that might be on the border of that, but I think is very useful for incorporating into classes, are some games. And in the face-to-face labs, I used to really enjoy doing like Stats Jeopardy and things like that. It’s a little bit more difficult to do in an online environment. But one of the things that I’ve done in the correlation module is to use a system put together by John Marden. He’s a Professor Emeritus at the Department of Statistics at the University of Illinois in Urbana Champaign. And he’s got this nice little system where he provides students with panels: four scatter plots and four correlation coefficients, and they need to match those. And so what I’ve done in previous semesters, and look forward to doing again, is having a competition across all the sections to see who has the longest sequence of correct panels, the winner of that gets a copy of a book by Tyler Vigan called Spurious Correlations, which if you haven’t seen it, his definitely worth a look. And there’s a website online as well, which is kind of fun. And one of the things that I’ve noticed with this particular gamified module is that students really work hard to get it. And at the end, they do, there are heroic efforts to win that book. And at the end, they really do know how to look at a scatterplot and get an idea of what the correlation coefficient might be.

John: For people who might want to go a little beyond using simulations in class, do you have any suggestions on where they might go to learn how to use, say, Shiny in R in order to create their own simulations? Is there a good reference out there?

Matt: I think there are some good references out there. They’re not, I think, specific to building simulations for teaching psychology. Although I have to say that one of the links that I’ll provide following the podcast will take you to an array of Shiny apps that were built specifically for teaching introductory statistics. And here’s the thing, they were built by undergraduate and graduate students for that express purpose. So this is a beautiful selection that were student built. But I think people who start working in R will look at some of the blogs that are out there and start being able to put these together themselves. But again, I think with all of these things, it would be starting off simple and going from there. Some of the ones that you’ll see out there are incredibly elaborate, and I know that they’re not in my skill set to build at the moment. So I would start with simple and go from there. But in the meantime, take a look at some of the other ones that are out there either to implement directly or try to emulate.

John: We always end by asking what’s next?

Matt: Well, for me, what’s next is a nice organized, gradual wind down of 2020. I think all of us are looking forward to 2021. I mentioned how grateful I am to have the opportunity to talk with you today. In preparation for this, I did lots of looking at things that are out there and I’m just really re-inspired to find simulations to put into my lab wherever I can. And also, as I’ve mentioned, planning on maybe building some assignments into my undergraduate teaching apprentice class about how they can use this. But I think I’m missing the contact with students in the online environment and in the lab, and I’m looking forward to being back in the classroom and using some of these things. But, I think, immediately what’s next, maybe another cup of lemon ginger tea.

Rebecca: Sounds like a good way to spend an afternoon.

John: This has been fascinating, and I’m looking forward to doing more of this during the spring myself. Thank you.

Matt: You’re very welcome. Thanks, John. Thanks, Rebecca. It’s a real pleasure to be here today.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

167. Supporting Persistence

Some students thrive in online courses and some students struggle. In this episode, Dr. Becky Cottrell joins us discuss the impact of student characteristics and circumstances on their success in online courses. We also discuss strategies that we can employ in our online classes to help all of our students be more successful. Becky is the online and hybrid course development analyst in the social work department at Metropolitan State University of Denver.

Show Notes

  • Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). Chicago: University of Chicago Press.
  • Tinto, V. (1997). Classrooms as communities: Exploring the educational character of student persistence. The Journal of Higher Education, 68(6), 599–623.

Transcript

John: Some students thrive in online courses and some students struggle. In this episode, we discuss the impact of student characteristics and circumstances on their success in online courses. We also examine strategies that we can employ in our online classes to help all of our students be more successful.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Becky Cottrell. Becky is the online and hybrid course development analyst in the social work department at Metropolitan State University of Denver. Welcome, Becky.

Becky: Thanks for having me.

Rebecca: Today’s teas are:

Becky: I’m drinking water today.

John: And I am drinking ginger peach green tea.

Rebecca: And I’ve gotten seasonal with my Christmas tea today.

John: I’ve got to bring that back. I’ve got a lot of it up in the office, along with some cinnamon sticks.

Rebecca: I beat you, John, I beat you this time. [LAUGHTER]

John: I saw your presentation at the OLC Accelerate conference, where you were talking about the research you’ve done on student outcomes in online and face-to-face classes at an Hispanic serving institution. Could you give us an overview of what prompted your interest in the topic, first?

Becky: Absolutely. I have been teaching online for more than six years. And I started working with a number of colleagues who really didn’t think that you could teach Spanish online. And I took that as a challenge and really wanted to teach a really great online Spanish class. And from there, it got me wondering who is taking online classes? I noticed a really big difference between my face-to-face students and my online students. And I wanted to know more about who they were and how they were doing in those classes. And combining that with the fact that we have seen an increase in student enrollments in online classes at our institution and around the country over the last many years, even before COVID, it really seemed important to me to know how students are doing in their online classes and what their grades are and what their outcomes were.

John: And that research becomes even more important when we put it in the context of COVID with the rapid shift online. Many people who were avoiding online instruction like the plague, have suddenly been forced to change their teaching modality.

Rebecca: …due to the plague. [LAUGHTER]

John: So, we can no longer say “avoiding it like the plague” anymore.

Becky: And students are complaining now and you hear students who don’t want to pay Harvard tuition rates for a substandard educational experience in an online class. But, are those experiences really substandard? I really want to know that.

Rebecca: That’s definitely a great question and a really relevant one right now.

John: So, this was your dissertation research?

Becky: It was. So, I just finished my PhD in Curriculum and Instruction. So I did a lot of research about what are student outcomes and what do they look like with different types of curriculum?

Rebecca: Can you talk a little bit about where your study was done?

Becky: Absolutely. So we use a pseudonym for the site. So, Russell University. It’s an urban university in the Mountain West and a very non-traditional population. So, lots of older students, lots of first generation students, veterans, working students, more students who are married… helping raise families. So, not your typical just-out-of-high-school students. It’s an Hispanic serving institution, and has been for the last few years.

John: How large was the sample that you worked with?

Becky: I started looking at every class that had online and face-to-face enrollments over two academic years, and at a large institution that ended up with 156,000 total course enrollments. But the statistical method that I was using doesn’t let one student be in the treatment group and the control group. So we had to aggregate students. And so I aggregated them down. There ended up being 28,000 students in the study. And from there, I just wanted to look at the ones who were taking mostly online classes, or mostly face-to-face classes. So those who were in that top 25% or bottom 25%, in terms of online enrollment, ended up being 7765 students over the course of two years.

John: That’s a nice sized sample. In many institutions, you have some students who are only online students, some students are only face-to-face. It sounds like there was a bit of a continuum there.

Becky: Certainly there were some who were all online or all face-to-face. It wasn’t something that I specifically looked at in my study, so I can’t pull out specific numbers of that. But yes, we definitely had students in the study who were entirely online and entirely face-to-face.

John: In terms of the online classes, were they developed with the assistance of instructional designers?

Becky: That’s a really interesting question. And the answer basically, is I have no idea. It wasn’t one of the things that I looked at in the study, I was looking more at student characteristics than course characteristics. That said, Russell University has a really robust online offering. Over the last 20 years, they have increased their online course offerings a great deal, and particularly in the last five years have really ramped up their efforts to develop courses and have really excellent quality matters certified courses at the university. That doesn’t mean that all of our courses meet that standard. But it has been an institutional goal and one of the things that they’ve worked on. but I was just looking at student demographics when I was looking at the study. Partly that’s hard because we have students who are taking maybe 20 different classes, and so they could have had one or two that were developed through an instructional designer, but the others may not have been. So, no real way of knowing.

John: The outcome you were looking at specifically was student success in the course?

Becky: Yes, so I measured student success in two different ways. The first way was looking at student grades, which we measured by course GPAs that was aggregated based on their course enrollments. And the other one was withdrawal rate. So, what was their percentage of withdrawals during the courses that they were taking during the two-year sample?

John: One of the things I found really interesting about your study is that you use a methodology that took into account sample selection in a way that so many education studies don’t. And you suggested the reason for that, I think, when you said that your online students were quite a bit different than your face-to-face students. Could we talk a little bit about that issue of sample selection in studies of this nature?

Becky: Absolutely. This is a really common problem in educational research, that you have something called selection bias. And I think that those of us who teach are aware that our students who enroll in 8 am classes are really different than the students who enroll in 2 pm classes. And we see some of those similar things with online classes versus face-to-face classes. It’s just a really different group and personality of those students. And what happens is students get to sign up for their own classes. There’s nobody randomly controlling them into different classes. They pick the ones that they want with the teachers that they want at the times that they want and in the course modality that they want. And we don’t know why. So that’s part of what I wanted to look at in this research is: what students are enrolling in online classes and what students are enrolling in face-to-face and why? Is there a balance between the groups? Are they really similar? Or are they really different? And so what I found was that there are different students who are enrolling in online classes versus face-to-face classes, which is not unexpected. As an example here, we found that students who are working full time were more likely to take online classes, which makes sense, they need to take the online classes because it fits better with their schedule and has greater flexibility to match their work schedule. But at the same time, what impact does that have on course outcomes? Does it mean that they are really motivated because they have a full-time job, so they’re going to get better course grades? Or does it mean that they are working full time and they’re managing a family and if something comes up, they’re going to put their schoolwork to the side because other things are more important. So selection bias, and the way that students self selected to classes, really changes how they might perform in those classes. Which brings us to that question of are those student course outcomes based on the online course modality? Or are they based on the characteristics that made students choose the online course modality?

John: When you didn’t control for student characteristics, what did you find in terms of comparing the outcomes in online classes with face-to-face classes?

Becky: One of the things that was really interesting here is that those students who were taking 75%, or more online classes actually had significantly better grades in their online classes than they did in face-to-face classes. So the online course GPA for those students taking 75% or more online classes was 2.55. And for those taking face-to-face classes was only 2.34. So definitely a significant difference and higher grades in online classes, which is not what I was expecting. Then, with regard to withdrawal rates, we had totally different results, which is that there was no significant difference in withdrawal rates among the two groups before balancing for those 15 different student characteristics.

Rebecca: Can you talk a little bit about what those 15 characteristics were and how you chose those?

Becky: Absolutely. I used Tinto’s student integration model to look at what characteristics he felt contributed to student success and persistence in the institution. So, I ended up with 15, different personal characteristics related to students. So, a lot of demographic characteristics: age, race, gender, those sorts of issues. We tried to get academic performance through GPA, transfer status, transfer GPA, ACT scores, SAT scores, those sorts of things. We also tried to determine institutional commitment through if they had a declared major. And the one area that we would have liked to have more, but wasn’t available in an institutional data set, was something related to like computer literacy and other skills that were related to performance in an online class, but it just wasn’t something that was available. So 15 different characteristics, including those demographics, academics, and just connection to the institution.

John: So you were using a nearest neighbor matching with, I believe, a two-to-one ratio?

Becky: Yeah.

John: Could you describe that, perhaps, for our listeners?

Becky: Absolutely.

Rebecca: …for people like me that have no idea what that even means? [LAUGHTER]

Becky: So the methodology that I used was kind of an interesting statistical method called the propensity score analysis. And basically what a propensity score analysis does is matches people who are in the treatment group with people who are in the control group. So it creates kind of an artificial match to say this is now one person and what would have happened if they’d been in treatment or if they’d been in control. So it takes all of those characteristics and assigns them a score, and from there can divvy them up and say they are likely to be in treatment or control and it recreates those groups. And that matching allows them to determine the probability of them being in treatment or control groups, which essentially controls for the characteristics that you’ve loaded into the model.

John: To simplify it a bit, you’re comparing people who are similar in characteristics and examining the outcomes when adjusting for those characteristics.?

Becky: That is a great explanation… very concise. And the idea of the nearest neighbor two-to-one matching is basically that for each person who’s in the online class, we found two matching people in the control group. So we tried to keep as many students as possible in the final outcome.

John: And there have been at least some studies that are found one-to-one or two-to-one gives you the best estimates with the least amount of bias from that procedure..

Becky: Absolutely, yes. When there’s a one-to-one match, you get a lot better balance, because you can obviously find a matching student in the online or the face-to-face class that is the best fit. But when you start matching more students, it’s not quite as good of a fit, so you don’t deal with balance quite as well. And speaking of balance, I’m going to jump in and tell you about this right now, just because I think that’s interesting, and one of the great parts about propensity scores is this idea that the first thing that a propensity score model does is say, “Are these groups the same? Are your online groups the same as the face-to-face group?” And what we found out is that they aren’t. And I thought this was a really interesting piece of my research. So they were totally different, different enrollment patterns. and there were about eight characteristics that were significantly different. And this is where I think it’s so fascinating. So we had more part-time students in the online classes… not surprising… but they had higher ACT scores, more transfer students, more credits taken, they were more experienced students, they had higher GPAs, they were more likely to have a declared major and they were all older. So the better students were taking online classes, which is so fascinating to me, and explains ultimately, why we had higher course grades in our baseline data. Students who are better students were taking online classes, where those beginning students who were younger, who had less experience, were taking the face-to-face classes. So I just thought that was fascinating, that it was imbalanced. But it really gave a good picture as to why we were getting the outcomes we were at the institution.

Rebecca: It’ll be interesting to have some follow up studies related to COVID-19 around those ideas, because just anecdotally, students who are newer to being online, or just newer college students, have struggled quite a bit with online learning or complained about it, or just don’t know how to manage their time and those kinds of things. And it seems related to the kinds of findings that you’ve had.

Becky: Absolutely. And I think across the country, we’re seeing that those upperclassmen stay enrolled and are succeeding through these COVID transition. But it’s the underclassmen who are taking a gap year or who are failing out of classes. So I think that these results speak to that, that those students maybe aren’t prepared for an online class,

John: What happened to your results in terms of student success, when you corrected for the sample selection?

Becky: This is so fascinating. After controlling for that balance, we had originally had, in our baseline data, better scores, better course grades in online classes, and after controlling for those characteristics, there was no significant difference in course grades between online and face-to-face courses, which is awesome, it’s really exciting to know that maybe we’re doing something right. And so that was really exciting. But, at the same time, our baseline data had said that there was a non-significant difference in withdrawal rates. But after controlling, we found that there was a significant increase in withdrawal rates, and online classes had higher withdrawal rates, by about 2%, than face-to-face classes.

John: I think that’s a fairly common result, that online students often have much higher withdrawal rates than face-to-face classes.

Becky: Right. The grades are really promising. And I’m glad to know that those course outcomes are doing well. But when we start looking at withdrawal rates, it brings up some really interesting questions about how are we engaging students and why do we have bigger withdrawal rates in those online classes.

Rebecca: I was just going to ask if your research led you to believe anything about those results? If it was this particular characteristic or a teaching method? Or are those just new questions that we need to continue asking? [LAUGHTER]

Becky: I think they are mostly new questions that we need to continue asking. But there are some implications in the literature that I think lead us to some possibilities here. One of the big ones is that sense of community and connection in online classes, students really want to feel that, and if they don’t, they’re more likely to drop out from those classes. And so it’s definitely a consideration as we’re looking at more online classes is how are we building community? And how are we engaging with our students in that online space to make sure that they’re able to connect with their instructor and connect with other students in the class? I think that another factor that we see is who are taking these online classes: so students who are more engaged with families, they’re older, they’re working full time, therefore taking fewer classes. I think that those factors can contribute to their persistence or not in these online spaces. So, definitely some of those issues are there and we know what some of those reasons are. And I would love to do some future follow up research on what really is happening at this particular institution.

Rebecca: I know you had also mentioned high-impact practices and trying to incorporate more of those, like inviting students to do research and things. I’m wondering if we have any data on how prevalent that might actually be in online learning compared to face-to-face learning. How often are those opportunities actually there?

Becky: I totally agree. It would be so interesting to look at what are those impacts? And what is the prevalence of those high-impact practices? I think there’s a lot of research about what we can do to do better. And I think that even from this research that for my dissertation was almost obsolete by the time I defended my dissertation, because COVID happened, but one of the things that we can be doing better, and I think we have started is providing greater access to student services in those online spaces that students maybe before didn’t have access to advising, registration… they didn’t have a good way to connect with people who are on campus. And I think so many of our institutions have had to move towards a much better practice with that. When we went online for months, they had to figure out how to do that. And I think that we’ll keep that around and providing better services to students. And that will definitely help keep them enrolled in classes and keep them from stopping out and persisting at the institution.

Rebecca: Nothing like a pandemic to really force some innovation, right? [LAUGHTER]

Becky: It’s true, but it’s been so much fun. I love seeing that innovation and how we’re benefiting our students. I also love seeing a little more attention towards online teaching, We were the ugly stepchild before and now everyone is excited to learn about this new thing and how they can do it better.

John: It’s gone from being an ugly stepchild to a savior in some way.

Becky: Yeah, absolutely. Think about the last pandemic with the Spanish flu. What happened to their education at that point? We didn’t have online learning. Did they have distance education? What even happened with that?

John: If this has happened 20 years ago, it would have been a completely different experience with a lot of colleges just completely shutting down or moving to some type of correspondence class instruction.

Rebecca: Which I don’t think would have gone well. [LAUGHTER]

John: Which would not have gone very well.

Becky: No, definitely 20 years ago, I think that right now we can say we have similar course outcomes in online and face-to-face classes. But 20 years ago, I would have been one of those students who was protesting at Harvard about paying tuition for a substandard educational experience, [LAUGHTER]

John: What are some of the things that you would recommend doing to help build class community?

Becky: I’m so glad that you asked about this, because this is one of the other personal interests that I have. I’ve been working with a faculty learning community for the last two and a half years around developing instructor presence in an online class. And so I love talking about this, I think that there are a lot of ways that we can really develop connections among instructors and students, and also among students. So one of the best practices that I’ve seen is making sure that teachers have an opportunity to connect one-on-one with their students, whether that’s sending out an email a time or two during the semester, or requiring students to meet with them, at the beginning of the semester or at midterms, throughout the semester, to be able to develop that one-on-one Zoom connection to just be able to have a little bit of face time with students. But I think that works really well. So making sure that there is an opportunity to connect on a human level. When we teach online, we tend to be really text heavy and dry. And taking that human element that we love in a face-to-face class and pulling it out in an online space is so valuable for students, and really helps them to connect with each other and with their instructor. It’s one of those inclusive teaching practices that we do really well face-to-face, but is a little bit harder to do online, and if we’re intentional about it, it can happen. In terms of developing community among students, I think that as much as there’s resistance towards group work, I think that you can intentionally use it to develop community in your classes. And this isn’t just a “Hey, you should write a paper together and divide up the work,” it’s intentionally using that as a community building opportunity. And letting students know that that’s your intention is you want that to be community building. So one of the things I’ve always done in my Spanish classes is have students meet in small conversation groups once a week to have conversation practice with each other. And there’s always a little bit of resistance, and students aren’t so sure that they want to do it. But I have them fill out a survey to let me know what time they’re available. And it’s just a group of three students. They meet every week, and they have a great time talking with each other and get that oral communication practice they need. It also ends up being one of their favorite parts of the class. They develop connections with other students. And I hear all the time about students who actually meet in person and go out for coffee. I had one student who was taking a class from Florida and another student who was in Denver, and the Denver student had to go to Florida for something and stopped and went to go visit the Florida student in person, they went and hung out together. So I think there are just really interesting human personal connections that can be made. And leaving space for that to happen is so important. I think we get too focused on academics and lose those moments at the beginning or the end of a class where we spend a few minutes talking about nothing or the weather or the football game last weekend. And leaving that space in an online class and making sure that you have some space for that, really helps to develop those connections.

Rebecca: I definitely have experienced that this semester with my students who have had persistent groups all semester. They have said multiple times how helpful that has been for them, and they just did a reflection activity and almost every single student said “Oh, being in those groups was the best part…” which we never hear about group work, right? [LAUGHTER]. But they got to know each other and they had support through the class and used that as a way to help each other out with the course material.

Becky: Absolutely. I love that. It’s so amazing when students can get that connection and really work together.

John: I had a similar experience in my online class where I had students work on podcasts. And the first time they met, generally, is when they met in small groups to have these conversations and recorded them using Zoom. And they were supposed to be 5- to 10-minute podcasts, but many of them ended up being dramatically longer because, essentially, they were getting to know each other. It was kind of nice to see that sort of engagement and that interaction where they were getting to form this community. It would have been nice if they had recorded just a shorter segment of it. But I did get to listen in on some of those initial meetings. And it was an interesting experience.

Becky: And I agree, I signed my students to only speak for 30 minutes, and they only had to record 15 minutes of that. But the timer would tell me how long they’d been in and many of them would be in there for 45 minutes to an hour, sometimes an hour and a half… that they would just spend that time together, practicing and talking. And it was great. It was just fun to see that connection, that they went above and beyond what we’d asked them to do.

Rebecca: So drop out rates for something that you mentioned that your research pointed to this was one of the biggest issues that we needed to be thinking about in terms of online education. So in addition to instructor presence and helping students formulate community, do you have any other recommendations for faculty or instructors to help mitigate that or get students to stay? …to retain students?

Becky: Absolutely. So we’ve talked about access to student support services, building a community, some of those high-impact practices that we don’t always think about in online spaces is making sure that students have the ability to collaborate with faculty, like on a research project, especially at a Hispanic serving institution. It’s a culture where those connections are really important. And making sure to provide those to students so that there’s an opportunity to connect with faculty on working on something meaningful is really important. So as faculty, we can make sure that we’re selecting students, when we’re thinking about TAs, research assistants, make sure that we’re thinking about some of our online students as well and see if that might be a good fit for them. And one of the things that I also think about in terms of improving retention is this connection and relationship between the faculty and the student is so important. But in order to do that, we know our faculty are overworked and underpaid, and to make sure that there’s institutional support for faculty, is really important. And so making sure that there’s access to instructional design and pedagogical training through some of the resources available at the institution is a big deal, making sure that there is a collaborative opportunity for faculty to work together and share best practices and generally just supporting faculty. As we hold on to faculty, it gives them more bandwidth to hold on to their students. So institutional support is a really big deal to benefit our students as well.

Rebecca: And one that we can’t underscore enough when faculty are feeling really strained. [LAUGHTER]

Becky: No, absolutely not, not in 2020. And here we are. I don’t know about other institutions, but we’re being furloughed. And so we’re asked to do more and have fewer resources.

John: …while being at further risk in terms of employment risk, as well as all the health risks out there.

Becky: Oh, there’s so much going on.

John: You mentioned forming connections between faculty and students, and one way of certainly selecting students to be TAs, and so forth. But, what are some of the things instructors can do in their courses to help form those connections within online classes,

Becky: One of the things that we’ve really found that is helpful is moving away from a really static discussion board. We see a lot of classes where instructors say, ‘Tell me three things that you learned from this reading,” or “What are three of the five methods that are used to do whatever it is”. And those are really boring discussion boards and do not foster community, but asking questions that really encourage students to engage in a debate, in a conversation, and teaching them how to engage with each other appropriately and respectfully in an online space is really important. So asking them to solve problems together, asking them to work together, not shying away from difficult conversations. This election year has had a lot of challenges, and engaging with those in a student class in a way that allows them to bring in their own unique perspectives helps them to connect. Some of that might be through a discussion board. Some of it might be through a tool like Flipgrid that allows you to have students have a video discussion where they get to record a short video and then reply to each other. That really fosters that sense of connection and community in an online space. So allowing for that to happen is really important. We can move away from a boring discussion board to either a better discussion board or some of these other tools that foster community.

John: Flipgrid or VoiceThread or other similar tools offer a lot more possibilities for connection and hearing each other’s voices and hearing their instructor’s voice I think should help to create that sense of community more so than just reading text on a screen.

Becky: …and videos also. That, if we are recording videos, we can see the instructor, we can see the other students… having a face to put to a name. And having just a little bit of personal information… knowing that I smile and laugh, and I am an engaging person, I think, helps to connect with the course.

John: Humanizing the instructor is a phrase that’s often used, letting them hear you, hear your voice and your sense of humor, letting people know you as a person rather than just as the author of these words that show up on the screen all over the place is helpful.

Rebecca: …and humanizing the other students in the class. If it’s just a name, it’s really easy to not really think of that name as a person, the more you see and hear, not only as an instructor, but also fellow students, I think, can be really beneficial. So I think that students eat up the media when it’s available to them.

Becky: Absolutely.

John: And helping them make connections to their own life in their discussion. If they’re going to have discussion boards, one way of doing it effectively might be to have them make connections, where they draw on what they’re learning and make connections from their own life and experiences and share them, which also is a nice way of forming that sense of human presence in the classroom.

Becky: Absolutely. With a PhD in curriculum, I feel like I hold in my two hands two different things. So on the one hand, I have the curriculum and the course objectives and the aligned assessments and all of those things, and I think they’re so important. In my other hand, I’m holding on to the importance of people like bell hooks and Paulo Freire, and that reminder that we need to be transgressing some of these lines of our existing education and decolonizing our educational experience and humanizing it to make sure that we’re making real personal connections with the content, with the instructor. And so those are the two things that I carry with me as I’m working in my own classes in this and I’m helping faculty develop their courses is, “How do you balance those two things?” That is so hard, and I think in online classes, we do really well with the alignment and the course objectives and the assessments. And sometimes that humanizing part feels like it falls by the wayside.

John: But they’re not necessarily substitutes, they could be complementary. If you design assignments well, where they’re engaging in these authentic interactions, while achieving the learning objectives, it’s more work trying to design that, but there are some things you can do that can work fairly well.

Becky: I think there are wonderful faculty out there who are doing really great things, those are just the two things I try to always carry with me to make sure that I don’t leave one of them behind.

Rebecca: I think it’s really important to think about those two. So, it’s a nice reminder. And I think actually a nice way to wrap up the conversation, because it’s the two things to keep in mind as you move forward. Having those little takeaways at the end is always helpful. So we always wrap up by asking what’s next?

Becky: For me, I am really excited to dig into some of this qualitative side of things that we’ve talked about today. As I said, I love that hard quantitative research, but I’m also really interested in the humanizing element of it and that instructor presence. So I’ve been working with this faculty learning community for the last two and a half years, and we have developed an online instructor presence self-evaluation tool that we are presenting at OLC in the spring. So we’re really excited to be able to share that with some other people about how you connect with people and how we engage in our classes. So we’re excited to move forward with some of that. And just see what is happening with COVID? How has that changed things? And how might we rethink how we’re teaching online?

John: It’s just something that people would be using on a longitudinal basis to track how their classes evolved? Or is it just used in general as an instrument to share with faculty?

Becky: What we’ve intended it as is a way for people to self assess. So we didn’t want it to be a rubric. We don’t want it to be point based. We wanted it to be conversational, and a way to go in and reflect on your own teaching and consider ways that you could improve. And so absolutely, the way that we’ve designed the tool is it has a “What are my strengths? and ”What could be improved?” area on each of it. And so it would be really interesting to come back and say, you know, I did this last semester, what does that look like this semester? What am I changing? How am I improving? S o I think it absolutely could be used longitudinally.

Rebecca: That tool that you’re talking about sounds really great. So I hope we can have you back so we can talk about that in the future.

Becky: I would love to… only if I can invite a part of our faculty learning community

Rebecca: Of course.

Becky: It was a group effort. It’s one of those things that we couldn’t have done it without each other. We’ve just been in each other’s support system. And when we first found out that our institution was going online, we had a meeting scheduled for that Friday, and we talked about canceling and everyone’s like, “No, these are the people that I need.” And so we all met that Friday that we were moving online, and we haven’t seen each other since in person, but we were just that group. We’re like, “No, I need my support group.” So, I would come back and talk about it, but only if I can bring my FLC with me.

Rebecca: [LAUGHTER] It sounds important to do so. Well, thank you so much for joining us today. It’s been a great conversation and we look forward to hearing more research from you, Becky.

Becky: Well, awesome. Thanks so much. It’s been a pleasure to visit with both of you.

John: Thanks for joining us. We’re looking forward to talking to you again.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

158. Distracted

It is easy to become distracted when materials or experiences seem irrelevant, unobtainable, or uninteresting. In this episode, James Lang joins us to explore strategies to build and strengthen student attention to improve learning outcomes. James is a professor of English and the Director of the D’Amour Center for Teaching Excellence at Assumption University and is also the editor of the West Virginia University Press series,Teaching and Learning in Higher Education, and the author of numerous articles and books on teaching and learning, including Small Teaching: Everyday Lessons from the Science of Learning and Teaching and Distracted: Why Students Can’t Focus and What You Can Do About It.

Show Notes

Transcript

John: It is easy to become distracted when materials or experiences seem irrelevant, unobtainable, or uninteresting. In this episode, we explore strategies to build and strengthen student attention to improve learning outcomes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is James Lang. James is a professor of English and the Director of the D’Amour Center for Teaching Excellence at Assumption University and is also the editor of the West Virginia University Press series,Teaching and Learning in Higher Education, and the author of numerous articles and books on teaching and learning, including Small Teaching: Everyday Lessons from the Science of Learning and Teaching and Distracted: Why Students Can’t Focus and What You Can Do About It. Welcome, Jim.

Jim: Thank you. Thanks for having me.

John: Good to see you again.

Jim: Yes.

John: Our teas today are:

Jim: I’m actually a tea aficionado. I get my tea from David’s Teas, which is a Canadian company. They, I think, have suffered a lot during the pandemic and closed most of their stores, but they still have a great online presence. And my favorite is Nepal Black.

Rebecca: Oh, that sounds good.

Jim: Yeah, it’s a great black tea. And I have many David’s Teas, though. [LAUGHTER]

Rebecca: I almost forgot about David’s Teas. I need to cycle back to that.

Jim: Yeah, it’s great stuff.

Rebecca: I’m on my last cup of a big pot of English Breakfast tea.

Jim: I love English breakfast. I love Earl Grey. You know, all the greens. I just love tea.

Rebecca: You’re in the right place, then.

Jim: [LAUGHTER] Yeah, exactly.

John: And, you may remember the collection of teas we had in our workshop.

Jim: Oh, I totally remember. Yes, that was like tea Nirvana in your center.

John: It’s sitting there kind of empty right now. But, we’re hoping we’ll be back there soon.

Jim: Yeah.

Rebecca: The collection of teas is lonely. [LAUGHTER]

John: Although every now and then some get pilfered from the office. And I’m drinking one of them right now. A blueberry green tea.

Rebecca: That sounds good.

Jim: Yeah.

John: We’ve invited you here to discuss your upcoming book, Distracted: Why Students Can’t Focus and What You Can Do About It, which I’m really looking forward to receiving when it comes out in October, I believe. Perhaps we could start by talking about the role of attention in learning. Why should we focus so much on attention?

Jim: So, in the book, I argue that we need to think about attention as actually the kind of foundational step for all learning; no learning happens without attention. So, I actually think it’s a value that we need to be more willing to kind of fold into our pedagogical thinking. If you look at the research on how people learn, almost all of it will tell you that the first thing that has to happen is the learner has to attend to whatever the content might be. And I also believe that it’s important for us to make attention a value in the way we form community in the classroom. We should be attending not only to the course content, but to one another. So, we’ve talked a lot in recent decades about the importance of having a learning community in the classroom, about having relationships between us and our students, and the students having relationships with one another. All of those things depend upon the attention that we pay to one another. So, to me, there’s a kind of cognitive aspect to this. But, there’s also a kind of ethical aspect to it. We owe that attention to one another, we need to be able to pay attention to the students and to the specific students in our room and not just sort of a generic idea of a student, we want students to listen to one another. When students are airing their ideas in the classroom, we as teachers want to be able to listen to them, but we want students to be able to listen to them as well. So, I think we really do need to pay more attention to attention in our pedagogical thinking. So, that’s kind of what the book is about. The kind of overarching point of it really is to get away from this thinking that attention is sort of the norm or that this is something we can just take for granted in the classroom. And that we should expect students just to be able to sit and pay attention, because that’s the normal modus operandi in the classroom. And instead, to recognize that attention is an achievement. It’s something that we have to work at. And as a result, faculty members have to think about how do they support student attention in the classroom. How are they deliberately cultivating it? And how are they deliberately sustaining it, both to the classroom content and to the other human beings in the room?

Rebecca: Like other kind of pedagogical approaches, it seems like talking about attention with your students might be a good thing to start off the semester, and explain what attention actually is. Do you have any recommendations for thinking through that with students?

Jim: Absolutely. There are great resources out there that can help us educate our students about attention and about distraction. And we have to start with those kinds of conversations about how we make the classroom a place where attention is a primary value. And again, this doesn’t mean like attention, where it’s just sort of me laser focused in on the teacher and being attentive like that for 50 or 75 minutes. We’re not built that way. That’s not how attention works. But we want to do our best to kind of continually renew the attention that we pay to one another. And I think that has to start with an explicit conversation with our students. And to say that “Look, you know, it’s important for me to hear your ideas. So, when you come in here, if you’re doing other things, then the contributions that you would make to this classroom, which I know are important, are going to get lost. And when your fellow students are speaking, I want us all to be paying attention and listening to what that student has to say.” So, I think we have to start with those kinds of conversations. And maybe not in the first day… I think there’s a lot we can do on the first day to try to engage students and set the tone for the course… but, sometime in that first week, to really have a conversation with students to say it’s important for us to pay attention to one another in the classroom. Here are the guidelines I’ve developed t do that and I welcome your input on those guidelines. And then, by the end of this week, we’re going to come to an agreement on these are the rules that we all will follow together in order to make sure that we are fulfilling our obligations to one another, in terms of building a community and paying attention to one another, and paying attention to one other’s ideas.

Rebecca: If we build the value of attention into our course, what does that look like,over the course of the semester? We talked a little bit about a discussion, setting some boundaries or some rules, but, how does that play out over the course of the semester?

Jim: Well, so two things. So, first of all, I do argue in the book, actually, that I think there’s value in having an explicit kind of guideline for how we will deal with both attention and distraction in the classroom. And that includes what we’re going to do with our technologies, but it’s not limited to that, and to develop some explicit guidelines that are shared with students that they’re invited to comment on that, then they actually will sign and say, you know, “I agree to sort of abide by this policy,” and then to revisit it, to come back to it in the middle of the semester, for example, at a midterm evaluations and say, “How are we doing with the guidelines? Do we need to update these? Or do we feel like everyone is kind of on board or are people slipping away? What can I do to help get everyone back and make sure that we’re still paying attention to one other? …because attention fatigues over time, that happens in an individual class session, but also happens over the semester, right? So, we’re going to get to a point of this semester, at which we’re all tired, we’re finding it harder and harder to pay attention to one another because there’s lots of stuff going on, and for the students, all their midterms and getting toward the end of the semester. So, it needs to be addressed initially, and it needs to be revisited. Now, from the teacher’s side, there’s a lot of things that we can do to kind of say, “Look, I’m doing everything I can to help support your attention in the classroom here.” And all those are kind of explicit pedagogical practices that we can take. And in the book, I talk about two creative ways of thinking about this, to think like a playwright and to think like a poet playwrights have long experiences of trying to guide people through experiences that unfold over time. So, a playwright has to think about “how do I maintain the attention of an audience for an hour, two, or three hours, sitting in a dark room, where the audience is supposed to be looking just at this stage and following the story?” How do they do that? They vary the structure, right? There are acts and scenes and intermissions, there’s rising and falling action, there are stories unfolding. Not only that, but like you go to the symphony or whatever, right? It’s the same thing, you’re going to have movements, you’re going to have pauses in the action, you’re going to have a movement that ends quietly, but then begins with a bang. The people that have had to think about “how do I pull the attention of an audience over time? …we can learn a lot from that. So, I think teachers need to think a little bit more like that, to think about the classroom experience as something that unfolds over time, and therefore needs to have a structure and variety to it. Right? So, that, essentially, I argue in the book for thinking about your classroom experiences, as kind of a modular one, where you’re going to have an opening activity that takes 10 minutes, and then there’s going to be something that goes on for 20 minutes, and then you’ll have a finishing thing. And not only to make those changes, because change renews attention, right? We know that from the research, change can renew attention. So, you have the changes. But then you also have the fact that these things are different. So, that like I’m doing something passive, like I’m listening to a mini lecture, but then I stop and do something. And then maybe I get that another passive experience. So, that’s the first thing is to think a little bit more like how we’re varying the structure of the classroom experience. And by thinking like a poet, what I mean by that is that one of the things that poetry and literature can do for us, it helps us see the world anew, right? Like, it takes everyday experiences and objects and things that we’re familiar with, and it shows them to us in a new light. So, we wake up to them or say, “Wow, like, I never thought about a peach like that, right? Like, that’s amazing.” There’s this incredibly beautiful and complex thing” or like a still life painting is trying to do the same thing for us, right? …to show the world back to us, in all its wonders and complexities and intrigue. And I think we need to do that as well. We need to think about like, what are the opportunities that we have to show students the amazing, wondrous, mysterious aspects of our discipline that can awaken their attention to what we’re trying to teach. So, in the book, I argue for a what I call signature attention activities, which might be something that you would do, you know, once a day, once a week, a few times a semester, but that are really kind of like creative pedagogical things that get students re-energized and re-engaged. And recognizing, like this everyday thing they might be experiencing, actually is an incredible, amazing thing that deserves their interest and engagement. So, thinking like a playwright, thinking like a poet… to me, those are two kinds of ways to try and develop new approaches to cultivating and sustaining student attention.

John: So, in terms of thinking like a playwright, would it make sense to break up each class period into a narrative or into a storyline where you have those modules that you talk about, but perhaps do something at the beginning to activate attention to provoke curiosity?

Jim: Absolutely. I mean, there’s lots of things that you can do, I think, at the beginning to kind of get them engaged. You can tell a great story, you can pose a problem or a question, but you have to do something other than just kind of “Okay, here we go. Here are the four concepts that we’re going to talk about today.” I think, if you really want people’s attention. Here you can expand it to other creative arts as well. When you pick up a novel, The first two pages, a novelist knows, they’ve got to draw you in in those first two pages, you’re going to put the book down, right? A television show, think about how many television shows, films, they begin with something that really is designed to capture your attention and draw you in and keep you engaged for the rest of that experience. We’re drawn to stories, we’re drawn to questions and problems. But, if we can think about foregrounding those, that’s a way of getting us engaged before we then go through and are doing the sort of harder cognitive work of whatever that classroom might be.

Rebecca: You mentioned some signature pedagogies to implement throughout the semester to focus our attention. In the spirit of small teaching, is there one that’s small and easy that faculty who maybe are under stress during a semester can implement right now.

Jim: The example I give it, the book… I’ll start with the one that kind of originally got me thinking about this. There was a faculty member actually across town for me at Holy Cross, an art historian, who since passed away. But she had her students go to the Worcester Art History Museum, and every week, they had to go to the museum and look at the same painting and write a different one- to two-page essay about that same painting over the course of the entire semester. They do 13 short essays about the same painting. And that, to me, is a great example of creative thinking about like, this is how you make attention a value. You know, you start and you look at it in a very surface oriented way. And then you just have to keep looking and looking and looking. And the more you look, the deeper you get into it. And the more you start to see all the sort of incredible stuff in there. So, I kind of encourage people to think about what is the thing in your discipline that’s like that painting that like you can go back to, or that you can develop some kind of strategy that’s going to get students to see it anew for the first time. So, one that is a little bit more kind of every day, I observed a theologian on my campus, who had her students engaged in an activity that was modeled on study of the Torah, the scholars studying the Torah use, which is she had the students get in pairs. And I was able to observe this class, they sat across from one another. And they were instructed to read out loud to each other the first few paragraphs of Genesis, but after every sentence, they were supposed to stop and say, “Okay, what do I see here?” Like, “What does this remind me of? What word is strange here? What do I notice here that connects to other things that we’ve talked about in the class?” And this went on for like 20 minutes. And some people only got like two paragraphs in like a 20-minute exercise of doing this. But, it was incredible to listen to what they came up with. And I stayed in the class and listened to some students afterwards. One student said, “I’m from an evangelical background, I’ve read these passages so many times, but I’ve never thought about some of the things that we talked about today.” And so it was a way to kind of reawaken them to something that was very familiar and that she could have got up there and given a lecture on things in the first book of Genesis, but the students uncovered it themselves, and were able to do that. So, I actually kind of talked through a process that was developed by someone at the teaching center at Brown University, which is trying to model like, very close looking at something in your discipline. And you start by just sort of doing that, “what is it?” Like, “What is here? Let’s really get in and describe it as much as possible.” And then the second thing we do is we say, “Okay, so what? Why is it important? What matters about it? What does it connect to in terms of other things that we know or are learning?” And the last thing is sort of “Where can we go from here?” Like, what questions does this raise that we can then go and think further about, or for example, that I might go and write a paper about or do some research about?” So, the careful look at it, then the thinking about how it connects outside of that thing to other things? And then the “Okay, now let’s go further, I’m going to develop my own kind of way of thinking about and understanding this thing.” You know, John Dewey, a long time ago, had students doing object analysis, where they would analyze like everyday things in their homes, or like that they encounter on an everyday basis, and trace those everyday things: “Who made it? What is the production of it, say about like, our economy and our world?” And you know, you can do that with anything, and almost any discipline, right? Like this t-shirt I’m wearing, right? Who made that t-shirt? That has huge implications for, like economics and politics and trade and inviting that kind of activity into the classroom seems to me like something that can help students see the discipline in a new way, and then re-engage their attention to show them this course actually has relevance and connections to things outside of the box of this classroom.

John: When in the classroom, one possible source of distraction, which I know you’ve written about before in the Chronicle and other places, is mobile devices. When we’re in a classroom environment, how can we help students use their mobile devices more effectively.

Jim: So, I think we have to be explicit about them. So, when we have those conversations at the beginning of the semester, I actually recommend in the book, an open source PowerPoint presentation that anyone can get and use. It was developed by a psychology instructor at the University of Toronto, which kind of shows students some of the issues that we face when we’re using our devices in the classroom. And of course, when students are using their devices off task in the classroom, it impacts their own learning, of course. We all know that. But, the bigger challenge is the way that impacts the students around them. And there is some pretty good research that shows that if a student is off task on a device, other students are drawn to that device, and that steals their attention away from whatever might be going on in the classroom. So, I think we have to talk to students about that, We have to say, “Look, you know, your device use is not just a personal choice that you’re making that has no broader implications. It does have broader implications, it has the potential to kind of tamp down the overall level of attention in this classroom.” And again, I think when we make that appeal, we need to do it on community grounds, right? Like, we owe each other our attention in this space. And we are all going to benefit from people’s contributions and those contributions are going to be richer, if we’re paying attention to one another, if we’re thinking together about the ideas. So, I’m not in favor, actually, of sort of full technology bans. I’m also not in favor of saying we should never have a technology ban. I argue in the book for a context-driven policy, which suggests that there may be times when we say no one needs their devices, right now, we’re going to talk about what this means. And you don’t need to take notes on that by hand or by device. There are other times when I may be lecturing, and you can use your devices, or you can take your notes by hand. There are times when we’re going to be having a discussion, you know, you can write down something if you’re so moved, but otherwise, I’d rather have us focus on one another here. We’re gonna be doing an activity and everyone’s going to go to the board, so you don’t need your devices for that. I’m segmenting off sections of the board here, and I want everyone to brainstorm a list of these five things. To me, it doesn’t make a whole lot of sense to say, we’re never gonna use technology in here, or there’s just an open technology policy, which we can use any time. It depends on what’s going on. And I think if we take that approach, that also helps us be better planners, because we have to think about “Okay, well, what is going to be happening in the first 20 minutes, and would that benefit from technology use? Or couldn’t some students benefit from that?” If so, okay, then great, I’m gonna be explicit about that. But, there may be these other times where it’s gonna do nothing but interfere. And at those times, I want to be able to say, you don’t need your device right now.

Rebecca: I think that makes perfect sense. Right now, I’m teaching synchronously online. So, I’m exploring some different ways of using technology and different ways that pure distraction might play out in a screen environment. Do you have any thoughts about how we can help students attend to each other more so in an online environment? Sometimes it’s a little more obvious, I think, in a physical environment of how to set things up, and maybe not as obvious in an online environment.

Jim: It’s definitely not as obvious whether people are distracted in their online environments, right? Because they can have their phone right next to the laptop. And I’m sure we have all done this in our zoom meetings, department meetings, or whatever committee meetings where things get a little slow, you pop over and you do something else for a little while, and then you come back in. And again, I think, even an online class, we can be explicit about that, when you’re stopping out like that you’re pausing your own thinking, and that’s going to lead to a sort of a less rich conversation for us all. When students have their cameras on, it’s a little bit easier to see obvious sources of distractions. But, of course, I think we do need to give students the option to not have those cameras on for a variety of reasons. To me, because I think about, as I’ve been doing workshops …and a lot of the faculty workshops that I do on other campuses, of course, have switched virtual… what I’ve seen a lot is that the people will actively engage with the chat room. So, when the chat room is there and is explicitly encouraged, that can be a way that keeps people engaged. In some ways, it’s not quite ideal, because people can also get off track in the chat room. I’ve definitely seen that happen as well. But, trying to find regular ways to make sure that people are engaged in parallel activities, or something that’s kind of supporting whatever it is that’s going on. You can still use polling, you can use chat rooms, you can use breakout rooms, you just have to think about the same thing that you think about in the classroom. How am I continuing to provide sort of variety and shifting from one kind of activity to the next. People always talk about like, well, you lose students attention over the course of a 50- or 75-minute lecture, you lose people’s attention over 50- or 75-minute discussion too. Anything that you do for a long period of time, your attention is going to fatigue. So, to me, there is no like one pedagogical technique online or in face to face that’s like this is going to keep people’s attention, guaranteed, for everyone in the room for this amount of time. That’s just not realistic expectations. So, we just have to think about how we are providing that kind of variety, giving people opportunities to actively engage. And kind of what I encourage people to do is what I did during the two years while I was researching the book, and what I’ve been doing over the last six months in my online environments, is just look at like when do people pay attention? When do people get off track? We can learn from those moments. That’s what I’m essentially trying to argue to faculty as well. And what I hope the book will do is get people together on campuses and say, “Okay, let’s just think about this collectively. When do our students drift off? And like, why is that happening? When do our students get really engaged? Why is that happening? And how can we take the sort of engagement moments and maximize what’s happening there, and take the distraction moments and use those as an opportunity to develop creative new approaches?” So, for the online classes, I just encourage people to think about what have their experiences been in your Zoom meetings and your webinars and things that you’ve done when you’re a participant. What’s helped you, and what’s brought you back, and what sent you away? I’ll just say one last thing about it. To me, in the Zoom context, or like a synchronous online class is one of the lines of research I follow in the book, is the use of names, we all perk up at the use of our names. So, if John was drifting right now, and I said, “John, what do you think?”, he’s gonna “What?” If he was drifting, he’s suddenly back in the room, right? So like, even if you don’t have cameras on, you can still be saying, “I’m regularly gonna invite people to post in the chat or to see if they have comments. And I’ll do that by calling your names.” Just even saying that it’s going to get people “Okay, you know what, I need to be kind of attentive here.” But once I actually say, “Hey, Rebecca, what do you think? You know, boom, you are like right there. So, there are simple things like that, that we can do that help. There’s no sure fire solution online or face to face, we just have to keep trying these different approaches.

Rebecca: One of the things I really love is the fact that there are names.

Jim: Yes, I know. Right? Exactly.

Rebecca: That’s amazing. [LAUGHTER]

Jim: Yeah, the first couple weeks in class, it’s hard to do that sometimes. Right? Because you’re still learning everyone’s names.

Rebecca: Yeah. What I noticed about what you were just saying, though, is using this object-based learning or close-look approach on ourselves or within the teaching arena. So this area that we want to study teaching…

Jim: Right.

Rebecca: …you’re actually offering up the suggestion that we do the same thing that we’re suggesting that our students should do within our disciplines.

Jim: Yeah, that’s true, actually, you’re taking a look at the classroom, but through this other lens. We look at it through all kinds of lenses. But, I think if you look at it through the lens of attention and distraction, to me, that’s like an avenue toward creative new thinking. And that’s kind of ultimately what we want here. This is basically the same approach I tried to take in Cheating Lessons, which is to look at like the issue of academic integrity, where does it happen and why? And then say, “Okay, once we understand that, what can we do differently? And how can we use that problem to improve education in general?” And that’s kind of what I’m trying to do here with attention to distraction? How can we use the problem of distraction to help us become better teachers in general?

John: Does your book also address issues of how we can help our students maintain focused attention when they were engaged in out-of-class activities?

Jim: This is a really challenging issue. So, one of the things I’d hoped to find in doing the research for the book was that strategies that people have touted as improving our general attentional capacities, that there are some of those that work. And the truth is, there doesn’t seem to be as much of that as we would like, especially evidence-based strategies that can sort of improve people’s attentional capacities. So, the one that’s been the most thoroughly researched in education in recent years has been mindfulness. So, if we practice mindfulness, to what extent can that actually improve our ability to pay attention? And there is some research that supports that, but it supports it if you are really all in on it, like you’ve got to be doing mindfulness on a daily basis for a significant chunk of time. And you’ve got to be really willing to make that commitment to mindfulness. When you do that, it can help. But we don’t have the ability to do that with our students, for the most part. And most of the experiments that you see being done in this area are like three- to five-minute little mindfulness activities in the classroom. I’m a fan of those, I think those can be really great and helping in the moment. We can help sort of in acute… like we can improve our attentional spins in an acute way. But, in terms of like developing strategies that are going to help students actually improve their attentional capacities in the long term, and outside of the classroom, I’m not sure we have anything yet that has proven to do that. Well, we do have one thing, but again, it’s nothing we should be doing in the classroom, it’s physical exercise, like physical activity improves your blood flow to your brain. And that improves all kinds of your cognitive functioning. But again, we can tell our students to do that, but it’s not like something we can enforce or get our students to do in the classroom. But, I look at some of the research in a great book on distraction called The Distracted Mind by Adam Gazaley and Larry Rosen, and they do a pretty good job of looking at like brain games and drugs and mindfulness and nature exposure. And their conclusion is, so far, we really only know one thing that is evidence based to improve people’s cognitive control, and it’s physical exercise. Everything else, we’re still not sure yet, like we’re exploring. There may be some positive studies here, but, we don’t really have enough to make it prescriptive yet.

Rebecca: Seems to me like something that could be useful to students outside of classes, just having them be aware of attention.

Jim: Absolutely.

Rebecca: …and what being attentive looks like, so that they can self monitor, if they so wish.

Jim: Absolutely. We can give them the sort of tools and instruction they need, and we can give ourselves the same. As the result of doing all this research, I’ve kind of realized that, in my own work life, there are things that I can do with my email and my Twitter feed open, like responding to emails and doing sort of committee work, that kind of stuff. But, if I want to write, I have to close everything out. And you know, since the whole pandemic thing, the weather has been better, I got in my backyard, I close everything out, and I just have Word open, and I do that for 45 minutes and then I get myself a 15-minute break, right? I take a walk around, I look at Twitter, that kind of thing. So, we need to do the same kind of look at our own attentional patterns and like habits and distractions. And we can encourage students to do that. We can help them understand how to do that. It’s up to them ultimately, of course, to decide whether or not they’re going to put those ideas into practice. We can also model for it in the classroom, though, and that’s another reason why I argue that there may be times when it’s a good idea to say to students “All devices away at this point for 20 minutes here, we are going to just brainstorm. We are going to think with nothing but our brains and the book or the problem, or whatever it might be, and the whiteboards, and let’s just try to come up with something. One of the things I suggest in the book is that devices and distractions are around us all the time. That’s our normal way of being. And we want to be able to prepare students for that world. That’s why I argue that we shouldn’t ban technology. We’re gonna be working with technology, like we need to know how to work productively with it. At the same time, it may be that there’s good reason to think that the classroom sometimes is an escape from all that, that the classroom is like an attention retreat, where we can go, put away all that stuff, and use our brains in a different way. And it may be that the more technology sort of intersects with our lives on an everyday, 24-hour, basis, that those spaces are really valuable, actually, and that they give students a taste of what it’s like to put things away and just focus our collective brains on something and see what emerges from that. And if we can give them the opportunity to do that in the classroom, then they may recognize, “Oh, you know, actually, this was really valuable. And there may be times when I want to do it myself outside of the classroom with a few peers, or even just by myself.

Rebecca: I certainly have had students in the past have experienced really stressful times, say like, they’re all in on a particular class or something, because it’s an escape, and it’s a place where they can focus and they put all their attention there. And I think a lot of students are doing that right now, during the pandemic, as well. I have a lot of students that are really focused right now on some of their schoolwork, because they’re stressed by other things that are going on around them.

Jim: Yeah, my last Chronicle column was a review of a book called Lost in Thought, by Zena Hitz. And one of the things she argues and that is that we need to recapture the value of just sort of getting lost in our own thoughts and engaging with ideas and the great thinkers and problems of the past and present. And part of the argument she makes is that when we do that, we have an opportunity to get away from our material circumstances, right? …like the world that we’re living in. And that kind of escape can be really valuable. It’s valuable, both for sort of mental health purposes, like, you know, you step away, and you get to sort of engage with something fascinating and intriguing, and get into kind of like a flow state or a thinking state. But, it’s also valuable, because it can give you a new perspective, like, that’s the moment which you might come up with, like a really creative idea. And I bet almost everybody listening to this podcast right now has had moments where they’re like in the shower, on a walk in the woods, riding their bike, whatever, and something suddenly hits them, and then a problem that they’ve been wrestling with opens up. What’s going on there is you are away from the other stuff. In those moments, that’s where the ideas sometimes emerge. So again, sometimes ideas emerge because you’re online, and you’re seeing all kinds of different stuff. And that’s great. But we want to have these other opportunities as well. And so the classroom should be able to provide a little bit of that for students as well.

Rebecca: I found that some students also respond really well to hearing examples from us of our experiences with attention or lack of focus and how we’ve wrestled with those things. I know that this morning, my class was talking about being tired or having anxiety, and I just expressed that I was also experiencing that as well. And all of a sudden, like we were all in the same place, we were all attentive to each other because we had this kind of common experience.

Jim: Yeah, one of the other major points I hope people take away from the book is just empathy, to recognize that attention is hard. And it’s especially hard in a time like this, when there’s so much going on in the world around us. When we have the pandemic, we’ve got an upcoming election, we’ve got Black Lives Matter. We have all kinds of things that are making us concerned or unhappy or frustrated or anxious. And so those things steal away our attention. And we have to be empathetic to ourselves. First, we have to recognize that our own attention is suffering right now. And then we have to bring that empathy to our students as well. A student who’s drifting away in the classroom. Sure, that can be because they’re looking at their Instagram, but maybe they’re looking at their Instagram, because they’re so stressed out. And this is kind of an easy thing that they do that gives them a quick little relief from everything else that they’re worrying about. Or maybe they’re drifting away in the classroom because they had a terrible night’s sleep, and they’re up with a sick relative. I mean, attention is drawn away, not just by our devices, but by all kinds of things. The more that we recognize that and the more that we are empathetic with our students, the more we can work with them to develop solutions.

John: You mentioned the importance of attention by both students and by faculty. We’ve talked mostly about students’ attention. Do you have any suggestions for faculty on how we can be more effective in maintaining attention to our students and their needs at any given time.

Jim: It’s just the basic stuff that we all think about in terms of the responsibilities that we have to build community in the classroom are essentially the ones I’m arguing for in the book as well. Names are important, knowing individual names, I argue in the book also for an activity like values affirmations in which students get to tell you what matters to them at the beginning of the semester. So, we can do our icebreaker activities in which hometown major, you know, all that kind of stuff. But, to get more substantive, and to get to know the students a little bit better. Invite them to tell you what matters to them what they’re good at, and to be able to kind of then keep those things in your mind and use them in the conversations that you have with students or in the feedback that you give to students. Giving individual feedback, thinking about how we’re doing that, using students’ names, and knowing a little bit about the assets that they bring into the classroom, I think there’s been a lot of good research on the ways that we can help foster community in the classroom. And to me, those are the things that are going to help foster attention as well. Attention is reciprocal. If I pay attention to you, you’re more likely to pay attention to me. If we’re sitting at a coffee shop together, and we’re there to meet and discuss something, and you pick up your phone, that’s the moment in which I’m going to pick up my phone as well. Whereas if neither of us does that, if none of us makes that initial move, we’re probably more likely to continue the conversation with one another and pay attention to each other. So, when our attention is drawn away from the students, when we’re not giving them our full attention, they’re not going to give us their full attention either.

John: Is there anything else from your book that you’d like to share with our listeners,

Jim: The only other thing that I talk about in the book that might be worth mentioning is the role that assessment can play in attention. And I do believe there is a role for assessment to play in supporting attention. And what I argue here is that your great students are going to try to pay attention to everything that happens in class. Your students that are struggling, that may have a harder time managing their academic work, those students actually can benefit from assessments which help them recognize this is a moment where I really should be paying attention in this class. And if an assessment is well designed, and it’s going to promote learning, then I think we’re only doing them a favor by helping students recognize “This is an important thing here, this matters.” And that can be low stakes. But, even low stakes can get some students over the threshold of “I’m going to sit here and check out” or “I don’t feel like I know what I’m doing here. So, I’m not going to say anything, I’m gonna hide in the back.” The students saying “Okay, actually, this counts a little bit. So, I better try and trying is going to help them.” So like, I don’t think there’s anything wrong with thinking about the role that our assessments can play in pointing students toward the activities that are going to help them learn. So, I argue for that in the book as well, that assessments do have a role to play in this process.

John: And it’s not just low-stakes assessments, I’ve been amazed at how much attention and enjoyment students get out of using things like Kahoot!, which is entirely anonymous, but just that feedback they’re getting on how well they’re doing and that somewhat competitive atmosphere with it, where there’s no harm if they make mistakes, but they become really excited about how they do on those.

Jim: Yeah, “I want to see if I got it, right,” like “I’m trying this, I want to see if I got it right.” Because that’s gonna tell me how I’m gonna do in the class. And so, those kinds of activities, I think, can be really helpful for engaging attention.

John: And it’s giving the students feedback, but also giving us feedback. So, we know where they’re struggling, so we can help address those needs.

Jim: Exactly.

Rebecca: I think projects are also another form of assessment that we didn’t discuss right here. But, I think even having small amounts of scaffolded projects where there’s something that like, is done and accomplished, and you can check it off, is another way of kind of feeling accomplishment, but also being aware that you’re focusing on the things that you’re supposed to be focusing on, to move forward in a larger scale project.

Jim: Exactly, and like a lot of this stuff, that scaffolding is good for all kinds of reasons. And one of those reasons is, as we just said, like I can go through, I can check it off, I know that this is important. So, I have to get it done before I can do the next thing. That’s going to keep their attention engaged throughout that process of doing a larger project.

John: And it reduces cognitive load…

Jim: Yeah.

John: …it reduces the amount of anxiety they have, and they’re getting guidance along the way. So, they don’t go off in a direction that it’s hard to recover from later.

Jim: Right. And anxiety and cognitive load are all connected with attention, [LAUGHTER] like anxiety, it steals our attention. When the cognitive load is too heavy, we lose our attention. So, all these things. You know, attention is like one of these things that, once you to start really thinking about it, it intersects with everything. [LAUGHTER]

Rebecca: That’s why it should be a value. [LAUGHTER]

Jim: Exactly. That’s what I’m saying. Yeah.

John: Yeah, well, I hope this gets the attention of a lot of our listeners [LAUGHTER]

Jim: Well done.

John: …so they can focus more attention.

Rebecca: We always end by asking, what’s next?

Jim: Well, I’m on sabbatical. So, I am writing a book. And for the first time, I’m kind of going back to my discipline. I’ve been doing sort of off-and-on research on George Orwell for a long time. My area Is 20th century and contemporary British literatures. So, I am using this sabbatical as an opportunity to try and get that book project going. And I hope to be able to have a book. or at least a good chunk of a book, by the end of my sabbatical. There also is a second edition of Small Teaching that we’re working on. And so that will be out at the end of 2021. So, that’s a second edition, which will have updated research, some updated recommendations for techniques, and, actually, there is going to be a chapter on building community. So they’ll be an additional chapter. And so I’m excited for that as well.

Rebecca: That’s like a lot of things to look forward to.

Jim: Yeah.

John: And living in the Orwellian world we’re in right, now… [LAUGHTER] I’m very much looking forward to that.

Jim: Yeah, definitely. There’s definitely a lot of relevance there. And that’s why I hope the book will get some attention. [LAUGHTER]

Rebecca: Nicely done. Thanks so much for joining us and sharing your expertise. And I know that I’m definitely looking forward to picking up your recent book, and I’m sure many of our listeners will too.

Jim: All right. Thank you. Thanks for having me.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

156. Social Annotation

Do you struggle to get students to complete readings or to deeply discuss readings in an online environment? In this episode, Margaret Schmuhl joins us to discuss how a social annotation tool can engage students in conversations with the text and with each other about the text. Maggie is an Assistant Professor in the Department of Criminal Justice at the State University of New York at Oswego. Maggie has also been working with us as the facilitator for our second cohort of faculty in the ACUE program here at Oswego.

Show Notes

Transcript

John: Do you struggle to get students to complete readings or to deeply discuss readings in an online environment? In this episode, we discuss how a social annotation tool can engage students in conversations with the text and with each other about the text.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted byJohn Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Margaret Schmuhl, an Assistant Professor in the Department of Criminal Justice at the State University of New York at Oswego. Maggie has also been working with us as the facilitator for our second cohort of faculty in the ACUE program here at Oswego. Welcome back, Maggie.

Maggie: Hi. Good to be back.

John: Good to see you, Maggie. Our teas today are:

Maggie: Well, I’m having an orange spice herbal tea.

Rebecca: That sounds nice and warming.

Maggie: It is. It’s very cozy for a cooler fall day.

Rebecca: I have Scottish breakfast, which apparently is my new default tea.

John: I’m having a ginger peach green tea, which I’ve been having a lot recently, too. We’ve invited you here to talk a little bit about how you’ve introduced the social annotation tool, Hypothesis, in your classes this summer and this fall. Could you talk a little bit about what prompted you to adopt Hypothesis for your classes?

Maggie: Yeah, so a couple of reasons. First, in my spring semester classes, I like to think that my students are very open and honest with me, especially when I asked them if they’ve done the readings. And for the most part, I get a very resounding “Nope,” like, “…haven’t done them.” And I just take a deep breath and carry on with the class, knowing that none of them have done any of these readings. And so, after feeling a bit frustrated for a long time with my classes not actually completing the readings that I’ve carefully curated for the class, I was looking for something that could keep them accountable to those readings, I had had colleagues who would assign reading summaries and such, and that seemed great, but I wanted to be able to see something, I wanted to see how they were understanding the reading. And I think John, you actually inspired me to consider Hypothesis because you had used it in some of your classes. So, when we had been meeting with the ACUE cohort, it was super interesting to me. So, I took a workshop, I think through CELT, with the Hypothesis representative, and it seems like a super easy functional tool. And I really liked that it was embedded right into Blackboard. So, it wasn’t necessarily throwing a lot of new technology out at students. And they didn’t have to create accounts, they didn’t have to go to a third-party website to use the annotation tool. It was something that I could throw right into the module, and all they had to do was click on it and start writing. So, it’s simplicity was super accessible, I think, for my classes.

Rebecca: One of the things that I think is really interesting about Hypothesis as a tool is that, if you’re using it for accountability purposes, it ends up being more of a dialogue with the readings rather than what can be perceived as busy work of summaries, or some of these other things that either just feel annoying to do, or annoying to read as a faculty member. And the same thing can happen with quizzing, too. It’s another thing to grade or another thing to look at. Sometimes that can be really effective. But, it’s a nice different way of doing it. And I think it’s really enjoyable as a faculty member to see how students are looking at materials.

Maggie: Yes, absolutely. Because with reading summaries, there’s an easy way out for students just to like look for a summary on the reading. But when you really want them to start asking questions about the reading, this tool helps them be able to locate certain segments of the reading that they may not have understood or something they found particularly interesting, or were able to connect it back to other classes or other information that we’ve talked about in just a really fluid way. So, yeah, I absolutely agree. That’s one of the big benefits I’ve found with this tool.

John: Did this replace some earlier activity? Or was this a new activity that you introduced in your class?

Maggie: So, actually, I haven’t had it replace anything. This has been more of a tool I’ve used, in addition to discussion posts, and so forth. But now that I’ve had a couple of courses under my belt with this, I do think I’m going to move towards replacing discussion posts. The discussion posts, from students’ feedback, they see discussion posts as just answering the questions that their professors wants, whereas the annotation allows them to pull out things that are interesting to them. And they’re able to engage, they think, in a more natural way than it is on the discussion posts. So, they’re reading through each document and, along the way, they see what their classmates are writing and where’s discussion posts, you have to go back into each of their classmate’s forums to see what they have written. And it seems a little more, I guess, artificial in discussion posts to just kind of comment like,”Oh, I agree. Here’s what I also wrote.” And it seems like a much more casual way of interacting that’s more akin to what we have in the classroom when we are face to face.

John: Do you think it encourages deeper and closer reading of the texts?

Maggie: Oh, definitely. I think a lot of my students have given me the feedback that they’re not just skimming the text anymore. They’re not just looking for the main findings or the points to summarize, but they’re actually considering each part of the text. And as they’re considering each part of the text, they’re using this tool to communicate to me their interpretations of the readings, but also the ways it connects back to their own experiences. So yeah, I found it to be quite invaluable for that kind of engagement.

Rebecca: Do you see it as a way to facilitate or to encourage community building around the content?

Maggie: Yeah, so I think that when I do replace discussion posts, there will probably be a little bit more of that. But, I already see where students are using the reply function. So when they create an annotation, they have an option to reply to another classmate’s annotation. And so I see dialogues begin to unfold between three or four students, whereas in discussion posts, if I tell students: “Okay, engage with someone else on their work,” they’ll pick one person, and they’ll respond to them. But again, it almost becomes like a text message to each other. And in a way, it seems, I think, more natural for them to just quickly write back and forth in response to each other’s questions, as opposed to having something a little more drawn out in a discussion post.

John: So how have students reacted to the use of Hypothesis.

Maggie: For the most part, my students have really enjoyed Hypothesis. Of course, there are some students who find it to be a little tedious. But, for the most part, when I asked them, whether they prefer discussion posts or their annotations, most of them prefer the annotations. They felt like they wouldn’t have completed the readings in a systematic way if it weren’t for Hypothesis. So, they’ve pointed to this level of accountability that the tool gives them to those readings, they actually feel like they’re retaining more information from those readings because of the way that they’re engaging with it. When I have a synchronous session, and we are diving into some of the issues that these readings bring up, engagement in those discussions are much greater than they used to be. I used to feel like I had to tailor questions so if they did the reading, or they didn’t do the reading, they could still participate. But, now I feel like we can actually dive into some of the nuances of that text in a way that we just couldn’t do before, when they didn’t do the reading.

John: it’s a whole lot easier, I think, for students to actually read the text when they have to actively be in the text to do their comments. So, it’s a little more difficult for them to evade doing the readings.

Rebecca: One of the things along those same lines that came up in a reading group discussion that we were holding yesterday was the idea of accountability and faculty talking through the concerns that they had about students being held accountable for things and that they seemed less accountable, or that employers have said that recent graduates seem a little less accountable than they had previously. So, it’s interesting to be able to use some of these tools to encourage accountability. But also, I think, it mimics a more professional experience about how you might engage with materials professionally. And so maybe it just feels more authentic, and therefore it’s easier to be more accountable.

Maggie: Oh, I love that, because I do think that, at least in the context of our careers as academics, we use annotation tools like this all the time, whether it’s in Google Docs, and we’re making comments and we’re working with co-authors and other faculty members on different projects and such, I definitely see where we use those tools and those skills that it’s a good skill set to encourage students to build.

John: Since we have this integrated into Blackboard with an LTI, it’s possible to do grading in the LMS. Have you been grading students on their participation?

Maggie: Yeah, so when I first started using Hypothesis in the summer, I was grading them, but at the time, the grading wasn’t embedded right into the Hypothesis platform. And so I was grading on a separate rubric and grading them sort of apart from each other. But now that I’ve been able to use the grading function right within, it makes grading much easier, because I can simply click on the student’s name, all of their annotations, and all of the replies that they’ve given to other students will show up right there so I can review them, give them a grade, move on to the next students, and it automatically loads right into the grading center. And when I’ve talked to students, they actually, not so surprisingly, said that if it wasn’t graded, they probably wouldn’t have done some of those readings. So, it certainly made me feel better by including this as a graded portion of their final grade, because I think without that incentive, they may not have engaged with it. But, I will say that I require students to do a minimum of three annotations, and I’ve several students who are doing 7, 8, 10, 12, just depending on the reading and their topic of that reading. They seem to be willing to move above and beyond that minimum standard, which I think is pretty cool.

John: I’ve seen exactly the same thing, that even though I did have some minimum specified, most of the students were doing2 to 10 times as much as a minimum when they were using Hypothesis.

Rebecca: Perhaps that attests to, in both of your cases, of actually helping students establish a habit of how to read or you get in the habit of using that tool to read and then you’re reading the whole document anyway, so you just annotate the whole thing.

Maggie: Yeah. And I was afraid that, as students were going through the readings, they would basically stop at the first page and put all of their annotations right on that first page, but I haven’t looked at all of their submissions. We do annotations every week on a reading, and so I’d have to pull it all out and compile that data to see what kind of patterns emerged. But, it seems to me that they are doing these annotations throughout the entire reading, they’re not just going a couple pages in and then being finished with it. Of course, there are students who are like that, but when I’m scrolling through that document, and I get to page 17, there’s still annotations there, which I find encouraging.

Rebecca: Probably, once a couple of students do it, and start modeling that, that becomes the standard of behavior, then people realize, like, well, even I’m not gonna read the whole thing, you got to read parts of the thing. [LAUGHTER]

Maggie: Exactly, yeah. And I’ve had a lot of student feedback that they like seeing what their classmates are writing about, because it’s given them insight into their perspectives on the reading and how it connects to their lives and their experiences. And I think it allows for an engagement in an online platform that I typically tend to enjoy in a physical face-to-face classroom.

Rebecca: And reading can seem like a really lonely activity generally. And if it’s difficult reading, it can feel extra lonely, especially if it’s asynchronous. So it seems like a good way to connect people through reading, which is not a way we generally think about being social.

Maggie: Absolutely. I’m teaching a class on the death penalty this semester. And so there are some Supreme Court cases that they are reading, and they are 200 pages long. Now, I required them to read one opinion and one dissent from the respective justices that are writing those cases. But, with that, they’re not so scared with the 200 pages of reading. They’re not just like totally shutting down and not doing it. They’re still engaging with the material, which is more than I can say wa’s happening in the classroom when we were face to face.

Rebecca: Can you talk a little bit about your future plans in using Hypothesis moving forward? Yeah, so another way I’ve used Hypothesis is for peer review, and I know that John has used Hypothesis for peer review too, so I think he probably has some comments as well on this. But, in my classes that are writing intensive, I like to incorporate some kind of a peer review feedback, because not only are we requiring them to write their papers with peer-reviewed research articles, and so forth, but I want them to understand what that process means by engaging in that. And I think when some of that feedback comes from their peers, they start to feel like the feedback isn’t so over their head, that that feedback is something that they can accomplish, and something that they’re perhaps a little less afraid of, than when it comes directly from me. So, what I really like about Hypothesis is I can create all of their submissions into a PDF, I can assign each student to review a certain number of papers, I typically tend to assign each student a particular paper so that not one paper gets all of the annotations over another. And I give them a requirement of making at least 10 or 15 comments on the paper. And before I know it, it’s actually eased up a lot of work on my own feedback, because they’re catching the things that I now don’t have to spend a whole lot of time, telling them to capitalize a particular word or explaining how to use commas in a particular sentence. And so it’s been really nice, because they can simply highlight, they can make little comments about when a sentence doesn’t sound right to them. And then I can come back, overlay my feedback over atop of it. And then the students have all of that in one place when they go to work on their final drafts and incorporate that feedback. So yeah, as far as planning in the future, I do plan to continue using that as a peer review feedback as well as in my readings. I’m teaching online courses next spring as well, and so I plan to go get some good scans of my readings so that I can allow for them to become annotatable. Is that a word? annotatable?

Rebecca: It is now. [LAUGHTER]

John: I had a similar experience when my students did peer reviews, and they really liked that ability. They like that they could see the comments, they could react to each other. They could reply to each other’s comments and sometimes they’d disagree about whether a change should be made and there were some really good discussions embedded right in the text, right at the point where it was occurring. And then I’d come in, and sometimes I’d say, “Well, you know, I think maybe the original actually worked pretty well here” or something similar. And it did make my work quite a bit easier, because the students were doing a lot of the basic editing. Initially, a lot of the comments were primarily grammatical. But after the first time we did that, we talked about how it worked. And the students were saying it would be nicer if we could get more substantive comments, actually suggesting ways in which we could improve the substance of the paper. And I was going to suggest the same thing, but they brought that up themselves. And they seem to have much more of a sense of ownership of the review process. And that worked really well. Did your students have any concerns or negative reactions about the use of Hypothesis?

Maggie: Yeah, so I have found that the tool seems to be much better suited for my upper-division seminar style classes. I think that, even though I find it to be really useful for my introductory survey courses, the students did not like it as much in those introductory courses. But, it’s hard to know exactly why. Some of them pointed to not wanting to actually engage with other people, which I kind of have to laugh and move on from those comments, because that’s part of the process of these courses, is engaging with other people. But, I do wonder if it’s between discussion posts, and low-stakes quizzes, if adding annotations in a lower-division course becomes a bit overwhelming for them. But, I do think that the benefits of being accountable to those readings and having better discussions because of those readings probably outweighs some of that concern. But, I have had some other student feedback. They didn’t like that there wasn’t specific feedback available for the grading function. So, when you grade in Hypothesis, you just give a number grade, it doesn’t allow you to submit a rubric to indicate different levels of content or grammaticals or whatever it is you want to grade in a rubric form. So I did have some students who wished that there was some more specific feedback available for that. But, it did make me wonder, and it kind of reminded me of some of the reading we were doing in our reading group, the Small Teaching Online, when they were talking about specs grading, I thought that these annotations might be a really good place for that… to incorporate some all-or-nothing kind of grading. But again, with low-stakes grading, it’s not a significant portion of their grades. So I guess that’s just one thing to keep in mind, is that sometimes students want some of that detailed feedback. And that tool doesn’t necessarily give you a place to comment on their annotations, except within the annotations, you certainly can comment by replying to their annotations, which I do.

John: But you don’t want to make it public because of FERPA, and so forth. But you always have the option of not using the grading feature within Hypothesis and just adding a column to the gradebook, attaching a rubric to it, and then just evaluating each student… looking at their comments using the same technique, and then just going to the rubric and adding that to the gradebook. So, there are workarounds.

Maggie: Yeah, and I’ve done it that way as well. That just brought to mind like, maybe I need to go back to using that method for some of these classes. The other thing is that sometimes scans aren’t the best. I do think it’s really better to use articles that are already searchable. Sometimes when you’re scanning material, making them searchable and accessible, is difficult. There’s really good scanners and technology that can help us with that. But, sometimes the students are highlighting certain segments of the text, and it’s jumping to other areas of the paragraph. And so I think with that it takes a little bit of time to complete. I’ve also had some students saying that they don’t like to highlight over other students highlights, but I think that’s more of a personal preference. So I just encourage them to reply, then, to those students’ annotations so that it’s about the same material. And that pushes them to engage with each other a bit. But while there’s certainly some areas that students want different features and improvement on, they overall very much like using this tool in Blackboard… at least that’s been my experience.

John: I suppose one nice side effect of this is the more people who use this, the more it will encourage the creation of accessible PDFs because basically the issue is that you need a text layer that contains all the text where it’s supposed to be basically.

Rebecca: Yeah, and if it’s a fully tagged PDF, it works better in Hypothesis than just an OCR’d PDF, for sure.

Maggie: Yeah, that’s fair. I think it is a great tool for faculty because it really does push them to make all of their readings accessible. So, in terms of accessibility, it’s a good way to push everyone to make their materials accessible.

Rebecca: We always wrap up by asking what’s next?

Maggie: Well, so in terms of using Hypothesis, I’m teaching some upper-division seminar courses next semester online, and so I plan to keep using this for both peer review and for reading comprehension. I’m hoping also that one day, we’ll be able to use inclusive access texts with Hypothesis so that we can move through some of the main readings, especially if we have a textbook, where students are able to annotate together.

Rebecca: I would like to be able to annotate images.

Maggie: Right. Yeah.

Rebecca: Well, thanks so much, Maggie for joining us. It’s always a pleasure to chat.

Maggie: Yes. Thank you for having me.

John: Thank you, Maggie. It’s great talking to you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

151. Video Conferencing

Although video conferencing tools are not new, the global pandemic has resulted in a dramatic expansion in faculty use of this technology in their learning environments. In this episode, Rick McDonald joins us to discuss ways in which we can use these tools to create productive and engaging learning experiences for our students. Rick is an instructional designer at Northern Arizona University who has extensive consulting experience in higher education and in K-12.

Show Notes

Transcript

John: Although video conferencing tools are not new, the global pandemic has resulted in a dramatic expansion in faculty use of this technology in their learning environments. In this episode, we focus on ways in which we can use these tools to create productive and engaging learning experiences for our students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca:
Our guest today is Rick McDonald, an instructional designer at Northern Arizona University, who has extensive consulting experience in higher education and in K-12. Welcome, Rick.

Rick: Hello, how are you today?

Rebecca: Great, thanks!

John: Today’s teas are:

Rick: I am a coffee drinker myself, but at least this early in the morning tea is more later in the day for me. [LAUGHTER]

Rebecca: I have Irish breakfast tea today,

John: And I’m drinking ginger peach green tea. We came through a really challenging spring semester, where people suddenly had to move online, and we’ve gone through a really difficult summer. We want to talk a little bit about video conferencing. In general, I think everyone’s become familiar with some form of video conferencing software. Zoom has suddenly become known by pretty much all faculty, one way or another, but there’s Collaborate and other tools as well. How can faculty become more effective in using video conference tools?

Rick: Well, I think, to start, we can all just relax a little bit but teaching with the video conferencing doesn’t have to be tremendously different. There are a few things that are absolutely different, and a few things to just consider that aren’t really such huge problems. First of all, when we’re teaching on video conferencing, we really need to know the software. Some schools are using multiple kinds of software. And I would choose the one that you know best. I would, again, relax and keep a nice and slow pace when we’re teaching over video conferencing, sort of frenetic pace can be very difficult for the remote student to stay engaged with, and at the same time making the class engaging, just like you would in your regular classroom. So, when we’re teaching, we try and engage the students in the classroom. When we’re teaching with video conferencing, we need to find ways to engage those remote students as well.

John: In terms of getting comfortable, one thing I’ve recommended to a lot of people is that, if they’re new to using video conferencing, they should work with other people in their department who may also be new with that, and take turns hosting meetings, so they get to play with all the tools. And if people do that a little bit, there’ll be a whole lot more comfortable, I think, once they arrive at their classroom. Is that something you’d recommend, too?

Rick: Absolutely. And really, I would recommend that those partnerships go on past the preparation stage, if it’s possible to find a faculty member who you can either team teach with, or you can assist when they teach their class and they can assist you when you teach your class. That can be really useful because, let’s say we have a very large classroom, we’re probably going to mute the mics of the remote students so that we don’t hear every dog barking and train going by 100 times. So, as we have been muted, somebody, if they have a problem during the class, we have to have some way of knowing about it. And generally that’s going to be through the chat. So, most of these applications have a chat that can go on simultaneously. And again, in larger classes, it’s not going to be very effective to be monitoring the audio and video of all of the remote students. So, if we use the chat and say, let the students know, “Hey, if you’re simply confused, put a bunch of question marks into the chat. If you have a question, ask it in the chat.” But if you have a partner who’s working with you, and monitoring that chat, that keeps you engaged, and you focused on your teaching, but the person monitoring the chat can say, “Excuse me, Rick, you know, I really didn’t understand that last point you made, could you please go back over it?” or “I didn’t hear it,” or as a partner can say, “Somebody online didn’t hear it” or “There’s a lot of confusion online right now. Could you please go back over that point?” I think that’s really useful. And if you can’t do that with a partner, it’s useful to try thinking about rotating it as a student role. I know there’s some negative issues with that; there’s some problems in that you’re adding something to a student that may have some difficulty keeping up with the content and monitoring the chat at the same time. But, I think it is really important to have a way to monitor and check for understanding and check for technical problems while you’re teaching, and it’s difficult to do that yourself.

John: If faculty want to keep tabs on how things are going with their students, what else can they do besides monitoring the chat?

Rick: In smaller classes, you can keep an eye on the videos as well, just like you would in your regular classroom. If you have a seminar or discussion-based class that’s smaller, then you’re probably going to have enough room to see the students and keep an eye on them and scrolling through them and just visually checking for understanding. Then there are other things that we can do. We can do live polls, we can do quizzes in our LMS and other activities that will help make sure that students are getting the materials that we want.

Rebecca: I’m newer to video conferencing and have been experimenting with recording so if I needed to share something with a student that was sick, one thing that I realized, for example, in using Zoom is that the polling doesn’t show up in a recording automatically. So, there’s things that, if you don’t test it ahead of time, you might not know how to do it or how to set it up. So, I really found being able to practice with colleagues in advance really helpful, because I’ve discovered some of those stumbling blocks that I didn’t realize were going to be stumbling blocks.

Rick: Right? Well, and that’s key. The technology and where we’re going to be teaching, it might not be our own technology. It’s easier for us to practice on our own computers and our own systems in our own homes in locations where we plan on teaching. But in this case, we are probably going to be teaching in a classroom, and that classroom is going to be designed and laid out by, depending upon the school, somebody in IT or in a teaching and learning center, something like that. And we don’t know how it’s set up. We need to go in there and test it. We need to know how to change the camera if we’re going to use a document camera, for example, we need to be able to switch back and forth. We need to know how to do all those things. And that practice is beyond us becoming familiar with it. Like you were saying there, where you did a recording, I really recommend that people go to every room that they’re going to be using and record a session. It doesn’t necessarily have to be a full lecture, but test what it’s like when you’re speaking at the podium and how you need to speak to be clear, make sure that the levels are right on the microphone for your particular voice. My voice is deep and loud, and it carries very well. So, generally, people can hear me, even if I’m a bit aways from the microphone, but that’s not true of everybody. You really need to know where the mic picks up and how well it picks up. You need to know where the frame is in your video. So, if you like to move around a little bit and walk back and forth from one side of the room to the other, that is probably not going to work in this environment. So, if you want to do it, you need to know where you are in the frame, so that you stay in view for those remote students. If you tend to walk around… and this is something that we’ve been taught to do as teachers, or have learned to do… that we want to walk around and engage the class. We want to make sure that people are paying attention. And we can really do that by moving around. Unfortunately, if we’re teaching to a group of remote students, when we move around, they might not be able to hear us as well. But they’re also then staring at a blank wall or the chalkboard or the whiteboard. And that makes it a lot harder to pay attention for those remote students, and even more so for anyone watching a recorded session.

John: And all that’s good advice, not just during a time of pandemic, but before any semester because one of the worst things you can do is go into class for the first day and set the example of fumbling with the controls and not being able to get this class started well, and that negative impression can have a pretty significant impact on how students see you and your class. So, you want to have a really good strong start, however you’re starting, and working with either the classroom or your computer controls, I think is really helpful, as you said,

Rick: I think we can expect some healthy skepticism from the students too. So we want to try and alay those by being prepared. It’s difficult for people who have never done this before, didn’t plan on doing it, would never have agreed to teach using this modality in any other circumstances. I think, fortunately, most people recognize that this is a big issue today and understand why schools are doing this. We may not all agree with every step that our administrations have taken, but I think we all do agree we’d like students to be able to learn this fall. My daughter’s starting college this fall in California in an art center, and he didn’t want to wait another year to start college. Personally, I would have been super happy to take another year. I would have just taken a year off. I’d be in, like Costa Rica or somewhere far away from here, if I was eighteen, [LAUGHTER] but there’s all kinds of life circumstances. People want to keep their careers moving on and it’s also a very different world today than it was when I was in school.

John: I think it’s a very different world than any of us were in school. [LAUGHTER]

Rebecca: Indeed. For faculty that are having to teach from home or from their offices, and they haven’t done that before, can you talk us through some ways we might want to think about setting up our workspaces to be more effective and efficient.

Rick: I think, first off, we want to try and find a room that is relatively quiet and well insulated, sound wise, or isolated. That can be difficult. When we’re teaching at home, our children are at home too. Ideally, if you’re in a lucky situation, there are other people to help keep the chaos away from the room while we’re teaching, as much as we can. Secondly, I think finding a room that is well lit is a good thing. And then go ahead and start your camera, set up the room, turn on the lights the way you think they’re going to be, and then see how it looks. In the room I’m in right now, there’s an overhead light, and if I turn that light on, it’s not actually going to light my face better, because the way the lights going to come down, it’s actually going to hit the top of my head, and then put most of my face in the shadow. So, in that case, it’s actually better for me to have the natural light coming in from the window. But, we need to sort of think those things through in a way that we haven’t before. So, it’s good to bring up any video app really, and look at it on your computer and then adjust the lighting. So, the computer itself is going to provide some lighting, but then you might need to bring in an extra lamp to put on one side or the other to sort of balance the light. The other thing you can do is, if you have a light that directional and adjustable that you might normally use for reading or something like that, if it’s bright enough, you can actually turn it away from you and face it towards the wall or towards a lightly colored object if your wall is dark, and what that’ll do is that’ll bounce the light off of the wall and onto your face, and a light like that can otherwise be too harsh, but that way it can light it and sort of balance your light, keep your face well lit. Things like that can be really useful. And then again, just making sure that your mic is going to pick you up. Generally, the mics aren’t a big problem when we’re teaching at home in our rooms. Sometimes a headset can be useful. Testing and finding what works best for you, I think, is key in just making sure that the video appears in a way that everybody can see well and clearly.

John: And this was implied in your discussion, but having a natural light is really good, but you don’t want that natural light behind you because then you get more of that shadow effect. If you have a bright sunlit window behind you, which I’ve seen in so many faculty at webinars, you just see a dark blur surrounded by this bright light and you want to arrange it so, if possible, that light is facing you. I had that problem in my office and I had to put up a blackout curtain over the window so I didn’t get washed out that way.

Rick: Right, if you can’t change where your desk is facing and the light is behind you, that’s not gonna work. Even if it’s in front of you, if the way the sun shines at certain times of the day is straight in, it’s gonna make you squint, you’re gonna end up washed out, so the details on your face will get washed out. So, then you might want to think about curtains in that case. We want to work on the lighting so that we’re clear, that people can see our faces and our mouths. That helps people understand what we’re saying, but it also helps them convey all the nonverbal communication that’s part of the way we speak, that nobody can see in this podcast. But when we’re doing our video conferencing, they can absolutely pick up all kinds of clues on whether we’re smiling, on how serious we are when we’re speaking, based upon our facial expression. And you can’t really see that if, like you said, you’re backlit, whether it’s from the window or whether it’s from where the lights are in your room. So, we really just want to straighten out the lighting as best we can right from the beginning.

Rebecca: Also thinking about time of day is key and remembering that in the fall, we’re gonna head into shorter days. So, you might have really good sunlight at the end of the day right now, that lighting is great, but it might actually be much darker. [LAUGHTER]

Rick: That’s absolutely true, especially for those of you up in New York. [LAUGHTER] It’s a little less of an issue for my friends south in Tucson or Phoenix or Corpus Christi. [LAUGHTER]

John: I noticed behind you there’s a painting and some artwork on the wall, but there’s nothing that’s really distracting, that’s taking the attention away from you. Is that something perhaps that faculty should also do? Not have something really distracting in the background?

Rick: Absolutely. Anybody doing any video conferencing, whether it’s for anything that besides your friends, it’s not only going to matter because it’s distracting, but you might have things that… I’m looking around this room and right now I think everything… over the past four months, we have made sure that everything behind us is non-controversial as well. Because you may have artwork in your home that’s beautiful and wonderful, but we don’t necessarily want to begin religious discussion at the beginning of our computer science class, or something like that, right? So, we want to just keep everything nice and clean and neat.

Rebecca: Like my bland gray walls behind me. [LAUGHTER]

Rick: Yeah, exactly. The bland gray wall works really well. [LAUGHTER] So does a nice piece of artwork, I think is perfectly fine… and really any artwork is fine. I don’t mean to be too prudish on these things, but especially if we’re teaching 18 to 22 year olds, sometimes they can be a little bit more easily distracted by things like that. Well, actually, really anybody… You see something that’s gonna upset you, it’s gonna upset you. So let’s think about that and just make sure that the room is welcoming, and, and ready for you to focus on your coursework and not on the room.

John: In a lot of ways, the easiest environment to teach in that sort of framework is when you’re in a room where you get to control all that, to control the sound and so forth. Many colleges are going to be using a system in which there is some type of a hyflex structure, without much flexibility in terms of how students choose to engage, where some students will be present in the classroom in reduced numbers and spread out across the room, while other people will be participating online synchronously. And some other people might only be available asynchronously because of other issues, maybe because of healthcare issues, maybe because they’re back at home taking care of relatives, or they themselves are perhaps in quarantine somewhere, and may not be able to always participate at the same time. in that environment. What are some of the challenges that faculty might face in trying to engage in say, active learning type activities, which require some interaction among the students in person, among the students online, and perhaps even between the online and the face-to-face students?

Rick: Let’s take that last example first. From a teaching standpoint, that’s ideal. We’re mixing our in-class students with the remote students. It’s helping us build community. And it’s great. And that can work really well. But, we need to think about the environment. So, if we do one person locally with one or a few students remotely, then the local student needs to have a computer, or perhaps they could do it through their telephone. And we probably want them to have a headset on because, if everybody in the classroom has a computer open, and is communicating with people from off site, we’re going to just sort of have a bit of chaos in all the sound coming from the speakers. But, if we can find a way to do that, if the room is suitable, or if there’s easy ways to break students out, that’s sort of the ideal. Otherwise, I think we’re looking at building breakout sessions within the remote students so that the remote students and… you mentioned Collaborate earlier… students can make their own Collaborates and then work together there and then come back to the central Collaborate that the class is in and we can do sessions like that and then have them present the results of their group breakout. They can communicate that back. That’s another way of doing it. And then the local students can obviously just meet in groups within the room.

In the LMS, we may find that the group tool is something we need to use for these video classes, though, because some schools are not actually doing the work of dividing the section up. So, if I’m going to have a third of the class come on Monday, a third of the class come on Wednesday, and a third of the class come on Friday, I’m going to need some way to decide that. And since most of the LMS tools do have groups, I can either randomly assign students or I could put signup sheets for the days. And then I could also use that group rule to do breakouts, whether they’re asynchronous or synchronous, it will help to have them set up. And so I can, again, either do it randomly or through sign up. And then there’s all kinds of group activities that people can do once we get into that asynchronous realm. In the synchronous realm, they’re meeting, they’re speaking, they’re coming up with a plan and then they’re reporting it back to the group and the asynchronous it might be different. They might meet, come up with something, and then post their work to the LMS. for everyone to review.

Asynchronous environments can still be very interactive and active through discussions, through group work online. There’s lots of different tools that you can use for that. And we can also engage the students with polling. There’s Kahoots!, I’m not sure everybody’s familiar with those. But in, Kahoots!, there are ways of doing polls and you don’t necessarily have to have your institution on board. So, if your institution doesn’t have a polling system, or it’s not built in… like Collaborate has a built-in polling system… I believe Zoom does as well. But, if you can do some kind of polling that can help the students stay engaged. You can also do little quizzes in a similar way with the polling… and just sort of checking for understanding, I think those are great ways of helping students stay engaged.

John: And in terms of Kahoot!s, you can do it synchronously for the people who are in the room and remote, and then you can have some discussion of their questions after you go through them. But, you can then set it up so that you can share the quiz online so that students, at least, would have the option of participating at asynchronously as well. They wouldn’t have the same real-time discussion capabilities of the students who were there synchronously, but at least they would have the same type of retrieval practice as an exercise with Kahoot!.

Rick: When you talk about the recorded version of your video conference or your streamed lecture. That is not an ideal way to learn or to teach, to watch a recorded session of a bunch of other people. People are going to tend to zone out and not be able to follow everything that happens. They’re going to be distracted by the other things going on and there isn’t going to be anything pulling them back in. Because when you say, “Okay, everybody do this poll…” well, on the recorded version, and they’re gonna do it whenever later, they may not pause it, they may not even notice that you told them to do something right away. Which doesn’t mean that I don’t think people should record their classes. I absolutely think we should. But, I think if we have a substantial number of students who are not able to attend live, then we are much better off with a very strong online learning component. At least in my opinion. A lot of these ideas that sort of flex idea came because people read work by Brian Beatty from San Francisco State where he coined the term HyFlex. When I was researching this when I started at NAU, I found that there is HyFlex, but there’s also been other people who’ve done very similar types of teaching, calling it different types of things, but it hasn’t been widely used. But, when you look at what they did, if you read the articles and research around this, which is relatively scant. But, what there is pretty much shows that all of the previous experiments with this involve having somebody there to assist the faculty member, whether it was a partner or a learning assistant or an educational technologist, somebody was there helping. And then the other thing that they really all did is build extremely good and strong online components. And in the San Francisco State one, they didn’t necessarily have to show up in person at all, they could do it entirely through the learning management system. And in my ideal world, schools would give faculty options so we would be able to teach one day a week live, and we would stream that for anybody who wanted it and everybody would have, say, one live session. And then in my ideal world, there would be an online component for the other half of the course for that week. And that would, I think, give students more actual flexibility in learning, but it would also, because the strong online component is so important, it would give them real incentive to create that strong online component.

John: And that would also have advantage if schools have to shut down at some point because if they do shut down, the face-to-face component will go away. And having that ready would make the transition a lot smoother, I think.

Rick: Absolutely. And if you are counting on everybody showing up every week, in the middle of a giant pandemic, you’re probably going to be disappointed. So, if you’re hoping to pass out papers, the one day a week that the students come to class, I think you’re going to find yourself with a lot of headaches. So, I think having your materials online… that’s the whole thing with an online learning course or a video conferencing course. And we didn’t really get into my background with that. I ran, for 13 years, a video conferencing system at a community college here. We’re the second largest county in the country and more rural than the largest county. And so at one point, we were teaching students over video conferencing who were living at the bottom of the Grand Canyon. So, those students, they’d have to hike out 12 miles or take a helicopter and then drive for four hours to get to our main campus. So, that was why it made so much sense for us and why we had a video conferencing program that went on to 2015. And that’s why it was like that. It was because there was this real reason to do it. But, when I was managing it, I would tell faculty, we have to build online components. And the reason is, the plan that had been made by the academic leadership was… well, we had this complicated system of faxing papers and collecting things through fax, we were already building online components. We started with WebCT and I said if we use WebCT for this we can do low-stakes testing through WebCT. We can distribute papers. When the students lose those papers, we don’t have to worry about finding a secretary or an administrative assistant, or another professor who’s at the other campus to run and print it out. And in the cases where we were working in even more remote areas, we didn’t have those types of resources. So, we really needed to use the online component. And that’s even more true if your students are going to end up staying at home or if somebody gets exposed and has to self isolate for a few weeks, they’re not going to be in person. So, having that online component really is going to make your life easier. And as you said, right now, when all these plans were being made, our state looked fine. But our state now is one of the highest rates of infection in the world. So, I don’t know what it’ll be like in a month. Nobody does.

Rebecca: I didn’t want to follow up a little bit on this conversation. We’ve talked a lot about what it’s like for faculty in planning, but not really entirely about the student side of remote learning, like what their systems might need to be like or what kinds of rules we might have in place? Or what kinds of expectations we have about participation in terms of a synchronous video component. Can you talk a little bit about that?

Rick: I think one advantage we have that this is happening in 2020 is that, if we’re looking at engaging in something that’s primarily video and audio, our telephones really today can do a lot of that, and even answering short polls we can do on our phones. So, the students do have that possibility. But, ultimately, a computer is a little bit more effective. And one of the things I am worried about, actually, is access to that technology for some students who may normally rely on computer labs at our schools. And when we’re thinking about it as faculty members, it’s tricky for those of us in instructional design and educational technology, who have been doing this our entire careers to remember that not everybody has all the tools that we do. And so I’m really hoping that schools are either making socially distanced labs available, or ideally having equipment that is available for checkout for their lower-income students who may not have all the equipment. And I think the other problem that we’re going to have for students is going to be quiet learning environments. A lot of students live with multiple people living in the same room. A lot of students live in environments that are a little bit noisier, and we’re gonna have to adjust to that and figure out, based upon the size of our class, like I mentioned earlier, do we need to mute them? How are we going to check for their understanding if they’re muted? Are we going to have all the video available? You mentioned what the students have at home. What is their internet connection? Do they have a strong enough internet connection? It probably needs to be at least in the megabit realm for this to work at all. And I think the other problem is that sometimes students are going to be on shared connections. And what I found in the spring, that we had switched from the telephone company, because I was able to get a much higher bandwidth to the cable company, which generally has been great. I’m working at home, my partner works at home and that really hasn’t been a problem. But I tell you what, when my two daughters were both participating in Zoom conferences, my spouse was on a Zoom conference, and I was on a Zoom conference, we were not all doing video, it just didn’t work. And so we had to mute some of those sections. And really, some students may not even want their video on. And so I think we’re gonna have to be open and accomodating for those types of questions that students might have. Because they may be a privacy issue. It may be a technology issue, and if they don’t have their video on, I don’t think we need to spend a whole bunch of time talking to them about their video and why isn’t it on, whether it should be on. I really feel like there’s so many different reasons that are valid for the camera to be off, that we should probably let some students participate without video feeds.

John: And the same argument can be made for audio because if they’re in a noisy environment, they may not be able to even speak without a lot of background noise. It’s one thing to invite students to turn on their video and audio if they can, but we probably shouldn’t require it.

Rick: I think you’re right. I think it’s also one of the real key differences between that built video conferencing environment that was pretty popular a good 10 to 20 years ago. Those rooms were purpose built. Every single room was purpose built, whether it was built for somebody teaching or whether it was built for the student receiving the mat. rials. Everybody went into a room that was, ideally sound isolated, that had a good mic setup. And that’s just not going to be the case when everybody’s at home.

John: We always end with the question: “What’s next?”

Rick: I think what’s next, globally… what a lot of us in instructional technology and instructional design really hope is that this fall is gonna go better than last spring. Because I can’t tell you how many, what I personally think are bogus, articles came out saying, “Look, it proves that distance learning doesn’t work.” No, it proves that distance learning needs preparation, and you can’t do it with a day’s notice. So, hopefully this fall, people will have much better experiences. I really hope people contact all the resources that are available at their schools. If they have instructional designers, those people can really help you build that online component. There are people who have been working in video at your school. I know there’s a number of people at Northern Arizona University with extensive experience. Reach out to those people, they can really help you. They can make sure that the room is the way you need it to be. I would say really reach out. But, as far as what’s next, I hope that what’s next is that people say, Wow, building an online component really made my life easier. And that they’ll start building online components all the time every year. And I’ve been pushing that to the point of obnoxiousness…. sorry, folks who worked with me… for decades now, that it’s more work that first semester you set it up, but every subsequent semester, using your learning management system, even for your in-person classes, is going to help. And now we’ve seen that it helps if there’s a global pandemic, but we can also see that it could help if there was a massive forest fire that went through your town, and everybody had to evacuate and you didn’t want to call this semester a loss. And there have been some, more in K-12, but some experiences where that really did happen. People were able to do it, and it’s also really critical. I don’t know how much you guys talk about K-12. But, that’s an environment, too, where preparing for emergencies is easier to see now. But, also where college students may sometimes forget things, 12-year olds and 13-year olds forget things a lot. And so having the work online for them can really help them. So, I’m really hopeful. That’s what I think is next. What I hope is next is that we have a much better experience this fall under such trying circumstances.

Rebecca: Well, thanks so much for your insights and some thoughts about preparing for the land of video moving forward.

Rick: Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

150. Pedagogies of Care: Sensory Experiences

This week we resume a series of interviews with participants in the Pedagogies of Care project. In this episode, Martin Springborg and Susan Hrach join us to discuss how sensory experiences can be used in an object-based learning framework to enrich student learning.

Martin is the Director of Teaching and Learning at Inver Hills Community College and Dakota County Technical College. Susan is the director of the Faculty Center for the Enhancement of Teaching and Learning and an English Professor at Columbus State University. Martin and Susan both contributed to the Pedagogies of Care project. Martin is co-author with Natasha Haugnes and Hoag Holmgren, of Meaningful Grading: A Guide for Faculty in the Arts. Susan is the author of the forthcoming Minding Bodies: How Physical Space, Sensation, and Movement Affect Learning.

Show Notes

Transcript

John: This week we resume a series of interviews with participants in the Pedagogies of Care project. In this episode, we examine how sensory experiences can be used in an object-based learning framework to enrich student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Martin Springborg and Susan Hrach. Martin is the Director of Teaching and Learning at Inver Hills Community College and Dakota County Technical College. Susan is the director of the Faculty Center for the Enhancement of Teaching and Learning and an English Professor at Columbus State University. Martin and Susan both contributed to the Pedagogies of Care project. Martin is co-author with Natasha Haugnes and Hoag Holmgren, of Meaningful Grading: A Guide for Faculty in the Arts. Susan is the author of the forthcoming Minding Bodies: How Physical Space, Sensation, and Movement Affect Learning. Welcome, Susan, and welcome back, Martin.

SUSAN: Thank you.

MARTIN: Thanks for having us.

John: Our teas today are:

MARTIN: Actually, it’s very hot in Minnesota right now. It’s like, it feels like 100, but it’s truly 93-94 degrees. So, I’m drinking iced latte with vanilla almond milk. It’s really tasty.

Rebecca: That sounds good.

SUSAN: Nice. I’m having a similar heatwave issue. I’m drinking sparkling water that has cucumber and strawberry flavor.

Rebecca: Well, that sounds good.

SUSAN: It’s my current summer favorite.

Rebecca: I, despite the fact that it’s 90 here, still am drinking hot tea because, I don’t know, I have a problem. [LAUGHTER] I’m drinking a summer berry green tea.

John: In our last podcast recording, you mentioned the summer berry green tea and I forgot that that was something they had at Epcot, and I saw my own, so I am drinking the summer berry green tea that I picked up in Epcot last November. It’s very good.

MARTIN: Can I ask a tea question, as long as I have two tea aficionados here?

Rebecca: We can try. [LAUGHTER]

MARTIN: So, my afternoon drink of choice is Earl Grey tea and coffee in the morning, Earl Grey in the afternoon. But I know there are different schools of thought on how you should steep this tea. So, just give me the definitive steeping on Earl Grey tea. That’s what I’m after.

Rebecca: I have a tea pot that does it itself.

MARTIN: Buy the tea pot that that does it for you.

Rebecca: It’s like you put in the kind of tea and it just does it.

MARTIN: Okay.

John: You specify the type and the strength and it brews it to that level. Yes, but, I think four to five minutes is normally recommended.

MARTIN: I’ve heard three, I’ve heard five. So, I’m like, should I just do four and split the difference?

John: Four is probably pretty safe, I think.

MARTIN: Alright.

Rebecca: Yeah, I clearly can’t handle it myself. so I have a tool to do that for me.

MARTIN: Thank you.

John: I have the same one. It’s a Breville tea maker, it’ll brew tea and you just pick the type, and it will even drop the basket in once the water has reached the appropriate temperature,

Rebecca: …and take it back out, it is the most expensive teapot you can possibly buy. So we invited you here today to talk about your contribution to the pedagogies of care project. Can you tell us a little bit about this project?

MARTIN: Yes. So, there’s a Teaching and Learning Series that West Virginia University Press has been engaged with for some time now. I want to say a couple of years we’re going on. So there are many authors within this series. Mainly the books are just short, to the point, for faculty, here’s how to do this thing. Tom Tobin, I’m just going to credit him and Tori Mondelli, both of them for starting this. Basically, when the crisis hit and we all were involved as directors for teaching and learning and other roles on our campuses, were responsible for helping faculty move courses online, and myriad other things, Tom and Tori got the gang together on Twitter and just said, “Hey, let’s put something together.” And that’s really how this thing started to form. We had a couple of meetings to talk about how we would do it, and we just did it. Everybody took on a part of it. And Susan asked me if I’d come on board with her object-based learning session, which I was happy to do. But now that the resource is out, it’s been made available to everybody. It’s an open educational resource, and anybody can use it for however they’d like.

SUSAN: One of the fun ideas that Tori and Tom suggested from the beginning is that it would be a multimedia collection. And so we tried to keep the videos and podcasts to no more than 20 minutes, or maybe a little bit over 20, but not much. And there’s infographics and PDF articles. And so I just thought it would be fun to have an audio-only entry and fun to collaborate. And so Martin’s area of expertise fit in nicely with the topic I wanted to address and we were off to the races.

John: It’s a really nice resource. I know we’ve shared it with our faculty and many teaching centers have shared it with their faculty.

MARTIN: Thank you.

SUSAN: It’s great to know.

Rebecca: Yeah, it’s definitely been popular on our campus. I’ve certainly been eating them all up and digesting what’s there and taking advantage. And in your particular entry, you talk a lot about object-based learning. Can you start by explaining to our listeners what object-based learning is?

SUSAN: Sure. Yeah, so I’ve heard it referred to both as object-based teaching and object-based learning, but it comes from the fields of museum education and art history and archeology where the object is the primary way into knowing more about a culture or a time period or an aesthetic sensibility. So new neuroscience of learning is affirming that that just works really well as a structure for human learning in general. So I take the sequence from a book that I have found really useful by Guy Claxton called Intelligence in the Flesh. But he identifies these three steps to learning: the first step is noticing, the second step is imitating, and the third step is practicing. And so object-based learning focuses mostly on that first step, noticing, as sort of the foundation for how you’re able to imitate well and then practice well after that. So, I first became familiar with this by going to a pre-conference workshop at POD in 2018. And Jessica Metzler, from Brown’s Sheridan CTL, did this great session called “Ways of Seeing” and she took us to the Portland Art Museum and we all sat around and looked at this sculpture from, I think it was the Anglo Saxon period. None of us had any idea what it was. And so it was perfect because it was an interaction with a primary object for us to be able to start a series of questions of inquiry.

John: Could you explain how this might be used in other disciplines? Certainly, we can see how statues might be used, but how might it be used perhaps in the STEM disciplines or in other fields?

SUSAN: So, if you think about just a sort of an experience that everybody’s had… just to be more concrete about this noticing, imitating, practicing… something as simple as tying your shoes. How did you learn how to tie your shoes. Well, you had to notice what your parent or somebody was trying to get you to notice, and then imitate what they were doing, and then practice a lot yourself, right? So any discipline that’s conducting an experiment or analyzing any kind of text, and I mean that in the broadest sense of the word, think about the way that you wrote your first scholarly article. You had to notice how other people did it, and then imitate them. And then just practice your own a lot. It’s just the sort of formula that works really well for almost any kind of learning. And it starts with noticing. And so, whatever object you might take to have your students notice carefully is the place to begin. For example, something that sounds kind of abstract, I taught a translation studies course about a year ago, and I structured the whole course on just that three-part premise. We just noticed a lot of things about how translators were approaching the task. And then we tried to imitate various approaches, that we had already noticed that they took differently, and then the students were able to start practicing their own versions of translation with, I think, a much more informed sense of what they were doing,

MARTIN: Well, my background, before I got into faculty development was in the visual arts, I taught photography and art history for about 20 years prior to getting into faculty development. One of the courses that I taught was co taught between myself and a creative writing instructor. And so I taught the photography side of that class or half of that class… and the creative writing for that part, the students use photographs as primary sources to really start that writing process for the various pieces that they wrote during the course. And so that’s another example of how the photo was the object.

John: It sounds like the first part of this is just helping students develop the skill of focused attention, so that they learn how to pay attention to things that they might not normally focus on. And, as part of that, you describe a sound walk activity as an example. Could you tell us a little bit about that?

SUSAN: Yeah, I’d be happy to. Sound walks are pretty much self explanatory. You send the students… I mean, you could do this indoors as well… but outdoors works better. Just out for As short of a period of time as you might feel you can spare, and tell them that their task is to just only notice what they can hear. And it’s best if they can immediately write down all of the series of things that they can hear. It’s okay for them to write down something that they can’t identify, that’s something they notice in the soundscape. But if you have them go outside, and then they’re walking, which builds in movement, which is automatically better for opening up our brain’s ability to absorb things, and then ask them to take this shift in their normal perception that just like triples the impact of their ability to notice things, to perceive them in a new way. And so it’s sort of like priming the brain for learning other things, because you kind of take in your brain out of its normal autopilot mode, it’s more open and receptive to noticing other things.

Rebecca: I’ve taken a similar approach in some of the things that I do in my classes as well because I teach primarily web design. And students often are familiar with websites, they go to them, but they go to them as, like a consumer, and not as a maker. So they don’t really notice unless they take the time to slow down and look in a different way.

SUSAN: That’s perfect. Yeah, that’s a great example. I mean, I think a lot of education in general is helping people to learn how to shift their perception of things, and then also to remain open. Once you’ve changed your mind once, that’s not the end, you’re going to continue to have that sort of open and curious attitude to be able to continue shifting your perception as a lifelong learner. So I feel like it’s just such a foundational skill in higher ed in general.

Rebecca: So Martin, can you describe some of the ways that you might use the same method in a more visual environment, rather than just in audio?

MARTIN: The object-based learning, as Susan mentioned, is pretty native to disciplines like art history, visual arts. Certainly, for example, in teaching art history, that’s an easy use, you’d bring students to a museum, and you have a guided time with them, where you guide them in that exercise of looking at something and applying it to something that they’re going to do back in the classroom or on their own time in preparation for the next class or a discussion. So, we together look at a piece or pieces, or they have their own itinerary, where they have pieces that they need to find focus on, make notes about. If you’re teaching that kind of class, reproduce in sketch form, and then bring that back to an assignment or assignments that they will produce back at the college. I feel like my discipline is an easier application for object-based learning than what we’re talking about the expansion of that into other disciplines. In our podcast, we talk about taking object-based learning and applying it to the STEM fields, for example.

SUSAN: And I want to add too, I mean, I think visual attentiveness is really its primary mode, but I sort of narrowed down for our podcast because we knew we wanted to keep it under 20 minutes, let’s just talk about two of the senses. But, you could do a lot with touch, I think. And I’ve seen some really great pieces, some museum ed pieces about physically handling objects, and the way that students can learn things about any sort of texture or object through just paying a little bit more attention to its tactile existence. And, I’m in literature, it’s not the first field you would think of as being tied to an object that way, but, you know, books, people have very deep attachments to the physical book. And I don’t think that we stop often enough to just talk about what that means. If you bring your students to the archives, for example, and they’re allowed to handle an older book, what does it smell like? What’s the texture of those pages like? What is the cover like? Those are all really interesting ways for them to find their way into being more curious about the object itself, the text itself. And for the most part, we just sort of present the thing as if the content inside is really all that we need to pay attention to. And really, it’s the full experience of that material object… the type font… the way it was produced… you know, all of those things about the history of the book are fascinating, and I’ve been lucky enough to be able to visit archives and deal with archival manuscripts. And it really did transform the way that I looked at early texts when you can look at the physical handwriting of the person who produced it, touch the paper that they touched, it’s a very human way into the study.

MARTIN: And these practices are not just good in theory, like “Oh, it’d be nice to bring a class out of the archives so they can smell books,” or have that experience of touching and interacting with those as primary sources. I don’t want to get us off on a tangent right now, but a project I’ve been working on for some time is photographing faculty teaching in the classroom, to just document what that looks like, and some very real examples of what Susan is talking about. So, I was just at Princeton photographing a class where they actually were down in the archives, and they had books that they were leafing through… old rare texts that were one of a kind to illustrate the points that the faculty member was trying to make in this humanities class. Another, I was at Caltech not too long ago, photographing a geology course, where the instructor was passing out rocks that the students could actually feel, touch, experience, as he was talking about that kind of rock. So, it’s used all the time. It’s maybe more prevalent than people actually realize.

Rebecca: I think one of the things that’s interesting is we often try to tell stories about our experiences. And those embodied experiences include all of our senses, but we often try to capture it in one medium, and we don’t always think about all the other senses. So, I think taking this time to notice, and notice in different senses. Maybe then, as a visual designer, it might be really interesting or important to to notice all the other senses instead of just the visual in studying something, because we tend to preference the modality that we create something in.

John: It’s all creating additional connections for people that make it easier, perhaps, to integrate the information.

SUSAN: That’s right. And I think even, just to build on what Rebecca was saying about how we tend to privilege one sense, and it’s often sight, but I think it’s helpful for students, even imaginatively, to start noticing how something might feel with their other senses. So, as an example, I did a little experiment with my Renaissance Lit students a couple of years ago, and I read them the description of the execution of Mary Queen of Scots, which is particularly violent, and it’s an exciting thing to read about, but it’s a little gory, and I asked them to respond to it by doing a little imaginative exercise about putting themselves in that room. And they could be anyone in the room. They could be just as a witness, they could be an observer, or they could be the executioner himself, or they could be themselves, sort of as time travelers. And then I asked them specifically to talk about what the temperature of the room felt like, what it smelled like, what sorts of internal sensations they were feeling as the execution unfolded. And I got this really great set of responses back from them. A lot of them are studying creative writing. So I, you know, was partly designing this exercise because I know that’s the writing that they’re interested in doing, but it was just really fun. And I think if you were teaching history, or really any field in which there’s some sort of story that you could read and have people kind of imaginatively place themselves at that moment, maybe the moment of something important that happened in your discipline, it gives them a more embodied way to connect, even just imaginatively, with it.

Rebecca: in this era of social distancing and virtual spaces and screens, do you have some suggestions of ways to incorporate object-based learning in new ways, than maybe some of the ways that we talked about which might really require being in close proximity or in small spaces like an archive that you might not have access to in the fall?

MARTIN: Well, there are primary sources all around us, we just need to step outside. And with a little guidance from the instructor, students should be able to have those experiences anywhere that they might safely explore in the world right now. So, it doesn’t really need to involve, for example, going to a crowded museum or another crowded space to find primary sources. You can, for example, go back to geology again. And you can easily go on a field trip yourself without human contact to locate the kind of rock or material that your instructor wants you to find and reference and be in the presence of and touch. That’s just one example.

SUSAN: Yeah, I love thinking of ways to get people out from behind the computer and the screen. I mean, I think the whole vision of online learning that we have right now involves people being planted at their desks behind their computer, and oh my gosh, we just need to find ways, like Martin said, of sending them out on field trips on their own, to do whatever might be productive. For you to ask them to leave their desks and go investigate. It could be something in their own kitchens. It could be something outside. I just recently had the opportunity to teach an introductory level interdisciplinary course, and I used this wonderful book I would recommend to anyone by Bonnie Smith Whitehouse that’s called Afoot and Lighthearted: A Journal for Mindful Walking. And she’s got 50 different writing prompts that you can assign as part of taking walks with the students. They’re super thoughtful. She’s got all sorts of great references to important thinkers and their philosophies about walking and why it matters, for example, to social movements. And so, it was so timely, in fact, with the recent Black Lives Matter protests and what just walking means for human beings in a bigger sense. What are we doing with our bodies when we use them in those ways? And so the course was based on physical movement and the creative brain, and I asked the students to pursue some sort of creative project and, oh my gosh, they picked the most fun collection of things. They were crocheting and building furniture and tie-dyeing t-shirts and baking and so they were doing these creative activities, but they had to walk and journal and then see what sort of effect that had on their creative process. And it was great fun, and I also felt like it was the sort of thing we all needed, me included, at this particular moment, I don’t think it was what any of them were expecting from an academic course. But, they did a lot of writing, and they put into the online discussion board, all sorts of sensory things. So, they would record 20 seconds of their walk through the neighborhood. And we could hear their footsteps and we could hear the lawn mower and we could hear the birds and it was just such a great way into students’ environments. That was unusual, and that made the course feel like it was jumping out of the computer in a way. So that was something I feel really lucky to have been able to just use as an experimental summer class. And we had a good time.

Rebecca: One of the things that you mentioned in your work is using podcasts as a way of noticing. Can you talk a little bit about ways that we might use podcasts?

SUSAN: Well, yeah, I think in a similar kind of way, to get students away from their desks and from sitting, there are so many great podcasts now, and there’s lots of educational podcasts that are connected to everybody’s discipline and touching on current themes that make it feel really relevant. And that material is just out there waiting for us to curate, and adopt, and include in our courses. And then, I think, if you can direct the students to listen to an episode of something that you find relevant for your discipline and tell them that the assignment includes you must take a walk while you’re listening to this or do some other sort of movement that does not require you to be mentally focused on the movement. So cleaning, I think, painting a room, or maybe driving long distances… I wouldn’t want somebody to be too distracted in their driving, but not doing homework for other classes… let’s put it that way… an activity you could participate in and listen to the podcast at the same time. I think that’s really kind of the ideal way for them to be able to experience an audio only delivery of content, and also have them not sitting in front of their computers.

Rebecca: What I really love about hearing about podcasts is it actually gets students to start doing some professional development. It’s modeling some of those kinds of things that they might do professionally as well, to continue knowing and learning and noticing new things in the field. It almost get them in the habit really early. [LAUGHTER]

SUSAN: Yeah, absolutely. I mean, this is, I think, at least the current mania for adult learners. And also, so many people are really attached to their books on… Well, I would say books on tape, but they’re not really books on tape anymore, or CD… they’re audio books. So I suppose it depending on the book, you could also assign students to read a primary text as an audio book and see how that shifts things, how it changes it up,

Rebecca: Especially when it might be in the author’s voice or something and changes how you’re understanding it or you’re hearing that person with their words and their emphasis.

SUSAN: Oh, so that reminds me of one other little exercise that I can recommend, which is, if you’re in a classroom, and you could do this with social distancing, and you have a podcast or an audio interview or something that you want the students to hear, you can have everybody listening to it as a group, but give them individual spots to stand along the whiteboard, or if you’re lucky enough to have portable whiteboards where they can be apart from each other, and have them standing and taking notes and doing whatever sort of sketching or doodling or things come to mind as they’re listening to the audio piece. And then when it’s over, everybody gets to share their notes together, and you can kind of see what everybody picked up on as a group. It’s really great. It’s a nice way to have them build on each other’s knowledge and also to sort of watch how other people take notes, and how other people process things. But I wanted to ask Martin, because I saw at some point in the spring that a number of museums had started making their collections more available as virtual tours, did you pay any attention to like which ones we might want to look at? Or do you remember which of the museums were doing that sort of virtual gallery walk?

MARTIN: I didn’t, and I haven’t been teaching for a while now. But because, through Google, I would frequently have my online students visit museums around the world, and then do virtual tours. So even though if the museum itself didn’t have that capability, you can go to Google Arts and Culture and do a tour through Google, that Google has set up for you. That’s a really great resource for anyone using Arts in the classroom to take advantage of. Of course, there are places like MoMA, etc… they have virtual tours set up already that are, in my opinion, they’re just a little more limited than what Google has available. But, since they’re produced by the museum themselves, they’re also a little better quality than what Google has to offer. But, at any rate, the student can go through a museum virtually and it’s experience, kind of like you’re walking the halls.

SUSAN: That’s great. I didn’t know about the Google Arts and Culture.

MARTIN: Yeah.

John: And more generally, there’s a growing number of virtual tours that are provided to historic sites, to other locations, where if you have even Google Cardboard, you can get that 3D experience with your smartphone, which provides a somewhat richer experience at times when travel may not be as likely or when people can’t afford travel in general.

SUSAN: Sure. That’s a good point, John,

SUSAN: John’s reference just now to visiting historic sites made me think about the way that I initially got interested in sensory learning, which was because I’ve had a number of wonderful opportunities to teach abroad. And it was such a striking difference to lead students through historic sites and have them walk in the footsteps of either a character from a story or the author of the story. I started designing these assignments called “You are There” reading experiences where we would go to the place and then read the thing that was written in that place or about that place. And I just enjoyed those learning moments for me and for the students so much that it became sort of a driving challenge for me to figure out how we can replicate that, when obviously, we can’t take everybody 3000 miles away to have a “You are There” reading experience. So, what can we do with our bodies on campus, in the environment that we’ve got, that would allow them to have a similar sort of portal into a distant world? Our archival library is focused on an American collection and I didn’t think there was anything in there for me, as a early modern British person, to be able to take the students to and then when I talked to the archivists, they said, Well, you know, our earliest two maps are colonial maps. They were made by French and British mapmakers and the dates were like 1592 or something and then it suddenly clicked for me… wait a minute, 1592, that is me. I can take my students to our library even though it’s focused on Americana. And we had a great session with those maps at the library because we could see how the French wanted to make the territory of Louisiana exaggeratedly large. And the British wanted to make their colonial territories exaggeratedly large. And so neither of the maps are particularly accurate, but they definitely show the bias of their creators and it was just really wonderful to be able to stand in front of these large-scale maps and have the archivists also talk about them as not meant for actual navigation. They were like propaganda pieces. So, you never know when you might find something on your campus that lends itself to a “You are There” moment.

Rebecca: It’s funny that you mentioned study-abroad things because I’ve also done a lot of classes with travel, and I did some similar kind of sensory work and had students experience a similar kind of space, like a cafe or something, in our town… like at school, and then do the same kind of activity abroad. And then we compared those different experiences. And we did it for different kinds of spaces, even wayfinding and the different ways you might get around. How you might get around in a building you’re not familiar with on campus versus how you might navigate in a different place where you might not speak the language.

SUSAN: That’s brilliant. I love that.

MARTIN: That kind of exercise is still completely doable. Even though we’re somewhat cooped up right now, you can still get out of your house, I had an assignment every semester in my photo class that had students go back home if it was possible, or go to another place of significance and do a guided looking and photographing exercise of that site, which is an exercise and learning experience that is completely doable still and safe. But, it’s so important to get out in the world and be guided through exercises like that.

Rebecca: I was in a webinar yesterday where they did an acknowledgement of the native land that they were on and then encouraged everyone to do the same that was participating in the webinar and took us to an online site that would actually tell you if you weren’t aware. And that’s another way of experiencing your space in a different way and thinking about it in a different way. Although not necessarily sensory, it still kind of gets to that place-based information, which I thought was really powerful and really interesting.

SUSAN: That is really interesting.

MARTIN: And with a place of significance, there’s no way to experience that in a book. You can’t really truly understand what Frank Lloyd Wright was trying to do with Prairie Design unless you go to a place and experience how it fits within the landscape. You can see lots of pictures of it for sure, and books, but you have to be there at some point. You have to be present at one of those sites to understand that kind of work.

SUSAN: But I think we can do a really good job with priming students to have that moment when they get to see Frank Lloyd Wright house have as big of an impact as it possibly could by doing things like Rebecca was saying about. You teach them how to just shift their perception in familiar environment. And then, I think, even just the looking at the photographs of a place that they may eventually visit leads to that really excited anticipation of seeing this thing that they’ve been guided to notice carefully and feel like they have a lot of prior knowledge and experience about before they get to see it in person. It helps to, for example, when you do finally get to go to a museum, feel like it’s just this huge thrill to see some object that you’ve been staring at in a book for a while. It’s a different thing than being guided through rooms full of paintings that you’re seeing for the very first time, and you don’t really have the context to appreciate why this is a big deal. I noticed that when I did a one-week Spring Break travel program, because I had been really skeptical about how that could possibly be a long enough time for students to understand cultural difference, for example. And, I mean, it is too short of a time for them really to go through the full journey of feeling alienated and rejecting the new culture and then coming around to understand partial differences in cultures, but we got to use our two months in the classroom before that spring break travel to get everybody pretty excited about when they would get to see these things in person. And they were completely thrilled… starstruck… about getting to see things that, if we had gone on your typical six-week summer program, I would have been standing in front of whatever saying, “Okay, here’s this important architectural piece, and here’s why you should care about it.” And everybody would be sort of zoning out because they just didn’t have enough prior context to appreciate why it matters. I mean, I think sometimes later on in life, people go, “Oh, hey, I saw that once. Now I understand why it was important,” but it’s hard to do that on the spot.

MARTIN: Totally agree. We can prime students to be completely raptured and excited. I saw that all the time with photographs and other pieces of art that they would experience only in books and then go see these larger-than-life-size things in front of them, that had only been 8 by 10, or 5 by 7 pieces of image on pages. And like you were talking about earlier… audio sources, so, like reading a poem yourself or having it read in class, and then hearing the poet read it… completely different meanings… and you’re completely blown away. People laugh at me because… I’m just going to go to this place… and this is a stupid thing. But, I always make this argument to my teenagers, “You should see the movie before you read the book, because if you read the book: first, it’s gonna ruin the movie; and if you see the movie first, it only makes the book that much better, because there’s so much more in it. And I’m gonna stand by that argument. I think it works.

SUSAN: I see exactly what you’re saying. I mean, I think what that speaks to is kind of layering sensory experiences together as a way of making them the most profound. I get that

John: More generally, we try to integrate new knowledge with our existing knowledge,and we need some sort of structure, some type of scaffolding to tie it together. And I can see that case. I’m not sure I’d make that argument about always watching a movie first. But, I can see the value of that. And if you re-read a book, you notice a lot of things you don’t notice the first time, in part, because you have that larger framework and structure. And I think that can be applied, to some extent, to learning in any discipline, because no matter what discipline it is, you’re trying to help students develop the ability to have focused attention on what that disciplinary lens has, in terms of what is important within that approach to viewing the world. And people need to be trained. And I think in any of these things, students come in and start learning a little bit and they notice some things. But if we want to continue their development in the discipline, we have to provide more scaffolding to help them learn to appreciate or learn to focus on more detailed things within the world around them. And I think that’s a process we need to work on, no matter what discipline we’re working on. And tying in more senses to that I think could be helpful. Just as an example that I think Rebecca and I can refer to, maybe need a little bit more so. When we first started recording podcasts, if we had a 20-minute podcast, it would take maybe an hour for me to edit it. And then now I’m spending about maybe 12 times as much time, maybe 20 times as much time editing many of the podcasts, because, initially, you just go through and you take out the obvious issues, but then you start noticing more things, you start noticing the sibilance after you’ve leveled things, you start noticing more background noises that you wouldn’t have noticed. earlier before we started recording. For the first year or so of our podcasts, we were recording in a place where there was a toilet flushing and sinks running all the time, and doors closing, and a coffee grinder and a blender. And at first, we didn’t really notice that because it was part of our everyday life. But the more I focused on the audio, the more those things jumped out. And that’s what we have to train our students to do in any discipline. In economics, what I try to do is help students see things in the world that they wouldn’t have otherwise noticed, it was just part of their environment. And sometimes I’ve had students do video projects where they actually go out and analyze behavior. And that type of experience of looking at it with this different lens helps them see the world differently in ways that essentially transforms their view of the world from that point onwards.

SUSAN: I’m so glad you refer to economics there because there’s a perfect example of a discipline where you’d say, “Okay, I don’t know how this connects at all, right? And you can definitely see how shifting their perception by paying attention to different things, noticing different things, is grasping the concepts that they need to learn in order to understand economics. But it’s also, I think, just really important to remember that perception is an embodied process. It’s hard to make that happen by just sitting still at your desk and listening quietly.

Rebecca: The other thing I appreciate about thinking about object-based learning and sensory experiences is that it reminds us that objectivity actually has a point of view, tight? [LAUGHTER] We often think that there’s no bias in objectivity, but it does. And it really brings the subject to the forefront in that there is subjectivity to everything that we experience around us and actually gets us to pay attention to that subjectivity rather than thinking that you follow some design principles and somehow you’re being objective and doing good work, rather than thinking about what that actually means as an experience of something.

SUSAN: Yeah. And I think a challenge about teaching as we become more and more expert at what we notice, is that it takes a lot of effort for us to remember what it’s like to be a novice, and I think that’s a source of a lot of grumbling and frustration among senior faculty. We teach new students all the time, but over decades, it can feel like “I have told them this 50 times already, why are they not learning it” …because you have said it 50 times already, but you haven’t said it to the same 50 sets of people.

Rebecca: It’s a good reminder. [LAUGHTER]

MARTIN: You do have to say that with each set of new students,

SUSAN: it can seem sort of shocking, sometimes, when you’re an expert at something that people can’t see what you can see.

John: I know I have had that experience where I’d just say something in class and I said. “Didn’t we just talk about it?” In the same room, I had, but it was a semester before. [LAUGHTER] Oh, yeah. Yeah, we do feel like we’re repeating ourselves a lot, but we have been over many, many years,

MARTIN: I’m coaching my faculty right now in using or applying the Transparency in Learning and Teaching framework that Mary-Ann Winkelmes has been talking about for a while now. And, hear of that is writing your assignments in a way that makes it possible for students who are not native to your discipline to understand what you’re talking about. So that’s in a document. You don’t have to say it 100 million times because it’s written and if they have questions they ask, but it’s transparent from the get go. Like this is what I’m breaking it down in a way that somebody who’s not like me is going to be able to understand.

John: And I think that’s especially important in a world in which we may end up doing more of our instruction asynchronously or online… where in the classroom, if you come up with explanations that aren’t quite complete, students can ask questions right then. But if you’re doing something in an asynchronous online environment, students are kind of left out there on their own. And it is especially important that we have detailed instructions that will fill in those gaps. And that you have a mechanism where students can ask you easily and get quick responses, either ask you or ask other students so that they’re not left out on their own trying to figure out what you meant, when it was perfectly clear to you, but it’s not so clear to a novice. And I think one of the things you mentioned in our earlier podcast with you that sharing this with colleagues and other disciplines might be a good way of getting that sort of feedback, where if they can figure out what you’re asking people to do, then students would be able to.

MARTIN: That’s very true.

SUSAN: My students always do a good job of letting me know where I haven’t been clear. [LAUGHTER] Even when I feel I have made the TILT so explicitly detailed, I’m always surprised.

John: I know in faculty development workshops, sometimes we’ll explain something which, because we’ve been talking about these things so much, it makes perfect sense to us. But ,then we have to go back down a little bit and explain what assumptions we were making and what the basis for that is. Because, when you’ve said the same thing many times, it’s easy to forget that people may be new to some of the concepts.

SUSAN: That’s right. And I appreciate what Jim Lang has, I think, tried to do with the series that he’s editing, which is about books written by human beings, for other human beings, is to try to get away from language that could potentially be offputting to people who really do care about their teaching and want to improve, but are a little resistant to talking about alignment, or maybe the other terrible “a” word, assessment.

John: Susan, could you tell us a bit more about your forthcoming book?

SUSAN: Yeah, so I’m super excited about my book coming out. It has been a year’s-long process for me. I’m not a neuroscientist, I had a chance to learn a lot about embodied cognition, which is sort of an emergent subfield in neuroscience and cognitive psychology. But it also borrows from centuries of philosophy. So Wittgenstein, for example, was interested in embodiment. And so it’s a work of integration. I’m trying to pull from a lot of different, maybe even an eclectic, set of sources in order to think about how… if we pay attention to the body… how does that change learning and classroom teaching in college? And so one of the first questions is, what is learning look like if it does not involve everybody coming in and sitting down in a chair? I’m sort of stimulated by thinking about how classrooms might be radically different by just turning inside out some of the things that we think of as normal. Why do we think sitting down in front of a desk is the way that we study something. I mean, just as an example of putting these things into practice, I’m standing right now, because my research convinced me, and as well as my lived experience, that we think better on our feet. And we think even better while we’re walking, which is why the peripatetics, the Greek philosophers walked as a part of their practice. So, it’s sort of a wacky book, it’s going to be for people who are willing to maybe try some unusual unorthodox things in a classroom. It asks us to pay attention to internal movement, as well as external movement and the senses, and then to think about our physical environments as well. So, I have a section on learning outdoors and thinking about the space of your classroom. And one of the things I lament about the age of PowerPoint is that we often walk into a room and it’s been turned into a cave because everyone pulls the shades down immediately, so that you can see the light of the screen better. And I mean, there couldn’t be a worse, less stimulating, mind-opening environment than a bunch of chairs facing a screen in a dark room. So, those are the sort of assumptions that that book is questioning and ways to kind of shake it up and follow what we’re learning about the brain to be better teachers.

Rebecca: I can’t wait to read it.

John: When is that coming up?

SUSAN: It will be out in spring 2021. I think it’s going to appear in the fall catalog from WVU press. So probably we can start orders in the fall.

Rebecca: Yay.

John: Excellent.

SUSAN: Yeah.

John: And Martin, we talked a little bit about your book in an earlier podcast, but could you tell us a little bit more about when that’s coming out?

MARTIN: So, it’s just for this podcast, in case folks just don’t listen to the other podcast, but listen to this one, the project I just briefly mentioned earlier, where I make photographs, of faculty teaching, that is the project that’s behind the book that Cassandra Horii and I are working on together right now. She’s the Center for Teaching, Learning, and Outreach Director at Caltech. So, she and I have been working on this project together for quite some time. I’m making photographs, we’re using the photographs to talk faculty about their teaching afterwards. The working title is What Teaching Looks Like: Post-Sscondary Education in America. And what we’re doing is really, we’re writing a series of essays, 10 in total, and then there are 10s of thousands of photographs that we’re condensing down into about 200 or so final pics that we’re actually using to illustrate the things that we’re talking about in educational development so much these days, including object-based learning. So, for example, those photographs I mentioned earlier, handing around rocks in a geology class, students poring over primary texts in Princeton in an archive. Those are the kinds of photographs that we’re showing in this book. So, that should be out next year.

SUSAN: I can’t wait to see that. Martin, I almost feel like maybe we can get our books shrink wrapped as a set, because I was lucky enough to be able to include some illustrations in my book. I can’t wait to see your pictures because it was really hard for me to find pictures of anything except students sitting down in desks all looking straight ahead. Like, that’s what the picture of teaching has been. But it sounds like your book is going to do such an important job of awakening us to what else it might look like.

MARTIN: So, we’re just blowing the lid off the stock photo industry in higher education. [LAUGHTER]

John: Excellent.

Rebecca: Yeah, I’m looking forward to both of these books, for sure.

John: Me too.

Rebecca: We always wrap up by asking, what’s next? You already talked a little bit about your books, but we didn’t ask our actual question of: what’s next?

MARTIN: What’s next, in reality for me is, while I do have a check-in with Cassandra tomorrow to talk about some of the essays that we’re writing for this photobook, the immediate pressing thing for me is preparing the faculty that I serve to teach online or continue teaching online throughout fall semester, and really, it’s a heavy lift, but I don’t want to make it sound like it’s too much of a drudgery to do that, but we’re preparing in actuality, and everybody’s doing this, for a semester that we don’t fully know yet what it’s going to look like. It’s frustrating. But, that’s what’s next, really.

Rebecca: Sounds like a good time.

MARTIN: Yeah.

SUSAN: Yeah. [LAUGHTER] I’m feeling that too. I mean, obviously, this has been such an intense period for faculty developers, I mean it’s sort of sinking into me more week by week that not, just within our own little communities, but the general public. I mean, there’s pieces in the New York Times now. I mean, they get it the general public goes, “Whoa, this whole educational enterprise, it’s experiencing some really challenging re-envisioning at the moment,” and so it feels like we’re doing really important work, but it’s hard. So to answer the question, “What’s next for me in that arena,” I’ve been pursuing a coaching course this summer in order to be more effective at one-on-one faculty development and helping people to set goals and pursue the things that will make them feel more fulfilled as faculty members, not just in the teaching arena, but in terms of their research and scholarly and creative activities, the service that they do for the institution… just being more intentional, I think, about carving out our careers. And coaching is a field that, it hasn’t been used much within higher ed, but I think has a lot of potential to help everybody.

MARTIN: What course is that Susan?

SUSAN: There’s a number of them. It’s certified through the International Coaching Federation. So, the coaching organization I’ve been taking the class through is called the Center for Coaching Excellence. It’s based in Minneapolis, actually. And so they offer a series of certification programs. And it’s been a real challenge. I mean, writing the book was really growing into new territory for me, and this is really new territory as well. It’s learning how to ask powerful questions. And so I’m still feeling very novice.

Rebecca: Feeling nervous is a good thing for developers to be feeling as we’re helping faculty go into new territory. [LAUGHTER]

John: And I think we’re all novices in many of the things we’re entering into this fall.

Rebecca: Well, thank you both for joining us today and the really powerful work that you’re doing and the conversations that you’re bringing to the table.

SUSAN: Thank you so much for the opportunity. I’m super excited to be on your podcast.

MARTIN: Me too.

John: We very much enjoyed talking to you and we look forward to seeing your work.

SUSAN: Thank you both. Thanks, Martin.

MARTIN: Thank you all.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Ryan Schirano.

[MUSIC]

145: Pedagogies of Care: Ungrading

This week we continue a series of interviews with participants in the Pedagogies of Care project. In this episode, Dr. Susan Blum joins us to talk about ungrading as a method to support and motivate student learning. Susan is an anthropologist at the University of Notre Dame and the author of several books and articles on higher education. Her newest book, Ungrading: Why Grading Students Undermines Learning and What to do Instead, will be released as part of the West Virginia University Press series on teaching and learning in December, 2020.

Show Notes

  • Blum, Susan (2020). Editor.  Ungrading: Why Grading Students Undermines Learning (and What to do Instead). West Virginia University Press.
  • Pedagogies of Care
  • Blum, S. D. (2016). ” I Love Learning; I Hate School”: An Anthropology of College. Cornell University Press.
  • Blum, S. D. (2017). “Ungrading.” Inside Higher Ed. November 14.
  • Noddings, Nel (2010). Caring in Education. Infed
  • Sackstein, S. (2015). Hacking assessment: 10 ways to go gradeless in a traditional grades school. Times 10 Publications.
  • Arcidiacono, Peter (2020). Differential Grading Policies. Tea for Teaching podcast, February 26. (the podcast that John referred to that discussed women and underrepresented minoritized groups in STEM classes)
  • Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of personality and social psychology, 77(6), 1121.
  • A Theory of Public Higher Education
  • Society for Values in Higher Education
  • School Stories

Transcript

John: This week we continue a series of interviews with participants in the Pedagogies of Care project. In this episode, we talk about ungrading as a method to support and motivate student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Susan Blum. She is an anthropologist at the University of Notre Dame and the author of several books and articles on higher education. Her newest book, Ungrading: Why Grading Students Undermines Learning and What to do Instead, will be released as part of the West Virginia University Press series on teaching and learning in December, 2020. Welcome, Susan.

Susan: Thank you so much. It’s a pleasure to be here.

John: Today’s teas are. Are you drinking tea?

Susan: I am drinking tea. I’m a tea drinker. I love the name of your podcast and I started my day with Mountain Rose Assam tea with milk and sugar. But now I’ve moved to Light of Day Organic Green Jasmine tea from Traverse City, Michigan.

Rebecca: It sounds like a lovely morning.

Susan: It’s as good as we can have during the pandemic.

Rebecca: It looked like you were drinking out of a lovely cup too, actually.

Susan: This is a Chinese made cup with lids that I’ve had for 35 maybe more years and I’m a China specialist by training and when I first went to China, and everybody was drinking out of covered tea cups, I came home and I thought I had to get some myself. So this is chipped and old, but it’s precious. So, thank you for noticing.

John: Very nice.

Rebecca: Yeah, thank you for describing it too. Sometimes we see things… We don’t always communicate all that to our listeners.

John: The visuals don’t translate well on a podcast.

Susan: It’s a white background porcelain mug with blue dragons and clouds and fish.

Rebecca: Yeah, it attracted my attention the second I saw it with your cup earlier. [LAUGHTER] I’m drinking Scottish breakfast tea and I haven’t quite decided what the difference between the breakfast and afternoon is. So I’ll have to report back next time.

John: That’s right. You were drinking Scottish afternoon before. I think the breakfast tea is supposed to be fairly strong. I’m not sure about their afternoon.

Rebecca: I’ll let you know if I can’t sleep. [LAUGHTER]

John: And I’m drinking ginger peach green tea today.

We’ve invited you here to talk about your contribution to the Pedagogies of Care project and your forthcoming book on ungrading. First, could you tell us a little bit about what prompted your interest in upgrading.

Susan: Well, for over a decade, almost two decades now, I’ve been investigating education. And I do that as an anthropologist. So, there have been a lot of dimensions of my inquiries. I began really thinking about plagiarism, which comes in part from work I had done previously on deception. And that comes in part from my own training as a linguistic and cultural and psychological anthropologist. So the plagiarism work made me really wonder what students were doing in school, what their purposes were, how they felt about it, what motivated them, and so forth. And that led to more research on student experience in college and what the purpose of college was. And that led me to really question what we were doing in the classroom and how we were actually meeting students, given what they need or what they want, or what we think they should want, which is a kind of strange conundrum, and how all of this fits into more general ways people grow up and become adults and are socialized into their societies. And so clearly it has to do with issues of social structure and social values and power. And when I think about power, I think about agency and I wonder who has the agency in learning? Is it the students? Is it the teacher? Where are the topics being generated? What is motivating the learning at all? What kinds of ways can we build on innate curiosity and desire to be competent and responsible people in social groups? And how do our pedagogical practices support or even contradict and prevent some of what we actually want? So my more recent book called I Love Learning, I Hate School: an Anthropology of College really explored a lot of the contradictory dimensions of what we claim we want and why those things don’t really work. And students are pretty aware of a lot of these things. So I really explore what I call and others call “the game of school” where if everybody’s going through the motions and the outcome is just a set of points and the learning…. it’s nice if you get it, but you don’t have to, you can sort of cram some thoughts into your head and do well on a multiple choice exam and get the points at the other side. Learning doesn’t happen. Coercion, fear, anxiety, lots of negative things happen. And that seemed to me to be tragic. It’s a waste of time, money, effort, and it doesn’t have to be that way. So I have been engaged for at least a decade in really rethinking my own pedagogical practices from top to bottom. You know, what do I teach? Why do I teach it? How do I teach it? What do I do? What do the students do? What do they do alone? What do they do together? And grading ended up being one of the kind of threads that connect to all these different dimensions of things. And I’ve also been part of a research project to study student learning in an internship. And there were no grades, but there were authentic outcomes of their practices. And so, I’ve been trying to make my classes as authentic as possible, rather than something people are doing simply for performance of competence, but to actually feel competent themselves. So, grades are thought to have three functions: sorting (which I reject), motivating (which we know doesn’t work), and communicating (which also doesn’t work). So I’ve tried to figure out how to make co-operative classrooms where everybody learns as much as they possibly can, for their own purposes, not for me, and I try to have students help generate their own goals so that they see this is not simply a task to be checked off, but as something that matters to them. I mean, I’m spending my life doing this work, and the thought that it’s just something to check off and get out of the way till they can get on to the real important stuff was very galling to me and actually, frankly, almost made me quit, which is kind of the topic of my next book. But the idea that I could actually change something that everybody thinks is central was so liberating to me. And it has really transformed the way I’ve been teaching. And so I’ve really been very pleased with the outcome. And since I published a short piece on Inside Higher Ed in 2017, I found that there are thousands of people at all levels of education, who are engaged in ungrading, throwing out grades, degrading, we call it different things, but we’re all engaged in the same enterprise. So editing this book… I want to be clear. I’m not the author of this book, I’m the editor. I have written the introduction and conclusion and a chapter but about 15 other people have also contributed to this book. And it’s been so gratifying and reassuring and stimulating and refreshing to know that all these other people are engaged in this too from all different directions. So I’m very excited about getting this out into the world so that we can provide some support and reassurance for people who might be interested in doing this but aren’t really sure how to make it happen.

Rebecca: Authentic learning is something that I’ve been really interested in for a long time. And so ungrading has always been really interesting to me, but I haven’t quite gone all the way there yet. And I’m certainly wanting to experiment in that space. Can you describe for folks what on grading look like and how that shifts the focus to learning?

Susan: Sure, and there are ways to do it partially or fully. So I’ve gone to total ungrading until the end of the semester when I am obliged to give a grade for my class. I wish I didn’t have to. I don’t actually think it’s meaningful or informative. But I’m required to do that. So that does happen, and I can tell you a little bit about how that happens, too. But ungrading really means you talk about what people are learning, maybe you have a conversation about what they’ve done well, what they haven’t done well, some things don’t actually have to be graded at all. We don’t have to assess everything. That doesn’t have to be the central activity of our teaching. And there can be what Nell Noddings refers to as free gifts. You can have people have experiences in the classroom, and the outcome is the enjoyment and the learning. And so that is its own reward. And if people perceive that they have been satisfied in their learning, then that’s an assessment. And you don’t have to translate that into some sort of numerical or letter reduction of what is, we hope, a fully human, rich multisensorial experience. I taught a class on food and culture last semester, which is a really fun class to teach. And students did activities, many of which they generated. I didn’t dictate everything. But one of the classes wanted to push one of the topics which was on technology and food. And they wanted to see what tools people use for cooking. So they had this idea that they would take pictures of what was in their kitchen drawer. This was before the pandemic. So take pictures of what’s in your drawer. So we talked about what was in the drawer. And then they had the idea that they would ask somebody older in their family to take a picture of what was in their drawer and talk about it. So then they had the idea that they could interview people about this. And anyway, it was wonderful as an experience. They interviewed their grandmothers and their mothers about what has changed and why do you have this tool? and what is the tool? …and we had so much fun talking about it, and everybody learned everything and it wasn’t graded. It just wasn’t graded. Because who wouldn’t want to do that? And so the motivation was purely intrinsic. And the assessment was when their classmates said, “Wow, that’s so interesting,” or when their grandmother said, “Wow, it sounds like you’re learning interesting things in school.” And so the measure of the outcome was part of the experience. And there was no need or use for anybody else to assess it. So that’s one type of grading is just not grading. You’re learning something, you’re enjoying it, you’re sharing it, and that’s what we’re here for. So there’s no point in doing more than that. But there are other kinds of assessments that are appropriate sometimes, and so for the assignments that are major assignments in my classes, I have the students include with their assignment, a self assessment. And these self assessments used to look a lot like grading, but they don’t anymore. They used to look like: “I did this right. And I did this right. And I had enough sources and I used the proper format. And I did this and that.” And then it was kind of a rubric where you could add things up. Now it’s much more: “What did you learn? What did you do well? What didn’t you do well? Why didn’t you? What do you need help with? What would you do differently? What are you taking with you?” So, it’s a reflection. So, it’s an assessment, but it’s much more of a reflection, which fosters metacognition, which we all say we want. And until this year, I had “adequate,” “not adequate,” and “exceeds expectations” or something which still kind of translates into like F,C and A. Now I just say, you know what you’re doing or you don’t know what you’re doing. And so sometimes in classes where things are new and hard, I teach a linguistic anthropology class where I have students do sometimes very difficult projects: ethnography, conversation, analysis, all kinds of stuff that they’ve never seen before and they admit is difficult. Sometimes they can say “I didn’t do this well.” And because it’s not a grade, there’s nothing at risk for them to admit that they actually haven’t quite felt secure about it. And that’s helpful information for me. It’s very honest, then we can say, “Well, actually, not that you know how to do this. And that’s okay. And we can work at it more, or I don’t expect you to because it takes two decades to master, or whatever it is.” So, then when I return their projects, I reflect on what they’ve done as their project, and sometimes I also reflect on their reflections. So there are a lot of layers here. So, that’s some of what ungrading looks like.

John: Since this relies on intrinsic motivation, what do you do to help build that? I imagine some classes students will come into them with a great deal of intrinsic motivation and in others that they see as just a gen ed requirement or something… a hoop that they perceive as a box they have to check off, which is something that, as you said, always bothers us. How can we perhaps help build that intrinsic motivation in classes when students are there when, as they perceive it, they’re just required to.

Susan: So I teach fundamentals of linguistic anthropology class, which counts as a social science requirement. So I get a lot of students in there for their gen ed requirement. It also counts as something among a set of choices for the major, but it might not be that they’re inherently interested in the topic. I personally think that everybody’s interested in language and communication. And everybody can become interested in anthropology, which is the study of people, but I don’t take for granted that they’re interested in it the way I’m interested in it. So, in that class, in particular, I have spent a lot of time really tweaking every dimension of the class, from the way they sit in the room, to who speaks first every day, to getting to know each other. I try to introduce play and fun. And I have teams and snack teams and students bring in interesting things for themselves. And anyway, this is really my laboratory where I work on how to create experiences that may allow students’ intrinsic motivation to flourish. Because I don’t think I produce intrinsic motivation. I just create conditions for it. So in that class, I now spend a whole week before we even get started just inviting them to ask big questions, to take charge of their learning, to think about what they’re curious about. Sometimes they work in groups that then they have a responsibility to each other. Also the social dimension… sociality, we know is part of it… I spent a lot of time thinking about the emotional and affective dimension of learning. I try to find really interesting things to do and read and try to connect it to their lives. Students are doing a lot of observations of things that are happening around them, which many of them have never done in an analytic or critical way; they’ve only done it in a reactive way. So, I think there are lots of ways to connect students’ own experiences beginning where they are, not with a deficit perspective, but with an asset perspective. You know, what do you know? What do you care about? …and then connect what we’re learning to something that you want to know more about? In that particular class. I have people write linguistic autobiography, and many of them say, “Well, I just speak English. I grew up in America.” And by the end of the semester, when they go back, and they look at that assignment again, they realize “No, there’s actually something to say because I speak this kind of variety…” and there are a lot of things to do that connect to students’ own lives that still get to the material. I’m not shirking my responsibility, but I also think there are lots and lots of ways to get there and they don’t all have to get to the same place. That’s perfectly fine with me. So, those are some ideas.

Rebecca: You talked a little bit at the very beginning about ungrading throughout the semester, but then, at some point, there is some authority that’s requiring a grade. Can you talk a little bit about how to negotiate that?

Susan: this was something I really worried about for years. And then in the summer of 2016, I came across Starr Sackstein’s book Hacking Assessment. She’s a high school teacher, and she has a chapter in our new ungrading book, and she talked about how to go grade free in a conventional school. So, that gave me confidence and ammunition to try to figure this out. So basically, I asked the students to suggest a grade for themselves. I have conferences, I actually try to do the mid semester and semester final to just say like, “If you were going to give yourself a grade, what would it be and why? What’s the evidence?” I’m not that fixated on the grades anymore. grades for me have become such uninformative flat measures of student experience that I find them very maddening. So if I had a student who came in who had never encountered the discipline before, and got very excited and tried some new things and didn’t do that well at those things, but learned a huge amount, to me, that’s a great accomplishment and a great gain, even if their paper wasn’t as good as somebody who’s a senior anthropology major whose paper it’s flawless. I want to say that both of them have had great learning experiences. And if they both say they earned an A, because they learned a lot. I’m actually okay with that. And I know one of the questions people always have is: “What about the engineers who design our bridges? What if the bridges fall down because one person learned a lot but they still don’t know it?” And one of the things we’re really excited about in our ungrading book is we have STEM teachers. So they are talking about what it’s like to teach computer science or math or chemistry and use an ungrading approach. So it can be done in slightly different ways. But for me, because I’m trying to get my students to see the world, reflect on it, analyze the world, that’s what anthropology does. If I get them there, then I am completely happy to give many of my students As. They don’t all ask for As; they don’t all think they’ve earned it. They come in with different standards and experiences about what grades mean anyway. and international students tend to have very, very high expectations for themselves. So they suggest well, modestly, “I only earned a B minus” but I might say that they really demonstrated great learning and accomplishment and it might be harder if they’re not a native speaker of English. So, I may bump it up. I can bump it up or down. They’re suggesting great, but usually, they’re pretty honest. And they learned a lot. They’ve worked hard. And I usually do accept the grade that they suggest.

Rebecca: How do you see the role of reflection and revision as part of the ungrading process? You mentioned handing in an assignment with a reflection, and then you reflect on all of that. What do they do next? Is revision or iteration a part of the practice?

Susan: It depends on the course. I’m teaching a writing course for graduate students again, and revision is obviously the heart of writing. Anybody can revise anything they want in my classes, and I’ve had students say, “I turned this in, but I procrastinated and I couldn’t really get it done, and I’m just not proud of it. And I’ll say, “Would you like to redo it?” And they’ll look at me like “What? What do you mean? I get to redo it. I’m not like branded as a failure my whole life?” No, if you want to redo it, I’m happy to read it again. I try, depending on the course again, to have things build on each other so that even if they’re not literally revising that assignment, they’re recognizing gaps or deficiencies or weaknesses or strengths that they can carry forward to future work that might rely on what we’ve already done. But I have not, at least in this laboratory class that I’m talking about, I haven’t really had one overall semester-long project. I have thought about doing that, and I haven’t done it yet. That could be something I do next spring. If we’re back in pandemic-ville, I may revise things completely again, just because why not?

Rebecca: I’m thinking about ungrading in the realm of, in the design world, doing sprints, so doing one project that builds on it, but having really distinct chunks that you get feedback on and can keep revising all semester. And so it definitely is in that same spirit. So I’ve been wrestling with how to completely implement that.

Susan: Well, I think in a skills-based discipline, there are certainly skills that you need to try and not be good at it first and then get better at. And that’s how we learn anything real. And it seems obvious to me now that punishing people for not knowing at the beginning is the wrong thing to do. So, having only the final product evaluated seems appropriate to me. But I know design… there are some things that people might all agree on, but there are other things that people don’t all agree on. And that’s true of real life. That’s one reason that a single scale of grading is such a distortion of how we really live our lives. People might make a movie and some critics love it, and some critics don’t love it. And to pretend that there’s a uniform single scale is to deny most of our actual experience outside school.

John: One of the things you mentioned in terms of international students is that they often underestimate the quality of their work. You also mentioned in your recording for the Pedagogies of Care project that some underrepresented groups in particular disciplines often experience the same problem. And we had a podcast recording related to that a while back that talked about how women and underrepresented minorities did as well in their introductory STEM classes, but they were more likely to drop out because they didn’t perceive the quality of the work as being sufficiently high. And that served as a major barrier. And I’m wondering how you address the issues of students who undervalue what they’ve learned or underestimate the amount of learning they’ve achieved. When you’re meeting with students and providing feedback and they undervalue their work, how do you address that with them?

Susan: Well, that’s where I’m grading is so perfect, because I can have a conversation. I have these short individual meetings with every student at least twice a semester. And I can say to them, especially if mid semester they say this isn’t very good, because I’m not smart or my grammar’s bad or I didn’t do this before or something, I can say to them face to face, or at least it used to be face to face, I can say, this was extraordinary. I loved what you did, this was such a contribution. So, I can just personally affirm their value and say, you might be focusing on this, but also notice this wonderful thing. And because also, students are constantly interacting in my classes, the students who may have fewer privileges coming in may get a lot of affirmation also from their classmates for their offerings. They may be quiet, they may not be willing to speak, but I try to make people comfortable, at least in small groups or pairs or something, so that they can make their contributions. So, I think it’s less of a problem when people can actually reflect and get comments back. Also, sometimes students exchange papers or exchange work, I tried to have an authentic audience, so that I’m not the sole audience so that people aren’t writing for me, but they’re writing maybe for real people. That’s something I’ve really tried to develop more. I think that when students read each other’s work, they tend, at my school anyway, to be very nice to each other. So they will get some kind of compliment. And I think then in that sense, there’s less of a potential for people to retain this idea that they are somehow deficient. But I also would like to say that schools can’t solve every problem. And teachers and classes, even the best, can’t solve every problem. And so we have broad racism and sexism, and ableism, and all kinds of other things in our society, and one particular teacher might make a difference, but these are really bigger questions that we need to address outside school also. As a professor, my realm is in my classroom, so I can try.

John: At the other side, what about the students who’ve come in who’ve read some material on the topic and have this fluency illusion where they perceive that they’ve learned it very well. I’m thinking of the Dunning-Kruger effect where the people with the lowest level of understanding often overestimate their competence the most. How do you address those issues?

Susan: I’m not so worried about that, really. I’ve had experiences like, that where students think that they’re kind of expert, and they’re not actually, but I don’t really see my role as like cutting down students’ confidence. I think there are enough forces out there trying to do that. So, I don’t really want to jump on the confidence destruction bandwagon, but I like to think that there will be some kind of real consequence where they will say something to somebody who knows more and that person will say,”Yeah, but X” or where they will interact with another student who will know more, and then the student who has this false sense of their own abilities will realize “Wow, I only noticed these three things and Julia noticed 25 things. I guess I have more room to grow as an ethnographer.”

John: This system, though, relies on intrinsic motivation. And you’ve mentioned using authentic assessments, also perhaps, to help build that. Could you describe some examples of authentic assessments that you use?

Susan: Sure. So in this linguistic anthropology class, one of the projects is in groups of three-ish, they have to create some kind of presentation about a particular language. So something they’ve heard of like Hindi, or something they haven’t heard of, like, I don’t know, Tzelta or something like that. And I give them some things they are supposed to include, but the form is completely open. So, I’ve had infographics, I’ve had lots of websites, I’ve had PowerPoint things. And one time, it worked really great… and I’m in this weird classroom that I like with a bunch of screens. The room is imperfect but they’re five screens around the room. And so I happen to have 10 groups that year. So the students plugged in their laptops and the other half of the students circulated and listened to the students as they were talking about their project. And then on these whiteboards that were next to the screens, they were writing praise, questions, suggestions and different kinds of questions. They had to figure out what kind of question it was: Was it a kind of application question? a factual question? or something like that. And the students really loved that project, because everybody saw what everybody did. And the assessment was basically peer generated. It was: “I liked this image.” “This was a really clear presentation,” or “I didn’t really understand what you meant here.” And so that’s assessment. It doesn’t look like assessment. It doesn’t say good and bad, but it’s getting feedback about what you’re doing that you can take with you. So if somebody says, “I couldn’t read the italic font,” next time, presumably they won’t read the italic font. But they’ve had 30, some people responding to their work, which is such excellent feedback, and so much more useful and meaningful than me just sitting there with something and writing a few things.

Rebecca: I like that poster session model idea. It’s a lot of fun and I think students really do respond to peer feedback in that way. I know I’ve been really successful when I’ve done class sessions that are like that poster session or fair-like atmosphere with those same kinds of categories to fill out, I think, is really super helpful. I’ve had really good experiences with that, too.

Susan: One of the things we’re all thinking about is how to translate all this physical stuff online. And you can. Like this past semester, that project ended up online. And so I had a Google Doc, where people were doing the same thing, and it worked fine. It wasn’t as fun as running around the room, but it was effective.

John: I’ve done the same thing the last couple of years in my econometrics class where students create posters, half of them present one day and half the next class day and I give a break, and a group of them can wander around and see the others on the days when they’re presenting, and I’ve invited members of my department. The Dean has come by in the past, and it’s been something they found so much more valuable than the PowerPoint presentations that they used to do, where they’d all be sitting there nervously, and then getting up and being glad to get through it, and then they’d sit there quietly waiting for the rest of them to be done. There’s so much more engagement when they’re up there presenting for the whole class period to anyone who happens to wander by. And it’s a form of a more authentic assessment, I think, that they value quite a bit.

Rebecca: …builds in more practice, too, because they’re talking about it multiple times. [LAUGHTER]

John: Yeah, having them talk for an hour about their project is much more effective than presenting to a silent audience much of the time. I liked it, they liked it, and they strongly encouraged that it continue.

Rebecca: So, we started talking a little bit about how to translate some of these things online. So, why is upgrading maybe particularly important to think about during this pandemic or in this transition to remote learning or the unknowns of the fall semester, as they currently stand? [LAUGHTER]

Susan: That’s a great question. We are in a very unpredictable moment. And every campus is trying to figure out what to do. The ones who are fully online have just made their claim, and so that’s a little bit more predictable at some level. The ones like mine that are committed to in-person except for exceptions or hybrid until we can’t do it anymore.

Rebecca: It’s a familiar story. [LAUGHTER]

Susan: Yeah. And I read everyday about what everybody around the country is doing. And we don’t really know. So anybody who is sticking to a rigid grading scheme is probably going to keep recalculating and recalibrating all semester long… if you’ve got participation, but then people lose their internet connection because they are stranded somewhere, then what do you do? Do you just have a different formula for that person? I think having precision in grading schemes has often been seen as equitable and comforting for students because it gives them security knowing what they’re going to do, but that presumes that you know what the conditions are going to be. I think, even in ordinary times, there are a lot of fallacies built into that and students’ conditions aren’t as uniform as we often assume they are. But, we know now during the pandemic, how widely and wildly variable people’s conditions are, and the New York Times has done stories about one student helping her family with a food truck and the other one is in the family second home in Maine, and there’s everything in between. There’s using the WiFi in the parking lot of the library or there’s using the WiFi in your beautiful six bedroom home. So the lack of uniformity just highlights all of the inequities and all of the unevenness of the conditions. So, if you’re sorting people, but they’re in wildly different conditions, you’re not actually doing a very good controlled experiment, and it’s certainly unequitable. Another dimension we should probably consider is that, in our current moment, everybody is experiencing some kind of stress and trauma. And the trauma-informed pedagogy is something that we all need to learn a lot more about. We know that one of the outcomes of grading in ordinary times (I don’t know what we’re going to end up calling this third condition) is that grades produce stress and pressure. Right now, with so many other stresses and pressures, we don’t really need grades to add to that. How we keep people accountable, how we keep them on track, how we keep them motivated, involved, connected to each other is really our challenge. And that’s what I think those of us who are really thinking about this are trying to spend every minute of the summer trying to figure out. But grading is not the best method for motivating people. So, I think that this is the perfect time to try ungrading.

Rebecca:So if we try ungrading, how would you recommend framing such things in our syllabi?

Susan: Well, I’ll tell you what I do. I have one sentence on my syllabus. My syllabus is not a contract. It’s not one of those punitive sort of legalistic syllabi. So, what we’re figuring out in this conversation is that everything is connected. But my syllabus has one sentence that says, “We will be practicing ungrading in this class, this will be explained.” And I begin most of the semester by having meaningful, enjoyable experiences where people are learning, and I don’t say it’s not graded, it’s just not graded. And then over weeks, I explain what oungrading is, and I show them this is what we’ve done, see how it works. And when I’m lucky, I have students who have had other classes with me and they can sort of support my claims that this is actually meaningful and they won’t just blow it off and they won’t just think it’s not important. I want to have an acknowledgement here before we end though that contingent faculty, graduate students, people of color, young women, people who are tenure track, people who are teaching lots and lots of classes, may not feel that they have the security to engage in something that’s unfamiliar. And it might be risky for some people. They may need to clear it with their chair or their Dean or somebody like that, who may say no, because it’s scary. That’s one of the reasons we’re trying to have this book so that a young contingent faculty member who really cares about pedagogy can say to the person who’s really holding their employment over their head, “Well, there’s research, too. Look at all these people who are doing it, they’ve done it, they’ve done it, okay, they’ve done it for years, and I would like to try it too. Can I try it in one class, maybe with a good outcome?” So, I don’t recommend starting from a completely conventional class last semester to a completely unconventional class online next semester. I think changing things bit by bit is probably the way to go.

Rebecca: I think that’s good solidI think that’s good solid advice, always: iterative practice with our classes. [LAUGHTER]

John: And you mean by that, perhaps, having some activities that are ungraded and then gradually expanding that as you become more comfortable and your department becomes more comfortable with that?.

Susan: I think that’s a great way of framing it. And it depends on the subject too. Some are much more amenable to ungraded, like writing or social science or something.

Rebecca: So we always end by asking: what’s next?

Susan: So, I’m part of a project called A Theory of Public Higher Education. It’s funded by Indiana University and the Society for Values in Higher Education and we are generating a theory of public higher education. We are going to be publishing our kind of manifesto. We’re finishing It this summer and fall, and it should be published next year. We’re very excited about that. It’s a group of six of us from all different institutions teaching all different subjects. It’s really led us to rethink what is higher education from the foundation up. Another project that I’m also really excited about is called School Stories. And I’d love it if your listeners would give it a look. You can find it at schoolstorieslab.com. And it’s basically crowdsourcing experience stories about being in school. So, it can be students, teachers, parents, administrators, it can be from any place in the world, from any level of school. Our only condition is that you have to be 18 to write the story, because otherwise, we get into problems. But we just launched last week, and we have worked on our web design, and we’ve worked on our IRB, and we’ve worked on every dimension of this and we’re really excited about it. There’ll be a new theme every week; this past week, the theme was racism. So what are people’s experiences of racism in school? We have a whole COVID sort of shell and context for what we’re doing now. So, please check that out. And then my next other project is a book I was writing really well until the pandemic hit. It’s about how your education, it’s called Progress Report about my own transformation in teaching, but it’s on hold right now, because I don’t know what to say, exactly. [LAUGHTER] I’m in a profound process of rethinking right now. So, I will write that but I don’t know what it’s going to be now.

Rebecca: It does seem like COVID-19 has transformed us all. We’re just not sure how yet. [LAUGHTER]

Susan: Right? I mean, we’re living through what we all perceive simultaneously as a huge transformation.

Rebecca: Well, thank you so much for sharing some insight into ungrading. It’s been an interesting conversation, and hopefully, it’ll provoke people to think a little bit differently about their plans for the fall and in the future.

John: Yes, thank you. We’ve really enjoyed talking to you. And this is a topic we wanted to get on the podcast for quite a while. So when we saw your contribution to the Pedagogies of Care project, it was an ideal match.

Susan: Well, thank you so much for your great questions and your welcoming demeanor and for your own little contributions to how to think about teaching, which I’ve kind of taken notes on, and to our listeners. Good luck to you and we’ll get through this.

Rebecca: We hope

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

144. Pedagogies of Care: Evidence Based Practices

This week we continue a series of interviews with participants in the Pedagogies of Care project. In this episode, Dr. Michelle Miller joins us to discuss how the use of evidence-based teaching practices can be an effective way of demonstrating that you care about your students and their success.

Michelle is a Professor of Psychological Sciences and a President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology research interests include memory, attention, and student success in the early college career. Michelle is the author of Mind’s Online: Teaching Effectively with Technology, and has written about evidence-based pedagogy in scholarly as well as general interest publications. She’s currently working on her newest book, Remembering and Forgetting in the Age of Technology: What the Science of Memory Tells Us about Teaching, Learning, and Thriving in a Wired World, scheduled as part of the West Virginia University Press series on teaching and learning, edited by Jim Lang. The tentative release date is 2021. She is also a contributor to the Pedagogies of Care project created by authors in this series.

Show Notes

Transcript

John: This week we continue a series of interviews with participants in the Pedagogies of Care project. In this episode, we discuss how the use of evidence-based teaching practices can be an effective way of demonstrating that you care about your students and their success.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Michelle Miller. Michelle is a Professor of Psychological Sciences and a President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology research interests include memory, attention, and student success in the early college career. Michelle is the author of Mind’s Online: Teaching Effectively with Technology, and has written about evidence-based pedagogy in scholarly as well as general interest publications. She’s currently working on her newest book, Remembering and Forgetting in the Age of Technology: What the Science of Memory Tells Us about Teaching, Learning, and Thriving in a Wired World, scheduled as part of the West Virginia University Press series on teaching and learning, edited by Jim Lang. The tentative release date is 2021. She is also a contributor to the Pedagogies of Care project created by authors in this series. Welcome back, Michelle.

Michelle: Hi. It’s great to be here.

Rebecca: Great to have you back. Today’s teas are:

Michelle: I am drinking fresh mint and hot water, which I think is my favorite summer tea of all when the mint is thriving all around here at the house.

Rebecca: Sounds nice and refreshing. How about you, John?

John: I’m drinking Tea Forte black currant tea.

Rebecca: And I’m drinking Scottish Afternoon. I haven’t quite run out of that yet.

John: We’ve invited you here to talk about your contribution to the Pedagogies of Care project and your forthcoming book. Could you start by talking about your contribution to the Pedagogies of Care project?

Michelle: Right towards the end of the spring semester for many of us, as you know, we in the teaching and learning community and professional development and scholarship of teaching and learning space, were in just vibrant discussion with one another, just talking each other through the experiences that we were having as part of the pivot to emergency remote instruction, which I think for most of us in higher education, that was a big part of what we did in March all the way through May of 2020. So we’d been talking about these and there’s this very vibrant group of authors that have come together under the West Virginia University Press’s project, as you mentioned, edited by Jim Lang. And so we had this group, which was already exchanging very rich sets of advice and ideas about where we were going and really talking about how to help. And so under the leadership of Tori Mondelli, who conceived of this whole project, and also Tom Tobin, who has also been a real leader as part of this group, we talked about how can we put together some resources that grow out of the work that we’re doing, that capitalize on some of the rich conversation and collaboration that’s already happening, and whatever format that takes, put that out there into the world, so that people can use that and there’s all different ways that it could be utilized. We’re not prescribing that but we really had envisioned something that was open, that was helpful, and that was really contextualized within this moment of real upheaval and crisis and new directions that many of us are involved in.

John: We’ve gotten some really good feedback. I shared that with the faculty at our campus just a few days ago and I got about a dozen responses within a couple of hours saying “These resources are really useful. Thanks for sharing.” We’ll include a link to that in the show notes. So, we went through this traumatic switch that was a bit of a struggle for everyone, students and faculty, what can we do now to better prepare for the fall?

Michelle: At the time that we’re recording this, we are, for me, about midway through the summer. So, it really is starting to get real, for many of us, what we are going to do in the fall. And we’re seeing more and more institutions who are firming up and starting to commit to real plans for what the format of instruction is going to be like, what enrollments are going to be like, and all those kind of locally specific pieces of information that are so important for determining what we’re going to be able to do. So, what can we do differently to better prepare for the fall semester? First of all, let’s honor that what the vast majority of faculty that I’ve talked to, what we accomplished in such a short space of time in spring, providing instructional continuity. This was amazing. I mean, we really enabled students who, in some cases, they were set to graduate, they were earning their degree in maybe a month or two, and we made it possible for them to get to that finish line through a tremendous amount of ingenuity and hard work on everybody’s part. So, let’s not sell ourselves short. That said, we are headed into a very different environment. And so what I’ve really suggested in some other things that I’ve written about and definitely in my Pedagogies of Care project is a focus on what does quality really look like? And for me, being a cognitive psychologist, social scientist, totally acknowledging that that’s my perspective… forr me that comes down to aligning with the best of what learning science has to offer. And the neat thing is that we are in an era right now when number one, we really have converged on a set of principles that are fairly non controversial, and if not always easy to implement, it’s fairly clear what we can be doing. And we have technologies, in some cases, that map onto them very well. They don’t do the work for us. But they can really help implement things and make things concrete that we’ve known in theory for a long time were very, very important. So, that’s one of the things that I think that we can focus on. So, there is that. I’ve also really emphasized the reevaluation that we won’t be able to simply do what we’ve always done. I think those of us who work in this space are always quite adamant that teaching, say online or teaching a hybrid course, is not a matter of just sort of capturing a lecture. If that were the case, this would be very, very straightforward. We should just lecture all summer, record it and post it, but that’s not what it’s really about. So, what I think that we can focus on as we do reevaluate, in our teaching, what are we trying to accomplish? We can step back and say, “Well, what do students want to get out of this?” And that I think can help us winnow down from all the things that we could potentially do. It will help us let go of some things that we will not be able to do. And help us find, if not an easy path forward, a more clear one that will allow us to serve our students and also take good care of ourselves during this time.

Rebecca: I think anything that helps us figure out what our priority can be, in terms of content or goals that we have for students, but then also methodologies that we’ll use and why, I think is key because I think we all need to scale back and be reasonable with ourselves because there is so much to accomplish if we want to do it perfectly. But we just don’t have that kind of time. You just said it was halfway through the summer and I almost had a panic attack.

Michelle: Right. Not that I’m counting but it is actually just about the midway through the summer. And you, know, when I started reflecting even more on this Pedagogies of Care concept, which is the kind of overarching ideal that we eventually rallied under as a group, it’s occurring to me that that applies to faculty as well. I mean, self care is a kind of a term that’s very cliched, and it gets kicked around, but I think that we also really do at this time need to be recognizing that, again, what we did, what we accomplished as faculty in the spring was tremendous, that it did require people working weeks and weeks and weeks, sometimes months without a break. And although summers are not really traditionally a break,or vacation for faculty in any conventional sense of the word, they are a time to recharge and for many of us were also taking care of research obligations and other things that went completely by the wayside for a while out of necessity. So we really do have to balance that too. What’s the degree of faculty burnout at this point? What’s the degree of faculty receptivity to brand new things. So, the things that we are looking at also need to be kind to ourselves. We need really good communication and collaboration more than ever before, I think, in university communities. I think that’s really also the thing that’s going to make this fall successful, is being able to recognize what faculty have been through and work with that. So yeah, I think that we should recognize this effort. And with that, I also think that evidence-based teaching, incorporating learning science and those principles… that ideally shouldn’t be yet another thing on the to-do list. I think that if that’s the way it’s coming across, then we’re going about it the wrong way. I mean, to me, frameworks are always a way to simplify. Again, we have this infinite landscape of things that we could do in any given class, all these different decisions to make and choices. We do have a framework for whether it’s learning principles or another framework… that should help and simplify. So I think it kind of fits in that big landscape of possibilities as well. That’s how I see it. It should help; it shouldn’t add to what’s becoming a pretty serious burden for faculty.

John: One of the things I’ve really liked in your discussion, as an economist, is you sounded at times, like an economist, when you were describing that, in terms of this is the most efficient way of helping students reach their goals… that if we use evidence-based methods of teaching, we can let students learn skills more efficiently without wasting as much time and getting closer to that point, making it a form of caring, I think, as you referred to it. That one way of demonstrating your care for students is by using techniques that are more efficient, that provide the largest return on students’ time… there’s the economics part coming in. So I really appreciated that. And I thought it was a really good argument that we tried to emphasize ourselves in our workshops.

Michelle: Oh, thank you. And you said it better than I possibly could have as a non-economist, but that’s exactly the core of that idea, that it is kind to students and perhaps it’s kind to faculty as well. We can pre-select some of these avenues and techniques that, if you’ve got an hour to study (and for many of our students, that hour of study might be fractured and jammed in among all kinds of caregiving tasks) that you’re going to get more from that. If, as a faculty member, you’ve got four hours that you can devote today to preparing for the fall… and as well, that’s going to be divided up among other tasks among your caregiving responsibilities… how can we cut to the chase for faculty so that they can make those choices? So I’m glad that that comes across.

Rebecca: I think it’s important when we are planning for the fall that we are getting down to those essential elements. Can you talk us through some of the steps that faculty might take to focus in on those essential items and the evidence-based practices so they can have a good framework moving forward, not just for the fall when they might be teaching remotely, and that’s what they’re not familiar with, but all the time?

Michelle: Coming down to essentials, and here too, I think, that that has really resonated with many faculty and also with instructional designers and others tasked with making all of this work. That’s what’s really resonated, like what are some of the essentials, and I’ll never claim to be able to I Identify the complete and exhaustive list of exactly what to do. But here’s what comes to my mind. I think that perhaps returning even to those learning objectives, which we may have put in a syllabus long ago, and they can be sometimes kind of abstract, but coming back to those and saying, alright, what does it really look like when students have achieved these? Are there any that need to be perhaps modified, or dropped altogether? So if we are going to have a semester of really focusing on essentials, this might be a good time to do that. Naturally, we will want to think about the content. And oftentimes we talk about in pedagogy and developing pedagogy, we talk about re-focusing away from just coverage of content, that’s something that a lot of us get behind. And it’s okay to be thinking about well what content is going to be in the course. But then really pivoting to look at what’s the engagement with that content? How are the students going to engage with the content and how are they going to engage with you? So that’s a piece of it, asking yourself that question. And I think then, starting to bring in those really concrete logistics. Now, again, typically those of us who talk about pedagogy a lot, we kind of discourage people from talking about very specific tools or technologies, until they’re really, really clear on some of those high-flown ideals of what they and their students want to get out of the course. But I think in this case, we probably want to hold off on th.t, we are going to have to say, “Well, are you going to be expected to teach online but synchronously? And if you want an example of that, the Zoom meetings, which we’re all pretty familiar with, at this point, where we’re in at the same time, but maybe you’re in a different place? So is that going to be a part of what you do with students? Because that is pretty new to many of us. And if so, there’s certain considerations you’re gonna have to have in mind say, ‘Well, how is that going to work?’” Especially, if you’re expected to also be teaching say, a face-to-face course at the same exact time, which I think is going to present challenges. And I think for many of us, it’s going to depend on your local institutional context, but I think you can’t go wrong right now with setting up a robust online component to your course. I think that with the level of uncertainty we have, or even with individual students… if they’re going to need to say quarantine or take care of an ill relative or something like that… having some asynchronous, so different time activities and materials online, is going to be essential. So I think taking those concerns and saying, “Alright, what is this physically going to look like?” I wouldn’t typically push that as much but I think that that’s important now. And I think in the preparation for this, too, another kind of bare essential point that I talk about in my resource for our project is media creation. So in some cases, people are going to want to create, say, a set of videos, or let’s say they’re demonstrating a process. Let’s say they’re teaching studio art. They might want to have some pretty involved videos or other kinds of demonstrations, or perhaps there’s not good written material out there that might replace a series of face-to-face lectures. Maybe they’re going to be wanting to write a fair amount of content or maybe record, even, podcast-style materials. That stuff eats up a lot of time. So I think really being real about what you absolutely need to do in that department and getting started now, that’s sort of the wisdom of experience that I would share with folks as well.

Rebecca: I think that’s really good advice, Michelle. As I’m thinking towards the fall, I made a list of “this is absolutely essential… if I don’t have this content made, we’re screwed if we’re online,” versus like, “this stuff does exist out there that I could use…if maybe isn’t my favorite.” And then there’s well established stuff that’s fine or whatever. Because it does take a lot of time to write, produce and plan some of that stuff… even if you’re using methods that aren’t burdensome, where you’re not worried about production quality and those kinds of details. It still takes time. You need quiet space. There’s a lot of constraints, especially if you’re like me and you have kids at home. [LAUGHTER] You got to find the quiet time to record the thing. [LAUGHTER] So I appreciate the balance there… really thinking logistically a little bit. Because if you have a finite amount of time, then you have to prioritize what can get done ahead of time.

Michelle: Right. And you know, it may not be the way to go. And I though I’d share with you an experience that I had, well, right in the thick of the great pivot, the transition to remote instruction. I was talking to a faculty member who does happen to teach studio art. They teach drawing and painting in a small-class atmosphere, a very intimate atmosphere that’s very hands on… and not somebody who works at my institution. I happen to know them. And she called me up partway through the great pivot week and was distraught. She was really on the verge of tears. And she was saying, “Well, this goes live next week, I need to somehow carry my course forward, my studio art course. And I just learned that my colleague, the guy down the hall, what he’s doing is he’s got these videos that come down from the ceiling, and then we have these close ups on drawing and these techniques and he’s doing all this. I can’t do this. I’m a single parent. I’m at home. I’m overwhelmed. I don’t know what to do.” And I said “Alright, it doesn’t have to look like that. Your colleague may be doing that. It doesn’t have to look like that.” And I said alright, what is working in your course? That’s another thing you can use to kind of cut to those essentials. So what is the strongest thing? What do your students need right now?” She said “Well, they’re absolutely overwhelmed and I think they need a lot of support.” And “Well, is there any kind of social peer-to-peer support?” And she said “Oh, well, we have since the beginning of the semester, I put them into these pods of three. And so they’ve been developing these social structures where they consult with each other every week. And so they have ways of communicating with each other in these pre-existing social groups. Do you think that could be useful? And I said “Yes, go with that.” So what your course is accomplishing really well right now is setting an atmosphere where students are talking to each other and I said, “Well, maybe you can kind of divide and conquer. You can hand off this project to where students are critiquing each other’s work in these groups. So, definitely kind of double down on that arrangement that you’ve already put into place. Your colleague down the hall, maybe multimedia is his thing and this is easy for him. But he may be struggling to say how do we get students to socially support each other form connections and feel connected to the class, even though it’s now in a remote format.” To me, that’s something to really capitalize on. So I took away a lot from that and I’ll be reflecting a lot on that as well. Your “solution” to the challenges we face is going to look different and it really should go with whatever is strongest for you. I think as academics, we kind of say, “Well, if it’s easy, that must be the wrong way to go about things.” But sometimes the path of least resistance maps well and aligns well onto what your strengths happened to be and what your students needs are.

John: Going back to that point, though, about creating media. If you create materials for an online format, you can always use that to support face-to-face if by some miracle things return to some sense of normalcy, it’s probably not going to, but that material will still be there and will be useful. So, a focus on that, I think, is really helpful. And that’s what we’ve been strongly advocating for our faculty as well.

Rebecca: Just as long as you don’t have specific deadlines… don’t put deadlines, dates or anything like that in them.

Michelle: Right? See, that’s just a practice that is so important to create reusable media. And it’s a seemingly small thing, but until you really get into this and get practice, you don’t realize how important that is… that yeah, if you are going to sink the time into that, make it reusable. And that’s an important point for reusability.

John: And going back there, I’d like to once again, we’ve done this many times, recommend Karen Costa’s book on 99 Tips for Creating Simple and Sustainable Videos. It’s a really nice resource. And it does focus on keeping it simple. Don’t do the fancy transitions. Don’t do something where a half an hour video is going to take you 30 hours of production time. Keep it so that it’s easy for you so that you can keep doing it without imposing a burden that’s going to make you stop doing this.

Michelle: Absolutely. And I’m so glad for that recommendation. I went out and got the book myself. I think I’m on Tip Number 80 as of this morning, so I’m almost there and I’m finding these wonderful… everything from very specific guidelines to much more conceptual things about why you want video in a course to begin with. So yeah, I’m with you on that. It’s definitely worth a read and definitely this summer. But maybe also, to kind of put this into a different focus as well with the focus on creating media and doing so purposefully in a way that is sustainable, let’s not lose sight of the active learning component. So that’s something that I’ve really kind of watched with some concern and definitely some interest as this conversation evolves. So active learning at this point, I mean, people sometimes perceive it as a buzzword, but it is such a robust concept. And I think it’s easy, at a point where we are kind of saying, “Well, how can we make all this work in some different formats” to lose sight of that. And so we may be creating wonderful videos, instructional videos, or all kinds of things and just merrily perking along with that, but we do need to remember how are students interacting with it, which is why a beautiful film of somebody demonstrating a drawing technique might, in some context, not even be as valuable as somebody who’s having students talk to each other because of that engagement. So I think that too, this is going to be so critical as we see more schools pushing for things like recordings of lectures, or even synchronously bringing students in during a live session you’re having with other students, I think that we do need to remind people who are in charge of these things, that education is just never something you watch, it is something that you do. So it is really tempting to say, let’s record everything we can, that’ll be equivalent, but active learning is not a luxury that we can just put on hold for a while. It really isn’t. And so I’m hoping that we don’t see that happen. I think there’s a very similar story that’s going on with Universal Design for Learning. Another concept I know you’ve engaged with so much on this podcast and is so important. And I think you’re too, it’s easy to say, “Well, you know, given all this going on, maybe we won’t have multiple ways of engaging with these great media that we’re creating, or maybe we’re going to kind of shut down this avenue over here for a little while.” And I really hope that doesn’t happen. So that’s another aspect of this balance between the quality and ambitiousness of what we’re doing and the feasibility and protecting ourselves as we face another very challenging semester.

Rebecca: I think that’s a really good reminder about focusing on the learning as the essential element as opposed to the teaching. It’s really about setting up the framework and the possibilities for students to learn, and designing those activities and making sure that we’re spending the time on that, rather than all the time on just delivering something.

John: But having those videos can free up time so that if you do meet synchronously, you can engage in more active learning activities rather than just lecturing to students online, which is probably one of the worst ways of structuring synchronous meetings. And if you really want to do a little bit more work, you could use something like PlayPosit where you embed questions in the middle of a video that could be somewhat open ended and that you could even grade. If you happen to have an institutional license you can embed it directly in your LMS. So the videos themselves can be made, with a bit of work, a little more interactive, and they can serve as a replacement for lecture that allows for more active learning, I think.

Michelle: Absolutely, and I too. I’ve seen some wonderful examples in practice of that technology, and there’s a couple of different ways to do this. So there’s multiple tools that allow you to put a retrieval practice or comprehension questions somewhere in the midst of this online lecture, presentation or video and what better way to help ensure that students are attentive to them, to give yourself some opportunities on the other end to say what’s the actual level of comprehension that’s going on out there. And for students to really solidify and practice the material. That’s all bedrock learning science stuff, right? Retrieval, active practice, and so on. And it just takes a little bit of ingenuity to take that one extra step to say, alright, what’s the level of interactivity here. And that’s something that I hear too, from faculty, it’s quite reasonable. They have taught purely face-to-face and don’t have that level of first-hand experience with something like online teaching. It’s just like, “Well, how do I know what’s going on out there?” And, again, there’s not a technology that’s going to just magically replace the experience of looking at the sea of faces that we experience in a face-to-face class. But think about it. That’s one way to do it. Having something like an online gamified quiz, like Kahoot!, which is currently my favorite quizzing app that’s out there. I ran this just the other day quite successfully in a remote synchronous environment. So, there are two that could help give you that information right away about what concepts are they struggling with. And having other ways of reaching out to students, if not talking to them individually in something like a meeting, a phone call, or even a text chat, having some other ways to kind of figure out on the ground what’s the mood level of the course? How are we feeling about things and are there individual students who are struggling for one reason or another who we can reach out to?

John: One way in which I saw interactive videos being used was several years ago, I took a MOOC on behavioral economics that Dan Ariely had put together and he’d often discuss experiments, but he set up the experiment and describe what the experiment did. But then the video pauses, and you’re asked to predict what the outcome would be. And that type of prediction is a really useful evidence-based technique that you can even do with videos if you can embed the questions in the middle of them. And I thought that was really useful. And it’s something I’m going to be trying to do a bit this fall. But in terms of evidence-based learning, could you talk a little bit about some of the main principles that people should be using to design their fall classes? What should people be focusing on?

Michelle: So, when I talk about bringing down just a vast literature of learning science and I’m going to necessarily boil this down to what I think are my favorites and the most applicable… So, of course, retrieval practice,I think if there’s one success story that our field has had, I mean it goes back even over 20 years that we got the data, determined how this principle works and started flowing it out to practitioners in the field, it’s this one. So that is, of course, the principle that when we actively pull something out of memory, it increases our ability to remember it in the future. And of course, we’d naturally think of tests, exams, and assessments as the avenue for this, but there’s lots of other ways that it can take place. So I always love to direct people to the website retrievalpractice.org. I’m not affiliated with it, but I think they have a wonderful compendium of ideas for how to bring this into classrooms at all different levels, all different disciplines, and so on. So if you don’t have retrieval practice, quizzing, students actively talking about what they remember, great time to bring that in. So you can’t go wrong with retrieval practice. Then, of course, the principle of what’s the timing of your study. So, spaced study, and pretty much by any measure, when we spread out student engagement with material… again, whether it’s through quizzing or solving problems, you name it, you’re going to get more out of that… efficiency… when it is spread over time. And I think that this is one of the real unsung benefits of online and technology assisted learning, even among people who are saying, “Oh, I’m just using the basic learning management shell to organize some materials and students turn their stuff in online. I mean, let’s not sell that short for how powerful that is, for being able to stagger deadlines, change the timing of when we are getting students to be working on different aspects of the course and so on. So while we don’t necessarily always want to bombard students with deadline after deadline, we do have to be mindful and help them kind of organize multiple deadlines. This is something that we could definitely build in as a design principle. So just to be very blunt about it, we always discourage people from the two midterms and a final course design. That’s something that a lot of us have experienced. It could work of course, like that can be fine. But from a memory and learning standpoint, that’s really not ideal. We want students engaging quite frequently. And then the practice… so the practice of this skill. So that advice, bring that up again, about it’s not all about content coverage. It’s about practicing the application of the content knowledge that they’re getting. We can almost always stand to build in more of these, I don’t think I’ve ever seen a course where I’ve said, “You know, you really need to present more content to the students. Don’t have them solving problems so often…” I have never seen that in practice, I will just go on the record and say that. So, if we want students to be doing X,Y, and Z. And again, go back to the front page of your syllabus and remind yourself what you’re hoping they’d be able to do at the end of the course. We want them to do that, what are the opportunities for them to actually try, and try in small bites? In my contribution to the Pedagogies of Care project, I give a very brief example of this in my own courses. So one of the things you have to do… bread and butter skills as a psychologist… is you have to be able to look at a psychology research study and kind of break down the structure of it. So no matter what’s being studied, there’s probably… we call them independent variables and dependent variables. So, things that are being manipulated, things that are being measured, and students have to develop that as a thinking skill and it’s really not easy. So I will oftentimes have them in, say a research methods course, very frequently, as part of whatever we happen to be doing, I’ll say, “Okay, here’s a really short description of a study. Maybe it’s an abstract or just a description, you pull out from me, before we talk about anything else about this study, you tell me, what are the independent variables? What are the dependent variables?” So it’s something that traditionally we’d always put on an exam. But, we didn’t always have students repeatedly practicing. So knowing that students absolutely had to master this before they got out of my research methods course. That’s what I did. So practice, and that kind of segues back into that active learning principle, which…yeah, you cannot go wrong with students getting involved. Once again, I don’t think I’ve ever seen a course where I said “You, the professor, need to get out there front and center, don’t emphasize the students so much.” So, they need to be doing the thinking, the practicing, and quite frankly, the work. That’s where the benefits come from. So with those: the retrieval practice, spaced study, practice of higher-order thinking skills, and a real active learning orientation, I think that that’s something you can take to the bank as a faculty member. You could build on that, but if you start with those, you’re probably going down the right path.

John: And I remember reading this really good book that talked about how using computer mediated instruction or using the tools within the LMS allows you to provide students with lots of feedback and lots of retrieval practice without necessarily increasing the burden on you, as the instructor. I think that book was called Minds Online: Teaching Effectively with Technology. [LAUGHTER]

Rebecca: Yeah. I feel like I might know that author, I’m not sure.

Michelle: Yes. [LAUGHTER] And thank you very much. That’s what I was trying to go for. So, thank you. It is wonderful that people are finding many of those points really relevant right now. So, yes, thank you so much for pointing that out. I think it’s great. [LAUGHTER]

Rebecca: I think one thing that I’ve been thinking about in terms of having more remote time then maybe in-person time is that I often provide a lot of structured activities around retrieval practice and spaced practice in my face-to-face class and if students are working more independently when they’re working remotely, I’m not there to [LAUGHTER] facilitate it synchronously, that structure needs to really be in place, maybe even more so than when you’re in face-to-face class, that they have that structure and that they know they should be doing those things on a regular basis. Of course, we should be reminding them to do these things on their own as well. But, I think focusing a little bit more on having that structure or those reminders in our courses, when they might be remote is actually really, really imperative,

Michelle: Right? And those are learning skills and abilities and principles that are going to serve our students well, no matter what they study or what they may do after they leave a course. And it’s kind of neat. There’s some indication from the research literature that particularly for students who come in who are not from advantaged backgrounds, that when they’re exposed to courses, which as you say, they remind them, “Okay, do this kind of practice. Here’s what you should be doing. Here’s why you should be doing that” …that benefit really does extend not just into that course, but into future ones because students can pick these things up on their own. So, if we do really want to be thinking about how can we set our students up for success no matter what the future holds, I think that’s a pretty high ideal that we can work towards. So yet another reason to incorporate these powerful practices and perhaps, yeah, to talk about how students can adopt them, no matter what.

John: For those faculty who are struggling to prepare their courses, what are some heuristics they could be using in terms of focusing their time where it would give the most benefit.

Michelle: This is something that has definitely been on my mind, both for my own preparation and to share with others. So heuristics, shortcuts, and helpful hints and approaches. So, I talked earlier about looking at what you consider to be your strongest points as an instructor and kind of the highlights of the course… the things that you know, are memorable, that advance learning that you feel really strong and competent with, with the caveat that, yeah, we do want to make sure that those do align with student learning. I think that that’s a great place to start. Say: “Okay, what’s the great parts of my course? Forget about what anybody else is doing. What do I really want to use?” And putting those front and center. If you have a short activity that’s working great, maybe that’s something that could be done every week, or somehow extend it. But the flip side of that is this, and this is another that I didn’t invent this… This is something you’ll see repeated time and again, in teaching advice, which is the pinchpoint heuristic, flipping it around and saying, “Oh my gosh, if there is one thing that students are struggling with conceptually, or it’s something that I know they should be doing, and they don’t do it to the level that they need to,” that you focus your efforts, kind of train your sights on that piece of it. Especially in the discipline. I teach, psychology. I mean, there’s so many fun things we could talk about with psychology, and it’s easy to kind of spend a whole lot of time and effort shooting the videos or setting up the learning activities online and making a quiz that’s about something that’s just cool to learn about. But that can’t squeeze out “Oh my gosh, everybody gets unconditioned stimulus and conditioned stimulus wrong and they do it every single year, and I know it’s going to happen.” So I need to be pulling out those things. You know what, if I’m going to spend the hours on a video or an extra module or creating an interactive quiz with multimedia, spend the time on the places where students are struggling. People who work with UDL, also talk about, “Well, here’s where you want to be especially conscientious to ensure that you do have the multiple means of representation and expression is around these areas that are really, really tough for students.” So what’s working great, where’s the point where you just say, if I could wave a wand and make one thing happen, that’s what I would do. So really looking at those two tracks. So that’s one heuristic. I think, as well, I’ll share with you something that I’m working on for my own courses… big caveat, that this is my courses. I will probably not be teaching a very large set of classes just because of the vagaries of course assignments. So I know I have that a little bit easier. But, here’s what I’m going to do as a framework. I’m kind of thinking of splitting it up so each week, students have a set of kind of general categories that they have to meet, they have to do some type of work or meet some kind of expectations in that area. So, I might, for example, have a column that corresponds to engaging with classmates about the topics for the week, and then a set of options for that week. So maybe you came to a face-to-face class, maybe you participated in an online discussion. And maybe there’s even a third option that I haven’t thought about yet. So just to really simplify things, I say, “Okay, check off in that area, what’s another column or category that you have to participate in, you have to do some type of demonstrating mastery of material” or I’m not quite sure what I’ll call it yet, but that could correspond to taking a quiz or maybe playing a Kahoot! in class or playing a Kahoot! remotely online. And I’ll probably also have a column that constitutes working towards whatever the term project is, and I’ll give them a set of choices again of what that term project can be like. But I am a very big believer in if you’re going to have a big project that there’s lots and lots of formative steps to that. So I tend to take that to extremes. And every week or so, students are doing something to show that they are moving towards and making progress in that area. So it is still a little bit general around the edges. But, to me, that really helped me feel like I had a handle on how am I going to manage choices? How am I going to manage multiple formats, and manage uncertainty with that focus on the purpose? Why do we have this do this week? Well, because it falls into these different categories, all of which are important for your learning in this class. So, those are a couple of the shortcuts that I would share.

John: One of the other things you talked about in your contribution to the Pedagogies of Care is the importance of getting help when you need it or where you need it. Could you talk a little bit about that?

Michelle: So this whole idea of getting help, I mean, it’s very simple on the face of it. I’m a faculty member, I want to do this thing in my class. I don’t know how… I call somebody… magic happens. And in reality, in higher education, what I’ve seen over and over at different institutions is that that is not a direct path at all between support, assistance, and collaboration, and the faculty member and the time and place when they need that. And so I think that this is going to be an issue that, if it’s not on people’s minds now, in leadership and pedagogy circles, if it’s not on their minds, now, it will be in six months to a year, I think that this is going to be one of the differences between institutions that make it through this fall in good shape and those that really struggle is what are those processes? So for faculty members, I’m really encouraging them to say, “Alright, where are the points, in this process, where you could get some kind of assistance that either you invest some time and you get the capacity to do something very efficiently in the future, say, like a workshop on how to do sustainable videos, or how do you actually find somebody to share the load? …actually delegate some of the work? For faculty, they should be reflecting on that, but at that point, that’s where things are going to get complicated depending on what the systems are in place at their institutions. So first of all, I think that institutions don’t always, and faculty ourselves, we don’t always make that distinction. When I say I want help, do I mean, I want you to point me to a great website or a book I can read? Do you want me to spend half a week coming to a workshop series? Or are you going to get in there and say, “Okay, you have the content, I can build these quizzes, you have a script for what you want to do for a complex video, I can shoot that for you, caption it, and put it online.” So what kind of help are we talking about here? And then figuring out how do you approach your institution to do that? So I’ve just really been continually surprised as I do visit different institutions. I mean, almost universally there are these amazing instructional designers and other people who just devote their professional lives to teaching and learning. They’re up on all the new technology. They know what was the great new video editing software that just came out last week? You know, they’re the ones who have that. And oftentimes there’s a disconnect there. People don’t know how, they feel inhibited, or maybe they’ve been actively inhibited. Some institutions, they say, “Well, there’s a process, and we’re going to put a lot of strings on how we’re going to divvy up these resources.” Others actually discourage instructional design and similar staff from even talking to faculty. And there’s a little kind of social piece to it as well, I think, just because we haven’t yet fully incorporated this into what we do… that it’s almost like, well, who makes the first move? If I’m an instructional designer and I know, here’s these courses over here that I could be helpful with, you know, just email people out of the blue… and likewise, faculty, they say, well, should I call the support line for this more complex project that I need help with or not? So I think that institutions will hopefully be sorting that out, but presuming that there isn’t a giant revolution in how we have collaboration between instructional designers and faculty, being aware of that and at least having something very clear in your mind for what you’re asking for, the worst that can happen usually is that somebody says no, but to have any chance you at least have to know what specifically do you want.

Rebecca: I think knowing that’s really helpful too. Because if you start talking to faculty, for example, in other disciplines, they might have a similar goal or they need similar structure in place, you could actually work with those faculty to put the structure in place and share the structure, swap out the content or whatever too. Sometimes we don’t think about those kinds of collaboration.

Michelle: Right, and what you’re describing, that’s something that is kind of non-traditional and new. We come into this with a very strong tradition of “my class is my class” and a kind of an ethos as academics that you do things the hard way, and you do them by yourself. But maybe this can be an impetus for us to really be getting creative with swapping, even things like a syllabus. You say, “Well, you know, maybe the way that I’ve gone about this, you can actually springboard this even if it is, as you said, in a different discipline.” Maybe we’ll even see faculty putting together some more unconventional team teaching arrangements. Traditionally, we know a team teaching is we’ve we’re experts in the same subject. And we’re going to create this class that sort of articulates, or we’re going to pass it back and forth. But maybe I should be collaborating with somebody from another area of psychology. Do they have to be in my sub discipline to just come in and say, help me with discussion forums, if I’m not very good at that, and then I can come into their class and help them with synchronous video, if they need help with that. Maybe if we have to, we will do it that way. So if that comes out of all of this, I think that would be a great benefit. And I want to say I have been really hesitant and cautious about engaging in this narrative of the silver linings and “Oh, isn’t this a wonderful experience? We’ll learn all these new methodologies of teaching will come out of this and we’ll all love online teaching and be fluent with it.” I don’t think that that’s an appropriate message for faculty right now. I think we do need to recognize that this has been somewhere between disruptive and catastrophic for most of us career wise, and not imply that we should all just constantly be thrilled to be learning new things. There are so many new things that we could be learning right now. But fall is coming. And we only have so much time. So I do want to put that out there, and that’s something that I think is an important thread that needs to be, and I hope it will be, talked about more as the dialogue unfolds. But even without saying, “Hey, this is a great time to do new things,” we can recognize that there will be innovation that happens, and it’s already happened. We’ve seen it happen.

John: And while this may not be a silver lining, I know in our teaching center, we’ve seen a lot of faculty who I didn’t even know existed on our campus, because as Jessamyn Neuhaus has talked about, people have broken down some of those barriers where they think they have to do everything themselves, and they’re more willing to request help when they desperately need help in ways that they weren’t willing to do before.

Michelle: Absolutely. I think that Jessamyn Neuhaus has been such a clear and fresh voice on some of these development issues. She’s absolutely right. She talks about it in her own style, which is totally unique to her, but it really gets it across, that we’re Professor SmartyPants, and we are not used to collaborating, working together, or just saying, “I don’t know.” So I guess we can also say, even if we don’t formally work in a teaching and learning center, if there’s something that you know, that your colleague does not, and you can help with, get out there, volunteer it, and let’s all really do this in perhaps a new spirit, where it’s not all just about, “Well, here’s what I know and you don’t know it, and I’m gonna feel uncomfortable coming in,” let’s have a real reset in terms of really open sharing. It’s not about playing the game of who knows more, or who figured out the latest thing. It’s really about serving the students and doing so in a way that we can sustain what promises to be a pretty challenging semester.

Rebecca: These have all been really great tips and things to think about as we move towards the fall, as the fall moves towards us… maybe that’s a better way of thinking about it. [LAUGHTER]

Michelle: I think that’s a frighteningly accurate turn of phrase there. [LAUGHTER]

Rebecca: I want to make sure that we get to talk a little bit about your new book, though, can you share a little sneak preview?

Michelle: Oh, sure. And this book, of course, well predates the era that we’re in. But it’s been something that I’ve wanted to write about for a very long time. And then when I was able to make the connection to James Lang and to his series, I think it was really meant to be. So, it is about memory and technology. So, much has been written in the popular press, and a little bit in the scholarly press as well, about cognitive processes and how those change or not in the presence of technology and with a frame for teachers, of course, so those of us who want to make up even just very specific policies, like should I allow note taking in class on laptops or not, to people who are really interested in this broader sense of teaching and learning in our contemporary era. So what I’ll be talking about in the book are issues such as well, first of all, what do we need to know about how memory works in the first place as a teacher or a person who is really into learning. So what do we now know about how memory works and how it can be improved? I also talk about why anybody should even care about memory, because that’s one of the angles of technology as well… this question of “Well, do you really need to know anything in the age of Google?” And there are people on both ends of that spectrum… probably no surprise that I come in somewhere in the middle of saying, on the one hand, it’s really important to be able to find information when you need it. And yes, we absolutely should be de-emphasizing memorization for its own sake. However, we also know from current research that memory in a subject area helps us think in that area. So there’ll be something for everybody in that section of the book as well. And then we will talk about what is the effect of having something like a smartphone, always at our fingertips? Does that create any kind of global change in memory? Does it change our memory for specific things that we might be doing or thinking about what we’re using that technology? And how, again, can we turn this to our advantage as lifelong learners ourselves and also for our students. Now, of course, you can’t talk about any of this without talking about attention itself. And so while it’s not a book about attention and distraction, per se, we’ll talk about “Alright, well, what’s the flip side of that?” And so how, basically, can we take all the advantages that technology has to offer for building memory and de-emphasize all the things that it does to offset and degrade our memories, and come out of this with the best of both worlds? I will get into a little bit at the end of the book as well into some of these bigger questions of how is memory itself changed when we live in a technological era when so much of our lives are recorded? And what does that say about things like generational differences, or what memory might look like decades from now? So I’m absolutely loving exploring all those themes, and I think they’ll be interesting for anybody who’s in the arena of teaching and learning but also with a lot of practical tips about again, how we can reap all the benefits that technology can offer for memory and for learning.

Rebecca: You’ll have a lot of disappointed listeners to know that that doesn’t come out until 2021. Right?

Michelle: Good things take time. And yes, we will see. It is a work in progress. And although we definitely have all the themes and all the ideas nailed down, it’s something I’m working on as we speak. So that’s part of why I’m so excited about the project. But yes, I got to finish it first.

Rebecca: We’re definitely excited for it to come. We always wrap up by asking what’s next?

Michelle: I am, as many of your listeners probably are, when this comes out, absolutely in the thick of redesigning my own courses for fall. Without getting into too many of the specifics, my institution has kind of laid out a set of parameters that they want us to meet. And so I’ll be re-envisioning my courses and to practice what I preach. I’m going to try to flow that out as much as possible to my colleagues, both locally in my own department, my own college, at my institution, and also nationally. So I’m kind of looking at some different ways that I can continue to engage people in this and share out what I’m learning as we go along. And I’m also pretty excited to be preparing some even more in-depth materials for some institutions who are looking for help in exactly this type of thing, how to get faculty interested in this whole topic of flexible teaching, some specific techniques that are useful for what I’ll call flexible teaching, key resources, things to do and not to do, and so on. So I’m excited to be coming back at it on all cylinders in the fall, and looking forward to engaging students in all the different formats that we now have and seeing where it takes us. So that’s what’s next for me.

John: Well, thank you. This has been wonderful talking to you again. We’ve always enjoyed these conversations, and our listeners have very much appreciated them.

Michelle: Oh, thank you.

Rebecca: It’s always really helpful to know too, that you’re not alone. We’re all going through the same kinds of contemplations, and so thanks for sharing some of your own stories about developing and planning for the fall too.

Michelle: Thank you as well.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

143. Pedagogies of Care: Creativity

Is creativity something you value in the work that students produce? In this episode, Natasha Haugnes and Martin Springborg join us to discuss ways to spark, motivate, and support creativity.

Natasha has served in faculty and curriculum development at the Academy of Art University and as an adjunct professor at the California College of the Arts. Martin is the Director of Teaching and Learning at Inver Hills Community College and Dakota County Technical College, Natasha and Martin both contributed to the Pedagogies of Care project and are two co-authors (with Hoag Holmgren) of Meaningful Grading: A Guide for Faculty in the Arts.

Show Notes

  • Haugnes, N., Holmgren, H., & Springborg, M. (2018). Meaningful Grading: A Guide for Faculty in the Arts. West Virginia University Press.
  • Pedagogies of Care
  • Haugnes, N., & Russell, J. L. (2016). Don’t Box Me In: Rubrics for Àrtists and Designers. To Improve the Academy, 35(2), 249-283.
  • Haugnes, N., & Russell, J. L. (2008, 2014) “What do Students Think of Rubrics? Summary of survey results: Student Perceptions of Rubric Effectiveness
  • Sawyer, R. K. (2011). Explaining creativity: The science of human innovation. Oxford university press.
  • Deci, E. L. (1972). Intrinsic motivation, extrinsic reinforcement, and inequity. Journal of personality and social psychology, 22(1), 113.
  • Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of educational research, 71(1), 1-27.
  • Inoue, A. B. (2015). Antiracist writing assessment ecologies: Teaching and assessing writing for a socially just future. WAC Clearinghouse.
  • Nilson,. Linda (2019). Specifications Grading. Tea for Teaching podcast. August 21.
  • Tharp, Twyla (2006). The Creative Habit: Learn it and Use it for Life. Simon & Schuster
  • Transparency in Learning and Teaching (TILT)

Transcript

John: Is creativity something you value in the work that students produce? In this episode, we discuss ways to spark, motivate, and support creativity.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Natasha Haugnes and Martin Springborg. Natasha has served in faculty and curriculum development at the Academy of Art University and as an adjunct professor at the California College of the Arts. Martin is the Director of Teaching and Learning at Inver Hills Community College and Dakota County Technical College, Natasha and Martin both contributed to the Pedagogies of Care project and are two co-authors (with Hoag Holmgren) of Meaningful Grading: A Guide for Faculty in the Arts. Welcome Natasha and Martin.

Natasha: Good to see you. Yay.

Martin: Thanks for having us.

John: Our teas today are:

Martin: I’m drinking coffee this morning.

Rebecca: Always… Always the rebels.

Natasha: Well, I had my two cups of coffee and now I’m on to Wild Sweet Orange Tea…

Rebecca: That sounds good.

Natasha: … and it’s delicious. Yeah.

Rebecca: I have iced Scottish afternoon tea

Natasha: Afternoon? Huh…

John: And I am drinking Tea Forte Black Currant Tea. We’ve invited you here today to discuss Natasha’s contribution to the Pedagogies of Care project and your joint work on Meaningful Grading in the Arts. Natasha, could you start by telling us a little bit about your contribution to the project?

Natasha: Sure. “Nurturing the ‘aha moment’” is the topic of the video made. And it was based on one of the tips in the meaningful grading book that I co-authored with Martin and Hoag. This video focuses on the “aha moment,” or that moment of insight in the creative process, and how to really nurture students and invite them into that moment. I focused on the “aha moment,” which could also be called the moment of insight in the creative process because it really is associated with kind of joy and happiness and magic. And there are a lot of cultural myths around insight and creativity in general, but especially these magic moments. People think they come out of the blue, that they’re come down from God, that they’re somehow related to some innate ability. And research shows us, and people who are creative practitioners know, that this is not entirely true. So, I just decided to kind of hone in on that moment. In my work at the Academy of Art University, I have worked with a lot of students and a lot of instructors who are often drawn to creative fields because of the joy and they really want to engage in that, the joy of the creative process. But then when the students get to school, and when the new instructors come to teach, they often get really drained. And they find that there’s so much hard work and there’s so much stress in the classrooms, even in things like painting and graphic design and moviemaking classes, students seem to get really rundown, and they don’t connect with those moments of joy. So, this results in frustration. At my own school, we were seeing pretty high dropout rates of students at a certain point and I actually ended up working with at-risk students in my role as the Resource Center Director at the Academy of Art University many years ago, and that taught me a lot about working with students and engaging them in their creative process. A lot of the students I worked with, they were sent to me by an instructor who would say “This student is just not engaging. They’re really sloppy in their work. They’re really lazy. They’re not putting the time in.” And when I talk to those students, I would find patterns that really ultimately meant that they weren’t understanding their creative process. They were doing things like brainstorming a whole bunch of ideas, and then trying to finish one, but then getting distracted and thinking, “Ah, I’m going down the wrong path, I’m going to do this other project, I need to take this other approach…,” and they would go down another path, and then they would abandon that path, and they would take yet another approach and pretty soon it’s time to go to class and the project they’re presenting for critique looks like it was done at the last minute. Again, this is really frustrating for the student and the instructor. And I realized I needed to learn a lot more about the creative process in order to work with these students and help them connect to that joy, help them understand how the hard work connects with the joy, and help the teachers understand how the hard work connects with the joy. I think it’s really imperative that our faculty understand creative process and define it so we can teach it to our students. And this is especially important for students whose livelihood depends on creativity, like a game designer, a graphic designer, even an illustrator can’t just go to work and hope that insight comes, they need to learn how to have some control over that, not only for their own work, but just so that they continue to enjoy what they’re doing.

John: It sounds like part of the problem is that people think that creativity is just something that people either have or don’t have, and they don’t see that it involves a process that includes a lot of work. What types of things can we do to nurture students in making the connection between the work that they do and that aha moment to get them to that point, so that we don’t lose them on the way.

Martin: One thing that I talk about quite frequently with faculty, no matter their discipline, but especially in the creative fields, and one thing that we go back to quite a few times in Meaningful Grading, is rewarding failure and grading process versus grading that final product. If you value the development of a creative process and you value your students diving into the waters that they’re sort of murky, they cannot be afraid to do that. And at the same time, they should also be aware that you’re rewarding that effort and their engagement and what can be kind of a scary process for them, especially if they consider themselves non-artists or unable to do art because they don’t have some innate knowledge of it. So, as you develop grading systems, making sure to work into those grading systems those things that you truly value about that process and about your course.

Natasha: I think it’s really crucial. And something that I try to point out in the videos is breaking it down, scaffolding the process for them, breaking it down into small accomplishable steps and explaining to them: “No, this is not creativity, this is not your whole project. This is what you need to do now. And here’s what you need to do, and you need to put the work in to do it. And then you can move on to the next step.” I think that’s really important, and it’s just really important for the instructors to do that. We often have the overview, we understand the process, we have faith that they’re going to get there, but the students don’t, necessarily, and so that’s kind of what leads to those patterns of procrastination that we see with the students who aren’t doing so well. They put things off, they don’t understand the importance of that early hard work that you really have to just put in in order to get the payoff at the end.

Rebecca: What are some ways that you recommend building in experimentation or risk taking into the grading system? Because those are often things that we value in creative fields, but are harder things or things that we don’t always build into our evaluation systems. We might focus more on the principles of design or something technical, [LAUGHTER] because those are easy to measure.

Natasha: You’re a graphic designer, aren’t you, Rebecca?

Rebecca: I am. [LAUGHTER]

Natasha: I think graphic design is actually a really great example of a place where you can get really bogged down with rules, right? I mean, you can approach graphic design almost as a mathematician and just kind of go “ka-chink, ka-chink, ka-chink, ka-chink, ka-chink” and you can create stuff that follows the rules, but doesn’t really have a lot of creativity to it. And I guess one piece of advice, this goes to a recommendation that I’ve included in the video, but really simplifying criteria. Again, if you can break down the steps and have each step just be assessed on one or two criteria, that allows students to kind of say, “Okay, I’ve met the goal, now I can do what I want. I’ve done what that teacher needs to see, [LAUGHTER] and I’m going to pass, and now I can really play with it.” In some research that I did with a colleague of the Academy of Art University quite a while ago, we did this big study, twice actually, called “Student Perceptions of Rubric Effectiveness.” We found a common pattern in students’ responses, the students that really liked the rubrics said that they liked the rubrics because it told them exactly what they did have to do. And then once they checked off all those boxes, they could just run with it, and that was very freeing to them. We can talk later that a rubric is not always perceived that way, for some students, it kind of acts like a creative constraint. But, I think if we can keep the criteria to a minimum, that can allow students to know what they have to do and then have fun with it.

John: One of the things I noticed in reading through your book, and also in what you were just talking about in terms of giving stories scaffolding, is so much of the advice that you give could apply in pretty much any discipline. While your focus is on the arts, students don’t have the same expertise that we do. And the tasks that they’re facing are much more challenging and require much more processing. And they don’t always come in with that growth mindset. Much of what you’re talking about basically, is how to help students move from this binary view that they’re either good at it or they’re not to recognizing that learning is work, and that they can get better as they develop. And it was nice to see how closely this was aligned to the advice we try to give in so many disciplines.

Natasha: I totally agree, John, and actually I was in a conference at the University of Missouri where they actually viewed this video, and the person who was facilitating the workshop that I was lucky enough to be able to attend from the comfort of my own home office here, she’s a scientist, and she actually put up a map of the scientific method and said, this is the creative process and this is not the exclusive domain of artists and designers by any stretch of the imagination. So, I love having those cross-disciplinary conversations. I actually teach writing and ESL, and so I see some crossover there. I guess I’m just reluctant to offer a lot more advice to teachers of physics and math and economics and things like that, simply because I don’t have as much experience with those instructors. I’ve been exclusively art and design skills for a really long time. Martin, maybe you can speak to that. You have a lot more majors at your colleges.

Martin: Especially in those foundational courses, you’d certainly get students coming in at a variety of levels. So, they have past experiences, or they don’t, and those with past experiences sometimes come in with quite a bit of knowledge or experience in the arts. So, they’ve had a lot of high school experience, for example, that puts them at a different level than the other students in your class that are truly beginners and don’t have any prior experience and consider themselves very much non artists. So, one thing that’s important to do, just getting to the practical here, if you’re in an arts course teaching at that foundational level… or really going back to your comments about this crosses disciplines, no matter what discipline you’re in, if you’re teaching that foundational level course, getting everybody at that same base level at the beginning. Purely speaking from past experience here on this one point, I taught photography for about 20 years. And in my intro courses, I would frequently have students come in that had high school experience, and they had learned something and could demonstrate that thing. But, at the same time, they learned it in a, I’m not going to say the wrong way, but in a bad way. They picked up some poor practices from their previous education in that, and so you have to make an effort to untrain that a little bit and get them to that same process that you want everybody to engage in, at that very beginning level. So, that step and that effort also makes those students who are truly coming in as beginners and don’t have any previous experience realize that “Oh, yeah, this is something that I actually have to learn and that everybody has to learn and these students who come in with previous knowledge, it’s not just some inherent skill or ability that they have in the arts. Another thing that I found really helpful, in sort of leveling the playing field and making it apparent to those truly beginning students, is using my past beginning students who have come into my courses with no experience, using their products as exemplars when I’m talking about how I want somebody to do something. So, if I’m talking about an assignment, I’m using examples from, and I’m pointing out the fact that these students came in from like, say, they’re nursing students or their automotive students, or this student came in with zero knowledge, and this is the thing that they produced, and it’s actually an ideal example of what I want you to produce in this assignment. So, using that, and going back to those examples shows those students who come in as true non-native or true beginners, that that level of achievement is possible.

John: I think that was an interesting point, too, that also shows up in other fields. I know people teaching computer science often will note that it’s much easier to teach people who are true beginners than those who had been self taught or perhaps picked up something in a course, where perhaps not an optimal pathway was given to them. The importance of unlearning things, perhaps, or breaking down the structures that people have and replacing them with stronger structures, can be as much of a barrier as people who are struggling just to get to that initial level. And that I imagine is particularly true in the arts.

Martin: Yeah. And going back to what I mentioned earlier about valuing process, maybe they do produce a product, that’s roughly the same result, like if they come up with the same result, but the process that they engaged in to get to there is so much more complicated and convoluted than what you’re trying to get everybody to engage in. So, they do need to go back and learn process. They do need to be at that same level as everybody else in your course.

John: One of the issues that often comes up in discussing creative fields is the importance of intrinsic motivation. Could you elaborate a bit on how we can help develop intrinsic motivation for students in these fields?

Martin: So, another thing that we talk about or that we bring up in Meaningful Grading frequently is the building of a community in an arts classroom and how important that is. That community is the intrinsic motivator. For example, if you make that a primary goal of yours in a course, you would then grade heavy on participating in that community at the beginning, knowing full well that the goal you have is to make that a more intrinsic reward for students and to back off on the grading or drop it all together, that participation component. So, that they not only learned that after they leave your course and after they leave an arts program that an arts community is vital. Like you can’t develop work in some sort of vacuum. As an artist, you have to be engaging with others, but also within your course, it’s just showing them and it’s creating that intrinsic value. Like, what’s bringing me back to this class day after day is not the grades that I’m getting from my instructor, but the vast resource that I have in these 30 other classmates that are able to give me feedback and support. And that also show me what they’re working on… that give and take. So, that’s one example of building in that intrinsic value.

Natasha: Correct me if I’m wrong, Martin, but a huge part of that community is critique. It’s critique discussions, right?

Martin: Exactly, hours and hours of it.

NATASHSS: …and helping students to understand that just getting that conversation, it doesn’t even have to be feedback, but a conversation, and engaging people to talk about your work does build intrinsic motivation. That’s the big payoff that we’re working towards.

Martin: And if you don’t have that tight community in that class, when you get to the middle or the end of that class, when you really want students to be engaging honestly in critique, it’s going to be like pulling teeth. You have to foster that community so that students feel comfortable, that they can open up, they can give opinions about other’s work, and accept opinions about their own work.

Natasha: I kind of want to get into a little bit of that intrinsic/ extrinsic motivation research. And I guess one of the things that got me into this field, and my obsession with grading and creativity, which people kind of look at me and they say “You talk about grading in art school, shame on you.” But the thing that was so confusing for so many of these at-risk students that I worked with before was they were engaging in those conversations, or they thought they were, with their instructor and their instructor would say things like, “Yeah, you know, you’re doing great, keep going.” And that can mean “Keep going. You got to keep working. ] 3 handclaps] But you’re not there yet.” But the student was hearing it as “Yeah, I’ve done it. Good enough.” Right? And so that student would say, “I got a D+, I don’t understand. Like, what’s going on? The teacher likes me…” or “the teacher said I was doing great.” And so they weren’t able to suss out the actual evaluation in those conversations, especially these new students. So, this is where it is so important to actually have grading systems that align with those conversations and that reflect those conversations. Keith Sawyer, he is like the creativity guru who I follow. He’s amazing. He wrote this book called Explaining Creativity. And there are a couple of pages in this book, Explaining Creativity, where he does essentially a synopsis of all the research on the effects of reward and grading on creativity. And there’s some things that we can look at here that are kind of important… that yes, we can extinguish intrinsic motivation with grades, we can do it by giving As for everyone. We can do it by just throwing grades that are completely unconnected to the actual conversations we’re having in class. And we can do it when we grade students and use a whole lot of really judgmental language and convey that judgment. That will all really decrease intrinsic motivation and creativity. But a lot of that early research on intrinsic-extrinsic motivation goes back to the Edward Deci studies, I believe, and he actually did more work on this later. And there’s a more nuanced conclusion that he came to later that when grades and rewards are perceived as information, when these grades and rewards are based on the quality of work that students are turning in, that can actually enhance creativity, and it can really build intrinsic motivation. But even when you’re using grades well, they shouldn’t be emphasized too much. This is the conversation that I often had in faculty development when I was working with new teachers. Oh, come to class, you’ll get five points. Five Points, that’s not why you come to class. You should not be coming to class to get the five points; you should be coming to class because the conversations are important. That’s why we want you here… and just changing the script in how we talk about grades. You need to have a grading system that has a lot of integrity. But, we should not be banging that over our students’ heads all the time, it should be kind of in the background just running along in the background. And what we communicate to students is the intrinsic rewards of all the work that we’re having them do.

Martin: And that’s why your grading system has to transform a little bit over the course of a semester, going back to that grading heavily on participation at the beginning of the course, where you have to get the students to the course to participate in the beginning for them to realize that there’s value in those conversations. If nobody shows up, they aren’t going to have conversations, but then that can change and it can evolve over the 16 weeks or 10 weeks or whatever length your course is.

Natasha: Yeah, and there are those students who really do care about grades I find in art and design school, there’s a certain subset of students who really don’t care, and that’s fine. And so they’re kind of on their own path, and they’re often doing well. But there are those students who really care and there are the students who are on the verge of failing out of school so they have to care. And I find that just understanding that, instructors need to leverage that knowledge to convince students to do stuff that we want them to do… that we know will do them good anyway, right? So if I say, “Okay, you’re going to be really a grade grubber… you want an A do these things,” and they’re the things that they need to do anyway. It’s a way of kind of tricking them into doing what we want. If you’re grading what’s important in your course, it’s going to work out, it’s going to work out for the students who really care about the grades, for the ones who don’t as much, hopefully, they’ll just be intrinsically motivated to understand why they need to engage. But grading what’s important is really crucial in that, I think.

Rebecca: One of the things we’ve talked a bit about is scaffolding and helping provide structures. So if we were to provide structure for faculty who are thinking about the idea of building a grading system that has the values that we’re talking about, things that really they care about or are important to their class, what are some of the steps you would recommend they go through to actually develop that system so it actually does reflect the values that they want?

Natasha: Well, [LAUGHTER] my answer to that is it’s ultimately working towards a rubric. And again, that can be kind of a bad word. I’m the one who’s been walking around the art and design school for 25 years saying, “Let’s build rubrics. And let’s do normings.” And I had a photography teacher tell me one time “Natasha, you gotta understand when you say “norming” to an artist, I mean, that’s like death, you know?” So I’m like, “Ah, sorry.”

Martin: There’s a reason why we don’t have assessment in the title of our book.

Natasha: Yes.

Martin: That was on purpose.

Natasha: It was by design. Absolutely. For the really grade-averse instructors, I start with a conversation. And I usually start with grading because that’s a really good entryway. And I’ll just say, “What are you teaching? And what does that look like? And what does it look like when a student does it? And what does it look like when a student doesn’t do it?” And really, that’s where you start. And then I think the next step is really getting real student work in front of this instructor or this department or this cohort of instructors who are teaching the same course… different sections of the same course. They need to look at the student work and they need to say, “Well, yeah, that one meets the criteria for this course that doesn’t.” Why? Why not? Having those conversations, that’s like the best investment that I think any department or any instructor can make into really focusing their teaching and to improving assessment is just think about how you’re making what you teach visible. And then what does it look like when it’s acceptable and when it’s not acceptable, when it meets the goals and when it doesn’t meet the goals? And then it just moves on from there. And if what you think is important, the quality of the color print in your poster, or the resolution on your screen of your logo, or whatever the heck you’re talking about, it might be process. So again, what does that look like? Well, I want to know that they’re listening to the feedback and really taking it in. Well, how can we make that visible? Maybe I have them do a little recording or do a short paper saying here’s all the feedback I heard, and here’s how I responded. “Joey told me I should change the concept, but I didn’t like that idea because…, so I’m not going with that…” and actually have them make that thought process visible. So, it takes some, again, creativity on the part of the instructor in the field of the teaching and learning. But usually, if there’s something really important that you’re teaching, you can have a way to make it visible and figure out what you’re looking for. And what does it look like when there’s evidence that the student has done what you need them to do? And what does it look like when that evidence is not there yet?

John: I usually meet with new faculty and generally ask them what would they like to see workshops on and, about six or seven years ago, one thing that was requested was a workshop on evaluating creative work. So I reached out and we got four people from different departments. We had someone from art, someone from music, someone from theatre, and someone from English. And they put together a presentation of how they evaluate creative work. And one thing that was in common was they all used rubrics, and they all talked about how there are certain fundamental skills or processes that students have to follow. And that’s what they embed in the rubrics and it surprised a lot of people in STEM fields who were attending because they were much less clear about what they were expecting from students and They expected something that would be much less well defined. And so one of the things they also emphasized, and you’ve talked about is that it’s telling students exactly what they’re expected to do and what types of things they need to demonstrate in the work before they can embellish on that. And that was a really important feature in all of their discussions, the same arguments show up in your book. That surprised many people outside of the creative fields.

Natasha: Oh, those are my tribe. That warms my heart to hear that, John. That’s exciting, yeah.

Martin: One of the added benefits of using rubrics is that time saved as well. Faculty time is a precious commodity. And if you can convince them or just show them how much time will be saved by simply having that rubric available, and using it as a guide, as you’re going through the assignments that are piled on their desk, it’s a convincing argument.

Rebecca: So, we talked a lot about building in values into our evaluation system. Can you talk about some of the things we should avoid doing.

Martin: I can speak to that a little bit. So, one thing that I’ve seen a lot of arts faculty members do… from a student perspective. So, coming up through the arts, one thing I’ve seen a lot of, and heard stories about, is the instructors bringing their personal bias, their own career and background, and that subjectivity in general, to the process of evaluating student work. So I’ve heard some pretty bad horror stories about that. For example, I’ll just go into one story quickly because I think just every faculty member who’s hearing this should know that this is never something that you want to repeat. So all the work, as you can imagine, all the prints, lining the board during critique and the instructor just, without words, just going across the board, pulling work down and throwing it out the window. Like if he doesn’t like it, right… if it doesn’t meet his criteria, which are a mystery, by the way…

NATASHAS: I’ve been in those classrooms. I’ve seen that.

Martin: Tell non-arts people about these stories, and they’re like, “no.” Yeah, it really happened. So remembering that you got to check your personal bias and your personal preference for art at the door and rely a lot on, or more on, having students engage in self evaluation, like did they feel like, and how do they feel like, they have made this, or communicated this, through their work, this issue that they think is important through their work. And if it doesn’t, like if you’re not understanding, then engage in a conversation about it. Like how they feel they’re getting there and where you think they’re not getting there. So using that as a starting point instead of your own, “I am the authority on art, and this is why this does not work.” That’s a huge demotivator.

Rebecca: I think one of those biases that a lot of faculty might bring to the door, is the history of white art created by white individuals.

Martin: This is the history of art, it’s all white male.

Rebecca: If students are creating their work from different cultural perspectives, and the faculty member is not up to speed on other cultural perspectives, we’re enforcing essentially a white supremacist point of view and system. So how do we engage in those moments in a way that’s productive, especially if we don’t understand the cultural background that something is based on?

Martin: Yeah, if students can’t place themselves in the history that you’re talking about, you’re referring to, how are they to imagine themselves in that world in the future?

Natasha: I’m gonna offer just one little tip here because yes, I hear you, Rebecca, and we see it everywhere in the overwhelming influence and sort of self-perpetuation of the white colonialist culture, even in our art classes. Something that we found when we did our rubrics research was that students, in general, really love rubrics, it helps guide their work. But what they really loved… even more than the grid of language… was samples, examples of work, examples of work that span the quality. Here’s an example of something where somebody tried really hard but they didn’t quite hit the mark. Here’s some examples of passing work. Here’s some examples of work that really hits it out of the park. And it’s really important not to have one example, especially in a creative field, because what happens then? The students who are not very competent will copy. Here’s an opportunity to allow for many different interpretations and really show those to your students. Consider using student work from previous semesters from a diverse range of students with diverse content. And that gives students something to connect to, it helps them see themselves in the class, it helps them understand that you, as an instructor, see them and value them. And that even though you have these criteria, there are many ways to reach those goals and reach those marks, those criteria that you’re putting out.

John: And so, by including a range of examples too, from different genres or different approaches, so that it does not become just a Western culture, perhaps. In recent podcasts we’ve done with Kevin Gannon, for example, he talked about decolonizing your syllabus and just suggesting that when you’re putting together your syllabus or searching for examples or exemplars, you could just do a little Google search on decolonize your [insert subject matter here] syllabus, and you can often find some good discussions of that with some good resources that you can build in.

Natasha: Yeah.

Martin: Yeah.

Natasha: This is incredibly important. In my work at California College of the Arts, there’s a very active group of instructors. They’re working on decolonizing the classroom, anti-racism, anti-racist pedagogies, and I’ve learned a lot since I’ve been in teaching there. I haven’t been there for a very long time. But I guess there’s a book called Antiracist Writing Assessment Ecologies: Teaching and Assessing Writing for a Socially Just Future by Asao Inoue. And he speaks quite a lot about assessment. And the point he makes about assessment is he says, in order to really decolonize your classroom, we need to be careful how we talk about quality, because quality so often is really culturally loaded. It’s so loaded that it is really hard for us to even untangle what we see and what we look for. And as a response to that, he really emphasizes grading on labor, grading on the work. And this, again, relates to some of the topics that are in this little video I put together although I don’t really call it this by telling students and taking all that quality judgment away from your rubric and from your assessment and just saying, create 50 of these things, [make 50 taglines, make 50 photographs, write five different thesis statements for your paper or write five different opening lines for your paper and just do that. And that’s the way of just asking for labor. You’re just saying do this work and it doesn’t have to look a certain way or be a certain way. But if you just put some effort into this, you will do well. This is a way of assessing work that actually pans out much better for students of color, students from cultures that are not traditionally represented very well in the faculty at colleges and university. So this is something I’ve been really taking to heart a lot. And in my writing class, I’ve actually, at CCA, where I teach freshmen composition to non-native English speakers, everything is graded on pass not passing yet. And so that really emphasizes the labor. If they’re not passing yet, the implication in that not passing yet grade is that you will do it again. Just do it again. Do it again. Nope, still not quite right, do it again. There have been a few students who have redone their essays four or five times, and it’s painful. But wow, they learn… they learn. And again, the trick is in not having five pages of criteria, but having a pretty narrow band of criteria that we’re looking for here that doesn’t get really niggly about the quality.

John: It sounds like it’s a specification grading system that you’re using. And it’s also building in something much more explicit than the “keep going” message that can be misinterpreted. So giving students the opportunity to try something to not quite get there, but to encourage them to continue working on it more explicitly than perhaps students always hear.

Natasha: And I’m glad you mentioned specifications grading, Linda Nilson has been a huge influence on the way I think about teaching and grading. She’s got a lot of really good thoughts out there for sure.

Rebecca: One of the things that’s really easy to evaluate is something that’s technical that has a right or wrong answer. How do we evaluate in a rubric format, things that are more qualitative, like the amount of experimentation or risk taking or other things that we might value in terms of creativity? Can you give us a concrete example?

Natasha: Actually, we have a a whole tip in our book about risk taking. There’s some really interesting ideas about ways you can really force students into making some mistakes and talking about them. There’s so much that comes up that seems, at first, like it’s going to be really hard to describe it in a rubric. But again, if we just get instructors and people who teach these disciplines together, talking about things, usually they can come up with something much more concrete, even if it’s not a cut and dry technical skill. Concept is one and I have some examples of like before and after for rubric wording. And often when we first write out a rubric, we might use some really sloppy language like “The concept is sloppy. It’s lazy. It just doesn’t work.” That just doesn’t work, right? [LAUGHTER] And so that might be the first draft. But then you start looking at some student work and talk with your colleagues. And you’ll find some more precise language will come out. Often when we talk about concept… I’m talking about the context of maybe an advertising campaign. But the concept is predictable. That’s a concept that is not acceptable is predictable. It’s the first thing that comes to mind when people think of this product. So, that is not a good concept. So there you go. Now we’ve made something a lot more understandable to the students and to the instructors when they’re using this rubric to grade later. And it can help you move forward in a way that that judgmental language won’t. It just makes the students feel bad. It makes the teachers frustrated, because we’re like, “Oh, it just doesn’t work.” But actually taking the time to look again at a range of work that doesn’t meet or that does meet the expectations for this thing that seems really nebulous at first usually you can manage to articulate it, and if you can’t, then maybe that’s not something you’re actually teaching in your class and maybe that’s outside of what you’re assessing. This is another tip that we come up with quite often. I think oftentimes instructors who fear grading, they think that they need to grade the art and you can’t grade art. No, you can’t grade art. You can’t say Picasso was better than Twyla Tharp. You can’t compare people and grade artists in a holistic way. Your grades should be based on what you’re teaching, and the objectives for your class. And we can communicate to our students, this is what we’re looking at here. You’ve also done this other stuff really well, but in our class, we’re really looking at this, so this is what your grade is based on. And that’s a really important factor in this whole endeavor, as well. One other little trap, I think, that faculty members can fall into when we talk about assessing grading or assessing creative work is that when we sit down to write our criteria out often the first thing we want to talk about is that incredible piece that that student two years ago did, it was amazing. It was mind blowing, it was so good and students need to see this and you get into those conversations. And that’s fun to talk about with your colleagues and you pull up that student’s work. And you talk about how great they were and what they’re doing now. Yes, that work should be shared with other students, that’s exciting. We have to celebrate those moments. But for the student in the middle of the pack in your class who’s kind of struggling, we need to think about what’s acceptable. That’s why it’s really important to really focus on that line between what meets expectations and what doesn’t meet expectations, because there are some students that just really need to work on that. [LAUGHTER] There are others that are going to blast through that and do really great things, but the ones that need our help are usually the ones that are hovering around that middle area.

Rebecca: So, we’ve talked a lot about rubrics and grading and evaluation, kind of assuming that we’re living in a perfect little world in some ways. But as we all know, right now, in this moment in time, there’s a lot of extra stress of COVID-19, protests related to Black Lives Matter, and any numerous other health things that are coming up because of COVID-19, remote learning. [LAUGHTER] All of these things, there’s lots going on. And so students are under more stress than normal. Students are often under a lot of stress, but this is like extra stress. So in these moments, what are ways that we can help promote creativity and also help our students really feel supported and being able to learn whether they’re on this point in the spectrum or they’re finding being creative really therapeutic and helpful, and all the way to students who just feel like they’re frozen because there’s so many things going on in the world, they feel like they can’t move forward.

Martin: I think now is a great time to be engaging students in creative process. It’s what gets us unfrozen. I’m speaking purely from my location at a Community and Technical College. If we can get students to engage in those often elective courses outside of their major or area of focus that allow them the opportunity to dive into those things that they are feeling a lot of stress about or anxiety about. It helps students be more successful in those courses that they do have to get through as a matter of course for their program of study.

Natasha: Oh, boy, these are hard times. I think, just most immediately from the video, the nurturing the aha moment, I think that it’s even more important than ever to break down our projects into small steps and help make those steps really kind of distinct from each other. I think that’s something that’s happening for students now, and for us, is we’re sitting and we’re staring at the screen all day long and it can become this big blob of existence where one thing bleeds into the other. And if we can really make the steps a little bit distinct, including a few steps where the students just disengage from all social media and anything online where they can actually be alone, without all of the electronic stimulation. I think those are things that can really help nurture their creativity. And also just I think there’s this funny paradox right now that we’re all alone. We’re all isolated. And yet, if you’re sitting there on your TikTok and Instagram and all day long you’re connected and that can be really, really stressful… and so convincing students to take a break from that, telling them we’re going through another step now. [LAUGHTER] And keeping things again really simple so that they can have that opportunity to use what we’re doing in our classes as a springboard to express themselves. Encourage them to incorporate what’s going on in their own life into the work that we’re doing, including examples and acknowledgments of what’s going on in the world. Really important. And it’s a fine line. I’ve just talked about this with my co teacher about how we’re going to be discussing Black Lives Matter, the latest George Floyd protests, and the Black Lives Matter protests, and the defund the police protests with our students who are mostly from Mainland China. Where do we even begin with that discussion? How do we do that without completely stressing them out, but also using it as an opportunity to feed their curiosity and acknowledge their own stress around these issues? So we need to let them know that we’re a safe space for everybody to engage and really help them break down things into small packages and celebrate their achievements. And again, let them keep working if they’re not quite there yet. Let them do it again. Let them do it again, let them do it again, I found myself being very forgiving on deadlines,

Martin: We also have to help faculty realize that they’re safe to engage in those redesigns and those conversations, and that comes from at that administrative level, engaging this at a college or institutional level. So that you aren’t leaving faculty to figure this out on their own. At my two colleges, for example, we have this new initiative that will run all the way through next year, and actually, for the next three years, probably called Equity by Design. And so we’re starting with a team comprised of administrators, directors, faculty, helping each other understand what this effort is going to be at a college level.

Rebecca: One of the things that you’ve both emphasize is kind of these small steps. And I think a small step for an expert might be different than a small step for a beginner. [LAUGHTER] Can we just take a minute or two to describe the differences between what an expert might think of as a small step and what might be in practice an actual a small step for a student.

Martin: One thing that we have been engaging in at my colleges is the TILT framework of Transparency in Learning and Teaching by Mary-Ann Winkelmes and her team. Mary-Ann came to one of our colleges in January and actually spoke and I’ve been facilitating communities of practice at both colleges on this topic this year. And in that work, there’s a realization as faculty review each other’s assignments and each other’s syllabi that you’re not starting at square one, you’re actually starting at square five, because we have to so often take a step outside of our disciplines to realize that, like you just said… So, what’s complex or complicated to one student is not for another and vice versa. So that transparency effort helps us to really outline the steps of an assignment, even those small steps. And so I’d encourage any family member struggling with whether or not to start at this point or that to review that transparency literature a little bit to engage with their colleagues, share assignments, and ask their colleagues whether or not they’re starting in the right place.

Natasha: That’s such a good question, Rebecca. The expert/novice thing is just something we grapple with all the time as instructors, especially if we’re teaching a new course… something that I’ve had to do in my own class… I was just thrown into a very new course for me a couple of years ago. And we did a new project on public service announcements this last semester, and I start something in class, I told the students “Now, choose a topic from this list of public service announcements that you’re going to create. And first thing you have to do is do some research. So let’s look at some websites.” And by having them do that in class and seeing what they come up with, I start to say, “Oh, right. [LAUGHTER]] They’re going to TikTok, you know, they’re going to these kind of places I didn’t even anticipate, and that allows me to then say, “Okay, I need to actually really scaffold this down.” I don’t want this to take two weeks of my time, I want them to find a credible source and then I ended up giving them a list of basically five places they should look. And you might say that is oversimplifying it, but again, this was just a step in the process of a larger PSA that they needed to make. So I needed to really like clamp that down. But I think if we can have students start in class and actually watch what they do, that gives us a lot of information about how big a step they’re willing to take on. And again, the little creative process chart that I put in the video that I created, I think a lot of creative practitioners, people who are really established, they’ve internalized this process, and they even don’t even want to put it on the line. They’re just like, “Oh, you bounce around, you know, you go back and forth and it’s not a linear thing.” And that’s not actually helpful to a new student who’s really nervous, who’s really stressed, who’s in school for the first time. They’re paying a ton of money to go to art school and their grandparents are really pissed because they should be an accountant. That’s intense. And so these students really need things broken down. And I think that just an awareness of our own expertise is a good starting point, and taking our cues from the students.

Rebecca: This has been really interesting. We always wrap up by asking what’s next? \

Natasha: What’s next, Martin? [LAUGHTER]

Martin: What’s next for me is to finish this book I’m working on with Cassandra Horii. We’ve been doing this project together for the past decade or so. I’ve been making photographs at colleges and universities across the country. We use those photographs that I make in classrooms in faculty teaching to help faculty think about their teaching practice. So we do this form of photo0based teaching consultation. So we’re putting those thousands of thousands of photographs together into a book. And we’re working with the same press that Natasha and I were with, West Virginia University Press, on that book. As far as my other life as an administrator in higher education, what’s next is figuring out what fall semester looks like. How are we engaging students? And in what space are we engaging them? Are courses going to be offered HyFlex, we don’t know? Are any courses going to be conducted face to face? Some of them have to be. You can’t teach arc welding at a distance. There’s some of that that has to be hands on. So figuring out exactly how we’re engaging students in this next phase is what’s next for me.

Natasha: I’m going really micro because these are really big questions. I’m going to keep working on the curriculum for my ESL class. I am now not in faculty development officially anymore at my university in an official role. My current role is that I coordinate and write the curriculum for one level of the English for non-native speakers at the Academy of Art University. And it’s exciting. So I’m working on actually integrating more of the anti-racist ecologies. I’m working on incorporating even more creative process readings and practices into my ESL course in the new zoom world, also really trying to figure out how to get students conversation practice in zoom. That’s the really tough one. So, I’m very much just kind of looking [LAUGHTER] about two feet in front of myself right now. And boy, as far as the bigger issues go, I don’t know. Let’s check in again in the fall. [LAUGHTER]

Rebecca: I think that’s fair.

Natasha: This afternoon, I’m going to make a creative genealogy for myself. I’m making a creative family tree, because I’m having my students do this next week when we start class and I’m going to do it for myself as a sample for them and also just to see what it’s like to go through that process. So that’s actually been really fun. That’s my fun thing that I’m doing.

Rebecca: It’s all about balance.

Natasha: Yeah.

John: Well, thank you. This has been fascinating. I really enjoyed reading through your book, and I’ve enjoyed your contribution to the Pedagogies of Care, and it’s been really great talking to you. Thank you.

Rebecca: Yeah. Thank you so much.

Natasha: John and Rebecca, it’s been a really fun conversation. Thanks so much for inviting us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]