306. Gender Bias and Timing of SETs

 A number of studies demonstrate gender bias in course evaluations. In this episode Whitney Buser, Jill Hayter, and Cassondra Batz-Barbarich join us to discuss their research that looks at the timing of when these gender differences emerge and theories for why they exist.

Whitney is the Associate Director of Academic Programs in the School of Economics at Georgia Tech. Jill is an Associate Professor of Economics in the College of Business and Technology at East Tennessee State University. Cassondra is an Assistant Professor of Business at Lake Forest College. Whitney, Jill, and Cassondra are the authors of an article entitled “Evaluation of Women in Economics: Evidence of Gender Bias Following Behavioral Role Violations.”

Show Notes

Transcript

John: A number of studies demonstrate gender bias in course evaluations. In this episode we discuss research that looks at the timing of when these gender differences emerge and theories for why they exist.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Whitney Buser, Jill Hayter, and Cassondra Batz-Barbarich. Whitney is the Associate Director of Academic Programs in the School of Economics at Georgia Tech. Jill is an Associate Professor of Economics in the College of Business and Technology at East Tennessee State University. Cassondra is an Assistant Professor of Business at Lake Forest College. Whitney, Jill, and Cassondra are the authors of an article entitled “Evaluation of Women in Economics: Evidence of Gender Bias Following Behavioral Role Violations.” Welcome Whitney, Jill, and Cassandra,

Whitney: Thank you for having us.

Cassandra: Thank you so much.

Rebecca: Today’s teas are:… Whitney, are you drinking tea?

Whitney: I am. I have some jasmine tea.

Rebecca: Always a good choice. Jill. How about you?

Jill: Harney and Sons Hot Cinnamon Spice.

Rebecca: Oh, that’s such a good choice. I love that one. It’s a family favorite at my house. How about you, Cassandra?

Cassandra: Yesterday, we made a sun tea on the porch. So it’s sweet peach tea.

Rebecca: This is a good variety. How about you, John?

John: And I have ginger peach black tea from the Republic of Tea.

Rebecca: So we’re combining choices here [LAUGHTER]. And I have Awake tea, despite the fact that it is early afternoon here.

Jill: I also had three cups of coffee this morning.

Rebecca: It’s one of the most popular kinds of tea, Jill.

John: We’ve invited you here today to discuss your research on gender bias instudent evaluation of instructors. Could you tell us how the study came about?

Whitney: Jill and I have been working on this for about six years, believe it or not. It’s been a long process for us. And actually at the very beginning we had a different third working with us. And the original three of us, we met at the conference, and we had just attended a session that talked about teaching evaluations. And afterwards, we just naturally began talking about this, because we all had these really, really strong feelings about teaching evaluations. All three of us at the time were young, young in our careers, young age wise. We were female PhD economists. And we were all earning tenure, or I think Jill had just earned tenure. But we’re all in this similar experience of having what we felt like was a very positive class climate, and a lot of camaraderie between ourselves and the students until the grades were returned for the first time. And then we could feel a definite shift and it was upsetting to all of us. We all got into this because we love teaching and we want to do a good job in that. It was just something that we were picking up on. So that was our anecdotal experience, Jill had a little data on it herself, because she would do mid-semester evaluations herself, just to gauge the class climate and see what students were needing. And I had an experience where in my first position, they did a surprise midterm evaluation, just to kind of see how the new professor was doing, that I didn’t know about. And I got glowing reviews from the students, everything was very, very positive, wonderful and six weeks later, same students but grades returned, evaluations looked a little different. And the comments were a bit different. So we had a little data to backup this idea too, and one thing if the people listening today haven’t read the literature, there’s an extensive literature on course evaluations. And it consistently finds gender bias in those. But the thing about that literature is it only looks at evaluations, which are typically done on the very last day of class, maybe even after that, maybe a couple of days before, but at the end of the semester. And we really haven’t seen anyone look into how these opinions of students evolve over the semester, or how students feel at the beginning or the middle of the semester. So that’s what we wanted to do with that. And in my opinion, and this is just me speaking here, Jill can have her own other motivations, or our other co-author that has worked with us before could feel differently. But for me, it was really important to acknowledge that society has come a long way in the past several years with gender bias. And I don’t think that modern students are shocked by female faculty any longer, I don’t think they have an explicit distaste for female faculty. Anecdotally, I feel that my students are actually happy when they meet me. And they have expectations of me to be warm, comforting, approachable. But I do think that when you expect someone to be more comforting and approachable, and they give you a grade back, that’s not always an “A” in a difficult quantitative subject like economics, you can get a bit of a Grinch Who Stole Christmas effect. I thought it was going to be one way and now my expectations are taken down. We all know no one likes that dopamine depletion of having expectations not met. So, to me, if we’re going to talk about gender bias, we really have to talk about it in this nuanced way, so that it doesn’t get automatically dismissed by people who don’t see an explicit bias and then say, “Oh, hey, there’s nothing here.” And then the last thing that I think is really important here for the motivation for the paper is that we have this expectation that bias would grow over the semester. So if bias grows over the semester, that means the earlier in the semester you evaluate, the smaller the bias will be. And one thing that the literature is missing is a very concrete objective way to deal with bias. What we were hoping to find was: move the evaluations up in the semester a bit, and you minimize or eliminate bias and that’s a concrete objective. Towards the end today, we’ll talk about what we actually found and whether or not we knew that. But that was one of the motivations.

Jill: So that’s how the original paper found in terms of motivation, but then Cassandra, she is a PhD in Psychology until she had read and she was doing work in the area. And she had reached out to Whitney and I. She had read our paper, she had read the results of our paper. And so then a second paper with Cassandra takes a more psychology approach in terms of a lot of what Whitney is talking about and Cassandra is going to talk about it later, with respect to the role-incongruity theory, social role theory, and she’s going to talk more about that later. And Whitneys described the motivation of that first paper, the second paper takes a very different perspective and looking at it from a more psych perspective. Cassandra, you might want to chime in?

Cassandra: Absolutely. I think you summarized it well, I joined the paper, as Whitney and Jill were trying to find a home for it. And we thought that our interests, though coming from very different backgrounds ,would blend nicely for this particular topic, as there’s a lot of scholarship in psychology that looks at understanding reasons behind this bias. And so I was brought in to really help kind of think about how do we frame that in a way that might appeal to even a broader range of audiences.

Rebecca: At the beginning of the paper, and Whitney, you’ve kind of pointed to this today about being a young faculty member, you also noted in the paper that women are underrepresented among economics faculty, especially at the level of full professors. Can you tell us a little bit about the extent of this under-representation?

Jill: Women have earned more than half the doctoral degrees for over a decade. But particularly among tenure-track faculty are underrepresented. In the paper we cite 36% of full professors are females. In economics, that’s a smaller percentage, 17 and a half percent of full professors are females, in the area of economics, although 35% of PhDs in econs represent females. It’s a smaller percentage of female faculty receiving full professor rank in economics. That’s what we mean by that under representation. In terms of economics, specifically, it’s oftentimes left out of the STEM fields, and depending on which university or college that you’re out at, economics can sometimes could be found in the social sciences and in the arts and sciences, or it can be found in the business school. So at my institution, Whitney’s institution, I believe, and Cassandra’s I think we’re all represented in the business school. But sometimes, you know, economics wanted to put in there with the social science field, it’s not thought of as being this more quantitative, heavy subject, and it oftentimes is, it is by nature of it. And so females in those more math heavy classes, like the STEM classes. I think my students when I started off, and I think Whitney was getting at this, with us being more junior faculty members. I can considered by students peer, instead of the professor in the course. And that made it tough, because to Whitney’s point about that returning grade feedback and the perception that students had of me a day one versus midway through the course, I was now coming across as someone that was handing back maybe less than 100% or “A” grade. So in my business school, my principles of economics courses are required. They might not even want to be in there, but they have to be in there to get a business degree. Earlier on, that was a challenge I faced, I’m 13 years into my career. I’m going up for full professor this summer. But starting off was really a challenge. And I remember having female mentors in my graduate program. They tried to prepare me for this, they tried to say it’s going to be challenging early on, you’re going to have to go against some of these perceptions, alot of the perceptions that we measure in this paper..

John: To what extent is the underrepresentation of women faculty due to a cohort effect where women have become a larger share of PhD economists in the last few decades, but that was less true 20 or 30 years ago and how much of it might be due to the impact of gender bias on evaluations on career pathways for women?

Jill: Really what this paper looks at, the standard evaluations of teaching and the bias or potential for bias, that exists there. So I’ll just speak to that and that where I currently am, evaluations of teaching are weighted heavily for retention of faculty, promotion of faculty, tenure and promotion decisions. And then when we’re hiring new faculty, looking at any previous course evaluations and experience with teaching. At every level in academia, these are used as some gauge for teaching effectiveness. I think one of the questions that we’re looking at and accrediting bodies are looking at is whether or not this is the measure that should be used. And looking at different measures that might be options for measuring teaching effectiveness, we know that they’re flawed, that our study is showing that they’re flawed, but also previous literature has suggested that they’re flawed as well. And so the fact that for most schools, this is the single measure that’s being captured… and I know that it’s different depending on again, at my institution, some departments don’t give them a whole lot of weight in tenure and promotion decisions. But certainly, my experience in my College of Business and Technology that these are weighted heavily. And so in thinking about a junior faculty member starting off, when Whitney and I met at the conference, if my evaluations were lower, I’m putting a lot of time into my teaching and improving and bringing up those scores. My male colleagues, in discussion just with them, didn’t have the same experience that I was having with respect to these SETs. And so we think about allocation of time and resources as a tenure track junior faculty member, I’m putting more in what I would consider just catching up, getting those SET scores higher, so that it’s reflected in my tenure and promotion packet. And that’s less time that I’m allocating toward research or other things. That’s my view on it. I think Whitney has a couple other thoughts on that.

Whitney: One of the things we tried to make clear in the paper is that the literature is very clear that evaluations do have a gender bias. And if these evaluations are being used, and they are, in hiring decisions, annual evaluations, promotion, tenure evaluations, and merit pay raise decisions, then they’re being used at every single level of advancement. It’s not one small piece. It’s a piece that’s used throughout and very integrated late in the process.

Rebecca: You mentioned at the top of our interview that the second paper shifts more towards psychology, and specifically describes ways in which both social role theory and role-congruity theory may explain the bias against female faculty in student evaluations. Can you briefly summarize these arguments for our listeners?

Cassandra: So social role theory was a theory that has been put forth for decades by Alice Eagly, a very prominent scholar in the social psychology world, as well as her colleagues. And this has been used as a framework to really understand the complexities and origins of gender gaps in our workplace in particular, whether that be inequities and experiences, the expectations that are different for women, and of course, the outcomes such as promotion at work. Essentially, social role theory suggests that the reason we see these gender inequities today in society or that they originated from men and women being distributed into social roles based on physical sex differences, so that women biologically were able to have children, men, on average, were physically stronger, which those differences 1000s of years ago, had an evolutionary benefit to a well functioning society, people were supporting in the ways in which they were best equipped to do so. And the assignment of men and women into these roles led get them to adapt role-specific qualities and skills. So women who were bearing children were friendly, helpful, sensitive, concerned with others, kind, caring. We refer to these now as more communal qualities, and men and the provider, the protector, role led them to have attributes such as ambition, being assertive, authoritative, dominant. These are qualities that now we label as agentic. So while technology of course has since caught up and made these biologically driven role assignments unnecessary, society continues to see a division of labor along these lines in the modern world and society at large. And society at large still holds the belief that women do possess these traits, and should possess these traits, these more communal qualities, and men do and should possess more of these agentic. Relatedly, role-congruity theory helps us understand the consequences when men and women fail to fulfill these expectations. And we know the failure to fulfill these expectations are more consequential for women, this experience of bias driven from the failure to behave in communal ways. In other words, violating these cultural expectations can be seen in all areas of society, but particularly in traditionally male-dominated positions, like college professors, or in male-dominated fields like economics [LAUGHTER]. And so women that are in these roles are already going to experience some degree of backlash for being in gender-incongruent positions. But that is especially true if they are also going to behave in traditionally more agentic ways, being more assertive, demonstrating their power, which we argued was what was occurring when you give critical feedback back to students.

John: To approach this, you gave evaluations to students at two different points of the semester. Could you tell us a bit more about the study design, how large the sample was and how many faculty and institutions participated in the study?

Whitney: Sure, we had a really rich data set for this study. That’s one of the reasons we were able to get two different papers out of it, and maybe even some future research, because we took all of this data, and we collected it in person on paper and entered it, which was an arduous process. As I said, we had been working on this project for about six years, about a year and a half of that was just data collection. And we have a lot of people to thank that did that for us for no author credit on this paper, so we had males and females across the United States gathering that data for us, that we’re really appreciative to have. So in the end, we wound up with about 1200 students in total, we weren’t quite 50/50, we were 60/40, favoring men, which is typical for economics classrooms, even though it is required in a lot of majors (that’s where you’re getting a lot of the women taking it). And like you said, John, we surveyed them twice. We surveyed them on the second day of class, we wanted as close to a first impression as possible without having a major sample issue with drop/ad. And then we surveyed them the day after they got their first midterm grade back. So we got the first impression, and then we got the way that they felt after they had had their first grade returned. We did this at five different colleges and universities, we had three male professors contributing data and four female professors contributing data. One of the big questions that people have asked us over the time is “Well, how does race play into this?” And that’s something that’s beyond the scope of our research, I will say that we only had one underrepresented minority in our sample, again, typical of economics professors, it was one of our male instructors. So, we would expect a downward bias from race and maybe an upward bias from gender, or getting those two, at least watching one another out in the paper. And when we asked these students about how they felt after their grades were returned. This was about four weeks into the semester, so still pretty early in the semester. What we did was we really wanted to ask about the specific qualities that had been hypothesized in the literature as drivers of bias or drivers of differences. So we just asked students to rate their instructor on a bunch of different qualities. Cassie really helped us out here because she came in and she says, “Well, you know, we can categorize these qualities into communal qualities and agentic qualities and neutral qualities…” which was really the way to approach it because of course, we get different things in communal versus just qualities. So we asked our students things like: “How knowledgeable do you find your professor? How challenging? Do you find them to be approachable? Do you find them to be caring? Are they interesting?” And then we asked a couple of very general questions: “Would you recommend the course?” All of this set us up to have a really nice dataset where we could look between genders and across time as well.

Rebecca: So I think everyone’s probably dying to know exactly what you found. [LAUGHTER]

Jill: I’m just going to provide an overview of the results because we do a number of different specifications and use different econometric methods in the findings. And so you can get all of those results there in detail. But in general, on the second day of class, we find that women are receiving lower ratings across the five agentic and gender-neutral instructor characteristics that we measured. They were rated higher on that second day of class on those more communal characteristics. And not all of those differences were statistically significant. Immediately after the first exam grade was returned to students, women were receiving lower ratings for all seven measured characteristics. Each difference was significant except for those caring and approachable, more communal characteristics. And then men were now having higher ratings in all the different aspects relative to time, or the second day of class. Over time, what we see was that men’s evaluations were getting higher on all characteristics from the second day of class to the period after the first exam was returned. And then in contrast, women’s evaluations were not trending upward. So we had a couple that were staying the same, but overall, they were going down. So those are just some overview findings. Again, those more specific results, by specification, can be found in the paper.

John: We will include a link to both papers in the show notes too, so people can go back and review them. To summarize, what you found is there was relatively weak evidence of significant gender bias on the second day of class, but that gap increased fairly dramatically after the first graded exam. So what do you attribute that change to, was it because of the feedback students were getting from grades as Whitney had mentioned before?

Whitney: We were attributing, and Cassie can talk about this with more authority on the theoretical point, but we’re attributing that to backlash theory, this idea that if I expect one thing, and I don’t get it, there’s this need to back off so that things go in congruence.

Cassandra: Exactly, Whitney is spot on there. What we thought this was evidence of was women behaving in gender incongruent ways, women are supposed to be warm and caring and friendly. And when you get a perhaps grade that maybe wasn’t an “A,” that feels harsh and critical, and a woman is asserting their power and dominance in the classroom, which again, they already are in a male dominated field profession. And those two things together combined can result in this backlash.

Rebecca: So if we take these findings, and think institutionally, what are some things that institutions might want to think about moving forward?

Whitney: That’s a good question. If you remember, from the very beginning, we were saying, we’re really hoping to find this nice objective concrete solution, we anticipated finding it through timing. And that’s what I would really like to do with future research is to be able to find something concrete and objective to treat this with. We weren’t able to do that because we found bias from the beginning. And we found that it came so quickly in the semester that it’s not something that we can just move back evaluations to midterm or something like that. Since we can’t do that, we’ve talked about other ways for institutions to take this. And one takeaway really is just an awareness that these gender biases exist and that these evaluations are flawed. This is really well established in the literature, but not necessarily in the general sphere of knowledge. When we published this paper, Georgia Tech did a little feature in their daily digest, and I had two female engineering faculty email me and say, “I knew this in my gut for years, but nobody’s ever quantified it.” That to me, is just evidence that it’s not in the general sphere of knowledge, even though the literature defines it well. Some of the impact of the concrete solutions that we have seen is we’re seeing a lot of schools and accreditors, like AACSB, they’re starting to require multiple indicators of teaching effectiveness and evaluation. So evaluations and peer reviews, or maybe something else to see the observation, something to that effect to where we have more of a global and inclusive way to look at someone’s teaching effectiveness. So this is a great takeaway, hopefully that will reduce the weight of the impact of evaluation just by having other factors in there. And just one final point that I want to make. And this is just a really big sticking point to me for the paper is that all of us are researchers, we all deal with statistics and statistical significance, and robust research methods. And then when those of us in Chair and Dean roles go to look at evaluations, all the sudden, all that training completely goes out the window, and we look at the difference between a 4.2 and a 4.4. And I know those differences sound really small, they are that small. And we say, “Oh, well, this person does better than this person, this person deserves to be hired over this person.” Never in our research, or in a formal presentation, would we ever compare two means that small without significance testing, number one, and without making sure they’re actually comparable, and say, “Oh, there’s a difference.” It’s just something that I think we need to recognize, we would not recognize this as good research or good methodology in any other area of our work. It’s just something that we should keep in mind as we move forward with this.

John: Now, you mentioned the use of peer evaluations as another way of providing, perhaps, more balance, but might they be subject to the same type of bias?

Whitney: Yeah, all the things that we would see for student evaluations, I can imagine how you would see with peer evaluations as well.

Jill: But there are creative ways to do peer evaluations that I think here at ETSU, we have a Center for Teaching Excellence. And I’m confident Georgia Tech, and Lake Forest has their own version of that. And so there are creative ways. And again, not that SETs are necessarily bad, but knowing what we know about the flaws in them, that, coupled with an additional measure or two, can be a lot more insightful, I think, to the teaching effectiveness, like true teaching effectiveness of instructors.

John: And one thing I’m wondering is if the measured effect might be larger in economics, because at least at many institutions, grades and economics and STEM classes are often lower, which might magnify the effect of this difference. It would be interesting if there was to be a study that also included some classes, maybe in humanities, to see if perhaps there’s less of an effect because of that role-incongruity issue there. It may not appear to be as severe in disciplines where grades across the board tend to be higher.

Whitney: I think you’re right about that, most people when they take economics, it’s a required class and certainly the grades are a big factor, then the two things that showed the most significance outside of our key variable of interest was interest in economics, and expected grade. Those were the things that across the board… now we still found gender bias controlling for those things, but it mattered.

Rebecca: So we talked a little bit about things that institutions might want to start thinking about: institutional policy and things that might shift how we use teaching evaluations. Are there any other strategies that institutions or instructors can use, or adopt, to try to reduce this bias in the short term?

Cassandra: That’s really the million dollar question. Because this type of bias exists in a lot of different domains, whether we’re talking managers and their subordinates, teachers and their students. One thing that’s often suggested or recommended is simply making people aware that this bias exists, and providing training on how to better approach evaluations, whether that’s how to use a rating scale and ensuring that you aren’t engaging in a halo effect, for example. Another strategy is requiring that people justify their ratings that are provided with qualitative comments… that if you’re just asked to fill out on a scale, on how competent is this person? Well, bias may creep in more if you aren’t asking for a justification of why that particular rating was given for competence. A last recommendation that I’ll share here is making these evaluations more public. So if there are a couple of people, say peers, that are evaluating myself or Whitney or Jill in the classroom, well, they need to come together, share and publicly disseminate their evaluations that they had given to us. This social accountability can help to mitigate bias and for people to ensure that the ratings that you’re giving are, in fact justified.

John: So we’ve got a long ways to go with this. It’s a problem that’s been recognized for quite a while with a lot of studies. But there hasn’t been that much done to address that. And those are some good suggestions that institutions may want to try. We always end with the question: “What’s next?”

Cassandra: [LAUGHTER] That’s a good question. Of course, I think that the three of us collectively would say we do hope that administration and decision makers start asking questions about their use of student evaluations of teaching and how they might seek to mitigate this bias, based on the recommendations Whitney had already shared. But we also hope that women faculty perhaps feel more empowered to advocate for themselves when it comes time for promotion and tenure decisions to be made. My Institution, a part of the promotion process, is writing letters, and going through interviews. So speaking to this, bringing an awareness to the people who are making the decisions that this exists, and that it is not just an opinion, that there is empirical evidence of its existence. But we are really interested in exploring more fully how providing feedback, particularly critical feedback, like in our study, where the professors are giving back grades might impact the perceptions of men and women in other contexts as well. So is this a phenomenon we would see, for example, between a manager and their team? Do people respond differently to critical feedback from a manager because of their gender? And how much are these differences, perhaps, driven by perceptions of how communal or agentic they are in their delivery of that feedback? So in other words, are we seeing the same pattern in other contexts? Ultimately, we hope that by better understanding how perceptions of communion and agency impact interactions that women have at work, particularly women in male-dominated or gender-atypical roles, this greater understanding will allow us to also discover ways to alleviate some of that backlash through more targeted interventions and training and perhaps better timing. Because at a minimum, it’s important to highlight the various ways gender bias continues to persist in our society. Because without that awareness, nothing can be changed.

John: Whitney, Jill?

Jill: I think that was great. [LAUGHTER]

Whitney: Yeah, I think, Cassie, you did a great job. And Cassie certainly helped us out with bringing formal language and theory to things that we felt as intuitive and we felt in our gut as important. We don’t have a lot of language for that in the economic space. And so blending these two disciplines together has been very helpful for looking at the situation.

Rebecca: Well, thank you all for joining us. And the research that you’re doing is really important and impactful. So we hope our listeners will use it.

Whitney: Thank you so much.

Cassandra: Thank you.

Jill: Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

305. 80 Ways to Use ChatGPT in the Classroom

Faculty discussions of ChatGPT and other AI tools often focus on how AI might interfere with learning and academic integrity. In this episode, Stan Skrabut joins us to discuss his book that explores how ChatGPT can support student learning.  Stan is the Director of Instructional Technology and Design at Dean College in Franklin, Massachusetts. He is also the author of several books related to teaching and learning. His most recent book is 80 Ways to Use ChatGPT in the Classroom.

Show Notes

Transcript

John: Faculty discussions of ChatGPT and other AI tools often focus on how AI might interfere with learning and academic integrity. In this episode, we discuss a resource that explores how ChatGPT can support student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by

John: , an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Stan Skrabut. Stan is the Director of Instructional Technology and Design at Dean College in Franklin, Massachusetts. He is also the author of several books related to teaching and learning. His most recent book is 80 Ways to Use ChatGPT in the Classroom. Welcome, Stan.

Stan: Well, thank you ever so much for having me on. I have been listening to your podcast since the first episode, you guys are crushing it. I recommend it all the time to my faculty. I’m excited to be here.

John: Thank you. And we very much enjoyed your podcast while you were doing it. And I’m hoping that will resume at some point when things settle down.

Rebecca: Yeah, we’re glad to have you here.

Stan: Yeah, thanks.

John: Today’s teas are:… Stan, are you drinking any tea?

Stan: A little bit of a story. I went over to the bookstore with the intent of getting tea. They had no tea in stock. I went to the vending machine on the same floor. The vending machine was down. I went to another building. I put in money. It did not give me tea. I’m stuck with Mountain Dew. I’m sorry. [LAUGHTER]

Rebecca: Not for lack of trying. Clearly. [LAUGHTER]

Stan: I tried. I tried.

Rebecca: I have some blue sapphire tea.

John: And I have Lady Grey.

Rebecca: You haven’t drink that in a while John,

John: no. [LAUGHTER]

Rebecca: Little caffeine today huh. [LAUGHTER]

John: Yeah well i am back in the office, I’ve returned from Duke and I have more options for tea again.

Rebecca: That’s good. So Stan, we invited you here today to discuss 80 Ways to Use ChatGPT in the Classroom. What inspired you to write the book?

Stan: Well, I’m an Instructional Technologist and my responsibility is to help faculty deliver the best courses possible. And in November 2022, ChatGPT came onto the scene and in December, faculty are up in arms, “Oh, my goodness, this is going to be a way that students are going to cheat and they’ll never learn anything again.” And as an instructional technologist, I see technology as a force multiplier, as a way to help us do better things quicker, easier. And so I didn’t feel threatened by ChatGPT. I’ve been looking at the horizon reports for the last 20 years. And they said, “AI is coming. It’s coming. It’s coming. Well, it’s here.” And so it was just a matter of sitting down in January, write the book, publish it, and provided a copy to all the faculty and we just started having good conversation after that. But the effort was that we should not ban it. That was the initial reaction; that this is a tool like all the other tools that we bring into the classroom.

Rebecca: Stan, I love how you just sat down in January and just wrote a book as if it was easy peasy and no big deal. [LAUGHTER]

Stan: Sell, I will have to be honest, that I was using ChatGPT for part of the book, it was a matter of I asked ChatGPT kind of give me an outline, what would be important for faculty to know about this, so I got a very nice outline. And then it was a matter of creating prompts. And so I’d write a prompt and then I would get the response back from ChatGPT. It was a lot of back and forth with ChatGPT, and I thought ChatGPT did a wonderful job in moving this forward.

John: Most of the discussion we’ve heard related to ChatGPT is from people who are concerned about the ability to conduct online assessments in the presence of this. But one of the things I really liked about your book is that most of it focuses on productive uses by both faculty and students and classroom uses of ChatGPT because we’re not always hearing that sort of balanced discussion about this. Could you talk a little bit about some of the ways in which faculty could use ChatGPT or other AI tools to support their instruction and to help develop new classes and new curriculum?

Stan: Yeah, absolutely. I guess first of all, I would like to say that this is not going anywhere. It is going to become more pervasive in our life. Resume Builder went out and did a survey of a couple thousand new job descriptions that employers were putting out. 90% of them are asking for their employees to have AI experience. As higher education, it’s upon us to make sure that the students that are going out there to be employees know how to use this tool. With that said, there has to be a balance. In order to use the tool properly, you have to have foundational knowledge of your discipline. You have to know what you’re talking about in order to create the proper prompt, but also to assess the proper response. With ChatGPT sometimes it doesn’t get it right… just how chat GPT is built, it’s built on probabilities that these word combinations go together. So it’s not pulling full articles that you can go back and verify, kind of like the human mind has been working. We have built up knowledge all these years. My memory of what happened when I was three, four or five years old is a little fuzzy. Who said what? I’m pretty confident what was said. I’m pretty confident, but it’s still a little fuzzy. And I would need to verify that. So I see ChatGPT as an intern, everybody gets an intern, now. They do great work at all hours, but you as the supervisor still have to verify the information is correct. Back to the classroom, students can’t or should not, or regardless of who’s using it, should not just hit return on a prompt, and then rip that off and hand it in to their supervisors or instructor without verifying it, without making it better, without adding the human element to working with the machine. And that is, I think, where we can do lots of wonderful things in the classroom. You know, from the instructor side of go ahead and use this for your first draft. Now turn on the review tools that track changes and show me how you made it better, as you’re working towards your final product. Instructors can go ahead and craft an essay, craft out some supposedly accurate information from ChatGPT. tThrow it in the hands of the students and say: “Please, assess this. Is this right? Where are the policies? Where are the biases? Tell me where the gaps are. How can we make this better?” And using it to assess it.” Those are some initial ways to start asking students or using it in the class. I don’t know if I’m tapping into all the things. There’s just so many things that you could do with this thing.

John: And you address many of those things in the book. Among those things that you address was having it generate some assignments, or even at a more basic level, having it develop syllabi, or course outlines and learning objectives and so forth, for when faculty are building courses.

Stan: Oh, absolutely. We have a new dean at our School of Business. And he came over and wanted to know, “Tell me a little bit more about ChatGPT, how we can use this. They’re looking at creating a new program for the college. And it’s like, “Well, let’s just start right there.” What are the courses that you would have for this new program and provide course descriptions, titles, and descriptions? Here comes the list of 10, 12 different courses for that particular program. Okay, let’s take this program, what are the learning outcomes for this particular program? So we just copied and pasted, asked for learning outcomes, here comes the list of outcomes. Now for these different outcomes, provide learning objectives. And it starts creating learning objectives. And so you can just continue to drill down. But this moves past the blank page. Normally you’d bring in a group of faculty to work on that program, what are your ideas and send everybody off, and they would pull ideas together and you would start crafting this. This was done in 30 seconds. And now okay, here’s the starting point for your faculty. Where are the problems with this? How can we make it better? Now go. Instead of a blank page, starting with nothing? That was one example. But even for your course, using ChatGPT, having a course description, you can ask it to say, provide me a course plan for 16 weeks. What would I address in this? What would be the different activities? Describe those activities. If you want it to have the activities use transparent assignment design, it’ll craft it in that format. It knows what transparent assignment design is, and it will craft it that way. And then going back to assessment, you can build content. So looking at that OER content, open education resources, that it can get you a jumpstart on that OER content. What are gaps that I want or taking content that’s there and localizing it based on your area to say here we are in New England, Massachusetts, specifically, I need an example. Here’s the content that we’re working with. Give me an example, a case study, and it will craft a case study for you. It allows you to go from that zone of drudgery to your zone of genius very rapidly. I’ve been working on a new book, and got down to the final edits, and I was like, “Oh, I’m missing conclusions to all these different chapters.” I just fed the whole chapter in and said, “Could you craft me a conclusion to this chapter?” And it just knocked it out. I mean, I could do it. But that’s my zone of drudgery, and I’d rather be doing other things.

Rebecca: It’s interesting that a lot of faculty and chairs and administrators have been engaged in this conversation around ChatGPT quite a bit, but many of them haven’t actually tried. ChatGPT. So if you were to sit down with a faculty member who’s never tried it before, what’s the first thing you’d have them do?

Stan: This is an excellent question because I do it all the time. I have a number of faculty members that I’ve sat down, looked at their courses and say, “What is the problem that you’re working with? What do you want to do?” And that’s where we start. We say “What is the problem that you’re trying to fix?” ChatGPT version three had 45 terabytes of information it was given. They say the human brain has about 1.25 terabytes. So this is like asking thirty-some people to come sit with you to work on your problem. One class was a sports management class dealing with marketing. And they were working with Kraft enterprises that has the Patriots, and working on specific activities for their students and developing marketing plans and such. We just sat down with ChatGPT and started at a very basic level to see what we could get out of it. And the things we weren’t happy with, we just rephrased it, had it focus on those areas, and it just kept improving what we were doing. But, one of the struggles that I hear from faculty all the time, because it’s very time consuming, is creating assessments, creating multiple choice questions, true and false, fill in the blank, all these different things. ChatGPT will do this for you in seconds. You feed all the content that you want, and say, “Please craft 10 questions, give me 10 more, give me 10 more, give me 10 more. And then you go through and identify the ones you like, put them into your test bank. It really comes down to the problem that you’re trying to solve.

John: And you also know that it could be used to assist with providing students feedback on their writing.

Stan: Absolutely

John: …that you can use it to help generate that. Could you talk a little bit about that.

Stan: We’re right now working with the academic coaches. And this is one of the areas to sit down. I’m also not only the Director of Instructional Technology and Design, but also my dotted line is Director of Library. So I’m trying to help students with their research. And the writing and the research go hand in hand. So from the library side, we look at what the students are being assigned, and then sit down and just start with a couple key terms or phrases, keywords that we want and have ChatGPT to give us ideas on these different terms. And it’ll provide ten, twenty different exciting ideas to go research. Once again, getting past the blank page. It’s like “I gotta do an assignment. I don’t know what to do.” It could be in economics, I don’t know what to write about in economics, it’s like, well, here pull these two terms together, and what does it say about that?” So we start at that point. And then once you have a couple ideas that you want to work with, what are some keywords that I could go and start researching the databases with, and it will provide you these ideas. It’ll do other things, it’ll draft an outline, it’ll write the thing if you want it to, but we try to take the baby steps in getting them to go in and research but getting pointed in the right direction. On the writing side, for example, I have a class that I’m going to be teaching at the University of Wyoming to grad students. I’m going to introduce ChatGPT. It’s for program development and evaluation, and I’m going to let them use ChatGPT to help with this. One of the things that academic writers struggle with is the use of active voice. They’re great at passive, they’ve mastered that. Well, this will take what you’ve written and say, “convert this to active voice” and it will rewrite it and work on those issues. I was working with one grad student and it was after playing with ChatGPT a couple of times, she finally figured out what really was the difference and how to overcome that problem and now she is writing actively, more naturally. But she struggled with it. With ChatGPT, you can take an essay, push it up into ChatGPT and say, “How can I make this better?” And it will provide guidance on how you can make it better. You could ask it specifically, “How can I improve the grammar and spelling without changing any of the wording here.” It’ll go and check that. So for our academic coaches, because there’s high volume, this is another tool that they could use to say, “Here’s the checklist of things that we’ve identified for you to go work on right away,” not necessarily giving solutions, but giving pointers and guidance on how to move forward. So you can use it at different levels and different perspective, not where it does all the work for you but you could do it incrementally and say, “here assess this and do this.” And it will do that for you.

Rebecca: Your active and passive voice example reminds me of a conversation I had with one of our writing faculty who was talking about the labor that had been involved previously of making example essays to edit of to work on writing skills. And she just had ChatGPT write things that [LAUGHTER] are of different qualities, and to compare and also to do some editing of as a writing activity in one of her intro classes.

Stan: Absolutely. What I recommend to anyone using ChatGPT is start collecting your prompts, have a Google document or a Word document, and when you find a great prompt, squirrel it away. Some of the workshops that I’ve been giving on this, I demonstrate high-level prompts that are probably two pages long that you basically feed this basic information to ChatGPT and it talks everything about the information that you’re going to be collecting, how you want to collect it, how you want it to be outputted, what items are you going to output, and you’re basically creating this tool that you can then call up and say, for example, developing a course, that it will write the course description, give you a learning outcomes, recommended readings, activities, and agenda for a 16 week, all in one prompt. And all you do is say “this is the course I want” and let it go. It’s amazing what problems that we can build this tool just like we build spreadsheets, we build these very complex spreadsheets, to do these tasks. We can do the same with Chat GPT, we just have to figure out what the problems we’re trying to solve.

John: Our students come into our classes with very varied prior preparation. In your book, you talk about some ways in which students can use ChatGPT to help fill in some of the gaps in their prior understanding to allow them to get up to speed more quickly. Could you talk about some ways in which students can use ChatGPT as a personalized tutor,

Stan: I’m going to take you through an example that I think can be applied for students. A student comes to your class. Ideally, they’re taking notes, one of the strategies that I use is I have my notebook, I’ll open my notebook, and I’ll turn on otter.AI, which is a transcription program. And I will go over my notes, I will basically get a transcription of those notes, I can then feed that transcription into ChatGPT and say clean it up, make a good set of notes for me. And it will do that. And then I can build this document and then I can review what we did in class, build a nice clean set of notes, and have that available to me. Over a series of setw of notes, I could do the same thing by reviewing a textbook and highlight and talk about, transcribe key points of the textbook or I can cut and paste. And then I can feed that information into ChatGPT and say, “Build me a study bank that I can build a Quizlet, for example, or I need to create some flashcards on what are the key terms and definitions from this content?” Here you go. Create some flashcards from that material. It could be that no matter how great the instructor is, I still don’t get it. They introduced a term that is just mind boggling, and I still don’t get it. And so I can then ask ChatGPT to explain that at another level. They talk about non-fiction, some of the best non-fiction books or the most popular that are out there getting on the bestsellers list, they’re written at a certain grade level. And I know that I write typically higher than that grade level, I can go ask ChatGPT to rewrite it at a lower grade level. I could, as a student, ask ChatGPT, to give an explainer at a level that I do get to understand. Those are certain ways that you can do this. And you basically can build your own study guides that have questions that have examples of all the materials, so you can feed that material in and get something out, just enhance it. And I think for faculty, this is also an easy way to create good study guides, that you can get the key points and build the study guides a lot easier, just going with the blank page and trying to craft it by hand, can be very difficult. But if you already have all your material, you feed it in there, and then say here, let’s build a study guide out of this year with some parameters, definitely much more useful.

Rebecca: We’ve talked a lot about how to use ChatGPT as an individual, either as an instructor or as a student. Can you talk a little bit about ways that instructors could use ChatGPT for in class exercises or other activities?

Stan: Absolutely. And I’m sorry, some of the examples other folks have actually contributed first, and I saw him and I thought they were just brilliant, but I don’t have their names right in front of me. So I apologize ahead of time. But as an instructor, I would invite ChatGPT into the classroom as another student. We call it Chad, Chad GPT and bring Chad into the classroom. So you could have an exercise in your classroom, ask the students to get into groups, talk about an issue, and then up on the whiteboard, you start getting their input, you start listing it. And then once you’re done, you can feed Chad GPT the same prompt and get the list from Chad GPT, and then compare it to what you’ve already collected from the students, what their input has been. And from there, you can do a comparison, like “We talked about that, and that, and that, oh, this is a new one. What do you think about this?” And so you can extend the conversation by what Chad GPT has provided? …and there I go, Chad, I’ll be hooked on that for a while. But you can extend the conversation with this or if students have questions that are coming up in class, you can field that to the rest of the class, get input and then say “Okay, let’s also ask Chad, see what Chad has to say about that particular topic?” Those grouping exercise we typically do the think-pair-share exercise, well part of that is each student gets to get Chat in that group. So, each group you can have Chad come in where they have to discuss, they have to think about it first, write something down, pair, discuss it, then add ChatGPT into the mix, talk about it a little bit more, and then share with the rest of the class. Lots of different ways that you can bring this into the classroom, but I bring it right in as another student.

Rebecca: Think-pair-chat-share. [LAUGHTER]

Stan: Yep. And that’s that mine that actually somebody was clever enough, they found that. I just happen to glom on to it. But yeah, definitely a great way of using it. It’s a new tool. We’re still figuring our way, but it’s not going away.

Rebecca: So whenever we introduce new technology into our classes, people are often concerned about assessment of student work using said technologies. So what suggestions do you have to alleviate faculty worry about assessing student work in the age of ChatGPT?

Stan: Well, students have been cheating since the beginning of time. That’s just human nature. Going back to why are they cheating in the first place? In most cases, they just got too much going on, and it becomes a time issue. They’re finding the quickest way to get things done. So ensuring that assignments are authentic, that they’re real, they mean something to a student ,is certainly very important in building this. The more it’s personally tied to the student, the harder it is for ChatGPT to tap into that. ChatGPT is not connected to the internet yet. So having current information, that’s always a consideration. But I would go back to the transparent assignment design, and part of the transparent assignment design that is often overlooked is the why. Why are we doing this. If you use ChatGPT to do this, this is what you’re not going to get from the assignment. So, when building those assignments, I recommend being very explicit that yes, you can use ChatGPT to work on this assignment, or no, you cannot, but here’s why. Here’s what I’m hoping that you get out of this. Why this assignment’s important. Because otherwise, it just doesn’t matter. And then when I have an employee that just simply hits the button and gives me something from ChatGPT, I’m going to ask, “Why do I need you as an employee? Because I could do that. Where’s the human element? …bringing that human element into it, why is thisimportant?” What learning shortcut or shortcutting you’re learning, if you just rely on the tool and not grasp what the essence of this particular assignment is. But I think it goes back to writing better assignments… at least that’s my two cents on it.

Rebecca: Thankfully, we have ChatGPT for that.

John: For faculty who are concerned about these issues of academic integrity, certainly creating authentic assignments and connecting to individual students and their goals and objectives could be really effective. But it’s not clear that that will work as well when you’re dealing with, say, a large gen-ed class, for example. Are there any other suggestions you might have in getting past this?

Rebecca: John? Are you asking for a friend? [LAUGHTER]

John: [LAUGHTER] Well, I’m gonna have about 250 students in class where I had shifted all of the assessment outside of the classroom. And I am going to bring some back into the classroom in terms of a midterm and final but they’re only 10 and 15% of their grade, so much of the assessment is still going to be done online. And I am concerned about students bypassing learning and using this, because it can do pretty well on the types of questions that we often ask in introductory classes in many disciplines.

Stan: That’s a hard question, because there’s certainly tools out there that can identify where it suspects it’s been written by AI. ChatGPT is original text so you’re not dealing with plagiarism, necessarily, but you’re dealing with, it’s not yours, it’s not human written. There are tools out there, but they’re not necessarily 100% reliable. Originality.AI is a tool that I use, which is quite good, but it tends to skew, everything is written AI. TurnItIn, they’ve incorporated technologies into being able to identify AI, but it’s not reliable. This honestly comes down to really an ethics issue, that folks who do this feel comfortable in bypassing the system for the end game, which is to get a diploma. But then they go to the job and they can’t do the job. And a recent article that I read in The Wall Street Journal was a lot of concern about employees not having the skill sets that they have, and how to convince students of this, that “why are you here? What’s the whole purpose of doing this? I’m here to guide you based on my life experience on how to be successful in this particular discipline, and you don’t care about that.” That’s a hard problem to fix. So I don’t have a good answer for that. I’m always on the fence on that because it’s hurting the integrity of the institution that students can bypass, but it’s harder. Peer review is another tool, you know, to have them go assess it. They seem to be a lot harder [LAUGHTER] on each other. Yes, this is a tough one. I don’t have a good answer. Sorry.

John: I had to try again, [LAUGHTER] because I still don’t have very good answers either. But certainly, there’s a lot of things you can do. I’m using clickers.I’m having them do some small group work in class and submitting responses. And that’s still a little bit hard to use ChatGPT for just because of the the timing, but it was convenient to be able to let students work on things outside although Chegg and other places had made most of those solutions to those questions visible pretty much within hours after new sets of questions have been released. So, this perhaps just continues that trend of making online assessment tools in large classes more problematic.

Stan: Well, I mean, one of the strategies that I recommend is master quizzing. So master quizzing is building quiz that are 1000s of questions large and randomly drawn from it. And they get credit when they ace it. And then the next week, they have another one, but it’s also cumulative. So they get previous questions too. And you have to ace it to get credit. Sorry, that’s how it is, cheat all you want, but it’ll get old after a while.

John: And that is how my course is set up. And they are allowed multiple attempts at all those quizzes, and they are random drawings. And there’s some spaced practice built in too, so it’s drawing on earlier questions randomly, but, but again, pretty much as soon as you create those problems, they were very quickly showing up in the online tools in Chegg and similar places. Now, they can be answered pretty well, using ChatGPT and other similar tools. It’s an issue that we’ll have to address, and some of it is an ethics issue. And some of it is again, reminding students that they are here to develop skills, and if they don’t develop the skills, their degree is not going to be very valuable. I

Rebecca: Wonder if putting some of those like Honor Code ethics prompts at the beginning or end of blank bigger assessments would [LAUGHTER] prime their pump or just cause more ChatGPT to be used. [LAUGHTER]

John: That’s been a bit of an issue because the authors of those studies have been accused of faking the data. And those studies have not been replicated. In fact, someone was suspended at Harvard, recently, and is now engaged in a lawsuit about that very issue. So the original research that was published about having people put their names on things before beginning a test hasn’t held up very well. And the data seems to have been… at least some of it seems to have been… manipulated or fabricated. [LAUGHTER] So right now, ChatGPT allows you to do a lot of things, but they’ve been adding more and more features all the time. There’s more integrations, it’s now integrated into Bing on any platform that will run Bing. And it’s amazing how well it works, but the improvements are coming along really rapidly. Where do you see this as going?

Stan: November 2022, was ChatGPT built on GPT3 , we’re now into four. And this is only half a year later, basically, that we got into four. I mean, it’s everywhere. For example, in selling books, one of the things that you want to do is try to sell more books. So I went back to Amazon, pulled out all the reviews that I had, sent them into ChatGPT and said “Tell me what the top five issues are.” In seconds it told me it just assessed it where this would take large amount of time for me to do this and it just did it nice and neatly. Everything is going to have AI into it. Grammarly AI is being built into it. All the Microsoft products are going to have AI built in. We’re not getting away from it. We have to learn how to use this in our professions, in our disciplines. With ChatGPT4, it was said somebody had drawn a wire diagram of a website buttons and mastered and text and took a picture of it, gave it to ChatGPT4 and it wrote the code for that website. It’s gonna be exciting. Buckle up, and we had consternation about January, we’re gonna have a lot more coming up. It’s just part of what we do. We have to figure out how to stay relevant, because this is so disruptive. In the long line of technologies that has come out, this is really disruptive. We can’t fight against it, we have to figure out how to do it appropriately, how to use this tool.

Rebecca: The idea of really having to learn the tool resonates with me because this is something that we’ve talked about in my discipline for a long time, which is design. But if you don’t really learn how to use the tools well and understand how the tools work, then the tools kind of control what you do versus you controlling what you’re creating and developing. And this is really just another one of those kinds of tools.

Stan: Well, even in the design world, I’ve gone to Shutterstock. And there is something that allows you to create a design with AI. So the benefit for a designer is they have a certain language, tone, and texture. Their language is vast, and for them to craft a prompt would look entirely different from me, a snowman sticks for arms, it’d be entirely different. But getting the aspect ratio of 16 x 9, everything that you craft into this prompt and feed it in, somebody who does design and knows the language would get something then a mere mortal like me putting that information in. So for somebody who’s in economics, you have a whole language about economics. Somebody who is trying to craft a prompt related to that discipline has to know the foundationals, the language of that discipline, to even get close to being correct in what they’re gonna get back. And students have to understand this, they cannot bypass their learning because they will not have the language to use the tool effectively.

John: And emphasizing to students the role that these tools will be playing in their future careers, might remind them of the importance of mastering the craft in a way that allows them to do more than AI tools can. And at some point, though, I do wonder [LAUGHTER], at what point AI tools will be able to replace a non trivial share of our labor force.

Stan: It’ll affect the white collar force a lot quicker. And I look at it… a nice analogy for the AI was in the Marvel, you have Iron Man, Tony Stark. And it is the mashup of the human and the machine. He’s using this to allow himself to get further and faster in his design, and to do things that we hadn’t thought about before. And I see this tool, being able to do this, that we’re bringing so much information and data to this, it’s mind boggling that suddenly you see a spark of inspiration that you couldn’t get there by yourself without a lot of labor, and suddenly it’s there. And you can take that and run with it. For me. It’s tremendously exciting.

Rebecca: So we always wrap up by asking, what’s next?

Stan: Great question. Right now, I’m getting edits back from my editor for my next book, it’s Strategies for Success: Scaling your Impact as Solo Instructional Technologists and Designers. I’ve been doing this for about a quarter century and mostly as someone by myself, helping small colleges on how to do this, how do I keep my head above water and try to provide the best support possible? So sharing what I think I know .

Rebecca: Sounds like another great resource.

John: Well, thank you, Stan. It’s always great talking to you, and it’s good seeing you again.

Stan: Yeah, absolutely. And also, free book… I’mgonna give a 100, first 100 listeners, but I can go more. Yeah, so there’s a link it’s bit.ly/teaforteachinggpt . And so it’s in that set of show notes to share, but the first 100 gets a free copy of the book.

John: Thank you.

Rebecca: Thank you.

John: We’ll stop the recording. And, and we’ll put that in the show notes.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

303. Higher Ed Then and Now

Teaching practices have gradually evolved as we’ve learned more about how humans learn. From one year to the next, these changes may appear small, but the cumulative effect is profound. In this episode, Todd Zakrajsek joins us to reflect back on the changes that have occurred in higher ed during our careers.

Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published four books in the past four years. His most recent book is a fifth edition of Teaching at it’s Best, a book he co-authored with Linda Nilson.

Show Notes

  • Zakrajsek, T. and Nilson, L. B. (2023). Teaching at its best: A research-based resource for college instructors. 5th edition. Jossey-Bass.
  • Zakrajsek, T. D. (2022). The new science of learning: how to learn in harmony with your brain. Routledge.
  • Harrington, C., Bowen, J. A., & Zakrajsek, T. D. (2017). Dynamic lecturing: Research-based strategies to enhance lecture effectiveness. Routledge.
  • EdPuzzle
  • PlayPosit
  • ChatGPT
  • Wayback Machine

Transcript

John: Teaching practices have gradually evolved as we’ve learned more about how humans learn. From one year to the next, these changes may appear small, but the cumulative effect is profound. In this episode, we reflect back on the changes that have occurred in higher ed during our careers.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Todd Zakrajsek, and I am with Todd here in Durham, North Carolina. Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published four books in the past four years. His most recent book is a fifth edition of Teaching at it’s Best, a book he co-authored with Linda Nilson. Welcome back, Todd.

Todd: Well, thank you, John. Well, this is exciting. And Rebecca may be a long ways away, but I have never been arm’s length from a person who interviewed me for a podcast before.

Rebecca: Isn’t that cool?

John: And we’ve really done that before either at a conference or at Oswego,

Todd: I feel very special.

Rebecca: Well, we can celebrate with our teas. So, today’s teas are:… [LAUGHTER]

Todd: I’m drinking a peach mango that I got from some teas that John brought, which are fantastic.

Rebecca: John, how about you?

John: I am drinking a Tea Forte black currant tea, which I brought from Oswego, in a new mug that was given to me by Claire McNally, when she visited this area last week.

Todd: Love Claire, she’s fantastic.

John: And it has kangaroos on it.

Todd: Yeah.

Rebecca: And I can’t see it. Let me see it, John. Oh, that’s a cool mug.

Todd: It’s a good mug. I got a mug from her university. But I didn’t realize I should have brought it. So I feel bad about that. But it is a podcast. So I didn’t think about what it would look like.

John: That’s true, we generally don’t do a lot of visuals on here.

Rebecca: And I have a blue sapphire tea in my Tea Rex mug.

Todd: Well, that’s a nice mug,

John: We’ve invited you back to talk a little bit about how some of the changes you’ve observed in college teaching across your career have impacted how you teach today. When did your work in higher ed begin?

Todd: Actually, it started when I was a graduate student. So back in 1987. So there’s no reason to try to figure out how old I am. Now I’ve basically specifically dated myself here. I started teaching, I got to teach an introduction to statistics course. And I had so much fun that I taught again the following year. And by the time I left my graduate program, I had taught more courses in that program than any other graduate student had ever taught in the psychology department there. I really loved teaching right from the beginning, when from the beginning, very concerned about student learning, and just getting rolling.

Rebecca: What was it about the teaching, Todd, that really got you hooked?

Todd: Just watching the studentsis. it’s the same thing as it is today, when you have an individual who’s struggling with something, and suddenly they get it and you realize that they may eventually get it on their own, but you realize how much you’ve helped them to move that along very quickly. And facilitating the learning process, I just really love that. That doesn’t mean I was fantastic at it. But I really did love it.

Rebecca: Sometimes the things we love the most are things that we’re not great at to start with.

Todd: That’s true.

John: My experience was similar, actually, I started in 1980, with a course where I had a fellowship, so I didn’t have to teach. But there was a sudden shortage in the department. And they asked me to fill in. And I was planning to go on into research. But it was just so much fun teaching that I’ve never stopped.

Rebecca: I taught as a graduate student too, and taught the whole time I was there. But I started a little bit later in 2003.

Todd: Alright, so that was a couple of years later.

Rebecca: Just a couple.

Todd: Yeah, I had kind of a funny start, I will mention that when I first started that after the first semester of teaching, my students got almost all As and Bs. And the department chair called me in and he said, “I’m not going to have you teach any more courses.” And I said, “why not?” And he says, “Well, you give grades away like candy, we have to have better standards than that.” And I said, “Well, how are you basing that?” And he says, “Well, you know, we looked at the grade point averages.” And I said, “Well, how about if I bring in my final exam, and just walk through it, and then you can tell me how it could change to be more rigorous.” And so it was great. I showed it to him at the beginning. And like the bottom of the first page, the students had to calculate a statistical value, then I had them explain how they came about that number. But if they had used a different test how might it been inappropriately found and what the interpretation might have been, based on the fact that they had done it wrong with a different test. I thought it was important for them to understand how these things can change. The Chair said, “I can’t believe you have your students in the first class actually talk about various tests like that.” And I said, “Yeah, I did. Then we turned the page he says “You did nonparametric tests?” I said, “Well, yeah, we did parametric tests, but then I thought they should know the equivalent.” And he said, “We never do that.” And then he turned the last page and he said “You had them do a two-way ANOVA? You’re only supposed to go through one-way ANOVA.” I said, “Yeah, but we’d finished everything and we still had a week left. And I figured I might as well introduce the next concept to them. And so I showed them how to do a two-way ANOVA and they ended up with all As and Bs. So if you could help me in how to push their grades down and give them lower grades, I’m perfectly happy to do that.” And he then set me up with two courses the next semester, but it’s that reliance on the teaching evaluations is always funny.

Rebecca: Todd, it’s just funny, as we’ve gotten to know you through the podcast [LAUGHTER] it sounds so perfect that that was your first experience. [LAUGHTER]

Todd: Yeah, I’ve lived my entire career on the edge. [LAUGHTER]

John: And those sorts of arguments are still occurring in a lot of classes today about rigor and the need to keep grades lower.

Todd: Yeah.

John: They’re less severe than they were a few years ago.

Todd: Yes, but also looking at how well a person’s teaching based on student evaluations. I mean, we should be looking at authentic assessment. Some things have changed through the years, some things have not changed through the years.

Rebecca: Well, technology is one of those things that has changed.

Todd: Woosh, yeah.

Rebecca: Can you talk a little bit about what tech was like in the classroom when you first started and how it’s evolved a bit?

Todd: Yeah, I know you have some listeners who have been teaching for a very long time. So those of you have been teaching for like 30 to 40 years, just stop and think back about what it was like when we first started. For those of you who have been teaching like Rebecca since 2003, let’s just mention that technology back then was mostly pens and chalk and chalkboards. So back then, of course, there’s technology, there’s always technology, but we were using overhead projectors. This was long before the internet came along to really be used in the classes. LCD projectors were not out yet. Canvas, Blackboard, Sakai, all those learning management systems were not around. We didn’t have any of the ways to email individuals, you couldn’t email your students back then. And there was no ChatGPT to write your papers for you.

Rebecca: But there were calculators that could do all the work for you.

Todd: Yes, but this is the cool part. Back when I started teaching statistics, I’m glad you mentioned the calculators, huge debate back then was whether or not the students should calculate the statistical values by hand using the calculator, because computers had just come onto the scene and we could punch the data into a computer and have a computer run an ANOVA for you. Should you calculate it by hand? Should you run it to the computer? And there was a huge camp that said you should do it by hand or you will never understand a statistical value. And I said, “You know, we’ve got the technology there. Why don’t we have the students use the computer to do the mundane stuff, and we’ll have more time to talk about the theoretical and the important implications.” But even back then we were having the discussions about whether to use the technology at hand or not. Oh, and by the way, we are also hanging grades on doors. So we would figure out the grades, we’d tack it to the door, and then the students who want to know what their grades were for the class would swing by and look at the door.

John: And they were sorted alphabetically, to make it easier for people to find where they were in the grade list.

Todd: Yeah, it was great. We listed them according to their social security number, [LAUGHTER] which was a little different back then. And yeah, we actually did that back then. But as John pointed out, they were listed by number so nobody knew whose number went with whom, except, surprisingly, they were alphabetical on the door. So not only could you figure out Armstrong’s exam score, you’d get Armstrong’s social security number as well. Yeah, times have changed.

John: And it was also back in the day of dittos and mineos as well, which was the only way of disseminating information on paper.

Todd: This is so much fun. We’ll get to some real meat of this thing. But that walk down memory lane has some fun stuff too. The dittos…

Rebecca: I remember dittos, just for the record, okay.

Todd: Yes. So you probably remember, if you dittoed just before class, and you handed it out in class, the students would all pull the ditto up to their face, so they could smell the ditto fluid. And they got that smell. I was running dittos one time in the graduate student office, and I noticed when I looked down because it ran out of fluid, and I had to put some more fluid in, and I looked down and I noticed that the floor was kind of eaten away by this ditto fluid. And then… this is the best part… About a month later, I was digging for something in the closet and I found extra tiles and I thought they should put these tiles down to replace the ones that are all eaten and on the side of the box it said these tiles were long lasting and durable, reinforced with asbestos. So that ditto fluid was eating through asbestos tiles. That’s some strong stuff.

John: …to make it a little bit more friable so that it would disseminate in the air nicely.

Todd: Well, there had to be something to help the faculty members who were running all their own dittos to not mind doing it, and one way of doing this is to have them use ditto fluid, because I’ll tell you, you may not have liked it when you started, but by the end, it was all right. [LAUGHTER]

Rebecca: It’s funny that we’re taking this walk down memory lane, because on our campus, I was in our historic lecture classroom today in Sheldon Hall.

John: What are some of the other changes that have occurred and how have they influenced how we teach?

Todd: Yeah, so it’s interesting, I did the walk down memory lane and we were chatting about this stuff. It’s all fun, but thinking about how the changes have taken place. I think that’s really important. So there have been massive changes. I think that we tend to forget, it’s so easy to communicate with students now. Heck, people are texting now so that you can text back and forth with students. But think about how that has transcended or gone through time. There was a time when I would have to call and leave a message for a student on an answering machine, and then they would call back and we would try to find a time that we could talk on the phone. If we wanted to have a conversation. I could either leave a note for the student or I could call and leave a message that says least come see me after class. So even having a conversation with a student was difficult, then it became easier with email because you could start emailing back and forth. And now we have Zoom. And the equity in the way that this has changed, just think about the difference of this, if I’m leaving a message for a student, they may not even have an answering machine, if they’re living off campus with limited means back then. So even getting in touch with a student would be challenging. Now I can have a Zoom conversation with a student who doesn’t have to hire a babysitter, who doesn’t have to find reliable transportation, who doesn’t have to drive across town and burn gas, and to do all of those things that it would take to have a 15-minute conversation that in the past would have been really hard, and even four or five years ago would have been challenging. The grades, why in the world would a person have to leave… and I was teaching in very northern Michigan, there were days that the wind chill was 75 degrees below zero… and students would leave their dorm rooms and walk across campus to see a grade on the door. It’s actually physically dangerous. And now we have learning management systems, we could post things for students. Interlibrary loan used to take weeks to get a document that you can now go on and get. People can lament all of these technological changes at times, but we’re actually creating more and more equity within the higher education system as we make certain things easier. Not saying that we’re anywhere near an equitable system yet, but we’re moving in a really good direction. And a lot of those changes are helping us to get there.

Rebecca: I’m thinking about all the times when I get to go to the door or meet after class, it really assumes that students are a certain kind of student, they’re full time, they have time. And our students now are working [LAUGHTER], and where they’re juggling a lot of different schedules and things.

Todd: Yeah, and I mean, we want to be careful too. And I agree with you 100%. But they were juggling back then too. But some of the things we were doing, for instance, I taught a night class. Now I would probably suggest if I was going to teach a class from 7 to 10pm that I would teach it through zoom, because there’s a lot of reasons that it’s good to do. But I had students that I noticed in class, would very quickly at the end of class would start talking to other students and I couldn’t figure out what it was doing because a lot of buzzing and stuff. And what I found was that there were certain students who were uncomfortable, and we were in a very safe campus, but they were uncomfortable walking to their car at 10 o’clock at night. So I started saying to the students, “Hey, I’m gonna park a car… and when we showed up, there were quite a few cars there… but I’ll be under the second light, I drive a little red Chevette, not a Corvette, a Chevette, but I’ll have my car there. If you want to park near me, we can walk out together.” And there were students that were not paying attention to almost any of the class because they were fearful of how they were going to get to their car safely. When you think about Zoom and stuff, it’s even safety factors, I would never have a review session now like I used to at 8 to 9 pm the night before the exam because I’m exposing people to potentially dangerous situations. Now we’d have zoom sessions. But I could tell you 40 years ago, there was no even concept of what zoom would be and how it would work. Even Star Trek didn’t have stuff like that.

John: And there was also, besides the inequity associated with people who were working, many campuses had a lot of commuting students who could not easily get back to campus for office hours. Or if they were just taking classes on Monday, Wednesday, Friday and your office hours was on a Tuesday, they’d have to come in that extra day, arranging childcare, or their work to be able to fit that into the schedule.

Todd: Yeah, it really did start to change that system. So we got a little bit more equity, and like you were saying too, the commuting students, the part-time students, the students taking distance courses. When I first started teaching, I was writing… oh my word, remember the correspondence courses? …and you mail away and get a packet of material, you take a test at a local library and, and they talk about distance education being not as good as on campus, but at least better than nothing. And now we’re finally getting to a system where we can stop assuming that those folks who are coming in for part-time courses and stuff are just getting something better than nothing. They’re actually getting something similar to full college courses, which some of those online courses are actually as good or better than college courses that are on campus. But all that’s changing with the technology. It’s crazy.

John: And there’s a lot of research that supports that in terms of the relative learning gains with online and face-to-face, as well as hybrid courses, which seemed to outperform others in a few meta studies that have been done. But those were options that just weren’t available back then. And the early online courses were often designed to be replicas of face-to-face classes, and they probably didn’t work quite as well. But we’ve learned since that, which brings us to the issue of research. During the time that you’ve been teaching, there’s been a lot of research on teaching and learning. While some of it was taking place, it wasn’t very widely disseminated to faculty.

Todd: Yeah, that is true, too. It’s so much easier to get technology out. It’s easier to gather data, it’s easier to write it up. It’s easier to edit it so all of those types of things that are happening now that couldn’t happen before. And as a result, we’re learning a lot more about how people learn, you know, the book I did on the New Science of Learning, looking at a lot of the ways that students learn. And part of it’s just the ease of getting to information. But also part of it’s just being able to investigate how people process information. I used to teach Introductory Psychology back then, we would talk about the stages of sleep. And nobody really knew, for instance, what REM sleep was about, we knew that you had to have it or else it caused some problems. Deep sleep we knew was important, we now have indications that deep sleep for consolidation is necessary for semantic memory. If your sleep is interrupted, you can get eight hours of sleep. But if you don’t get deep sleep, the information doesn’t get consolidated. Procedural memory, how to give shots and kick balls and do anything procedurally looks like it’s more solidified during REM sleep. So again, the different types of sleep are associated with us learning long term, different types of information. We never knew that before all this technology was running around. In fact, back then I gotta say, I remember from my intro psych class being told that you were born with a certain number of neurons, and as you live through life, neurons would die. And if you killed them by drinking or doing something like drugs or something, they were gone forever, and you would never get more. And if you broke a connection, it was broken forever. That’s just simply not true. But it’s what we thought back then. So technology has really allowed us to look better at how people learn, different ways of helping them to learn and different ways they can even study. By the way, before we move on, we now have this physiological demonstration that staying up all night and cramming the night before the test. Even though it gets you slightly higher grades on the test, we now know that because the information is not consolidated that it won’t be there a week later or two weeks later. So we’ve always told students, you shouldn’t do it, but now we can actually show them why it doesn’t work.

John: And the LMS itself has offered a lot of ways of giving more rapid feedback to students with some automated grading with some things to give them more low-stakes testing opportunities. And those were things that we just couldn’t easily do back when you started teaching.

Todd: No, John, that’s a really good one. And we know that one of the most consistent findings right now in all of learning and memory stuff is that the more often you do something, the easier it becomes, long-term potentiation. Which means the more frequently you retrieve information from your long-term memory, the easier it is to retrieve. And just like you’d mentioned, we can now do LMS systems that are set up so that you could do practice quizzes, you could do dozens or hundreds of practice quizzes and keep pulling that information out over and over and over again. That was just not possible before this. And so the LMS helps with that, it helps by giving feedback, really good feedback so that students know what they’re doing well, and what they’re not doing well. And it helps faculty members to design feedback specifically for certain types of projects, and so that I can more easily give more feedback without spending a lot more time on it. So LMSs have done a tremendous amount of work. And that’s not even mentioning the fact that you can have all of the LMS systems loaded with the content. So students can log in and get their information without leaving their house. If there’s fiscal challenges with your class, you can put in articles, the students may not have to buy a book, they could read the articles. And so we’ve got students who were able to come to classes because they can afford to be there. By the way, I remember being on a committee when I was a graduate student, and we were looking at financial aid and different financial systems. And I remembered asking the Chief Financial Officer, I said, “What increase in tuition does there need to be before you start to see students drop off because they can’t afford to be here?” And this was about 40 years ago, but he said $100 for a year, if they have to pay $100 this year more than last year, some students won’t come back. If we look at the price of textbooks now, textbooks can cost $400. So, a book like that is definitely going to make a difference between some students being able to take the class or not. So LMS systems make this possible.

John: And they also make it easier to share OER resources that don’t have any cost for students, or some less expensive adaptive learning platforms, giving all students that first-day access. I remember, not so long ago, when I was still using textbooks in some classes, students would wait several weeks before they got that book. And that put them at a severe disadvantage. And the people who were being put at a disadvantage. were generally the students who came in with the weakest backgrounds because they came from lower resourced school districts.

Todd: Yeah, if they had the resources, they would have the better background foundational material, but they’d be able to buy the books. And you mentioned OERs. So open educational resources are really another thing that are really valuable because back then, before the technology, you couldn’t produce something that would be readily available like throughout the world. And so this project that’s going on now where they’re doing introductory level books in all the different disciplines, you can get an OER introductory psychology textbook that students can log in and read. None of that was possible before the technology. So even the creation of OERs has changed so much.

Rebecca: Well, speaking of digital materials, libraries have changed significantly too over time from having completely physical collections and interlibrary loans and things that take a lot of time to having a lot of digital resources, which changes access to research and materials that you can populate into your classes, but also can aid students in the work that they’re doing. Can you talk a little bit about the change in libraries and how that’s impacted how you’ve taught?

Todd: Yeah, you know, libraries have been fascinating to watch over the last 40 years, because it used to be the biggest challenge librarians had before them was which books to put on the shelves because there was a finite amount of shelf space. And there were lots and lots of books. And so that was the big thing. We used to take out journals that weren’t used very much to make room for other journals. Through time, little by little, they started digitizing all that stuff. And I can remember chatting with librarians, one conversation I had was back around 2001. I said, gonna be interesting, because there’s gonna come a day where there’ll be no books in the library, and the Dean of Libraries said “Well, there’s always going to be books.” I said, “Not always, potentially.” But even if we reduce them, I said, “What is your foresight? How is the library going to change?” And so he had a couple of ideas. But what it basically boiled down to our conversation is, I always felt like a library was like the brain of the campus, it had the books, and it had all of the information that you could go and get. As the books left, and things were diversified in a way that you could find this stuff, you could get all the information right from your dorm room, or from your apartment, when the internet came along, you could get anything you needed, then the library was still a physical space that was in the middle of campus. And what it should become is a learning commons, a place where people go to share and to learn from one another. And I think that’s what’s really changed is individuals still just pile into libraries and use the space, but they use it in different ways. They go there to meet other individuals to work, which they did before. But they took away that aspect of going there for the book part. And it meant all of those shelves got emptied, and they started pushing them out. And you can go into libraries right now that have very few shelves. But they have webcams, they have smartboards, they have spaces where folks can plug in their computers and share with one another. They’ve got screens set up so that you can project and have students sitting around a table, they’ve got Google Glass set up, all of these types of things that bring students together to use technology to learn from one another.

John: And they have cafes to help support that to make it easier for people to gather.

Todd: Yeah, you could swing by and get a cup of tea.

Rebecca: It’s funny, even when I was in high school, my sister and I would rely on going to the library to have access to a computer so that we could even type of paper, because we didn’t have one at home. And that kind of place of having the technology started a long time ago, but it’s amped up quite a bit over the last 20 years.

Todd: Yeah, and I agree completely. And the computers that are there. I mean, even right now, with the books dissipating, there a’re still large numbers of computers. And oftentimes, they’ll even be an area in a library that’s carved out with really high-end computers. But it gives students an opportunity to go. We make this assumption that everybody has a computer and they don’t. But libraries give them that opportunity.

John: Yeah, for those students working on smartphones or Chromebooks, that gives them access to all the tools that students with $2000 or $3000 or $4,000 computers.

Todd: Yes, because smartphones can work for lots of things. But they’re a little tough to write a paper on

John: When I started teaching, and probably when you did too, the predominant mode of instruction, which actually still is often the predominant mode of instruction in many departments, was lecture. That’s changed quite a bit since then. Could you talk a little bit about the shift from lecture-based courses to courses that involve much more active learning activity?

Todd: Yeah, or they just involve a lot more of everything. The concept of flipped classrooms, which was almost impossible 30, 40 years ago, because you really couldn’t get the information to the students. Yes, it was kind of possible, but whoo, if it was hard now, it was really hard back then. But the ability to get information out to students that they can read it before they come to class. But coming back to the lectures… So I’m going to take this moment and those of you who know me know that I’m going to do this, is that we still have no evidence that lectures are bad, but there’s something that we need to really keep in mind. I think this is vital. I do think it’s important for us to be able to talk about buzz groups and jigsaws and fish bowls and lectures and Socratic lectures, discussion lectures, all those different methodologies out there so that we know what we’re talking about when we chat with one another. But I do think it’s time that we stop talking about lectures being more effective than one thing or fishbowls being more effective than something else and look at the components of what is valuable in a learning experience.

John: And a good reference for that is a book on Dynamic Lecturing, which you happen to be a co-author of.

Todd: That is true and in fact that there’s the Dynamic Lecturing. And then there’s a chunk in that about The New Science of Learning. And then there’s a whole chapter in that about Teaching at its Best, because that’s a good point, John, thank you.

Rebecca: It’s almost like you’re trying to slip it in everywhere you are.

Todd: Because the research… people keep talking about one methodology being better than another. Here it is, folks, you can be a hideous lecturer, you can be a phenomenal lecturer. And if you’re a hideous lecturer, you’re not going to learn anything. If you’re a phenomenal lecturer, students will learn from you but they won’t learn all the time, it depends on some student factors. I’ve actually been exposed to group work in flipped classrooms that were awful. And so that concept is we start thinking about and this is why it’s going to come back to the technology, we think about the elements that need to be there, that are necessary for learning to take place. I’m just going to do this, because it’s not the topic I’ll make it very brief, is let’s just go with three things. If you don’t have your attention, as a teacher, if my learners aren’t attending to what I’m saying, if they’re on their phone or thinking about bacon, then they can’t process what I’m presenting. And if you’re having a think-pair-share, if they’re not attending to the person they’re sitting next to, you have to have attention. Number two, they have to have some value. If I’m hearing somebody or I’m reading something, and this has no value to me, it’s really hard to get it into your long-term memory and to learn it. And number three, I have to have a clue of what’s happening, I got to understand some aspects. Now if we think about attention, value, and understanding, now we can flip back to the technology. This is why gaming works. Gaming draws the attention, it increases the value, because you want to win the game, and it has understanding. We have all played games. You open up the old board games, and now it’s digital, where you don’t have a clue what the game is. It’s like, if you advance a player four pieces and the opponent advances five pieces, you have to go back three spaces, unless it’s a Tuesday. When those instructions are that complicated, you don’t understand. So we can use technology to help with attention, we can use technology to help with the value of what’s going on. And we can use that technology to help with understanding. Those are things that were very difficult before. And they allow us to do things like a mini lecture and then shift over to an active learning exercise, and then say, take all this information and create a Zoom session tomorrow that will go over it again. So the technology has really helped us to be able to do all of these things to get at the core of learning, a topic I barely care about. [LAUGHTER]

John: That’s an important one, because people often see this as this binary issue where you lecture or you use active learning. And there are some really effective ways to combine them. And in fact, in that book on dynamic lecturing, it was suggested that lecture can be more important in introductory courses, when students don’t have as much of a knowledge base.

Todd: You’re absolutely right. Discovery learning is a really great way to learn if you’ve got a lot of time. I can just put you into a room with some other people and say, “Here’s some data, and here’s some things we need to know. Go.” And if you don’t have any foundational knowledge at all, it takes forever to figure it out, you go online, you know what to look for, I could do a five-minute lecture, and at the end of five minutes, set it up and say, “Now go and work with your neighbors. In fact, here’s what we’re going to do, we’re going to have you each work in small groups in class, I’m going to open up a Padlet. At each table, I want you to go in and add your information or put it into the column that corresponds with your group number.” As an instructor, I can watch everything develop in front of me. While I’m in the room, I can look at my laptop and see it and walk over to a table and say, “looks like you’re struggling a little bit.” I’ve lectured, I put them into small groups, I’ve had them use technology, I’ve created a little bit of competition on who can come up with what and I’ve had a way for me to monitor it and give them feedback. That is so different than what teaching used to look like. So pulling it all together, that’s what we do.

Rebecca: The tools to be able to monitor have been really helpful in my own teaching and being able to get a better pulse on what’s going on and get a nice overview and then be more targeted in how to interact with small groups rather than just kind of wandering around more aimlessly like I think I did initially. [LAUGHTER]

Todd: Yeah, and this is all going to be great until we get our cognitive load headbands that I’m waiting to be developed. So anybody who’s listening, take this idea, run with it, you can make a bazillion dollars and then take me out to dinner or something. I want a headband and the headband has a light and it measures brainwave activity. And then as I’m teaching, if you start to be a little bit like it’s a little bit too much, you’re moving out of that zone of proximal development, the light turns from green to a yellow. And then when it hits red is like when you’re trying to put together an Ikea bookcase and someone comes by and says “What do you think of this?” and you say, “Errr, I’m working on an Ikea bookcase right now.” …that shutting down with that red light. I’m telling ya, that’s going to be the technology we’ll want next.

Rebecca: It would be so helpful. [LAUGHTER]

Todd: You can actually look and see somebody else’s zones of proximal development and their cognitive load. Whoof. Which by the way, there’s a little party game that they’ll do periodically at parties. It’s like if you’re a superhero, what would you want your superpower to be? And I was in a room one time and one person said they wanted to fly and somebody else said that they wanted to be invisible, which real quickly in my head, I’m thinking, what could you possibly gain that wasn’t illegal or creepy if you’re invisible. So aside from that, transporting and everything else, and they got to me, and I said, “I want to be able to see people’s zones of proximal development. If that were my superpower, I’d be the best teacher.”

John: I bet that went over really well at those parties. [LAUGHTER]

Todd: Yeah, my friends all said “You are amazingly smart and quite insightful.” They used different words, but that’s what I heard. [LAUGHTER]

Rebecca: They didn’t start with what is that? [LAUGHTER]

Todd: As soon as I start talking, most of my friends just shake their head and drink whatever beverage they have near them. [LAUGHTER] So yeah, it’s good times, good times. They’re all impressed. They don’t say it all the time, but I know they are.

Rebecca: I think one of the things that often happens with technology is that it allows us to get things quickly and move through things quickly. But sometimes, as you just noted, learning doesn’t happen quickly.

Todd: Yeah.

Rebecca: Can you talk a little bit about speed and the difference between maybe not having all the technology and all the things really quick versus maybe now where we have it at our fingertips, but do we always want it at that speed?

Todd: So there’s another study that I’m waiting to see. This is an easy study, folks, somebody can run this one quickly. We all know that students are listening to any recorded lectures or recorded material that they have to watch, 1.7 is about the best speed that we tend to see people listening. 2.0 is a little bit fast for some folks. 1.0 is like normal speed, that’s no good, too slow. So what I’m curious about is the space between words and between sentences that our brains, because they move so fast, we can listen faster than somebody can talk. And we have all this other stuff going on is I can be thinking and processing while you’re talking to me. But if I bumped that up to one seven, I think we close the gaps. And I hear it a lot faster. But what I don’t think is happening is the cognitive processing while I’m listening. The active listening component to it. So I think technology can create concerns in those directions. And students who do try to process material too fast… we’ll wait and see.

John: And that’s especially important in flipped classrooms where students do watch these videos outside. One of the things I’ve been doing with those, though, is embedding questions in the video so they can watch them as quickly as they want. But then they get these knowledge checks every few minutes. And then if they find they’re not able to answer it, they may go back and get their attention back and watch that portion again.

Todd: Yeah, I think that’s a really good way to go. EdPuzzle’s kind of a fun technology to use. I don’t know if that’s the one you used.

John: I’m using Playposit, which is a bit more expensive, it works beautifully. I love it, they did just double the price this year, though, it was bought by a new company.

Todd: This is the tricky spot now as the prices are going up. You know, inflation is a terrible thing to waste. Anytime somebody can raise prices now it’s like, “ooh, inflation”. So you know, prices double, inflation is 8% with runaway, now it’s back down around three. But when inflation was 8%, they doubled the price and say, “Hey, we’ve got to,” but yes, it’s some of them are expensive. There’s lots of things that are less expensive. Oftentimes we pay for functionality that help us but the freemiums kind of thing. So stuff that’s inexpensive. I just wanna let everybody out there know just about anything you want to do in class or can think about doing it, there’s a way to do it for either free, or probably under $100 a year, which I know $100 can be expensive for some people, it’s about eight bucks a month. And so things like Padlet that I think might be up around 140 now, so maybe $12 a month, can change how much time you spend doing things, and how much time for students. But yeah, I love the embedded questions to help slow things down.

Rebecca: I think that the cognitive load can happen really quickly if we’re piling lots of information in but not always providing the time to process and use that information in some way in the kind of activities that you were talking about. Or knowing when everybody’s red light is going off in the class.

Todd: Or when people try to do multiple things. I mean, now you’ve got the technology around. So if students are trying to listen to an assignment while they’re texting their friend and have a TV on, I mean, we’re living in an age where there is a lot going on, and people believe they can process lots of things. Evolution doesn’t happen quite that fast. And so I think we have to be careful with that one.

John: One other thing that’s happened is back when you and I both started teaching, the only way students generally communicated their learning was either on typed pages or on handwritten notes. Now we have many more types of media that students can use. And also we’ve seen a bit of an expansion of open pedagogy. How does that help students or how does that affect student learning?

Todd: Wow, that’s really changed a lot as well. Blue Books. Remember the blue books? I think they still sell blue books in the library. They may cost more than the I think it was eight cents when I started, but the concept of writing things down, you turn them into the faculty member, the faculty member would grade them and turn them back. One of the big things that I caught years and years ago was so much wasted cognitive energy in terms of what they produced. I’d read a paper from a student and think this is amazing, and no one will ever see it. It was written for me, I graded it. And now it’s done. I think the technology has changed so many things. One of the biggest things, I would encourage all the listeners, any faculty member out there is, whenever possible, create something that will take the students’ work, the things that they’re doing, and use it to make society better. It’s not that hard. There’s assignments that you can do on Wikipedia. Anybody who wants to complain about Wikipedia, if you don’t like it, I’m gonna go back to Tim Sawyer, who is a faculty member of mine, my very first time I ever did TA work. I was complaining about some students. And he said, “You can complain three times. And after you’ve complained three times, either stop talking,” he was a little bit ruder about that, “or do something about it… just shut up or do something.” And so I complained about Wikipedia for a while, that it wasn’t all that effective. And I thought, well, if I don’t like the page on cognitive load on Wikipedia, I could give an assignment of my cognitive Psych class to go on to Wikipedia and fix it. And so you can have Wikipedia assignments, there’s so many things you could do. Here’s one for you. If you’re doing one on communication, you could have your students go and take pictures or short videos somewhere on campus of something that’s meaningful to them, and then jot down why it’s meaningful, take that compilation of stuff and send it over to the office on campus that does publicity. What better way of drawing students to campus than to have all of these students that have said, I love sitting by the pond because… and in the past, we would have had students write a paper about someplace on campus that you think is effective, put it in the blue book, we would grade it, we turn it back to the students. And that is a waste of possibilities. And so I think we do have lots of ways that we can get the students involved in helping through technology,

John: One of our colleagues in SUNY, Kathleen Gradel, had an assignment for a first-year course, where the students went out, took pictures, geocoded it and added it to a map layer that was then shared with other first-year students about useful resources on campus and their favorite spots on campus, which is another great example of that type of authentic learning.

Todd: Yes, for the authentic learning, there are just so many possibilities because of the technology. If anyone doesn’t have ideas, ask deans, ask the provost, ask the president on your campus, like what kind of information would be helpful, either for the next round of accreditation or for just helping the campus and we can design those things. Another one I did was we took students to the museum. We’d go to the museum, almost any class could kind of find some way to tie museums in, and through the museum, not only would they write stuff that the folks at the museum who did curation would help use, but also just helping the students to see how issues from the museum, how artifacts and things can be used in their own life, to better understand.

Rebecca: When I first started teaching, community-based learning was popular, in fad at the time, and I think having the experience of being a student in a class like that, but then also a faculty member teaching classes like that has really informed the kinds of projects that I do. Maybe they’re not always community-based learning, but they’re often community oriented, whether it’s the campus or even the surrounding community that the campus is situated in to help students get connected. There’s so many nonprofits that need partners and love, there’s always a project that can be done. [LAUGHTER]

Todd: There is. And I used to be a director for a service learning component of the campus. And yeah, there’s just so much out there that we can do to help others.

Rebecca: And students always had such a strong connection. And they didn’t want to fail because other people were depending on them. And so there was a real investment in the work that they did on projects like that.

Todd: I will admit that I’ve never experienced it myself. I’ve never even heard of anybody that if the students are doing some kind of authentic learning, that their authentic learning is then used to help somebody else. I have never heard students say “What a waste of time” or “I hate that class,” or “those assignments are just busy work.” They’ve never used those terms.

John: One common sort of project is to create resources that could be shared with elementary or secondary school students in the disciplines. And again, they can see the intrinsic value of that.

Todd: Yeah. Students could write short manuals on how to learn and then pass that on to the first-year students. And so upper-division students could be helping the lower-division students because not everybody can get a copy of The New Science of Learning, third edition.

John: …available from… [LAUGHTER]

Todd: Available at… used to be Stylus. Since Stylus was sold to Routledge, now it’s available at Routledge. [LAUGHTER]

Rebecca: Given the historical background that we’ve walked through today, what if we think about the future? Where do you see technological changes or learning theory changes impacting the future of higher ed?

Todd: Yeah, we’re living at an interesting time. I like to point out to folks that when you go back to Socrates, Plato kind of time there was a thought that if you wrote something down, it would weaken the mind so we shouldn’t write things down. Luckily, some individuals wrote things down or we never would have known. We’ve gone through several iterations of those kinds of things. Samuel Johnson, I believe it was, who said “With the ready availability of books, teachers are no longer needed. If you want to learn something, you could go get a book on it.” Well, that was a couple of 100 years ago. And we still have faculty members, we have students writing things down, we’re reading, I don’t imagine how you could teach without writing things down and having books. The internet came along, as we were discussing earlier, while we were teaching, we watched the internet show up. And there were people who said, “Well, with the internet, there’s going to be no need for teachers anymore, because students can get whatever they want.” I can’t imagine teaching without the internet right now. So as we’ve gone through each of these iterations, there’s been this fear that maybe we’d be supplanted by some technology followed by “I don’t know how I’d work without that,” it’s a little trickier now, because with generative AI, we’re talking about not just something being available, but actually creating something. I don’t know what that’s going to look like. But there’s some real possibilities that the generative AI ChatGPT, could do things like help students who have writer’s block, get started. And that’s an individual that maybe could produce something really cool, but just can’t get started. I didn’t publish my first book until about seven or eight years ago, because I’m one of those individuals who has a terrible time from a blank screen. I just have a terrible time with that. And so now, I don’t use ChatGPT to actually write anything significant. But I will tell you that I will use it for the first paragraph. That’s all, just one paragraph. And then I completely rewrite that. And there’s no actually trace of it. But it’s something that gets me going.

John: So can we count on more than a book a year going forward? [LAUGHTER]

Todd: No, no, no, no, you can’t. So exhausting. But the concepts that will help students that can do that, I think that’s going to be helpful for them. So there’ll be a type of student who couldn’t have produced before, but now can. We are definitely going to run into some challenges, though, with students who are going to just use generative AI and use artificial intelligence to actually create and to hand something in instead of doing the work. So I do think we’re in a challenging time right now. And I wouldn’t make light of that. There’s actually something that I find fascinating from this. Right now, more than ever before, we can actually have artificial intelligence create something for us, especially in higher education, this hasn’t been done before. The tricky thing is that we were the ones to be able to make that possible, because we learned things. If we let a machine do that work for us, we’re not going to be put into the situation or our students coming along, will not be put into a situation where they’re intelligent enough to do the things that need to be done when they need to be done. And so I do think we’re facing a real dilemma right now. If my students, for instance, always do use some artificial intelligence to create a paper and hand it in, if I can’t catch it, they may end up with an A in that portion of the class. But there’s going to come a day when they’re going to have to write something or be able to read something and tell if it’s written well. And so I’m a little bit nervous, we’re entering a phase where by bypassing some cognitive processing that needs to be done, we may be limiting what we’re able to do in the future. Wrapping this up, though, I don’t want to be the person who says if you use a calculator, you’ll never understand this statistical test. So I don’t know where the balance is. But I do think we’re going to have to have decisions coming up that we’ve never had before.

John: Generative AI is drawing on that wealth of knowledge that has been produced. And for that to continue to grow in the future, we do need to have some new materials being created. So that is an interesting challenge, unless it goes beyond unless….

Todd: …unless it creates it. So that was one I thought about, by the way, sometimes you’re sitting around just thinking about stuff and it’s interesting. I was thinking how do I acquire new information. And the way I acquire new information is I go read articles, I read books, I read a ton of stuff. And then I say I think this is valuable, I don’t think that’s valuable. And then I put it together and say here’s what I’m thinking. And now I’m looking at this generative AI who goes out and scans the environment and pulls these things and then creates something new. It doesn’t have the cognitive processing that I have at this point, but…

John: it’s in the early stages.

Todd: We have some folks who are very concerned out there, especially in European countries that are starting to put some guardrails out, because at the point that it keeps grabbing stuff, and then generating and then it grabs the stuff it generated, then it’s going to be interesting. But as of right now, I just read another article, I think it was yesterday, that they’re going out and grabbing the most popular or most frequently written things and then putting it down as if that is right.

Rebecca: The way that you might prioritize as a human with an expertise in something, is going to be really different than a system that’s prioritizing based on popularity, [LAUGHTER] or like how current something is like when it was last published. That’s a really different value system that really changes priorities.

Todd: Yeah, and I think it changes how we teach. I think the way we teach is going to fundamentally shift because we’re going to have to work with students with all these things being available and explain to them and talk to them about the learning process and the value of the learning process. And keep in mind, this isn’t just about ChatGPT writing papers, everybody’s freaked out about that right now. We shouldn’t lose sight of the fact that you could get fresh, cleanly written papers that have not been plagiarized at all, we’d be able to do that for 20 years. There are paper mills, I can either write away or contact somebody and say, “Please write me a 10 page paper on Descarte, and they would write it, I could turn that in. What actually has happened recently is that everybody can do it, even those who can’t afford to have a paper written at $10 a page or whatever it’s costing. And so equity comes back again. [LAUGHTER] Now we’re an equal opportunity cheater. So we have to be careful with that. But I think the way we teach is going to change because all that information is going to be available, kind of like the internet on hyperspeed. And then what do you do with that? It’s going to be really intriguing. I think it’s an exciting time.

Rebecca: So Todd, this episode’s gonna come out right at the beginning of this semester. So you’re saying we need to be thinking about how to change our teaching. ChatGPT’s here, what are you doing for the Fall differently?

Todd: Well, I think the biggest thing is what we were just talking about, looking more at the learning process, which has been a big thing for me for the longest time, is explaining and talking through the learning process, I can hand you all this information, but if I hand it to you, you don’t learn. In fact, one of my favorite examples came from a friend of mine, and it was the gym, if you want to get in better shape, I could pay somebody to go do sit ups for me. And then I could somehow log in the book at the gym that 100 Sit ups were done, use the passive voice there, and somebody else did them for me, I’m not going to get in better shape unless I do the situps. So I have to do the work, I have to run, lift weights, do the situps in order for me to be able to gain. We need to just turn that into a cognitive process for our students to really gain cognitively they have to do the work. And so I think more than ever, it’s how do we convince students of that? And for the faculty members who say, “Well, that’d be great, but my students just want the grade.” If that’s the case, we have a bigger problem than whether or not some technology can write a paper for them.

John: So how do we convince students that it is important for them to acquire the skills that we hope they get out of college?

Todd: I think this is probably going to come down to the community building, it’s been there forever. If you really want your students to do the work, the best thing you can do in my view, and that’s why I’m gonna say, Rebecca, I don’t think a lot for the way I teach, has changed. You build a community, you build relationships, you talk to the students about importance of things, if you’re sincere about that, and they get that then yes, there’s going to be some students that are going to mess with the system, they have always been there. But you’re also going to get a lot of students who will say, “Yeah, that’s a good point.” And then they’ll do the work. I don’t teach as many undergraduates as I used to, I’m teaching more faculty than ever because of being the faculty developer. But there were years that I would have to tell my students don’t put more time than this in on your paper, you have other classes, you need to do the work in the other classes. Because, and I’m telling you, I am very proud of this, my students would spend a ton of time on this stuff for my class, because they didn’t want to let me down. And I would say you’ve already got an A, I’m proud of what you’re doing, please go work on your other classes. That kind of scenario happens when you build community. And I’m not saying it’s easy, I would never say it’s easy. And it’s not going to happen for everybody. But it is the foundation of good teaching.

Rebecca: So we always wrap up by asking, what’s next?

Todd: There’s just so much going on right now. I think that what’s next for me is I am still in that headspace of coming kind of back from the pandemic, anybody who says, “Yeah, but the pandemic’s all over,” wait for November, we won’t know, we’re going to see. But I still think that’s next is kind of thinking about how we teach and learn in this environment. So moving in that space, it’s probably not surprising. I’m working on the next book here. One of the things I want to do now is the last couple of books that I’ve done had been pretty heavy books. And now I want to write something that’s a little bit lighter. So it’s going to be more of a quick guide with more narrative and having some fun, I love telling stories. I love having fun with people. So I’m going to try to create a book that’s kind of like a science of learning and teaching at its best but really accessible and more of a story-based kind of way of looking at things.

Rebecca: Who is your audience for that book?

Todd: Anybody who will read it? [LAUGHTER] Anytime I write anything, I have to have the audience firmly in mind and think about who am I talking to. And I really believe there is a pretty big overlap with students and faculty who don’t know specific things. And I’m not saying this in a mean way toward any of my faculty colleagues at all. But there’s a lot of people who aren’t taught about things like long-term potentiation and deep sleep in terms of semantic memory, and looking at depth of processing and those types of things. So the same type of thing we can say to a student, we know you shouldn’t cram, but here’s why you shouldn’t cram… faculty learn a lot from that as well. And so my audience for this book is going to be faculty and students, students, because I think it’ll be more fun to read about how to learn in a narrative form like that. And faculty because it’s more fun to learn when you read in that kind of a format for some people. we’ll see.

John: And if faculty design their courses to take advantage of what we know about learning, it can facilitate more learning.

Todd: Wouldn’t that be cool? We could just keep rolling, rolling. What a great amount of work. I mean, a huge amount of work that faculty do. They’re hard working folks that are just cranking away all the time. Number one, making their life a little bit easier by helping to understand things would be great. And just having a little bit more fun would be fun, would be nice way to go to0.

Rebecca: Hey, anytime you can save time, so that we can have more play in our lives is better.

Todd: Yeah, just to do whatever you want to do.

John: Yeah, ending on a note of fun is probably a great way to end this.

Rebecca: Well. It’s always great talking to you, Todd. Thanks for chatting with us and going on the Wayback Machine.

Todd: Oh, you know, I love the Wayback Machine.

Rebecca: I love it too.

Todd: For those of you who don’t know about that, you should check out the Wayback Machine

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

302. Flipped Team-Based Learning

Flipped classrooms allow for class time to be used to put content into action. In this episode, Tina Abbate joins us to discuss the team-based approach that she uses in her classes to help develop the real-world skills important in her field.

Tina is a Clinical Assistant Professor at Stony Brook University’s School of Nursing. She holds a collection of credentials including a PhD, MPA, an MS, and is a registered nurse (RN). She teaches in-person and online undergraduate nursing classes at Stony Brook and conducts research on active learning strategies and the retention of information. She works as a nursing supervisor at two local hospitals.  She is the recipient of the 2023 SUNY FACT2 Award for Excellence in Instruction and was a recipient of the Stony Brook University Award for Excellence in Teaching an In-Person Course.

Show Notes

Transcript

John: Flipped classrooms allow for class time to be used to put content into action. In this episode, we look at one instructor’s team-based approach that emphasizes real-world skills important to the field.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist….

John: ….and Rebecca Mushtare, a graphic designer….

Rebecca: ….and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Tina Abbate. Tina is a Clinical Assistant Professor at Stony Brook University’s School of Nursing. She holds a collection of credentials including a PhD, MPA, an MS, and is a registered nurse (RN). She teaches in-person and online undergraduate nursing classes at Stony Brook and conducts research on active learning strategies and the retention of information. She works as a nursing supervisor at two local hospitals. She is the recipient of the 2023 SUNY FACT2 Award for Excellence in Instruction and was a recipient of the Stony Brook University Award for Excellence in Teaching an In-Person Course. Welcome, Tina.

Tina: Thank you. Thank you so much for having me here today.

John: We’re very happy to see you again. We saw you at the SUNY Conference on Instruction and Technology (or CIT) about a month or so ago. And our teas today are:…. Tina, are you drinking tea?

Tina: I am. I am drinking a chai tea. Very good.

Rebecca: That sounds nice and warming.

Tina: Yup.

Rebecca: It’s a little chilly here, although it’s summer and it was hot yesterday. It is not hot today. [LAUGHTER]

Tina: Yes, for sure the weather has been very odd.

Rebecca: So I have my tea for teaching mug today. And in it, I think actually a mix of a couple of different black teas because I switched when I had a half a cup left. [LAUGHTER] I’m not sure what we call this today, but it’s a mix of black teas.

Tina: That sounds delicious.

John: Well, it sounds like a great tea to have while discussing blended learning.

Rebecca: A high quality blend. [LAUGHTER]

John: And we’re having a real cold spell here in Durham, North Carolina. The temperature has dropped down to 87 today, and I am drinking a tea forte black currant tea.

Rebecca: That’s a nice summer tea.

John: It is.

Rebecca: So we’ve invited you here today to discuss your use of active learning tools. But before we jump into that, we were curious about your wide range of degrees, credentials, and certifications. We didn’t even list them all. Can you share a little bit about your pathway into your current position at Stony Brook?

Tina: Sure. Well, when I went back to grad school, I certainly didn’t intend to get three graduate degrees. I had gotten into Binghamton and gotten into their BS to PhD program because I wanted to do research and my ultimate goal was to do executive leadership position at a hospital because I really enjoyed the leadership role of nursing. So just to backtrack, I graduated Binghamton University in the year 2000 and started right in the NICU (neonatal ICU) at Stony Brook and I worked as a NICU nurse for six years. And in that time, I knew that I wanted to go back to school. And like I said, I got into the BS to Ph. D. program at Binghamton. They awarded me a fellowship program. So I moved from Long Island. My daughter was one at the time. And I started my education there at Binghamton, continued it for the graduate program. And about a year into my doctoral studies they had asked if I wanted to teach clinical and I’ve taught in other capacities. I used to teach violin and piano when I was younger and I never really thought of teaching as a career goal for me. However, I was a poor graduate student, and I said, “Sure, I’ll do it.” And I had about six students in the NICU. I was teaching clinical, and, I don’t know, something came over me. I found my professional soulmate, something clicked so hard for me in that clinical that I wanted more. So I continued asking for teaching assignments. And it’s hard to articulate the feeling that you have, but I felt like I found my niche. And so I did clinical instructoring for about six years and then I moved into the classroom setting. So at that time, I still worked as a nursing supervisor, so I enjoyed the leadership role. And Binghamton started a dual master’s degree program, where you get your master’s in nursing with a concentration in whatever you wanted, I chose education. And the other part of the dual degree was a Master of Public Administration. So I was the first cohort to move through that program. So I graduated first with my Masters of Science in Nursing and my functional role was educator. Then two years later, I completed the Master of Public Administration, and then eventually the PhD. And it all just aligned so perfectly in my current career, because obviously I’m an academic at heart through and through. So those degrees have assisted me in that role. I still work in administration. I teach research, I teach leadership and management. So all of the degrees I’ve utilized and I still utilize actively every day. So this pathway was kind of carved out for me, I think, and I just feel very fortunate that I’m able to apply all of the degrees that I’ve gone for.

John: At the SUNY Conference on Instruction and Technology, you gave a presentation on how you structure your courses. And you mentioned that you were using a flipped team-based learning class structure. Can you tell us a little bit about how your classes are structured, and what a typical class day would look like in one of your classes?

Tina: Sure. So any class that I’m involved in or coordinate, the structure that I utilize is a flipped team-based learning approach. And this essentially requires students to prepare prior to coming to class. It has some benefits there, there’s flexibility, students can learn at their own pace, it really amps up the student responsibility for learning, as we know, and then it also gives us the opportunity for higher level learning because they’re interacting with the concepts outside prior to class. And the team-based part of it I like is because that increases that collaboration amongst students. We know that nursing healthcare is a team sport, so I like to engage the students in teamwork so that they can collaborate and work on their team dynamics, and their own personal team skills. So how my classes operate is, prior to each class, students complete a set of videos, and they’re interactive videos, they’re accessible videos for all types of learners, and it carries weight in their grade. So basically, in these pre-class videos, students get a little voiceover content from me about a concept, and then they get tested on it using a variety of types of questions: matching, true-false, multiple choice, hotspots, you name it. As they move through the videos, they are taking notes on a note-taking guide. So all the concepts are there for them to just follow along, take notes. So they’re seeing, hearing, they’re doing something as they move through the videos. And that note-taking guide eventually acts as a study guide for them, because they have to take a quiz every single class. So they complete these videos before class. And then I start each class with a micro-lecture review using Kahoot!, which is just a game-based learning platform. And in this micro-lecture review, I’m really drilling down to the concepts and helping these students reconcile any last residual confusion that they may have about these concepts. And then after the Kahoot!, they take a quiz. Now, since they’ve interacted with the concepts so many times prior to taking this quiz, I push the level of the questions in these quizzes. There are 15 questions and I try to push the level as high as I can. And the students are able to rise to the occasion because they are not hearing the information for the first time when they walk into class. They have a vague sense of the concepts, we nail it down, and then they take the quiz. After the quiz, the rest of class is comprised of team-based activities. And that’s how every class looks like for me.

Rebecca: Can you talk a little bit more about the embedded questions that you have in the videos and how students have responded to that aspect of a flipped classroom?

Tina: Absolutely, I use a program called Articulate 360. Articulate 360 has many different types of functions in it. But I focus more on the storyline aspect of this product, where I’m able to set up these video clips. So if you already have voice overs, you can basically chop up that voice over into different bits, put it into a story, the type of file that they reference there. And then in between each clip, you can embed any type of quiz question that you could possibly imagine. And you can set up different parameters. So for example, I like to elevate the stakes a little bit, so the students, for these pre-class videos, the grade that counts is their first pass. So it’s not like they can retake the video for a higher grade. It’s whatever they get at the end of that first pass of the video is the grade that counts. And they have two opportunities to answer each quiz question correctly. And I also embed a lot of feedback, so if they get the answer wrong, they’ll see a pop up with some review, and then if they still got it wrong, or they got it right, then there’s an explanation that pops up for the right answer. So I do survey my students in the middle of the semester using a Google form. And then at the end using the university platform, and the feedback about the videos has been very positive, they really do appreciate even though it means extra work, I’m still not giving them 20 chapters to read. I’m giving them something that passes along a bit more quickly and has a better chance of sticking in their memories. And they also appreciate the note-taking guide because it also becomes a study guide, not just for the quiz, but for the final exam at the end.

Rebecca: Like I’ve counted four or five layers of countability on that same content. [LAUGHTER]

Tina: Exactly.

Rebecca: We’ve got the note taking guide. We’ve got the embedded questions, and we’ve got the Kahoot!, and then we’ve got the quiz, and then the exam at the end.

Tina: Yeah, so it’s all about building on these concepts, having the knowledge and then being able to apply it in the classroom

John: In your presentation, you mentioned that you were de-identifying the names of students taking the Kahoot!, but maintaining a leaderboard in the classroom. Could you tell us a little bit about how that works.

Tina: So Kahoot is based on answering the questions correctly or incorrectly. And part of the score is how quickly you answer the question. So ideally, you want to answer quickly and answer the questions correctly. So at the end of the Kahoot!, they get a score. And just again, to raise the stakes, students have to hit a certain benchmark of points to receive full credit. And I try to push that benchmark a little bit, not to make it impossible, but just to make it a little bit challenging for them to give them something to work towards. So for example, in one semester, they have to reach 70,000 points to get the full credit, and then it’s prorated from there. So every time I have a class, I load the data into this program that was built by our Center for Excellence in Learning and Teaching on campus. One of the computer scientist was able to put this leaderboard, showed me how to upload the files, which are basically just CSV files. And what it does is this leaderboard shows their rank in the class, their total score, and the score for that week, so that they can monitor their progress. And everybody else is de-identified and random words, but they can see their name, and they could see their rank in the class.

John: And one of the advantages, I think, of using Kahoot! is it does provide some practice in developing automaticity. So that students can practice retrieving information quickly, which I would think would be especially important in health-care situations.

Tina: Absolutely. And I’ll have some students that come to me and they just absolutely despise Kahoot! because of the stress. And if you’ve ever taken a Kahoot!, and I have, it is stressful, you have to really think on your feet very quickly, especially since your score is based on how fast you answer the question. So what I tell them from the beginning is if you really are struggling with Kahoot!, and you don’t like Kahoot!, Kahoot is really for you, it’s meant for you, because I want you to think of a situation in a hospital setting. If a patient is deteriorating, we call something called a rapid response. And a team of people flow to the room to address whatever issue it is, perhaps the patient’s having difficulty breathing, whatever, chest pain, this now has become a very emergent situation. And in that situation, you have to be, as the primary nurse or a nurse assisting someone else, you have to have laser focus, and someone may ask you to just go get a piece of gauze. And if you’re new in the role you may be so flustered, just by getting that piece of gauze. So, this is really like a precursor to that. So I tell the students to use Kahoot! as a mechanism to help with your laser focus in situations where the outcome is dependent on what you’re doing.

Rebecca: Another thing that seems really relevant to a healthcare setting is the team-based learning aspect of your course. Can you talk a little bit about how you arrange the team-based activities and also how you set your students up for success on teams.

Tina: So with team-based learning, as we know, it’s simply a collaborative learning strategy and how the team activities look depends on the course. So I can talk to you about my research course. That happens every fall semester, and I have 160 students, this is the graduating class. These are the seniors, they’re in the last two semesters of the program. And what we do in that course is the team-based activity portion of class is working on a project. So I’ll tell you a little bit about the project which is experiential in nature. Stony Brook University is attached to Stony Brook University Hospital. So every year, I pick a unit, I meet with the manager, and they give us a clinical problem to solve. So for example, this fall the students and I will be working with the surgical ICU and the clinical topic is nurse wellbeing. So, as we know, we’re in this post-pandemic world and wellbeing has really moved to the forefront. Things like burnout, compassion fatigue is very prevalent in the healthcare environment and just globally as humans. I think we’re just a little tired of living in this fight or flight for so long. And now we’re trying to come back from this. So this fall semester, the students will be working in teams to find a solution for the surgical ICU for nurse wellbeing. So what we do is we search for articles together, and that’s how they get to their solution. We use a framework we use Melnyk’s seven steps for evidence-based practice. So in undergrad nursing, even though it’s called the nursing research course, the students are expected to utilize the research that has been done on a topic to make changes to their practice. Our expectation is not for them to actually conduct research. That’s a PhD level thing, but according to our essentials in baccalaureate nursing, that our accrediting body tells us what curriculum to teach to the students, the expectation is that they know how to read the research, how to critique it, how to appraise it, how to synthesize it, and how to use the research to develop solutions. So from there, they work in teams of eight throughout the semester, they develop their solution, they put it into a video project, the six-minute video project, and I choose the top two projects. Those top two projects then move on to the implementation phase. So then the unit will implement and evaluate the solution. And in addition to that, we put in for posters at conferences. For example, last year, we had two posters at ENRS. I was assigned the course of research, I was like, “Oh, boy, how am I going to make this interesting?” …because we know that research content can be a bit dry. So I ran the course for a couple years, and I knew that I had to do something with it. And that’s where I started moving towards this more experiential learning opportunity for the students. And so far, it’s been going really well.

Rebecca: So I heard you say something about teams of eight, and I almost maybe had a heart attack, [LAUGHTER] just thinking about how big that team is, and how to manage that. Can you talk a little bit about some of the structures you have in place to help a group that size, which is relatively large, be successful?

Tina: Sure. So teams of eight… that means I have 20 teams in total. And we’re all reviewing the same articles. So then I know the answers to all the questions. And basically, Google Drive is my answer. Every team has their own folder, within that folder are subfolders, I have them buddy up and be assigned to a certain number of articles. As a team, they have like individual and buddy responsibilities, which is clearly articulated in a contract that they review and fill out at the beginning of the semester. So they have individual responsibilities, buddy responsibilities, and they have team responsibilities. And every single class looks the same. So by the second class, they’re already into the mode. I don’t throw them any curveballs, every class structure is exactly the same, so they know what to expect. And they have appraisal forms to fill out. They have tables to fill out as a team to keep all of their literature organized. And the structure that I have in place seems to be working because there’s very little confusion now that I’ve kind of worked out all of the kinks. And I also always keep instructions projected just to make sure that they are apprised of the flow of class.

John: You mentioned Melnyk’s, seven steps of evidence-based practice. Could you give us a brief overview of that framework?

Tina: Absolutely. So there’s many evidence-based practice models out there. Stony Brook goes with Melnyk, and there are seven steps and actually I begin with step zero, step zero is igniting that spirit of inquiry. And that’s one of my main end goals of the course is for them to stay curious about how they can improve practice as a nurse for their patients. So that’s step zero. And then basically, what we do is we take the clinical problem, and then we frame it in the form of a question, a PICO question. And that helps us to find our articles. So once we find our articles, we go through the articles, we decide what we’re going to keep, what we won’t want to keep, then we start to critically appraise these articles, review them, read them, understand them, the students put that information into a literature review table, which is just the main elements of each article. After we’ve appraised all of the articles, the next step is to synthesize all of the articles. So what is the bigger picture? For that synthesis class students do complete synthesis tables. And when they create these synthesis tables, now the beginnings of their proposed solution begin to emerge. So then students put their solution together based on the synthesis table. And then the next step in this process is to implement the solution and then evaluate the solution. And of course, dissemination is always the last step.

John: You also mentioned that you use collaborative testing on exams. I’ve done this with a two-stage exam process where people take the exam individually first and then submit that but then take it again as a group. It also appears to have been and that’s been tremendously successful. It’s also appeared to have been really beneficial in terms of student learning, and it’s just so much more fun to watch the students work in groups on exams, than it is to go over the exam the next day with the whole class. That collaborative exam format has been so much much better than I ever expected it to be. Could you tell us a little bit about how you do collaborative testing on your exams?

Tina: I absolutely adore collaborative testing. If you have to assess students using exams, this is really maximizing the use of exams. So in my courses, students take collaborative exams in teams of three. And as we know, the research says that collaborative testing may decrease test-taking anxiety, the students have to take a large licensing exam at the end of the program. So it may help some of these students with that, like you said, immediate feedback on test performance, it really scales back the number of questions, I don’t even do exam reviews anymore, because the immediate feedback that they get, they’ve reconciled any confusion on the exam, that an exam review is no longer required. It increases student engagement and collaboration. I love how they, like you said, they debate, they discuss, that peer instruction. There are some people out there who can read a book and retain 100%. But generally speaking, you’ll have a better chance of retaining more information if you’re teaching someone else versus reading a book. Of course, that just varies learner to learner. So that could be really something to hone in on when it comes to collaborative testing. So, yes, the traditional way is to take the test individually, and then they take it again, in a team. And in our program, the clinical courses like medical surgical nursing, pediatrics, all of those, I would always recommend to do individual than collaborative because you really want to assess that individual on their performance and understanding of the concepts. And so I teach research, and I teach leadership and management, these are non-clinical courses, I skip the individual part, and take them right to a collaborative exam. So for example, for my research course, the students don’t know who they are paired up with, or in a team with, until about an hour before the exam. They get two articles, a quantitative article and a qualitative one. And then they have a set of questions to answer. Essentially, we’ve been preparing for this type of exam throughout the semester. So they end up doing really well. In my transitions to professional practice, where I’m teaching leadership and management, that is a traditional final exam, multiple choice, select all that apply, type of questions. And again, I actually do it on Zoom, they go into breakout rooms, they share their screen, and they take the exam, there’s a scribe who enters the answers. And also when it comes to accommodations, kind of as a side note, I’ve been able to set up strategies for individuals that do have accommodation so that they can maximize their experience as well.

John: When I first tried this, I was so excited about how the students were reacting with the collaborative exam that I took a short video clip while they were doing it and sent it to Rebecca. She was working with me in the teaching center at the time. It was just a remarkably positive experience.

Tina: Do you notice a difference like I would say an estimate of 10 points between the individual and the collaborative mean.

John: Generally, yeah. And the group one is virtually always higher than each individual score, except in one case in my class, where one student had a higher score than his group, and that’s because during the group discussions the student gave in to peer pressure within the group. I encouraged him to be more assertive when he’s confident about his answer. But that only happened with one student on one exam.

Tina: that’s pretty rare. I just love just watching them engage like that. So I’ll pop into like the breakout sessions, and they’re collaborating and negotiating and it’s just fantastic.

Rebecca: You mentioned earlier about your research class having a project coming up about well being. And I think that’s a topic that we’ve been talking a lot about in higher ed in a lot of situations. Can you talk a little bit more about that project and some of the research that’s going into it and some of the outcomes of it?

Tina: Absolutely. I mean, wellbeing is such a hot topic right now in probably every type of job you could think of. And it’s interesting wellbeing is kind of always been in the background. And I think the pandemic really shoved it into the forefront where it really should have been. That really needs to be, in my opinion, the top priority of any workplace because if your employees are well, it has a positive trickle down effect. So it has gotten to the point now where our accrediting body who tells us the essentials that we need to teach to our students, they have added a wellness component, and we’re adopting these new essentials in the next year. These are new essentials for us to follow. So it made it into the essentials, which is very telling. And now faculty are charged with teaching students, monitoring students, about their wellness and wellbeing. So this was pretty timely, because of the pandemic, the clinical topic that we’ve been doing for this EBP project has been things like compassion fatigue, burnout. And now this year, we’re doing wellbeing. Last fall, we worked with the cardiothoracic ICU. And the EBP project topic was compassion fatigue. And we wove in a lot of wellbeing into the solution, which is actually kicking off on July 1. So this year, instead of doing compassion fatigue, which has a bit of a negative connotation, let’s flip it to the positive. And like I said, we’re working with the surgical ICU, and we want to customize a wellness solution for that unit. So in the meantime, by proxy, I can teach the students about their own wellbeing and their own wellness. So I have a lot of content in there, so that they’re learning about this clinical topic to help develop a solution, but they’re also learning about it for themselves. And I do a few things with them, and definitely evolving this as we move along. And I’m lucky enough that I have the graduating class in the fall and the spring. So I move it through from the fall to the spring semester. So in addition and educating them on the different ways to promote your own wellness, we start each class with a mindfulness activity. I have a sound bowl that actually a student gifted to me, we do meditation, mindful breathing, every class is something different. This year, I’m inviting students to lead some of these sessions. So I want it to grow so that other students can participate and lead us and it’s literally three to five minutes at the beginning of every class, all lights down, devices off, phones flipped down, and we just take the time to be as present as possible. And I also help them keep an eye on their level of burnout. And I give them the professional quality of life survey at the start of every class. And halfway through, I’ll do a comparison of statistics between the different cohorts. Because I have the traditional cohort and I have the accelerated one, we look to see how our scores are doing over time, just to have that educational component to it. And then also the Insight Timer app, that’s an app that you don’t have, I would highly recommend that you download it. It has so many mindfulness type of activities that you can do. There’s a journal, you can track your progress. They have classes, and even the paid version, which is I think, maybe $60 for the year, they offer so many different bells and whistles, it’s really just a phenomenal app to use if you’re looking to promote your own wellness. So the other thing I wanted, I attended that CIT conferences, is I would love to use ChatGPT to develop a wellness assignment. So I’m still thinking about the inner machinations of how that would work. But hey, you know, if AI is here, might as well see if we can use it to promote wellbeing.

John: And it’s nice to have that focus of using ChatGPT positively because this is something that’s going to be part of students’ lives going forward, maybe not this specific tool, but AI tools are not going to disappear and using them for good would be a nice alternative for the concerns that many faculty have about the use of these tools. During your presentation at the CIT conference, you also mentioned using a variety of edtech tools. What are some of the tools that you use in your classes?

Tina: Sure. So I’ve trialed some apps here and there, I’ve used Plotagon. I used Go Animate for Schools, which is now VYOND, just for them to create case scenarios in their leadership and management class. And based on feedback, the one that they really liked is now a bit pricey. So I tried a free version of an app, and it really didn’t go well based on feedback. And that’s how it works in education. You try something out and you survey the students and if the experience over time is really not positive, you need to move on to something else. But things that really have stuck is I told you about Kahoot! and Articulate 360. How I communicate with the students. I use GroupMe. I prefer to communicate with them using that application over Brightspace or traditional email. They join via QR code and I have them all in one group chat and I can post quickly. They could send me direct messages, they could post questions in our group chat. And it just seems to really streamline communication because we’re all competing for their cerebral real estate, they have a lot going on, a lot of deadlines, so I find that this GroupMe app is really helpful. And I also try not to spam them with too many messages, thoug. It really seems to work. And then again, Google Drive, I can’t even begin Google Drive for everything, whether I want to survey them or whatever it is, Google Drive has it for us.

Rebecca: So speaking of Google, [LAUGHTER] you mentioned earlier using a Google form for a mid-semester evaluation. Can you talk a little bit more about that, and how you’ve used that to make adjustments in your class for the latter half of the semester?

Tina: Sure. So a Google Form is a pretty nice way to just give a quick survey to your students, I do that in the middle of the semester. And I have to tell you, that’s where I get my best data, because they are in the throes of it. And my response rate is typically over 90%, as compared to at the end, where they’re kind of just fizzling out, tired, maybe a bit over it, generally speaking. So I don’t get the response rate in the final that I do in the mid semester, when I analyze it, very short, a couple Likert questions: What do you like? What don’t you like? …and if there’s enough of a theme in the qualitative questions, or in the Likert scales, I’m able to make changes prior to them departing from me, instead of waiting for the next cohort to come in. For example, some things that came up was: “It can be a bit loud in the classroom.” So I’ve done something to control the volume in there, because it’s a very active classroom, or we feel like we’re sitting around too long during the TBL activities. So now I have a mechanism for them to let me know when they’re done with their activities, so that they’re not sitting around waiting. So those types of things. If they say, “let’s skip the final exam,” then that’s not anything that I can honor. But I’ve gotten some really good raw feedback that’s helped me evolve my classes. I’m just always so grateful for the student experience, because they inform me where this needs to go. Another way that I use a Google Form is with team-based learning. Michelson says that you should have the team members evaluate each other on their team performance. And typically, this is done at the end. But I like to do it in the middle of the semester, where they’re evaluating each other so that they have an opportunity to remediate, and then by the end, hopefully, their team’s performance scores have gone up. The challenge, though, with a Google Form is it’s very hard for me to share the feedback back to the students, it requires a lot of copying and pasting. And there’s a lot of room there for error, human error. So currently, I do bring in the students that are rated poorly just to give them some one-on-one guidance on how to improve their team performance. But in the meantime, to work around that I did trial a product called Kritik that offers that ability where the students will get their feedback back. But I reached out to our Center for Excellence in Learning and Teaching. And right now what we’re doing, we have a sandbox, and we’re working on trying to do a Kritik-like type of peer evaluation in Brightspace, using PeerMark. And we’re getting very close to ironing out some of the finer details. So I’m going to finally have an evaluation where every student can see their feedback from their team members based on their performance, so they know what they’re doing well and where they need to improve.

John: You teach both face to face and also online. Do you use many of the same techniques in your online classes that you use in your face-to-face classes? How do you modify your class for online delivery?

Tina: So I do everything the same, except that it’s in an asynchronous format. So students really have to be self disciplined in an asynchronous online type of environment. The online classes that I teach are post-licensure undergrads, so they have their two-year Registered Nurse license, and they’re looking to get their four-year degree. So some of the assignments, we tailor a little bit differently just because they have nursing experience, whereas my pre-licensure students do not. So maybe the assignments vary a little bit, but the structure is the same, using Articulate. I don’t use Kahoot! with them, only because I don’t have them in front of me, but they do have the quiz. And they have the TBL activities and things of that nature. So it’s the same, but it’s just in an asynchronous format.

Rebecca: I know that we mentioned in the intro that you do some research on some of your teaching practices. Can you tell us a little bit about some of that work?

Tina: Sure. So a colleague and myself got IRB approval, and we’re just starting to do some research on this evidence-based practice project that the students do in my class. And we’re just starting off with a cross-sectional study. We have a valid tool that’s been out in the literature that measures their perceived knowledge, skills and attitudes regarding evidence-based practice. So, I’m not building logistic regression models or anything yet, but starting off with a cross-sectional study to understand pre and post, the beginning of class and at the end of their research class, if there’s any impact or change in their knowledge, skills, and attitudes regarding evidence-based practice. So that’s where I’m starting. And I’d like to move on from there eventually.

John: And speaking of moving on, our last question is: what’s next?

Tina: So, I just would like to continue publishing and presenting. And continuing my research. Like I mentioned earlier, I’d like to introduce an AI tool for wellbeing, and Stony Brook just purchased several VR headsets. And because my courses include a lot of content about compassion, wellness, well being, I would love to develop a simulation about empathy. I think that would be a fantastic use of VR, apart from like, typical clinical scenarios. And that’s really my plan for now.

John: Well, thank you for joining us. And when you do have some results from your research, we’d love to have you come back and talk about it.

Tina: Thank you. Definitely. I really appreciate you inviting me. This is a wonderful opportunity for me. Thank you.

Rebecca: Yeah, thank you for letting us use your class as a little case study for folks to think about ways that they could change, improve, and reconsider their own classes. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

301. A Return to Rigor?

Some faculty have advocated a return to “rigor” to address perceptions of growing student disengagement in our classes. In this episode, Kevin Gannon joins us to discuss an alternative approach that provides students with cognitive challenges in a supportive environment. Kevin is a history professor and the Director of the Center for the Advancement of Faculty Excellence at Queen’s University of Charlotte. He is also the author of Radical Hope: a Teaching Manifesto, which is available from West Virginia University Press.

Show Notes

  • Gannon, Kevin (2020). Radical Hope: A Teaching Manifesto. Morgantown: West Virginia University Press.
  • Gannon, Kevin (2023). “Why Calls for a ‘Return to Rigor’ Are Wrong.” The Chronicle of Higher Education. May 22.
  • Imad, M. (2022). Trauma‐informed education for wholeness: Strategies for faculty & advisors. New Directions for Student Services, 2022(177), 39-47.
  • Freire, P. (1970). Pedagogy of the oppressed, New York (Herder & Herder) 1970.
  • Boucher, Ellen (2016). “It’s Time to Ditch Our Deadlines.” The Chronicle of Higher Education, August 22.
  • Vygotskii L.S. (1984). “Problemy detskoi (vozrastnoi psikhologii).” In Sobranie sochinenii v 6-ti tomakh, vol. 4, pp. 243–432. Moscow: Pedagogika
  • Cavanagh, S. R. (2023). Mind over Monsters: Supporting Youth Mental Health with Compassionate Challenge. Beacon Press.
  • Jack, Jordynn and Viji Sathy (2021). “It’s Time to Cancel the Word ‘Rigor.’” The Chronicle of Higher Education. September 24.

Show Transcript

John: Some faculty have advocated a return to “rigor” to address perceptions of growing student disengagement in our classes. In this episode, we discuss an alternative approach that provides students with cognitive challenges in a supportive environment.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist….

John: ….and Rebecca Mushtare, a graphic designer….

Rebecca: ….and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Kevin Gannon. Kevin is a history professor and the Director of the Center for the Advancement of Faculty Excellence at Queen’s University of Charlotte. He is also the author of Radical Hope: a Teaching Manifesto, which is available from West Virginia University Press. Welcome back, Kevin.

Kevin: Thanks. Great to be back with you all.

Rebecca: Today’s teas are:…. Kevin, are you drinking tea?

Kevin: I am drinking Cheerwine because I’m in North Carolina now. And this is how we roll in this state. And it’s so damn hot outside, a hot beverage is the most unappetizing suggestion right now.

Rebecca: All right, we’ll let it slide. [LAUGHTER]

John: I would have been tempted to have Cheerwine because it does have that whole North Carolina flavor, which I had never heard of until I came down here, the first time in 1987. But it’s incredibly popular.

Rebecca: What is it?

Kevin: It’s a cherry soda, basically. You got to be ready for sugar. You got to get your pancreas in shape and then prepare to go, but it’s quite tasty.

John: It’s a very inexpensive and popular cherry soda.

Rebecca: Interesting. It sounds like medicine.

John: No, it’s more sugary than medicinal.

Rebecca: Okay.

John: …but Duke, some time before last year, removed all of the soda from the vending machines and every place where they serve beverages on campus. You only have choices of healthy drinks: water, fruit juices, iced tea, they have of course, because it is North Carolina.

Rebecca: Is it sweet tea then?

John: It is not sweet tea, it is unsweetened tea. So I have a Tea Forte black currant tea that came down with me in my new Duke University mug.

Rebecca: And I have an Irish Breakfast tea because it’s 90 degrees outside and I have a hot tea because… I don’t know why. [LAUGHTER]

John: And I would not be drinking tea if I were not sitting in a very nicely air conditioned classroom here at Duke. So we’ve invited you here today to discuss your May 22 Chronicle article where you address the arguments that some people have raised advocating a return to rigor as a solution to what seems to have been a substantial reduction in student engagement Since the start of the pandemic. What do you think is the source of the disengagement that faculty have been perceiving?

Kevin: Well, I think there’s a lot that goes into it. But I will say that I think it’s important for us to remember that there’s: A) no one cause or explanation for it, which leads to B) there’s no one solution that’s going to fix it all. And we know this, I think, but in the day-to-day practice or dealing with this room full of disengaged students, it’s hard to remember that sometimes. And so I think the root cause of this disengagement comes from the fact that we went through and are still dealing with the effects of rolling trauma on a global scale. And we can talk about trauma-informed pedagogy all we want, but it’s not going to erase the fact that trauma happened and for some of our students, and for some of us, continues to happen as well. And of course, what we saw as a result of that was the pandemic laid bare so many of the other things that were already in place that were unsustainable, and didn’t let us hide from those things anymore. So whether we’re talking about the so-called racial reckoning of the summer of 2020, or we’re talking about the ways in which people from different socio-economic groups experienced the pandemic and or healthcare as a result of that. And we’re in an age now of sort of creeping authoritarianism, pseudo-fascism, whatever you want to call it. I don’t think anybody could realistically expect folks to bring all of their cognitive energies to bear in a classroom with all of this going on in the background. It’s like trying to read a book while you’re in the spin cycle of a washing machine. There’s no way. And I think we can talk about creating spaces that are sanctuaries from that, but I don’t think that we alone can solve all of the things that are leading to disengagement. In fact, disengagement, the diversion of cognitive bandwidth, defense mechanisms, these are all things that are actually, I would argue, fairly healthy responses to everything that we’re seeing around us. And we need to have the sort of empathy and understanding of what many of our students and ourselves are going through to allow space for that to happen.

Rebecca: Yeah, sometimes it feels like there’s a request to just snap out of it.

Kevin: Right. One of the refrains I’ve had over and over is we’re not going to pedagogy our way out of like systemic collapse. And so I think Mays Imad actually put it very well, when she talks about learning as a sanctuary. Our classes can be a sanctuary from this, and that’s important. And we should be doing that work and we should be providing those spaces and curating those spaces and nurturing those spaces for ourselves and for our students, but to put all the pressure upon educators to get students “reconnected,” despite everything else that’s happening around us, I just think it’s unrealistic, and it sets us up for failure. And the same is true, I would argue, for K to 12. Teachers, during the pandemic, educators were expected to sort of fill in the gaps of all of the missing social services over and above “just education.” And I think that when we talk about this disconnection, there’s a danger of us moving into that space where we’re being expected to solve systemic problems, when we are not in a position where we’re able to do so. And in fact, we are suffering from the effects of those problems, in many ways the same as our students.

John: And at the same time, I know our campus at least, and I think, throughout the US, we’ve seen an increase in the diversity of our student bodies. There’s many more first-generation students coming into our classes, there’s many students from historically minoritized groups who simply were not generally attending, and those students often come in with less knowledge of, as you note in your article, the hidden curriculum of education. We have to help them learn that curriculum. And that brings us to that whole question of the push to rigor. In the article, you describe two approaches to adding more rigor, one is adding more logistic rigor, and the other is adding more cognitive rigor to classes. Could you talk a little bit about those?

Kevin: Sure. And so when we think about this idea of rigor in the way that we normally talk about as faculty members, I do think that there are these sorts of two camps that rigor falls into, I don’t think that we… and I’m speaking broadly here, and certainly implicating myself, in some of this… I don’t think we always do a great job specifying which one or the other we’re referring to. So it’s very easy for me as a faculty member to say, “I’m making a very rigorous class.” And maybe all of that is one type of this rigor as opposed to a balance. And so I think when we look at rigor, what it basically boils down to is there’s sort of two broad ways in which a course could be challenging, it could be difficult. One of those is what we talk about as faculty as the good stuff and what I call cognitive rigor, complex thinking, higher order thinking, the ability to critically interrogate information, the ability to step outside of one’s own perspective, all of the things that we know higher education should be doing. And then there’s the other kind. There’s, for lack of a better term, I call logistical rigor. And that’s where you see things like inflexible policies, volume of work, not necessarily difficult work, but so much of it that the sheer volume in itself is what makes the difficulty exist for students. The classic story, I keep saying it’s apocryphal, but when I tell this story at various workshops I do at other campuses, people always swear it happened to them. So I think it is real. But the apocryphal story of the big lecture class where the professor strolls out of the first day and says, “Look to your left, look to your right, only two of the three of you will be here by the end of the semester,” like that’s that sort of logistical rigor that I think we see a lot. The problem is, as I note in the article, we often mistake one for the other. We often say that our classes are rigorous. And we think in our faculty braids that they’re cognitively rigorous, but the way our students are experiencing them is actually through logistical rigor. And so if you look at some of the research, and I linked some of it in the article, what really surprised me as I dove into this, were the vastly different perceptions that students and faculty had about a rigorous class. There was an article that listed the top 10 features of a rigorous class according to students. The top five of those were what I would call logistical rigor, the number of pages that were assigned to be written during the semester, the reading load, the pace of the scale of work, none of the good cognitive rigor stuff came in until the bottom five on that student list, and the numbers there were significantly lower. And this is just one study, but there’s a pattern across when we look at student perceptions of rigorous courses, of difficult courses, it’s a pretty clear thread that students are experiencing difficulty as logistical rigor. And so when we as faculty say, “Well, we’re really after these cognitively rigorous courses and that’s what it’s all about.” Well, that’s not what our students are seeing, which leads me to wonder if that’s really what we’re doing. And my suspicion is, is that no, that we’re often creating these logistically rigorous course spaces, and thinking that it’s cognitively rigorous, and of course, those are two very different things.

Rebecca: Well, and I think sometimes that code word of rigor, and I intentionally use the word code here is that it’s often used to weed certain students out and then we wonder why particular disciplines aren’t diverse or don’t have new faces as a part of the fields and disciplines, as if having structure or support or scaffolding is somehow the antithesis of rigor.

Kevin: Right. And this is really the crux of it, and of course, everybody’s brains will first go to STEM courses because STEM fields have really been struggling with this. But as a humanities guy, there are humanities fields, including some sub-fields in my own discipline of history. I would argue philosophy wrestles with this as well, where rigor in this logistical sense is exactly as you put it, the sort of weeding out, its code for “Some of y’all should be here, and some of y’all shouldn’t, and by the end of the semester, we’re going to have that sorted.” And of course, is that what we want to be doing? Is this how we reproduce our disciplines? If your answer to that is “yes,” I would argue that you’re probably in the wrong line of work. So we need to be thinking, what is it that we’re actually doing? There’s a difference between saying “our uses of rigor are counterproductive” and “we should dumb everything down.” Those are two separate things and that’s not what I’m saying. But our uses of rigor are doing the very things that you point out, Rebecca, that we’re putting barriers in front of students, we’re closing off pathways and opportunities for them to engage in our discipline. And given this moment of where we’re at in higher education right now, I think that’s a horrible, horrible strategy. Rigor, it has become such a loaded concept, because it has become this stand in for weeding out or culling or all these other awful metaphors that we use in higher ed to talk about kind of thinning the herd, so to speak, and that language matters.

John: When you were talking about the difference in faculty and student perceptions of making courses more rigorous. It reminds me of the discussion that we often see about active learnin. When faculty are surveyed in terms of the extent to which they use active learning activities in the class and the proportion of time that they lecture. When students are surveyed on the same questions, we get a remarkably different picture, suggesting that faculty are doing a lot more lecturing, and a lot less active learning than they believe that they are. And it might be nice if we could get a little bit more dialogue going back and forth between students and faculty and getting perhaps more student feedback in general. But it does suggest that we’re seeing a disconnect between what students observe and what faculty think they’re doing in their classes.

Kevin: Right. And a lot of times will be talked about, “Well, I do X in my class,” like I know in my own case, there are a lot of occasions where what I say I’m doing is actually more aspirational than actual. I would love to be doing these things. And on a good day, these things are happening. And maybe they’re working. And I get that. There are some days in some classes where the stuff that we know is most effective and most desirable, just doesn’t work the way that we would hope or the way that we would want. But that doesn’t mean that we stop trying. I think there are plenty of opportunities in place for us to have that sort of dialogue you’re talking about with students to see: are the actual experiences of my students aligned with what I think they are? And this is why we do assessment. This is why we do, at least in a perfect world, student ratings of instruction, if this course was designed to get you from point A to point B, and I want to say that you got to point B, I need to be able to prove that. And I need to be able to describe what that experience was like for you as students. And so how do I bring student voice into this. So you could do informal midterm feedback, you cn do weekly reflection papers, you could do check ins with students. The faculty development world has, I’ve seen it referred to as the small group instructional diagnosis, which is a unwieldy term for a kind of guided reflective discussion for midterm feedback and input from students about how a course is going. I think there’s a lot of tools already there that, working together…. and this is the other hard part….working as colleagues, working outside of my old office and department, and with my faculty development people or with other academic support, can I bring these folks into the process where they could work with my students as well, and help me gather that data? Am I doing, in actuality, what I say that I’m doing? What is my students’ experience of this course? And is that in alignment with what I have designed the course to be?

Rebecca: We all hit barriers like time and things that cause us to slip into old habits occasionally. So those aspirational moves certainly occur for all of us. But I also think that that transparency piece about like, “Why are we doing this active learning thing?” or “Why are we slipping into this old habit that’s maybe not the most ideal?” …can actually be really healthy, because then students can also share that and have that dialogue going back and forth so that they know where they’re at in something and vice versa, like we know where we’re at in terms of the classroom.

Kevin: Well, and it’s an excellent model, too. I think modeling transparency with our students in any way that we can about the course design, about the content, about the ways in which we might be collectively engaging with that. All of that is to the good. We want students to understand that learning doesn’t just happen by accident. We want them to get into this place where they’re thinking metacognitively, and to me really the only effective way to do that is to have this sort of radical ethic of transparency. If a student says “Why are we doing x in this course?” If my syllabus has, here’s all the stuff that we’re going to do this semester, and I can’t come up with a good answer to that, that’s a problem. And so this helps, as you suggest, keep us in this place where we’re ensuring that what we’re doing is in alignment with our goals and our values as disciplinarians, in other words, as members of our discipline, as well as effective instructors and human beings. Does this stuff align with what my professed core values as an academic and as an educator are. The only way we’re able to do that, I think, is to be in this place of transparency to model what that looks like for our students.

John: One issue where the logistical question comes up is that during the pandemic, a lot of faculty relaxed deadlines, and sometimes dropped deadlines entirely. And that certainly provided students with the flexibility they needed. But one concern is that some students would end up getting further and further behind in the course. And there’s a lot of research that suggests that without some structure in terms of deadlines, and getting things done in reasonable periods, the quality of student learning tends to deteriorate. What sort of policies might give students some flexibility, but still make sure that they’re progressing through the course, so they can keep up with other activities that they’re supposed to be doing as the course progresses.

Kevin: And so this is the key issue. And so I think it’s important to establish that, when I suggest that rigor, at least as we sort of traditionally used it is actually failing us, is getting in the way of actual learning, I am not suggesting removing structure from a course. And I wish I could remember which of his writings it’s from but there’s this piece of one of the Paulo Freire books or essays that I’ve read, where he talks about what we would call learner-centered instruction. And he basically says if we’re in a rowboat, and I teach you to swim by throwing you overboard, that’s learner centered, but that’s not necessarily helping you learn how to swim in that moment. And his point is structure is a necessary part of what he would call a liberating pedagogy, that it’s not just throw people in and say, “Okay, go learn, you are the agents here.” And so I think it’s important to realize that you can do this work well and meaningfully only if there is a structure in place, but it has to be a structure that’s explicit, that’s decipherable for students. Back to your notion of the hidden curriculum, if it’s hidden, [LAUGHTER] and the students are running up against these sort of invisible barriers without being able to name what they are, that’s a problem. But if there’s structure in the course, where students are able to see “here’s what I’m accountable to, and here’s how I’m accountable to other students in the class,” then you’re in a place where you could do what we might call that desirable level of difficulty. And so I think there are ways to bring in structure and maybe more structure than what we had during the pandemic. Getting through what we got through was a victory in and of itself, and whatever we had to do to do it, we got through it at least relatively unscathed as higher education. I think that’s a really important win to acknowledge. But it doesn’t mean that we have to go all the way back the other way now. And so I’m a big fan of the sort of nuts and bolts level of policies that build in flexibility, but don’t get rid of structure entirely. And so Ellen Boucher wrote a great piece in The Chronicle back in 2016, and had the headline, “It’s Time to Ditch our Deadlines,” which is unfortunate, because that’s actually not what she argues. She advocates for a two-day grace period, no questions asked, for her students. And if you needed more time than that, then you had to have a conference with her and come up with a plan, like “Okay, I can’t get this paper to you on the due date. I’m going to take the two days.” While I need more than the two days we’re going to have a conference and as the instructor, I’m going to work with you and say “Here’s the game plan. Here’s your next step. Here’s what you’re going to be accountable for and when you’re going to be accountable for.” So I’m doing extensions, but I’m not just saying turn it in whenever. I’m not leaving students to figure out “Okay, what are my next steps? What are my next actions?” Because in the case of deadlines, we know that when students are failing to meet deadlines, as you mentioned, this stuff just snowballs, and it becomes worse, and then they just ghost us, because the whole thing has become so overwhelming that the avoidance reflex kicks in. And so something like Boucher suggests where right off the bat, here’s a two-day policy, if you need it, just tell me you’re taking it, I don’t need to hear about whatever stomach ailment you had, or I don’t need the graphic email describing your symptoms. Just tell me you’re taking the two days if you need more than that we got to talk first. And so that’s an example of a policy that has a structure there but still it explicitly packages in that type of flexibility. And I think that’s a good model for where we need to be, understanding that for different students life is happening in different ways. And yet, there’s still we can’t just say “Okay, turn things in whenever,” because as most of us design courses, stuff builds on each other, right? That’s the whole point of scaffolding and getting rid of deadlines entirely or not having that sort of structured accountability in place does prevent the type of things from happening that should be happening. And so I think finding ways to preserve structure but flexibility within that structure, which, I get is… as I listen to myself say that, part of me goes, “Well, that’s a really just kind of wishy-washy answer.” But I do think, in this case, that moderating it, there was a reason we got rid of so much structure during the pandemic, with this recognition that it was absolutely necessary for the way that everybody’s lives were unfolding and happening. That didn’t go away, like people’s lives still unfold and happen in very complex ways. And that’s true for our students and us. So we can’t just say, “Oh, we’re going to swing all the way back, 180 degrees to the other side, and have deadlines by God.” I just think that’s an incredibly counterproductive thing. And I think that the folks who have been trying that are the ones who’ve been seeing a lot of resistance, and not a lot of success in terms of their students meeting those things. And that’s where we hear some of this frustration that’s coming out in the discourse.

Rebecca: I think the other thing that sometimes rubs up a lot against rigor is this idea of relationships between students and other students and students and faculty in the classroom and that sense of belonging… somehow these are like diametrically opposed. It’s not like rigors over here and belonging is over here, and they can’t possibly happen in the same place.

Kevin: Yeah, and doesn’t that speak to what we’ve seen with the sort of debate such as it is over active learning? The conversation starts from an erroneous proposition that you could either do active learning or you could have a “real class,” like you could do this namby pamby arts and humanities, sit in a circle and sing Kumbaya and braid each other’s hair kind of stuff or you could do real learning and manly-man stuff. And I’m exaggerating, actually only slightly. [LAUGHTER] A lot of times the conversation about rigor and challenge starts from this erroneous sense of mutual exclusivity, that you can have a compassionate flexible pedagogy or you could be rigorous, but you cannot do both. When it actuality it’s and you have to have one to have the other. You cannot have a challenging learning experience where your students can actually meet those challenges if you don’t have a compassionate empathetic pedagogical space, because the whole point about rising to challenges is you can’t do it by yourself. This is what the Vygotski talks about the zone of proximal development, learning is social, you need other people around you, you need an instructor, you need classmates. Well, why is that? It’s because we help each other when it comes to the point of really challenging and pushing ourselves cognitively to get to that achievement of that goal, that desirably difficult goal. We cannot do that if students don’t feel that they belong in that space. We cannot ask students to take intellectual risks or to try something that they have never tried before, if they’re at a place where they don’t feel secure in doing so. Because we wouldn’t ask that of ourselves either, if we’re being honest. And so rather than posit: “you could be rigorous or you could be flexible and compassionate,” it’s “you can be flexible and compassionate and then you can be challenging.”

John: At Oswego this fall we’re going to be using for one of our reading groups, Sarah Rose Cavanagh’s book that you reference here, her newest book, which is Mind over Monsters, and you cite that basically as suggesting the importance of bringing both of those things together. And it’s an excellent book, by the way, which I would recommend to anyone interested in addressing some of these questions.

Kevin: Absolutely. It’s a brilliant book. And I think it’s a vital intervention in this very conversation that we’re having.

John: A term that you use in the book, which was a technical term I hadn’t quite seen used in this context, was that many faculty when they tried to introduce rigor, essentially are adding more “hard-assery”, I think was term that you uses rather than actually more cognitive challenge. Why is that happening? Why do people do this?,

Kevin: Yeah, and the phrase I use is performative hard-assery. [LAUGHTER]

John: Oh that;s right. Sorry, I forgot. [LAUGHTER]

Kevin: Well, I’m glad that that resonated, because I’m really proud of that phrase. But I really do think that that is where a lot of the rigor conversation is. My classes are hard, my students write, they read a bunch, they do all these things. And it’s like, “Do they? and what is the result of them doing those things?” It comes back to the student perception of rigor as the more pages I decide to read and write, students don’t talk about what they learned, and they talk about what they had to do. They don’t talk about what they became. They don’t talk about how it made them feel. They don’t talk about how they changed. They just talk about things they have to do. And I think that that’s a really important distinction when we think about the student experience in all of this. And I think it’s very easy for us to say, “Well, I’m doing my job because I’m assigning my students a ton of work, and it’s hard and I grade hard.” And again, I’m not saying we shouldn’t be challenging. I think we should absolutely be challenging. But how are we structuring those challenges? Am I giving my students a challenge just sheerly through volume. Is my class just kind of a death march through this enormous swamp land of content that’s just going past them a mile a minute, or Is it challenging in the sense that they are taking the time to be deliberative thinkers, to be critical consumers of information, because those two things look a lot different. And again, especially coming at this sort of not quite post-pandemic stage that we’re at, is a very real desire to bring some structure back, but thinking about bringing that structure back in terms of just assigning more stuff, because from our own graduate school experience, that’s how we structured our very lives. And so if we think about structure, and again, I’m saying we very intentionally here, that’s the first place our mind goes to. And so I brought back structure, because I’ve assigned a whole bunch more work. And now I also get to complain about how students aren’t doing the work, because I’ve built in this sort of self-fulfilling prophecy, it makes me feel good, it makes me feel like I’m “doing my job,” but am I? Are students learning? Is this advancing learning, because chances are, it’s probably not,

Rebecca: As you’re describing these two scenarios, the marching through the marsh, for example, I’m thinking of the really long checklist that accompanies it. And then in this other environment, this luxurious amount of time to contemplate something and wrestle with something and think through it, and how there needs to be space around that sometimes, to really have the time to process and understand what it is that we’re trying to grapple with.

Kevin: On a micro level, we think about this as instructors all the time, when we think about trying to foster an effective discussion in class. We know that asking a good question is the essential piece of having a good discussion. If I ask a complex question that requires a fair amount of cognitive heavy lifting, I need to give my students time, I can’t expect my students to answer it right off the bat. If it’s a good question, there’s going to be some silence afterwards, as students think about and chew on it a little bit before they decide how they want to respond. And it’s true on a course level too. Are we providing space for our students to do this work, to do this processing? Or is it just more, more, more, more, more, faster, faster, faster, faster, in the name of rigor, in the name of structure, in the name of challenging, but it’s really kind of the cognitive equivalent of trying to drink from a fire hose, like what’s really happening there?

Rebecca: Sometimes not much. [LAUGHTER]

Kevin: I have actually stood in front of a fire hose, not willingly, but I’ve been hit by a fire hose. And I can tell you, it’s not pleasant, it hurts and you’re really wet and miserable afterwards. And I would argue that those are not the things [LAUGHTER] we want associated with learning spaces. And yet, this is, a lot of times, where we, and again, speaking broadly, where we kind of lay it…this sort of, we’re gonna fire hose everything out, and it’s up to the student. And I’m exaggerating slightly for effect, but again, not very much.

John: If the solution to the student disengagement is not dumping more work on students and having more students fail along the way, as many people seem to see it, what can we do to get students a bit more engaged with the class? Because that’s been a complaint. I’ve heard from a lot of people at many institutions in the last year or so.

Kevin: So I think there’s two things I would use to answer that question. And the first I would say, engagement’s not going to be 100% all the time. And if we are thinking that it was somehow that way, magically, before COVID, we’re deluding ourselves. And so we have to give ourselves permission to fall short in that category, not every student is going to be engaged in everything at every time, no matter what we try to do, because that’s the world we live in, that surrounds the spaces we’re in. And so let’s be realistic in what it is that we’re after, how do we engage students in a meaningful and at least most of the time kind of way? And that’s where I think we can do a lot. And so there’s a couple approaches that I think hold a lot of promise. One is we do have to be challenging, we do have to provide challenge, people like to meet challenges. If students think they know something already, they’re going to hear it and “Oh, I already know this, I’m already checking out.” So we have to put in this level of difficulty, of mystery, of complexity, but we have to provide support in helping them meet those sorts of challenges. And we have to be clear and transparent about how we’re providing that support. The idea of a safety net under the trapeze artists, the trapeze artists who’s doing incredibly complex and really, really difficult things that they’ve practiced a lot to do, but they’ve had a net underneath them, just in case it didn’t go well. And at a much lower risk sort of way, that’s what we’re doing. I’m asking you to do difficult things, things that you have probably not been asked to do before. You may fall short of the goal, but that’s okay, because here’s the supports underneath you. This is a space where it’s okay for that to happen. And so depending on the type of class and the discipline you’re in, that might look a little different. But when people talk about desirable difficulties, in other words, challenges that people can actually meet, even if it takes a lot of effort, but there is a solution. And again, I reference Vygotsky’s zone of proximal development, a lot in this. It takes a while to get there, but you get there and that’s what the important part is. So how do we create teaching and learning spaces where the challenge is centered, but the ways in which students are being supported in meeting those challenges are also at the center. Right now we’re very good at centering challenges. I would argue we’re less proficient at centering those other things. And so how do we support students in meeting those challenges of what should be a demanding education? And so Cavanaugh calls it, in her book compassionate challenge. I suggest in the article that the question we should have as our litmus test is: does this advance learning? Does this thing I’m doing advance learning in the sense of what are the goals? What are the outcomes that we’ve established for the course? Those sort of transformations, those promises, to use Ken Bain’s words, that we’ve made to students? Are we getting there? Do you know how we’re getting there? Are you able to assess as a learner yourself what’s working in getting you there? Those are the sorts of things that need to be at the center of the teaching and learning experience. And if we’re just doing challenge for challenges sake, or that sort of performative hard-assery shtick, our students are not going to be interested in having that conversation with us or with themselves even, about what’s working for them in terms of the strategies they’re adopting, and the things that they’re doing to meet the challenges that they’re faced with on our course.

Rebecca: Thinking about procrastination in relationship to what we’ve been talking about. And sometimes procrastination reads as lack of motivation, or a lack of engagement. And sometimes the reason for the procrastination is that there isn’t the deadline, or there isn’t the structure or there isn’t the milestones to move you along. How do you see the relationship of procrastination to rigor and this idea of engagement?

Kevin: That’s a great question. Because I think if we err too much on the side of the so-called logistical rigor, or it’s like just really hard, inflexible policies and strict deadlines and this high volume of stuff, we’re actually creating the circumstances that procrastination will become an epidemic among students, because what we’re giving is an unrealistic amount of work to do in the time that’s allotted. And it’s very easy to get from there to just sheer avoidance, I can’t do this, I don’t see a tangible way through this, I cannot see myself getting through this gauntlet. So eff it, basically, is how that works. And so I’m going to do other things, whether I’m doing this consciously, or subconsciously, or some mixture of both. That’s like the perfect storm. When it’s all extrinsic motivation and when it’s all insurmountable barriers, at least from the perspective of the student, that’s like the perfect storm for avoidance. And I’m someone who personally struggles with this all the time. My avoidance reflex is keenly developed over the 50 years of my life. [LAUGHTER] And I do not do well with unstructured time. And so getting back to this question of how do we find that balance, I think structure is important, not an overwhelming or a suffocating amount of structure. But there needs to be something in place to help our students fit themselves and their work and their lives into the framework of the course. We need to be able to give them the tools to do that without pre-determining every outcome or stifling every option. But I think thinking about what are the causes of procrastination, what are the effects of procrastination, because one of the things that when folks talk about student disconnection is this phenomenon that we’re seeing more and more of students just kind of ghosting, just dropping out, like they were in class for six weeks, and now they’re gone. And I think a lot of that is things have built up to the point where they seem so overwhelming that there’s no realistic solution in place. And a lot of that is exacerbated by that cycle of procrastination. And so by the time we get to the point where the student is feeling so overwhelmed that they just want to leave everything, which they do, it’s way too late. So we have to be intervening in the earlier part of that process where it’s procrastination that is creating the conditions that this sort of overwhelming volume is going to grow out of a little bit down the road if we’re not able to intervene. So I think thinking about procrastination is the way you frame it in these very explicit and sort of fraught of mind terms is a really important part of all of this.

John: Since I’m at Duke, I’ll mention a study that Dan Ariely had done a while back where he worked with one of his colleagues at MIT at the time, and they were giving students writing assignments where they had to write three papers over the course of the semester. In one class, they had fixed deadlines for submitting these papers that were evenly spaced. In the other section of the course, they were given the option of setting their own deadlines, which could be at any time during the semester. If they chose, they could set them all at the end of the term. And what happened was that students who had either fixed deadlines or who set the deadlines evenly over time ended up performing better than the students who chose to put the deadlines at the end. And I should also note, there was a one-percentage point penalty for each day they were late. So it was a small penalty, but it was a non-trivial penalty. So the logical thing is to put all the deadlines at the end and then try to get them done evenly. But the people who had deadlines later did the work later and did lower quality work. So those deadlines can be important as long as there’s some sort of incentive structure with it. And I think that has helped encourage me to not drop deadlines entirely. Usually I allow some scores to be dropped or allow some deadlines to be flexible, but warn students that if they don’t meet the deadlines they’re going to have trouble with these in-class activities that are going to be done based on the things they were supposed to have done before they come to class. But it’s a challenge. And I haven’t found a good balance

Kevin: That speaks to exactly the type of balance that we’ve been talking about. We don’t have to choose between strict, rigid, inflexible deadlines, or no deadlines, or complete student set deadlines, like the Elen Boucher piece that I referenced earlier. Here’s the structure, here are the deadlines. And then here’s the wiggle room that comes along with them. And so your desire to sort of have the deadlines but to balance them with flexibility in your classes, and to have students understand, this is why you need to have these things completed, or at least aiming for this particular juncture, because you’re going to need it in the next phase of the course, etc. This is all part of what we talked about supporting students to meet these challenges, this is the type of support. Support could be encouraged through our course design, as well as the actions that we’re taking on a day-to-day basis. And so again, I want to be really clear that I think rigor as we’re using it kind of higher education wide, has outlived its usefulness as a word. It has too much baggage, it has been wielded in exclusionary, inequitable, and sometimes very horrible ways. But that doesn’t mean that we don’t talk about challenge. And so I think like Cavanaugh does, and like a lot of other folks who have landed on this idea of thinking about ways that we can challenge our students. And the way to do so is to create structured environments where the structure facilitates rather than suffocate students as they endeavor to meet those challenges.

John: And at our teaching center, we’ve given hundreds of workshops over the years, but never once have we advocated rigor, or even used the term rigor in our framework we always refer to challenge and the benefits of that. And you cited a Chronicle article that was the basis of a podcast episode with Sathy and Jack. And we’ll include a link to that in the show notes.

Kevin: The headline was, “It’s Time to Cancel the Word ‘Rigor’” and playing on everybody’s sort of obsession with the boogeyman of cancel culture. And I think that that got in the way a little bit of folks engaging with the very real truth of the article was that, as Rebecca alluded to earlier, rigor has been wielded rather than used it’s been wielded like a cudgel, it’s been a barrier, it’s been exclusionary. And when we talk about rigor now, I think everything has a history, this is my own discipline talking here, I don’t think rigor with the amount of baggage it’s carrying, I just don’t think it can be constructively used when we’re talking about challenging students. And because students have experienced rigor, defined that way and referred to with that word, to the point where there’s that kind of baggage with it too where it’s sort of the pedagogical equivalent of hazing, as opposed to anything else. It helps us as educators ask ourselves, are my challenges cognitive or just logistical? Are my challenges supported for students? Or am I just sort of asking them to close their eyes and jump off the cliff and trust that no bad things are happening. And it helps me as an instructor hold myself accountable to ensure that again, I’m not doing the things that I would rather avoid and that I am doing the things that I tell students I’m doing to help support their learning. In the tradition of first-year student essays everywhere, look at the Webster’s definition of rigor, it talks about things like extreme inflexibility and rigidity. There are connotations, you know, rigor is for corpses. So I think that it’s a concept that has no usefulness for the questions that we’re trying to answer and the knots that we’re trying to untie at this particular moment in higher ed.

Rebecca: There’s a lot to think about. Thanks. [LAUGHTER] So we always wrap up by asking what’s next?

Kevin: Well, right now, I’ve actually thinking a lot about the course spaces in which some of these things that we’ve talked about play out all the time, and that’s the intro or survey courses. And I think the project that’s kicking around in my head right now, and I’m getting dangerously close to actually starting to write stuff, is thinking or rethinking the intro/survey course, sort of a critical interrogation. What are they supposed to be? What are they actually functioning as in reality? How large is the gap between those two things? And what are some ways in which people are creatively answering some of the problems that the survey course presents in terms of not just teaching but designing effective spaces as well as some of this comes out of my own field in history. We’re wrestling with again this death march through content. World history in two semesters, Plato to NATO in an academic year. Is that really what we’re after here? Or should we be doing something else? And so from my own discipline, I’ve developed an interest in thinking about this and thinking about the ways in which other fields and disciplines are wrestling with similar types of questions, which of course, then leads to the larger question of what is the point of these things? And are we doing the things that we say these courses should be doing? Because of that, thinking a lot about not just teaching and learning, but about first-year student success, about things like just and inclusive teaching, things like student-centered pedagogy, a lot of really interesting and fun things that are kind of swirling around. So the short answer, rather than that very long-winded one is I’ve researching survey and intro courses to see if there’s better ways that we might be doing it.

Rebecca: Sounds like a great project. Maybe you need a deadline so you get started on it. [LAUGHTER]

Kevin: Let’s not get carried away because I am going to take that two-day grace period, I can tell you that right now.

John: Well, thank you. It’s always great talking to you.

Kevin: Well, thanks for having me back. It’s a real treat to be with you two again.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

300. Episode 300 Reflection

This is episode 300 of the Tea for Teaching podcast. Whether you are a new listener or have been with us for all 300 episodes, we are very grateful that you’ve joined us on our podcasting journey. In this episode, we celebrate this milestone by reflecting on what we’ve learned and how the podcast has evolved.

Show Notes

Transcript

John: This is episode 300 of the Tea for Teaching podcast. Whether you are a new listener or have been with us for all 300 episodes, we are very grateful that you’ve joined us on our podcasting journey. In this episode, we celebrate this milestone by reflecting on what we’ve learned and how the podcast has evolved.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Now that we’ve reached Episode 300, we thought we’d take the opportunity to reflect on some of the changes that we’ve seen in higher education since we launched the podcast in 2017.

John: Today’s teas are:…

Rebecca: I have Blue Sapphire.

John: I heard you just stocked up on a trip to your favorite tea store.

Rebecca: Yes, I was really excited to stock up on my favorite and I have a couple new ones too. So maybe in some upcoming episodes, we can try those out.

John: And I have Bing Cherry Black tea from Harry and David’s which is made by the Republic of Tea. When we first got started on the podcast, much of the focus was on specific teaching practices and techniques and interesting projects. Most of our guests were people that we knew or guests who were within our professional networks.

Rebecca: in the spring of 2020, as we know, [LAUGHTER] the focus shifted to the challenges associated with remote and online teaching, and the challenges facing remote learners and instructors.

John: As we became accustomed to pandemic teaching, we focused a bit more on faculty concerns as we transitioned into the transformed higher ed landscape. Historically, higher ed had been designed to serve the elites of society, and while higher ed gradually became more open and students have become much more diverse, many residual practices have worked against serving the students that we have. During the pandemic, faculty became much more aware of the inequities facing our students as well as faculty and staff.

Rebecca: Yeah, so one of the things that we’ve been talking about quite a bit is this more holistic focus on the needs of our students and faculty as humans, and really generating and creating a much more inclusive higher ed environment. How do you see that moving forward, John?

John: One of the things we’ve talked about is addressing the needs caused by the increased demands on time for faculty, staff, and students. As we developed new teaching techniques and tried to build more structure into our courses, it put much more demands on faculty in terms of redesigning their courses, in terms of paying more attention to the needs of students, and providing students with more feedback. And that has led to issues with burnout, which we’ve addressed in a number of podcasts.

Rebecca: And you’ve never experienced that, have you, John?

John: The day is not over yet. [LAUGHTER]

Rebecca: Well, how do we think about supporting faculty as we move forward with all these demands on their time in trying to prevent burnout? We’ve talked about this in a couple of episodes, but as we enjoy summer and have a little bit of more downtime for some of us… maybe not you, John… and move into the fall, what are some things that we need to be thinking about for our own classes to prevent burnout?

John: One of the things that we’re trying to be careful with when we recommend new teaching techniques to faculty at the teaching center here, is that they change approaches gradually, that small changes, incremental changes, are much easier to accommodate than the type of rapid changes that people had to do when they first moved into remote teaching. And so I think we have to be careful in making sure that we maintain a balance and we don’t burn out ourselves, because we’re not going to be very effective in supporting our students if we’re struggling to get through each day ourselves.

Rebecca: Yeah, we need to be present, just like we want our students to be able to be present and have the supports around them to be present in their learning. I think one thing that we’re also talking about in grad studies in our office is really this increased stress on faculty, and how do we support faculty, but also how do we support graduate student populations through things like accountability groups, or ways where there’s another human for accountability, but also for support, and not necessarily a mentor model, where there’s a power dynamic, but really a peer-to-peer approach to connect people together.

John: And we’re running two reading groups this fall to address some of these needs. One of the reading groups is on Sarah Rose Cavanagh’s new book, Mind over Monsters. And the other one is the second edition of Jim Lang’s Small Teaching. We had done that a few years ago, but we’ve had a lot of new faculty since then. And while we try to reach as many faculty as we can in our workshops, there’s a lot of faculty who are still teaching in pretty much the same way as faculty were teaching a century or so ago. And we’re hoping that by encouraging small modifications in teaching approaches, it might encourage more faculty to participate in introducing active learning activities and evidence-based teaching approaches.

Rebecca: It’s really easy to slip back into past practices when we’re tired [LAUGHTER] and overworked. And it’s not surprising that people have kind of slipped back into assignments and stuff that they’re really familiar with to reduce the cognitive load around new stuff and the many stressors around. So having that added support to help faculty re-engage with some of those ideas is, I think, a really great idea at this juncture. And I love that Mind over Monsters is one of the reading groups as well, because mental health is such an increasing concern, not just for students, but also for faculty and staff.

John: And we’re very much looking forward to both of these reading groups. Among the things we’ve talked about more frequently since the start of the pandemic are the challenges faced by underrepresented and contingent faculty.

Rebecca: I think when we’re introducing new techniques, and we’re thinking about supporting students around mental health, or we’re thinking about evidence-based practices and engaging in active learning, we need to remember that contingent faculty or underrepresented faculty have different barriers or different obstacles in implementing these things… or even more pushback from students and implementing some of these techniques. So we really need to be cognizant of supporting each other and realizing that we don’t all have the same kind of supports in place. I think some populations of faculty are just overly criticized. And when they try something new, it’s not accepted in the same way that a more dominant group’s adoption of those same techniques might be.

John: And that’s true both by students as well as by their faculty peers. And one of the things that’s come up in many of the podcasts we’ve discussed are the biases in both student and faculty evaluation of teaching.

Rebecca: Yeah. One of the things that I think is on the minds of our faculty too, is, as we’ve seen increased diversity of our students, we’ve seen diversity in levels of preparation. And I think those inequities have always been there. But again, maybe it’s more visible now than it had been in the past. How do we work through that in our classrooms, especially in these more introductory classes as students transition into college?

John: Well, I think those inequities have always been there, but they certainly grew with remote teaching, because our students face very unequal resources in their school districts and in their households. And when people are physically in the classroom, they’re at least exposed to the same infrastructure within their institutions. But when students were taking classes from home, as we talked about in many, many episodes, during a pandemic, they had very unequal network access, they had very unequal computing facilities, they may have been sharing a computer with multiple family members, they may have been forced to work. And as a result, the inequities in prior education and prior learning became much more dramatic during the period of remote teaching. And that disproportionately affected students from low-income households and low-income school districts. And what we have to do is provide resources, I think, for all students to be successful. And while we always should have been teaching, or providing resources and support, for all students, those needs have become much greater now, because while we are bringing in a much more diverse student body, we’re also losing students who come in with less preparation at some of the highest rates we’ve ever seen before. And we have to make sure that we’re providing the students that we accept with the support they need to be successful. And there’s lots of ways of doing it, you can build in some additional resources, you can connect to YouTube videos, and such things and provide support to students, you can use mastery learning quiz systems, and many other techniques. But we have to work towards having more faculty building that in because while many faculty are doing these types of things, and trying to build more support and more structure into their classes, it’s not a universal phenomenon.

Rebecca: And maybe even acknowledging that some students in the class are quite literally working harder to get to the same level.

John: The last few years when I’ve been teaching my large intro class, that’s something I’ve mentioned explicitly. I said, everyone here has all the resources they need to be successful. But if you had taken an AP introductory microeconomics course, or something close to that, you’re not going to have to work as hard to attain mastery of many of the concepts. If you have not been exposed to these things, or if your background in working with math and using graphs is not as strong, you’re going to have to work a bit harder. And that’s not a message that a lot of students appreciate hearing. But if we want to get all of our students to the same level at the end, the students who do come in with a weaker background need additional support to get there. And using tutoring when available, encouraging students to come in and talk to their professors and use office hours, all of those things can help but we’ve got a ways to go. What are some of the things you do to try to provide support for the increasingly diverse student body?

Rebecca: Yeah, I think the reality is that what we give each student isn’t the same, because what they come in with is not the same. So I often am trying to assess where students are and then pushing them right at that moment where they’re at, rather than expecting everyone to be at the same point. And I can do that a little more efficiently in a small class than you can in a bigger class. But I think we need to use those smaller classroom spaces to be able to do that so that everyone feels challenged, but also has what they need.

John: My introductory course is a prerequisite for all upper-level economics classes, and most of my students will be moving on to upper-level classes, so they have to reach at least a minimum level of proficiency in the discipline in order to be successful in their future classes. In other classes, instructors can be more flexible, and just try to get the most learning gains in their students, no matter what their starting points were. In my introductory class, at least, I have to pretty much take the students where they are, and try to get them all to the same place, while making sure that they’re all challenged. And that’s a very challenging goal to reach.

Rebecca: …and the difference between teaching those introductory classes versus higher level classes within a discipline, for sure, I think one of the most efficient things we can do is making sure that all students know the most effective ways to learn, because they don’t necessarily know those things coming into college, or even into graduate school. What we need to just remember is learning isn’t something that we just magically know how to do, we need to learn strategies and techniques that are effective.

John: We’ve had a number of guests over the last year or two who’ve talked about books that they’ve provided, or resources they’ve created to help students be more successful. Because one of the things that’s been pretty obvious for quite a while is that the study techniques and the learning strategies that students use are not generally consistent with what evidence suggests is most effective. And as a result, students are not using their time as efficiently as they can, by engaging in strategies that they perceive as being useful, that really result in very little increase in long-term recall… strategies such as highlighting, repeated rereading, and so forth. And one of the things that might be helpful is if we all could shift students a little bit in the direction of using evidence-based learning strategies, and some of that could take place through course design, by building infrastructures that incentivize the use of these techniques.

Rebecca: Yeah, and I think the moment that students realize that they don’t have the most efficient way, or the moment that a student begins to struggle is different, depending on some of that background, that they have. Students that come in well prepared may have never really struggled in high school, and maybe eventually, maybe even in the first year of college, you don’t struggle, but maybe it hits a little later on in their education, maybe not until graduate school. And then other students might struggle the second they get to college, because there’s not as much structure in place as there was in high school. So I think we need to be underscoring these techniques at all levels, and not just in their first year.

John: And one other thing that’s been discussed in many podcast episodes, is the importance of making the hidden curriculum of higher ed transparent to students, so that we don’t expect students to know what a syllabus is or how it could be used, that we shouldn’t expect students to know what is expected on a term paper in a class without making those expectations explicit and transparent to students. Because in general, we see a lot of students coming in, and they see it as a game where they’re trying to guess at what instructors are asking. And many of those guesses, especially for students who have not been in college prep classes before, are wrong. And they wasted a lot of time and effort that could have been spent more productively developing their understanding of the subject matter.

Rebecca: And the reality is that there’s differences between disciplines and between courses. And so the more we can be explicit about expectations within our own discipline, and within our own courses, and beyond the classroom experience of higher ed, because there’s expectations in other spaces as well, like student clubs, athletics, and all of the rest of the co-curricular activities that support student learning are incredibly important. And those are also not obvious.

John: One of the things that we’ve talked about much more on the podcast, and higher ed in general has been addressing much more extensively since the pandemic, is alternative grading approaches. Because traditional grading approaches and traditional course structures generally incentivize students to cram and to focus on maximizing their grades, rather than maximizing learning, so that if we really want students to shift to evidence based learning strategies, it would be really helpful if we could shift students emphasis away from grades and faculty emphasis away from high-stakes assessed activity and shift it more to activities that result in deeper learning, more long-term learning. And we’ve talked to many guests who have shifted to using strategy such as specifications grading, mastery learning systems, portfolio assessments, and ungraving, which has become one of the most talked about topics in higher ed in the last few years.

Rebecca: Yeah. And I think one of the things that comes up in a lot of those conversations is concerns over students just wanting the right answer and not learning and not critically thinking about the subject matter and the knowing of why and how, and doing analysis. And I think every instructor [LAUGHTER] has a desire for some of those kinds of conversations to come out in their classes, rather than just regurgitation of things that they’ve said in class.

John: Part of the issue is that when we get students in college, they’ve already had 13 years of experience in K through 12, where grades were the primary area of focus. And as a result, it’s hard to shift that focus from grades to learning. Besides alternative grading, we might use some other strategies such as encouraging students to be more reflective on their work, to spend some time in reflection-based activities and metacognitive development type activities.

Rebecca: Yeah, I know, this is a space where I was maybe a little hesitant at first thinking like, “Oh, these are just quick assignments that have no meaning,” but quickly realizing actually the value in really good well designed reflective activities that challenge students to think through how and why they learned something and what it is that they actually got out of an activity. And I’m often very surprised about how much learning occurs that is not visible, despite the fact that I teach studio classes, so I’m with my students much more than the average instructor. So I actually do observe a lot of learning. But in the reflection activities, I’m hearing a lot about how students are spending their time or things that they really struggled with and worked through that I wasn’t aware of. It also helps me understand where they’re not aware [LAUGHTER] of their own learning, or where they’re using strategies that aren’t as effective and helps with interventions. I know you’ve done a lot around metacognition, especially in your lower-level classes, but also in your upper classes.

John: In at least a couple of my courses. I’ve been using the metacognitive cafe discussion forum, which was actually the topic of our second podcast, Judie Littlejohn and I jointly developed this quite a few years back. And it’s been remarkably effective. It’s basically a low-stakes discussion forum that I’m using in my online classes, where students will reflect on their learning and share their learning strategies and will also read a bit about retrieval practice and spaced practice and the benefits of sleep in learning. And every time I do it, even though it’s only a trivial portion of the grade, it’s 5% of their total grade for participating in that activity, the students report that it was the most valuable learning experience they had in the class. A large proportion of the students at the end of the terms say they wish that they had learned these things back in elementary school, that they had been using practices that were not efficient and they didn’t realize that because they’ve never been taught how to learn. And it’s something that students have found really valuable. And the other nice thing about it is because, in this particular case, it’s done in a discussion forum, it helps them build community and helps them get to know each other, because they’ll often talk about the challenges they face. In online classes, many of the students have families where they’re taking care of young children, they may be working different shifts, they may be faced with other challenges that normally wouldn’t come up in a content discussion forum in an online class. But when they share that, and they share those challenges, and they share their career expectations, and they talk about how what they’re learning might be useful in their expected careers, besides the sense of connection, it also helps students see the relevance of what they’re doing and sharing that with other students helps build a little bit more intrinsic motivation in learning.

Rebecca: It also seems like there’s a bit of an immediacy in that context as well, because the information can immediately be put into action in a real lived experience and not something that may feel abstract, which sometimes happens within a discipline when it feels like maybe it’s not a thing I’m going to do anytime soon, professionally. So I think this really highlights the reason why we need to help students hook into everything that we’re doing to make it feel like they have a personal, professional, or educational connection to their own goals.

John: One of the topics that I use in each class where I’ve done this, at a point where students face the first really challenging material in the class, is just asking them to discuss how they deal with challenges. They share useful strategies, but one of the main benefits of that is it normalizes the sense of struggle, that when students are struggling with concepts, they often feel that they’re alone on this, but when they hear that other people are struggling with exactly the same issues and exactly the same concepts, it normalizes it, and again, it helps them understand that challenge is an important part of learning, which is not the message that they’ve generally received throughout their prior educational experience before coming to college.

Rebecca: It seems to me like this is the same reason why our reading groups work so well for faculty development as well is this connection among peers, but also that the challenges we experience are not in isolation. [LAUGHTER]

John: One of the things I’ve always enjoyed about the reading group is when people will come up with a technique or describe something they’ve done in class and people in completely different disciplines who might never have considered that will make connections and see how they could do something similar in their classes. That understanding that we’re all facing the same challenges makes it much easier to deal with some of the day-to-day stresses that we might have in our classrooms.

Rebecca: I know that one of the things that has come up in our reading groups, and also in our conversations about the future of higher ed and where we’re going to be going over the next few years is student engagement, and then specifically, the role of AI tools like ChatGPT [LAUGHTER] in the conversation. So if students don’t feel motivated, and they’re relying heavily on these tools, how do we get students to re-engage with the idea of learning?

John: Well, going back even just a little before the introduction of ChatGPT, which kind of hit higher ed by storm in late November of 2022, we did see a dramatic increase in the use of sites such as chegg.com, and various other sites out there, where the use of those tools became normalized a little bit, which made it much more challenging to give online assessments. And I think that’s where most people are concerned right now about things like ChatGPT, because with other places, you could at least locate where answers were coming from. And you could address that with the students and attribute it to the specific sites where they got their answers, which was, again, a bit of a challenge. But ChatGPT is raising some challenges for assessment that are going to be difficult to deal with, because it’s much more difficult to determine who is the author of specific items submitted online for assessment. And a lot of people are struggling with that right now. I know I’ve been struggling with it. In my spring 2023 online class, the quality of student writing on essays improved fairly dramatically over the course of the semester. And that seemed to correlate with the spread of the use of ChatGPT a bit. AI tools are really powerful, and they can be really useful. And they have a lot of potential value in education and in providing support for personal and work productivity. Right now, I think, more people are focused on the challenges, but we’re going to have to start thinking about ways in which we can productively integrate this and prepare students for a world in which the availability of AI tools will be ubiquitous.

Rebecca: And you teach in some really challenging contexts, really large classes in person, a number of online sections, and I know ChatGPT is keeping you up at night. What are some things that you’re thinking about… maybe haven’t resolved… but that you’re really thinking about redesigning or rethinking or retooling in the fall to just respond to the moment that we are currently living in.

John: As of 2020, I had shifted all the quizzing to online quizzes and tests and midterm exams and so forth. I’m seriously thinking about in my large face-to-face class, moving back to at least in in-class midterm and an in-class final exam. I really appreciated the fact that I could let students do it at their own pace, and that it took some of the anxiety and stress away when students did not have this two-hour time limit to complete an exam in the classroom. But with the size of the class, a large proportion of the testing is done with multiple choice exams, or algorithmic questions, and those are types of things that ChatGPT answers really, really well. Not too long ago, someone posted that ChatGPT 4 received a score of a 99th percentile on the Test of Understanding in College Economics (the microeconomics version of that), and those are the same types of questions that I’d be giving students on these quizzes. And while I had 1000s of questions that I had created that students were selecting from, all of those questions now are vulnerable to the use of AI tools, which makes it much more difficult to assess in that large class. Right now, the only thing I’ve really thought about doing differently in my large class is moving back to at least a couple of in-class exams. Now some of the things I was doing, such as polling questions embedded in the class activities and working on problems in class, where students submit that in real time, are generally much less subject to that type of issue. I know there are tools where students can scan the questions and so forth, they get responses back a bit more quickly, but it wouldn’t be as easy for them to do in real time when they’re in a polling environment. One of the main benefits of that is when I use polling, it was always tied with peer-to-peer discussions. And those peer-to-peer discussions is where most of the learning actually occurred from those in-class problem-solving exercises. For my online class, I’m not sure what I’m going to do. One thing I have done in the past is I’ve had students do podcast projects. And again, it’s pretty easy for chat GPT to generate scripts, but these projects are pretty heavily scaffolded. Students submit a proposal and they go through a number of steps to get there. And projects that are scaffolded like that, are probably a little bit less sensitive to the use of AI tools to generate the entire project. What are you thinking about in terms of your classes, or in terms of the graduate program?

Rebecca: I think we’ve talked a lot about the concern over the validity of our assessments and wanting there to be accuracy, not just for our sakes, but for students’ stakes in the value of their degrees moving forward. Part of it, I think, is really engaging in conversations around ethics around these tools, and not necessarily discouraging the use of the tools entirely, or banning the tools. I think that just motivates people to want to use them more, but rather to use them in ways that are productive, or interesting, but are also well documented… [LAUGHTER] like students are disclosing what they’re doing. And we can analyze the use of the tools in particular ways because maybe it could save time in particular places and not take away from certain kinds of learning, as long as we keep the learning objectives [LAUGHTER] up front. And then we assess when we’re using particular tools to determine whether or not it’s taking away from the learning. But I think these are hard conversations to have, and certainly not things that I want to be policing.

John: And I’d much rather not be policing these things. Sometimes students haven’t given me much choice in that. One example that I’ve seen recently is students submitting exam responses that asked him to analyze recent data, where the response said something to the effect: “as an AI tool, I do not have access to this data.” And when a student submits work like that, it’s pretty clear that they haven’t even read the essay responses they’re submitting on that graded assessment activity. And we want to make sure that students do actually interact and engage with their learning materials. Perhaps we can also design assessments that are not as vulnerable to AI-generated text. This semester, with my online classes, one thing I have shifted to, instead of having them discuss general debates or issues in economics, I have them focus on interpreting videos online, for example, where economists are debating certain topics, or doing readings that are not in the training database for ChatGPT, which means it’s much harder for AI tools to generate responses when they don’t have access to the underlying content that’s the focus of the assessment activity.

Rebecca: Would hyperlocal situations or examples also be a strategy because there’d be less widely available information on something like that.

John: Definitely. Information on the local community or the campus community or other local things, information that would just not be part of the training database is a good place in which we can ask students to connect the materials their learning to real-world events so that you maintain that sense of relevance while ensuring that the students are actively engaging with the work themselves rather than using a tool.

Rebecca: One of the things that I’ve used historically in my design classes, and it’s a little easier again, because I teach studio classes and see students more often so I tend to have a hunch as to what they’re working on, because I’m seeing them working on things, is really documenting process and not just using language, but showing through a video and showing steps along the way that might not be as easy to capture as an end product using an AI tool.

John: In general, open pedagogy projects, too, could be less vulnerable to having work being done entirely by AI tools. So videos would be an example, wikis perhaps might be.

Rebecca: I think that things that combine text and image are more challenging to have an AI tool create, at this moment.

John: That may very well change…

Rebecca: …by the time this episode comes out. [LAUGHTER] I think one of the things that I’m hearing us say actually, is that a lot of the strategies to reduce intellectual integrity or academic integrity issues around ChatGPT are also the things that are more likely to engage students and foster their learning anyways because they’re more authentic assessments, they’re probably more project based, they’re probably more long term with milestones along the way. And these are things that students often deeply engage in. And I think when they can connect to their local community, whether that’s the campus or the community that campus is situated in, or even their own hometown, in different ways around the discipline, those are all ways that students get a hunger to want to learn more.

John: And going back to our earlier discussion of the importance of shifting students’ focus from grades to learning, students are using tools like ChatGPT to raise their grades, even though they recognize it does not support their learning. If we can shift students’ focus to recognize the value of learning as improving skills that they’re going to need later in life, that should reduce the incentive for students to use shortcuts to avoid learning material.

Rebecca: If we’re not just looking for the right answer, but the journey to an answer, and even if it’s an incorrect answer, being able to understand why it’s not correct, and allowing that to be the learning is a really different way than our education has historically worked. The future of higher ed seems really stressful, John. [LAUGHTER]

John: It does, but it always has. That’s nothing new. But certainly the last few years have seen a lot of rapid change that… I hate to use the word unprecedented… but that have been relatively unprecedented.

Rebecca: And I think it really does speak to this need to connect with other colleagues, where we can share some of the challenges that we’re facing and brainstorm together to improve our teaching, but also to improve the level of stress we’re experiencing. [LAUGHTER]

John: One of the other things that we’ve talked about, especially within the last year or so is growing faculty concerns over student engagement. When students first came back to the classroom, there was a lot of excitement about being back. But since then, faculty generally seem to be noting that the level of engagement of students has shifted or has changed somewhat. More students are not completing assignments. Students in some classes have been disappearing from class as the semester progresses. And there’s a lot of concern that students are not as fully engaged with their coursework as they had been prior to the pandemic. So Rebecca, how are we going to solve this?

Rebecca: That’s a good question, John. I think one of the things that this aligns with is the higher incidences of loneliness, and mental health. And so finding ways to connect students to each other, and establishing those peer networks, I think, is one of the most important things that we can do in our classes. And it’s something that I’ve maybe always done in some way. But I’m being much more intentional about moving forward, because I’m feeling like even if students are in the same room, they’re still feeling really isolated. And so we have to be intentional about creating those opportunities for students to experience connection and feeling like they want to show up for each other and for themselves,

John: …using more group activities in class where the work of each student depends on the contribution of the other members does help create that sort of pressure on students to be there for their peers, to be there for the rest of their team. And that could be very useful.

Rebecca: Yeah. And I think the key to that, though, is not just assuming that students know how to interact with one another, or how to depend on one another in a team context, but really scaffolding those learning opportunities that really start with making connections and establishing relationships, because it’s the relationship that’s going to cause the pressure to show up for someone. And I think when we’re seeing high incidence of like ghosting, for example, it’s because the individuals don’t feel connected to the people that they’re ghosting.

John: And there have been a lot of studies done recently that show the importance of a sense of belonging in student persistence. So helping students form those connections is really important, because we have so many students who go to college, build up a huge volume of debt, and then disappear without getting the degree which does serve as a signal that they’ve actually accomplished something as a result of their education. And they end up with more financial struggles than they would have had had they not started. So we do want to help students form those connections for their own sakes, for their own future success. And one of the books we used in a past reading group was Relationship-Rich Education by Leo Lambert and Peter Felton. And that summarizes a lot of the research on the importance of building community and building connections, and also provides some really nice examples of ways in which institutions can transform to help facilitate those connections.

Rebecca: As instructors, we have a lot of power in that space to help students feel a sense of belonging. We can do really simple things to make someone feel seen and if they feel seen, they’re more likely to feel like they belong. So personalized messages, getting to know your students a little bit, being approachable, calling students by name, all of those things help students feel like they’re a part of a particular community. There’s so much to still learn and to come together around. And so I know that we’re looking forward to having many more guests and many more conversations to help work through many of the questions and concerns and things that we’ve raised today and have been raised by our colleagues. Now, John, we always wrap up by asking what’s next?

John: Well, what’s next for me is I’m heading down to North Carolina to teach at Duke again next week. And I’m looking forward to this. And this time, I’m going to try avoiding getting run over by a car. So I can actually teach my classes down there and spend some time away from the hospital this summer,

Rebecca: #life_goals. [LAUGHTER]

John: Small goals are sometimes more achievable. And Rebecca, what’s next for you?

Rebecca: This summer, I’m looking forward to doing some more work on our graduate student online orientation, which we put together as we transition to our new course management system in the fall and also working with some colleagues on an accessibility online module.

John: And we’re looking forward to talking to more of our wonderful guests. I’ve really enjoyed the experience of interacting with so many great people doing some really good research and doing such good work in higher ed.

Rebecca: We’re grateful for all of our guests and all of our listeners. So thanks for listening

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

299. My Professor Cares

Students from marginalized groups often question whether or not they should be in our classes and disciplines. In this episode, Michal Kurlaender joins us to discuss an easy to implement intervention that faculty can use to improve retention and student success. Michal is a Chancellor’s Leadership Professor in the School of Education at UC Davis and is a co-Director of the California Education Lab. She is a co-author with Scott Carrell of a National Bureau of Economic Research working paper entitled “My Professor Cares: Experimental Evidence on the Role of Faculty Engagement.” (This article is forthcoming in the American Economic Association journal, Economic Policy.)

Show Notes

Transcript

John: Students from marginalized groups often question whether or not they should be in our classes and disciplines. In this episode, we discuss an easy to implement intervention that faculty can use to improve retention and student success.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Michal Kurlaender. Michal is a Chancellor’s Leadership Professor in the School of Education at UC Davis and is a co-Director of the California Education Lab. She is a co-author with Scott Carrell of a National Bureau of Economic Research working paper entitled “My Professor Cares: Experimental Evidence on the Role of Faculty Engagement.” This article is forthcoming in the American Economic Association Journal, Economic Policy. Welcome, Michal.

Michal: Thank you for having me.

John: Our teas today are:… Michal, are you drinking any tea?

Michal: I am not drinking tea, but I did have some not too long ago today.

Rebecca: Do you have a favorite?

Michal: I’ve come back to Earl Grey. I used to be an Earl Grey person. I left it for a while, and it’s just made a comeback for me.

Rebecca: Nice. It’s a classic. I have Christmas tea today, despite the fact that it’s June.

John: And I’m drinking a ginger peach black tea today.

Rebecca: I have my Christmas tea because we had our presidential announcement today. And it was like celebration tea.

John: White smoke has come out of the towers [LAUGHTER] and we have a new college president here.

Michal: Congratulations.

Rebecca: So we invited you here today to discuss the study of the impact of specific faculty behaviors on historically underrepresented minority student success. How did you decide on this specific intervention?

Michal: My colleague and collaborator Scott Carrell and I do a lot of work to try to understand College Access and Success. And in particular, we’re interested in understanding inequalities in graduation rates at more open access institutions, like the California State University system, which has, across its system, some more selective campuses and some more open access institutions. But in particular, what we’ve noticed for years is that the graduation rates of students of color, particularly male students of color, black and LatinX men, were really much lower than other groups. And this was a puzzle to us, largely because the eligibility to get into four-year colleges, including the CSUs is quite substantial. These are primarily B-plus students who have finished a comprehensive set of courses required to be eligible for the CSU. And so to see their graduation rates lag so much behind other students was really troubling to us. And so that’s why we decided to focus particularly on the CSU system. And we focused on one campus in particular, that’s a less selective CSU campus.

John: What was the intervention that you used?

Michal: We didn’t go in knowing what intervention to use, we actually started with a focus group with particularly men of color at this campus and asked them what their challenges were. In particular, what we learned was that their challenges were not necessarily social or more broadly campus level, they were primarily in the classroom, and that is they felt disconnected from their instructors and from what to do to be successful. These were all students who reported feeling quite successful in their high school, feeling quite connected to their high school instructors who encouraged them to go on to college. Many of them were in competitive fields like STEM and engineering. And then they felt like they really struggled in college, and in particular, how to seek help and how to understand what instructors wanted from them. And so we came in quite agnostic, I would say, about what could work, what is helpful here? Is it more writing centers, more coaching, more nudging? We didn’t know. And what we came out with is feeling quite sure that we needed to tackle the classroom. And in particular, I think what we wanted to think about was this untapped source of potential support or hindrance in that is faculty, and think historically, we know that many times we just think of the classroom as kind of untouchable, and we put other support centers, writing centers, and tutoring centers, and other supports for students. And we kind of leave the classroom alone and leave faculty, including ourselves… we’re both faculty… to do what they will. And instead, here we wanted to really think about, could we intervene with faculty to provide more support for students?

Rebecca: Yeah, it’s funny how we often overlook that particular option given that’s a key touchpoint with our students, right?

Michal: Exactly. So we came out deciding that we’re going to do an intervention that was classroom based, and that was going to try to work with faculty to give students more information about what it takes to be successful in the classroom, how to seek help. And then we decided to pilot that to see if the proof of concept worked. And we piloted it at a large, more selective institution on a small-scale pilot, and found some promising results and then launched it at full scale. This article describes that whole kind of research process as well, which we think is also an important contribution to the literature… to not just immediately do something, but to actually think about the way in which it might function and just really to understand from students what they tell us they need.

John: Your initial pilot used a light-touch intervention, could you describe that intervention?

Michal: So initially, in the pilot, what we did was a slightly underpowered pilot in the sense that we took students who didn’t complete their first homework assignment in a classroom where you have to complete a set number of homework assignments. And you could miss one, but historically, we knew that students who missed the first one often struggled in the class. And so that’s the point of randomization that we took for the pilot. That is we took those who didn’t submit their first homework assignment and to half of them we sent an email saying, “Hey, we noticed you did and submit your homework assignment. Just to remind you to do well on this course, here are some things that might be helpful.” We also provided some information on what’s coming up and reminding of the office hours and how to seek help. And then we did two others. Importantly, the two other emails provided information that showed the students that we knew how they were doing in the class so far. So after the midterm, and before the final, and we found again, it was underpowered, but we found positive effects among the treatment group. And then that was conditional on some other ex post characteristics that we added to the pilot, but then we decided to launch it at the CSU campus that we worked with at full scale across a random sample of introductory courses.

John: For those listeners who are not familiar with statistics, when you mentioned that this is an underpowered test, could you just explain that in terms that…

Rebecca: …Rebecca can understand? [LAUGHTER]

Michal: Absolutely. So they were underpowered statistically, to detect a statistically significant difference between the treated students, those who got the emails, and those who didn’t. And so for that, we need a large enough sample size of treated versus control students, particularly if we’re going to add other kinds of observations about them, like their gender, or their race, or their prior academic achievement. And so when we say something’s underpowered, we might see the positive effect that is better achievement in terms of the final or in satisfactory progress in the course. But that difference may not be large enough in statistical terms to consider it statistically significant, even if the mean differences are actually in the direction that you expect. So to get that, to be able to actually detect significant effects, you need a large enough sample size. And that’s when we launched into the population that we were specifically focused on, which was the population at this less selective campus.

Rebecca: At that less selective campus when you scaled things up, did you keep the intervention the same? Or did it change? What did that scale up look like?

Michal: Great question. So the first thing we did was we really focused on introductory courses. This was also piloted in an introductory course. But we wanted to focus on particularly large classes, especially because the information was going to come from the instructor and we were doing a randomized control trial, that is some students are going to get this treatment and others are not. So the class had to be large enough for it to not be weird that some students were getting it and others were not. If you’re in a classroom of 30, that might be strange if you’re talking to someone next to you, and they get this email from an instructor, but basically what we did is we recruited faculty, we randomly chose 30 large undergraduate courses. And then we recruited those faculty and said, “Will you be part of an experiment with us? …and here’s what you need to do.” And the important thing here is that we’re not trying to dramatically change faculty style in the classroom, we all have our own style, the own way we write a syllabus and what we expect from students. What we wanted, we had several key principles. The key principle first was that faculty need to directly communicate with the students showing them that they know who they are. So it very much said in an email, Dear Rebecca, or Dear John. They needed to provide information that was specific to their class, it couldn’t be quite generic. We provided them some templates, but the goal was for them to personalize them and say, “Here’s this upcoming unit, here’s what to look out for, here’s how I would study for it, Here are my office hours, and so it provided information. So the way we changed it is… it was a semester-long courses. and so we requested that faculty sent a minimum of three emails to students, one after the first assignment or exam, sometimes earlier if they didn’t have an exam, and two later, we wanted it within the first three weeks of the semester, one after the midterm, and one before the final. And the important thing is, in the second two emails, those were further personalized to sort of say, again, “Dear Rebecca, I see that you’ve gotten a 72 on your midterm, it’s not too late to improve your grade in this class. Here are the things that you could do. And so it was personalized also just showing that we know how you fared in this class. And so again, the goal was to let faculty, in their own words and in their own course formats, personalize these emails, with the principal being information to students, personalization to students, and help seeking behavior advice.

John: And this process is a personalization, was this done in a mail-merge type format? Because I would think to scale this intervention would be a lot easier if you did do it either using the tools within the LMS or using some type of a mail merge.

Michal: Great question. So again, this was a grant-funded study and where we could provide some support to faculty, some faculty didn’t need additional RA support from us and either knew how to do a mail merge, it really worked with their course management system, like Canvas or Blackboard and found it very efficient to work on the own. Others you may or may not be surprised, did request our help from our graduate students. And we did provide support including one actually helping a faculty member directly write individual emails for students to support. You’ll probably ask me how the faculty feel about this. And I will say we actually asked them how long it took. It didn’t take more than a minute an email and so we do kind of try to guesstimate the investment on the faculty’s time to do this, and it very much varies on their comfort level with the course management system.

Rebecca: In the scaled up version of this study, did you continue only interacting with students who had struggled and missed their first assignment or is that a shift from the pilot to the big study?

Michal: Yeah, thank you for catching that. No, it is a shift. We did open it up. We believed actually, theoretically, our priors were that anyone could benefit from this. So if you were a B or on the cusp, we have lots of researchers suggest students, especially in introductory courses, some students, particularly first gen students might take a B or a C as a signal that they shouldn’t be in a particular major. We really did want to encourage across the achievement distribution for everyone. As to John’s earlier question, as you scale this up, or as people have talked to me since this experiment said, what if I want to do this, but I teach 400 people, you could one year, one term, try it with your lower achievers, another term, try it with those at that C range, or others. And so we did in this initial intervention want to do it across the board.

John: How large was the sample on the scaled up version?

Michal: It was 20 faculty, some were in multiple classes, we had 22 classrooms overall, and roughly 3000 students.

John: Excellent. How large was the estimated effect in the scaled up version?

Michal: First, it’s important to note, as we’re talking about findings, that our findings are really concentrated on first- and second-year students or new students, and who are from underrepresented minority backgrounds, so URM students, and we find that their treatments are about five percentage points more likely to earn an A or a B in the course by comparison to control students. So again, just important to note, we find overall positive effects for the whole of treated students. But they’re only statistically significant for the URM students that we target, that our intervention aimed to focus on.

Rebecca: Was the impact limited to just the classes the students were in, or was there an effect beyond that individual class?

Michal: Yeah, so that’s a great question, and we do find what are called spillover effects. And that is that those same students, those URM students, had a positive effect of being in the treatment group, even in courses that were not part of the experiment. That positive effect was much smaller in magnitude, it was like three quarters the size of the effect of the actual treated course. But still statistically significant at the ,10 level.

Rebecca: it seems so easy, just three emails.

Michal: Yeah, it does. It takes an investment. But yeah, I think it does beg the question, I think, for me, and maybe this is something that you want to talk about a little bit later. But we do so many things to introduce first-gen students or to get students in the classroom and again to provide support externally, but we do tend to sort of assume that they’ll just survive or just be okay in the classroom without training faculty on how people might experience their classroom differently. And so, again, we do test for other subgroups. I focus on first-gen, because it’s a concept that is helpful for people to think about students who don’t have at home, at least, people to tell them what to expect in the college environment and how you might go to someone’s faculty office hours, and they’re not there, or they’re there, but they’re sort of like, “Yes, did you have a question about the material?” Whereas to know, you could go just to review the material or you don’t have to have a question, you can just show up, things like that. Being able to feel comfortable asking questions in class and others who don’t or just step in at the end of class and sort of say, I found this part of the lecture really complicated, will you be reviewing it again the next day? So things of that nature. And so I don’t think these emails did those, but they sort of remind us that there are things that faculty can do to remind students that they see them, that they see that they’re in a classroom, and that they know that they may be enjoying parts or struggling in parts, and that there are some actions that they can take to be more successful in their classroom.

John: Did the effect size vary with class size? Was there a more substantive effect in larger classes than smaller classes? Because I would think it might be easier for students to feel more lost in a larger section, especially as a first-gen student?

Michal: It’s a great question. We aren’t able to test that, our sizes were all pretty similar, and we didn’t have enough. We actually chose the largest classes that exist in this campus, which don’t get much bigger than 150. I think it is worth testing. Absolutely. I can tell you from the pilot, that, in particular, that was a class of 400 students, and both for the pilot and for the full scale up the types of emails we got, that faculty got back. So that’s one thing we could talk about is how did students respond more generally, and many students emailed instructors back, and in particular, in the pilot, but also in the scale up, we got those emails back from the faculty who were in our experiment, and they very much appreciated the email and said, “No faculty members ever emailed me before,” or “especially in a class of this size,” or “I so appreciate the email. I’m working really hard.” So the first and sort of overwhelming finding from these emails is just gratitude from students that a faculty member emailed them, particularly we noted that in the pilot when the class was quite so large.

Rebecca: Yeah, I can imagine that just something that feels personalized, whether or not it’s super personalized or not, just feels personalized, really helps students feel seen.

Michal: Exactly that. That’s right. I should also say in addition to just grades and you asked me about graduation outcomes, we also included a survey, both in our pilot and in the scale up, to try to get at some of the mechanisms and in particular, we asked questions like, “Did you feel this instructor supports your learning? Did you feel you could reach out to your instructor?” And we do find consistent with our intervention that students in the treatment group reported more positive outcomes on these dimensions.

Rebecca: That’s so fascinating, because it’s so easy. Like it just isn’t that complicated.

John: In your study, you also examined how this effect persists over time, which certainly relates to the graduation thing. What did you find in terms of the persistence of this effect over time,

Michal: We did look at long-run treatment effects. That is, we waited to see what happened several years later, we presented on this paper in the shorter term outcomes. And we tracked these students and worked with this institution closely, we really wanted to know did it affect the outcome we care most about, which is graduation? In other words, we care about student success in a particular classroom, and maybe they’re slightly better grade, or not dropping out of the course. But we really care about their longer term outcome of finishing. Again, for this specific group of interest. We note that the treatment results in that 7.3 percentage point increase in persistence, one semester later, and then a higher four percentage points difference in graduation. So we do find positive effects on the likelihood of graduation.

Rebecca: We’ve talked a bit about the impact on students and how students have responded, how did faculty respond to participating in this intervention?

Michal: First, I’ll just say, again, we had to recruit faculty to do this. And so we do track the faculty who said no, and we did as much reconnaissance work, if you will, to understand that we did need to self select faculty, keeping in mind that if we did self select faculty who just had a proclivity to help students or being this intervention, if anything, we perhaps under reported some of our findings, but we do as much effort to understand how representative our faculty are, which we did determine they are, and in the paper, there’s some details about that. And they represent faculty from a real diversity of disciplines, from music to engineering, across the board, humanities and social sciences, we had the whole range, we had the range of faculty. So you know, “I do this a little bit sometimes. And the two of you are, I would have said, I’d do the same with a student who doesn’t show up or doesn’t complete to sort of track them down. But I’ve never done it systematically. And I’ve always wondered if it even matters.” We had everything from that kind of faculty member to a faculty member that’s like, “Well, I’ll do this, but I don’t think it matters. I mean, at the end of the day, the students who want to put in the work put in the work.” And so we had a whole range in their attitudes. We did offer a stipend to do this, because we did believe it takes time. And we wanted to sort of show that faculty who do have a lot of demands, especially at teaching institutions, that this was going to take some time. And so we haven’t done it again without an incentive, or with an institutional incentive, that’s part of like performance evaluations or something. So that’s yet another thing that in terms of where to take this in terms of where institutions might take this, for our perspective, it was externally funded, they were only accountable to us in their efforts to do this. And so we talked to them multiple times in the term, again, some were in both waves of the study, because we did it over two terms. And then we surveyed them at the end and really got some details from them. Some of this is in the paper around how they felt. And I would say most expressed similar to what you expressed, Rebecca, which is like “Wow, this some emails and I made this difference, especially in underrepresented minority students lives and in their classroom, and it felt really good about the impact.” …keeping in mind, we talked to them also, before we knew the results. And so at that point, they just were sort of documenting how much time it took to do the emails, and what kind of emails they got in response from students and most felt, I would say, humbled by the thank yous that something so small, like an email, got so much gratitude back from students. We did have some faculty that sort of said, this takes a lot of time, and I’m not sure it’s much of a help. In our last survey with faculty, we actually provided them the full scale results and said, “Here’s, by the way, what we learned from the study,” and then asked them to respond or to reflect on that. And many said, “Wow,” like, again, similar to your reaction, “a few emails could make such a big difference, I will be sure to continue.” We ask them directly if they will and we report this in the paper. Most do say that now that they know the positive impacts of this study, they’re likely to continue with these emails.

Rebecca: I imagine the workload for a faculty member isn’t necessarily in drafting those initial emails, but maybe the responses to the emails [LAUGHTER] you might receive back.

Michal: It’s a good question. I don’t know how many continued the conversation once an email was sent, the standard emails, part of the experiment and the student wrote back and said, “Thank you so much.” I’m not sure they continued. We did a lot of qualitative coding, which we don’t report in the paper, we report some but then we did a lot more internally. And there definitely were a lot of hardships described among students who did reply, the extent to which faculty replied with those hardships offering extensions or any other kind of augmentation to their requirements, we don’t know.

John: In an introductory microeconomics course I use the Lumen Learning Waymaker package which actually does automate emails to students based on their performance on weekly quizzes and so forth. And even though the students in the class realized it was automated, they’d still write back to me and I’d respond to it. And they’d often apologize, saying, “I had a bad week and I know I need to work harder.” But it did often start a dialogue that might not have occurred otherwise. And I’ve often wondered how large that effect was, but because it’s automated for the whole class, it’s hard to measure the differential effect of that. So it’s nice to see this result, that that type of approach works.

Michal: Yeah, I think that what you’re describing is exactly right, this sort of feedback. Our whole intervention is built around theories, not just from kind of behavioral economics or nudging or information source, but on the education literature on feedback, and the important role of feedback, and the timing of it. And it’s most useful if it’s not just performative like feedback, like your grade on an assessment like a “C,” but that actually gives you more information about how you’re doing or what to do to improve. And so this kind of thing you’re describing John is exactly right. I think we know that more touch points with students through assessments, as opposed to all hanging on a midterm and final also support students to get more feedback about how they’re doing.

Rebecca:I think sometimes students know that they’re struggling, you get a grade back, you know, if you’re doing well or not, but I think a lot of students need more coaching around what to do to improve or to better understand how the grade is calculated, to just take the time and attention. It’s there. It’s in the syllabus. But sometimes they don’t realize what they should prioritize. And including some of that kind of messaging makes a lot of sense. I know that when I’ve done that with my students, they’ve been really appreciative because they didn’t realize that they were putting all their energy into something that didn’t really matter as much.

Michal: Yeah, that’s right. And I think coaching is a great choice of words around what to do with it. I also think many students come to our universities with really uneven or unequal preparation for those courses. And so I think a lot about students who came from a high school where they took an AP course in economics or chemistry that might as well have been a college level course. Many of these questions are great on a grade curve. And so that C might be an excellent grade for them, given the type of preparation that they had, but they don’t know that necessarily, and they might, to them, signal that maybe this isn’t the right major for them. And so I also think coaching around what to do with the grade when you’re kind of passionate about a subject and not to give up on yourself too quickly. Many are juggling jobs, we know for some, it’s their only work is to get through this term, and others are doing this while working and taking care of family members or whatever. And so that grade that they got often conveys information that we as faculty don’t necessarily know anything about how they’re interpreting,

John: I would think just a signal, as in the title of your paper, “my professor cares” might create a sense of connection and belonging that might otherwise be missing for someone who is a first-gen student who might not feel that they belong in the institution.

Michal: Yes, I agree. And I think there are more and more studies coming out, particularly in social psychology, but elsewhere about the importance of belonging. We know from the K-12 literature that it’s having a teacher who cares about you matters, actually. And again, nothing dramatic happens once you get to college. But we assume it’s completely different, where in fact, I think having an adult or particularly your instructor care, you feel a sense that that instructor cares about your learning, or how you’re doing in their class, irrespective of the grade per se, just that you’re making progress in the class or feeling comfortable in the class, I think is really important. And I think it’s hard to test. And most of the belonging literature has been on survey type research, “I feel like I belong here.” And it’s not as much in the classroom, although there’s increasingly more studies about belonging in the college classroom beyond just a university at large.

John: A while back, we interviewed Peter Arcidiacono, on a paper that looked at the impact of differential grading between STEM and non-STEM courses. One of the things you just said reminded me a little bit of that, because one of the things that was noted in that paper is that many of the people, particularly female and underrepresented minority students who switch their majors out of the STEM fields had some of the higher grades in the class, but it was below their expectations. And I’m wondering if this type of intervention might have an effect of letting them know that that performance in that discipline may not be all that bad. Since we’re probably not going to eliminate grading differentials between STEM and non-STEM disciplines, perhaps some type of personal communication might help preserve some of those connections so we don’t lose as many people in the STEM fields where the returns to education are the highest.

Michal: Absolutely, I think well said. That’s exactly right. And I think that is among the reasons we wanted to get across the grade distribution, not just those who are really, really struggling. And also because we do think that students might give too much meaning from a signal of a grade early on in their academic pursuits where they can get through a certain amount of courses and then maybe where the fun stuff of their major where they really see that utility of a particular course for a future career path matters. And so I think that’s right and I do think Peter’s work and other people’s work looking at the impacts of particular kind of grades and grading distributions or signals of grades, I think, are really important. I think that’s an area that’s blossoming in economics and in other fields to sort of better understand heterogeneity or differences between subgroups around how particular information like an assessment, grade or a test score.

John: One of the things we’ve been seeing in a lot of studies is that many changes in education, such as using more active learning techniques, providing more course structure, benefits all students but disproportionately benefits those students who are historically underrepresented. And it seems like this study just provides more evidence of that, that good teaching practices benefits everyone, but especially benefits the people who are most at risk in higher ed. And those are often the people who can get the greatest benefit by persisting in higher ed.

Michal: Yes, I think that’s exactly right, if we’re really going to address disparities in college outcomes, and I think one really important source to go to is the importance of information gaps. And that would be particularly for first-generation students, for students of color, but also for students who come from unequal K-12 backgrounds. Colleges and universities often know and often are recipients of systematically particular high schools in their states, especially public flagships, community colleges, others, and so they are aware, they offer relationships with those K-12 high schools that are feeders to their institutions. And that is an important source of information that they can provide to high school students as they enter college for a kind of a warmer handoff, if you will, but that also faculty teaching introductory courses can provide. And so I think, again, if our goal is to address inequalities that we see in college outcomes, then I think information, particularly for those for whom their information gaps, is particularly key. Students want to be seen and not necessarily yes, there’s the anonymity of a large lecture hall, that maybe don’t want to be called on. But that doesn’t mean you don’t want to know that your faculty member sees you and knows whether you’re doing well or struggling, or how you feel about the class or how to succeed on the next exam or in the next course in that sequence.

Rebecca: So if there’s other faculty who think, “Hmm, three emails, that seems easy,” What recommendations would you make to those faculty?

Michal: Yeah, I mean, I think what the first recommendation I would make is to try it, to do it. I think thinking about how you communicate in your syllabus that about your forms of communication are important. So if you’re going to do an email, I think one thing that we would have loved to test and if we were to continue further is the format for the information. And so I think letting also students know that you want to hear from them over email or through other means, I think is useful. So first, deciding on the kind of medium like how you’re going to communicate this, I think email makes sense. When faculty start texting students, maybe we’d move to a text them information. But that’s not the case for most of us, so it’s through the course management system or email. I would say focus on again, what we know from the literature on feedback is that for it to be as specific as possible about what students can do with this information, and so that is looking at your syllabus closely, knowing… we typically do as faculty… where students trip up in the material, what’s complex, what’s up ahead. And so giving that kind of feedback as well about how to prepare for the next assignment or exam, what has tripped people up in the past, what you know, might work for them is really helpful. Again, other research has suggested the importance of going to office hours might matter. But that means you need it to show up, you have to think about how you structure your office hours. Incidentally, we did try to track that… quite hard to do whether it actually promoted more office hours in the pilot, we believe it did promote more office hour usage. More broadly, it’s something we’d like to test, the actual help-seeking behavior of students. So I would say faculty should do it if you’re teaching a 400-person class and you can’t imagine doing this for 200 students or even 100 students, maybe start with students who you see as struggling based on that first assignment, as we did in the pilot and see what you can learn from that… maybe do as John suggests, which is kind of get savvy on a mail merge and think about ways to do this so it’s more efficiently done, so that you can reach as many students as possible.

John: We just switched recently to Desire to Learn’s Brightspace platform, and it does have intelligent agents and it does have replace strings so you can automate an email conditioned on the grade on either your overall course grade or on a specific item. And if you do it on overall course grade, (which I just set up, by the way, for my summer class last week), students get an email saying, “I see that you’re struggling, there are some things you might want to try.” It would be nice if I could put in their grade without having to go to mail merge, but I don’t think that would be possible. And I’d like to scale this up. In any case, I’d encourage them to contact me during my office hours or to make an appointment to talk to me. The first iteration of that went out last week. None of the students responded, but it’s a small summer class. So I’m curious to see how this might work. And your paper helped encourage me to try this. I had other reminders out there, but this was one that I thought might be useful, using a specific grade trigger.

Michal: That’s great.

John: We always end with the question. What’s next?

Michal: Well, I think our lab and Scott Carrell and I continue to do this work, and in particular, we’re also spending a lot of our time these days doing work at community colleges in California, which serves one in four community college students nationwide. So also persistence outcomes are quite weak at community colleges, historically, and we’ve seen real declines in enrollment at community colleges since the pandemic. And so we are definitely doing some work at community colleges. We continue to track and follow graduation rates, particularly inequality in graduation rates at CSU. And we’d love to launch another intervention. So stay tuned on that. I can tell you, we are quite committed to understanding the college classroom beyond college settings more generally, and so hope that the college classroom continues to be a source of important information for the field about how to better support student success,

John: You’re doing some really wonderful research. And it’s really nice to see some of the attention that this got because your article has been mentioned in The Chronicle. It’s been mentioned in Inside Higher Ed, and I’ve seen people tweeting about it ever since it came out. It’s good to see this research becoming popularized.

Michal: Well, we appreciate it, especially since Scott and I did not succeed on that front, that is becoming those people who do good social media. Other people are better at that than I am. And I’m always a little troubled when I talk to more junior faculty around “Do you need to do all that?” …and 10 years ago or so I would say “No, just do good work and it doesn’t matter.” And now I confess the sort of buzz that some people are better able to develop around their findings in their papers seems to matter and so it’s really nice when it happens to you because we didn’t do it ourselves. [LAUGHTER] So I appreciate my friends Sue Dynarski and others who’ve done a really nice job promoting this paper.

Rebecca: Well, thank you so much for joining us and for sharing your work with our audience.

Michal: My pleasure. Thanks for having me.

John: And we will include a link to your study in the show notes and we encourage our audience to read it.

Michal: Wonderful.

John: Thank you.

Michal: Thank you so much. I really appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

298. Sustainable Professional Development

The rapid changes in teaching practices and teaching modalities during the last few years have provided challenges for both academics and the professional developers that support them. In this episode, Judie Littlejohn, Jessamyn Neuhaus, and Chris Price join us to discuss ways of moving toward more sustainable models of professional development.

Judie is the Director of Online Learning, an Instructional Designer, and an adjunct instructor of history at Genesee Community College. She is also the Chair of the SUNY Faculty Advisory Council on Teaching and Technology, or FACT2. Jessamyn is the Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at SUNY Plattsburgh. She is the author of Geeky Pedagogy: a Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers. She is the editor of Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. Chris is the Academic Programs Manager for the SUNY Center for Professional Development. He coordinates and leads system-wide professional development in the SUNY system. Before moving to this position, Chris served as the Director of the Center for Excellence in Learning and Teaching at SUNY Brockport. Chris is also a member of the POD Core Committee. POD is the Professional and Organizational Development Network serving those who work in educational development.

Show Notes

Transcript

John: The rapid changes in teaching practices and teaching modalities during the last few years have provided challenges for both academics and the professional developers that support them. In this episode, we discuss ways of moving toward more sustainable models of professional development.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Welcome to this live podcast recording session at the SUNY Conference on Instruction and Technology. The topic of this panel session is “sustainable professional development.” Our panelists are: Judie Littlejohn, Jessamyn Neuhaus, and Chris Price.

Rebecca: Judie is the Director of Online Learning, an Instructional Designer, and an adjunct instructor of history at Genesee Community College. She is also the Chair of the SUNY Faculty Advisory Council on Teaching and Technology, or FACT2.

John: Jessamyn is the Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at SUNY Plattsburgh. She is the author of Geeky Pedagogy: a Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers. She is the editor of Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. (Jessamyn will be joining us as soon as she finishes a presentation in a nearby room.)

Rebecca: Chris Price is the Academic Programs Manager for the SUNY Center for Professional Development. He coordinates and leads system-wide professional development in the SUNY system. Before moving to this position, Chris served as the Director of the Center for Excellence in Learning and Teaching at SUNY Brockport. Chris is also a member of the POD Core Committee. POD is the Professional and Organizational Development Network serving those who work in educational development.

John: Judie, Jessamyn, and Chris have all been extremely active in helping faculty navigate the challenges of the last few years. Welcome, Judie, Jessamyn, and Chris.

Rebecca: This wouldn’t be a complete episode of Tea for Teaching if we didn’t ask about tea, so, our teas today are: … Chris, are you drinking tea?

Chris: English afternoon.

Rebecca: Well, you’ve made the correct choice.How about you, Judie?

Judie: And I have Darjeeling, which is my current favorite.

John: And I have a Tea Forte black currant tea today.

Rebecca: And I have Irish breakfast.

John: Jessamyn, you’ve arrived just in time to be asked about your tea.

Jessamyn: Well, I specifically arranged to have to be somewhere else when I was supposed to be here so as to demonstrate in this meta way, unsustainable faculty development [LAUGHTER] where you’re in two places at the same time.

John: During the past three years, faculty have engaged in professional development at unprecedented levels in response to the global pandemic, and here in SUNY, with the SUNY-wide transition to a new learning management system. From what you have observed in your roles, how has this changed instructional practices? We’ll start with Chris.

Chris: Yeah, so prior to the pandemic, online learning was was more optional. You were either an online instructor or a college had online instruction as part of its mission, but it was a choice. And so after the pandemic, the whole “Should I do online?” just went away and it became “How am I going to do it?” or “How are we as a campus going to move things online? And so the validity of online instruction, that question, just kind of went away. We know, though, that during a pandemic, it wasn’t the ideal time to transition to online instruction, it was just everybody doing it all at once by the seat of their pants, and instructional designers and everybody that supports online instruction were definitely stressed out. So it wasn’t an ideal situation. But I think where it’s left us now, is that we don’t have to have those debates anymore about whether or not online is a valid mode of instruction or not. And so it’s allowed us to take a step back and be a little bit more creative with how we incorporate online instruction into our courses, our programs, our colleges and universities. And so, for example, if you think about HyFlex instruction, before the pandemic, I had never heard of HyFlex instruction, and it really kind of has emerged since then as being something that many colleges and universities are considering as an integral part of their instructional plans. If you would have told me before the pandemic that folks were teaching courses asynchronously online, online synchronously via videoconference, but also face to face, I would have really questioned whether or not that was even possible, and now campuses are doing it all over the place. We just presented today about a Tools for HyFlex courses project that we’ve been looking at, investigating the different types of video conferencing tools that faculty are using in their courses. And so, just to sum up, yes, the pandemic definitely, a lot of residual trauma that folks have from it, but I do think that overall that now the acceptance of online as a mode of instruction has definitely been a positive.

Rebecca: How about you, Judie?

Judie: I do agree with what Chris said, we’ve seen a huge growth in HyFlex courses on our campus as well, and the exponential use of Zoom. But I think these changes are not finished. At the beginning, just before the pandemic really sent us all home, we started training everybody in Zoom on our campus. We were fortunate that that final week, we were on spring break, so we were able to literally herd everybody into computer labs. And each day, we were allowed fewer chairs in the lab as people had to start to spread out a bit. At that time, the focus really was on Zoom, and on using our LMS for the faculty who weren’t using it yet. But then, I think, as people became more comfortable with Zoom, and also we knew we were changing LMSs, the focus really shifted to the LMS training. On our campus, we don’t have the dedicated lab space and personnel to teach all of our faculty a new LMS at the same time, so we took advantage of all the SUNY trainings that were offered. We were in the first cohort onboarding to the system-wide Brightspace digital learning environment, and a lot of our faculty attended every virtual workshop that SUNY offered, and thank goodness that we had those opportunities for the faculty because we could not have recreated them all ourselves. So there was a huge growth, I think, in faculty participating in webinars and trainings offered by SUNY. And it all translates, I believe, into course quality. I think, as far as instructional practices go, the students are having better learning experiences because of the way the content is arranged in the new LMS. But along with that, though, I feel like most of the faculty are more open to engaging with students in Zoom, even if they are in 100% asynchronous courses. They have more virtual availability, I’ll say, than they would have had previously in their face-to-face office hours. So those are the two big things, the course quality, and the virtual availability, that have really increased through all this.

Rebecca: How about you, Jessamyn?

Jessamyn: I guess I’m thinking of something a little less tangible than the other two, which I totally agree with what was just said. I would say the awareness of students’ life circumstances, and how life circumstances can impact learning. I remember very clearly, my first semester after the emergency pivot and a faculty member saying to me, like it was just percolating through his head, and he said, “But what if my student doesn’t have access to wi-fi?” …like he was thinking it through. I saw it happening and the change happening, really thinking about how life circumstances impact student’s ability to engage in the courses. And I would say that it was a good shift, that awareness, but it’s also been very overwhelming. Because once you start down that road, it piles up fast, all the ways, and all the challenges that individual students might be facing.

Rebecca: The tremendous pressures of the pandemic broke down silos between faculty, departments, disciplines, and institutions to initially support the transition to remote teaching and to support the increased interest in inclusive teaching practices. How do we sustain this culture of sharing and prevent the silos from being rebuilt? Judie?

Judie: First, I think the way the silos were breaking down just as SUNY was working on new, updated general education requirements, is really helpful as far as sustaining this, because, with the DEISJ requirements, which are Diversity, Equity, Inclusion and Social Justice, they help us maintain this level of empathy and concern. I don’t want it to sound like the faculty didn’t care about the students before, but I think the level of empathy really increased throughout COVID and the new general educational requirements will help to maintain that. Faculty will continue discussing how they are meeting all these updated requirements in our current programs, and they’re sharing ways to incorporate the DEISJ requirements into different fields that may not have overtly considered those topics previously. So, yes, I think it’s fortuitous timing with SUNY’s shifts in DEISJ requirements, as far as keeping those silo walls broken down so that people can continue having conversations and sharing.

Rebecca: So what I was observing when I was hearing both Jessamyn and Chris is that in professional development we should be implementing the good pedagogy that we’re trying to have our faculty also implement in the classroom.

John: So faculty have become more aware of the challenges that our students face in terms of mental health and wellness, access, and family and community resources, resulting in more interest in inclusive teaching practices. How do we sustain interest in these issues while faculty, staff, and students all feel overwhelmed, and overworked?

Rebecca: You don’t feel that, do you John? [LAUGHTER]

Jessamyn: Well, I would never underestimate the power of acknowledging the situation, acknowledging the reality. So I started both of my sessions by saying, “I know this is a tough time, I know that you are tired, I know that we are all tired.” I also have been taking care to frame and remind people that I would never recommend anything to you that’s not going to make your teaching life, working life, better, I would never tell you to do something just for the sake of checking a box or because I think it’s the right way. But this will make your life easier, your working life better. And finally, I’m really leaning into cut down on content, as much as you possibly can. That’s sort of an underlying advice I’m giving, is talk less and listen more, for me as well, but for faculty. And it’s not that creating more student-centered course design is less work, per se, but it definitely changes the energy in the room and feeds our teacher souls in a different way than nonstop lecturing does. Those are just a few ideas.

Chris: So, I’m gonna take a little bit of a different track here. One piece of advice that I would give is that you got to figure out ways to let things go, things have to go, we can’t burn the candle at both ends, but that, of course, is easier said than done. I’ve been privileged to be involved with, the last few years, online groups of tenure-track faculty throughout SUNY. These are sessions that are open to anyone in SUNY. They take place every couple of weeks. They’re about an hour. They’re not recorded. They’re in Zoom. But we just call them open mic sessions. And I do it with two colleagues at Empire State University. Alan Mandel and Shantih Clemans came to me and the three of us kind of put this together. And, and one of the things we keep hearing from new tenure-track faculty is that as the issue of service comes up again, and again, and again, they’re not really sure how to navigate that. And I think we know this, and I think the research bears this out, that women, minoritized folks, bear a disproportionate service burden on campuses. And so how do we get around that? We need everybody doing service, our colleges and universities just cease to exist if folks weren’t doing service. So I think what we need to do is do a better job of recognizing and rewarding service and distributing it more equally across campuses. Again, I know, that’s probably easier said than done, but I think that that might alleviate some of that disproportionate burden on many of our faculty. The other thing too, is I think, we need to encourage more of a team-based approach to instruction. So that, there are some things like you got to let go. And you got to say, okay, you know, maybe I don’t need to be tweaking every single thing in my course. And now I know instructional technologists and designers are also overwhelmed. But, I think they would all appreciate it if faculty didn’t come to them in an emergency and they came to them more proactively to figure out how to troubleshoot things in their courses. So those are two just very practical suggestions.

Judie: For me, on my campus, as far as sustaining interest in maintaining that level of empathy with the students when I’m overworked and overstressed… we’re short staffed right now, so I do a lot of things in a day and the day is long, and I also teach, but I teach as an adjunct, so I can’t grade or give feedback during my working hours, and I share this with faculty too. So when I what I do every single day when I get home, I go right to my computer, which sounds counterintuitive, but I just keep my day going and I sit right down and I grade all three classes for whatever the people did in the last 24 hours. So I grade every single day, all three courses, because I’m in that place, right? I’m home from work. This is what our students do. They’re working all day, they’re taking care of their kids, they’re doing whatever. And then they get home and they have to sit down at their asynchronous course, and try to rewatch the video, read the lecture, take the quiz, and do all of that. So I feel more connected with the student in that way. Because I do it every day, there’s not that much to grade. If I have classes of 25 students and they don’t all do their work on the same day, it’s spread out throughout the week. I’m real forgiving of deadlines and things like that, I suggest deadlines to keep them on track to finish on time, but I never penalize late work. And I don’t want to their I don’t want to judge their excuses, either. Because I always feel like, I end up being the judge of the best excuse if somebody needs an extension. So everyone has their extension out front. And I grade their work as it comes in and give them their feedback. And it just helps me keep my empathy that way, I think, because sometimes you’ll look and start to read a response or an essay and think, “oh, here we go… this again,” or you know [LAUGHTER] and then I always think, somehow, in my home space like that, it’s easier for me to take a step back and say, “I don’t know what their day is.” I always say “I’m not wearing their shoes…” Who knows what the students have had to go through throughout the day. And one thing that helped me a lot early on in the pandemic was we have a respiratory care program on our campus, and our respiratory care faculty were right there on the frontlines of COVID, like nobody else. And they’d never taught online. And they didn’t use the LMS. So I was constantly helping them get their materials online and their grading online and their announcements and so forth. And I could see the announcements that these people were writing to their students, and just how powerful they were, and they’d start off with, “Please take care of yourself. Please get some rest. We understand you have to take time…” and just how much concern and true love for their students that they were able to demonstrate in these announcements. And some of them I kept as just examples of how to show empathy towards your students. And I think having that sort of an example for me, as an adjunct, helped me set my own daily practice to be able to keep that empathy for my students. So I think just finding a good example like that and being able to share it, is how we can help keep it going forward.

Rebecca: So along the lines of resources, budgets and resources for professional development are limited, I hear. Where should professional developers place their focus, and their efforts?

Judie: So for me…we did assign these by the way, we don’t just randomly know when to talk… [LAUGHTER] I take a project management approach to a lot of these things, just because I like to keep records and details and know who’s done what and where we stand. So definitely, from that perspective, I start with a needs analysis, and who has what, who needs to know what, and where are the gaps, and how can we fill the gaps? So I don’t think there’s a way to identify that one key thing that all faculty need right now, aside from entire LMS training. So, we are in cohort one, so this summer, everybody’s teaching their courses for the second time. So we’re kind of over that hurdle. I tell them, we climbed the mountain and our sliding down the back side and able to focus more on course improvement than how does the LMS work. But beyond that, I’m still trying to get some of our own training materials that existed in our former LMS. we have to Brightspace-a-size things. So, [LAUGHTER], we have to Brightspace-a-size, for example, the RSI, the regular and substantive interaction training, and bring that up to the right standards in Brightspace, so that I can identify who wasn’t able to complete it before and things like that. But by and large, though, it definitely is a needs analysis, that we can’t throw all of our professional development energy into one bucket. We got to figure out what are all the buckets and who needs who needs a ladle full from each?

Jessamyn: Yeah, I’m gonna piggyback on that because I think campus culture really is key to answering this question. The campus culture around faculty development efforts, really has an impact on what’s going to land and what’s not. I’d also add to that, that we, as professional educational developers, should be keeping track better track of where our time goes, where our efforts go, when we make a yearly action plan. We should also be doing monthly activity reports just for ourselves, and to save us time at the end of the year, compiling everything we did, and that can help us see where our time and energy and hours are going and if it’s unsustainable. So if you’re spending… well, I know I can say 100% certainty… what’s your biggest time suck, audience? I bet it starts with E…

Rebecca: …and ends with mail.

Jessamyn: [LAUGHTER] and ends with mail… and ends with “oh my god, another three hours doing email.” So really sort of self reflecting and reporting on that, I think is important. But the last thing I would say… just I’m going to put a plug in for where professional developers should focus their efforts on inequitable teaching labor, and systemic discrimination facing historically minoritized faculty in the academy, and faculty with… a term I learned… invisible and visible disabilities, because we might have some power to move the needle there. Like, for example, student evaluations of teaching do professional and personal harm to faculty every single semester, or every single trimester. There might be a way that, in our role as professional faculty developers, we can mitigate that, change some policies or provide people with actual actionable strategies for lessening that impact. That’s on the top of my list right now.

Chris: Yeah, I’m gonna pick back up what Jessamyn said, because I think the most important resource a campus has for professional development, are the people who work in professional development on the campus. You may have no budget, but at least there’s someone, and hopefully it’s a full-time person, or someone with a significant course release to do that work on your campus. So, that is a resource, and so doing needs analysis is important. And knowing what you’re doing is important, because it helps figure out what is it that you can do most effectively. The Center for Professional Development, now that I’m there, I do a lot more program planning a lot less working directly with faculty. And we don’t really have a budget, like a lot of the things that we do are things where we find people to deliver programs, and then we charge a fee for the program, so that, you know, we can pay the people who do the do the programs. But we do a lot of other things that are free, and they’re mostly relying upon folks giving their labor for free to do it, because mostly they enjoy doing it. So I mentioned that new faculty open mic series for new tenure-track faculty, and that everyone who’s involved with doing that does it because they’d love to do it. And it really energizes them as well as the participants. And so, I think, having those groups of folks coming together… Yes, people do want resources, they do want curated things… to learn how to use Brightspace, and that’s really important, and I think it’s also important to bring faculty together whenever you can, just to talk about a topic… any kind of opportunity you can have them critically reflecting with one another and doing it in a way that’s meaningfully tied to things that they’re constantly focused on in their teaching. I think that’s low hanging fruit, frankly, just when you get faculty together. It’s helpful if you have a budget for food and that sort of thing, but that’s not always possible. It’s even better if you have stipends to incentivize them to do that, but that’s usually not the case. And so, I think that that, mostly when you bring folks together and talk about things that really matter to them, usually has good results and doesn’t usually cost much.

John: On campuses, there has always been some differences in the degree to which faculty participate in professional development: some faculty will go to every possible workshop or training program, others you only see if there happens to be a global pandemic and they need to learn how to use Zoom or their gradebook in the LMS. What are some of the major barriers to professional development and to faculty engagement with professional development?

Chris: It’s just not valued, recognized, and rewarded enough, right? And let’s face it… not as much on community college campuses. I have an example I’m gonna talk about in a minute, on one of the community colleges where it is recognized, valued, and rewarded, and what a significant impact that’s having on that campus. But, and I’m not going to call any campuses out, but on many of the four-year campuses, it’s kind of folks who do it, actively do it because they feel like they need to do it and they do it because they care about their students. Not that the others don’t care, but that’s front of mind for them, they really know that they need to be continually improving to reach their students, to engage their students, especially those who are hard to engage. And, they they do it on their own time, and they burn the candle at both ends, because they have to do the other things that are recognized and rewarded. So that’s the first thing. One of the things when I first thought about with the last question is rather than just strategies for developers to do that don’t cost much, we’re advocates as well. Jessamyn, you alluded to this, I think we have to be advocates with administration, with faculty leaders, for the value of professional development, professional learning, and you need the folks at the very top level to buy into it. So the campus I was thinking of, Rockland Community College, has a Provost Faculty Scholars Program, where they’re incentivizing their faculty to go through the Lumen Circles program, which SUNY invests in and makes freely available to campuses, which are critical reflective practice groups, and they give them a small stipend to do it. They have them report back on campus. And again, the Provost is actively involved in the development of the program, the promotion of the program, and it goes into their tenure and promotion dossiers, and it’s recognizes you are doing the thing that we want you to do. And they have a lot of folks on campus doing it. So it gets back to that critical point of tipping the campus in favor of being a learning organization and not just everybody for themselves.

Rebecca: Juide, how about you?

Judie: The major barriers to faculty engagement with professional development, I’d say, as I am on a community college campus, and we don’t have Teaching and Learning Center, we don’t have any faculty developers full time. So we are a really lean ship, I guess, on our campus. Faculty development happens in pockets in different disciplines or out of my office, basically. And it comes down to time and money. Community colleges are broke right now. Our funding is FTE based, and as we see the enrollment go down, or the population decline in the northeast, or enrollment goes down, every enrollment we lose is money we don’t have. And we can’t offer… I mean, we could give them a sandwich maybe… [LAUGHTER] but we can’t give people money for engaging in anything. And to be totally honest with you, that always baffled me that people expect to be paid to learn something, when usually, we expect our students to pay to learn and here we’re offering training, but you don’t want to do it unless we pay you to learn it. Like it’s wrong to me, it’s an ethical dilemma, maybe for me. But we also on our faculty… I’m certainly not slamming them or saying that they’re all greedy, and they won’t do anything if they don’t have any money for it. They teach five and five, and they are on Senate, and they’re on search committees and they’re club advisors. And nothing happens if they don’t do it. They’re a huge driving force on our campus, and they have just as many responsibilities as all the rest of us and then to try to pile you got to come to this training, and you got to finish this project. And if you do this, and you do that, like there just aren’t the hours in the day. So, untill something traumatic or tremendous happens like COVID or like an LMS change, it’s very hard for them to find the time and when they do find the time like you said, Chris, they’re burning the candle at both ends. So we have to balance burning out faculty versus helping them improve their practice. And we just try to do that by being available all the time and having drop-in hours all the time and troubleshooting where we can and offering consistent programs so they can come when they are able to attend.

Jessamyn: I think one major barrier is actually I’m just going to echo what Chris said about it being acknowledged and valued. And following up on Judie’s point about the financial compensation, I would say and not being rewarded, but not necessarily financial, like rewarded in the sense of being acknowledged as a vital part of one’s job as a college instructor, and therefore, part and parcel of how you’re evaluated and reviewed. And there’s even some kind of professional development that should be considered important campus and department service. So like the department delegate program I mentioned earlier, where one member of a department comes in, does some stuff at the Center for Teaching Excellence and takes it back to their department, that is valuable service, it needs to be in people’s employment review and their tenure file, not as a individual extra thing they did for their own pedagogical fun, but as service contributing to the betterment of the department and the campus and for students. So that’s a sort of top-down barrier to faculty engagement. And I think we also need to own, as faculty developers, something Flower Darby mentioned in her podcast with you that faculty development and advice to people about teaching can’t be a one-size-fits-all framework, that teaching contexts are very different, impacted by social identities and embodied identity. So that’s on us, on the people providing faculty development and advice about teaching, to better acknowledge disparate teaching realities. That’s Roxanna Harlow’s phrase: “disparate teaching realities,” inequities facing our instructors. So I guess that’s like the top-down, bottom-up and then sort of in the middle academic culture generally, academic culture that tends to not value pedagogical learning the same way it values research insights or knowledge production. It should be part and parcel. I mean, we’re a bunch of smarty pants academics, right? Like, we love knowledge, that’s why we’re here, it’s that this should be part of that. This should be respected and seen as part of the work we do, as knowledge creators in this world.

Chris: I just want to address the whole issue of if you’re lucky enough to have a budget or a grant to pay people to do professional development. So I’m totally of two minds on this. On the one hand, I totally agree with Judie, in that there is an ethical issue here, you’re being paid to do something that you should be doing anyway, just as part and parcel of your job. And yet, in many cases, campuses are investing in the resources that you’re taking advantage of. And that just added cost and makes it less sustainable over time, and more expensive, and it creates that culture of I’m not going to do it unless I get paid. And so there is a real, I guess, hazard there when you introduce the piece around paying people to do things that they ought to be doing anyway. But on the other hand, I’ve kind of come to see it as a way to help people be accountable for doing that work and whenever I’ve been in a situation where we’ve paid people to do professional learning, it’s usually not compensation for their time. It’s more just a way to kind of say, “Hey, here’s just a little sweetener to make it more of a priority and to remember, “Oh, yeah, I’m getting that money, I better do the thing I said I was gonna do. When we have the grants that we have through system, we gave everybody a stipend for participating. And, we had almost everybody complete all the surveys and give us all the data that we wanted for it. And so, I think it’s,, used correctly and used very judiciously, it can be effective, but it’s a problem that I think many of us would be lucky to have and most of the campuses that are using it are usually using grant funds and not just their budgetary state funds to do that sort of thing.

Jessamyn: One thing I’ve done for the department delegates which was a big investment of time, so it wasn’t like they came to one workshop, it was this big thing they did all semester. I was able to offer an honorarium. So honorarium sort of in the sense of like, what it really means. It was a miniscule amount of money. But it was symbolic of that I wanted to try to acknowledge in some way and honor the time and effort that they’ve made.

Rebecca: So now we’re gonna move to audience questions. There was a question early on, before we actually started recording about strategies to reach hard to reach students. And so I’m wondering what you all might be able to offer to that question.

Judie: On our campus, we have an early intervention system. And we have different departments that are involved in that, and they work together to create reports that can be disseminated between our success coaches who work with all students and our online specialists who work with what we call the truly distant students. And so if I can’t reach somebody in one of my classes, or another faculty member isn’t getting a response from a student, we can turn to these other resources, and ask other people to try to help us connect with the students to find out what’s going on. Sometimes that helps a lot when you reach out and say,” Hey, can you just check in with me? I want to know how you are.” But some you just never reach and they have different reasons of their own for just stepping away. And unfortunately, that happens sometimes.

Chris: I think this is where the siloing is the worst on campuses, between folks who work in the academic side of the house, and those who work in student supports, student affairs. That’s where I like to see student success initiatives. There’s a recent report by Achieving the Dream and it was about professional learning as essential for any student success initiatives. And those of you don’t know Achieving the Dream, they do a lot of work with Guided Pathways and they work in a lot of community college campuses. And so, that was the gist of the report is that, if you’re going to have a student success initiative, you have to involve the faculty and the faculty have to see their participation in those initiatives as part of their own professional learning, because they’re working with students daily in different capacities, they see students, you know, whole students, not just the student in my section, and those are the things that we said, faculty are doing more and more of now, trying to see their students as whole people and not just when they see them three days a week or whatever.

Jessamyn: Yeah, I really agree with both of you that there has to be a systemic response. And like, Kevin Gannon, I think on your podcast, I heard him say there are things we can pedagogy our way out of, and student success is one of those things that the broader system needs to support as well. I would just add one, one thing, and the only reason it’s on my mind is because I this semester was added to what was called as called the Student Success Consortium on my campus, and at the end of one meeting, they turned to me and said, ”So, if you could do one thing, what should we do?” I was like, “Oh, no pressure,” but I did think of something, and that is: if, on an individual basis, what instructors can ensure that students have numerous on-ramps when they get off track, and that a lot of students disappearing and ghosting is because they flub something, or they’ve made one misstep or they’ve missed one assignment. And if it’s not drilled into their heads: “there’s a way back, there’s a way back, there’s a way back,” then, it’s really easy to spiral into, “I’m done, I failed, I can’t. So I’m just going to avoid it.” So I ended up saying, ”that was one thing I could change was to ensure every class offered students clear, consistent, and repeated reminders about the ways to get back on track when you stumble.

Rebecca: Lots to think about for sure. There is one question that’s come in, and it’s an anonymous question, it says: “To the best of your knowledge, is it against the state rules to lobby vendors to provide professional learning programs to various colleges, or maybe even your take on doing that?”

Chris: So we just did these two projects, Tools for HyFlex courses and Tools for Large Courses, and we also work with Lumen Learning to do the Lumen Circles. So, I’m no expert on procurement, [LAUGHTER] but my understanding is that if whatever is spent is within a certain threshold, it’s acceptable to engage with vendors to try to meet needs that you have, and the POD organization actually, a few years ago, had in EDUCAUSE a guide for faculty development folks, for educational developers, to working with vendors, because when I first started doing this work, I tried to keep them at arm’s length, and like, we could do this in house, and I had this kind of can do attitude. But as I’ve gotten a fewer years under my belt, I have a different attitude now, and that we can certainly work productively with partners who are working in a space where they’re trying to make money off of whatever they’re doing, be it educational technology, or a faculty development kind of thing. And you just have to, as Jassamyn said earlier, do it within the context of your institution and what you need it for. You shouldn’t just think you’re gonna buy something off the shelf, and just think “I’m just gonna plop it in here, it’s just going to work.” I’ll bring Rockland up again, they’re incorporating the Lumen Circles in their campus with already pre-existing programs, they hired somebody to coordinate the faculty going through it. And so it just needs to be mindfully incorporated into the things that you’re doing. And so that applies, like I said, to either faculty development, educational technology, or anything beyond the LMS that you want to bring on your campus to help faculty work with students.

Rebecca: Sounds like a way to extend some sustainability to the work that you all do by leaning on some other resources.

John: We have a second question, which is, “Is there a repository that lists free webinars for faculty development?” and the person submitting said they pass out a list of those through email on their campus. How can you find free webinars?

Judie: Oh, I think lists like the POD listserv. I’ve been to so many AI webinars lately, and I haven’t paid for anything and then once you become familiar with who offers them, you get on their mailing list. For example, I did the Project Management Conference at Arizona State University, and those people are awesome. And they run all kinds of programs, and they’ve never charged anything to me. Oh wait, let me rephrase that. [LAUGHTER] They don’t say we like that, Judie Littlejohn, she’s free. though. They advertise their free offerings externally. And so I take advantage of those. But, once you start to discover these, it feels like they just lead to more and you can see like on LinkedIn, the different social media outlets, and even reading different articles. In Inside Higher Ed, you’ll read articles about different opportunities that are free. And that’s where I find them. I don’t try to assemble them in a list though. That’s an interesting idea. So, every Friday, I send an email out to all our faculty, it’s just our Brightspace update (that’s what I call it), but it’s not always Brightspace related what I list under there, but it generally is. But I will if I find things, especially the SUNY offerings, I always list exactly what faculty are welcome to take advantage of that they don’t have to go through us for, they can do that on their own. A lot of people are very appreciative of those. So it’s helpful, but I do like that idea of trying to put a list together to share or have posted on a site that people could go and look at when they want to.

Jessamyn: I’m gonna add a cautionary note, though, I do think people appreciate resources, but you can also overwhelm them with resources, when people are stressed and cognitive, you know, have a cognitive overload. And here comes Jessamyn like, here’s just five books you should read and two webinars you should attend and come to my workshop next week. I’ll have people that are like, “Uhhhh, guess what, I’m gonna ghost you now [LAUGHTER] and I’m gonna just avoid the whole thing, because it looks too overwhelming.” So, it’s an overused word, but I curate, like I really, really carefully curate before I send out any recommendations. I’ve cut way back, even on providing things through my site to people, because it got overwhelming. Again, campus culture really matters, but I would curate carefully.

Chris: Yeah, we have just at the Center for Professional Development, we do have a faculty development playlist on our YouTube channel. So if you just look up SUNY Center for Professional Development on YouTube, and faculty development playlists, we do have a bunch there, everything that we do gets tagged that way. The POD google group is a good thing. What I usually do when I’m looking for something is I do a Google search for whatever I’m looking for and then I look to see if it’s on a Teaching and Learning Center website. There are some that are really good, Vanderbilt is good. I know that there was maybe some issue of some of those resources being taken down. I hope they’re not. Carnegie Mellon, I know, has a lot of great resources. So there are a few teaching and learning centers that put a lot of time and effort and energy into curating great faculty development resources. So that’s what I would do. That’s what I do. [LAUGHTER]

Jessamyn: I’m gonna throw out another kind of maybe radical idea for the attendees here at CIT. I think webinars can be very valuable, but I also have seen and in my personal experience, building our pedagogical toolkit often happens conversation by conversation, or seeing somebody do something. So at a conference that we’re at right now, a lot of the presentations, sometimes it’s the content, and sometimes it’s just what somebody did while they were presenting the content, or the conversation I had with you… it could be via Zoom or it could be an email, but an interaction that flips a switch, “Oh, I could do that.” It seems like a lot of pedagogical learning, like all learning happens incrementally, it takes a lot of practice and hearing it many times and trying it out. Webinars definitely have a place for sure, especially for Brightspace-a-zation… [LAUGHTER] Brightspace-a-size, yes and LMS stuff for sure. But some of the things that we do in our classroom, it’s less about being buried under content and more about connecting with someone else talking about what’s working, what’s not working, repeat.

Rebecca: Well, thank you so much for wonderful advice throughout our panel today. But we always wrap up by asking: what’s next?

Judie: Well, I can answer that one. Jessamyn said some phrases that I’ve never heard on our campus such as yearly action plan, and monthly activity reports. And I thought, “Oh, my goodness, this is a thing.” So maybe we could, as we’re trying to add more staff into the Online Learning Office this fall, I’d like to try to come up with things like that and have a more predictable approach, perhaps for the, you know, from the faculty side, of what sort of professional development opportunities we might be aiming for. And when they can expect them to happen. I like that idea.

Chris: And I’m always thinking about how to connect folks across SUNY who do this work so that they can share with each other because I do think there are unique things about working within SUNY that would make it so worthwhile for folks in SUNY to connect to our supporting faculty. POD is great. It’s an national and international network, but I do think there’s lots of affordances of just talking occasionally amongst ourselves.

Jessamyn: Well, I knew this question was coming. And I am going to take this opportunity to give the world premiere of my working title for my next book project that I am working on this summer. It’s called Snafu edu:Teaching and learning when things go wrong in the college classroom, and I’m working on a manuscript, it’s for West Virginia University Press in their teaching and learning in higher ed series. And I know nobody in this room, nothing ever goes wrong in your classroom, but for the rest of the people who teach, things go wrong.

Rebecca: Snafu.edu?

Jessamyn: Snafu edu.

Rebecca: I like it. I like

John: Have you grabbed the web address yet?

Jessamyn: I have. [LAUGHTER]

John: Okay.

Jessamyn: I have.

Rebecca: Well, thank you, everyone. We appreciate you participating today.

Jessamyn: Thank you, John. Thank you, Rebecca.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

297. The Road Forward

The opening session of the 2023 SUNY Conference on Instruction and Technology, which took place at SUNY Oswego, included a keynote address in the form of a live podcast interview with Flower Darby. This podcast episode is a recording of this session, which included both a live and a remote audience. Flower is an Associate Director of the Teaching for Learning Center at the  University of Missouri at Columbia. She is the co-author, with James Lang, of Small Teaching Online: Applying Learning Science in Online Classes and a co-author of The Norton Guide to Equity-Minded Teaching.

Show Notes

Transcript

John: The opening session of the 2023 SUNY Conference on Instruction and Technology, which took place at SUNY Oswego included a keynote address in the form of a live podcast interview with Flower Darby. This podcast episode is a recording of this session, which included both a live and a remote audience.

When colleges shut down in the country in mid-March of 2020 we reached out to Flower Darby to provide some guidance for people who were moving to remote instruction, for the most part, for the first time. She joined us on a special episode… in fact, it was the only time we released two episodes in one week… and she provided advice to faculty on emergency remote instruction, resource sharing, and strategies to keep courses going. Today, we are pleased to have Flower back with us to reflect on the impact of the past three years and map a road forward for teaching and the academy.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Flower Darby. Flower is an Associate Director of the Teaching for Learning Center at the University of Missouri at Columbia. She is the co-author, with James Lang, of Small Teaching Online: Applying Learning Science in Online Classes and a co-author of The Norton Guide to Equity-Minded Teaching. Welcome back, Flower.

Flower: Thank you, John. Thank you, Rebecca. Thank you all for having me and for being here and focusing on how we can map that all important road forward. So good to be here with you all.

Rebecca: So it wouldn’t be an episode of tea for teaching if we didn’t ask about our teas. So our teas today are: … Flower, are you drinking tea?

Flower: I have my iced tea in here. And it is a Hawaiian Islands Passionfruit blend.

Rebecca: Nice.

John: And I have a Tea Forte black currant tea today.

Rebecca: And I brought out the Jasmine dragon pearls green tea for today’s episode.

John: When we last talked on March 17 2020, you provided some suggestions on how we could maintain instructional continuity during that two-week shutdown we were going to have to deal with [LAUGHTER] and one of the things you asked in the conversation was what are we going to reflect on looking back when we’re through this immediate crisis situation? So we’re going to turn this question back to you. What sticks in your mind the most from the period of pandemic teaching?

Flower: Yeah, thank you, John, it’s such a big question. And I have reflected very deeply on this over the past three years of ongoing research, conversations with 1000s of educators around the country and around the world. And really, there is one thing that has become crystal clear for me. And that is the centrality and the importance of holistic wellbeing for both ourselves, our faculty, our staff, and our students. I’m excited about how technology use has advanced as a result of the pandemic. But that’s not actually the central focus for me. It is about the importance of our wellbeing, of our students’ holistic wellbeing, and of how we need to center and highlight and forefront social connections and those kinds of relational aspects of teaching and learning with technology in person wherever we are. That extended period of isolation and loss and grief and challenge and distance in education really brought to the front for me the importance of connectedness, and being intentional in how we connect with our students and help them connect with each other as well.

Rebecca: I think you’ve started hinting at this already, Flower, but how has higher ed been transformed by this pandemic experience?

Flower: Yeah, thank you, Rebecca. So I do think that there has been a lot of work done to enhance technology implementations, to provide better support for faculty who are just in the trenches trying to figure this out, whether it’s using new features in Canvas or your learning management system that you may not have used before. Here at the University of Missouri all of our classrooms are now called Zoum rooms. And that’s Zoum, because we’re at Mizzou, which is spelled M-I-Z-Z-O-U. And it’s a challenge for faculty to be in these high-tech teaching spaces and it’s wonderful that the university is making this commitment. But for me again, it is not about the tech, we have seen good tech advancements. We just heard from the Chancellor about some really amazing innovations and that is wonderful because it is going to keep moving us forward. But it is true, I do believe the truth of transformation that is beginning, and that we need to continue, is focusing on this notion of holistic wellbeing and relational teaching and learning.

John: In 202o, one of the things we talked about in that last conversation was the insufficient systemic institutional support for teaching centers, instructional designers, and the effective use of technology in teaching and learning. Have leaders in higher education made long-term sustainable investments in this work.

Flower: Yeah, great question. I think some have, and of course, my answer is going to be, “And we could do more here.” Here again, I just mentioned that example locally here at the University of Missouri with all this investment in the Zoum rooms. What I’m hearing sometimes, though, is that the challenge in managing all the bells and whistles, especially if you do have students who are in the room in front of you, and also Zooming in, it is not something that we necessarily prepare college faculty to do in our graduate programs. I’ve had so many countless conversations over the last few years where the fact is, as a professor, I did not necessarily set out to be a tech whiz as well. So I’m seeing encouraging signs and improvements and a greater awareness of the potential for effective use of teaching with technology to enhance inclusive and accessible, equity-minded student success. And I’m going to say we can do more.

Rebecca: As you just mentioned, prior to the pandemic, many faculty and students were resistant to online instruction. And while this resistance has faded, in some cases, one of the concerns that you expressed in our earlier discussion was the physical isolation experienced in online learning and you mentioned this a bit earlier today. Has the pandemic helped us to develop new ways of encouraging that relationship building online?

Flower: I would say yes, and I would say greater awareness and receptivity to the importance of building relationships online. I think that once again, we still have work to do in this area. But I do believe that prior to the pandemic, there was less awareness on the part of faculty and no blame no shame, I would say it’s the way that institutions and graduate programs may or may not prepare faculty for effective online teaching. I would say there was kind of less institutional awareness of the importance of those relationships online. Now we have seen what happens, we felt it, we’ve lived it in our bodies, when we felt disconnected from our students, when we’ve tried to teach to black boxes with a name, or Anna’s iPad on the Zoom screen. We’ve kind of lived out that disconnect and that isolation. And yet, we know there was abundant research to show that we can have really engaging interactions in our online spaces. And we know that it increases access. I’m here with you today from beautiful Columbia, Missouri, because we have this option available, although I would love to be in the room, but we have other folks joining online as well. We see the value, we know the importance of those connections, those relationships. And one thing that I have really focused in on in the last couple of years is being intentional in the way that we create and structure those opportunities for rapport building and to close the distance because we know that it’s not going to happen by accident online, whether asynchronous or synchronous, intentionality is required. And I encourage faculty to rethink how they use their class time, what the activities are in asynchronous modules, maybe even the kinds of assessments that are in our syllabi, and whether we’re offering points for those kinds of activities. Basically, my argument is that we can do more to design for intentional social connections… that I would say would apply in all class modalities.

John: During the pandemic, the inequities that our students face became much more visible. When we were connecting to students who were zooming in from home and they had trouble accessing the one computer they had to share with four or five other people, or when they didn’t have good network access, or when they were struggling to try to work to pay some bills, and so forth, those inequities became really hard to ignore, for faculty. And campuses did a lot to mitigate that, by loaning computers, by loaning hotspots, and providing other resources. But now that we’ve moved back to more on-campus instruction, and with staffing shortages and budget cuts very common in higher ed, do we run the risk of falling back on some of the exclusionary practices that we had practiced in the past?

Flower: I would say yes, we run that risk. But I actually want to take a little tangent here and tell you about a conversation that I had just on Friday. This past Friday, I was at a conference in Portland, Oregon for teachers of accounting, and I had a very heartwarming and encouraging conversation with one individual faculty, and I think it’s highly representative of where our heads are, and more importantly, where our hearts are now. Of course, I was presenting on the importance of social and emotional connections… that’s what I do… and she shared a story…in fact, I have a couple of poignant stories from that event. This particular one said “You know, I used to be the hardline accountant, you follow the rules and you make the deadlines or you get out,” and then she said “until I was watching my students trying to take their exams via Zoom, and I saw one of my students, his little brother was like hitting him on the head with like a paddle while he’s trying to take his exam. And I saw all kinds of other things.” This experience, the pandemic did give us a view, a window into our students’ lived experiences. The other one, just very briefly, another faculty member, a caring, passionate, dedicated, instructor was talking about how she had one student in one particular semester who lost 13 members of her family to COVID. How much community was built as the entire class was caring deeply. So I do believe there is lasting change. The first accountant I was telling you about, she’s like, “I’ll never go back to that hardline approach. I have more empathy for my students now. I see what they’re dealing with to make this happen.” Now, that said, yes, we do run the risk of falling back into exclusionary practices. I’ve been thoughtful, reflective of how we want to go back to the way things have always been, and I get it, I’m back on a physical campus, I spent the entire time of the pandemic myself working remotely, and I hungered for a physical campus with real live embodied people and students on the pedway. And I’m loving this experience. So I get it that we want to come back to our tried and true, our comfort zones, our methods we’ve relied on. And if we slip back into, for example, less than equitable teaching methods like large lectures and high-stakes exams, we are absolutely sliding back into exclusionary practices. So I would encourage us to not waste this crisis, which is not my unique phrase, but I think it’s definitely apt in this time. Let’s keep pushing forward so that we can become more inclusive and equity focused as higher ed as a whole,

Rebecca: It can definitely be easy to slip back into habits, but I know many of us are really committed to that change and the equitable work that you’ve been talking about, Flower. What can we do together to redesign higher education to be more equitable? What do we need to do?

Flower: Yeah, great question, and sometimes I think, huge picture as in, it’s way too big to change, and other times, I really focus on the circle of control. So I definitely think that, if you haven’t seen it before, I think it’s Stephen Covey has these circles, the inner one is control, here’s what I have influence over, then there’s a broader circle, that’s influence, here’s what I have some influence about. And then the third circle is concern, there’s not a lot that I can do. And so I think we can focus primarily on our control circle. I do think, and we make this argument in the newly released Norton Guide to Equity-Minded Teaching, we make the argument that we do need to advocate for systemic change, and we need to do that in community. So we can work towards that. And then there are influences and aspects that we cannot really necessarily change. We can be concerned about them and mindful. So what can we do? Well, here’s what it comes down to for me, and I’ve thought a lot about this. I would argue, as I just mentioned a minute ago, we have to stop doing things the way we always have. So if you think about the history of higher education in the United States, it is based on centuries of tradition in Europe. And it was designed to be available to elite white men. It is exclusionary in its very nature. So we have this opportunity to say we know that our student body is diversifying. And we know that is so important because with diversity comes strength, comes creativity, comes new perspectives, comes better solutions. So let’s stop doing things the way we always have. Let’s stop with those large lectures. And I do sometimes think about things we can’t change, like, untenable work conditions where contingent faculty are going semester to semester, or we’re being asked to teach these large enrollment classes with very little, if any, TA support. Those are things that are challenging that are, for me, are in that circle of concern. But I think a general mindset to think about is, if I’m tempted to slip back into doing things the way I used to do, maybe that’s an opportunity to ask myself, whether there’s a more equity-minded way to do some things, a more inclusive way, a more active way to help students really process and interact with the materials that we’re teaching.

Rebecca: You’ve talked a little bit about burnout, stress, and mental health concerns that continue to challenge our students, faculty, and staff. And we’ve talked about needing to humanize the learning experience through the pandemic. What role… and you already hinted at this, [LAUGHTER] but I’m hoping you can dig into it… what role does holistic wellbeing play into the future of the academy?

Flower: You know, clearly by now, you know, I’m glad to say it needs to play a central role. And I don’t think as an academy, we are quite there yet. We do work and exist in an overworked culture. We absolutely do. And I would say we’re high-achieving individuals. If we choose to be here, we’re passionate, we’re focused, we work hard, but in general, we need to give grace to ourselves to take more opportunity to support our own wellbeing. We need to extend that grace to each other in community… to say “it’s okay, go ahead and take that weekend off. I don’t need the manuscript on Monday. It’s alright.” And then collectively, again, I think the more that we’re having these conversations, that can help. So just recently, my good friend and colleague, Sarah Rose Cavanaugh, had a piece come out in The Chronicle. And the headline was, “They Need Us to Be Well,” and it’s about how, if we want to support student success, if we want to advance equity and become even more inclusive than we already are, we actually have to start with ourselves. And I don’t think that we do this very well in this culture. I have had to work, to learn to give myself permission to take a weekend off. It’s something that we, again, for me, it’s about giving permission, it’s about supporting each other in these decisions as well. And then, broadly speaking, I hope that we are seeing the beginnings of a culture change. Questions about the validity and the feasibility of teaching these large-enrollment sections or teaching online classes with very little attention paid to interactions between people. I think we do have this opening for these conversations, and I’m doing everything I can to advance those conversations.

John: In response to the pandemic, faculty face unprecedented changes in the way in which they were teaching. And since then, we’ve had a number of other changes in our practices. Here at SUNY, we’ve had a transition to a new learning management system that came immediately after these transitions. And I know other campuses happen to be doing the same thing at the same time. But one of the things that’s happened recently is the introduction of new AI tools, such as ChatGPT and image generation tools. And one of the questions a lot of faculty have is how they might be able to accurately assess student learning in the presence of these tools, while also appreciating the affordances that these tools provide. And how will these AI tools transform instructional practices and the future of the Academy?

Flower: Yeah, great question. And sadly, I don’t have a crystal ball, [LAUGHTER]

Rebecca: Dang.

Flower: …however, I do have some thoughts [LAUGHTER] to share and some, again, recent research, there was a survey that I heard of… just yesterday, I saw a summary of it, it was run by the Washington Post. And it was essentially saying that many faculty are not really thinking about ChatGPT generative AI, all these new tools. We’re doing a little bit of the “I don’t want to deal with that quite yet.” And again, that’s an overgeneralization. But that’s what the survey results really did show. So how these tools will impact our work in education? My real answer is: “Yes, they absolutely will,” I have come to conclude that they are like the advent of the Internet. Remember, way back in the early to mid-90s, when it didn’t used to be possible to do an online search for something? I think this tool, this change,is along the same lines of the smartphone. Remember, when we didn’t always have a super powerful computer available to us in our pocket or our bag? For me, this is a seismic change, and it will change the way we do things. Now, that’s scary… and exciting. [LAUGHTER] Our world is changing, and we have to be willing to embrace it. Are we worried about whether students are going to do their own work? Absolutely. Do we know that we need to equip our students to use these tools in order to thrive in their careers? We know that too. Right now we’re in a very unique moment of trying to walk a middle ground here, trying to see what are the opportunities of these tools? How do we help to understand whether our students are actually doing their own work? I don’t have those answers. What I do know is that this is another big, and quite frankly, painful opportunity to think deeply again, about the way we do teaching and learning in higher education. The pandemic was this for me, ChatGPT, and Gen AI is this for me. We can think deeply about the way we do things. We’re gonna have to change some things. And that deep reflection and change process is undeniably painful, undeniably scary, and can be deeply meaningful and rewarding as well.

Rebecca: That’s a little too much seismic activity going on there. [LAUGHTER]

Flower: It is. It’s a tough moment. It’s a really tough moment in higher ed. I want to just be honest about that.

Rebecca: So at this moment, we’d like to move to some audience questions. And we do have a first audience question. And that first question, Flower, is moving forward, which pandemic modifications, or temporary adjustments, should we adopt as best practices to meet the needs of modern learners?

Flower: I have one that comes very easily to mind. And thankfully, it’s not a big huge effort or overhaul to our course design and pedagogy. And that is, let’s check in with our students more often. Let’s check in more frequently. During the pandemic, in an effort to engage those black boxes that were on our Zoom screens, many of us developed new ways of using Zoom polls, of asking quick questions in the chat box, assigning collaborative activities in Google Docs or Padlet, these kinds of things, and the students have unwaveringly told us that they appreciate us checking in more often. So, whether it is a matter of if you are lecturing in person, every 10 to 12 minutes or so ask your students a quick question, quiz them on the concept, or ask them “How are you doing? Are you with me?” …here again to think about the Zoom example, which I know not everybody is really doing as much of, and that’s probably good. But I know one instructor who would use to say, “Are you with me?” and her students knew that they could use the emojis in the chat box… one thumbs up was like, I’m not doing too good. Two thumbs up, pretty good, pretty good. Three thumbs up, I got it. And she told me that if she saw a range of those one thumbs ups coming in, she’d be like, “That’s it. We’re not going any further until we kind of talk this through a little bit more. What are your questions?” That’s the kind of informal checking in with our students that I’m encouraging us to do. Again, this can take the form of an activity that happens during class, a quick poll, it can be a show of hands, I saw a great example, just last week, those of you here in the room and watching the video will see this, for the audio recording, I’ll just describe it. I heard this great example of a biology instructor asking her students a very simple check your understanding question during a large lecture, and she had taught her students that they should answer the question based on a number of responses, the responses were numbered 1, 2, 3, 4, 5. And she said, choose the best answer and just put it right here, on your chest, so that you can hold up the number of fingers right against your chest. And what this does is it doesn’t let anybody have to feel really awkward or insecure about “I’m not sure if I’m holding up the number one, and maybe I’m wrong.” By holding it right here, we’re doing a couple of things. We’re providing safety, we’re building trust, we’re giving students an opportunity to retrieve the information that they have just been taught. And again, you can adapt this approach to anything: How are you doing? Are you with me? Hold up a one if you’re feeling terrible. So this idea of the informal check-ins with students… you can also do online anonymous surveys using Survey Monkey, Google form, whatever is in the learning management system. Ask your students: How are you? What do we need to do more of? The stop, start, continue, survey is a really great model. What should we stop doing? What should we start doing? What should we continue doing? And then very importantly, if you do those kinds of surveys, which are really powerful and equity minded, you want to circle back to your students and say, “Alright, here’s what you told me. No, we’re not going to stop having our weekly homework assignments. Those are important. But here are some changes that I can do.” So informal check ins with your students, it does a lot of different things… key for me is demonstrate pedagogical caring, that we do care about our students. And we want them to be successful. And it helps our students to communicate to us if we need to slow down or provide another explanation, those kinds of things.

Rebecca: So I’m wondering, Flower, if you can talk a little bit more about some of these same kinds of practices, but in an online environment, in an asynchronous online environment, where sometimes it’s a little more difficult to figure out how to adopt some of these practices?

Flower: Yeah, great question, Rebecca. And thank you for kind of bringing us back to online because I have another huge takeaway for me [LAUGHTER] from the pandemic is that online, asynchronous, I think, is one of the most challenging formats to teach in. And I myself still struggle to see the students in my classes as real embodied people. It is so easy to fall into the sense that they are names on a screen, that they are tasks on a to-do list. And we know that our students don’t think we’re real. They tell us repeatedly… well, they don’t tell us they tell each other, they tell the media. My own daughters have said to me online teachers aren’t real. I kid you not. [LAUGHTER] During the pandemic they’re like “My teachers aren’t real people.” So very important, asynchronous while recognizing the limitations, and I don’t actually mean of the format in this case. I mean, the demands on people’s busy lives, because we know students who choose asynchronous online frequently need the flexibility. And maybe they don’t necessarily see the value of all the discussion forums and those kinds of things. So how do we do this relationship building, this increasing interaction? Certainly, I would argue that, as instructors, we need to be communicating with our students more often than not, and that can be announcements, it can be interacting in the discussion boards, not a ton, not dominating, but posting a guiding question here, or “that’s a great point” kind of there. We can be responsive in our assignment feedback, we don’t have to write a tome of comments. But even using an emoji or a quick comment to say, “I see you. I appreciate your work.” Some learning management systems make it very easy to record assignment feedback. Now, all of these, we need to hold in balance with the point I was making earlier about self care and holistic wellbeing for ourselves. I am not saying that we should become 24/7 chatbots, who are always available to our online students. I am saying they need more than what we might do. And we can also foster these connections to support their wellbeing with each other. So here’s one very quick example. I love an activity called “share one photo” and what this is, you can create this in an asynchronous class as an individual assignment, or as a discussion forum getting to know you kind of opportunity and if you do it… well, in either format, you could do this more than once. It could be an ongoing or an every other week, something like this. It’s a great way to intentionally structure social connections and relationship building. And what you do is you ask your students to look in their photo library on their smartphone, don’t go out and take a new photo, look in your photo library, choose one photo that is meaningful to you, write a line or two about why you chose to show that and submit it. And it’s worth points. Because it’s not just that we only focus on the class content, we focus on building relationships to help our students thrive. This can be really powerful, you will get different responses. If it’s an individual assignment, you make it more vulnerable images, if it’s a discussion post, you’ll have opportunities for students to connect with each other, like I was just saying, it doesn’t always have to be just you. But this is a way for students to choose what they want to share. It demonstrates to them that you care about them as people and not just names on a screen. And it can be a really powerful and fun way to see a little bit more about who your students are as people. I would certainly encourage that we do the same. Let’s also share one photo, help our students see us as real people as well.

John: One of the questions that has come in is from someone who is in a nursing education program, and the instructor notes that they use a lot of high-stakes exams and assessments in that. Do you have any suggestions on how they can move away from that? And I’ll just add a little bit to it, given that they do have high- stakes assessments as a criteria for licensing.

Flower: Yeah, great point, I was gonna make the same point there, John, thank you. There are disciplinary considerations to all of the recommendations that people like me come in and make. [LAUGHTER] And if you do have those accreditation exams, then part of your curriculum needs to be preparing your students to be successful on those high-stakes accreditation exams. So for me, a lot of times it’s about keeping things in tension, or in balance. We know that they need practice, they need to develop a comfort level with higher stakes, higher pressure situations. And honestly, I’m thinking about on the job, when you’re dealing with a patient, there could be a healthcare crisis that you need to be able to respond to. So, for me, preparing future nurses to deal with the pressure is part of the learning outcomes. But maybe, while they’re students, maybe we can balance that just a little bit. Maybe it’s not just about those high-stakes exams, maybe we balance out the grading scheme to award more points, as an example, for a weekly written reflection, where students can explain how they’re thinking about the processes that they’re learning about. If we have to, and I’m going to qualify… if we have to, for disciplinary reasons, have those bigger exams, because I’m going to invite us to think about: “Do we have to have those exams in this case?” Yeah, maybe. But maybe what we can do too, in a very equity-minded way is to offer retakes, offer test corrections, and a critical part there is to again structure a way for students to articulate where they went wrong, what they learned through this process. So kind of explain, how do I get this wrong? What did I need to do differently. So for me, it’s about balancing the grading scheme, thinking about equity-minded grading in terms of maybe you could build in the drop one exam, drop your lowest test, there’s a lot more that we write about that in the Norton Guide, which by the way, I want to say is actually available for free as an ebook. And, of course, I don’t have the link right in front of me. But maybe in the podcast notes, you can place the link to finding out more about that book, because it is freely available and has lots more of these kinds of ideas in it.

John: It’s a great resource, and we will share a link to that in the show notes.

Rebecca: We have a question that came in from Kristin Croyle, who is one of our previous guests on Tea for Teaching, and also is our Dean of the College of Liberal Arts and Sciences here at SUNY Oswego. And she asked, “What do you recommend for campus leadership approaches to support student learning and faculty staff wellbeing and what should we institutionally start, stop, and continue doing?”

Flower: Oh, I love it. Thank you so much.

Rebecca: No small question there.

Flower: [LAUGHTER] I know, right? Well, first of all, I’m going to actually focus on faculty. When we support faculty wellbeing it can translate most effectively to student wellbeing and success and equity. But one thing we haven’t really talked about today is how our own identities as instructors impact our day-to-day experience. And that can be a big question. It can be related to identities, our social identities, involving our race, ethnicity, gender, sexual orientation, it can involve lots of things. It can even involve the work… and I did this for literally decades… being an adjunct faculty and teaching just wherever I could, mostly online, so at least I wasn’t a freeway flyer, but recognizing that our individual faculty have a lot of demands, a lot of needs, and one size does not fit all from a campus leadership perspective. So recognizing and elevating the importance of attending to our individual faculty to maybe working with them to adjust teaching loads. There’s a lot in the recent media and literature about how some instructors who hold some marginalized identities end up doing a lot of emotional labor that many other identities may not, in terms of supporting students who are underrepresented, extra demands on their time. So let’s stop treating all faculty the same. Pie in the sky, let’s also stop with these untenable working conditions. But that’s a big one. Let us start paying attention to the individual wellbeing of the individuals who are doing this hard and important work of supporting our students and helping them to learn and grow and graduate and make a better life. And let’s continue having these conversations. This is the way we’re going to enact change.

John: We have a question from Christine Miller. And she asks, “While respecting academic freedom, how do we spread the good news of equitable and inclusive practices to resistant faculty and support these practices with our adjunct faculty?

Flower: Yeah, this is a great question. And the opportunity to work on the Guide to Equity-Minded Teaching gave us as an author team, which I do want to give a shout out to my brilliant co authors Isis Artze-Vega, Bryan Dewsbury, and Mays Imad. It gave us lots of opportunity to wrestle with this. And here’s exactly where we landed. I used a phrase earlier that has served me quite well. And that is to not blame and not shame. Let’s recognize that our colleagues, maybe those whom we support, whether we’re perhaps in an instructional design role, whether we’re in a leadership position, let’s recognize that every person is on their own individual equity journey. And we don’t want to judge somebody for perhaps not being in the place that we are, for being a little bit more resistant. The way that I think about it is that maybe we haven’t given them an invitation to slow down and think about things from a different perspective. Maybe they haven’t had that opportunity to see their accounting student trying to take an exam while their little brother was hitting them on the head. So let’s meet our faculty colleagues where they are. We talk about this with our students, too. Let’s meet them where they are, let’s help them to find a way in to what we’re encouraging them to learn and think about. Let’s not blame. Let’s not shame. Let’s extend grace. Let’s support each other. Let’s ask questions. Let’s tell stories. Because, as I just mentioned, the one with the accounting student and the exam, these are things that get people to think about things differently. So it’s a really important question. I’m asked this a lot. And we think about polarized political situations, you think about legislation that is being enacted around the country or being debated. And yet, of course, I’m gonna say this work is worth doing. It’s all the more worth doing. Let’s be strategic, and let’s be supportive of each other and not get frustrated with somebody who isn’t quite where we want them to be yet. We’re all on a journey.

Rebecca: There’s another question. There’s actually a bunch of questions that we’re not going to get to because they all came at the same time. But there’s another question here that says, “You mentioned the different circles of control for advocating for equity and advocating in communities. How do you seek out or help build those communities on your campus, and then build consensus on what that community can influence on the campus?”

Flower: For me, it’s about being intentional to dedicate time and I will try to be brief in my answers, so hopefully, we can get to a few questions if possible. Let’s be intentional with our own personal time to create those communities and work together. One example that has a long history is a book club. So maybe a group of folks on the community, on the campus, want to choose our new book, or any range of other really great books and set aside time in your semester to connect with other colleagues, working with your centers for teaching excellence, working with your instructional designers. These are ways that we can individually choose within our own circle of control to establish community with our colleagues and support each other in this work.

John: We have an anonymous question, which is: “If you were to create or select an emoji to represent [LAUGHTER] the road forward in higher education, what would it be?”

Flower: Wow, that is a good one. The first image that flashed into my mind is the big mountain [LAUGHTER] with a path going up. And I kind of like that because it can represent a couple of different nuances. It can be: we have a long way to go, we have an uphill battle. But it can also be: we have this amazing opportunity and challenge ahead of us and we can ascend and climb this mountain together. I’m going to leave it at that.

Rebecca: Well, thank you CIT audience members for your questions and engagement. And also Flower, for all your answers to not the easiest questions. But we always wrap up with one last question, as you know, and that is: “What’s next?”

Flower: Yeah, thank you. You may have asked me this question in March of 2020. And I have the same answer. [LAUGHTER] I am resuming work, working hard on a manuscript on effective teaching, applying emotion science to teaching with technology. I was working on that manuscript. This amazing opportunity to join the author team for the Norton guide came along, So I had to pause on the emotion science book, but I am picking it up in earnest, and I think again, it holds a lot of keys for how we can enhance equitable higher education for ourselves and our students.

Rebecca: I know it’s something we’re definitely looking forward to here.

Flower: Thank you.

John: And thank you for joining us. We really appreciate it, and it’s great to talk to you again.

Flower: Yeah, very nice to be here with you all and I hope it’s a wonderful rest of the conference as well.

Rebecca: So let’s give Flower a warm thank you. [APPLAUSE]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

296. ChatGPT Chat

Since its arrival in late November 2022, ChatGPT has been a popular topic of discussion in academic circles. In this episode, Betsy Barre joins us to discuss some of the ways in which generative AI tools such as ChatGPT can benefit faculty and students as well as some strategies that can be used to mitigate academic integrity concerns. Betsy is the Executive Director of the Center for Advancement of Teaching at Wake Forest University. In 2017 she won, with Justin Esarey, the Professional and Organizational Development Network in Higher Education’s Innovation Award for their Course Workload Estimator.

Show Notes

Transcript

John: Since its arrival in late November 2022, ChatGPT has been a popular topic of discussion in academic circles. In this episode, we discuss some of the ways in which generative AI tools such as ChatGPT can benefit faculty and students as well as some strategies that can be used to mitigate academic integrity concerns.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Betsy Barre. Betsy is the Executive Director of the Center for Advancement of Teaching at Wake Forest University. In 2017 she won, with Justin Esarey, the Professional and Organizational Development Network in Higher Education’s Innovation Award for their Course Workload Estimator. Welcome back, Betsy.

Betsy: Thanks. It’s so good to be back.

John: We’re really happy to talk to you again. Today’s teas are… Betsy, are you drinking tea?

Betsy: Yeah, actually, I was really excited. I’ve Chai spice tea. I was really excited when y’all invited me back because I’ve actually made a decision to stop drinking coffee as much as I have in the past. So I thought I’d be into all these exotic teas by the time that we recorded this, but nope, just a boring chai tea for today. But maybe next time when I come back, I’ll have some interesting teas for you.

Rebecca: We’ll make sure we ask you to level up next time, Betsy.

Betsy: Great.

Rebecca: I have a cup of cacao tea with cinnamon.

Betsy: Nice.

John: And I have a pineapple ginger green tea today.

Betsy: You all are inspiring me. I love it.

Rebecca: Did you say pineapple, John?

John: Pineapple.

Rebecca: Is this a new one?

John: No, it’s been in the CELT office for a while. It’s a new can of it, It’s a Republic of Tea tea.

Rebecca: I feel like it’s not one of your usual choices.

John: You said that the last time I had this. [LAUGHTER]

Rebecca: Yes, I just don’t associate this tea with you.

Betsy: You have a block.

John: I think I’ve only had it on the podcast two or three times.

Rebecca: Just a couple. [LAUGHTER] I just don’t remember. Clearly. Okay. [LAUGHTER] We’ll move on. We’ve invited you back, Betsy to talk about ChatGPT. We know you’ve been writing about it, you’ve been speaking about it, and everyone’s concerned about it. [LAUGHTER] But maybe we can start first by talking about ways that faculty might use tools such as this one to be productive in our work.

Betsy: When I discovered ChatGPT, the way that I discovered it, which was back in December, was that I had a colleague who sent a screenshot of asking them to draft a syllabus. And so my first encounter was actually with ChatGPT, doing something that would help teachers. It’s also the case that I’m a teaching center director, so, of course, I’m thinking of these things, but it has certainly shaped what was possible. And it blew my mind what it was capable of doing in a great degree of detail, actually. And then about a month later, I was working on a curriculum project where I was having to draft learning outcomes. And that’s a task that we do in the teaching center a lot, and always getting it precisely right and not really sure what’s the different ways that we can phrase this so that it’s actually measurable. And so I just started playing around with what was its capabilities in terms of learning outcomes, and I saw that it was actually pretty impressive and generative there. And then back then, when there was only GPT3, I kept trying to see if it could do curriculum maps for us. And I really had to force it and really think hard about my prompts to get it to actually map outcomes to courses and curriculum. But then when GPT4 came out, I tried it again. And I thought I was going to have to do it step by step. But this time, I tried with a philosophy curriculum, and I said, I want 15 courses, I want them to have three to five outcomes each, students need to take a certain number of courses, we want them to have each outcome three to five times and just sort of gave broad guidance. And it gave me a full curriculum as well as a map. And it was actually a very good philosophy curriculum. So it came up with the outcomes. It came up with the courses, I was floored, and it was my first request. So there are many other things I think we can use ChatGPT for in terms of our teaching, but the curriculum was really, I think, one of the most complex things that I’ve seen it do.

John: I saw you do that. And so I experimented to have it develop a whole major program, with course descriptions and learning outcomes for the program, as well as for each individual course. And it did a remarkably good job of it.

Betsy: Yeah, I was amazed because I didn’t really give it much of a prompt. And it had within the philosophy major, like comparative philosophy, issues of diversity, environmental philosophy. So it wasn’t the typical things that you would expect in a philosophy major, it was actually quite innovative in some ways. And I appreciated that. From the perspective of a teaching center consulting with administrators and faculty on curriculum, one of the things we often see is that the little blurbs in our handbooks or bulletins for students to see the descriptions of the courses, they’re about 150 words. And often they’re very much teacher centered. So here’s the topic of the course: in this course, you will study this, this, and this. And one of the biggest challenges is how do we turn those into outcomes. And so I actually tried to do that too, is I went through our bulletin and just threw in those 150-word descriptions of the topics, and had them develop three to five outcomes that were measurable. And it did pretty remarkably. And so I think that could be a useful starting place. Again, with a lot of this stuff, you don’t want to just take it as is, but a useful starting place to help our faculty and our curriculum committees brainstorm. And in about a week, we are going to do a course design institute at Wake Forest. We do it every summer, and I’m really eager to have my colleague Kristi Verbeke, and my other colleague, Anita McCauley, experiment with using ChatGPT as part of the process in the course design institute to see if it helps them speed up or get more ideas as they’re generating various aspects of design of their course, not just outcomes, but all the way down the line of the steps of course design.

Rebecca: Sometimes it can be really hard to get started, but as soon as you have a start, you know what you want.

Betsy: That’s right. And one response you might imagine to the fact that ChatGPT can draft learning outcomes is you might imagine someone saying, “Well, that’s a clear sign that it’s pretty easy and meaningless tasks to think to be able to draft the learning outcomes.” But what I have found is that when I, not just my colleagues, but when I have a really concrete learning outcome that’s measurable, it helps me design the course better, like it’s just so much easier to think immediately of an assignment. But when it’s vague, and it’s kind of like, I don’t really have it really clear, in my mind, it’s so much harder to do all the other steps. And so even if we think it’s a somewhat trivial task, having ChatGPT help our colleagues come up with really clear learning outcomes will help speed up everything else, at least that’s my hypothesis and we’re gonna see how that goes this summer.

Rebecca: We’ve played around a little bit of using like those course descriptions that might appear in a catalog and turning it into marketing language, which is very different.

Betsy: Oh, that’s so interesting. And has it worked well?

Rebecca: Yeah, I think it’s definitely a starting place to move it to a different kind of language.

Betsy: So I’m teaching a first-year seminar in the spring, and I’m an ethicist, so I teach a course on sexual ethics. And the last time I taught it, I had a pretty conservative title. And it was interesting, I only had women in the class, cis women in the class, there were no men that had signed up, which had not been the case before when I’ve taught that class. So I actually used it to say, I want to attract a diverse group of 18 year olds, or 20 year olds, first-year students to this class, what are some titles or some quick summaries that I might use, and it was really fun to see some of the ideas it gave me. I ended up mashing a bunch together, again, taking pieces of it as an expert and pulling it together. But it certainly got me thinking in a way that would have taken me much longer if I didn’t have that help.

John: I used ChatGPT to create an ad for the Tea for Teaching podcast just to see how it would work and I posted it on Facebook, and I got quite a few responses from people saying, “I use this all the time in my work.”

Betsy: Yeah.

John: This is a tool that’s out there and that came up really quickly, but it’s still a really early stage of this. And a lot of faculty are really concerned about issues of academic integrity, and so forth. And we can talk a little bit about those. But we have to prepare students for the world in which they’re living. And the world in which they’ll be living is one where AI tools are going to be ubiquitous. So you do a lot of work with ethics. How can we help students learn how to ethically use ChatGPT, in college and beyond?

Betsy: Yeah, I think it’s actually a fabulous question. And one of the things I’ve often said, a lot of folks come to me to talk about ChatGPT in terms of teaching and learning. And of course, I have lots of thoughts about that. But I actually have been particularly consumed with reading about the much bigger questions about what AI means for humanity, to be quite frank. There are really dramatic and important questions that we need to think about. And in fact, I think sometimes what I have seen is sometimes people will think that that’s just hype: “Oh, that AI might take over the world, or that it might have these dramatic effects.” But if you actually talk to people who are experts in artificial intelligence, they’re really worried. And when the experts are really worried, it makes me very worried. So when we think about preparing our students, on the one hand, you can think about it as preparing them to use a tool that they need to use for their career, kind of like, “I need to teach them how to use Excel, or I need to teach them how to do basic productivity tools.” And that’s really important. Don’t get me wrong. In fact, like a lot of students don’t learn how to use Excel, and they don’t learn how to use these productivity tools. I have colleagues that I’m teaching these things to where I’m like, “Oh, you didn’t realize you could use this, it makes your life a lot easier.” But I think the bigger issues are preparing them to think about the potential implications to really understand what the tool is doing and what that means for how we understand human intelligence, how we think about consciousness. I mean, what it means for whether we want to have a world in which there are artificial intelligences that we might have moral obligations to. I mean, all sorts of huge, huge questions. Now, I don’t think all teachers need to address those issues. Just like all teachers probably don’t need to teach the technical stuff. But I certainly think when we are thinking about curriculum, it’s essential that our institutions think about helping our students think critically and philosophically about what artificial intelligence means. And I think perhaps my guess is like some of our students or like, our faculty haven’t played around with it a lot or kind of like, “it’s just another thing like Grammarly, it’s not that big of a deal.” But we have found at Wake Forest that when we invite experts in, so linguists, or computer scientists, or machine learning folks, or ethicists to come and talk about these tools and how they really work. Folks have their eyes opened, and then realize, “Oh, this is a bigger deal than we thought it was and we might need to think about regulation [LAUGHTER] and what comes next.” So policy issues, not just ethics issues as well. So we don’t have an answer except for the fact that we need to be talking about it. I have some ideas myself about what I think regulation should be, et cetera. But I do think our students shouldn’t just be seeing it as a tool to make their lives easier, although it is, it also is important for them to think through the implications for society. And then I guess also as another ethical piece, obviously, is that, as we address the issue of academic honesty, helping our students think about their reasons for choosing to take liberties that they were not authorized to do and thinking about their own character. And that’s going to have to be an approach that is somewhat different than just punishment to help our students behave in ways that we wish them to.

Rebecca: I know that my colleagues and I have had some really interesting conversations around AI related to visual culture and creating visual items, because a lot of the libraries of images are copyright protected. And what does it mean when you’re taking something that has these legal protections and mash them up into something new? And then whose property is it? So they lead to really interesting conversations, and so you start thinking about it as a maker and your work being a part of like a library of something, and then also, when you’re using work that’s created, what does that mean? So one of the things that we’ve been talking about is there’s policy at all levels, like what’s our departmental policy around these things? And what kind of syllabus statements or things might we do to be consistent across courses?

Betsy: Yeah, and I think one of the most important things, and it’s gonna take some time, is for all of us to get clear on what we think our policies or our positions are going to be about what is appropriate and what’s not appropriate. And then once we do, to really communicate that to students, because I think they’re in a place right now where it’s all over the map. And many instructors aren’t actually sharing that with them. And so I think that gets us in a fuzzy situation where students assume “Well, if this professor said this, then this professor would be okay with it.” And often it’s very different. And so how do we at least have a conversation at the beginning of the semester with our students about what we think? And I actually think, as you point out, Rebecca, it’s a learning opportunity too for students to co-construct some of those positions. So let’s talk about the reasons why we might want to not just say it’s a free for all. We can talk about the value of art and the value of our work as artists, and what does it mean to just use somebody else’s work without acknowledging it? And maybe there are ways to acknowledge it. And unfortunately, one of the challenges of these image generators is that we don’t necessarily know what it’s drawing on. And so that’s one interesting regulation is: could there be a way? I mean, I don’t know. It’s tough. So one of the challenges with the science of this stuff is that often those who create it don’t know how it’s working. [LAUGHTER] And they will tell you that, that it’s a black box. And so to be able to get in there and say, “Well, I will reveal it to you.” I think sometimes folks assume they’re not telling us because they want it to be proprietary. But often, they’re not telling us because they don’t actually know how the algorithm was developed or is doing its work. And so that’s a really tricky situation. But when we did a number of series of workshops for our faculty this semester, and one of them was we brought in some experts, and we had some copyright experts and some lawyers that came in and talked about this, and really fascinating questions about copyright in our work that, again, is a great opportunity for students to learn that question in a real live way that they see happening.

John: Going back to the whole issue of copyright, in terms of human history, that whole concept is relatively new. And when artists created new work, they started by copying the work of others, and they added their own twist. And in general, in pretty much all academic disciplines, the work that people are doing now is built on the work that others have done before, this. Is what ChatGPT is doing, in part, just the same type of thing that humans were doing, except instead of spending years learning how to do this, and building on it slowly over centuries, it’s doing it in a few milliseconds.

Betsy: Yeah, and I’m not an expert on arts, and so I’m sure there are lots of experts, and Rebecca, you can jump in here as well. But I would say that there are certainly questions about: Is it harming? That’s the question, often with ethics, we’re asking. Is itt harming anyone to engage in this practice. And even if we don’t know we’re using somebody else’s work, we often are. Our ideas build on one another, etc. But of course, in a capitalist society where artists make money based on their work, there become new questions about how do I preserve my livelihood in this particular context? Now, again, if there was a different context in which we supported our artists, so that they didn’t need to make money off of their work, because we gave them a basic income, there may be a different question involved there. And so actually, I mean, I think the economic questions, so I’m tying you both together here. So economics and art, this is great. The economic questions are really interesting about what does this mean for the future of labor? And how do we think about work in the future? I mean, granted, now, it seems like it’s not going to be immediate, but there might be long-term implications for all of us that we need to rethink as well. So I don’t know, Rebecca, you have thoughts about that?

Rebecca: Yeah. I mean, I think it’s interesting. I mean, John’s pointing really to the printing press is when copyright came about, when there it was easier and less time consuming to make copies of things. And then in 1998 copyright law changed again, because of the ability of making digital files, copies, so easy.

Betsy: Napster. [LAUGHTER]

Rebecca: Yeah, copyright law hasn’t kept up with technology over time. So there’s constantly these conversations about technology and creative work and what is it mean? I come from computer art. So generative art is a thing that we do, and that’s algorithm-based and you would argue that the machine is collaborating, in some ways, you write the algorithm. So I think there’s a trajectory of this has been happening for a long time. But it does raise a lot of interesting questions. And I think it’s really important for our students to grapple with, and really critically think about, and for us to critically think about together. In some ways, it’s nice because it gives us something to have a good constructive conversation around and really sort through it together.

John: And then maybe a less positive note, in terms of the economics behind this, there have been a lot of stories of people taking two or three jobs on and using chatGPT, to do two or three times as much work as they did before. And one of the issues I’ve addressed with my students in my labor economics class is, if we have these tools that can do the work that college graduates used to do, will there still be a demand for college graduates to do these tasks. Most technological change in the past ended up replacing less skilled workers, and provided a really nice return to those who had college degrees. But this type of innovation might very well be hitting a little bit more heavily on college graduates than most previous innovations.

Betsy: Yeah, it’s hard to actually talk about this, because I feel like every week it gets better and better. And so I could say, “Well, currently, here’s the set of skills, if we’re an expert, we can use it to sort of level up a bit.” So as I shared with just the curriculum mapping, I’m able to ask it things, and then because I’m an expert, I’m able to do things with it and ask appropriate prompts that push it right in the direction I want it to go and then I produce this wonderful outcome, ehereas sometimes I tried just to show my faculty to put in questions from like physics or something and I couldn’t really assess whether it was a appropriate answer or not, or how I had to push it. And so there’s part of me that thinks that there will be roles for expertise. But then again, how good will it get? Who knows? Will it eventually out compete us? …which is somewhat of a worry… but I do think that, at least currently, there’s still a role for the expertise to play a role. But you’re right, it’s going to just make it more efficient so that we can do more. And then the question is, if we’re doing more will we need fewer workers? Or will we just be more productive? All sorts of interesting questions there. I will say just a funny little story about this point about computer art and economics is our office is called the Center for the Advancement of Teaching. And our acronym is CAT. [LAUGHTER] So we make lots of jokes about that. And we have a serious logo. But all along we’ve been thinking like, we should have a fun, funny cat logo, like with an actual cat, we just haven’t had the money for it. We have this wonderful designer we work with, she’s amazing, we got a quote, we’re going to do it if we have money left over at the end of the year, but I was just playing around with Midjourney and like, what can it do for me? …and I mean, it’s not as good as hers will be, I don’t think, but it was pretty remarkable, especially since this is just a fun logo, it’s not like our serious logo, that I could just use it instead of paying somebody to do it. So this is the real sort of challenge is that it’s really maybe just the most advanced things that we’re still going to rely on experts for but maybe some of the basic stuff that we would have paid, we’re no longer going to do and what does that mean, economically?

Rebecca: There are some existing sources prior to AI that were like people who didn’t have degrees or didn’t have a background in design, who would whip up something for five bucks. [LAUGHTER] And sometimes it looks like it was whipped up for five bucks.

Betsy: [LAUGHTER] You probably would think that about my Midjourney examples. I’m sure Rebecca, I’m sure. Yeah. It’s so funny. So, this is the other thing too, is the students will eventually get better at this, like googling prompts. I went to this website that was like, “Here are these professional designers who design logos, ask it to do it in that person’s style?” Or “Here’s some language that you can use for the prompt like: vector, flat,” like, well, this sort of thing, or a mascot logo, which I didn’t even know that was a thing. But I guess if I want a cat logo, it’s a mascot logo, learning those things, which I never would have prompted, it actually helped me get something that was a little bit better. But it is fascinating. Yeah. And I think that’s true, in my experience with the tool in general is that the more you use it, the more you learn what it’s capable of. And I do think that a lot of our faculty have not really spent a lot of time experimenting with it for a variety of reasons. They’re busy, et cetera. But I often encourage them to really spend as much time as possible with it to really understand what it’s capable of doing. I was sharing with John, before we started this podcast, that plugins are now possible with ChatGPT. And the plugins just take it to a whole next level beyond even GPT4. And I’m still starting to play around with that. And I think it’s just something that, again, faculty need to be prepared for, because right now they’re saying, “Oh, it can’t cite things, or it can’t search the web.” Well, now it can. And what do we do about that? How do we keep up with it if we aren’t paying attention to it?

Rebecca: I think one of the things that you said earlier and alluded to when you were talking about the logo is needing expert language and expert concepts to be able to curate the prompts. So if students want to use tools like this in a productive way, they then also have to have a certain level of expertise, presumably, to do a good job. If we want to encourage students to be productive and use a productive tool in a productive way, what can we do to coach students? Do we want to coach students in this way?

Betsy: The question about whether we want to coach students is a really interesting one. There are folks, I think, who are anxious that if you teach them how to use it, they’ll use it in inappropriate ways. And my sort of response to that is we’re gonna have to address their desire to violate norms in a different way. That’s a different issue. That’s an issue of character. It’s an issue of ethics, because I think they are likely to do it anyway. Now it’s true, if they don’t really know how to do it, we might find it easier to detect it. But I’m guessing that, in a year, it’ll be harder and harder to detect it, even if they don’t know how to do it. But as of right now, I would say I think it is useful to teach them. I wasn’t teaching the spring, but I am teaching in the fall. And I’m really excited to think about. I’m not going to totally redesign my course, some people have done that, I’m not going to do radical changes, but just to engage in the conversation with them in the ways that it can be used. And I think some of the most important, honestly, are using it to explain material that they didn’t understand or using it to interpret my prompts if they’re weird, or my expectations. So helping the students use it to help them with their learning. So giving me feedback on my work, and I have a whole list of things that I would recommend, which is somewhat different than what we immediately think about when we think about students using ChatGPT. We think about them using it to write their papers, or to start, brainstorm, or give an outline. And all of those things might be great. But I actually think, as a person who’s interested in pedagogy, and particularly in student learning, there are only so many hours in the day, I have so many students, I can’t be with them one on one for 15 hours a week. But if there’s a way in which they can have like a tutor, who’s there with them to say, here’s what I think might be the explanation of that thing you didn’t understand, or let me help you interpret this paragraph and put it in the words for a sixth grader, or I’ll give you an analogy related to sports, if that’s what you know. [LAUGHTER] All of those things are amazing opportunities for our students to accelerate their learning. And that’s what we want. So it is true that these tools can be a threat to learning if students are just using them to write their papers in a literally copy and paste kind of way. But I also think there’s real opportunity to help them accelerate their learning. And again, you have to be careful, because it’s not perfect. And that’s your point about expertise. But I think, frankly, sometimes the advice they get from their friends, or if they go to the internet and Google it or YouTube, they’re not getting great advice either. So it doesn’t have to be perfect to be better than what they’re currently doing, is I guess what I would say? So I think that’s important. And then we could talk if you wanted to about how they might use it as a writing tool. I think it’s trickier there, of how we could ultimately and I’m sure you’ve heard this before in the previous ChatGPT podcast you did, it ultimately depends on your learning goals. What are your goals for your course, how you want to use it, but I do think there are certainly legitimate ways in which we can help students use it to help them learn more.

John: And just following up a little bit on that. I’ve heard of a number of faculty who are encouraging students to use it to create tutorials on specific topics where they may have a weaker background. And that’s certainly a very good potential use of this.

Betsy: And I even have experimented with like, “Okay, so give me feedback on this and then give me a learning plan,” like, “give me an improvement plan,” like, “what should I do? What steps should I take to get better at this skill?” …and it’ll actually give you pretty good plans. And it also can help them with time management, you know, “I have this many things I need to do help me prioritize what I should work on next.” And that’s good for us, [LAUGHTER], but it’s also really good for our students who really struggle with time management, I think. So I really do think there are a number of things that students can use it for, that I would feel comfortable with, but I also think it’s a really useful exercise for everyone listening or for any instructor to think through the possibilities. And you may decide these things are okay, these things aren’t okay, and it may differ for each class. But that is really important to do before you can actually communicate to your students what is and is not okay. And if you want them to actually do what you ask them to do, you have to have good reasons, I think. So you can’t just give them a rule, you should justify that rule, like with kids a little bit, you got to say why you think it’s the case to hopefully bring them along of why they wouldn’t want to just use the tool straightforwardly.

John: One thing I’ve used with some students, especially when I’ve talked to him about some of their uses of ChatGPT is, if all they’re learning in the course is how to type a prompt into ChatGPT and copy and paste that in, what types of skills are they acquiring there that’s going to be useful when they leave, because they could be replaced by anyone typing in those prompts.

Betsy: Right? What makes them unique. So one frame for this is the things we’re teaching students in school are useful for them. We want them to learn so that they can be productive in the market. That’s one way we often frame the work that we’re doing. But I think this gives us an opportunity to open it up a little bit wider where we think about the purposes of education beyond just what is going to be useful in the market. And so I sometimes will use the example of pottery… we’re coming back to art… is that I took a pottery class, after COVID, but it was like the first thing I wanted to do after we were back in person and so I took a wheel throwing class, and I was absolutely terrible. But the idea that I would go to Target and just cheat by bringing in something from Target that was made by a machine. No, there’s a reason I’m doing it. I want to actually learn the craft and the craft has meaning in and of itself, apart from the fact that yes, I could get a much better bowl [LAUGHTER] from Target than anything I will be able to create, but I’m really glad I’m doing it. And I think that’s really what’s gonna start happening is we’re going to start to see that there’s actually intrinsic value to some of these tasks, apart from their value for the, you know, am I gonna make more money later, etc, that we actually think that learning and thinking and the creativity that comes with producing is a value in itself. And that’s going to take a while to turn our students in that direction again, because there’s so market driven right now. But if things start changing in the market, and there are fewer and fewer jobs, they may be open to that conversation.

John: And maybe with the growth of alternative grading systems that try to shift the focus away from extrinsic rewards to intrinsic rewards, this could be quite complimentary.

Betsy: Yeah, and I know that we don’t want to talk all about academic honesty, but it is a real question. And so I don’t want to dismiss the faculty who are anxious about this. I was sharing before the podcast was recorded as well that the news about Chegg losing so much money in the past few months, was a real indicator to me that perhaps my optimism [LAUGHTER] about students not using it was ill placed, that in fact, that’s pretty good indirect evidence that a lot of students are now using ChatGPT to do what Chegg used to do for them. And so it’s not good for us, but it’s not good for the students. And so we do need to think about it. But I do think there’s sort of two broad approaches. One is like the punishment and enforcement approach, and then the other is prevention. And I think focusing on prevention is really where we need to go. And so referencing focusing on the intrinsic value of the work, maybe pulling away from those high-stakes graded assessments is a way to think about motivational changes of how we prevent students from engaging in these and I sometimes will use the example again, of the pottery class, like the idea that I would be motivated to cheat in a pottery class is absurd, like, why would I cheat at that class, because I’m just doing it for my sake. Now, if I was doing it so that I could get more money or so I could get this grade so that I could get into something else I wanted, then I might be willing and tempted to cheat in the pottery class. We know that students cheat because of the grade, they don’t cheat just because they think that’s the fastest way to learn. [LAUGHTER] They know they’re not learning, but they’re like, “I need this grade, because I need this degree so that I can get this job.” And so really bringing them back, decreasing that external stuff, and taking them back to the value of learning may be the only way we’re really going to tackle this. Now, it’s easier said than done. We’re all in a system where grades matter, and students need to get degrees and so it’s a longer conversation. But I do think revisiting some of the literature on cheating, even before ChatGPT existed is going to be really valuable for all of us.

Rebecca: So you’ve talked about moving towards more low-stakes opportunities, and we’ve hinted towards alternative grading, what are some strategies that faculty can use to assess student learning? We’re concerned that we’re not able to see whether or not a student is learning if they’re using tools like that, those are the conversations that we’re having.

Betsy: Yeah, so there’s things you can do to hopefully try to prevent it. But those may not always work. So I have a lot of ideas for how to prevent it. You can give them extrinsic reasons for using the tool itself. Like for example, I mean, this is just a simple one, but let’s say you’re teaching a math class, and you have an in-person final, and you tell them, “you’re going to be preparing yourself for the in person final by doing the homework yourself.” So there’s a kind of extrinsic reward of the final that’s in person will hopefully motivate the students to do the practice problems themselves, because they need to actually learn the thing that will get them that reward. But I do think if they do it, so again, lots of motivational things to talk about. But if they do it, first of all, how do we know is a really interesting concern. And I think that one interesting point that I’ve raised when I’ve had some conversations with folks is that I think a lot of people think when we’re talking about we need accurate assessments of student learning. The first assumption is that what we’re talking about there is we need to have grades that are just, so when we pass them on to like jobs, or to future courses, that we have just grades. But I actually think there’s a real learning reason why we want accurate assessments, is that if I can’t accurately assess your skills, you’re not going to learn. I actually want to know where you are really struggling, so then I can adapt my teaching to better help you learn. And if it looks like you’re doing great, I’m moving on, I’m moving on, I’m not going to actually help you learn that thing. And so it’s really important for learning as well that we have really accurate assessments of their skills. And so if they are using it, so how do we detect it? Tough one, but I think that’s where multiple measures. So you might imagine you have some in-class things that are happening, you’re not just lecturing. So this is a good reason for active learning as well. Because you’re engaging your students in class, you actually hear them speak in class and explain things to you in class. And if they’re struggling there, and then all of a sudden, they have this beautifully written paper, I think that’s a useful comparison. It’s no guarantee that that’s the case because sometimes students need time to reflect, particularly English as a second language learners need time to build their arguments, etc, rather than just being on the fly in class, but it is interesting evidence and that there’s people talking about oral exams and other possibilities or at least having conferences with the students about their work. So it’s not an exam, but just like let’s meet to chat about this. Now, of course, if you’re teaching a huge class, that’s not possible and available to you, but those that are teaching smaller classes, it might be. So I think we’re gonna have to be creative. I have not found a silver bullet here, I have heard lots of great ideas of things that could be possible, but all of them have trade offs, all of them come with downsides. And this is kind of my mantra all the time, when I think about pedagogy issues is that we should not get too absolutist about this, all of us are going to make different choices. And they’re all gonna have different downsides, and they’re all pretty reasonable, because right now, there is no obvious solution of what we all should be doing. I think some may choose to do oral exams, some may choose to do in-person, others may choose to say, “I’m not gonna pay as much attention as some others are.” And all of those things I think are reasonable. They’re just different approaches. And we should keep paying attention and be open to changing our minds if it seems like it’s not working. But I don’t feel like it helps us to be in sort of like one strong camp or the other when we think about the issues of academic honesty, and ChatGPT. So again, I don’t have an answer. Just lots of questions for you. But did you find anything that was useful over the past semester for you?

Rebecca: We teach really different things so our approaches are going to be very different. In my classes, we’re doing creative work. And so historically, and we continue to do this, documenting your process is part of the project. And so we see a project evolve over time. And that maybe involves using the use of AI as part of an input during that time, but documenting that as something that makes that happen. And we do critiques, we show things in progress, and we talk about it, and there’s feedback that’s recorded at those moments. And then if we’re not responding to feedback, then we’re not growing. So we have some systematic ways of demonstrating some creative process there and having to discuss and determine decision making around design decisions or creative decisions, like “Why did you make that decision?” And if it’s just like a random choice, then let’s be intentional about it. And now you need to maybe rethink that choice and make it more intentional. So those kinds of authentic learning opportunities really do kind of push it in a direction where it’s a lot more difficult to use AI as the entire thing. [LAUGHTER] It might be a part of the process, but it wouldn’t be the final output.

Betsy: John, I want to let you respond too, but what you’ve done is that one of the things about that, because that is certainly like doing authentic learning and process-based stuff. As you put it, Rebecca, it’s more difficult. But that doesn’t mean, and this is important, some people will say is that like, ChatGPT will tell you it’s process too, or you can ask it to give me processes, etc. So I do think, actually, one of the things that I think is I appreciate about your example is there’s a lot going on in class, there’s a lot going on, and it’s harder for folks who are doing asynchronous online courses. But if there are ways in which we actually see the process, and that’s kind of the authentic too, is that we’re actually not assessing a product, we’re literally live with them watching the process, I think we might be more likely to get some accurate things. And then if we just said, “Okay, we want you to write about your process, we actually want to see the process as most important.” So John, what about you,

John: The classes I’m most worried about are my large class, which has up to 400 students in it, and an online class that’s on the same topic with generally 40 to 50 students in it. And there’s some challenges there. In the large class, one of the things I’ve done since the start of the pandemic, is to shift all the assessment to online activities. I used to have a midterm and a final that were cumulative, they weren’t a tremendously large portion of the grade, there were lots of low-stakes tests that they could do over and over again. But the validity of those I suspect is going to be a bit different now. Because ChatGPT can do quite well with multiple choice questions and short answer questions and even algorithmic questions. So I’m probably going to bring back at least a midterm and final in person in my large class, just for the reason you described, the motivational thing… that you can practice these things as much as you want to learn it, but you’re going to be tested on this. And the greater your ability to recall and apply these concepts, the better you’ll be able to do on these things. And I wish I didn’t have to do that, because there’s so much advantage of letting students do things over and over again until they master things. But I’ve looked at some of the times on some of the quizzes I used this time, and students were turning them in [LAUGHTER] much more quickly than would have been possible had they not been relying on some sort of assistance.

Rebecca: Well, John, they’re just learning it so much better.

Betsy: Yes, that’s right. That’s right. That’s right.

John: And a nice side effect is you no longer get any spelling or grammatical errors.

Betsy: Yeah, you can read it faster as well.

John: Yeah, it makes it easier. [LAUGHTER]

Betsy: Yeah, no, and I do think as much as I think that we should trust our students, and I don’t want to be overly alarmist. There’s a lot of evidence that our students are doing it and even our students who would prefer not to do it, I think are doing it because they perceive that all the other students are doing it. So this was the same problem in the pandemic with academic honesty is that you have some students who will never cheat for whatever reason, [LAUGHTER] a small number… you have some students who will always cheat, they’ll find ways, they’ll pay somebody, whatever… they’re gonna find ways, and then there’s just a whole bunch of students in the middle who If the context really matters, and if they assume that all the other students are doing it, it puts them at a disadvantage to not do it. And we shouldn’t put our heads in the sand or assume that all of our students are not doing it. We shouldn’t also assume that our students are these horrible people, because they’re doing it, we need to recognize they’re doing it, and how can we help them create the conditions where they would be motivated to not do it to get themselves in trouble. And I do think your point, John, about the 400 students, about teaching an async online course, and even Rebecca, some of your description of what you’re doing in class. One thing that occurs to me, and I don’t have any illusions that this is going to happen, but I do think what these push against is our traditional model of how higher education happens. So we assume for the longest time that it was a lecture that took place. So that’s why 400 didn’t matter versus 20, we also assume that most of the learning would take place outside of class, because you would just come to a lecture and then you would go read the book and learn and teach yourself, basically. It’s kind of this old school model of like, the professor is just there to give you information, you’re going to teach yourself before the exams. And I think I can imagine a world in which, if we really want to see process, we need to be with our students more than three hours a week, and we need fewer students in the course. But that would be such a radical change to the economic model of higher education. I can’t imagine how expensive that would be. But it is more similar to K through 12. And in some ways, I think K through 12 folks have more of an advantage because they’re with the students so much more that they can actually watch them. And homework is less important. One of the most important things I always tell my students and have for years is that most of your learning will take place outside of class, and to emphasize that to them. And I think maybe now that creates a challenge because we’re not with them. And so we can’t sort of see whether they’re doing what we want them to do. So we really have to lean into the intrinsic motivation pieces of what is it that motivates them to want to do well, but with 400 students, they don’t know you really well, John, so they don’t feel guilty about like I have this relationship with my professor. It is tough. And I guess I would say on this point about academic honesty, and maybe we don’t have to keep talking about academic honesty. But I’ve seen a lot of faculty feel really guilty about their approach to this on both sides, like either they’re too overly harsh, or they have ignored it too much. And they’re super anxious about whether they’ve taken the right approach to academic honesty. And I think the most important thing I would say to instructors is this is really hard. Don’t beat yourself up about it, like you’re trying your best. And none of us have a perfect system. If we did, we’d be able to sell that, and it would be great. [LAUGHTER] We don’t have a perfect system. Some of us are maybe leaning one direction, and others are leaning in the other direction. And it’s really demoralizing when our students cheat, and then that makes us depressed as well. But also know that you’re not the only one that all of us have students who cheat and that’s unfortunately, part of the educational process. So do your best. [LAUGHTER] Pay attention. But don’t worry if it’s not a perfect outcome.

John: One of the things I was struggling with just recently as I was grading exams is how do I evaluate the work which is clearly the student’s own work, versus one that probably wasn’t the student’s own work. I don’t want to penalize students for actually trying.

Betsy: I think some people say like, “Ah, let’s just ignore it. It’s not my job to be a cop.” But I think the reason we want to actually do that is an ethical reason, which is that I don’t want the students who actually put forth the effort to be disadvantaged. So I think that’s the right impulse, John, yeah, for sure.

John: One thing I hope that doesn’t happen, though, is that we move to proctored exams online, and that we don’t move to more use of high-stakes in-person exams and so forth, because that would go against so many other things that we’ve been arguing in terms of equity and inclusion and so forth.

Betsy: Yeah, and also, these detectors. So TurnItIn, which most folks are using now, because many schools have TurnItIn, attached to their LMS. And so even before schools have had an opportunity to make a choice it’s default turned on. So your institution has to choose collectively to turn off the AI detector in TurnItIn. And so I think that’s important, too, to think about, like, are we just going to move to these detectors as a way of punishing students? And are they reliable enough? We don’t know. So there are all sorts of good equity questions. Actually, there’s a paper that I read in preprints, about how the detectors seem to flag international students more than those who speak English as a native language, in part because their grammar is better. [LAUGHTER] And so it’s more formulaic, because we’re teaching them the formula of how to speak English. And we need to be mindful of like, how do we balance these things,our equity concerns… and really they both are equity concerns, as you point out, John… so there are equity concerns about more high-stakes testing and in-person testing, etc. But if we just ignore it, there are also equity concerns for the students who do the work versus the probably the privileged kids who are just going to be like, “Whatever, I’m going to pay my $20 a month for GPT4, and be able to get the better answer to be able to use it.” So how do we come up with some sort of solution that balances those and we probably won’t be able to have… at least I don’t think there’s one… where there aren’t some harms? And so it’s really about like, which harms are we willing to tolerate while we work for a better solution? And that’s the hard part of ethical reasoning is that there’s not a solution where no one is harmed usually in these dilemmas.

Rebecca: One interesting thing you said like spending more time with your students, which I have the luxury of doing in a studio art space, we spend twice as much time with students for the same credit hours…

BETSY. Interesting.

Rebecca: …which is valuable. We see process, we get to know our students really well, it’s a relational space [LAUGHTER] for forming relationships. And that really does change the dynamic. But that’s a really big time investment, while the cost of faculty in the spaces but also students of certain backgrounds, or if they have to work, it becomes much more difficult for them to take those kinds of classes, because they’re offered at particular times, and they’re longer, they’re harder to schedule their job around, and that kind of thing. So there’s equity issues in that space, too, as you’ve alluded to, about being in person.

Betsy: Yes, being in person, and then also the point about extra time. I was on talking about workload before. One interesting thing related to workload is that we know, from the research on student learning, that time on task increases learning. And so sometimes I think when we talk about making things accessible to students who are working 40, 50 hours a week, what we’re really doing is reducing the work that’s required of them, which is fine, if it’s just about getting the degree, which I think you can make interesting ethical policy arguments that that’s really important, because economically, it allows them to advance, etc. But if it’s about learning, we actually shouldn’t be reducing the amount of time they’re spending because they’re going to learn less. And so then there’s that tricky question of if we need students to spend 40 hours a week on school, what do we do? We have to compensate them so that they’re not having to work, there’s much larger policy issues at stake here beyond just like, well, we just got to expect them to buckle up and do it, they got to work 80 hours a week now instead. So these are all tough things. And in the context that we are in, where we don’t have those amazing, policy based governmental solutions in the United States, we have to make compromises. And we may say, “Well, maybe a little less work for the students who are working is the compromise we’re going to make for the greater good in this situation.” But recognizing it is true that they’re probably learning less if they’re not putting in the 40 hours. So but maybe with ChatGPT, we can speed it up, I don’t know. [LAUGHTER] So, interesting things about efficiency.

Rebecca: So, we always wrap up by asking “What’s next?”

Betsy: So I’ll speak to related to artificial intelligence a little bit. So I am, in July, going to the Council of Graduate School summer workshop, New Dean’s Institute. They’ve invited me out to share a little bit about ChatGPT, and I’m really excited to be thinking about what’s distinct about graduate education with respect to these tools, it kind of merges my interest in faculty use, as well as thinking about student use as well, for learning. In addition to that, we’ve been talking a lot about how to prepare for the fall, when the faculty come back. We were sort of… just like, COVID… sort of flying by the seat of our pants in the spring, like, here’s some things we’re gonna roll out for you, we’d like to be a little bit more intentional in the fall. And so as I’ve alluded to in this session, I really do think focusing on motivation for students is going to be really important instead of detection. And so we’re gonna do a reading group, we’re gonna go back to Jim Lang’s Cheating Lessons, which still holds up pretty well, actually. And we’re going to do a reading group of faculty on that. And then we’re also going to read the Grading for Growth book that’s just coming out in July, which we’re super excited about alternative grading. I’m teaching in the fall, as I said, so excited to actually try some of these things out and see if my ideas are actually practical [LAUGHTER] or not. And hopefully, I guess I just say, what’s next? I hope there’s some regulation. So we didn’t get into a lot of details about this, because we were focusing on teaching and learning. But I know Sam Altman, and Gary Marcus were before Congress. And I do hope that we actually see, unlike with social media, that we see some movement for some regulation about the development of these tools. So I think what we have now… fine, let’s figure out how to use them. But it’s really anxiety inducing to me that these tools will develop skills that nobody planned emergently like, it’ll just, “oh, now it has this new skill.” And the more that we build these tools out, we don’t actually know what we’re going to create. And I think [LAUGHTER] that is a little worrisome to me. And so I hope that what is next is more regulation on the tools.

John: We should note that we are recording this several weeks before it’s actually released. And we hope that at the time when this is released, [LAUGHTER] we haven’t reached that AI apocalypse that so many people have been worried about.

Betsy: That’s right. That’s good, John, thank you.

Rebecca: Well, thank you so much for joining us, Betsy. We always enjoy talking to you.

Betsy: Thanks for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]