Category: learning

326. UDL in Action

Posted in accessibility, active learning, authentic learning, diversity, equity, feedback, higher education, inclusion, learning, pedagogy, social justice, teaching, universal design
January 31, 2024

Universal design for learning, or UDL, is a framework to help us design more equitable learning experiences. In this episode, Lillian Nave joins us to discuss how she has implemented a UDL approach in her first-year seminar course. Lillian is the Coordinator of the Center for Excellence in Teaching and Learning for Student Success at the Appalachian State University Hickory Campus and a senior lecturer in a first-year seminar course at Appalachian State University. She is also the host of the ThinkUDL podcast. She is the recipient of several teaching awards and often serves as an invited speaker on UDL issues.

Show Notes

Transcript

John: Universal design for learning, or UDL, is a framework to help us design more equitable learning experiences. In this episode, we discuss how one faculty member has implemented a UDL approach in a first-year course.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Lilian Nave. Lillian is the Coordinator of the Center for Excellence in Teaching and Learning for Student Success at the Appalachian State University Hickory Campus and a senior lecturer in a first-year seminar course at Appalachian State University. She is also the host of the ThinkUDL podcast. She is the recipient of several teaching awards and often serves as an invited speaker on UDL issues. Welcome, Lillian.

Lillian: Thank you very much. And I applaud you on all of the monikers that I currently have attached to my name that often don’t make sense. So, very good. And of course you said Appalachian in the way we say it down here in North Carolina and I don’t have to throw an apple at you. [LAUGHTER]

John: Well, I’ve heard it so many times on your podcast that I wanted to be sure we got that down.

Rebecca: Today’s teas are:… Lillian, are you drinking tea today?

Lillian: I am and I brewed it specially for our interview today, it is Bengal spice, which is a Celestial Seasoning herbal tea that is caffeine free. I went caffeine free a while ago, and it’s fantastic and I don’t have to put sugar or honey or anything in it. It has cinnamon sticks and it’s delicious and my best friend in the world got me hooked on it. And it always just makes me feel warm inside and makes me think of the lovely conversations I have with the people who I’ve drank tea with in the past. And I also have iced tea because I continuously drink iced tea and this one happens to be actually just a Crystal Light [LAUGHTER] iced tea as well.

Rebecca: And I did notice, of course, and must state that there was a Tea for Teaching mug involved.

Lillian: Yes, exactly, and I appreciate that. John saw me at a conference and I was so happy to get my Tea for teaching mug that I made sure I was drinking from it today.

John: And I am drinking a peppermint spearmint blend today… also caffeine free today.

Rebecca: I am drinking a highly caffeinated afternoon tea [LAUGHTER] to make up for everybody’s caffeine deficits.

Lillian: Well, I have plenty of chocolate throughout the day that is not caffeine free.

John: So we’ve invited you here today to discuss your work on Universal Design for Learning. We probably don’t have many listeners who aren’t familiar with UDL, but for those who are not as familiar, could you provide an overview of Universal Design for Learning.

Lillian: Of course, and I’d be glad to. Universal Design for Learning is a way of thinking, I would say, about teaching and learning that relies on three main concepts that include: engaging your students, providing accessible materials for your students, and varying the ways that your students can explain to you that they’ve learned something. And it is all based on neuroscience and also a lot of research that tells us that all of our students are different. Variability is the norm. And so we have students in higher ed that come from all different backgrounds, different cultures, different preparedness levels, different abilities and disabilities. And in order to reach all of our students, it’s important to think about that variability and universal design for learning gives us some really specific things to look out for. And three areas or categories: the engagement, the representation, and providing multiple means for those and multiple means of the student to express their knowledge. And so that’s like the general overview, but there are like so many weeds, that I can have a podcast [LAUGHTER] and talk to people about all of the different intricacies of UDL. But in general, that’s it in a nutshell.

Rebecca: I know we’ve talked about it a bit on our podcast before in terms of the difference between UDL and accessibility. But for those who haven’t heard those previous episodes, can you talk a little bit about the difference between those two ideas?

Lillian: Yes. So, accessibility is a part of Universal Design for Learning. You cannot have universal design for learning without attention to accessibility. But accessibility alone is not universal design for learning, it’s a part of it. And I like to think of UDL as like a three-legged stool. If you take out one of those legs, the stool doesn’t function anymore, it will fall over. And one of those legs is multiple means of representation. It’s typically the center column that’s purple if you look at the UDL guidelines at UDLguidelines.org. CAST puts those out. And that middle column about representation is about providing multiple forms of representation, so different texts or audio files, or making sure your font is readable too, especially for dyslexic students. So accessibility is about making sure that all students can access materials, that they have the ability to make sure they can actually get at that information. And this is less common now, but there used to be like those PDFs that a professor copied out of their book when they were a student and it has all their like markings and stuff in it. And then they would just like make a read really bad copy of it, and hand it out when everybody still used paper all the time, or maybe it’s on a screen. And those oftentimes were not accessible. And that means if a student needed a screen reader to read it, or even if they just needed to make the font bigger, they couldn’t because it wasn’t an accessible document, a screen reader couldn’t read it, meaning if a student was dyslexic or even blind, they had really no way to get at that information. So when you make your documents accessible, and you make your class accessible, you are making sure that everybody really just is on an even playing field, and they can get at that information. But in addition to accessibility, you want to make sure you’re also giving lots of opportunities for students to express what they know in different ways. And you’re also engaging with the students. And those two other things are not included in accessibility. That’s what makes up UDL.

John: How did you become interested in UDL?

Lillian: Well, I started working with our Center for Teaching and Learning in 2016 as a faculty fellow and I started doing faculty development as a one-course release for quite a while. And in doing that, we became part of a grant that was in three North Carolina institutions that was called College STAR. And STAR stands for supporting transition, access, and retention. And it was a two-pronged approach where there was a lot of student development, so tutoring centers and things like that. And then there was this other side, and that’s the part I got interested in and got pulled along with and that was supporting faculty. And to do that we used Universal Design for Learning. And I was just part of that grant. And that ended up me being the Universal Design for Learning Coordinator at Appalachian State. And I started going to different departments and introducing that at workshops and that sort of thing. So I became kind of the UDL girl or UDL lady for App state, which I think is about 2016 when that started, and then I saw it, like I’d never heard of it before, and I’d been teaching since 1997. And then I thought, oh, boy, this is really good, this makes a lot of sense. And so I started implementing it as well in all my classes.

Rebecca: So we’ve talked a little bit about UDL principles broadly. So can we dig into maybe a specific example, like your first-year seminar course? That sounds pretty interesting.

Lillian: Absolutely. So my course right now, I teach one called intercultural dialogues, and I get first-year students, and so I love it that they’re small, under 24 students. This past year, I was at our new campus in Hickory, and I only had 18 students there. And we get to work on intercultural competence, which is one of those major things that colleges want our students to know. It’s a 21st-century skill, and it is about understanding our own cultures, and then understanding that other people have different cultures. And then how can you mindfully act and interact with somebody from a different culture. And I have heard some students, at the end of the term, say “this is something everybody should know, like, this is really, really important.” They’ve seen how important it is. But at the beginning of the class, like nobody wants to be there. Nobody wants to take a first-year seminar course. They’re usually there because it’s Tuesday morning and they have an open slot. I’m in that unenviable position of teaching first-year seminar, and it’s that Gen Ed requirement that nobody wants to take. So I want to make it interesting, and I want to make it worthwhile for them, and so we learn about our own culture and then I also match students up with students abroad. We’ve worked with students in Doha, Qatar, in China, in Morocco, in Germany, Thailand, and Japan in the past year, and they work on intercultural competency skills and talk about things like power distance in the classroom, or some of the UN Sustainability Development Goals, like gender equality by 2030. And so they’ve done some things with students in China about that. So the class itself is about intercultural competence. And I have infused a bunch of UDL into this class over the years. And so it wasn’t all at once, but it took a while. And so that means I have multiple ways for the students to get at that information. So there is never just one way to do the reading. It’s either accessible or I’ve recorded a voice audio file for students. So we have a lot of commuter students at this campus, and so actually, we have zero students who live on campus in our new campus in Hickory. It is a new campus, it just started and so there are no dorms, there’s only one building and we’re all in that one building. And so students come in, they have jobs, and so UDL is very helpful for me to think about those students who have various commitments of their time. So they could listen to the audio file rather than read the book, because they’ve got a 40-minute commute to come into campus or something like that. So they’ll have an audio or video. I’ve used H5P, which is on our learning management system, which is like an interactive video, there’s VoiceThread, which is another way to be kind of interactive for students to participate. And so there’s always multiple ways for them to get at that information. And then there’s multiple ways that I ask them to tell me what they know. And they’ve done concept maps, so we have very little like, “write me a paper,” there’s very little of that my class. And that’s also a culturally competent type of teaching thing, because we often in an individuated, Western academic model, we prioritize reading and writing, reading and writing, reading and writing. That’s always what it is. And yes, we need to have very good readers and writers. But there are lots of other ways to learn and express your learning that might be more prevalent in other countries and in other cultures. And so they’ve had to draw some of their answers, they’ve had to give me a visual representation, they’ve written a poem, drawn a cartoon, and tell a story. One of their first assignments is to bring in something that expresses who they are, a cultural artifact. And then the last class that we just had recently, I asked everyone to name everybody else’s cultural artifact, so they learned about each other that way. And it was things like a keychain or one student brought in the T-top of his T-top convertible car. [LAUGHTER] Because the car was really important to get him around and all that stuff. And engagement is the last part. But it’s really the first part that gets students interested in the activities and in the learning and why they should learn. So I start off with a liquid syllabus, which is a syllabus that students can access outside of our learning management system, and they can see what we’re doing, and they get a video of me talking to them. And that’s supposed to be engaging. And then they have a lot of authentic assignments working with students overseas. And this year, because I was finally back in an actual classroom and not doing remote teaching, as I have been since 2020, I took students up to New York City. And Appalachian has this amazing loft that actually anybody can go stay at so all of your listeners could go and stay at Appalachian’s loft, and it’s very inexpensive. It’s like $70 a night per bed, and it’s these two rooms of 10 beds each. And we took students up there to actually learn a lot about culture. We went to the Tenement Museum, they looked around, and it was very cool. So experiential type of education as well. So that took like years before I got to that rendition of how I teach that course. But those are all UDL principles that guided me.

John: You mentioned collaboration with students in other countries, what types of collaborative work did those students do in your classes?

Lillian: They had specific zoom meetings that they had to do personally one on one. So one of them is that students got matched one on one with fellow students in Morocco. And they were supposed to talk about the difference in power distance in an educational setting. So power distance is the amount of power that people in a group expect and believe should be shared or held by the people in that group. So in an educational setting, if you’ve got a first-year student who comes into a large lecture hall on a college campus, there is a larger power distance for that instructor who will pretty much lecture to those students, students don’t raise their hand all that often, there’s not a lot of back and forth, there’s not a lot of flexibility, they probably don’t even know the students’ names. And so that would be an example of a larger power distance. A smaller power distance might be in a classroom, like in my classroom, I say, “You can call me by my first name, you can address me in this very informal way. We’re not going to have a lecture, we’re going to be in small groups, and then we’re going to share our ideas.” And that way, there’s a lot more voices, there’s a lot more talking. And that can happen in various times throughout the semester. I may do a lecture, I may not. And so my students were talking with students in Morocco to find out about their understanding of what power distance was. And do you call your professor Dr. Smith? Or what do you address them by? Are there rules about when you can address your professor and those sorts of things. So that was one of them. And then we worked with students in China, and this one was a series of three Zoom conversations. And all of our students had to set all these up. They were all in English because our partner students wanted to do this in English, and most of them had never spoken to a native English speaker. So this was a really good goal for them. And in China, they can’t have Zoom, not allowed. And so the students had to receive our invite from our students, and their first session was kind of an introduction: who they are, what they’re doing, for about an hour. The second was a second list of questions, which was about: Who takes care of children? Who goes to work? Who do you live with? Do you live with an extended family? Do you live with a nuclear family? And it was really about gender roles. And one of those things that’s a national cultural dimension is something called achievement versus nurturance. And that continuum has also been called in the past: masculinity versus femininity. And it’s how much a culture believes that men and women should adhere to somewhat stereotypical gender roles. So are there women CEOs, and stay-at-home dads? In some countries, that happens in some cities. That happens a lot more than in other countries. Do you put more emphasis on earning a higher wage? Or on having the flexibility to work from home? Like, where are you on that? And so the students talked about that. And then in the last session, they talked about: if you could change anything, what would you see that might improve your country from where it is now and that sort of thing. So they got to do some really authentic conversations with people around the world, and the students in China were 12 hours ahead. So my students were meeting sometimes at two in the morning, but they were up, [LAUGHTER] it didn’t matter.

Rebecca: So it sounds like there was a lot of coordination with counterparts around the world to make sure that you designed experiences for both sets of students that met, maybe not the same learning objectives, but learning objectives that were relevant for each population.

Lillian: That is exactly it. And it was my colleague in China, who said she wanted to do something about the sustainable development goals from the UN. I said, “Okay, well, let’s try and look at that.” And it worked for each one. And it is a lot of coordination for the faculty. And so I would meet with my fellow faculty member several times throughout the semester. And so we got the dates right for when we’d have Thanksgiving, nobody else had Thanksgiving break, and we have holidays, and they started a month early or a month late, and so there was a lot of coordination. And then they had to give me the list of all the students and I needed gender, too, because some students wanted to stay within their own gender, women, especially, in Morocco where some were less likely to speak to male students, who wanted to stay with female students. So we wanted to be culturally sensitive to those types of things. So there was a lot of beginning coordination to set those things up.

Rebecca: I wanted to circle back to one other thing you said too, in that you mentioned your classes developed with all these UDL principles over a significant period of time.

LILIIAN: Yes.

Rebecca: I want to know how you got started, what was the first thing you implemented? And how did that set a trajectory for the others?

Lillian: The first thing, way back when, probably 2016, 2017, like the big aha moment for me, was not doing the same thing all the time, and not having to grade everything, meaning maybe we were just going to do some honestly experimental assessments in class that were kind of fun and authentic. And I didn’t have to grade everything. And when I kind of let go of that, it opened me up to some more ideas. And then I thought, well, I don’t need them to write a paper, because I really have a specific goal in mind. And the goal doesn’t necessarily need to be a paper that then I’d have to read, [LAUGHTER] it made my life easier too. Maybe they just needed to demonstrate their understanding of these concepts. And so like one of the first things we do is draw an iceberg, and talk about how the culture that we see here, taste, feel, smell, all those five senses, that’s about 10% of what makes a culture. And when we think about culture, it’s usually just those things, it’s like, “Oh, you eat this special meal on Lunar New Year, and you have these special foods, and the kitchen always smells this way, or we dress up in cultural clothing, or whatever.” But that’s really only 10%, the tip of the iceberg. And then we have to get really deep into what our values and beliefs and assumptions we make. And that’s typically the hard part of the class. And so I just had students either draw an onion or an iceberg. And then they had to kind of point to where this was, what are your deeply held beliefs and assumptions, and that culture is so much more. And it’s a lot easier, I think, to conceptualize it as a drawing than it is to write me a paper about what your deeply held beliefs are, [LAUGHTER] and where they align with the things that I can see on the outside.

John: You mentioned that the first thing you had done was reducing grading and doing more formative assessments, which is beneficial for students too because it takes some of the pressure off and gives them the opportunity to try something, make mistakes, and learn from that without any penalty. Is that something that you’d recommend for someone who’s interested in exploring UDL, as a first step, if they’re not already doing that?

Lillian: Absolutely, I think it frees up both the student and the faculty member to kind of see what works. And so much of Universal Design for Learning is about feedback, feedback from the students. And that is a major portion of UDL. And I should have said that at the very beginning, that you really want to be figuring out what works for the students and what works for you. So I do think that’s a great way to think about it. And also, the flip side of that coin, to me as well, was whoever is doing the work is doing the learning. So if you are always lecturing to your students, it’s hard, like, you got to put together this great lecture, I always felt like I had a top hat and a cane, you know, walking into my lecture, and yadda-dat-dah, I’m gonna, like dazzle you with my art historical knowledge, and make it interesting. And I was doing a lot of work to do that. And I have slowly moved into kind of the other end of this continuum, from lecturer into facilitator. And if I can facilitate the students working together, or a lot of feedback back and forth with me or with each other, then they’re actually risking some things like “talk to your neighbor about this,” and they don’t have to raise their hand in front of the large class, they’re actually trying and risking and doing these smaller things. And that’s where I see the learning happening. If they’re just listening, that’s fine, that’s great too, but the more they can participate in their own learning, the better it is, that I’ve seen, certainly in my classes, the more they can do, the more they’re learning. But it doesn’t mean I have to evaluate every single thing that they hand in, or that they produce. And I can certainly, on the spot, kind of tweak things and say, “Okay, let’s turn it into this direction,” or something like that. But it was the: “I don’t have to grade everything they do” and “The person who’s doing the work is doing the learning.” And it was like freedom for me to try all of these things that were totally not what I had done as a student, or had valued as a student or an instructor because I was very much in that: “Alright, you’ve got a 15-page research paper, a midterm, and a final, and that’s the art history course.” And I don’t do that anymore.

Rebecca: Well, it sounds like not only is there a benefit to the faculty, in terms of workload, joy, [LAUGHTER] etc, but also an emphasis on self efficacy for students and building confidence.

Lillian: Oh, yeah. Exactly. And they’re trying things out, and they’re seeing what works. And that feedback is really important, a big part of UDL.

John: And you mentioned that it took you time to build to where your courses currently are. Is that an approach you’d also recommend to faculty who are beginning to introduce UDL principles, because it can be a lot of work completely redesigning or transforming your teaching?

Lillian: Absolutely. I don’t know if I could have done this stuff early on in my career, because I was worried about how I looked and was perceived. I was very young, and so I think I needed to feel like I was in charge. And that power sharing was too difficult for me as a young instructor. So I understand that. And now I feel much more comfortable in the classroom. And I feel that being a facilitator is really helpful for the students. And sometimes they just want to sit and listen, but that happens too. But it took a long time to get there. And the course has evolved over a long time. And you try new things. Tom Tobin and Kristen Behling talk about the plus one mentality, just trying one new thing. And that’s what happened, is when I started this course, we weren’t speaking with students in other countries. That just sort of happened when I went to a conference and made some friends in other countries and said, “Oh, I bet this would make a lot of sense to add this in.” It’d be really authentic, which is one of the engagement principles is having really authentic learning experiences. And I used to be like, “Oh, you’ve got to plan everything out, and it has to be perfect.” And now I see that I fumble through a lot of things. And every once in a while something sticks, and it’s good. It’s a practice. They say being a doctor is more of a practice. I think being a teacher is very much a practice to see what works and what worked in my class five years ago, doesn’t necessarily work now. Things that were really fun and hot at some point, you know, like making memes is pretty fun right now, but we didn’t do that 10 years ago, and we probably won’t do it in another five years, like, what did you learn? Let’s make a meme out of it. It’s evolving.

Rebecca: It’s interesting that we’re talking about a course about culture. And you’re describing how the culture of higher ed or institutions or our classrooms also evolve. And that evolution requires risk both on the part of the instructor as well as on the part of the students, and that the UDL principles are really allowing that risk to happen on both ends.

Lillian: Yes, and sometimes there are forces outside of our control that make that more difficult, and it’s not an enviable position. So things like I wish I didn’t have to grade, but we still have to have grades in the end. And so how does that fit into your course. And so I know a lot of folks are using ungrading. I know you’ve talked with Susan Blum and Josh Eyler about various different kinds of grading, which I think is like a later on kind of thing for UDL. Start with accessibility, make sure your stuff is readable and devourable by all of your students, and then start kind of playing around with it. And then maybe I think that ungrading or different kinds of grading structures might be the last step on that process. But to each their own.

John: One of the issues involving student variability is that some students might be resistant to some of the approaches that you’re using, because there have been a number of studies that show that students often prefer to be lectured at. And it seems like they’re learning more that way, despite the evidence that that’s less effective. How do you persuade students to be open to trying new approaches to learning?

Lillian: Great question, I know exactly what you’re talking about, like students, they’re like, “this is how we learn best,” and then you actually poll their knowledge. And students in an active learning situation who kind of hated it are much more knowledgeable than the students that were just in a lecture where they really liked it, because that was kind of safe. And in the last year or so I’ve heard myself saying this a lot when I do speaking, and when I’m talking to folks, is I think everybody needs to be uncomfortable in the classroom, some of the time. We don’t want the same students to be uncomfortable all of the time. So that means varying those different ways that we assess students, so it’s not the “alright, every week, you’re writing a paper,” oh, that also gets boring. But for your great writers, it’s fantastic. But are we really finding out what that student knows? Are we finding out that they’re a good writer. And so maybe that’s a poem, or maybe it’s a concept map, or there are other ways to assess that info. And so I tell my students, like, I’m really conscious about that, like, you probably aren’t going to like some of these things, but your neighbor isn’t going to like the next thing. And so having those opportunities that you have to step out of your comfort zone, to get into the learning zone, but not all the way out to that outer edge of the target, which is the panic zone. And that’s actually an intercultural competence idea that I learned in that field. When you study abroad, the only way you’re really learning is if you’re in that learning mode, like if you go to Germany, but you’re living with a bunch of Americans, and you never speak German, and you go to McDonald’s, and you’re at an English speaking school, then have you really learned much about German culture? So you should go outside of that comfort zone. Maybe you’re living with a local family, and you have to speak German, but you don’t want to go and you are in like a chaotic household and they don’t speak English, and they haven’t made sure that you have any food, and you don’t feel safe, and all the classes you are way ahead of you in your German speaking. And so you’re not learning much either, you’re kind of panicking. So it’s that learning zone we have to be in and so I think in our classrooms, we need to do that too, have multiple different ways for students to express what they know, which is one of our UDL guidelines. And I am very overt when I tell students that. And I found that with student evaluations, like I would get student evaluations where they asked like, “Did you practice critical thinking skills?” And they’d be like, “No, like, I totally didn’t at all.” [LAUGHTER] And then the next year… and I think I learned this at like an academic conference… the next year, throughout the course, I’d be like, Okay, we’re gonna do this critical thinking exercise, this thing that we’re doing right now, this is about critical thinking, you’re going to use your critical thinking skills because this thing that we’ve done, that’s a critical thinking skill. Guess what? The evaluations… way up. [LAUGHTER] Exactly like you’re pulling back the curtain and you’re saying, like, here’s actually why we’re doing it, and this is what you’re doing. And so I think explaining that is really helpful. And then the students know why they need to do something like what I’ve asked them to do, why am I writing a poem or why am I drawing an iceberg? And I think we do need to tell students that and not just have them guessing, because then they’re going to be in the panic zone and not learn so much.

Rebecca: That’s a really good point to remind students that being uncomfortable and taking risks is actually part of the learning process. Can’t remind them too much.

Lillian: Yeah, exactly. It’s necessary. And they do want to just sit and doodle. And not that doodling is bad. But they just want to sit and listen and have us do all of the work. But it’s like, I know this stuff, so why do I need to explain it? You could just watch a video of me talking. We need to get you into grappling with this and doing the stuff that I know you don’t want to do and you don’t want to be here because it’s a first-year seminar and you’re a first-year student. So my heart is with all of those folks that teach Gen Ed [LAUGHTER] to the students who don’t really want to be there.

Rebecca: Switching gears a bit now, can you talk a little bit about how you started the ThinkUDL podcast?

Lillian: Yes, it’s going back to that College STAR grant. We were getting into doing like workshops, and so I was working with other universities in North Carolina. And we had a PI, the head of it was at our East Carolina school. And I said, “Do we have any multiple ways to do this. They were doing research and then some workshops. And it just made sense to me like, we need a podcast, like this would be so off brand for UDL not to have multiple means for us to get this information out. And a podcast has the added bonus of being asynchronous, so people can listen to it whenever they want. I’ve always had transcripts, too. So if you don’t want to listen to my voice, which is totally fine, you could read the transcript and you can get that information, you can see the resources. So there are multiple ways to get it. But there was money from that original grant that sent me and my shout out to Tanner, who was my first sound engineer, and we went to a podcast convention in 2018 in the summer in Philadelphia. And seriously, I didn’t know a thing at all. I didn’t even listen to podcasts then. The only one I’d heard of was the Teaching in Higher Ed with Bonni Stachowiak, and then like, “Okay, we’re gonna try it.” And the very early episodes are, I think, awful. But luckily, Tanner kind of cleaned them up. But there was a Chris Farley on Saturday, live long, long time ago, so I’m showing my age here. But he would interview, in these sketches. He’d interview people that were amazing, like Paul McCartney from The Beatles, right? And he would just fumble the whole time. Like, “Wow, so you were in the Beatles? Wow. Yeah. That’s great. So can you tell me like, what’s it like being a Beatle?” And that’s what I felt like the whole time, [LAUGHTER] like “Wow. Okay. All right.” So it took a while. But the grant helped it and for about three or four years it was grant funded. And now I’ve turned it into its own nonprofit. And Texthelp is now a sponsor. And so they do the editing for me because the grant ended. And Tanner, he was part of that grant. So i had to kind of move on. That’s how it started.

John: So you’ve been doing this for a while now with the podcast, and we’ve been listeners since the very beginning. What do you enjoy most about podcasting?

Lillian: Well, ours came out around the same time. So the nice Tea for Teaching, right? It’s like 2018. And so I enjoy talking to people. If you’re still doing it, you have to enjoy talking to people. But that’s the best thing. I talk to people just all around the world because I do want it to have a worldwide focus. And so I have listeners, the top five are in the US, Canada, Australia, Ireland, and the UK, and so English speaking countries, yay. But you can see like, all over the world on six continents that people are listening, and I had a listener in Australia say, you know, I was walking on the beach, near my home, on the coast of Australia. And that just blows my mind that people are actually listening. But mostly it’s like, I get to just learn about new ideas all the time. And you would think UDL like it’s so focused on UDL, like, there would not be enough, like I should be done with this, but there’s so much.There are 31 of these checkpoints in Universal Design for Learning, and as you mentioned, it’s really overwhelming. Like if you were to go and just look at the guidelines, it’s like a whole bunch. And it’s like, how am I going to do that? You can’t, you can’t do it all. You can’t just redesign your course right away. And so there’s just all these little conversations I can have to help people understand what you can do. And then I get to talk to really interesting, witty, awesome, brilliant people all over the world. And that’s the best part. If I could just do that, like that was just my job, I would love that.

Rebecca: Definitely something that John and I enjoy too. It’s kind of an introvert’s dream to talk to [LAUGHTER] a lot of individuals one on one rather than having to network through a conference or something like that. It’s a good opportunity to have really in-depth conversations with folks that might not have the opportunity to have otherwise.

Lillian: Yeah, my brain is always seeking out the new. And so I love like, “Oh, that’s a neat idea.” And then I’ll send them an email, and sometimes they write back, and “oh, I really love to talk about this, it’s cool.” And so I’ll read their article or their book or whatever. And then there’s something else shiny that I get to go talk to other people about. And it’s just been helpful for folks. And honestly, I just didn’t expect there’d be listeners, and there are listeners. And so that’s just really fantastic.

John: We started out as primarily to meet our campus audience needs for commuting faculty, and so forth. And then we were amazed at how it caught on and spread. And it’s given us that opportunity that you both mentioned, to talk to some really interesting people doing some really interesting work. Before that in the teaching center, we talked to people at a workshop, and we might hear from one faculty member for three or four minutes, maybe 10 or maybe they’d come in for a consultation. But usually that was about a problem or an issue they were facing, but it just provides a wonderful chance to connect to people that we normally wouldn’t be able to talk to. And we see an interesting article, and then reading through it and getting to talk to the people doing the research in depth, it’s really a valuable experience.

Lillian: Yeah, everybody should be a podcaster just to have these conversations. You don’t even have to record them. It’s just really neat. And so it’s given me that like, “Hey, I have a podcast,” like a reason for me to be intrusive in somebody’s email. Like, I really want to talk to you about this. This is really cool. Would you talk to little old me? If so I have a podcast. [LAUGHTER]

John: And I’m amazed at the number of people who say yes.

Lillian: Absolutely, me too, like, “Wow, you’re actually going to talk to me. That’s so fantastic. I appreciate that.”

Rebecca: We’re definitely a part of a really wonderful community of practitioners.

Lillian: Yeah, it makes a very thankful and it’s so cool, because I have listened to Tea for Teaching for a long time. That’s actually my most listened to podcast for teaching and learning. I enjoy the fact that there’s two of you, and you kind of go back and forth and just interesting topics. So I’ve enjoyed yours, ever since the birth of our podcasts in 2018. They’re siblings. [LAUGHTER]

Rebecca: They are.

John: So we always end with the question. What’s next?

Lillian: I am so excited about what Appalachian State is doing. About two years ago, they bought a building in a town called Hickory, North Carolina. And our main campus is in Boone. And it’s a beautiful campus up in the mountains. And I happen to live halfway between these two cities. And so I was going up to Boone to teach and it is about 42 minutes to get up. And you have to go over the eastern Continental Divide, over the Appalachian Trail, over the Blue Ridge Parkway in order to get from my house into Boone to teach. And that’s great. And it’s the most beautiful commute I think in the world. But it also gets foggy and icy and weather and I was enjoying it, and it was where I would listen to podcasts. But Appalachian State is now the first university in the North Carolina system that now has two campuses. And so we’ve opened this campus in Hickory, it is a commuter campus, and some brave students, we have about 250 to 300 that have started this past fall of 2023. It has birthed this campus. And so I get to teach there, and I get to do some faculty development. And it’s really exciting to be on the ground floor of a new campus. And it’s the only one in the North Carolina system. We’ll have other campuses, but there’s no multi-campus university for us. It’s like being in a startup, except I don’t think I get stock options. That’s the only bad thing. [LAUGHTER] And so meeting new faculty, some faculty are teaching for the very first time. And so there’s no like institutional culture that they’re jumping into at this new campus, although we are very much a part of the Boone campus. It’s new. And there’s only a very small number of faculty there. So it’s like being at a small liberal arts college in a state system. And it’s just really cool. And I’m loving meeting the faculty there and helping with teaching and learning and UDL and all that stuff. So that’s like the next big thing is Appstate Hickory, and it’s really exciting.

Rebecca: Well, I hope you have a wonderful adventure. [LAUGHTER] It sounds like a really fun opportunity.

Lillian: Yeah, in fact, I’m in a group of faculty, we have like a community of practice, a peer mentoring circle we call it, and we’re calling ourselves the Hickory Adventurers, because like, we don’t know what’s going on, [LAUGHTER] and we’re trying to figure it out together.

John: You get to help shape what’s going on, which is a really nice place to be.

Lillian: Yeah, it’s fantastic. I’m excited. And I think I’m just that kind of person. It’s new and shiny. And I’m there.

John: I think we’re both that way a bit. And that is one of the risks of having a podcast, you get to hear about all these great things that people are doing, and there’s always a tendency to try to do many of them. And that can be a bit overwhelming, not just for us, but also for our students.

Lillian: Yes I know I have to peel it back [LAUGHTER] just a bit, don’t go overboard.

Rebecca: Well, thanks so much for joining us, Lillian. I know we’ve wanted to talk to you for a while.

Lillian: Absolutely. I’m so glad and when you contacted me, I was super excited. So thank you so much for having me on Tea for Teaching. I’m gonna show my mug that nobody can see ‘cause it’s a podcast, but I love my Tea for Teaching mug, and thank you for having me.

John: Well, thank you for joining us. It was great talking to you and we’ll look forward to more conversations in the future.

Lillian: Great.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

325. Looking Forward to 2024

Posted in access, accessibility, diversity, edtech, educational technology, engagement, equity, grading, higher education, learning, motivation, pedagogy, teaching
January 24, 2024

As we enter this spring semester, we take a break from our usual format to discuss what we are looking forward to in 2024.

Show Notes

Transcript

Rebecca: As we enter this spring semester, we thought we’d take a break from our usual format to discuss what we’re looking forward to in 2024.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: So what tea are you drinking now, Rebecca?

Rebecca: I’m drinking Ceylon tea.

John: And I have ginger peach black tea from the Republic of Tea.

Rebecca: So it’s a new year, John, and we’re gonna have a nice positive episode. So what are you grateful for?

John: I’m grateful that we survived another year. I’m grateful for the continued return to face-to-face instruction and a continuing return to a more normal classroom environment and college environment after all the disruptions from COVID that lasted for a while. And I’m also grateful for the initiatives that we’re using on our campus, and most campuses, to provide more focus on equity and trying to reduce some of the equity gaps that we’ve been seeing. And I’m seeing a general interest in that across a wide range of faculty and the administration. And we’re just in the process of running some workshops, and we’re getting some really good attendance at workshops that focus on techniques that faculty can use to improve equity and reduce some of those gaps. And I’m looking forward to seeing continued expansion of more equitable practices. And also, we’ve had quite a few people trying to implement the TILT approach that Mary-Ann Winklemes has talked about on a past podcast episode, and we’re hoping that that, combined with the increased structure that many people are trying to use in their classes, will help provide all students with more equitable outcomes.

Rebecca: I’m really glad that you have brought up equity, John, because I was just reflecting on a couple meetings that I was in just this week, and thinking about how equity oriented many of our colleagues are. And it’s really exciting to see them really advocating for policies, instructional practices, and many other things that are really equity oriented, and thinking about inclusion, and access, and all the things that you and I have talked about for a long time and have cared about and tried to implement in our classes. I’m also really grateful for, and I know you are too, for the many guests that we’ve had who’ve shared their expertise with us and with our audience. When we do these weekly episodes, it’s so great to have the opportunity to talk to such experts, to learn from them, to stay fresh with what’s going on, and to be able to share it with everyone else. It’s an experience that I didn’t know that I wanted, and I’m glad that we get to continue doing it.

John: And one other thing I’m grateful for from last fall is I attended my first POD conference, and I got to meet dozens of guests that we’ve talked to before. And we’ve talked to them, we’ve seen them on camera, but it was so nice to meet them and talk to them in more detail and in more depth in person.

Rebecca: I felt that way when I went to EDUCAUSE for the first time this year and connected with a number of colleagues focused on accessibility and growing that network and really connecting beyond just names and emails and other ways that we’ve communicated.

John: What are some of the major things you’re watching in the higher ed landscape? We’ve seen a lot of changes going on in the last few years, what are things that you’re going to be focusing more of your attention on in the next year?

Rebecca: I know that some of the things that we’re working on in grad studies and that I’m personally really involved in are kind of some increased accessibility resources for our colleagues at Oswego as well as SUNY. I’m looking forward to building out some of those resources, sharing those resources, and wrapping up a couple of research projects related to accessibility and getting to share those out. And I’m really excited that the higher ed landscape generally is having a lot more focus in this space because students with disabilities have been often overlooked in our diversity, equity, and inclusion initiatives. And there definitely is a push to be a little more inclusive, and to have that population represented in these efforts and initiatives. We’re also really focused on ideas of belonging for a wide range of students, thinking about how do we get our online students connected to each other and to the larger population of students and to see them as members of our community, extending some of those features and opportunities for international students and really thinking about what some of their needs are to be successful at our institution, especially with our kind of rural location, and what can make things really excited. So I’m really looking forward to finding ways to support our students not just in the courses that I’m teaching or in my instructional role, but also in policies and procedures that we’re implementing at the institution,in grad studies, but more broadly as well. How about you?

John: One of the things that I’m really following closely is the development of AI. This came about a little over a year ago and it’s been a really disruptive influence. It offers a lot of tremendous possibilities, but it also provides some challenges to traditional assessment, particularly in asynchronous online courses. So I’m looking forward to continued development, it seems like there’s new tools coming out almost every week. And it offers some really nice capabilities to narrow some of the equity gaps that we have by providing low-cost assistance for students who may not have come into our institutions quite as well prepared. It offers a possibility of students doing a little bit more retrieval practice for those classes where instructors are not providing those opportunities. It offers students who, again have a somewhat weaker background, to take more complex readings that they may have been assigned and simplifying it and creating a more accessible format to help students get up to the level they need in their classes. And so it provides tools that can help students improve their writing, and so forth. The challenge, of course, is that it can also be used as a substitute for learning in some classes unless assignments are designed in a way, and assessment techniques are designed in a way, that reduces the likelihood of that and that’s one of the things we’ll be working on a lot this year, ways of coming up with more authentic assessment and ways of providing more intrinsic motivation for the work that students are doing, so that students can see the value of the learning rather than focusing entirely on grades. And one of the things we’re doing is we have a reading group coming up early this semester on Grading for Growth by Robert Talbert and David Clark. And we’re, in general, encouraging faculty to at least consider the adoption of alternative grading systems, which shift the focus away from students trying to maximize grades to maximizing their learning. And there’s a wide variety of tools that could be used for that, ranging from mastery learning quizzing systems, which many faculty have already been doing through specifications grading, contract grading, labor-based grading, and also ungrading.

Rebecca: Yeah, it’s really exciting that we’re going to focus on that semester, something that I’ve been interested in for a long time and have been using in my classes as well. I’m glad you mentioned AI, there’s certainly a lot of promise, and I’ve been really excited by how many faculty, staff, administrators who’ve actually really been engaged in the conversation around AI. I think sometimes there are new innovations and things and people kind of brush it aside and don’t always think it applies to them or isn’t relevant to them. But I think this is something that’s relevant to everybody. And most people are seeing that and engaging in the conversation, struggling in the conversation, but at least we’re doing that in community. And I think there’s some power in that as we think through policy, in assignments and all the things that we need to think about to provide an enriching experience for our students, but also engage and use the tools and the power that they offer.

John: One other thing I’m following is the development of a wide variety of new edtech tools. We saw an explosive growth in the development of tools and expansion of their capabilities in response to the COVID pandemic, but that growth and expansion hasn’t dropped. And we’re seeing more and more tools that have often been designed based on research about how students learn. And I think we’re going to see expanded use of many of these tools in the coming year.

Rebecca: And I think a lot of faculty got used to experimenting with these tools during remote education and are continuing to use them in physical and virtual classrooms, which I think is really exciting, maybe even more exciting to me was attending big conferences like Middle States and actually having a presenter use some of these edtech tools as part of a plenary. So rather than having more of a lecture style session, it was more of an interactive session, which doesn’t always happen at conferences of such scale, or these more leadership conferences. So it’s exciting to see that we’re modeling some of these practices at the highest level so that the wide variety of individuals involved in higher ed are experiencing learning and engaging with these kinds of tools. Along the same lines, I’m also excited that many of these tools are starting to actually attend to accessibility, in part because higher ed institutions are really pushing back to third-party providers and requesting them provide information about accessibility, and even refusing to adopt tools if they aren’t meeting basic accessibility principles, which I think is really excited and really important.

John: And I saw something very similar both at the POD conference where you might expect to see people creating more interactive workshops, but I’ve also been seeing it in the workshops that we’ve had in the last couple of weeks here. We have a record number of faculty presenting in workshops, they’re using polling, they’re using tools like Mentimeter and they’re doing many more interactive activities than in past years. If we go back a few years, many of the sessions that were presented were essentially straight walkthroughs through PowerPoint slideshows with not a lot of interaction with the participants. And our workshops here have been both in person and remote over zoom. And people have been working really effectively to bring all the participants into the discussion and into the activities, regardless of whether they are in person in the room or remote. And it’s been nice to see that. Much of that I think did grow out of the experiences of COVID, and people just getting more comfortable trying new tools.

Rebecca: Come to find out, practice helps us learn things.

John: Also, our campus enabled the AI Companion in Zoom, which will provide meeting summaries for people who arrive late or people who come in at the end of a discussion. And I think that’s going to offer some nice opportunities for people who may have missed part of the discussion early on in a session, or in a workshop, or in a meeting, because so many of our meetings now take place over Zoom.

Rebecca: And there’s lots to be watching that are also highly concerning, but John and I resolved that we weren’t going to focus on those today.

John: So this will be a relatively short episode. [LAUGHTER]

Rebecca: So continuing on this theme of gratefulness and positivity, John, what are you looking forward to trying this year, or focusing on in your own work this year, or committing to this year?

John: One thing, and this partly follows up a couple of podcasts we’ve had in the last year or so. In our campus, many departments are working to build in some of the NACE competencies into their classroom. And there’s some really significant advantages for that. If it’s done well, it will help students recognize the intrinsic value of the things they’re learning in class and recognize that these are skills that they’re going to need later, which again, helps provide much more motivation for students to learn than if they just see a series of activities that instructors ask them to do, and they don’t see the value of that. So by making the connections between what we’re doing in the classroom in terms of the development of critical thinking skills, teamwork, and all those other NACE competencies, it offers some really serious benefits for students and for faculty. Because if students are more engaged in the activities and understand the purpose of them, I think they’re going to be much more likely to focus on the learning rather than again, trying just to get the highest grade. And that’s also very consistent with the TILT approach that we mentioned earlier. If students understand why you’re doing things, they’re going to receive the techniques and engage in them more productively than if they didn’t see the value of those tasks.

Rebecca: Yeah, I’m glad that you mentioned TILT as well. We mentioned it earlier, but I was just remembering that one of the things I wanted to mention while we were talking today is a commitment to thinking about TILT, not just in a classroom context, but all the other places that touch a student experience. So thinking about policies and procedures and ways that we can use a TILT approach to really improve transparency and clarity for our students and provide some equity and access by doing so. The other thing that I’m committed to trying to do is get back to more play, we’ve had some episodes on Tea for Teaching focused on play. And they always get me really excited about some of the things that I’ve done in the past in some of my classes and that I’ve done with some of my colleagues… and that the burden of transitioning during COVID to remote learning, some of these things have taken time and maybe attention away from play. And I’m hoping to take some time in 2024 to put some more attention back on being a little more playful.

John: So, you think education could be fun?

Rebecca: Maybe.

John: Okay. [LAUGHTER]

Rebecca: I’ve moved to doing some exercises and activities again a little more recently that get to some of these more playful ways of creating and making and thinking through complex problems. And every time I do that the students appreciate it. I have more fun, they have more fun, and I think a lot more learning gets done.

John: Since we want to focus on the positive, we’ll leave challenges for future episodes.

Rebecca: We’ve got all of 2024 to do that, John.

John: And we really appreciate, as Rebecca said, all of the wonderful guests that we’ve had since the beginning of this podcast, and we appreciate our audience too. So thank you for hanging in there with us.

Rebecca: Have a great 2024.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

324. Unmaking the Grade

Posted in blogging, engagement, feedback, grading, higher education, learning, motivation, pedagogy, reflective practice, teaching, ungrading, writing
January 17, 2024

A growing number of faculty have been experimenting with ungrading. In this episode, Emily Pitts Donahoe joins us to discuss her ungrading approach and the documentation of this process on her blog. Emily is the Associate Director of Instructional Support at the Center for Excellence in Teaching and Learning and Lecturer in the Department of Writing and Rhetoric at the University of Mississippi.

Show Notes

Transcript

John: A growing number of faculty have been experimenting with ungrading. In this episode, we discuss one instructor’s ungrading approach and her documentation of the process.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Emily Pitts Donahoe. Emily is the Associate Director of Instructional Support at the Center for Excellence in Teaching and Learning and Lecturer in the Department of Writing and Rhetoric at the University of Mississippi. She experimented with ungrading and chronicled her experiences in her Unmaking the Grade blog. Welcome, Emily.

Emily: Thanks. I’m really excited to be here.

John: We’re very pleased to be talking to you. We talked recently at the POD conference, and I’m looking forward to this additional conversation.

Emily: Yes, where I got this lovely tea for teaching mug from John, which I’m so excited about and drinking from right now.

John: And since this is only audio, Emily was holding the mug [LAUGHTER]…

Emily: I’m showing it off.

John: So our teas today are:… Emily, are you drinking tea in that mug?

Emily: I am. [LAUGHTER] I am drinking tea out of my tea for teaching mug. I’m a big tea drinker. And so today I’m drinking my favorite tea, which is a strong black tea called Scottish morn. And I got this tea from a tea shop called Apothica in Niles, Michigan, which I used to go to when I lived in South Bend, so highly recommended if you’re in that area.

Rebecca: Sounds wonderful. I was ready for you to say that you had coffee or something in the tea for teaching mug so it’d be completely blasphemous… [LAUGHTER]

Emily: Never.

Rebecca: …because coffee is one of the most frequent flavors. [LAUGHTER] Emily, you’re doing it right. [LAUGHTER] I have blue sapphire tea today.

John: And I have Irish Breakfast tea today.

Rebecca: So, we invited you here today, Emily, to discuss your experiences with ungrading. Your blog is based on your spring 2023 course experiences but your ungrading experience predates this course. Can you tell us a little bit about your initial experience with ungrading?

Emily: Sure. So I have a background in Writing and Rhetoric and English literature, and so a lot of what we do in Writing and Rhetoric, I think, is already pretty aligned with some ungrading goals and practices. So in my courses, students have always had the opportunity to revise their work based on feedback, and include it in a final portfolio, and most of their grade is based on their revised work. So it’s not just that they get graded and then that’s the end of it, they always have a chance to improve based on feedback. The first time that I stopped putting letter grades and percentages on student work was in spring of 2022, when I was teaching a general education literature course at Notre Dame, and so the course was on pre-modern and early modern literature, but with a focus on how the texts that we were studying might help us examine or wrangle with some of the questions that we’re preoccupied with as a culture and society today, and thinking about how those texts might relate to student lives. So it was a course for kind of all levels and all majors, which I think made it a good course to experiment with. So I had first-year students all the way up through senior-level students from engineering and business and English and psychology and all kinds of different majors. So like all teaching experiments, I think there were definitely some kinks to work out after that first course; they never go exactly right the first time. [LAUGHTER] But I would say that, overall, it was a huge success. And that’s not because I did it perfectly, or even because I did it particularly well, but because I think ungrading really helped some of my students move beyond this idea of the school as a points game, to help find their interest and their motivations to study the material that we were studying for their own purposes, and to focus on developing the skills and knowledge that they wanted to develop rather than on attaining a specific grade.

John: And the second time you did this, you created a blog describing the process. What prompted you to create the blog?

Emily: So, I was inspired by a post written by Robert Talbert, who’s the co-author of a recent book, Grading for Growth, which I really highly recommend on alternative grading, and he and David Clark, his co author, also had a blog and substack newsletter on Grading for Growth. And so in December of 2022, Robert posted a stop, start, continue for the ungrading community. And if you’re not familiar with stop, start, continue, it’s an evaluation exercise used for mid-semester reflection, very often in classrooms where the instructor or another facilitator will ask students: “What kinds of teaching practices or classroom practices would you like to stop, start, or continue in the class?” And so Robert’s post was a stop, start, continue for the ungrading community. And one of the things that he recommended that the ungrading community start doing was getting into the weeds and writing in detail about the daily experiences and specifics of upgrading, so: what we’re doing, what kind of successes we’re having, what challenges we’re encountering, how we’re adjusting in real time, and he recommended keeping this as the kind of blog or like a captain’s log of weekly reflections. So when I read his post, I thought, “Well, I could do that. It didn’t sound that hard, and it sounded like a lot of fun.” So that was what prompted me to start writing weekly reflections to share some of the methods that I was using, successes and challenges that I had, and even some of my doubts or misgivings about some of the things that I was doing in the class.

John: And just as an aside, we were so impressed by Grading for Growth, that we’ll be using it in the spring reading group here, both at SUNY Oswego and at Plattsburgh with Jessmyn Neuhaus. So we’re very much looking forward to that. And Robert and David will be giving a keynote address at the start of our workshop series in just a few weeks in early January. So we’re very interested in doing more of this on our campus as well.

Rebecca: Can you talk a little bit about your decision to delay posting the blog until after the semester ended?

Emily: Sure. So the selfish reason is that it’s supposed to be an unfiltered look at the ungraded classroom. And part of me thought, what if I messed this up so badly that I’ll be embarrassed to talk about it. So it was partly that, but I would say the more important reason is that I thought it might be weird for students if I was talking in public about what was happening in the classroom in real time. And I was concerned that it would damage the relationship of trust that I wanted to build with my students if they knew that I was reporting on our conversations to a larger audience about the class. So I offered to hold things for that reason. But the other thing that I wanted to do was make it clear to students that I was writing these reflections that might later be published, and then to get their input on them. So some of what I shared on the blog, I didn’t really want to share without getting students consent to do that. But I also wanted to get their input on some of the things that I was talking about. So I believe really strongly in taking a students as partners approach to higher education and learning experiences. And I try to employ that in my educational development work in the classroom. So that means bringing students into conversations about teaching and learning and asking for their experiences and their expertise as students really and then using that to inform our practice. So delaying the posting allowed me also to get some input from students and to be able to share some of their thoughts and opinions on the blog whenever I could.

John: In reading through your blog, your spring 2023 class seemed really interesting. Could you describe that for our listeners?

Emily: Yeah. So this is the second course in our sequence of first-year writing courses here at the University of Mississippi, but it’s a little bit different. So for the second course in the sequence, students have the opportunity to take either Writing 102 or Liberal Arts 102, and I was teaching Liberal Arts 102. And so the goals of each course are really the same. But Liberal 102, as we call it, is conducted within the context of a research area in a specific discipline. So that means that it can have a very specific theme. So some of my really excellent colleagues in the department and in other departments have taught courses like writing about true crime, or the rhetoric of sports, or I believe there’s a course on fashion. So because one of my areas of expertise is teaching and learning, the course that I taught was called Examining Higher Ed Teaching and Learning in a College Classroom. And so I think this is a great course for first-year students as they’re entering this next phase of their education to reflect on where they’ve been and where they want to go. And so what we did was looked at questions like: What’s the purpose of a college education? Why are we all here? What kind of benefits does it provide to individuals or to society? What kind of collective benefits does it provide? And then how are those benefits enacted or engendered in the classroom. And so we explored a lot of debates around higher ed in the US and students have an opportunity to reflect on and draw on their own experiences as students and their expertise as students, and then integrate that with larger areas of research on education or current events in education. And then they communicated their ideas about education to audiences outside of our classroom. And so it was really, I think, an ideal course for ungrading because we could talk about grades, not only as a matter of course policy, but also as a core subject matter. So in the beginning of the semester, we read Alfie Kohn’s piece “The Case Against Grades” and talked about it both as a way to introduce the course grading system and as a kind of larger prompt for reflection about grades as an issue of concern in higher education right now.

John: And it sounds like a great opportunity to have students reflect on what you’re doing as you’re doing it. How did students respond to this?

Emily: So I think every time I’ve ungraded a class now, and I’m currently on my third ungraded class, students have responded a little bit differently every time and of course, every individual student is also different. I think there are three broad categories of response that I see. One is just enthusiasm; some students are really excited about ungrading, and usually that’s students who feel that, for one reason or another, their grades in the past haven’t been representative of their learning or who feel that some of their creativity or their risk taking has been stifled by their desire to get a good grade. So I wouldn’t say this is a lot of students, this is probably a smaller group of students, but some students are really excited about it. I would say another group of students are really hesitant because ungrading is a big unknown for them, especially for students who are dead set on getting that “A” grade. It can be really nerve racking not to know kind of where you’re at in the middle of the semester. When I get negative feedback on ungrading, it’s almost always students who say,”This is really interesting, but I never know where I’m at.” And so they’re really concerned about how they’re going to measure their progress. But they’re not thinking about their progress in learning, they’re thinking about their progress toward a specific grade, which is really understandable because grades are important. And so that’s also a smaller group of students. But I would say, by far, the biggest reaction I get from students is something like cautious optimism. So I always start ungraded courses with a conversation about grades and learning, and I ask students to share with me their experiences of grades and share with each other. And very often, they have a lot of negative experiences to share, and sometimes positive experiences as well. But we talk about the relationship between grades and motivation and grades and learning and they have a chance to reflect on that. And so I use these conversations as a jumping off point to explain why I use the system that I do and how I think it will benefit them as learners. And I think students find it really helpful to be able to talk about their experiences with grading openly and to be heard by a teacher. And I think that that alone makes them more willing to buy into the system. So once it’s explained fully, and once students start to see the potential benefits, I would say that most of them are cautiously optimistic.

Rebecca: I think that largely aligns with my experiences and explorations when ungrading as well. In your class, you included five different assessments and opportunities for revision. Can you talk about and describe these assessments?

Emily: Sure. So the first thing that students had the opportunity to do were weekly writing practice assignments. So these are what you might think of as formative assessments, they’re preparation for class discussion and major assignments that mostly involve reading or short writing prompts. And these students weren’t able to revise, they either kind of did it or they didn’t. But then students had an opportunity to do more longer major assignments, which are more typical assessments for the writing class, a series of papers and a multimodal kind of project or two, with an imagined audience kind of outside our classroom. And so these students could revise up to three times if they wanted to do that. And so I had to, for this class, adjust a little bit and do slightly fewer assignments than I might have done in a traditionally graded class, because the expectation is that students would be doing more intense work on each assignment in their revision. I also asked students to do self assessments periodically throughout the semester. So students would answer a series of questions about their progress toward the learning goals, about their goals for the remainder of the semester. And they would also propose a current grade for themselves based on evidence that they provided. And this is a similar activity to what we did at the end of the semester to determine their course grades. They were also assessed on their final portfolio, which had revised versions of their major assignments. So their work… basically as good as they could make it… their best version of their major assignments, plus another final self assessment of their work in the class. And then they were also assessed on class engagement, so their class attendance, their preparation for and participation in class discussion, the timeliness of their work, their support for the learning community and fellow students and things like that.

Rebecca: Can you talk a little bit about how you provided your feedback on those assessments?

Emily: Sure. So for the writing practice assignments, usually I just wrote a sentence or two in response to students work to let them know if they were on the right track, or they may be needed to adjust slightly as they prepare for their major assignments. For the major assignments, I did do a lot of longer written feedback, and a lot of this looked like feedback that I would give in a traditionally graded class. But I think it was a little bit more oriented toward future growth, rather than reflecting on past work. It was giving them a lot of direction on how they might want to revise to make their work better in the future, not just for future papers, but for this particular paper. And with that feedback, I also gave them a rubric, where I marked kind of where I thought they were at along the specific assessment categories and criteria that we were using for that assignment. So for example, we might have a category for audience and purpose. Students could see whether I think their work on that category was still developing, whether it was at the level of proficient or whether it reached the level of excellent, so I had these different categories and these different kind of progress metrics, where I would indicate this is a skill that I think you’re still working on, or this is a skill that you’re doing really well in. And I also provided feedback to students through our individual conferences. So we met once at midterm, I met with each individual student, and then once at the end of the semester, to have a face-to-face conversation about how students were doing and if they needed to make any adjustments to reach their goals for the rest of the semester. And I didn’t give any feedback through letter grades or points. Those are not really great forms of feedback. They don’t actually give us a lot of information about our progress. So most of the feedback that I provided was kind of qualitative feedback.

Rebecca: You just talked a little bit about rubrics as part of a feedback that we were providing students, can you talk a little bit about how those rubrics were developed?

Emily: Yeah, so I’ve tried several different methods of creating rubrics. And one of the things that I like to do is co-create them with students or get students input on rubrics. And I attempted to co-create rubrics with students in two different classes and have done it a few times. And sometimes it’s a huge success, and sometimes it really does not work very well at all. And in my spring class, it didn’t work very well at all. [LAUGHTER] So basically, what I’ve done previously is ask students what they think makes a good argument or a good piece of public writing, or whatever it is that we’re working on. Or I ask them what kinds of things they believe their writing should be assessed on, and how we would determine whether or not their writing is successful. And then we have a conversation about that. I make notes on the board as we’re talking. And then I go away and distill those notes down into a rubric with about five categories in which student work is assessed. And then there’s the three metrics, so developing, proficient, and excellent. And sometimes this works really well. But in my spring class, I think the students were really at a loss when I asked them about assessment, and even when I asked them what good writing looks like. And I think that’s because that no one has ever asked them to think about that before. They’re used to being told what their work should look like by someone else, and then trying to conform their work to those expectations, or to someone else’s standards. And obviously, we need to have standards and expectations for student work. But one of the things that that does is get the students only thinking about what the teacher wants and not what they want. They’re not thinking about what they want their work to look like, or what their standards are or what they’re trying to accomplish. And so I think, ultimately, if we’re preparing students for the real world, whatever that is, this is what we want them to be able to do, not just to blindly follow somebody else’s standards, but to create and work independently in a self motivated way, and then assess their work independently. So I think we have to kind of start with baby steps in that process. So in the spring course, the rubric activity really turned into a conversation about how students have been assessed in the past and why they’ve been assessed that way. So when I ask them about their experiences, how their writing had been assessed, they would say things like, we were graded on the word count, or we were graded on whether or not we included two sources in every paragraph. And then we use that as a jumping off point to talk about the principle behind those what they viewed as arbitrary rules. And so if your teacher is concerned about word count, it’s because they want you to make a substantive argument. And what does a substantive argument look like? If your teacher is concerned about including sources, it’s because they want you to make an argument based on strong evidence. And so what would strong evidence look like? And so we use that conversation to think about the assessment criteria for their college level work, and how that might be similar or different from what they experienced in the past. So in that way, I do think that conversation was helpful. And I do think based on my few experiences, that it helps to have students extrapolate criteria from some examples, rather than from thin air, kind of as I was asking them to do, but either way, having a transparent conversation about what students are being assessed on and why and a purpose behind the criteria is key to their success.

John: In one of your blog posts, you describe some of the writing assignments that you used in your class, and they seem quite interesting. Could you describe some of the writing prompts that you gave to students?

Emily: Yes, I was really excited about some of the assignments that I developed for this class. And I think I’m going to keep using them or versions of them in future classes. So for this class, I had a menu of assignment options. So the students could, for each major assignment, choose the kinds of projects that they found most compelling, or they would really draw on your strengths. And I thought this was important, partially because we were working some with generative AI in the class. And I’m sure everyone is aware by now some very serious data privacy concerns, ethical concerns around the use of generative AI. So I didn’t want to mandate that students work with AI if they didn’t want to, and some really didn’t want to. So I tried to offer them several options. And I was really happy about some of the options that I came up with. So obviously, one of the things that I was thinking about going into this semester was how to deal with generative AI and I have a lot of thoughts about that. But one big way to think about how to discourage inappropriate use of AI or encourage appropriate use of it is to think really carefully about assignment design. So one of the things that I did to help students navigate AI was lean into some multimodal work, think about argumentation in media other than writing or that kind of worked alongside writing. So we didn’t abandon writing entirely, but I did include a photo essay as one possible assignment in a menu of options. And for this course, I asked students to take a look at a book of photographs by Cassandra Horii and Martin Springborg that’s called What Teaching Looks Like. It’s a really, really cool book with candid photos of college teaching. And so we use this book to talk about visual rhetoric and also to use the photos as a launchpad for this discussion about college teaching practices and students’ experiences in the classroom. And so the assignment that students did based on their work with this book was to create their own photo essay called “What Learning Looks Like.” So instead of what teaching looks like, what learning looks like, with a specific audience and purpose in mind. So that was one of the assignments that I liked. Another one that I thought was really promising was something called “share your story,” which asks students to tell a personal narrative about their educational experience, and then connect that to a larger issue of concern or a body of research in higher education. So of course, this is not totally AI proof. ChatGPT can make up a story, but it can’t tell my students’ own stories. And I think most of us really like to tell our stories to other people. So I think that provided a little more motivation for students to do their own work. And then the last assignment that I really liked was an assignment that actually asked students to work with ChatGPT. So they created a prompt based on a template that I provided for an argumentative piece that they then fed to ChatGPT, so they gave ChatGBT the prompt. And then they took the ChatGPT output and critiqued it. So they annotated it, noting what pieces they thought were strong, whether or not the piece had weaknesses, and what those weaknesses were, where it might need revision or overhaul. And then they had to totally rewrite the piece that ChatGPT produced and make it their own. So they had to do a substantial revision. And then they had to annotate their own work and tell me why they made the revision decisions that they did. And so I want to clarify that I do think it’s important that students learn to generate first drafts on their own, because drafting is an essential part of the writing process, and it’s where a lot of the thinking happens. But I like this assignment as just one assignment in the sequence, because it does help students learn about generative AI and develop some AI literacy. And it also helps students get over that terror of the blank page. So a lot of students procrastinate because they don’t know how to get started. They’ll open their laptop to begin a paper and stare at the blank screen. And I find it really difficult to get over that first hump of starting work, and just close their laptop and go away and try it again later, usually the night before it’s due. So I think starting in this way, with a ChatGPT prompt and an essay gives them a jumping off point, and it’s an easy way to help them start building the confidence they need to do their own first draft.

John: We should mention that we did talk to the authors of What Teaching Looks Like, and we’ll include a link to that discussion in the show notes. One of the things you described in your blog is that having this ungraded environment encourages students to be perhaps a little more open and honest with their instructors, but that could lead to some challenges in terms of additional emotional labor. Could you describe the challenges that you faced in this class with that?

Emily: I think one of the big themes in the spring class in particular was that I asked students to share with the class and for audiences beyond the class about their previous experiences in school, or their current experiences, and I think this also happens in a lot of ungraded classes. One common method of introducing an unfamiliar grading system is getting students to think about their previous experiences with grades. And so one thing that happened in this particular class is that students’ related a lot of past educational trauma to me, and usually that involves bad experiences with previous teachers. And I’m really glad that they were able to speak honestly about that sort of thing. And I think it helped them to have a teacher take them seriously when they related those stories. At the same time, it was pretty difficult to navigate those conversations, because I was managing their emotions, my emotions, and also doing that when sometimes I only knew one side of the story. And so that was a little bit difficult. Another thing is that students in this class didn’t seem to feel that their grade depended on telling me what I wanted to hear. So I think they were a lot more honest about their views than in other courses that I’ve taught. So in my traditionally graded classes, or I think, in any class where students are discussing hot button issues, they tend to think that they’ll be graded more harshly if they express views that the instructor disagrees with. So very often, I think, they try to say what they think you want to hear rather than what they really think. And that didn’t seem to happen as much in my spring course. I had several students endorse viewpoints that I definitely disagreed with whether they knew that or not. And I think that’s good, because students have to sort through their own views and values. But it also required me to do a lot of thinking about how I would approach and address students not just whose views I disagreed with, but whose expression of those views might be damaging to others or to themselves. And so, we need to do some work around community building and relationship building in the class. And then of course, when you build relationships with trust with students, they’re more likely to tell you about the personal problems that they’re facing, whether that’s their mental or physical health or their personal relationships or family emergencies or grieving. And it’s really just a lot. And I do sometimes lay awake at night worried about my students. And there’s a lot of care work happens, I think, in all classes and also especially ungraded classes. And so there’s a lot of work in referring students to other resources and helping them navigate campus resources, and also just a lot of kind of management of your own emotional state [LAUGHTER] that has to happen. So I did want to be honest on the blog about some of the emotional toll of the work of teaching in general and of ungrading too.

Rebecca: You’ve described the care work, you’ve described individual conferences with students, you’ve described students’ anxiety over not knowing where they are sometimes and allowing substantial revision. Can you talk a little bit about how all of those things play into workload and how you’ve managed things?

Emily: Yes, this is a good question. And I feel like I have a very complicated answer to whether or not ungrading has increased my workload, because I get asked this question a lot. In my traditionally graded classes, giving all that feedback felt like a waste of time for a few reasons. First, because I had the sense that students weren’t really reading the feedback. So we know from research that when students receive a letter grade and also feedback on their work, they tend to see the grade and then ignore the feedback, or at least receive the feedback as a justification for a grade, rather than something that’s going to help them improve their work in the future. So I also spend a lot of time in traditionally graded classes worrying about how students would receive my feedback if they got a low grade on their paper. So how could I write feedback that would be appropriately honest, but also appropriately encouraging, so that students didn’t just see a C-minus or a D on their paper, and then give up and throw it in the trash. So I don’t really worry about those things since I’ve adopted ungrading. I provide feedback honestly, and with the mindset of a coach rather than a judge. And I provide it with at least some confidence that students will read it and use it. So they have plenty of opportunity to revise. And in fact, their ultimate achievement of the course is measured by their growth in specific areas and their demonstration of learning that arises from taking a piece of writing from not so good to much better. And so the expectation is that what they turn in the first time is not their best work and they’ll only get to their best work after they incorporate feedback. So I do spend a lot of time responding to student work now that I’m ungrading. But the process is more efficient because it accomplishes my goals rather than wasting my time. And it’s more enjoyable, because it causes me less angst. So I guess it is more work to provide feedback, but it’s also more efficient and enjoyable work. I think I feel kind of the same way about individual conferences with students, that it does take quite a bit of time to do those and it would be, I want to acknowledge, so much more difficult, it may be impossible to do if I was teaching more classes and more students. I’m very lucky that I teach small class sizes. And because I work in educational developments and work in a teaching center for most of the time, I only teach one course at a time. So I think there are ways to do this with larger courses, but I’m very fortunate in my course to be able to conference individually with each student twice in the semester. And those conferences are incredibly time consuming, and they can be really draining, but they are also really joyful. And I think it’s really important that students are able to have those one-on-one conversations with me. And they are much more, I think, effective in accomplishing a lot of the goals that I have for student learning than just simply doing written feedback or peer review or things like that. Having that face-to-face time to give students some individual attention is a really both enjoyable and effective part of the learning process.

John: A question that often comes up from people who have not tried ungrading is how well do students’ perceptions of their learning align with your perceptions of the learning when you do have to assign those midterm or final grades in the class?

Emily: Yeah, this is a really good question. And I would say I’ve had to do a little bit of work on my process to make sure that our expectations are aligning well. Sometimes, there are cases where students don’t automatically start off knowing what I expect from their work and what good work looks like, even when I thought that I was clear about that. So that is an issue that I’m working on. I would say for the majority of students, they do understand what good work looks like. And when they’re asked to provide evidence for their course grade, most of the time they know what good evidence looks like and are able to demonstrate to me in really, sometimes ways that I hadn’t anticipated, that they really had learned in the class and progressed and improved their work. For those students who struggle, and I think it’s more frequent for first-year students to struggle with this than more advanced students, for students who do struggle, I think it is important to be able to show them models of student work early in the semester so that they can get a sense of what a successful assignment looks like or what’s kind of level of expectations we have for what student work counts as really excellent work. So that’s been really helpful. And I’ve also made some changes to my class this semester to help clarify for students a little bit what kind of evidence might be good evidence for specific grade proposals in the course. So if they are really shooting for an A or B in the course, what kinds of actions or behaviors or demonstrations of quality work do they need to be able to show in order to attain that grade?

Rebecca: I know one of the things that I struggled with in some of the classes where I’ve done ungrading is that sometimes the midterm conference feels like it comes too late. So I experimented, I think once with like a one-third, two-thirds, three thirds approach, but then that was so much more conferencing.

Emily: Yeah.

Rebecca: So I’m curious about your timing and how the midterm time works in the classes that you’ve been teaching.

Emily: Yeah, I think that’s where the self-assessment assignments come in. So I do think midterm is quite late for students to be getting the first level of feedback. And so I have students do a self-assessment form, I guess, a quarter way into the semester. So we start that process really early of having them look back at the work that they’ve done, and propose a grade for themselves based on that work. And when they do those self assessments, I don’t conference with students every time they do a self assessment, but I do look at where their grade proposals are. And I’m able to say if our expectations are very, very different, or assessments of that students work is very, very different, I’m able to reach out and say, “Hey, I don’t think we’re aligned here, and here’s why.” And being able to intervene early on that I think is really important.

John: We’re recording this near the end of the fall semester of 2023. And you’re currently teaching another ungraded class, could you tell us a little bit about what types of changes you made from the spring class to the fall class?

Emily: Sure, there are a few changes that I’ve made, because there were some things that I think did not go very well in the spring and I wanted to try to improve that. So what didn’t go well was that I had a real problem with attendance, there were quite a few students who struggled with their attendance, especially in the latter half of the semester, and quite a few students who struggled to submit their work on time kind of throughout the semester. And that made it difficult for me, but I think more importantly, it made it really difficult for the students who once they missed an assignment, they found it really difficult to get back on track. So my challenge is really to figure out how to motivate students to attend class regularly and submit work on time without penalizing them for absences or late work or without a kind of point system to encourage them to show up to class and to submit their work on time. So that was one thing I wanted to address. And then the other thing is that students’ anxiety about not knowing where they stand in the midst of the semester at any given point. So what I developed to address all of those challenges was a course progress tracker. And so this is a document, a pretty comprehensive document, inspired by David Clark’s Grading for Growth post about grade trackers that I read during the summer while I was designing this course. And so the document is really a series of tracking worksheets in three different categories. So the first category is readings and assignments, students can see each week at a glance what work they have to complete, and then they can check off boxes as they complete that work, and then note, if there are assignments they’re submitting late, they can record those late submissions. There’s a category for attendance and engagement where students can check off the classes that they attend, and then make notes about their in or out of class engagement during each week of the semester. And then there’s a section… the most important section… for learning and growth where students can remind themselves of what the course goals are, and then track their progress along those goals, so they can see and make notes about where they’re still developing, where they’re doing excellent work, and how they’re improving their writing as the semester goes along. So, so far, it’s been going really well. And I’ve had fewer challenges with attendance and late work this semester than last semester, though, I can’t say to what extent that’s just a result of a different population of students or the fact that it’s fall instead of spring. So I will add that caveat, but I have surveyed students about their use of the progress tracker last week, and I’m really looking forward to diving into that next week. So the last thing that I think the progress tracker does, which I didn’t totally intend, but which has been really excellent, is that I think it helps students a lot with their self-assessment work, which they also struggled with, to some extent last semester, or I should say, maybe I struggled to teach really well. So I had a realization at the end of last semester, that when I sat down to think about a student’s work over the course of this semester, I was really thinking about three things. And so the first thing is quantity. How much work did students do? How much labor did they put in? How many assignments did they submit at a satisfactory level? How many class days did they attend? How much time did they spend on their major assignments? So that was one category. The second was quality. So how good was the work that they were doing according to the standards that we laid out? Was it still kind of developing work? Or was it excellent work? And then the third thing was growth. So how and how much did student work improve over the course of this semester? And can they demonstrate that they’ve gained knowledge and skills that they didn’t have before? Or can they demonstrate that they’re better off from having taken the course. And so while we’ve had conversations about those things in the spring, I never articulated to myself or to my students, that particular model; I never kind of said it in quite this way. So the last section of the progress tracker includes a guide to determining final letter grades for students. And it gives them space to think about quantity, quality and growth, and how that might contribute to the grade that they propose for themselves, if they’re interested in the specific letter grade. And it gives them a sense of what kinds of evidence they could provide if they want to show good evidence for a specific letter grade. So you can provide good evidence for an A if your work rises to the level of excellent on multiple assessment categories. Or if you attended class every time that you were able and submitted the vast majority of your writing practice assignments. So all of these things are good pieces of evidence if you want to propose an A grade for yourself. So there’s some flexibility there, I don’t prescribe exactly what students have to do for a specific grade, but I do make suggestions and say, here’s a guideline for you. If you’re confused about what kinds of work you need to be doing, or what you need to do to demonstrate that you’ve attained a certain letter grade, you can take a look at this guide, and learn a little bit more about what the expectations are. Currently, it’s just something for the students to use for themselves. I have provided time in class for students to fill out sections of it, because I think that if I did not ask them to do it in class, they might struggle to keep up with it outside of class. But I think maybe in the future, I will consider asking them to keep up with it. And then periodically checking in on those progress trackers throughout the semester so that I can intervene early, if there are any issues and maybe leave comments on the documents to let students know if I think they’re struggling in a specific area, or if they’re doing really well in a specific area. I am playing around with that idea for my next version of the class.

John: So we always end with the question: “What’s next?”

Emily: So this spring, I won’t be teaching an undergraduate course, but I will be working with graduate students who are preparing to teach their own Writing and Rhetoric courses. And I’m really looking forward to talking with them about teaching. And I’ll also be continuing to write about ungrading on the blog. So specifically, I’m hoping to share a bit more about what I learned from my experiences this semester. And I’ve been collecting data from my current students about their use of the progress tracker, about their use of AI in their writing this semester, and about their feelings and impressions of ungrading. And what I’m planning to do throughout the spring is share some of those student thoughts with the readers of the blog, and I’m really excited to be able to share students’ ideas about ungrading and other topics as well.

Rebecca: Sounds great. We’ll look forward to reading that for sure.

John: Thank you, Emily. It’s great talking to you and we look forward to future conversations.

Emily: Thanks, this has been fantastic.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

323. Explore First Study Abroad Program

Posted in access, diversity, engagement, equity, first-generation students, higher education, inclusion, international education, learning, pedagogy, study abroad, support network, teaching
January 10, 2024

Compared to continuing-generation students, first-gen students experience a higher risk of not completing a college degree. In this episode, Sue Roberts, Marianne Young, and Beth Hanneman join us discuss a study-abroad program for first-gen students that is designed to build their confidence, sense of belonging, and help them understand the connection between their education and their career goals. Sue is the Associate Provost for Internationalization at the University of Kentucky. Marianne is the Assistant Vice President for Smart Campus Initiatives at the University of Kentucky. And Beth Hanneman is the Associate Director of Career Advising and Career Education, also at the University of Kentucky.

Show Notes

Transcript

John: Compared to continuing-generation students, first-gen students experience a higher risk of not completing a college degree. In this episode, we discuss a study abroad program for first-gen students that is designed to build their confidence, sense of belonging, and help them understand the connection between their education and their career goals.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Sue Roberts, Marianne Young, and Beth Hanneman. Sue is the Associate Provost for Internationalization at the University of Kentucky. Marianne is the Assistant Vice President for Smart Campus Initiatives at the University of Kentucky. And Beth Hanneman is the Associate Director of Career Advising and Career Education, also at the University of Kentucky. Welcome, Sue, Marianne, and Beth.

Sue: Thank you, we’re glad to be here.

Rebecca: Today’s teas are:… Sue, are you drinking any tea?

Sue: I am not drinking tea right now. But if I were in my normal space, I would be drinking tea. Yes.

Rebecca: Do you have a favorite kind?

Sue: I do. It’s a rooibos tea, a red bush tea.

Rebecca: Wonderful. Marianme, how about you?

Marianne: I’m not currently drinking too, but I have one on the ready. It is a lovely wild sweet orange.

Rebecca: Nice. What about you Beth?

Beth: So I went to Montana this past summer for a yoga retreat and fell in love with Huckleberry. So I now drink a wild Huckleberry tea at least once a week. And that’s what I’m having this morning.

Rebecca: I have never had that. I think it’s a first on the podcast.

John: It is. I am drinking an Irish Breakfast tea.

Rebecca: And I have a Jasmine Dragon Pearl this morning.

John: Dragon pearls?

Rebecca: Dragon pearls. [LAUGHTER]

John: Ok, so we have a mythical tea. [LAUGHTER] We read about the Explore First Study Abroad program in an article in the Chronicle recently. And so we invited you here to talk about that. Could you give us an overview of this program?

Sue: Sure. It’s a new program for the University of Kentucky, run for the first time in summer 2023. We took 60 students, 60 undergraduate students, all of them first-generation students to either London or Dublin for a three- week Education Abroad program focused on career readiness.

Rebecca: Can you talk a little bit about this program came about?

Sue: It came about in many different ways, actually. There were conversations happening on our campus for probably two or three years before COVID, even, among different colleagues, some in the Career Center wondering how they could produce really good career readiness, education abroad programming. So I’m in the office of first-generation student success thinking about how we could do a better job in introducing first-generation students to education abroad. And then in the International Center, in the Education Abroad office itself, there were lots of conversations about how we could partner with colleagues across campus to develop programs that would reach this demographic.

John: One of the things we were really intrigued about was a program that was designed to benefit first-gen students as well as providing those career readiness skills. Could you describe a little bit how this program integrates that career focus?

Beth: Yeah, I’ll take that one. So over in the Career Center, we have the national association called NACE and they have NACE Competencies. And so when you start thinking about any major that a student has, you want them to make sure that they have those skill sets. So when they actually get into the workforce, they’re able to be analytical, they’re able to have communication skills. And what I love about the first-gen program and Explore fFrst within that, is the idea of how do you do that in a global setting. And so when you start thinking about designing this curriculum, and me and Marianne had the privilege of helping to work on that. It was this idea of, we had at least find out what the foundation is that the students had. And so sort of thinking about block scheduling, where a lot of times professors may say we’re going to do one topic and then go to another, I did more of what we call spiral curriculum, where you introduce a topic, and we brought it in here in the United States before we left, so maybe it was resumes where they had to create a resume and work on a resume ahead of time. And then when we’re overseas, we re-introduced resumes to say, “How would you put this Explore First experience on your actual resume? …kind of the same when it came to networking. What does it look like to navigate, to connect with people? Okay, great. We do that here in the United States. But then how do you do that in a global setting. So it was one of those things where you can actually see that reinforced. I also thought that was really cool when it came to interviewing as well, of having that prep for the students within that area.

Rebecca: We have a couple of other episodes of tea for teaching that also talk about the NACE competencies that we’ll link in our show notes for folks that are interested in that particular detail. So we just talked about NACE. Can you talk a little bit about the benefits that students get from it being a study abroad experience?

Beth: Yeah, at least for myself, the world is definitely interconnected. And so what I loved about how we interacted with employers is that we first looked at those that were connected here to Kentucky, and then we reached out with those that would be overseas. So for example, we have a company named Alltech that’s located here in Lexington. And so our students on the Dublin trip was able to actually go see their location. So it wasn’t one of those things that “Oh, hey, we’re just randomly seeing a company.” We’re very methodical about how that connection is. Another company, Compass, for example, does food services for our University of Kentucky health care system. So it wasn’t one of those things of just like, “Oh, we’re just gonna go see a company.” It was one of those elements of not only is making this connection abroad, but how does this actually impact me back home. And we would actually talk about that. Students would be like, “Alright, yes, you see this in another country, how does cultural awareness make a difference? How does being able to navigate and learn about yourself influence who you’re going to be back at UK?” And one of my favorite questions we asked in the interviews that they had to do for one of the assignments was: “If you’re back on the college campus, and you run into the university president, what are two things you tell them about the program?” And it was really cool to hear from the students that like, “I didn’t think I could learn so much about myself in three weeks, let alone three months about a career,” and others have never been on a plane before to navigate what that looks like. So even if you go into the job market, most likely somewhere along the line, you’re probably going to have to travel somewhere and do connections, but to have that support to have other people with them, when they did it for the first time was really impactful.

Rebecca: I love that reflection question. [LAUGHTER] So many benefits to that.

Marianne: I think the global stage also provides a really unique opportunity to just boost confidence with these students, as Beth was talking about, some of our students had never been on a plane. And not only did they navigate getting on a plane, they navigated getting into and past customs and immigration and all of that. And then the most surprising piece to me, in terms of what this looked like in terms of that confidence boost is at the beginning of our core sessions with the students, they put together kind of a: “these are what my goals are currently.” And by the end of the program, not only physically the confidence that you saw as they stood in front of the class, and they were presenting what their new goals were, for some of the students it expanded the opportunities that they were considering. They had never considered what it might be like to be in a leadership position in a global organization, or for the other students, it solidified what it was that they were wanting to do. And that confidence that they had of “Yes, I’m on the right path,” I think came from them navigating situations that they hadn’t been in before. And then being able to connect with different companies and different leadership individuals within the company, who they could see like, “Oh, my gosh, they were first generation as well, and now they’ve moved abroad, and they have this position in the company,” and the confidence that came from being able to navigate an international city as well as “I have confidence in how I’m going to navigate my career pathway,” that was so amazing to see in the sestudents.

Sue: I will agree with that. I visited, I think, three of the four programs. The program was split into four different groups, two went to Dublin two to London. So there were 15 students in each and I think I’ve visited three of them over the course of the summer, and to see that confidence grow, almost hourly, was incredible. And I will say that I think it translates, we’re hoping anyway, that it will translate, into greater understanding and kind of sense of purpose as a student. So you can see the point of your degree, you can see the point of why you’re struggling through this or that course to make it through to graduation. And of course, we want to see good results in terms of retention rates and graduation rates.

Rebecca: I’ve had the opportunity to teach a couple of travel courses where I’ve had students who had never traveled before, some within the US and some travel abroad. And I agree that seeing the confidence growth in students is such a rewarding experience for the instructors as well as for the students. It’s a really powerful experience. But one of the things that I really love that you’re describing is this connection to alumni, and those really specific intentional connections between the businesses locally as well as abroad. That’s a really beautiful component of your program.

John: And one of the challenges that all colleges face is the relatively high DFW rates generally experienced by first-gen students. And by making clear to the students the salience and the relevance of the material that they’re learning, and how this can open up these possibilities to them in a very obvious manner that they may not naturally see, I can imagine this could be really transformative.

Marianne: I think one of the great moments we had as we were visiting one of our employer partners, as Beth was talking about the spiral curriculum, he had talked about LinkedIn profiles and been helping them and then we get to this employer visit. And they start talking about LinkedIn profiles. And it is almost the exact lecture we had given that morning. And students are turning around and saying, “Oh, my gosh, you told us that this morning, and here’s this employer saying this exact same stuff. You were right.” And so we revisited again, and then the next employer, and so it was the aha moment of “Oh my gosh, this is actually something that I’m going to need and I’m going to use as I navigate my career.”

Rebecca: Can you talk a little bit about what a typical day for a student looks like in this program.

Marianne: There was no typical day, but this will kind of give you some activities they may have been shuffled around a bit. We’d start off with maybe some class time reviewing the visit. Maybe we’d have the day before talking about the experiences that the students had, answering any questions they may have had, how they navigated a challenge, just really taking time to connect and then help them build on the information that we were presenting. Once we kind of move through class time, we might have visited an employer in the afternoon and done a site visit where we not only learned about the company but also some of our employers took us through exercises like design thinking at some of our tech companies, or walking us through interviews and resume reviews in terms of a mock opportunity if you’re applying to their company. And then we may also have a cultural visit in the afternoon, visiting a significant landmark in the area or learning about the history and the culture of the particular space that the students were in. And then in the evenings, our students would maybe get together and cook in the residence hall, or they might go out to dinner together. And so generally we had class time and some cultural visits or an employer visit.

Beth: Yeah, exactly what Marianne said, and when you think about over the three weeks, because we were there for three weeks with these students, the beginning part of the classes were more kind of prep and foundation to get them to know what to expect. And so we broke teams of three, and so we would have one person would be the person who was in charge of introducing the group, we had one person that was the photographer, one person that was in charge of the “Thank you” at the end for each site visit. And that was really cool to see him learn collaboration, but also kind of change up and like, “Oh, well, this person is really interested in architecture, so we’re gonna have him be the speaker for this one, and then I’ll do the photography. And this one over here, we as a group, we actually wrote the thank you note together of what that means to follow up within it, kind of expectations within it.” So that was really cool in terms of curriculum, and kind of how we set that beginning. And then closer to the end like Marianne said, was more like review: “What did you learn from it and reflection?”

John: Do students travel with faculty or staff from the University of Kentucky?

Sue: We structured this program so that each group of students was accompanied and led by two professional staff members from the campus. Typically, it was a person from the Career Center, and a person from the Office of First-Generation Student Support. So it gave the students oftentimes very familiar people who understood where they’re coming from, and the skills they brought to the table, but also a person with the career advice ready for them kind of as needed. So it worked really well, I think, to have those program leaders on the ground with the students. And we weren’t hosted by a university, although we did visit universities in both locations. But we worked with a education abroad partner provider called AIFS. And they have provided the classrooms, they assisted us find student accommodation, and they worked on us with the cultural visits.

Rebecca: I think I remember also reading that you did some work prior to going abroad and some coursework there. Can you talk a little bit about what that looked like?

Marianne: We had an opportunity and clusters with the students, we broke them into smaller groups to help prepare them for what to expect as they were preparing for their education abroad experience. And so we covered a variety of topics of what about your luggage? What is a carry on? What is it checked bag? How do you move through security? What can you expect in an airport? What can you expect in terms of customs and immigration? We talked about how to prepare for the weather, how to think about budgeting, and being prepared for different costs of things, or how you might be able to consider all of the different pieces and parts of preparing for souvenirs that you want to get… all those different things, a variety of cluster topics to make sure that our students not only had a connection with the person that they were going to be traveling with, but also the other students. And then it gave us space for any of the questions that students may have had, as they were preparing for this experience.

Beth: It’s just a great way, because a lot of times when I’ve done education abroad, you might meet at the airport for the first time. But what I loved about this program is that we literally got to know each other prior, to at least kind of understand maybe some of the things that they were concerned about as a student. And so then as a staff member, we can be like, “Okay, let me give you some extra resources.” I remember one particular student, I ended up calling her mom and her was on the phone, because she’s like, “Okay, Beth, I know you went through the whole situation of what we need to do for the airport, but my mom has questions.” And I was like, “Sure, no problem. Let’s talk.” And so her mom was like, “Oh, nice to meet you Beth. Her mom dropped her off at the airport, gave me the biggest hug. When her mom picked us up. She was like giving everybody in the group a hug, because we were extended family for them for three weeks.

Rebecca: Can you talk a little bit about the timing of the program?

Sue: So last year, it was the first time through, so I think we had to do things very quickly. We had to assemble a team, we had to get these clusters stood up. We had to recruit students, we had to find the scholarships for them and all the rest of it in kind of a hurry. So this year, we’re spacing it out a bit more. So we’re recruiting for summer 24 right now. And we’re going to repeat the program, we’re going to take 30 students in two groups to London and 30 to Dublin in two groups. So we’re excited about that. And then their travel occurs during the summer vacation, early in the summer vacation, so May and June this year.

Marianne: The interesting piece of the timing as well was I was on the first group that went to Dublin, and we were actually able to connect with the group that was coming after us in the airport as they were preparing to go out, so my students were able to pass along some words of wisdom and some thoughts to the group that was coming after them. And so that was a really fun piece with the timing to be able to catch up with each other in the airport.

Beth: In terms of curriculum, the way it was set up, there was one class that everybody was part of. So we actually met as a group virtually, with all 60 students ahead time. So everyone had that basic foundation. And then when we’re actually over there, that was when we did the actual classes for the individual pieces. So I did Dublin two and London one. And so at the same time, we’re having curriculum in terms of giving feedback for the next group coming over.

John: Now, you mentioned that there was some time spent acquiring funding, could you talk a little bit about the funding that was used for this program?

Sue: Sure. And this is a little unusual, I think. It was a surprise to us when the Kentucky State legislature or general assembly awarded some funding to all Kentucky institutions of higher education to support displaced students, so refugee students primarily. As part of that program, they allocated a certain amount of funding also to support education abroad and exchanges. So when we realized that, which was only in the summer before, so summer 22, we thought, “Aha, we could finally make this happen. This thing we’ve all been talking about in different ways about connecting careers, connecting first-generation students with education abroad, and making a real difference for those students, and we could do it at a actually a quite a big scale for education abroad, 60 students is quite significant.” So we seized the opportunity. The trick, then, of course, was to find matching money, because it didn’t cover everything. So we found some money internally at the university, we used education abroad scholarships, our administration at the very highest levels awarded us some funding to kind of back us up if we needed it. So it was a team effort. And this upcoming year, we don’t have the state money. That was a one time deal, so we’re funding it all from our university funds.

Rebecca: So cost is always a concern, I think, for first-gen students, and may be one of the reasons why they don’t even think they might ever go abroad. I know that was the case for me, as a student. I didn’t think I was ever going to get a chance to go abroad, but I got to when I was in graduate school. So can you talk a little bit about what this meant to the first-gen students or if you had trouble initially even getting them to think that they could really actually participate in this program?

Beth: Yes, one of our students on Dublin two got the email about applying and that the program was going to be there when we started doing the recruitment. And he was like, “Is this a scam? Is this true?” We went over to the first-gen office to verify because he’s like, “it sounded too good to be true.” And so once we got the word out that it was, of course, students applied and said, “Okay, let’s do this.”

Sue: You’re absolutely right that it’s the number one barrier of first-generation students is finances. And I think it was the majority of our students on this program are Pell recipients. So first-gen is a very big category. But we did try and make a difference, particularly for students whose financial means were limited. So it was a tremendous opportunity for them. And as Beth said, some of them were very disbelieving at first of the opportunity, they couldn’t understand how this was happening, because they hadn’t thought that this was something that they could ever achieve. And that was another reason why we kept it to three weeks, because many of these students… well, in fact, I would say all of them… were working during the summer. So three weeks away from your summer job and earning is a big deal, and others had family obligations as caregivers, and so on. So that’s the reason why we ended up with a three-week timeframe was partly to be sensitive to those needs. And then in terms of the financing, London is an expensive city, and Dublin is an expensive city. And these are young people, these are primarily first- and second-year university undergraduates. So budgeting to shop, to eat, to go out is difficult, and especially when it’s a different currency and options are different, and the prices are different. So that was a big element that we’ve reflected on since this first time through, and I think we’ll be maybe just a little bit more supportive. We were supportive, but maybe just a little bit more supportive of how to budget when overseas because they were responsible for their own spending money on this program.

Rebecca: What about things like food or other personal costs for travel, was that included in the program?

Sue: So there were a few meals included. There were a few like welcome dinners and things like that included, but by and large students were responsible for their own catering as it were for their own food. And it turned out that was kind of an issue because it takes a bit of knowledge of the stores and what’s available and what’s affordable. It takes a bit of knowledge of how to cook in a budget conscious way and not just grab the processed thing. And of course these supermarkets are unfamiliar and I would say one thing we did think a lot about upon return and when we debriefed this was giving the students a bit more time to adjust to that situation and to learn their options and to cook because sometimes we were so busy with all these things we were doing every day that they didn’t have much time to plan their suppers, or to go shopping and cook so I’ll let Beth and Marianne chime in because they were on the ground, they know more, but that was the kind of impression I got.

Marianne: So, we had some bargain shoppers in my group that found meal deals at like close Tescos, things like that. And so it was always kind of a competition of “Did you know that the sandwich was included in the meal deal?” And so it was a pretty inexpensive breakfast and lunch and sometimes dinner opportunity for students that became somewhat of a game, to figure out, like, what are you going to put in your meal deal this time? And so a lot of our students were supportive in terms of sharing different deals that restaurants were doing. Did you know that on Tuesday, they have a student special where you can get this, this, and this. And so they were really great about sharing some of the tips and tricks and things that they picked up along the way. And then also sharing meals. So going to the grocery, and he brought me to purchase something, while I may not eat this whole thing, but if you split it with me, then I’ll get the next one. And so they were really great, that was really fun to see them kind of helping and sharing deals that they found along the way.

Beth: And one of the things that we did is we actually offered a time if someone wanted to come with me to the grocery store. We did a local one so they could see how close something was if they by chance needed something quick to eat. But then we actually went with them, which was one was a little bit farther away have more of the discount kind of larger what they would think of a supermarket type of thing here in the United States, just because, over there, you kind of buy more what you need, versus “I’m going to have lots in the refrigerator and freezer within it.” And we actually had a couple of times where we had meals together because it was someone’s birthday and we wanted to celebrate. And very similar to what Marianne said that they would be like, “here’s a deal,” or “hey, I’m going to make soup anybody else want?” And we had one student who loved to cook, and others are like, hey, awesome, I will help clean up, or pitch in for some money if you would be willing to be the person that actually wanted to make the meal, and they actually collaborated teamwork that way too. So it was really cool to see.

Beth: One thing I wanted to share is the fact I think people think that education abroad is three weeks, and then you’re done. For us, this is a lifetime connection for these students. So we actually had dinner with both groups separately, so they could get together and meet each other and see each other, I’ll get messages being like “Guess who I saw on campus?” and they have a competition of how many times they see a certain student, be like “I saw them three times this week, Beth, I saw this one four,” but even moreso is this confidence coming back to have to go after the dreams that they want. We had a student who, I’ll be very honest, had a really rough home life and had a lot of confidence issues and got over there had a chance to start talking, had a really good conversation with a couple employers. And we did a networking night where we invited all the employers back and they could come and network with the students. And she had a really good conversation with some of our staff that we actually had come over from UK to kind of see the program as well. And she followed up, wrote a note back to say thank you for this conversation, I appreciate it. This is what I’m looking for in terms of career or job, and that person connected her with to somebody back here in Lexington, she then goes and reached out to that person, met with them, interviewed, and now has an IT position that she never thought possible a year ago. So that’s really cool to actually see them do this steps, it’s one thing for us to say go do, it’s another for them to actually gain the confidence to go and actually obtain it.

Rebecca: What a great story.

John: One of the things that I think you’ve all mentioned is that this creates a really tight bond among the students as well as the connections to the staff members they traveled with. And that sense of belonging has been shown in a lot of studies to be a major factor in student retention. And I think programs like this can create really strong bonds that can help students be more likely to succeed.

Sue: I will agree with that. There were two young women on one of the programs who became fast friends. And I assumed they had been friends forever, and I just was chatting with them, and I said, “You know, you must have known each other a long time.” They said “no, we just met.” And one of them said that she was so excited to come back to UK in the fall… so this semester… and I said “well, weren’t you excited anyway?” And she said, No, she wasn’t, that her freshman year had been a little lonely, and she had not made a good connection with her roommate and was struggling a little bit to fit in. And she said “Now I have a best friend on campus, and we’re gonna have fun the next few years together. And I thought that was awesome.

Marianne: And I’ll brag on our students. They were phenomenal. I mean, when I told people that I was going to go to a foreign country with 15 college students, and we were going to travel the city and we were going to do all of the things they looked at me like, “Why would you take college students to an international city?” They were fantastic. They were supportive. They were curious. They were beyond what you could even imagine in terms of the questions that they were asking and the way that they engaged with the curriculum. We have phenomenal students here at the University of Kentucky and I was lucky to get to take 15 and I have them running across campus yelling my name saying “Oh my gosh, this is the first time that I’ve seen you since you’ve been back on campus” or seeing each other on campus is such an honor to be a part of just that family that we now have. And, like I said, they were the best students you could ever imagine traveling with. I do it again in a heartbeat.

Sue: So one thing that really impressed me about these students, actually taught me a lot about this category “first-generation” which is kind of thrown around is that these students are really amazing. I mean to get to the University of Kentucky, to be studying their majors, to be making good academic progress is an accomplishment for these students and they have the resilience, they have the resourcefulness, they have the curiosity, as Marianne said, to make the most out of education abroad. So these are not students who took this for granted. When they were in London or Dublin, their eyes and ears were open all the time. And they were busy taking it all in, reflecting on it, and they absolutely were some of the very best students I’ve ever seen on an education abroad program.

Rebecca: So do you have plans to evaluate the success of this program?

Sue: So yes, we built assessment into this program from the beginning. So we had a researcher from the College of Education who helped us and administered pre- and then during, and then post-surveys of each student. So she collected information about their responses and reflections to the assignments and to their learning, and also something about their intercultural learning as it were. So we’re going to also track these students and see how they do compared to the peers who didn’t get the opportunity to study abroad in terms of their academic progress and their graduation.

John: Excellent. We look forward to hearing more about how well this program works. We always end by asking, what’s next?

Sue: Well, we want to do this again. [LAUGHTER] So we think it’s working. We’ve got good results so far, assuming that the next summer is also successful. We hope to just keep tweaking it, making it better and better for the students and maybe building it in as kind of a signature program for first-generation students here at the University of Kentucky.

Rebecca: An incredible thing to invest in and offer first-gen students. Thanks so much for sharing the details of your program.

Sue: Thank you for having us. And thanks for your interest in this endeavor.

John: It sounds like a wonderful program.

Sue: Yeah, it’s been a blast to develop and to be part of I must say. It’s been fantastic. And I don’t think we mentioned but perhaps we should that over one quarter of our undergraduate students identify as first generation. So this is a significant population at our university which serves students of all sorts from the state of Kentucky and outside.

John: Excellent.

Sue: Thank you both.

Marianne: Thank you all

John: Thank you for taking the time to join us and to share this wonderful program.

Sue: Yeah, appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

322. Accessibility Challenge

Posted in access, community of practice, equity, higher education, inclusion, learning, pedagogy, professional development, reflective practice, social justice, teaching
January 3, 2024

Digital accessibility can be intimidating for faculty and staff. In this episode, Michele Thornton, Laura Harris, and Kate DeForest join us to examine one example of a gamified approach to professional development. Michele is an Associate Professor of Management at SUNY Oswego, Laura is the Web Services and Distance Learning Librarian at SUNY Oswego. and Kate is the Digital Content Coordinator at SUNY Oswego.

Show Notes

Transcript

John: Digital accessibility can be intimidating for faculty and staff. In this episode, we examine one example of a gamified approach to professional development.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Michele Thornton, Laura Harris, and Kate DeForest. Michele is an Associate Professor of Management at SUNY Oswego, Laura is the Web Services and Distance Learning Librarian at SUNY Oswego. and Kate is the Digital Content Coordinator at SUNY Oswego. Welcome Michele, Laura, and Kate.

Kate: Hello. Thank you.

Michele: Thanks for having us.

Laura: Thank you.

John: Our teas today are:… Michele, are you drinking tea?

Michele: I am. I’m having a London fog. It felt like a good choice given the kind of rainy day.

John: … and Laura?

Laura: I am drinking the Comfort and Joy tea by Republic of Tea.

Rebecca: Sounds nice for a chilly day.

John: …and Kate?

Kate: I’m enjoying a nice chai tea.

Rebecca: Look at the variety in this group. [LAUGHTER] I have Harsha today, which is a black tea.

John: That sounds rather harsh and a bit different than the comfort and joy. [LAUGHTER] And I have, in the spirit of the season, a Christmas tea today.

Rebecca: So we invited you here today to talk about the article that we co-authored, titled “10-Day Campus aAccessibility Challenge” in a recent special issue of the Journal for Post Secondary Education and Disability. The accessibility challenge was an initiative developed by the workgroup on accessibility practices at SUNY, which you’ve all been an active member of. So first, Can each of you briefly describe how you got involved with accessibility work at SUNY Oswego and some of the specific projects that you’ve worked on? And we’ll start with Michele.

Michele: Thanks, Rebecca. My first introduction was by being part of our initial cohort of faculty accessibility fellows in 2019. So that was a year-long fellowship, where myself and a handful of other faculty members from across the campus were able to learn the importance of things like Universal Design for Learning, build skills and capacity around principles of digital accessibility, and become part of the growing community on campus that was really advocating for a more accessible and inclusive campus.

John: And we do have an earlier podcast episode on the origins of that project, and we’ll share a link to that in the show notes.

Rebecca: Kate?

Kate: Okay, I was hired as the digital accessibility analyst and remediation specialist in 2018. I was primarily focused on assessing and remediating online course materials at that time. When I was hired, I was invited to be in the work group, and quickly became one of the main resources for remediation and accessibility at that time. I’ve been involved with creating our digital accessibility website, many of the written and video tutorials, launching the 10-day accessibility pilot program and other subsequent programs, and currently involved with creating accessibility course modules.

Rebecca: Laura?

Laura: One aspect of my job is to support online learning and teaching. And in that role, Rebecca and a former colleague invited me to be part of the workgroup focused on facilitating the creation of accessible materials for online courses. Over the years, the scope of that workgroup has broadened, and now we focus on accessible practices in general. One of the projects I’ve really enjoyed is providing training on different models of disability.

John: Rebecca, you’re one of the people who put together the accessibility challenge. So could you explain your role in it?

Rebecca: Sure. I am one of the two founding members of our workgroup on accessibility practices at SUNY Oswego, and was the first facilitator of our Faculty Accessibility Fellows Program that Michele was in. So Kate, before we jump in too far with our accessibility challenge discussion, can you first help our audience understand what we mean on our campus by accessibility?

Kate: Sure. So the bare bones basic definition, as paraphrased, would be allowing a person with a disability and a person without a disability, the same or similar experience in the same or similar manner. And we are speaking of it in the digital capacity, so using websites and digital content, digital documents, and allowing people to basically experience them in a very similar way.

John: And before we go any further, could someone tell us what the accessibility challenge is? Michele, can you set the stage for us by providing an overview of the accessibility landscape at SUNY Oswego and the circumstances that led to the development of the accessibility challenge, and also, what exactly that accessibility challenge was?

Michele: For me, it’s hard to separate out the genesis of the accessibility challenge from two other important existing factors. The idea came to us in fall of 2020. And so if we can all put ourselves back there, we had just come out of the first spring of the onset of the COVID19 pandemic. That took our campus, like many others out there, into this rapid switch to online learning, and one that really brought the harsh light on many accessibility barriers that parts of our campus had previously not really observed or had much experience with. We’ve already talked about and highlighted a couple of different ways that our campus has been thinking about accessibility with the workgroup, our Fellows Program, and so we have this long tradition of campus leadership and support around promoting accessibility. But the unprecedented need that the pandemic really illuminated revealed that we needed to move even quicker to build a more robust, more skilled, more engaged community that would be prepared to meet the challenges that our students and faculty were facing. We had historically been offering a lot of different trainings and one-on-one faculty support, but we felt like we needed something much more concentrated, quick, that would be fun and enjoyable for folks to participate in. The pandemic was just in its first 12 months, and people were stressed and feeling isolated, nervous or afraid. We wanted to create an opportunity to connect as well as learn from each other. So we’ve talked about this as a 10-day challenge. It was essentially two weeks of an online community engaged learning experience, where folks signed up and took different asynchronous and sometimes synchronous online courses to build their skills and capacity around accessibility.

Rebecca: So Laura, can you talk a little bit about some of the design considerations that went into the challenge?

Laura: Sure, Rebecca. The workgroup often discusses training and professional development opportunities we can provide to faculty, staff, and students. And as Michele indicated, when we had those discussions in late 2020, we knew that many people were feeling overburdened, disconnected, and disenfranchised. And we didn’t want to add to people’s mental loads, we wanted to craft something that was fun and supportive. So one of the theories underlying our thinking is self-determination theory, which suggests that experiences that support individuals’ experiences of autonomy, competence, and relatedness, or connectedness, will provide the best kinds of motivation. I want to talk about competence first. I think promoting competence around accessibility practices has always been at the core of what we do. However, the increase in online instruction caused by the pandemic made fostering this competence a necessity. We just didn’t have, and we do not have now, the staffing to remediate every course. Moving on to relatedness, I feel that offering a challenge instead of regular professional development opportunities really allowed us to foster that sense of connection. With challenges, there’s a shared goal, and when it comes to pulling people together, a shared goal can be more powerful than a shared interest. I also think that, with challenges, there’s often a more intentional effort at providing support and encouragement to participants. For example, the organizers behind National Novel Writing Month host live Q&As and have collected pep talk letters addressed to their participants written by well-known authors. So while competence and relatedness were things we considered early in the process at the macro level, the ability to foster autonomy and agency came through when we were planning the details. One of the other theories we incorporated into the design of the challenge is Universal Design for Learning (or UDL). I would argue that agency and choice are at the core of UDL, which focuses on providing learners with multiple ways of achieving a learning objective. So we made a point to offer multiple ways for Challenge participants to learn about and apply various accessibility concepts.

Rebecca: So Kate, can you describe the challenge and how it worked?

Kate: Sure, we wanted to basically simplify accessibility and allow all content creators to understand basic accessibility principles. So this challenge was centered around creating accessible Word documents through Microsoft Word and Google Docs. We broke it down, broke down accessibility into bite-sized pieces, and we focused on one topic each day. So we started off by introducing accessibility, we covered what it is, why it’s important, and who it benefits, because that’s sometimes lost in translation, depending on the definition that people think of accessibility. Then we went in and we focused on specific skills. So some of the topics included properly structuring content such as how to semantically make headings and lists. We covered writing alternative text for images, captioning videos, effectively using color, and providing descriptive hyperlinks as some of the basic principles. And then the last couple of days of the challenge ended with some self reflections, sort of what did the participants learn? Did the program help boost their competence around accessibility? …that type of feeling. So we send daily emails to the participants, and these emails give a brief background of today’s topic, again, who it benefits and why it’s important. We provided written and video tutorials that explained how to do the task, and then asked the participants to incorporate that principle into the documents of their choice. We also provided other related articles and sources of information, as well as links to live zoom sessions that were being offered that same day around that particular topic.

John: Laura, can you talk a little bit about the timing of the challenge and how participants were recruited to participate in it?

Laura: So for many years, the Center for Excellence in Learning and Teaching has provided a series of professional development opportunities that are offered for faculty and staff, and are usually led by faculty and staff. These usually are about two weeks before and after the spring semester. The ones that we have in January we refer to as the winter breakout sessions, we have been offering professional development on accessibility practices during the winter and spring breakout sessions for the last few years. So it really just made sense for us to offer the sessions related to the challenge at the same time. As far as recruitment goes, we worked with the Office of Communications and Marketing, they did a news story that was shared with the entire campus, they emailed all faculty, staff, and students. And we also communicated through some smaller communication channels, like the email list for the Center for Excellence in Learning and Teaching.

Michele: If I could just build on a couple of things that Laura said, I think that we really credit the success of our first challenge to the timing of it and the relationship with the Winter Breakout sessions. I think there’s two important reasons for that, is that this period of time is between semesters. So we know that while faculty certainly are anticipating professional development at that time, they also tend to have kind of allocated some time and space to work on it. Secondly, I do think that if we had tried to replicate this at another time during the year, and rebuilt the structure and rebuilt the marketing around it all separately, that would have been just an even heavier lift for our committee that was working on it. So that connection to our existing structure mattered for sure.

Rebecca: We also had students participating. So, previous professional development had focused primarily on faculty and staff. And this particular initiative really invited students into the scope as well. And most students aren’t taking classes at that time. So there was a little more flexibility for students to take on the challenge as well.

John: One of the things we should note is that the timing of doing this in the early stages of COVID probably helped because faculty were using much more digital content, and were aware of how much they didn’t know, in some cases, about using digital content effectively. So I think all those things came together to help make the program remarkably successful. Laura mentioned that we had been doing workshops for a while, but we should probably credit Rebecca with that, because that was one of the first tasks she took on, actually even before she became associate director of the teaching center. We asked her to do a number of workshops, I think, in your very first year here actually on accessibility.

Rebecca: Yeah.

John: …and that hasn’t stopped since then.

Rebecca: No, it hasn’t. [LAUGHTER] I think one of the other things that we might want to also point out about the success is not just the timing in terms of the January sessions, but also that our involvement in accessibility on campus was starting to mature. We already had campus-wide initiatives leading up to this particular one. And the first cohort of faculty accessibility fellows had completed in 2019. So we also had those fellows to support this initiative during COVID. So Michele, you spearheaded the evaluation of the challenge, can you talk a little bit about the methods used and the results of the challenge?

Michele: I think when we first came up with this idea, the first thing we did was tried to see if there were other models out there that we could pull from. And while I think there’s other examples of challenges, we knew early on, we had a hunch, that what we were doing was kind of novel and unique. And because of that, it was important for us not just to document what we were doing, but to have some attempt to gauge the impact that it was having, I think this would be helpful for us for a couple of reasons. One, just our ability to improve on future iterations or efforts that we would do in this space. But also, I think that you might start getting a sense that all of us feel really passionate and strongly about this. And so the ability for us to advocate beyond our own campus and share what we were doing and help others understand the impacts of it was important for us in terms of documenting and gathering data. So we took all sorts of approaches to gathering data, everything from monitoring the open rates on the emails that went out across campus, to looking at our website traffic through Google Analytics, but the majority of the data came from a pre- and post-set of survey questions. So we did everything from ask folks when they first started to reflect a little bit around their motivation. This gets at some of the things that Laura was sharing about our initial design, about why people were participating and what they were looking for from the challenge, all the way up to then in the post-survey asking folks, as Kate mentioned, to reflect on their experiences. From those surveys, we were really able to pull out key qualitative and quantitative data to get a sense of what motivated folks to join, really understand how their confidence changed and increased in their ability to do things like define accessibility, to be able to make a Word document digitally accessible, but also just understand what they enjoyed most. And over and over again, I think the thing that we learned that probably was maybe most surprising and really nice kind of unexpected benefit was folks reporting that they really enjoyed being part of the learning community together. and the sense of being part of something that was bigger than just “Hey, I’m learning some new skills to teach my class or to send out more accessible emails,” but understanding that they were sort of joining and connecting into this broader movement that was happening on campus was, I think, one of our most surprising and also exciting takeaways.

Rebecca: I think one thing that we didn’t mention that might be worth noting here, especially after you were talking about the surveys is that we did prime our audience at the academic affairs retreat in August, leading up to the fall 2020 semester, by having a few minutes on the agenda to talk about accessibility and to get the academic community aware of digital accessibility. And then the challenge followed up only a few months later.

John: Kate, can you talk briefly about some of the iterations of the project that follow that initial 10-Day Challenge.

Kate: So the initial challenge was held in January of 2021, as we mentioned, and March of 21, we held a presentations challenge, which focused on creating accessible presentations using PowerPoint and Google Slides. This was a weekly challenge, meaning that participants received one email each week for four weeks that focused on one topic. We also provided them with background information, written and video tutorials, and additional resources in the same manner that we did for the initial 10-Day Challenge. And then in January of 22, we ran a five-day accessibility challenge. This was formatted in a very similar manner to the 10-Day Challenge, but we basically split the content into two and created two tracks: we had a beginner level and an intermediate level. Each day, again, highlighted one topic, we provided participants with background information, the tutorials, and additional resources. But this time, participants could choose what content to work on for each day, whether they wanted to stick with the beginner track or go to the more advanced track or do a combination of both.

Rebecca: So Michele, can you talk a little bit about the newest iteration of the project that’s currently in progress?

Michele: I am so excited [LAUGHTER] to talk about the newest iteration of it, because really, this new version has given us a way to massively scale up what we started as kind of this idea of doing a 10-day challenge. Last fall, the entire SUNY system migrated to a new common learning management system. And when that happened, members of our team started imagining a way that we could use that common system to develop an asynchronous customizable version of our challenge that could be deployed across the 60+ universities, schools, campuses across SUNY. We applied for and got a nice grant to support the development of this idea. And there is a really fantastic team, committed faculty, staff, and even students here at Oswego now working to bring this to fruition. And our goal is to pilot it broadly this coming spring, but here at Oswego, we’re going to come back to our roots with the winter breakout session, and we’ll launch the first iteration of it and be able to get some feedback in the next couple of weeks here.

John: And the grant that funded it was a SUNY Innovative Instructional Technology Grant. We should credit SUNY for providing this competitive grant program. Could each of you provide a bit of advice for anyone thinking of doing this at their own institution?

Michele: I’ll start, and I think that we used a pretty big variety of resources. So we didn’t just kind of create material from scratch, we also linked out to things like existing resources from Deque, we pulled in literature and other articles that folks have read. So I think that the biggest takeaway, I would suggest, is that you can do this as big of a scale or as small, you could have a three-day challenge, it doesn’t need to be 10. But starting anywhere, and recognizing your capacity and reaching out to use other existing resources is a good way to supplement if you perhaps don’t have as big of a pool to draw from in terms of internally on your own campus.

Laura: I would add to that just starting wherever you are with whatever resources or personnel you have, it doesn’t have to be a fancy initiative. It can be sort of a grassroots within your own department or as small as you want it to be. But just getting started and sharing out whatever information you have is something, it’s movement in the right direction.

John: Following up on Kate’s comment, this is very consistent with Tom Tobin’s plus one strategy, start with some small changes, and then build on those every semester as you move forward.

Rebecca: In the show notes, we’ll link out to an overview of our challenge as well as the article that we wrote on the challenge.

Michele: You know, the only other thing that I would say is that it’s important to find support and partners and maybe places that you might not expect on campus. So thinking about how to connect this with your campus DEI efforts more broadly, or working with your accessibility resources. Again, we talked about so many different areas where we got support, even the communications and marketing team really helped us, but our CTS team, I think finding those collaborators is a big part of how to ensure something like this can be successful.

John: And CTS is Campus Technology Services.

Rebecca: So we always wrap up by asking: “What’s next?”

Laura: So something that I’m working on right now is creating a guide that talks about the accessibility features that are available from each of our major database vendors like EBSCO, and ProQuest, just to make those features a little bit more discoverable… accessible… [LAUGHTER] to our users. We try to talk about those in our instruction that we do, but they’re not always obvious.

Kate: So I’ve presented at a number of different conferences, and I’ve talked about the accessibility initiative at Oswego, and this particular challenge and some of our iterations. So I hope to continue that and just kind of share the good word about what Oswego is doing and some of our projects that we’ve been working on and how we can help other campuses or help other departments or people implement similar types of projects.

Rebecca: And how about you, Michele?

Michele: Well, I think I gave it a little bit away, that most of my accessibility work right now is focused on making sure that our new iteration of the Challenge gets off the ground, and we’ve got everybody all hands on deck with that. Beyond that, though, Rebecca, you and I are working on a fun project when we find ourselves with time to take a similar approach in terms of documenting the impacts of accessibility work on our campus. And we interviewed all of the first few cohorts of our campus accessibility fellows, and we’re in the process of trying to figure out what we’ve got there and how that I think shares the story about how Oswego is maturing in its process of working to achieve accessibility, and a more inclusive environment.

Rebecca: Well, thank you all for your work in accessibility, and for sharing that today.

Michele: Thanks for having us.

Kate: Thanks for having us.

Laura: Thank you for having us.

John: Thank you. It’s great talking to all of you and we’ll be seeing you during the winter breakouts very shortly.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

321. College Students with Disabilities

Posted in access, accessibility, edtech, educational technology, equity, higher education, inclusion, learning, pedagogy, social justice, teaching
December 27, 2023

Sharing student narratives about their experiences can help us to understand how our instructional and policy decisions impact the student experience. In this episode, Amy Fisk joins us discuss to discuss her research project with Rebecca on the perceptions that students with disabilities have of their learning experiences.

Amy is the Assistant Dean for Accessibility at the State University of New York at Geneseo. Amy oversees the Office of Accessibility Services, which coordinates accommodations and support services for students with disabilities. Prior to her role at Geneseo, Amy coordinated a support program for students on the autism spectrum at SUNY Purchase.

Show Notes

Transcript

John: Sharing student narratives about their experiences can help us to understand how our instructional and policy decisions impact the student experience. In this episode, we discuss the perceptions students with disabilities have of their learning experiences.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Amy Fisk. Amy is the Assistant Dean for Accessibility at the State University of New York at Geneseo. Amy oversees the Office of Accessibility Services, which coordinates accommodations and support services for students with disabilities. Prior to her role at Geneseo, Amy coordinated a support program for students on the autism spectrum at SUNY Purchase. Welcome, Amy.

Amy: Thank you. Thank you so much for having me.

Rebecca: Today’s teas are:… Amy, are you drinking any tea?

Amy: I drink tea every morning. So I have Bigelow French Vanilla black tea.

Rebecca: It’s a good way to start the day. How about you, John?

John: In honor of the holiday season, I have Christmas tea today.

Rebecca: I’m drinking Blue Sapphire from my favorite tea shop in Canandaigua.

Amy: Where’s that?

Rebecca: It’s right on Main Street. You should go there.

Amy: I should.

John: We’ve invited you to talk about the article: “Perspectives among college students with disabilities on access and inclusion,” which you co-authored with someone else… Rebecca, I think it was.

Amy: That name sounds familiar.

John: …which was published in College Teaching earlier this year. Before we talk about the article, could you tell us a little bit about your role at SUNY Geneseo.

Amy: So I oversee our Office of Accessibility Services, or OAS. I meet with students to coordinate accommodations and other kinds of support services for our students with disabilities. I monitor policies and procedures within our office. And I often work with faculty and staff on issues related to accessibility and inclusion. So for example, I might do trainings across campus, work with administrators on various committees, and having a voice on issues related to disability, education, awareness, and accessibility.

Rebecca: So prior to this project, Amy, and I didn’t actually know each other. Do you want to share the origin story?

Amy: I started my position here a month before COVID became a thing. So I was kind of thrown into some challenges I did not anticipate. But one of the things I had been thinking about, many of my colleagues were thinking about, was: How are we going to support our students with disabilities? We’re really kind of concerned about their trajectory during this challenging time. We wanted to just get some more information about students’ experiences during COVID. I started talking to Nazely about this, and she says, “You know, I know someone who does research who also might be interested in a potential collaboration.” So that’s how I got connected to Rebecca. And ultimately, we shared an interest in learning more about the impact of COVID on our students within our respective roles on our campuses. We knew that this was a really challenging time for all of us, but especially for our students with disabilities who had already been experiencing barriers pre pandemic. And so we really wanted to hear from our students about their experiences, and what can we learn about access and inclusion moving forward, even when the dust settles and we talk about things post COVID?

John: A lot of the studies that have been done have been quantitative studies. And your study is a qualitative study. Could you talk a little bit about how this qualitative research complements the quantitative research that’s been done?

Amy: Sure. So ultimately, we wanted to gather students’ stories, and many of our findings from our studies are reflective of findings from past studies on challenges and barriers students with disabilities face compared to students without disabilities. But we wanted to identify these specifically within the context of remote learning. And also within the context of navigating this challenging time just in life, we really wanted that student narrative. And we also wanted to assess the positive things that were happening, the practices that were helping our students feel successful, to really help inform tangible takeaways and recommendations to our readers. And we hoped for this information to be relevant, like I said, when the dust settles and regardless of teaching modality. And I think it’s important to highlight that despite the obvious challenges that COVID brought, it has highlighted the importance of accessibility in higher education and really gave us an opportunity to reassess what we’ve been doing, our everyday policies and practices, and we really wanted to highlight that from the student perspective. Beyond that, we also wanted to talk about the needs of our students with disabilities within the context of access and inclusion. So, often disabilities and identities, that tends to be left out of conversations related to diversity, equity, and inclusion. So it wasn’t just about how we need to provide appropriate classroom accommodations, but what are the ways that we can be more inclusive, promote a sense of belonging, proactively provide equal access? So those were the things that we had in mind as we were designing our study.

Rebecca: One of the things that comes up in a lot of conversations, at least more recently in higher ed., is this growing number of students who are registering for accommodations and also the mental health crisis. Can you talk a little bit about that to provide some context for our discussion today?

Amy: Sure. So the mental health needs of college students with disabilities was becoming really apparent before COVID hit, really significant needs related to depression, anxiety, other severe psychiatric impairments. And the studies that had been done around the time of COVID really highlighted those issues of more and more students connecting to their disability services offices, self-identifying as a student with a disability where they have a clinical levels of depression, anxiety, other debilitating mental health needs. And that theme came out in our study as well.

John: Did all the discussions of the challenges of COVID help encourage students to become more willing to declare their mental health challenges or their other needs that perhaps they might have been more reluctant to state prior to this time?

Amy: I think so. I think there is a shift in our culture, and it being okay to talk about mental health and mental illness, for students to say, “I’m having a really hard time, I’m struggling,” because mental health is a spectrum. We all experience a variety of emotions throughout the day, throughout an hour, and throughout our lifetime. And I think it’s becoming slowly de-stigmatized in talking about mental illness and the importance of promoting mental health, especially among our college-age population. A lot of college campuses are really taking seriously the wellness of their students on campus just because of the rise in numbers of students needing that extra support, because colleges across our country are noticing a pretty significant increase. And I do think COVID has propelled that de-stigmatization of talking about mental health.

Rebecca: We’ve talked in the past on this podcast with Kat MacFarlane about some of the barriers that students face in just even approaching and asking for accommodations, having to register with an office of disability services, or whatever the equivalent is on the campus, and having to self identify. And then a lot of students don’t actually choose to do that for a wide variety of reasons, some associated with stigma, but we are seeing increased registrations. So does that mean that there’s increased disability?

Amy: Yes, I think there are a variety of factors and more students connecting with disability services offices. One, I think high schools are better preparing students with disabilities to enter the post-secondary environment. Two, I think our offices are becoming more visible on campus. Again, I think there’s also a de-stigmatization of disability and accessibility services offices, and we’re becoming more visible and relevant on college campuses. And third, I think colleges are starting to talk about disability as an important facet of diversity more and more, I think there’s certainly room for improvement, but I think that conversation is starting to happen. So more students are finding their way into our offices.

Rebecca: So three key themes emerged in our research about the perceptions of students with disabilities, of our institutions, and their experience, and of belonging. And so those three themes are accommodations and accessibility, building relationships, and community, and then course structure and design. Perhaps we can take them one at a time here. Let’s start with accommodations and accessibility. Can you first start with what’s the difference between accommodations and accessibility? Because we know that this is often something that’s confusing to folks.

Amy: Sure. So an a ccommodation, by definition, is designed to remove some sort of barrier that an individual with a disability is experiencing. So an academic accommodation, for example, might be having extra time to take an exam, because timed tests can be a barrier for some students. Maybe it’s a notetaking accommodation because they need assistance accessing that lecture material. Sometimes it’s ensuring that the course materials themselves are accessible, that they can be read through a screen reader. Sometimes the accommodation is related to a course policy such as attendance for a student with a more severe chronic medical condition. So it is an individualized process to assess what an appropriate accommodation would look like. But the purpose of it is to remove some sort of barrier so that this person has equal access to their environment. And so accommodations, though necessary, is something that we’re legally required to provide for the ADA. It’s really a reactive way of ensuring equal access. It’s a floor, it’s a minimum. Accessibility, on the other hand, is about inclusion from the start, so that individual accommodation may not even be needed. And something I like to highlight is that accessibility is not about lowering standards. It’s sending a message that everyone belongs in this space and that inclusion matters.

John: What were some of the most common barriers that students reported facing related to accommodations and accessibility in your study.

Amy: Some of those barriers for students just not receiving their approved accommodations during remote learning, including extended time on tests, for example, or online course materials just not being accessible, or having to continually remind instructors about their accommodations, explain why they needed the accommodation in the first place, negotiating terms of pre-approved accommodations. And this was particularly true among students with what we might call an invisible or non-appearance disabilities such as learning disabilities, ADHD, mental health disabilities, these students are less likely to be believed and questioned about the validity of their disability or their need for accommodation. So those were some pretty significant barriers for students and just not receiving the accommodations that they were approved for.

Rebecca: I think one of the things that was also highlighted as a result of our study taking place, whilst COVID was in full force is how many campus resources students with disabilities and other students depend upon every day. So we had students reporting things like” I didn’t have access to a printer to pull up text when it was more of an image instead of an accessible text that could have been expanded digitally, or having access to a quiet space, like the library.”

Amy: Yeah, that was really significant. And then that is where you also saw some other equity disparities. So there were some students who live down near New York City in very populated areas, there was a lot going down there at the time. COVID, if we recall, some students did not have quiet spaces at home, whereas other students had quiet home offices and their parents may have been at home with them, helping to support them. And then other students who didn’t have a quiet space whatsoever took on more caretaking responsibilities, didn’t have access to WiFi. So those equity disparities continued to widen during COVID beyond the disability barrier, so that was something significant, I think that needed to be highlighted,

Rebecca: What are some of the factors related to accessibility and accommodations that actually resulted in positive perceptions?

Amy: So our students actually reported some very positive interactions with their instructors. So when receiving a student’s letter of accommodation, or like an accommodation notification that would come through our office, some would reach out and ask the student “How can I support you? How can I help provide this accommodation?” One student even noted how they appreciated that the instructor didn’t call them out in class, because that had happened before. So I think just preserving the students’ dignity, reaching out to the student, those were the kinds of things that our students reported as making a significant difference.

John: I know you’re study focused on the status of students during COVID, but in your role addressing these issues now, have the changes in faculty behavior persisted? Have faculty continued to become more sensitive to some of the accessibility and accommodation needs of students as we move back to more classroom instruction?

Amy: So in conversation with colleagues, other disability service providers across SUNY, but also across the country, I think we’ve seen a mix. I think there are some who just wanted to go back to normal, and didn’t we all. I think COVID, again, was a very challenging time and faculty too didn’t have a ton of support, and also really struggled with having that emergency shift to a remote learning modality and some didn’t have the skills or support to really deliver courses in the way that would have facilitated student success. So they were really looking forward to getting back to that in-person modality, back to the pedagogy that we’re used to, and that may have posed some new barriers for students coming back to college campuses. Conversely, we also saw instructors taking some of those learned lessons from the remote learning period and applying them when we did come back to campus. So I do know a number of instructors who, for example, are still utilizing the lecture videos they created during COVID and post them on their learning management system for students who may not have been able to attend class that day, for example, so they can still get the lecture material or recreating their course materials and documents so that they are accessible, creating videos, captioning their videos, modifying some course policies to be a bit more inclusive for students. So I think there has been a change in realizing we can still have students be successful and meet the learning objectives, but in a different and more inclusive way.

Rebecca: I think one of the things that we can highlight that also came out in the student experience, and students reported this in our study, is that some of them actually experienced better access during COVID. Not all but some, in part because some of the technology caught up. And when we first went remote, Zoom didn’t have captions available by default and now it does. And so a lot of these things have become norms that people with disabilities have fought for for a long time and never got.

Amy: And I would say that’s true as well with regard to course policies that may have been amended as well, introducing more self paced work, which is also something that students really appreciated during the remote learning period.

John: I just recently returned from the POD conference where there were many, many discussions of this very issue. And in general, the results there were pretty much the same as what you’ve described, that a lot of the changes that faculty made to better accommodate students needs persisted, but some faculty have moved back to old practices and the results are a little bit mixed. But on average, there seemed to be, in a number of studies, some substantial improvement in faculty responses to student needs.

Rebecca: Based on what students have reported, what recommendations do you have for faculty related to accommodations and accessibility to continue the forward movement as opposed to regressing?

Amy: So, actually I did a talk with faculty of one of our academic departments at the start of the semester, reviewing our office, some of the logistical pieces of implementing accommodations, that sort of thing. But before I started really getting into that, we had a discussion about how accommodations, and the dialogue about accommodations with students are approached, how it’s discussed, how it’s communicated, something as simple as taking time to actually review the portion in your course syllabus related to an accommodation, so maybe an accessibility statement. That tells students that this is important, making sure that your online materials are accessible from the start, that tells students that accessibility and inclusion is important. And students are more likely to engage in a reciprocal dialogue with you about their needs when they feel like they’re heard, when they feel like they’re a valued member of the class, that their accommodations are important and not burdensome. That’s a term we heard a lot in our study, that they’re not a burden, or it’s some sort of requirement that the faculty has to fulfill. And so I think this is probably true for most students, regardless of disability. But students in our study specifically noted how they appreciated when the instructor showed empathy and understanding and flexibility, recognizing that students have significant issues outside of the classroom. We all do, between family, finances, things that are happening in our world today. And I think this is important to acknowledge as well, given that we’re seeing an increase in students from various diverse backgrounds coming into the college environment.

Rebecca: And as we’ve talked about many times on the podcast, flexible doesn’t mean not having standards. [LAUGHTER] And it doesn’t mean a free for all. In fact, a lot of our students benefit from structure, which we’ll talk about, I think, in a few minutes, because that ties to one of our other themes. You talked a little bit about faculty workload related to this, and sometimes the perception that faculty put off is that it’s a burden to provide these accommodations. And the reality is that a lot of our students need very similar things. And so if we think about the common requests for accommodations, or digital accessibility strategies, from the start, we often don’t have a lot of one off things that we do need to accommodate, because we’ve already built it into our courses. That’s not to say that there aren’t accommodations that we need to provide additionally, but it may result in less work, ultimately, to really think about these accessibility principles upfront.

Amy: Right. And I think something as simple as making your course lecture materials available on the learning management system available to students. That can help reduce a lot of barriers for many students who might struggle with keeping up with the pace of the lecture and they end up missing material. A student who may have missed class that one day and just needs that material, other students who need to kind of re-teach themselves the material because, perhaps, they had challenges with staying focused during class. I think there’s a variety of reasons why students would benefit from that, but something as simple as that. Often, when students come to see me, there are maybe students who hadn’t needed accommodations previously, but they encountered a particular course where the policies were such that there were new barriers that arose and if the policy was different, perhaps they wouldn’t need that accommodation. That’s a concrete example of the difference between accommodation and accessibility. Some of our course policies and course design may be inadvertently barriers to students with or without disability. So this might include use of pop quizzes, not making lecture materials available to students, not permitting use of technology, not allowing students to even take breaks in class. And so although the purpose of these policies is probably to make students engaged and have accountability in the course, which these are things, of course, we want… again, we’re not lowering standards… students still need to go to class and do the work. But I think some of these policies actually might be having the opposite effect, and it does for students who request accommodations, rather than focus on learning in the course.

John: I think that many faculty who had only taught in a face-to-face modality before COVID, were able to avoid issues of accessibility by not creating digital content. When they moved to remote teaching, though, they were forced to begin developing digital materials and often received some training in creating accessible digital content. Do you think that that training received during COVID helped encourage more accessible practices by faculty in general?

Amy: I think so again, I think some of these practices have shifted over time, and I think COVID has shed light on the benefits of accessibility, not just for people with disabilities, but for all people. I mean, again, use of captions and subtitles can be beneficial for a lot of folks, whether you’re sitting in a busy Starbucks, whether you have a lot going on in the background, maybe you’re trying to juggle work and family, maybe, again, you’re hard of hearing, and so you need access to those captions. Again, accessibility is for all, not just about or for people with disabilities.

Rebecca: The second theme that kind of emerged in our research was building relationships and community, can you share some insights with faculty about the role that they can play in helping students with disabilities feel connected and included? And you highlighted some of those already: providing accommodations and showing students some dignity and respecting their dignity.

Amy: So again, I think engaging with a student and even something as simple as taking the student aside and asking, “How can I make this course more accessible to you?” speaks volumes to the student, that they are valued, they belong, that their needs aren’t burdensome, and they’re more likely to engage in a reciprocal dialogue with the faculty member when they feel like “Oh, they care about me and my success in this course.” I actually knew about a professor who did an anonymous Google form, asking students “How can I make this course more accessible to you? Are there barriers? In reviewing the syllabus, do you have concerns about something within the course?” One of my students actually told me about this, and said how it really made them feel seen and valued. And they were more likely to reach out to the instructor when they needed help, because some students fail to do so out of shame. They’re in a very vulnerable position to talk about their disability related needs to a faculty member, to an authority figure. And so when you do something as simple as asking a student, “How can I make this accessible to you? Are you experiencing barriers right now?” really opens that line of communication with the student and helps them build a positive relationship with that instructor and for maybe other instructors. It also helps to build a sense of community so that other students know that this is really important, and that inclusion matters. And that’s also sending a message to all their students within the classroom that we appreciate and respect diverse learners here in this classroom. I think that’s a teachable moment for our students as well.

Rebecca: So one of the other things that I think emerged is a desire to be connected with peers, but that faculty can play a really important role in facilitating that connection. So I think oftentimes, we just assume in a classroom that at the beginning of class students are socializing and getting to know other folks and have those contacts, but students really reported that having more structured ways of connecting with peers was really beneficial to them outside of class. And that’s something that I think we might take for granted as instructors in the classroom, that it would just kind of organically happen. But that structure, that scaffolding around that really bubbled up as being pretty important to our students,

Amy: Yeah, that peer-to-peer interaction for an even if it was virtual. One of our students said, “Our instructor had a virtual whiteboard that we could all do group work even when it was asynchronous, which is pretty neat.” So that helps set the stage for positive peer interactions, for peers to ask peers for help and mentorship, which is important. Often, students just feel that going to office hours is the only way that they can receive help. And when you provide opportunities to work together, learn together, that really helps, again, open up a line of communication among peers as well, which is a skill that we’re trying to teach our students.

John: And that was especially severe during COVID. But also, when we returned to the classroom, and students were asked to sit at least six feet away from any other student, it certainly reduced the amount of interaction and it has made it a little more challenging for all students to interact with others. That’s been improving, but I think, perhaps, that experience may remind faculty of the importance of building those types of connections. Because even before COVID, there were always some students who may not have felt as much a part of the class community. But I think we’ve all learned the importance of community during that time.

Rebecca: I think that’s just another example of something that students with disabilities have pointed out as being really important to them. But it’s also important to many other students, too.

John: The third theme that emerged from your research was course structure and design. And most of your findings in that particular category align with many other studies involving inclusive pedagogy and Universal Design for Learning. Can you highlight some of the common barriers that students with disabilities faced in terms of course structure and design.

Amy: So one of our students in the study commonly referred to one of their course LMS pages as a scavenger hunt, where they spent more time trying to find the materials and the information on the course rather than on the assignments themselves. So students in general benefit from an organized LMS and an organized syllabus for deadlines, instructions, policies are very clear and concise, but for students with disabilities, this is particularly important. Many of my students with ADHD, health or chronic medical conditions, or a learning disability, they need to plan ahead, because it might take the students double or triple the time to finish a task. So if students don’t know when their next test is, or if instructions aren’t posted a few days before something is due, we’re really not setting them up for success. And I also talked about some of those other policies and course design that might be inadvertent barriers to our students. And so some of our students reported that they did benefit from self-paced tasks, or on untimed learning assessments, having some autonomy and options for completing assignments in a different format, such as doing a presentation or a podcast, instead of a paper, working in groups or choosing to work individually on a project. Those are some of the specific practices our students highlighted as being really helpful. And again, we’re not lowering standards, they have to meet the same standards and learning objectives, as every other student, just perhaps meeting those same standards in a different way. And that’s what Universal Design for learning is all about.

John: One time in a workshop, a faculty member mentioned that they have students do a scavenger hunt in the LMS, to find various course policies, or to find materials. And I cringed at that and I suggested that it might be better to design your course in a way where the students don’t have to struggle to find things so they can focus their cognitive efforts on learning materials, rather than engaging in scavenger hunts, trying to navigate the course. Has that improved recently?

Amy: I think it has, again, in conversations with some of my colleagues who do this work and talking with faculty, I think it’s a mixed bag as it relates to how instructors are approaching course design in their policies. But other faculty are seeing that changing their pedagogy, changing their policies, changing the way they interact and see students and helping to meet those student needs have evolved, because perhaps they themselves have experienced accessibility barriers during COVID as well. And so it’s become more relevant, because they have that lived experience. And they’re seeing that adopting some of these inclusive practices are actually helping to keep their students engaged, that the students, even if they’re struggling, are more likely to tell their faculty member “I’m struggling and I need help, but I want to stay in this course, what kind of flexibility could be provided?”… rather than, we’ll use a college student term, ghosting [LAUGHTER] the class. So I think things are changing in a direction that speaks to some degree of flexibility and helping students meet those same standards, where the focus is more on learning, rather than adherence to an arbitrary policy.

Rebecca: I think the students really underscored maybe without realizing things like the transparency and learning and teaching or TILT, where being really clear and explicit about what the expectation is and how to get there and how you’re going to be assessed really helps and supports students… that structure and those guardrails is what all of us need. How many times have we worked on a paper the second before a deadline? We work on deadlines, and so if we help students with intermediary deadlines, we’re actually helping them and that doesn’t mean that we’re not flexible,and flexibility doesn’t mean not having those.

Amy: It’s about scaffolding. It’s about recognizing that not all students are coming from the same background and experiences and privilege. They’re not on the same playing field, and so providing those scaffolded learning opportunities… that can really help even the playing field, just providing those scaffolded learning opportunities.

Rebecca: And it’s really some of this scaffolded accountability, so it’s not all due at once, It’s helpful to faculty to remind them that there’s feedback throughout a process on a larger assignment, but also it’s helpful for students to hit individual deadlines to evolve their work as well.

John: And that’s something that is found, as you noted earlier, by Mary-Ann Winklemes in her research on Transparency in Learning and Teaching, and also by Viji Sathy and Kelly Hogan in their research on the importance of structure in reducing equity gaps. While transparency and structure benefits all students, it especially benefits the students who have equity gaps of some form, and it sounds as if that’s also true for students with disabilities.

Rebecca: Yeah, I think none of this is really new, but oftentimes students with disabilities aren’t necessarily included in those studies about equity always, it’s not always one of the groups that’s pulled out separately.

Amy: Part of what’s next is also hearing about the experiences of students with disabilities from other diverse backgrounds, including students of color, students from lower SES backgrounds, students in the LGBTQ+ community, that those experiences are different and that intersectionality is really key in understanding students’ experiences in the classroom and how we can be more accessible and inclusive because, again, accessibility is not just related to are we providing a legally required accommodation, but are we creating a sense of belonging in that space, and giving students an equal opportunity to demonstrate their knowledge and be successful, which is ultimately why we’re all here, I would hope.

Rebecca: So we always wrap up by asking what’s next?

Amy: I think it’s important to not just put a focus on what individual faculty can be doing in their classrooms to support students with disabilities. But how are we promoting access and inclusion at the institutional level, supporting students with disabilities and students from other diverse backgrounds is a whole campus responsibility and faculty needs support in doing that work as well. So I’m hoping what’s next is working with administration, other campus leaders and identifying ways we can really help move that needle in a meaningful way. Making accessibility into larger DEIB (or diversity, equity, inclusion and belonging) campus initiatives, our campus-wide policy, strategic planning, campus-wide faculty and staff training, and other professional development opportunities, hiring diverse faculty and staff on our campus. So not just about talking the talk, but walking the walk when it comes to access and inclusion in higher education.

Rebecca: I think that’s definitely a theme that we’ll see throughout all of higher ed. I hope that we’ll all go home and arm and move in this direction collaboratively.

John: Well, thank you for joining us. It’s been great talking to you and we’re looking forward to hearing more of your future work on this topic.

Amy: Well, thank you so much for having me today. I appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

319. AI in the Curriculum

Posted in edtech, educational technology, gender bias, higher education, learning, teaching
December 13, 2023

In late fall 2022, higher education was disrupted by the arrival of ChatGPT. In this episode, Mohammad Tajvarpour joins us to discuss his strategy for preparing students for an AI-infused future. Mohammad is an Assistant Professor in the Department of Management and Marketing at SUNY Oswego. During the summer of 2023, he developed an MBA course on ChatGPT for business.

Show Notes

Transcript

John: In late fall 2022, higher education was disrupted by the arrival of ChatGPT. In this episode, we discuss one professor’s strategy for preparing students for an AI-infused future.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Mohammad Tajvarpour. Mohammad is an Assistant Professor in the Department of Management and Marketing at SUNY Oswego. During the summer of 2023, he developed an MBA course on ChatGPT for business. Welcome Mohammad.

Mohammad: Hello, and thank you for having me here.

John: Thanks for joining us. Today’s teas are:… Mohammad. Are you drinking tea?

Mohammad: Yes. So I love tea. And from where I’m coming from, originally from Iran, tea is a big thing. So we have a big culture around tea. And it’s very interesting because we go to a coffee shop and we drink tea there. So we call it a coffee shop, but the most we get was tea. So I love brewed tea, and it’s kind of a time-consuming process, and it needs devices, tools that they don’t have here. So for a while, I tried tea bag, but I couldn’t connect well with that, so I decided to switch to coffee. But when we drink tea, we have rock candy. So we try to sweeten it with rock candy instead of sugar, because I love tea, and I’d love to drink my tea with rock candy. Now I drink coffee with rock candy, [LAUGHTER] which is a very funny mix, but it works for me. And time to time when I go to a restaurant that has middle eastern food, I get tea there and I really enjoy it. So that is a luxury for me. So it happens once a month when I get brewed tea, but I also like herbal tea, so I like mint tea and other types of herbal tea. I tried to get them mostly before bed.

Rebecca: Today I have blue sapphire tea, brewed fresh this morning.

John: And I have an English breakfast tea, but after a conversation we had earlier, I have some rock candy with saffron in it as a sweetener. So it is very good. So thank you for that suggestion.

Mohammad: Good. Good. Yeah, I have a big mix of rock candy with different flavors with different taste so I will bring you some, so you can try a different one. Good that you have the saffron one. I will bring a different version to you.

Rebecca: So we invited you here today to discuss the course that you offered last summer on ChatGPT for Business. Can you tell us a little bit about how this course came about?

Mohammad: So this course had a very interesting story. It was spring semester 2023, and ChatGPT was out from, I think, end of 2022, November, December. I was using that, and I really enjoyed how powerful the system is. I was following AI even before ChatGPT, and I was expecting such a thing to happen, but to be honest, I wasn’t expecting it to happen in 2022. I was thinking like 2027, 2030. But it happened, and I was so fascinated by the technology, by the quality of the answers that it provided. I was using it every day, to be honest, and I was trying different things with it, trying to find biases in it, trying to find how it can help me. And then it was the break, we had a week of break, spring semester, because reading break or spring break. And I made the first modules of the course event without discussing it with my department. I was so interested, I said, “Okay, let’s try,” and I said “worst scenario is I’m going to put it online for everyone to enjoy it. If the school doesn’t approve this course, then I will put this on YouTube.” So I’ve made the first module, and then we had a faculty gathering at this Italian restaurant in Liverpool, New York, called Avicolli. We were there and the director of our MBA program was there as well, Irene. So I told Irene, I have this idea of ChatGPT for Business, and I have worked this much on it. And she was so supportive, said: ”That’s a wonderful idea, let’s go for it.” So I sent her a proposal, and everything worked very well. And the school was so open to try new things, which I was very happy about. And then we made the course and submitted the proposal. It was approved, and we offered it in summer. That was the story, actually.

John: Could you tell us a bit more about the course? How many students were enrolled in it? What was the modality?

Mohammad: So, for our MBA program, most of our MBA students are professionals. They have a career already, they’re working full time, and then they’re getting their master’s degree, their MBA, actually, to move forward with their career. Many of them already have master’s degree, they may be doctors, they may be nurse supervisors, so the modality that we use for summer courses is mostly asynchronous online, which means we record the session, we put it online, they take online exams, and we go that way, we communicate online. For this course, I designed it in three modules. In the first module, we discuss the ethics and foundations of AI. We discuss how ChatGPT was trained, what was the data that they use? What are the biases that can happen? How can we use this system ethically because there are so many things that we can do with AI, which are very good things. And there are so many not right things that people can use AI for. So we wanted to make sure about the ethics first. And every course that I want to design on AI, I will start with ethics and foundations, because I think that’s the most important element. So we discussed the biases in AI, for example, gender biases, racial biases that may happen if we solely rely on these systems that are trained on biased data from internet, let’s say. So we discussed that. The second module was on prompt engineering. So as we know, prompt is the query that we sent to the AI, that’s the ChatGPT or Bard. So the quality of question that we ask is directly related to the quality of answer that we get from the system. So we want to make sure we ask questions that give us the best answers. And most of the time it’s not one question or one prompt, it’s a sequence of prompts. So we call it a prompt flow. So, at the first round, you may not get the best answer. But as you improve it, you will get closer and closer to what you want. And that’s what we did in the second module. So we designed an eight-step method for prompt engineering. And there are different stages actually in it. So for example, in one step, you have to anonymize the data to make sure that privacy of your client is considered. You want to set the context for the system, so it understands its role in helping you do the job, etc, etc. So we call it the Kharazmi prompt engineering method, which is named after the person who developed the algorithm, actually. So we made that 8-step method, and it worked very well for my students. In the third module, we went one step further. So as you know, these large language models are very efficient and very effective in writing code in different languages. So one of the things that I tested ChatGPT for in 2022, early 2023, was writing codes with it. So I gave it a task and asked it to write the code for me in R, Python, Stata. And it was so good at writing efficient code in these languages. I even used it to optimize my code. So I intentionally, for example, gave it a for loop in R, to see if it can optimize it. And as you know, in R, we can use sapply(), or lapply() to optimize for those. And it was so good at getting it. So I found that it’s very helpful with coding, with programming. And we made the third module actually on data analytics, which requires a lot of coding. And many of the MBA students, because of their background, they’re coming from degrees, or fields that have nothing to do with programming or coding. They have to use it time to time, they have to read the output, but they may not have written their own code. So in my class, I had a student who said, the last time I wrote the code was 20 years ago, that was like the diversity of my class. And I had the students who had taken economics, and they did a lot of coding. So we made the third module on data analytics and how we can use ChatGPT to write us the code and help us with data analytics. And it was wonderful to see that the students with no background in programming tin either R or Python, were able to write code, they’re able to debug code. So I intentionally gave them codes that had some intentional error. So I removed a part or I removed a small comma there, and they were able to debug it in a couple of seconds. And that was one of the fascinating parts of this course. And interesting, I had a student who told me that our company was moving actually from one software to another. And they used ChatGPT and what they learned in that class to migrate their code from one language to another. So with regards to enrollment, we had a lot of interest. So we had so many people who registered for the course and we had so many who were in the waitlist, but we had to make it small cohorts because we wanted to give very personal attention to each student to make sure that everything goes well. So we limited the enrollment to 12. And we promised the rest that we will offer this course again, and you will have a chance to take it. So we had a cohort of 12 MBA students, and understand the MBA students, as I mentioned, they’re professionals. So in class we had a very high profile journalist, three times Emmy Award winner journalist, we had a neurosurgeon, we had a CFO, we had an activist who was running for office. They had so many different backgrounds that helped actually enrich the learning for everyone. I was learning from how they are using the system for their own specific niche. And that was wonderful, I would say, learning process for everyone.

Rebecca: With the diversity of students that you had in your class, can you talk about some of the kinds of activities that they did individually or together?

Mohammad: As I mentioned, the course was asynchronous, because of the course that we have at SUNY-Oswego, most of our MBAs are professionals. So we intentionally try to make, especially summer courses, asynchronous online. But the level of enthusiasm in this class was so high. So we set up weekly meetings. And most of the time we did it during lunchtime, because everybody was working, that was the best time. In my situation, I think we set the time for 6pm, so 6pm we were on Zoom discussing the module that you have learned that week. So there was a lot of interesting discussions in those sessions. I think one of the best discussions that we had was about ethics of using AI. People from different areas were talking about how these biases can affect, let’s say, patients it has, how these AI tools can be used for fake journalism, making fake news, and what are the dangers of that. And then we discuss the inherent biases in the system. So ChatGPT was trained on data that was on internet, data on internet was created by human beings, human beings are prone to biases, those biases will be transferred to the system. So we discussed that. And we had a very healthy discussion about the need for diversity in data, and diversity on the teams who work on this data to train the models. Because if the team members are diverse and sensitive to different issues that may happen, they will make an effort to fix it. So I think the most interesting part for me was the discussion of ethics, and the wrong and right ways that we can use AI and how we can mitigate those biases or harmful uses of AI.

John: Many people in academia are talking about AI and the need to train students in the use of AI. Could you talk a little bit about some of the ways in which AI tools are already being used in business applications.

Mohammad: I will go from academia point of view and how students are using it day to day. And then some of the uses of AI in industry. So in academia, the very basic things that students use AI for are about, let’s say, summarizing a big text. And that’s what I teach them actually, in any course that I have. I’m teaching marketing research, I’m teaching principles of marketing, any course that they teach, I remind them that, okay, you have this big article, and you want to read that, you don’t have time for it, ask ChatGPT to summarize it for you. It helps us read more and more articles, more and more books. So that’s one of the things that people can use it for. The other thing that I have seen many of our international students actually use AI for, to improve their writing skills. So you’re an international student, you have wonderful idea, but you don’t have the best writing skills, writing experience in English. You can write wonderful articles in your own language, but when it gets to English, your vocabulary is limited, you may make grammar errors. So they use it to improve their writing. And in all my courses, I tell them, I’m more than happy to see you use AI to improve your grammar, to improve the flow of your writing,and to check for any writing errors in your text. So that’s totally fine, If they use it for. And there are many other things that the students use it for, for example, they use it to generate individualized examples. So let’s say you’re a student, you have a small problem with one of your courses, let’s say calculus. There is no good example in the textbook, let’s say. But you can ask AI to generate an example that will help you understand that specific niche research problem that you have. So that’s what I see from different areas, use AI for their coursework. When it comes to industry, it’s an abundance of AI use. So many marketing teams are using AI to generate content, especially a start. Because then you’re a startup and you’re a small business, you don’t have a marketing department. You’re one person, you’re the CEO, you’re the CFO, you’re the HR, you’re the marketing manager, you have to do all those jobs, and these LLMs, these large language models, these AI systems, help entrepreneurs to do the marketing and many other aspects of their business on their own. If you want to create content for your social media, ChatGPT can do that for you. You want to make a job posting, ChatGPT can take care of that for you. And then you can focus on improving and developing your business.

Rebecca: I want to circle back to some of the ethics questions that you were grappling with in class. I’m hoping that you can share some more details about the kinds of conversations that you had with students around ethics? Because this is a topic that I think comes up a lot for faculty, in particular, in thinking about how they might want to encourage or discourage students from using tools like ChatGPT.

Mohammad: Definitely. So what we did at SUNY Oswego was we set up an AI committee, I’m talking about the School of Business, I’m sure other schools are doing the same. So we set up an AI committee to make sure that we have a certain policy or certain plans on how we want our students to be trained and use AI. Because it’s the new computer, it’s the new calculator, it’s the new Wikipedia. We cannot stop people from using it. So we want to train them on the use of AI with integrity, we want to make sure that they are using it in an ethical way. So what we did was, we developed three different policies for courses. For some courses, very fundamental courses, we don’t want the students to use AI, because we want them to learn the tool. For example, in calculus, we want them to learn the mathematics behind doing the calculation. Or let’s say in marketing, we want them to understand the fundamentals of what’s the target market, how we can pick the target market, how we can make a fit between our business offering, and what the target market needs and wants. For those fundamental courses, we either ban use of ChatGPT, or we make it very limited to certain purposes, for example, you can use it to fix the grammar in your writing, you can use it to improve the writing of your assignment. Then we have a second level use of AI. Some courses, we are fine if a students uses it to generate some ideas for them to help them do assignments, create examples for them. And then we have a third layer, which is we ask them to use AI. So we tell them in the syllabus that you’re not only are allowed to use AI, you are expected to use AI, text to text AI, text to image AI, text to voice AI, all of that to improve the quality of assignment that you submit, to improve the quality of the projects that you do for this course. For example, for ChatGPT for Business, in the syllabus, it said that you’re learning text to text AI, but you’re expected to use other types of AI when you do your assignment. And many of my ChatGPT for Business students actually use that and they develop logos and many visuals for the assignments totally generated by AI.

Rebecca: Can you talk a little bit about what came up in those conversations in class about the ethics and how they’re using it in different ways. So if they’re using it for images, or they’re using it to write code, or all these other varieties of uses that you’ve outlined.

Mohammad: So one of the discussions that we had around biases, we discussed how gender bias may be inherent in those AI systems. And when we talked about it, it’s not just ChatGPT, any AI system can be prone to those biases. For example, our facial recognition systems, they’re mostly trade on Western pictures, faces from Western people. So they may not do well when it comes to let’s say, African Americans. And they may cause a lot of bias. We have cases actually of that in the news. So that was one of the things that we discussed. And one of the conclusions that we had in those discussions was that it’s not just about the data to train the model it’s about the team that is working on that. The team needs to be diverse enough. If you have African Americans, if you have different ethnicities, if you have different genders in the team, then we’ll be more sensitive to these biases, and we make sure that these are not happening. The other thing was about gender bias. So let’s say the system was trained on data that we had on the internet, go check the Fortune 500 list, the CEOs of Fortune 500 list, the majority of them are male CEOs. So if you train the system on that type of data, it will assume that males are better at doing those jobs, which is wrong. We had a very healthy discussion about that, or different ethnic backgrounds. So if you check the top 100 US companies, only eight of them have African-American CEOs. So when you train your system on that data, you are making inherent biases in the system. The bias is in the DNA of that system, let’s say. So we want to make sure that we at least have those biases in mind, so we are not solely relying on AI for any purpose that you’re using it for. So AI is now being used and ChatGPT… companies are using that, but sooner or later, governments will start using AI. They will use it for let’s say immigration purposes. Just imagine how those biases can affect people’s lives actually. Health care will start using that. So there are so many dangerous decisions that doctors can make. There’s so many things can go wrong with solely relying or blindly relying on AI. And that was one of the biggest things that we discussed. So we want to use it to be more efficient, and sometimes be more effective. But we want to use it with supervision, somebody should check the output, someone should read the output carefully. That person should be aware that these systems are prone to many errors, many biases. So that was one of the discussions that we had. The main thing, I think, that we discussed regarding biases and errors was gender biases and ethnic biases in AI. And then we discussed the wrong ways of using AI. One of the main things that we discussed was fake news. So somebody can make fake news, make a fake Twitter account, and keep posting with the same language that a certain politician is doing. And, as we know, it’s not just text to text AI, you have text to voice AI. So we can give it a sample of a person’s voice, and it can generate the same voice. So just type the speech for the AI and we’ll read it with the same voice. So there’s so many things that can go wrong, especially when it comes to disinformation and fake news.

Rebecca: it seemed like one of the other ethical areas that you talked about, based on what you had said previously, is about data, the data inputs that train the systems, and also the data that you’re putting into the system that you might be analyzing. So there’s privacy issues, copyright issues, etc. Can you share a little bit about how those conversations unfolded as well.

Mohammad: So, for example, one of the ways that people are using it, especially many doctors are actually using ChatGPT to ask it questions. For example, what are the side effects of this new medicine that I’m using. So sometimes you’re inserting private information to the system. So in the prompt engineering session that we had, one of the steps was anonymize, we write the prompt for the system, then we check it for any private information. It can be a name, it can be an address, it can be even a vehicle plate number. All of those should be removed from your prompt, before you submit it to the AI, because you never know what happens to that data. So one of the things that we did was to make sure that no personal or private data is being inserted into the system, at least for the systems that we have right now. In future, we may have private GPTs. So your organization may have an institutional GPT, that makes sure that all the data is private, it may change then. But the systems that are general purpose right now, Bard, ChatGPT, any other system, we want to make sure that the data that we insert into the system is totally anonymized, no private information is being sent to the system, even an email address. We use placeholders for that in our course, to make sure that even emails are not being fed to the system. The other important question that you raised was about copyright. So there are two things with corporate. First, the systems were trained on content that was generated by a person. So what if I asked AI to generate content similar to that? So write me a Harry Potter story, for example, exactly use the same language that JK Rowling was using? What happens then? That’s a big question. The other concern is who owns the copyright for the output that we get from Ai? For example, in my courses, I’m redesigning all my PowerPoints. And I’m removing all the images that I was using before with images that AI has generated. So when AI generates those images for me, who owns the copyright? Is it ChatGPT? Is it is Dall-E? Is it Midjourney? Or is it the person who directed the system to make those content? So at least for ChatGPT, based on what they wrote on their website, they don’t assume any copyright for themselves. The person who’s generating or giving the prompts will own the content. So at least we know that’s the answer to that question for one system, but what happens in future? There should be lots and lots of discussions on copyright, who owns the copyright of the output? And if the system was trained on somebody else’s writing, somebody else’s art, who owns the output? If I prompted it to write a JK Rowling Harry Potter for me, do I own the copyright or do the original writer usually will get the copyright of something that I’ve prompted to ChatGPT? So I think one of the biggest questions that we have had is regulations. How do we want the regulations to evolve in a way that accommodates all these questions that we have today? I think the pace of change is very fast. So policy makers, those who are setting the rules, should be very fast in responding. The technology’s not waiting for anyone, they have to be as fast as these changes in the system are, otherwise there will be chaos, there will be a lot of unanswered questions, and it will go in any direction that we cannot expect. So one of the big things that should happen, I would say, is regulation. We need to regulate the system in a way that fosters improvement, but at the same time, protects people.

John: In addition to all discussions of regulations that are going on globally, there’s also quite a few lawsuits going on in terms of potential copyright violations, which could have some really devastating implications on the development of AI. So a lot of this, I think, we’ll have to just wait and see, because it’s going to be challenging.

Rebecca: A number of interesting cases too of folks trying to register things with the copyright office that were generated by AI that have been denied. So lots of interesting things to be watching for sure.

Mohammad: Definitely.

John: Another area of a lot of concern, and a lot of research that’s beginning to take place is to what extent AI tools will enhance the productivity of workers, and to what extent it may end up replacing workers. And there are some studies now that are finding both of those. Were your students very concerned about the possibility that some of their potential jobs might disappear, or substantially alter, as a result of AI tools.

Mohammad: So I think the best saying with regards to jobs is that nobody will take your job, let me say it in different words.The CEOs who can use AI will take place of CEOs who cannot use AI. So it’s not, “you’re going to lose your job to AI,” it’s mostly about those who are not equipped, those who don’t know how to use AI, will be replaced by the ones who know how to use AI. In short term, there may be some changes in the job market, some of the jobs may be automated, but new jobs will be created. For example, now we have a lot of companies looking for prompt engineers, something that wasn’t there before, like a year ago we didn’t have such a need in the market. So the other thing that will happen is that we need to train people to use AI. But at the same time, the pace of change is so fast. So we train people for a year to take AI jobs. And by the time they finish their education, the system has changed. Now you have to retrain them. So that’s one of the things that is happening and educational institutions should find a way. They should keep updating and updating their curriculum, I would say every day, to keep up with the changes in technology. The other thing that I personally expect and hope to happen in the long run is that we will work less. In the Industrial Revolution, our working hours were reduced, we could do the same amount of productivity with less work. Same thing may happen 10 years from now, five years from now. Instead of working nine hours, we may two hours, three hours, a day, and then be even more productive than what we are right now. Because this system can make us be more efficient. There is a good metaphor that people use for AI, they call it human algorithm centaurs. So in Greek mythology centaurs are half human, half horses. They can be as fast as a horse and they can have the human intelligence and human capabilities. Now we have half human, half algorithm, we can do so many things much faster, much more effectively than before, and will increase the productivity manyfold. So I’m expecting a better life actually for human beings, morat the same time being more productive than before.

Rebecca: It’s interesting, some of the kinds of conversations I’ve had with my students who are design students about AI, have really been about is it going to replace a designer? Well, maybe in some contexts, people are going to use AI to create designs or visual elements, it’s not going to have the same thought [LAUGHTER] and strategy necessarily behind them that a designer might use. But what they’re mostly discovering is that AI is really helpful in making the process faster. So generating more ideas, finding out what they don’t want to design [LAUGHTER] and getting just a place to start and moving forward and developing their work more rapidly. And so that really gets to that efficient idea that you were just talking about.

Mohammad: That’s very true. And I agree with you, sometimes you are just thinking and you cannot start. AI can give you an idea to start with. And then you come up with ideas that you wanted. So regarding the design jobs or any job. I have students who will come to me and say, “Should I change my field to AI?” I said, “No, do what, whatever you’re interested in, if you’re doing design, keep doing design; if you’re into, let’s say, marketing, keep doing marketing; if you’re in finance, keep doing finance; but use AI in your field. If you’re doing design, see how you can use AI to design better. If you are doing marketing, see how you can use AI to make better content, to make better decisions.” So I think it’s not AI replacing people, it’s AI enhancing people. So in any field, we have to equip ourselves with the skills of using AI to do our jobs better.

Rebecca: From an experience I’ve had with my students, we’ve definitely discovered that if you don’t have the right language around the thing that you’re trying to make, it doesn’t do a good job. [LAUGHTER] So you need some disciplinary background or some basic knowledge of the thing that you’re trying to do for it to come out successful.

Mohammad: That’s very true. So one of the limitations of AI that we discussed in our classes was about different languages. So most of the content that was used to train ChatGPT was written in English. So think of other languages that didn’t have that much content on the internet. AI is not as capable in those languages. So that’s one of the things that we need to think of. So this is a system that is super capable in English language, but when it comes to languages that don’t have that many speakers, then it falls behind. So I tried it and I learned that sometimes the system tries to think in English and then translate it in the other language, and it makes so many mistakes in that process. So that was one of the things that came to my mind of what you mentioned..

John: We’re recording this in the middle of November. And in just the last few weeks, we’ve seen a lot of new AI tools come out, we’ve seen ChatGPT expand the size of the input that it’s allowed, and we now see this market they’re offering for GPTs, as they’re calling them. And the pace of change here is more rapid than in pretty much any area that I’ve seen, at least since I’ve been working in various tech fields. It would seem that this would be a challenging course to teach in that the thing you’re studying is constantly changing. Will you be offering this again? And if so, how will the course be different in your next iteration of the course?

Mohammad: That was a very good question, actually. So yes, the course is being offered in January 2024. And as you mentioned, one of the biggest challenges with this course, I would say the biggest challenge with teaching AI, is to keep the content current. So that’s not just what happened today. When I was teaching this course in summer, I made the second module, and then open AI announced the plugins. Now I had to redo the content to make sure that I can use those plugins because they were so powerful. The plugins that ChatGPT introduced were so powerful, and there are so many companies who were making different plugins. So I remember, for the second module, I had to start and re-record my content. I updated my content. I recorded everything 1am, 2am before the session in the morning, because everything was changed. So I had to incorporate that into my class. Same thing is happening with new developments. So what I learned is that every day I have to update my content, I have to update my course. So ChatGPT API was one of the things that I was thinking of as the fourth module and was working on that. Now. I think GPTs is one of the modules that needs to be there. That’s like the app store of Open AI. So, that’s a big game changer. As you mentioned, it has a larger memory right now we can provide it larger context. So that’s another capability that AI has, and it changes the way that we prompt it the way that we ask it questions. So keeping the curricular updated, I think is the biggest challenge. And this is something that we should have in mind. Every week, every day I see something new. I update my slides, update my content to make sure that everything is correct. Because if you don’t do that, let’s say two months, three months, if you don’t update your content, then you have to redo it, you h ave to start over. So that’s definitely one of the things that I do and GPTs is one of the things that I will definitely incorporate into my course for January 2024.

Rebecca: Iterative change definitely seems like a good way to go to manage that, for sure.

Mohammad: We don’t know what will be announced in December. [LAUGHTER] So, I always count on a big change.

Rebecca: But yeah, buckle up and be ready, right?

Mohammad: Yeah.

John: And we welcome our new AI overlords…

Rebecca: Yeah.

John: ///n case, by the time this is released that they have taken over.

Rebecca: Can you talk a little bit about how your colleagues in the School of Business have responded and whether or not more faculty in the School of Business are incorporating AI.

Mohammad: I see that many of my colleagues are super interested in this new technology. So what I like most about SUNY Oswego in general is that everyone is so open to accept new technology, accept new things, accept innovation, and everybody’s trying to absorb the new innovation that we have seen and incorporated one way or another into their work or into their courses. So as I mentioned, we have the AI committee, and in our meetings we have very good discussions about how we should update our curricula. I know that some of my colleagues are already doing that, are already using AI to generate let’s say, visuals for their content, or teaching or talking with the students about the ethical users of AI. So I think at least the ecosystem that they see at SUNY Oswego is very open to accepting innovation, and is very fast to incorporate it into their curricula and educate the students, or at least have discussions with the students about how to use that and how to equip themselves with the skills that they need for future.

John: Just a few weeks ago, your department scheduled a symposium on AI, could you talk a little bit about that?

Mohammad: So we wanted to take a lead in AI education at SUNY-Oswego. So we’re very focused on teaching the students and equipping them with the skills that they need to take future jobs. And we are making a big move toward AI. So we wanted to make sure that our students are exposed to the new developments in this field and understand the importance of this area. So we set up an AI symposium, Bridging Bytes and Business to show them how technology, how AI, how computer, is changing the way that we do business. So we set up a hybrid conference or symposium. They had two panels. The first part was online with scientist discussing the new technology, discussing how AI is evolving. What are the biases, what are the errors that we have in this AI? And they were discussing what is the next big thing that will happen in AI? So in the first round, we had Suroush Saghafian from Harvard. He has a lab that works on developing AI, we had Diane, Diane is a three times Emmy Award journalist, and she was one of our MBA students, actually. And she talked about how AI is used in journalism, what are the challenges of, let’s say, disinformation generated by AI, how journalists need to address those concerns. And we had Saeideh who is a computer scientist. Saeideh worked for Yahoo, Meta, and Google. And she gave us her knowledge, her experience with what these big companies are working on for the next big thing that is happening. So we had a very healthy discussion about the science part of AI. And then we had the business leaders from upstate. We had Michael Backus from Oswego Health, we had John Griffith from insurance, and we had Mohamed Khan from Constellation Energy. So they were discussing how their companies, how their industry is using AI, and what they expect students to know about AI before they go to the job market. What are the skills that they need to have? So we had this very successful symposium, and since it was a hybrid symposium, we’re broadcasting it online. It was kind of a webinar. So we had many attendees from all over the country. So we had attendees from all over U.S. I think we had California, we had Texas, we had Arkansas, New York, obviously, we had people from Canada joining us, Ireland, United Kingdom, France, Germany, and interestingly, we had attendees from Australia. It was 2 am there, I think, but they joined us, and they stated very last minute of the symposium. And that made us very happy and very proud of SUNY Oswego on taking the lead in providing this type of discussions actually around AI. And we’ll keep doing that. We’ll keep having more and more symposiums and panel discussions to keep our students current and to encourage our students to learn more and educate themselves more about AI.

Rebecca: So we always wrap up by asking: “What’s next?”

Mohammad: So we have big plans. One of the things that we’re doing is ChatGPT for Business, it will be offered again in January 2024, and hopefully summer, but aside from that, we are going one step further. We are designing a new course, more advanced than ChatGPT for Business. That course is Prompt Engineering for Artificial Intelligence. So in that course, we’ll focus on different ways that students can use prompt engineering for different purposes: for HR, or marketing, for finance, for different fields. So that course will be an advanced level to ChatGPT for business. And we are going to offer degree in our MBA program on strategic analytics and artificial intelligence. So we are incorporating AI into actually all courses that we offer in that program. And then we will have a micro credential on prompt engineering, because that’s what industry is looking for. They want somebody who is good at asking the right questions from ChatGPT, Bard, and any other AI that you’re using. So they need somebody who is good at writing good prompts for them. So that’s what we are focusing on right now, to equip our students with those skills, with the knowledge that they need to be effective and efficient prompt engineers. And I believe we will be among the very first institutions in North America to offer those courses and those degrees, actually.

Rebecca: Well, thank you so much for joining us and sharing the work that you’ve been doing.

John: We’re always curious about where this is going, and I’m sure we’ll be back in touch with you again in the future. So thank you.

Mohammad: Thank you very much. I really appreciate the wonderful podcast that you have. I time to time listen to your podcast, and I actually bought a book on ChatGPT, based off of one of your podcasts, one of the guests that you had, they wrote a book on ChatGPT, 80 Ways that ChatGPT can help you with your courses, I think. And I’m still reading that book and I’m enjoying that. So thank you for the wonderful podcast that you have.

John: And we’ll include a link to that book by Stan Skrabut, and we’ll also include a link to the recording of that symposium as well in the show notes for this episode.

Mohammad: Thanks so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

318. Reducing Equity Gaps

Posted in active learning, diversity, equity, evidence-based practices, first-generation, grading, higher education, inclusion, learning, pedagogy, race, STEM, teaching
December 6, 2023

Gender and racial equity gaps exist in economics and other STEM fields. In this episode, Tisha Emerson joins us to discuss research on strategies to reduce these inequities. Tisha is the chair of the economics department and the James E. and Constance Paul Distinguished Professor at East Carolina University and is the incoming Chair of the American Economic Association’s Committee on Economic Education.

Show Notes

Transcript

John: Gender and racial equity gaps exist in economics and other STEM fields. In this episode, we discuss research on strategies to reduce these inequities.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Tisha Emerson. Tisha is the chair of the economics department and the James E. and Constance Paul Distinguished Professor at East Carolina University and is the incoming Chair of the American Economic Association’s Committee on Economic Education. Welcome, Tisha.

Tisha: Thanks for having me. I’m so glad to be here.

John: I’m very happy to be talking to you. And our teas today are:… Tisha, are you drinking any tea?

Tisha: Yes, my favorite tea, I found a couple years ago, it’s a Golden Moon brand of Tippy Earl Grey.

Rebecca: That sounds nice.

Tisha: It’s delicious.

Rebecca:I have Harvest Memories today.

John: I don’t remember that one.

Rebecca: Well, it is from my little favorite tea shop in Canandaigua, New York, and it’s autumn flavors.

John: Autumn flavor?

Rebecca: Autumn flavors.

John: Do they have leaves dumped in there?

Rebecca: I mean, it almost looks like that. [LAUGHTER]

John: Ok. [LAUGHTER] Well, that’s one way of getting rid of all the leaves that have been falling. And I have English breakfast tea today. So we’ve invited you here today to talk about some of the research you’ve done, and also your new role and your past role with the AEA’s Committee on Economic Education. One of the things that we’ve observed is that there’s some significant equity gaps in economics and in STEM disciplines in general in terms of race and gender. And in one of the studies that you’ve done with this, in a 2023 paper in the Southern Economic Journal, you and KimMarie McGoldrick examined retake behavior for students who are not successful in their initial attempt at completing an introductory microeconomics class. Could you give us just a general overview of this study?

Tisha: First, let me say that this started a long time ago. And we got access to this great data, the MIDFIELD dataset, that was actually originally funded by the NSF to study the gap in engineering education, but it’s student transcript data. So we said, well, you could use that for economics too. And we have. And so this particular study looked at almost 180,000 students who take Principles of Microeconomics for the first time. And what we ultimately wanted to do is to think about the likelihood of success, and actually more so, failure, because a lot of papers already look at success and they stop there, because you have too small of a sample, fortunately, of the failures to continue on to look at them. But when you start with 180,000, you have enough to continue. So we find, as many others do, that, of course, aptitude matters, but that, unfortunately, females tend to be much more likely to be unsuccessful in their Principles of Microeconomics class, as are underrepresented minorities, or URM. And then we follow those students and we say, okay, so if you were, in fact, unsuccessful, and we define that as a grade of a D or an F, so grades that wouldn’t really let you continue in your study of economics, or you withdraw. And we say, if you’re in that category, what do students tend to do?… or first of all, for those students that are unsuccessful, what helps predict that? So given that you are in the unsuccessful category, what is more likely to cause that? And we saw that students who were carrying more concurrent credit hours were more likely to withdraw. Which makes sense, because if you have constraints that lead you to want to be full time, if you start with 12 hours, and the course is not going well… any of them… and you were to drop, then you would not be full time anymore. And so it gives them sort of this flexibility, they have more concurrent credit hours. But we found that students who got D or Fs, given that they were unsuccessful, they tended to have taken more related courses. So things like accounting (financial accounting in particular), calculus, and macro principles. And so we didn’t actually see a lot of gender or racial differences there. But then those students who were unsuccessful (D, F, or W), on retake decisions, we found that women were much less likely to retake if they were unsuccessful. But we did find that women who were of higher aptitude, who were unsuccessful initially, they were more likely to retake. So that was a positive. And we did find that underrepresented minorities were more likely to retake the course, but then more likely to be unsuccessful again, on their second attempt. So there’s some good news, but also, a lot of not so good news.

John: So you were able to follow students over time, how long a time period was there in the data set that you were analyzing?

Tisha: We had over 20 year period of data, but that doesn’t mean we followed any particular student for that length of time. Once we saw that they were unsuccessful, we made sure that they had at least one more semester at the institution so they did in fact have a chance to choose to retake the course or not and then we gave them three years post their initial unsuccessful attempt. And if they didn’t retake it in that period, then we said they chose not to retake.

Rebecca: In a 2019 study in the Journal of Economic Education, you and KimMarie McGoldrick use the same dataset to examine the decision to switch into an out of the economic major. In this study, you find that very few students selected economics as a major when entering college and that 83% of economics majors actually selected the major after taking their first economics course. What did you find in terms of the gender and racial composition of those that selected the economic major?

Tisha: So what we found was that, for the few students who started out as economics majors, so going into their principles class they already had said they were majoring in economics, we found that females were as likely to persist or not. That’s good. And we found something similar for underrepresented minorities, that is the females and URM students who went into the principles class planning to major in economics, they were as likely to persist as not.

John: What happened to those students who were not majoring in economics.

Tisha: Well, unfortunately, females were not as likely to persist and decide to major in economics. There was a larger proportion of females who opted to remain in their original major. So, they come into the principles class, they have some major other than economics, they were much more likely to stay in their other major and not switch to economics. Men were considerably more likely to switch, on average, than females were. But on a more positive note, we did find that students from traditionally underrepresented racial and ethnic minorities and Asian students were more likely to switch to economics.

John: And one thing we should note, and you mentioned this in your study, is that because this datset was focused on engineering, the schools in this sample have relatively large engineering programs and some students switched to economics because economics was perceived as being easier than their original major. So some of this may not apply as broadly to liberal arts institutions that do not have engineering programs.

Tisha: That’s possibly true. And in fact, business majors were even more likely to switch to econ. So it was about 10% of those switching in were switching from engineering and 27% were switching from business. So at the liberal arts schools, to the extent that they don’t have business or engineering, there wouldn’t be that source of majors coming in.

Rebecca: Both of these studies are really interesting. But, how do we think about some strategies to create a more inclusive environment in economics and STEM classes? So what did these studies tell us? What should we be doing?

Tisha: Well, I don’t know that they tell us what we should be doing. They tell us that what we’re doing is not really working.

Rebecca: Fair.

Tisha: …which is unfortunate, because that’s what we’ve been doing for a really long time and we’ve been really unsuccessful at attracting women and underrepresented minorities. So there’s a new sort of strain in the literature that’s really trying to address some of this. And there’s a study, for example, by Porter and Serra, and that was published in 2020. They did a randomized control trial at Southern Methodist University, where they, just by happenstance, picked a couple of very charismatic female alums and they randomly selected which principles classes they would speak to. And this was just like a 15- or 20-minute exposure, where they talked about majoring in economics, and how that helped them in their career. And they found that this significantly increased the number of women from those courses who decided to major in economics and additionally, not necessarily just major, but take more economics courses. And I thought it was really fascinating that there was no effect on men in those treated classes, it was just the women. So that suggests possibly that there’s some room for role models. Other work that looks at role models from other directions, like some of my own work with KimMarie and John Siegfried, didn’t find any evidence that supports the idea of role models. So I think, still mixed, and the exact sort of interaction that students may or may not have with these people I think is going to be important.

John: You’ve done a number of studies on these types of issues, and one of them was looking at the effect of classroom experiments. Could you talk a little bit about what types of classroom experiments have you looked at and what’s the impact of those on students?

Tisha: Sure, I talk about that a lot, because I really love the classroom experiments. I think that they’re a great pedagogical technique. And so since your audience is more general, maybe I’ll explain just what we mean by classroom experiments. And that’s basically the idea that we’re going, in some cases, simulate markets or other decision environments for the students that will mirror the types of environments that we’re talking about in class. So for example, if we’re talking about a market, then we will have students participate in an experiment where some are buyers and some are sellers and they negotiate and trade. In my classes, we always do the experiment first, I don’t talk about any of the concepts. And then we collect the data as we’re going through the experiment on the decisions that they’re making: prices, quantities that are traded, then I’m able to talk to the students about “Well, look, this is what actually happened, this is your actual data that you generated, and now let’s talk about the theory.” And this is what the theory would predict the outcome to be. And they can see how well they match up. And they are able to discuss in class, this is why I chose to behave in this way. And they see how it fits in to the economic theory. And I did a study back in 2001, which was published in ‘04 in the Southern Economic Journal, where we were looking just at the effect of experiments and we looked to see if there was a differential gender effect. And in fact, there was. So you often see that women underperform men in these principles classes. But what the experiment did is it closed that gender performance gap. So it raised sort of everybody’s performance, but it raised the females disproportionately, so that they were closer to the same outcome as the men if they were in a classroom with experiments. So I do think that there is definitely room through pedagogy to improve women’s outcomes, and make them more attracted to economics, but also make economics more attractive to them. There’s a large literature that suggests that women are much more grade sensitive than are men, various people have gone about showing this in different ways. And if you can bring up their performance so that it’s on par with men. And it’s not really even that comparison, but you’re bringing up the females’ performance, then they’re getting better grades, and they’re going to be less likely to want to leave the study of economics.

John: And we’ll throw in a reference to an earlier podcast we did with Peter Arcidiacono, who had looked at that very impact in STEM fields in terms of the decisions of people to move from one major to another. And in particular, in that study, they found that women did better on many of the tests but still left because the grades were relatively lower than in other classes. So one issue I think might be worth addressing is the issue of grade differentials, that if we’re going to continue to assign grades that are lower than most other departments, there’s a good chance we’re going to lose majors in general, but disproportionately more female majors, based on what the research literature tells us.

Tisha: There’s this really cool study, McEwan, Rogers, and Weerapana, 2021. They’re at Wellesley, and so it’s all females, but they were trying to address grade inflation. And what that meant was, of course, there are some disciplines that have higher grades, they had to bring their average grades down, and economics is treated in the opposite direction, and that it actually drew more students into these harder grading disciplines, like economics and the other STEM fields. And so yeah, that’s certainly an issue. I don’t think we’re gonna across the board be trying to treat grade inflation. I don’t think this is published yet. But I was reading another study out of Wellesley, where they were allowing students… I think it was maybe for their gen ed classes… to take them all pass fail. And again, you saw students flowing into these harder grading disciplines.

John: I know a number of schools during a pandemic made all of their first-year classes pass fail, which is a good way of letting students get into a discipline without worrying so much about the grades so they can explore things without worrying about the impact on their GPAs. That might be another useful strategy.

Tisha: Yes, I agree, and then this is really anecdotal, but I’ve heard other people observe the same thing, that women, they’ll be in your principles class, and they’re doing well, but maybe they’re going to get a B plus. And they say, “Oh, either I’m not gonna take another economics class,” or I’ve even had students withdraw at that point, and I’m like, “but you’re getting a B plus, I don’t understand,” but they’re so sensitive to grades. And then there will be male students who are getting a C, barely, and they’ll say, “I’m going to major in economics.” and I say, “Okay, are you sure?” It’s hard to understand sometimes.

Rebecca: It’s interesting to think about withdrawal policies that have really kind of loosened up due to COVID. So our institution changed the policy more recently so that students can withdraw through the last day of classes without documentation. And so students who are doing really poorly might take that option, but as you’re discussing students with higher grades withdrawing, that sounds really concerning that policies like that could be kind of counterintuitive, or really have some other negative impacts. Classes all cost money, [LAUGHTER] and there’s financial impacts to withdrawing.

Tisha: It’s hard to really predict how this is all going to play out. And I have to say, I’ve never heard of such a generous withdrawal policy. At my previous institution, I think they could withdraw up to sort of two weeks out and not even have a W necessarily on their transcript. So this all seems like a lot. There’s a lot of moving parts. But on the positive side, so let me just say, first of all, part of me feels like it’s a little bit too generous. But part of me also thinks back to the work that KimMarie and I did. And if you don’t have that D or F, that you have to overcome, it’s a lot easier to persist, not just in your major, but towards graduation, whether you switch majors or stay. And part of me feels like that’s helpful to the student. So in some ways, I like a generous withdrawal policy, I definitely want students to know what the withdrawal policy is, what the deadlines are, so that they can make a good decision.

John: Yeah, the only concern I have with the policy is that it can lead to students making slow progress towards a degree and they could run out of funding before they complete their degrees. But on the other hand, it does provide a bit of a safety net for students who are adjusting. And as our student body has become more diverse, and we have more first-gen students, and we have more students coming from schools that did not prepare people particularly well for a college environment, it provides a safety net, which has allowed students to take some chances early on and not be harmed. And I’ve seen some students struggle in their first year or so, drop classes, and come back and be very successful, when before they might have been deterred from further study in the discipline.

Tisha: I agree, I can see that.

Rebecca: So, definitely one of the things that I’m always thinking about when students are talking about withdrawing is making sure that they know all of the ramifications. So what does it do to your GPA? What does it look like on a transcript? What are the financial implications? …so they can make a well informed decision. And depending on who they’re talking to, they are not necessarily always getting all of the information, and that can be problematic, too.

Tisha: Yes. So I think that points us to the issue of how important advising is, so that there are well informed advisors and that students have access to them, not just at the end when it’s time to register for next semester, but they have a relationship with their advisor, and feel that they can go in and ask those questions, because as John said, a lot of those students can’t go to their parents, they’re first gen, their parents don’t have advice for them. And even if your parents went to college, they don’t necessarily know all of the rules and regulations of the institution that you’re at. It’s really hard, I think, for students to have all the information that’s necessary.

John: I’ve had trouble keeping up to all the changing information [LAUGHTER] over the last few years. In terms of those requirements. You’ve done some work with cooperative learning. Could you talk a little bit about that?

Tisha: Sure. So that really came out of the fact that I had done a lot of efficacy work around classroom experiments. And KimMarie and I are really good friends. She is a cooperative learning expert. So I should say that she is the expert on CL. And she wanted to do an efficacy study, so we decided to team up and do that. So what we did is unlike a lot of some of the other work in efficacy, is that a lot of the work is comparing lecture. So you don’t do anything to this active learning technique. And with cooperative learning, the students are working on exercises. And when we talked about it, we said you know, it’s really not fair, and not even interesting to compare a student who is in a lecture-based class where they don’t get to see any of these problems to students in a cooperative learning class that are working on these problems in class in teams. So we thought a lot about the collaborative learning approach. And we’re trying to isolate what we thought could be the mechanism that might be driving different outcomes. And what we decided was we were going to compare the active team-based learning to individual learning, working on problems. So in our control, the students were exposed to the same problems, they had the same amount of time in class to work the problems as in the treatment, which was the collaborative learning. It’s just that in the treatment, they were working in teams in this think-pair-share share framework, and we didn’t find a significant difference. So at the end, we said that the cooperative component didn’t seem to be driving any difference. Although there are other people who’ve done work comparing collaborative learning to lecture based and they did find significant positive effects from cooperative learning.

John: One type of thing that I’ve done that seems to have been fairly successful is using clicker questions. Following the methodology of Eric Mazur, students are given a challenging question where typically half or so them will get it right the first time when they’re asked individually. But then they get to talk it over with the people around them, preferably someone with a different answer, they get to debate and argue it and I normally see between a 10 and 20 percentage point increase in the correct answer. And the really absurd answers tend to disappear down to about nothing. So that’s not a very formal study, but that second stage of that process where students are engaged in some peer instruction seems to have had a pretty significant impact. And I point that out to students, that when they talk to each other about it and explain it to each other, they do much better than when they’re working on their own. And I encourage them to try doing that outside of class, to work with other students, because there are a variety of studies… I haven’t seen too many in economics, but there are quite a few studies that find that that type of peer instruction, either in the classroom or outside, can be fairly effective.

Tisha: Yeah, so I’ve observed that too. I haven’t used clicker questions, but just going in and observing my colleagues when they teach, and they are using clicker questions. And I have to say, I’m stunned, because I don’t know how the right answer bubbles up as opposed to the person who had the wrong answer convincing the other of the wrong answer. I don’t know how that doesn’t happen. It’s like magic to me, almost. So part of me wants to think a bit more about what is the mechanism and how is that working? That’d be really cool to do a study on that.

Rebecca: One of the things that’s always curious to me about that particular dynamic is sometimes students discover, as they try to explain something to someone else, [LAUGHTER] that they really don’t know what they’re talking about. [LAUGHTER] They realize, huh, I can’t actually explain this concept, so maybe I’m the one that’s not correct. [LAUGHTER]

Tisha: Maybe that is what’s happening. Because everyone always says, “You have to really know your stuff to teach it,” and so if you are trying to explain it to your partner, you do have to know the material.

John: And when they’re raising objections and you don’t have a counter argument, that can help break down your misconceptions, because getting rid of those incorrect perceptions can be as important as trying to build new ones.

Tisha: Yeah, I could see that and maybe the people who are initially giving the wrong answe are just guessing. So maybe they’re not trying to explain and convince their partner of their answer, I guess. I don’t know. So maybe it’s not magic.

Rebecca: [LAUGHTER] Feels like magic.

John: It does. And it works.

Tisha: When I see it happen, it does seem like magic. In general, I think that active learning is helpful, because I think that women are going to be more interested when you show them the applications, and that when you have active learning, there’s some community building that happens. So you’re illustrating relevance, you’re building this community, giving them a sense of belonging in the classroom, which is a lot of what I think actually happens with the experiments. And then actually also with the experiments, they’re kind of discovering some of the theory for themselves, which is a growth mindset. And there is research that shows that if you can show relevance, belonging, and growth mindset, and develop these in your classroom, that students are going to do better, but that it also is more appealing to females and underrepresented minorities.

Rebecca: There’s lots of things for all of us to think about, even if we’re not necessarily in economics classrooms. Definitely good food for thought. Switching gears a little bit. Can you tell us a little bit about your work on the Committee on Economic Education?

Tisha: So I’m not officially on it at the moment, I’m the incoming chair. But I have served on it in the past. And I’ve been shadowing KimMarie McGoldrick, who is the current chair for the last year. So the committee is doing a lot of really great work. I think. When I first served on the committee, we basically just organized sessions for the American Economic Association meetings. And when I say “just” I don’t mean “just…” Compared to now, it’s just but we have, I think, seven sessions that we organize for the meetings every January. But while I was first on the committee, then chair Mike Watts, who was at Purdue, he said,”I have an idea. We should have a conference, so an all economic education conference,” and we started it while he was chair, and it’s CTREE, so the Conference on Teaching and Research in Economic Education. So the committee is in charge of that. It’s every end of May, beginning of June, depending how the calendar falls. So our next CTREE will be in Atlanta, not that far. It’s on the East Coast, maybe you can join us. And it will be May 29 through the 31st of 2024. We have many other things that we also are overseeing: an educate program, which is professional development for college faculty, and that includes two-year college faculty, in course design, different active learning pedagogies, and we talk about how to bring diversity and inclusion into the classroom. KimMarie actually started this this year, we have a newsletter called EconEdNews. And that features different pedagogies, it features the winner of the AEA Distinguished Economic Educator Award, which the committee also oversees, and some centers that we have for Econ Ed across the country… just a lot of information that’s in there, we have two issues a year. So we’ve had our first two, and our next one will come out in March of ‘24. So just a lot of great work, I think, that the committee is doing and I think really thanks to Mike who started us with our own conference. And then Sam Allgood succeeded Mike and added and then KimMarie is an overachiever. And she added a lot more in her six years as the chair.

John: And going back earlier, early surveys of economic instruction suggested that economists were using much more chalk and talk than many other disciplines were using. So these efforts have been fairly important in shifting people away from that. It’s a somewhat overdue change, but it’s nice to see that happening. And I would love to go to CTREE. But there’s two barriers that I’ve run into one is we run a series of workshops here, right around the same time, and the other is I teach at Duke in the summers. So I either am doing workshops here, or I am in North Carolina teaching econometrics now down there, so I haven’t been able to get there. I would love to go to a CTREE meeting.

Rebecca: I think I would have no clue what’s going on if I went. [LAUGHTER]

John: Oh I think you would.

Tisha: I think you would, I think you’d have a nice time. Everyone there is just so lovely and very engaged in teaching, and just really cares about the enterprise.

Rebecca: It’s nice to see education groups out of these professional organizations to really formulate communities of practice around teaching.

Tisha: Yes, I agree. I think economics, unfortunately, was late to the game. But we’re increasingly appreciating the importance and the importance not just of focusing on four-year colleges, but two-year colleges. So going back to the point of diversity, and the lack thereof in the discipline, the students in four-year colleges that I’ve worked with have predominantly still been very much what economists already looked like. And if we really want to improve the diversity in economics, I think we need to reach down to the high schools to the two-year colleges, and make economics more interesting and accessible.

Rebecca: We always wrap up by asking: “What’s next?”

Tisha: There’s lots of things. I guess, with regard to the committee, I’m hoping to just kind of not start anything new in my first term, and keep everything running as well as KimMarie has it now. But then for my own research, I am really interested now in looking at comparing pedagogies. So it’s been a lot of efficacy work looking at a particular pedagogy compared to sort of the standard chalk and talk. But now, how do they compare to each other? Is there a better pedagogy for students? And my first study, and I have the data, so I just have to start working on it will compare experiments to cooperative learning. And I told KimMarie, if experiments lose, then she won’t hear any more about this. I won’t write it up. I won’t talk about it. Because I don’t want to say that experiments lose, but no, I mean, seriously, that’s the next thing that’s first on the agenda. But also, I have a deep interest in diversity questions. KimMarie and I with Scott Simkins have a paper looking at HBCUs compared to PWIs and the study of economics there. And that was inspired by some work that showed that HBCUs contributed disproportionately to the STEM pipeline. And since economics is part of STEM, we thought, “Well, that should be true for economics as well.” So we did some work in that area. And we show that there are some positive contributions from HBCUs to the economic pipeline. But I think there’s a lot more to do there. So I would like to dig down in that. I’ve heard a lot of people at HBCUs talk about secret sauce, and I want to learn what that is. And not just at HBCUs but look at MSIs as well, more generally. And I do want to look at some questions at two-year colleges, so that’s for the next five to 10 years…

Rebecca: You’re going to be busy. [LAUGHTER]

John: For those who aren’t familiar with the terms PWI are predominantly white institutions and MSI are minority serving institutions.

Tisha: That’s correct.

John: Those terms are becoming more and more common, but just to make sure all of our listeners are familiar. Okay, well, thank you. It’s great talking to you, and it’s nice meeting you. And we will keep trying to get KimMarie McGoldrick on the podcast. I’ve been asking her for about four years. She’s been a little resistant, but we’ll see if we can get her here in the future. Well, thank you.

Tisha: Thank you so much. I hope you have a good afternoon.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

317. Beware the Myth

Posted in higher education, instructional communication, learning, neuromyths, social justice, teaching
November 29, 2023

One of the most persistent neuromyths is the belief that students learn more when instruction is tailored to their specific learning style. In this episode, Shaylene Nancekivell and Xin Sun join us to discuss their research on possible negative consequences of the learning styles myth.

Shaylene is an assistant professor of psychology at the University of Manitoba. Xin is a postdoctoral fellow in the Department of Psychology at the University of British Columbia. Shaylene and Xin are co-authors of a study entitled “Beware the myth: learning styles affect parents’, children’s, and teachers’ thinking about children’s academic potential,” published in the NPJ Science of Learning journal this fall.

Show Notes

Transcript

John: One of the most persistent neuromyths is the belief that students learn more when instruction is tailored to their specific learning style. In this episode, we examine possible negative consequences of the learning styles myth.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Shaylene Nancekivell and Xin Sun. Shaylene is an assistant professor of psychology at the University of Manitoba. Xin is a postdoctoral fellow in the Department of Psychology at the University of British Columbia. Shaylene and Xin are co-authors of a study entitled “Beware the myth: learning styles affect parents’, children’s, and teachers’ thinking about children’s academic potential,” published in the NPJ Science of Learning journal this fall. Welcome Shaylene and Xin.

Shaylene: Thanks for having us.

Xin: Thank you.

Rebecca: Today’s teas are. Shaylene, are you drinking tea?

Shaylene: I am drinking a latte actually [LAUGHTER], the Chestnut Praline Latte from Starbucks, I needed coffee. When I do drink tea, it’s usually chai tea.

Rebecca: Alright, sounds good. You’re recovering here [LAUGHTER]. How about you Xin?

Xin: Coffee is my go to.

Rebecca: It’s a very popular flavor of tea on this podcast…

John: It is.

Rebecca: …for sure [LAUGHTER]

Xin: If it’s tea, it’s bubble tea for me [LAUGHTER].

John: I have a Lady Grey tea today.

Rebecca: And I’m back to some blue sapphire tea. I went to the tea store this weekend and I’ve stocked up, so we’ll have some variety soon, John.

John: Okay. [LAUGHTER] We’ve invited you here to discuss your paper that examined the effect of identifying students’ learning styles on perceptions of children’s academic potential. Before we discuss this, though, we should probably just remind everyone about the myth of learning styles. It’s something which has been remarkably persistent, but what does research tell us about the relationship between identified learning styles and how people actually learn?

Shaylene: Yeah, so I’ll field this question. I think the first thing that people should realize is that the learning style myth is really hard to get a handle on Coffield and their colleagues identified about 81 different versions of the myth that kind of float around both academic and non-academic spaces. So when people hear the term learning styles, I think the first thing we should all point out is that everyone sort of means a different thing. That could be true in everyday conversations, that could be true like school to school. And so yeah, what learning styles are is actually really hard to pin down. The element that’s most common to most learning style myths is something called the meshing hypothesis. So this idea that if we figure out the right modality that people can learn in and we match instruction, somehow, to this magical modality, that people will learn better. And this is the thing that’s like, most common across all myths. And the modalities that people talk the most about are things like visual, auditory, and kinesthetic. So learning with your hands, your eyes, or your ears, that’s kind of the version that’s most common. We don’t really know where exactly the learning style myth came from. There was like old books by Dunn and colleagues, talking about how to set up ideal learning environments for children that mentioned this term learning styles, but this whole matching to like visual auditory, kinesthetic version of the myth where researchers aren’t too sure how everyone came to some sort of agreement that this was the version of learning styles that should stick around and we don’t really know where it came from. Regardless, researchers have tried to disprove it. So they’ve done studies where they have given people learning style surveys or tried to identify people’s learning style in some way, and then matched instruction to it and they don’t find any effects on learning. Overall, the best thing to do is to match the instructional style to the kind of content you’re teaching. And that’s kind of the best thing to do instruction wise, and then in terms of studying for students, the best thing to do is try to take in information in as many different ways as possible, and just deploy best practices in studying, so things like spaced practice, and paraphrasing, putting things into your own words, those kinds of things that we know are really good for studying.

Rebecca: So although we have no idea where it came from, why oh why is it so persistent?

Shaylene: I have two other research papers where I’ve tried to dive into that. The one that’s most relevant is actually a paper that Xin also co-authored with me that I did during my postdoc at Michigan. And what we did was we walked participants through an actual research study. We said, like, “Okay, this is the data they’re going to collect, these are their hypotheses, do you agree if they find data against or for the hypotheses that it’s testing the learning styles, and we found that participants agreed it was a good test of learning styles. And then we told them at the end of the study that it turns out, this is a real study or version of it, at least, that researchers have done, and they found no effects on learning outcomes. And there were a proportion of people that were very willing to revise their beliefs upon thinking through this exercise, they’re really kind of open to like, “Okay. I told you that this would prove or disprove it, I’m on board, maybe I’m wrong.” But there was also a pretty big proportion of participants that even after walking through this exercising and agreeing that that was a test of learning styles, did not want to revise their beliefs. And what we found is It seems to be entrenched in their identity somehow. They provide these really personal anecdotes of ways learning styles had helped them or family members. So that actually is kind of what led us to this paper we ended up doing, which is that there seems to be some sort of link between our academic self-construal and our learning styles. And that kind of creates almost backlash against wanting to revise their beliefs, because then you’re not just revising a belief about learning styles, you have to revise a whole host of beliefs about maybe why a family member succeeded or failed at school, or why you yourself succeeded or failed at school. And that’s a little bit harder to do for any person, and not just research participants.

John: We should probably mention an earlier podcast that we had with Kristin Betts and Michelle Miller, who had conducted an international survey of neuromyths. And my recollection was that the learning styles myth was the most widely held myth of any of the neuromyths that they were examining. So it is out there. I know a lot of our faculty on our campus still teach it, despite all the evidence, and despite some of us encouraging them to please stop that. But it’s something that just won’t go away, and most students come into college believing it. So in your study, though, you’re looking at some of the possible harmful consequences of a belief in learning styles. Could you provide an overview of your study?

Xin: So in our paper, we conducted three studies. And the first study, we included a child sample and a parents sample. So the children they were around 6 to 12 years old, so elementary school age. And in the second and third study, for each study, we included a parent sample and a teacher sample. So for all of these five adult samples, they’re all around 100 participants each. So what we did was for study one and for across the studies, we first sort of describe the participants to hypothesize students. So the first one, we wouldl describe them as all the students learns the best with their hands. And the second or the other students would be described as all the student learners the best with your eyes. So basically corresponding to a hands-on learner and a visual learner. And in study one, we asked participants to rate how smart do you think these students are? So one on one, are they very smart, like just smart, a little bit smart, or not so smart. And so parents and children, they went through the same kind of protocol. And what we found in study one is, on average, both parents and children, they believe that the visual learner is smarter than the hands-on learner in terms of their smartness rating. And in study two, instead of asking participants to rate individual learners, we ask them to compare directly, so which learner do you think is smarter? And a lot more participants believe that the visual learner described as like learning best with their eyes, is smarter than the other hands-on learner. This is true across the teacher and the parents sample. And then in study three we wanted to ask “Well, so what do you mean by being smarter? So does that mean that this visual learner would do better in some of the school subjects?” So we then, in study three, asked parents and teachers to predict grades of these two described hands-on and visual learner across like some common school subjects, including math, science, language, social studies, as well as some of the, what we call non-core subjects, like arts and music. So what we found is, on average, both the parent sample and the teacher sample rated grades as being higher in the visual learner compared to hands-on learner in many of the subjects including math, language arts, social studies, and they also rated the hands-on learner as having higher grades in some of the other subjects, like arts and music. This is generally what we found. Shay, do you want to add on to it?

Shaylene: Yeah, sure. And I think kind of the big take home is like a lot of the work that’s tried to argue against learning styles has made like a wasted resources argument, which I think is completely valid. We should be spending instructional time, especially in teachers college pre-service classrooms like teaching teachers best practices, same thing with our students and study practices. But I don’t think that argument has persuaded people. So one big motivation for us to do this paper was I think the learning settlement actually can create more damage than just a wasted time or wasted training session for teachers. When we provide these labels, and we label children this way, it’s providing unintended messages likely to the parents, or if you apply these labels, like teacher to teacher, or even label a child in a classroom of their peers, people are hearing other things than just this kid is a hands-on or a visual learner. They’re making judgments about maybe what that child is going to be good at or not good at, or when or how they’re going to succeed in the classroom. And that, to me, is highly problematic. We want to meet kids where they actually are, not where they are based on construall that’s resultant of a myth.

John: The efficiency argument never troubled me that much in that when instructors believe in the learning styles myth, they tend to use multimodal instruction, which is helpful for everyone. My concern with it had more been the impact on students’ perceptions of themselves in a somewhat different way. I’ve often had students say, “Well, I can’t learn by reading this textbook, because I’m a visual learner” …and encouraging them to use their eyes to read didn’t seem to make much of an impact. Normally, I referred them to some of the studies on learning styles. But it’s something that students really deeply believe… that they can only learn in certain ways, which may deter them from trying new ways of learning and that’s what’s always concerned me. The effect on perceptions is something I hadn’t really thought about until I saw your paper. I’m really pleased to see this avenue of research because it provides a really strong argument against the persistence of the learning styles myth. Ultimately, studies like this might help persuade people to stop doing this.

Rebecca: So given your results, what are some of the implications we need to think about in terms of pre-service teachers, but also, the students that we have in our college classrooms who may have had this kind of impact on them throughout their education so far.

Xin: I feel Shay kind of touched on that point, which is, on the one hand, that what people generally feel that the learning style myth, the consequences are that it’s a waste of time, it’s a waste of energy. But it’s fine to just have it there because I wouldn’t do any other harm apart from that, but in reality from our study, you can really see that it not only like we add these things, it’s just an add on, but also it creates, I would say, biased, thinking about students, like what they can do, what they in the future might become, is sort of ingrained, for example, in parents’ or teachers’ thoughts. So I think this is not a good sign to like students’ development. And I feel you’rr right, John, that there are studies also, like qualitative evidence showing that students like from elementary school, some of the students would say, “I don’t want to do this kind of math, for example, because you showed me in a certain way, like, I’m a visual learner, you should show me in another way or something.” So I feel like the learning style method, if you believe in it, it might set limits for yourself. So I cannot do such and such because I am a certain person. I think this is also an important future direction for us to study, which is what’s you think about yourself in terms of learning self myth and what you can do.

John: What you’re showing is that it’s not just the students’ self-perception, it’s also their teachers and the parents who may be affected by this. And all those things, I think, could have some impact on student major choice, the decision to go to college. Is that one of the things you’re thinking about following up in terms of this study?

Shaylene: Yeah, I think a really important future direction for this work and something that actively writing grants… so fund us… to look [LAUGHTER] more into, are things like how does labeling someone with a certain learning style affect program recommendations, how I even write recommendation letters? When I get those boxes, and it’s described the students strengths and weaknesses, am I actually describing their real strengths and weaknesses or am I describing strengths and weaknesses based on essentially what the… I use this word a little bit loosely… but stereotype almost and, I think we’re also very interested in a group about how these might intersect with other aspects of people’s identity. So what does it look like if you label someone identifying as a woman or a man with a certain learning style or different racial identities? So, yeah, again, how does that intersect with, because we know there’s a whole host of academic beliefs that can come to the surface, when you find out that someone maybe is a young woman. They might get a double hit, if they’re perceived to be a young woman and a hands-on learner… they’re never destined to be good at math, for example. So we’re also just interested in exploring more about identity because identity, of course, is intersectional, and has more than just this learning style element. We’ve kind of studied it to the best of our ability in a vacuum in this paper, but it doesn’t actually exist in a vacuum in the real world. And the other thing we are looking into right now, planning some projects, looking at more implicit beliefs. So to kind of put another step forward to kind of prove that this is akin to like a stereotype. So there’s certain research paradigms you can do to look at more implicit beliefs, and to look at more people snap judgments, as opposed to like these more reflexive explicit beliefs that we have in our survey. So yeah, those are all promising and I think, really important, future directions to flesh out in the future.

Rebecca: I know that one thing that I was thinking about, while I was reading your paper, is how guidance counselors and others who help students make choices about what they might do, college, or even before that, if they want to do a technical program or something, and how that might inform the decisions or the advice they give. I know that as a student, I excelled in places like math and science, but I also deeply loved art. And if I showed a preference towards wanting to do that more kinesthetic work in art, I was discouraged. It’s like “Oh, you’re book smart.” So it’s almost like those same stereotypes that you’re talking about, or kind of found, I think, sometimes, perhaps lead to odd advice, or don’t necessarily encourage a student to pursue something that might be a worthwhile endeavor for them, because it’s deemed not worthwhile.

Shaylene: Yeah, for sure. I think that’s the fear. And, I think, at least anecdotally, when I talk to people, I get stories very similar to yours. In our one paper, in Cognitive Science, we coded them, but we read like, at least a few dozen online participants. People didn’t have to write that much. It was just like, “explain your thinking.” People wrote us essays about learning styles and how they had affected their life or their child’s life, like completely spontaneously and these are like online workers that were paid a whole dollar for our survey. So I think, at least anecdotally, we kind of have evidence to suggest this is likely happening. And so it will be a very important feature direction. But I also think, as academics, we need to be careful, because I think educators are overburdened, and they’re doing their best. And so I think a real take home for this is that we shouldn’t criticize them for holding beliefs that literally 90% of the population have, and that we should do more to support our educators. And yeah, I don’t want the takeaway here to be [LAUGHTER], teachers are doing bad or evil work or anything like that. They are so overburdened, they have no time for professional development. And so it makes sense that they would hold beliefs similar to their peers that aren’t educators, but we can do more to give them evidence-based ways to support their students. So not just telling them don’t do this, but giving them a replacement. And that’s something I try to do a lot too when I talk to my students about learning styles is, I think, sometimes, especially when you’re busy, it’s easy to say, “Oh, just don’t study in that way.” But like, what are they replacing it with? Because it’s filling a hole that they have. So making sure that we give them something else to put in that spot. So they’re gonna be career counselors, that’s like part of their job, how can they do that in an evidence-based way? So something else I’m very interested in, just in general in the neuromyth literature is, what are the different ways we can communicate neuromyths effectively? That’s something I would love to do more research on as well. Because the message “Don’t do this,” I don’t think is the right one.

Rebecca: No, we know this… just interacting with small children…[LAUGHTER] …you tell them not to do something, that’s the exact thing they want to do. We all do that [LAUGHTER].

Shaylene: Yeah, like try this instead.

Rebecca: Yeah. [LAUGHTER] Redirection.

Shaylene: I have a list of places that learning styles permeate the educational system. It permeates teacher education, it permeates university learning centers is a big one. So like where post secondary students go to get study advice. It’s like a very common part of post-secondary learning centers. It obviously permeates the classroom. It has global endorsements, so you see it across Latin America, Greece, obviously Canada, the United States, large parts of Europe. And then the other place you see it is, unfortunately, in the academic literature, almost like the education adjacent literature. So it might be someone that isn’t an education scholar, but maybe they’ve published a paper on teaching in their field, because they’ve developed a new teaching technique or something for teaching medical students. And you see learning styles permeate those papers. And we do cite a lot of those, which makes it extra hard for an educator, because if you do a literature search on learning styles, and you don’t know the subtle differences between these fields and these journals, you will find tons of papers supporting learning styles… academic papers, that’s the other place it permeates and you see these field- specific beliefs. And Xin pulled out a great quote for our intro that was like talking about these people thought they discovered this unique learning style of medical students that was a combination of two different learning styles or something, I think. It permeates this education-adjacent academic literature as well.

Xin: Yeah, yesterday, I saw this learning styles quiz on Canada’s government website. They have a job bank, where they help people to do career planning. And then there’s a bunch of career quizzes saying, like “discover your learning styles.” So they officially fill this as a tool for people to find their job, or at least which industry they would ideally work in. So that’s shocking to me. [LAUGHTER]

Rebecca: Step one do no harm [LAUGHTER].

John: Now, in terms of follow up studies, are there any large data sets out there that would have identified learning styles in it to facilitate a study of the impact on major choice or decisions to go to college and so forth?

Shaylene: Not to my knowledge, a lot of the learning style literature kind of falls into a few broad buckets. So the first bucket is this really wonderful rich neuromyth literature that has been around for a long time. And there learning styles is usually just one or two survey items in a broader survey to assess educational literacy like educational sciences literacy, or neuro literacy or that kind of thing. So you’ll see learning myths listed with things like right and left brain learners, and drinking a lot of water can increase your brain size. So just general, like neuro literacy things. And then those are actually are pretty big datasets. And there is some stuff where they’ve correlated that with academic success, or teachers’ prior knowledge of neuroscience and then like willingness to endorse the myths, but it’s more of like a catch all type surveys. And then there is this other literature I’ve talked about that kind of takes learning styles to be real, and looks at how it’s related to major selection and that kind of thing. So they’ve kind of studied this, but kind of the wrong lens. Why do people pick chemistry? …because they have this perception of themselves. So it’s related, but it’s not quite right. And then other literature is actually really small. And that’s kind of where I’d place our paper is kind of diving deeper into the learning style myth itself. And there’s other great researchers doing this work. Newton just had a paper come out this year looking at the endorsement of learning style myths in higher ed. He’s published a bunch of papers, they’re really great. Couple papers by our research group, looking at learning style myth in particular, but that literature is really small, there’s not a lot of people doing that work.

John: I was just thinking, if, there was information on it in the Baccalaureate and Beyond or one of the NCES data sets, for example, that would be a nice way to examine the effect of student perceptions of their learning style on decisions to go to college or majors and so on.

Shaylene: Maybe we should look. It’d be worth looking.

John: I’ve worked with those data sets a lot, but I never specifically looked for that variable, but I might this afternoon. [LAUGHTER] But yeah, it would be nice to be able to examine that.

Shaylene: Yeah. For sure, I mean, even at the institutional level, it would be worth doing something like we have the undergraduate research pools that at least would be a bigger sample then that’s even in our paper included questions like on mass screening or something. But it’s kind of hard, because unless you get into one of these nationally representative surveys, it’s hard, still looking at people that chose a psychology course, or whatever, so your sample is still going to be a little skewed. And the government for the most part thinks they’re real. So I don’t know what they would include if they included it.

John: So, obviously it would be best if we could just get rid of the myth, but we still have whole generations of people who grew up with that. What might be some ways of breaking people from this belief? I think you had a prior study that looked at strategies for trying to encourage students to move away from this myth. Could you talk a little bit about that?

Shaylene: So I guess the prior paper that we did suggests that, for some people, providing them with evidence and walking them through the evidence and helping them think about it is enough, and they will revise their beliefs upon being presented with that evidence. And that kind of fits with other work I’ve done earlier, suggesting there’s a ton of variability in how people think about and construe learning styles. So I think getting the evidence out there and really helping people think about the evidence critically and what it means. Like, “What was the study? What was the hypotheses? How did they test it? Was it a good test? This is what they found…” That went pretty far. Our sample was all online workers that did that study, and they had a variety of educational backgrounds, too. They weren’t all familiar with research or that kind of thing. But there was a huge proportion of people that this belief is like really entrenched in how they perceive the world, how they perceive their academic journey or other people’s academic journey. And so one thing that we actually are really interested in is how maybe stories of personal anecdotes that we could provide people that maybe help them think through the potential harm of the myth. So, “Oh, well, it might have benefited someone in this case, but it could actually create harm in this case, because maybe this person really imagined themselves as being an electrician or being a plumber and those are really great jobs. But now they feel like that’s something they won’t be good at because they’re not the kind of learner, they haven’t been told that they learn in the right way, they should go to more math or science university instead, because that’s what they’ve been pushed to do.” That would be really not great for that person’s life outcomes, because that’s maybe not their interest or passion. So just helping people maybe think through how these stereotypes might unfold and the potential consequences of them, I think, is maybe a promising future direction. Because we know based on that prior work that just giving people this is what the literature has found isn’t going to revise everyone’s beliefs. So maybe like challenging their anecdotes with a thinking exercise almost might be an interesting direction to go.

John: And your study could be a nice basis for that type of thing, because it does describe harm that can take place as a result of a belief in this, which was not something that people generally considered before.

Shaylene: Yeah, it was hidden, I think, but maybe if it becomes more visible, people would be more willing to revise their beliefs. And you see it in like dialogue. I see it even when I get reviews back from my papers on this subject, reviewers will say like, “Well, it’s a preference. And we know that matching instruction to students preferences can benefit them because they feel more motivated. So is it really harmful?” I’ve had reviewers push back and write that kind of thing. And so this is maybe a thing we can point to, that even a preference, it might seem benign, but applying that kind of label is not benign, it’s not neutral.

John: It might be hard to get past reviewer 2. [LAUGHTER]

Shaylene: Yeah, I mean, the consequence of this permeating academia is that permeates our reviewers sometimes. But, I will say we had a really great experience at NPJ. So this is not about [LAUGHTER] that journal. Just to be clear, we had a lovely experience there. The reviewers were great. They definitely made the paper better. Everything good to say about that journal.

Rebecca: Well, we’ll always wrap up by asking what’s next?

Shaylene: What’s next for us is fleshing out the stereotypes more. So we’ve already talked about some directions, you guys brought some up and we did, looking at how it might affect actual daily practices. So how someone writes a recommendation letter or the kinds of programs children get recommended to. The kinds of careers people think others would excel at. I have learning style surveys where most people that believe in the myth think that learning styles can predict career outcomes, but diving deeper into like exactly what that means, because it was just like a one item on our broader survey. And then also looking at things like we talked about earlier, like how learning styles intersect with people’s identities, because most people aren’t just black shadows.And that’s what we did in our study, we just like provided little black silhouettes. So I think kind of moving our findings into like, a little bit more of an ecologically valid realm, I think, is the next step.

Xin: And building on that is also what we’ve discussed. We could also switch gear from the career guidance to parent or teachers’ perspective on what the recommendations they have on learners, but also towards learners themselves. What do they think? “Oh, I have these kinds of learning styles. What does that mean to me? Am I gonna go for certain jobs, but not the others?” Yeah, if we ask children these kind of questions on “What is your learning style? And, what’s going on, going off of it?” That will be very interesting.

Shaylene: Xin and I have another paper looking at beliefs about… it’s Xin’s work, really, I helped… She has a wonderful paper looking at people’s beliefs about language and language acquisition and policy endorsement. And I actually think that’s also another promising future direction. So I wonder if the degree to which we believe in learning styles, how that affects beliefs in streaming of children from a really young age, which we know isn’t always beneficial, especially if you stream children too young into more academic streams and less academic streams? That does happen in some countries. And so that’s, although less common in Canada, very common in other places. Xin, you talked about like the Chinese education system has different…

Xin: Oh, that’s from high school, though…

Shaylene: Yeah. So even that, the degree to which you might endorse, it’s more extreme streaming of students like different schools even. So that’s probably another important direction is, how does this affect things you would vote for or endorse as a citizen?

Rebecca: Thank you for your really important work and helping us all think about the kind of harm that some of these beliefs can actually cause.

John: Yeah, students face enough barriers that having learning styles serve as a barrier to their learning is something that could be pretty easily avoided. Great work and I’m looking forward to seeing more of your future work on this and other topics.

Shaylene: Great, thank you so much for having us. It’s been a really fun conversation.

Xin: Thank you.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

316. Help-Seeking Behavior

Posted in equity, first-generation, first-generation students, higher education, inclusion, learning, support network, teaching
November 22, 2023

Continuing-generation college students are often better prepared by their family and peer networks for academic success than first-gen students with more limited support networks. In this episode, Elizabeth Canning and Makita White join us to discuss their research on differences in academic and non-academic help-seeking behaviors between first-gen and continuing generation students.

Makita is a graduate student in Washington State University’s Experimental Psychology Program. Elizabeth Canning is an Assistant Professor in the Psychology Department at WSU.

Show Notes

Transcript

John: Continuing-generation college students are often better prepared by their family and peer networks for academic success than first-gen students with more limited support networks. In this episode, we discuss differences in academic and non-academic help-seeking behaviors between first-gen and continuing generation students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Elizabeth Canning and Makita White. Makita is a graduate student in Washington State University’s Experimental Psychology Program. Elizabeth Canning is an Assistant Professor in the Psychology Department at WSU. Welcome, Makita and welcome back, Elizabeth.

Elizabeth: Thanks for having us.

Makita: Hello.

John: Our teas today are:… Elizabeth, are you drinking tea?

Elizabeth: I am drinking coffee this morning. It’s morning over here in the Pacific coast.

Rebecca: How about you Makita?

Makita: I have a hibiscus berry tea. I don’t usually drink tea, but I got some just for you guys.

Rebecca: Awesome. [LAUGHTER]

Rebecca: Sounds like a nice treat. I have some chai today. John?

John: And I have an English breakfast tea today, because I got a long band practice tonight. [LAUGHTER] So, a little more caffeine will help.

Rebecca: [LAUGHTER] So we invited you here today to discuss your recent study entitled “Examining Active Help-Seeking Behavior in First-Generation College Students,” which was published in Social Psychology of Education. Can you tell us a little bit about how this study came about?

Makita: Well, for me, this was actually my master’s thesis. When I became a researcher, part of my dream is to change the world, make a difference, and I’m really passionate about people getting access to the things that they need. So when I was in undergrad, and I was exposed to a lot of first-generation college students, and when I was hearing my parents talk about their experiences as first-generation college students, I started to notice that things were a little different when you didn’t have people around you who could tell you about what college was supposed to be like. And then when I started reading the literature on first-generation college students, and I saw how, in my opinion, excessively large, the gap was between first-generation and continuing-generation college students. That really captured my attention. So then, when I went to a school, where they have a really high population of first-generation college students, it felt really appropriate to look at first-generation college students. And also I’m honestly really interested in help-seeking behaviors. You probably have experienced yourselves where you see one person who is very persistent and active in getting someone’s attention, basically, very annoying, consistently waving their hand… [LAUGHTER] …trying to come up and get someone to basically give them whatever they need or they want, versus another person who maybe is a lot more quiet and sitting there hoping that someone will help them. And I wanted to know why. What makes one person act so differently from another? So I was really interested in first gens and help seeking. And then, at the same time, Elizabeth had recently been to a conference where they talked about first-gen forward initiatives, which is where colleges encourage faculty and instructors to self identify if they themselves were first-generation college students, to encourage other first-generation college students at the university to feel more comfortable, maybe talking to them or going to office hours, things like that. And we combined those two things together into this study, where we could look at help seeking in first-generation college students and a shared identity to try to see if that would change how help seeking looked.

John: And you mentioned some of the gaps that are observed. Could you talk a little bit about some of the equity gaps between first-generation students and continuing-generation students in terms of their academic performance and success?

Makita: Yeah, so for first-generation college students, we tend to see that on average, they earn lower grades, and they’re more likely to drop out of college. And they’re also less likely to engage in academic success behaviors, like going to office hours, or trying to talk to their instructors after class, things like that. And there are a lot of different reasons for that. They quite honestly don’t have someone at home who can teach them those implicit unspoken rules about college, about what’s expected of you and how you should behave. So they have to learn that on their own and that can take a little extra time, which is pretty valuable when you’re in college, that time. And then, a lot of the time, when you’re a first gen, you’re also coming from a lower income family, which may require you to work while you’re also going to taking classes and it means that you’re a little less likely to live on campus, which can influence you in all kinds of fun ways. But, we were, as I said before, really interested in whether or not there was a difference in the type of help-seeking strategies that students were using and how frequently they were help seeking. We wanted to see if that was maybe part of the reason for this gap.

Elizabeth: Yeah, I’ll just jump in too, that one of the big inspirations I think, for this work came from a sociologist, actually. Her name is Jessica Calarco. I think she’s at the University of Wisconsi- Madison now, but she wrote a fantastic book, it’s called Negotiating Opportunities. And she did a lot of field work with children, looking at at how children from lower- or upper-middle class families act differently in the classroom and how children approach their teachers or how they seek out resources and things. And she found that more low-income students are much more passive in their health seeking behaviors than upper-middle-class students. And so we had kind of read this work and thought it was really interesting and wanted to see if the same was true at the college level, and how that might look with those types of behaviors in a college setting. So we wanted to see if that gap that they found with children was the same for higher ed.

Rebecca: Can you talk a little bit about the methods of your study?

Makita: Yes, so this was a little complicated. But I’ll try to go through it in the most straightforward way I can.

Elizabeth: I will say for a first-year Master’s student, this was about the most difficult kind of laboratory experiment to design for, like right off the bat. But Makita did such a great job. And I think it turned out so well. But the complexities of it actually make it super interesting. So hopefully, we can explain it in a way that is easy to understand.

Makita: So we designed this study when COVID was in full swing, and we were in lockdown and, as a result, the entire study is set up to be conducted through Zoom. So the way it would work was a participant would join a Zoom room, where they would interact with one of our many undergraduate research assistants. And the research assistant would introduce themselves as a lead experimenter, and they would give the participant a phone number and an email, maybe like, “in case we’re disconnected or something goes wrong, you can reach out to me.” And then they’d give this participant 10 questions GRE style math test, and they would have 10 minutes to take it. And immediately after they finish taking this math test, the research assistant would have a short, partially scripted conversation with the participant, they would say things like, “Oh, don’t worry if you didn’t do that well. I didn’t do that well, the first time I took a GRE test, I didn’t even know I was supposed to take it until like right before I took it, I did really bad. But I did way better the second time I took it.” And then in the middle of that partially scripted blurb, they would, in one condition, say “I’m the first in my family to go to college, so nobody at home knew anything about graduate school.” And that was our main intervention. So for half of the students, they heard the research assistant was the first in their family to go to college, and for the other half, they didn’t. So immediately after that, they’re given another survey. And in that survey, they are told, “Hey, you’re going to take another math test after this one. And if you can improve your score by about 20%, or by about two questions, then you will be entered into a raffle to win a $20 gift card.” So we’re incentivizing them to want to do better. And then the survey says, “Do you want to go over your answers from exam one?” If you say yes, then the survey instructs the participant to reach out to their experimenter or this research assistant so they can do so; if the participant says no, then the survey instructs them to reach out to the research assistant to get the link to test number two. So in both cases, this participant has been instructed to reach out to the research assistant to get their attention so that they can move on to the next step. But in one case, they are getting academic help, they are going over their questions from exam one. And in the other case, they are just getting the link to exam two. The thing that makes this study fun is that the experimenter isn’t going to answer, the experimenter is going to leave their Zoom camera black, ostensibly off, while they are potentially off doing something else, or distracted, or maybe something’s gone wrong, and they are disconnected, who knows? And they’re going to ignore any attempt to get their attention for about eight minutes. And during that time, what they did was they recorded any attempt to get their attention. We were looking specifically for any additional behavior outside of the Zoom room, something active and persistent added on to that, like calling us or emailing us or texting us, using that information that was provided to them at the very beginning of the session. So after that eight minutes had past, the researcher would come back and say “Oh my gosh, I’m so sorry, something went wrong with my computer,” and then they would either help the student go over some answers or give them the link to the next test. And then the student eventually took the final survey with some demographics and final questions in it.

Elizabeth: So it’s a pretty lengthy paradigm. I think our research assistants had so much fun doing this study.

Makita: They really did.

Elizabeth: They had a lot of acting training to make it believable. But in terms of designing it, in a laboratory experiment, you have to kind of make some trade offs with making everything standardized, but also making it at least somewhat realistic to what might happen in the real world. So we were kind of playing off the idea of being an instructor and having a syllabus where you have lots of information about how to contact the instructor. And we have exams all the time, as instructors in our courses. And so it might be the case that they would need to go over the questions that they missed for the first exam before the second exam. And so we were trying to kind of mimic that type of setting in this one-hour laboratory study. But again, this is a sort of a different experience, where you’re not really getting graded, and it’s not going to affect your GPA. So we had to add a little bit in there around incentivizing them to want to do well in this sort of hypothetical situation. But again, I think our research assistants had a great time collecting these data.

John: What did you find in terms of the differences in help-seeking behavior between first-gen and continuing-generation students?

Makita: So we had a couple of different measures of help seeking in this study. The first measure was whether or not the student wanted to go over their answers. And we found that regardless of condition, so regardless of whether or not the experimenter self-identified as a first-generation college student, first gens overall sought less help than continuing-generation college students, which lined up with what we saw in previous studies. Things got a little more interesting when we started looking at the active help seeking behavior. So students who said, “Yes, I want to go over my answers,” we categorized as academic help seekers and students who said, “No, I don’t want to go over my answers,” we categorized as non-academic help seekers. Then, if students used some kind of additional method of help seeking during that eight-minute waiting period when the experimenter wasn’t responding, they were categorized as active help seekers. So we had active academic help seekers and active non-academic help seekers. And what we found was that students, our academic help seekers, weren’t really impacted by the identity of the experimenter, but our non-academic help seekers were. So in our control condition, when first-gen students were seeking non-academic help, about 13% of them used active help seeking, but in our intervention condition, it was more like 43%. So that was a really big jump, and it was really cool to see that. In other words, when first-generation college students had a help provider available, who was also a first-generation college student, they were more likely to reach out to them in this active persistent way on top of sending a message in the Zoom chat, they were also emailing or texting or calling. When the research assistant identified themselves as a first-generation college student that made our first-generation participants feel more comfortable with reaching out to them in this non-academic realm.

Rebecca: So when we think about this study, what are the implications as we talk to educators or higher ed leaders and actions we might take or ways that we might think about it?

Makita: It’s always difficult to try to generalize from a laboratory study to the field or to real life. In this case, because the person who was performing the role of the help provider was a peer, most things that we can generalize this too would also involve peers. So for instance, if we have a upperclassman teaching assistant for a really difficult math class, maybe if that person self-identifies as being first generation that might make first-generation college students in that class feel more comfortable with asking for help in regards to, say working Canvas or Blackboard or accessing their homework or figuring out how to get the right textbook, things like that. Based on these results, we wouldn’t expect it to necessarily help with their academic performance. But overall, engaging in this type of help over time, might.

Elizabeth: Yeah, I would just add that it kind of highlights two nuances of help seeking that we’ve kind of overlooked in the literature so far. There is a lot of evidence that suggests that first-generation college students seek help less often than continuing-generation students. But not a whole lot of people are talking about the types of help seeking. So, there’s passive type of help seeking versus active type of help seeking. So, differentiating that might be really helpful and understanding where we might need to intervene and help these students. And then what we’re seeing in this study is that we also might need to break it down into the type of assistance that the students are seeking. I think a lot of times, we’re just assuming that they’re seeking help for academic reasons, like I don’t understand this content, explain it in a different way, or related to the content of a class. And what we’re seeing here is that we might actually need to break this down into what’s related to the course content and what’s related to more of this sort of navigational type of awareness that first-generation students might not have the background knowledge to address. And so this non-academic help-seeking behavior might really benefit first-generation college students. And there’s a number of different scenarios in which that might be helpful. So applying for scholarships and financial aid, navigating course seeking and course maps, and figuring out the requirements for different degree programs, applying to graduate school, applying to different research opportunities. All of those things are academic-adjacent, but they’re not academic in the sense of the course material that they’re learning in that course. So it might be the case that all of these other types of help-seeking behaviors, it might be important to intervene in those areas to help first-generation college students.

Makita: And something really interesting about this study is that if we hadn’t separated help seeking into academic and non-academic, we wouldn’t see the difference that we found, if we had just examined it as basic help seeking without separating it, then the nuance of this situation would be lost. And as Elizabeth said, in many of the studies that we read, and that we looked at, they look at help seeking as this just basic block, they’re not separating it out into active or passive or academic or non-academic. And it seems like that actually might be really important because how well an intervention is working for a different type of help seeking might be something that we actually didn’t notice in some of these previous studies.

John: And there’s been a lot of studies recently indicating the importance of providing students with more structure, particularly first-gen students, which might help students get past that barrier. But there’s also been a lot of studies that have investigated the effect of a sense of belonging and comfort in the institution. And having that peer connection to someone else with a similar identity as a first-gen student can help break down some of those barriers and help them overcome that, so that they’ll be more comfortable seeking help in the future, I would suspect.

Elizabeth: Yeah, absolutely. I think anytime an instructor can talk about their personal experiences, overcoming challenges that they have gone through, I think first-generation identity is something that is not as visible as other types of identities. And so that might be something that we need to provide to students if we feel comfortable doing so and talking about that in a way that might relate to students and that belonging, right, making them feel like you can have the same types of success, the same types of career plans, the same types of goals in college, as other students, no matter what your background is.

Rebecca: I think it’s interesting for instructors to just think about taking the time to be explicit about that. It’s an identity that you might take for granted or not think about exposing, but it might be worth planning to expose that really early on in the semester to see how that might benefit students.

Elizabeth: Absolutely.

John: One thing we did on our campus last year was we had a committee that was looking at challenges for first-gen students. And one of the things they did was they created some images that could be used in your signature line indicating that you were a first-gen student. And they distributed that pretty widely. And a lot of faculty and staff members have included that to help form those types of connections. It sounds like, based on your research, can be fairly important.

Elizabeth: Yeah, I think it’s a great initiative. I think here on our campus, we have some stickers and some like door hangers that you can put on your office door and things like that. But I like the email signatures a lot, because that kind of gets blasted to everyone. But yeah, I don’t think it can hurt. I mean, it’s a pretty simple type of thing that you can do. There’s not any evidence that it would be negative for anyone, at least at this point. So it’s sort of like a no-cost way of helping potentially a few students along the way. So yeah, I think it’s a great practice.

Rebecca: Are there follow up studies that this is making you kind of itch to do?

Makita: Oh, yes, definitely. For me, the ideal next step would be to try it in a real classroom, have either an instructor, or a TA for a lab, self identify as first gen to half of the class and not self ID the other and then see if help seeking changes. It would be really, really cool if we could do another behavioral measure of help seeking instead of just self report, but it gets a little complicated when you try to figure out how to track whether or not someone is emailing the TA or the instructor without accidentally infringing on someone’s privacy. So we still have a lot to go when it comes to actually figuring out how to run a study like that. But that would be my ideal.

Elizabeth: Yeah, I think one of the things that we really need is a really good measure of help seeking, whether it’s self report, or whether it’s a way to assess that in some kind of behavioral data. Right now, there’s several different help-seeking skills out there where students respond to them. But they’re not as nuanced as what we’re seeing here around this academic versus not, this passive versus active. So a measure where we could really look at that nuance, I think, would help the field in general in terms of looking at whether interventions are increasing help seeking in various ways. And then of course, the behavioral measure is really, really interesting to us in terms of what students are actually doing. And we’re still kind of figuring out what that might look like, is that something that we can pull from, like their website data. So how often students are looking in the LMS for certain material, how often they’re clicking on things, whether or not they’re going to tutoring centers and office hours and things like that. So we’re still trying to figure out how to measure the behavioral side. But at the very least, we definitely need a really strong self-report measure of help seeking to move this research along.

John: We always end with the question, what’s next?

Makita: What’s next is I’m going to go back on campus and teach an introductory psychology class.

Elizabeth: Great. Next for me is I have meetings all day. [LAUGHTER], but I’m looking forward to the weekend. Actually, something that’s exciting for me is next week I’m going to a conference in Indiana that’s bringing together a bunch of educators looking to build up infrastructure for conducting research in education. So we’re going to be talking about barriers to doing types of different research in education and how we might solve those in the future. So I’m excited for that. I think it will be a great brainstorming opportunity to figure out how to make this type of research easier to conduct in different educational settings.

Rebecca: That sounds awesome. I hope that you’ll share back what you’ve learned and decided. [LAUGHTER]

Elizabeth: Yes, that’s the plan.

John: Well, thank you. It’s always great talking to you, Elizabeth, and it’s really nice meeting you Makita, and I hope we’ll be able to talk to both of you in the future.

Elizabeth: Great. Thank you so much for having us.

Makita: Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]