321. College Students with Disabilities

Sharing student narratives about their experiences can help us to understand how our instructional and policy decisions impact the student experience. In this episode, Amy Fisk joins us discuss to discuss her research project with Rebecca on the perceptions that students with disabilities have of their learning experiences.

Amy is the Assistant Dean for Accessibility at the State University of New York at Geneseo. Amy oversees the Office of Accessibility Services, which coordinates accommodations and support services for students with disabilities. Prior to her role at Geneseo, Amy coordinated a support program for students on the autism spectrum at SUNY Purchase.

Show Notes

Transcript

John: Sharing student narratives about their experiences can help us to understand how our instructional and policy decisions impact the student experience. In this episode, we discuss the perceptions students with disabilities have of their learning experiences.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Amy Fisk. Amy is the Assistant Dean for Accessibility at the State University of New York at Geneseo. Amy oversees the Office of Accessibility Services, which coordinates accommodations and support services for students with disabilities. Prior to her role at Geneseo, Amy coordinated a support program for students on the autism spectrum at SUNY Purchase. Welcome, Amy.

Amy: Thank you. Thank you so much for having me.

Rebecca: Today’s teas are:… Amy, are you drinking any tea?

Amy: I drink tea every morning. So I have Bigelow French Vanilla black tea.

Rebecca: It’s a good way to start the day. How about you, John?

John: In honor of the holiday season, I have Christmas tea today.

Rebecca: I’m drinking Blue Sapphire from my favorite tea shop in Canandaigua.

Amy: Where’s that?

Rebecca: It’s right on Main Street. You should go there.

Amy: I should.

John: We’ve invited you to talk about the article: “Perspectives among college students with disabilities on access and inclusion,” which you co-authored with someone else… Rebecca, I think it was.

Amy: That name sounds familiar.

John: …which was published in College Teaching earlier this year. Before we talk about the article, could you tell us a little bit about your role at SUNY Geneseo.

Amy: So I oversee our Office of Accessibility Services, or OAS. I meet with students to coordinate accommodations and other kinds of support services for our students with disabilities. I monitor policies and procedures within our office. And I often work with faculty and staff on issues related to accessibility and inclusion. So for example, I might do trainings across campus, work with administrators on various committees, and having a voice on issues related to disability, education, awareness, and accessibility.

Rebecca: So prior to this project, Amy, and I didn’t actually know each other. Do you want to share the origin story?

Amy: I started my position here a month before COVID became a thing. So I was kind of thrown into some challenges I did not anticipate. But one of the things I had been thinking about, many of my colleagues were thinking about, was: How are we going to support our students with disabilities? We’re really kind of concerned about their trajectory during this challenging time. We wanted to just get some more information about students’ experiences during COVID. I started talking to Nazely about this, and she says, “You know, I know someone who does research who also might be interested in a potential collaboration.” So that’s how I got connected to Rebecca. And ultimately, we shared an interest in learning more about the impact of COVID on our students within our respective roles on our campuses. We knew that this was a really challenging time for all of us, but especially for our students with disabilities who had already been experiencing barriers pre pandemic. And so we really wanted to hear from our students about their experiences, and what can we learn about access and inclusion moving forward, even when the dust settles and we talk about things post COVID?

John: A lot of the studies that have been done have been quantitative studies. And your study is a qualitative study. Could you talk a little bit about how this qualitative research complements the quantitative research that’s been done?

Amy: Sure. So ultimately, we wanted to gather students’ stories, and many of our findings from our studies are reflective of findings from past studies on challenges and barriers students with disabilities face compared to students without disabilities. But we wanted to identify these specifically within the context of remote learning. And also within the context of navigating this challenging time just in life, we really wanted that student narrative. And we also wanted to assess the positive things that were happening, the practices that were helping our students feel successful, to really help inform tangible takeaways and recommendations to our readers. And we hoped for this information to be relevant, like I said, when the dust settles and regardless of teaching modality. And I think it’s important to highlight that despite the obvious challenges that COVID brought, it has highlighted the importance of accessibility in higher education and really gave us an opportunity to reassess what we’ve been doing, our everyday policies and practices, and we really wanted to highlight that from the student perspective. Beyond that, we also wanted to talk about the needs of our students with disabilities within the context of access and inclusion. So, often disabilities and identities, that tends to be left out of conversations related to diversity, equity, and inclusion. So it wasn’t just about how we need to provide appropriate classroom accommodations, but what are the ways that we can be more inclusive, promote a sense of belonging, proactively provide equal access? So those were the things that we had in mind as we were designing our study.

Rebecca: One of the things that comes up in a lot of conversations, at least more recently in higher ed., is this growing number of students who are registering for accommodations and also the mental health crisis. Can you talk a little bit about that to provide some context for our discussion today?

Amy: Sure. So the mental health needs of college students with disabilities was becoming really apparent before COVID hit, really significant needs related to depression, anxiety, other severe psychiatric impairments. And the studies that had been done around the time of COVID really highlighted those issues of more and more students connecting to their disability services offices, self-identifying as a student with a disability where they have a clinical levels of depression, anxiety, other debilitating mental health needs. And that theme came out in our study as well.

John: Did all the discussions of the challenges of COVID help encourage students to become more willing to declare their mental health challenges or their other needs that perhaps they might have been more reluctant to state prior to this time?

Amy: I think so. I think there is a shift in our culture, and it being okay to talk about mental health and mental illness, for students to say, “I’m having a really hard time, I’m struggling,” because mental health is a spectrum. We all experience a variety of emotions throughout the day, throughout an hour, and throughout our lifetime. And I think it’s becoming slowly de-stigmatized in talking about mental illness and the importance of promoting mental health, especially among our college-age population. A lot of college campuses are really taking seriously the wellness of their students on campus just because of the rise in numbers of students needing that extra support, because colleges across our country are noticing a pretty significant increase. And I do think COVID has propelled that de-stigmatization of talking about mental health.

Rebecca: We’ve talked in the past on this podcast with Kat MacFarlane about some of the barriers that students face in just even approaching and asking for accommodations, having to register with an office of disability services, or whatever the equivalent is on the campus, and having to self identify. And then a lot of students don’t actually choose to do that for a wide variety of reasons, some associated with stigma, but we are seeing increased registrations. So does that mean that there’s increased disability?

Amy: Yes, I think there are a variety of factors and more students connecting with disability services offices. One, I think high schools are better preparing students with disabilities to enter the post-secondary environment. Two, I think our offices are becoming more visible on campus. Again, I think there’s also a de-stigmatization of disability and accessibility services offices, and we’re becoming more visible and relevant on college campuses. And third, I think colleges are starting to talk about disability as an important facet of diversity more and more, I think there’s certainly room for improvement, but I think that conversation is starting to happen. So more students are finding their way into our offices.

Rebecca: So three key themes emerged in our research about the perceptions of students with disabilities, of our institutions, and their experience, and of belonging. And so those three themes are accommodations and accessibility, building relationships, and community, and then course structure and design. Perhaps we can take them one at a time here. Let’s start with accommodations and accessibility. Can you first start with what’s the difference between accommodations and accessibility? Because we know that this is often something that’s confusing to folks.

Amy: Sure. So an a ccommodation, by definition, is designed to remove some sort of barrier that an individual with a disability is experiencing. So an academic accommodation, for example, might be having extra time to take an exam, because timed tests can be a barrier for some students. Maybe it’s a notetaking accommodation because they need assistance accessing that lecture material. Sometimes it’s ensuring that the course materials themselves are accessible, that they can be read through a screen reader. Sometimes the accommodation is related to a course policy such as attendance for a student with a more severe chronic medical condition. So it is an individualized process to assess what an appropriate accommodation would look like. But the purpose of it is to remove some sort of barrier so that this person has equal access to their environment. And so accommodations, though necessary, is something that we’re legally required to provide for the ADA. It’s really a reactive way of ensuring equal access. It’s a floor, it’s a minimum. Accessibility, on the other hand, is about inclusion from the start, so that individual accommodation may not even be needed. And something I like to highlight is that accessibility is not about lowering standards. It’s sending a message that everyone belongs in this space and that inclusion matters.

John: What were some of the most common barriers that students reported facing related to accommodations and accessibility in your study.

Amy: Some of those barriers for students just not receiving their approved accommodations during remote learning, including extended time on tests, for example, or online course materials just not being accessible, or having to continually remind instructors about their accommodations, explain why they needed the accommodation in the first place, negotiating terms of pre-approved accommodations. And this was particularly true among students with what we might call an invisible or non-appearance disabilities such as learning disabilities, ADHD, mental health disabilities, these students are less likely to be believed and questioned about the validity of their disability or their need for accommodation. So those were some pretty significant barriers for students and just not receiving the accommodations that they were approved for.

Rebecca: I think one of the things that was also highlighted as a result of our study taking place, whilst COVID was in full force is how many campus resources students with disabilities and other students depend upon every day. So we had students reporting things like” I didn’t have access to a printer to pull up text when it was more of an image instead of an accessible text that could have been expanded digitally, or having access to a quiet space, like the library.”

Amy: Yeah, that was really significant. And then that is where you also saw some other equity disparities. So there were some students who live down near New York City in very populated areas, there was a lot going down there at the time. COVID, if we recall, some students did not have quiet spaces at home, whereas other students had quiet home offices and their parents may have been at home with them, helping to support them. And then other students who didn’t have a quiet space whatsoever took on more caretaking responsibilities, didn’t have access to WiFi. So those equity disparities continued to widen during COVID beyond the disability barrier, so that was something significant, I think that needed to be highlighted,

Rebecca: What are some of the factors related to accessibility and accommodations that actually resulted in positive perceptions?

Amy: So our students actually reported some very positive interactions with their instructors. So when receiving a student’s letter of accommodation, or like an accommodation notification that would come through our office, some would reach out and ask the student “How can I support you? How can I help provide this accommodation?” One student even noted how they appreciated that the instructor didn’t call them out in class, because that had happened before. So I think just preserving the students’ dignity, reaching out to the student, those were the kinds of things that our students reported as making a significant difference.

John: I know you’re study focused on the status of students during COVID, but in your role addressing these issues now, have the changes in faculty behavior persisted? Have faculty continued to become more sensitive to some of the accessibility and accommodation needs of students as we move back to more classroom instruction?

Amy: So in conversation with colleagues, other disability service providers across SUNY, but also across the country, I think we’ve seen a mix. I think there are some who just wanted to go back to normal, and didn’t we all. I think COVID, again, was a very challenging time and faculty too didn’t have a ton of support, and also really struggled with having that emergency shift to a remote learning modality and some didn’t have the skills or support to really deliver courses in the way that would have facilitated student success. So they were really looking forward to getting back to that in-person modality, back to the pedagogy that we’re used to, and that may have posed some new barriers for students coming back to college campuses. Conversely, we also saw instructors taking some of those learned lessons from the remote learning period and applying them when we did come back to campus. So I do know a number of instructors who, for example, are still utilizing the lecture videos they created during COVID and post them on their learning management system for students who may not have been able to attend class that day, for example, so they can still get the lecture material or recreating their course materials and documents so that they are accessible, creating videos, captioning their videos, modifying some course policies to be a bit more inclusive for students. So I think there has been a change in realizing we can still have students be successful and meet the learning objectives, but in a different and more inclusive way.

Rebecca: I think one of the things that we can highlight that also came out in the student experience, and students reported this in our study, is that some of them actually experienced better access during COVID. Not all but some, in part because some of the technology caught up. And when we first went remote, Zoom didn’t have captions available by default and now it does. And so a lot of these things have become norms that people with disabilities have fought for for a long time and never got.

Amy: And I would say that’s true as well with regard to course policies that may have been amended as well, introducing more self paced work, which is also something that students really appreciated during the remote learning period.

John: I just recently returned from the POD conference where there were many, many discussions of this very issue. And in general, the results there were pretty much the same as what you’ve described, that a lot of the changes that faculty made to better accommodate students needs persisted, but some faculty have moved back to old practices and the results are a little bit mixed. But on average, there seemed to be, in a number of studies, some substantial improvement in faculty responses to student needs.

Rebecca: Based on what students have reported, what recommendations do you have for faculty related to accommodations and accessibility to continue the forward movement as opposed to regressing?

Amy: So, actually I did a talk with faculty of one of our academic departments at the start of the semester, reviewing our office, some of the logistical pieces of implementing accommodations, that sort of thing. But before I started really getting into that, we had a discussion about how accommodations, and the dialogue about accommodations with students are approached, how it’s discussed, how it’s communicated, something as simple as taking time to actually review the portion in your course syllabus related to an accommodation, so maybe an accessibility statement. That tells students that this is important, making sure that your online materials are accessible from the start, that tells students that accessibility and inclusion is important. And students are more likely to engage in a reciprocal dialogue with you about their needs when they feel like they’re heard, when they feel like they’re a valued member of the class, that their accommodations are important and not burdensome. That’s a term we heard a lot in our study, that they’re not a burden, or it’s some sort of requirement that the faculty has to fulfill. And so I think this is probably true for most students, regardless of disability. But students in our study specifically noted how they appreciated when the instructor showed empathy and understanding and flexibility, recognizing that students have significant issues outside of the classroom. We all do, between family, finances, things that are happening in our world today. And I think this is important to acknowledge as well, given that we’re seeing an increase in students from various diverse backgrounds coming into the college environment.

Rebecca: And as we’ve talked about many times on the podcast, flexible doesn’t mean not having standards. [LAUGHTER] And it doesn’t mean a free for all. In fact, a lot of our students benefit from structure, which we’ll talk about, I think, in a few minutes, because that ties to one of our other themes. You talked a little bit about faculty workload related to this, and sometimes the perception that faculty put off is that it’s a burden to provide these accommodations. And the reality is that a lot of our students need very similar things. And so if we think about the common requests for accommodations, or digital accessibility strategies, from the start, we often don’t have a lot of one off things that we do need to accommodate, because we’ve already built it into our courses. That’s not to say that there aren’t accommodations that we need to provide additionally, but it may result in less work, ultimately, to really think about these accessibility principles upfront.

Amy: Right. And I think something as simple as making your course lecture materials available on the learning management system available to students. That can help reduce a lot of barriers for many students who might struggle with keeping up with the pace of the lecture and they end up missing material. A student who may have missed class that one day and just needs that material, other students who need to kind of re-teach themselves the material because, perhaps, they had challenges with staying focused during class. I think there’s a variety of reasons why students would benefit from that, but something as simple as that. Often, when students come to see me, there are maybe students who hadn’t needed accommodations previously, but they encountered a particular course where the policies were such that there were new barriers that arose and if the policy was different, perhaps they wouldn’t need that accommodation. That’s a concrete example of the difference between accommodation and accessibility. Some of our course policies and course design may be inadvertently barriers to students with or without disability. So this might include use of pop quizzes, not making lecture materials available to students, not permitting use of technology, not allowing students to even take breaks in class. And so although the purpose of these policies is probably to make students engaged and have accountability in the course, which these are things, of course, we want… again, we’re not lowering standards… students still need to go to class and do the work. But I think some of these policies actually might be having the opposite effect, and it does for students who request accommodations, rather than focus on learning in the course.

John: I think that many faculty who had only taught in a face-to-face modality before COVID, were able to avoid issues of accessibility by not creating digital content. When they moved to remote teaching, though, they were forced to begin developing digital materials and often received some training in creating accessible digital content. Do you think that that training received during COVID helped encourage more accessible practices by faculty in general?

Amy: I think so again, I think some of these practices have shifted over time, and I think COVID has shed light on the benefits of accessibility, not just for people with disabilities, but for all people. I mean, again, use of captions and subtitles can be beneficial for a lot of folks, whether you’re sitting in a busy Starbucks, whether you have a lot going on in the background, maybe you’re trying to juggle work and family, maybe, again, you’re hard of hearing, and so you need access to those captions. Again, accessibility is for all, not just about or for people with disabilities.

Rebecca: The second theme that kind of emerged in our research was building relationships and community, can you share some insights with faculty about the role that they can play in helping students with disabilities feel connected and included? And you highlighted some of those already: providing accommodations and showing students some dignity and respecting their dignity.

Amy: So again, I think engaging with a student and even something as simple as taking the student aside and asking, “How can I make this course more accessible to you?” speaks volumes to the student, that they are valued, they belong, that their needs aren’t burdensome, and they’re more likely to engage in a reciprocal dialogue with the faculty member when they feel like “Oh, they care about me and my success in this course.” I actually knew about a professor who did an anonymous Google form, asking students “How can I make this course more accessible to you? Are there barriers? In reviewing the syllabus, do you have concerns about something within the course?” One of my students actually told me about this, and said how it really made them feel seen and valued. And they were more likely to reach out to the instructor when they needed help, because some students fail to do so out of shame. They’re in a very vulnerable position to talk about their disability related needs to a faculty member, to an authority figure. And so when you do something as simple as asking a student, “How can I make this accessible to you? Are you experiencing barriers right now?” really opens that line of communication with the student and helps them build a positive relationship with that instructor and for maybe other instructors. It also helps to build a sense of community so that other students know that this is really important, and that inclusion matters. And that’s also sending a message to all their students within the classroom that we appreciate and respect diverse learners here in this classroom. I think that’s a teachable moment for our students as well.

Rebecca: So one of the other things that I think emerged is a desire to be connected with peers, but that faculty can play a really important role in facilitating that connection. So I think oftentimes, we just assume in a classroom that at the beginning of class students are socializing and getting to know other folks and have those contacts, but students really reported that having more structured ways of connecting with peers was really beneficial to them outside of class. And that’s something that I think we might take for granted as instructors in the classroom, that it would just kind of organically happen. But that structure, that scaffolding around that really bubbled up as being pretty important to our students,

Amy: Yeah, that peer-to-peer interaction for an even if it was virtual. One of our students said, “Our instructor had a virtual whiteboard that we could all do group work even when it was asynchronous, which is pretty neat.” So that helps set the stage for positive peer interactions, for peers to ask peers for help and mentorship, which is important. Often, students just feel that going to office hours is the only way that they can receive help. And when you provide opportunities to work together, learn together, that really helps, again, open up a line of communication among peers as well, which is a skill that we’re trying to teach our students.

John: And that was especially severe during COVID. But also, when we returned to the classroom, and students were asked to sit at least six feet away from any other student, it certainly reduced the amount of interaction and it has made it a little more challenging for all students to interact with others. That’s been improving, but I think, perhaps, that experience may remind faculty of the importance of building those types of connections. Because even before COVID, there were always some students who may not have felt as much a part of the class community. But I think we’ve all learned the importance of community during that time.

Rebecca: I think that’s just another example of something that students with disabilities have pointed out as being really important to them. But it’s also important to many other students, too.

John: The third theme that emerged from your research was course structure and design. And most of your findings in that particular category align with many other studies involving inclusive pedagogy and Universal Design for Learning. Can you highlight some of the common barriers that students with disabilities faced in terms of course structure and design.

Amy: So one of our students in the study commonly referred to one of their course LMS pages as a scavenger hunt, where they spent more time trying to find the materials and the information on the course rather than on the assignments themselves. So students in general benefit from an organized LMS and an organized syllabus for deadlines, instructions, policies are very clear and concise, but for students with disabilities, this is particularly important. Many of my students with ADHD, health or chronic medical conditions, or a learning disability, they need to plan ahead, because it might take the students double or triple the time to finish a task. So if students don’t know when their next test is, or if instructions aren’t posted a few days before something is due, we’re really not setting them up for success. And I also talked about some of those other policies and course design that might be inadvertent barriers to our students. And so some of our students reported that they did benefit from self-paced tasks, or on untimed learning assessments, having some autonomy and options for completing assignments in a different format, such as doing a presentation or a podcast, instead of a paper, working in groups or choosing to work individually on a project. Those are some of the specific practices our students highlighted as being really helpful. And again, we’re not lowering standards, they have to meet the same standards and learning objectives, as every other student, just perhaps meeting those same standards in a different way. And that’s what Universal Design for learning is all about.

John: One time in a workshop, a faculty member mentioned that they have students do a scavenger hunt in the LMS, to find various course policies, or to find materials. And I cringed at that and I suggested that it might be better to design your course in a way where the students don’t have to struggle to find things so they can focus their cognitive efforts on learning materials, rather than engaging in scavenger hunts, trying to navigate the course. Has that improved recently?

Amy: I think it has, again, in conversations with some of my colleagues who do this work and talking with faculty, I think it’s a mixed bag as it relates to how instructors are approaching course design in their policies. But other faculty are seeing that changing their pedagogy, changing their policies, changing the way they interact and see students and helping to meet those student needs have evolved, because perhaps they themselves have experienced accessibility barriers during COVID as well. And so it’s become more relevant, because they have that lived experience. And they’re seeing that adopting some of these inclusive practices are actually helping to keep their students engaged, that the students, even if they’re struggling, are more likely to tell their faculty member “I’m struggling and I need help, but I want to stay in this course, what kind of flexibility could be provided?”… rather than, we’ll use a college student term, ghosting [LAUGHTER] the class. So I think things are changing in a direction that speaks to some degree of flexibility and helping students meet those same standards, where the focus is more on learning, rather than adherence to an arbitrary policy.

Rebecca: I think the students really underscored maybe without realizing things like the transparency and learning and teaching or TILT, where being really clear and explicit about what the expectation is and how to get there and how you’re going to be assessed really helps and supports students… that structure and those guardrails is what all of us need. How many times have we worked on a paper the second before a deadline? We work on deadlines, and so if we help students with intermediary deadlines, we’re actually helping them and that doesn’t mean that we’re not flexible,and flexibility doesn’t mean not having those.

Amy: It’s about scaffolding. It’s about recognizing that not all students are coming from the same background and experiences and privilege. They’re not on the same playing field, and so providing those scaffolded learning opportunities… that can really help even the playing field, just providing those scaffolded learning opportunities.

Rebecca: And it’s really some of this scaffolded accountability, so it’s not all due at once, It’s helpful to faculty to remind them that there’s feedback throughout a process on a larger assignment, but also it’s helpful for students to hit individual deadlines to evolve their work as well.

John: And that’s something that is found, as you noted earlier, by Mary-Ann Winklemes in her research on Transparency in Learning and Teaching, and also by Viji Sathy and Kelly Hogan in their research on the importance of structure in reducing equity gaps. While transparency and structure benefits all students, it especially benefits the students who have equity gaps of some form, and it sounds as if that’s also true for students with disabilities.

Rebecca: Yeah, I think none of this is really new, but oftentimes students with disabilities aren’t necessarily included in those studies about equity always, it’s not always one of the groups that’s pulled out separately.

Amy: Part of what’s next is also hearing about the experiences of students with disabilities from other diverse backgrounds, including students of color, students from lower SES backgrounds, students in the LGBTQ+ community, that those experiences are different and that intersectionality is really key in understanding students’ experiences in the classroom and how we can be more accessible and inclusive because, again, accessibility is not just related to are we providing a legally required accommodation, but are we creating a sense of belonging in that space, and giving students an equal opportunity to demonstrate their knowledge and be successful, which is ultimately why we’re all here, I would hope.

Rebecca: So we always wrap up by asking what’s next?

Amy: I think it’s important to not just put a focus on what individual faculty can be doing in their classrooms to support students with disabilities. But how are we promoting access and inclusion at the institutional level, supporting students with disabilities and students from other diverse backgrounds is a whole campus responsibility and faculty needs support in doing that work as well. So I’m hoping what’s next is working with administration, other campus leaders and identifying ways we can really help move that needle in a meaningful way. Making accessibility into larger DEIB (or diversity, equity, inclusion and belonging) campus initiatives, our campus-wide policy, strategic planning, campus-wide faculty and staff training, and other professional development opportunities, hiring diverse faculty and staff on our campus. So not just about talking the talk, but walking the walk when it comes to access and inclusion in higher education.

Rebecca: I think that’s definitely a theme that we’ll see throughout all of higher ed. I hope that we’ll all go home and arm and move in this direction collaboratively.

John: Well, thank you for joining us. It’s been great talking to you and we’re looking forward to hearing more of your future work on this topic.

Amy: Well, thank you so much for having me today. I appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

320. Gender Differences in Faculty Retention

Women and men leave academic positions at different rates and for different reasons. In this episode, Aaron Clauset and Katie Spoon join us to discuss their research on the magnitude of and differential causes of gender differences in faculty attrition.

Aaron is a Professor of Computer Science at the University of Colorado, Boulder. He is a nationally recognized expert on network science, data science, and complex systems and he is the recipient of the 2016 Erdos-Renyi Prize in Network Science. Katie is a computational social scientist and a 4th-year PhD candidate, also at the University of Colorado, Boulder.  Aaron and Katie are two of the authors of a paper on “Gender and retention patterns among U.S. faculty,” which has received a great deal of attention and has been discussed in the Chronicle of Higher Education, Times Higher Education, Science Careers, and Nature News.

Show Notes

Transcript

John: Women and men leave academic positions at different rates and for different reasons. In this episode, we examine the magnitude of and differential causes of gender differences in faculty attrition.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Aaron Clauset and Katie Spoon. Aaron is a Professor of Computer Science at the University of Colorado, Boulder. He is a nationally recognized expert on network science, data science, and complex systems and he is the recipient of the 2016 Erdos-Renyi Prize in Network Science. Katie is a computational social scientist and a 4th-year PhD candidate, also at the University of Colorado, Boulder. Aaron and Katie are two of the authors of a paper on “Gender and retention patterns among U.S. faculty,” which has received a great deal of attention and has been discussed in the Chronicle of Higher Education, Times Higher Education, Science Careers, and Nature News. Welcome, Aaron and Katie.

Rebecca: Today’s teas are:… Aaron, are you drinking any tea today?

Aaron: I am drinking coffee.

Rebecca: Always a popular tea choice. How about you, Katie?

Katie: I made tea specifically for this. And it’s

Rebecca: Yay!

Katie: …and it’s Trader Joe’s Ginger Tumeric herbal tea?

Rebecca: That sounds yummy.

John: That’s really good. I’ve had that. And I have Prince of Wales today.

Rebecca: And I have some Christmas tea, a little ahead of schedule.

John: We’ve invited you here today to discuss your paper on “Gender and retention patterns among U.S. faculty.” Could you give us a general overview, of the focus of the study?

Katie: Yeah, so we were interested in this topic of gender retention in academia, because it’s really hard to identify faculty who have left academia. So a lot of studies have had to look at a very narrow or specific population of faculty. And so while there’s a lot of studies on this topic, the majority seem to focus on assistant professors, STEM fields, and higher prestige institutions. And so we were really interested in doing a systematic study across career age, institution, and field.

Rebecca: Can you talk a little bit about what data you used for this study.

Katie: So we have a data use contract with Academic Analytics Research Center, which is a company that contracts their data with research teams, and it’s a census for all of the tenure-track and tenured professors at US PhD-granting institutions across, I think, over 100 fields that we grouped into broader domains. And we have data from 2011 to 2020, for all faculty within that period, so we can see if anybody left their job during that decade, then we have that information. And then our study has two parts, we first look at just the rates women and men faculty are leaving their jobs using that employment census. And then we emailed a frame of around 70,000 of those professors from 29 fields, and got about 10,000 responses to a survey looking at the underlying reasons why faculty were leaving.

John: So that’s a remarkably large sample. How did you measure faculty attrition in a sample,

Katie: We calculated attrition risk as just the number of people who left divided by the number of people who could have left in a given year. And then we calculated that across career age. So that’s the year since a faculty member received their PhD. That was kind of our simplest calculation of attrition risk. And then, from that analysis, you could kind of see broadly that women were leaving their jobs at higher rates than men at every career age and the gap seems to grow over time. And so then we did a logistic regression analysis, looking at the odds of leaving a position where we adjusted for other factors.

John: What other factors did you hold constant in the logit regression?

Katie: So we were adjusting for career age, employer prestige, so the institutional prestige, and whether they were trained for their PhD in the US or abroad, and adjusting for those factors, we found women were more likely to leave than men. And that varied a lot across career age, field, and institution.

John: We’ve long known that women are not very well represented in the upper ranks of faculty, but many people would try to attribute that to the fact that we’ve seen a general increase in the number of women entering disciplines. But what you’ve demonstrated is that we’re losing a lot of people along the way, that we don’t see as many women full professors, not just because there weren’t that many women entering that stream a decade or more ago, but also because we’re losing so many people, which I think is a really important result, which I don’t believe had been measured as carefully before. Is that a correct interpretation?

Katie: Yeah, I think definitely both hiring and retention are important for overall gender equity in academia. If you hold the right It’s of attrition constant from our study, then even if you start with a 50-50 cohort of men and women as assistant professors at career age one, then even after a whole career, that falls to about 40% women, and a lot of people leave during that time, like a lot of men too. The majority actually have left over that time, but women are leaving faster, which I think is important to keep in mind.

Aaron: So the gender representation in the U.S. academy is complicated, in part because the academy is really big and diverse. I mean, there’s lots of different fields and lots of different institutions. And this is partly why the literature on this topic is really messy, is because many of the studies look at one institution, they may look at R2 institutions, or fancy R1institutions, or only this field or that field. And so it’s been sort of hard to get a sense of what’s going on. And there’s lots of sort of claims about why it is that you don’t see as many women in the most senior ranks in academia. So the idea that it’s a sort of a demographic inertia thing that in order to be provost at a university, well, you got your PhD probably 20-30 years ago. And so the gender ratio at 20-30 years ago is what sort of sets the maximum gender ratio 20-30 years in the future for those senior people, and of course, that’s a part of the explanation. That’s certainly not wrong. But we were very interested in understanding this attrition aspect of people who are leaving academia. And not just like changing universities, I think that’s a critical piece of the way we measured attrition is that if you went from Princeton to Yale, we don’t count that as an attrition event, you have to leave the entire set of PhD-granting universities in the US in order for us to count you as an attrition event. So these are people who are not just leaving one academic job for another, but like they’re leaving the American academy entirely. And so the results of looking at the data, because this sort of broad census lets us see that the rates, overall, women are leaving at higher rates than men. But it gets really messy when you start looking at individual disciplines. Some disciplines have higher rates than others, some places, it doesn’t seem like there is a gender difference in the rate. And when you drill down to individual fields, it gets even messier because fields have different norms and different sizes and things. So the real power of the study is in part because it has this just enormous view, a quarter million faculty in the data set, which lets us look at differences in these small rates in a really powerful way.

Rebecca: You’re just mentioning that there’s messiness across disciplines and fields. Can you talk about whether or not there was any similarities in STEM and non-STEM fields?

Katie: I think, maybe surprisingly, we found the gender gaps in retention were actually larger in non-STEM fields than in STEM fields. And that was especially true at the full professor level, the gender gaps overall grow with rank. And so that kind of surprised us, I think, because it’s a little counter to what the literature has mostly focused on, which is assistant professors, and especially in STEM. So we actually find that full professors in non-STEM fields are the group with the largest gender gap in retention.

Aaron: This is an important finding, in part because there’s a narrative that sort of underlies a lot of discussion about faculty attrition and gendered attrition, in particular, that once you get tenure, it’s fine. You can’t get fired, tenure is the goal you’re looking for. Everybody wants it. And once you get it, somehow, everything gets better. And so this finding in the paper, that it’s actually tenured women who are leaving at higher rates than anyone else, it kind of inverts that narrative and shows that we’ve been maybe looking in the wrong place for the real effects of this thing.

Katie: And I think, yeah, more on that point, for us seeing that the effect was larger for full professors kind of points to maybe something else going on besides professional things, or work-life balance in the early career, which is what led us to do the survey and check out what was happening underneath the rates.

Rebecca: There’s often a stereotype that women maybe earlier on in their career leave because of childbirth or raising children and things like that. But that’s definitely counter to what you’re suggesting.

Aaron: Women certainly still leave the Academy for work life-balance reasons. So the observation that the rates sort of run counter to the narrative that has been explored very thoroughly in the literature, Katie was saying it really points to there being more to the story, not that the previous story was necessarily wrong, but like there’s just more going on. And also, things have certainly changed in the past 30 or 40 years. Gender norms change. Men are much more involved in child rearing now than they were 40 years ago, and so our study is at a particular decade of time. So we aren’t able to say things as much about what was going on in the 1980s. But today in the 2010s, when the census data on attrition was collected, we can see that there’s more going on and, as Katie was saying, that’s where the survey really helps us go beyond just looking at the rates of things to try to look at the reasons for these rates.

John: To investigate the causes of attrition, you then did a survey. Could you talk a little bit about the sample that you used, how large it was, how you determined it, and what you found?

Katie: So we took the larger census employment data and we selected 29 fields that we considered broadly representative. We wanted a mix of fields across different sizes, different domains of study, as well as fields that had commonly been studied in the literature, like engineering or computer science versus fields that were less commonly studied, like education. And we then attempted to get emails for all of those people, and then ended up with a frame of about 70,000 people who we then contacted. We got roughly 10,000 responses from both current faculty members, as well as former faculty members who either left academia or retired. And then we also looked at faculty who switched institutions. I think the survey really showed us that while there were no visible gender gaps in certain fields, or at certain ranks, in the first part of our study, where we were looking at the rates that women and men leave, we found that the reasons women and men leave are still very gendered, even in those cases. So for example, I think early career STEM faculty, there was no visible difference in the rates that women and men left. But in the survey, we did find that women were more likely to feel pushed out of their jobs and less likely to feel pulled towards better opportunities. And that, while there was a lot of variation across domains in the retention rates, that there wasn’t much variation in that pattern of feeling pushed or pulled, it was like varied more universally, I guess, across domains.

Aaron: That push-pull dynamic is really key, and it was something that had been studied previously in the literature. And so we wanted to incorporate that into the survey. And I’m just going to read to you so that your listeners can hear what we meant by a push and a pull. So the question we asked for the push was,:”I am unhappy, stressed or otherwise less than satisfied in my current position,” and the pull was “I am drawn to, excited by, or otherwise attracted to a different position.” And so the idea of those pushes and pulls is really about whether the person wants to stay in their current position or not. Are they drawn to something else? Is it an opportunity that they’re pursuing to leave their current position? Or do they feel like they’re being forced out, that they’d rather stay where they are, but they feel like they cannot stay. And so that push and pull really sort of illustrates the idea that it’s up to that person to decide whether they feel like they can stay or want to stay in their current job, or whether they are choosing to move to a different opportunity. And people could choose both. I mean, leaving an academic job is a big decision. Sometimes you might feel unhappy, and you start looking for an opportunity and then you find something that looks more exciting. And so survey respondents could check both. They said, “I could feel pushed and pulled at the same time.” And so our results on the push-pulls were just focused on the ones who said either push or pull.

Rebecca: Given these findings, what are some things that colleges and universities might do to reduce the gender differences in attrition rates and maybe make it more attractive to stay in academia?

Katie: Yeah, so I think a key finding of our study is that, even where the rates may look the same, looks like there’s gender equity, the reasons could be very different. So I think the first thing would be to also investigate: are the reasons people are leaving gendered, even if doesn’t look like they’re leaving at different rates? And then I think, as far as the particular reasons, we did kind of look into what pushes people were experiencing specifically. And while professional reasons like productivity and funding, as well, so work-life balance, which is things like caring responsibilities and the workload. While both of those categories are definitely still important, especially for faculty at different career stages, we found that climate, the workplace climate, so how someone feels when they go to work around their colleagues, they feel that they don’t belong or fit, if there’s dysfunctional leadership, that those workplace climate reasons were the most gendered, both for former faculty who actually left a position, as well as for current faculty who were considering a hypothetical decision to leave your job. And so I think focusing on the climate reasons, especially as a first step, would be a good way to reduce the gendered impact, specifically, but for retention overall, I think that all the reasons are probably important and listening to faculty, taking people seriously when they say this is why I left, is good.

Aaron: One of the workplace climate findings from the survey that was, I think, most striking is that climate was listed as a very important reason by women at all career ages, whereas the work-life balance issues were most salient early in the career and then it sort of tails off for everyone, men and women. And then professional reasons are sort of high but they also just sort of tail off over time, but the climate reasons are high for young women professors and also senior women professors. So this suggests that this is a place where universities are really not moving the ball forward and trying to improve things because you have both the young faculty and the senior faculty all saying consistently this is a huge issue for their stress. And they’re feeling like they belong, and they feel pushed or pulled out of their academic positions. So figuring out how to move that needle is critical. And there are some efforts. So for instance, Harvard has this CAOCHE survey that many universities do on climate, but it’s typically done at the institutional level, but climate is typically a departmental thing. So if you look at like the computer science department versus the sociology department, they’re very different social sort of contexts. And one department could have a good climate, one is very supportive and inclusive of women, and the other could have a very dysfunctional one. And so these sort of large scale institutional instruments for measuring climate, they may not really be up to the task of allowing an institution to improve in this way. So figuring out how to get down to the level at which climate is actually operationalized, which is the department level, I think, is a key need for institutions of higher education.

John: One of the results that we haven’t yet talked about is, when you were looking at the initial study of attrition rates, when you included a control for the major field, you found that there was no significant gender difference, holding academic field constant, for people at the more junior levels. But at the associate and full professor level, there was a significant gender effect. That result I thought was kind of surprising. I would have guessed that there would have been more women leaving earlier in the career. And I think that’s what many people would suspect because of issues such as work-life balance, and so forth. But I thought that result was really surprising. It certainly has to do with climate, I think, as you’re suggesting, but why would it be more severe at the associate and full professor level than at the more junior level? Or is it just the cumulative effect, perhaps, [LAUGHTER] building up and getting worse over time.

Katie: We added fixed effects for fields to try to get at if you have a woman and a man at the same institution and in the same field, so they’re probably in the same department then or very similar departments. We were trying to get at that average field effect. Our main results are like, if you were to just select someone in the natural sciences, what is their odds of leaving if they’re a woman compared to a man? And women are distributed across fields unevenly. And so basically, adding fixed effects for field lets us adjust for that variation in gender ratio, or other kind of field norms. And so I think the fact that the effect goes away at the assistant professor level, like the effect is not very large to begin with, for assistant professors. And so it’s possible that like, adding that field effect just reduces the gendered effect for assistant professors, whereas the effects are larger for tenured faculty. So really, it’s just saying that even when you adjust for someone’s field and all of these gender ratio, and other field level variables, we do still see something going on. We can’t say for sure why that is, but I think the survey really suggests very strongly, that is pointing to the climate.

Aaron: Our study was not designed to get at the causality of any of the differences in the rates from like a causal inference perspective. And so we can speculate about why senior women might feel like climate is a larger issue in terms of wanting to leave, and then many more of them are leaving, etc., and we can speculate about that. And certainly, if we all have experience in academia, we know women who fall into this category, we probably heard some of their stories. And those stories are real, and probably many more women have those stories than we might imagine. And so we can speculate about that, but the study itself was specifically aimed at looking at the rates for leaving for attrition and then asking in the survey, the reasons that people felt like they could or would or did leave their academic positions. When I was visiting UMass-Amherst last week, and I was talking with some of the folks in their NSF funded ADVANCE grant program. We had a very nice comment session about this specific question, about what it is that the senior women are experiencing. And several of the members of the ADVANCE team there sort of talked about the idea that… it makes a lot of sense… that over the course of a 20- or 30-year career, you might experience sort of the accumulation of these sort of devaluation experiences, and that might slowly sort of erode your belief in the system and your belief in a sense of belonging, even though you are tenured, or you are a full professor, or maybe even distinguished professor… you know, you’re at the top of your career, and yet 20 to 30 years of being devalued by your institution and your male colleagues. I mean, who wouldn’t consider moving on to something else, maybe considering early retirement, for instance. And so it’s not hard to imagine what’s going on with the senior women. But I think that it’s really important that future work really dig into that specific question. Because there is this sort of narrative that once you make it as a full professor, you get to be the boss and everybody else who’s more junior than you. But that’s not what we’re seeing in the data, we’re seeing that the senior women are saying that climate is as much of an issue for them, as it is for the junior professors who have been there for three or four years.

Rebecca: Your point earlier too isn’t lost on me that there’s fewer women in administrative roles. And this is the pipeline for those roles [LAUGHTER] too, that you have to be more senior often to be in those roles. And it’s really concerning about the future of the academy when we’re thinking about how few women are continuing on.

John: So, your study focuses only on PhD granting institutions. One source of attrition, though, that could occur, potentially at least, is faculty may be leaving for comprehensive or four-year institutions and so forth, which wouldn’t be picked up in the data. And that might be a nice thing for someone to explore as a future area of study, to see if some of this might be a shift in institution types. I suspect it’s not, but we do know that there’s a much larger proportion of women in community colleges and a much larger proportion of women in four-year colleges than in PhD granting institution. So it’s possible some of it could be attrition in that form. But that still doesn’t explain why there is such a gender difference in the climate, which seems to have a much larger cumulative effect than immediate effect.

Aaron: This makes me think of our results on prestige. Because nothing makes sense in academia, except in the light of prestige. And one of the things that our research group here has been very active in is in looking at how prestige structures academic careers, and the production of science and so on. So we really wanted to know, like, is prestige lurking on here as well. And we found some really interesting things.

Katie: Yeah, I think the main effect for prestige… it’s gendered slightly… but the main thing is that faculty at lower procedure institutions overall, regardless of gender, are much more likely to leave, then faculty at higher prestige institutions. And again, our analysis is looking at all-cause attrition, so we don’t know if they are leaving to go to a different type of institution or leaving academia altogether, but it’s definitely a huge effect. And then we do find a small gendered effect on top of that, where women are even more likely than men to leave low-prestige institutions, especially full professors.

Aaron: And if we think about how many professors are at those elite institutions, I mean, the Ivy League or the Ivy League plus, it’s a really small number of institutions that receive a huge fraction of our attention in academia. And that’s the prestige game right there. And so if you think about the number of people who are at the elite institutions where the attrition is lowest, it’s not many, most tenure-track faculty are at these less elite institutions. And so they’re experiencing these higher attrition rates. And we don’t have an explanation for why. But we’re very curious to know what is it about these lower-prestige institutions that makes attrition overall higher and more gendered?

Rebecca: Many questions to continue exploring, for sure. [LAUGHTER]

Aaron: Yeah, it’s a mystery.

Rebecca: So why do we care about attrition, anyways? We talk about it with students all the time, we talk about keeping students and retaining students. Why is it important to think about retaining faculty and keeping them in the pipeline in higher ed.?

Katie: I think there are a lot of practical reasons why it’s good to retain women. Studies have shown that people tend to ask different types of scientific questions based on their identity, such as their gender or race. We also know that having women in leadership roles, there’s been some studies showing that that can influence younger women’s perceptions of being a scientist and things like that. But personally, I think the most important reason for doing this is that scientific talent is not inherently gendered, and so there shouldn’t be differences in how women experience their workplaces. This is what I think.

John: And if people are going to invest in acquiring a PhD, that’s a pretty large investment for them as an individual. And it’s really unfortunate to have that wasted, especially if they’re feeling pushed out. If they were moving on to some other position where their skills could be better used, that would be an improvement in efficiency for them and for society, but if they’re leaving because of a chilly climate, that’s probably not the best use of society’s resources.

Rebecca: It’s a lot more resources to bring on new people over and over again too.

Aaron: Every faculty search is a big lift. And if you hire a bunch of talented women, and then they don’t feel like they belong in the academy, and they leave, then not only do you have to hire new people because you have the faculty leave, but every one of those incredibly talented, highly trained, highly productive women, their contributions to science have now been lost in some sense. And that seems like a moral tragedy for all of us.

Rebecca: I think a lot of our institutions focus a lot on having policies and strategies that are supposed to be equity minded. This is an equity gap that isn’t always attended to.

John: And also there’s a lot of research that suggests that when faculty are happy and excited about their work, that enthusiasm tends to get conveyed to students. And we’re losing that we have a lot of people who are unhappy being in the positions they’re in.

Rebecca: Wait, is that why we’re in higher ed, just teach students and get them excited about our fields. [LAUGHTER] Sometimes we lose track of that. [LAUGHTER]

Aaron: In some ways, that’s like the only reason to be in higher ed.

Rebecca: Exactly.

Aaron: As a tenure-track professor, our job is teach the next generation of students and then also make discoveries about the world and so on. And if we didn’t want to do the teaching part, then we would go be a research scientist somewhere. And there’s plenty of people who can do that. But it’s that combination of the teaching and the research that makes being a tenure-track professor in the American academy really special and sort of the envy of the world. In fact, the American system, to a large extent, is being copied and has been copied by the rest of the world’s higher education systems, because it’s such a powerful synergy. And you’re right. If your faculty are happy, the students will feel that and they’ll be excited about continuing on, and they’ll get infected with the excitement of science and discovery and so on. And in some ways, it’s kind of confusing that institutions are so behind the curve on faculty attrition, and especially gendered attrition, because it’s not like this is a new phenomenon. I mean, our paper is the latest in a long history, you know, 40 plus years of studies looking at gendered attrition at different places in the pipeline. And the fact that it’s still something to study is kind of an indictment of all of higher education.

John: And on that happy note…. [LAUGHTER]

Rebecca: I was going to say, on that powerful note.

John: …we’ll move to our final question, which is: “What’s next? “

Katie: Well, we actually have a follow up study that we’re working on right now. Aaron had kind of mentioned a little bit earlier about this accumulation of feeling devalued across a career. We asked our survey respondents what needs to be different about their positions to reduce their stress, or what needed to be different for them to stay in their positions. And we got over 6000 free-text responses to that question. We then read them all and we’re working on writing up the results. But it’s very powerful reading these stories from people and really narrowing in on like, what exactly is going on in the climate? What is going on for older women? for women of color? for mothers? …and so we’re excited to get that out.

John: Excellent. Well, thank you. It’s been great talking to you. And we’re looking forward to seeing that next study.

Aaron: Thank you.

Rebecca: Yeah, this is important information. I’m glad that we’re able to share it in this space.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

318. Reducing Equity Gaps

Gender and racial equity gaps exist in economics and other STEM fields. In this episode, Tisha Emerson joins us to discuss research on strategies to reduce these inequities. Tisha is the chair of the economics department and the James E. and Constance Paul Distinguished Professor at East Carolina University and is the incoming Chair of the American Economic Association’s Committee on Economic Education.

Show Notes

Transcript

John: Gender and racial equity gaps exist in economics and other STEM fields. In this episode, we discuss research on strategies to reduce these inequities.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Tisha Emerson. Tisha is the chair of the economics department and the James E. and Constance Paul Distinguished Professor at East Carolina University and is the incoming Chair of the American Economic Association’s Committee on Economic Education. Welcome, Tisha.

Tisha: Thanks for having me. I’m so glad to be here.

John: I’m very happy to be talking to you. And our teas today are:… Tisha, are you drinking any tea?

Tisha: Yes, my favorite tea, I found a couple years ago, it’s a Golden Moon brand of Tippy Earl Grey.

Rebecca: That sounds nice.

Tisha: It’s delicious.

Rebecca:I have Harvest Memories today.

John: I don’t remember that one.

Rebecca: Well, it is from my little favorite tea shop in Canandaigua, New York, and it’s autumn flavors.

John: Autumn flavor?

Rebecca: Autumn flavors.

John: Do they have leaves dumped in there?

Rebecca: I mean, it almost looks like that. [LAUGHTER]

John: Ok. [LAUGHTER] Well, that’s one way of getting rid of all the leaves that have been falling. And I have English breakfast tea today. So we’ve invited you here today to talk about some of the research you’ve done, and also your new role and your past role with the AEA’s Committee on Economic Education. One of the things that we’ve observed is that there’s some significant equity gaps in economics and in STEM disciplines in general in terms of race and gender. And in one of the studies that you’ve done with this, in a 2023 paper in the Southern Economic Journal, you and KimMarie McGoldrick examined retake behavior for students who are not successful in their initial attempt at completing an introductory microeconomics class. Could you give us just a general overview of this study?

Tisha: First, let me say that this started a long time ago. And we got access to this great data, the MIDFIELD dataset, that was actually originally funded by the NSF to study the gap in engineering education, but it’s student transcript data. So we said, well, you could use that for economics too. And we have. And so this particular study looked at almost 180,000 students who take Principles of Microeconomics for the first time. And what we ultimately wanted to do is to think about the likelihood of success, and actually more so, failure, because a lot of papers already look at success and they stop there, because you have too small of a sample, fortunately, of the failures to continue on to look at them. But when you start with 180,000, you have enough to continue. So we find, as many others do, that, of course, aptitude matters, but that, unfortunately, females tend to be much more likely to be unsuccessful in their Principles of Microeconomics class, as are underrepresented minorities, or URM. And then we follow those students and we say, okay, so if you were, in fact, unsuccessful, and we define that as a grade of a D or an F, so grades that wouldn’t really let you continue in your study of economics, or you withdraw. And we say, if you’re in that category, what do students tend to do?… or first of all, for those students that are unsuccessful, what helps predict that? So given that you are in the unsuccessful category, what is more likely to cause that? And we saw that students who were carrying more concurrent credit hours were more likely to withdraw. Which makes sense, because if you have constraints that lead you to want to be full time, if you start with 12 hours, and the course is not going well… any of them… and you were to drop, then you would not be full time anymore. And so it gives them sort of this flexibility, they have more concurrent credit hours. But we found that students who got D or Fs, given that they were unsuccessful, they tended to have taken more related courses. So things like accounting (financial accounting in particular), calculus, and macro principles. And so we didn’t actually see a lot of gender or racial differences there. But then those students who were unsuccessful (D, F, or W), on retake decisions, we found that women were much less likely to retake if they were unsuccessful. But we did find that women who were of higher aptitude, who were unsuccessful initially, they were more likely to retake. So that was a positive. And we did find that underrepresented minorities were more likely to retake the course, but then more likely to be unsuccessful again, on their second attempt. So there’s some good news, but also, a lot of not so good news.

John: So you were able to follow students over time, how long a time period was there in the data set that you were analyzing?

Tisha: We had over 20 year period of data, but that doesn’t mean we followed any particular student for that length of time. Once we saw that they were unsuccessful, we made sure that they had at least one more semester at the institution so they did in fact have a chance to choose to retake the course or not and then we gave them three years post their initial unsuccessful attempt. And if they didn’t retake it in that period, then we said they chose not to retake.

Rebecca: In a 2019 study in the Journal of Economic Education, you and KimMarie McGoldrick use the same dataset to examine the decision to switch into an out of the economic major. In this study, you find that very few students selected economics as a major when entering college and that 83% of economics majors actually selected the major after taking their first economics course. What did you find in terms of the gender and racial composition of those that selected the economic major?

Tisha: So what we found was that, for the few students who started out as economics majors, so going into their principles class they already had said they were majoring in economics, we found that females were as likely to persist or not. That’s good. And we found something similar for underrepresented minorities, that is the females and URM students who went into the principles class planning to major in economics, they were as likely to persist as not.

John: What happened to those students who were not majoring in economics.

Tisha: Well, unfortunately, females were not as likely to persist and decide to major in economics. There was a larger proportion of females who opted to remain in their original major. So, they come into the principles class, they have some major other than economics, they were much more likely to stay in their other major and not switch to economics. Men were considerably more likely to switch, on average, than females were. But on a more positive note, we did find that students from traditionally underrepresented racial and ethnic minorities and Asian students were more likely to switch to economics.

John: And one thing we should note, and you mentioned this in your study, is that because this datset was focused on engineering, the schools in this sample have relatively large engineering programs and some students switched to economics because economics was perceived as being easier than their original major. So some of this may not apply as broadly to liberal arts institutions that do not have engineering programs.

Tisha: That’s possibly true. And in fact, business majors were even more likely to switch to econ. So it was about 10% of those switching in were switching from engineering and 27% were switching from business. So at the liberal arts schools, to the extent that they don’t have business or engineering, there wouldn’t be that source of majors coming in.

Rebecca: Both of these studies are really interesting. But, how do we think about some strategies to create a more inclusive environment in economics and STEM classes? So what did these studies tell us? What should we be doing?

Tisha: Well, I don’t know that they tell us what we should be doing. They tell us that what we’re doing is not really working.

Rebecca: Fair.

Tisha: …which is unfortunate, because that’s what we’ve been doing for a really long time and we’ve been really unsuccessful at attracting women and underrepresented minorities. So there’s a new sort of strain in the literature that’s really trying to address some of this. And there’s a study, for example, by Porter and Serra, and that was published in 2020. They did a randomized control trial at Southern Methodist University, where they, just by happenstance, picked a couple of very charismatic female alums and they randomly selected which principles classes they would speak to. And this was just like a 15- or 20-minute exposure, where they talked about majoring in economics, and how that helped them in their career. And they found that this significantly increased the number of women from those courses who decided to major in economics and additionally, not necessarily just major, but take more economics courses. And I thought it was really fascinating that there was no effect on men in those treated classes, it was just the women. So that suggests possibly that there’s some room for role models. Other work that looks at role models from other directions, like some of my own work with KimMarie and John Siegfried, didn’t find any evidence that supports the idea of role models. So I think, still mixed, and the exact sort of interaction that students may or may not have with these people I think is going to be important.

John: You’ve done a number of studies on these types of issues, and one of them was looking at the effect of classroom experiments. Could you talk a little bit about what types of classroom experiments have you looked at and what’s the impact of those on students?

Tisha: Sure, I talk about that a lot, because I really love the classroom experiments. I think that they’re a great pedagogical technique. And so since your audience is more general, maybe I’ll explain just what we mean by classroom experiments. And that’s basically the idea that we’re going, in some cases, simulate markets or other decision environments for the students that will mirror the types of environments that we’re talking about in class. So for example, if we’re talking about a market, then we will have students participate in an experiment where some are buyers and some are sellers and they negotiate and trade. In my classes, we always do the experiment first, I don’t talk about any of the concepts. And then we collect the data as we’re going through the experiment on the decisions that they’re making: prices, quantities that are traded, then I’m able to talk to the students about “Well, look, this is what actually happened, this is your actual data that you generated, and now let’s talk about the theory.” And this is what the theory would predict the outcome to be. And they can see how well they match up. And they are able to discuss in class, this is why I chose to behave in this way. And they see how it fits in to the economic theory. And I did a study back in 2001, which was published in ‘04 in the Southern Economic Journal, where we were looking just at the effect of experiments and we looked to see if there was a differential gender effect. And in fact, there was. So you often see that women underperform men in these principles classes. But what the experiment did is it closed that gender performance gap. So it raised sort of everybody’s performance, but it raised the females disproportionately, so that they were closer to the same outcome as the men if they were in a classroom with experiments. So I do think that there is definitely room through pedagogy to improve women’s outcomes, and make them more attracted to economics, but also make economics more attractive to them. There’s a large literature that suggests that women are much more grade sensitive than are men, various people have gone about showing this in different ways. And if you can bring up their performance so that it’s on par with men. And it’s not really even that comparison, but you’re bringing up the females’ performance, then they’re getting better grades, and they’re going to be less likely to want to leave the study of economics.

John: And we’ll throw in a reference to an earlier podcast we did with Peter Arcidiacono, who had looked at that very impact in STEM fields in terms of the decisions of people to move from one major to another. And in particular, in that study, they found that women did better on many of the tests but still left because the grades were relatively lower than in other classes. So one issue I think might be worth addressing is the issue of grade differentials, that if we’re going to continue to assign grades that are lower than most other departments, there’s a good chance we’re going to lose majors in general, but disproportionately more female majors, based on what the research literature tells us.

Tisha: There’s this really cool study, McEwan, Rogers, and Weerapana, 2021. They’re at Wellesley, and so it’s all females, but they were trying to address grade inflation. And what that meant was, of course, there are some disciplines that have higher grades, they had to bring their average grades down, and economics is treated in the opposite direction, and that it actually drew more students into these harder grading disciplines, like economics and the other STEM fields. And so yeah, that’s certainly an issue. I don’t think we’re gonna across the board be trying to treat grade inflation. I don’t think this is published yet. But I was reading another study out of Wellesley, where they were allowing students… I think it was maybe for their gen ed classes… to take them all pass fail. And again, you saw students flowing into these harder grading disciplines.

John: I know a number of schools during a pandemic made all of their first-year classes pass fail, which is a good way of letting students get into a discipline without worrying so much about the grades so they can explore things without worrying about the impact on their GPAs. That might be another useful strategy.

Tisha: Yes, I agree, and then this is really anecdotal, but I’ve heard other people observe the same thing, that women, they’ll be in your principles class, and they’re doing well, but maybe they’re going to get a B plus. And they say, “Oh, either I’m not gonna take another economics class,” or I’ve even had students withdraw at that point, and I’m like, “but you’re getting a B plus, I don’t understand,” but they’re so sensitive to grades. And then there will be male students who are getting a C, barely, and they’ll say, “I’m going to major in economics.” and I say, “Okay, are you sure?” It’s hard to understand sometimes.

Rebecca: It’s interesting to think about withdrawal policies that have really kind of loosened up due to COVID. So our institution changed the policy more recently so that students can withdraw through the last day of classes without documentation. And so students who are doing really poorly might take that option, but as you’re discussing students with higher grades withdrawing, that sounds really concerning that policies like that could be kind of counterintuitive, or really have some other negative impacts. Classes all cost money, [LAUGHTER] and there’s financial impacts to withdrawing.

Tisha: It’s hard to really predict how this is all going to play out. And I have to say, I’ve never heard of such a generous withdrawal policy. At my previous institution, I think they could withdraw up to sort of two weeks out and not even have a W necessarily on their transcript. So this all seems like a lot. There’s a lot of moving parts. But on the positive side, so let me just say, first of all, part of me feels like it’s a little bit too generous. But part of me also thinks back to the work that KimMarie and I did. And if you don’t have that D or F, that you have to overcome, it’s a lot easier to persist, not just in your major, but towards graduation, whether you switch majors or stay. And part of me feels like that’s helpful to the student. So in some ways, I like a generous withdrawal policy, I definitely want students to know what the withdrawal policy is, what the deadlines are, so that they can make a good decision.

John: Yeah, the only concern I have with the policy is that it can lead to students making slow progress towards a degree and they could run out of funding before they complete their degrees. But on the other hand, it does provide a bit of a safety net for students who are adjusting. And as our student body has become more diverse, and we have more first-gen students, and we have more students coming from schools that did not prepare people particularly well for a college environment, it provides a safety net, which has allowed students to take some chances early on and not be harmed. And I’ve seen some students struggle in their first year or so, drop classes, and come back and be very successful, when before they might have been deterred from further study in the discipline.

Tisha: I agree, I can see that.

Rebecca: So, definitely one of the things that I’m always thinking about when students are talking about withdrawing is making sure that they know all of the ramifications. So what does it do to your GPA? What does it look like on a transcript? What are the financial implications? …so they can make a well informed decision. And depending on who they’re talking to, they are not necessarily always getting all of the information, and that can be problematic, too.

Tisha: Yes. So I think that points us to the issue of how important advising is, so that there are well informed advisors and that students have access to them, not just at the end when it’s time to register for next semester, but they have a relationship with their advisor, and feel that they can go in and ask those questions, because as John said, a lot of those students can’t go to their parents, they’re first gen, their parents don’t have advice for them. And even if your parents went to college, they don’t necessarily know all of the rules and regulations of the institution that you’re at. It’s really hard, I think, for students to have all the information that’s necessary.

John: I’ve had trouble keeping up to all the changing information [LAUGHTER] over the last few years. In terms of those requirements. You’ve done some work with cooperative learning. Could you talk a little bit about that?

Tisha: Sure. So that really came out of the fact that I had done a lot of efficacy work around classroom experiments. And KimMarie and I are really good friends. She is a cooperative learning expert. So I should say that she is the expert on CL. And she wanted to do an efficacy study, so we decided to team up and do that. So what we did is unlike a lot of some of the other work in efficacy, is that a lot of the work is comparing lecture. So you don’t do anything to this active learning technique. And with cooperative learning, the students are working on exercises. And when we talked about it, we said you know, it’s really not fair, and not even interesting to compare a student who is in a lecture-based class where they don’t get to see any of these problems to students in a cooperative learning class that are working on these problems in class in teams. So we thought a lot about the collaborative learning approach. And we’re trying to isolate what we thought could be the mechanism that might be driving different outcomes. And what we decided was we were going to compare the active team-based learning to individual learning, working on problems. So in our control, the students were exposed to the same problems, they had the same amount of time in class to work the problems as in the treatment, which was the collaborative learning. It’s just that in the treatment, they were working in teams in this think-pair-share share framework, and we didn’t find a significant difference. So at the end, we said that the cooperative component didn’t seem to be driving any difference. Although there are other people who’ve done work comparing collaborative learning to lecture based and they did find significant positive effects from cooperative learning.

John: One type of thing that I’ve done that seems to have been fairly successful is using clicker questions. Following the methodology of Eric Mazur, students are given a challenging question where typically half or so them will get it right the first time when they’re asked individually. But then they get to talk it over with the people around them, preferably someone with a different answer, they get to debate and argue it and I normally see between a 10 and 20 percentage point increase in the correct answer. And the really absurd answers tend to disappear down to about nothing. So that’s not a very formal study, but that second stage of that process where students are engaged in some peer instruction seems to have had a pretty significant impact. And I point that out to students, that when they talk to each other about it and explain it to each other, they do much better than when they’re working on their own. And I encourage them to try doing that outside of class, to work with other students, because there are a variety of studies… I haven’t seen too many in economics, but there are quite a few studies that find that that type of peer instruction, either in the classroom or outside, can be fairly effective.

Tisha: Yeah, so I’ve observed that too. I haven’t used clicker questions, but just going in and observing my colleagues when they teach, and they are using clicker questions. And I have to say, I’m stunned, because I don’t know how the right answer bubbles up as opposed to the person who had the wrong answer convincing the other of the wrong answer. I don’t know how that doesn’t happen. It’s like magic to me, almost. So part of me wants to think a bit more about what is the mechanism and how is that working? That’d be really cool to do a study on that.

Rebecca: One of the things that’s always curious to me about that particular dynamic is sometimes students discover, as they try to explain something to someone else, [LAUGHTER] that they really don’t know what they’re talking about. [LAUGHTER] They realize, huh, I can’t actually explain this concept, so maybe I’m the one that’s not correct. [LAUGHTER]

Tisha: Maybe that is what’s happening. Because everyone always says, “You have to really know your stuff to teach it,” and so if you are trying to explain it to your partner, you do have to know the material.

John: And when they’re raising objections and you don’t have a counter argument, that can help break down your misconceptions, because getting rid of those incorrect perceptions can be as important as trying to build new ones.

Tisha: Yeah, I could see that and maybe the people who are initially giving the wrong answe are just guessing. So maybe they’re not trying to explain and convince their partner of their answer, I guess. I don’t know. So maybe it’s not magic.

Rebecca: [LAUGHTER] Feels like magic.

John: It does. And it works.

Tisha: When I see it happen, it does seem like magic. In general, I think that active learning is helpful, because I think that women are going to be more interested when you show them the applications, and that when you have active learning, there’s some community building that happens. So you’re illustrating relevance, you’re building this community, giving them a sense of belonging in the classroom, which is a lot of what I think actually happens with the experiments. And then actually also with the experiments, they’re kind of discovering some of the theory for themselves, which is a growth mindset. And there is research that shows that if you can show relevance, belonging, and growth mindset, and develop these in your classroom, that students are going to do better, but that it also is more appealing to females and underrepresented minorities.

Rebecca: There’s lots of things for all of us to think about, even if we’re not necessarily in economics classrooms. Definitely good food for thought. Switching gears a little bit. Can you tell us a little bit about your work on the Committee on Economic Education?

Tisha: So I’m not officially on it at the moment, I’m the incoming chair. But I have served on it in the past. And I’ve been shadowing KimMarie McGoldrick, who is the current chair for the last year. So the committee is doing a lot of really great work. I think. When I first served on the committee, we basically just organized sessions for the American Economic Association meetings. And when I say “just” I don’t mean “just…” Compared to now, it’s just but we have, I think, seven sessions that we organize for the meetings every January. But while I was first on the committee, then chair Mike Watts, who was at Purdue, he said,”I have an idea. We should have a conference, so an all economic education conference,” and we started it while he was chair, and it’s CTREE, so the Conference on Teaching and Research in Economic Education. So the committee is in charge of that. It’s every end of May, beginning of June, depending how the calendar falls. So our next CTREE will be in Atlanta, not that far. It’s on the East Coast, maybe you can join us. And it will be May 29 through the 31st of 2024. We have many other things that we also are overseeing: an educate program, which is professional development for college faculty, and that includes two-year college faculty, in course design, different active learning pedagogies, and we talk about how to bring diversity and inclusion into the classroom. KimMarie actually started this this year, we have a newsletter called EconEdNews. And that features different pedagogies, it features the winner of the AEA Distinguished Economic Educator Award, which the committee also oversees, and some centers that we have for Econ Ed across the country… just a lot of information that’s in there, we have two issues a year. So we’ve had our first two, and our next one will come out in March of ‘24. So just a lot of great work, I think, that the committee is doing and I think really thanks to Mike who started us with our own conference. And then Sam Allgood succeeded Mike and added and then KimMarie is an overachiever. And she added a lot more in her six years as the chair.

John: And going back earlier, early surveys of economic instruction suggested that economists were using much more chalk and talk than many other disciplines were using. So these efforts have been fairly important in shifting people away from that. It’s a somewhat overdue change, but it’s nice to see that happening. And I would love to go to CTREE. But there’s two barriers that I’ve run into one is we run a series of workshops here, right around the same time, and the other is I teach at Duke in the summers. So I either am doing workshops here, or I am in North Carolina teaching econometrics now down there, so I haven’t been able to get there. I would love to go to a CTREE meeting.

Rebecca: I think I would have no clue what’s going on if I went. [LAUGHTER]

John: Oh I think you would.

Tisha: I think you would, I think you’d have a nice time. Everyone there is just so lovely and very engaged in teaching, and just really cares about the enterprise.

Rebecca: It’s nice to see education groups out of these professional organizations to really formulate communities of practice around teaching.

Tisha: Yes, I agree. I think economics, unfortunately, was late to the game. But we’re increasingly appreciating the importance and the importance not just of focusing on four-year colleges, but two-year colleges. So going back to the point of diversity, and the lack thereof in the discipline, the students in four-year colleges that I’ve worked with have predominantly still been very much what economists already looked like. And if we really want to improve the diversity in economics, I think we need to reach down to the high schools to the two-year colleges, and make economics more interesting and accessible.

Rebecca: We always wrap up by asking: “What’s next?”

Tisha: There’s lots of things. I guess, with regard to the committee, I’m hoping to just kind of not start anything new in my first term, and keep everything running as well as KimMarie has it now. But then for my own research, I am really interested now in looking at comparing pedagogies. So it’s been a lot of efficacy work looking at a particular pedagogy compared to sort of the standard chalk and talk. But now, how do they compare to each other? Is there a better pedagogy for students? And my first study, and I have the data, so I just have to start working on it will compare experiments to cooperative learning. And I told KimMarie, if experiments lose, then she won’t hear any more about this. I won’t write it up. I won’t talk about it. Because I don’t want to say that experiments lose, but no, I mean, seriously, that’s the next thing that’s first on the agenda. But also, I have a deep interest in diversity questions. KimMarie and I with Scott Simkins have a paper looking at HBCUs compared to PWIs and the study of economics there. And that was inspired by some work that showed that HBCUs contributed disproportionately to the STEM pipeline. And since economics is part of STEM, we thought, “Well, that should be true for economics as well.” So we did some work in that area. And we show that there are some positive contributions from HBCUs to the economic pipeline. But I think there’s a lot more to do there. So I would like to dig down in that. I’ve heard a lot of people at HBCUs talk about secret sauce, and I want to learn what that is. And not just at HBCUs but look at MSIs as well, more generally. And I do want to look at some questions at two-year colleges, so that’s for the next five to 10 years…

Rebecca: You’re going to be busy. [LAUGHTER]

John: For those who aren’t familiar with the terms PWI are predominantly white institutions and MSI are minority serving institutions.

Tisha: That’s correct.

John: Those terms are becoming more and more common, but just to make sure all of our listeners are familiar. Okay, well, thank you. It’s great talking to you, and it’s nice meeting you. And we will keep trying to get KimMarie McGoldrick on the podcast. I’ve been asking her for about four years. She’s been a little resistant, but we’ll see if we can get her here in the future. Well, thank you.

Tisha: Thank you so much. I hope you have a good afternoon.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

316. Help-Seeking Behavior

Continuing-generation college students are often better prepared by their family and peer networks for academic success than first-gen students with more limited support networks. In this episode, Elizabeth Canning and Makita White join us to discuss their research on differences in academic and non-academic help-seeking behaviors between first-gen and continuing generation students.

Makita is a graduate student in Washington State University’s Experimental Psychology Program. Elizabeth Canning is an Assistant Professor in the Psychology Department at WSU.

Show Notes

Transcript

John: Continuing-generation college students are often better prepared by their family and peer networks for academic success than first-gen students with more limited support networks. In this episode, we discuss differences in academic and non-academic help-seeking behaviors between first-gen and continuing generation students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Elizabeth Canning and Makita White. Makita is a graduate student in Washington State University’s Experimental Psychology Program. Elizabeth Canning is an Assistant Professor in the Psychology Department at WSU. Welcome, Makita and welcome back, Elizabeth.

Elizabeth: Thanks for having us.

Makita: Hello.

John: Our teas today are:… Elizabeth, are you drinking tea?

Elizabeth: I am drinking coffee this morning. It’s morning over here in the Pacific coast.

Rebecca: How about you Makita?

Makita: I have a hibiscus berry tea. I don’t usually drink tea, but I got some just for you guys.

Rebecca: Awesome. [LAUGHTER]

Rebecca: Sounds like a nice treat. I have some chai today. John?

John: And I have an English breakfast tea today, because I got a long band practice tonight. [LAUGHTER] So, a little more caffeine will help.

Rebecca: [LAUGHTER] So we invited you here today to discuss your recent study entitled “Examining Active Help-Seeking Behavior in First-Generation College Students,” which was published in Social Psychology of Education. Can you tell us a little bit about how this study came about?

Makita: Well, for me, this was actually my master’s thesis. When I became a researcher, part of my dream is to change the world, make a difference, and I’m really passionate about people getting access to the things that they need. So when I was in undergrad, and I was exposed to a lot of first-generation college students, and when I was hearing my parents talk about their experiences as first-generation college students, I started to notice that things were a little different when you didn’t have people around you who could tell you about what college was supposed to be like. And then when I started reading the literature on first-generation college students, and I saw how, in my opinion, excessively large, the gap was between first-generation and continuing-generation college students. That really captured my attention. So then, when I went to a school, where they have a really high population of first-generation college students, it felt really appropriate to look at first-generation college students. And also I’m honestly really interested in help-seeking behaviors. You probably have experienced yourselves where you see one person who is very persistent and active in getting someone’s attention, basically, very annoying, consistently waving their hand… [LAUGHTER] …trying to come up and get someone to basically give them whatever they need or they want, versus another person who maybe is a lot more quiet and sitting there hoping that someone will help them. And I wanted to know why. What makes one person act so differently from another? So I was really interested in first gens and help seeking. And then, at the same time, Elizabeth had recently been to a conference where they talked about first-gen forward initiatives, which is where colleges encourage faculty and instructors to self identify if they themselves were first-generation college students, to encourage other first-generation college students at the university to feel more comfortable, maybe talking to them or going to office hours, things like that. And we combined those two things together into this study, where we could look at help seeking in first-generation college students and a shared identity to try to see if that would change how help seeking looked.

John: And you mentioned some of the gaps that are observed. Could you talk a little bit about some of the equity gaps between first-generation students and continuing-generation students in terms of their academic performance and success?

Makita: Yeah, so for first-generation college students, we tend to see that on average, they earn lower grades, and they’re more likely to drop out of college. And they’re also less likely to engage in academic success behaviors, like going to office hours, or trying to talk to their instructors after class, things like that. And there are a lot of different reasons for that. They quite honestly don’t have someone at home who can teach them those implicit unspoken rules about college, about what’s expected of you and how you should behave. So they have to learn that on their own and that can take a little extra time, which is pretty valuable when you’re in college, that time. And then, a lot of the time, when you’re a first gen, you’re also coming from a lower income family, which may require you to work while you’re also going to taking classes and it means that you’re a little less likely to live on campus, which can influence you in all kinds of fun ways. But, we were, as I said before, really interested in whether or not there was a difference in the type of help-seeking strategies that students were using and how frequently they were help seeking. We wanted to see if that was maybe part of the reason for this gap.

Elizabeth: Yeah, I’ll just jump in too, that one of the big inspirations I think, for this work came from a sociologist, actually. Her name is Jessica Calarco. I think she’s at the University of Wisconsi- Madison now, but she wrote a fantastic book, it’s called Negotiating Opportunities. And she did a lot of field work with children, looking at at how children from lower- or upper-middle class families act differently in the classroom and how children approach their teachers or how they seek out resources and things. And she found that more low-income students are much more passive in their health seeking behaviors than upper-middle-class students. And so we had kind of read this work and thought it was really interesting and wanted to see if the same was true at the college level, and how that might look with those types of behaviors in a college setting. So we wanted to see if that gap that they found with children was the same for higher ed.

Rebecca: Can you talk a little bit about the methods of your study?

Makita: Yes, so this was a little complicated. But I’ll try to go through it in the most straightforward way I can.

Elizabeth: I will say for a first-year Master’s student, this was about the most difficult kind of laboratory experiment to design for, like right off the bat. But Makita did such a great job. And I think it turned out so well. But the complexities of it actually make it super interesting. So hopefully, we can explain it in a way that is easy to understand.

Makita: So we designed this study when COVID was in full swing, and we were in lockdown and, as a result, the entire study is set up to be conducted through Zoom. So the way it would work was a participant would join a Zoom room, where they would interact with one of our many undergraduate research assistants. And the research assistant would introduce themselves as a lead experimenter, and they would give the participant a phone number and an email, maybe like, “in case we’re disconnected or something goes wrong, you can reach out to me.” And then they’d give this participant 10 questions GRE style math test, and they would have 10 minutes to take it. And immediately after they finish taking this math test, the research assistant would have a short, partially scripted conversation with the participant, they would say things like, “Oh, don’t worry if you didn’t do that well. I didn’t do that well, the first time I took a GRE test, I didn’t even know I was supposed to take it until like right before I took it, I did really bad. But I did way better the second time I took it.” And then in the middle of that partially scripted blurb, they would, in one condition, say “I’m the first in my family to go to college, so nobody at home knew anything about graduate school.” And that was our main intervention. So for half of the students, they heard the research assistant was the first in their family to go to college, and for the other half, they didn’t. So immediately after that, they’re given another survey. And in that survey, they are told, “Hey, you’re going to take another math test after this one. And if you can improve your score by about 20%, or by about two questions, then you will be entered into a raffle to win a $20 gift card.” So we’re incentivizing them to want to do better. And then the survey says, “Do you want to go over your answers from exam one?” If you say yes, then the survey instructs the participant to reach out to their experimenter or this research assistant so they can do so; if the participant says no, then the survey instructs them to reach out to the research assistant to get the link to test number two. So in both cases, this participant has been instructed to reach out to the research assistant to get their attention so that they can move on to the next step. But in one case, they are getting academic help, they are going over their questions from exam one. And in the other case, they are just getting the link to exam two. The thing that makes this study fun is that the experimenter isn’t going to answer, the experimenter is going to leave their Zoom camera black, ostensibly off, while they are potentially off doing something else, or distracted, or maybe something’s gone wrong, and they are disconnected, who knows? And they’re going to ignore any attempt to get their attention for about eight minutes. And during that time, what they did was they recorded any attempt to get their attention. We were looking specifically for any additional behavior outside of the Zoom room, something active and persistent added on to that, like calling us or emailing us or texting us, using that information that was provided to them at the very beginning of the session. So after that eight minutes had past, the researcher would come back and say “Oh my gosh, I’m so sorry, something went wrong with my computer,” and then they would either help the student go over some answers or give them the link to the next test. And then the student eventually took the final survey with some demographics and final questions in it.

Elizabeth: So it’s a pretty lengthy paradigm. I think our research assistants had so much fun doing this study.

Makita: They really did.

Elizabeth: They had a lot of acting training to make it believable. But in terms of designing it, in a laboratory experiment, you have to kind of make some trade offs with making everything standardized, but also making it at least somewhat realistic to what might happen in the real world. So we were kind of playing off the idea of being an instructor and having a syllabus where you have lots of information about how to contact the instructor. And we have exams all the time, as instructors in our courses. And so it might be the case that they would need to go over the questions that they missed for the first exam before the second exam. And so we were trying to kind of mimic that type of setting in this one-hour laboratory study. But again, this is a sort of a different experience, where you’re not really getting graded, and it’s not going to affect your GPA. So we had to add a little bit in there around incentivizing them to want to do well in this sort of hypothetical situation. But again, I think our research assistants had a great time collecting these data.

John: What did you find in terms of the differences in help-seeking behavior between first-gen and continuing-generation students?

Makita: So we had a couple of different measures of help seeking in this study. The first measure was whether or not the student wanted to go over their answers. And we found that regardless of condition, so regardless of whether or not the experimenter self-identified as a first-generation college student, first gens overall sought less help than continuing-generation college students, which lined up with what we saw in previous studies. Things got a little more interesting when we started looking at the active help seeking behavior. So students who said, “Yes, I want to go over my answers,” we categorized as academic help seekers and students who said, “No, I don’t want to go over my answers,” we categorized as non-academic help seekers. Then, if students used some kind of additional method of help seeking during that eight-minute waiting period when the experimenter wasn’t responding, they were categorized as active help seekers. So we had active academic help seekers and active non-academic help seekers. And what we found was that students, our academic help seekers, weren’t really impacted by the identity of the experimenter, but our non-academic help seekers were. So in our control condition, when first-gen students were seeking non-academic help, about 13% of them used active help seeking, but in our intervention condition, it was more like 43%. So that was a really big jump, and it was really cool to see that. In other words, when first-generation college students had a help provider available, who was also a first-generation college student, they were more likely to reach out to them in this active persistent way on top of sending a message in the Zoom chat, they were also emailing or texting or calling. When the research assistant identified themselves as a first-generation college student that made our first-generation participants feel more comfortable with reaching out to them in this non-academic realm.

Rebecca: So when we think about this study, what are the implications as we talk to educators or higher ed leaders and actions we might take or ways that we might think about it?

Makita: It’s always difficult to try to generalize from a laboratory study to the field or to real life. In this case, because the person who was performing the role of the help provider was a peer, most things that we can generalize this too would also involve peers. So for instance, if we have a upperclassman teaching assistant for a really difficult math class, maybe if that person self-identifies as being first generation that might make first-generation college students in that class feel more comfortable with asking for help in regards to, say working Canvas or Blackboard or accessing their homework or figuring out how to get the right textbook, things like that. Based on these results, we wouldn’t expect it to necessarily help with their academic performance. But overall, engaging in this type of help over time, might.

Elizabeth: Yeah, I would just add that it kind of highlights two nuances of help seeking that we’ve kind of overlooked in the literature so far. There is a lot of evidence that suggests that first-generation college students seek help less often than continuing-generation students. But not a whole lot of people are talking about the types of help seeking. So, there’s passive type of help seeking versus active type of help seeking. So, differentiating that might be really helpful and understanding where we might need to intervene and help these students. And then what we’re seeing in this study is that we also might need to break it down into the type of assistance that the students are seeking. I think a lot of times, we’re just assuming that they’re seeking help for academic reasons, like I don’t understand this content, explain it in a different way, or related to the content of a class. And what we’re seeing here is that we might actually need to break this down into what’s related to the course content and what’s related to more of this sort of navigational type of awareness that first-generation students might not have the background knowledge to address. And so this non-academic help-seeking behavior might really benefit first-generation college students. And there’s a number of different scenarios in which that might be helpful. So applying for scholarships and financial aid, navigating course seeking and course maps, and figuring out the requirements for different degree programs, applying to graduate school, applying to different research opportunities. All of those things are academic-adjacent, but they’re not academic in the sense of the course material that they’re learning in that course. So it might be the case that all of these other types of help-seeking behaviors, it might be important to intervene in those areas to help first-generation college students.

Makita: And something really interesting about this study is that if we hadn’t separated help seeking into academic and non-academic, we wouldn’t see the difference that we found, if we had just examined it as basic help seeking without separating it, then the nuance of this situation would be lost. And as Elizabeth said, in many of the studies that we read, and that we looked at, they look at help seeking as this just basic block, they’re not separating it out into active or passive or academic or non-academic. And it seems like that actually might be really important because how well an intervention is working for a different type of help seeking might be something that we actually didn’t notice in some of these previous studies.

John: And there’s been a lot of studies recently indicating the importance of providing students with more structure, particularly first-gen students, which might help students get past that barrier. But there’s also been a lot of studies that have investigated the effect of a sense of belonging and comfort in the institution. And having that peer connection to someone else with a similar identity as a first-gen student can help break down some of those barriers and help them overcome that, so that they’ll be more comfortable seeking help in the future, I would suspect.

Elizabeth: Yeah, absolutely. I think anytime an instructor can talk about their personal experiences, overcoming challenges that they have gone through, I think first-generation identity is something that is not as visible as other types of identities. And so that might be something that we need to provide to students if we feel comfortable doing so and talking about that in a way that might relate to students and that belonging, right, making them feel like you can have the same types of success, the same types of career plans, the same types of goals in college, as other students, no matter what your background is.

Rebecca: I think it’s interesting for instructors to just think about taking the time to be explicit about that. It’s an identity that you might take for granted or not think about exposing, but it might be worth planning to expose that really early on in the semester to see how that might benefit students.

Elizabeth: Absolutely.

John: One thing we did on our campus last year was we had a committee that was looking at challenges for first-gen students. And one of the things they did was they created some images that could be used in your signature line indicating that you were a first-gen student. And they distributed that pretty widely. And a lot of faculty and staff members have included that to help form those types of connections. It sounds like, based on your research, can be fairly important.

Elizabeth: Yeah, I think it’s a great initiative. I think here on our campus, we have some stickers and some like door hangers that you can put on your office door and things like that. But I like the email signatures a lot, because that kind of gets blasted to everyone. But yeah, I don’t think it can hurt. I mean, it’s a pretty simple type of thing that you can do. There’s not any evidence that it would be negative for anyone, at least at this point. So it’s sort of like a no-cost way of helping potentially a few students along the way. So yeah, I think it’s a great practice.

Rebecca: Are there follow up studies that this is making you kind of itch to do?

Makita: Oh, yes, definitely. For me, the ideal next step would be to try it in a real classroom, have either an instructor, or a TA for a lab, self identify as first gen to half of the class and not self ID the other and then see if help seeking changes. It would be really, really cool if we could do another behavioral measure of help seeking instead of just self report, but it gets a little complicated when you try to figure out how to track whether or not someone is emailing the TA or the instructor without accidentally infringing on someone’s privacy. So we still have a lot to go when it comes to actually figuring out how to run a study like that. But that would be my ideal.

Elizabeth: Yeah, I think one of the things that we really need is a really good measure of help seeking, whether it’s self report, or whether it’s a way to assess that in some kind of behavioral data. Right now, there’s several different help-seeking skills out there where students respond to them. But they’re not as nuanced as what we’re seeing here around this academic versus not, this passive versus active. So a measure where we could really look at that nuance, I think, would help the field in general in terms of looking at whether interventions are increasing help seeking in various ways. And then of course, the behavioral measure is really, really interesting to us in terms of what students are actually doing. And we’re still kind of figuring out what that might look like, is that something that we can pull from, like their website data. So how often students are looking in the LMS for certain material, how often they’re clicking on things, whether or not they’re going to tutoring centers and office hours and things like that. So we’re still trying to figure out how to measure the behavioral side. But at the very least, we definitely need a really strong self-report measure of help seeking to move this research along.

John: We always end with the question, what’s next?

Makita: What’s next is I’m going to go back on campus and teach an introductory psychology class.

Elizabeth: Great. Next for me is I have meetings all day. [LAUGHTER], but I’m looking forward to the weekend. Actually, something that’s exciting for me is next week I’m going to a conference in Indiana that’s bringing together a bunch of educators looking to build up infrastructure for conducting research in education. So we’re going to be talking about barriers to doing types of different research in education and how we might solve those in the future. So I’m excited for that. I think it will be a great brainstorming opportunity to figure out how to make this type of research easier to conduct in different educational settings.

Rebecca: That sounds awesome. I hope that you’ll share back what you’ve learned and decided. [LAUGHTER]

Elizabeth: Yes, that’s the plan.

John: Well, thank you. It’s always great talking to you, Elizabeth, and it’s really nice meeting you Makita, and I hope we’ll be able to talk to both of you in the future.

Elizabeth: Great. Thank you so much for having us.

Makita: Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

315. Unessays

An unessay assignment provides students an opportunity to demonstrate their learning in innovative and creative ways. In this episode, Jessamyn Neuhaus and Maggie Schmuhl join us to discuss how they have employed unessay assignments in their courses.

Jessamyn is the Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at SUNY Plattsburgh. She is the author of Geeky Pedagogy: a Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers. She is the editor of Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. Maggie is an Associate Professor of Criminal Justice and the Associate Director of the Center for Excellence in Learning and Teaching at SUNY-Oswego.

Show Notes

  • Neuhaus, J. (2019). Geeky Pedagogy: A guide for intellectuals, introverts, and nerds who want to be effective teachers. Morgantown, WV, USA: West Virginia University Press.
  • Neuhaus, J. (2022). Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. West Virginia University Press.
  • Neuhaus, J. (editor) (2022). Using the Unessay to Teach History. Teaching History: A Journal of Methods. Vol. 47. No. 1.
  • Cavanagh, S. R. (2023). Mind over Monsters: Supporting Youth Mental Health with Compassionate Challenge. Beacon Press.
  • Jessamyn’s American Promise Unessay Assignment
  • Pittman, Chavella (2022). “Strategizing for Success: Women Faculty of Color Navigating Teaching Inequities in Higher Ed” in Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. Ed. by Jessamyn Neuhaus. West Virginia University Press.
  • Pittman, Chavella (2023). Navigating Teaching Inequities. Tea for Teaching Podcast. Episode 291. May 31.
  • POD Network Conference
  • Teaching, Engaging, and Thriving in Higher Education Series. University of Oklahoma Press.
  • American Historical Association Annual Meetings

Transcript

John: An unessay assignment provides students an opportunity to demonstrate their learning in innovative and creative ways. In this episode, we explore ways in which different faculty have employed unessay assignments in their courses

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Jessamyn Neuhaus and Maggie Schmuhl. Jessamyn is the Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at SUNY Plattsburgh. She is the author of Geeky Pedagogy: a Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers. She is the editor of Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. Maggie is an Associate Professor of Criminal Justice and the Associate Director of the Center for Excellence in Learning and Teaching at SUNY-Oswego. Welcome back, Jessamyn and Maggie.

Maggie: Hello.

Jessamyn: Hi. Thanks for having me.

Maggie: Good to be here.

John: Our teas today are:… Jessamyn, are you drinking any tea?

Jessamyn: I am. In preparation for this very question, I’m drinking a wild sweet orange.

Maggie: Oooh.

Rebecca: Sounds nice.

Maggie: That sounds very fall. I am drinking a bread pudding tea. It’s a black tea from a little tea shop in Sacketts Harbor.

Rebecca: That’s a first on our podcast for sure.

Maggie: I think it’s called Tea Thyme, the little shop. It’s quite lovely.

Rebecca: Nice. I have Irish breakfast this morning. So we’ve invited you both here today to discuss unessay assignments. Could you tell us a bit about what an essay is?

Maggie: I guess I’ll say an unessay is anything but an essay. Jessamyn probably has some more nuance? [LAUGHTER]

Jessamyn: Not really, that is the basic premise. In classes where you might traditionally ask students to show their learning via a standard essay, either like the five-paragraph essay or a research paper, an unessay project asks students to demonstrate their learning, increased understanding, via any other kind of format.

Rebecca: Jessamyn, can you tell us a little bit about why you decided to use unessays in your class?

Jessamyn: Sure. And this also relates to… I know, we were maybe going to talk about what was appealing for historians, in particular, about the unessay. And some points for history professors generally and me, what attracted us to considering using the unessay. A couple of things about our discipline: first, that the unessay, it’s a good way for students to explore some key historical thinking skills in different new ways for them, so things like citation, research, and historiography. So trying to understand what other historians have written and argued about the past. There’s also been a pretty significant sort of public history turn in the world at large, like trying to better understand different audiences for historical research. Some other more general reasons were issues of equity and inclusion, especially around Universal Design for Learning, trying to offer students some options as a way to increase their learning, increase engagement. And the final thing I’ll say, just for me personally, is that I wanted to make assessment and grading a more joyful and happy part of my teaching life. It had really started to drag me down. And I didn’t want to approach assessing student learning as a soul sucking slog, not just for personal reasons, but also that it was interfering with my ability to accurately assess. If I was absolutely dreading reading a stack of essays, then something was amiss, and I wasn’t going to be able to, in my view, really accurately assess what has this student achieved.

Rebecca: About you, Maggie?

Maggie: I think really a lot of what Jessamyn said echoes in the reasons I decided to incorporate the unessay in my classroom. I was actually inspired by a talk that Jessamyn gave to our faculty, I guess it was over a year ago now, about the unessay and I immediately thought of my senior seminar class. So in criminal justice, in our department particularly, it was common for students to write a 20- or 30-page long research paper, and they dreaded it. It was a source of anxiety for students. It was something in the end that I’m not entirely certain they felt proud of. They just were trying really hard to get the A in the class, to make it perfect, to spend lots of late nights trying to perfect this project and I don’t think it brought about the kind of learning gains that I was hoping to see with my students, it seemed like there was a lot of going through the motions. And the unessay just gave us an opportunity to really turn that on its head and give some of that autonomy back to students and some of that creativity back to learning.

Rebecca: I’m loving these themes of joy, creativity…

John: …which are not terms we often associate with either writing papers as students or with grading them as faculty, and Jessamyn, you were the editor of a fall 2022 issue of Teaching History that focused on unessays. Could you tell us a little bit about ways in which you and other historians have been using them in your classes… some examples of ways in which students have demonstrated their knowledge?

Jessamyn: Sure, and I’ll just give a little promo here, Teaching History: A Journal of Methods is an open access journal. So you can just Google Teaching History: A Journal of Methods, and it’ll take you to the open access platform. It’s hosted by Ball State University. And so that issue is still totally available for anyone interested in reading it. I would say, like some specific projects that were explored in that issue, there was an article about using maps as historical documents, there were a number of projects that were exploring how to present history and historical thinking in a public history setting. So along the lines of museum exhibits, and especially in a digital format, the upsurge in digital humanities is something several of the authors were looking at. And then there’s a whole range of examples that were discussed in the two interviews I did for everything from board games, to graphic novels, to quilt, to the annotated recipes. All those were discussed in that issue.

Rebecca: What are some specific projects, Jessamyn, that you’ve done in your classes,

Jessamyn: A couple that have come up that I used… it’s interesting… Maggie used the unessay with her senior students. And I have helped students in the history department here, they also have to do a major research project. And I’ve been the advisor on a number of non-traditional projects is what I called them at the time, and now would say unessay. So at the senior levels, students did major research projects, like a fictional interview with a historical figure, historical fiction would be another example. I’ve had a couple of students do online exhibits and one person do an in-person exhibit. But I also use the unessay assignment in my intro level class, which is a basic history survey class. And those projects ranged from again, the board game, I had a number of posters, I had one student who was a trained professional baker do a set of cookies with icing, comparing different protest movements, the Bonus Army March in the 1930s with the more recent climate change activism. And one of my very favorites was a video recording that a student did, showing step by step how she made a family recipe. And since the theme of the class was the American Promise, and the way that throughout our history, as a nation, we’ve struggled to achieve the ideals of equality and liberty for all. And we get closer and we fall short. And her recipe was from her family’s background from India. And she took us step by step through it. She had music playing and was describing the importance of this particular dish in her own family history. The unessay was encouraging them to see their location in the American Promise. And it really struck me, in my previous life as a historian of food and food waste. So that was interesting. But most striking to me was, this was a student who had been very, very quiet in class. I felt like I didn’t know her very well. She was doing great work, but she was sort of on the edge of things it seemed like, and with this unessay, it completely changed my perception of her. I felt like I got to know her so much better and so did the other students as well. Having students share their unessays is an important part of the work and It really changed just the whole classroom dynamic in that way. So that was really notable.

Maggie: Nice.

John: Maggie, what are some of the projects that your students have done as unessay assignments?

Maggie: Yeah, so I mentioned before that I assigned the unessay for my senior seminar class. So it’s a sort of capstone project. And there were some really awesome projects that came out of this and I probably don’t have time to talk about all of them. So cut me off at some point when I’ve been going on for too long. But I had a student who wanted to be a elementary school teacher, and had gone through our program and decided that being a practitioner in the field of criminal justice was not exactly where they wanted to be. And so his project was to create lesson plans for second graders to communicate some of the harms of incarceration and how it affects youth when their parents or caregivers are incarcerated. And so it was a really wonderful project, all the students had worksheets, kind of like you can think about having when you were in second grade, and little matching puzzles and games, and it was a really cool experience. And for him getting interested in a master’s of education program, it was a way that he could then take that project and show to his grad school applications and advisors, like, “Hey, this is something that I’m really passionate about. I’ve been working towards this. And yeah, it was a lot of fun.” We had a good time, even some spelling words thrown in there. [LAUGHTER] It was great. Another student project was a choose your own adventure story. There’s various examples of this. But in my youth, it was the Goosebumps choose your own adventure where you read to a certain part in the story, and then it gives you the option of what you want your protagonist to choose. And then you have to flip to a different area of the book and see the fate of that character. And so the students started a really great project, unfortunately, didn’t get to finish it for various reasons, but started a really great project about reentry. And so the main character was someone who was being released from prison and had to navigate all of the barriers and circumstances around reentry that made it difficult for him to assimilate back into society. So that was a big fan favorite amongst the class. They really enjoyed that. Of course, we also had some board games, one in particular, was a monopoly game that examines varying socioeconomic status, and how that influences someone’s path through the criminal justice system. So the players started off with different amounts of money. And as they would pull community cards and have to purchase property and such, they would be challenged to need to pay a particular court fine, and how that would set them back from being able to find a job. It was a very well thought out game of Monopoly. But the goal to show and highlight the inequalities related to income and how that affects criminal justice processing. I could probably keep going, but it really gave a lot of opportunities for students to combine what they were interested in. There was a student who had run their own podcast for a period of time. So she conducted a podcast series on Amber Alerts and their effectiveness. And it was something that when she had been traveling down to Texas, and had received a number of Amber Alerts in the short time that she was there, she was like, “Are these things really working? And how can I explore that through a medium that she was already quite drawn to?”

Rebecca: Unessays always sound like so much fun?

Maggie: Yeah. And they’re fun for us. I know Jessamyn touched on that, too. It’s a whole different experience as someone assessing the work that they’re doing when you know that they’re passionate about it. And it gets me excited about assessment, which I don’t think is something we can typically say.

Rebecca: There’s not a big line of faculty raising flags saying assessment is fun.

Maggie: Right. [LAUGHTER]

Jessamyn: I also wanted to follow up on a point Maggie’s touched on a couple times about student autonomy and agency. I think for both our student populations, this is a really key academic, professional, and personal skill that many of our students need to build, that they arrive to college with a sense of themselves as very passive recipients of education and have not had the opportunity to really see themselves as producers of knowledge. as having agency in academic settings. So for me, one of the great glories of the unessay, is that it’s starting point really is, what do you want to do? What do you want to do to demonstrate that you understand this topic, and for a lot of students, it’s scary at first. A few might think that you’re trying to trick them. And I know this was something we did want to talk about. It does require, on our part, a fair amount of structuring and transparency and communication with students, maybe even especially the highest achieving students. One of my favorite students of all time went on to become a high school social studies teacher. And she did a fantastic job in the intro class with the unessay and then told me like a year later, “You know, I hated that assignment when I did it.” And it was because she was very, very nervous. She was a straight A student who was really nervous about doing something different. But she said, “Now, it’s my favorite, looking back.” So providing that structure and support and transparency, a lot of careful thought on the front end.

John: And one of the things we’ve been talking about in one of our reading groups, Mind over Monsters this semester, at both of our institutions, is the importance of autonomy, in terms of increasing student motivation and giving them more sense of control. It sounds like it’s really helpful in motivating students.

Jessamyn: Yeah, I think so. In both of our disciplines, criminal justice and history, there is a major learning curve to helping students see themselves as having agency in this particular discipline, especially the way they may have interacted with it before. And for me, coming from a lot of their previous experiences with history has been learning what happened and not actually doing history or thinking about those processes and the unessay could really help them move towards that particular desirable learning outcome.

Rebecca: I think one of the challenges that faculty who have been engaged in unessays before see before they enter in is the complexity of having students doing many different kinds of things at one time and managing those projects and also assessing those projects. Can you talk a little bit about how you help provide that structure for yourselves and for students?

Maggie: Yeah, so for my criminal justice class, I spent a lot of time the summer before I implemented this project, creating very detailed instructions and expectations. Jessamyn, I went to your website, and I use some of the self assessment tools, and I adapted them for my class as well. It was a very mindful process to think about how to give students the structure they need to feel confident in their ability to carry out this project, but also give them enough flexibility and creativity to take this project where they want it to be. I also spend a lot of time reading about different alternative grading practices to also incorporate in this, so I use kind of a combination of ungrading and specifications grading as well to help students assess their work. And I met with students quite often throughout the semester, particularly we had a midterm and a final exam meeting where we talked about the strengths of their projects, where they needed to go and what areas they needed to work on, whether it was engaging more deeply in the research and how to communicate that effectively in their projects. It was a lot of work for the faculty, but it was worth it in my opinion. It wasn’t so onerous and it wasn’t boring. It was exciting to meet with students, even if it meant spending my entire midterm week in the office with the students. It was almost every minute of my day for an entire week doing this, but it was, I think, a unique opportunity for making those connections with my students.

Jessamyn: I have been lucky. I have a relatively light teaching load and small classes. So that did make a difference in how I thought about structuring and implementing the unessay. There would be challenges on a larger scale. I would also say that there’s components of the unessay, that it’s not just do anything you want, and then nothing, and then in the end they turn in something. No, there’s stages and components like usually a proposal process and I also was able to meet with students. You could meet with students in small groups as well, if it’s a larger size class to go over the proposals, and many unessay assignments also do have a written component like a bibliography, an intro. At the midterm and end there could be a reflective assignment or sort of summary of work achieved. I did co-create with students the assessment guidelines for their projects. So taking the autonomy and agency a step further, I didn’t leave it totally up to them, we worked together on it, but asked them, “So this is your idea, this is the project you have in mind. How will it show these important skills? How will it show that you’ve met these course learning objectives?” And we work together on it. And that part is always great for me, I just really enjoy trying to de-center myself a little bit as the instructor which again, is something I can do from a place of certain privileges. I have tenure, I’m white, I’m cisgendered. But trying to enlist students as partners more, decentering myself a little bit, and asking them to reflect on what would show that you have achieved this learning outcome with this unessay assignment. A nice bonus is always students, at least one student, will say this is so hard, teaching must be really, really hard. [LAUGHTER] So bringing them in on that assessment process can help them even reflect on what does grading and assessment mean? What would show me that you’ve successfully achieved this learning, this desirable outcome?

John: Do either of you use rubrics for the unessay assignments?

Jessamyn: Well, when I co-created assessment guidelines with students, some of them used a rubric form. A lot of my students found rubrics helpful for things like deadlines. And I did set deadlines for certain benchmarks. And that was something that was easy to map out and help them structure their work time and stay on track as well.

Maggie: Yeah, I created a checklist with that self assessment that was on Jessamyn’s website, geekypedagogy.com. I did have students go through that and rate themselves, and then we got together in a meeting to discuss where we felt the accuracy of each of those points. But I think it gave them some comfort in saying, “Okay, I know I did multiple revisions of this, and I incorporated all of the feedback I was given. And here’s the changes I’ve made, or here’s the areas that I think I probably could have done better, or I could have focused my time a little bit better in this way.” And it just gave them, I think, a nice guideline to be able to talk about the strengths and areas of improvement for their project.

Jessamyn: I’ve used rubrics before, when I was co-creating the assessments. That’s fine with me, if that’s what they want to use. I will say that this part of it, while being very cognizant, like I said, of my own privileges and positionality and aware of that, your teaching context can really shape what kinds of assessments you can implement, for sure. But I also think that it’s at least worth considering the idea of our academic egos and our investment in our discipline and expertise. And what the unessay may challenge us to do is let go of the thing we know how to do absolutely better than anything in the world. So as a historian, a trained historian, I know how to assess student writing better than any other task I do as a teacher. I know so much about it. And it’s easy for me to do, not joyful, [LAUGHTER] but not difficult. It doesn’t push me to think about anything outside my comfort zone. And when I assign an unessay, I have to let go of that and think about things differently. And again, keeping in mind that I know bandwidth is limited, the incredible pressure on a lot of people in the pandemic pivot and for all various reasons, our limited energy, it might be too big an ask, too big a lift. But I would encourage listeners, if it’s possible, to consider maybe it’s not about me. [LAUGHTER] It’s not about us, and what makes us most comfortable as the person with the PhD in the room. That being a little uncertain. how can I assess did students meet this learning outcome with a quilt, with a board game, with a podcast, that it’s okay for us to feel a little bit discomforted. That’s about maybe getting out of the students way a little bit.

Rebecca: When I hear folks talk about unessays, both of you as well as others that I’ve heard talk about unessays, I see a big emphasis on process and the behaviors of a discipline, rather than necessarily the outcomes of something. So often in a research paper, we might assess on some of the polish of something like that, rather than really seeing always the process that goes into the thinking. But I feel like a lot of times with these projects, you end up knowing a lot more about a student’s thinking, because you’re talking with them about their projects in an ongoing way. Can you talk a little bit about that process piece with unessays?

Jessamyn: Well, one thing that jumped into my mind, Rebecca, when you were asking your question is writing a thesis statement for a research paper. Now, this happens to be something I’m incredibly good at helping students do, crafting their thesis statement. But, just like you described, it’s often so much about the academic writing skill that I’m helping them with, and when they have to write a thesis statement for the research paper they’re working on, that’s a great skill to have, if you’re going to be an academic. If you are engaging in scholarly writing, you have to know how to do that. And for a few students, that’s important. And that’s not like written communication skill. I’m talking about the highly academic scholarly skill of summarizing the argument you’re about to make in a paper. So for students who want to go on to be graduate students, I’m all in. We’re going to sit and we’re gonna go through this word by ever loving word to figure out the best way to state your argument. For a lot of my students, though, is that the best investment of their time, and the best way I can help them? There might be other ways to, just like you said, have them articulate and demonstrate their thinking, which, and I’ll just throw it out here in the age of generative AI and ChatGPT, this is at the heart of how we help students navigate is their original ideas really matter, their ideas, and their thinking matters. So if I can support a student being able to demonstrate their thinking and ideas in a way that matters to them, that they feel articulate doing, then, mission accomplished. That’s what I want. That’s what I want for students.

Maggie: Yeah, and I would say that, in my class, the 20- to 30-page research paper does serve students who are going to go on to graduate school. But like said, I don’t know if that is the best way to help them make the connection between what we’re doing in the classroom to what they are going to be doing on a regular basis in their various careers that they find themselves in. I wanted to make this project less about a means to an end for them, I wanted them to really think about how they start with an idea and how if they spend enough time thinking about that idea, and really focusing on how to communicate that expertise that they gained throughout the research on this idea, that they can create something that they’re proud of and they are hopefully less focused on the grade at the end of the semester and more focused on “Okay, I was actually able to start in one place, develop that project, develop this in a way that I can now go to my future employer or in an interview and say that I’ve developed these skills along the way that are mine and mine alone. They’re not ones that have been imposed on me by an instructor.”

John: We have increasingly diverse student bodies in terms of their prior preparation. Some have come through and had lots of practice writing throughout their high school careers and have developed those skills, others haven’t had as much preparation in writing. And we have lots of courses, though, that give them that but writing skills are not always one of the most important learning objectives for many of our classes. And it seems to me that perhaps using unessays can help provide a more equal playing field for students who may not have as strong of writing skills, but might be fairly comfortable creating videos or might be able to engage in some other creative activity that can demonstrate what they learned much more effectively than if they were constrained to having to do it in a written format.

Maggie: Well, I would say that was my experience for sure. I had a group of students make a series of TikTok videos and so they were quite excited about their ability to use something that they are on almost every day, if not every hour of their day, to use that as a platform to communicate the time topics that they were most interested in. And so it did mean that I had to learn a little bit about TikTok, I had to create a profile so I could follow my students and be able to see the videos that they had created. There’s an up voting system, too, I had to navigate [LAUGHTER] but yeah, in terms of equity, it does give students the ability to make decisions about how to present information in ways that they’re comfortable, but still challenging. I think that no matter what mediums students picked, it was challenging regardless. So it didn’t feel like students were just picking something that “Oh, well, I do this all the time.” It really was a strong motivator. And I think in a lot of the conversations we’ve been having in our reading groups about how to channel anxiety about something into a more direct way to accomplish a challenging task. I think it gave even students who are perhaps underprepared for academic writing, or find a lot of anxiety around that to shift their focus away from what they envision that they struggle with to then start in a place of, “Oh, I know I can do this, and I can communicate it in this way.” So I think it gave them some of that confidence back, that perhaps they find disappears when we impose a pretty lengthy paper or academic medium on them.

Jessamyn: There’s two aspects of the unessay and equity minded teaching I’ve been thinking of. One is the universal design for learning concept of providing some choice. It doesn’t have to be unlimited, it doesn’t have to be infinite choice. But providing some choice does increase accessibility. The second kind of key equity minded principle I have in mind with the unessay is that we should not be grading and assessing people’s life circumstances, we are grading and assessing their learning and their ability to show that they have reached certain benchmarks and outcomes. Just to put it in here as a caveat, I do think all kinds of writing is very, very important. Writing skills are very, very important. And a lot of the unessay assignment as I use it does include some clear written communication, because that is a skill we need in this world of ours. So the equity piece comes in though, that if I’m going to be assigning a grade assessing student writing, then it’s on me as the instructor to teach that and create the conditions in the classroom, the practice and the feedback, and more practice and making mistakes, and more feedback to do that writing piece. And if I’m taking into account the wide variety of academic backgrounds, what needs to happen in the course, for students to be able to demonstrate their learning accurately via a written essay, that’s a lot of writing work that I’m going to be doing to provide that equitable classroom. So the unessay as an option, that’s something a lot of people do offer the unessay as an option alongside a traditional essay option, the unessay as an option could increase equity in that way. I think it does convey the idea that our diversity is an academic asset. That all said, it’s interesting that the unessay could also exacerbate inequities without structure, without support. Students who have had very rich and varied past educational experiences may leap at the opportunity and see it just solely in terms of its benefits. Other students with less of that in their background might be extremely nervous about it as a whole new frontier for them. So without some structure, transparency, without all that prep work, Maggie, that you were describing, it could actually exacerbate some of those inequities.

John: And I think the scaffolding that you both described is really important in relieving some of that uncertainty, because I know some students when I’ve given similar types of assignments are really anxious about it and letting them do it a little step at a time and master that step seems to help relieve the pressure on them by just reducing the uncertainty.

Maggie: I know one of the things I incorporated in the project, and I think Jessamyn did this as well, that I gave students unlimited rewrites on any of the written work. So I did have a bibliography and I did have a proposal and a sort of artist’s statement reflection at the end of it. I think, again, I took that from geerkypedagogy[.com] and I gave students the opportunity to just keep rewriting and rewriting. And yes, it was a bit more work on my end. But it was so refreshing that students were actually taking my feedback into their work and they were making improvements. I think that whenever I’ve done drafts in the past, they might go through and make just the easy corrections and leave some of the harder things aside. But in this project, because we started a lot of those elements very early in the semester, we spend weeks and weeks just revising. And it really did give students, I think, a opportunity to make sure they fully were understanding and achieving those outcomes.

Jessamyn: Yeah, that point Maggie about your feedback actually being read and applied is so important for students and for faculty. It’s discouraging and disheartening to provide students a ton of feedback and then feel like it hasn’t been read. It’s up to us, though, to provide the opportunity for students to apply the feedback. If it can’t be applied, it’s very difficult to take it seriously [LAUGHTER] and to see it as meaningful and relevant. And I think another aspect of what you are describing that listeners might be interested in… if we haven’t totally convinced you yet [LAUGHTER] to use the unessay… is that it does get students’ attention, it cuts through the static. We get into a routine we’re just trying to get through, check that box for students who are overworked, overloaded, for whatever reason. I really feel like it kind of cuts through a fog that many of us, students and faculty, are in just by asking us to do something a little bit different.

Rebecca: Sometimes in those cases, and you guys have alluded to this before, that doing something different if they’ve not done it before, is risky and scary. And so all that scaffolding that you’ve built described as so important for students to be able to take that first risk and maybe take other academic risks in the future.

Jessamyn: Yeah, we can’t underestimate the impact of our traditional grading systems and traditional school systems where the stakes are sky high. You can’t overestimate them, financial aid, everything, is so tied to a student’s GPA, their families support, staying in school is often tied to GPA, their own sense of their selves as students, who they are, is defined by the grade they get. So asking them to, Rebecca, like you said, consider the process, not the end point. It is a big ask. So we can’t expect instant student buy-in, that they will immediately jump on board and with ease and joy, even if we are feeling that way about it.

John: So it sounds like it’s been a lot more fun, even though it may sometimes take more work for both of you. How have students responded? You’ve mentioned a little bit about student responses. But how do students generally respond at the end of the semester? Is it something that they find to be equally joyful?

Maggie: Jessamyn touched on this, but there was a bit of student buy-in that I had to get. And there were a couple of students who still wanted to write a traditional research paper. But in the end, they did a podcast element as well with it. So it was the sort of combination of the two, but it was the first time I had ever implemented an unessay in my classes. So I didn’t have a lot of student examples to show them. But I did spend a lot of time creating my own examples of what an unessay would look like. And I think for each of those scaffolded assignments I created a bibliography and a presentation and I tried to create an unessay myself so that they could see what it would look like. I think that gave them a little more confidence in knowing what was expected of them. But really, it took a lot of conversations with students on why I was doing this, what I thought the importance of shifting away from a traditional research paper, what gains they were going to have from that, and every student, at least to my knowledge, really enjoyed the process and much preferred the unessay over their 20- to 30-page research papers.

Jessamyn: The student resistance you just mentioned, Maggie, reminds me that I mentioned teaching context before, but it is important to underscore that any non-traditional teaching and learning practice and assessment, you can expect greater student resistance or reluctance when you quote unquote don’t look like a professor. So when you don’t fulfill the stereotype of what a professor looks like, if you’re navigating any kinds of intersectional biases, you may have to take into account the student resistance component and student biases. It’s a point that Chavella Pittman has emphasized for like an ungraded practice, that it’s very important to take those considerations into account. I will add that the biggest benefit besides my own grading joy is what I see… it’s extremely hard to measure… I couldn’t put it on my end-of-the-year file assessments of my own teaching, but students gain confidence, students gain some agency, some autonomy. I can see it in how they talk about it. If you can have a showcase or a gallery walk, you can see that they are proud of what they have achieved. When we’re doing senior presentations, that’s always very clear, like students’ sense of accomplishment is very notable. And it’s not that that doesn’t happen with the research papers. Well, it can, but the range of students who express and demonstrate to me that they’ve gained some confidence in themselves as students, as thinkers, as knowledge generators, as producers, it’s really heartening.

Rebecca: So we always wrap up by asking what’s next? Jessamyn, you want to go first?

Jessamyn: I’m headed to the POD Network Conference. I’m gonna do a session, with Chavella Pittman and Thomas Tobin, on faculty development that acknowledges teaching inequities. And I am working on a book on teaching and learning when things go wrong in the college classroom, and I just recently signed a contract for that book with the new series at University of Oklahoma press called Teaching, Engaging, and Thriving in Higher Education.

John: Maggie and I will both be at POD and will try to get to your session. And we’re looking forward to that book.

Maggie: I suppose what’s next for me feels like such a loaded question right now because I’m anticipating applying for a sabbatical. And so I’ve been really ruminating on this question of what is next for me. I’m entering into a mid-career stage, and so I feel like my answer is “I’m not quite sure yet.” But as far as the unessay goes, I do plan on bringing the unessay to my courses next semester. I’m teaching two elective courses on death penalty, and women and crime. And I think really any course, but particularly these courses, would be well suited for an unessay project.

Jessamyn: That reminds me, I am lucky enough, I’m going to be going to the American Historical Association Conference this spring and doing a workshop on the unessay, so any historians listening, I hope I see you there.

John: And we will include links to Teaching History, to that specific volume, as well as to the other resources that have been mentioned here. So check the show notes to find more of those resources. And thank you both for joining us again.

Maggie: Thank you so much.

Jessamyn: Thank you.

Rebecca: It’s always wonderful to talk to both of you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

314. Handbook of Online Higher Ed

Since its start in the late 1990s, asynchronous online instruction has spread throughout the world and has been the subject of extensive experimentation and study. In this episode, Safary Wa-Mbaleka, Kelvin Thompson, and Leni Casimiro join us to discuss their new handbook that examines effective practices in online learning from a global perspective.

Safary is an Associate Professor of Leadership in Higher Education at Bethel University in St. Paul, Minnesota. He has authored and co-authored more than 40 scholarly journal articles and more than 20 books and book chapters. Kelvin is the Vice Provost for Online Strategy and Teaching Innovation at the University of Louisville. Kelvin developed the BlendKit Course open courseware as part of the Blended Learning Toolkit, and he co-hosts TOPcast: The Teaching Online Podcast. Leni is a Professor of Education, the Associate Dean of the AIIAS Graduate School and Chair of its Education Department and the Director of AIIAS Online, the virtual campus of the Adventist International Institute of Advanced Studies (AIIAS) in the Philippines. Kelvin, and Leni are frequent invited speakers on topics related to online instruction. They are the co-editors of The Sage Handbook of Online Higher Education.

Show Notes

Transcript

John: Since its start in the late 1990s, asynchronous online instruction has spread throughout the world and has been the subject of extensive experimentation and study. In this episode, we discuss a new handbook that examines effective practices in online learning from a global perspective.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Safary Wa-Mbaleka, Kelvin Thompson, and Leni Casimiro. Safary is an Associate Professor of Leadership in Higher Education at Bethel University in St. Paul, Minnesota. He has authored and co-authored more than 40 scholarly journal articles and more than 20 books and book chapters. Kelvin is the Vice Provost for Online Strategy and Teaching Innovation at the University of Louisville. Kelvin developed the BlendKit Course open courseware as part of the Blended Learning Toolkit, and he co-hosts TOPcast: The Teaching Online Podcast. Leni is a Professor of Education, the Associate Dean of the AIIAS Graduate School and Chair of its Education Department and the Director of AIIAS Online, the virtual campus of the Adventist International Institute of Advanced Studies (AIIAS) in the Philippines. Kelvin, and Leni are frequent invited speakers on topics related to online instruction. They are the co-editors of The Sage Handbook of Online Higher Education, which we’ll be talking about today. Welcome Safary and Leni and welcome back, Kelvin.

Safary: Thank you.

Leni: Thank you.

Kelvin: Good to be here.

Safary: A pleasure to be here.

Rebecca: Today’s teas are? Safary, are you drinking tea?

Safary: I’m having water this morning.

Rebecca: A key ingredient to tea it might add. [LAUGHTER]

Rebecca: How about you, Leni?

Leni: I used green tea, particularly this Japanese matcha. [LAUGHTER]

Rebecca: Nice. How about you, Kelvin?

Kelvin: I have deconstructed tea. That’s also called water.

Rebecca: [LAUGHTER] Popular globally.

John: And speaking of globally, Rebecca and I are both drinking Moon Bird tea, which is a gift from one of our listeners in France who sent this to us a few weeks ago. So again…

Safary: Wow.

John: …thank you, Myriam.

Rebecca: Yeah, it has a nice hint of pear and elderflower.

John: …which is also a green tea.

Rebecca: Yeah.

John: We’ve invited you here today to discuss the SAGE Handbook of Online Higher Education. Could you tell us a bit about the origin of this book project?

Safary: The origin of this project is actually something that has to do with me having worked with Kelvin several years ago at the University of Central Florida. And right after that, I decided to work in the Philippines and that’s where I met with Leni Casimiro and we worked together. And at both institutions, we were working with online education. And eventually I was transferred to work for two years in Kenya. During the COVID-19. I happened to be in Kenya, and I quickly saw the great need of people wanting to have online education. The resources went up in the place. The things were scattered all over the place. And immediately the idea came that we needed a project that captured the whole world because now this was a worldwide phenomenon, it was no longer something peculiar to Kenya or Philippines or U.S., the whole world was in need of a tool like this. And that’s how I reached out to Kelvin and to Leni.. Thankfully, they both agreed to be part of the project. And I think, from my perspective, that’s where it came from. I don’t know about them… how they think about this? [LAUGHTER]

Leni: Well, for me, it’s really a big project that we did, combining the different parts of the world. You see where Kelvin comes from, representing the West, I represent the opposite, the East. And although Safary comes from the East as well, but he can represent the African continent. And so this really makes the book a global project, really a blend of different perspectives. And so I can say that online learning is represented all over the world in this particular book. And this is indeed, a big surprise to all the readers and a big discovery for everyone.

John: Speaking of readers, what is the intended audience of this book?

Kelvin: Well, I mean, honestly, I would say anyone, anywhere, around the whole planet, who in any way touches online or digital education, should access this book. It’s great for libraries and institutions to acquire and be in their communities. It’s a big book. There’s stuff in there for everybody. So I think it’s a great resource.

Rebecca: Speaking of the size of the book, the handbook contains 50 chapters. Can you talk about how you selected those chapters?

Kelvin: I think the scope and the sequence and the layout of the chapters and the sections sources originally to Safary’s proposal with the publisher, but it was intended to be rather comprehensive with sections like fundamentals and student support and administration and instructional design, instructional delivery, regional specifics, particular regions around the world, and how online education might differ a little bit in, say, the African context versus the European context. But over time, as we were recruiting authors, and as the writing process started, you get a little bit of evolution, the sections might morph a little bit, the distinctives of a given chapter might adjust based on interest and specializations of the authors. So that’s a little bit of the insight into the evolution. But I credit Safary for the vision, which I would say, is probably about 80 plus percent of what he originally had envisioned in the layout. That’s my guess. Safary, would you agree with that?

Safary: Yeah, the thing is that, when you work on a huge book like this, especially a handbook for Sage, they want to have the complete plan when you submit your proposal. Before I can get my co-editors to agree with me, they need to have kind of ideas, okay, this is what I have in mind. So usually, when I work on a handbook like this, I come up with a rough draft. And Kelvin and Leni were very good in catching certain things that I wouldn’t have caught because of their expertise, their experience, and their regions that they represent. And so in the end, what we have here is a product of the Table of Contents was really the product of these three brains that are speaking today.

Leni: I really liked the way Safary has chosen the chapters of this book. Well, we can say that he really originated the choice of these chapters. As you can see, from the perspective of a reader, when you look at the content, you can look in the sequencing, and you will find that you are actually looking into the step-by-step development, or the step-by-step process of engaging in online education. I will say it’s almost like a manual, almost every step that you will go through in undertaking online education in your institution is covered in this book. That’s why it’s really a very important book for every school to have.

John: We had some challenges coming up with a brief intro for each of you, because each of you has done so much with online education in many different roles in many different places. But you also have an editorial board for this book, which is a little bit different than many other books that we’ve seen in terms of handbooks. What was the role of the editorial board in putting this handbook together?

Safary: Yes, we had an editorial board. When you have a project of this magnitude, it is really important to have experts from different parts, especially at the global perspective of experts, and of course, experts on the different topics that are represented in the handbook. As much as we have experience with online education, we cannot assume to know it all… areas where we definitely need help. And so we selected very well known, very well recognized experts from different parts of the world. As far as online education is concerned, all the names that are there are people who are very well respected in the field of online education within their respective countries. The role they played was, for them to be our experts in checking the accuracy and the quality and the completeness of the chapters that were submitted to us. So basically, each chapter went to two to three reviewers and the editorial board members were the primary reviewers to help us really catch everything… and the work they did, I know that some chapters had more feedback than others, but I can say that contribution they gave through their feedback was very substantive in improving this handbook. I don’t know, Leni, how you found that when you’re working with the editorial members who are assigned to you?

Leni: Yeah, actually the editors we chose, I can say they are truly excellent and helpful. During the early parts of the writing of the chapters we lead editors are having like a tug of war with the chapter authors. They tend to bargain their thoughts with us, but when the editorial board came into the picture, it gave a more balanced outlook into writing the chapters. And so we really appreciate their services. The other thing is that this editorial board members are experts in the area and so we can truly depend on them. Their feedback were truly much valued and contributed much to the excellence of the contents of this book.

Rebecca: So the handbook is divided into seven sections. Can you provide a brief overview of each of those sections to give us the lay of the land?

Leni: Oh Yeah, seven sections, it’s nice to give an overview for people to know what the book contains. First section, of course, is the fundamentals of online education. It contains the introduction to the topic of the book, online learning, and some variations in online delivery, like blended, MOOC and ERT, emergency remote teaching, we just really call it ERT, and that became popular during the pandemic. The second section, online education around the world. This section is the most colorful part of the book, at least for me. Because it tours us around the world and gives us a view of how online education grew in varied contexts like US, Canada, Europe, Asia, Latin America, Africa, Australia, and the Middle East. The third section, Online Instructional Design, this section now brings us to the T-cell of online learning, the design of online instruction with focus on how learning happens online. This is now the more serious part of the book. While we came from the most colorful, we now go to the serious part of the book. And then the fourth one, Online Instructional Delivery, this section focuses on the hammer and nail of online learning, the actual online teaching, and this is the most exciting part. Because this is now the delivery, the previous one was the most serious part, this one is the most exciting part. And then perhaps, Kelvin, can you say about the fifth section [LAUGHTER] Instructional Technology for Online Education?

Kelvin: Here’s what I would say about that, if you’ve got the most serious, you talked about the most exciting that you talked about, maybe the fifth instructional technology for online education is the most invisible, maybe that’s what it is. Nobody thinks about plumbing until it doesn’t work. [LAUGHTER]

Leni: Thank you. So that’s technology, I would say this section is essential, because you cannot teach without knowing how to use technology [LAUGHTER]. And the sixth section, Online Education Administration and Management, I would say this is the driver’s seat of online bandwagon [LAUGHTER]. Online education can never prosper without the support of the school administration. So, leading school reforms, like entering the field of online education requires certain strategies to be certain of success. Therefore, I would say this section will indeed equip the readers with those skills, perhaps Safary, can tell us what section seven is?

Safary: I would say the last section is the Customer Service, given that the students are the customers. So the customer service, how to make sure we deliver the best customer service to the online students. And so it discusses all those different aspects of how to really prepare, plan effective service to the students, because many times when people are migrating from face-to-face to online or integrating online education, they forget that online students actually need serious support. And this support definitely needs to be defined. And people who are dealing with the students need to be trained. And so the last chapter actually deals exactly with that.

Leni: For me, because I was looking at the table of contents, and I was smiling in my mind, wow, this is really neatly done. And so this works came to my mind, and I said, Oh, the seventh section, this section focuses on the heart of every online classroom, the students. And so because the students are the reason why we offer online learning, thus we ought to know how we should support them.

Rebecca: One of the things that I love about working on collaborative projects that are really big, and then you have these opportunities to reflect together, is how you summarize what you did. It’s probably really different than while you were right in the middle of it. And it’s fun watching the facial expressions and things as you guys are describing the different sections.

John: With 50 chapters, there’s a great deal of breadth and depth on these topics. In section one, though, you address two topics which are not always considered as part of traditional, at least, online education, which is the use of MOOCs and ERT, emergency remote teaching. But these have played fairly important roles. Could you talk a little bit about the role of MOOCs and Emergency Remote Teaching in the larger environment of online higher education?

Leni: As I see it, MOOC and ERT are connected to the overall theme of the book, because technically they are both delivered online. Online learning can be synchronous or asynchronous. And it’s mostly taken asynchronously while ERT is done synchronously, because it is generally a replication of the face-to-face classroom through the web. However, there are certain arguments in the field as to whether can we classify these two under online learning, because they are believed to not use the principles of effective online teaching. And they say, is their instructional design in ERT? There are more questions to raise to the point that some people believe they should not be called online learning. But for me, we have a common denominator, course delivery through the web. Maybe we can hear from my co-editors here, Kelvin and Safary, what they think about it?

Kelvin: I was thinking, John, when you asked that question, I think the combination of Emergency Remote Teaching and Massive Open Online Courses, it’s part of the popular conception of what online education is, it’s sort of like what a layperson might think, is, it’s just one big thing. So if you didn’t address Emergency Remote Teaching, Massive Open Online Courses, maybe even Blended or Hybrid learning, those mutations, it might not provide quite the same way in for the broadest possible audience. But then, once we’ve ushered you into the house, through the front door, I hope we do a good job of taking you on a more detailed guided tour through the nuances and everything that online education can be, without just being stuck at that surface level.

Safary: If I may add something to the ERT. Personally, the reason why I wanted to see this chapter there was that outside of the United States and maybe Canada, and a little bit of Latin America, when ERT came, Emergency Remote Teaching came, many people call it online education. And as we know, online education, the way we know it traditionally, is much more than translating your face-to-face class to a Zoom class or Google meet class. And let’s face it, that the word there is emergency. This was an emergency modality, which obviously emergency is never the best option, it means better than the chaos that you’re going through. And so many people who didn’t know online education, they came to believe that Emergency Remote Teaching means online education. And many people who were against online education to start with, it was like, “Okay, we have already said that this thing is really bad because it was an emergency.” So it was very important to distinguish what Emergency Remote Teaching is. And in the future, if somebody wants to use that for another calamity that happens, then they know what steps to take, but it does not replace what is known, what we define as quality online education.

Rebecca: One of the parts of your book, The second section is about online education around the world. And getting that tour around the world is not something we typically get the opportunity to have. So can you talk a little bit about what some of the global differences in how online higher education is structured and practiced across continents and regions?

Safary: This section came up as we were trying to make the book global. We really wanted to hear the voices of the people from around the world and not just the United States… the United States being the lead on online education, no question about that. We wanted to know where things are in different regions that were represented. We had to even go online to try to track people down from different countries. It was not easy finding people from certain regions where we didn’t have a network. So as a result, we’re able to bring on board chapters from different parts of the world. We had a chapter from the United States, we had a chapter from Europe, from Canada, from Asia, from Latin America, from Africa, from Australia, and from Middle East. So we were able to see what was happening in each one of them. And these chapters we had, they were kind of similar in a way where we wanted to know what is happening, what are the challenges, what are the achievements that people have in those regions, so that people from those regions who decided to do more work on online education, they have a place where they can learn of what is happening in the whole region from this book. They can have this as a reference to understand what was happening in their region. It is true that when you have one chapter, for example, I co-authored a chapter on Africa, because I was still in Africa at that time. It’s a chapter that’s covering 52 countries, you cannot really cover 52 countries, we just had to have illustration from some of African countries, because there’s no way we have data on all the 52 countries, but at least, there were some common themes that were coming up from a different African countries if I can speak from that specific region.

Leni: I can speak from the perspective of an Asian because I come from Asia. And I would say, we cannot deny that online education started in the West. But because we live in a connected world, it spread easily. Basically, I can see a lot of similarities around the world. The only differences I noticed, because your question says what are some of the global differences in how online education is structured and practiced? Now, I would say the only differences I noticed are the approaches to online learning, depending on the level of their maturity, in using this modality, and the resonance of the context they serve. Institutions that have been engaged in online learning for a long time definitely deal with issues that are different from those of newcomers, the needs of the context they serve also differ, so the strategies utilized also differ. One thing I would highlight, though, is that you can clearly see the creativity and continuity of people in different parts of the world in running online education. And we still can learn from each other. That’s why I said a while ago, the section on the global online education is really colorful.

Rebecca: One of the things that I think is really interesting about that section, is that it can also give us insight as instructors that teach a global audience about what the contexts are that students might be coming from. And that’s something that we often don’t have a little bit of insight into.

Safary: I think that is a very good point. Now that we have online education, people are teaching in many different countries. I remember just a couple of weeks ago, I was approached by one of my former students who wanted me to teach a class in the Caribbean. If things worked out for me, for that class, I would have just glanced at that chapter that covers a little bit of the Caribbean and see what I need to watch out for. So that is definitely a good point for the section on the different regions. In this handbook.

John: When online education first started, there wasn’t really that much known about what would work effectively. And as online education evolved, we saw the role of instructional design become an important part of the practice of online education. And section three deals with online instructional design. And that’s helped facilitate and inform online education, along with a lot of research that’s been done since the early stages. How have instructional design practices evolved since the early stages of online education in the latter part of last century?

Kelvin: That’s a good question. And I guess I’ve been in this field watching this first hand and touching it for about 25 years now. So I sometimes say not exactly the first floor of the building, but just one step above. And what I would say is that when I started in the late 90s, what we saw a lot was adaptation of traditional instructional systems design models and practices, that is constructs that were used quite often in corporate education. See if this takes you back to the past: CD ROM development, military learners. Those kinds of methods, practices, and models were adapted to this online context. And some of that’s constrained, like you’re making a system, like it’s a bounded system that was, quite often the context, like a CD ROM. And now you’re talking about the internet, a network open system. And I remember some of those early days, like, “Okay, what can we learn from these models? How can we adapt those?” Over time though, we learned that this is a unique context, which then began to have its own models and practices and processes and research and iterations and development. And I think of even things like much newer developments, like alongside of constructs like inclusive pedagogy, we see practices and thrusts, like inclusive design, as being a very specialized subset. So we’ve got a very robust research and professional practice literature that has grown up and these, arguably, two and a half decades of online education experience to draw upon. And I guess I’ll just say this, about that. Throughout my time in this field, what I’ve seen is that online tends to make the formerly invisible, visible; formerly implicit, explicit. And I think that evolution of instructional design and development field, it has learned from that. Online education has drawn us along in what does it mean to bring learners in from really anywhere and bring them together in a learning community, and how do we excel in that. That’s been a really rich progression over these last two and a half decades.

Safary: If I may add to that, the reason why we had this section was that many people who are new to online education, they think that online education is about uploading all the files that you have been using face to face, and then let the students read that, and that’s online education. It leads to a lot of frustration from the students because there was no instructional design for online learning. And so we needed to have a section that would guide people into that. And also for instructional designers in college and universities where they already have instructional designers. Some of them have not gotten a degree in instructional design. So they have limited knowledge. They just happen to know a little bit more than everybody else, but they don’t really have a solid foundation. And so that section helps to kind of guide people in the proper instructional design for online learning.

Rebecca: So sections four and five focus on online instructional delivery and instructional technology. These are topics that we love to talk about and have episodes of this podcast on. But given the time constraints, we probably can’t dig in fully here. But can you help us identify some of the most important changes that have occurred in how well designed online courses are taught?

Leni: That’s a nice question. Kelvin also said a while ago, he was mentioning about the early years of online instructional design, I would say, perhaps 1998 to 2000, those are the early years I’ve been involved, still in the planning stages of online delivery. Most of the online courses we developed were primarily text based, and are delivered asynchronously. That was after the military, Kelvin used, online learning, it was already in the university. Why text based? Because even our students, in the context we are serving also did not have the capability or the capacity to access videos or higher level technology tools. That’s why we designed the way they can access us. And so, yes, it was primarily text based and asynchronous. However, through the years, I would say two forces caused the major changes in the way we design online courses, first, technological developments, particularly in instructional technology. And second, changes in the needs or nature of our stakeholders, the students. Well, technological developments without a doubt have increased the repertoire of instructional media that we can use in designing truly engaging online courses. But as I’ve said earlier, technology is not the heart of online learning… it’s our students. And we saw how the nature of our online students change over time as well. While many of them were happy with plain text based asynchronous online courses during the early days, now they want more real-time meetings. And the flexibility they want is indeed tremendous, I tell you. We notice that there is a greater demand now for more flexible and personalized learning approaches. And these topics are dealt with in this handbook. I know Kelvin has written on this. And some other chapters also addressed this flexible learning, personalized learning approaches. These are now the needs and demands of the new generation of online students.

John: This is bringing me back to a time when I started back in 1997 teaching online when many of the students had 300 baud… [LAUGHTER] …or 1200 baud modems, and you couldn’t do much more than text. And I remember putting in some flash-based videos, and many students couldn’t access those because they didn’t have the download speed, especially students in more rural areas. So there was a lot of resistance to online education when it was first introduced, which is one of the reasons why I think instructional design practices became a part of early online education to help ensure the quality of that. And we do have, in most institutions a fairly elaborate process of instructional design assistance and instructional design review for online courses, which is something that’s never really happened in the same way for most face-to-face courses. Might it be time to start applying some of the techniques and practices of design that’s being used for online course delivery to in-person course delivery?

Safary: I remember about 15 years ago, I was training faculty on online teaching in the Caribbean. And I remember many of them, at the end of the training, saying, “I have improved my face-to-face teaching because of the training that I have been going through for online teaching.” So I definitely believe that if people get the proper training in online teaching, they can use that knowledge to improve face-to-face teaching. Because let’s face it, many people are teaching not because they have a degree in education, but because they have a degree in whatever field they come from, they have never learned how to teach. And so when they go through the training for online teaching, they discover a lot of principles that they should have even been using face to face. So I definitely agree with you on that one.

Kelvin: Yeah, it’s true. I say it all the time online makes the formerly implicit, explicit; formerly invisible, visible. And I think that’s why online has been a vehicle for applying thoughtful design and teaching practices and the improvement thereof. Once you sort of concretize the elements that make up an online education experience, then you can see well, how are they arrayed? Are they lined up properly? Does this cause lead to the desired effect, and you can work on improvement, no offense to anyone in this, but when we just are dealing in the ephemeral, we will walk into a space, four walls and a door, and we say words into the air, it’s much harder to see how those parts fit together or don’t. And it’s harder to be reflective. So, I think that’s the reason that online education has brought more emphasis to potential improvements, continuous improvements, and so I welcome it as a vehicle for a more thoughtful process in general. I love this elegant turn of phrase Caroline Boswell says she frames teaching as a student success intervention. Or as I put it, I’m one of those odd people who sees a connection between teaching and learning. And not everybody does.

Rebecca: You’re kind of queuing up our next question perfectly Kelvin. The final section of your book is really about student support. And our students are often distributed when we’re teaching online. So what are some of the biggest challenges in terms of supporting students that are in these online programs or online courses?

Kelvin: Yeah, I would welcome Leni’s and Safary’s viewpoint on this as well. But to me, I’ll keep it simple and say that the biggest challenge is the diversity of student profiles. The different backgrounds, the multifaceted demographics, and resource or not resource, or technological connection or not technological connection, that diversity makes it awfully hard to assure kind of an equitable experience for everyone. So that’s the gap that emerges, that student support is trying to offer… not to mention the diversity of approaches to design and development in the actual experience. But I’m curious what Leni and Safary would say to that.

Leni: I would go for the opposite, on the side of the teachers, I would say the greatest challenge in student support is personalizing your support. It’s related to your diversity. Almost every online student has her unique needs and contexts. So considering different personalities and backgrounds as well, you may be able to personalize your support. But in the name of efficiency, you’ll find yourself dehumanizing the process. What do I mean by this? Well, machines can never replace human touch. And human touch is what every online student needs.

Safary: If I may speak a little bit from experience I had in Kenya during the COVID-19, we migrated our classes to the online delivery. And I quickly realized that… and this was something that was going on in all of Africa, I know this because I was involved in different international association for online education all over the continent…. and so we were meeting and discussing some of these issues. The major challenges that were going on at that time, I don’t know about today, were dealing with infrastructure, because most universities that didn’t have online education platforms, or online education structural systems, so the technology was not in place. Many students there were not access because the internet was extremely slow, some were using loads of data to access the materials and they would run out. Some had issues with electricity. These are things we take for granted in the West. These are the not issues that we will discuss even in textbooks of online education, but they are real issues that cannot be ignored. And so that was a major challenge in supporting online students, because the infrastructure was not in place. And I think the issue is still the same. But more and more work is being done. I remember, for example, in Kenya, what the government did, they gave the free data access to all the faculty in the whole country, as long as it was used only for instructional purposes [LAUGHTER]. If you want to use it for something else, it wouldn’t work. I mean, that was quite creative, to try to help people to help education move forward, because everything was just stuck because of COVID-19.

John: Over the past year, we’ve seen a fairly explosive growth and use of generative AI large language models, including chat GPT, Claude, and a few others that have come out very recently. And that opens up a lot of interesting opportunities, but also some challenges for online education, particularly concerning the assessment of asynchronous learning. How do you see online education adapting in response to the widespread availability of tools like this, which will only become more powerful over the next few years?

Kelvin: It’s sort of the very definition and epitome of disruptive innovation or disruptive technology. And just to be clear about this, I don’t think it’s limited or focused on asynchronous online education, I think it’s everything. For me, it’s really an opportunity to address learning and assessment of learning much more meaningfully, and I’ll use one of Leni’s words, more personalized and relational. I think one of the things we’re seeing with the injection of these various forms of artificial intelligence into the learning setting is the value proposition of the human. And I think it was Cathy Davidson, years ago, from HASTAC said something like, “If we faculty can be replaced by a computer we should be.” That is, if all you’re offering is something that is easily rendered more efficient and scalable by a machine then, well, what are you doing it for? I think that the opportunity to really gauge learning, which is a very personal and a meaningful thing, we act like it’s something that’s kind of homogenized and industrialized, but learning… I don’t know what learning is, frankly, I can’t crack open a human and see what all is happening with the connections and making of meaning in all the background experiences. All I can do is get insight, but in dialogue, in the creation of artifacts I get a glimpse. If we’re product oriented, to the exclusion of the process, and to the exclusion of the human context, well, that can be certainly disrupted, maybe stolen by artificial intelligence in machines. But if we keep the emphasis on humans, on “Well, John, tell me about this…” that’s more meaningful. I learned a practice a long time ago from a faculty member that I studied under, where she adopted a practice of a learning summary. And in any course, again, that’s just one artifact, but it gives a glimpse into the articulation of what learning is really about. So I think we need to push the envelope in “What does authentic assessment mean? What does meaningful learning look like?” Now, that’s hard to do at scale. Are you going to have a personal oral defense with every student for every assignment, probably not. But if we see artifacts, and products, as breadcrumb trails leading to a destination of a more substantive dialogical process, well, then maybe that’s something. So I don’t think we know yet how this is going to play out. And I think your listeners are gonna find cold comfort from me in getting to an easy solution. But I think the future of responding to generative AI is to lean more into the human and the relational than less.

Rebecca: So we always end by asking what’s next?

Safary: Well, as far as this project is concerned, what is next really, we want to continue building a community of online higher education scholars, practitioners, so that this momentum that has been created by this book can continue, because this is one of the few maybe rare books that really have so much global contribution to online education. Many of the books that are written, they’re usually kind of regional to a specific region of the world. And so this is the first time we have a network of, I think, around 100 people who contributed to this, coming from many different countries. And I feel this has created synergy on the discussion of online education in a way that we should not let that go. So one of the things that we have been talking about is the possibility of holding a summit on online higher education in the next few months, once everybody has gotten a chance to hold a copy of this book, and to bring different experts together from different parts of the world, and try to address online education from different parts of the world, while addressing common issues such as assessment, which is one of the major controversial issues anywhere have been, everybody talks about the challenges of online assessments. So that’s things like this, and probably this artificial intelligence, which is a new thing, we may want to go deeper into that… we’re not able to dig too deep with that, although we addressed it in the book. But we didn’t go too deeply because it was still kind of new ChatGPT was just coming out when we were finishing the handbook. And so that is one of the things that we are looking into, there is another handbook in the making with SAGE that will focus specifically on instructional design in higher education. So that would be like an extension of this project. So we want to continue building on this work, because we consider it’s very important.

Leni: I’m really optimistic about the next steps on this because it’s like a seminal book that really got there’s a global perspective, as Safary says it’s not the same as the other online learning books. So we can also see a lot of developments coming up. And so I will say, this book is just step one, the next steps will really be coming up definitely, because the field is always growing. We have seen its growth, and it will still grow. And so there’s more to follow, I believe.

Rebecca: Well, thank you all for joining us. I know that our listeners will really enjoy the handbook and all that it has to offer.

John: Well, thank you, and it’s great talking to all of you and we’re looking forward to reading the book.

Safary: Thank you so much for the opportunity. Really appreciate that and wish everybody a wonderful reading experience.

Kelvin: Thanks for having us, Rebecca and John.

Leni: Thank you very much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

313. Supporting Neurodiverse Students and Faculty

Many discussions of inclusive teaching practices ignore the role of neurodiversity in higher ed. In this episode, Liz Norell joins us to discuss strategies that faculty and institutions can use to create a welcoming environment for neurodivergent students and faculty. Liz is a political scientist and the Associate Director of Instructional Support at the University of Mississippi’s Center for Excellence in Teaching and Learning.

Show Notes

Transcript

John: Many discussions of inclusive teaching practices ignore the role of neurodiversity in higher ed. In this episode, we discuss strategies that faculty and institutions can use to create a welcoming environment for neurodivergent students and faculty.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Liz Norell. Liz is a political scientist and the Associate Director of Instructional Support at the University of Mississippi’s Center for Excellence in Teaching and Learning. Welcome back, Liz.

Liz: Thank you. I’m happy to be here.

Rebecca: Today’s teas are:… Liz, are you drinking tea?

Liz: I am not. I am drinking some vitamin water, tropical mango flavor.

Rebecca: So there’s some stuff mixed with water. That’s tea, right?

Liz: Yes, sure.

John: And I am drinking Prince of Wales tea today.

Rebecca: Oh, I like that one. John. Haven’t had that in a while. And I have Awake today. It’s Monday… [LAUGHTER] when we’re recording.

Liz: That feels fair.

John: But it’s getting really boring. [LAUGHTER]

Rebecca: I know. [LAUGHTER]

John: We’ve invited you here today to discuss issues related to neurodiversity in higher education. Before we start, though, could you define neurodiversity? And how is this different from neurodivergence?

Liz: Sure. So I think these two terms get kind of conflated with one another a lot. And so I tried to be really explicit in talking about neurodiversity versus neurodivergence. And there are a lot of different perspectives on these two terms, and some of the baggage that they carry with them. But when I think about neurodiversity, I just think about a diversity of brains. And so any group that has more than one person is going to be neurodiverse. We all have different brains. But neurodivergence is a brain that works differently than how we typically think of brains working. And there are lots of diagnoses that get put under the umbrella of neurodivergent. Some of the most common ones are autism, ADHD, there’s Tourette’s. There’s lots of other ones, dyslexia, dyspraxia, OCD sometimes gets lumped in there, bipolar disorder will get lumped in there. So neurodivergence is just a brain that works differently than the way we typically think of brains working.

Rebecca: If we’re thinking about college student populations, and I know that this is partially a guess, because we don’t actually know, but how many do we think are neurodivergent?

Liz: The estimates that I’ve seen have been around like one in six, maybe, but I think that it’s probably closer to 30, or 35%, honestly. I think a third is probably a reasonably good estimate. It’s a large number. And I will say that a lot of those students either may not know that they have some sort of neurodivergence, or they may not ever tell us that they have a neurodivergence and we’ll talk more about that a little bit later, I think.

John: What proportion of neurodivergent students have accommodations through campus disability services?

Liz: Who knows? If we don’t know the denominator, it’s hard to know what proportion of people would be registering with accommodations. But I think there are certainly a lot more students who are registering with disability services with an official diagnosis, but there are some barriers to that. And the first one is that it’s really expensive and time consuming to get a diagnosis. So I should say, and I should start pretty early in our conversation by saying that I recently went through the diagnosis process to get a diagnosis of autism at age 45. And it took me a year from reaching out until I had the diagnosis. And I was able to navigate that, because I have some experience interacting with medical teams. I had good insurance, but it still took me a year to get an appointment, and to get the diagnosis. And so there are a lot of students who may not have the tools or the time or the resources to go through that, even if they suspect it. And I went 44 years of my life without even suspecting that I might be neurodivergent. I think there are a lot of barriers to that. And then once you have the diagnosis, it can be very intimidating to disclose that, to go through the campus accommodation process takes so much time and advocacy. And that comes from a population that’s already taxed in terms of their bandwidth and their resources and their just ability to get these things done. So I don’t think that thinking about the numbers of people who seek accommodations is even close to representing the population of students who might have these conditions.

John: It would seem that there’s a bit of an equity issue here in that students from wealthier households, students from continuing generation households, are much more likely to have the resources to go through the process of having the need for accommodations being documented.

Liz: That’s right. And I think this gets into some of the language around neurodivergent versus the neurotypical. A lot of people who are neurodivergent, who have some sort of condition or way of thinking or way of operating have been socialized to think that there’s something wrong with them, this kind of medical model of disability, instead of the more like social model of this is a socially constructed difference. And so to seek out a diagnosis requires a kind of self containment, I think, to recognize that this is not something wrong with me. A lot of people who have a neurodivergent brain probably feel like they should be able to act like a neurotypical person. So they don’t want accommodations because they feel like that is somehow making them less than their neurotypical students. And it’s this medical model that has infused so much of our talk about disability, and especially pernicious here where we know that there are real struggles that students have when they have these neurodivergent brains, that we are just not accommodating well in the classroom.

Rebecca: I do want to mention at this juncture, that we do have an Episode 221 – Disability in Higher Education with Kat Macfarlane that really talks in detail about the accommodation process. And so that’s a really great place to learn more about that process in high detail that was kind of the subject of most of that episode. Most of our college faculty generally haven’t been trained to address issues of neurodiversity. Can you talk about some of the common challenges our neurodivergent students face in classrooms?

Liz: Yes. And it’s absolutely the case that we have not been trained in this. And I think also many of my faculty colleagues, past and present, have this idea that an accommodation is somehow like special treatment that’s making a class easier for students, when we should, John, as you mentioned, be thinking about this as an equity issue. So accommodations are meant to provide equal opportunity for success. And if you’re bringing some of these conditions into the classroom, you’re already operating at a deficit. So what are those? Well, it can be things like being really easily overloaded by sensory information. So we see this a lot with autism and ADHD, where, as someone who now understands herself to be autistic, I think about this phrase, “the lights are too loud,” like, it just feels very harsh. And when people are talking over each other, I get very flooded very quickly. This has been the case my whole life. If there are unfamiliar foods or drinks, that can be really overloading and so background noise, people who are close to each other, uncomfortable seating, these are all things that can show up in our classrooms that can cause someone with a neurodivergent brain to go into a kind of overload. And that, of course, reduces their ability to pay attention and to learn and to retain information. Unclear communication is a huge challenge for people with neurodivergent brains, because it’s often the case that there’s some sort of like inability to recognize sarcasm, or the ability to get some nonverbal communication. Oftentimes, people with neurodivergent brains will interpret everything very literally. And so they miss out on some of the nuance. And for me, it’s been this like obsession with choosing the just right word, because I need it to be precise. And I can get really fixated on that sometimes, in a way that feels very pedantic. But that is really just me very much trying to communicate clearly. When there’s unclear terminology–write professionally, or be collegial, or work well with others–like I don’t know what any of that means. I have no idea. And there’s an assumption that there’s some shared social norms that may not be as visible to people with a neurodivergent brain. There’s a lot of, of course, well documented social aspects to neurodivergence. So just like not really knowing how to work with others in an effective way, or feeling like that sense of, I’m different. I’m broken, I’m not as good as… that I mentioned earlier, can carry over into social dynamics. And then the last one that I think is really important for us to think about in terms of higher ed is executive function. So executive function is that ability to kind of be a taskmaster of your own attention and brain. And so things like prioritizing work, time management, how to take notes, how to make decisions, how to cope with the ups and downs of life, due dates, all of those things like managing systems is really hard when you have a neurodivergent brain. And we often assume that our students have those skills. And so we don’t scaffold them. We don’t help them. We don’t point them to resources, and that can be really hard. So those are just like four big clusters: sensory overload, communication, social interaction, and executive function.

John: How can faculty anticipate or design with neurodivergence in mind, particularly when many students with disabilities choose not to self identify?

Liz: Just being aware of these things that I’ve just mentioned, is hugely helpful. And I think the hard work is really just awareness. So for example, I have heard lots of my colleagues and myself at earlier points in my career, lament about students who are distracted by their cell phones or their laptops, who seemed to need to go to the bathroom three times during a 50-minute class, or who otherwise seemed to be just kind of like disconnected from class. We see that as a sign of disrespect and as of not paying attention. But a neurodivergent brain often really struggles to sit still and make visual contact with another person or object. And so it’s often the case that our neurodivergent students can learn better if they are doing something, the more physical, the better. So for some of my students, it’s things like knitting in class or coloring or doodling. This is actually not them disengaging or not paying attention. It’s them doing something that allows them to focus their attention on what you’re saying. So I like to think about performance of attention, what we often think of as paying attention. So if a neurodivergent student is going to perform attention, they’re probably not actually listening to anything you’re saying, because they’re using all of their brainpower to do the things that you think mean they’re paying attention. With that said, this sort of notion that we have to reorient our thinking about what students are doing and what that means in terms of their engagement with us, I think being really clear about scaffolding what tasks are needed, providing clear deadlines. So Karen Costa, who is just brilliant, talks a lot about ADHD, and she is a person with ADHD. And she talks about the need for more structure, not less, that flexibility can be useful, but you need a lot of structure. And for people with neurodivergent brains, it can be really helpful to have lots of small deadlines that are low stakes with some grace around them, but like clear structure is really important… messaging to students, like, here’s how you do this class. So if that’s working in a group that’s giving specific roles, and asking the students to decide who’s going to be the note taker, and who’s going to be the recorder, and who’s going to be the crazy idea person, and who’s going to be the let’s bring it back to the text person. So just kind of delineating some specific roles, communicating clearly and in multiple modalities. So especially if you’re doing a lot of audio lecturing, or giving up directions, making sure that those are also available in writing. So students can come back to them later when they know that you said something, but they don’t remember what. And then just being really aware of that sensory environment in which you’re learning. So if that’s in a physical classroom, thinking about ways to give students permission to make themselves comfortable, if that’s getting up and walking around a little bit, and just sort of saying, like, I know that that helps some of you concentrate, fine with me if you do that. Telling them that they can get up and leave the room for a couple of minutes if they need a break. If they want to bring in things to play with, color or fidget, create whatever, that’s fine too. A lot of my students like to just sit on the floor. I mean, that sounds like a disaster for me and my middle-aged body, but when you’re 18, it’s like easy to get up off the floor, and so if you want to sit on the floor, sit on the floor, it’s cool, if that helps you be more comfortable. So I think it’s awareness and then just messaging to students that they can do what they need to do. And I just want to say one more thing about this. And that is, even if you are not neurodivergent, even if you are what people define as neurotypical, you can talk about students you’ve had, people you know, friends, family members, colleagues, you know people who are neurodivergent, talk about some of the ways that they have given themselves permission to make their environments work for them. So that you’re messaging to students that you understand and that you support those kinds of self-advocacy efforts. So you don’t have to do all of that on the first day of school, [LAUGHTER], first day of class. It’s a lot, but I often include something on my syllabus that says you may have accommodations or you may not, but if there’s something that I can do to make this class easier for you to participate in meaningfully and be successful in, according to your own goals, then I will do it, as long as it’s something within my power to do. So. You don’t need formal accommodations to ask me to do something to help you.

Rebecca: One of the things that I heard you mention and often come up a lot in inclusive pedagogy and other spaces is the idea of scaffolding. Here, again, we have this assumption that everybody knows exactly what that is all the time. But part of that is really about helping students understand their priorities, perhaps within a class, and also how to manage their time related to certain kinds of tasks. Can you talk a little bit about that component of scaffolding and what that might actually look like in practice?

Liz: Yeah, it’s hard for me to tease that out without thinking about lots of other things too. Because, we, as faculty, are coming into the classroom with certain ideas about what should be important, and what students should want to do in order to be successful, whatever that means. And I have really had to learn over my teaching career to check myself on that, because my priorities are not the same as my students. I remember once I was grading a student’s final exam they had done, it was very, very early in my teaching career, I’m embarrassed to even say what they had done for their final, but it was it was multiple choice and I was grading it by hand, and I told them, they could just stick around. I would grade it real quick, and then give them their final grade, because I had done everything else before then. And I looked at the student who had come to class every day and had really meaningfully participated and said, “Your final grade as a C,” and I was so apologetic, like, “Oh my gosh, I’m sorry. “And this student was like, “YES! I PASSED!” And it was a real moment for me of just like saying, “Okay, you cannot put your own values and goals onto students, because that was literally the student’s dream–was to get a C and not have to take the class again,” I think when we’re talking about students, it’s really having that very frank conversation, like, some of you are here because you are being required to take this class, and I know it wasn’t your choice, and I’m going to try to make it as the least awful it can be for you. And I’m going to ask you to try to invest at least enough time to give it a fair shot. But I don’t expect that everybody has the same goals and so let’s take a moment and reflect on what success for you looks like. And then what do you need to do over the course of the next 15 weeks, or whatever it might be, in order to make that goal. And so we sort of assume that our students’ goals are to get a good grade and to move on to whatever the next thing is. But maybe it’s not, maybe they’re just taking the class for fun, maybe they don’t care about the grade. Maybe they’re an adult learner who has a curiosity about something. After I graduated undergrad, I took an ethics class online–this was like 2001 or 2002 maybe, so it was very early days of online teaching and learning. And I took it through the community college just because I had never taken it, and I thought it might be interesting. It really wasn’t interesting for me. So I just like stopped paying attention. But I did not consider that a failure. Like I got a little bit of information. I also took a macroeconomics class, because I had never taken a macro class; I had taken micro. And it was all online, and I did all the work, but I didn’t turn in any assignments because I just didn’t care about those. I just wanted to learn it. So, I think having these conversations can be really helpful in students figuring out what it is that they want to get out of a class

John: Going to that point you made about structure. This is something we’re seeing an awful lot… certainly in the work on inclusive teaching, in Viji Sathy and Kelly Hogan’s book, as well as the work of Mary-Ann Winkelmes on transparency and learning and teaching. There seems to be a convergence that by providing students with structure and support, it can do a lot, it can benefit pretty much all students. In past discussions, when we spoke to those authors, much of the focus was on the benefits to first-gen students and to students who were historically minoritized. But it’s kind of nice to know that the same inclusive teaching strategies also addresses issues of neurodivergence.

Liz: I had an experience at my last job, where I just kept asking my dean, “Give me a set of rules and I will follow them.” And she would say, “Well, just use your best professional judgment.” And I don’t know what that means. I think people with certain kinds of neurodivergence just want you to tell them what to do and they will do it. Give me a set of very clear expectations, and I will meet them and this can work for everyone… just like clarity. And it doesn’t have to be punitive. I’m very fond of Cate Denial’s Pedagogy of Kindness. And this notion that like it should be kind, which can be as Sarah Rose Cavanagh says like a warm demander. I want you to have expectations of me, I want to know that you care about me. But just be really clear, because clarity is kind. And yes, it helps with all of those things. So the first generation students, the historically minoritized, the neurodivergent, lots of different kinds of people are benefited by this. And even the third generation middle class neurotypical student, also [LAUGHTER] benefits from clarity.

Rebecca: Imagine that, not spending all of our cognitive energy trying to figure out what people want.

Liz: Exactly. {LAUGHTER]

Rebecca: What are some of the challenges that neurodivergent faculty face in their careers? We’ve talked a lot about students, but we also know that faculty also exist.

Liz: We do. And we’re not all like cut from the same cookie cutter. I can tell you, just from my own experiences, that higher ed can be really hostile to those who are neurodivergent. I’ve had really great experiences, and I’ve had some really challenging ones. I think that it’s helpful when we’re aware of these things for students, because we often have the most power over their educational experience. But we also share power with our colleagues. And so knowing what some of these things are can help us understand the behaviors of our colleagues that we might have been inclined to read as subversive, or unprofessional, or lacking in collegiality. Those words get used a lot, for a lot of different kinds of identities and traits. Neurodivergence is certainly one of them. So as a woman who’s neurodivergent, that intersectionality means that I’m always on the lookout for that kind of language of like unprofessional and not collegial and you’re being difficult in some way. Well, or maybe I just don’t understand what it is that you’re asking of me. I also think we need to be really careful when we think about this idea of fit. So especially in hiring, we are looking for someone who will fit. But fit often means like me. And it can be very exclusionary to people who have some sort of neurodivergence, because they may not act the way that you do. But that’s actually a strength, I think. When you look at the different kinds of neurodivergent conditions, ADHD brains are so good at hyperfocus. They just don’t always do a good job of like, understanding time, right, there’s a kind of time blindness. But they’re so good at that. And autistic brains are so excited about the things that they’re excited about. And that energy is so captivating. And so these are not weaknesses, these are strengths that can really help us appreciate things in our work that we wouldn’t if we didn’t have those around. So when thinking about working with colleagues, all of the things that I said before, sensory overwhelm, communication, social interactions, and kind of executive function, we should be thinking about those things with our colleagues as well. So when I design a workshop, for example, in our Center for Excellence in Teaching and Learning, I’m thinking about, “Okay, how do I create a space where the chairs could be moved away from the rest of the group so that people can have a little bit of space and kind of get away from that? Can I dim the lights a little? Can I ask, make sure everybody’s using a microphone, but also let people know that if they want to put in their loop earplugs, as I do, you can do that to kind of limit some background noise. Can I make sure that everything I say is also written down somewhere so that people have something to refer to later? Can I talk about my own experiences in a way that normalizes other people doing the same?” All of those things can be used to make the environment more inviting for our colleagues. The last thing I want to say is that it is so exhausting, as a person with a disability of any kind to constantly having to advocating for yourself. So the more that non-disabled people can lend their support and their voices to advocating for easier pathways to accessibility, the less you’re taxing your disabled colleagues. So thinking about what can I do, that if I did it, would make it easier for a disabled colleague to come behind me and ask for the same?

John: Near the end of the summer you posted on the social media site formerly known as Twitter, something about a podcast and puzzles set of workshops. Have you started that? And could you talk a little bit about that?

Liz: Yes, so I’m really excited about this. And a lot of people who are not at the University of Mississippi are also excited about it. And I’m just trying to get the people on campus excited about it. So the idea here is that, and this was specifically created as a neurodivergent-friendly space. So faculty and staff can come to our center for an hour every other week. So this Wednesday will be our next one. And we play a pedagogy podcast. So we played an episode of this podcast, and we do a puzzle or some other kind of individual or parallel play is what it’s called. So I’m working on a puzzle. It’s in the next room, and I’m not done with it. And it’s driving me crazy, because I don’t like unfinished puzzles. But I have committed to not working on it, except when everybody else is here. But it’s a Funko Pop puzzle of Ted Lasso, so it’s really fun. And we’ve had two of these now, and it’s a small but mighty group who are into this, but the lights are low, it’s indirect, diffuse lighting, there’s lots of different kinds of seating, and one person comes and colors, a couple of people have come and done puzzles. But the idea is that it’s just a way to get together in a social space without the social expectation of small talk. So you can just come and show up and listen to a podcast and leave. If you want to stay and talk you can, you don’t have to, I am hoping that this becomes a movement of podcasts and puzzles. And I’m going to stick with it as long as it takes for me to make it so here, but we have probably like five or six people who have come to one of them. And I hope many more who will as it continues to spread. It’s kind of hard to describe in a website or an email, but I think once people come they see the brilliance of this… I say with all possible modesty. [LAUGHTER]

John: Have people actually finished a puzzle during the course of one of these meetings?

Liz: This Wednesday will be our third meeting. So I think the Funko Pop Ted Lasso crossover puzzle will finish this week, I hope. It’s going to drive me absolutely batty if it doesn’t, and then we’ll move on to another one.

Rebecca: That sounds like a lot of fun.

Liz: It is. And it’s a good way to kind of model why I think so many of our neurodivergent students would really thrive at a kind of way of learning that’s very different than what we’re used to in higher ed. And that’s probably why it’s hard for people to imagine why we’re doing this or what it looks like. But we have writing groups where people come into our room and do their own writing. And just that body doubling of having someone else there, while I’m trying to do something is enormously helpful. And so in this case, like I took two things that I love, that I never make time for because I feel guilty about all the other things I should be doing. So puzzles and listening to podcasts about teaching, and I just put them together. And I hope that more people will see ways to create these spaces that are perhaps a bit unconventional to higher ed. But that can open our imagination to the ways that we can model learning in different ways than the more traditional models that we’re used to seeing.

Rebecca: I like the analogy with the writing group, in that it’s really holding people accountable to do a particular thing, which is to attend to teaching in a different way, by listening to a podcast as opposed to a different kind of workshop or something and allowing them to do something with their hands.

Liz: I also have this very large bucket of fidget toys that I take to every workshop. And I say just borrow a fidget and just play with it and see how that changes your experience of the workshop. And if you find it to be soothing, imagine what normalizing this in your classes might do for your students. So my colleague who’s just a couple of doors down I have one of these little like pop balls that make like these really satisfying noises. And the first time I brought this to a workshop, she said, “Is that the sound I’ve been hearing?” I just play with it all the time.

John: Do you have any other advice for faculty and campuses who wish to better address neurodiversity?

Liz: There’s this phrase in the autism world and the disability world and I’ve been hearing it more and more and it is, “Nothing for us, without us.” And so I can tell you my perspective as someone who is neurodivergent, there’s so much expertise on your campus, and you should talk to those people. So that might be in the disabilities support services area. It might be students in your class, but just like have these conversations and find out, what can I do from my position, whatever it might be, that can make this place more welcoming to people who are neurodivergent? And I think when you’re asking that question, just like with anything else that we might be doing, then people are going to assume good intent. And they’re also going to be much more forgiving, if you make a stumble of some kind, whatever that might be. I don’t know. And so just talk to people, ask them. I feel like this is the most obvious advice that we give as faculty developers, but it’s ask your students, just ask them, they just want to be asked. And so if I was to give any advice that would be that just ask your students: “What can I do that would make this easier for you?”

Rebecca: I know, one thing that we talk a lot about on our campus is that access is really the doorway to belonging. If you don’t have access…

Liz: Yes.

Rebecca: …you’re not going to feel like you belong.

Liz: Just to know that someone is thinking about what you might need is enough to make them feel like they’re included, and that you’re listening when they tell you what they need, would be helpful.

Rebecca: So we always wrap up by asking: “What’s next?”

Liz: I have so many writing projects that I’m just sort of getting started with. So I just recently finished a manuscript on my book, The Present Professor, that I mentioned the last time we talked. And so that’s going through the publishing process and will eventually go out in the world, I assume, knock on wood. So I’m filling my time while I wait for progress there on, it seems like, about a dozen other writing projects, all of which are just kind of me thinking. I’ve been really interested lately in talking about the role of learning outcomes, and what we decide rises to the importance of a learning outcome. And if I may say this one controversial thing that I just keep saying to everyone I know, I don’t think you as a student should be able to fail a class for doing something or failing to do something that is not a learning outcome of the course. So if turning in something two days late means that I fail the assignment, then shouldn’t that be a learning outcome, timeliness? I don’t know. It just feels to me like, if we’re going to assess learning, then we should be assessing learning, and not all the other things that are performance of learning. So I’ve been thinking a lot about that and a whole bunch of other things. That’s what’s next, something, many things.

John: We had a similar conversation with Kevin Gannon, not too long ago who talked about…

Liz: …performative hardassery…

John: That was the technical term…

Rebecca: [LAUGHING]

John: …but it was…

Liz: Yes

John: …in terms of rigor, who distinguished between cognitive rigor and logistical rigor.

John: Well, thank you. It’s great talking to you and we look forward to more conversations in the future. And when your book gets closer to coming out, we’d be very happy to have you back on to talk about that as well as any other topic that comes up in between now and then.

Liz: Absolutely. I so appreciate the work you guys do, and I’m grateful and honored to be a part of it.

Rebecca: It was great talking to you. Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

310. Community Effects of Incarceration

Some students receive substantial support on their educational journey within their homes, communities, and schools; others face substantial barriers. In this episode, Arpit Gupta joins us to discuss his recent study that examines the effect of community incarceration rates on the academic performance of children in affected households and on their classmates.

Arpit is an Associate Professor of Finance at the Leonard N. Stern School of Business at NYU. Arpit has published extensively in highly ranked finance, economics, science, law, and management journals on topics ranging from housing markets, infrastructure investment, bail, local journalism, racial housing gaps, incarceration, and remote work.

Show Notes

  • Gupta, Arpit and Hansman, Christopher and Riehl, Evan (2022). Community Impacts of Mass Incarceration. May 3.
  • Norris, S., Pecenco, M., & Weaver, J. (2021). The effects of parental and sibling incarceration: Evidence from ohio. American Economic Review, 111(9), 2926-2963.
  • Lazear, E. P. (2001). Educational production. The Quarterly Journal of Economics 116(3), 777–803.
  • Chetty, R. (2016). Improving opportunities for economic mobility: New evidence and policy lessons. Economic Mobility Research and Ideas on Strengthening Families Communities the Economy, edited by Brown, Alexandra, Buchholz, David, Davis, Daniel, and Gonzalez, Arturo, 35-42.
  • Chetty, R. (2021). Improving equality of opportunity: New insights from big data. Contemporary Economic Policy, 39(1), 7-41.

Transcript

John: Some students receive substantial support on their educational journey within their homes, communities, and schools; others face substantial barriers. In this episode, we discuss a recent study that examines the effect of community incarceration rates on the academic performance of children in affected households and on their classmates.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist;hellip;

John: ;hellip;and Rebecca Mushtare, a graphic designer;hellip;

Rebecca: ;hellip;and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Arpit Gupta. Arpit is an Associate Professor of Finance at the Leonard N. Stern School of Business at NYU. Arpit has published extensively in highly ranked finance, economics, science, law, and management journals on topics ranging from housing markets, infrastructure investment, bail, local journalism, racial housing gaps, incarceration, and remote work. Welcome, Arpit.

Arpit: Thanks so much for having me.

John: It’s great to see you again. It’s been a while since we last talked… 20 years or so.

Arpit: Yeah, it’s been a while. So I owe my economics career to John having him teach me at a very formative time in my life. Very happy to be back here.

John: Back at the TIP program, way back. And you would have probably done that anyway, because you had a lot of interest in it even back then. Today’s teas are: …are you drinking any tea, Arpit?

Arpit: …just drinking water at the moment.

Rebecca: It is the foundation of tea.

John: It’s one of our more popular teas.

Rebecca: I have an Awake tea today.

John: I have a Darjeeling tea today.

Rebecca: So we’ve invited you here today to discuss your May 2022 working paper on community impacts of mass incarceration, co-authored with Christopher Hansman and Evan Riehl. Could you tell us about the origin of this study?

Arpit: Yeah, so Chris, Evan, and I were all graduate students at Columbia University. Chris and I were also roommates. And we had a third roommate who was a public defender. So we would just come home and hear interesting stories of his experience at work and things he was seeing. One of the things that he brought home and kind of talked to us was the fact that bail was an interesting process. And there was an interesting random assignment across bail judges. And so that was our first project, it kind of stemmed directly from talking to this roommate and his collaborator on that project. And another thing that he was mentioning is that the way he saw it is that incarceration spells really had rippling effects, not just directly on individuals concerned, but kind of affected broader communities in different ways. And we felt that that was a really interesting insight that has been explored in some other non-economics research. And we wanted to just explore this concept further, because we felt it was an important essential public policy question. And so we spent many years to try to get the right data and setting to explore further at these broader community impacts of incarceration.

John: So earlier studies had found that incarceration of a parent had significant effects on education for children within the household. Could you just talk a little bit about those effects before we talk about your contribution to this literature?

Arpit: Yeah, absolutely. So there is a pretty broad literature on this topic. And I would sort of separate some of the papers that are not in economics from the papers that are in economics. There are a number of great studies that, for example, will track cohorts of people across generations to kind of see what are the rippling long term implications of incarceration. There are a variety of these papers that explore I would sort of describe are the multi dimensional aspects of incarceration on different outcomes for individuals and families that are concerned. And I would sort of characterize this non-economic literature as really highlighting the disproportionate spatially concentrated incarceration. And that’s kind of the key insights of this broader sociological literature, that you think of incarceration as something that affects a lot of people in very concentrated ways and bad ways. The economics literature has taken a little bit of a different approach and has primarily focused on the direct impacts of incarceration, with some literature starting to look at how that also affects household members. A lot of literature has been in Scandinavian countries where they have a different justice system and really good data. Some of those papers have actually found positive effects of parental incarceration on children outcomes, which might make sense if you’re removing, for example, a negative role model from a child’s life or if the criminal justice system itself offers positive remediation, restorative justice, and so forth, that kind of improves someone’s outcomes after they’ve returned from prison. The closest paper to our study in the United States is going to be a paper by Norris and Weaver, which focuses on the effects of incarceration for students in Ohio. And there, they argue that incarceration of a parent improves the odds that the child is going to be involved in the criminal justice system in the future, so that they are less likely to be arrested in the future. And they find more mixed evidence on the education impact. They don’t find much evidence for negative education impacts. But that’s done on the kind of little bits of a smaller sample with larger standard errors.

John: Your study, though, goes a little bit further, because you’re looking not just as the effect on children within the household, but also spillover effects into their classrooms and schools, from incarceration of adults in the household. How did you separate out the effect of differences in incarceration rates from all the other factors that might influence such outcomes in those communities?

Arpit: Absolutely. So this is going to be, of course, a key distinction between how economists think about the problem versus other disciplines. We’re thinking upon the question of identification. How can we identify whether the negative impacts or positive impact you’re looking at can be attributed to incarceration, or are just reflective of other background trends. Let me start first with actually how we think about these effects in aggregate, because that gets at like the community dimension of the problem, which is kind of our central focus. So the big question that we’re really interested in is what happens to a community, when a lot of people within that community are behind bars? How does the impact on that set of individuals spill over and impact the overall community. And of course, this is an even harder identification problem than just looking at the attacks on one person, because you wonder what the omitted background factors that can affect entire communities. But we find that when a county has a relatively more strict set of judges, that actually has a large impact on the overall performance of all the students in the area. So that suggests that there are large impacts of incarceration that kind of broadly affect all the students in a particular area. And that motivates us to think about what is the size of the effect of incarceration on children’s outcomes, and what are the mechanisms by which they’re affected? But we then dig more deeply into thinking about the effects on the directly affected children, those whose parents are themselves incarcerated. There, we similarly use judicial variation, and we also look at the spillovers onto other children in the classroom. So the key innovation, the key contribution, I think, of our analysis is to take this question that has been studied before, but adopt it to the problem, thereby thinking about the more aggregate consequences and the mechanisms by which incarcerations affect broader communities.

John: And you also use an event study approach too, to provide more support. Could you talk a little bit about that part of the analysis?

Arpit: So we use those in both our direct and indirect analyses where we were trying to understand what is the impact on a student if their family member is incarcerated. And the event study approach basically looks before and after that arrest and looks at the outcomes for the children as measured by outcomes such as the test scores, the suspension rates, misbehavior rates, and so forth. So we’re interested in a little bit of a multi dimensional set of outcomes for children, because we want to know both how is this child doing, we want to know whether there are behavioral disruptions that may stem from having a background incarceration at home, that may then affect other children, because if you’re misbehaving in the classroom, that’s something that will negatively potentially affect other children’s learning in the classroom. The event study is looking within the child before and after that arrest period. And we also do that same event study analysis at the classroom level, basically. So looking at what happens to the performance of other students in the classroom, when one of the students’ family members is arrested.

Rebecca: How big was the impact of incarceration on children in the affected households and in the classrooms.

Arpit: So for one individual child, the effects on math and English scores is something 5% of the standard deviation. So it’s an effect that is sizable enough, if you think about many educational interventions as having very heterogeneous effects, and it’s very hard often to kind of get meaningful moves in child performance. But the really big part of the analysis, I think, was trying to reconcile those direct effects, the ones that are one to one and a half percent of the standard deviation against the overall impact of incarceration on the whole community. So what happens if I take a whole county and I change the mix of judges and I have much more incarceration? What is the overall educational impact there? So when we looked at that overall community level perspective, we actually found that changing a one standard deviation in the county level stringency is actually affecting test scores by between one and a half to three and a half of a standard deviation. So we’re basically getting very big aggregate effects that the individual effects alone can’t explain. And so we think that there’s scope for these spillover effects, by which one directly affects how a child behaves in a certain way in the classroom that then spills over to the other children in the classroom that thereby amplifies the effect, so as to generate larger negative overall effects. And one channel that we use to identify those is to look within the classroom itself, not gonna measure all the potential spillovers between children, but it’s one area where we think there’s spillovers, and we think that those spillovers can also account for some fraction of the overall community effect.

Rebecca: Can you translate some of that standard deviation talk [LAUGHTER] to people that don’t know anything about statistics.

Arpit: For example, at the county level, when we are thinking about a one standard deviation increase in the stringency we’re thinking about a 15 to 20% increase in incarceration. So that’s kind of the range of variation that we’re looking at at the county level when thinking about what are the typical shock to incarceration, and that’s a kind of pretty substantial increase in the incarceration levels we’re seeing as a consequence.

John: So you’re finding the effect on any one other student is relatively small, but the aggregate effect on all the students in the class is relatively large. Is that correct?

Arpit: That’s right. So when we look at those other students in the classroom, we’re getting effects for those students in response to the incarceration of a peer’s family members, they’re on the order of 0.3, 0.4 percent of a standard deviation. You should just basically think of that as a really small number. And the only way we’re kind of getting the power to analyze this is that we’re looking at this North Carolina data, which is really great, a lot of people have worked with it, exactly because it is so comprehensive. So we’ve got all the student rolls, we’ve got all the arrest records, all of these are matched together. And so using this really holistic sample allows us to try to quantify these effects that are pretty small for any one individual child, but they’re just a lot of exposures that can aggregate up. And so we think that this classroom disruption channel can explain something like 15% of that relationship between aggregate incarceration and test scores. So it kind of all adds up to explain a more meaningful fraction of this overall relationship between what happens when a lot of people in the area go to jail and what happens to student performance in that area.

John: What sort of mechanism are you hypothesizing might be the cause of the spillover effects to other students in the classrooms?

Arpit: So let’s start with what we can measure in our data. So what we observe is that children who are affected whose family members are incarcerated are looking at increases in suspension days, they’re absent more often, they’re involved in more fighting incidents, typically it takes two people to fight. So that sort of tells us that there are other people involved in the classroom for these affected students. And so we think that this relates very closely to the idea that there are classroom level externalities, and there is a large literature, actually papers by Lazear and others that highlight the importance and implications of classroom level externalities, classroom disruptions, when it comes to learning. It also comes up, by the way, when I talk to people in North Carolina who are teachers. One thing that they really bring up is that children come into the classroom with all sorts of backgrounds that change behavior in the classroom, and that impairs the learning experience for other students in the classroom. So that’s what we can measure most cleanly, is the existence of these behavioral disruptions by students affecting how they behave in the classroom, and influences, through that channel, the learning experience of other children. That doesn’t need to be the only mechanism that’s going on here, there can be other spillover channels between children that we can’t observe in our data. There can also be other channels outside of peer interactions between children through other community interactions between people as well, that we also can’t measure in our data. So we think of this project as really trying to open a set of analysis that we’re considering and thinking about the broader web of social interactions, when incarceration happens.

Rebecca: What are some of the public policy implications of the study?

Arpit: So the challenge, of course, is that you’re measuring one side of the equation, we’re measuring sort of the cost of incarceration, and so you have to balance those against some of the possible benefits of incarceration, because children are also affected by crime in the local community, as well. And so it’s a difficult trade off to try to balance both the costs and benefits of incarceration in tandem. So I don’t think our results actually have a clear takeaway. I think the biggest thing that I personally kind of took away from the analysis is that if we have different techniques, if we have different ways of trying to reduce and address crime, it would be ideal if we were able to lean on ways that rely less on the incarceration channel, which impose these additional externalities and costs and burdens on local communities, and instead found other ways of trying to address and mitigate and reduce crime. So for example, when it comes to a different setting, when it comes to thinking about bail, which is a topic we’ve also researched before, there is sometimes a choice between arresting the individual and putting them in jail, compared to something like house arrest, compared to something like electronic or digital monitoring. These systems are also not perfect. There are also a lot of costs and tradeoffs there. But to the extent that you can find ways of deterring, mitigating, crime that don’t rely as much on the incarceration channel, I think that lowers the spillover negative effects on local communities, I want to mention that, when we look at these multi-dimensional impacts of the original incarceration event on the student, we actually find, consistent with prior literature, that to the extent that we can observe juvenile offenses, we don’t observe increases in crime, if anything, there are decreases in criminal activity. That, again, is consistent with some of the prior literature. And the way to interpret that, I think, is to again think of there as being multiple dimensions by which people are affected. So you can observe that there’s a negative role model effect, you observe someone going to jail for a crime… Well, I’m not going to commit that crime, but you may still act out in the classroom. So we shouldn’t think of the responses to these kinds of disruptive background events as happening on some uniform spectrum of good behavior or bad behavior, but it’s much more multi-dimensional in how people respond to stressful situations.

John: Did you find a difference in the effect whether it was a male or a female in the household who is incarcerated in terms of the impact on children?

Arpit: Everything I’ve said, so far, I’ve been trying to be careful in sort of saying , these are individuals in the household, because really, what we’re doing is the household level match. So we’ve got the address, and so what we really know is that this is someone that lives at this address that is arrested. We view that mostly as a strength of our approach, which is trying to identify household members. It sort of recognizes the intergenerational and complicated family backgrounds many families have, but it does make it a little more challenging to establish the sort of true relationship between individuals. And so one thing that we kind of did there is sort of try to identify probable female parents or guardians, male parents or guardians, or simply assign kind of age ranges and things like that. We did find the effects on children were much larger when we were looking at the incarceration of a female payment. So that kind of makes a lot of sense, if you think that mothers and female guardians kind of play uniquely important roles within the household. And when it comes to the child themselves, the effects were actually pretty similar between boys and girls.

John: In the US, we have one of the worst rates of intergenerational income mobility, might this type of an issue be one of the causes of that, in that in low-income communities where incarceration rates tend to be higher, it’s putting children in those communities at further disadvantage, which can have some long-term consequences.

Arpit: One thing I want to mention is where we’re kind of taking the paper is to adopt the community frame and think about other community outcomes that might potentially change as a result of incarceration. So I do think that probably one of the reasons that we have this, not just low on average in the United States, a low rate of social mobility in the United States, but also it’s very regionally varying rate of social mobility differences across the United States. I remember when the first Chetty map was released that showed the geography of economic mobility in the United States. My home state, North Carolina, is actually incredibly low for social mobility. And that’s surprising, actually, because North Carolina is where everyone’s moving to. It is incredibly economically dynamic, it has lots of job centers, but moving there is low cost of housing. It has a lot of features, which you might expect should be associated with high economic mobility. And in fact, like much of the south and very regionally varying patterns across United States, you actually observe pretty low social mobility. And I do wonder whether one reason for that is that we have these very high rates of incarceration across much of the United States. And that’s not an easy thing to just stop incarceration, because we all know that the system of criminal justice, that is also there to protect in low- income communities from the negative consequences of crime. So the public policy challenges of how to figure out what to do about this are really complicated. But we want to know why is it that people that grew up in the same state that I did, don’t necessarily have great opportunities compared to people who grew up elsewhere. So we’re hoping to use the setting, use this analysis to dig a little bit deeper into this question. And one fact that is kind of already out there that I think is very related, is that analysis by Chetty and others, which looked at the geography of social mobility, found that a big correlate, something that associates strongly with social mobility across United States is the presence of two-parent households. So the number of absent fathers, that associates very strongly with the lack of social mobility in an area. Of course, that is not a causal statement, you could imagine things go the other way. So lack of social mobility kind of impacts in different ways. But I think that’s a diagnostic that is suggestive of the idea that something about incarceration affects broader communities, affects the family formation, affects family stability in ways that impact people’s ability to build stable relationships. And all of that kind of has really persistent negative impacts.

Rebecca: As an educator, this study makes me think about if I’m a teacher in a classroom, I’m kind of experiencing the phenomenon that you’re studying, and the kinds of things that I might consider doing for classroom management or the way that I might better understand even just what’s happening or what I’m observing, I think is food for thought for educators to just be more aware of what’s happening in their communities.

Arpit: The other kind of question, I think, for economic policy is about these measures of teacher value add, which are being thought of as ways of assessing or even compensating teachers for the increase in test scores, that they’re resulting in the classroom that they have, right? And this makes sense to economists we want to value and grade people based on the incremental add that they’ve done to a population kind of coming in. But one thing I actually hear a lot from teachers is they’re very worried about this possibility as something that affects them as a teacher, because they’re saying, “Well, it’s not my problem, if I happen to have a classroom in a particular year where the children are going through a lot of stuff at home, they’re not necessarily going to learn as much, that might affect other children in the classroom as well. And that’s something that I will potentially be judged for, something outside of my control.” And that is a very strong problem for this whole teacher value add methodology, because these kinds of background events don’t necessarily follow a predictable sequence. And so they can kind of happen at various times over students’ lives, over a teacher’s career across different classrooms. And so it’s very hard statistically, to separate out whether a student is doing well or badly because of the teacher, or because of some background events. It also impacts, I think, how we statistically evaluate and think about evaluating teachers.

Rebecca: I imagine it also impacts classroom management and observations of classroom management and other tools that we use to evaluate teachers currently… behavior in that class is different than others, or they have different traumatic experiences impacting their behavior. That’s not necessarily being observed by an observer.

John: And we have put probably far too much weight on teacher and school compensations and budget tied to student performance, because, as you said, there’s so much that’s outside the control of the teachers or the school districts.

Rebecca: That’s also the schools that tend to struggle to get teachers and things too, right?

John: And we’re penalizing those teachers and those school districts, often, that face the most severe challenges and need the most support. You mentioned this dataset from North Carolina is a very rich one, but you had to do a bit of work to get all that data together. Because there is a lot of data on student outcomes, but you also have to tie this to incarceration. Could you talk a little bit about how you matched the household data, or the incarceration data, to the schooling data?

Arpit: Oh, man, this is my favorite part of the project, because it allows me to reminisce about my sort of a Moby whale moment of a project. So I think all of us as researchers need to sort of think about what are the projects that we really want to see live, what are the ones that we’re really going to go to bat for, and this is one of those projects for me. I just felt that this needed to be answered. And so, together my collaborators, we really just spent a really long time trying to figure out how to get the right data for this. So you have to put together the criminal justice records for a given area, you need to put together the education records, and then you need to figure out how to link the two of these. So some states you can get one, some states you can get the other and it’s very hard to find a set of states where the two of them match. So we tried a whole range of states, a whole range of datasets, many times we got very close, but were stopped at the last minute. And finally, we were able to work with the state of North Carolina, which has an excellent set of education records, has these great criminal justice records, and were able to figure out a way of merging and matching the two sets of documents at this household level, have a pretty good sense of the direct linkages between the children in our sample and the criminal defendants and then using the classroom identifiers in the dataset to identify other spillover effects, looking at the broader geographic implications. So all of that wound up working out for us at the end, but it was a long haul to get there. And I think it’s definitely a lesson that I took away from this project that if you want something to do well, you really got to work at it. There’s no substitute for putting in the shoe leather for calling people, cold calling people, emailing people and just hearing no, no, no, no, no again and again and again until you’re able to figure out something that works.

John: And the matching between households for the students and the incarcerated people was based on household addresses. Is that correct?

Arpit: That’s right. So that match was done by the North Carolina education folks. They took their records, they imported the criminal justice records, matched that at the household level, and then gave us a data set that had removed all identifiers that we could work with for research.

John: It’s a wonderful data set and it’s a really impressive piece of work.

Arpit: Thank you very much.

Rebecca: So we always wrap up by asking what’s next?

Arpit: For us on this project, we’re really trying to see if we can think about some of these broader implications of incarceration on communities outside of the educational impacts that we’ve been talking about so far. So thinking about the impacts on family structure, thinking about whether it spills over into the usage of other government programs, whether it has employment effects, kind of housing market access, I think that there are a whole range of different outcomes, particularly at these broader community levels that I think are shaped by the number of people in that local community that are impacted by incarceration. So I think those are the overall community spillovers, we’re interested in understanding.

John: Well, thank you. This is some really impressive work. And I have to say I’m really impressed by all the work that you’ve been doing in so many areas. You’re doing some wonderful work on some really important topics.

Arpit: Thank you very much, John. I had an economics teacher growing up who inspired me to work on these topics.

Rebecca: Well, thank you so much. We’re looking forward to sharing this with our audience.

Arpit: Thanks.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

308. Design for Learning

We tend to design courses for ourselves because we are the audience we know best. In this episode Jenae Cohn joins us to explore how user-experience design principles can help us create effective and engaging learning experiences for the students we have right now. Jenae is the Executive Director of the Center for Teaching and Learning at the University of California at Berkeley. She is the author of Skim, Dive, and Surface: Teaching Digital Reading. Her newest book, co-authored with Michael Greer, is Design for Learning: User Experience in Online Teaching and Learning.

Show Notes

  • Cohn, J. (2021). Skim, dive, surface: Teaching digital reading. West Virginia University Press.
  • Cohn, J., & Greer, M. (2023). Design for learning: User Experience in Online Teaching and Learning. Rosenfeld Media
  • Global Society of Online Literacy Educators
  • Horton, S., & Quesenbery, W. (2014). A web for everyone: Designing accessible user experiences. Rosenfeld Media.
  • Web Accessibility Guidelines
  • Copies of Design for Learning may be ordered at the Rosenfeld Media website. The discount code for listeners is TEA20. It’ll be available on Wednesday, 9/27 and will give listeners access to 20% off the book for one month (i.e. 30 days).

Transcript

John: We tend to design courses for ourselves because we are the audience we know best. In this episode we explore how user-experience design principles can help us create effective and engaging learning experiences for the students we have right now.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Jenae Cohn. Jenae is the Executive Director of the Center for Teaching and Learning at the University of California at Berkeley. She is the author of Skim, Dive, and Surface: Teaching Digital Reading. Her newest book, co-authored with Michael Greer, is Design for Learning: User Experience in Online Teaching and Learning. Welcome back, Janae.

Jenae: Thank you. I’m so glad to be back.

John: It’s good to see you again.

Jenae: …Good to see you, too.

John: Today’s teas are… Jenae, are you drinking any tea?

Jenae: I sure am. I’m always prepared to drink tea. Especially when I’m talking to the two of you. But I went for a classic English breakfast tea this morning. Do you both have some tea with you?

Rebecca: Yeah, I have English tea time.

Jenae: We’re matching…

Rebecca: Yeah…

[LAUGHTER]

John: And, I’m not. I have [LAUGHTER] ginger peach black tea today.

Jenae: That sounds really good, though.

John: It is.

Rebecca: Sounds like a good way to start the day, for sure. So we invited you here today to discuss Design for Learning. Can you talk a little bit about how this book project came about?

Jenae: Absolutely. So my colleague Michael and I have a lot of shared interests. Michael and I both are trained in rhetoric and composition. And we both are people really interested in online writing, online reading, and online learning, broadly speaking. We both served on the board for the Global Society of Online Literacy Educators, which is an organization dedicated to supporting folks who teach reading and writing online, broadly speaking. Through that organization, we got to know each other better. And we just realized how much we wanted to talk about what it really meant to create quality online learning experiences. And something that kept cropping up for the two of us. And I should say that both of us have had like a hodgepodge of jobs in and around higher education. We kind of joke that we were both sort of like these misfits in higher ed, people who have kind of done a bit of teaching, a bit of admin. He’s worked in publishing, I did a lot of work in instructional design and just higher education pedagogy. And something we noticed, just in the various roles that we were in was that educators, professors, faculty could learn a lot from user experience frameworks. And we were reading a lot about UX and UI in the work that we were doing around instructional design and for him publishing, and it just dawned on us like, why are we not bridging these conversations between the work of thinking about designing learning interfaces and the work of building really good, high quality learning experiences. I think we notice that in higher ed, there is this tendency to kind of try and reinvent the wheel around defining what a good teaching experience, especially what a good online teaching experience is by just creating really kind of exhausting templates and tons of checklists and rules. And we really thought those are useful, but wouldn’t it be more useful just to remember that students are people navigating devices online? And can’t we use the frameworks that help inform those design decisions to inform the design of learning experiences to make them better? So that was really the genesis of this project. We started off thinking we’d write a bunch of blog posts and then it struck us that blogs and articles were great, but wouldn’t it be even better if we wrote a book [LAUGHTER]. So we put it all together, and it resulted in this book.

John: So who’s the intended audience of this book?

Jenae: We really are targeting a broad audience with this book, I’d say primarily folks who do instructional design style work in mind. So in higher ed, that could be faculty, a lot of faculty play the role of instructional designers, as well as facilitators and teachers, of course. But, we also hope that this book would really reach folks who do dedicated instructional design support. We also hope that this would just reach people who are having to teach online or do trainings or workshops online, and who are still really struggling with it. This book, I would say, was written before the pandemic happened. We were, I would say, drafting and conceptualizing it before the pandemic. And of course, the pandemic shaped the drafting as we went, there’s still COVID-19 out there, so I don’t want to say we’re beyond the pandemic. But in this moment where we’re beyond perhaps like a peak point of the pandemic, let’s just say. There may be folks who are still wanting to be more intentional about what it means to provide more equitable access to online learning experiences, who want to be designing in a more intentional way, and who want to be really thinking critically about how to create more sustainable online learning experiences, as well, that really work. I think we were also on a mission with this book to prove that really, anyone can do this, you just need to keep some known principles in mind that, again, this is not totally new territory, and scholars and user experience and human computer interaction have been thinking for a very long time about how to make information accessible online, and how to make sure that information and interactions are easily navigable. And so that was really the literature we wanted to tap into. So that’s all to say that I think people who benefit from reading this book is really anyone who wants to be creating a better online learning experience for whatever teaching situation they’re in.

Rebecca: I’m, of course, super excited about this book, because I’m a UX designer. I love that you use that framework to write this book. Can you talk a little bit about why you chose this approach?

Jenae: Absolutely. I’m so glad that you appreciate this book exists. We’ve gotten really good reception from the UX community on it as well. I would say that we use this framework because we felt like it really centered the learner in an important kind of way. I think that in a lot of teaching situations, people who educate or design learning are often more thinking about the content: What information do I have to deliver? What are the main things that I need to make sure people know how to do? Those aren’t bad things to focus on, we need to cover content, and we need to make sure that there are clear outcomes. But I think it’s most important to really think about how is someone engaging with that content? How are they understanding it? What are their opportunities to understand that content in a variety of different ways. And I think what a user experience framework allows us to do is to center that reminder. Learners have these embodied experiences that shape how well they’re going to be able to learn, how well they’re going to be able to interface with the information. And if we’re talking about that in an online context, in particular, it’s impossible to do so without addressing what it means to, again, engage with and use these online environments effectively. So I think a UX framework really just allows us to be more centered in reminding ourselves who really benefits from the learning experiences we design, and who really needs to have access to [LAUGHTER] the information to be successful. And I think UX frameworks just really help us center that.

John: Can you talk a little bit about how this approach centers the user in terms of practical ways in which that’s built into the design process?

Jenae: Sure, one way to sort of think about that is to really take a step back and try to remind yourself just who is taking your class in the first place. Starting there, starting from the place of trying to be curious about who your learners really are. I think that it’s easy to make assumptions, I’ll just say in higher education, in particular, since I think that’s primarily the audience for this particular podcast. I think a common misconception, for example, is that all students entering their class are traditional college age, 18 to 21 years old, but like, I should put a big asterisk on that and say, that’s probably not the traditional age at most institutions anymore. But that’s the stereotype of kind of who a college student is. And there may be some assumptions about what their prior learning experiences were like that brought them into a college classroom… about the prior knowledge that they had. And so what I think a user-centered design encourages us to say, “Do we know that? How do we know that? What information do we need to gather to remember who’s actually coming into our rooms?” And I’m not suggesting that any educator has to like, do deep dive demographic data work to find out who their learners are. But I think most of us can kind of anticipate the range of people who are coming into our classes. We might anticipate just the different types of learners that we may encounter. And by that, I mean, it’s worth, I think, before you start designing, just trying to remember, what are the different motivations that students have for coming into the class? What are their purposes for being there? What are the main things that students are going to want to do by being in your class or your training or your workshop at any given moment? So starting by just sort of trying to map out who those people are? And then try to anticipate, okay, given this motivation, or this purpose that this learner may have, what kinds of things might they be looking for… literally looking for my online course? What things will they click on first? Which links are they going to want to access most frequently? Which resources are going to benefit them on the site most? And then trying to design your learning management system course site, or if you’re not using learning management system, your course website, broadly speaking, to really privilege the resources, the links, the activities, the pages that are going to be best aligned with what you anticipate your users or your learners may need. And Rebecca, I’m sure, can speak to this given her expertise, too, but UX design really is a whole process of trying to consider how the visual information, how even like the tactile information, say how your keyboard was set up, how your device is set up, how that allows you to most easily use and engage with products, so to speak, that you’re building. And in this case, we want to think about how can you build the best online course that you can, in a way that allows users to most easily find the information you anticipate they will most frequently need?

Rebecca: So one of the things I’m hearing you say, is really thinking about the wide variety of learners that we have and the different needs that they have and trying to address that. One of the things that’s really popular in UX design and that you talk about in your book are personas. Can you talk a little bit about how learner personas can help us think through the different kinds of learners that we have in our class in a really practical, tangible way. You just kind of provided that theoretical framework, but I love that the personas is such a practical application of that.

Jenae: Yes, thanks for asking that. Rebecca, I was debating whether to dive into that with the last question. But let’s dive into it now. So for those who aren’t familiar, personas are an exercise where you really try to create a character sketch, I would say, of the user you’re imagining is going to engage with your course or in this case, try to imagine an example of a student who’s going to be in your class. And by creating a character sketch, I mean, I encourage instructors, if they have the time to sit down and say, “Okay, what might be the name of someone in my class? What might be their age? What might their prior experiences with learning my topic have been? Why are they here? What brings them to college? Or what brings them to this class in the first place? What are going to be some of their biggest challenges? What are going to be some of their biggest hopes? What are the things that they’re going to be most excited about doing in this class?” And again, it’s a bit of an imaginative exercise. And so I think it’s easier to do with more teaching experience. But it’s also not impossible to do even if you’ve had relatively limited experience. It’s really just an exercise in trying to think through who might be the real people that you are engaging with, I do want to say that there’s been a lot of conversation in the UX community, and again, Rebecca, you may have some thoughts on this, too, about sometimes the stereotypes that personas can perpetuate. For example, I think there have been concerns in the UX community that when you try to characterize, say, an older user, of an online interface, a stereotype might be that they struggle more or are more challenged with using technology than, say, a younger user. And that that might be a challenge to anticipate. And so I want to be mindful, for example, that if you are going to be in the practice of building personas, which we talk about in the book, because I do think is a useful exercise to kind of try and make concrete for yourself who is going to be on the receiving end of your experience, that you do try to check yourself a little bit on reinforcing stereotypes to the best of your ability. It’s easy to do, stereotypes exist because we notice patterns sometimes in how people behave. And that can sort of reproduce some harmful assumptions about who those users are. But again, to the best of your ability, attempt to anticipate what the needs might be based on what you do know about who might be in the room, just again, kind of reminding yourself that you’ll want to think about your personas in nuanced ways, and not necessarily make assumptions about who they are. And I would say one solution to that, how am I supposed to write a generalized description of a persona, while avoiding all possible stereotypes about who they might be? I would say again, time allowing, try to run your personas by other people, and just see what their reactions are to reading them. For example, if you have a trusted colleague, or a friend who teaches a similar class, or who you work with regularly, just show them what you’ve drafted and say, “Does this feels like a real person to you?” And to attempt to ask diverse people about how your persona sketches are landing or how realistic they feel to them. That’s always a good way to kind of gut check, and just make sure that as you’re anticipating your users’ needs, you’re not falling too much into your own biases about who the people are that you’re supporting in your course.

Rebecca: So one of the things, I think, people do sometimes run into when they’re making personas is to create the ideal student that doesn’t exist, and also to recreate themselves. And so one strategy that I often recommend is thinking about creating aggregates of people that you do know. Because then they’re more realistic in terms of the way they might interact. So if you’ve taught a class before, you might have a real pool of people you could draw from [LAUGHTER] and to create a persona from, obviously, that’s more difficult when it’s a new place. And I was also going to offer up in terms of thinking about disability and thinking about accessibility, that there’s a book called A Web for Everyone, and they have a lot of resources. It was published quite a while ago, but they have a lot of resources still online, they have some personas for people with a wide range of different kinds of disabilities. And sometimes that can be really useful in just thinking through kinds of scenarios that you might not think of on your own.

Jenae: That’s fabulous. I would love to see that resource about sort of supporting accessibility, especially. That’s such a huge issue in designing online learning experiences, particularly. I’m so glad you mentioned those resources. That’s fantastic.

John: And while there may be those types of biases that you might have, those who’ve taught classes multiple times do know some of the types of problems that past students have had. So those issues that they’ve experienced in the past could be built in. But one of the other things you suggest is doing a pre-course survey, so that you get some more information about the actual students in the room rather than those who may have been thinking about when you initially designed the course? Could you talk a little bit about that survey?

Jenae: Yes, I’d be happy to talk about the pre-course surveys. So this is a practice that, I think, has multiple benefits. So in a pre-course survey, I think instructors have this wonderful opportunity just to ask students what their motivations are for engaging with the class, what brought them here, how they would characterize some of their prior experiences with learning similar topics, if any, and just to voice what concerns they have, or what things are exciting to them about the term ahead. I’m giving a lot of examples of possible questions and I just want to acknowledge that not all instructors will want to ask all of those questions all at once. But those kinds of questions that really get at motivation and concerns, I would say, in a nutshell, can be really critical, both for adjusting, I think, those persona expectations. So, creating personas should be an iterative process, I should say, as well. It’s not a one and done thing where you anticipate who your learners are prior to the course starting and then you’re like, “Okay, I figured it out, I know who all the students are. Knowing who the real students are, can then allow you to go back to what you anticipated. I think, and both of you, Rebecca and John, were speaking to how you could use prior information from prior terms to inform your kind of current term or current course. Great, you could sort of just align your prior understanding with this current information you might get from these surveys to then go into your course website, or your course learning management system, your syllabus and say, “Okay, is this design going to work for the group of people who are actually here based on what I’m reading?” …recognizing, of course, that nothing’s gonna be perfect for everyone. But you can do the best you can to try and make the materials as good as possible for the group that you have. In front of you. I would say that you want the survey to feel less burdensome for your students to complete. I’m giving a lot of examples of questions that I think are ideal as open-ended questions. Some of these, you could turn into multiple choice or kind of Likert-scale style questions, because you can just use it as an opportunity to take the temperature. “On a scale of one to 10, for example, how confident do you feel in your understanding of your ability to pick up new quantitative concepts?” …for example, if you’re teaching in a STEM-style discipline. Or “On a scale of one to 10, how comfortable do you feel as a writer or with writing tasks?” …if you’re teaching something more humanities- or writing-centric. You can get really creative in trying to solicit some feedback. And I also encourage instructors to be judicious in what they’re asking in these pre-course surveys to kind of try and ask questions, with the end goal of helping you as the instructor make small tweaks to the design of the course. Think about this information as a way to say “Okay, are there certain links I should put on the homepage that I didn’t think needed to be on the homepage? Or should I reorganize the menu on my learning management system in a way that highlights some resources more than others based on the information I’m getting in the survey? Should I reorganize a module to introduce some content before other content, because I’m seeing a trend in the surveys about less confidence in one area of the course than I was expecting in another?” So thinking about how the answers might inform your design, a research-based perspective really, I think, can make your course really even stronger. And I think it’ll feel better, both for you and the students because it helps the students see that you’re curious about them, you want to know who they really are. And we know that engaging personally with people really matters for good teaching. But the instructor too, it can be really frustrating. If you design something and it doesn’t land with your students. You feel like you spent a lot of time building something that didn’t work. That’s a really disheartening experience. So getting the feedback might allow you to avoid [LAUGHTER] having or feeling so disappointed if the information didn’t land the way you were expecting it to. And this isn’t foolproof. There’s always room, again, for iteration. But I do think the surveys can at least help you anticipate a little bit better how the progression through your course could go.

Rebecca: I can imagine that some of those surveys with open-ended questions could lead to better understanding how students name things or label things which could give you a lot of clues about the actual user design of a course by just how you might name or provide quick descriptions of things. In your book, you talk a lot about instructional text design, which obviously has lots of skills in online learning from instructions for assignments to just how we might label a folder [LAUGHTER]. There’s lots of skill there. Can you talk a little bit about the basic principles that you’d recommend for course designers to follow when they’re writing instructional text?

Jenae: Absolutely, and I realized, as you were talking and responding, I was nodding along. And then it struck me. It’s like, “I’m on a podcast, no one’s going to know that I’m nodding and agreeing with you right now.” So [LAUGHTER] for the listeners sake, like I was nodding along quite vigorously with that entire response. Instructional text, I think, is one of the most underrated and one of the most important things to design for any online course experience. I think that online course designers have a real tendency to rely too heavily on video and on images. There’s an assumption that if you’re working online, everyone’s just using video all the time, or everyone’s just wanting to engage with the flashiest multimedia possible. That is still important. I mean, we have two chapters in the book, all dedicated to video. So I don’t want to undermine that. It is important to engage with multimodal artifacts and building multimodal interventions, when you’re teaching in a multimodal environment like the internet. However, for students who may have low internet access and low bandwidth, for students with disabilities, text remains one of the most accessible and easiest ways to find information in an online course. I’d also say text is one of the most mobile-friendly pieces to think about. And we know that increasing number of students are accessing their courses or coursework through their smartphones. I’ll answer your question directly now, but I wanted to provide that context. I would say when it comes to designing instructional text, I really encourage instructors to think about two big things, to think about the hierarchy of the information that they’re writing, and to think about the discrete chunks of information that they’re wanting to communicate. So when I talk about the hierarchy of texts, I think it’s important when we’re writing to consider what are the sections of our text? Most academics, most instructors, are used to, when they’re reading or writing, creating headers, and sub-headers, and paragraphs that denote a certain order of information. And when you’re teaching online, especially, I think even more critically about how are you labeling the text? How are you indicating which things are instructions versus content? How are you labeling the order of the content that you want students to read in? How are you even labeling the order of instructions, like there is usually multi-tiered sets of steps. So using header text and different layers of header text, is a really important web accessibility measure. And again, it helps readers see visually and if they’re using a screen reader tool, it helps them navigate that text more easily. So I should take one step back and say when I’m referring to header text, I mean that when you’re working in a rich text editor, on any website, you can typically see an option to select different layers of headers, like the header ones are usually the highest, biggest level header, header twos go below that header, threes go below that. So just being mindful that just increasing text size is not the same thing as using headers is one really, really simple way to create hierarchy. And again, to denote the correct order of reading the text information. And when I say chunking text, this is as simple as just thinking about paragraphing, making sure that you are spacing out pieces of content in really critical ways. So anyone who’s read a piece of writing with super long paragraph knows, that’s a lot harder to kind of discern, it’s a lot harder to see how one idea moves from one to the next. Shorter paragraphs are typically easier to get a sense of when you’re moving from one idea to a new idea. And so even though long paragraphs have their purpose, perhaps especially in scholarly writing, or even in more, I would say kind of creative writing, in some cases, when you’re doing really instructional or technical work, which you’re often doing when you’re designing a class, shorter is better, more chunked is easier to access, because you’re assuming that people are doing things with your information. So those are the two qualities I would just be thinking about with instructional text. There’s a whole other component that we didn’t really address in the book, but I’ll just stick to very briefly here, which is also thinking about just the visual appearance of your text. A lot of accessibility folks speak to some best practices and guidelines around font face, and font size, and some of these factors when you’re designing text as well. I’m not an expert, I should say, in like type of graphic design or font size, but I want to point out anyway, because I think if you are designing online, it’s important again to do the best that you can to try and anticipate those needs. So I think as a general rule, making sure your font sizes are not super teeny tiny, or super large. Making sure that you’re using standard font faces: Arial, Helvetica any sort of sans serif font is typically considered a best practice. The rules around this change all the time, Web Accessibility Guidelines change as technology evolves, so I never like to give super hard and fast rules, and again, it’s not my area of expertise. But it’s another piece to keep in mind that visual and verbal information is intertwined. Text is a visual medium, online learning experiences are largely a visual medium, by default. And so the more mindful we can be of what that looks like, and the more mindful we can be of how the visual experiences we design online, are compatible with accommodations for disabled users. We just anticipate our users’ needs, our learners’ needs more proactively, and it raises the boats for everyone. It just gives everybody a deeper chance to succeed if we’re just thinking about these interface choices in more deliberate ways.

Rebecca: I love that you’re really talking about how the instructional text is also part of digital accessibility. It’s important to have plain language, it’s important to chunk your content and these sorts of things. So I’m really excited that you’re incorporating that into the work that you’re doing.

Jenae: Thank you. It is exciting. I think it’s one of these things that, when Michael and I were first discussing this book, it was a real lightbulb moment for us that there was such a robust literature out there that discussed all these great principles for making sure that online information was easy to find. And it just was striking to us that a lot of folks in teaching professions weren’t getting access to that information or exposure to that information. And we started thinking about this, again, prior to the pandemic, kind of in the mid 2010s. And even at that point, online courses were growing, mobile access was becoming a more common way that students were engaging with courses. So, why not tap into these existing sets of conversations that are industry best practices, for engaging with online interfaces, in spaces like higher ed, and in spaces just like learning and development, where these dialogues seem not to have met each other as fully as they could.

Rebecca: Our chief technology officer and I were having a conversation about some of these things yesterday as we’re talking about our student body is diversifying and that we have far more students with disabilities who are able to attend college and have access to college in a way that maybe they haven’t in the past. And as you were talking about headings and paragraphs and things, something that people might not know, is that if you use a screen reader, you’re not necessarily visually interacting with the text. Instead, you’re thinking programmatically, and so just like kind of vision centered [LAUGHTER], the user might skim headings visually, it’s the same way someone might use a screen reader. So by choosing a heading level two, it allows someone to find that section easier. And by breaking things into paragraphs, and delineating that’s that kind of content that allows a screen reader user to be able to jump to a particular part of the content. When we don’t do that, a screen reader user has to listen to everything from the top to the bottom of the page.

Jenae: Great example. Yes, and that’s such a frustrating experience to have to do that. If we can be just a little bit more attentive to the information architecture of sort of what we’re trying to communicate and convey… information architecture is a technical term, but it’s also a metaphor [LAUGHTER]… we have architecture and we have design to help create solid foundations for places that we live. Similarly, when it comes to information, we need to be building solid infrastructure to help people navigate their way through a course. One of my colleagues a while ago used a metaphor for online learning design I’ve never forgotten and we’ve alluded this a bit in the book, which is that when you’re building something online, it’s like you’re just building a whole house [LAUGHTER]. When you walk into an in-person classroom, the architecture is literally there, and you make assumptions about the room in the space, the second that you walk in the door. When you’re designing text online, or just when you’re thinking about the whole online learning experience, it’s a total blank canvas, you have to build that architecture and those hierarchies. If you’re not attentive, you’re absolutely right, the consequence is that it can be a big overwhelming mess of information. And I think it’s a useful practice for instructors, even when they’re not teaching online, to think about these things. It’s also just a great exercise and getting really very focused on what information do you want to prioritize when you’re communicating assignment instructions or when you’re picking out content-based readings for your students? What do you want them to focus on? What are the big things you really need them to learn or pay attention to? And so if your course design, your visual design can align with the hierarchy of choices you’re making as an instructor or the priorities that you’re setting, it just makes it easier for everyone to have equal access that information so that more time can be spent for students to focus individually on how they’re processing, applying, doing higher-order thinking with that information. They don’t have to spend so much energy just trying to intake the kind of basics before they have the opportunity to really work with it and apply it meaningfully,

John: You provide a lot of other information in your book, and we encourage people to read your book. If they want to find out more about creating videos, about providing effective webinars, and so forth, there’s some really nice hints and suggestions throughout. But one of the things you end with in there, is ways in which instructors can continuously improve their courses, in terms of soliciting feedback to make the course better each time. Could you talk a little bit about how you would encourage instructors to continuously work on developing their courses?

Jenae: Sure. So I really like that section of the book, because what I hope that section communicates is that thinking about your course design is a reflective and an iterative process. I don’t think a course is ever really fully perfect and done, there’s always things you can do, and modify each time you teach or offer the experience. So, I don’t think getting feedback on the course has to be hard, I don’t think it has to take a ton of time. We talk about multiple ways of getting information about how the course is working. And I’m going to start with I think some of like the easiest and most passive ways to get information and then we’ll sort of work our way to some of the more perhaps active or personalized interventions for getting information about the course. So one thing I think is worth really paying attention to, after you finish teaching a course, are some of the analytics that are available in your learning management system or your course website. And I recognize that some folks are really reluctant to look at the analytics, because there is a surveillance economy implicated in the tracking of course analytics. Every site on the web tracks your movements, every site in the web knows how long you’ve stayed on a certain page, what things you’ve clicked on. And a learning management system is no exception to that. Unfortunately, that information can get weaponized to discriminate against students, discriminate against users in problematic ways. In the web, outside of learning, for example, analytics can be gathered and sold to advertising companies to spread information about your activities for profit. So I just want to note that context, but you can also use this information for good and for some useful things as well. So seeing which resources students are clicking on the most in your class can be really useful information for you to say, “Huh, seems like a lot of people found that resource useful.” You don’t have to necessarily identify which individual students looked at which resources but you can look at this data, typically in aggregate, and again, most learning management systems have an analytics dashboard, you can access to look at this. I think that’s incredibly useful just to see what was clicked most often and what wasn’t. You might also want to track, for example, which pieces of information students did spend more time on. It could indicate a couple different things, it might indicate that something was really challenging, if students spend a lot of time on one particular piece of content over another or if they found it useful. You’d have to contextualize that data based on what you were seeing in the course. But I think if you’re willing to look at that information, again, in the context of how your term went, it might just give you some passive information that could surprise you. I would even look at, for example, with assignment submissions, how many delays were there on certain assignments versus others? In which assignments did students request more extensions more than others? Again, this is just information that might help you inform whether the pacing was appropriate for the course, whether assignments were sequenced appropriately. That kind of thing. If you want to get more active, if you gave a pre-course survey, you can do a post-course survey. Most institutions, of course, have formal evaluations of teaching, but we know that institutional student evaluations of teaching can be fraught. Sometimes they ask the kinds of questions we don’t always want to ask or find most useful as instructors. So if you do your own very brief post evaluation, you could focus it on the design of the course itself. I think it’s worth asking students at the end of the course, how easy was the course site to navigate? How accessible did the materials feel for your ability to learn? You could return to some questions from your pre-course survey. If you asked a Likert scale about rating your confidence with learning something on a scale of one to 10 at the start of the course, you could ask them by the end, “How does your ranking change?” Even referring back to the original data that they might have submitted to you with the pre-course survey. So those are another way to ask them. I think if it’s possible to, what I love to at the end of the course is even a little brief post interview with students if possible. We mentioned this a bit in the book. Again, it’s time consuming. But if you have a small-ish class where you could have conferences at the end of the term, and have a moment with just a five-minute conversation to ask students: “How did it go? What aspects of the course design did you like most? Which were most challenging to you?” That’s another way to get information. Finally, I’ll just do one more technique we write about in the book, which is never discount your own reflection on your experience as well. This is another form of user research. Even though you are not the end user for the course, you are the designer, and so I think it’s always useful just to jot down a few notes and treat that as research when you’re done, too. What did you notice about user interactions on your course site throughout the term? What things surprised you? What things went exactly as you expected? You can use those notes to iterate and improve your experience for the next time that you offer it. So those are just a few techniques and many, which again, are drawn from the field of user experience research surveys, and interviews, for example, are pretty common user experience research practices… other UX research practices that, again, just depending on your time, depending on your resources, it’s great if you can see students engaging in the course as well, asking them, just really seeing what it looks like for them to interact in the course. That’s a good way to get at good information about it. I just want to encourage anyone who’s teaching not to shy away from getting that kind of feedback, because it does make, I think, teaching more satisfying when you’re getting more information about what’s working and what isn’t.

Rebecca: So, you know this question’s coming…[LAUGHTER] We always wrap up by asking: “What’s next?”

Jenae: Yes, I do know it’s coming, and it’s funny, because I was thinking about it. What am I [LAUGHTER] doing next? So to be honest, I don’t have a clearly defined project, I’m doing a lot of little things, I might be taking a little break, because I have written two books in about two and a half years [LAUGHTER]. So that’s been a lot… wonderful. I think I’ve been bitten by the writing bug, for sure. And so I suspect there’s more writing in my future, but nothing immediately next. I’m still very curious about what it’s going to mean to keep designing really good online learning experiences in the future, I don’t think we’re done with that conversation. I’m really curious about how that’s going to evolve in the context of creating more inclusive and equitable learning environments for students. So I imagine those are topics I will continue to explore to some extent, but we will see how, of course, with AI too, and the impacts of that on online learning, I’m sure there’s gonna be a whole set of ways to think about these topics that will continue to evolve. So I’m kind of keeping my eyes open and my ear to the ground on how things are developing. And we’ll just kind of see what ideas emerge from there.

Rebecca: Well, it’s always a pleasure to talk to you, Jenae. Thanks for all the work that you do.

Jenae: Likewise, thank you, again, for having me and for engaging with these excellent questions. And if you listened to this podcast, we’ll put in the speaker notes, I’ll give you a little gift of a promo code. If you’d like to buy the book, we can give you a 20% off discount with thanks to Rosenfeld Press who published this book.

John: Well thank you. We’ll be sure to include that in the show notes and it’s always great talking to you.

Jenae: Wonderful, and likewise, thank you again.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

306. Gender Bias and Timing of SETs

 A number of studies demonstrate gender bias in course evaluations. In this episode Whitney Buser, Jill Hayter, and Cassondra Batz-Barbarich join us to discuss their research that looks at the timing of when these gender differences emerge and theories for why they exist.

Whitney is the Associate Director of Academic Programs in the School of Economics at Georgia Tech. Jill is an Associate Professor of Economics in the College of Business and Technology at East Tennessee State University. Cassondra is an Assistant Professor of Business at Lake Forest College. Whitney, Jill, and Cassondra are the authors of an article entitled “Evaluation of Women in Economics: Evidence of Gender Bias Following Behavioral Role Violations.”

Show Notes

Transcript

John: A number of studies demonstrate gender bias in course evaluations. In this episode we discuss research that looks at the timing of when these gender differences emerge and theories for why they exist.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Whitney Buser, Jill Hayter, and Cassondra Batz-Barbarich. Whitney is the Associate Director of Academic Programs in the School of Economics at Georgia Tech. Jill is an Associate Professor of Economics in the College of Business and Technology at East Tennessee State University. Cassondra is an Assistant Professor of Business at Lake Forest College. Whitney, Jill, and Cassondra are the authors of an article entitled “Evaluation of Women in Economics: Evidence of Gender Bias Following Behavioral Role Violations.” Welcome Whitney, Jill, and Cassandra,

Whitney: Thank you for having us.

Cassandra: Thank you so much.

Rebecca: Today’s teas are:… Whitney, are you drinking tea?

Whitney: I am. I have some jasmine tea.

Rebecca: Always a good choice. Jill. How about you?

Jill: Harney and Sons Hot Cinnamon Spice.

Rebecca: Oh, that’s such a good choice. I love that one. It’s a family favorite at my house. How about you, Cassandra?

Cassandra: Yesterday, we made a sun tea on the porch. So it’s sweet peach tea.

Rebecca: This is a good variety. How about you, John?

John: And I have ginger peach black tea from the Republic of Tea.

Rebecca: So we’re combining choices here [LAUGHTER]. And I have Awake tea, despite the fact that it is early afternoon here.

Jill: I also had three cups of coffee this morning.

Rebecca: It’s one of the most popular kinds of tea, Jill.

John: We’ve invited you here today to discuss your research on gender bias instudent evaluation of instructors. Could you tell us how the study came about?

Whitney: Jill and I have been working on this for about six years, believe it or not. It’s been a long process for us. And actually at the very beginning we had a different third working with us. And the original three of us, we met at the conference, and we had just attended a session that talked about teaching evaluations. And afterwards, we just naturally began talking about this, because we all had these really, really strong feelings about teaching evaluations. All three of us at the time were young, young in our careers, young age wise. We were female PhD economists. And we were all earning tenure, or I think Jill had just earned tenure. But we’re all in this similar experience of having what we felt like was a very positive class climate, and a lot of camaraderie between ourselves and the students until the grades were returned for the first time. And then we could feel a definite shift and it was upsetting to all of us. We all got into this because we love teaching and we want to do a good job in that. It was just something that we were picking up on. So that was our anecdotal experience, Jill had a little data on it herself, because she would do mid-semester evaluations herself, just to gauge the class climate and see what students were needing. And I had an experience where in my first position, they did a surprise midterm evaluation, just to kind of see how the new professor was doing, that I didn’t know about. And I got glowing reviews from the students, everything was very, very positive, wonderful and six weeks later, same students but grades returned, evaluations looked a little different. And the comments were a bit different. So we had a little data to backup this idea too, and one thing if the people listening today haven’t read the literature, there’s an extensive literature on course evaluations. And it consistently finds gender bias in those. But the thing about that literature is it only looks at evaluations, which are typically done on the very last day of class, maybe even after that, maybe a couple of days before, but at the end of the semester. And we really haven’t seen anyone look into how these opinions of students evolve over the semester, or how students feel at the beginning or the middle of the semester. So that’s what we wanted to do with that. And in my opinion, and this is just me speaking here, Jill can have her own other motivations, or our other co-author that has worked with us before could feel differently. But for me, it was really important to acknowledge that society has come a long way in the past several years with gender bias. And I don’t think that modern students are shocked by female faculty any longer, I don’t think they have an explicit distaste for female faculty. Anecdotally, I feel that my students are actually happy when they meet me. And they have expectations of me to be warm, comforting, approachable. But I do think that when you expect someone to be more comforting and approachable, and they give you a grade back, that’s not always an “A” in a difficult quantitative subject like economics, you can get a bit of a Grinch Who Stole Christmas effect. I thought it was going to be one way and now my expectations are taken down. We all know no one likes that dopamine depletion of having expectations not met. So, to me, if we’re going to talk about gender bias, we really have to talk about it in this nuanced way, so that it doesn’t get automatically dismissed by people who don’t see an explicit bias and then say, “Oh, hey, there’s nothing here.” And then the last thing that I think is really important here for the motivation for the paper is that we have this expectation that bias would grow over the semester. So if bias grows over the semester, that means the earlier in the semester you evaluate, the smaller the bias will be. And one thing that the literature is missing is a very concrete objective way to deal with bias. What we were hoping to find was: move the evaluations up in the semester a bit, and you minimize or eliminate bias and that’s a concrete objective. Towards the end today, we’ll talk about what we actually found and whether or not we knew that. But that was one of the motivations.

Jill: So that’s how the original paper found in terms of motivation, but then Cassandra, she is a PhD in Psychology until she had read and she was doing work in the area. And she had reached out to Whitney and I. She had read our paper, she had read the results of our paper. And so then a second paper with Cassandra takes a more psychology approach in terms of a lot of what Whitney is talking about and Cassandra is going to talk about it later, with respect to the role-incongruity theory, social role theory, and she’s going to talk more about that later. And Whitneys described the motivation of that first paper, the second paper takes a very different perspective and looking at it from a more psych perspective. Cassandra, you might want to chime in?

Cassandra: Absolutely. I think you summarized it well, I joined the paper, as Whitney and Jill were trying to find a home for it. And we thought that our interests, though coming from very different backgrounds ,would blend nicely for this particular topic, as there’s a lot of scholarship in psychology that looks at understanding reasons behind this bias. And so I was brought in to really help kind of think about how do we frame that in a way that might appeal to even a broader range of audiences.

Rebecca: At the beginning of the paper, and Whitney, you’ve kind of pointed to this today about being a young faculty member, you also noted in the paper that women are underrepresented among economics faculty, especially at the level of full professors. Can you tell us a little bit about the extent of this under-representation?

Jill: Women have earned more than half the doctoral degrees for over a decade. But particularly among tenure-track faculty are underrepresented. In the paper we cite 36% of full professors are females. In economics, that’s a smaller percentage, 17 and a half percent of full professors are females, in the area of economics, although 35% of PhDs in econs represent females. It’s a smaller percentage of female faculty receiving full professor rank in economics. That’s what we mean by that under representation. In terms of economics, specifically, it’s oftentimes left out of the STEM fields, and depending on which university or college that you’re out at, economics can sometimes could be found in the social sciences and in the arts and sciences, or it can be found in the business school. So at my institution, Whitney’s institution, I believe, and Cassandra’s I think we’re all represented in the business school. But sometimes, you know, economics wanted to put in there with the social science field, it’s not thought of as being this more quantitative, heavy subject, and it oftentimes is, it is by nature of it. And so females in those more math heavy classes, like the STEM classes. I think my students when I started off, and I think Whitney was getting at this, with us being more junior faculty members. I can considered by students peer, instead of the professor in the course. And that made it tough, because to Whitney’s point about that returning grade feedback and the perception that students had of me a day one versus midway through the course, I was now coming across as someone that was handing back maybe less than 100% or “A” grade. So in my business school, my principles of economics courses are required. They might not even want to be in there, but they have to be in there to get a business degree. Earlier on, that was a challenge I faced, I’m 13 years into my career. I’m going up for full professor this summer. But starting off was really a challenge. And I remember having female mentors in my graduate program. They tried to prepare me for this, they tried to say it’s going to be challenging early on, you’re going to have to go against some of these perceptions, alot of the perceptions that we measure in this paper..

John: To what extent is the underrepresentation of women faculty due to a cohort effect where women have become a larger share of PhD economists in the last few decades, but that was less true 20 or 30 years ago and how much of it might be due to the impact of gender bias on evaluations on career pathways for women?

Jill: Really what this paper looks at, the standard evaluations of teaching and the bias or potential for bias, that exists there. So I’ll just speak to that and that where I currently am, evaluations of teaching are weighted heavily for retention of faculty, promotion of faculty, tenure and promotion decisions. And then when we’re hiring new faculty, looking at any previous course evaluations and experience with teaching. At every level in academia, these are used as some gauge for teaching effectiveness. I think one of the questions that we’re looking at and accrediting bodies are looking at is whether or not this is the measure that should be used. And looking at different measures that might be options for measuring teaching effectiveness, we know that they’re flawed, that our study is showing that they’re flawed, but also previous literature has suggested that they’re flawed as well. And so the fact that for most schools, this is the single measure that’s being captured… and I know that it’s different depending on again, at my institution, some departments don’t give them a whole lot of weight in tenure and promotion decisions. But certainly, my experience in my College of Business and Technology that these are weighted heavily. And so in thinking about a junior faculty member starting off, when Whitney and I met at the conference, if my evaluations were lower, I’m putting a lot of time into my teaching and improving and bringing up those scores. My male colleagues, in discussion just with them, didn’t have the same experience that I was having with respect to these SETs. And so we think about allocation of time and resources as a tenure track junior faculty member, I’m putting more in what I would consider just catching up, getting those SET scores higher, so that it’s reflected in my tenure and promotion packet. And that’s less time that I’m allocating toward research or other things. That’s my view on it. I think Whitney has a couple other thoughts on that.

Whitney: One of the things we tried to make clear in the paper is that the literature is very clear that evaluations do have a gender bias. And if these evaluations are being used, and they are, in hiring decisions, annual evaluations, promotion, tenure evaluations, and merit pay raise decisions, then they’re being used at every single level of advancement. It’s not one small piece. It’s a piece that’s used throughout and very integrated late in the process.

Rebecca: You mentioned at the top of our interview that the second paper shifts more towards psychology, and specifically describes ways in which both social role theory and role-congruity theory may explain the bias against female faculty in student evaluations. Can you briefly summarize these arguments for our listeners?

Cassandra: So social role theory was a theory that has been put forth for decades by Alice Eagly, a very prominent scholar in the social psychology world, as well as her colleagues. And this has been used as a framework to really understand the complexities and origins of gender gaps in our workplace in particular, whether that be inequities and experiences, the expectations that are different for women, and of course, the outcomes such as promotion at work. Essentially, social role theory suggests that the reason we see these gender inequities today in society or that they originated from men and women being distributed into social roles based on physical sex differences, so that women biologically were able to have children, men, on average, were physically stronger, which those differences 1000s of years ago, had an evolutionary benefit to a well functioning society, people were supporting in the ways in which they were best equipped to do so. And the assignment of men and women into these roles led get them to adapt role-specific qualities and skills. So women who were bearing children were friendly, helpful, sensitive, concerned with others, kind, caring. We refer to these now as more communal qualities, and men and the provider, the protector, role led them to have attributes such as ambition, being assertive, authoritative, dominant. These are qualities that now we label as agentic. So while technology of course has since caught up and made these biologically driven role assignments unnecessary, society continues to see a division of labor along these lines in the modern world and society at large. And society at large still holds the belief that women do possess these traits, and should possess these traits, these more communal qualities, and men do and should possess more of these agentic. Relatedly, role-congruity theory helps us understand the consequences when men and women fail to fulfill these expectations. And we know the failure to fulfill these expectations are more consequential for women, this experience of bias driven from the failure to behave in communal ways. In other words, violating these cultural expectations can be seen in all areas of society, but particularly in traditionally male-dominated positions, like college professors, or in male-dominated fields like economics [LAUGHTER]. And so women that are in these roles are already going to experience some degree of backlash for being in gender-incongruent positions. But that is especially true if they are also going to behave in traditionally more agentic ways, being more assertive, demonstrating their power, which we argued was what was occurring when you give critical feedback back to students.

John: To approach this, you gave evaluations to students at two different points of the semester. Could you tell us a bit more about the study design, how large the sample was and how many faculty and institutions participated in the study?

Whitney: Sure, we had a really rich data set for this study. That’s one of the reasons we were able to get two different papers out of it, and maybe even some future research, because we took all of this data, and we collected it in person on paper and entered it, which was an arduous process. As I said, we had been working on this project for about six years, about a year and a half of that was just data collection. And we have a lot of people to thank that did that for us for no author credit on this paper, so we had males and females across the United States gathering that data for us, that we’re really appreciative to have. So in the end, we wound up with about 1200 students in total, we weren’t quite 50/50, we were 60/40, favoring men, which is typical for economics classrooms, even though it is required in a lot of majors (that’s where you’re getting a lot of the women taking it). And like you said, John, we surveyed them twice. We surveyed them on the second day of class, we wanted as close to a first impression as possible without having a major sample issue with drop/ad. And then we surveyed them the day after they got their first midterm grade back. So we got the first impression, and then we got the way that they felt after they had had their first grade returned. We did this at five different colleges and universities, we had three male professors contributing data and four female professors contributing data. One of the big questions that people have asked us over the time is “Well, how does race play into this?” And that’s something that’s beyond the scope of our research, I will say that we only had one underrepresented minority in our sample, again, typical of economics professors, it was one of our male instructors. So, we would expect a downward bias from race and maybe an upward bias from gender, or getting those two, at least watching one another out in the paper. And when we asked these students about how they felt after their grades were returned. This was about four weeks into the semester, so still pretty early in the semester. What we did was we really wanted to ask about the specific qualities that had been hypothesized in the literature as drivers of bias or drivers of differences. So we just asked students to rate their instructor on a bunch of different qualities. Cassie really helped us out here because she came in and she says, “Well, you know, we can categorize these qualities into communal qualities and agentic qualities and neutral qualities…” which was really the way to approach it because of course, we get different things in communal versus just qualities. So we asked our students things like: “How knowledgeable do you find your professor? How challenging? Do you find them to be approachable? Do you find them to be caring? Are they interesting?” And then we asked a couple of very general questions: “Would you recommend the course?” All of this set us up to have a really nice dataset where we could look between genders and across time as well.

Rebecca: So I think everyone’s probably dying to know exactly what you found. [LAUGHTER]

Jill: I’m just going to provide an overview of the results because we do a number of different specifications and use different econometric methods in the findings. And so you can get all of those results there in detail. But in general, on the second day of class, we find that women are receiving lower ratings across the five agentic and gender-neutral instructor characteristics that we measured. They were rated higher on that second day of class on those more communal characteristics. And not all of those differences were statistically significant. Immediately after the first exam grade was returned to students, women were receiving lower ratings for all seven measured characteristics. Each difference was significant except for those caring and approachable, more communal characteristics. And then men were now having higher ratings in all the different aspects relative to time, or the second day of class. Over time, what we see was that men’s evaluations were getting higher on all characteristics from the second day of class to the period after the first exam was returned. And then in contrast, women’s evaluations were not trending upward. So we had a couple that were staying the same, but overall, they were going down. So those are just some overview findings. Again, those more specific results, by specification, can be found in the paper.

John: We will include a link to both papers in the show notes too, so people can go back and review them. To summarize, what you found is there was relatively weak evidence of significant gender bias on the second day of class, but that gap increased fairly dramatically after the first graded exam. So what do you attribute that change to, was it because of the feedback students were getting from grades as Whitney had mentioned before?

Whitney: We were attributing, and Cassie can talk about this with more authority on the theoretical point, but we’re attributing that to backlash theory, this idea that if I expect one thing, and I don’t get it, there’s this need to back off so that things go in congruence.

Cassandra: Exactly, Whitney is spot on there. What we thought this was evidence of was women behaving in gender incongruent ways, women are supposed to be warm and caring and friendly. And when you get a perhaps grade that maybe wasn’t an “A,” that feels harsh and critical, and a woman is asserting their power and dominance in the classroom, which again, they already are in a male dominated field profession. And those two things together combined can result in this backlash.

Rebecca: So if we take these findings, and think institutionally, what are some things that institutions might want to think about moving forward?

Whitney: That’s a good question. If you remember, from the very beginning, we were saying, we’re really hoping to find this nice objective concrete solution, we anticipated finding it through timing. And that’s what I would really like to do with future research is to be able to find something concrete and objective to treat this with. We weren’t able to do that because we found bias from the beginning. And we found that it came so quickly in the semester that it’s not something that we can just move back evaluations to midterm or something like that. Since we can’t do that, we’ve talked about other ways for institutions to take this. And one takeaway really is just an awareness that these gender biases exist and that these evaluations are flawed. This is really well established in the literature, but not necessarily in the general sphere of knowledge. When we published this paper, Georgia Tech did a little feature in their daily digest, and I had two female engineering faculty email me and say, “I knew this in my gut for years, but nobody’s ever quantified it.” That to me, is just evidence that it’s not in the general sphere of knowledge, even though the literature defines it well. Some of the impact of the concrete solutions that we have seen is we’re seeing a lot of schools and accreditors, like AACSB, they’re starting to require multiple indicators of teaching effectiveness and evaluation. So evaluations and peer reviews, or maybe something else to see the observation, something to that effect to where we have more of a global and inclusive way to look at someone’s teaching effectiveness. So this is a great takeaway, hopefully that will reduce the weight of the impact of evaluation just by having other factors in there. And just one final point that I want to make. And this is just a really big sticking point to me for the paper is that all of us are researchers, we all deal with statistics and statistical significance, and robust research methods. And then when those of us in Chair and Dean roles go to look at evaluations, all the sudden, all that training completely goes out the window, and we look at the difference between a 4.2 and a 4.4. And I know those differences sound really small, they are that small. And we say, “Oh, well, this person does better than this person, this person deserves to be hired over this person.” Never in our research, or in a formal presentation, would we ever compare two means that small without significance testing, number one, and without making sure they’re actually comparable, and say, “Oh, there’s a difference.” It’s just something that I think we need to recognize, we would not recognize this as good research or good methodology in any other area of our work. It’s just something that we should keep in mind as we move forward with this.

John: Now, you mentioned the use of peer evaluations as another way of providing, perhaps, more balance, but might they be subject to the same type of bias?

Whitney: Yeah, all the things that we would see for student evaluations, I can imagine how you would see with peer evaluations as well.

Jill: But there are creative ways to do peer evaluations that I think here at ETSU, we have a Center for Teaching Excellence. And I’m confident Georgia Tech, and Lake Forest has their own version of that. And so there are creative ways. And again, not that SETs are necessarily bad, but knowing what we know about the flaws in them, that, coupled with an additional measure or two, can be a lot more insightful, I think, to the teaching effectiveness, like true teaching effectiveness of instructors.

John: And one thing I’m wondering is if the measured effect might be larger in economics, because at least at many institutions, grades and economics and STEM classes are often lower, which might magnify the effect of this difference. It would be interesting if there was to be a study that also included some classes, maybe in humanities, to see if perhaps there’s less of an effect because of that role-incongruity issue there. It may not appear to be as severe in disciplines where grades across the board tend to be higher.

Whitney: I think you’re right about that, most people when they take economics, it’s a required class and certainly the grades are a big factor, then the two things that showed the most significance outside of our key variable of interest was interest in economics, and expected grade. Those were the things that across the board… now we still found gender bias controlling for those things, but it mattered.

Rebecca: So we talked a little bit about things that institutions might want to start thinking about: institutional policy and things that might shift how we use teaching evaluations. Are there any other strategies that institutions or instructors can use, or adopt, to try to reduce this bias in the short term?

Cassandra: That’s really the million dollar question. Because this type of bias exists in a lot of different domains, whether we’re talking managers and their subordinates, teachers and their students. One thing that’s often suggested or recommended is simply making people aware that this bias exists, and providing training on how to better approach evaluations, whether that’s how to use a rating scale and ensuring that you aren’t engaging in a halo effect, for example. Another strategy is requiring that people justify their ratings that are provided with qualitative comments… that if you’re just asked to fill out on a scale, on how competent is this person? Well, bias may creep in more if you aren’t asking for a justification of why that particular rating was given for competence. A last recommendation that I’ll share here is making these evaluations more public. So if there are a couple of people, say peers, that are evaluating myself or Whitney or Jill in the classroom, well, they need to come together, share and publicly disseminate their evaluations that they had given to us. This social accountability can help to mitigate bias and for people to ensure that the ratings that you’re giving are, in fact justified.

John: So we’ve got a long ways to go with this. It’s a problem that’s been recognized for quite a while with a lot of studies. But there hasn’t been that much done to address that. And those are some good suggestions that institutions may want to try. We always end with the question: “What’s next?”

Cassandra: [LAUGHTER] That’s a good question. Of course, I think that the three of us collectively would say we do hope that administration and decision makers start asking questions about their use of student evaluations of teaching and how they might seek to mitigate this bias, based on the recommendations Whitney had already shared. But we also hope that women faculty perhaps feel more empowered to advocate for themselves when it comes time for promotion and tenure decisions to be made. My Institution, a part of the promotion process, is writing letters, and going through interviews. So speaking to this, bringing an awareness to the people who are making the decisions that this exists, and that it is not just an opinion, that there is empirical evidence of its existence. But we are really interested in exploring more fully how providing feedback, particularly critical feedback, like in our study, where the professors are giving back grades might impact the perceptions of men and women in other contexts as well. So is this a phenomenon we would see, for example, between a manager and their team? Do people respond differently to critical feedback from a manager because of their gender? And how much are these differences, perhaps, driven by perceptions of how communal or agentic they are in their delivery of that feedback? So in other words, are we seeing the same pattern in other contexts? Ultimately, we hope that by better understanding how perceptions of communion and agency impact interactions that women have at work, particularly women in male-dominated or gender-atypical roles, this greater understanding will allow us to also discover ways to alleviate some of that backlash through more targeted interventions and training and perhaps better timing. Because at a minimum, it’s important to highlight the various ways gender bias continues to persist in our society. Because without that awareness, nothing can be changed.

John: Whitney, Jill?

Jill: I think that was great. [LAUGHTER]

Whitney: Yeah, I think, Cassie, you did a great job. And Cassie certainly helped us out with bringing formal language and theory to things that we felt as intuitive and we felt in our gut as important. We don’t have a lot of language for that in the economic space. And so blending these two disciplines together has been very helpful for looking at the situation.

Rebecca: Well, thank you all for joining us. And the research that you’re doing is really important and impactful. So we hope our listeners will use it.

Whitney: Thank you so much.

Cassandra: Thank you.

Jill: Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]