256. Sharing Our Stories

Students do not always recognize the expertise of faculty who do not match their cultural stereotype of what a professor looks like. In this episode, Sarah Mayes-Tang joins us to discuss how she has used personal narratives to address these student biases. Sarah is an Assistant Professor in the Mathematics Department at the St. George Campus of the University of Toronto. She is also the author of a chapter in the Picture a Professor project, edited by Jessamyn Neuhaus.

Show Notes

  • Neuhaus, Jessamyn (forthcoming, 2022). Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. West Virginia University Press.
  • Peterson, D. A., Biederman, L. A., Andersen, D., Ditonto, T. M., & Roe, K. (2019). Mitigating gender bias in student evaluations of teaching. PloS one, 14(5), e0216241. (A study that suggests that reminding students of bias in course evaluations may reduce bias.)
  • Perusall
  • Ogawa, Y. (2009). The Housekeeper and the Professor: A Novel. Picador.
  • Borges, J. L. (1998). The library of Babel. Collect

Transcript

John: Students do not always recognize the expertise of faculty who do not match their cultural stereotype of what a professor looks like. In this episode, we examine one professor’s strategies to address these student biases.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Sarah Mayes-Tang. Sarah is an Assistant Professor in the Mathematics Department at the St. George Campus of the University of Toronto. She is also the author of a chapter in the Picture a Professor project, edited by Jessamyn Neuhaus.
Welcome, Sarah.

Sarah: Thank you so much. It’s wonderful to be here and to see both of you. [LAUGHTER]

John: Thanks for joining us. Our teas today are… Are you drinking tea?

Sarah: I am drinking tea. I have a…

Rebecca: Yay!

Sarah: …I wouldn’t miss it. …it’s a chocolate mint black tea by Sloane tea. They’re a Toronto tea company.

Rebecca: Awesome.

John: Very nice.

Rebecca: I have a very standard [LAUGHTER] English breakfast today.

John: And I have a Prince of Wales tea today.

Rebecca: Oh, I haven’t had that in a while. John. We’ve invited you here today to discuss your chapter in Picture a Professor. Your chapter’s entitled “Sharing Our Stories to Build Community, Highlight Bias, and Address Challenges to Authority.” Can you tell us a little bit about this chapter?

Sarah: Yeah, sure. I think that my chapter might be the most obvious kind of strategy in this book. So a lot of the authors are sharing really inventive, or new strategies that I hadn’t thought of. Mine is all about just talking to other people about the challenges that we face when we don’t look like other professors in the academy, or at least what students might picture as their idea of a professor, what might you picture when you Google a professor? So my strategy is all about talking to people. First of all, starting by talking to colleagues, in particular, colleagues that might face similar challenges. So first of all, I should say, I’m a white woman, so I can’t speak to the full challenges that, for example, people of color might face, but I’m a math professor, and I present pretty feminine, and I teach mathematics in I guess, like a pretty serious math department. And so I certainly don’t look like what students expect when they come into a big math class. So for me, and I think for a lot of other people that I work with, it really came as a huge shock, when students started to question even my basic mathematical ability, 18 year olds dealing with probably their own insecurities about mathematics, but it was coming out as like, she doesn’t know what she’s doing. And then the reaction from my superiors who are mainly white men, would be to act more authoritative, basically act ways that were more like them. And the way that I felt was just like, there was something very, very wrong with me. I felt very ashamed. And even though I sensed that it had something to do with my identity, I knew they wouldn’t question me in the same way, if I was a typical looking professor, I also thought I did have to change something about myself. And there’s such tremendous shame in that. And it wasn’t until I, at the end of the year, whispered a little bit about it. And then another colleague said this exact same thing happened to me, the exact same thing. And the whole year, we were going through parallel experiences. And knowing that changed my life, it changed my profession. I would have left the academy if it hadn’t been for that. And then over time developing a group of cheerleaders who I could go to, and then kind of gain more confidence. My chapter also addresses being able to speak to colleagues and being able to speak to our students, because it’s important that they understand the challenges that we face, because we don’t just have white men who we teach, we teach a variety of students. And I think if we can talk about our personal challenges, and they can see that we also have faced challenges that they might be facing, then that can really be very transformative. So that’s kind of a brief summary of some of the things that I talk about.

Rebecca: Can you talk a little bit about how that unfolds in a classroom when you’re having those kinds of conversations with students in a math class?

Sarah: Yeah, so sometimes it unfolds very naturally, by some prompt that might happen. Yeah, there might be an extreme example. This past semester, I had another professor, he came in, and it was like, clear gender issue. And so I used that, in the next day actually, it took me a little while to react to it. And then the next day, we had a very deep conversation about gender in the classroom. But it might be before student evaluations, that has taken me a long time to come to, but how do you address the research about what students do in evaluations? Sometimes I assign reading about mathematicians’ experiences, I try to assign readings of biographies of diverse mathematicians, and then we relate it to their own experiences. And then if it’s appropriate, I don’t want to be all about me. But if it’s appropriate, I sometimes talk about what I’ve experienced. So those are some of the ways that it comes out. But I try to make my classrooms not just about the mathematics, because that’s really where the transformative stuff happens.

John: In terms of the teaching evaluations, have you addressed that issue specifically with students in terms of gender bias on evaluations before the evaluations? And has that helped? …because there is some research that making students aware of biases tends to reduce the amount of bias that shows up in the actual evaluations.

Sarah: It’s really hard for me to say if it helps.

John: …there’s no control group.

Sarah: Exactly. So even though I teach gigantic classes, you’d think that I’d be able to do some sort of like statistical thing, I have no idea if it really helps. I do get comments, after, if I do address it. I know that some students will say, “I’m not just saying this because she’s a woman.” So there is some backlash in that. So it’s unclear. I try not to do it right before the student evaluations, but like a few weeks before. I also do evaluations throughout the semester. And yes, it’s difficult to see if it impacts the evaluations or not. However, what is meaningful to me is not whether it impacts my evaluations, I think, but again, reaching the students who might not fall into those majority groups and helping them see that some bias stuff may be going on and it’s not all in my head and that is impacting my experience, and there’s actual research behind that.

Rebecca: I can imagine that students in a math class don’t expect to talk about identity. Can you talk a little bit about the student response to some of these conversations that you’ve had with students.

Sarah: It varies. I find that students are more and more open to it. I’ve taught a lot of first-year classes. So as they go through first year, they’re more open to it. Because at first, they’re like, “I just need all the math and they find a big change between high school and university in terms of the contact hours. So like you’re wasting our time talking about this stuff. But by upper-year classes, they find it such a refreshing change, because they’ve been in so many math classes, where it’s just all content, content, content. A lot of lectures. And so I really didn’t react to any of that backlash. And it’s almost like a breath of fresh air. And another aspect of identity that I think has been meaningful, like, has maybe come very naturally, the idea of like, “Are you a math person?” …because that’s another type of identity that’s really common in our society. And even that, it certainly linked gender and race, but it’s something that isn’t directly gender or race. And so talking about how that fits into their identity has also been a key to unlocking more personal conversation and getting them to really reflect on themselves in a mathematics classroom. Yeah, and I think one of the keys is like having them watch a mathematician talk about their work and how their identity is linked to their work. And then they comment, for example, Perusall or something where they can annotate the text, and then they start to get involved in some conversations, I can bring those comments into class and then we can have some pretty dynamic conversations.

Rebecca: I can imagine teaching first-year students in math with a societal “I’m not a math person” problem. I know, I teach in art and design, so we have a lot of students that claim that identity, “I’m not a math person, I don’t do math,” and are afraid. Can you talk about some of the ways that you have reduced the fear, allowed people to see themselves as being math people, even though they’ve never seen themselves in that way? I know you’ve had some really interesting things that you’ve done.

Sarah: Yeah, I love reaching those people. And it’s a lot more difficult now in my job at a big university than it was when I was teaching at a liberal arts school, where all students are required to take a math course. So maybe I’ll talk a little bit about my experiences at the liberal arts school to start. So I was at Quest University Canada, a small school, about 500 to 700 students. It kind of started as an experiment. And so we are encouraged to do all sorts of things. And we had a lot of students who were so afraid, just as you describe, of their mathematics course. And they were putting it off, putting it off, putting it off. And I think one of the things that really traps them is the idea that everything has to build on the thing before and a lot of them got lost somewhere in early elementary school and they never recovered. And it was some sort of threat to their identity, probably some like quick quiz or something. Someone said something, and they were lost, forever. So it’s trying to show them that well, there’s actually parts of mathematics that math majors don’t see until their fourth year that you can do right now. And that’s usually how I try to approach it. So I think one of the things is just addressing that head on, talking about their experiences in mathematics and telling them we’re going to do something different. You’re not going to see numbers, you’re not going to see arithmetic even, this is going to be about shapes and space and ideas, and maybe even accessing points of connection with individual students. So I can give examples of particular projects if you’d like or particular courses.

Rebecca: I’d love to hear an example of a project.

Sarah: Sure. I’m a firm believer that the things that you think are going to be total train wrecks can either turn out to be the best things you’ve ever done or they could be trainwrecks. But definitely my best things have been the wild ideas. So I was teaching a course on mathematical creativity. And it was going to attract a lot of students that were totally afraid of math because they had to take it as part of a series of courses on creativity. So they got to take a social science course on creativity and a chemistry course on creativity. But they also had to take this, in their minds, terrible math course on creativity. So I was really excited to teach it. But how would you describe the feeling of creating something new in mathematics. And for me, and for most mathematicians, if you hear like these quotes about mathematics, they’re like mathematicians will say “math is like poetry. Math is like…” …they give all these analogies with very creative analogies. But most students don’t access that until graduate school, because there’s not this freedom of exploration. So I spend a lot of time just wondering, how do I feel creative? How do students feel creative? And it was really only on research that I felt really creative? So, how can I model research for students? So what I ultimately did was, I asked them, first of all, I didn’t tell them where we were going, cause there’s going to be a two-stage project centerpiece of this course. And first stage is you have to define something from geometry, but it can’t be like anything you’ve ever defined before. So one group defined, they called it like an ice cream cone shape. So it was a triangle with like a circle in it. And then we really worked on making their definition mathematically. So how does the triangle touch the circle, another group to defined a caterpillar shape in a precise way. And then the second part of the project, after they have their definition they couldn’t change was to discover as much about that object as they could. And they were only graded on how, on their journals, how much time they spent thinking about it, and how much they talk to other people. And I’m telling you, the ideas that these students had, and the level of mathematics that these totally math-phobic students did, was incredible. It was what I would expect from fourth-year students. And they were starting to use the word theorems and proofs. I said, you don’t have to prove anything, I just want you to like discover things, but they were coming to it naturally. And it was amazing. I could just gush about all the things that they did forever, like all of the discoveries that they made for themselves. And I still hear from these students about the impact that this project had from them, I don’t know, six or seven years ago now. So yeah, that’s one of my favorites. But at the time, I thought, Oh, this might go really poorly.

Rebecca: It’s amazing how the freedom to explore and discover can really open up the freedom to see yourself in a new way, or to be a researcher in a different way. As you were talking, I’m remembering an opportunity I had as an early faculty member workshop. And it was a multi-day workshop with mathematicians, and I was the non-mathematician, to help develop curriculum. And I had never hung out with math folks that much before. But it was really interesting. And we had really interesting conversations about creativity and the overlaps of our work that neither of us had recognized before. So it’s really interesting how those opportunities to have those conversations, whether with students or with colleagues can open up so many possibilities.

Sarah: Yeah, there’s so much and I’ve learned so much from my first year-students who are interested in some very diverse things, and they brought a lot to, like I was gonna say, my teaching, but like, also, just me personally, [LAUGHTER] I think they really enrich my life.

John: And you have taught some interesting classes, including a first-year seminar course in math and literature and poetry. And another one was women’s mathematics. Could you tell us a little bit about each of those classes and how you’ve used that to get students more engaged with math?

Sarah: Sure. So both of them are at the University of Toronto, I will probably do it as U of T at some point, which is not University of Texas for American listeners. And in part they were written to try to attract students who might not traditionally sign up for a math course, all of our first-year courses are massive at U of T, except for these first-year seminars, which are capped at, I think, 25. So really, students’ opportunity for a small class experience. So the math of literature and poetry, I think some of the seeds were planted, actually, by one of the students in that math creativity class. She was a poet, and she identified herself as a poet as a first-year student, and she was also very afraid of math, but she kept finding linkages. And she said, “You know, I think this can help me with my poetry.” And I am totally not a poetry person at all, or I certainly wasn’t, I’m maybe more now. And so that started to get me thinking about like, maybe if I combine math with like a poetry course, I could engage some other students. And she’s one of those students that I still talk to you and she just got her MFA in poetry and still using her math. So I talked to her actually, in developing this course. She helped me a little bit on the poetry side, also a key part in the math and literature and poetry is I had a TA from English because, again, I’m not a specialist. So I needed someone to help me and she was wonderful, a PhD student in literature. So I think another source of inspiration that was also integrated into it was that I had taught novels previously and seen how novels helped students relate to mathematicians or see themselves as mathematicians. I was just amazed at like how much empathy they had for the characters. So we read novels like about mathematicians, like the Housekeeper and the Professor, a really great Japanese novel in translation. And then there was mathematics from novels. So for example, one of our key novels, or a story, of the Library of Babel by Borges and you can actually ask, what is the shape of this library? What could the topology of it be in mathematical language? So then that was a key for investigating topology.

John: Was a library closed or open?

Sarah: Yeah, [LAUGHTER], exactly, that sort of question. We can start to narrow it down. So that was the math in literature and poetry course, in terms of content. The woman’s mathematics course is still kind of growing in my mind. It’s been in the works for a really long time. I just like us to center it almost like an experiential learning course, where the object of study was the university or like the mathematics in our university itself. And so as a result of both history and modern mathematics, and all sorts of things, I decided one of our units was going to be on data visualization, which is a little bit more number focus than I often have in first-year seminars, but people are often surprised that like Florence Nightingale was not just a friendly nurse, she was [LAUGHTER] an amazing data scientist. And she was really one of the first people to bring some of these amazing data visualizations, and she’s an amazing statistician, all these things. And there’s also a lot of women in this space currently. So their project was like, well take some data about the math department, maybe, or students in the math department and find an appropriate visualization for it. And they generated stuff that we really haven’t seen, like, what does it look like in our departments to have 13% woman faculty. You can say it all you want, but to actually see it with like the people, it is actually pretty startling to me. And then another project with that course, was we worked with a university archivist, and went into the university archives. So our university has a long and storied history, we hold ourselves up as a great research university. So we have many illustrious women in the past who have studied here, but people don’t know about them. And since I would say, we have a pretty bad situation with women in our department now, people kind of assume, after this archives project, I would go around and I would ask people, “When do you think the first woman president of the math student union was?” and people would say, “Oh, there’s never been a woman president. Like, are you kidding?” Because that’s the way it looks like now. But the answer is actually 1910 or something. And there were strong women, like way back when. And so students went to the University Archives, they looked at student records, they looked at faculty records, they looked at photos, and they told some stories from that. So that’s a project that’s gonna continue for a future class.

Rebecca: Sounds really interesting, and a great way to get students engaged with many different mathematical ideas, but also really engaged with this idea of identity related to disciplines.

Sarah: Yeah. And another thing that it did is it also helped them see themselves as part of the university because it was the first semester of the academic year, they were first-year students, and so it helped them see themselves as part of the community.

Rebecca: We don’t, in our curricula, look at the history of our university as part of what’s informing our work or informing the students. And so I can imagine that that’s a really unique kind of experience that could happen in any discipline, that would be a really interesting opportunity for students to just better understand their traditions that they’re coming out of.

Sarah: And I think our university archivist wants to work with classes, like they’re so excited to see, especially first year classes, get us to coming there and being part of that aspect of the university. I’m always a big fan of all of our librarians and I know that you guys are too,

Rebecca: You should see my notepad right now. It actually says “go see our archivist” because him and I had a conversation about a project we could do with my class. [LAUGHTER]

Sarah: Yeah, they’re wonderful.

John: Often, archivists are working in rooms all alone by themselves. And in fact, ours do work in the basement. And the opportunity to engage with students is good for them, as well as for students. That sounds like a really engaging project.

Rebecca: I’m ready to sign up for all of these classes. So I hope you have room.

Sarah: You are welcome to come and even speak and spread your wisdom, I would love that. [LAUGHTER]

John: Speaking of room… you also have taught some introductory calculus classes with up to 3000 students in them and you transformed them into an online environment along with a colleague during COVID. Could you tell us a little bit about how that course was structured and how it changed when you went to remote teaching?

Rebecca: It sounds so daunting.

Sarah: Oh, okay. So COVID for everyone has been really, really tough. And especially that I always have to go back and think what year was that? 2020 to 21 academic year…, everyone had it really, really tough. So we all like deserve, like hero badges or something, and I’m ready for a break. So I think we all need to catch up on our rest still from that. But I was fortunate because before the pandemic was on anyone’s radar, we had already arranged kind of a transition point in my job where I was going to be going from coordinating this gigantic introductory calculus class to not coordinating it. And the new coordinator, my colleague, Bernardo Galvao-Sousa, he was going to take over it. We were going to have a one year overlap, so he could kind of see how I did it and just like everything was gonna go normal and then he was gonna take over. So that overlap year was to be the 2020 to 21 academic year. So it was fortunate that we were both able to work on it, I don’t know how it would have happened otherwise. I had already been in a period of transforming this class, it had been basically the same from no one really knows, like, as far as anyone could remember, it had been the same. And I was basically brought on and hired at University of Toronto, to bring it to the 21st-century. So over the past three years before that, I had been changing it completely. And then, of course, we went online, which required it to be rethought because you can’t teach a course for 1000s of students the same way. So what was the course like before? Well, we have a lot of rules in our university for first-year courses, in that they have to have a midterm, they have to have a final exam worth a certain weight, there were three one-hour lectures a week, one one-hour tutorial, kind of the whole structure was pretty traditional. But I had been introducing some innovative projects, we were shaking things up in how we did them in tutorials. And the whole curriculum was really modernized. So I’ll give an example of one aspect of the course, the applied communication task, and how we transformed that aspect of the course, to put it online and still give students hopefully as good of an experiences as they could online. So applied communication tasks were this word project. So first semester, they were three separate projects, second semester it was one project, they were applied, and they were about mathematical communication. And I’m a big believer that I don’t really invent many new ideas, I just kind of like look at the needs of my students in my place and try to adapt things from elsewhere. So I had talked to every department that took my students after so this was like a calculus course for science students. And so they were going, the majority to life sciences, but they were also going to chemistry, they were going to physics, some are going to earth sciences. And then some are going to psychology and some at some other smaller departments, not smaller departments, but smaller portions are going to other departments. I guess economics was another big one. And I talked to them about like, what skills don’t students have that they should have from calculus? And one of the big themes was that students were afraid, they were just afraid to approach math in new context. So they could solve all the problems that were traditional, but they couldn’t if they saw a scientific paper, and there’s math, they were like, “I’m not familiar with this math, what do I do?” So I really took that as inspiration like, well, we should have students do that very thing. So as an aside, I put questions like that in exams, like, you know, take problems from scientific papers, give them information and put them on exams. But then also, in the second semester, have them find a scientific paper that has a mathematical model. And ultimately, the goal is to communicate something about that scientific model. Now, what form should that communication be in? Well, one common form that scientists use is a scientific poster. And the advantage of that is that it could be kind of an event, it can be kind of a grand finale for the course in tutorials. So we had a bunch of mini-poster sessions with about 100 students each. And so each of the posters presented models. They got into groups, kind of halfway through the semester, they combined some of their papers, but that took them through the experience of talking to a librarian and having to deal with databases. It got them through finding what’s important and what’s not. Well, I don’t understand… really this is way over my head in terms of math… what can I say from this model, and so all those skills like that, and then also the kind of communication. And it also combined oral communication where they have to talk about their poster and written communication, they had to write about their poster and they really worked on different drafts of different parts of their poster, and they have to read. First semester, the projects prepare them for that. They had a project that was focused on written communication, that was writing a proposal to their city council based on population projections from their hometown. They had a reading task and that’s changed a little bit over the years. So that’s what it was. What we did online is we basically kept the same projects, except instead of having the sprinkled in the tutorial, like every second or third tutorial was about the project, now we knew that they’re at home, they do not have any resources, any people around, we really need to make these be focused tutorial and make the structure very, very, very clear. Because otherwise, this really complex project is just gonna get completely confusing. We structured the first semester in that the first three tutorials were focused on writing. And the second three tutorials were focused on reading. And the third three tutorials were focused on oral communication. And then within that, the first tutorial had the same structure, the second tutorial had the same structure. And the third tutorial had the same structure. So they kind of had something much more predictable. And there was like a lot more evenness, and we didn’t try to give them as many skills as we did in the in-person, we cut down the expectations, we trimmed as much as possible. And then something similar in the second semester, we trimmed a lot, we focused a lot, we didn’t aim as high on the exams, in terms of all of those questions from scientific papers. We didn’t have exams, instead of exams, we had three different types of quizzes, the fun type was reflection quizzes, which had them reflect on their learning sometimes, or maybe conduct some sort of experiment at home, and then use that and make a model or something to like, go on a walk, this was in the deep COVID In the fall of 2020. And so like go on a walk, if you can’t go on a walk outside, go on a walk around your house and find something to model. So some people are modeling bird chirps or whatever. And then you create your own scientific models. And if you have two to three thousand students spread around the world, obviously, cheating is a huge concern. So we tried as much as possible to make it interesting. And for me, like, yes, academic integrity is big, but it was the perception of academic integrity amongst the students. Like we really wanted to keep them engaged.

John: So how did you assess and evaluate all those quizzes? Did you have a large team of TAs?

Sarah: We had very limited TA hours actually. So I think that’s another part of big course stuff that we don’t talk about a lot. It’s actually something I’ve been writing about, I’m just not sure where to send it because we don’t talk about it. It’s like management, like how do you manage a large organization? So we have about 50 people. How do you distribute your resources, and we have very limited resources. So we wanted to do these quizzes, we want to them very well, we have very few TAs and we still wanted TAs to teach tutorials. What are we gonna do? So what we ended up doing is redistributed our instructor resources. And normally students would be in classes of 200 in person. And we had them in classes of 400 online, because we figured the difference between an online class of 200 and an online class of 400 was not going to make a big difference. And technology, I can go into all the technological challenges. Now, the technology is all there. But August 2020, breakout rooms for this large of groups, impossible. So we had to do all these Zoom, and it’s crazy stuff. That’s how we managed is we had instructors who were like just in charge of quizzes. And that’s how we did it. And then every third quiz was kind of the automatically graded kind.

Rebecca: I think it’s important to bring up some of these logistics or project management skills the faculty have to have, especially when coordinating such big courses. And I appreciate that you’re sharing some of those things, because you’re right, we don’t talk about it. Just like we don’t talk about those same experiences that we have as young female faculty in the classroom or whatever kind of identities that impact our experiences.

Sarah: Exactly. Yeah. And both of these are the things that really keep me awake at night. It’s not the actual teaching, it’s the “How am I going to possibly grade?” [LAUGHTER] Or like, “How can I negotiate with my chair for more hours per students?” Or “What are you going to do with that one TA who’s behaving inappropriately with students?” It’s all of these extra things. Very, very, very different if you’re doing even a class to 500 versus a class of a few 1000 is quite different because you can’t see it all.

Rebecca: Yeah, managing an equitable experience is a really different kind of thing. It just keeps scaling up. So finding that equity piece is a challenge.

John: But it is impressive that you did those reflection quizzes at that scale, because that’s something I’ve wanted to do, but have been a little reluctant to do in a class of 400 that I teach in the fall. And now this is suggesting maybe I should do some of that. [LAUGHTER] Providing the feedback is the main concern that I have.

Sarah: Yeah, well, I think for gigantic classes. I don’t know however, we defined gigantic, like I guess gigantic versus the thing that you want to do. It’s often like what’s really the priority here and then what can you sacrifice, like, there’s always going to have to be a sacrifice. So I can’t provide the same feedback on a quiz to a group of 2,000 students as I can for a group of 20 students, or the classes that I had this year were in the low hundreds. And I can’t provide them the level of feedback that I had like on everything. But using peer feedback can be helpful, or just explaining to them, I can’t provide you feedback on this. If you want more feedback, you’re going to have to seek it, which is hard I know and not ideal. However, these are the things that we face, or just like deciding that the grading scale is going to be really generous and loose. I experimented this last semester in class of like 300-ish students with not ungrading, but more this [LAUGHTER] direction, letting them determine a lot more of their achievement levels, trusting them to say, “Oh, yes, I have mastered this actually.”

Rebecca: Well, now you’ve piqued our interest and we need to know more about it.

Sarah: Yeah, like, I’m still kind of thinking about how to describe it or characterize it because I started off with a structure. And then I really let the semester go on and adjusted as I saw my students change and as I changed myself. So I don’t have a lot of eloquent ways to discuss it. This is a upper-year course for group theory. And I wanted to do a lot of things that I just didn’t have the resources for. So I had to make a lot of tough decisions. And also, we are in a super grade-intensive university. And by the time they’re in like, third year, this is so ingrained in their mind. And this particular course has a very high percentage of international students, probably over 80% international students. And in my university, I think that they tend to be more concerned about grades because they have to be and somehow, just like not giving them grades on anything. [LAUGHTER] Like saying, “Okay, you’ve either mastered this, or you’re excellent on this, or you’re not there yet” was really difficult at first, a lot of them dropped the course immediately, because they didn’t understand it. They were like, “what percentage is this?” And I’m like, “Well, there is no percent.” “Well, is it 100?” I think they did not understand the concept of it at all. So I wanted to focus on oral skills, and oral skills are so hard to assess. But I want to give them the opportunity to develop their oral skills, I didn’t really want to assess them as much as I wanted to make sure that they were speaking about math and they were talking about math to other people. They could reattempt any assignment they wanted. So, they did a test, they could show me that they had actually learned the material on the test. But they had to talk to other people about it, they had to demonstrate they had spoken to other people, a lot of the main things like videos, and one group organized a mini conference on the topics for the weekend. They did a lot of amazing thing as a result of this. And the TAs provided very targeted feedback. So we’ve provided feedback on the skills that we knew students needed feedback on. So, they needed feedback on particular cognitive skills that they were not able to assess, like research has shown that they are not able to assess their own proofs, or students are not able to provide that same feedback. That’s what we assess. But we didn’t bother assessing things we didn’t care about.

Rebecca: It’s an interesting way of thinking about it. I know that I also was experimenting a little bit with ungrading this past semester, and also found that international students are the most like, “I don’t know what this means.”

Sarah: [LAUGHTER] Yeah, you have to just admit, sometimes, you don’t really understand it. Also a good opportunity for discussion for students, and talking about what that means when we don’t really understand.

Rebecca: Yeah, I think it’s good to model that.

John: We always end with the question, what’s next?

Sarah: Well, I just submitted my tenure file two days ago.

Rebecca: YAY!

Sarah: So I need to catch up on a couple of things, but then rest. I have not had a good opportunity since the beginning [LAUGHTER] of the pandemic, so I think that that’s going to be my answer. Yeah. [LAUGHTER]

John: That sounds like a wonderful plan. [LAUGHTER]

Rebecca: And great advice for everybody listening. [LAUGHTER] We can all use a little rest.

Sarah: We all need that reminder.

John: And we should note that this is the first time that Rebecca and I have recorded in the same room since March of 2020. So this is sort of a return to normalcy for us.

Rebecca: Yeah. So it was nice to share this experience with you, Sarah.

Sarah: Yeah, it was so nice to talk to both of you and to see you together. [LAUGHTER] So, I know that listeners can’t see you, but I have enjoyed seeing you and speaking with you.

Rebecca: Well, thanks so much for joining us. Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

254. Teaching Up

Creating an environment where members of the learning community can be taken seriously as their own authentic selves requires planning. In this episode, Celeste Atkins joins us to discuss how shifts in context, like reframing an assignment, can impact the way people engage with each other and the content.

Show Notes

Transcript

John: Creating an environment where members of the learning community can be taken seriously as their own authentic selves requires planning. In this episode we discuss how shifts in context like reframing an assignment can impact the way people engage with each other and the content.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Celeste Atkins. Celeste is a Sociologist, the Assistant Director of Faculty Mentoring Initiatives, and a Lecturer in the College of Agriculture and Life Sciences at the University of Arizona. She is also the author of a chapter in the Picture a Professor collection, edited by Jessamyn Neuhaus. Welcome, Celeste.

Celeste: Thank you.

John: Today’s teas are:… Celeste, are you drinking tea?

Celeste: I am an iced coffee person. So I actually drink Jot and I make my own vanilla lattes every day.

Rebecca: Wow, that sounds fancy.

Celeste: It’s really easy. Jot is a coffee concentrate, you use a tablespoon full of it and then I use a tablespoon full of vanilla sugar and eight ounces of milk. And it’s delicious and easy and quick.

Rebecca: and caffeinated. [LAUGHTER] I have Jasmine black tea today.

John: And I have ginger tea today.

Rebecca: The title of your chapter in Picture a Professor is “Teaching Up: Bringing my Blackness into the Classroom.” In addition to your chapter in Picture a Professor, you’ve also published other chapters that grew out of your dissertation: Teaching Up: Developing an Intersectional Andragogy. Can you tell us a bit about your dissertation research?

Celeste: Well, I have a background in sociology, but my PhD is in higher education. And so I spent close to a decade teaching at the community college level. And my dissertation grew out of my own experiences as a Black woman in a conservative Arizona town teaching about racial privilege, heterosexual privilege, and those types of things. So what I wanted to do was take an intersectional approach, because there’s literature on faculty of color, there’s literature on women, there’s literature on queer faculty, but not much takes an intersectional approach to see what we have in common and what we don’t. And so I interviewed, I believe, 18 sociology faculty from across the nation at different levels, in different types of institutions, about their experiences as part of a traditionally marginalized group teaching up, so teaching about privilege when they themselves are oppressed in some area. And so we had women, we had queer faculty, we had a couple of faculty who identified as disabled, and quite a few faculty of color.

John: On your website, you note that the chapter in Picture a Professor is based on some unexpected findings from the research in your dissertation. Could you tell us a little bit about the unexpected findings that you talked about in this chapter?

Celeste: Sure. So actually, this chapter is about the part of my dissertation that spoke the most to me, but surprised me the most, which is, when I started to look at differences intersectionally, I found that Black women, in particular, focused on bringing their authentic selves to the classroom. And for some of them it was after they got tenure, for some of them, it was after they felt they had sort of sold their soul in a way. And for me, what I found in my teaching, and why this resonated with me was: I started teaching, I got a lot of feedback, “you’re too aggressive,” “you’re too assertive,” “you’re too scary,” blah, blah, blah. And so then I tried to be like a Disney princess and be really, you know, flowers and butterflies, and very welcoming and soft, and it was fake. And my students didn’t like it, because it wasn’t me. And they could tell it wasn’t authentically me. So after a year or two of that not going well, I decided to just be me. I found a different book that was more intersectional and I started talking about what it’s like being a fluffy Black woman and how it affects how I live in the world. And I would make jokes about it, and I would address it. And then students really responded to it because it was who I am, and my authentic self. And so what these other sociology faculty were doing that’s so important, is modeling different ways of being professional. Because one of the things that’s so hard about hegemonic academia is it’s very heteronormative, it’s very white, it’s very male, it’s very middle class. And so a lot of us do a lot of code switching. And I used to joke about my best friend in college, she worked for a talent agency and I worked in HR and so we would call each other and like, “Good afternoon, may I speak with Michelle, please?” And she’d go, “Who’s calling?” And I’d go, “This is Celeste. What’s up girl? Hey, what are we gonna do this weekend?” As soon as we knew it was each other on the phone, then we would be our authentic self. And a lot of us spend time code switching. But what that does is reinforce the idea that our authentic selves is not okay in academia. And so this chapter about bringing our Blackness to the classroom is about when we show our true selves not only do we find different ways to connect to our students, but we also expand for many their ideas about what faculty are, about what professional is, about what an academic does. I can be an academic and not talk in $5 words, I can be an academic and be very gesture-y and very outspoken and out there and still do quality academic work, and in some ways, reach students that a lot of others who are so concerned at fitting in this rigid box of what is considered proper academia miss.

Rebecca: Can you talk a little bit about the kinds of classes that you teach that we can start talking about what that looks like for you and how your chapter addresses being in those classes?

Celeste: Well, I’ve gone through a lot of changes during my dissertation journey. And I actually have another chapter coming out about how I felt like I was kind of pushed out of teaching. It is very challenging to be a woman of color, the only Black woman faculty at my institution for part of my tenure, and teaching about these topics in a place that not everyone agrees with. And so I have actually transitioned out of full-time teaching, but I spent my career teaching intro to sociology, human sexuality (which is very fun), race, and gender. And now for the College of Agriculture and Life Sciences, I teach a diversity class. It’s fully online, but what I’ve done, based on what I’ve learned from my teaching experiences, is I’ve created it in a totally different way. So there’s no book, and it is the closest I could get to a series of guest lectures. So it’s based completely on YouTube videos and I give a little introductory video explaining the concepts. And then I find people who are either experts in the field, or who are personally oppressed in that way to share their stories. Because what I learned is, it’s one thing to hear about the concept, it’s another thing to humanize the people who are going through it. And so we’re actually doing some research on that to see if that approach is more effective. And so that’s currently what I’m working on.

John: And there’s a lot of research that shows the power of narrative. And when they’re personal stories, it has much more resonance with people than when they read about something in a book that seems a bit more distant. So, that sounds like a wonderful approach.

Celeste: I really found that especially when I was teaching human sexuality, I would bring in queer folks, I would bring in trans* folks, I would bring in polyamorous folks. And it went from “Ooh, that weird stuff”or “all those ‘those’ people” to “Wow, they remind me of…” and “They’re just like…,” and that I found was so important in breaking down stereotypes and really making a change.

Rebecca: You talked a little bit about negotiating your identity in the classroom at the beginning and making adjustments and not feeling authentic. How do you feel like you’ve been able to really be your authentic self [LAUGHTER] now? How are you able to arrive at that moment? And what does that look like?

Celeste: Well, when I was teaching face to face, what I would do is literally address the elephant in the room, we would do those, you know, the things that students love so much: come up and talk about yourself. But I would say “Look, I’m a big Black lady. And we’re gonna talk about stereotypes and those types of things”. But people say that I’m intimidating and people say these things, but my students who know me know that I’m here to help you. I’m not here for the money. If you knew what I made, you’d know that. And so I use a lot of humor. I make a point to break stereotypes, especially with my images. And then I make a point to be humorous about the images. So we’ll be talking about deviance, and I’ll say “So not holding a knife to a white lady’s throat is that… what kind of deviance [LAUGHTER] is that?” But I’m also very careful to never show single mothers that are Black. I’m also very careful when I do gangs. I have memes that I use and one is this white guy with a really long beard, riding a pink bike talking about biker gangs, or I have one meme of Sesame Street when I talk about gangs. And so I’m really, really careful to break stereotypes. And I also make sure that when I’m choosing my test questions, I’m choosing the ones that, again, reinforce breaking those stereotypes.

John: So you’ve talked a little bit about bringing your own identity into the classroom and how that evolved over time. How do you help students express their identities in class?

Celeste: I’m really, really careful about how I do examples. I very deliberately find diversity for my images. And again, I try to find things that people don’t think about. So when I’m doing, let’s say, relationships, I’ll show like an older lesbian couple, nobody thinks about old people still being in love [LAUGHTER] oftentimes, when you’re talking to young students. And another thing that I do is I bring in stories of my friends who are very diverse, and the people that I’ve known. And I feel like if you create a safe learning environment, and I do a lot of steps to do that in the beginning, that then students will feel safe sharing. One time, we were talking about border patrol, and we were talking about racial profiling. And I was trying to get across to one student who was either in border patrol or headed to be in border patrol that if you only focus on Latinos, then yes, you will only find drugs on Latinos. If you’re not stopping white people. If you’re not stopping Black people, then you’re not going to find drugs on them. And the argument was, “Well, it’s the cartels. And it’s this, and it’s that.” And finally, another student of mine, who is Latino, and whose father is Latino, but a border patrol officer, talked about being stopped, talked about being afraid, talked about this dynamic of “Yes, there are good officers who aren’t, and yet still, this happened to me, even though my dad is.” And so I tried to create that kind of space where students can shift each other’s ideas by sharing their own narratives.

Rebecca: You mentioned just a moment ago about setting yourself up to be able to have that space for students. Can you talk about some of the steps that you do take to create that environment?

Celeste: Yes, when I was teaching face to face, it was basically the first week, and usually these were two day a week classes, were centered upon creating a safe learning environment. So we would talk about community agreements, and then I would take it further. And I use some things that I learned at WRITCHE and don’t ask me what that acronym is for, but it was something about teaching about sexuality. And so what we did when we went to that workshop was we anonymously answered all of these questions on a survey. And so what I did was I create a survey about: Have you ever had or helped create an unwanted pregnancy? Have you ever used a food bank? Have you or anyone you know ever been to prison? Or Is anyone you know, undocumented? We lived on the border. And so what I would do is I would have my students take this, and I would go to great pains to make it truly anonymous. So I made everybody do a checkmark and not a big X and not a square, and everybody used pencil, and then we would go outside, and we would shuffle all the papers and pass them out. And then we would step in, step out to show who did it. So how many people have been part of an unwanted pregnancy? And we’d have… so I’d say then when we’re talking about reproductive rights, remember, it’s not those people, it’s people in this class. How many people have a family member who’s undocumented? Okay, when we’re talking about this, you need to keep it in mind. So it makes it really personal without outing people that, in this classroom, there are queer people. In this classroom, there are parents. In this classroom, there are people who have been to prison. And so we do that. And then I did a version of the opportunity walk. I know that there are mixed responses to the opportunity walk, but the version that I use starts with basically what we call ascribed statuses in sociology, so the things you can’t control. And so when they get to a certain point, I say “Now stop, look around, these are the things you had no control over.” And I talk about, as a Black woman, I’d be kind of back there in the back as well. And then we talk about the things that they have control over: education, those types of things, speaking up, being an ally, that’s an important one, because that starts to push you back again. And so we look at that. And we end that, and I say, “I want you to think about, again, where you were, it has nothing to do with you. So therefore, when we’re talking about privilege, it’s not about you, you didn’t tell the stork, ‘please bring me down to a rich white family,’ we have no control over any of these social categories that we’re born into. And so when we’re talking about that, then we’re trying to understand.” And then later on in class, I do another exercise called the “oops exercise,” again, talking about intersectionality. And pointing out that even if you’ve got privilege, if you’re white, male, heterosexual, well educated, at some point you were young, and therefore you were oppressed by age, and we like you enough that we want you to live long enough to be oppressed again by age, right? So even the most privileged people experience oppression in at least one category. And so those are the ways that I tried to make it a space where both we can share our own stories, and where we understand that privilege. While it’s challenging, and while we want to think the world is fair, it really isn’t. And we have to look at how we have privilege without it being a personal failing.

John: What other suggestions do you have for creating a more inclusive classroom environment where everyone is part of the class and where everyone’s voice is taken seriously and is heard by the class?

Celeste: I think it’s a balancing act. And I think it depends a lot on the identities or the perceptions of the faculty person themselves. So as a Black person, as a big Black woman, I find it necessary (and luckily, it’s part of my typical approach anyway) to use a lot of humor to make myself seem approachable. And it’s very frustrating because I used to co-teach with a guy who called himself my token old white guy, and he was an English professor. And I would say something about sociologically sound principles that are from my discipline that are scientifically proven, and students would go “well, I don’t …:.” and then he would say the same thing as a frickin English professor, and they would go “Yes, you’re right.” And it’s frustrating. But [LAUGHTER] the reality is, that’s the way it works. So sometimes I do that, sometimes I use my colleagues that way. And sometimes I’m that way, as a cisgender, straight woman, then I provide that added, “It’s not the chip on my shoulder” when I’m talking about issues affecting the queer community. So I think that’s important. I also think it’s really important to listen to your students. I have yet to find a school that has student surveys that address what I want to learn. So I create my own. And then I have students give them back, I have them give them back on the last day of school where I like to be done. So their grades are done on the last day of school. And so this won’t affect your grades. I’m going to give you your grades in a minute. And you can be completely honest, and what would make this class more comfortable for you? And I change my classes based on that feedback. And when you work for a while in one institution, then students tell them and so the feedback at my former institution, students either loved me or hated me. And the ones that loved me were like, “She’s awesome. She’s funny. She does really cool stuff. But she don’t take no crap. So don’t go in there and try to BS her and don’t be late, because she won’t take it.” And then the other ones are like, “Oh, she’s so hard.” Yeah, because I don’t take late work, because I’m trying to also prepare you for real life.

Rebecca: Can you talk a little bit about some of the ways that you design assignments to make them more personally relevant to students?

Celeste: Oh, yes. One of the things that was really interesting when I started graduate school was I started a minor in a certificate for college teaching. And I was like, “Oh, this will be an easy minor. I’ve been teaching college for quite a while now, so I’m good.” But part that really helped me was designing effective writing assignments. And I saw such a difference when I stopped having students write a paper and started having them do things like write a letter. So in my race class, I would have students, instead of writing a paper explaining to me privilege and intersectionality, I would have them write a letter explaining to someone in their lives, privilege and intersectionality. And if you looked at my website, some of my students did some amazing, amazing letters. And they were students that I wasn’t, in some cases, expecting that type of understanding. But when they’re explaining it, using their experiences is very different than how I explained it. But not only are they showing their understanding, they are teaching me other ways to reach other students. And so I found that very, very helpful. And part of what I do is I build reflection into all of my written assignments. So, what did you learn? How will it help you? Because my argument to students is that sociology is something that they can use no matter what their end goal is in life, you can always interact with people better. And so how will this help you in your civic life? So those are some of the ways that I try to make it more relevant to students.

Rebecca: It’s amazing how a small shift in the frame of a writing assignment can make all the difference, that content is really not any different. It’s just framed in a different way.

Celeste: Yeah. Because when you say it’s not a paper, and you say it’s a letter, then they start to write from their own, instead of trying to regurgitate what I said. When I say it’s a paper, they think I want to hear me, and I hear me talk enough. [LAUGHTER] So I really want them to show me their perception. So, to me, that was the most powerful change I’ve ever made.

Rebecca: Audience matters, for sure.

Celeste: Oh, yes.

John: Much of your work now is in faculty mentoring and faculty development. Could you tell us a little bit about your roles there?

Celeste: Sure. So once I started to feel that I was losing my empathy for students [LAUGHTER] and getting very frustrated in teaching… especially, it’s hard to teach online about race and hot topics, because they don’t really see you as a human being. And they feel really empowered to say things that they wouldn’t say, especially to my face, but they wouldn’t say in a class. And in a classroom setting, first of all, students will call each other out. So I don’t always have to be that person. And second of all, I can revert to: “Hello, we’re going to treat each other with respect, we agreed to this, we wrote a contract about it, we have community agreement.” It’s much more challenging to do that online. And so I began to feel like it was taking too much out of me to try to teach about these in a fully remote setting as I was during the pandemic. At the same time, I was working as a graduate assistant, paying for my tuition, and I happened to land a job in the Office of Instruction and Assessment. And I started to learn about faculty development as a career, which I really didn’t even know existed. And I began to think that is something that I can do. I’d been department chair, I’d been mentoring new faculty, I had done a lot of workshops on time management and classroom management. And so I began to shift my ideas into that was what I wanted to do. At the same time, I was working full time, working at least two jobs, because I was also a graduate assistant, sometimes three or four, and a single mom to a four year old when I started graduate school, and having some challenges with a cohort of students that were half my age who had very different ideas about social justice than I did, like we both wanted the same end result, but had very different ideas about how to go about it and was feeling very isolated and made a friend. And after a couple of years, where both of us sort of mentored each other, we both ended up in assistant director positions. And we started to think about the power of our relationship and how we could help people find that in a less organic way. Because it just happened to be magic. It just happened to be she worked in the office, she had really cool artwork, I walked in and asked about it. And when you see us together, you see this big Black lady and this little… she looks 12, but she’s not… and she’s got blue hair, and people are like, “How are y’all friends?” But at the core, we’re both about helping people. We’re both about social justice. We’re both about making the systems better. And so we bonded in a lot of ways, and we help each other in a lot of ways. And we actually complement each other in a lot of ways. For example, I hate rewriting and I would have not published all those chapters if it weren’t for the fact that she loves editing. So I would write it, she would edit it, and then I would fix it. And that’s how I got through. And we collaborated on a lot of things. And so we had been sort of building out this framework around peer mentoring, and how can we create, systemically, an environment where people could find their sort of match. And during that time, they were also, in the Office of the Provost, hearing that mentoring needed to be focused on and talking about creating a mentoring Institute. So she encouraged me to apply for this position, it’s a brand new position. And so I, in November, received this position, which is Assistant Director of Faculty Mentoring initiatives. And my main goal is to facilitate the creation of the MENTOR Institute. And I like acronyms. So MENTOR is actually an acronym for Mentorship through Effective Networks, Transformational Opportunities, and Research. And that’s really what we want to create. We want to create a place where we share social justice minded inclusive best practices about mentoring, and where both faculty and students and hopefully, eventually staff, will be able to do training and expand their knowledge and do research about mentoring best practices.

Rebecca: Sounds like a really great opportunity to start something new, but something that’s so needed in so many institutions. The mentorship piece is crucial for people, but also it’s so not facilitated. [LAUGHTER]

Celeste: Well, what we found is it’s just very different. In a huge R1 Institution, each college does things their own way. And so what we want to do is synergize and illuminate the great work that’s already been done. We have pockets of really excellent mentoring, and then to help facilitate for those who are going: “Yes, we need to institutionalize this, but we don’t know where to start.” And so it’s been really interesting. It’s been fun. It’s been a lot of work. [LAUGHTER] I’m currently working on our first workshop that’s going to premiere in fall, when everyone comes back, on mentoring practices. And I’m also conducting focus groups with graduate students to sort of understand what’s going well, and where we can fill in those gaps.

Rebecca: Sounds like really important and exciting work, but definitely work nonetheless. [LAUGHTER]

John: Do you have any other reflections on your work on the Picture a Professor project,

Celeste: I just want to say a couple of things. One is that I really hope that people will take the time to look at this book, because I think that part of what’s needed for the culture shift in academia is a shift in how we picture a professor, what a professor is. I spend a lot of time with people going, “where’s the professor?” It’s me. Hello, I’m the professor. And I also want to encourage people who are in graduate school to look for these types of publishing opportunities. I’m still working on my first sort of solo first-author publication in a peer-reviewed journal. I was part of the task force for the American Sociological Association, where we focused on contingent faculty. And as that I earned a first-author credit just because my last name starts with “A,” but I found it really challenging in any other ways to publish in peer-reviewed articles. However, I published three or four chapters of my dissertation by looking for edited anthologies that were coming out in the area that I was publishing. It’s still peer reviewed. It may not carry as much weight, but for me, it was a little bit more of a user friendly way to learn how to publish, to learn how to do rewrites, to learn how to do those multiple versions of wait a minute I thought I was done with this… [LAUGHTER] until it gets accepted, and it builds your CV. So I wish someone had told me that. I just happened to luck into it. And once I got my first chapter, then I started looking for other chapters. So that’s some advice that I wish someone had given me.

Rebecca: Well, thank you so much for all that you’ve shared with us. We always wrap up by asking, what is next?

Celeste: Well, I’m gonna be 100% honest, because I found bringing my authentic self was the only way to do it. And literally what is next for me is an epic road trip with my daughter.

Rebecca: That sounds awesome.

Celeste: She’s been a trooper for four years while I was in graduate school. She’s been a trooper for two years of a pandemic. And my little extrovert [LAUGHTER], who was stuck at home with just me and her. And I’m pretty much an introvert. So we are going to go on a road trip for two and a half weeks across seven states. And we are going to work on my bucket list, which is I want her to see all 50 states with me before she goes to college. So we’re working on breaking that down. And then professionally, it’s our first workshop. And we also facilitate faculty development communities for promotion. And we are looking into creating some sort of grad student communities in the fall as well. So, that’s what’s next for me.

Rebecca: That sounds like lots on the horizon. Have a wonderful road trip. That sounds wonderful.

John: It does. And thank you for joining us. It’s been great talking to you and we’re looking forward to sharing this episode with our listeners.

Celeste: Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

242. Student Podcasts

Student research papers have been ubiquitous in higher education, but there are many ways in which students can demonstrate the skills that they have acquired. In this episode, Megan Remmel joins us to discuss the use of student podcasts as a more engaging alternative to traditional research papers. Megan is an Assistant Professor of Political Science at Bradley University.

Show Notes

Transcript

John: Student research papers have been ubiquitous in higher education, but there are many ways in which students can demonstrate the skills that they have acquired. In this episode, we discuss the use of student podcasts
as a more engaging alternative to traditional research papers.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners. [MUSIC]

Rebecca: I guess today is Megan Remmel, an Assistant Professor of Political Science at Bradley University. Welcome, Megan.

Megan: Hi, thanks for having me.

John: Our teas today are… Megan, are you drinking tea?

Megan: I am not drinking tea. I have rooibos this morning. But I am now currently drinking my coke zero sugar.

John: It’s not that much different than many teas.

Megan: Yes.

Rebecca: Many other rebels join us as well. [LAUGHTER] I have English afternoon today, John.

John: And I have ginger peach black tea, a return to an old favorite as we move towards the end of our semester here.

Rebecca: We’re both getting to things that are comforting.

Megan: Ginger is calming. So, [LAUGHTER] you might need that at the end of the semester.

John: Oh, very much so.

Rebecca: So we invited you here today to discuss a podcast project that you’ve been using in your state and local politics class. Can you tell us a little bit about your project.

Megan: So I was trying to think of a way to make students try to approach things in a more neutral way. Because obviously, in poli sci, there’s a lot of soapbox standing. And I’d previously been using just plain old policy analysis papers. And students don’t love them. And so I was trying to think of a way to get them to do the assignment that I wanted them to do, and having some guidance, because they’ve listened to podcasts before, so they kind of know what some of these are structured like. And so I was hoping that that would help tone down some of the opinionation that can come out of these things. And so I YouTubed, and I found John’s YouTube page, [LAUGHTER] and found his podcast project and contacted him just out of the blue asking him if he had any materials he was willing to share with me. And he did. And those came in very handy in terms of being able to guide students in the project. But it was just me trying to give them a different way to do something. Some of them still opted to do a paper this semester, but I wanted to give them the opportunity to try something different, to maybe be able to say to someone in an interview that they’ve done something in this more kind of digitalformat. So it was trying to open up the possibilities for them in class.

John: And was this a face-to-face class? Or was this an online class or a hybrid class?

Megan: So the first time I tried to do it was last spring, and that was a hybrid class and was admittedly a disaster. But that class was a disaster for numerous reasons, I think hybrid being the prime driver of that. So this class was entirely in person. I did allow them to work in groups if they wanted to, and some of them did, and some of them didn’t. And I had them do two rounds of podcasts. The others who wanted to, wrote a paper and the percentages were equal. And so a number of them who worked in groups the first time around did not work in groups the second time around.[LAUGHTER] So they got to choose their own topics. I gave them a list of I think 10 topics from that section of the course. And so there were restraints, but I let them propose if they wanted to do a topic that was of interest to them. Somehow I managed to have a Sports Communication major in the class, and when we talked about special purpose districts, I mentioned to him that there are stadium districts where cities are basically using taxpayer dollars to do massive overhauls of stadiums. And so that’s where he went. So it was still in political science. It was still state and local politics, but it was something of much greater interest to him personally than say, term limits and state legislatures.

Rebecca: Imagine that.

Megan: I know, shocking, right?

John: So, the students worked in groups, how large were the groups that they work in for these podcasts.

Megan: So, I proposed having them work in twos. This class is a 300-level political science class, but it has historically been required for the history secondary education students and criminal justice students. and poli sci students have gotten more interested in state and local politics, but I don’t think they think it’s as sexy as national level or international politics, so I think that they realize that’s where the jobs are, they’re getting more interested. So the audience is not kind of a typical political science class. Because of that, I have these history secondary education majors, who because of how tight their curriculum is, are in classes all the time and know each other really well and work together and collaborate pretty frequently. And so I did allow one group of three to work together. So I basically increased the requirements. So if they worked in a group of one, they had to have eight peer reviewed sources that they could point to in the script. If they were a group of two, they had to have 12 sources, and then this threesome had to have 16 sources. And then it went from a 10-minute requirement to 15 and a requirement to 20 minutes for that three-person group and the three-person group was actually probably the best podcast I got. And I obviously can’t attribute it to whether it was just the number of them or they’ve also been some of the best students in the class this semester, just generally, so I wasn’t surprised that they did a good job anyway.

John: So you mentioned a script. Did you have students submit a script before they recorded or was that done after the fact?

Megan: So kind of both. I had them pick a topic and then I had them submit either an outline or a script and kind of gave the pros and cons, where an outline is obviously a little more freewheeling and allows for a little more conversational style in the recording, whereas a script would be much more definitive, they wouldn’t be scrambling for words necessarily. So they’d probably have fewer filler words and they could be sure that they weren’t fading off and losing track of what they were saying. So I gave them the option of either, I think the students who wrote scripts just generally did better. So I don’t know if in the future when I do this again, if I’m going to get the option of an outline, or if I’m just going to make them write a script, because those seem to just perform better, but with the script I made them include work cited, and they had to tell me where in the script or where in the outline which source connected to that material. So I was trying to make sure that they were still using peer reviewed sources, they could obviously use stuff from outside of that. But I wanted to make sure they were still using peer reviewed sources, the way that my policy analysis paper kids were. But letting them do it in this less structured style, in comparison to like an eight to 10 page policy analysis paper.

Rebecca: How did students respond to having these options?

Megan: I was a little surprised at how few students wanted to do the paper. [LAUGHTER] I don’t know if they’re just burned out from… I assume the last two years has just been a lot of online writing assignments, for instance, and so they were just scrambling at anything that didn’t involve them having to write in such a structured way. So I have relatively few students select the paper option. So I’d say it was probably three quarters picked the podcast and a quarter picked the paper. And the ones that picked the paper… my pattern deducing… seem to be the seniors in the class. And I think they just wanted to get their paper done and be done and not necessarily have to coordinate with other people. And maybe they have prior experience with less successful group work, for instance, and they were just: “I’m going to trust myself.” But that was kind of the pattern, where the underclassmen were more likely to do the podcast and the ones that seem to be picking the paper were the seniors.

John: Was there any apprehension about recording a podcast? Because when I’ve tried doing this, I know students are often a little bit anxious about things like, “I don’t know if I have the technical skills or have theequipment to do that.”

Megan: Yeah, well, what was great was in the material you sent me, you sent me a lot of options that students could use. So for instance, regardless of what they submitted to me, in terms of the outline, I have them use, I think it was Otter’s transcription. And so they were using some of the sources that you sent to me. And so I think they felt more comfortable. And as they went, I think, obviously, from the first round to the second round, the quality of the recordings went up. And some of them realized that there’s ways on their smartphones to record and that it will partly transcribe for them. So I think they got better as they went, I didn’t try to ding them too much for production value in the rubric. So there is stuff in there just about like, “Please don’t have insane amounts of background noise [LAUGHTER] in your podcast. Maybe don’t record it in your car…” or something like that. So I tried to have a kind of minimum standard, but I wasn’t going to hold it against them if it was kind of fuzzy audio, for instance. But they actually didn’t seem all that apprehensive about the idea. They were better at it than I would have felt.

Rebecca: So the burning question is: “Did they move away from so much opinion and they’re more neutral? Or did they stay pretty opinionated? [LAUGHTER]

Megan: So actually, it went better than I thought it was going to, because spring 2021, when I tried this the first time around, I could not get them out of being on their soapboxes. And when I created the instructions for the policy analysis paper, I frame it as though you were working for a state legislator who knows nothing about the policy topic you’re writing about. And they want a policy brief from you and then recommendations at the end. So, the recommendations part is the “opinion” part. But it’s got to be based in all of the research that you’ve talked about earlier. So if it were about legislative term limits, political science agrees on very little, but this is one thing there’s kind of universal agreement on is that they are bad, and they backfire and do the exact opposite of what we want. So if that’s what the research is finally saying, then the idea is that you would recommend to the state legislator to vote against instituting term limits in the state. So I found that they were generally able to do that… it took the scripts, that initial round, to be like, “some of this language is getting a little feisty,” and “some of this, I’m not seeing any citations behind it, so, as far as I’m concerned, it’s reading like your personal opinion.” So I think that stuff was pretty necessary to get them to tone it down. I also had them submit draft recordings before the final recording. So I could ensure that the script was improved upon for the recording and so I could direct them if they were starting to go a little too far into the opinion editorial page of the newspaper. And so they were generally pretty good at it. If anything, I think they might have been overly cautious by the end of it, in that they had all this evidence about something leading to something and it was kind of repetitive so… confident that that’s actually what’s happening and still feeling like they have to do a both sides-ism. So I think I’m gonna have to try to work on that to instill in them that “No, you can take a position at the end, it’s just got to be based on the evidence you presented earlier, instead of just constantly pontificating,”

John: …and once you have your students do that, could you have them work with some journalists out there? [LAUGHTER]

Megan: Yeah, as somebody who gets interviewed pretty frequently with local media, I get a little frustrated with the both sides-ism. And yesterday, I got interviewed a lot about the Roe draft. And the reporters kept wanting to talk about the leak. And I was like, “No, the leak is not the important part, guys.” So yes, I understand some frustration there.

Rebecca: Can you talk a little bit about how students either shared or heard each other’s podcasts or whether or not the podcasts were shared more broadly.

Megan: So, inside Canvas, which is the learning management software, Bradley uses, for the ones who gave me permission to share, I posted the files inside of Canvas. And then, strangely, and I don’t know if it was because they were maybe afraid of the quality of it, some of them let me share one podcast, but not the other podcasts. And it wasn’t necessarily like, they wouldn’t let me share the first and they would the second, I just think they thought I like this one better, and so you can let people share this one. And I didn’t like this one, and you can’t share this one. So I just put them on the Canvas website. A couple of them told me, the ones who got maybe low Bs, for instance, on the first one, because everyone did pretty well… the ones who got maybe high Cs, low Bs, on the first one, they told me, they went and listened to some of the other podcasts just to kind of see what the universe looked like. And one of them went, “Yeah, I realized I need to step up my game.” [LAUGHTER] And so I think it was useful from that perspective, I don’t necessarily think they were listening to it to learn about the topic that their classmates have done. But I do think it made some of them realize the quality of their work could have been improved If this is the comparison point.

John: I think that’s a useful benefit of any type of peer review of other people’s work, that when they get to see what other people are doing, they might feel better about their own work. But more typically, they realize that there were things they could have done better. And that’s, I think, a useful experience for everyone, including faculty.

Megan: Well, I think students don’t even realize that for all intents and purposes, we have to use them as guinea pigs from semester to semester to make a class better and to improve. So I realized that maybe the pure hybrid format of spring 2021 was not a good time to maybe experiment with assignments. And so it made more sense to try something now. And it’s unfortunate that those kids maybe didn’t get the best experience, but they are our little guinea pigs, and we also need to learn from their work to see how we can make their work better by improving our assignments.

John: One of the issues I’ve had when I’ve done this, I’ve only used it in online classes so far, mostly because my face-to-face classes are relatively large and I couldn’t listen to two or three hundred of these. But one of the issues I had was that for many students in the online classes, during the depths of the pandemic, it was the only time they really got to talk to other students at the same time and I ended up with these incredibly long draft recordings, sometimes, like 30 or 40 minutes for a podcast that was supposed to be quite a bit shorter. And it did add to the amount of time it took to provide feedback. And included in the rubric was a great penalty if it was too short or too long. So I had to remind them of that. It was a tiny penalty, I think the length was only like five or 10% or so of the rubric score, but I felt bad docking them for that, because when I listened to it, it was clear that they were just enjoying getting to know each other and they were having these great conversations and getting to know their classmates. On the other hand, the focus could have been a little bit tighter. And that is one of the trade offs about having a script versus something which is a little more freeform. But it was really encouraging to hear the connections that students were forming. Although, after many hours of this, I would have appreciated them being a little more concise in some of that discussion.

Megan: Well, to your point. I’m curious, I’m not teaching the summer, but I am teaching an online Intro to American Government class this fall. And when I’ve taught it online before I just used forum postings. And it’s a lot of “I agree with this person,” even though you have directions that tell them not to do this, “I agree with what this person said.” And I’m kind of wondering, and thinking about tweaking this for the fall of kind of doing these voice responses, in hopes that it might limit some of that just repetitive nature and get maybe something a little bit more substantive. Plus, it’s more interesting for me than just reading the same post over and over and over again. And because it’s in an online environment, and it’s asynchronous, though I do have weekly benchmarks so they can access everything all at once, I think it would allow them to have a little bit more of the interaction than they get into the standard asynchronous typical shell. So it’s nice to hear that. I think I would also then have to say it was only supposed to be a 300 word post. So that’s only like maybe two paragraphs so we really don’t need to give a War and Peace sort of opinion. But maybe that would give them some of that more conversational style and make them feel like they’re at least possibly getting to know some classmates, ideally with the idea that maybe they can talk to each other and go over course material instead of being in their own little silos.

Rebecca: There’s something about hearing a voice, or seeing a face that can make all the difference. Of course, from your end, if you just make sure they have to post things in accessible format, you can either listen or read, whichever might be faster. [LAUGHTER]

Megan: What are you implying Rebecca? [LAUGHTER] It was more fun to grade these for sure, it was way more fun to grade these than a standard paper, without a doubt, because I could listen and giggle. [LAUGHTER] Because some of them would throw in a little snarky bit. And you don’t get to have that in these really structured formal papers. So, for me, grading wise, it was definitely more enjoyable.

John: For me as well, it was much more fun. And my impression was students had a lot more fun with it than they would have had doing a written assignment.

Megan: Yeah, I still feel like I’m going to continue to allow them to do a written paper. I know, if I had been in their shoes, I probably would have still pursued a written paper. And I had a couple students in there who were just quiet as church mice. And were probably never going to have the self confidence to even attempt a recording. So I think I still need to provide the option, which I did not do last spring. And I think that’s another reason why it might not have been successful is just given the… I mean, you guys know… class personalities vary wildly. And so that class was just very quiet and reserved and not super engaged. And so podcasts probably not the best approach in that class, in hindsight, but I didn’t know that before they got into the class, and I had built the syllabus.

Rebecca: Yeah, those surprises do happen.

Megan: They do, they do. And it’s a little difficult to overhaul your syllabus quite that radically in the middle of the semester.

John: There is something to be said, though, for pushing students a little out of their comfort zone. And in fact, this podcast, in part, got started because of a similar experience that I had, where I was teaching in the Duke Talent Identification Program…

Megan: I remember that.

John: …and they asked me to be on a podcast they had just started. And I said, “Well, I’m really busy, I don’t really have time for this, and I don’t think I’d really be the best person.” So I gave them a list of people’s names who they should contact to be on this. And they said, “Okay, we’ll contact them, too. But we’d like to interview you.” And after trying to get out of it for a while I agreed to do it, [LAUGHTER] and then realized it wasn’t all that bad. And then I came back from Duke that summer and Rebecca and I were talking and I said, “You know I did this podcast and maybe this is something we might want to consider.” And it’s one of the factors that led into this. I wouldn’t have probably had been doing the podcast had I not been pressured a little bit.

Megan: [LAUGHTER] I feel something similar. 18 year old me would not have done the podcast option. 35 year old me who’s done probably 80 media interviews over the last few years…much more competent doing it now.[LAUGHTER]

Rebecca: I think one of the things that I really responded to that you were saying, Megan, is that you were offering options. And that there were two that you did two podcasts or two papers and that if students chose a paper the first time but then heard podcasts, there’s a second thing. So they could do perhaps one of each, right?

Megan: Yeah, so there was more flexibility. I did not have any of them do that. But at least it was a possibility for them. I feel like I don’t know if they misread the syllabus, but it was once I picked a path that is my path, I am locked in for that path. But there was the possibility of it. So maybe some of them in the future will get maybe a little more courageous and go from a paper to a podcast.

Rebecca: Or maybe they go from a podcast to a paper

Megan: …to a paper.

Rebecca: whatever works for them. [LAUGHTER]

Megan: Also true, whatever is most appealing to their preferences.

Rebecca: I really like what you were both saying too, about personalities of students coming out. And that when they might be writing a more traditional paper, it’s just like entire personhood just disappears. And that having that kind of positionality a little bit come out and their personality come out helps us to get to know our students better and to help them get to know each other better when they’re reviewing each other’s work.

Megan: Yeah, there were some students in the class who in class itself were really quiet and then I would hear these little snarky asides in their podcasts and be like, where’s that in class, I want that in class, please give it to me in class.

John: And they would often make connections to their own lives. They were trying to connect their own experiences to what they were learning in class, at least in the podcasts they were doing for me, and those are exactly the type of connections we try to encourage students to make so that they recognize the salience of what they’re studying. I think that was really helpful.

Megan: Yeah, with my history, secondary education students. In the first half of the class, we talked about state-level interest groups. And so I gave them the option to talk about the NEA and the AFT. And most of them picked it because I think they all know that they’re probably future members of one, if not both of those organizations. And I don’t think any of them realized how different those two organizations are, I don’t think they realized not only currently how different they are, but the histories and the motivations behind them are entirely different. And so I think some of them may not join both of those organizations now, [LAUGHTER] when they become teachers, because I don’t think they like the motivations of one group necessarily versus another group. So I do like that maybe this is actually going to impact their workplace environments, and actually how they choose to behave. Same thing with the second half of the course, we talk about tax policy, which I love… shockingly, not of super interest to them… and obviously, property taxes are one of the major sources of education funding for K through 12. And so a lot of them picked that. And they knew maybe that it was bad, I don’t think they realized how bad it was. And at least in theory, some of them seem to have a little fire lit under them. How quickly that the real world maybe extinguishes that is a different story. But at least for now, I think there’s a lot of desire, at least within these particular students, to try to change school funding formulas, for instance. So I actually looked at the roster in advance of the class starting and looked at the majors of the students to try to find topics that were relevant to state and local politics as a political science class, but that students of those majors would actually gravitate toward.

Rebecca: To me that seems like one of the most meaningful choices that you made in your assignment design, because that really hooks a student and keeps them engaged.

Megan: Yeah, forcing them to talk about a topic they do not care about is hugely problematic. I teach our research methods class, and basically, if it’s a quantitative social science paper, it counts. I don’t care what topic it is. And they’re just mind boggled. So one of them, he’s a political science major, but he’s writing his paper on how video games affect stress levels in people. And so they just get to poke around in stuff that they don’t feel like they have permission to poke around in otherwise.

Rebecca: The other thing that I found interesting as a design faculty who does similar things, maybe not a podcast, but we do things that are out in the public, and we might share them, is that I often give models for students to look at that are professional, we might even analyze those together. But it’s not until they see each other’s that all the light bulbs go on. [LAUGHTER] It’s something about seeing a peer get it that all of a sudden helps bring the rest of the students along. And so they’re always clamoring for getting to see each other’s work. And it does improve the overall quality of the work, in my experience overall….

Megan: Yeah.

Rebecca: …despite the fact that they might have these professional models to look at.

Megan: Yeah, I did, because of, again, John’s instructions, I found state and local related podcasts, and linked to some of them. So they could see how they’re talking about policies, but not being super opinionated about them. And I’m looking forward to now that I have permission to share some of these, I teach this class every spring next spring, being able to give them these models of colleagues basically having done this work. So that, yeah, it doesn’t have the same production quality, and there’s no intro music and ad breaks, but they can see that their classmates have managed to do well on this. And they too, can do well on this. I mean, I always provide sample papers, I get permission from students and remove all their identifying information and post those so that students can see like “You can write a research design in my research methods class, it is possible. This was an A, this is what it takes to get an A.” So I’m glad that a few of them gave me permission to share their podcasts. And I think I’m going to share some of the better ones and some of the less better ones so that they can see for themselves, the spectrum of possibility. And if they’re cool with just putting in somewhat minimal effort, then that’s what this podcast sounds like. And if you want to put in the effort that’s gonna get you an A, that’s what this podcast sounded like.

John: One of the things that my students have commented on at the end of the class was that some of them have decided that they really enjoy podcasting, and they started their own or they plan to do one in the future…

Megan: Wow.

John: …and a few of them have also said, “I never listened to podcasts before, but now I’m listening to these podcasts.” So I was really impressed. But it did have these other side effects that I didn’t really anticipate it having.

Megan: I can’t say any of them have told me that but I’d love for at least for them to listen to some more podcasts because clearly, that’s all I listen to in the car. [LAUGHTER] So many podcasts.

Rebecca: I’ve had similar experiences. John, although I haven’t taught a podcast class. I’ve introduced students to podcasts as part of learning materials.

Megan: Yeah.

Rebecca: …and having assignments… and many of them say that they really enjoy that format more than others, but they may have never really experienced it previously.

Megan: Yeah, I can assign them a 10-page article or I can assign them a 30-minute podcast. They definitely like the 30-minute podcast better. They seem to actually listen to it in a way that they don’t with the reading. So yeah, I have been more and more frequently been trying to find either like 5-minute local NPR stories or outright organized podcasts for them to listen to,

John: I’ve been doing more of the same. And I try to find podcasts that have both the audio and a transcript, so that people can choose a modality depending on where they’re working and reading. In some cases, it may be hard to find the time to listen to audio, or they may be constrained in some way and they prefer reading the text. And in other cases, students would much prefer listening to a podcast while they’re walking or exercising, or doing something else. So they have appreciated the choice when it’s used as a basis for discussions or some other assignment.

Megan: I hadn’t even thought of that. But I’m gonna have to think about that for the syllabi for the fall.

Rebecca: The transcripts are really helpful too, because if you are listening, and then you hear the name of something, or you’re not really quite sure how to spell it, or whatever, the transcripts can be really helpful… if those transcripts have been edited, of course.

John: One thing I’ve always been amazed by are the number of people who say they only read the transcript, they never listen to the podcast. Because what I enjoy about it is the narrative and the ability to focus on a conversation while I’m driving or walking or something similar. And reading the transcript would be very much like all the other reading I do, and it just wouldn’t seem as interesting.

Megan: I agree, but to each his own.

Rebecca: There’s a few podcasts that I listen to regularly that I might go back and revisit in transcripts to pull out some notes of things that I wanted to remember. And so I really get very frustrated when podcasts don’t have transcripts for me to do that.

Megan: [LAUGHTER] Fair.

Rebecca: To support my needs.

Megan: This is about me. Thank you very much.

John: It’s important for accessibility purposes as well.

Rebecca: Yeah, definitely. So Megan, you shared a couple of things that you might want to do differently in the future related to this assignment, do you have any other thoughts about how you might frame or structure the assignment a bit differently to continue producing excellent podcasts and your classes?

Megan: I’m curious about the idea of… It’s a 300-level class, and I feel like providing them the topics was a little hand holdy. And so I’m trying to figure out if I should let them pick the topics, because then they have to work a little harder to figure out what might constitute state and local politics. Because I think you can see that in pretty broad ways. But I worry that if I do that, then they’re just going to take some really black and white literal approaches. So I’m not entirely sure what I’m going to do on the topics front of things. And it might just, again, depend on who’s in the class. I think if it were more of a straight up political science class, I’d let them maybe wander a bit more. But since that’s not the audience of this class, typically, I might try to give them a little more structure, since they’re probably of all people gonna go, “I don’t know what you want for me in a class that’s not directly my major.” So that’s thing number one. I am also trying to figure out for the groups, for when they recorded together, given that some of the groups fell apart in the second round, I’ve never had great success with peer review, because they don’t seem to want to be telling the truth about each other. And so I’m trying to figure out how to get around that because it was pretty obvious with one group in particular that someone was doing all of the heavy lifting, but that person wouldn’t fess up to it. And so I’m trying to figure out if I should allow them to work in groups, and if so, how to try to ensure that the workload is being done somewhat more equitably. Because when I’ve had them work on group projects that have involved writing, for instance, I can follow the Google Doc, because I’ve required them to give me access to something like that and I can see who’s adding what. I’m less able to do that in this scenario. So I’m trying to figure out how to… maybe in the script writing process, still have them share it as a Google doc so I can follow the trajectory of who’s adding what, when, in an effort to try to get a better handle on making sure that one person is not being carried through the assignment. And that’s just, I think, a general struggle with trying to grade group work assignments. But that’s the only thing that I can think of right now and I’d love advice. But the only thing that I can think of right now to try to address that a little bit. And I knew it was gonna be a problem, I just didn’t know quite what to do with it. [LAUGHTER]

John: While you can follow the editing history, that can be a bit of a tedious path through that. What I’ve generally asked students to do is just to use a color code where they pick a dark color, so there’s still good visual contrast to meet accessibility issues. But they each have their own color that they write their text in. So when they write a section of a document, just have them block it and choose their color. And then when you read through it, all the dark blue will be from one person, the dark green will be from another person, and the purple will be from a third and it makes it a whole lot easier to evaluate the individual contributions. And that’s worked really well.

Megan: That is beautifully simple. I appreciate that. Thank you.

Rebecca: I do a couple of things, also, because I do a lot of group work, and group work that’s not always visible. I often have shared documents where it makes sense and ways of documenting it. But I’ve also done things like having students keep timesheets, just recording what they’re doing and when and how long it takes with the frame that it might be helpful for them to better understand where they use their time. And so that sometimes is helpful. We also tend to do things in design more like a process document. So it documents the process and milestones and summaries of what they’ve contributed at various points. And so I find those kinds of documents really helpful to understand what people are doing. And I always request them to provide a little bit of information about why they made certain decisions. And as soon as you do that, then you know who did it, it becomes very clear. And the other thing that I do is a Google form as review of the other collaborators. But I do things like ask questions about how willing they were to accept feedback, what was their greatest contribution? So it’s framed in a little bit different way than maybe a traditional rating system? Like, did they do all the things?

Megan: Yeah, it sounds more, what did you do versus what did your partners do?

Rebecca: Yeah. It’s more aligned with how you might do evaluations in a workplace or something rather than maybe traditional peer-to-peer evaluation.

Megan: Those are awesome. Thank you.

Rebecca: Megan, was there anything else that you wanted to make sure we talked about?

Megan: No, I just want to thank John for being so willing to share his materials with me because I would have been floundering about how to start.

John: I was really happy that someone requested it.

Megan: It was great. Thank you so much.

Rebecca: Well, we always wrap up by asking: “What’s next?”

Megan: I think I’ve got a lot of material to work with with state and local politics. Given the recent political landscape, maybe too much material to work with and state local politics. I think one of the things in the future and I’m nowhere near this yet, I’m interested in letting them explore alternate methods of this podcasting style. So maybe actually interviewing local candidates, working with political parties, the League of Women Voters is actually very, very active in Peoria, and they still subscribe to being non-partisan, so it would be nice to try to team up with them and see if they’ve got some sort of outreach campaign they’d like to do. So I’m thinking of trying to really expand, but next year is busy. So that might be the year after.

Rebecca: Well, thank you so much for joining us.

Megan: Thank you for having me.

John: Thank you very much. And I’m really thrilled that someone actually found that material useful. [LAUGHTER]

Megan: It was, it was great. [LAUGHTER]

John: It’s an activity I’ve been doing with my class for three years now, and it’s been working really well, and I’ve really enjoyed it. And many of the students really have to.

Megan: Yeah, time two was the charm for me.

Rebecca: Yeah, you have to have a practice round.

Megan: Yeah, unfortunately, those students were guinea pigs, but I learned from them.

Megan: Thank you for having me. I appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle.

[MUSIC]

236. ePortfolios

As David Wiley has noted, “disposable assignments” often have small impacts on student learning. In this episode Nikki Wilson Clasby joins us to  discuss how one campus has used ePortfolios to create authentic learning experiences in their English composition courses.

Nikki is the coordinator of the English Composition Program at SUNY New Paltz.

Show Notes

Transcript

Rebecca: As David Wiley has noted, “disposable assignments” often have small impacts on student learning. In this episode we discuss how one campus has used ePortfolios to create authentic learning experiences in their English composition courses.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Nikki Wilson Clasby. Nikki is the coordinator of the English Composition Program at SUNY New Paltz. Welcome, Nikki.

Nikki: Thank you. It’s good to be here.

John: Thanks for joining us. Our teas today are… Nikki, are you drinking tea?

Nikki: I am certainly drinking tea, yes.

Rebecca: Woohoo!

Nikki: Would you like to know what it is?

Rebecca: Yes!

John: Yes!

Nikki: [LAUGHTER] So this is an exotic blend called Tetley, a very strong British brew, which we Brits love, unless you’re a PG Tips fan, but Tetley’s pretty up there. And I have it with 2% milk which is the best way to drink it.

John: Most of our colleagues on campus from England tend to drink Yorkshire Gold.

Nikki: Mmhmm.

John: They seem to prefer that to the other options.

Nikki: Yes, well I am a Yorkshire lass, but I have to say Tetley has that kick that I need. [LAUGHTER]

Rebecca: Good to know. I think today I have Scottish afternoon tea.

Nikki: Ooh!

John: And continuing with the theme I have an Irish breakfast tea from Twinings.

Rebecca: Oh!

Nikki: Oh very nice, that’s a good one too. I like that one.

Rebecca: This crew needs some strong stuff today.

Nikki: We need some scones now. [LAUGHTER] Then it will be complete.

John: We’re recording this at 12:30 today, and I’ve already had five meetings today including a class.

Rebecca: This is my second pot.

Nikki: I’m impressed. [LAUGHTER] You can come and have tea with me any day.

Rebecca: [LAUGHTER] Perfect. So we’ve invited you here today to discuss the use of ePortfolios in the Composition Program at SUNY New Paltz. But first can you tell us a little bit about your role at New Paltz?

Nikki: Yes, I am the Coordinator of the Composition Program. I stepped into this role two years ago, and I am also a lecturer. And so I teach mainly our upper-level writing and rhetoric courses where I specialize in visual rhetoric. And I also teach courses in what we call Practical Writing and Design which is a new course dealing with a sort of blend of graphics and writing. And I also teach a FIG, a First-year Interest Group, for the Communication Disorders. And I run practicum for our TAs.

Rebecca: So you’re not busy or anything?

Nikki: I’m not busy at all, no. Plenty of time for drinking tea. [LAUGHTER]

John: And plenty of reason to drink that tea with the caffeine.

Nikki: Which is why I drink Tetley, yes. [LAUGHTER]

John: So, we invited you here, though, because we heard about the common use of a WordPress site for the creation of student ePortfolios. And I think the first question we have to ask is… How did you possibly get agreement within a department on the use of one platform?

Nikki: [LAUGHTER] Actually it’s pretty simple, there’s no drama involved here. So in 2019 the composition committee reworked our three-credit English 180 Composition II course to a four-credit GE course, which we retitled English 170 Writing & Rhetoric. And we had been using print portfolios for a long time, and so during the process of revamping our course the composition committee reviewed how we could improve our portfolio assessment. And Matt Newcomb, who was the coordinator at the time, and I had been long advocating for ePortfolios. So during our meetings we decided that it would be a good time, seeing as we were revamping this course, to introduce ePortfolios into our curriculum. And we’d looked at options in Blackboard, but they were just too…

John: Awful? [LAUGHTER]

Nikki: Yes, too awful, but just by happenstance and pure serendipitous coincidence the university at this time decided to just opt for a CampusPress system, and they adopted Hawksites. And so it was made readily available for us to use. And so the timing was perfect, we just jumped on it right away and said, “Yes, this is what we want. This is the way we’re going to go.” So because Matt and I had been advocating for ePortfolios for so long it was pretty simple to get our program on board with the project.

John: And we should note that the Hawks are the campus mascot for SUNY New Paltz. And that Hawksites is just a campus-wide instance of WordPress, I believe.

Nikki: It is, yes. That’s exactly what it is. We just gave it the name Hawksites. Yes, it’s a campus-based university blogging website and ePortfolio tool.

Rebecca: Are students developing these in this more beginning course, and then working on the same portfolio throughout their entire curriculum?

Nikki: Well, this is what’s really interesting. So the faculty are allowed to use the ePortfolios as a tool for however they want to integrate it into their program, so they have free rein to do whatever they want with it. And we all use it in different ways and to different degrees, depending on our comfort level with technology, and how it fits into our curriculum. But as a composition program we use the ePortfolios for assessment purposes, so I can tell you a little bit about how that is organized. So whereas we can have free rein to use them however we want, we do have some very specifics that we need for our assessments. Would you like me to tell you about those?

Rebecca: Love to hear about those.

Nikki: [LAUGHTER] Okay, so for English 160 which is the basic, very first writing course, students have to go through this ePortfolio assessment at the end of the semester to determine whether or not they are fit to move on to English 170. So this is the tool that we use to make sure those students are ready for the more vigorous program. So for that 160 assessment process the students have to upload to the ePortfolio, or at least be able to visibly show on the ePortfolio, they have to have two of their strongest assignments, and they also have to have their revisions for those assignments. They can choose whichever ones they want to put on, but they have to be two major assignments. The only requirement is that they have to show that they have been able to write in different modes, different genres for different rhetorical situations. And there needs to be an element of research in their citation, you know the beginning stages of that research process. And obviously we’ll be looking for the standard of their writing as well, that’s why the revision aspect is really important. So that’s what they’re required to do for the ePortfolio. And then the 170 students, they have different requirements. But let me just backtrack just for one second. So across the board for 160 and 170 as part of the ePortfolio requirements, all the students have to create a reflective cover letter that goes up front in their ePortfolio. They write that reflective cover letter at the end of the semester and we give them questions, guidelines as to what to tackle. And what we want from them is a sort of critical overview of their progress during the course. And they have to cite examples of their writing to prove their case. So it’s a persuasive letter, and our assessors read that first, so they’ll read that reflective cover letter. And that gives us a very clear sense of what the student understands about their writing process, and that makes us feel a little bit better about whether they’re ready for 170 or not. It shows that they’re applying the techniques and skills that they’ve learned throughout the semester to that cover letter. So the 170 students, we have a very specific framework for our 170 Writing & Rhetoric program, it’s based upon a wicked question. So a wicked question might be… What should we eat? Or how do we save the world? Or what does it mean to be human in a digital landscape? And all professors can choose whichever kind of wicked question they want, that they’re excited about, and then they base all of their assignments around that wicked question. That gives us a lot of flexibility for Writing & Rhetoric, which is wonderful. So the semester is divided into two sections. We basically require two major assignments that are argument research based, and then each of those two large assignments has two smaller assignments that help students gear into those big assignments. So, for instance, you might have a proposal with an annotated bibliography that leads to a research paper. So students have to choose one of those sections. So in the ePortfolio we want to see two smaller assignments leading to a large assignment. We don’t need revisions at the stage for 170, we acknowledge that revision is part of the process, and that they will be revising anyway for those papers. So that’s the structure of the assignment sequences for those two ePortfolios. And then beyond that we add other things into the ePortfolios as we see fit. So, for instance, this semester we have our internal assessment which is for our 170 students, and that’s on basic critical reading. So that’s kind of how our ePortfolios are set up. And then at the end of the semester we have a system set up where we review each other’s ePortfolios based on a common rubric that we have put together. So that’s basically how it works.

John: It sounds like a great approach in ensuring standardization across their classes and making sure that all students meet the requirements to move on.

Nikki: It is, it’s very effective. We have lots of conversations afterwards about who’s on the cusp, borderline cases, and so it’s very democratically pieced together. And then of course we have to work on individual cases of students who are failing for various reasons. And it’s a pretty good system, and it’s been very effective over the last two years that we’ve been using it.

John: This is more of a technical question about the organization… Is each site organized by class or is it by students? In other words, does a student have their own WordPress account that they use and create an ePortfolio that is unique to them across all their classes? Or is there a class site where all the students in the class post their work? Or is it some combination of the two?

Nikki: So what happens for us is that in our composition program, we have a template on Hawksites, and our students create an account through Hawksites, and they are given the template that they have to use. So they are essentially creating their own account on Hawksites for our classes, and it’s unique to them. It’s not something that we share with other classes, this is specifically for our class. Does that make sense?

John: Mmhmm.

Rebecca: So just to clarify, if a student was in another class, in another subject area, they may have an additional site.

Nikki: Correct, students can create as many accounts as they want on Hawksites for individual programs. I have about 10. [LAUGHTER] It’s fantastic. It’s such a good resource, we love it.

John: I’m hosting a variety of WordPress sites as well for different purposes.

Rebecca: Me too. One thing I think that’s always important to ask when we’re talking about ePortfolios, is whether or not that student work is public to everyone, public just to members of the community, your classroom community? Or are they private? Or do students have a choice of the privacy settings?

Nikki: The students have a choice of privacy settings. But what we encourage students to do is to select the option that allows only people within our university that have a university ID and login to be able to access it, and only the people that the students give the link to, or the people that the faculty give the link to. And this allows us to share those ePortfolios amongst the people who are going to be assessing. So there is some choice for the students, but it also gives us the option to share easily amongst our colleagues. But I want to add something into this too, that within our template that we had created at Hawksites, we have a permissions policy embedded into the site. And that is a basic form and students can sign it, they can say yes or no. We ask the students, “Would you mind If we shared your portfolio for teaching purposes? Would you mind if we shared some of your work for research or for teaching methods?” And students can pick “yes” or “no” for all of those. And that’s nice to have that there on the ePortfolio, so whenever we’re looking for examples we can check the permissions pledge and see who’s agreed and who’s not agreed, and then of course we respect the students that haven’t signed it. So there are some levels of privacy within our cohort of teaching. There is a blog function on our Hawksite. It’s up to the professor whether they use the blog or not. But as they stand, the students can’t see each other’s ePortfolios, those are private. But there is a blog function within Hawksites, and faculty can choose whether they decide to share that blog function with other students or not. I have used that function for a different project, but I haven’t seen anybody take advantage of that because we also have Blackboard which has its own blog function too.

Rebecca: How have students responded to the idea of using ePortfolios?

Nikki: That’s a really good question. It very much depends on the instructor and how the instructor teaches the ePortfolio component. I can tell you that for TAs who are new at trying to grapple with this technology and pedagogy, some of them have in the past waited till the very end of the semester to have the students upload their work. It’s too stressful for students, they can’t handle it. It’s a lot of work to put a good portfolio together. So I make the TAs have the students sign up for an account within the first two weeks of the semester. And I encourage the TAs to find ways to get the students to engage with their ePortfolio on a low-stakes non-graded level just so that they can learn how to use all of the functions and the tools. And also get them in the habit of using their ePortfolio as a working kind of document, and not something that just gets shoved to the end of the semester. So it really depends on how it’s taught. But if you do teach it with those kinds of sensitivities in mind, and you don’t stress the students out, I find my students in particular love using their ePortfolios. They enjoy engaging with them, they enjoy seeing their work look professional on the site, they enjoy the option of using a more web-based writing process for embedding videos, hyperlinks, uploading images, embedding their beautifully designed Google slide presentations into their site. So they do enjoy that process. I give them time in class to do it so it’s very therapeutic for them. But they also appreciate learning some of the real-life skills that comes with curating an ePortfolio, and they recognize that this will help them later. So the enthusiasm for it is pretty high, and most students feel very proud of their ePortfolios by the end of the semester because they have something to show for all of their hard work, and it looks good. So they’ve adopted it really well. My worry about students is they do all this work, and they hit the submit button for grading, and then that paper disappears down the black hole never to resurface, and then they just move on. And it’s a shame because that work is good work, and we want our students to feel like they have a stake in the writing process, they have a stake in scholarship and research, and the ePortfolios provides a really nice platform for allowing them to think of themselves in that respect, and not just the humble student that struggles, if you like. [LAUGHTER] Helps them feel a little bit more professional.

John: David Wiley refers to those types of assignments that students post in Blackboard, or submit their paper at the end of the term and never see again, as disposable assignments. And having something that looks professional that they have access to, and that they can share and feel good about, is something that students really value. I’ve had students write some books in my class, and they really enjoy seeing this final product. It’s something that they can share with their friends, with their parents, with potential employers, and link to on their resumes and so forth or on LinkedIn, and they’ve appreciated that tremendously. I think you do some of the same, right, Rebecca?

Rebecca: Definitely the students love it when it’s like… It’s a real thing, a real shareable thing, with real audiences. [LAUGHTER]

Nikki: Exactly, and that’s the key thing, right? Especially in rhetoric having that real audience, it’s super, super important.

John: And in my experience it leads to a much higher quality of work when they have that non-disposable assignment. Have you seen the same?

Nikki: Yes, I totally agree. There’s a level of accountability that goes on there. So when you’re racing off an essay at the last minute and submitting it, it disappears into the black hole. With the ePortfolio it comes back to hit you in the face, and you can’t put that stuff on the web, you have to go back and revise it. And it’s really nice being in the classroom and having the students respond to your comments and make those revisions. And then you kind of hear the penny dropping, it’s like, “Oh boy, I really didn’t do this very well, I better snatch this up for the ePortfolio.” And it is very reassuring to see that in action. So yeah, it’s lovely.

John: So how have other composition faculty responded? Are they all comfortable with it? Was there any resistance?

Nikki: That’s a really good question. I know, for me, I’ve been involved in this sort of work for a long time, I came from Iowa State University here and we’ve been working with ePortfolios for a long time. And that switch, going from the paper portfolios which I hated [LAUGHTER] sorry, I hated them… Going from the paper portfolios to the ePortfolio, that’s a big mind switch to go through. So we had to work with our faculty, encourage them to set up a Hawksite of their own so they could experiment, help them feel comfortable sharing those Hawksites in the classroom so they could use that as a teaching tool. So initially there was some learning to do, and that’s great, I mean that’s great, that’s fine, perfect. So it took a while to make that switch to ePortfolios, but now that we’ve made that switch, I think we all recognize that it’s so much more accessible, it’s so much easier to organize, it’s so much easier to assess. We’ve only been doing it since the fall of 2019, but I don’t hear any complaints [LAUGHTER] about the ePortfolio. It is part of what we do now. So it’s good.

John: And that was perfectly timed to be ready for the pandemic.

Nikki: You know, it was perfectly timed for that. And what I like about it, and I think what we all agree we like about it, I encourage the faculty to have the students post the links to the faculty right from the very beginning. So that way we can just go in periodically, and we can just monitor what’s happening on there, and then we can give direct feedback to students about it. So it is, it’s a wonderful tool.

Rebecca: One of the things that you’ve mentioned is this template that you share out, and you mentioned some of the permissions that you allow students to choose. Can you talk about some of the other features of the template itself that you share with students? Like, what are some of the things built into it?

Nikki: Yes, so the template has the tabs already constructed so that students don’t have to work out how to recreate those. Obviously we teach them how to generate new tabs, but the basics are already there. So it has a homepage tab, so we encourage students to post a photograph of themselves and think about how they want to present themselves to a general audience as a student. So they have that, we work on that side of things. And then we just have the tab for the reflective cover page. Then we have the tabs for the individual assignments and their revisions,and then we have the permissions tab. And then we also include on the ePortfolio, this is a new feature, during the pandemic we had a lot of issues with attendance and accommodating students who were sick and who were in quarantine, so what we did was we posted the essentials of the course policies on the ePortfolio. And we had students acknowledge and sign that they had read them, and that they understood what those different policies were for attendance, for assignments, for what they needed to do if they were sick, all of those things we put on their ePortfolio site. So it became a quick reference guide for students that they can just pull it up, and they could see what was required of them. But also for us as faculty when students were suffering, or not keeping on track, or getting to the end of the semester and things were not looking good. We could pull that up, we could see who had signed the pledge and we could say, “Look, policy said that you needed to do this, this, and this, and you didn’t do those things.” And so that helps stem the flow of the great appeals at the end of the semester which I have to deal with. So that worked well for keeping students on track, and keeping that information transparent and clear.

John: And you have it in writing, digital writing.

Nikki: We have it in writing, and the students sign it. So it helps them take accountability for their part in this process. They can’t say, “Oh, I didn’t know about that,” when it’s on the ePortfolio and they’ve signed it. It’s like, “Mm, well apparently at some point you did read this.” So that helps. Not all students read it [LAUGHTER] of course, but at least it’s there though, that’s the important thing. Those documents are not buried somewhere else, they’re visible, they’re right up there. And I think that’s really, really important, and I’m really glad that we decided to do that, especially over the pandemic. It’s been helpful.

John: And I know I always read all the terms and conditions when I sign up for a new software package, and so forth.

Nikki: Of course we do. [LAUGHTER] The other thing that I want to add in there is that, for me, some of the professors do this too, but I have my students create a writing journal tab in their ePortfolio, and they have weekly writing journal prompts in there. And I do that so that students have a safe space just to write, and to reflect on what we’re doing in class, and to apply those ideas to material that they’re interested in. And I set that up because I wanted them to feel like they owned their ePortfolio, that it was their ePortfolio, that it was their personal sort of diary, if you like, of all of their work. So that tab is very important for my classes, and my students enjoy doing that kind of work.

Rebecca: You also mentioned earlier that the work looks professional, so I’m assuming then there’s some stylistic things that are built into the portfolio as well. There’s at least a base look for things, no?

Nikki: There is a very basic look, and I would love to be able to include more design tools in the ePortfolio because we don’t have a choice of font style, we can move our images around [LAUGHTER] to a couple of places. It’s very, very rudimentary, and it would be really lovely if we could add a few more tools in there to make it look even better.

John: So everyone in composition has agreed to use templates, but it sounds like they might use them all differently. Is that correct?

Nikki: Yes. Thankfully, even though the design elements are pretty rudimentary, there are some tools to change the actual overall look within the basic template. The students can change the background image, they can change colors, they can personalize it in a way that suits them which is really nice. So yes, that’s fun, and those are good skills to teach the students as well.

John: What about different sections of the course? Is there a standardization in terms of how the platform is used? Or does that vary from instructor to instructor to some extent?

Nikki: It varies from instructor to instructor depending on their comfortability with technology, and how they want to integrate the ePortfolio into their program. I’m not a standardization sort of person, but we do have… the basic elements for assessment are standardized, they have to have those specific elements for assessment. But apart from that they are free to use those ePortfolios as they wish, and that’s the way that I want it to be.

Rebecca: One of the things that might be helpful for listeners too, earlier you were talking about your assessment process, and that people from other sections review work, that you’re reviewing work of the students of other instructors. So I’m wondering if you’d talk a little bit about the logistics of how that actually works. Because I think for some folks it can be such a big undertaking, so hearing stories of how other people organize those sorts of things can be helpful.

Nikki: Yes, so first of all students have to be eligible for an ePortfolio review, that’s the first step. So students have to have completed all of the assignments and all of their requirements, like the library instruction, oral presentation, all of those things. The student has to have at least a D to be able to be eligible. So that sort of weeds out some of the stuff. And then what the faculty do is we take seven portfolios per class, and that’s a random selection, so you take the first student on your roster, and then every fourth student gets to go in that pile. So each faculty member has seven students randomly selected for ePortfolio assessment. Plus, we have then any student who is borderline, any student that is just clinging on there, or any student that a faculty member is really unsure about, so that goes into the pot too. And then my assistant and I, we create ePortfolio partners and we specifically place, for instance, seasoned faculty members with new TAs. And that’s the way that we do it, so we choose who assesses whose work. And that makes it a very organized system and a fair system, especially for the new TAs who are not sure about what to do, at least they’re working with someone who has experience. So that’s how we do it, and the assessments can take place whenever is convenient for that particular pair, as long as all of the results are all tabulated and submitted by a specific time period. And then after that time period we’ve got some space here to work on ePortfolios that have issues. So once that rudimentary assessment is done then anything anyone is concerned about can be given to my assistant and I, and we’ll go through case by case any of those borderline cases that we’re worried about, we can work on those. So that’s kind of how it works, and it’s a really good system. It works really, really well, it’s very efficient, it’s fair for everybody. At the end of the semester, you know we’re tired, the faculty have already been through all of the ePortfolios and given their verdict, and then we double check with those seven to make sure, it’s really a calibration thing to make sure that everybody’s on the same track. And I need to preface this by saying that all faculty members have to go through a standardized calibration training at two points during the semester, so we make sure that everybody knows how to use the rubric and can apply it effectively. So with those checks and balances it actually works out incredibly well.

John: Are there standard documents that you share with people, and then you see how they evaluated to compare against the benchmarks, for the calibration?

Nikki: Yes, we do. We have a standardized rubric, and then during our retreat sessions we will selectively pick, like, a very, very borderline portfolio for people to assess. We put people in groups, and then we make them grade the ePortfolio with the rubric, and then we discuss it and we talk about what’s working, what’s not working. And if there’s any huge discrepancies in the assessment of those ePortfolios we talk about what was going wrong with those discrepancies. So it’s pretty organized, and it’s pretty efficient.

John: It seems like a really nice way to provide equitable and fair assessments that adhere to the standards that you’re trying to meet. I’m impressed.

Nikki: Thank you.

Rebecca: You talked a little bit about students needing to meet standards to go through the portfolio assessments. Does that essentially equate to their ability to continue on in that particular program?

Nikki: Yes it does. If they’re not meeting the basic requirements for an ePortfolio review, technically it means they’ve failed. And so what we do with those students is we then decide… How did that student fail? Did they fail on their own merit? Or did they fail because they tried and tried and tried but just couldn’t get it? So we have standardized measures here that says, “Okay, so if a student has been trying really hard, and they just didn’t get it, then we will allow that student to repeat the course.” So we have checks and balances there for those students.

John: So this program is used universally in the Composition Program. Have similar practices been adopted by other departments at New Paltz?

Nikki: I’m ashamed to say I don’t know, and the reason for that is because I don’t get out much to see. [LAUGHTER]

John: That’s not uncommon especially during the pandemic.

Nikki: Yeah, I just don’t know, and I feel embarrassed to say that, but I came out of teaching a 4/4 load into this position. So that’ll be one of my next step projects is to figure out who else is using them on campus and talk to them about how they’re using those ePortfolios.

Rebecca: So another thing that is worth considering is… You mentioned that students can choose some privacy settings and things. How long do students have access to these portfolios after they’ve created them?

Nikki: As far as I know students have it for as long as they’re a student.

Rebecca: So we’ve talked about assessment being a primary motivator and maybe some professional skills as being a good motivator for putting ePortfolios in place. But are there other advantages to using student ePortfolios that we should be thinking about?

Nikki: So apart from the ePortfolios for the students being an opportunity to see themselves as professional communicators, to help boost their ethos and their confidence. I think we talked a lot about what the students get from this, but from a faculty’s perspective the ePortfolios are a fantastic tool because they are so accessible, they’re easy to coordinate for assessment, we don’t have to wade through buckets and buckets of paper. And also we don’t have to, [LAUGHTER] I know this sounds like a really minor thing, but when we used to do the paper portfolios we’d do the portfolio assessment, and then we would call the students into our offices to break the news to them whether they’ve passed or not, and give them their paper portfolio back, and a lot of students didn’t come. So we ended up with piles, and piles and piles of portfolios in our offices and it’s like, “Well what do we do with those?” I found it really distressing. You know, if you’ve got four classes and 120 students, and every semester, and then they just pile up, that was distressing. So to switch to the e-system just feels better on my soul, [LAUGHTER] for the planet doing this. But the ePortfolios, they’re just such a good tool for faculty for teaching, for training other faculty, and for sharing what we do with our students with each other, and sharing ideas and seeing what the possibilities are. The ePortfolios just offer so much more potential for pushing what writing and rhetoric is, and what we do with it in the classroom. So from a pedagogical point of view, I can’t imagine going back to paper portfolios. It’s just a fantastically amazing, creative, and soul-satisfying tool to have at your disposal.

John: That’s a really nice, positive note to end on.

Rebecca: So we always wrap up by asking, What’s next?

Nikki: Well, so last summer my colleague Rachel Rigolino and I used Hawksites to develop an online tutor training site because we need more TA tutors in our system to help with the writing program, and that was really successful. And so what we want to do now is to extend that. We would like to develop a Hawksite for our TAs so that we can put all of their innovative teaching ideas into a Hawksite, so that it’s accessible to everybody for sharing ideas. And that’s a really big project. So that’s our next big step, to do that.

Rebecca: Sounds like it’ll be really helpful, and really exciting to work on.

Nikki: I think so. I think it will be vital. [LAUGHTER]

John: This sounds like a really good program, and thank you for joining us, and thank you for sharing this with us.

Nikki: Thank you, it was a pleasure. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle, Annalyn Smith, and Joshua Vega.

[MUSIC]

213. Wicked Students

Much of the training that students receive in college involves working with well-defined problems that can be resolved using the tools and techniques of a specific discipline. In this episode, Paul Hanstedt joins us to discuss strategies that colleges can use to better prepare students to collaborate on the “wicked problems” they will face in the future.

Paul is the Director of the Houston H. Harte Center for Teaching and Learning at Washington and Lee University. He is the author of Creating Wicked Students: Designing Courses for a Complex World, General Education Essentials: A Guide for College Faculty, which is about to go into its second edition, and numerous publications related to general education and writing across the curriculum. He has worked with many colleges and universities in revising their general education requirements.

Shownotes

  • Hanstedt, P. (2018). Creating Wicked Students: Designing Courses for a Complex World. Stylus Publishing, LLC.
  • Hanstedt, P. (2012). General Education Essentials: A Guide for College Faculty. John Wiley & Sons.
  • Hanstedt, P. (2021). Helping Students Understand Our Codes: Designing Inclusive Open Curricula. AAC&U Liberal Education Blog. April 29.
  • Jessica Tinkenberg – twitter
  • Standards of Learning (SOL) – Virginia Department of Education
  • Warner, J. (2018). Why They Can’t Write: Killing the Five-Paragraph Essay and Other Necessities. JHU Press.
  • Blum, S. D., & Kohn, A. (2020). Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). West Virginia University Press.
  • Hanstedt, P. (2020). Higher ed needs to redesign gen ed for the real world. Inside Higher Ed (Opinion). February 10.
  • Zull, J. E. (2002). The Art of Changing the Brain: Enriching the Practice of Teaching by Exploring the Biology of Learning. Stylus Publishing, LLC.
  • Kate McConnell
  • Textbook used at Plymouth State in a wicked problems seminar:  LeBlanc, C. (2019). Tackling Wicked Problems. Plymouth State University.
  • An article by the instructor on the course:  LeBlanc, C. (2019). What is “Tackling a Wicked Problem”? Desert of My Real Life. May 10.

Transcript

John: Much of the training that students receive in college involves working with well-defined problems that can be resolved using the tools and techniques of a specific discipline. In this episode, we examine strategies that colleges can use to better prepare students to collaborate on the “wicked problems” they will face in the future.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners..

[MUSIC]

Rebecca: Our guest today is Paul Hanstedt. Paul is the Director of the Houston H. Harte Center for Teaching and Learning at Washington and Lee University. He is the author of Creating Wicked Students: Designing Courses for a Complex World, General Education Essentials: A Guide for College Faculty, which is about to go into its second edition, and numerous publications related to general education and writing across the curriculum. He has worked with many colleges and universities in revising their general education requirements. Welcome, Paul.

Paul: Thank you for having me. I’m delighted to be here.

John: Today’s teas are… Paul, are you drinking tea?

Paul: I am drinking tea.

Rebecca: Yes, rejoice! [LAUGHTER]

Paul: I’m actually normally a coffee drinker, but I’m drinking tea. [LAUGHTER] I was part of the new educational developers for the POD organization and Jessica Tinklenberg sent out a care package afterwards. And in my care package was a Stash Tea Jasmine Blossom. I’d already had a 16 ounce cappuccino today, I thought I’d better keep it mellow [LAUGHTER] for later in the day.

Rebecca: Sounds nice and relaxing.

Paul: Yeah.

Rebecca: I have Yunnan Jig again. I think it’s becoming one of my new favorites. I don’t have my Golden Monkey around, so I’m going to have to drink something different. But it is also a golden-tipped tea.

John: I’m drinking a peppermint spearmint blend today… taking it easy on the caffeine.

Rebecca: I don’t know why, John, it’s halfway through the semester. What is wrong with you? [LAUGHTER]

John: I need some sleep, it’s been a stressful semester. We’ve invited you here to talk about a couple of things. We saw an article that you posted on April 29 on the AAC&U blog on “Helping Students Understand Our Codes: Designing Open Curriculum.” But we also wanted to talk to you about Creating Wicked Students. When that book came out, we both looked at it and read through it. But we were at a really early stage in the podcast, so we weren’t quite ready to ask people that we didn’t know who were already on our campus or that we were comfortable inviting because they knew of us already. And that’s long been on our list of topics that we wanted to discuss. So maybe we can start with Creating Wicked Students. What is a “wicked student”?

Paul: Good question. Maybe I should define a wicked problem first. A wicked problem originally came out of city planning. And then it was adopted by engineers because they looked at it and they said, “This is exactly what we’re dealing with all the time.” A “wicked problem” is a problem where the dynamics are in flux, they’re shifting. What the problem looks like on Tuesday and what the problem looks like a week from Thursday can be completely different. Oftentimes, the data is incomplete. Oftentimes, previous solutions don’t apply. Very often, wicked problems are problems that must be solved, you can’t let it sit. There’s oftentimes contention about how to solve it, how to fix it. And almost always—and this is where the gen-ed person in me gets all excited—the solutions for a wicked problem are going to require drawing from a bunch of different fields. So the best example, and it’s a horrible example, but everyone will understand it immediately, is COVID. COVID is the perfect, horrible, wicked problem. The dynamics have been changing constantly. Even now, we’re still waiting to get around the bend and say, “Oh, okay now we’ve arrived.” If it were purely a science problem we’d be in great shape because we would have been done early this summer. But there’s politics in play, there’s economics in play, there’s culture in play, there’s religion in play. There’s messaging and communication and images and memes and social media and technology. All of these things are creating a dynamic quality to COVID that makes resolving it very, very difficult. And, of course, we must resolve it. So that’s a wicked problem. A wicked student is a person who can solve that problem [LAUGHTER] when they graduate from college. But in engineering, what they said when they saw wicked problems is, “Our students need wicked competencies, because they’re going to face, once they leave university, they’re going to face these wicked problems in the field.” So much of what we do in education is tame, static, the answer is at the back of the book, there’s clarity. So a wicked student is a student who can move beyond simple answers, move beyond solving problems where they know the answer because they’ve been told the answer, or they know the answer because it’s at the back of the book, or they know the answer because they apply an algorithm and arrive at a solution that’s clear. A wicked student is a student who can deal with uncertainty, who can struggle, who can ask good questions, who can draw from different fields, who can collaborate well, and who can, when they fail, pause, step back, deliberate, reconsider, and try again. I have a colleague in biochemistry, Kyle Friend, who talks about what he wants his students to finally know is that he gets it wrong 70% of the time, and that’s normal. For so many of our students, getting it wrong is bad, it’s failure. And, frankly, that’s our fault because we create that dynamic.

John: Would you say that most curricula is well designed to create wicked students who are able to handle these problems that are not so well defined, that cross disciplinary boundaries?

Paul: No. [LAUGHTER]

Rebecca: The end. [LAUGHTER]

Paul: Yeah. So many of the systems that we’ve put in place—in Virginia, it’s the Standards of Learning tests—they ask for certainty and clarity. I’m running a writing pedagogy seminar on my campus right now and using John Warner’s wonderful book Why They Can’t Write. And he makes a point that the five-paragraph theme, which for those of us with composition and rhetoric backgrounds, is the root of all evil. He says, “Actually no, it’s just the tip of the iceberg. It’s the symptom of the larger problem.” And the larger problem is our desire to mass produce education, to make education easy, to make our grading easy, to make learning easy so students can feel like they’ve mastered it, because they’ve got the five facts, or the 10 facts, or the 11 dates, or the four algorithms that they need to understand in order to do well in the class. So no, I think so much of what we do is tame.

Rebecca: So if we’re saying, “Yes, please,” to these wicked students, what do we need to do in our classrooms? What can we, as faculty, implement in our course structures?

Paul: Yeah, how long you got? [LAUGHTER] So…

Rebecca: I have all day, if you’ve got all the answers!

John: And there’s a good book on that, too, that people can refer to.

Paul: There is a good book on that, yeah. So any number of things. One is finding ways to bring uncertainty, lack of clarity, into the rhythm of the classes that we teach. Years ago, I was running a workshop and a gentleman named Dan Clark who was then at Western Oregon University, he’s not there anymore, and I forget the institution he’s at. But he talked about Monday morning riddles. You know, you walk in, and whatever field you’re in—whether it be psychology, or accounting, or politics—give a problem and put students in groups and have them not come up with one solution but come up with three solutions, three ways to approach it. And ask them to be able to articulate why. And then put them all on the board and discuss them. And the point is not: there is one answer, and there’s one way to get to that answer. And oftentimes there is, let’s recognize that. But learn to play, learn to ideate, learn to work with multiple paths, because sometimes the first path you take isn’t going to work. And I should say, really quickly, I’m always aware that there are people in the room in various fields, whether it be French or chemistry, where they’re saying, “Well, there are facts, and they need to know the facts.” Yes, it’s not about knowing the facts, it’s about whether or not 100% of our time in the class is about content delivery and content reiteration, or if we create space in there for application of that content, and sometimes application of that content and then discussion about that application, where clarity and certainty isn’t necessarily achieved because that’s not the point. So it can be in our daily practices, we can create exam questions like that. I can, in a literature class, give students a poem and say, “We’ve been studying the Romantic poets, which one of the Romantic poets wrote this poem?” The fact that they’ve never seen the poem isn’t the point. Actually, I might choose to include a poem from a poet they’ve never read. In fact, I might include a poem from a poet who’s not even a Romantic. Thing is, can they analyze? Can they think? Can they explore? Are they being deliberate about their exploration? And what I’m grading them on is not their ability to get to the answer, it’s their ability to travel to make that journey. I could keep going if you want me to. [LAUGHTER] We can talk about paper assignments, we can talk about text selection, we can talk about the goals we create for a course. When do our goals move beyond content delivery? When do our goals move beyond application? When do our goals move into our ideals? I mean, I didn’t go to grad school and then take a job where I was diving underwater for nine months a year in order to just make sure that students could identify passages from Dickens that we’d already discussed. There’s more to it than that.

Rebecca: One of the things that I really love about Wicked Students is, it’s a value system that we want students to embody. And you’ve talked about one way of getting at that and offering space for these messier problems or the journey. What are some other ways within the structure—thinking about syllabus, learning outcomes, assignment design—that might continue to embody those values? It’s one thing to say, like, “We want to do that.” But what does it look like when we’re actually doing it?

Paul: Right. Well one of the things, and everybody’s doing it these days but why not, here’s my nod to Susan Blum [LAUGHTER] and her book Ungrading. One of the things, if we’re going to have this uncertainty, if we’re going to ask students to take intellectual risks, if we’re going to ask students to not freak out when they don’t arrive at the perfect answer, we need to be sure that in our syllabus, and in the day-to-day work of our course, there are plenty of opportunities where the grade is not of consequence, where they are not hurt by the ability to get it wrong. So I talk about ungraded work that happens in the class, I talk about minimally graded work that might happen outside of the class, proportionally graded work where the first time they do it, it’s only worth 2%, and the last time they do it, it’s worth 20%. But they’ve progressed and they’ve gotten feedback. So that would be another one of the ways, even in things like large lectures, the pause… you’re 20 minutes in, you’ve been delivering content to 270 students, and now you pause. And maybe you use clickers, maybe you use PollEverywhere. Maybe you simply put that question up on the screen and say, “So how might this apply here? What do we think? What are the ways to approach it? This is considered a difficult problem, how would you deal with it?” Maybe you simply say to them, “Pair up with the person next to you or jot some notes, think-pair-share, with the person next to you.” And then you just do a little bit of pointing to the room, “You up there in the red sweater, let me know what you’re hearing. Keep in mind that we’re not looking for the perfect answer, we’re looking for thoughtful answers.” So those would be two ways. There’s a lot of other ways. I mean, I like the idea of co-writing our goals with our students. And letting them know upfront that, with this class, it is not about static information, it’s about construction of knowledge, construction of ideas. The goal here finally is, not just for me to tell you what I know or what other people think, but for all of us to approach this material, whether it’s in literature, or in chemistry, so that we take the thinking just a little bit further, or maybe a lot further.

John: How can we sell this to our students, especially our freshmen students who come in from a curriculum that’s very structured, that’s very well defined by disciplinary boundaries, and so forth, and where all the problems have nice, neat, easy solutions?

Paul: And oftentimes, they’ve gotten into university because they’ve been really good at playing that traditional game, right? And here we are changing the rules. [LAUGHTER] Well, part of it is you need to create spaces. You can’t just give them a big assignment at the end of the semester that’s wicked, or a test question that’s wicked, without creating those spaces for them to experiment, fail, and get used to it. But what I like to, frankly, do is walk into class on the very first day, and sometimes… I like to be very dramatic… and I don’t even give my name. [LAUGHTER] I just put that painting up on the screen, and I ask that question that can’t necessarily be answered perfectly, and then I ask them to write. I was at a workshop a couple years ago at a college up in Maryland, where a meteorology professor talked about on the very first day, he hands his students a meteorology map. He says, “Tell me what’s going to happen next? What’s the weather going to be like?” And he goes, “They get it wrong 97% of the time, but that’s not the point. The point is for them to work with it and play with it. And then as the semester goes along, you can periodically hand that back to them, and they can see themselves making progress.” So I think the first step is to begin. So often we try to just say, “If I just repeat it, then they’ll get it.” It’s hard to use words to combat 18 years of experience and 13 years of educational experience. We need to create alternative experiences where they live, where they see it, where they understand.

Rebecca: I know one of the things that we emphasize in design is process over product, often. We have some students who can make a fantastic product, but they do no process.

Paul: Mm-hmm.

Rebecca: And so it always ends up being a really long conversation about, “Well, what was your journey here? Why are you doing that? How does that meet the creative brief? Or how does that meet the problem? How does that solve the problem?”

Paul: Right.

Rebecca: It just looks good… it doesn’t mean that it does anything. So I think we’re always working on ways to emphasize that process piece more, because it tends to get overlooked. It’s like, here’s the beginning, here’s the end, but that muddy, messy middle part—which is the most important part, especially to develop wicked students—is often overlooked.

Paul: I really do like multiplicity ideation, multiple iterations. And maybe it just happens in the planning stage where you say, “Rather than just going into the task…” because there’s very few fields where you just jump into the task… “come up with plan, give me three different plans, three different ways to approach this, three different ways we might design this, three different ways we might interpret this poem, three different ways that we might design this experiment, three different ways that we might think about how to approach culture in rural Germany, three different ways about how to solve this medical problem.”

Rebecca: So three is really important, Paul? [LAUGHTER]

Paul: Three, yeah. [LAUGHTER] I guess I’m sort of focusing on three. Well, I mean, there is a certain sweetness to three. Five is way too many oftentimes, and two is probably not enough. So yeah, I don’t know why I got on three. [LAUGHTER] But thank you for teasing me about that, I appreciate that actually. [LAUGHTER]

John: And as you move through the course, you also suggest that the degree of complexity should be adjusted from the beginning to the end of the term. Could you talk about some ways in which you might do that in a given class?

Paul: Sure, yeah. So the classic example, and I use it all the time, comes from my colleague, Chris Connors, who’s in geology here at W&L. And he talks about how early in the geology course, they’ll provide a rock sample to students. And they’re asking for, in a simplified form, recommendations about drilling or mining. And in the early data sets, there will be one or two pretty clear paths, and a minimum of noise, being data that is meaningless or simply distracting. And if the student’s been paying attention in those early data sets, they should be able to get it, they should be fine. Later data sets, middle data sets, they’re more complex, more noise, and a multiplicity of reasonable answers. Some of which are going to be more self evident than others, and some of which need to be constructed. But the students, if they’ve been growing, they should be okay. And part of what they need to do is not just say, “This is the recommendation I’m giving,” but, “Here’s the why this recommendation, rather than these recommendations.” Latter data sets are just wide open, lots of noise. And the paths of recommendations, the ideas, the thinking, the conclusions that are being drawn, really have to be constructed almost from scratch by the students individually. And so that’s one example of how progression could occur. Also worth noting that you can increase the degree of difficulty by the level of interaction that you have with students: the amount of coaching that you offer, the number of tips or hints that you provide, how much you’re walking around the room, the degree to which you’re having them collaborate versus doing it individually. What I love about the geoscience example, too, is that in many ways it’s, again, as someone who works in literature, that’s what we do in a literature classroom. When I’m teaching Victorian literature, I begin with a relatively simple Victorian text. And then by the end of it, I’m handing them Eliot, which is millions of words, and a lot of noise. And they’re having to build these maps, construct these meanings. And I’ll just point out that construction of meaning, that production component where the student is taking it just a step further, they’re not handing to me a reading of Middlemarch that they’ve read somewhere else. They are building a path, they are collecting the data, they are doing that work. And finally, that’s what we want. If we’re going to graduate these students into this complex, messy, messed-up world, we had better have people who can build that path thoughtfully and deliberately themselves. Otherwise, we end up with a situation where some large portion of our nation might be being misled by other people, hypothetically.

Rebecca: Definitely hypothetically.

Paul: Yes.

John: One of the barriers though, I think, is that our classes and majors and programs tend to be structured very much within very narrow disciplinary boundaries. Would it help if, perhaps, we had more interdisciplinary courses, and activities, and seminar courses, in most colleges?

Paul: Yes. And I don’t want to take us down a rabbit hole unless you want to go with me [LAUGHTER] of general education, and over the last 40 years, the movement has been away from distributional models that say, “Two of math and science and two of social sciences and two of arts and humanities.” Distributions still exists, but the movement has been towards more integration, more blending, more of a recognition that it’s not about what content we’re delivering, but how that content connects to other content. How, when I teach a class on artistic and literary responses to science and technology, I’ll have a student reading Coleridge’s Eolian Harp and giving me an analysis that ties it to string theory in physics. How do we make connections? How do things blend? So, sometimes, yes, majors need to do what majors need to do, and I want to come back to that in just a moment. But gen ed can do a better job of just replicating what majors are doing. Gen ed, it shouldn’t be just about exposure, it should be about synthesis, it should be about reflection, it should be about building something, making connections. And gen ed is going a step further and really looking at the way high-impact practices, for instance, rather than having models that are driven by distribution, they can be driven by high-impact practices: e-portfolios, which is about synthesis, study abroad, internships, undergraduate research, first-year experiences, senior capstones, all of these things that are synthetic. They’re asking students to make these connections. They also push students towards agency, towards really doing the work themselves. A student abroad, one of the reasons that’s a high-impact practice is because from dawn to dusk, they have to cope. Just leaving their dorm room, leaving their dorm, is a complex act. Getting lunch is a difficult, challenging thing where they have to assume agency and do it. So right now, though, I just also want to step back and talk about majors because I referenced that earlier. Let’s face it, increasingly, majors are becoming interdisciplinary. They’re either doing it by building interdisciplinary majors—biochem, gender studies, environmental studies—but even within majors, they’re recognizing the ways that they need to bring things into play. I mean, psychology has shifted from being a social science to more of a hard science, because it’s bringing neuroscience into play. Again, there’s this idea that the world is a messy place. The academy, we’ve got our little hallways and our little cells and our little silos and our little blocks that we built, those are very convenient for us. They’re not necessarily helpful for the students, there’s some convenience and some value to it for the students. But if that’s all they experience, and if that’s all they see, and if we allow ourselves to fall into that, I don’t know that it helps.

Rebecca: So one of the things that often is a hard sell to students is general education.

Paul: Yeah, [LAUGHTER] I’ve noticed that!

Rebecca: It’s advising time around here, so helping students identify things outside of their major that maybe they might be interested in exploring or might be useful to them in these wicked problems as they leave the institution and go into the profession. What’s some advice that you might have in making that sell towards general education? Is part of the issue the way our general education is structured, so it’s not a good sell? Or is it partly just communication?

Paul: Yeah I’ll deal with the communication question first. It is definitely communication, we spend a lot of time on the “what,” and very, very, very little time on the “why.” I mean, in so many places, if it’s not literally a box check, it’ll have a brief description of the humanities, and then it’ll be on to just a list of courses. And frankly, I would say that if you’ve got a particular learning outcome or goal for general education that you want students to meet, and you’re trying to impress them with how important this idea, this value, this goal is, and then you’ve got 700 courses that can fulfill that requirement. I’m sorry, that’s a mixed message because it basically says, “This goal is incredibly important, so important that anything can fulfill it.” And again, oftentimes what’s happening there is the system is set up to please the department or the anxieties that departments might have or to get butts in the seat. So figuring out how to better communicate that “why,” both on the front end and in advising. We have ready answers for why students should take particular courses in our field. Do we have ready answers for why a poet should take a mathematics course? For why a student in sociology should study dance or yoga? We need to work on that. There is nothing wrong with talking to your colleagues and coming up with some answers so that when students come into play, you can sell it convincingly. The other question about the structure, I definitely think that that’s a factor. If we’re selling them isolated little building blocks, it communicates to them what matters and what doesn’t matter, why it’s happening, why it’s not happening. You know, they get distribution in high school, and call me naive or idealistic, but I think even the most cynical, jaded, least-engaged student in the world comes to college and wants something different. They want it to feel like a challenge, that they’re moving up, that they’re going somewhere else, this is something more. And if we don’t give them that, then they come away with that idea that college is just a transactional thing. That’s a failure.

John: I think this brings us quite naturally to your blog post on open gen ed requirements. Could you tell us a little bit about the concerns you raised about having a completely open or very open gen ed curriculum?

Paul: Sure, full disclosure—and that blog was edited to fit a certain word count, this full disclosure was in the original version—my son goes to an institution with an open curriculum. I actually think… and I’m not going to name the institution, I’m tempted but I’m not going to… I actually think this institution does an extraordinarily good job of advising. So that students, no matter what they’re studying, are pushed and encouraged to go into and take courses in different fields. So it can work really well. A couple of weeks ago, my daughter and I were visiting colleges, she was someplace where the guy ended the tour by saying he was there to study economics, he said, “If you told me a year ago, I’d be taking two classes in classics and really enjoying them, I would have laughed in your face.” That’s a win, that’s a well-done job. So open curriculums can work and I want to be careful not to just dismiss them completely. Part of the issue with them, on the faculty side of things, I think, it’s a missed opportunity because it means that faculty from different disciplines in different departments don’t need to be in conversation with each other. I think that’s a loss. And that’s unfortunate, because the more we talk to each other, the better our teaching all-around is going to be. But really, for me, the major concern has to do with students. Not all students come into college recognizing the value of that liberal arts component, that liberal arts dimension that is so much a part of the American educational system and has been almost from the start. My son grew up in a household where we went to Hong Kong for a year to help them revise their university curriculum from a three-year system with just a major to a four-year system that included liberal arts and general education. That’s been written into his DNA, written into his family history. We talk about stuff like this at the dinner table because I’m such a geek. He has an advantage because he walks in and even if he is really interested in studying physics, he knows that there’s a value to taking German courses and philosophy courses and courses in art history. So for him, in an open curriculum, there are not going to be any signposts. As bad as a really highly-structured curriculum might be, there are a lot of signposts, “This matters, this matters, you need to do this, you need to do this.” In an open curriculum with no signposts, the student who has that level of knowledge, that insider knowledge, they’re going to be okay. The student who walks into that setting, for whatever reason, without that knowledge, maybe it’s because they’re from an immigrant family that isn’t familiar with a tradition of the liberal arts, that phrase doesn’t even necessarily mean anything to them. Maybe they’re from a first-generation family. A young woman that I know actually came from a family where her parents had both gone to college, but they were running a business, and that was really their focus, a nd so this idea of the liberal arts and the value of liberal arts just wasn’t part of the conversation that they had. Perhaps it’s because students come from a historically or structurally marginalized background, I don’t know. And of course, we want to be careful about essentializing about any groups. But the fact of the matter is, if we have an open curriculum, and we don’t have some guardrails in place, we’re not paying attention, it can be inequitable. Certain students are going to benefit powerfully, other students are not going to benefit as much. That’s wrong. We’re educational systems, we’re educational institutions, our goal is to progress everybody forward, not just to allow the privileged to continue to be privileged, and the less-privileged to continue to be less privileged.

John: A lot of first-gen students come in with a very specific career goal and a very narrow definition of what courses they think are going to be relevant for that, not recognizing the wicked nature of some of the problems they’ll be facing or the world in which they’ll be living in a few years. How can we build a system that would correct for that if there is going to be a more open curriculum?

Paul: Yeah, well, I’ve already mentioned advising. So part of it is we need to have, as an institution, and as instructors and colleagues, a language of practice, of shared practice, shared understanding, shared values, that we communicate to the students. And one would hope that that happens not just in the advising session, because that’s really kind of too late, but in everything—from the initial literature the student received, to the alums and the alumni stories that we tell in the alumni magazine or the people that we bring to campus to have conversations with our students, the internships we set up with alums, and all of those kinds of things. So that’s one thing, advising, but also advising writ large. I also think we need to create some spaces where reflection and integration is going on. So maybe it’s a completely open curriculum, but maybe in order to sign up for your classes next semester, or maybe in order to complete your work as a sophomore, or as a junior, or you do your senior thesis, you have to have a reflective component that kind of puts some of the pieces together, that does that sort of integration, that synthesis. Students, they’re so busy, and they’re doing so many things. Sometimes we can reflect, but oftentimes we’re just doing. And so where do we build those moments to make sense? James Zull in The Art of Changing the Brain talks about how there are educational experiences, and they’re really just data until the students integrate it and reflect on it. And at that point, they move from data to knowledge, meaningful knowledge. So where do we create those spaces to make meaning out of the experiences that they’re having? Kate McConnell, AAC&U, says, “Assess and then disaggregate your data.” So those are three things you can do: advising, reflection, assessment and disaggregation. Finally, when I talked about this in the essay, and this is one of my favourite metaphors, my colleague, Rich Grant—who’s a physicist at Roanoke College and he’s actually now the acting Dean of the college—he talks about how students come to college with all these experiences, some personal, some educational, some social, some related to social media, and they’ve got this box of puzzle pieces. And again, on some level, they want meaning, they want purpose, they want to know how all these pieces fit together. If all we’re doing in college is just dumping more puzzle pieces into the box, I don’t think that’s success. Some students will thrive with that, other students will not. If our goal is to make sure that everybody thrives, what can we do that’s deliberate and thoughtful?

Rebecca: I was reflecting a little bit on what you asked, John, and then also the way that you’ve been talking about gen ed, and wicked problems, Paul, and I was thinking: I was a first-gen student who didn’t give a crap about gen ed. I remember. I had a couple things I was interested in, so I curated in a way that followed my interests, but I really didn’t see a lot of value in it until later on, when I had an opportunity, really, to work on a messy problem, to work on a wicked problem. And that was in a class that was cross-listed between geography and architecture…

Paul: There you go.

Rebecca: …and worked with community organizations. And then all of a sudden, I was like, “Oh, I need to learn all of these things.” Like then I had like a to-do list of things I needed to learn.

Paul: Yeah, and so they did a great job there of bringing the messiness of life into the classroom. I’ve got a friend who’s an architect, and a very good one, and every time he moves up the ladder in his organization, the problems get messier and messier and messier. And almost from the second stage, he wasn’t prepared for it. So how do we bring that messiness into the classroom, into the academy, into the curriculum? That’s a great example, I love that.

Rebecca: It’s making me think, like, having a really messy problem really early in a college education would be really important to getting students on board with the idea that you need to learn a lot of things…

Paul: Mm-hmm. [LAUGHTER]

Rebecca: …to have a lot of different kinds of experiences.

Paul: And there are increasingly places that are doing that. I believe Plymouth State is. In the first year, they’ve got a seminar that deals with a messy problem. In the Netherlands, almost all of the university colleges, the small liberal arts colleges and the larger universities, have problem-based and project-based curricula for gen ed, where students are taking these things on from the beginning.

Rebecca: That’s interesting, too, because that means a gen ed class has been designed as a gen ed class from the start, and not necessarily a major class that slots into a category.

Paul: Yeah. In gen ed reform, really, one of the stickiest questions is, “How do we determine what counts as a gen ed class or not?” And in some institutions, if it’s a gen ed class, it has a gen ed prefix and designator, and it is a gen ed class and nothing but. And, on the other extreme is, “Well, anything can count as gen ed.” And then, in between, you’ve got models that have core courses and distribution courses. And the core courses are gen ed specific, regardless… And the line that I constantly give to gen ed committees as they’re doing revisions is, “If you’ve got gen ed goals, you have structure, an apparatus, a process, a protocol in your course approval that honors those goals, unapologetically.” And that’s hard, because you’ve got departments pressuring you. But if we’re going to do it, we have to do it, we can’t just give it lip service and let whatever count.

John: Early in my career at Oswego, I spent one semester serving on our faculty assembly when we were discussing putting together the final stages of a gen ed curriculum. And much of the discussion was basically departments being very worried about losing FTEs, losing lines, and so forth. How can we get past that in these discussions?

Paul: Yeah. One of the things is, in the design process, and in the communication process, about whatever new gen ed models you’re bringing forward for conversation, you have to foreground that there are generally going to be multiple paths into the curriculum. So in the distribution model, well the history department has their own, that’s great because it’s clear, and it’s simple, and there’s assurance, and every department has their own, so everybody’s happy. If you’re going to make an integrative model that’s going to counter that, you need to show the history department that, look, in this model you have your one course. In this model, you can enter it the first year, which is really going to appeal to some people who prefer working with first-year students. You can enter at the capstone level, the senior or junior year, which is really going to appeal to some people who enjoy doing that. Or there’s these interdisciplinary courses over here, which is going to appeal to some faculty who enjoy doing that. Or there’s these hobbyhorse courses over here, that might really be interesting for people who have never been able to teach the bizarre, esoteric topic of their dissertation in a gen ed course, but now you can. And the beauty of that last one, in particular, my colleague, Hedley Freake who is retired from the University of Connecticut, he’s a chemist and he says, “If you give me an option between teaching Chem 101 yet again, or teaching gender and nutrition in developing nations, I’m going to take that latter course all the time, because it’s more engaging to me, it’s more engaging to students. And this is shocking, but the more engaged students are, the happier they are, and the happier I am.” But you have to really be deliberate about foregrounding those multiple entries and the benefits of those multiple paths. Again, it’s the simplicity of the algorithm, and all the experience we have with that safe, reassuring algorithm versus, “Oh, this is new, and it’s different.” And how do we clarify that idea with more than just words? We just had this conversation on my campus yesterday.

Rebecca: Sounds like a wicked problem to me.

Paul: It is, education is a wicked problem! [LAUGHTER] And it’s terrible, right? Because I was thinking, it’s not because education’s the solution. But clearly, we’ve got podcasts, we’ve got conferences, we’ve got discussions. It’s a problem, it’s a challenge, it’s fluid, it’s dynamic. It’s changing constantly, the opportunities, the challenges, it’s all… Gosh, you teach your 9 o’clock class and it goes perfectly, you teach the same thing at 11 o’clock and it’s a disaster. I mean, you know… [LAUGHTER] Why? Because students are wicked. [LAUGHTER] There’s a chemistry there that we can’t control, a dynamic that we have to work with. I love that, and this is it. I mean, wicked problems are engaging. They engage us emotionally, intellectually, socially. They’re exciting, the brain reacts to them, we like puzzles. The part of the brain that wants to solve, that appeals. And I do think that that, again, if we’re thinking about changing the way students think about education. If education is static—you hand me this and I hand it back—that’s disengaging. And a certain number of our students are not going to see the value of that. If, on the other hand, we say, “Here’s a problem, can you solve it?” That’s powerful.

Rebecca: And it’s fun.

Paul: It’s fun.

John: For them and for us.

Paul: Yes.

Rebecca: Dare I say education is fun? [LAUGHTER]

Paul: Wickedly so. [LAUGHTER]

Rebecca: So we always wrap up by asking the big, huge question, “What’s next?”

Paul: What’s next? I think we’re all still recovering. I mean, of course we are. But even those of us who haven’t been ill, the brain, our sort of cognitive power, is still depleted. So naps, naps are next. I’m very excited. I mean, I just got the contract for the revision of the General Education Essentials book last week. And I’m just delighted to be able to take that on because it came out in 2012, I wrote it in 2011, and in those 11 years my thinking has really evolved. And so I’m looking forward to seeing where that goes. I am always with the wicked pedagogies and the wicked problems thinking about, when I do workshops or something, I’m always collecting or trying to collect the examples that people use. So I kind of think, ‘Where can I go with that? Is there another way to approach this? Are there different modes of communicating that can be brought into play?’ I don’t know. So recovery and exploration and maybe trying to find some of the joy that was threatened, if not, depleted.

Rebecca: Yes. More joy, please. [LAUGHTER]

Paul: Yes please. And chocolate.

Rebecca: Yeah. It’s a good side.

Paul: It is, yeah.

Rebecca: Well thanks so much, Paul. We really enjoyed having this conversation and you always bring great ideas to the table.

Paul: Well, thank you. I very much enjoyed chatting with you.

John: We’re really happy that you’ve joined us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

206. U.S. Regulations for Online Classes

To be eligible for U.S. federal financial aid funding, colleges and universities offering distance learning programs must satisfy new federal regulations that went into effect in July 2020 and July 2021.  In this episode, Russell Poulin joins us to discuss how these requirements have changed and what these changes mean for faculty and institutions offering online classes.

Russ is the Executive Director of the WICHE Cooperative for Educational Technologies (WCET), and the Vice President for Technology Enhanced Education at the Western Interstate Commission for Higher Education.

Shownotes

Transcript

John: To be eligible for U.S. federal financial aid funding, colleges and universities offering distance learning programs must satisfy new federal regulations that went into effect in July 2020 and July 2021. In this episode, we examine how these requirements have changed and what these changes mean for faculty and institutions offering online classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners..

[MUSIC]

John: Judie Littlejohn is joining us as a guest host for this episode. Judie is the instructional designer for Genesee Community College, and has been a guest on several of our past episodes.

Judie: And our guest today is Russ Poulin. Russ is the Executive Director of the WICHE Cooperative for Educational Technologies (WCET), and the Vice President for Technology Enhanced Education at the Western Interstate Commission for Higher Education, which, of course is WICHE. Welcome, Russ.

Russell: Oh, it’s so great to be here. Thanks for asking me.

John: Our teas today are:… Russ, are you drinking tea?

Russell: Oh, yes, I’m drinking tea and I live in Longmont, Colorado, which is near Boulder and we have Celestial Seasonings here, and so, I’m drinking Earl Grey that has probiotics in it. That’s a new product for them.

Judie: So you have Earl Grey, but I’ve got Lady Grey black tea from Twinings.

Russell: Oh, very nice!

John: And I have pineapple ginger green tea from the Republic of Tea. We’ve invited you here to discuss the new federal regulations concerning regular and substantive interaction in distance-learning courses. These regulations went into effect in July 2021. We also would like to talk a little bit about the requirements for identity verification that went into effect a year earlier, but before we discuss this, could you tell us a little bit about WCET?

Russell: Oh, I’d love to, and thank you for asking. And so WCET is part of WICHE, the Western Interstate Commission for Higher Education, which is a regional higher education compact focused on the west, but back in 1989, they started WCET to focus on educational technologies, distance ed, online learning – which wasn’t a thing yet then – and, even from the start, we had other states that came in, and now we’ve grown and now we have, members through all 50 states and Canada and even Australia. And our members, are institutions and organizations and corporations interested in the use of distance ed.

Judie: How did WCET become the go-to source for information on regular and substantive interaction?

Russell: Well, we were hearing from members that there was some confusion about exactly what they were supposed to do with this, and that there wasn’t complete clear guidance from the US Department of Education and so Van Davis, who worked at Blackboard at that time, and now works for us here at WCET. And what we did was that we went through all of the guidance, there were some guidance that had been given, and also the findings against several institutions and then tried to put together what is it that they were looking for? What is it that is expected? And then we put together a blog post of our findings, and put that out there, and for like, four or five years, no matter what else we published, that ended up being the top blog post for the year because people were seeing that that was the only place where somebody had compiled this all together and knew what to do. And then in 2019, I was named to the Department of Education’s negotiated rulemaking subcommittee that worked on new rules, and there are people who actually get together and write these rules – you can imagine how exciting that is – and I was part of that and worked on it. And one of the issues that we worked on was this distance education issue and updating this regular and substantive interaction part of it and so you could say I was in the room where it happened! [LAUGHTER]

John: So basically, there was a gap out there that needed to be filled, and there was a lot of concern because of the sanctions that were placed against Western Governors University?

Russell: Yeah, it’d been unclear, and then also yeah, there was these findings against Western Governors University that their courses were all correspondence courses, and that they were expected to pay back – really, when you got to the end of it – it was a billion dollars which they didn’t have in their pockets at the time, and so it really caught the attention of a lot of people and this was a rule that was first written in 1992, had incomplete guidance. It was being administered in different ways and so it was time for a real update and more details about what is it exactly that they’re expecting? And so we worked on that, and we also were able to put in that there’s greater reliance in terms of the relationship between the accrediting agency and the institution, because there were times where it seemed like the auditors were sort of overruling that, and that doesn’t make sense because if you’re meeting what the accreditors want, why is this out of balance?

Judie: So what is needed, then, to differentiate a correspondence course from a credit-bearing online course?

Russell: So, it’s really good to go back and make that distinction and remind people that this is all about federal financial aid and what qualifies for federal financial aid. And they always want to get into, “Well… interaction, the academics, and the pedagogy of it.” Well, okay, put that aside [LAUGHTER] for the moment, we’re talking about, for federal purposes, what qualifies for federal aid and what does not? And the idea was that distance education is something that qualifies for full federal aid, whereas correspondence response education… that you might not get full aid for those courses or if you have enough correspondence students or enough correspondence courses… if over 50% of your students or courses are labeled as correspondence… then your institution is now ineligible for aid, and that’s what happened with Western Governors University. And so, it’s good to be clear, especially as we’re seeing more and more distance ed, hyflex, hybrid sorts of courses about what fits and what does not, and what activities work. We need to know what happens, so that we know that we’re in line, so that we keep our federal financial aid for students.

John: The technology for offering online courses has changed quite a bit since the initial regulations were developed. Under the current revisions, what is meant by interaction in that definition of regular and substantive interaction?

Russell: Yeah, that’s a good point that this rule was written in 1992, prior to online learning [LAUGHTER]. There’s been a lot of changes. But you really do need to parse out the parts of it, and so thanks for asking about the interaction part of it because back in the old definition, prior to July 1, that there’s this really odd thing that, the only thing that counted for interaction was something that was done by the instructor. So, providing a lecture, providing a video, providing an assessment or providing an assignment, any of those sorts of things. Those are the only things that counted, so our interaction here… “Yeah, you said something and I’m the instructor that didn’t count what you said” That’s weird. So, now, there are parts of it (freshly hidden in the regular part of the definition) that if you go down there, that you’ll see that there’s the expectation that you respond to students and student requests. So, at least we’ve made that move that the student being part of the interaction is now a part of what counts, and that’s the great move forward.

Judie: And what is expected for substantive interaction, like how do you define the substantive part?

Russell: So going back again to the old one, that it was all just about content, that if you’re teaching a history course, as one might, that you’re talking about what happened in a particular battle, or what happened in 1792, or those sorts of things, particular things about the course, and interactions about basketball or other extraneous things didn’t count. But under the new rule, we’ve gone to an activity-based notion of it, and I’m going to read these, so stay with me on this, and they’re very short. So one, one of the activities is providing direct instruction. Two, is assessing or providing feedback on coursework. Three is providing information or responding to questions about the content or competencies. Four is facilitating a group discussion. Five is other instructional activities, as approved by the accreditor, that’s a really wide open one. But for these that, if you have any type of online course, I sure hope that you’re assessing what the students are doing, I sure hope that you’re responding to them when they ask questions, and so, you’ve already got two of those right there. The one that’s really in question is the one about direct instruction, and…everyplace else they talk about synchronous or asynchronous, but I know through some things that we learned about the Department of Ed’s thinking of that, and we asked this question directly of them during a webinar about direct instruction. At that time, they said that it was only synchronous education, and sometimes people freak out when I say that, and we are asking for clarification on that in writing, but remember that it’s two of the five, so even if they define it as just synchronous instruction, you’re probably meeting at least three of those five already if you have any type of quality course.

John: One of the issues, I think, that you’re also seeking to resolve is whether synchronous online office hours would count. Has there been any feedback on that yet?

Russell: Yeah, it seemed to us quite clear about the office hours, that that is something that would count and it’s something that did not count in the past. And so, it’s an interesting change for that and one of the reasons that they did that was one of the groups I was representing was competency-based education, and so, we’re going to get into notion of regular here in a moment, you know, how do you define regular for something that, by definition, is irregular in terms of competency based? And it’s based upon student pacing over the faculty pacing, and so, there’s a nod to that in terms of if you have regular office hours and have that posted on the syllabus or somewhere that that would count. And we’ve had some pushback from our financial aid friends, because that’s new to them. They said, “Well, that’s never counted before.” And so again, this is something where we’ve asked the question of the department and hope to get that in writing yet again so that that’s reaffirmed and everybody’s under the same understanding for that one.

Judie: You started to talk about regular, so how do you define “regular” interaction?

Russell: So regular? Yeah, as I was alluding to, that was the hardest one to do, because remember, we’re writing these for, depending on how you count, four to six thousand institutions. Some have short courses, some have competency-based, you think of every variation that you have out there. So saying “meeting once a week,” or doing something once a week just didn’t work because it didn’t fit all those different ones, and once a week would not be enough in a five- or six-week course, that’d be too little. So, we tried a formula and that was a disaster. So anyway, so we have these words, and then there’s a lot more, again, back to the relationship with the accreditor on this and what works. So, these are a little bit vague, but you need to work with your accreditor and how they’re defining these. So there’s really two parts to this. The first is going to be predictable and scheduled, and so this is something where either you have it in your syllabus, and then you have the syllabus at the start and here’s when things are going to happen, and then with that, that you may actually have a two- or three-week break where an instructor is not putting things out or you don’t have interactions, because it makes academic sense that you have the people out doing a paper or group work for two to three weeks and doing that. So that once a week would actually not work there, right? Or the predictable part of it was that maybe it’s not exactly in the syllabus, but what you do is you say, “Every Wednesday, we’re gonna meet or have office hours, or we’ll do something at webinar times.” So something predictable. So, that’s the first part of it. The second part of it was about monitoring the students’ academic engagement. And that was something we really brought in with competency-based education in mind, where what you’re doing is that you are actively following the student and making sure that they’re not out there on their own, and that you’re making sure that the student’s not floating and that you’re seeing that, “Okay, do they need interaction? Do they need some intervention?” Or the second part of that is what I alluded to before, or that the student says, “Okay, I need help with this,” or “I’m ready to move on to the next part of my competencies.” So that’s the other part of it is the bringing that together in terms of monitoring the engagement, so something that’s predictable, scheduled, and then also, outside of that, that you’re actually monitoring and interacting with the student.

John: Are there any requirements concerning the extent to which there should be interactions with individual students as compared to interactions with the class as a whole?

Russell: Well, both of them count. So if you’re giving a lecture, or if you’re doing a group discussion and doing that, that that’s a group thing, and so that counts. And also, remember, the second part of the regular was that you are responding to student requests, and so that’s on an individual basis. And when there is a federal financial aid review, or ”audit” as it’s called, of your institution, what they’ll do is they’ll go and take a sample of classes, and that they’ll look to see what happened in that course? Were there group interactions? Were there individual interactions? That they’ll look to see what happened and then they’ll look to see, does it meet the regulations? And have you developed faculty? Have you let them know that these are the expectations of them? That they’re looking for those sorts of things, and did it actually have an effect in the courses?

Judie: So what would be some examples of regular and substantive interaction that we could build into a course?

Russell: Yeah, that’s a really good question. You’re an instructional designer, you plan the whole course out, right? And you’re going to get more points on the regular side for the predictable and scheduled if you have a syllabus, that… it doesn’t have to be detailed down to every last thing that you’re going to do… but at least you’re showing that what are the expectations along the way? That, when are you going to have assessments? That, if you’re – let’s say that for some courses – that you’ll send out a video with a different lesson every Monday or every Friday, what you’re doing is that you’re hitting the marks on the regular, another is that you are showing that you have some expectations, in terms of the feedback and the feedback loop. And sometimes institutions have this as a policy, sometimes they leave it to the faculty member. But the more classes that you have where you show that the faculty person… you don’t have to respond immediately… but they’re showing that they will respond in 72 hours, not counting a weekend… that they will respond to the students. And there’s that expectation that they will do that because I had a question from one, that they had a faculty member that they put the discussion out there at the start of semester and then graded it at the end of the semester [LAUGHTER]. And first of all, that’s terrible teaching practice, let’s start with that, nevermind the regulations, but that faculty person thought it was good. And the other is, that there’s this expectation that, as you go along that you’re working with the students. So those are some examples of things that you’d want to do there.

Judie: So just a couple of different things that I like to try here is, I really like to try to encourage faculty to give feedback prior to the next due date. That just makes sense, so that if a student is making an error, they’re not going to repeat the same error because they haven’t gotten their feedback yet. So I just think, pedagogically, it makes sense to give the feedback as soon as possible. But I also like to just have faculty create a communication plan when they’re developing their course. I think faculty plan in their heads, you know, “I’m going to send this announcement, I’ll do this feedback, I’ll do X, Y, and Z.” But when they sit down and really map it out in a communication plan for their course, I think that really helps get into that regular schedule. And whether it’s date driven, or day-of-the-week driven or at specific points throughout the course when students reach different milestones, I think that really helps them. My understanding is that those types of things would help people meet the regular and substantive interaction definition. What do you think of that?

Russell: I love both of those ideas, and actually that, really, if you work with the instructional designers, look at what works in terms of good pedagogy, that these are the things that you’re talking about is that having a plan ahead of time and being open with the students so that they know, and then getting back to the students in between assessments so they have the feedback in terms of, they know where they might be falling behind a little bit in some areas and so that they know, “Oh, I don’t quite get that concept. And then, I have a math background and that was so key in mathematics that, if you don’t get this one, you’re not going to do any better on the next test right? And that’s probably true in so many different fields as well. And so, I love both of those ideas in terms of doing things and where you’re informing the student, and then keeping them engaged and then constantly moving them forward.

Judie: It’s great to hear that kind of feedback from you, thanks! [LAUGHTER] I hear a lot from faculty now, especially during COVID, when many are teaching in Zoom. And so they’ll record a live lecture with their students and, assuming that FERPA rules are followed, and there’s no students caught in the video or audio, they want to just show that recording again in the next semester, and want to know if they’re meeting regular and substantive interaction that way. And I tell them that, when they’re giving their lecture, that is regular and substantive interaction when they’re engaged with their live students. But I say that once you make a recording, and put it in the course, it becomes course content, because it’s no longer a unique experience with those individuals talking about their understanding of the course content. And I see like a real fine line there and I wonder what you think of that? Or how that might be interpreted? What do you think of using old recordings versus always expecting some sort of fresh and unique interaction with the students?

Russell: Yeah, I think if that’s all that you did, I think that you’d have a hard time in terms of meeting the regular and substantive interaction, and this is the one where we get back to the direct instruction question on that one, and we did gather that question plus several others and pose them to the Department of Education, because we felt that even since they released the rule that we were hearing different things from them than from the accreditors. And so I’m a little hesitant to give you a yes, that works or not, under the new rules or not until we get a better answer from the Department. My feeling was that, under the old rules that I felt a little bit better about that that probably was problematic. It might still be, but I’m kind of curious to see what they say about the synchronous versus asynchronous going forward. I think that, if that’s all that you relied on, I think that that’d be problematic that you would need to have other sorts of interactions that might make that work.

Judie: Thanks.

John: But videos that were custom created for that week’s activities, or that provide feedback for the class would count, right?

Russell: Yeah, the ones that are custom created.

Judie: I encourage them to make like small targeted videos for clarification. Like, to address a specific topic that they know that students struggle with, as opposed to just making an hour video of you standing at the front of the classroom, talking to people that future students don’t even know.

Russell: Yeah, and I think if you just use the video over and over again, and I certainly saw this in some engineering courses where they’re using the same ones that, what happens is that you have to update your materials every once in a while, too [LAUGHTER].

Judie: Oh, sure

Russell: You need to be doing that. And so I remember witnessing a course where they were falling behind on some facts or raised a lot of questions about advances that had happened after the video had happened, or were quite clearly dated, because they were talking about things that were going on in space as a future thing, instead of a past thing. And so I think, if nothing else, that you’re going beyond whether you meet these rules or not, that you’re diminishing confidence of students in terms of the value of what they’re receiving.

Judie: Sure, that’s a good point.

Russell: One other issue that we didn’t touch on so far had to do with the definition of an instructor, and that was a difficult one for us. And that was another one that Western Governors University got hit on this one. It was a finding against them. And that it seemed like some of the definitions meant like an instructor, that there was one person that was in front, and that a lot of institutions have gone to team teaching or bundled instruction, or using GAs or TAs or there’s several people in the course and with the unbundled instruction that WGU did that they had one content expert providing the content of the course and another one doing the assessment and maybe somebody else doing some of the advising. So you broke it up and that they weren’t counting that even though it was approved by their accrediting agency. So that is one where we have worked on “instructor” and we’re very clear in this, that it is what is approved by the accrediting agency that that is what counts. T hat was sort of alluded to before, now it’s very clear. And so if you have a non-traditional sort of model for your instructors or faculties, you may want to talk to your accrediting agency about how they view that and get something in writing about that.

John: I think that is a pretty common issue where there’s often a master course developed by the content expert and then again, there are instructional teams that work on the whole course, but the division there can vary quite a bit. And I know there’s a lot of interest in institutions in trying to scale online education to make it more efficient, and this is an area that certainly needs to be addressed with the accrediting agencies.

Russell: Definitely, definitely, yeah.

Judie: So these rules that we’ve been talking about also addressed student identity verification. So is the student identity verification related to the regular and substantive interaction? Or is this another area that requires a more precise definition?

Russell: This is actually a whole other area that was in a different part of the regulations, and this is one that actually went into effect in July of 2020, and it’s part of what the accrediting rules are, and there’s a whole list of things that the accrediting agencies are supposed to be looking for when they’re doing your accrediting reviews. One of the things that they’re supposed to be doing is making sure that the institution has, really, policies and processes to make sure that the student who enrolls in the course is the same one who’s completing and submitting the assignments in there. So it really is about academic cheating, and that this is only in the distance-ed world that they have to do this. I have to tell you, in a subcommittee, we tried to expand it and got beat back. So, sorry we lost that fight for you. But it’s still in distance ed where the accreditors are expected to check for that to make sure. And then the big change that happened in that is that, previously, there were some, what were considered ‘“examples” in there. And one was that you had some sort of ID, some sort of login ID for that or that you did proctoring, and that those were meant as examples. And those were taken out, because all too often, what would happen is that an institution would say, “Well, we have an ID!” and they would do nothing else. And so that was clearly insufficient in terms of doing it, and so, they’re raising the bar. The intent is that you have a plan, and that you’re executing the plan, and that you’ve worked that out with your accreditor, and that when that financial aid review happens, that you will be able to demonstrate what you’re doing and if it’s effective. Another part of it, there’s a second section to it, that also talks about that if you have additional costs, and so, let’s say that you’re using a proctoring software, and that that costs so much per student, that you have to notify the student at time of registration, that there’s an extra cost for that. And this is something that a lot of institutions have fallen short on, because what they’ll do is that they’ll notify the student in the syllabus, and so the first day that the student starts they see the syllabus, and all of a sudden they have to pay more money. And the idea is that the student should be able to have a choice at the time they’re picking between which course that they might take or know that they’re going to have an additional cost for participating in that course. And that rule is out there, and it’s a good one, because you’re being clearer to the consumer about what’s going on.

John: So authentication with a password to a course management system is not sufficient, and some type of proctoring software is, but there’s a lot of concerns raised with proctoring. Are there any other ways to authenticate students that meet the requirements without moving to software proctoring solutions?

Russell: Yeah, and I think that over the last year that we’ve seen, the concerns about proctoring software have risen to the fore, and there’s some good ones and that they do some good things, and so you should not throw them all out. But yet, you should pay attention to the concerns about that. But there are ways that we have worked on this in terms of different ways that faculty can work in terms of their assessments. One of the things we talked about is face-to-face proctoring, As distance ed grows, though, that gets to be harder and harder to find enough proctoring sites and the ability to do that, but that is an option. Some of the other things that have been proposed have to do with how you do assessments, and that having more frequent assessments and doing things where it’s easier to take the course than it is to cheat. That if you do like one or two big assessments per term, that it’s a lot easier to get someone to do those for you or the big papers and all that, so that’s one. There’s others where getting people involved in terms of group coursework, or other sorts of authentic assessment type of things, where you get involved in different sorts of things, where you have to stay engaged more and more often through the course, and it’s harder to get somebody else to do that for you. There are people who will take the whole course for you… that’s a problem. But the more barriers that you can put up, and we really do love our instructional designers, but the more that we can do to help faculty with thinking about assessment strategies and effective assessment strategies, the better off that will be with all of this. Aso there are other areas like… oh shoot, I’m blanking on… ICAI, they have a lot of strategies as well out there, and for some reason, it’s early in the morning here and I’m blanking on their name, but there’s an institute for academic integrity that has a lot of good resources on this issue.

Judie: Yeah, I think it’s great to encourage all faculty to work with their instructional designers on authentic assessment.

Russell: Yes, yes, yes! That success will be more sure if you work with your instructional designer.

Judie: So, do you think that this authentication concern is only for assessments? Or is it for day-to-day coursework and interactions, too?

Russell: Yeah, it’s really about any type of quiz or paper or anything that you’re going to be evaluating the student on. That’s really what it’s looking at, because there’s the opportunity for cheating or something bad to happen there. So there are other things that you do in terms of papers, you know, with Turnitin or other sorts of activities that people do or trying to create papers that are more authentic or real: “Write something about your hometown or work on a project in your hometown where there’s not a lot of papers.” [LAUGHTER] So, you can do those sorts of things where it’s harder to plagiarize.

John: It’s really nice to hear that open pedagogy projects, videos that students create, where they’re actively engaged in it, group projects, and all those things can serve the same role without moving to the extreme of proctoring.

Russell: Yes.

John: It is good to note that any courses that require proctoring must list that up front so that students are aware of the cost. And the other issue with that is, as colleges enroll more first-gen students and more students from the lower income quintile, many students won’t have computers or networks that will necessarily support proctoring software. If students are working on their course through their smartphones, most proctoring solutions don’t work with smartphones. But it’s pretty easy for the student to take a video of themselves talking about something, so allowing faculty to have more options for authentication is something that allows for a more inclusive learning environment.

Russell: That was a huge lesson from the move to remote learning due to the pandemic, that you had so many students who did not plan to be in a remote course that uses online tools… that they were using, as you said, cell phones or different types of tablets that were not compatible with some of these proctoring software solutions, or that they didn’t have the adequate bandwidth for taking the test in that way with a full video… that that was a real problem, privacy issues with it. So there were all sorts of things that were problematic with that. And so, being upfront with the students is very good and we do often cite that there wasn’t a federal finding, but it was a student in Nevada, who was in a course and they were not notified that they were going to be using proctoring software. The student was pre-law, decided to flex his pre-law muscles, and got a whole bunch of students behind him and took it to the institution and ultimately went to the Board of Regents there, and all of those students had all of their fees repaid. It wasn’t a federal finding, but what was shown was that they were out of compliance with the federal rule, and the Board of Regents decided to remedy that.

Judie: So looking at regulations like this, we can see that online classes are held to a higher standard than face-to-face classes. Do you think similar requirements should also be implemented for face-to-face classes?

Russell: Well, I think what’s happening is that the vast majority of courses are now digital courses. And whether it is fully online, whether it’s hybrid, hyflex, blended, or just the old term that the OLC used to use of “web enhanced,” that you use the web a lot, even though that you meet face to face Monday-Wednesday-Friday, that we’re seeing all these digital tools going throughout. And what happened with the pandemic? It really took off, right? And so there’s even more of that is gonna happen. And some surveys that one of our organizations, Every Learner Everywhere did, that there was more interest and uptake from faculty in terms of, “Well, I’ve done this now I should do it again.” And so I think what’s happened is that we’re gonna see more and more use of digital technologies through, if not every course, the vast majority of courses. Well, the thing that happens is that these same sorts of problems are in all of these. And you and I know, keep this as our secret between us [LAUGHTER] but, all this stuff was happening even without technologies, right?

Judie: Right.

Russell: A lot of the cheating scandals in some of the service academies in the last few years had very little to do with technologies other than sharing some information. So we know that’s going on. So it’s gonna be interesting. This is one of the things that we’re looking at that I bet we’re going to see a lot of new guidance coming out of this Department of Education that recognizes that and may expand this out quite a bit more, because there are people in the department who have huge concerns about consumer protection issues and the use of online or digital learning regardless of where it’s used, and that they’re seeing that some of these rules need to be applied more broadly. So I wouldn’t be surprised to see that we get guidance that says exactly that coming out and that we have James Kvaal was just finally approved as the Undersecretary there. I think that that staff has been working on these sorts of things, and have been waiting for him to be approved. I would not be surprised that in the next three to six months that we don’t start seeing some new guidance coming out or answers to our questions. I don’t think they wanted to answer our questions until James Kvaal was in. And so I think that we’ll see clarifications and guidance about some of these things where we’ve had questions before, and how do they apply in a hyflex setting? How do they apply in a blended setting?

Judie: That’s good news.

John: And whether the rules are expanded or not, they’re just good practice… that regular and substantive interaction is good pedagogy.

RUSSEL: Yes! [LAUGHTER] Yeah, you really nailed it with that. And that was one of the things that surely the people that run the subcommittee and then the main committee were trying to look at: “What do we do that makes sense in terms of best serving the students?” And we have to remember at the end that these are consumer-protection practices that have to do that, there are also federal-financial-aid protections that aid is going to worthy activities, and so we need to remember that in all this.

John: We always end with the question: “What’s next?”

Russell: Well, I kind of previewed thata bit, that I really do believe that we’re likely to see several more clarifications coming out, or maybe some surprises coming out in the next few months. And so I think it would be good for people to pay attention to what’s going on, and we certainly write about whatever comes out in our WCET Frontiers blogs. So, be watching for that. And on something completely different, that we’re getting together some folks to work on the issue about veterans and their housing allowance. And just quickly on that, that veterans who take all their courses online, get about half or a little bit more of the housing allowance of veterans who take just one course face to face… they could have it all online, but just one course face to face. And it really is antiquated thinking, and it’s something that we need to get fixed. Because, I could be the same student in one term, take all my courses online, and the next term take just one course face to face. I have the same housing cost, I have the same family [LAUGHTER] I still need to eat, but somehow my aid is less. And so we’re working on that one.

John: And you shared many resources with us that we’ll include in the show notes, so those will be available on the website. Well, thank you for joining us, this was really helpful, and I think it’s going to benefit a lot of institutions and a lot of faculty and instructional designers as they plan for future semesters.

Russell: Well, it was a great pleasure being here with you today, and having a little bit of tea in the morning is always good. And so thank you, Judie, thank you, John for inviting me and for having me here.

Judie: Yes, thank you. This was fun. Take care.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

186. Super Courses

Students often see our classes as boxes that they need to check in order to graduate. By reframing our courses around fascinating big questions that students can connect with, we can help our students recognize the value of these learning experiences. In this episode, Ken Bain joins us to explore examples of courses that do this well.

Ken is an award winning teacher, the founder of the teaching centers at Northwestern, New York, and Vanderbilt Universities. He is the author of two very influential prior books, What the Best College Teachers Do and What the Best College Students Do. His newest, Super Courses, was released in March 2021

Shownotes

  • Bain, K. (2004). What the Best College Teachers Do. Harvard University Press.
  • Bain, K. (2012). What the Best College Students Do. Harvard University Press.
  • Bain, K. (2021). Super Courses: The Future of Teaching and Learning. Princeton University Press.
  • Andrew David Kaugman, Books Behind Bars
  • Deci, E. L. (1972). Intrinsic motivation, extrinsic reinforcement, and inequity. Journal of personality and social psychology, 22(1), 113.
  • Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of educational research, 71(1), 1-27.
  • Perusall
  • Hypothes.is

Transcript

John: Students often see our classes as boxes that they need to check in order to graduate. By reframing our courses around fascinating big questions that students can connect with, we can help our students recognize the value of these learning experiences. In this episode, we explore examples of courses that do this well.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Ken Bain. Ken is an award winning teacher, the founder of the teaching centers at Northwestern, New York, and Vanderbilt Universities. He is the author of two very influential prior books, What the Best College Teachers Do and What the Best College Students Do. His newest, Super Courses, was released in March 2021. Welcome, Ken.

Ken: Thank you very much. Glad to be here.

John: We’re really glad to talk to you. You visited Oswego a few years back and people are still talking about your visit.

Ken: Oh, wonderful. I had a wonderful visit.

John: Our teas today are… Are you drinking tea?

Ken: No, my doctor won’t let me do that, and I haven’t had a good cup of tea in… oh my goodness… many, many years.

Rebecca: Oh, that would make me so sad.

Ken: Yes, indeed, it does. Me too. I can’t drink tea… anything that has caffeine in it.

Rebecca: Ah, total bummer.

Ken: Yes, it is. [LAUGHTER]

Rebecca: I have about the strongest caffeinated Irish breakfast tea you can have. [LAUGHTER]

Ken: Oh, my goodness. Well, the last cup of coffee that I had was in 2002. I remember the date. That’s because it….

Rebecca: Oh, no…

Ken: …a traumatic experience, to go cold turkey.

John: Actually, that’s how I started drinking more tea. I had to cut out caffeine, so I started drinking herbal tea.

Ken: Well, I do drink herbal tea from time to time. I just don’t happen to have a cup right now.

John: I am drinking Tea Forte black currant tea today.

Ken: Oh, wonderful.

John: It’s really good.

Rebecca: So we’ve invited you here today to discuss Super Courses. Could you tell us a little bit about the origin of this book project?

Ken: Well, my wife and longtime collaborator, Marsha Marshall Bain, suggested that we do a course around the Invitational syllabus, as we’ve come to call it, what we used to call the promising syllabus. And we began collecting those syllabi from around the world and began looking at them. And in the midst of that endeavor. Peter Dougherty, who is the longtime director of the Princeton University Press, contacted us one day and said, “Would you come down to Princeton, and I’ll buy you lunch?” So that sounded like a great invitation. And we went down. And he asked us what we were working on. And I told him and he said, “Oh, you’re looking for super courses.” And that triggered a whole avalanche Of reconsiderations of what we were doing, and we shifted the Invitational project over to the super course project, and began looking for courses that offered what we had been calling a natural critical learning environment. And we began that project back in, I guess, late 2007.

John: Maybe before we talk about your new book, you can talk just a little bit about the concept of an Invitational syllabus, since that was the origin of this project.

Ken: Oh, sure. It’s the idea of inviting your students into the class, rather than requiring them to come. And rather than focusing upon topics, it focuses upon big and enticing questions, so that the Invitational syllabus begins with an intriguing question, an important question, a beautiful question that students find so enticing, that they say, “I want to be a part of that.” And it becomes a self-motivating experience. So that’s part of what we meant by a natural critical learning environment, is the creation of that self-motivating experience where students would pursue things, not because someone was threatening them with a bad grade, or because they were just looking for credit, but because they became deeply interested in the question.

Rebecca: Can you expand a little bit upon the idea of the natural critical learning environment beyond just the Invitational syllabus?

Ken: Sure, we now have identified, oh, I guess, about 20 some odd elements of what we call a natural critical learning environment. And the first and most foundational of those elements is that it’s organized around those intriguing questions. And its intention is to foster what the literature calls deep learning, that is learning in which students think about implications and applications of what they’re learning and the possibilities of what they’re learning. It’s learning where students look behind the words on a page and think about all of those implications and applications and possibilities and how things are connected to each other. So that’s the foundational element and the chief goal of the natural critical learning environment is to create an atmosphere where students can, and will likely, pursue that deep approach to learning and they develop what we call deep intentions to learn. But, how do they do that? How do we get them to that point? So, what is the natural critical learning environment? Well, it’s an environment where they can try, they can come up short, and get feedback, and try again, without penalty, without any kind of situation where they are punished for coming up short. In other words, if you think about it, it’s the kind of learning environment that we expect, as scientists and as scholars. We try out things, and if they don’t work, if the data doesn’t confirm our hypothesis, we modify it and try again. And we’d be terribly insulted if our first effort out of the box was… and people would say, “That’s nonsense. That’s crazy. Go away.” We try things, get feedback, and try again, and so that’s what the natural critical learning environment does. It’s also an environment where students can work with each other. People learn in community arrangements, where they work with each other to grapple with the problems. And they learn… and this is another key element of the natural critical learning environment… they learn by doing. Sometimes that means learn by teaching. And by teaching, we don’t mean necessarily that they stand in front of a mirror and deliver lectures. In fact, the teaching that they develop often doesn’t even include lectures, it includes a way of fostering very deep learning on the part of other people by creating dialogues, creating exchanges around big questions that move students toward a deeper understanding and a deeper application.

John: Going back to that question of the big questions, because that’s an important part of the approach. I think you talk about that both at the level of the course as a whole in the Invitational syllabus, but also when you’re devising individual components of your course. Could you elaborate a little bit on what faculty should think about when trying to select those questions?

Ken: Yeah. And it’s more than selecting them. It’s framing them, and framing them in a way that will intrigue students. Now, some of the best super courses we came across were questions that sometimes began with questions that were much larger than the course and much larger than the discipline. But in the course of students pursuing those big questions, they discover that “Well, I need to learn chemistry to answer this question,” or “I need to learn history,” or maybe “I need to learn both,” because many of the super courses were multi-disciplinary, built around a big and complex and interesting fascinating question. And then the students would devise ways of trying to answer that question. And the professor would build an environment where they could progressively tackle those questions. They can run from the very simple to the very complex, one of my favorites, and one that I’ve talked about so much, and actually written about, going all the way back 10, 15 years ago, is one that we do mention, briefly, in this new book, but it’s joined by other really exciting examples. First, that old example, it comes from 2006. And there was a professor at Princeton at the time, he was a political historian and political scientist, and who wanted the students to examine the impact of that period we call reconstruction, in period from roughly 1865 to 1877, and to ask themselves, what kind of impact did that period have on subsequent political and social developments and political institutions? Now, as a historian, that’s a very intriguing question to me. [LAUGHTER] But I don’t think you will find many undergraduates who are just dying to pursue that question. So she didn’t ask that question initially. Instead, she built a course around a question that she knew was already on the minds of her students. Now you think about what was on the minds of students in the fall of 20006? …A big question. When I ask American teachers these days, they often can’t remember. [LAUGHTER] I recently did a workshop in China, and in far southwestern China, and they got it immediately. They remembered. But, the question was, basically, “What in the world happened with that disaster we call Katrina?” Now, there’s a lot of evidence that that question became the dominant question in American politics in 2006, 2008, and helped determine the outcome of the election in 2008. You look at Mr. Bush’s numbers of approval, they fell off the cliff after Katrina. So what caused that disaster? So she organized a course, she called “Disaster: Katrina and American Politics.” And students signed up immediately. It became an extraordinarily popular course. Well, how do you get from there to an examination of political history? Well, it happened on the first day. She went into the class, and the first question she asked her students was, “When did the disaster begin? Did it begin in August of 2005, when the storm surge hit New Orleans? Or did it begin in 1866 with the beginning of reconstruction in the Crescent City?” And with that question, she transformed their interest into her interest, and it became the driving push of the whole course. But let me give you another broader example of the book, A guy by the name of Andrew David Kaufman, who teaches at the University of Virginia, about a dozen years ago, organized a program he calls Books Behind Bars. His field is Russian literature, late 19th and early 20th century Russian literature. And that literature is quite famous for asking big questions, questions about: “What’s my purpose in life? What’s my destiny in life?” So what he does in this course, is the help students go into a maximum security correctional facility for young people, people the same age as the UVA students in the course. When they go into that prison, and they help those other young people confront those questions, by reading Tolstoy, by reading Turgenev, raising the questions, and then struggling with them in a class that they do for them once a week, ensure they learn Russian literature, by teaching Russian literature, and not by lecture, but by creating an environment, a natural critical learning environment, where their students, the residents in the correctional facility, will learn just as deeply as they will. And it’s a transformative experience, for both sides, and it changes lives, and it’s self motivating. That makes sense?

Rebecca: Yeah, these are really powerful examples. And I love that both of them have really strong ledes with the course title.

Ken: Yeah. And if you’ll notice also that both have appeal to a sense of altruism. And we discovered that many of these super courses do just that, even in fields like physics and engineering. They do things to help other people. One of our favorites is a course that some high school girls in a high school in northwest Los Angeles, developed for themselves. And the only help they had was they were invited into this program and invited to come up with a project. And they live in a relatively poor neighborhood. They said, “Well, the biggest problem in our neighborhood is homelessness. And we see the homeless out on the street and in the park and under interstate 5 that runs near the high school. And what we want to do is we want to create a portable tent that is solar powered, so they will have heat and the cooking facilities and light and so on and so forth in their tent.” Now to do that, they had to learn engineering. So they organized their own courses, they organized their own sequence of topics that they would pursue. Now they have some guidance. The teachers over there kind of giving them hints or answering questions: “Should we pursue this next?” But they learned everything from electrical engineering to programming, and lots of things in between. But they also learned just the basics of being an engineer. That’s transformative. They created it, and therefore they took ownership of it. Now, the super courses, and the super institutions that we studied, immerse a lot of what they do in the research on human motivation, research pioneered by Edward Deci and Richard Ryan. And they argue, in their well documented research papers, that human beings have three basic needs and that if you meet those psychological needs three basic psychological needs… we have physical needs that go beyond these… the three basic psychological needs. And if you meet those psychological needs, people are just naturally motivated to try to learn. You don’t have to stimulate it, it just occurs naturally. And the problem is often that the way we set up schooling for people doesn’t meet those needs, it actually counters those needs. And so we get classes full of uninterested students, students who are signing off and not really becoming involved. And to address that situation, many of these courses deliberately use Deci and Ryan’s work to build an environment, where, what shall I say, where people are just naturally driven to do what they need to do. Those three needs, by the way, are: a need for autonomy, that is, we like to be in charge of our own lives. We don’t like teachers being in control of our learning. We want teachers to help us with it, that’s different, but not to control that. And beyond autonomy, there’s also a sense of competence. So people, if they feel like if they don’t know something, they can learn it. And they feel that what Carol Dweck calls a growth mindset, that mindset that says, “I may not know this, yet, but I can learn it. I may not know calculus, yet, but I can learn it.” While the person with the fixed mindset says, “I’m not a writing person,” “I’m not a computer person,” “I’m not a whatever person,” and they give up. And they don’t try to learn and to push the envelope. And then finally, it’s the sense of relatedness, people like to be part of a broader effort, and an effort in the super courses that’s often larger than the classes itself, larger than the discipline… that they take on these large projects, because they believe that it can make a difference for themselves, and for other people whose lives they will affect.

John: One of the things that’s challenging, though, for a lot of faculty, is that we do have to assign grades for all of this. So we know ultimately, students are going to get these grades. And that tends to lead to more of a reliance on extrinsic motivation. What can faculty do to provide that sort of encouragement and to help create a growth mindset when students are going to struggle with some of the material at first?

Ken: Yeah, it’s to give them lots of opportunities to try, fail, receive feedback, and try again. Now, that seems really daunting to many faculty members. They say, “Now look, I have to two hundred students, I can’t do that, for all of the students.” But there are ways of doing that. And that’s one of the things that we explore in the book. It is difficult to describe, so I won’t attempt to do so in the conversations here. But, the courses develop ways for students to give feedback to each other. Sometimes they have students make an argument about their own learning, and then have each other to assess that argument and make an argument. And that second part that I mentioned, is really an important part of the natural critical learning environment, that it’s an environment where people deliberately learn to give themselves and each other feedback. So they set up the whole system of marks around that idea that students were going to give each other feedback on how well they’re doing. And they’re going to give themselves feedback. And that they learn to assess their own efforts and work through those. Often, in the course of the term, credit is often given for participating. That is, if you do the work, you get credit for. And only at the end, do you approach anything like a summative judgment that we usually call a final grade. One of the things that we do in the book is to explore the history of grading. And we do that to help people see that grading is, for one, a fairly recent invention in education. The idea of putting a number or a letter on someone else’s thinking, that didn’t emerge until fairly late and really didn’t become entrenched until the late 19th to the early 20th century, and that changed everything. So I want people to see this in that kind of context, that there’s nothing natural or automatic about having the traditional approach to grading. And so what people have done in the super courses, is find ways of saying “Okay, now you’ve joined a community, you’re going to be helping each other to learn and you have responsibilities toward that community and to help each other to assess each other, to give each other feedback… substantive feedback, not scores, substantive feedback to one another. And we’ll try to give you feedback as well, maybe as a group, maybe individually in smaller classes, but to give you that opportunity of trying, coming up short, and being able to try again, without that affecting your overall final grade. And then the final grade is based upon an accumulation of lots of things, and perhaps a final project, a final paper, a final presentation, or something of that sort, rather than just simply accumulating, you get 10% on this and 15% on this and 40% on this aspect of the grading.

John: How can we help students embrace the concept though, of productive failure, that process of trying something, making mistakes, and then learning from that experience. Because that’s something that many of our students don’t naturally come to, because many of them haven’t seen it before up to the point when we have them in class.

Ken: Yeah, exactly. And I think one of the keys is to provide them with very dramatic and enchanting learning environments, at the very beginning of their experience, so that the students say to themselves, “This is going to be different.” Let me give you an example of what I’m talking about… my favorite example of this category. It came from a program that was offered, again, in a secondary school. It is called city term. And it brought students from around the world together right outside of New York City, and they use New York City as their classroom. And the first Saturday in the program, they’re invited to go on a scavenger hunt. And they’re given a list of items that they might look for in New York City. One of them, for example, that they often use was: find the first wooden escalator. So students go off in groups, and there’s a teacher that goes with them, but the teacher doesn’t interfere, and just keep them safe. And beyond that the students go wherever they want to. It’s a wild and exciting adventure. And then at the end of the day, they end up in Central Park on a picnic, and they discuss with each other. “How did you find that escalator at Macy’s? What questions did you begin to ask yourself? Who did you talk to? How did you reason through the process?” And by sharing ideas with each other, what they’re actually doing is learning good research techniques. That’s a wild way of learning good research techniques, to say the least. But it’s something that the students will always remember. And they will latch on to that. And they will latch on to the course now, because that first experience was something that was quite dramatic to them. Now, we don’t all have the opportunity to use New York City as our laboratory or our classroom, and to take students on a scavenger hunt. But we can imagine creating a first assignment, and I’m reluctant to use that word “assignment,” because we found often that these courses don’t talk about assignments, they talk about opportunities, and invitations to students. It’s so exciting that it begins to break down all of their sort of stereotypes in their mind about what’s going to happen in a class. So in Andrew Kaufman’s Books Behind Bars course, they’re first asked to apply for the course. So they have to explain why they want to be in the course, and then helps to begin to break down barriers. And then the first day in the class, he begins to break down the barriers by first telling them about a three minute story about a young man who read a little short story by a guy by the name of Tolstoy, Leo Tolstoy, and as we called him in the West, and about halfway through the story… in other words, about a minute and a half end of the story, the students began to realize that it’s a story about Gandhi. And it’s a story about an important transformation in Gandhi’s life as a result of reading a piece of literature. And so Professor Kaufman says to his student, “We want you to think about a point in your life, when a piece of art, maybe a piece of literature, maybe a painting, maybe a song, but some piece of art had a deep impact upon you and your thinking.” And the students began to discuss with each other. And they began to realize first, that this isn’t going to be a course where the teacher just talks to them and they take notes and then later take a test on whether or not they can recall the notes that they took. But it’s going to be a class that they will dominate, that they will do most of the talking and most of the thinking, and by creating a different kind of environment, you then can move to ultimately getting them to think about such questions as how are you going to assess yourself? How do you know whether or not you’re making progress, and whether or not you’re learning and you’re learning deeply. And the key point here is helping students to learn what it means to learn deeply, that learning deeply is not the traditional strategic learning… “oh, I learn this for the test. I’ll make an A on the test and I’ll make an A in the class.” No. it’s self-driven learning, where you begin to look behind the scenes, where you intend to look for ways in which this course can transform you and transform your thinking.

John: So essentially, I think what you’re saying is we need to help encourage students to develop more reflection on their work and on their learning process.

Ken: Yeah, I think so. I think that’s the key is getting students to think about their own learning, and help them with categories that will enable them to think deeply about their own learning… categories like deep learning, versus strategic learning, or surface learning. The strategic learner just wants to make straight As [LAUGHTER] and they’ll do whatever is necessary to make those straight As. The surface learner just wants to learn enough to pass the course, to be able to perform on an exam or write a paper or whatever it is that’s required of them. But neither one of those two leads to deep intentions, that I deeply want to understand how calculus works, and how it can help me in understanding the world in which I live, and how this applies to me, and my field and how it applies to my major, even though I’m not a mathematician, and how I can change the way in which I think. So developing those deep intentions. and fostering that deep development of intentions, becomes extremely important.

Rebecca: One of the things that you were just mentioning in terms of the strategic learners and the surface learners is how much many of our courses are probably structured with them in mind, rather than a deep learner in mind, and that we perpetuate these kinds of learners rather than deep learners based on our class structures.

Ken: Yeah, exactly. Exactly. And I think we began to break away from that, by the way in which we frame the questions we raise. You think about the example of Kaufman’s course. Or let’s take an example from physics. Eric Mazur, at Harvard, has pioneered a lot of the elements of the natural critical learning environment and super course. And the students in his course do not learn physics, by listening to lecture, boring or otherwise. They learn physics by doing physics. They do three big projects. And each semester of the course… it’s a two-semester course, some students take only one semester… but in either semester, they do three big projects and they’re massive projects. And they work with a team. Each student’s is in a team of about five or six students. And they work together to try to attack a problem. And each of the projects has a back story. For example, you’ve been contacted by a charity that was created in Venezuela, by a well known philanthropist and musician, who became quite convinced that music, and symphonic music in particular (being part of a symphony orchestra), is a transformative experience that can help very poor people rise out of their poverty, and to develop a different mindset that enables them to conquer some of the economic circumstances they face. It’s a program that now has about a million students worldwide who are engaged in it. But it has a problem, namely that some of the students are so poor, that they cannot afford to buy real instruments. Now, you’ve been studying waves and music is made up of waves. So, your team has been invited to create new kinds of instruments that can be made from things that you find in the junkyard. Now that new instrument has to be able to be tuned over to different octaves, has to stay in tune for a specific amount of time, but by creating these new kinds of instruments, you can help the young children. Now, that’s a compelling project. They learn physics and the physics of waves actually doing the project and demonstrating that they can do it. Another project is more fun than anything else. Do you remember the old Rube Goldberg cartoons?

John: I do. I don’t know if Rebecca does. [LAUGHTER]

Rebecca: I know what they are. [LAUGHTER]

Ken: Ok. There was this guy. He was an engineer and he was also a cartoonist from San Francisco. He used to draw these cartoons, where you would attempt to do something very simple, like crack an egg, by a very elaborate piece of machinery, where balls would roll down ramps and would trigger gates that would open up other gates that would cause other balls to fall, and so on, and so forth. So it’s a very elaborate, unnecessary project. So on the Rube Goldberg project, so students are invited to create a Rube Goldberg type device to crack an egg… [LAUGHTER] something you can do very simply by knocking the egg against the edge of your kitchen counter, or your kitchen table. But no, you had to create this very elaborate project. Well, to do that, all of Rube Goldberg drawings and inventions were based on physics. That ball rolling down a ramp at this angle will acquire this speed, and it would open this door and would result in this hammer hitting this hammer that would cause this ball to roll down this ramp… and that would, etc, etc, etc. So you had to know the physics in order to create these absurd projects. But there’s a lot of fun in that. A lot of fun. And the first project they do, I think, is one was just strictly on fun. They build a racing car, and then race each other, like many of them did when they were back in the fifth grade. And they have great fun in doing that. Another one is one where they have to design a lock that other people can’t open. And they get points for 1. being able to keep other people out of their safe, and for being able to crack the lock of other peoples’ safes. So a lot of wild times. Now there’s a textbook that stands behind all of this. And the textbook is written by the professor. And it lays out everything that he might otherwise have said to them in lecture, and more. Now, how do you get students to read that textbook, 1. by creating these enticing projects, but another way you do it is by making reading of the textbook into a social experience. So Mazur and his colleagues created a program which is now available to everyone, called Perusall. And, using Perusall, students can read material together. So in the sense that, “Okay, we’ve got an assignment, we need to read to chapter two. We’re all reading it together, you’re reading it on your computer, I’m reading it on my computer. And as I read along, I may have questions. So I’ll highlight that text, I’ll raise the question.” It’s like writing in the margin, but everybody can see what you wrote. He organizes the groups into groups of 15 or 20 students apiece, and they can read each other’s comments and they can respond to each other’s comments. And to participate in the class, to be a participant in the class, what it means is that they will keep up with offering the comments to each other, and their comments on the text itself, making comments on what’s written there, to raise questions. to answer a question, so on and so forth. And what they found is that reading completion goes from 40 something percent at best, all the way up to 95 to 100% completion.

John: I’ve been using Hypothesis in my classes for the last three years, and nd it’s a very similar type too…

Ken: Yeah, there are several others out there.

John: And students really enjoy that, too. They enjoy seeing what other people are raising questions about. They enjoy answering questions for each other and posing questions to each other, and It seems to make the reading process much more engageing when it becomes this social activity.

Ken: Exactly. Well, that’s the whole idea. And then of course, behind that is this set of really intriguing, interesting, fascinating projects. And there’s a course at an engineering school in Massachussetts that offer a course, for a long time. They no longer offer the course unfortunately, but they offered it for over a decade, I think it’s called the “history of stuff.” [LAUGHTER] And the first day of class, the students go in and they see on the front table stuff. It’s the kind of stuff you might find by going down the aisles of Walmart [LAUGHTER] and picking things off the shelf, just a wild assortment of things. And they’re invited to come down and pick out one of those items. And then to begin to explore it, explore its history. Why was it created? What was it created out of? What materials? What kind of implications does its creation have for society? Does it just clutter up society and create a backlog of unrecyclable material that creates environmental problems of one type or another? Or exactly what is it? Students had a great deal of fun just exploring stuff. And the course ended up by looking at some of the history of technology through the lens of a well known American patriot, Paul Revere. But Paul Revere was also an expert in metals. And so they explore engineering of metals through the eyes of Paul Revere. And it becomes a way of mixing disciplines in a way that makes each discipline more intriguing and more interesting. Rather than “Oh, you study this, then you study that, you make no connection between the two.” Say, one more example?

John: Oh, sure.

Ken: There was a course we looked at in southwest China, and we went to the school, Southwest Jiaotong University, in Chengdu, and it was a course organized by a young woman who teaches physical education. And first day of class, the students are invited to think about what kinds of sports they enjoy doing. is in rock climbing. Is it soccer? Is it basketball? What is it? Now, can you imagine creating an exercise device that will make you a better soccer player, or rock climber, or whatever it is that you want to do… your favorite sport? And then the whole class goes to a sports equipment store, and they began to look at the equipment that’s already there. And then began to think about, “Okay, how can I create something better?” Now, as part of the team, it’s not just this PE teacher, but it’s also other people. Ah, you need someone from biology perhaps, to help them think about… “Well, what kind of exercising do the human muscles need? What kind of social environment do you need to create here?” So, you need other experts. And if you want to make this a product that can go on the market, maybe you need a marketing professor, who can help you devise a marketing plan of the new product that you’re creating. I told this story to my broker several years ago, and his response to me, was: “In Communist China?” And I said, “Yes, they have a market economy just like we do. And they’re interested in marketing and learning marketing. And they have marketing professors, just like we do.” And so they creates this environment where students learn by doing. And they learn by mixing disciplines, rather than keeping them apart. Much more interesting.

Rebecca: I love the move towards more interdisciplinary work….

Ken: Yeah.

Rebecca: …something that I feel really connected to, but it really gets people I think, more excited about different disciplines when they’re more intertwined, because we understand how they’re related to one another.

Ken: Yeah, exactly. So if you’re going to study the human brain, for example, how do you make it interdisciplinary? One of the professors, we studied taught in the medical school and taught medical students about the brain. But she was asked to create a course for undergraduates. This was at Vanderbilt. And so the course on the brain for undergraduates mixed every discipline you can imagine, together, because, as she argued, everything is connected to the brain. So you might be studying music, if that’s your interest, or whatever your interest might be. You might be studying ethics, if that’s your interest, and then you’re encouraged to think about what part of the brain handles ethical questions? What part of the brain helps you to appreciate and understand music? What part of the brain helps you to do this or to do that? So they’re studying all the aspects of the brain, but they’re also studying all these other disciplines, from Holocaust studies to a wide variety of other things, and raising deep ethical questions along the way. And she offered this course for 10 years, and it was a transformative experience for most of her students, the overwhelming majority of them, breaking down stereotypes and prejudices and helping them to also think more deeply about how their brain operates.

Rebecca: Sounds like there’s a lot of classes I should sign up for.

Ken: Exactly. I thought at one point, I was talking to some high school students about where they want to go to school. And I said, “Well, it’d be wonderful if you could go to a school that would mix all these super courses together. Because they’re strung out all over the world.” Maybe there’s a way of doing that virtually I don’t know.

John: Or maybe, as a result of your book, and other similar work, perhaps more faculty will start doing this type of thing and more of their courses.

Ken: Yeah, and perhaps, in designing a curriculum that includes professors from a wide variety of different disciplines, and students from each of those disciplines, working together in small groups, to tackle problems of physics, and then later tackle problems of the brain or tackle problems of history, or tackle problems of well, you name it… and have a opportunity to sort of tour super courses around the world. That would be a wild experience.

John: I still remember examples that you used here when you spoke at Oswego. And I remember examples when I first read your first book on what the best college teachers do, a while back, in large part because you weave in narratives, along with the theory and the reasoning behind these concepts. And I think the use of narrative helps makes the story much more interesting and helps raise curiosity and makes things much more memorable. Is that something faculty should strive to do in their own classes?

Ken: Yes, I think so. And the professor, I was just mentioning at Vanderbilt, I think, did that and created a course that was part history and part neuroanatomy and part philosophy and part literature and part music, but they’re all around narratives of one kind or another. Yeah, I think so. I think creating that narrative. Human beings love stories. And if we began to understand things, in terms of stories, then it becomes much more memorable to us. And we remember what we learn. And if you think about learning, it contains at least these three major aspects: we’ve got to encounter new ideas and procedures, and so forth, but we’ve got to encounter new material, there’s the encountering part. And the second is the making sense of it part where we relate it to other things that we’ve learned. And then the third aspect of it is retaining it long term. So we remember what we remembered, what we learned. And I think encountering all of this in stories, makes it much more memorable. But I think what the super courses do is they have students read stories to learn physics or history or other kinds of things. But they don’t tell them those stories orally, for the most part, they do not use lecture, to do that first aspect, that is of introducing the material. Usually, that’s all that happens in the classes, you’re introduced to the material and lecture, and you never get around, you never have time in class, for those second aspects of the “making sense”part of and the things that you might do to retain it. So super courses are built in a way that they spend their time working on those other aspects. Because the first one, the one of conveying the new information and ideas to the students, that can be done with reading, with films with other ways. But the part of struggling with meaning, with the teacher and with each other, that’s much more complex, and that requires a different kind of approach. And that’s what the super courses offer.

John: This project began with a collection of syllabi, and we should probably note that those syllabi do make it into the book as an appendix. So, not only do you have the stories of how these classes work, but it provides faculty with examples of how these things are implemented. In an Invitational syllabus.

Ken: We took excerpts from some of the syllabi, not all of them. But from a few, to give people illustrations of what we’re doing: one from math, one from the sciences, and one from the humanities.

Rebecca: We always end by asking, what’s next?

Ken: [LAUGHTER] Good question. Well, I have on my agenda, and I’ve been working on, a book aimed at parents. And the working title of the book sort of summarizes the whole idea of the book. Although the working title is 11 words long, andt hat is way too long. But we’ve got to find a way to achieve the same thing with a shorter title, but it’s: How to Help Your Kids Get the Best Out of School. Now we chose those words carefully, because the first task is defining what we mean by the best. And, in part, it means learning to learn deeply. So how do you help your kids to do that? And we chose one of many words we might have used for kids. We said “kids,” ‘cause it’ss short and to the point. We’re trying to shorten bold type as much as possible. And I’m working on that with a colleague, Mindy Maris, and we have a due date with Harvard press of 2022. So, that’s coming up rapidly. So we’ve got a lot of work to do over the next year and a half. But we’ve already done quite a bit of work in organizing that, and so forth. So that’s the next major project.

Rebecca: That sounds like an exciting addition to the collection that you’ve already have out, and rounds out the offerings.

Ken: And then somewhere in the far distant future I play out entirely, I would love to take all the we’ve learned and how to understand the best in any field. And that was a process in itself. How do you define the best and how do you collect evidence that something is better. I’d love to do a book that might be entitled: What do the Best Coaches Do? [LAUGHTER] and describe good coaches in a wide variety of different sports. But that would be my swan song, if I ever get around to it.

Rebecca: Thanks so much for joining us today and sharing some insight into your newest book. I know that a lot of our listeners will be looking forward to reading it soon.

Ken: Well, I look forward to hearing feedback from your listeners. And as we said, toward the end of the book, we hope that at some point, every reader will say “I wouldn’t do it that way. I’d do it this way.” But when they say that, we hope that they will base that judgment on strong evidence that that presents, whatever alternatives they come up, presents a better learning environment than the one we describe in the book. But, we hope this idea of a super course, is something that is organic, it continues to grow. And five years from now, somebody will summarize something about super courses today, meaning the super courses, 2027 or 2030. And may describe a much different book than the one that Marsha and I wrote. But it’s an organic process. And we’re looking forward to the conversation w e hope that the book stimulates.

John: Well, thank you. It was great talking to you again, and we’re looking forward to sharing this with our listeners.

Ken: Thank you.

Rebecca: Thank you so much for your time. We really appreciate it.

Ken: Oh, my pleasure. My pleasure. Anytime.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

181. Capstone Experience

Imagine a course in which the faculty member is a coach who guides students through a real-world project with messy data and the problem-solving that comes with it. In this episode, Dr. Kathryn Berkow joins us to discuss how a course with no content can provide students with a rich learning experience full of analysis and insights. Katy is an Assistant Professor of Management Information Systems at the University of Delaware. She is also the host of the On Cultivating Student Engagement in Higher Ed podcast.

Show Notes

Transcript

John: Imagine a course in which the faculty member is a coach who guides students through a real-world project with messy data and the problem-solving that comes with it. In this episode, we explore how a course with no content can provide students with a rich learning experience full of analysis and insights.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Kathryn Berkow. Katy is an Assistant Professor of Management Information Systems at the University of Delaware. She is also the host of the On Cultivating Student Engagement in Higher Ed podcast. Welcome back, Katy.

Katy: Thanks so much for having me on again.

John: We’re really happy to talk to you again. And our teas today are:

Katy: I’m having a decaf Earl Grey.

Rebecca: …good classic. I’m having a Scottish afternoon tea, which is my new regular.

John: And I am drinking vanilla almond tea. It’s a black tea from the Republic of Tea.

Rebecca: I really like almond teas.

John: This is my first, it was a gift from my son.

Rebecca: So we invited you back Katy to talk about your capstone course that you offer. Can you tell us a little bit about the course?

Katy: Absolutely. I love talking about this class. It’s one of my favorite courses to teach and I teach it every spring. And so having the break in the fall teaching my other courses gives me a lot of renewed energy for this capstone. It’s a senior capstone experience for our business analytics minors in their final semester as they prepare for their next steps after graduation. And the students have a variety of majors and a variety of perspectives in the class. And analytics is only their minor focus. And so in course design I have had, I would say, two influences. I have a colleague who designed a similar capstone for our management information system students that relies on semester long projects with an external sponsor. But also, before coming to the University of Delaware, I worked in the financial services industry as a quant analyst. So I worked through lots of long, larger scale, analytical projects. And so I modeled the course after my colleagues set up quite a bit, but I adapt it from my own professional analytical experiences. So when I first started teaching this course, I shied away from saying what I’m about to say, but now that I’ve seen enough students through the course to know that they learn so much over the course of the semester, I feel very comfortable sharing this: there is no content in this particular course, and I think that’s what makes it so much fun. So we spend the complete semester working on a large scale, real, unwieldy project that is truly representative of the type of projects students will face in their professional careers as data scientists. The students work on the project throughout the semester, they report to me or my co teacher as their manager every week. And we provide feedback on performance, suggestions and resources for how to move forward. When the students are stuck, we’ve usually seen something like that before. And we can brainstorm together how to get unstuck. And sometimes all the students need is confidence that the direction they’re planning to go in is a good one. But because there’s no content, the projects can really unfold and be the focus for all of us throughout the semester, and each student team gets a unique experience. So the two important things that I really want students to know after finishing this course, are: one, that there’s no perfect answer to a complex problem… there are only a degrees of good, better, better than that… when it comes to analytical solutions, which I’m sure is true in so many areas; and number two is that each unique problem needs its own thoughtful solution. We’re not trying to teach students in this class how to think about every problem, just how to think analytically now that they have the analytical tools. Not what to do in every situation, but how to think through each complex new situation they face.

Rebecca: Do students in this class all work on this same project, or do you have small groups working on projects.

Katy: So each student team is working on a project throughout the semester, and this semester, for example, I’ll have 42 students in the class working on nine distinct projects.

Rebecca: Do the students define their own projects, or do you have a predefined project.

Katy: So I create the projects for them with community members. Because there’s no content in the course, the project is the critical design component. So each year, I start getting ready months in advance curating these projects for the semester. In the past, we’ve worked with our own athletics department on a variety of projects, a large retail banking institution, a service provider of home repairs, a few local nonprofit organizations, and lots of others. And so the variety is exciting, because the students all have different interests as well. And I tell these organizations that all I really need from them is a sufficiently challenging research question. I mean, everybody’s got lots of questions, but we have to really hone in on one theme, and then enough data to support finding an answer to that question, perhaps. It’s not been too challenging to get project sponsors interested, because I’m offering free analytics. [LAUGHTER] And so I might contact someone through a friend of a friend and say, “Here’s a few naive questions, I think we might be able to help you answer if you have the data,” and people generally seem excited to have an introductory conversation. So, for example, some of the organizations that we’ve worked with want evidence of program efficacy. They might have survey data or some measurement of before and after metrics on the students or participants in the program. And we can use that data to answer questions. Others have said we want suggestions for how to price the seats in our new stadium which is super open ended and they’ll provide some data about ticket sales, for example, and so it’s a very open ended data-driven question but it’s not standard. And sometimes those non standard questions are even more fun. So I write up a project description after a couple of meetings, discussion, thought with the project sponsor. That might be two to five pages, it’s not a lot of information, and then I get the data into a format that students can work with, which sometimes is me doing nothing to prepare the data. I do want the students to struggle a little bit with formatting the data since data cleaning is a big part of learning to be a successful data analyst, but I provide lightly processed data and a project description to the students as their starting point. So, I ask them what project they’d like to work on after introducing the projects to them on the first day of class and I try to fit students to a project that they’re most interested in, but really sometimes i’m surprised that some projects are more popular than others and it’s not the ones that I expect

John: Paul Handstedt has a book called Creating Wicked Students: Designing Courses for a Complex World and what you’re doing sounds very much like what he’s advocating. Giving students really challenging problems where there’s no clear solutions, is a really good way of helping them pull together all the things they’ve learned. So in a sense the content is really everything they’ve done up to the capstone but you’re giving them an opportunity to apply it in ways that they’ll need to, if they’re going to be successful in their career

Katy: Absolutely. Completely. Because I did read that book and really felt inspired and I think I was already doing this style of course at the time, but it made me feel like I was headed in the right direction, that giving students this opportunity to try solving a problem that has no answer and most of the problems they try to solve in their careers don’t already have answers or we’d just be using those existing solutions. And so it is really good practice I think for whatever’s next especially in the field of analytics. Ee don’t know what the technology looks like or the methods are going to look like as we move forward and so they really need to be able to think critically about ethical concerns, methodologies, how to work with their data in a really honest and skilled way that can be applied to lots of wicked problems.

Rebecca: So, it sounds like in a process like this, which sounds similar to kinds of courses that I also teach, you act more as a coach rather than more of a traditional teacher, kind of coaching them along on how to respond to the data, respond to what’s happening in the moment. Is that correct?

Katy: Absolutely, and that’s something that I love about this class. I know a lot of your guests have talked about removing themselves from the sage on the stage position and becoming the coach or cheerleader in an active environment and that is one of the things I love most. When students are excited and driving the questions, I get even more excited to talk about what those answers might look like with them.

Rebecca: Do students have the opportunity to talk to your community partners or is it always through you?

Katy: So we staged the course in three sections and we have three presentations associated. So, the students will get started, spend maybe three weeks or so working with the data, getting to know it, generating some questions and some initial discovery points, and then they’ll present those results to their sponsor. And actually teaching online has been much easier for the project sponsors because they can easily attend presentations and provide feedback. Usually in that first round, the discovery period, the students present something and the sponsor can say, “Oh, let me explain that.” It’s sort of a back and forth where there’s a lot of sort of correcting any misunderstandings or answering questions. Phase two, the students are working toward what I call an initial solution. This solution might be a basic model that makes some assumptions that are maybe not appropriate, but just to get started. And it underscores what I love most about the class which is the idea of iterative solution. Presentation three is going to give them an opportunity to refine the solution, completely abandon what they did for presentation two, or improve it in a way that makes it more realistic, more robust, more an answer to the project sponsor’s research question. So, absolutely, the students get to interact with their project sponsors during those presentations where they are leading the show, they’re having the conversation, they’re hearing the direct feedback as though the project sponsor is their manager in that moment. And then I can sort of serve as a liaison, saying “I think what you’re saying is this,” “maybe we can put that on our to-do list,” and sort of just offering the support to both sides, to help everybody come together for a solution

John: How do you assess the contributions of individual students in the group project?

Katy: This will be the fourth time we offer this class and this has evolved quite a lot. A few things have remained the same. Those presentations I mentioned are a clear part of assessment in the class. But each week, I require that each team submit a status state. And that will be a highly detailed list of achievements toward answering their research questions, and also a list of highly detailed to-dos for the next week. So, even though it’s not a class in project management, they’re getting that scope of accountability for moving their project forward. And I require that each student’s name appear next to achievements, and next to to- do items. And that gives me an opportunity to really see what’s happening. But we also have weekly meetings. The students, instead of meeting in a classroom setting where the entire class is together, I’ll meet with each group for maybe 30 minutes. If they need more time, certainly, we can have an office hours setup. But 30 minutes is usually plenty for us to discuss anything from the status update, for them to get feedback from me, and for me to say things like, “It doesn’t really seem like we’ve done enough this week, what’s going on team?” …just like their manager would say in real life, maybe they haven’t had a chance to have that really, in-person accountability conversation before. The need for that is very rare. Most of the time, I’m saying things like, “Wow, I’m so blown away by what you’ve achieved this week. How can I best assist this week with your to-dos? What do you want to talk about in this meeting?” but those are the three components. I recently transitioned to specifications grading, which has been a ton of fun in this particular class, because the senior students are so independent and really prepared to graduate, that this gives them a lot of flexibility. So I require excellence in answering the research question as the C-level component. So the team grade is the C, do a great job on this project. And then I can add in individual components that will scale toward a B and an A. So for this semester, for example, the students will earn a B if they complete a data ethics module where they have to think and write about some ethical dilemmas in data science. They write reflections on visiting speakers who are analytical professionals that we have come to class via zoom, and they evaluate their own performance. And then to earn an A, I ask that the students take on a particularly challenging component of answering their team research question. And I don’t give a lot of guidance there except to say, discuss it with your instructor so that everybody’s on the same page. So I can help them determine what is sufficiently challenging to be truly deserving of an A, because every project is so different, I don’t want to spell out what they need to do. But an example would be if you’re working toward a C in the class, you could work with a team member to generate a model that answers a particular, say, sub question of your focus. But if you’re working toward an A, you need to be maybe developing that model on your own. The one thing that I do promise students is that there will be no surprises. I know it sounds like there’s some looseness in the specifications. But we’re talking about it every week in our meetings. And if students are not on track to earn that excellence, C-level grade, then they know about it with plenty of time. And I’ve really never had to give students much more motivation than “Hey, I haven’t really seen much from the team this week. Let’s talk about those to-dos for next week.”

Rebecca: It sounds like those meetings are really important in terms of processing learning, not only just moving the project forward, but also just processing “what are they learning?” …and how might they move that forward in the future, not just in their projects? And you also mentioned some sort of self-evaluation or reflection, can you talk a little bit more about that component?

Katy: Oh, sure. At the end of the semester, I ask them to do this performance evaluation of themselves in much the same way that I have to write one in my role and have done in the past. It gives them a little bit of practice self reflecting. I’m not really judging performance based on their performance assessment. I’ve already seen what they’ve done in the meetings. When they speak with confidence about something complicated, I know that they’ve learned a lot. So I really just ask them to be honest, and say, “What do you feel like were your greatest growth points? What do you feel like you still need to work on as you head into a professional role as a data analyst?” And I also ask for feedback about the course. {What were the elements that you felt contributed most to your growth this semester, and what things didn’t contribute anything?”

Rebecca: What have your student responses been to working on these projects.

Katy: So ever since the first time the course was offered, the students have expressed sincere appreciation for the help in making a transition from student to professional. It seems like by the time this course pops up in their schedules, they are really ready to start becoming more independent, and squeezing into a seat in a classroom just doesn’t feel comfortable for them anymore. So they express that this course and other capstones like it that are problem based really give them an opportunity to be in the driver’s seat and have more independence in their senior spring. Many also have said that they’ve learned the skills of analytical thinking, data cleaning, planning, modeling, but now they’re seeing for the first time how those things go together in a sequence and on a complete scale over much more time. This isn’t just a week long project where everything is abbreviated and if they’re going into an analysis role this is going to be what their career is like. So, I couldn’t be happier than to hear those two things. It makes it feel like a success. But, I’d also like to add something really selfish here. I get so much personal fulfillment from teaching students, at any level, but this class really gives me the opportunity to stand back and coach as you said earlier, Rebecca, rather than be that sage on the stage in front of the class with the slides. And I just get so much enjoyment from seeing students take off, watching them steer the ship for the first time, getting their answers. I also teach them at sophomore levels, the intro course, and so i’m lucky enough to see them again at the senior level and just saying that it makes me feel really proud is such a dramatic understatement because seeing them ready to leave the University of Delaware and become professionals… it’s really really fulfilling in a selfish way.

Rebecca: My experience with classes like that, too, is that students really appreciate the opportunity to try out this pretend career for a moment. [LAUGHTER] It’s a safe place to try that rollout and understand where they fit in the bigger picture or what specific role within an area or field might be a good fit for them in a way that an internship doesn’t, because it’s maybe a little more flexible or you have more of that direct contact with the faculty member during that process instead of just in the work environment. I’ve had students that have said that in those scenarios they’ve really appreciated that opportunity to fail without repercussions.

Katy: Yeah that’s a great point. It’s like we’re the coach but we’re also the bumper in the bowling alley… that we’re just trying to keep them on track and rolling forward. And I think what you said is true, especially as I think about what their roles might look like… different positions as data analysts… maybe they’re the best at doing the analysis, building the models, or maybe they’re the best at communicating the results and being the liaison between people who are super analytical and people who are not super analytical. So, this class, just as you said, gives them an opportunity to try on the different hats that might be available and see where they fit best in a comfortable supportive environment.

John: When you mentioned watching your students grow from seeing them as sophomores to seeing them as seniors, I have very much the same experience. I used to teach a wider variety of classes but in recent years, since i’ve been working in the teaching center, I primarily see them just as freshmen in a large intro economics course and then I see them generally as juniors and seniors in any econometrics class and in a capstone course, and seeing the change in them and seeing how they become confident with their material and seeing the work that they’re able to produce is really impressive. It’s a really nice feeling.

Katy: Absolutely and when you’re talking to people from other universities or if you’re talking to a panel during accreditation, it’s really, really nice to be able to speak to the entire scope of the educational process. When you see them one time as sophomores, or even one time as seniors,you only get that point in time feedback, but when you get to see the whole development process it just makes you so proud of what the students are learning and you have just such appreciation for all that your colleagues do along the way as well. It just makes it feel like there’s a symphony happening here and that you can see it much more clearly.

Rebecca: So, Katy, you’ve talked about this is the fourth time you’ve taught it. So, the first two times I believe were in person, then you had a time that started in person that shift out not in person. I imagine this time is remote.

Katy: Absolutely.

Rebecca: So, can you talk about that transition or transformation?

Katy: Sure. Last spring we were in person for five weeks and then we transitioned online and I was actually having one of my group meetings with students when the email came out. So, the email came out announcing that we were closing maybe at like 4:55 and we were meeting at five. So, I corralled the team into my office and I said “What’s going on? Just read it out loud. Let’s get this out in the open.” And it was so helpful for me to see the students processing the information live because I was processing it too but thinking in the way that they were thinking in that moment was so helpful to me. One student, for example, raised this question of “But i’m supposed to be on spring break when they say we’re coming back to school…” And i’m just like “Uh huh, let’s take it one day at a time. this is obviously new information for all of us.” But that’s how she was processing and the next student said “Are we going to not have a graduation?” …and I don’t know the answer to that. I said “Well I don’t know if this is going to be two weeks oor three months or longer, but we’re just going to take it one day at a time.” it was just so fascinating to hear what was happening in their minds. One of the questions that came up was “What’s going to happen in this class?” …like they felt almost more concerned about this class than some of their other ones transitioning online. And I said, “Well that you certainly don’t need to worry about. I feel less concerned about transitioning this particular class online than I do my other ones. Because all we do is have conversations. We have conversations together in this small group, we have conversations as a class where we’re listening to a speaker, and we have conversations with your project sponsors. And we can do all of that on Zoom.” I was lucky enough to have had some experience with Zoom beforehand. And so I just felt really confident that there might be one or two things I had to think through, but that most of the things that we were doing could easily be achieved via Zoom. So it was the other classes that were more, I’ll say, traditional in delivery format that I was worried about. So this, I feel like the students are getting exactly the same experience, they just don’t get to shake their project sponsors hand, which, you know, is a little bit disappointing. The networking component is really nice in person, but it’s not necessary. And I think meeting in these small groups, I still get to know the students just as well, and really can serve in the capacity of whatever they need, whether it’s mentor, thinking about helping them to find a job, or just project mentor, whatever is needed, I can do via zoom, because it’s in this sort of small group protected setting. And so it has been maybe the greatest challenge to transition other courses, not this one. So I really feel good about how this one has transitioned online,

Rebecca: I was shaking my head up and down the whole time you were talking, Katy, because I felt the same way about my project based courses. And in some ways, some of the logistics got so much easier being online. There’s some classes that I do that were project based, for example, our Vote Oswego project that I do with a political science faculty member in the fall, we do things with our classes together, it was so much easier to find a space we could all fit in on Zoom, we could easily get in and have the space to go into small groups without getting too loud. And some of those logistics actually were really fine online. And then even with some other projects that we were doing, having our community partners join us more easily, in a lot of ways, by being on Zoom, rather than having all the logistics of going to campus in person and finding a time that’s going to work, because there’s all the travel time involved and what have you as well.

Katy: Absolutely. One of our visiting speakers comes from Denver every year, just to be with us. She happens to be a graduate of the University of Delaware and loves spending time with our students and giving back in that way. But when I had to call her and say, “Hey, we don’t need you to come all the way to Denver anymore, we’re going to be virtual,” maybe that was a relief. [LAUGHTER] But now it just opens the door. I’ve heard a lot of your guests say it opens the door for so much flexibility in the future to just bring in more voices into the classroom and have an opportunity to learn from just a variety of different people because the commute time is zero now.

John: Going back to your point about students being a little anxious of how the course was going to proceed, I had a very similar experience with my capstone course compared to another more traditional type class. I met with them on the Monday before the shutdown began, and it wasn’t announced and there had been no discussion, at least that I had heard of on campus. But it was pretty clear that campuses all over the world were shutting down. So it was pretty clear we were going to as well. And I said “We should be prepared that this might be our last time meeting in person.” And they said, “Well, what are we going to do? How are we going to keep doing all these meetings and keep going?” And they wanted to make sure the class would be successful. I said, “Well remember how I told you, if you couldn’t be here sometime, you could just come in on Zoom? Well, that’s what we’ll probably be doing in the event of a shutdown.” Most of them had laptops there, I said, “Go to the Zoom website, create an account, [LAUGHTER] just so you’ll have it there, because you may want to create your own sessions for work within your groups.”And they downloaded Zoom to their smartphones and to their laptops. And I said,”Let’s just try connecting and just make sure you mute your microphone so we don’t get any feedback issues.” And they were pretty relaxed about it. And nothing really about the course changed other than the fact that they were meeting over a computer. They were doing all the presentations, they were doing all of their group work in breakout rooms instead of gathering tables around. And actually in a lot of ways it was easier because when they were meeting in small groups in the physical classroom, they were all looking over the shoulders of the people who were actually doing the writing at the time. And it was just so much easier for them just to share the screen and discuss it from wherever they were and have a much clearer view.

Rebecca: I think it’s really interesting that both of you are talking about how concerned the students were about being able to fulfill that particular class or that particular project and I think it really attests to probably their commitment to that project. And Katy, in your case, it might be just because it’s somewhat high stakes, right? There’s a client or a partner involved that you want to satisfy and it feels really satisfying to do real work for real people. And I’ve had this experience as well with community based projects that students just are all in on those kinds of projects, and just don’t want to see them fail at all.

Katy: That’s true. I’ve never had a problem getting students motivated. I told you once in a while, I’ll say like, “What’s happening this week?” and then you hear there were some tests or something like that. But honestly, the students are highly motivated all semester, because they’re getting to interact with those project sponsors during the presentations. And they’re going to be accountable to that person’s face or group of people during the presentation. And so sometimes I worry, as we get closer to the end of senior spring, that students are going to lose their motivation, and it really doesn’t happen. You know, they’re tired by the end, they’re ready to be done with the project, because this might be the longest project they’ve ever worked on. But they really deliver. And they always really impress me, I don’t feel any stress at the end of the semester with grades, because they, number one, they know what’s happening. We’re very clear all semester long about where their grade is headed, but also because they’re driving it. And this is a group of students who’s elected to have a minor in addition to their major studies. And they’re just highly motivated. Most of the class is earning an A by the end because of the excellent work they produce.

Rebecca: Sounds like a really fun class to teach.

Katy: I have so much fun. The hardest part’s the project. Once the projects are made, the rest is easy. I just show up and ask questions.

Rebecca: Mm hmm. I love my projects classes,

John: Is there any type of artifact that the students create that they can share with potential employers as a demonstration of what they’ve learned.

Katy: I would say I always encourage the students to include their project on their resume. But I do ask students to sign a nondisclosure agreements for their project sponsor. Tthat makes everybody feel more comfortable. Sometimes the data is sensitive if it’s got, for example, young children participating in a program, and other small businesses might not want to share their data. And so I have the students sign something. But I tell them, if you’re interviewing, you can certainly describe your involvement in the project in a loose way, you can talk about the specifics of the modeling you did, and how you contributed to the client’s end goals without saying, “Oh, I worked for this specific company.” And they also do create an executive summary. So that’s an artifact. They share their presentation slides, of course, with their customer, and they create an executive summary. But the goal of that artifact is to deliver information to their project sponsor, and not necessarily to serve as a portfolio. When I offer this class in the graduate level, where I have professional students who might be working, I don’t also ask them to work on a project for another organization. They collect their own project from publicly available data, and they generate a description of what the impact they could have studying this publicly available data might be. They create a digital portfolio using a WordPress blog. And then that’s something that they could really share as an artifact. So if it happened to be publicly available data they were studying, then they could certainly share that with a prospective employer.

Rebecca: It’s also a great opportunity at that capstone level to have conversations about the way that the profession works. Whether it’s nondisclosure agreements or copyright or whatever it might be related to what you’re doing. These are important times to have those conversations before the students graduate and move into their professional lives.

Katy: Absolutely. Because the data in our classes is often either… I’ll call it simulated, which is just made up by me, as an example, or some publicly available data that’s already in a nice format. So, it’s good for them to have the exposure to working with data that is just in a different format, because it comes from a different place. Or to see that not every dataset looks the same as the one your teacher might have curated in your sophomore level introductory course,

John: We always end with the question: “What’s next?”

Katy: So Monday is the start of our semester at the University of Delaware. So what’s next for me is diving into nine really exciting projects this semester with our seniors this year. So I’m really excited about that. But for the course overall, there’s going to be a big change coming up. And that is that I mentioned we have minors in business analytics in the course. We recently added a major in business analytics. So I think the exciting thing to look forward to is that we’ll have an even greater mix of students coming up either next spring or the following spring that will include people who have had even more training in analytics during their time in the University of Delaware, and it’ll increase the variety of solutions that we can provide to our project sponsors. So that’s really exciting, as is being part of a growing program. But in addition to that, I’d like to concur with many of your other recent guests who have said that they’re focused on what the future looks like in their classrooms. Certainly what’s next is going to involve some changes. And it seems like there are lots of opportunities to reimagine our courses when we have the option of being in person, but also using the new tools we’ve learned for engagement and flexibility. So in the broader sense of what’s next, I really don’t know, but I’m thinking about it a lot.

John: Our students got an interesting email. For the first time since I’ve been at Oswego, we had about a foot of snow and it was coming down pretty quickly and instead of getting a notice that classes were canceled, they got a note that all classes will take place remotely today. [LAUGHTER] So one change is, I think, for those students who used to look forward to the occasional snow day in upstate New York, those days are probably gone pretty much everywhere.

Rebecca: It’s not just the students that look forward to those days. [LAUGHTER]

Katy: Absolutely, I agree. But you’re right, that’s a big change. And we’re seeing that at the University of Delaware as well, our winter session courses were not affected by snow.

Rebecca: Well, thanks so much, Katie, for joining us and sharing other great stories from your classroom. There’s so much to learn from your practice and I’m glad that you were able to join us again.

Katy: Well, thank you so much for having me back. I really enjoy talking with both of you. But also I enjoy learning so much from all the guests that you talk with on your show.

John: And I really enjoy your podcast as well and am looking forward to hearing more episodes. And this sounds like a really great project you have there.

Katy: Thanks so much. Yeah, I hope everyone will check out the On Cultivating Student Engagement in Higher Ed podcast and I’m hoping to put out a second season this spring.

John: And we will share a link to that in the show notes.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

177. Blogging in Unexpected Disciplines

Maybe you’ve seen professional development sessions about digital portfolios or blogs and thought, “that is not relevant to my classes.” In this episode, Dr. Kathryn Berkow joins us to discuss how she has used blogging in her Business Analytics class to allow students to share their learning journey. Katy is an Assistant Professor of Management Information Sciences at the University of Delaware. She is also the host of the ON Cultivating Student Engagement in Higher Ed podcast.

Show Notes

Transcript

Rebecca: Maybe you’ve seen professional development sessions about digital portfolios or blogs and thought, “that is not relevant to my classes.” In this episode, we look at one example where blogging has been used to share students’ progress on business analytics projects with an audience.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Kathryn Berkow. Katy is an Assistant Professor of Management Information Sciences at the University of Delaware. She is also the host of the ON Cultivating Student Engagement in Higher Ed podcast. Welcome, Katy.

Katy: Well, thanks so much for having me. I’m a huge fan of your podcast.

John: And we’re fans of yours.

Rebecca: Our teas are:

Katy: I am drinking tea today, I just made a cup of something called Scandinavian detox, which is an herbal caffeine free blend.

John: And I’m drinking ginger peach green tea.

Rebecca: Oh, back to an old favorite. Me too. I’m back to the Scottish afternoon tea… just delivered.

John: We’ve invited here to discuss how you’ve been using blogs in your business analytics class. Could you first describe the class in which you’re doing this?

Katy: Absolutely. So I teach business analytics at the University of Delaware. And the course that we’re talking about today is a reformulation of a traditional second course in statistics, and it’s for sophomores and juniors. So this is a required course in the business school for most business majors. For some, it’s their last statistics class they’ll ever take. And for others, it’s creating the opportunity to introduce perhaps more study in analytics. There’s a wide range of students in the class, lots of different majors, lots of different interests. And the topics that we focus on, are sort of three-pronged instead of a traditional two-pronged approach. So we focus on introducing a programming language, which is always a topic that introduces some fear for many students. We talk about statistical modeling, which also brings some fear. And the third, the part that introduces the reformulation of the class is the communication around the results of building a statistical model. And the potential impact on the research question and what the potential is for using these analytics. We really want to connect the students to the impact that their analytics could have on an organization.

Rebecca: Statistics isn’t necessarily the first topic that comes to mind to meet a blog assignment. So can you talk a little bit about how you decided to introduce blogs into your course?

Katy: Absolutely. And I completely agree with you, Rebecca, this didn’t come on suddenly for me. There were two things that were happening concurrently. The first was, I was experiencing a lot of frustration with my multiple choice homework assignments. When I started teaching the course, I had traditional exams, and these multiple choice questions. And there was always a problem with the multiple choice questions. I’m not putting enough time into writing them well, and generating the multiple questions for each topic that’s going to create that ability to randomize selection for different students. And there was always a problem, you know, a student would email and say, like, “I think I got this wrong, but I don’t know why.” And we would discover that there was actually a problem with the question. But the second thing that was happening as I was becoming more and more frustrated with my homework assignments was that I attended a workshop about digital portfolios. And it was the kind of workshop where you go to several sessions over the day, and you have a great choice. And this was one that I was like, “Well, that sounds interesting, but I’m not sure it’s going to be useful for me,” just like you said, I’m not sure that’s a statistics class was an obvious digital portfolio class. But I just kind of listened, I loved the idea and filed it away. They were giving examples of using it for writing courses or art classes to feature students’ work. But eventually, this moment came where I realized that the two could absolutely go together and highlight the communication in my class that’s missing from the multiple choice homework questions. The students were not getting practice at communicating the impact of their results, or the meaning of their results, in multiple choice questions, really. So the idea to create a space for them to not only report results, but to talk about them in their own words, really made me excited. And I just immediately started developing this new assignment, knowing that I was taking a risk, but I was really excited about it.

John: And so as they’re reporting this, is this something they’re doing over the course of the whole project?

Katy: Yeah, absolutely. So the way the project begins, that first week of the semester, I have students select their own data set of interest from some resources I provide. So an example of that would be like I send them to Kaggle. As a first source. Kaggle.com has lots of publicly available data that people have put up on the site to share. And I’ll have the students select a data set of interest, and work with that particular data set all semester long. So they start by generating an interesting research question to them. And you know, I’m just asking them to put some context around. So some students will select something very serious. Last semester, some students wanted to work with COVID-19 data, which is so relevant and interesting, but others did not want to work with COVID-19 data, they wanted to work with something that was maybe feeling like more of a break from what we were experiencing. So, a lot of students choose sports related data. And whatever the students choose, they’ll work with that data set. The intention is for them to work with that data set all semester long, and apply every skill to that data set and report out. So, there are a variety of different topics, and I also include In my syllabus that we should just be conscious that we’re not all going to choose the same type of data. Some of us are going to choose very serious topics and we should all treat the discussion of those topics in a serious way, and some of us are going to choose some things with a little bit of levity like studying the winners of the bachelorette and what their characteristics are like, and that we should appreciate the levity in that too, and just appreciate each other’s choices.

Rebecca: One of the things that I know that I’ve appreciated in doing projects like this in my class, is the ability for students to practice that professional communication and have an audience. Can you talk a little bit about who the audience of the blog is, and whether or not it’s public facing?

Katy: Oh, sure. So the blogs are intended to be public, because they’ll submit the links to their posts as their homework assignment. And then we use Canvas as an LMS. So Canvas loads those assignments right up, and I can look at them right away, which is great. But because the blogs are public, I also include some notes about that in my syllabus, pointing out that you wouldn’t want to report personal information, for example. You can keep it as private as you’d like to, in that you don’t have to identify yourself as the author if you don’t want to. But I encourage them to keep a professional tone, especially if they want to use this as a potential portfolio to share with future employers as evidence of their study and their work. And I tell them, they can spend as much or as little time as they’d like making it look beautiful, with some really beautiful images. Some students choose not to do that at all and it’s just a plain white background with a title. And that’s fine, too. So I tell them, I am their audience, but that they should consider that this is a public space, and that they might have a broader audience. But I also include in the syllabus, if anyone is uncomfortable with the idea of a public facing blog, they can request a different option. I’ve never actually had to offer that option. But I would offer like, let’s create a Google Doc that is more like with chapters instead of a blog with individual posts.

John: What platform are students using? Do they get to choose our own? Or are they all working in one platform?

Katy: So, I suggest WordPress because it has a free option. And because that’s where I built my example blog, which I’m happy to talk about as well. And I can provide more detailed instructions and troubleshoot with them a little bit more easily. But if they want to choose Wix or something else that they’re more comfortable with, I’m completely fine with that so long as they know that I’m not an expert in any of these other platforms.

Rebecca: Do you want to talk a little bit about that example you just mentioned?

Katy: Absolutely. So the example blog that I built was sort of my first experience with understanding if this could work in the class. As I was trying to formulate what we would do, I was writing my own version of the blog, which later turned into a resource that can help students as an example. So I selected a data set about understanding diabetes, and its relationship to age and blood pressure. And this is a very common data set on Kaggle. So I was writing. The goal is for students to take each topic that we learn in class, and there’s about, say, 15 topics over the course of the semester. So if we learn to build a basic regression model, the students will then apply the basic regression code that we’ve learned in class, and we’ve seen a couple examples and apply it to their own data set. So now I’m thinking about COVID-19 and I want to understand the relationship between, say, rates of cases and age groups, something like that. So they now have my example blog to go to and read and see how I structured what I did. And they’ll say, “Okay, Katy built her model relating blood pressure to age, and I’m going to do something similar.” And one of the reasons why I like creating the example is so that they can see where they’re going. But I also really liked that it serves as sort of a textbook support where we don’t have a textbook in our class. So in the later, more complex topics, I also write a lot about how I’m interpreting my results, and why, so that they have a little more support as they try to make those interpretations. And of course, if they’re having any trouble making those connections, they can always come to office hours and ask questions. But those who have used the example blog as a support really report that it’s useful.

John: You mentioned that you don’t have a textbook, but I’ve seen on your website that you have created videos that serve as a substitute for a textbook.

Katy: Yes, as we’ve transitioned to online remote teaching, I had actually been very lucky to have had experienced teaching online before. And I wasn’t using those videos that I had used in the online version of the class in my on campus class at all. But there have always been students not able to attend class for various reasons. You know, “I’m an athlete and I’m going to be traveling, I’m going to miss class” or “I’m just sick and I’m going to miss class.” So for lots of reasons, students have needed an additional resource and, without a textbook, that’s always been a concern. So I wanted to provide those additional supports but, of course, as I’m sure you’ve considered before as well, does providing the video reduce my attendance in class? There’s this constant struggle. But as I have been doing this online remote teaching, I think that I don’t care, I just want to provide all of the resources, so that every student can find this material in whatever way is easiest, most flexible, most accommodating to their lifestyle. And so that means putting out the videos in what I’m calling a digital textbook. And I can also refer it to students even when we’re on campus as an additional support if they need it after class.

Rebecca: How have your students responded to working through this information in this way? And have they had the opportunity to look at each other’s blogs and comment and do some peer feedback as well?

Katy: That is a great question. I have not incorporated any peer feedback of the blogs into class… except this semester, as we were doing the remote teaching, I was meeting in small groups, so the students would watch the videos from the online class. And then we would come together in conversation. And the only peer feedback that I’ve ever incorporated is having the students present their issues as they work through their blog posts during these small group sessions, and then having others weigh in. And I found that to be so rewarding, it was just so much fun to have them identifying each other’s problems. And there were lots of students in each group studying similar types of data. So one student is studying hockey, “Oh, I’m also studying hockey, did you find this…” and so it really was fun to make those connections. So in that way, we have had some peer oversight and I think I’d like to find ways to continue that kind of small group discussion about the blogs. But overall, the feedback on the blogs has been positive. The way that I know that is I’m looking at course evaluations, I would say, in maybe 40% of my course evaluations, students are mentioning the blogs in general. So, not everyone by a longshot. And when they do mention it, it’s in two ways, in general: 1. I really loved the blog posts, or 2. the blog posts were really hard, but I knew I was learning. And so seeing comments like those, it really makes me feel like I’m achieving the goal of helping the students to get to a place where they’re ready to apply the skill in a new setting. So that’s the beauty of the blog, that it captures what we really want them to do, which is to get into their professional setting, ready to apply it to a new situation, because they have my example blog, they have their example blog, they can remember how to do this given those skills.

John: You provide students with a sample blog, but do you give them any other guidance on what they should include in their blog post?

Katy: Absolutely. So in each problem description, so for each blog post assignment, I’ll give a bulleted list of items that they should include. But in addition to that, because I probably go on a little bit in my bulleted list, I also provide a rubric. And Canvas, as an LMS is really helpful in this way, because I can include the rubric right in the assignment and tell students how I’ll be calculating the points for each assignment. So I can have one point for including an image showing the results, I can have one point for correctly interpreting the intercept, etc. So the students will know as they’re writing, what are the things that I’ll be looking for. But even better than that, they can also see where their blog posts needed improvement. So after it’s been graded, and by the way, it’s very easy to grade in Canvas, because it can take me just a few minutes to read through a post and know whether the student has done the five or six checklist items in the rubric and just click, and I’m done in just a few minutes per student, which is great if you have a large number of students say whether it’s 50 or 150. It’s relatively quick. And I’m very lucky to have the help of a TA with that. But the beauty of the rubric in canvas is that the students can see which buttons I’ve clicked so they can know “Okay, I got a five out of six on this post. And where I wasn’t effective was at describing what the meaning of the intercept was. So now I can go to tutoring or to office hours or talk to the TA about how I could change that for a project or an assessment.”

John: How many posts do they have to make as you’re working through the project.

Katy: So the students will work on a post for every topic over the course of the semester, which ends up being 14 topic related posts. I also have them create a welcome post that’s just “Hey, I’m here blogging as their first assignment…” that just officially sets up their blog and creates their first post walking them through the process. And they create one additional post about “data and impact” I call it so they report which dataset did I select? Why? What’s the potential impact? So I have them create a research question they’ll be focused on through the semester and describe what impact it could have. Now this is a business analytics class, but I think of it more like an organizational analytics. So even if you’re studying the bachelorette, you can position yourself to think about, “Okay, I work for Netflix, and I’m trying to understand, is there a theme here? Do we need to have more diversity and inclusion in our bachelorette or bachelor? So what are the dynamics around the situation we’re studying?” And obviously, some things lend themselves more to a traditional business analytics context than others. But I want the students to know that any organization can have any research question. So you can create that context. And then they do a final post with conclusions at the end of the semester.

John: Do you have a generic rubric that works with all of the posts? Or is there one for each of those stages?

Katy: Great question, John, you’re pointing out that there was quite a bit of setup on the front end of this. So it was good that I was really excited about it when starting the experiment, [LAUGHTER] because I had to build that example blog, I had to build each of the 17 total assignments, and that kind of semi-custom rubric to each one. So each one includes the requirement that an image must be posted, for example. But, if we’re studying linear regression assumptions, I’m going to have one point associated with each assumption that needs to be tested correctly. So there is a lot of customization. However, after creating, because these are so generic or agnostic to the material the student is studying, after creating that first batch of assignments, I’ve been doing this two years now, I have never needed to change it. And I really haven’t had any issues with these assignments when it comes to “this wasn’t clearly enough defined,” for example, because while the students do tend to become frustrated with open-ended assignments, they hear me say, a lot of the time, business analytics is an open-ended kind of structure. It has a lot of art, and a lot of science. And I can’t answer the questions, always in a direct way because your real life is not going to have direct answers to every question. And so over the course of the semester, I do find students tend to get more comfortable with this ill-defined nature. But they also have the support of specific rubrics to give them some sense of structure when possible.

John: Would you recommend this approach to other faculty outside of the more traditional writing fields?

Katy: Absolutely, but I won’t pretend to know how. What I would say is that I encourage just hearing what other people are doing, like I get so much value out of listening to your podcast, and to Bonni Stachoviak’s podcast, just hearing what people are doing in their classrooms that’s engaging their students, because one of the reasons I created my own podcast was to generate more sharing, because what happens in our classrooms is so opaque to others. As I told you, I attended that session a couple of years ago, maybe three or four years ago now. And I didn’t really think it applied. And I know we all say like, “Oh, well that’s not for me,” or “that’s not gonna work,” I just feel like the more ideas we have circulating in our minds that you know “don’t work,” and the more issues we face on our own in our own classes about, especially now, how to engage students under these changing circumstances, I just feel like having a glossary of ideas that you can pull from and maybe adapt to your own class is so helpful. So I think there are lots of potential applications to this, that I could see. In fact, I use this in my other courses. So I teach a capstone course at the senior and graduate level. And when we’re not using proprietary data related to an organization, which I do in my senior capstone, like in the graduate course, I do have students select publicly available data, I also use a blog reporting structure. So I did need to develop a similar framework with rubrics. But it creates this sort of agnostic space where no matter what project you’re working on, you can report out on that topic, and all you need to do is share a link with me and then I can see your status update for the week. And you can really tell a lot by how students are writing about what they’re doing. Now in, say, a finance class where there’s a correct answer to every question, I’m not sure a blog is going to be a perfect fit. But it’s just really nice to have it as one option for reporting something because probably there’s going to be a report or a reflection or writing assignment in some space, where digital reporting is a really nice way to reduce paper, make our classes much more streamlined for remote delivery or flexible delivery as we move into the future. I do think it has a potential for tons of applications that I certainly haven’t thought of yet. But I really like this delivery method in class.

John: And you mentioned reflection, and that’s something that’s useful in any discipline, and that’s one way in which it could be fairly easily integrated. You’ve made my life a lot more complicated this week, because I’m supposed to be preparing my syllabi this week. We’re recording this in late January. We’ll be releasing it a little bit later. And when I saw the description of what you were doing, I was thinking this is something I should try this semester. So I’m probably going to be trying to implement some of these ideas in my econometrics class this semester.

Katy: Well, that’s great to hear. I hope it works.

John: I hope I can get it all together.

Katy: Well, I’d be happy to share anything by the way, like if you want to see my example blog or if you want to see an example rubric or an example assignment description, I’d be happy to share any of those things.

John: That would be very helpful. Thank you.

Rebecca: It is funny that sometimes we get these really great ideas to try out these new things, and John and I certainly experience this frequently. As podcast hosts, we’re exposed to so many great ideas, and then we have so many. But it’s really great, even though that sometimes there’s a lot of time involved in the setup of these things, they often play out in saving time over time….

Katy: Absolutely.

Rebecca: …presuming you stay with it, right? [LAUGHTER]

Katy: Well, absolutely. And that’s one of the things that has been the neatest for me, because one of the issues with the multiple choice questions is that they were so specific, and there was so much opportunity to make a mistake, missing a dollar sign or writing the wrong variable name. With the blog assignments, they’re so open ended. All I’m saying is: “Apply using linear regression with two variables to your dataset.” How can I get that problem description wrong? I can’t. So the beauty of the blog is that the nature of the description of an assignment is so open ended, but so directly flowing from what we did in class that the students really know how to accomplish each step.

Rebecca: I’m sure they also can see the practical application and how all of this skill sets not only from the statistics part of it, but also from the blog component of it, or the digital reporting of it, is practical for their future careers and things. And I think that tends to give students a little bit more buy-in to these kinds of assignments.

Katy: I think you’re right. And it achieves two other critical things. One of them is that it brings the sort of inherent motivation that we know is so important for student learning right up front, the students are choosing what they want to study. So they’re going to have a lot more excitement. And honestly, I see that when I’m talking to them. I don’t know anything about professional hockey, but when my students are working with hockey data, and that’s the area that they have the greatest interest in, I see them light up when I ask questions about like, “Now, how did you make this interpretation? Why do you think this is related to this?” So inherent motivation is a big deal. But the second thing that is a really fun byproduct is that having read what students are doing and having spoken to them in office hours, it brings up a ton of fun examples to talk about in class. So I might remember that Adrian is working on, you know, the COVID-19 data, and I can sort of bring that up casually in class. We found this weird thing that happened in Adrian’s dataset, and why do you think this is happening? Or what do you think is a good solution to this problem that Adrian’s facing and we can talk about things that I had never imagined would come up in class just because of a quirky dataset that was publicly available.

Rebecca: I could see how that also puts students in the position of being an expert or a co- teacher in some ways, which I can see as being another motivating factor.

Katy: Absolutely. That’s such a great point, because they’re also experts in each other’s conversation. So in the small groups that I was doing in the remote teaching in the fall, one student would say, “Oh, well I bought him an idea for your data set based on our conversation last week.” And even if you just watch hockey, and you’re not working on a hockey dataset, but you really like it, you can be an expert in someone else’s space. And clearly I can’t, because I’m upfront with the students about what I know and what I’m interested in and what I’m not. So like, I don’t watch a lot of hockey. so you have to help me understand. And that sort of puts me in a backseat in the same way it puts them in the front seat.

John: Earlier, you mentioned that one of the motivations for doing this was that you wanted to move away from multiple choice tests. Have you eliminated multiple choice tests from this course?

Katy: So yes, I have… sort of. In the remote teaching environment, I have relied more on team projects and multiple choice quizzes. I was trying to model after folks I’ve heard on your podcast and others say that having smaller-stakes quizzes is much more inclusive, gives students more of an opportunity to learn and to course correct, and so I was having these multiple choice quizzes. But I’m finding that the multiple choice quizzes are really an obstacle for students. And I am as frustrated as they are with them. They’re doing great on the blogs, but they’re not doing as well on the multiple choice questions. And I suspect it’s because of the poor design in the multiple choice questions and not because the students don’t know what they’re doing. They couldn’t do a great job on the blog if they didn’t have the skills. So as we head into the spring, where I’ll be teaching remotely again, I am planning to drop the multiple choice quizzes completely. That’s not to say I’ll never use multiple choice questions again. In my in-person class, I have used multiple choice questions or a multiple choice final as part of assessing students. But for now, it seems that the multiple choice quizzes really have not been effective. So this semester, I’m going to be grading students on their blog posts, team project, and participation, which is just an interaction metric

John: Are the team projects the same as a project they’re reporting on the blog?

Katy: Oh, great question. So when I assign a team project to students, it is on a data set that I’ve selected. So at the beginning of the semester, they are downloading a folder of lots of different data sets that they’ll need for examples working through the course skill videos that we’ve talked about. And I’ll choose a data set that they might not have seen yet and write some questions that are somewhat open ended, “build a model,” and I’ll try to randomize across teams like which variables they should use in their model. So they’ll all be predicting sales, for example, but they might have different predictor variables. So team one is going to choose variables, 1, 2, 3, etc. So they’re all working with something a little bit different, but they’re trying to end up in the same place: predicting sales. And these are totally open ended. But they get to work together, and that helps. And I’ve also been using specifications grading in this class over the last semester, which has been a lot of fun. The students have started to appreciate it toward the end of the semester, though, maybe I shouldn’t have selected a semester that was so much in transition to experiment with that. [LAUGHTER] Maybe it created a little stress at the beginning. But by the end of the semester, the students were saying things like, “I really liked that I could choose my level of interaction with the course.” The reason I bring this up is because I would require revisions in team projects. So every team had to score 100%, which of course is alarming to students until you say, “there will be revisions required until you’ve scored 100%.” So I want us to move past these topics, but we are going to make sure that we learn them. And I think that was a really neat experience for me to be able to coach students through finally getting mastery, which I have never done in class before. If a student didn’t master a topic, we just move on and maybe there will be an opportunity down the road. But it really was nice to see students come to a place where they had achieved a particular benchmark.

Rebecca: Were your collaborative assignments done during class time, or were they outside of class?

Katy: So this semester, I allowed students to have an entire day that they could complete the assignment anytime between 12am on that day and 11:59pm. I also encouraged my international students to do it during that 24 hours, but to make sure it was done during that eastern time 24 hours. And they could work on it during the class time, during the scheduled class time. Because it was a day that we would have had class, but they didn’t have to. So they had the flexibility to choose a time that worked for them. If it was late at night, great. If it was early in the morning, great. Or if it was during class time. That’s fine with me too.

John: We’ve really been enjoying your podcast. And you’ve mentioned this a little bit. But can you tell us a little bit more about how you decided to start this podcast?

Katy: Absolutely. First, I want to thank you both for listening. I’m such a fan of your podcast and others that are similar. And just like the digital portfolio idea, the possibility of creating a podcast has been in the back of my mind for a while, listening to some others like yours. So just as I have learned so much from hearing what others are doing via podcasts like yours, I just feel like I can’t get enough examples of creative ideas that people are using to engage their students. And not just big ideas like “Oh, implement a new grading system.” But small ideas like “Oh, I spend the first minute of my class engaging my students in this way,” or “I have this micro assignment that we do only occasionally but builds community.” And as I mentioned before, what we do in our classrooms can feel so opaque. It’s really that I just wanted to contribute in a time when we can all use more sharing of the things that are working for us, and not every idea is perfect for every person. But just more sharing of what’s working so that we have more options to choose from when teaching feels so different and so challenging to so many of us right now. And so the podcast is generating, selfishly, tons of new ideas for me, which even if that’s the only thing that comes out of it, that’s enough. But it’s also doing the service of drawing attention to the great ideas around me at the University of Delaware this semester, and hopefully, in a broader sense in the future. And so I just really appreciate being able to feature great ideas that are happening around me that might not otherwise be heard about except in documentation of excellence, for example.

John: And often those are only seen by a few people on review committees and sharing it more broadly, both within your campus and across the whole academic community, raises the visibility of that work much more extensively.

Katy: That’s my hope because the more people I talk to, I just feel so impressed by the exciting ideas that I’m hearing from different people and to be able to amplify their voices. I couldn’t be happier to be able to do that.

Rebecca: Well, you’ve talked about some really exciting things that you’re working on. But we always like to raise the ante by asking: what’s next?

Katy: So next, for me would be a second season of the podcast. I’d really like to incorporate more ideas from a broader range of places across the University of Delaware, but also bringing in others from outside just to add to the discussion. And also, I’m really thinking a lot, listening a lot to your podcast, and thinking a lot about what the future looks like in my classroom. I will have taught remotely now for…. this will be my second semester in the spring teaching remotely. And I’m learning so many things about how we can make learning more accessible to more students under the current circumstances. But I think a lot of it applies to what our future looks like. So I’m really not sure when we go back in the fall, I’m hopeful we’ll be back in the fall safely, but I’m really not sure what my class is going to look like. I can’t imagine that I’m just gonna slide back into what I used to do without incorporating some new things. And finally, I am expecting a baby in June. So there is a new chapter ahead for me that will bring some other fun changes. And so that is what’s next.

John: Congratulations.

Rebecca: Congratulations. Yeah.

Katy: Thank you very much.

Rebecca: …definitely exciting times on so many fronts.

John: Rebecca had that experience fairly recently.

Katy: Oh, congratulations.

Rebecca: Yeah, my daughter is three and a half now, but yeah, it was a great adventure and continues to be a great adventure. So I know you’ll have a great time.

Katy: Yeah, I’m certainly looking forward to understanding all the things that are ahead and totally different and maybe they’ll also inspire some new ways of thinking about learning from seeing it through a different set of eyes

Rebecca: It definitely did for me. [LAUGHTER]

John: I think it does for everyone who raises children. And actually Josh Eyler, after watching his daughter learn and experience the world…. it inspired him to study more about learning, which is ultimately the source of his book on How Humans Learn.

Katy: I think I heard him talking with you about that on your podcast.

Rebecca: Well, thanks so much, Katy. This has been a great conversation.

Katy: Well, thank you both so much for having me. It’s really been a pleasure chatting with you.

Rebecca: Hopefully we’ll have you back in the future to hear more about the fun things you’re doing in your classroom.

Katy: That sounds fantastic. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]