300. Episode 300 Reflection

This is episode 300 of the Tea for Teaching podcast. Whether you are a new listener or have been with us for all 300 episodes, we are very grateful that you’ve joined us on our podcasting journey. In this episode, we celebrate this milestone by reflecting on what we’ve learned and how the podcast has evolved.

Show Notes

Transcript

John: This is episode 300 of the Tea for Teaching podcast. Whether you are a new listener or have been with us for all 300 episodes, we are very grateful that you’ve joined us on our podcasting journey. In this episode, we celebrate this milestone by reflecting on what we’ve learned and how the podcast has evolved.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Now that we’ve reached Episode 300, we thought we’d take the opportunity to reflect on some of the changes that we’ve seen in higher education since we launched the podcast in 2017.

John: Today’s teas are:…

Rebecca: I have Blue Sapphire.

John: I heard you just stocked up on a trip to your favorite tea store.

Rebecca: Yes, I was really excited to stock up on my favorite and I have a couple new ones too. So maybe in some upcoming episodes, we can try those out.

John: And I have Bing Cherry Black tea from Harry and David’s which is made by the Republic of Tea. When we first got started on the podcast, much of the focus was on specific teaching practices and techniques and interesting projects. Most of our guests were people that we knew or guests who were within our professional networks.

Rebecca: in the spring of 2020, as we know, [LAUGHTER] the focus shifted to the challenges associated with remote and online teaching, and the challenges facing remote learners and instructors.

John: As we became accustomed to pandemic teaching, we focused a bit more on faculty concerns as we transitioned into the transformed higher ed landscape. Historically, higher ed had been designed to serve the elites of society, and while higher ed gradually became more open and students have become much more diverse, many residual practices have worked against serving the students that we have. During the pandemic, faculty became much more aware of the inequities facing our students as well as faculty and staff.

Rebecca: Yeah, so one of the things that we’ve been talking about quite a bit is this more holistic focus on the needs of our students and faculty as humans, and really generating and creating a much more inclusive higher ed environment. How do you see that moving forward, John?

John: One of the things we’ve talked about is addressing the needs caused by the increased demands on time for faculty, staff, and students. As we developed new teaching techniques and tried to build more structure into our courses, it put much more demands on faculty in terms of redesigning their courses, in terms of paying more attention to the needs of students, and providing students with more feedback. And that has led to issues with burnout, which we’ve addressed in a number of podcasts.

Rebecca: And you’ve never experienced that, have you, John?

John: The day is not over yet. [LAUGHTER]

Rebecca: Well, how do we think about supporting faculty as we move forward with all these demands on their time in trying to prevent burnout? We’ve talked about this in a couple of episodes, but as we enjoy summer and have a little bit of more downtime for some of us… maybe not you, John… and move into the fall, what are some things that we need to be thinking about for our own classes to prevent burnout?

John: One of the things that we’re trying to be careful with when we recommend new teaching techniques to faculty at the teaching center here, is that they change approaches gradually, that small changes, incremental changes, are much easier to accommodate than the type of rapid changes that people had to do when they first moved into remote teaching. And so I think we have to be careful in making sure that we maintain a balance and we don’t burn out ourselves, because we’re not going to be very effective in supporting our students if we’re struggling to get through each day ourselves.

Rebecca: Yeah, we need to be present, just like we want our students to be able to be present and have the supports around them to be present in their learning. I think one thing that we’re also talking about in grad studies in our office is really this increased stress on faculty, and how do we support faculty, but also how do we support graduate student populations through things like accountability groups, or ways where there’s another human for accountability, but also for support, and not necessarily a mentor model, where there’s a power dynamic, but really a peer-to-peer approach to connect people together.

John: And we’re running two reading groups this fall to address some of these needs. One of the reading groups is on Sarah Rose Cavanagh’s new book, Mind over Monsters. And the other one is the second edition of Jim Lang’s Small Teaching. We had done that a few years ago, but we’ve had a lot of new faculty since then. And while we try to reach as many faculty as we can in our workshops, there’s a lot of faculty who are still teaching in pretty much the same way as faculty were teaching a century or so ago. And we’re hoping that by encouraging small modifications in teaching approaches, it might encourage more faculty to participate in introducing active learning activities and evidence-based teaching approaches.

Rebecca: It’s really easy to slip back into past practices when we’re tired [LAUGHTER] and overworked. And it’s not surprising that people have kind of slipped back into assignments and stuff that they’re really familiar with to reduce the cognitive load around new stuff and the many stressors around. So having that added support to help faculty re-engage with some of those ideas is, I think, a really great idea at this juncture. And I love that Mind over Monsters is one of the reading groups as well, because mental health is such an increasing concern, not just for students, but also for faculty and staff.

John: And we’re very much looking forward to both of these reading groups. Among the things we’ve talked about more frequently since the start of the pandemic are the challenges faced by underrepresented and contingent faculty.

Rebecca: I think when we’re introducing new techniques, and we’re thinking about supporting students around mental health, or we’re thinking about evidence-based practices and engaging in active learning, we need to remember that contingent faculty or underrepresented faculty have different barriers or different obstacles in implementing these things… or even more pushback from students and implementing some of these techniques. So we really need to be cognizant of supporting each other and realizing that we don’t all have the same kind of supports in place. I think some populations of faculty are just overly criticized. And when they try something new, it’s not accepted in the same way that a more dominant group’s adoption of those same techniques might be.

John: And that’s true both by students as well as by their faculty peers. And one of the things that’s come up in many of the podcasts we’ve discussed are the biases in both student and faculty evaluation of teaching.

Rebecca: Yeah. One of the things that I think is on the minds of our faculty too, is, as we’ve seen increased diversity of our students, we’ve seen diversity in levels of preparation. And I think those inequities have always been there. But again, maybe it’s more visible now than it had been in the past. How do we work through that in our classrooms, especially in these more introductory classes as students transition into college?

John: Well, I think those inequities have always been there, but they certainly grew with remote teaching, because our students face very unequal resources in their school districts and in their households. And when people are physically in the classroom, they’re at least exposed to the same infrastructure within their institutions. But when students were taking classes from home, as we talked about in many, many episodes, during a pandemic, they had very unequal network access, they had very unequal computing facilities, they may have been sharing a computer with multiple family members, they may have been forced to work. And as a result, the inequities in prior education and prior learning became much more dramatic during the period of remote teaching. And that disproportionately affected students from low-income households and low-income school districts. And what we have to do is provide resources, I think, for all students to be successful. And while we always should have been teaching, or providing resources and support, for all students, those needs have become much greater now, because while we are bringing in a much more diverse student body, we’re also losing students who come in with less preparation at some of the highest rates we’ve ever seen before. And we have to make sure that we’re providing the students that we accept with the support they need to be successful. And there’s lots of ways of doing it, you can build in some additional resources, you can connect to YouTube videos, and such things and provide support to students, you can use mastery learning quiz systems, and many other techniques. But we have to work towards having more faculty building that in because while many faculty are doing these types of things, and trying to build more support and more structure into their classes, it’s not a universal phenomenon.

Rebecca: And maybe even acknowledging that some students in the class are quite literally working harder to get to the same level.

John: The last few years when I’ve been teaching my large intro class, that’s something I’ve mentioned explicitly. I said, everyone here has all the resources they need to be successful. But if you had taken an AP introductory microeconomics course, or something close to that, you’re not going to have to work as hard to attain mastery of many of the concepts. If you have not been exposed to these things, or if your background in working with math and using graphs is not as strong, you’re going to have to work a bit harder. And that’s not a message that a lot of students appreciate hearing. But if we want to get all of our students to the same level at the end, the students who do come in with a weaker background need additional support to get there. And using tutoring when available, encouraging students to come in and talk to their professors and use office hours, all of those things can help but we’ve got a ways to go. What are some of the things you do to try to provide support for the increasingly diverse student body?

Rebecca: Yeah, I think the reality is that what we give each student isn’t the same, because what they come in with is not the same. So I often am trying to assess where students are and then pushing them right at that moment where they’re at, rather than expecting everyone to be at the same point. And I can do that a little more efficiently in a small class than you can in a bigger class. But I think we need to use those smaller classroom spaces to be able to do that so that everyone feels challenged, but also has what they need.

John: My introductory course is a prerequisite for all upper-level economics classes, and most of my students will be moving on to upper-level classes, so they have to reach at least a minimum level of proficiency in the discipline in order to be successful in their future classes. In other classes, instructors can be more flexible, and just try to get the most learning gains in their students, no matter what their starting points were. In my introductory class, at least, I have to pretty much take the students where they are, and try to get them all to the same place, while making sure that they’re all challenged. And that’s a very challenging goal to reach.

Rebecca: …and the difference between teaching those introductory classes versus higher level classes within a discipline, for sure, I think one of the most efficient things we can do is making sure that all students know the most effective ways to learn, because they don’t necessarily know those things coming into college, or even into graduate school. What we need to just remember is learning isn’t something that we just magically know how to do, we need to learn strategies and techniques that are effective.

John: We’ve had a number of guests over the last year or two who’ve talked about books that they’ve provided, or resources they’ve created to help students be more successful. Because one of the things that’s been pretty obvious for quite a while is that the study techniques and the learning strategies that students use are not generally consistent with what evidence suggests is most effective. And as a result, students are not using their time as efficiently as they can, by engaging in strategies that they perceive as being useful, that really result in very little increase in long-term recall… strategies such as highlighting, repeated rereading, and so forth. And one of the things that might be helpful is if we all could shift students a little bit in the direction of using evidence-based learning strategies, and some of that could take place through course design, by building infrastructures that incentivize the use of these techniques.

Rebecca: Yeah, and I think the moment that students realize that they don’t have the most efficient way, or the moment that a student begins to struggle is different, depending on some of that background, that they have. Students that come in well prepared may have never really struggled in high school, and maybe eventually, maybe even in the first year of college, you don’t struggle, but maybe it hits a little later on in their education, maybe not until graduate school. And then other students might struggle the second they get to college, because there’s not as much structure in place as there was in high school. So I think we need to be underscoring these techniques at all levels, and not just in their first year.

John: And one other thing that’s been discussed in many podcast episodes, is the importance of making the hidden curriculum of higher ed transparent to students, so that we don’t expect students to know what a syllabus is or how it could be used, that we shouldn’t expect students to know what is expected on a term paper in a class without making those expectations explicit and transparent to students. Because in general, we see a lot of students coming in, and they see it as a game where they’re trying to guess at what instructors are asking. And many of those guesses, especially for students who have not been in college prep classes before, are wrong. And they wasted a lot of time and effort that could have been spent more productively developing their understanding of the subject matter.

Rebecca: And the reality is that there’s differences between disciplines and between courses. And so the more we can be explicit about expectations within our own discipline, and within our own courses, and beyond the classroom experience of higher ed, because there’s expectations in other spaces as well, like student clubs, athletics, and all of the rest of the co-curricular activities that support student learning are incredibly important. And those are also not obvious.

John: One of the things that we’ve talked about much more on the podcast, and higher ed in general has been addressing much more extensively since the pandemic, is alternative grading approaches. Because traditional grading approaches and traditional course structures generally incentivize students to cram and to focus on maximizing their grades, rather than maximizing learning, so that if we really want students to shift to evidence based learning strategies, it would be really helpful if we could shift students emphasis away from grades and faculty emphasis away from high-stakes assessed activity and shift it more to activities that result in deeper learning, more long-term learning. And we’ve talked to many guests who have shifted to using strategy such as specifications grading, mastery learning systems, portfolio assessments, and ungraving, which has become one of the most talked about topics in higher ed in the last few years.

Rebecca: Yeah. And I think one of the things that comes up in a lot of those conversations is concerns over students just wanting the right answer and not learning and not critically thinking about the subject matter and the knowing of why and how, and doing analysis. And I think every instructor [LAUGHTER] has a desire for some of those kinds of conversations to come out in their classes, rather than just regurgitation of things that they’ve said in class.

John: Part of the issue is that when we get students in college, they’ve already had 13 years of experience in K through 12, where grades were the primary area of focus. And as a result, it’s hard to shift that focus from grades to learning. Besides alternative grading, we might use some other strategies such as encouraging students to be more reflective on their work, to spend some time in reflection-based activities and metacognitive development type activities.

Rebecca: Yeah, I know, this is a space where I was maybe a little hesitant at first thinking like, “Oh, these are just quick assignments that have no meaning,” but quickly realizing actually the value in really good well designed reflective activities that challenge students to think through how and why they learned something and what it is that they actually got out of an activity. And I’m often very surprised about how much learning occurs that is not visible, despite the fact that I teach studio classes, so I’m with my students much more than the average instructor. So I actually do observe a lot of learning. But in the reflection activities, I’m hearing a lot about how students are spending their time or things that they really struggled with and worked through that I wasn’t aware of. It also helps me understand where they’re not aware [LAUGHTER] of their own learning, or where they’re using strategies that aren’t as effective and helps with interventions. I know you’ve done a lot around metacognition, especially in your lower-level classes, but also in your upper classes.

John: In at least a couple of my courses. I’ve been using the metacognitive cafe discussion forum, which was actually the topic of our second podcast, Judie Littlejohn and I jointly developed this quite a few years back. And it’s been remarkably effective. It’s basically a low-stakes discussion forum that I’m using in my online classes, where students will reflect on their learning and share their learning strategies and will also read a bit about retrieval practice and spaced practice and the benefits of sleep in learning. And every time I do it, even though it’s only a trivial portion of the grade, it’s 5% of their total grade for participating in that activity, the students report that it was the most valuable learning experience they had in the class. A large proportion of the students at the end of the terms say they wish that they had learned these things back in elementary school, that they had been using practices that were not efficient and they didn’t realize that because they’ve never been taught how to learn. And it’s something that students have found really valuable. And the other nice thing about it is because, in this particular case, it’s done in a discussion forum, it helps them build community and helps them get to know each other, because they’ll often talk about the challenges they face. In online classes, many of the students have families where they’re taking care of young children, they may be working different shifts, they may be faced with other challenges that normally wouldn’t come up in a content discussion forum in an online class. But when they share that, and they share those challenges, and they share their career expectations, and they talk about how what they’re learning might be useful in their expected careers, besides the sense of connection, it also helps students see the relevance of what they’re doing and sharing that with other students helps build a little bit more intrinsic motivation in learning.

Rebecca: It also seems like there’s a bit of an immediacy in that context as well, because the information can immediately be put into action in a real lived experience and not something that may feel abstract, which sometimes happens within a discipline when it feels like maybe it’s not a thing I’m going to do anytime soon, professionally. So I think this really highlights the reason why we need to help students hook into everything that we’re doing to make it feel like they have a personal, professional, or educational connection to their own goals.

John: One of the topics that I use in each class where I’ve done this, at a point where students face the first really challenging material in the class, is just asking them to discuss how they deal with challenges. They share useful strategies, but one of the main benefits of that is it normalizes the sense of struggle, that when students are struggling with concepts, they often feel that they’re alone on this, but when they hear that other people are struggling with exactly the same issues and exactly the same concepts, it normalizes it, and again, it helps them understand that challenge is an important part of learning, which is not the message that they’ve generally received throughout their prior educational experience before coming to college.

Rebecca: It seems to me like this is the same reason why our reading groups work so well for faculty development as well is this connection among peers, but also that the challenges we experience are not in isolation. [LAUGHTER]

John: One of the things I’ve always enjoyed about the reading group is when people will come up with a technique or describe something they’ve done in class and people in completely different disciplines who might never have considered that will make connections and see how they could do something similar in their classes. That understanding that we’re all facing the same challenges makes it much easier to deal with some of the day-to-day stresses that we might have in our classrooms.

Rebecca: I know that one of the things that has come up in our reading groups, and also in our conversations about the future of higher ed and where we’re going to be going over the next few years is student engagement, and then specifically, the role of AI tools like ChatGPT [LAUGHTER] in the conversation. So if students don’t feel motivated, and they’re relying heavily on these tools, how do we get students to re-engage with the idea of learning?

John: Well, going back even just a little before the introduction of ChatGPT, which kind of hit higher ed by storm in late November of 2022, we did see a dramatic increase in the use of sites such as chegg.com, and various other sites out there, where the use of those tools became normalized a little bit, which made it much more challenging to give online assessments. And I think that’s where most people are concerned right now about things like ChatGPT, because with other places, you could at least locate where answers were coming from. And you could address that with the students and attribute it to the specific sites where they got their answers, which was, again, a bit of a challenge. But ChatGPT is raising some challenges for assessment that are going to be difficult to deal with, because it’s much more difficult to determine who is the author of specific items submitted online for assessment. And a lot of people are struggling with that right now. I know I’ve been struggling with it. In my spring 2023 online class, the quality of student writing on essays improved fairly dramatically over the course of the semester. And that seemed to correlate with the spread of the use of ChatGPT a bit. AI tools are really powerful, and they can be really useful. And they have a lot of potential value in education and in providing support for personal and work productivity. Right now, I think, more people are focused on the challenges, but we’re going to have to start thinking about ways in which we can productively integrate this and prepare students for a world in which the availability of AI tools will be ubiquitous.

Rebecca: And you teach in some really challenging contexts, really large classes in person, a number of online sections, and I know ChatGPT is keeping you up at night. What are some things that you’re thinking about… maybe haven’t resolved… but that you’re really thinking about redesigning or rethinking or retooling in the fall to just respond to the moment that we are currently living in.

John: As of 2020, I had shifted all the quizzing to online quizzes and tests and midterm exams and so forth. I’m seriously thinking about in my large face-to-face class, moving back to at least in in-class midterm and an in-class final exam. I really appreciated the fact that I could let students do it at their own pace, and that it took some of the anxiety and stress away when students did not have this two-hour time limit to complete an exam in the classroom. But with the size of the class, a large proportion of the testing is done with multiple choice exams, or algorithmic questions, and those are types of things that ChatGPT answers really, really well. Not too long ago, someone posted that ChatGPT 4 received a score of a 99th percentile on the Test of Understanding in College Economics (the microeconomics version of that), and those are the same types of questions that I’d be giving students on these quizzes. And while I had 1000s of questions that I had created that students were selecting from, all of those questions now are vulnerable to the use of AI tools, which makes it much more difficult to assess in that large class. Right now, the only thing I’ve really thought about doing differently in my large class is moving back to at least a couple of in-class exams. Now some of the things I was doing, such as polling questions embedded in the class activities and working on problems in class, where students submit that in real time, are generally much less subject to that type of issue. I know there are tools where students can scan the questions and so forth, they get responses back a bit more quickly, but it wouldn’t be as easy for them to do in real time when they’re in a polling environment. One of the main benefits of that is when I use polling, it was always tied with peer-to-peer discussions. And those peer-to-peer discussions is where most of the learning actually occurred from those in-class problem-solving exercises. For my online class, I’m not sure what I’m going to do. One thing I have done in the past is I’ve had students do podcast projects. And again, it’s pretty easy for chat GPT to generate scripts, but these projects are pretty heavily scaffolded. Students submit a proposal and they go through a number of steps to get there. And projects that are scaffolded like that, are probably a little bit less sensitive to the use of AI tools to generate the entire project. What are you thinking about in terms of your classes, or in terms of the graduate program?

Rebecca: I think we’ve talked a lot about the concern over the validity of our assessments and wanting there to be accuracy, not just for our sakes, but for students’ stakes in the value of their degrees moving forward. Part of it, I think, is really engaging in conversations around ethics around these tools, and not necessarily discouraging the use of the tools entirely, or banning the tools. I think that just motivates people to want to use them more, but rather to use them in ways that are productive, or interesting, but are also well documented… [LAUGHTER] like students are disclosing what they’re doing. And we can analyze the use of the tools in particular ways because maybe it could save time in particular places and not take away from certain kinds of learning, as long as we keep the learning objectives [LAUGHTER] up front. And then we assess when we’re using particular tools to determine whether or not it’s taking away from the learning. But I think these are hard conversations to have, and certainly not things that I want to be policing.

John: And I’d much rather not be policing these things. Sometimes students haven’t given me much choice in that. One example that I’ve seen recently is students submitting exam responses that asked him to analyze recent data, where the response said something to the effect: “as an AI tool, I do not have access to this data.” And when a student submits work like that, it’s pretty clear that they haven’t even read the essay responses they’re submitting on that graded assessment activity. And we want to make sure that students do actually interact and engage with their learning materials. Perhaps we can also design assessments that are not as vulnerable to AI-generated text. This semester, with my online classes, one thing I have shifted to, instead of having them discuss general debates or issues in economics, I have them focus on interpreting videos online, for example, where economists are debating certain topics, or doing readings that are not in the training database for ChatGPT, which means it’s much harder for AI tools to generate responses when they don’t have access to the underlying content that’s the focus of the assessment activity.

Rebecca: Would hyperlocal situations or examples also be a strategy because there’d be less widely available information on something like that.

John: Definitely. Information on the local community or the campus community or other local things, information that would just not be part of the training database is a good place in which we can ask students to connect the materials their learning to real-world events so that you maintain that sense of relevance while ensuring that the students are actively engaging with the work themselves rather than using a tool.

Rebecca: One of the things that I’ve used historically in my design classes, and it’s a little easier again, because I teach studio classes and see students more often so I tend to have a hunch as to what they’re working on, because I’m seeing them working on things, is really documenting process and not just using language, but showing through a video and showing steps along the way that might not be as easy to capture as an end product using an AI tool.

John: In general, open pedagogy projects, too, could be less vulnerable to having work being done entirely by AI tools. So videos would be an example, wikis perhaps might be.

Rebecca: I think that things that combine text and image are more challenging to have an AI tool create, at this moment.

John: That may very well change…

Rebecca: …by the time this episode comes out. [LAUGHTER] I think one of the things that I’m hearing us say actually, is that a lot of the strategies to reduce intellectual integrity or academic integrity issues around ChatGPT are also the things that are more likely to engage students and foster their learning anyways because they’re more authentic assessments, they’re probably more project based, they’re probably more long term with milestones along the way. And these are things that students often deeply engage in. And I think when they can connect to their local community, whether that’s the campus or the community that campus is situated in, or even their own hometown, in different ways around the discipline, those are all ways that students get a hunger to want to learn more.

John: And going back to our earlier discussion of the importance of shifting students’ focus from grades to learning, students are using tools like ChatGPT to raise their grades, even though they recognize it does not support their learning. If we can shift students’ focus to recognize the value of learning as improving skills that they’re going to need later in life, that should reduce the incentive for students to use shortcuts to avoid learning material.

Rebecca: If we’re not just looking for the right answer, but the journey to an answer, and even if it’s an incorrect answer, being able to understand why it’s not correct, and allowing that to be the learning is a really different way than our education has historically worked. The future of higher ed seems really stressful, John. [LAUGHTER]

John: It does, but it always has. That’s nothing new. But certainly the last few years have seen a lot of rapid change that… I hate to use the word unprecedented… but that have been relatively unprecedented.

Rebecca: And I think it really does speak to this need to connect with other colleagues, where we can share some of the challenges that we’re facing and brainstorm together to improve our teaching, but also to improve the level of stress we’re experiencing. [LAUGHTER]

John: One of the other things that we’ve talked about, especially within the last year or so is growing faculty concerns over student engagement. When students first came back to the classroom, there was a lot of excitement about being back. But since then, faculty generally seem to be noting that the level of engagement of students has shifted or has changed somewhat. More students are not completing assignments. Students in some classes have been disappearing from class as the semester progresses. And there’s a lot of concern that students are not as fully engaged with their coursework as they had been prior to the pandemic. So Rebecca, how are we going to solve this?

Rebecca: That’s a good question, John. I think one of the things that this aligns with is the higher incidences of loneliness, and mental health. And so finding ways to connect students to each other, and establishing those peer networks, I think, is one of the most important things that we can do in our classes. And it’s something that I’ve maybe always done in some way. But I’m being much more intentional about moving forward, because I’m feeling like even if students are in the same room, they’re still feeling really isolated. And so we have to be intentional about creating those opportunities for students to experience connection and feeling like they want to show up for each other and for themselves,

John: …using more group activities in class where the work of each student depends on the contribution of the other members does help create that sort of pressure on students to be there for their peers, to be there for the rest of their team. And that could be very useful.

Rebecca: Yeah. And I think the key to that, though, is not just assuming that students know how to interact with one another, or how to depend on one another in a team context, but really scaffolding those learning opportunities that really start with making connections and establishing relationships, because it’s the relationship that’s going to cause the pressure to show up for someone. And I think when we’re seeing high incidence of like ghosting, for example, it’s because the individuals don’t feel connected to the people that they’re ghosting.

John: And there have been a lot of studies done recently that show the importance of a sense of belonging in student persistence. So helping students form those connections is really important, because we have so many students who go to college, build up a huge volume of debt, and then disappear without getting the degree which does serve as a signal that they’ve actually accomplished something as a result of their education. And they end up with more financial struggles than they would have had had they not started. So we do want to help students form those connections for their own sakes, for their own future success. And one of the books we used in a past reading group was Relationship-Rich Education by Leo Lambert and Peter Felton. And that summarizes a lot of the research on the importance of building community and building connections, and also provides some really nice examples of ways in which institutions can transform to help facilitate those connections.

Rebecca: As instructors, we have a lot of power in that space to help students feel a sense of belonging. We can do really simple things to make someone feel seen and if they feel seen, they’re more likely to feel like they belong. So personalized messages, getting to know your students a little bit, being approachable, calling students by name, all of those things help students feel like they’re a part of a particular community. There’s so much to still learn and to come together around. And so I know that we’re looking forward to having many more guests and many more conversations to help work through many of the questions and concerns and things that we’ve raised today and have been raised by our colleagues. Now, John, we always wrap up by asking what’s next?

John: Well, what’s next for me is I’m heading down to North Carolina to teach at Duke again next week. And I’m looking forward to this. And this time, I’m going to try avoiding getting run over by a car. So I can actually teach my classes down there and spend some time away from the hospital this summer,

Rebecca: #life_goals. [LAUGHTER]

John: Small goals are sometimes more achievable. And Rebecca, what’s next for you?

Rebecca: This summer, I’m looking forward to doing some more work on our graduate student online orientation, which we put together as we transition to our new course management system in the fall and also working with some colleagues on an accessibility online module.

John: And we’re looking forward to talking to more of our wonderful guests. I’ve really enjoyed the experience of interacting with so many great people doing some really good research and doing such good work in higher ed.

Rebecca: We’re grateful for all of our guests and all of our listeners. So thanks for listening

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

299. My Professor Cares

Students from marginalized groups often question whether or not they should be in our classes and disciplines. In this episode, Michal Kurlaender joins us to discuss an easy to implement intervention that faculty can use to improve retention and student success. Michal is a Chancellor’s Leadership Professor in the School of Education at UC Davis and is a co-Director of the California Education Lab. She is a co-author with Scott Carrell of a National Bureau of Economic Research working paper entitled “My Professor Cares: Experimental Evidence on the Role of Faculty Engagement.” (This article is forthcoming in the American Economic Association journal, Economic Policy.)

Show Notes

Transcript

John: Students from marginalized groups often question whether or not they should be in our classes and disciplines. In this episode, we discuss an easy to implement intervention that faculty can use to improve retention and student success.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Michal Kurlaender. Michal is a Chancellor’s Leadership Professor in the School of Education at UC Davis and is a co-Director of the California Education Lab. She is a co-author with Scott Carrell of a National Bureau of Economic Research working paper entitled “My Professor Cares: Experimental Evidence on the Role of Faculty Engagement.” This article is forthcoming in the American Economic Association Journal, Economic Policy. Welcome, Michal.

Michal: Thank you for having me.

John: Our teas today are:… Michal, are you drinking any tea?

Michal: I am not drinking tea, but I did have some not too long ago today.

Rebecca: Do you have a favorite?

Michal: I’ve come back to Earl Grey. I used to be an Earl Grey person. I left it for a while, and it’s just made a comeback for me.

Rebecca: Nice. It’s a classic. I have Christmas tea today, despite the fact that it’s June.

John: And I’m drinking a ginger peach black tea today.

Rebecca: I have my Christmas tea because we had our presidential announcement today. And it was like celebration tea.

John: White smoke has come out of the towers [LAUGHTER] and we have a new college president here.

Michal: Congratulations.

Rebecca: So we invited you here today to discuss the study of the impact of specific faculty behaviors on historically underrepresented minority student success. How did you decide on this specific intervention?

Michal: My colleague and collaborator Scott Carrell and I do a lot of work to try to understand College Access and Success. And in particular, we’re interested in understanding inequalities in graduation rates at more open access institutions, like the California State University system, which has, across its system, some more selective campuses and some more open access institutions. But in particular, what we’ve noticed for years is that the graduation rates of students of color, particularly male students of color, black and LatinX men, were really much lower than other groups. And this was a puzzle to us, largely because the eligibility to get into four-year colleges, including the CSUs is quite substantial. These are primarily B-plus students who have finished a comprehensive set of courses required to be eligible for the CSU. And so to see their graduation rates lag so much behind other students was really troubling to us. And so that’s why we decided to focus particularly on the CSU system. And we focused on one campus in particular, that’s a less selective CSU campus.

John: What was the intervention that you used?

Michal: We didn’t go in knowing what intervention to use, we actually started with a focus group with particularly men of color at this campus and asked them what their challenges were. In particular, what we learned was that their challenges were not necessarily social or more broadly campus level, they were primarily in the classroom, and that is they felt disconnected from their instructors and from what to do to be successful. These were all students who reported feeling quite successful in their high school, feeling quite connected to their high school instructors who encouraged them to go on to college. Many of them were in competitive fields like STEM and engineering. And then they felt like they really struggled in college, and in particular, how to seek help and how to understand what instructors wanted from them. And so we came in quite agnostic, I would say, about what could work, what is helpful here? Is it more writing centers, more coaching, more nudging? We didn’t know. And what we came out with is feeling quite sure that we needed to tackle the classroom. And in particular, I think what we wanted to think about was this untapped source of potential support or hindrance in that is faculty, and think historically, we know that many times we just think of the classroom as kind of untouchable, and we put other support centers, writing centers, and tutoring centers, and other supports for students. And we kind of leave the classroom alone and leave faculty, including ourselves… we’re both faculty… to do what they will. And instead, here we wanted to really think about, could we intervene with faculty to provide more support for students?

Rebecca: Yeah, it’s funny how we often overlook that particular option given that’s a key touchpoint with our students, right?

Michal: Exactly. So we came out deciding that we’re going to do an intervention that was classroom based, and that was going to try to work with faculty to give students more information about what it takes to be successful in the classroom, how to seek help. And then we decided to pilot that to see if the proof of concept worked. And we piloted it at a large, more selective institution on a small-scale pilot, and found some promising results and then launched it at full scale. This article describes that whole kind of research process as well, which we think is also an important contribution to the literature… to not just immediately do something, but to actually think about the way in which it might function and just really to understand from students what they tell us they need.

John: Your initial pilot used a light-touch intervention, could you describe that intervention?

Michal: So initially, in the pilot, what we did was a slightly underpowered pilot in the sense that we took students who didn’t complete their first homework assignment in a classroom where you have to complete a set number of homework assignments. And you could miss one, but historically, we knew that students who missed the first one often struggled in the class. And so that’s the point of randomization that we took for the pilot. That is we took those who didn’t submit their first homework assignment and to half of them we sent an email saying, “Hey, we noticed you did and submit your homework assignment. Just to remind you to do well on this course, here are some things that might be helpful.” We also provided some information on what’s coming up and reminding of the office hours and how to seek help. And then we did two others. Importantly, the two other emails provided information that showed the students that we knew how they were doing in the class so far. So after the midterm, and before the final, and we found again, it was underpowered, but we found positive effects among the treatment group. And then that was conditional on some other ex post characteristics that we added to the pilot, but then we decided to launch it at the CSU campus that we worked with at full scale across a random sample of introductory courses.

John: For those listeners who are not familiar with statistics, when you mentioned that this is an underpowered test, could you just explain that in terms that…

Rebecca: …Rebecca can understand? [LAUGHTER]

Michal: Absolutely. So they were underpowered statistically, to detect a statistically significant difference between the treated students, those who got the emails, and those who didn’t. And so for that, we need a large enough sample size of treated versus control students, particularly if we’re going to add other kinds of observations about them, like their gender, or their race, or their prior academic achievement. And so when we say something’s underpowered, we might see the positive effect that is better achievement in terms of the final or in satisfactory progress in the course. But that difference may not be large enough in statistical terms to consider it statistically significant, even if the mean differences are actually in the direction that you expect. So to get that, to be able to actually detect significant effects, you need a large enough sample size. And that’s when we launched into the population that we were specifically focused on, which was the population at this less selective campus.

Rebecca: At that less selective campus when you scaled things up, did you keep the intervention the same? Or did it change? What did that scale up look like?

Michal: Great question. So the first thing we did was we really focused on introductory courses. This was also piloted in an introductory course. But we wanted to focus on particularly large classes, especially because the information was going to come from the instructor and we were doing a randomized control trial, that is some students are going to get this treatment and others are not. So the class had to be large enough for it to not be weird that some students were getting it and others were not. If you’re in a classroom of 30, that might be strange if you’re talking to someone next to you, and they get this email from an instructor, but basically what we did is we recruited faculty, we randomly chose 30 large undergraduate courses. And then we recruited those faculty and said, “Will you be part of an experiment with us? …and here’s what you need to do.” And the important thing here is that we’re not trying to dramatically change faculty style in the classroom, we all have our own style, the own way we write a syllabus and what we expect from students. What we wanted, we had several key principles. The key principle first was that faculty need to directly communicate with the students showing them that they know who they are. So it very much said in an email, Dear Rebecca, or Dear John. They needed to provide information that was specific to their class, it couldn’t be quite generic. We provided them some templates, but the goal was for them to personalize them and say, “Here’s this upcoming unit, here’s what to look out for, here’s how I would study for it, Here are my office hours, and so it provided information. So the way we changed it is… it was a semester-long courses. and so we requested that faculty sent a minimum of three emails to students, one after the first assignment or exam, sometimes earlier if they didn’t have an exam, and two later, we wanted it within the first three weeks of the semester, one after the midterm, and one before the final. And the important thing is, in the second two emails, those were further personalized to sort of say, again, “Dear Rebecca, I see that you’ve gotten a 72 on your midterm, it’s not too late to improve your grade in this class. Here are the things that you could do. And so it was personalized also just showing that we know how you fared in this class. And so again, the goal was to let faculty, in their own words and in their own course formats, personalize these emails, with the principal being information to students, personalization to students, and help seeking behavior advice.

John: And this process is a personalization, was this done in a mail-merge type format? Because I would think to scale this intervention would be a lot easier if you did do it either using the tools within the LMS or using some type of a mail merge.

Michal: Great question. So again, this was a grant-funded study and where we could provide some support to faculty, some faculty didn’t need additional RA support from us and either knew how to do a mail merge, it really worked with their course management system, like Canvas or Blackboard and found it very efficient to work on the own. Others you may or may not be surprised, did request our help from our graduate students. And we did provide support including one actually helping a faculty member directly write individual emails for students to support. You’ll probably ask me how the faculty feel about this. And I will say we actually asked them how long it took. It didn’t take more than a minute an email and so we do kind of try to guesstimate the investment on the faculty’s time to do this, and it very much varies on their comfort level with the course management system.

Rebecca: In the scaled up version of this study, did you continue only interacting with students who had struggled and missed their first assignment or is that a shift from the pilot to the big study?

Michal: Yeah, thank you for catching that. No, it is a shift. We did open it up. We believed actually, theoretically, our priors were that anyone could benefit from this. So if you were a B or on the cusp, we have lots of researchers suggest students, especially in introductory courses, some students, particularly first gen students might take a B or a C as a signal that they shouldn’t be in a particular major. We really did want to encourage across the achievement distribution for everyone. As to John’s earlier question, as you scale this up, or as people have talked to me since this experiment said, what if I want to do this, but I teach 400 people, you could one year, one term, try it with your lower achievers, another term, try it with those at that C range, or others. And so we did in this initial intervention want to do it across the board.

John: How large was the sample on the scaled up version?

Michal: It was 20 faculty, some were in multiple classes, we had 22 classrooms overall, and roughly 3000 students.

John: Excellent. How large was the estimated effect in the scaled up version?

Michal: First, it’s important to note, as we’re talking about findings, that our findings are really concentrated on first- and second-year students or new students, and who are from underrepresented minority backgrounds, so URM students, and we find that their treatments are about five percentage points more likely to earn an A or a B in the course by comparison to control students. So again, just important to note, we find overall positive effects for the whole of treated students. But they’re only statistically significant for the URM students that we target, that our intervention aimed to focus on.

Rebecca: Was the impact limited to just the classes the students were in, or was there an effect beyond that individual class?

Michal: Yeah, so that’s a great question, and we do find what are called spillover effects. And that is that those same students, those URM students, had a positive effect of being in the treatment group, even in courses that were not part of the experiment. That positive effect was much smaller in magnitude, it was like three quarters the size of the effect of the actual treated course. But still statistically significant at the ,10 level.

Rebecca: it seems so easy, just three emails.

Michal: Yeah, it does. It takes an investment. But yeah, I think it does beg the question, I think, for me, and maybe this is something that you want to talk about a little bit later. But we do so many things to introduce first-gen students or to get students in the classroom and again to provide support externally, but we do tend to sort of assume that they’ll just survive or just be okay in the classroom without training faculty on how people might experience their classroom differently. And so, again, we do test for other subgroups. I focus on first-gen, because it’s a concept that is helpful for people to think about students who don’t have at home, at least, people to tell them what to expect in the college environment and how you might go to someone’s faculty office hours, and they’re not there, or they’re there, but they’re sort of like, “Yes, did you have a question about the material?” Whereas to know, you could go just to review the material or you don’t have to have a question, you can just show up, things like that. Being able to feel comfortable asking questions in class and others who don’t or just step in at the end of class and sort of say, I found this part of the lecture really complicated, will you be reviewing it again the next day? So things of that nature. And so I don’t think these emails did those, but they sort of remind us that there are things that faculty can do to remind students that they see them, that they see that they’re in a classroom, and that they know that they may be enjoying parts or struggling in parts, and that there are some actions that they can take to be more successful in their classroom.

John: Did the effect size vary with class size? Was there a more substantive effect in larger classes than smaller classes? Because I would think it might be easier for students to feel more lost in a larger section, especially as a first-gen student?

Michal: It’s a great question. We aren’t able to test that, our sizes were all pretty similar, and we didn’t have enough. We actually chose the largest classes that exist in this campus, which don’t get much bigger than 150. I think it is worth testing. Absolutely. I can tell you from the pilot, that, in particular, that was a class of 400 students, and both for the pilot and for the full scale up the types of emails we got, that faculty got back. So that’s one thing we could talk about is how did students respond more generally, and many students emailed instructors back, and in particular, in the pilot, but also in the scale up, we got those emails back from the faculty who were in our experiment, and they very much appreciated the email and said, “No faculty members ever emailed me before,” or “especially in a class of this size,” or “I so appreciate the email. I’m working really hard.” So the first and sort of overwhelming finding from these emails is just gratitude from students that a faculty member emailed them, particularly we noted that in the pilot when the class was quite so large.

Rebecca: Yeah, I can imagine that just something that feels personalized, whether or not it’s super personalized or not, just feels personalized, really helps students feel seen.

Michal: Exactly that. That’s right. I should also say in addition to just grades and you asked me about graduation outcomes, we also included a survey, both in our pilot and in the scale up, to try to get at some of the mechanisms and in particular, we asked questions like, “Did you feel this instructor supports your learning? Did you feel you could reach out to your instructor?” And we do find consistent with our intervention that students in the treatment group reported more positive outcomes on these dimensions.

Rebecca: That’s so fascinating, because it’s so easy. Like it just isn’t that complicated.

John: In your study, you also examined how this effect persists over time, which certainly relates to the graduation thing. What did you find in terms of the persistence of this effect over time,

Michal: We did look at long-run treatment effects. That is, we waited to see what happened several years later, we presented on this paper in the shorter term outcomes. And we tracked these students and worked with this institution closely, we really wanted to know did it affect the outcome we care most about, which is graduation? In other words, we care about student success in a particular classroom, and maybe they’re slightly better grade, or not dropping out of the course. But we really care about their longer term outcome of finishing. Again, for this specific group of interest. We note that the treatment results in that 7.3 percentage point increase in persistence, one semester later, and then a higher four percentage points difference in graduation. So we do find positive effects on the likelihood of graduation.

Rebecca: We’ve talked a bit about the impact on students and how students have responded, how did faculty respond to participating in this intervention?

Michal: First, I’ll just say, again, we had to recruit faculty to do this. And so we do track the faculty who said no, and we did as much reconnaissance work, if you will, to understand that we did need to self select faculty, keeping in mind that if we did self select faculty who just had a proclivity to help students or being this intervention, if anything, we perhaps under reported some of our findings, but we do as much effort to understand how representative our faculty are, which we did determine they are, and in the paper, there’s some details about that. And they represent faculty from a real diversity of disciplines, from music to engineering, across the board, humanities and social sciences, we had the whole range, we had the range of faculty. So you know, “I do this a little bit sometimes. And the two of you are, I would have said, I’d do the same with a student who doesn’t show up or doesn’t complete to sort of track them down. But I’ve never done it systematically. And I’ve always wondered if it even matters.” We had everything from that kind of faculty member to a faculty member that’s like, “Well, I’ll do this, but I don’t think it matters. I mean, at the end of the day, the students who want to put in the work put in the work.” And so we had a whole range in their attitudes. We did offer a stipend to do this, because we did believe it takes time. And we wanted to sort of show that faculty who do have a lot of demands, especially at teaching institutions, that this was going to take some time. And so we haven’t done it again without an incentive, or with an institutional incentive, that’s part of like performance evaluations or something. So that’s yet another thing that in terms of where to take this in terms of where institutions might take this, for our perspective, it was externally funded, they were only accountable to us in their efforts to do this. And so we talked to them multiple times in the term, again, some were in both waves of the study, because we did it over two terms. And then we surveyed them at the end and really got some details from them. Some of this is in the paper around how they felt. And I would say most expressed similar to what you expressed, Rebecca, which is like “Wow, this some emails and I made this difference, especially in underrepresented minority students lives and in their classroom, and it felt really good about the impact.” …keeping in mind, we talked to them also, before we knew the results. And so at that point, they just were sort of documenting how much time it took to do the emails, and what kind of emails they got in response from students and most felt, I would say, humbled by the thank yous that something so small, like an email, got so much gratitude back from students. We did have some faculty that sort of said, this takes a lot of time, and I’m not sure it’s much of a help. In our last survey with faculty, we actually provided them the full scale results and said, “Here’s, by the way, what we learned from the study,” and then asked them to respond or to reflect on that. And many said, “Wow,” like, again, similar to your reaction, “a few emails could make such a big difference, I will be sure to continue.” We ask them directly if they will and we report this in the paper. Most do say that now that they know the positive impacts of this study, they’re likely to continue with these emails.

Rebecca: I imagine the workload for a faculty member isn’t necessarily in drafting those initial emails, but maybe the responses to the emails [LAUGHTER] you might receive back.

Michal: It’s a good question. I don’t know how many continued the conversation once an email was sent, the standard emails, part of the experiment and the student wrote back and said, “Thank you so much.” I’m not sure they continued. We did a lot of qualitative coding, which we don’t report in the paper, we report some but then we did a lot more internally. And there definitely were a lot of hardships described among students who did reply, the extent to which faculty replied with those hardships offering extensions or any other kind of augmentation to their requirements, we don’t know.

John: In an introductory microeconomics course I use the Lumen Learning Waymaker package which actually does automate emails to students based on their performance on weekly quizzes and so forth. And even though the students in the class realized it was automated, they’d still write back to me and I’d respond to it. And they’d often apologize, saying, “I had a bad week and I know I need to work harder.” But it did often start a dialogue that might not have occurred otherwise. And I’ve often wondered how large that effect was, but because it’s automated for the whole class, it’s hard to measure the differential effect of that. So it’s nice to see this result, that that type of approach works.

Michal: Yeah, I think that what you’re describing is exactly right, this sort of feedback. Our whole intervention is built around theories, not just from kind of behavioral economics or nudging or information source, but on the education literature on feedback, and the important role of feedback, and the timing of it. And it’s most useful if it’s not just performative like feedback, like your grade on an assessment like a “C,” but that actually gives you more information about how you’re doing or what to do to improve. And so this kind of thing you’re describing John is exactly right. I think we know that more touch points with students through assessments, as opposed to all hanging on a midterm and final also support students to get more feedback about how they’re doing.

Rebecca:I think sometimes students know that they’re struggling, you get a grade back, you know, if you’re doing well or not, but I think a lot of students need more coaching around what to do to improve or to better understand how the grade is calculated, to just take the time and attention. It’s there. It’s in the syllabus. But sometimes they don’t realize what they should prioritize. And including some of that kind of messaging makes a lot of sense. I know that when I’ve done that with my students, they’ve been really appreciative because they didn’t realize that they were putting all their energy into something that didn’t really matter as much.

Michal: Yeah, that’s right. And I think coaching is a great choice of words around what to do with it. I also think many students come to our universities with really uneven or unequal preparation for those courses. And so I think a lot about students who came from a high school where they took an AP course in economics or chemistry that might as well have been a college level course. Many of these questions are great on a grade curve. And so that C might be an excellent grade for them, given the type of preparation that they had, but they don’t know that necessarily, and they might, to them, signal that maybe this isn’t the right major for them. And so I also think coaching around what to do with the grade when you’re kind of passionate about a subject and not to give up on yourself too quickly. Many are juggling jobs, we know for some, it’s their only work is to get through this term, and others are doing this while working and taking care of family members or whatever. And so that grade that they got often conveys information that we as faculty don’t necessarily know anything about how they’re interpreting,

John: I would think just a signal, as in the title of your paper, “my professor cares” might create a sense of connection and belonging that might otherwise be missing for someone who is a first-gen student who might not feel that they belong in the institution.

Michal: Yes, I agree. And I think there are more and more studies coming out, particularly in social psychology, but elsewhere about the importance of belonging. We know from the K-12 literature that it’s having a teacher who cares about you matters, actually. And again, nothing dramatic happens once you get to college. But we assume it’s completely different, where in fact, I think having an adult or particularly your instructor care, you feel a sense that that instructor cares about your learning, or how you’re doing in their class, irrespective of the grade per se, just that you’re making progress in the class or feeling comfortable in the class, I think is really important. And I think it’s hard to test. And most of the belonging literature has been on survey type research, “I feel like I belong here.” And it’s not as much in the classroom, although there’s increasingly more studies about belonging in the college classroom beyond just a university at large.

John: A while back, we interviewed Peter Arcidiacono, on a paper that looked at the impact of differential grading between STEM and non-STEM courses. One of the things you just said reminded me a little bit of that, because one of the things that was noted in that paper is that many of the people, particularly female and underrepresented minority students who switch their majors out of the STEM fields had some of the higher grades in the class, but it was below their expectations. And I’m wondering if this type of intervention might have an effect of letting them know that that performance in that discipline may not be all that bad. Since we’re probably not going to eliminate grading differentials between STEM and non-STEM disciplines, perhaps some type of personal communication might help preserve some of those connections so we don’t lose as many people in the STEM fields where the returns to education are the highest.

Michal: Absolutely, I think well said. That’s exactly right. And I think that is among the reasons we wanted to get across the grade distribution, not just those who are really, really struggling. And also because we do think that students might give too much meaning from a signal of a grade early on in their academic pursuits where they can get through a certain amount of courses and then maybe where the fun stuff of their major where they really see that utility of a particular course for a future career path matters. And so I think that’s right and I do think Peter’s work and other people’s work looking at the impacts of particular kind of grades and grading distributions or signals of grades, I think, are really important. I think that’s an area that’s blossoming in economics and in other fields to sort of better understand heterogeneity or differences between subgroups around how particular information like an assessment, grade or a test score.

John: One of the things we’ve been seeing in a lot of studies is that many changes in education, such as using more active learning techniques, providing more course structure, benefits all students but disproportionately benefits those students who are historically underrepresented. And it seems like this study just provides more evidence of that, that good teaching practices benefits everyone, but especially benefits the people who are most at risk in higher ed. And those are often the people who can get the greatest benefit by persisting in higher ed.

Michal: Yes, I think that’s exactly right, if we’re really going to address disparities in college outcomes, and I think one really important source to go to is the importance of information gaps. And that would be particularly for first-generation students, for students of color, but also for students who come from unequal K-12 backgrounds. Colleges and universities often know and often are recipients of systematically particular high schools in their states, especially public flagships, community colleges, others, and so they are aware, they offer relationships with those K-12 high schools that are feeders to their institutions. And that is an important source of information that they can provide to high school students as they enter college for a kind of a warmer handoff, if you will, but that also faculty teaching introductory courses can provide. And so I think, again, if our goal is to address inequalities that we see in college outcomes, then I think information, particularly for those for whom their information gaps, is particularly key. Students want to be seen and not necessarily yes, there’s the anonymity of a large lecture hall, that maybe don’t want to be called on. But that doesn’t mean you don’t want to know that your faculty member sees you and knows whether you’re doing well or struggling, or how you feel about the class or how to succeed on the next exam or in the next course in that sequence.

Rebecca: So if there’s other faculty who think, “Hmm, three emails, that seems easy,” What recommendations would you make to those faculty?

Michal: Yeah, I mean, I think what the first recommendation I would make is to try it, to do it. I think thinking about how you communicate in your syllabus that about your forms of communication are important. So if you’re going to do an email, I think one thing that we would have loved to test and if we were to continue further is the format for the information. And so I think letting also students know that you want to hear from them over email or through other means, I think is useful. So first, deciding on the kind of medium like how you’re going to communicate this, I think email makes sense. When faculty start texting students, maybe we’d move to a text them information. But that’s not the case for most of us, so it’s through the course management system or email. I would say focus on again, what we know from the literature on feedback is that for it to be as specific as possible about what students can do with this information, and so that is looking at your syllabus closely, knowing… we typically do as faculty… where students trip up in the material, what’s complex, what’s up ahead. And so giving that kind of feedback as well about how to prepare for the next assignment or exam, what has tripped people up in the past, what you know, might work for them is really helpful. Again, other research has suggested the importance of going to office hours might matter. But that means you need it to show up, you have to think about how you structure your office hours. Incidentally, we did try to track that… quite hard to do whether it actually promoted more office hours in the pilot, we believe it did promote more office hour usage. More broadly, it’s something we’d like to test, the actual help-seeking behavior of students. So I would say faculty should do it if you’re teaching a 400-person class and you can’t imagine doing this for 200 students or even 100 students, maybe start with students who you see as struggling based on that first assignment, as we did in the pilot and see what you can learn from that… maybe do as John suggests, which is kind of get savvy on a mail merge and think about ways to do this so it’s more efficiently done, so that you can reach as many students as possible.

John: We just switched recently to Desire to Learn’s Brightspace platform, and it does have intelligent agents and it does have replace strings so you can automate an email conditioned on the grade on either your overall course grade or on a specific item. And if you do it on overall course grade, (which I just set up, by the way, for my summer class last week), students get an email saying, “I see that you’re struggling, there are some things you might want to try.” It would be nice if I could put in their grade without having to go to mail merge, but I don’t think that would be possible. And I’d like to scale this up. In any case, I’d encourage them to contact me during my office hours or to make an appointment to talk to me. The first iteration of that went out last week. None of the students responded, but it’s a small summer class. So I’m curious to see how this might work. And your paper helped encourage me to try this. I had other reminders out there, but this was one that I thought might be useful, using a specific grade trigger.

Michal: That’s great.

John: We always end with the question. What’s next?

Michal: Well, I think our lab and Scott Carrell and I continue to do this work, and in particular, we’re also spending a lot of our time these days doing work at community colleges in California, which serves one in four community college students nationwide. So also persistence outcomes are quite weak at community colleges, historically, and we’ve seen real declines in enrollment at community colleges since the pandemic. And so we are definitely doing some work at community colleges. We continue to track and follow graduation rates, particularly inequality in graduation rates at CSU. And we’d love to launch another intervention. So stay tuned on that. I can tell you, we are quite committed to understanding the college classroom beyond college settings more generally, and so hope that the college classroom continues to be a source of important information for the field about how to better support student success,

John: You’re doing some really wonderful research. And it’s really nice to see some of the attention that this got because your article has been mentioned in The Chronicle. It’s been mentioned in Inside Higher Ed, and I’ve seen people tweeting about it ever since it came out. It’s good to see this research becoming popularized.

Michal: Well, we appreciate it, especially since Scott and I did not succeed on that front, that is becoming those people who do good social media. Other people are better at that than I am. And I’m always a little troubled when I talk to more junior faculty around “Do you need to do all that?” …and 10 years ago or so I would say “No, just do good work and it doesn’t matter.” And now I confess the sort of buzz that some people are better able to develop around their findings in their papers seems to matter and so it’s really nice when it happens to you because we didn’t do it ourselves. [LAUGHTER] So I appreciate my friends Sue Dynarski and others who’ve done a really nice job promoting this paper.

Rebecca: Well, thank you so much for joining us and for sharing your work with our audience.

Michal: My pleasure. Thanks for having me.

John: And we will include a link to your study in the show notes and we encourage our audience to read it.

Michal: Wonderful.

John: Thank you.

Michal: Thank you so much. I really appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

233. Guided Notetaking

Many college classes contain a substantial lecture component, but our students arrive at college with little or no training in taking effective notes. In this episode, Tanya Martini joins us to discuss how guided note taking can be used to promote equity and student success. Tanya is a Professor of Psychology at Brock University in Ontario.

Show Notes

Transcript

John: Many college classes contain a substantial lecture component, but our students arrive at college with little or no training in taking effective notes. In this episode, we examine how guided note taking can be used to promote equity and student success.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Tanya Martini. Tanya is a Professor of Psychology at Brock University in Ontario. Welcome, Tanya.

Tanya: Thanks very much for having me.

John: Our teas today are… Tanya, are you drinking tea?

Tanya: I am drinking tea, yes. So, my tea is a pomegranate white.

Rebecca: Oh that sounds nice. Nice and light.

Tanya: Yeah. My friend was joking with me because I said I was doing this podcast about tea and teaching, and he said, “So, like, basically two of your favorite things in the world.” I said, “Pretty much!”

Rebecca: You’re in excellent company then. [LAUGHTER]

Tanya: Are you also drinking tea?

Rebecca: Always, always.

Tanya: Always.

Rebecca: Today I have English afternoon. I was in a rush, so I couldn’t get a fancy pot going.

John: And I’m on my fourth cup of tea for the day, and it’s a peppermint spearmint blend.

Tanya: Oh, that’s nice, that’s really nice. My mum was English, so we always drank tea when I was growing up. And then I met my husband when we were hiking in Wales. So we basically have a household where the kettle is never cold.

Rebecca: Yeah, my house is like that, too.

Tanya: Is it?

Rebecca: When John’s like, “Fourth cup?” I’m like… “Fourth pot? What are you talking about?” [LAUGHTER]

John: We have had three kettles today, because we’ve had a number of people come into the office earlier today.

Rebecca: We’ve invited you here today to discuss how you’ve been using guided note taking in your introductory psychology course. Can you give us a little background on the course first?

Tanya: Sure, it’s at Brock. So there’s 1500 students in the intro psych class, and about 220 or 230 of them would be psych majors in their first year. And the rest of them are taking it either as a requirement for another degree program like education or nursing, or they take it as a social sciences elective. So I would say about 85 or 90% of them are genuinely in their first year, they’ve just arrived from high school. We don’t have a lot of non-traditional students in the class, a few, but the majority of them are 17, 18 year olds, and they’re just making their transition from high school to university.

John: What’s the modality for this class? And how is it taught in terms of dealing with that many students in one class?

Tanya: Yeah, it’s a good question, because different people will handle that sort of volume of students in different ways. We made a really intentional choice even before I came on board that we wanted all psychology students to have the same experience. So rather than trying to split the students up into different groups taught by different instructors, what the instructors do is they split the weeks. So our course runs 12 and 12, like 12 weeks in the fall and then again in the winter. So it’s a 24 week, full-year course for us from September to April. And in the past I would say decade or so it’s three instructors, but it’s usually kind of two primary instructors and what we call a junior partner. And so the two, kind of, main instructors would teach 10 weeks each and then the junior partner would teach four. And it’s been kind of a good model. I started as the junior partner and it gives people a bit of a chance to decide… Is this a course that I could see myself moving into? Or is this really not something for me? Because it’s not for everybody to be in a course that’s that large. So in terms of structuring it, that’s how we do it. And it gives everybody a very consistent kind of experience. And then, like most Canadian universities the last couple of years, the big, big courses have been asynchronous online. So that’s what we did last year, what we’re doing currently this year. In normal times, it’s run as a three hour per week course, where two hours is allocated to a lecture. We don’t have enough seats in any space for all of those people. So what we do is we run the same lecture three times a week to 500 people, and we would do a two-hour lecture. And then we run 75 small group seminars every week, so 18 to 20 people usually. And those seminars are run by third and fourth year undergraduates. So we have, like, a peer mentorship model in the seminars. And it works actually pretty well, because what we find is that the first-year students, our impression, anyway, from the evaluations, is they like having other students leading those discussions. It makes them feel comfortable, and not just with the content, but I think they feel comfortable if they’re struggling with the transition. They feel as though this is kind of a senior mentor who can help them to find the resources that they need. But I really like the fact that our department and our faculty kind of puts the resources behind it because it’s quite resource intensive to run that many seminars in a course. It’s nice because it helps students to make the integration into university and help them cross that bridge from high school. Because they’ve got this one person who knows who they are, knows them by name, knows that if they go missing somebody will be checking in on them to make sure that they’re okay. And it also just helps them to meet people and make new friends in the first year. And then a few years ago, we got permission to have the teaching assistants, those third and fourth year students also take a course in facilitating good discussions. So that they’re not walking into the seminars feeling ill prepared. And it’s a great space for us to teach them some transferable skills that are very applicable to the workplace. So it’s kind of a nice model. Very resource intensive, and I’m grateful for that, but it works really well.

Rebecca: Can you talk a little bit more about the seminar for your TAs?

Tanya: Yeah, sure. It’s a full-year course. So it runs basically in parallel to the first-year course. And what we do is, I really try to model it as… it’s not a kind of “how to TA intro psych course,” I really try to sell it and emphasize its applicability to the world more broadly. So we tackle a number of different topics that are relevant to those seminars, but are also relevant to the working world, too. So we talk about diversity in groups. And so if you’re facilitating conversations in groups that are diverse, what are some of the advantages to that? And what are some of the challenges? And how do you overcome those kinds of challenges? How do you manage the balance of the discussion so that you don’t have decisions being made based on the people who talk the most or who talk the loudest? How do you get over those kinds of challenges? We talk about active listening and what that means, and we talk about cultivating a level of self awareness so that if you start to feel kind of anxious or upset, you recognize that in yourself, and you know what to do to bring it down. We talk about… How do you facilitate things when it starts to get challenging and heated? What kinds of things can you do? And then in the second semester, we move into talking about things, like, How do you give a good presentation? So if we move away from the facilitation skills, we talk about good multimedia presentations, the students do the equivalent of a 15 minute TED Talk. So it’s all meant to be very applicable to a broad array of settings. And most students find that it’s a great community, because we hire 18 students every year to fulfill the seminar leader role. And we tend to find that they coalesce into a really nice, tight community, they’re very supportive of one another. So you get a great exchange of ideas and lots of support, if you’re feeling anxious, or things aren’t going well, or whatever. So that course runs separately, but it’s a really nice counterpoint to the very, very large first year course. So I love having the big, big class on the one hand, but then also this smaller class where I really get to know those 18 TAs really well.

Rebecca: In our notes for this episode related to this course, one of the things that you talk about is how diversity doesn’t necessarily promote inclusion, or the other way around. Can you expand on that a little bit more? It totally caught my attention. I was, like, I want to know more about that!

Tanya: Yeah, it’s such a great point. So I have the luxury of being on sabbatical this year for the first time in eight years, and it’s such a great gift. And one of the things that I really wanted to do was, to do more reading about diversity and inclusion and thinking a little bit about… How do we incorporate that into the syllabus? And what does it mean for our seminars? Are we doing the best job that we can be doing? And I have been taking, like, a number of workshops, and I went to this great workshop being run by an organization that promotes diverse viewpoints. And the woman who was running it is from an organization in the States called The Village Square, where they purposefully bring people together who are on both sides of the political divide, I think, in that case, left and right. And I had a conversation with her, and she’s, like, taking this really great course, this… it’s a MOOC basically, Massive Open Online Course, and she said, “I think you would really like it. It’s all about bridging differences.” And so that comment about you can have diversity without any inclusion comes from this course that I’m taking. And they just talk about how you can have diversity, like, you can purposefully hire or take on people who look different from one another, or different religious backgrounds, political backgrounds. But unless everybody feels comfortable articulating their opinions and talking about where they’re at, unless they all feel comfortable with one another, you don’t really get true inclusion. Because inclusion sort of necessitates that not just you have a roomful of diverse people, but that all of those people are contributing to the conversation. And I thought that that was a really valuable and important point to take away. Because although I think, of course, you have to make some effort to bring those diverse people into the room, the job doesn’t stop there. Because at that point then, you have the sometimes challenging job of getting all those people to a place where they all feel as though their opinion matters, and they’re comfortable articulating it. And they go on to talk a little bit about how in some cultures, for example, there’s much greater value placed on listening rather than talking all the time. And so you have to work with that and work around that and try to draw people in so that the discussion is somewhat balanced. And at that point you’ve kind of done a better job of reaching inclusivity, as opposed to just diversity.

John: This is a really great set of skills you’re providing the students, who no matter what they do in the future are going to be working with diverse groups, as well as having to do presentations in some form or another. You mentioned hiring these students, what type of compensation do they receive? Are they paid for this work? Or are they receiving credit for it as a result of the courses they take?

Tanya: Yeah, so that’s another great question. And it’s a model that has changed over time. So at Brock, and at a number of schools in Ontario, teaching assistants are unionized. And for a long time this was a unionized position, they would get paid for something like 300 hours a year. And then, like this won’t surprise you at all, we went through financially difficult times as budgets were shrinking. And there was a period in the last decade where enrollments were shrinking just because of the demographics of Ontario. There was a much smaller pool of 17-year-olds in high school, and we were all competing for those people. And I was really only the junior partner at that time. But at that time, the Dean really started to make noises about the amount of money that it cost to mount this course. And so what we decided to do was to implement this strategy where we would make it a hybrid. And what we did was we rolled some of the things that used to be paid tasks into a course. So on the one hand, it would be disingenuous of me to say this was all driven by pedagogy, because it was driven to a large extent by budget. So what we did was we created this course, and about one third of the hours, we rolled into course related stuff. So a lot of the seminar preparation material, where we would pay them for that before it kind of became part of the course and talking through the seminar preparation with your colleagues and me giving some scaffolding around that. So now they get paid for about 200 hours. And the rest of it is rolled into this full-year course from September to April. So they get course credit for that. What it does mean, though, just from a logistical point of view is we used to have a model where people could do the course a number of times, they could TA the course a number of times. In fact, they were more likely to because they would get seniority in the union. But now you can only TA when you’re taking this course, and you can only take it once. So we don’t have the continuity we used to have, but we get a fresh group of 18 people. And I try to look at it as the upside of that is we can deliver these skills to more people because it’s a unique cohort every single year.

Rebecca: Are most of the courses in your college the two semesters back-to-back, like full-year courses?

Tanya: No. In fact, it’s kind of a rarity. Almost all of them are a half-year course, so 12 weeks, from either September to December or January to April. Intro psych is one and then we have a research methods course that’s a full year. But, in general, the guiding principle has been one of breadth. But I think what we have always felt is that a full year for first year is a good idea. Not so much because it’s such an important year for us and there’s so much to deliver. I mean, there is, but I think just in terms of integrating students into the department and to the university, and again, helping them to make that transition, giving them the full year course is really helpful in terms of allowing time and space for that to happen.

John: We saw an article in The Chronicle recently describing your use of guided note taking and it seemed really effective. Could you describe how guided note taking has been used in your classes and how your use of it has evolved?

Tanya: Basically in terms of historical context, what we find, and this won’t surprise anybody, in a class of 1500 people, you get everything from A to Zed. So we have students who are amazingly well prepared for university, and they know how to take notes, and they know how to read dense text, and they’re very skilled and savvy about taking exams, and they have great study strategies. And then on the other end, we have students who are quite ill prepared. And sometimes not even ill prepared, but maybe they’ve been out of school for a long time, and so those skills are kind of rusty. And so retention is an issue for us. We’re always trying to make sure that we’re a course where we’re bringing everybody up to the same place. By the end of the 24 weeks we’re trying to get people ready so that everybody is kind of on a level playing field. And when I joined the course, laptops were just starting to be a big thing. And what we were doing at that time, and this predates my involvement in the course, is we were supplying notes to students. But they were the PowerPoint notes. So you print the PowerPoint notes, you’ve got the slide, and they’ve got a bunch of lines on the side, and students would make these notes. And we would supply these paper notes for a fee, and they would buy them at the bookstore bundled up. And then they could just put them in a binder, and bring them to the course. And so some of the students, as I was coming into the course, were just kind of ignoring them and they were taking notes on a laptop, other people were taking notes using these paper copies. So it was kind of quite a mix at that time. But one of the things that really started to become clear to me was students really weren’t sure what they should be writing down. And my slides are somewhat sparse. Yeah, I’ll have the main points on there. And students would come up during the break, and they’d say, “Well, I kind of missed this point.” And sometimes I would be looking and it was just full of dense text, like, they were trying to write down verbatim everything I said. And then some of them would come and they hadn’t written down anything because they had the picture of the slide there, and they assumed that if it wasn’t on the slide it probably wasn’t worth talking about or remembering. So it really struck me how there was so much diversity in what people interpreted to be good note taking. And I was talking to my colleague about that, and I said, “Sure,” that this is really a good thing. And so that was my first sense that maybe we needed to do something and talk more explicitly about note taking. And could we do something that would scaffold their note taking? And so that was where the first iteration of our guided notes came from. And basically, the first iteration was just, like, a hierarchical overview of, here’s what the lecture looks like from the top, here’s the three main points or the four main points, and here are the subpoints. Just like on a PowerPoint slide, you get the main points and the subpoints. So that they could kind of see at a glance, these are the big things. And then what the guided notes looked like was almost like a series of small exam questions, because that’s really what the lecture is, it’s the answers to a number of different questions. And so if I was talking about, say, four things that determine whether you will pay attention to something in the environment, the question in the guided notes would be: “Name and describe four things that determine whether you pay attention to something in the environment.” And might even give them a table to fill in. And so they started to see the lecture as providing answers to a number of questions, some of which are connected, and giving them clearer space in which to put it in. And then we had students in that first iteration commenting on the fact that they didn’t want to take notes on a computer, or they didn’t have a computer. And so we started creating a digital copy and a PDF copy with spaces, and they could either write with their pen or they could type on a computer. And that’s what we did for four or five years, that’s kind of what it looked like. And then one of the things that I wanted to do during this sabbatical was just think a little bit more about them, because I started to think it would be a good idea to have something similar for the text. So I co-authored the textbook that we use, and I started to think, you know, it’d be really good to have something comparable to the text. Because in as much as they struggle to take notes during lectures, sometimes I think they struggle just knowing what to extract from the textbook. There’s so much material there, and sometimes they come in with all the highlighting. Okay but, [LAUGHTER] like, it’s the whole page.

Rebecca: It’s a rainbow page. [LAUGHTER]

Tanya: I know, the rainbow page! And so it was sort of clear that sometimes we’re struggling to extract what the central stuff was. And rarely did I see people making marginal notes or anything like that. And so I started to think that would be helpful. So that was one of my sabbatical tasks. So I went into the note taking literature a little more deeply, and I started to create them, and they’re a little more in keeping with the Cornell Notes. So they take the form of the Cornell Notes, and they are a little more sophisticated. And that was what prompted the rework of the lecture notes that they commented on in The Chronicle. So they look more like Cornell Notes now with the features of Cornell Notes. And the other thing, it’s such a great blessing that a sabbatical is, I was reading about diversity and inclusion, and went back into some texts that I hadn’t read for a while, like Dan Willingham’s book, Why Don’t Students Like School? And he talks about the fact that, one of the things that you could do and might be advantageous to do is to sort of frame the lecture around a series of questions and make them explicit. And so it really pushed me… this whole reformulation of the lecture notes has really pressed me to think about my lectures a lot more carefully. And so what I started to do is think… Okay, well, if I was going to distill this section on attention into, like, six or eight main questions, what are they? Because Willingham’s contention is, instead of it being like this steady stream of information just being thrown at you, if you can kind of organize it into a series of questions that are somewhat interesting, that makes things, first of all, organized a little bit better in your head, but also just a little more interesting to listen to. So the guided notes now kind of look like Cornell Notes. And they’ve got this accompanying, what I call “the roadmap,” which is basically just a graphic organizer. I did a lot of reading about graphic organizers. It sort of shows, these are the main points, these are the main questions. And then as we’re answering this question, we’re tackling this study and this study. And you’re not meant to write on the roadmap, it just gives you a visual of… here’s where we are in the grand scheme of things, here’s where we are in this particular unit.

Rebecca: For those that aren’t familiar, can you talk a little bit about what Cornell Notes are?

Tanya: Sure, yeah, I would love to talk about Cornell Notes. And there are obviously slight variations, but the way Cornell Notes are set up is that if you’re thinking about a page, there’s two main columns, usually. And in the left-hand column, in my notes, I list the main questions or the main points for my textbook notes. It’s like, here are the learning objectives for this chapter. And then the students’ written notes are on the right-hand side. And I scaffold those by putting in the sort of smaller questions that were always in the guided notes. Like, name and describe the four things that guide whether you’ll pay attention to something in the environment, is there. But what I do is also to include, and sometimes it’s structured differently, but I’ll section on, like, What are your questions about this? So that students have a space to write down if something’s going by, and they know that they haven’t understood it, it’s a space where they can very clearly plant a flag and say: “I need to go and check on this,” or “I need to ask the instructor, or I need to go to office hours, I need to talk to my TA.” So there’s a very clear space there for… What questions do you have? And I always frame it that way, too, because I think it’s a subtle difference. But when you say, “What questions do you have?” You’re basically saying, “It’s totally normal to have questions. So what are they?” So our Cornell Notes look like that. They’ve got a section for here are the main questions over here on the left-hand side, and as we’re answering that question, here are the things we’re going to cover. And you can make your notes on that right-hand column. And then before we move on to another question… What questions do you have about this? What do you need to check on?

John: You mentioned at the start of this, the wide diversity in students’ note taking skills, and so the students who are going to do well would probably do well, anyway. But I would think that this would benefit some of the students who would come in with weaker skills, and they are now able to focus on the things that are much more relevant and important for learning the course content. As well as it’s providing them with some clues about how it all fits together with the questions and with the structure you provide. Has this reduced some of the equity gaps in your classes?

Tanya: Yeah, I think that’s a great question. And the answer is, I don’t know, because I don’t have data. But one of the things that I found really interesting, and I was saying I did a lot of reading about note taking and so on, and I hadn’t really thought about guided notes as being an equity issue. But Viji Sathy and Kelly Hogan, who are both in North Carolina, have written extensively about how providing structure of this kind is incredibly beneficial in terms of students who have come from usually some disadvantage and have less preparation, or their preparation for university hasn’t been as strong. And so I don’t have any data, but certainly Kelly Hogan in her intro biology class, and I really admired the fact that she actually dug around in the data. And what she found was that, quite unbeknownst to her, the number of Ds and Fs was not spread equally across different groups. And so she found that people who were Caucasian ancestry, like they were getting a relatively small proportion of Ds and Fs. It was slightly higher among her Latino and Latina students. And then it was really quite high among the African American students. And I found that quite surprising too, but certainly she’s got data to support the idea that providing this kind of structure is very beneficial in terms of diminishing those kinds of gaps. But what I will say, and I’m not sure if this would resonate with other people, one of the things that has always concerned me a little is when you provide this kind of structure in a first year course, and then you send them off into the world in second year, with me knowing that none of my colleagues are providing these kinds of support… Are you in fact just pushing off the retention issue and all the problems into second year? And so one of the things that I think is really important is not just creating these guided notes and the scaffolding, but really talking very explicitly about: If you find this helpful, let’s talk about why it’s helpful. And let’s talk about how you can transfer that skill into another course where they’re not necessarily going to give you guided notes for your textbook and for your lecture. So what are these things doing for you? How are they scaffolding? And I think having that explicit conversation is really important. Otherwise, I feel like we run the risk of just pulling the rug from under them, but doing it a little bit later on. So one of the things we talk about is, How do you decide what’s really important? So if it’s in a textbook, being cued into what’s in bold type, and what’s in italics, and so on. But I also try to kind of just get back to the structure issue. Because it’s interesting how you teach for a long time, and it seems so obvious, but it’s not obvious to them. So it’s like, if I give you this chapter, why don’t you just try mapping out the headers and the subheaders? So that you get some sense of how this material is organized. And you’ll see in chapter eight, there’s like four big topics, that’s the big text, the main A headers, and then there might be two subheaders under this one. And then if you kind of create that map for yourself, it gives you the big overall picture. And one of the benefits of teaching intro psych is we talk a lot about memory. And how is memory structured? And what do you do to facilitate memory? And we talk about how memory is organized in this kind of way, and hierarchies are important and useful. And so I’m really trying hard not just to provide the scaffolding, but to help them to understand why the scaffolding works. So that in second year, if somebody has dismantled it and the scaffolding isn’t there, at least you have some sense of, “Okay, how can I recreate that effect for myself?” And I think that that’s something that is still kind of a work in progress. I don’t know if I do that well enough yet, or enough yet. But I think that that’s a really important component of providing the support in the first place.

John: And that’s something I think that most students have not been exposed to along the way. So helping them develop those types of skills would be really helpful, I think. And we should probably mention that Viji Sathy and Kelly Hogan have a book coming out at some point in the near future, and I’m very much looking forward to reading that.

Tanya: Me too.

Rebecca: I’m sure many of our listeners will be. We have it on our radar, like the second it comes out, we’re going to know. [LAUGHTER] Have your students responded to this scaffolding that you’re providing through note taking? Have they given you some informal feedback that you could share?

Tanya: Yeah, so, so we’ve had informal feedback verbally, and sometimes it’s solicited and sometimes it’s not. So if students come to my office hours, and they bring their notes, I don’t hesitate to ask them, “How’s it working for you? Does it matter to you?” Occasionally students will tell me they don’t use the notes, but they will refer to the notes before a test. And sometimes they tell me that it feels like it’s the only thing that is keeping them going through the lectures, which sometimes does feel like it’s quite a lot to take on board. But on the course evaluations, we see a number of people… because sometimes I asked about it explicitly on the course evals, and what we get is uniformly positive feedback. Sometimes it’s quite effusive, and sometimes it’s not, but we don’t get people saying, “This is terrible, don’t do this again.” Even if they’re not using them, they genuinely appreciate the fact that we think about the fact that this is sometimes a skill that’s challenging, and that we’re trying to support them as best we can. So yeah, the informal feedback and the formal feedback has been uniformly positive. The question of the data is an important one, and I don’t have any data to supply. But I kind of lean on the fact that Kelly Hogan’s work, to me, is pretty compelling if you can move students in that way. That’s a big deal to me, because, as I said, we have a lot of traditional students. They’re 17 and 18-years-old, but they come from such different places, and the extent to which high school has prepared them well varies a lot. And I’m not sure what your experience has been like, but I think the pandemic has just amplified that a lot. The kinds of experiences they’ve had, the extent to which they’ve been in the class versus not in the class, the extent to which teachers have had the resources to rise to the occasion, or not, has really created significant inequities, I think, in high schools in Ontario. I don’t doubt that everybody’s doing the best job they can, but they don’t always have the same resources to work with. So I think we’ll only see more of that in the future as they move through high school and arrive on our doorstep.

Rebecca: I think tools like the note taking that you’re talking about, are just really great ways to help students filter out the mass amount of noise in an already very noisy world. And it seems particularly noisy these days. [LAUGHTER]

Tanya: Yes.

Rebecca: There’s so much going on, that helping provide that structure to really narrow focus seems really helpful. I know that my students have been certainly reporting that there’s just so many distractions, and things that are occupying their minds, that the ability to have structure like this in place can be really helpful to getting through a class, or being successful in a class in ways that maybe if that structure wasn’t there wouldn’t happen at this moment in time. Maybe it would happen in a different moment in time, but not at this moment in time.

Tanya: No, I totally agree with you. I think that that’s true. And I also think that, I’m not sure about your experience, but my experience is, even in the normal times, many of them are not accustomed to sort of sitting through it, our lecture. I do try to break it up quite a bit, actually, so that we’re trying different class activities. Or I’ll do something where, in a class of 500, I can divide them into groups, and we try to replicate an experiment. And like, “You’re going to be the group that has these instructions, and you close your eyes now, and you guys will have these instructions, and then…” So we’re always trying to mix it up a little bit. But sitting for two hours and trying to take stuff on board, it’s a lot. And I’ve been doing different things on this sabbatical, some of which have involved me sitting for hours, and struggling by the end of it.

Rebecca: I love that you’re taking classes on sabbatical, it was something that I did when I was on sabbatical as well. It’s a nice reminder of what it’s like to be a beginner in things again.

Tanya: It’s totally true. Especially, I think, the last couple of years. Doesn’t it feel like there hasn’t been two minutes to spare to do anything that elevates your teaching? I was trying to really struggle just to get through, and you know we were pivoting this really big course online, and suddenly all the assignments are different, and no in-person exams. It just felt like… it was so intense. And so this has been just such a great gap to kind of go, “I have time to do some reading,” and “I have time to take this course that I think will be beneficial to me, and it might be beneficial to my students.” So yeah, it’s been so amazing [LAUGHTER] just to be able to remember what it’s like to go learn some new things and get excited about that. I also started taking karate lessons, right? Which I thought, this is the best example of why you have to really have a lot of patience with beginners. Because I’m now at that stage where I’m a complete beginner myself, and hopelessly uncoordinated, and there’s all these black belts who are incredibly patient with me. But it’s sort of like, “Oh yeah,” everybody who teaches first year students should actually go do something brand new every now and again and remember what it was like to feel like you don’t know anything, and you’re not really very good at it. So that’s been a good reminder, too.

Rebecca: I love it.

John: When you were talking about the course for the teaching assistants, you emphasized how the skills they’re learning are going to be useful in the rest of their lives. Since most of the students in your psychology class are not going to become psych majors, and I think many of us have experienced challenges in getting students engaged in content in gen-ed classes. What type of strategies do you use to help emphasize the relevance of the skills they’re acquiring or the knowledge they’re acquiring in your class?

Tanya: Yeah. So that’s been kind of the focus of our research for a number of years now, and it was born of just some of those lightbulb moments. I’m sure you guys have had them too, where you have these experiences, like, “Ohhhh!” So I’m just going to describe one of them to you. So it started to become clear to me… now I’m going back probably eight or nine years, where I was having a good discussion with a student who had worked as a TA for me, she had worked as a research assistant for me. She was one of these young women who just had it all going on, like really smart, and really thoughtful, and a good thinker. And we were having this conversation as she was getting ready to apply to graduate school, and she was putting her application stuff together, her letter of intent. And we got to talking and, just to paraphrase, she was basically communicating that she really didn’t know, she spent four years and tens of thousands of dollars, and she really didn’t know how to articulate or leverage what she had been learning. So even though I had zero doubt that she had all these skills, she didn’t know how to talk about them, and she didn’t know how to talk to somebody else and persuade them that she had skills. And I suddenly started to think, that’s a huge problem. [LAUGHTER] That’s a really, really big problem. And that led to some of my early research, where I started to look at that question among our site undergrads at Brock, and it wasn’t an anomaly at all. And what I found was that students, when you ask them, “What do you think you’re getting out of these assignments that they’re giving you? Most of them would say, “Well, the instructor wants me to learn about topic X. They want me to learn about the content.” And very rarely did they ever say, “They want me to improve my communication skills,” or, “They want me to improve my teamwork skills,” or whatever. And so for me, that was really telling. And so I started just trying to be a lot more intentional about being explicit with students about the skills. And so in PSYC 1F90, what the main project that they do is, they would gather some data as a group, usually on some really famous study that works pretty well consistently, and they would gather data from themselves as participants. And I would get the data from the TAs, and I would combine it, and I would analyze, it’s very simple analysis. And then they would take the findings, and they would write it up like a mini psychological paper with an introduction, method, results, and discussion. And I would talk a little bit about that, and I would talk about the fact that it builds communication skills and writing skills. And so I think that I’m doing well, and then I had my second lightbulb moment when I had a history major come in. And we were talking about the assignment, and she was very polite, like really polite, but kind of candid. And she basically said she thought the assignment was pointless. And I said, “I was talking in lecture, it builds communication skills.” And she said, and again I’m paraphrasing, “Well, maybe that’s the way psychologists communicate, but that’s not the way historians communicate.” And I thought, that’s really interesting. [LAUGHTER] Because suddenly it became clear that she was kind of focused on the superficial aspects, like, it’s got our introduction, method, results, and discussion, and that’s not what historians do. And I suddenly realized it’s not actually enough to say, “Well it builds communication skills.” You actually have to talk about how there’s some transferability of communication from one discipline to another, right? So, absolutely true, historians probably write papers that look quite different. But let’s talk about the fact that all communication—written, verbal, whatever—you have to think, first of all that, What’s the main thing you want to talk about? So what are the main points you want to make? And what are the subpoints? And how are you going to order those points? What’s the logical flow of information that you want to present? And then you need to think about your audience and think about their characteristics. Do you need to persuade them? Or are you going to be preaching to the choir? What do they know in advance? What background do you need? All these things, it doesn’t matter, I said in an essay that I wrote for the Times Higher Education, it doesn’t matter whether you’re like a cop giving testimony to a jury, or whether you’re writing a psychology paper, or whether you’re given a TED talk, or whether you’re writing a history paper, or whether you’re writing an auditing report for Price Waterhouse. Those things matter cutting across different disciplines. And I never realized that it was really important to actually get to that level of detail in explaining the skills to get buy-in, but that’s what I do now. We’re doing a different project going forward, and I’ve got a whole section of the instruction sheet. It’s like, How can you leverage the skills from this assignment in a job interview? And I don’t know if they’ll read it, but I’ll be talking about it in class anyway. And so this is where I’ve arrived in terms of students and skills. And I know some people will probably say, “Wow, you spend a lot of time in class talking about skills,” but I actually think it’s really an important element of what we’re doing. And I think, left to their own devices, it’s not always obvious to students that this is going on, that you’re acquiring this, as well as some knowledge of “what are the factors that influence whether you pay attention to something in the environment.”

John: Faculty often complain about the silos that we created within our discipline, and we forget that students create their own silos where they think of each thing as something entirely separate from the rest of their academic career. And if we can make it clear to students that the material they’re learning is going to be useful for them, they’re going to be much more engaged and much more interested and enjoy it a lot more, I would think.

Tanya: Yeah, I think it’s really important in terms of buy-in, I really do. And you’re absolutely right about the silos, and I think I had been quite naive to that. I think I really didn’t realize it until this wonderful young woman took the time to articulate it to me, and that was incredibly helpful. So you’re right.

Rebecca: While you’re talking about this, Tanya, I’m also thinking about how often I experience students focusing in on the thing that seems newest to them, the thing that’s most unfamiliar. And that’s what they get hung up on, and they think the entire class is that one thing that they don’t understand. And then the things that seem familiar—like writing seems familiar, we’ve written before—that’s the thing they pay the least attention to, and it’s often some of the things that we want to pay the most attention to. And so it’s funny how the newness, or our fear of not knowing how to do something, can put our attention there, and forget about the details of other things that also have a learning value to them that aren’t always recognized.

Tanya: Yeah, and I think you’re right about that. I have to constantly remind myself that they might not see that. It’s very obvious, but they might not see that. And to make it explicit, and to take time to talk about it, and answer questions about it. And that was something that I didn’t do nearly enough of in the early days of my teaching. Like, I really just thought, I don’t know, I thought if it was obvious to me it should be obvious to you. [LAUGHTER] But students over the years in articulating some of these things have really crystallized for me how important it is to be explicit. And for sure, some of them absolutely do get it, they don’t need me to explain it. But I come back to the variability in a class of 1500, and even really, I mean, I don’t even think that’s totally wrong for upper-year courses. Sometimes students don’t necessarily always see things that we think that they will. It helps me to think to myself… Okay, what’s the most important thing here, in terms of content but also in terms of skills? And then let’s make sure that I talk about that, and talk about how it moves from this particular course and this particular assignment. Like, can you see how you might use it out there in the world? And I usually try to talk in terms of multiple examples, like my students are going into law enforcement, they’re going into business, they’re going into nursing. So I try to draw from a few different disciplines, just to give this sense of… Yep, it transfers over here, and it transfers over there, and it transfers over here, just so they get in the habit of thinking about that.

John: We always end with the question, What’s next?

Tanya: What’s next for me is, I’m still working on the diversity and inclusion piece, and so that’s taking up lots of the space between my ears. Trying to think more about… How, in a class that is so diverse, how do you create the kind of inclusivity that we’re looking for? Recognizing that students don’t participate for lots of different reasons, and we want to kind of take them gently into the water in some cases. So I was mentioning that we’re moving away from that old assignment where they would write kind of a mini psychological paper. And then what I’m trying to get them to do is to work on a persuasive message, and they’re going to present it in a couple of different ways, so they choose a topic they agree or disagree with. And then what I’m working on is… It’s got an element where they have to find some sources to support them, and there’s an information literacy component. So I’m doing a lot of research on that, and how do you promote good information literacy at a time when information is just boundless, and some of it’s good and some of it’s not good? And then they have to present it in a screencast. So we talk to them about good multimedia, and they do a very short screencast with their preliminary ideas. And then they have to write an op-ed. And so one of the things I’ve been dialoguing with my co-instructors about is, just lots of students are incredibly nervous about putting a screencast together. I try to talk a little bit about how this is kind of a gentle introduction to presentations. So we’re not going to ask you to give a flawless presentation in front of 20 people or more, and if you don’t like your first recording, you can try re-recording. So I acknowledge that there are people who have significant anxiety, for example, and so on that diversity, on a side of that kind of diversity, trying to create an assignment that will gently lead them into improving their sense of confidence about their ability to present. And looking at the seminars, and asking myself, How can I draw on this Massive Open Online Course? Which is all about, How do you facilitate good communication among people who don’t agree? How can I bring that into the TA course and help them to inspire really good, inclusive conversations among the first-year students? So I’m thinking about inclusivity in a really broad kind of way. I’m thinking about it on the side of my assignments, I’m thinking about reworking my syllabus. I’m thinking about… How do we bring it into the seminars? In more than just, “Well I’m going to introduce you to some African Canadian scholars,” or “I’m going to draw in people who are diverse into the course.” It’s more about, how do we get diverse participation? And the sense from students that we’re all part of this community, and we’re all going to contribute to this community that is the intro psych.

Rebecca: Sounds like you’ve got a lot on the horizon for sure.

Tanya: Yeah, it’s an exciting time, it really is. It feels like it’s a time where you just have an opportunity to do some really good work, and I’m going to really try to capitalize on it.

John: Well, thank you. It was really enjoyable talking to you and you’ve given us a lot to think about.

Tanya: Thank you very much. It’s been such a pleasure to speak with you today.

Rebecca: Thanks, Tanya.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle, Annalyn Smith, and Joshua Vega.

[MUSIC]