Category: large-class instruction

373. Cross-Institutional Peer Observation

Posted in community of practice, faculty learning community, higher education, large-class instruction, learning, pedagogy, peer observation, professional development, reflective practice, teaching
December 25, 2024

Peer observation and peer feedback can be useful resources for faculty professional development. In this episode, Anna Logan, Ann Marie Farrell, and Martina Crehan join us to discuss a cross-institutional, cross-disciplinary peer observation process. Anna is an Associate Professor in the School of Inclusive and Special Education and the former Dean of Teaching and Learning at the Institute of Education, Dublin City University. Ann Marie is an Assistant Professor, also in the School of Inclusive and Special Education at Dublin City University. Martina is Head of Teaching Enhancement Unit at Dublin City University. She has over 20 years experience in professional development and as a curriculum innovator.

Show Notes

Transcript

John: Peer observation and peer feedback can be useful resources for faculty professional development. In this episode, we examine a cross-institutional, cross-disciplinary peer observation process.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Anna Logan, Ann Marie Farrell, and Martina Crehan. Anna is an Associate Professor in the School of Inclusive and Special Education and the former Dean of Teaching and Learning at the Institute of Education, Dublin City University. Ann Marie is an Assistant Professor, also in the School of Inclusive and Special Education at Dublin City University. Martina is Head of Teaching Enhancement Unit at Dublin City University. She has over 20 years experience in professional development and as a curriculum innovator. Welcome back, Anna and Ann Marie and welcome Martina.

Ann Marie: Hi. Thanks for having us back.

John: It’s good to see you again. Today’s teas are:… Anna, what tea do you have today?

Anna: Today I am having a very traditional Barry’s Red Label tea. Barry’s is a very famous Irish tea from County Cork. So I’m having that.

John: And Ann Marie?

Ann Marie: I am having an earl grey tea today. That’s actually my favorite tea. So that’s what I’m having this afternoon.

John: And Martina?

Martina: So I’ve gone a little bit more exotic with a rooibos tea. So it makes me think of beautiful warm blue skies. [LAUGHTER]

Rebecca: And I have an English afternoon today,

John: And I have… something I missed out last time… an Irish breakfast tea today, I would have had Barry’s if I was able to get back to the office this morning, but it’s a Twinings, but at least it’s Irish breakfast this time. [LAUGHTER]

Rebecca: at least the right part of the world, John. So we’ve invited you here today to discuss your joint work in setting up a cross-institutional process of peer observation of teaching. Can you tell us a little bit about the origins of this project?

Martina: Thanks, Rebecca. So this project was a collaboration between three Irish universities, Dublin City University, Maynooth University and RCSI, and our story began, hard to believe now, actually, as far back as 2017 and it started with a conversation between myself and two colleagues who worked as faculty developers in three institutions. Have to mention my co-conspirators here, Muireann O’Keeffe and Morag Munro, and we’d all been involved in peer observation of teaching work in one form or another in our respective institutions. We were very aware of how valuable a tool it could be in scaffolding professional dialog and reflection and practice, and we became very interested in the potential for cross-institutional and cross-disciplinary peer observation. A lot of the literature at the time was very focused on peer observation of teaching in a single institution, but we were coming across more and more publications which really focused on the power of cross-disciplinary and cross-institutional. We were really lucky to have some funding at the time that was available via a partnership between the three institutions. So we initiated a pilot scheme. We focused it on large-group teaching as the common focus for the participants, and designed a process where we supported the participants with an induction. We also supported them in terms of the observation process itself, reflection, professional conversations and so on. And we sought volunteers, and were really lucky to be joined by, I think, 10 volunteers across the three institutions from a range of different backgrounds, two of whom were Ann Marie and Anna, and I think they might want to speak about their motivations for joining us.

Ann Marie: Well, from my perspective, I was just very interested immediately by the idea of having an opportunity to work with somebody outside of DCU, outside of my own university. And my background is teaching. I was originally a primary school teacher, and so that’s my background. And since I’ve been working in higher education, I’ve been really interested in the pedagogical aspect of my work. So when the invitation went out, it just seemed like something that was really interesting to me, and it fitted in with some of the work I was already doing. So that’s what motivated me to join up.

Anna: And I love teaching. And of course, this was something that was focused on teaching. I think Martina might have mentioned that it was focused on large-class teaching, which was something that I was really interested in. So that was another motivating factor. When I read a little bit about it, or was approached, the structure and the opportunity to engage in reflection on my teaching, particularly the teaching in a large class, was really useful. I am very committed to improving my teaching all the time, and I really was motivated by the opportunity that I could identify aspects of my teaching that I had a hunch weren’t great, that I was a little bit concerned about my pacing, for example, and so I knew this would give me potentially a good opportunity to get feedback on that and develop my skills in terms of teaching those large classes.

John: When we do peer evaluation of teaching on our campus, it could be for formative purposes, but it’s often more for summative evaluation, which is perhaps not the most useful form of peer observation, and I really like the idea of having people from other institutions observe teaching, because it takes away any possibility of that type of summative evaluation. Have people appreciated that aspect, that they’re getting reactions from people outside their campus who will not be involved, even indirectly, in evaluating their teaching for retention or other purposes.

Ann Marie: From my perspective as a participant in the cross-institutional peer observation of teaching project, it was absolutely fantastic to have somebody coming from a completely different context who didn’t know one single thing about the culture and the ideas and the way we do things, even generally as an institution, never mind specifically, in my classroom. And I was paired up with a guy who was teaching a completely different subject area and he was doing something around digital media and literacy for trainee doctors at RCSI, the Royal College of Surgeons in Ireland, so it’s a medical university in its entirety. So he was coming from a completely different perspective, and even when we visited each other’s institutions, that was really useful. Like one of his comments is, a lot of my students are all student teachers, they’re all resident in Ireland, most of them go home at the weekends, whereas his students were mostly International, maybe went home once a year. So he was very struck. He came in to see me on a Friday morning, and there were about 40 suitcases lined up at the door as the students were coming to my session, and then were running for buses to go home. And he was struck by that, and even just the profile of the class. So what’s really useful about that kind of cross-institutional thing is that you see your own students, your own organization, in a much broader context. And I think that does influence how you view Teaching and Learning then as well.

Anna: And I absolutely agree with Ann Marie’s observations in relation to that. I also think that the cross-disciplinary nature that Ann Marie has highlighted was really very, very important. And coming back to your observation, John, that this really wasn’t about summative assessment, this was about mentoring and peer support and mutual peer support. I was paired with somebody from a business background, so I was visiting and observing in a large undergraduate business classroom. And it struck me at the time that I was authentically a learner in that place. There wasn’t a competition. I wasn’t going in with my lens as a teacher educator with a particular interest in inclusive and special education. So it was, I suppose, in a way, less competitive. And maybe this wasn’t a conscious thing, it wasn’t, “how do they do it in this institution?” Because it was an entirely different discipline. It meant I was an authentic learner about leadership and management education, and hopefully, I think my peer was equally a learner in my classroom about inclusive education. So I think it was both the cross-institutional and the cross-disciplinary which really supported that mutual peer support aspect, rather than somebody coming in in an evaluative summative role. It wasn’t designed as such, and it didn’t slip into that space because we were in different disciplines and in different institutions.

Martina: I think generally as well, just to add to that, John, in Ireland, in higher education, probably for the most part, we’re quite lucky where peer observation of teaching in most institutions has maintained that very formative developmental aspect. It’s actually very rarely used here in that much more summative, evaluative way. So I think we’re very lucky from that perspective. But as I was listening to Anna and Ann Marie speaking there, I’m not sure if somebody used this phrase in some of the research that we’ve conducted on this, but I remember somebody talking about the cross-institutional and the cross-disciplinary piece allowed you to tune into a different frequency of teaching and learning. And I thought that was just a lovely way of summarizing all of that which kind of resonated with me and has stayed with me, I have to say.

Rebecca: One of the things that I heard you both speaking about is being a non-expert in a classroom on the subject matter, which helps you be in the classroom as a learner, and probably helps you give really different feedback on the teaching of your peer, in a different way than if you are an expert on the subject matter, which I find to be really interesting and really helpful. And I know that when I’ve had the opportunity to be in other classrooms of different disciplines, have found to be one of the most exciting and rich experiences as an observer is to be in those really different disciplines: (A) because it’s really interesting to learn something new, [LAUGHTER] but (B) because the methodologies are often really different and give you some really interesting and new ideas. I’m interested to hear a little bit more about some of the logistics of how this process was put in place, because you’re talking a little bit about visiting the different institutions and coordinating three different organizations, and that sounds like a really complex situation to put together. Can you talk a little bit about what that was like for both the organizers as well as the participants.

Martina: So, I think we were very lucky in that, from my perspective, from an organizational point of view, I had a colleague in each of the other institutions who was as enthusiastic and as motivated in this. So from a logistical perspective, we kind of shared the organization, and we shared the locations for some of the induction workshops and some of the conversations that we needed to have with the participants, which Anna and Ann Marie will be able to talk to you about from their perspective and as a participant. We took the responsibility of matching the individuals, so that made it, I think, a little bit easier for the participants, and then it was really up to them to organize: When will I visit? What day? What time? How will that work out? So we left that in the hands of the participants so that they could make their own decisions, and hopefully that made it a little bit easier for them. So I think from the organization point of view, the logistics were possibly maybe a little bit easier than for the participants.

Ann Marie: Well, from a participant’s perspective, I just found it was really well structured. So we had like a planning or a support day with Martina, Muireann, and Morag at the beginning of the process, where they went through models of reflective practice. They went through principles of the type of feedback, or considerations we as participants needed to take into consideration when we were working with our partner, like, for example, establishing ground rules with your partner in terms of what the nature of the feedback, maybe what you would request as feedback that the person being observed might ask for particular feedback and so on. So it was just really well supported from the outset, and then when Martina then sent us off in our pairs, I linked in with my partner, and we had at least one, if not two, phone conversations, and that was to agree when we were going to meet, who was going to go where and when, basic things, like giving directions, all of that sort of thing. And then when we met, we gave each other a tour of each other’s University. And that was really interesting, and it broke the ice. It was a lovely kind of, I suppose, relationship-building thing, like we spent a little time in both institutions before we did the feedback or the observation or whatever. And in our case, the chap I was paired with came and observed two of my classes. He asked could he come and see a second one? And I said, “Yes,” I was delighted. His classes were mainly completed at that point in the semester, but he had recorded them, so I watched one of his recordings, and then we sat down in terms of feedback. And it was interesting, because I was far more nervous about giving feedback than I was about receiving it, because I am a teacher educator. I go out to schools all the time. I’m constantly giving feedback on teaching, and that’s a very particular context in terms of going out to one of our students who’s doing a course. But this was different. So I didn’t want to become the teacher educator. I wanted it to be equal. So I was quite nervous actually about giving the feedback. But the whole process of the actual observations and the feedback was just so collegial. It was eye opening, like he saw things that he could only see because he was sitting at the top of the room, and I do walk around when I’m in a tiered lecture theater. I walk around, but I cannot sit down at the back for 30 minutes or an hour and watch, whereas he did, and he had very interesting things to say about what students were doing while class was going on. So that actual process of the two of us working together, I just found it really, really, really useful. So that was my main takeaway around that.

Anna: My experience was different. You asked about the process, John, so just talking about the process, my experience of the process was slightly different. Yes, there was that initial face-to-face workshop, which was really useful. But there was a flexibility in this. Even though it was structured, there was quite a lot of flexibility. Martina said, after that first day, it was really handed over to us in our pairs to work it out. And I think one of the things that was great about it was the flexibility, because we had to be quite flexible, my partner and I, for a number of reasons. First of all, my partner hadn’t been able to come to that initial first day, so Ann Marie mentioned the kind of the building and the breaking the ice. So actually, we had to do that. I was on Skype at the time, because this was before COVID and I certainly wasn’t comfortable in Skype. I’d never used it before, so there was quite a lot there, but we had a long conversation and a long getting to know you chat on Skype, which did help us to break that barrier. Now again, Ann Marie, you mentioned the logistics of when this would happen in the academic year, when it would happen in the semester. So there were constraints to do with that which are to do with when a module started, when it ended. So I was very fortunate in that my partner was still teaching at that time, so I could get out and go to her university. Again, we built in time to have a chat, as Ann Marie had said, while we had to do it on Skype again, we were very clear about what we wanted feedback on, and I would totally relate to Ann Marie’s observation. I’m really used to giving feedback. I was very much conscious of stepping away from that. And again, it was a different discipline and so on. So I went out to and I observed the lecture live in a large lecture theater, and we had our feedback session after that, but it was a bit different for me, because my teaching in that particular module was in blocks, and it just didn’t suit my partner to come out. But again, as Ann Marie said, we could record. So I recorded the session and shared the recording with my partner, and then we came together again online and had our conversation. So the flexibility was great in terms of the process, and that structure at the beginning, clear guidelines, ground rules, all that sort of thing. Then flexibility was really important. Because if it had had to be you have to see a face-to-face lecture, or it has to happen within a certain period of time, there was that flexibility which made it very achievable too, I think, for us

John: Now, you’ve already addressed some of this in terms of your responses, Anna and Ann Marie, but have you heard any other responses from faculty in terms of the general reaction to this process? Has it been popular? Have people been anxious or concerned about it? How have faculty, more broadly, responded to this.

Martina: I can address this to some extent from the perspective of the pilot project, and then maybe other forms of peer observation within single institutions which I’ve been involved in. And I think for many people, particularly when they start on this process, it’s a mixture of fear trepidation, a sense of the unknown. What am I signing up to? What am I jumping into? Because realistically, you know, for most of us in any teaching context, we go into our lecture theater, our classroom, our lab, and we close the door. It’s us and the students. So that concept of having somebody sit at the back of the classroom or the front of the classroom and observe us is challenging, and I think particularly in an Irish context, it puts us in mind of the cigire, which is our Irish word for inspector, who might come in, particularly if we’re in teacher training as primary level or second level, will come in and assess us in that more evaluative way. So it is certainly daunting for people at the start, so the structures and the supports that we put around it are really, really key in that. And one of the things that participants usually respond to quite well is that sense of support, the fact that it is developmental, that it is not something that is evaluative, that judgments are being made on, and the fact that there is a reciprocity. So if I’m paired with Anna, then she will come into my classroom, I’ll go into her classroom. And the philosophy is really centered around a conversation about our teaching and learning practice. And I think once people have experienced that for the first time and get that sense of the philosophy and the ethos, it really kind of helps that progression, and it helps that sense of comfort and that sense of familiarity. So for the most part, I mean, there are people, I think, who probably feel a little bit more challenged about it for a longer period of time, but I think having those supports in place are really, really key to ensuring that participants engage in this in a productive and a positive way and see it as something that helps their practice and is going to develop their practice.

Ann Marie: And I suppose, from my perspective, since engaging in that pilot project, I’ve invited a lot of people into my class for a variety of reasons, and sometimes not about the teaching and learning piece, but more about the content. So I’ve invited staff in if they so wished, if we’ve guest speakers, or if I’m in doing a session on special and inclusive education, other teacher educators who might be interested. And sometimes we do have some other members of faculty who come in and join for some of those and that’s not quite the same thing, but I would be much more confident in doing that now, and I also think it is really important for very new members of staff. I think for the most part, new lecturers are thrown in teaching. They’re the one group of teachers who perhaps do not have a teaching qualification, and they’re in what are arguably some of the most complex situations and some peer observation of teaching, but maybe initially, one way where new members of staff can come in and be invited in to observe large-class teaching, just to help them get off the ground themselves, or that has, in my case, morphed into co-teaching, where somebody has come in initially a new member of staff has come in to observe, and that’s then kind of the teaching, and that very large class context, in my instance, is supported then in a very collegial, safe way, before they get up and stand in front of 400 students on their own. So it can lead to that. I suppose it’s like having an open-door policy. And it might start off as one way, but if the person ends up co-teaching one or two sessions with you, actually, you can then talk about teaching and learning in that context and give feedback to both of us, [LAUGHTER] rather than the person, in terms of evaluating how the class went or whatever. So it has potential way beyond just two people observing each other.

Rebecca: So Martina, you mentioned that this was a pilot project. Is this something that you’re hoping to continue across the three institutions and expand beyond a pilot?

Martina: I think we would love to. Obviously, there are very fundamental issues like funding, which impact future plans in this regard. So certainly we would love to continue. We’d love to expand. I think, as the three of us here are all working in the one institution now… I was actually working in one of the other institutions at the time of this particular project… what we’re really, I think, focusing on and thinking about is taking the learning and taking some of the elements of that project into how we facilitate peer observation of teaching within one institution. So focusing on the cross-disciplinary aspect, so between schools, between faculties, and really taking some of that learning and embedding it within the existing processes that we have here. And as Ann Marie just mentioned there, specifically also focusing on the role of this for new educators in the way that she described, I think it’s hugely powerful in terms of those first steps into teaching, which can be so daunting and so challenging for somebody new to practice.

John: One of the things I was thinking, and this relates to a conversation we had in an earlier podcast with Anna and Ann Marie, is that by having people from different disciplines, the focus of the observation will be on the pedagogy and on teaching techniques, rather than on the content. Because when we observe people teaching the same courses or in the same discipline, we often tend to think about how we might be doing it, rather than focusing on what is actually being done in the classroom. Is that one of the benefits of this cross disciplinary aspect of the peer observations?

Martina: Yes, absolutely, I completely think that it is, John, for all of the reasons that you’ve stated. I think it really helps people to focus on the process rather than the product of teaching. And I think what’s also useful is it also encourages, and I don’t know Ann Marie and Anne if you might have experienced this, but I feel that it encourages people in the observation setting to also focus on what the students, what the learners, are doing in that teaching space. So not just what the great teaching that Anna and Ann Marie are facilitating, but also: What are the students doing? How are they engaging? When are they being active? When do they appear to be coming a little bit more passive? And that, I’m sure can be really, really helpful then for the individual who is being observed, because it’s that other set of eyes, and it’s a different lens through which you’re seeing what is happening in that teaching space.

Ann Marie: I would agree with that. I think when you observe somebody who is from the same discipline or sub-discipline as you, you can get very caught up with the content and the curriculum, and you lose sight of the teaching and the learning. Whereas if you don’t know anything about the curriculum or about the concepts that are being taught, you have to focus on the teaching. You have to focus on the learning. You have to focus on the actions that have been taken by the teacher and by the students in that context. So it is very helpful, and I think it can be accomplished, though, even within a discipline. So at university level, in particular, the more specialized, narrow experts in some tiny nano aspect we are, the more hireable we are, and the better we are at research and so on. So people within education, for example, who might be in different sub-disciplines, I think, while there would be some overlap in terms of content, in many cases, there isn’t. So even within a discipline, I think you could enable peer observation of teaching, to observe somebody in another sub-discipline, albeit that you’re all coming from the same main discipline, if that was a factor in an institution or whatever, but it definitely forces you to look at the teaching rather than the content of what has been taught.

Anna: And I would absolutely agree with that. And just again, I think Martina’s focus was why it was cross-Institutional, why it was cross-disciplinary. The shared concern was teaching large classes. So that was a shared concern. I would completely support Martina’s observation. In that business classroom. I felt I was authentically a learner, and that, at least to some extent, gave me insights into how the strategies that the lecturer were using were being experienced by my fellow students in the classroom, and also because it was about leadership theory in business education, and that, as both Ann Marie and Martina have said, meant that I wasn’t getting distracted about… I mean, it’s natural for us to think, okay, but I’m teaching that I’m going to use that strategy. It’s great. I’m going to use it. Or if I were doing it, I’d be using a different case study, or I’d be using a different student response system, or whatever it was. Instead, I was really pared back to the things that I recognized about this dynamic of working with a large class, the strategies that the person I was observing was using to seek, maintain attention, to support students’ reflection, for example. So it’s kind of paring it back to the core teaching and learning strategies, and as I say, authentically, putting me in the learner place as much as I could be. The other thing that I thought was interesting, just thinking about it further too, Ann Marie’s observation was, while our disciplines were very different, and indeed, the time that we had spent working and teaching in higher education were very different. We’re very different terms of our experience, my peer spontaneously… I had asked for feedback on certain aspects of my teaching… but she spontaneously said that she liked how I had given feedback and that she was going to use that similar kind of approach. So again, that was affirming, but that was maybe particularly valuable because it was coming from a very, very different discipline,

Rebecca: Building on what we’re talking about, can you share some advice to other folks that might want to set up a similar model of feedback at their own institution?

Martina: I think there’s some key factors which are important, regardless of whether it’s cross-institutional or within one institution. Obviously, if it’s cross-institutional, you need to secure your trusted partners in each institution, and there are funding and logistical implications. But I think that there’s a few common ingredients for success, and one of those is something that I think we’ve all touched on in the discussion, is that creation of that sense of safety and trust in terms of the system. So having a very clear process and plan, making sure that you start with participants around the rationale for, and the philosophy of, peer observation, that introduction into the ethos, very clear guidance in terms of the stages of observation, the feedback conversations, how to reflect, giving people templates to assist them to do that, so making it as easy as possible, so that academic staff feel that it’s safe, but it’s also feasible. It’s also something that’s achievable and that they can build into their practice. So I think that structure and that support is really, really key. I think keeping the conversations alive then about peer observation of teaching and giving people opportunities to hear from those who have already experienced it. So just a couple of months ago, we ran a session during our teaching and learning week here at DCU, where Anna and Ann Marie and another colleague spoke about their experience of peer observation of teaching and what it was like, and what they gained from it, and the challenges and the opportunities. So it’s really important for people to hear those voices from the field, so to speak. And the third thing I would say, is maintaining a focus on the scholarship of teaching and learning around these and the opportunities for research. So we’ve been very lucky in terms of this project. It spawned, I think, probably three publications at this point in time, but one of them, which is the most important one, of course, was a co-authored publication by myself and the two other faculty developers, but also all of the participants. And I think that was a really, I think, lovely and enjoyable part of the process, but really, really powerful in terms of keeping that focus on not just the research output and the research impact of this, but actually what that could contribute to the scholarship of the whole area. So those are my three pieces of advice.

Ann Marie: And just to add to what Martina had said there, the scholarship and the joint publication for us… first of all, it was so generous of Martina and her two colleagues to enable that. Obviously we are expected to write and to research and to publish, and because of the way Martina, Muireann, and Morag structured that, and built in the scholarship that enabled that, that ticked a box for us. And sometimes the teaching and learning piece for staff… staff can see it as something other, and they don’t have time for it, because they have to do research, whereas we need to be looking at those two things being aligned, and I learned a lot from Martina and her two colleagues did that time, and I’m actually using it in a project I’m involved in now where I’ve built in opportunity, hopefully, to get ethical approval. But anyway, I’ve built in an opportunity for intended focus groups of staff here in my faculty to get publications based on their work, on their focus group for themselves, and to get publications for themselves. And I think that’s important, because the reality is we do have to do that, and we cannot ignore it. And the more we can align those two things from the beginning of a project like this, the better.

Anna: Absolutely, I’d reiterate that. So in terms of impact, I would encourage any colleague to go for it. You’ll gain more than you’ll put into it. A practical consideration, I think, spending that time using the kind of reflective models that Martina and her colleagues supported us with to enable us to identify what aspects of teaching you want feedback on. If there’s training available before, the kind of model that Martina used, absolutely go for it. I would strongly encourage it outside of your discipline, and, if possible, outside of your institution. And then finally, to reiterate Martina’s observation about the alignment with scholarship and to encourage Ann Marie on that, that has had a huge impact. The experience of engaging in this pilot peer observation project has had a huge impact on my own practice and then subsequently as a leader of teaching and learning within my faculty. So I found myself talking about this experience a great deal. Actually, it happened at a time when our institutions were undergoing a lot of change. So it was a really positive experience. And therefore I have found myself talking to lots of colleagues in my capacity as a leader in teaching and learning about this and encouraging them, even in their own informal sort of ways, to avail of every opportunity to get into other people’s classrooms and to engage in that feedback piece. And in terms of the alignment with the research, just to echo Ann Marie’s observation of how valuable that was, and that has had a huge impact on other aspects of my practice, for example, leading projects during the COVID pandemic, which were about transition of placement online, and I was really conscious from the get go to build in the opportunities for scholarship, to secure the ethical approval, and we’ve had several outputs in Scopus quartile 1 ranked journals on foot of that. And it’s continuing to impact on my practice in the different role I have in leading placement in my faculty at the moment, and again, adopting that role of kind of having participant researchers, very much focused on participant researchers and building on all those opportunities, as Martina said, not just to tick the box of publication, but the reality is we do need to do that, but to be quite explicit in colleagues about how collectively we are contributing to the furthering of the scholarship of teaching and learning, as well as enhancing our own skills and competencies as higher education teachers.

John: We’ll include a link to those studies in the show notes so that people can access them. I believe one of the things I saw in at least one of those papers was a suggestion that this has led to faculty forming wider support networks, that many of these relationships, once they were formed in a classroom, have continued since then and provided people with more peer support, which I think in a time when higher education is changing quite rapidly, could be really important. Have you found that often happen, that people have continued the relationships after the initial peer observation?

Anna: I just was one observation on that, and one of the reasons that I was motivated to engage in the study was because it provided an opportunity to work with our colleagues in the university’s teaching enhancement unit. And again, that was a really positive so that has absolutely encouraged me, personally, to continue to engage with colleagues. I’ve engaged with, actually, colleagues who were involved with Martina in leading the institution in other contexts. So that relationship and also encouraging, again, in my own role as a leader in teaching and learning, encouraging other colleagues in the faculty to similarly engage as much as possible with colleagues in the teaching enhancement unit in relation to other aspects of enhancing and supporting teaching and learning.

Rebecca: Well, we always wrap up by asking, what’s next?

Martina: So from my perspective, and we’ve mentioned this a little bit already, it’s really, I think, taking those learnings and those elements that we’ve used in that pilot project and implementing them within a system within the institution. So I think our focus on the future will be very much from that cross- disciplinary perspective, so getting different schools and different faculties to work together on this. So that will certainly be one of the things that we’ll be focusing on in the future. I think also maintaining that scholarship of teaching and learning. This is something that I think is ripe for research in so many different contexts, and I think maintaining that collaborative and that participatory approach to the research as well. So that will certainly be some of our key focus in the future.

Ann Marie: For me, in terms of just my own practice, I suppose I have continued on in the way I described earlier, with newer members of staff and existing members of staff coming in to classes sometimes as well. But maybe going forward, it’s something that maybe could be a bit more formalized or a bit more structured. At the minute, it’s ad hoc, so maybe in our own school, that it’s something that could be more structured, more obvious, and spread out a bit more, although I have to say a lot of staff in the school I work in already would have people. They’re very welcoming for other members of staff to come in and sit in and observe and listen to their sessions and so on. So I’m very lucky from that perspective that the doors are open anyway, but maybe it is something we could do a little bit more in a structured way, for new people coming in.

Anna: And just to add to that, again, just thinking about Ann Marie’s right, in our own school of inclusive and special education, there will be a lot of awareness and knowledge in terms of the programs that we offer in the school. But just in terms of teacher education, initial teacher education does tend to be quite packed programs with lots and lots of different elements. And we’ve come out of faculty quality review and come out of a significant review of our initial teacher education programs, and maybe peer observation might also provide a context for colleagues to learn about disparate elements of a program that’s a very large program with a view not just to professional development for those involved in the peer observation, but also developing a kind of a shared understanding of what the program looks like. The more we know about what each other is doing in a program, the more we can scaffold our students’ learning and gets us a little bit out of our silos as well.

Rebecca: Well, thank you so much for joining us. It’s always great to hear about the work that you’re doing, and peer observation is important work, and it’s helpful to hear of specific models of how this can work.

John: We’ve really enjoyed learning more about this process, and we look forward to future conversations. Thank you.

Martina: Thank you.

Ann Marie: Thank you, both of you, for inviting us.

Anna: Thanks. John. T hanks. Rebecca.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

368. Pedagogy for Large Classes

Posted in classroom community, engagement, evidence-based practices, evidence-based teaching, feedback, higher education, large classes, large-class instruction, learning, pedagogy, teacher training, teaching
November 20, 2024

Building a sense of community is an important component of inclusive teaching. In this episode, Anna Logan and Ann Marie Farrell join us to discuss strategies that can be used to cultivate belonging and increase student engagement in large classes. Anna is an Associate Professor in the School of Inclusive and Special Education and the former Dean for Teaching and Learning at the Institute of Education, Dublin City University. Ann Marie is an Assistant Professor, also in the School of Inclusive and Special Education at Dublin City University. Anna and Ann Marie are both recipients of multiple teaching awards. Since 2018, they have been running an annual Pedagogy for Higher Education Large-classes Symposium.

Show Notes

Transcript

John: Building a sense of community is an important component of inclusive teaching. In this episode, we discuss strategies that can be used to cultivate belonging and increase student engagement in large classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Anna Logan and Ann Marie Farrell. Anna is an Associate Professor in the School of Inclusive and Special Education and the former Dean for Teaching and Learning at the Institute of Education, Dublin City University. Ann Marie is an Assistant Professor, also in the School of Inclusive and Special Education at Dublin City University. Anna and Ann Marie are both recipients of multiple teaching awards. Since 2018, they have been running an annual Pedagogy for Higher Education Large-Classes Symposium. Welcome Anna and Anne Marie.

Anna: Thank you.

Ann Marie: Thank you.

John: Today’s teas are: … Anna, are you drinking any tea?

Anna: I am. Generally I drink Irish Breakfast tea. But on Earth, this occasion, I got something special. I’ve never had it before. It’s called Emperors Clouds and Mist, so it’s a green tea, I don’t know. Hopefully it’ll work out alright. And I’m having it in a keep cup, a glass keep cup.

Ann Marie: Well, I did not go down the fancy route, so I am drinking Barry’s tea with, I don’t know what type of actual tea it is, except that it’s Barry’s tea, and it is a staple of pretty much every cupboard in every kitchen in Ireland. So it’s standard ordinary tea.

John: And we have some of that in our office, actually. We have a nice variety there.

Rebecca: On this occasion, I got some Irish Breakfast tea. [LAUGHTER]

John: And I should have, [LAUGHTER] but I am drinking a ginger peach green tea.

Rebecca: A good favorite of John’s, of course. So we invited you here today to discuss your work related to large-class instruction. Could you each tell us a little bit about your background and interest in large-class instruction?

Ann Marie: Well, I suppose we’re both teacher educators. All of our students are either student teachers or are qualified teachers who come back to do some of our postgraduate courses. But particularly in relation to student teachers, some of our classes are quite large, and we were anxious, I suppose, to model good teaching practices in that context, especially as our students will themselves be teachers. So a number of years ago, we began to look at our work in the large-class context, particularly on a four-year Bachelor of Education primary teaching program, which, in each year group, has over 400 students, and we were teaching all 400 together in a large tiered lecture theater. So that was, I suppose, my interest in that area.

Anna: And I suppose, just to add to that, for many years I’ve taught a small number of classes that are in a large cohort, that kind of 400-plus size, and always enjoyed it, always got a buzz from the experience. And then, I think a couple of years ago, I was one day teaching that large cohort at nine o’clock in the morning in that tiered lecture theater, a quick 50-minute lecture. And later that day, I was teaching a group of six doctorate in education students, and I remember being really struck by the kind of very differential model of resourcing there: these six, eight doctoral students, really experienced practitioners, getting a lot of support and so on, versus perhaps that introductory class. But then a number of years ago, Ann Marie, who we go back a long, long way, invited me also to co-teach some of her sessions with the large cohort. And I really enjoyed that work so much, so I’d already enjoyed the dynamic of working with the large group. I had questions about it, about the equity issues associated with it. We might come back to those a little bit later. And also, I just really enjoyed working with Anne Marie and the dynamic of working hard with that large group.

Ann Marie: The pupil-teacher ratio was still one to 225.

Anna: We go back a long, long way.

Ann Marie: We were in college together with our primary degree, and then 10 years later, just totally by chance, we ended up back in college together on a postgraduate diploma course. So we go back to the 1980s as students, and then later as colleagues. [LAUGHTER]

Anna: And we’re still talking to one another, which is good. Always helps. [LAUGHTER]

John: That’s wonderful. Can you tell us about your Pedagogy for Higher Education Large Classes project?

Ann Marie: Yeah, I suppose I can tell you a little bit about that. The Pedagogy for Higher Education Large-Class symposium (or PHELC) symposium, began really as a one-off event. We were invited to run a pre-conference workshop at a wonderful conference held annually in Valencia every year called the Higher Education Advances Conference, or the HEAD Conference. So in 2019 we ran that pre-conference workshop face to face, and again, a pre-conference workshop for the same conference in 2020 online when the pandemic hit. And in the course of that, we set up a website. I suppose, we set up a model for the symposium that then became an event in its own right. So we’ve run it now for six years, and we’ve attracted people both online… the last five have been entirely online… and we’ve attracted people from all over the world who are engaging with very large-class numbers and large-class cohorts, albeit in different contexts. But many of the challenges, and more importantly, many of the possibilities that have been experienced by those higher education teachers, is our focus at that symposium. So we look for papers based on research, but also based on reflective practice, with the aim of reducing the fear and angst around large-class teaching and giving people who are doing some really good work in that area, a platform to share that work with others. So we also set up a website, and most of the presentations and all of the papers, we asked presenters to write an accompanying paper along with their presentation. They are all available on phelc.ie, PHELC.ie, for anyone to look at. The Symposium is free, we usually get funding here in Ireland, and I suppose our main aim is just to enable access to anyone who wishes to participate, either to listen or to present, that there’s no barriers. So hence the online, and hence no conference fee for it.

Rebecca: But Anne Marie, you said it started as a one off, I think it’s still going.

Ann Marie: Oh, it’s still going. We’ve done six [LAUGHTER] and our initial intention was just to do the pre-conference workshop in Valencia in 2019 but it became evident that there was an interest in that area. The Conference Committee invited us back again the following year to do a second workshop on this topic, but we just got a lot of very good feedback from participants. We also had put in a huge amount of work into developing the website, developing the workshop or the symposium, and we love large-class teaching. We had done a lot of work in that area ourselves, and it just allowed for our pedagogical practice and that to be mapped onto the scholarship of teaching and learning, and to give people an opportunity to engage in scholarly discussions, be it at the symposium or in terms of the papers that they submit, that they could align their teaching practice along with their academic writing, researcher-type practice. So we’re still going. Now we are at a crossroads, Anna and I were just wondering if we will continue with our annual symposium, or maybe we might move to a symposium, perhaps that maybe every second year, and broaden out maybe to do some more focused publication and research, perhaps every other year, we’re not sure. So we’re due to carve out a day or two to work out what we’re going to do going forward, but we will be doing something going forward.

Anna: And we were always committed, John, to open education resources, so all the publications are available as OERs and so on. So again, I think reflecting Anne Marie’s observation, we tried face to face, we tried HyFlex, we tried online, and what we’re really interested in kind of equity and making sure there’s no barriers. So we have had people from all across the world, and we’re committed to the resources we produce being available to everybody.

John: Excellent, and we’ll include a link to the website in the show notes, so if people would like to check out those papers they can do so. One of the things we’ve noticed, in the U.S., at least, is that as we brought in more first-gen students and more low-income students, we see increased DFW rates and one concern that a lot of people have with large classes is that it might be easier for students to get lost. What’s your perspective on addressing that issue of students feeling disconnected from their class and their instructor, when they’re in such a large-class environment, and I teach a very large-class as well.

Anna: Yeah, that’s a really interesting question, John. And just thinking a little bit about it, and it’s interesting, it was actually some colleagues that I met on my sabbatical where I first heard that term “weed out,” because it’s not a term that would be much used, in the Irish context anyway. On the one hand, you know, looking at the data, I suspect that that notion of attrition or non-progression or drop out is potentially maybe slightly lower in the Irish context. The latest figures coming from our higher education authority are putting us at about 85% progression from first year, which is, I think, pretty good. That said, it’s interesting that the rates of progression are considerably lower among male students, but also among students who come from socio-economically disadvantaged groups. And coming back to your question and the question of relationships. I suspect that that failing to build relationships, or build connections with classmates or with lecturers or to feel that they belong in the community, whether it’s large or small, is very likely to be a significant factor in that dropout. Again, I think interestingly, it reminds us when we look at that, and it certainly was something I know we discussed at our most recent PHELC Symposium the importance of the relationships when teaching at scale, and that not making any assumptions. Sometimes, maybe as lecturers, we assume that students are well connected, that they have a good network. That’s very often not the case. So while, as I say, the data might not be quite as strong and we don’t have maybe that concept of a “weed out class,” I think it still doesn’t preclude the fact that a concern to foster a community that is relationship rich is important, regardless of the context.

John: I noticed in last summer’s PHELC symposium, there was a paper that you presented Ann Marie with Peter Felton on relationship-rich education in large-classes. Could you talk just a little bit about that?

Ann Marie: I suppose myself and Peter, in his context, would look at very large classes from the perspective of building a relationship, that it is possible to build a relationship, whereas sometimes, I think it can feel, sometimes, to some higher education teachers, that it’s almost impossible to have a relationship but you don’t know anyone’s names, or you might know very few names. But you can build a relationship with a really big class in different ways, even though you don’t know their name: very simple things like smile when people are walking in the room, say hi as they’re passing you heading up to the seat, have a wander around the room, don’t stay stuck down at the bottom, move. So like when I’m in a class, I share my proximity. I wear a headset. I move away from the podium. I’m up the back while I’m speaking, looking over the shoulders to make sure they’re not booking flights or watching YouTube while I’m chatting. And we use different teaching approaches and strategies in that large-class context to mix up how we relate with the students like Anna and I co-teaching. So when we co-teach, we tend to use a workshop format for at least some part of that session, which then allows myself and Anna to walk up the steps of the room. We chat to small groups. So we literally, physically bring ourselves closer to the students. You can have a bit of fun in class. If you do get to know some of the students, you can invite those students to help you change the dynamic, the colder dynamic, to make it warmer, then we can talk about that in a minute, in terms of chatting with Class Reps and that sort of thing. So I suppose in the first instance, it being the potential of having a relationship, like, if you take it completely out of teaching, really huge groups like Coldplay, Robbie Williams have a massive relationship with 10s of 1000s of people standing in front of them, and they don’t know any of them, but there is a feeling, there’s an empathy generated in that really large context. And I’m not saying that we have to be Robbie Williams or Chris Martin at all. That’s not what I’m saying, but it’s just really in the teacher’s perception of what is possible there. And I know with my work with large-classes, they are my favorite. My final-year students, I’d say I know names of 10 of them. There are 430 in the room. I love that class because there is such a two-way flow in communication in the room and between sessions on back channels. It could be something like we have, in the Moodle we use here, an in-built chat function, then a module page on Moodle, but something like Padlet where students can ask questions anonymously without having to put their hands up in a really big group, allows to build up that relationship as well. And it doesn’t just mean you answer the questions in class if they ask them in class, but it also allows them a way to reach you between classes, so if something might occur to them afterwards. So I would check that kind of a back channel between the sessions, and whether I answer the questions in class or not, I answer them in writing, one way or another, on that back channel. And you don’t even need technology in class. If people are a bit reluctant to speak, you can use a scrap of paper or post it, or something like that. It’s just to get the student voice and to capture that whatever way you can, so that you’re capturing their voice and their feeling that you’re listening and that they are engaging at that level in the class. So all of those sorts of strategies build up a teacher-student relationship. The way you would in small class of 20 may be a bit more naturally, but you just need to think about it in a different way. That’s about perception, around how you perceive or what you perceive as a possibility in that very large context. And it means that, in my own case, when I look at like I’m currently, this semester, working with 430 final-year Bachelor of Education student teachers, and they’re my favorite class. I love them, and I feel I have a relationship with them, and I feel I have a good relationship with them, even though, out of the 430 I probably could name 10 people outside, but I feel I have a relationship with them in spite of that. But just I think it is how we as teachers perceive numbers and what’s big. What does a large-class mean, actually? So for me, I think 400 is large, and then I visited somebody working in Sweden, who has over 600 in his classes. So mine suddenly seems very little compared to his.

Rebecca: I think your connection of using performance or concerts or theater as an example of where you can feel connection to performers is a good example of how students can feel connected to their faculty in these large settings.

Anna: Ann Marie has mentioned, and she’s right, that it’s about our perceptions of size and what’s possible and the possibilities. There’s also a little bit of it that is about students’ expectations and students’ perceptions, and sometimes a lot of our large-class teaching is in your traditional tier lecture theater with a podium at the front. And it’s no accident that Ann Marie has a great relationship with her final-year students, because she works with them in that large group in first year. But it’s also, I think, about over time, building their expectation that they are valued, that, as Ann Marie said, simple things like smiling, that there’s an expectation that we want to have a relationship with them, that we value that, that it isn’t that disembodied voice that’s standing at the podium and lecturing at them, that it is about relationships and it is about community and assessment is another way in which we can set that up, too.

Rebecca: We’ve talked a little bit about the differences in how you might build relationships in large versus small settings. And Anna, you also talked about the difference in your experience of going from a big class to a small, intimate doctoral setting. [LAUGHTER] Can you talk a little bit more about, from the perspective of faculty, the difference in instruction between teaching these large classes and teaching small classes?

Anna: Yeah, again, I think it is, apart from thinking about the equity challenges and the message that that conveys, sometimes, some of the literature talks about information transmission approaches in a large class, versus dialogic teaching and exploration and so on. And it sometimes strikes me, and having worked closely with Ann Marie over the years as well, that actually the difference in the large class, as Ann Marie said several times, it’s about kind of not just ruling anything out, but thinking, well, if I do want to create a relationship, or I do want to elicit learners voice in that context, how am I going to do it? Rather than just saying, “Well, it’s just not possible, and it has to be that information transmission piece.” For me, it strikes me that in order to do it well, the instructor, the educator, has to be much more deliberate, much more careful in their planning, not just thinking about their learning content, but thinking about the things we’ve spoken about, like the relationships, like, how am I going to elicit students voice? How am I going to build in formative feedback? Because it’s not, perhaps, as natural as in that context, where I was working with six or eight doctoral students who were coming to the session with me, having set prior reading and that they were anticipating and fully expecting. Their expectation was that was going to be dialog, lots of exchange of views, and that’s why I think two little things can make a difference that break down that barrier in the large-class, in terms of setting our expectations of what’s possible, for not just for ourselves as higher education teachers, but also for our students as well.

John: In large-classes, students are often somewhat reluctant to volunteer responses or to ask questions. You’ve already addressed that a little bit in terms of having a backchannel. Are there any other strategies that you use to help students become more comfortable participating in larger classes?

Ann Marie: Yes, so I usually, when I have a large-class, email the Class Rep prior to the module beginning, and I try to meet them, either online or face to face for 10 or 15 minutes, at least for the first couple of weeks. And when I meet them, I request that they will help me to teach in the first couple of weeks by asking and answering questions in class. In most cases, they kind of step back a bit in horror and because they don’t like speaking in the large-class context at all. And these are Class Reps who have put themselves out there anyway, but they’re still not comfortable in that context. So what I usually do is I go through the forthcoming session, I tell them the questions I’m going to ask, and I give them potential answers to those questions. And then for other parts of the session, I give them questions to ask me, and then I get one student to monitor the back channel so that I’m not reading out the questions that have been posted. It’s a student reading it out, one of the reps as well. And I have a little mic called a CatchBox that’s in a foam dice type item. And it’s a microphone that can be thrown around the room like a foam football, and it allows the students to speak in their normal voices without need to raise them. So the reps that I have set up then models answering and asking questions. They model the use of the CatchBox. And even in the first session, you find that some people that I have not planted who are on the edge of courage tip over and have the courage to ask the questions they’re dying to ask. And by the time they get to the third session, I don’t need to set up anyone anymore. There will always be a few people who will ask questions, and the fact that the student is monitoring the Padlet or the backchannel of the typed and posted questions by default, means you get student voice as well. So I have found that a really useful way of gathering the student voice, and also, if you do set a task in the class, or if Anna and I are co-teaching in the workshop, because they’ve spoken, we do something like a think-pair-share where they’ve chatted one or two others, they have a bit more courage to respond as well, because they’ve already sussed out the response. They’ve already sussed out the question. They’ve teased out a couple of things between themselves, plus we’re walking around the room answering questions. So sometimes when that happens, I might be chatting to a group, two or three, during the task, and we have a little chat, and then I say, “Right, I’m going to come back to you now with the CatchBox, and I want you to say the whole class exactly what you’ve just said to me,” and again, you get a little worried look, but one of them will do it. So again, I think Anna, what you said there is right, the very deliberate actions in that big setting really help to gather the student voice.

Rebecca: You’ve talked a bit about student reps. Can you talk a little bit more about what they are and how you’ve designated them,

Ann Marie: They either volunteer or sometimes there might be a vote amongst the student body. So every year group, on every program, has student representatives, and the number of representatives vary depending on the size of the year group. So in a year group with 400 students, there are a minimum of six class representatives, and they’re volunteers. They’re linked with the students union, and their role is, I suppose, as a liaison or a communication between the union, between the Program Chair, and the staff of the program, and the students. So they’re a conduit for information between staff and students and between the university and students.

John: You mentioned using a CatchBox, and I’ve always wanted to use one. I saw them at a conference in SUNY Oneonta a while back. Do you toss it to specific students, or do you just toss it in the crowd to get responses?

Ann Marie: So the first time we bought it, I was delighted. I walked into the large class, and I asked a question, and I threw it in to the middle of the crowd, and they threw it back at me, [LAUGHTER] a few times, [LAUGHTER] so I never did that again. [LAUGHTER] So that is why I model it, that is why I set up the first couple of sessions. So the students I choose, or the rest, or whoever’s helping me, they scatter through the room, and then I make it very clear that when I ask a question, they have to put their hand up high so I can see them. And then I will throw the Catchbox to them. They throw it back to me. If I’m on my own, if I’m the only teacher in the room, with two aisles, I’ll stand on one aisle, and I might ask a random student to stand on the other aisle, and I’ll throw it across to that student if I’m trying to reach somebody on the far side. So in my experience, I would caution against just randomly throwing it into the group, because people are very happy to use you as target practice. It doesn’t hurt if it hits you on the head, by the way. [LAUGHTER]

Anna: In those large-classes, we tend to use a headset mic because we want to do all that moving around the room in terms of building relationships. And when we first started that, before we had the Catchbox, we used to have hand-held microphones, the ones that move around. But I think we’re all the same. We tried to put that up to a student who we know has something wonderful to add from the work, but they all recoiled in horror, but I do think they enjoy the Catchbox. But again, as Ann Marie said, it’s about being deliberate and setting it up. That’s why it works.

Rebecca: So you both talked a lot about co-teaching and working with each other. Can you talk a little bit more about the benefits of this practice?

Anna: Co-teaching is interesting because you mentioned at the beginning, we’re both in the School of Inclusive and Special Education. And I know the language about university structures is different, but basically we’re in a department of 18 academics, and we’re all focused on inclusive and special education. And I know it’s the same in the United States and in many countries in the world, but one of the models of support for children in elementary and high schools, for example, that is very often used as a way to support more inclusive approaches, rather than withdrawing children from their mainstream classroom, is co-teaching. So the concept of co-teaching, as in, let’s define it, let’s use a standard definition of two or more adults in a single physical space delivering instruction. So that’s the definition of it that is used in the compulsory education sector, in elementary schools and high schools. We’ve looked a bit at the literature. The definition of what constitutes co-teaching is not at all as clear, it seems to us, in higher education, because co-teaching can be used just to describe a number of faculty members, sequentially teaching separately, within a module or within a course, for example. But, I suppose, coming back, co-teaching is an important concept. It’s kind of a threshold concept as part of looking at inclusion for a lot of our work with undergraduate students in those large courses and also larger courses. So a few years ago, I would have been responsible for teaching content in relation to working with special needs assistance, but also delivering some input on co-teaching. So it’s a concept that we want our students to grapple with, but we also enjoyed teaching the large class. So in the interest of modeling, walking the walk, talking the talk, Ann Marie invited me into the classroom to model that co-teaching. That was a really interesting experience for us. We did a little bit of research about it. We got students’ perspectives, but we also brought in some observers to observe us co-teaching. We know the literature about co-teaching, that for it to work well, it has to be very carefully planned. And we did plan. Initially, Ann Marie, I think we spent a lot of time really thinking, mapping out that 50-minute session. Typically, we’re talking about a 50-minute session, later we did longer sessions, but we really mapped it out, and what exactly I would be doing at a particular point in the lecture, what Ann Marie would be doing, what students would be doing. And we also thought about the kind of different models of co-teaching, because literature talks about lots of different models of co-teaching. So we thought about those, and what were we actually teaching? The really interesting thing was that the observers told us we needed to be more explicit, because, again, it’s no more than that kind of notion of maybe students come into the university and come into that tiered lecture theater, and they think about it like a theater, for example, and they think, “Well, I’m here to be passive and and I’m then going to get these wonderful thoughts from the professor on the podium at the front of the stage. So they also seem to be, in some instances, a little bit thrown by the fact that… Ann Marie, you might talk a bit more about that… about the two of us being in the room. It kind of threw them out of their comfort zone in terms of what was the expected model. So our observers noted that and reminded us that we needed to be explicit. This is why we’re modeling this. We’re modeling this because we think it’s really important for you to understand this concept and how it works, because you’re going to be a teacher. That’s taken me well away from the original question, which was about why we were using it. We were using it because we wanted to replicate workshop type activities that we would typically engage in with a much smaller group. And one of the supports for doing that was to have more than one teacher in the room.

Ann Marie: Anna said one of our reasons for using it was to model the process of co-teaching, because our students are going to be teachers. But other people have used co-teaching to model, let’s say, social workers. I can’t remember the author, but there is a paper on social work education and social workers using that co-teaching model to model different perspectives around a case conference or something like that. So it also enables, if you’re a higher education teacher, engaging in something that’s a contested concept or a contested idea. You can use the co-teaching model to show that contestation, because the two people are very likely to be coming into the room with perhaps different views and different insights. And you can model that carefully in that context. The most important thing, I think, around the co-teaching, is that you’re fairly comfortable with the person you’re co-teaching with, and that it is carefully planned. It’s a fair bit of work when you first do it, but once the planning is done for a session, like Anna and I started this about 14, 15, years ago, and where we have continued to co-teach for subsequent years, the work’s done. We know what we’re doing, but it was a heavy investment of time the first year we did it, but after that, it’s worth it, absolutely worth it, because it enabled us to do workshops with very large groups that I would have struggled to do on my own.

John: And it can provide a really rich experience for students. When I was a college student, two of my most memorable classes were team taught by two instructors. One was on bioethics, where there was a biologist and a philosopher, and the other one was on the philosophy of social sciences, where there was a philosopher and a sociologist teaching together with very, very different perspectives on all of it. And the ability to go back and forth and present different perspectives is really healthy. You had mentioned, though, doing some work with think-pair-share and some group activities. Could you talk more about what other type of small group activities you do in your large-classes?

Ann Marie: So usually what we do in those classes, it depends on the workshop, or it depends on the focus. So for example, on Monday, a couple of days ago, myself and another one of my colleagues, co-taught a workshop with the ED 4 group, and the focus of the workshop was on their assignments. I use universal design for learning in my own pedagogical practice with these students to model how to use it in teaching and learning, and part of the Universal Design is in the assignments. So I give them an awful lot of choice in terms of topics. I use authentic assessment. I’ve asked them to write for an audience that is not me. They choose their audience and so on. But they’re also allowed to work with who they want and how many they want, between two and five. Some people are very comfortable working in small groups, others are very happy to work with four others or whatever, but I give them all those choices. And then last month, the workshop we ran was for one and a half hours on planning for that assignment. So my objective was they walked out of there at the end of that session, before our reading week next week, with a plan of action to tackle that assignment. So the way that was structured was, I suppose, almost like a dance between myself and themselves. So I brought them through each criterion, they were supported by a writing frame that I gave per group, and I asked them to complete each step of the plan for their assignment based on the criteria and the expectation of the structure of that so that’s an example of a type of in-class task where they work together, in that case, choosing who to work with. And we stepped it so that we have six steps, but they completed one step, myself and the other teacher, Christina, wandered the room, and we came back after each step and summarized what we were seeing and filled in the gaps and then moved to the next step. That was a full tiered or stepped task. In other workshops or in other contexts, we might ask students to just talk about a key concept, they might not necessarily write anything. Give them five six minutes to talk about it. We’ll wander the room and then come back together and ask one or two to share with that. Sometimes, we might use the backchannel on our Moodle page for them to post what they’ve taught or things like, what does the word exclusion mean to you in a school context? So it just depends. We don’t structure the group because we can’t. They’re too big. The other thing I would say, though, is, in terms of very large-classes, it often, sometimes with me and often exclusively in other modules, they’re split into small groups, so they’re in workshop groups of 13, 14, or 15 workshop groups. In some modules, they’re never together as a big group, and there’s a bit of thinking, because it’s in a large-class, but what you’re managing there is the consistency across teachers’ input in class, formative feedback and so on. So yes, the group is smaller, but you’ve a whole curricular planning thing there to consider that I don’t have to consider when I have a big group. However, when the big class is split into those smaller groups of 20 or 30, you know who you have in your group, so you can plan for those groups of 30, whereas in my big class, I don’t know who’s talking to whom, so I can’t manage it at that level.

Anna: And we mentioned the think-pair-share, we would use that quite a lot, but using the same activity, but with lots of different kinds of materials. So it might be looking at samples of children’s work, for example, and within that group in the large lecture theater, we might roughly divide the room into four areas, and each quadrant, let’s say, would look at a particular piece of work, and then we’re looking for them to listen carefully when we take feedback from, it might be the same child’s piece of work, so they’re invested in kind of hearing of the other groups, while they might actually only be actively working in their small groups or in their pair on one piece of material, but they’re actively taking feedback from the activities that the other learners have been engaged in in the large-class. The other thing that Ann Marie uses really effectively is the panel idea within a large-class, bringing in a panel of speakers. I think that’s particularly effective in that large-class.

Ann Marie: It’s highly engaging. So with one of my groups, the module is on inclusion and inclusive pedagogy and inclusive practice in education, all contested terms. So to model the multiple perspectives, I bring in panels of guest speakers. So for example, I have one panel of parents and young people who talk to my students about their experiences of primary school. Now you could hear a pin drop [LAUGHTER] in those classes. It’s highly impactful, 10 times more impactful than me summarizing what the literature says about parental and student experience of inclusion. It literally demonstrates multiple perspectives. I have a second panel then, with teachers in different teaching contexts, and a third for a different team. So in other words, with the very large class, that’s this particular final-year group, every session is different. So I use three sessions where I have panels, I’ve two or three workshops where we have asked students to engage in a particular task, and a couple of sessions that are much more like a traditional lecture, albeit that we try and find a space for them to do something in that session, just to break it up a bit. So no two sessions are the same, and I try as well with those very large classes. Your question to Anna was about losing students, particularly in first year. I do wonder sometimes, is it not so much the class that’s large, but the program, it’s not just your module and your class is large, but lots of classes are large. There’s a multiplicity of experiences those students are trying to navigate. The large-class context can be the glue that brings some of those isolated learning experiences that are at a modular level back to the class. So for example, I think, after reading week next week, two of the co-taught sessions are with people who are teaching other modules with this same group. So they’re coming in to my session, we’re going to co-teach the session. They’re going to come in and make the links with what they’re doing in their modules, and I’m going to make the links within our module so that they’re getting an opportunity to see two of us working across the program, as well as somebody like myself and Anna, who are within the same module, but these are people in different modules. I do think that goes back to helping students make the links and the engagement for them to see that the two teachers in two completely different modules are now working together.

Anna: The other thing I would just add, in terms of Ann Marie’s panel, for example, in the panels that Ann Marie brings in, there is something particularly impactful of those real stories, real lived experiences. When they are heard in a very large context that enhances the impact of the experience for example. Because we were just talking there a couple of days ago, Ann Marie and I, and sometimes we talk about engagement, and I don’t know whether it’s the case in other universities, but a big focus on active engagement. There’s almost an assumption that that means that learners have to be up and about doing something, writing something, engaging with a student response system like Vevox or Menti or whatever it is, listening and listening to a powerful story like that, particularly when you’re part of a very large group, and it’s a powerful story, it enhances the impact of that, in fact, being in that large group.

Ann Marie: It does. And by the time we got to our third panel last week, one of the students in the group was on the panel. She came to me with something that had struck her, and she agreed, with a little bit of persuasion to be one of the panelists. And that’s powerful.

Rebecca: It always strikes me how, when I’ve had guest speakers come in and talk to students, that they can say the same thing that I’m going to say, but if someone else says it, it’s truth.

Ann Marie: Because it is their truth. So all of our speakers are speaking from personal or professional standpoint. It’s their lived experience, whether it’s personal or professional, whereas sometimes we might be talking about someone else’s experience sort of a step away. And I suppose that’s another thing, especially in the arts classes, is sharing yourself with the students and just giving them an insight into your experience. In my case, my past teaching experience, my own experience of my son’s engagement with school and little hurdles he’s had to hop over and those sorts of things and that, in terms of relationships in class, makes a difference as well. You suddenly were human, rather than the person who popped in for 50 minutes, dropped the pearls of wisdom and then runs out the door again.

John: What types of formative and summative assessment techniques have you found to be most successful in large-class instruction?

Ann Marie: I suppose some of the things we’ve mentioned are based on formative assessments and feedback. So a lot of those workshop type scenarios, and even when students are working on different case studies and different items and materials on the Moodle page, we will then provide feedback on all of those items, including the ones they’re not using. And sometimes I’ll use a little screencast, just a 3, 4, 5 minutes, to explain and fill in the gaps by feel from the class that they maybe they haven’t got it or whatever. Summative assessment can be trickier, and I think one of the challenges higher education teachers face with very large cohorts, the biggest one might be around the assessment and being fair, being manageable and so on. So I’ve grappled with this. I’m still grappling with it. I’ve two very, very large-classes, and I do something different with each and that’s because of the amount of content in the module that I’m part of. Other people who also have assessments in it and so on. With that fourth-year group, I have found that taking on the perspective of Universal Design for Learning in designing assessment and coupling that with authentic assessment approaches and viewing assessment as learning rather than of learning has changed how I experience the assessment process. So with that group that I have at the moment, for example, I have 430 students. I’m going to get, this December, about 140 assignments across all of them, a fair bit of work in that, in terms of marking and feedback and everything. I won’t deny that, but the most interesting set of assignments that I get, because they’re all doing something different, and they all present, so I’ll get websites…. They’re all creating a resort for a named audience, but they can create it in whatever way they like, and they really seem to buy into it, because they’ve got so much choice. So with that group that works, the multi-layered choices work, who to work with, what to do, who to write for, because they’re fourth years. But with my first-year group, which is 450 in it, I don’t offer those same range of choices. In that case, they have a fair bit of assessment in that particular module, so I have to cut my cloth to be fair to the student, and I’m not entirely happy with the assessment I have. It’s very applied. It’s based on case studies. We do similar, aligned case studies inside the class, in a workshop style format. They’re the co-taught sessions that Anna and I do, and then they do something similar themselves, but I still haven’t nailed that large-class first-year group, giving them options around what to do, who to work with, isn’t as fair as it is for the fourth years just because of where they’re at in the course of the size of the module they’re engaged with. So that’s my next task now, is to have a think about that and changing that before I meet that particular first-year group. It is challenging, but I would say particularly with my fourth-year group, the Universal Design for Learning Framework and applying that range of choices slowly and iteratively over time to see what worked, has completely changed my assignment/assessment marking experience, completely.

Anna: The other side of that coin, of course, is the feedback piece, which, as Ann Marie has said, the assessment piece at scale is particularly challenging. So is feedback. And again, it’s a bit like sometimes I do a lot of group feedback using exemplars as proxies for feedback. Sometimes, I think lots of higher education teachers have the same experience. Students nearly need to be hit over the head and told, this is feedback. This is relevant to you, that unless it’s actually on their assignment. And I know investing a lot of time, Ann Marie has said, in that first-year model where we’ve done the co-teaching, we model the task, we scaffold the task. There’s a lot of connections between the Moodle DLE in the classroom, so there’s lots of exemplars there. And that took an awful lot of time, sort of building that up when we started first, a lot of time, but it doesn’t change. I think of the example, Ann Marie, where we were giving feedback, asking students to write S.M.A.R.T. learning targets. That’s kind of a universal element, for example, in teacher education. And the investment in time in giving collective group feedback on the DLE, on various iterations of less and more specific, measurable, achievable, realistic, time-bound targets. It’s a lot of investment time initially, but it doesn’t change.

Ann Marie: Then it’s done. It took us hours the first year, then it’s done. You get the same type things back from following year groups.

Rebecca: So we always wrap up by asking: what’s next?

Ann Marie: Well, we are rethinking PHELC. So we’ll watch the space for that. We just need to think about where we might go with that, how we might expand it, maybe beyond us, and what we might do above and beyond our symposium. And then, as I said, in terms of the large-class teaching, it’s always what’s next. Like no two years are the same. We change. I love changing it. I’m basing the feedback from students as well. Each iteration of a module, the delivery of a module, might be a little bit different, and I am hoping, particularly with that fourth-year module, that the assessments that some of the really excellent ones, I’m developing a repository for those, which I’m hoping will become a site for publication at undergraduate level, in the first instance, within the university, but maybe further afield down the line. That’s a long term, big project that I’ve just dipped my toe in the water off. So I’ll let you know. [LAUGHTER]

Anna: And I suppose, just beyond our own work, one of the things we mentioned was the co-teaching and how much we have enjoyed over the years working together, and that has mainly been in the large class. So for me, and I mentioned that, one of the things we’ve done was we invited people to come in and observe. So one thing that I’m really interested in is that opportunity to be in other large classes, to be that observer, to maybe support other colleagues in their large-class work and so on, and any opportunity I can get to co-teach and to work with other people out there, embracing.

John: Well, thank you. This was a fascinating conversation, and we look forward to talking to you more in the future.

Anna: Thank you, John, and thank you Rebecca, for the opportunity. This was great.

Ann Marie: Absolutely. It’s been lovely to speak with you. Thank you for inviting us.

Anna: Indeed.

Rebecca: Well, thank you for some great ideas.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

349. Growth Mindset Messaging

Posted in continuing-generation, diversity, engagement, equity, first-generation, first-generation students, growth mindset, higher education, inclusion, instructional communication, large-class instruction, motivation, STEM education
July 10, 2024

First-generation college students, on average, have lower GPAs and higher dropout rates than continuing-generation students. In this episode, Elizabeth Canning, Makita White, and William B. Davis join us to discuss a growth-mindset intervention that has eliminated this equity gap in a large STEM class.

Elizabeth is an Associate Professor in the Psychology Department at Washington State University. Makita is a graduate student at WSU’s Experimental Psychology Program, and William is a Professor of Molecular Biology and the Interim Vice Provost for Academic Excellence and Student Achievement at WSU.

Show Notes

Transcript

John: First-generation college students, on average, have lower GPAs and higher dropout rates than continuing-generation students. In this episode, we discuss a growth-mindset intervention that has eliminated this equity gap in a large STEM class.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Elizabeth Canning, Makita White, and William B. Davis. Elizabeth is an Associate Professor in the Psychology Department at Washington State University. Makita is a graduate student at WSU’s Experimental Psychology Program, and William is a Professor of Molecular Biology and the Interim Vice Provost for Academic Excellence and Student Achievement at WSU. Welcome back, Elizabeth and Makita, and welcome, William.

Bill: Thank you. It’s a pleasure to be here.

John: Today’s teas are:… Elizabeth, are you drinking tea?

Elizabeth: I am drinking tea. I’ve learned from past podcasts with you that I should bring some tea along. So I’m drinking some sweet and spicy black tea today.

Rebecca: Sweet and Spicy… [LAUGHTER] sounds interesting.

Elizabeth: It has some cinnamon and some orange. It’s very good.

John: Very nice. And Makita?

Makita: I’ve got classic Earl Grey today.

Rebecca: Can’t go wrong with a classic. [LAUGHTER]

John: And Bill?

Bill:I have a Raspberry Zinger herbal tea today.

Rebecca: I’ve moved on to Awake tea… this time of day. [LAUGHTER]

John: We recorded another podcast much, much earlier today. So that’s probably needed a bit more. [LAUGHTER] And so that I won’t be awake all night tonight, as I have been for much of the last couple of nights, I have just a pure peppermint tea.

Rebecca: Probably a better choice for this time of day. [LAUGHTER]

John: And for this time of the year.

Rebecca: So we’ve invited you here today to discuss your March 2024 study: “Growth Mindset Messages from Instructors Improve Academic Performance Among First-Generation College Students,” which is quite a mouthful. [LAUGHTER] But before we discuss the study, can you talk a little bit about the difference in outcomes between first-generation students and continuing-generation college students?

Makita: Sure. So when we say first generation, we usually mean someone who’s the first in their family to attend college. For this study, we specifically defined it as a student for whom neither parent has earned a four-year college degree. And these students typically have to face and overcome a lot of social and cultural barriers in order to be successful in college. And although almost a third of the people who go to college are first-generation, they don’t always seem to do as well as continuing-generation college students. On average, they tend to have a little more difficulty adapting to college, they typically earn lower grades and they have higher dropout rates. They also act a little bit differently, they’re less likely to do things like go to office hours, or communicate with their instructors, whether that’s by email or in person to seek clarification on course materials. And just in general, they tend to be less likely to try to access helpful academic resources when we compare them with continuing-generation college students, which would be students for whom at least one parent has earned a four-year college degree. And on top of all of that, the students are also usually… I mean, they have to be… dealing with having less familial guidance than their peers do when it comes to navigating higher education. So if you’re a continuing-generation student, you have a parent who might tell you something like, “Hey, it’s a good idea to go to office hours,” or “You should be asking questions” or “there’s a financial aid department,” important stuff like that. They also may be dealing with a mismatch between the values and culture that they grew up in, compared with the values and expectations of an American university, which is typically very individualistic and may not be as supportive as what they are used to. And of course, they’re also more likely to be working, living off of campus, and they usually get less financial support from their families. But there are other factors that may exacerbate the difficulties that they’re dealing with. And this study is about one of those.

Bill:And one of the things that, just for context at our local institution to keep in mind, is WSU has a lot of, for lack of a better term, brand loyalty. It was not uncommon for me in my class to have fourth generation Cougars. And if you think about that legacy of your great-grandparents had attended this institution, as had other people, that sort of familial knowledge that gets passed along, is a huge benefit for that population of students as compared to the first-generation students as Makita was talking about.

John: One of the things that you mentioned, Makita, was differences in help-seeking behavior. And one of the studies on that is one that we talked about with you and Elizabeth in an earlier podcast, and we’ll include a link to that in the show notes. Could you describe this current study?

Elizabeth: Yeah, so this study we conducted in 2021, I believe. It was the second semester online, so the pandemic year. I know we’re all sort of crossing out that memory from our minds… [LAUGHTER] …that whole year. As instructors, it was a little tumultuous. But Bill and I got together and we designed this study together in his class. He was teaching at the time the introductory biology class at WSU. And what we decided to do is test out some actionable techniques that instructors can do to communicate a growth mindset to their students. So a lot of the research that my lab does looks at how growth mindset messages impact different students from different backgrounds. And at this time, there was a lot of suggestions about how to implement growth mindset ideas in your class, how to change course material, how to integrate it throughout, but there wasn’t really any experimental evidence of specific strategies that instructors can do. So we decided that we would just test it out in Bill’s class. So what we did in this study is we randomly assigned everybody in the course to one of two conditions: a control condition or a growth-mindset message condition. And after the first two major exams in the course, students received an email from Bill, and in the control condition they received a message that is pretty typical of an instructor after an exam has been conducted in their class. It says things like: “exam grades are posted online, here’s how we calculated them,”
“here’s what it means,” and then “feel free to talk to me in office hours, or ask questions about it.“ In the growth mindset message condition, we included all of that information, but we also included a fairly lengthy paragraph… I’ve gotten comments on… but it’s a big chunk of a paragraph that basically says, in Bill’s words, “Here’s what I believe about student performance and exam performance and improving in the class.” And there we integrated growth mindset theory, we talked about how abilities can be improved, we talked about how academic struggles are normal to experience, and that this struggle is part of the process of learning, that is something that is controllable. That you can put in effort that you can use different strategies to make improvements. And I think we used, Bill, if correct me if I’m wrong, we started with a draft of this email that you had already been doing in your class. And we just sort of infused it a little bit more. So Bill really wrote like the first draft of it, and then we kind of went back and forth a little bit to edit it and infuse it a little bit more with growth mindset. But it’s really written in the voice of the instructor. After we implemented this email after the first exam, we did it again after the second exam, changing the language a little bit. We assessed their performance across the semester. So we looked at their exam grades, we looked at their final course grade, and we found that students who were in the growth mindset condition earned higher grades in the course. And this was predominantly driven by first-generation college students. So first-generation college students performed much better in the course, when they received those growth-mindset messages from the instructor.

Rebecca: Can you talk a little bit about the effects and how significant they were?

Elizabeth: Yeah, so when we look at the group differences in the control condition, what we find that’s very consistent across big STEM courses across the university, that continuing-generation students are getting better grades than first-generation students. So in this course, they’re earning about three-fourths of a letter grade higher than first-generation students. So this is a really, really big gap between the continuing-generation students and the first-generation students. It’s not specific to this one course. So we see this in a lot of different datasets. But what was really remarkable in this study is that that difference in their grades was completely eliminated in the growth-mindset condition. So first-generation college students earned about three-fourths of a letter grade higher in the growth-mindset condition compared to the control condition. So this is a pretty large effect on final course grades. And of course, it’s pretty significant in terms of the curriculum that they’re a part of. Bill, if you want to talk a little bit about the students who took this course and how it plays a role in their career trajectories.

Bill:Yeah, thank you for that. I was looking forward to jumping in a little bit. And this is a little bit about my journey too, in this course. So, at this point, just for context, this was the 16th semester that I had taught this course. Largely I had been either the sole instructor or I was the primary instructor of record for the course. It’s taken by about 450 to 550 students each semester, little bit of differential between fall and spring in size, over 50 different majors, or pre-major students take this class. It’s nominally a freshman majors biology course. But there’s a substantial number of students who are neither biology majors or life science majors. And there’s a sizable number of students who are not freshmen. So, in reality, the highest population of my course normally were sophomores, freshmen were the largest population in that spring semester. And that was pretty typical, because it is oftentimes taken as the second semester of an introductory biology sequence. So yeah, and to be honest, over the years before this study, I had been trying to narrow those opportunity gaps between first-generation and continuing-generation students. I knew they existed. I had done many things with the course experimentally over the years, and never had really closed that gap. This was one of the first times, if ever, I saw that gap completely close in anything that I tried, and I tried tinkering in the lecture in the lab, big changes, small changes, a lot had gone on before this.

John: You mentioned that the class is normally about 450 students. Could you tell us a little bit more about the specific course in which you implemented this? You noted that this was during the pandemic. Was this offered remotely? Was it synchronous? Was it asynchronous? And how many students were there in this particular sample?

Bill:So, the course was taught in a synchronous Zoom manner. So both the lecture and the lab were taught using zoom. I believe, if memory serves, I had the largest Zoom Room on campus that semester. I can’t remember the exact number, I want to say my enrollment that semester was somewhere close to 550, or 600

Makita: The final sample is 417, I believe.
Bill:417. That’s right. That’s how many students, so it must have been around probably 450, at the beginning of the semester, or something like that. Yeah, we had synchronous Zoom every day with students. In fact, I learned pretty quickly how to crash Zoom my first semester I taught remotely. [LAUGHTER] So by then, it was a little bit more of a pro, and at least knew how to keep it running for 50 minutes at a time.

Rebecca: And true badge of honor.

Bill:[LAUGHTER] Well, there are many lumps that we took in the pandemic and learning how to teach and Zoom was one of them. [LAUGHTER]

John: I had 350 students in the fall of 2020, and it was not my most enjoyable teaching experience. What are the factors you controlled for? Because you did control for some other variables to separate out the effect of the growth-mindset messaging?

Elizabeth: Yeah, so we controlled for three things in our analytic models. We controlled for students’ prior college GPA… so this was just their self report… GPA that they’ve had in college. Since this was in the spring semester, all the freshmen had had a semester under their belts at that point, to have an idea of what their college GPA was. We included this just because we were looking at performance. So we wanted to filter out or control for their prior performance, that’s a big predictor of their performance in the class. We also controlled for their self-reported personal mindset beliefs. So we asked the students themselves, what do you believe about the nature of intelligence? Do you think it’s something that can improve? Do you think it’s something that you’re just born with? So we measure their own self-reported mindset beliefs, and we control for that and all of the analyses, because we want to look at the effect of the intervention above and beyond student’s own beliefs about ability. And then the last thing we controlled for was their race or ethnic status. So what they self reported to us on a survey. Here, we controlled for that because there is some overlap between race, ethnicity, and first-generation status. We did look at the different groups separately, we don’t have enough power to really look at race ethnicity as a moderator at the intervention. So we control for it in our analyses to look at the effect of first-gen status above and beyond race and ethnicity,

Rebecca: I can imagine, as an instructor who’s fiddled quite a bit in their class to overcome these equity gaps, that it might have been a little surprising [LAUGHTER] how much of a change happened with such a relatively small intervention.

Bill:It was shocking. It’s funny, because one of the things that Elizabeth has really talked about is how we took materials that I actually already possessed in some ways, and modified them slightly in some cases, and more radically in others, but just that transformation was enough to start to really make an impact. And you could see it happening. And that was amazing to me. I remember when I halved my opportunity gaps in my class with an intervention. And I remember dancing and celebrating, [LAUGHTER] and then this one was just on that scale. This was huge and amazing to me.

John: One of the interesting things in this study is you were looking at the effect in terms of grades on subsequent exams, but you didn’t find much of an effect on the second exam from the first intervention. But when you had two of those messages coming out, that’s where you found the relatively large effect. Why do you think the second message was having more of an effect than just a single message?

Elizabeth: Yeah, so we have a couple of theories. We think that one reason could be that they just need the message more than one time. So they need two doses or more than two doses to sort of really get people’s attention and make them change their behavior. That c ould be one reason. Another reason is that it might be enough to just have one message, but it would take time for behavior to change in order to see a difference in their performance. So it could be that after the first exam, they started engaging with the material differently and then that then sort of snowballed to affect their third exam scores, and then again, their final course performance, but we don’t really know exactly what’s going on. Another reason could be that towards the end of the semester, students get a little antsy, and they start waking up a little bit to, “Oh, I should probably be doing something different if I want to have a grade that looks like this.” So it could be timing, it could be dosage, it could be the effects take longer to materialize in terms of performance changes. One of the interesting things that we found here in this dataset is we did try to look at behavior in the way that we had access to, so we pulled all of their activity on the course website. And this semester, our campus was using Blackboard as the course sites. And we pulled all of the Blackboard data. So we looked at whether students in the growth mindset condition were engaging with Blackboard differently. And we found that they were. Students, on average, who were in the growth-mindset condition engaged with the course material 40 instances more, on average, than students in the control group. This is around a 12% increase in webpage engagement. So after they get this message from the instructor saying that intelligence is something that can grow, abilities can improve over time, and here are some strategies that you might implement to realize those changes, they’re going to the course website, they’re clicking on things, they’re re-reading their notes, they’re looking over the course lectures, they’re engaging more with the resources that are posted there. And then not surprising to any instructor ever, [LAUGHTER] that when students are engaging with the course materials that then lead to greater performance in the class. So we found a mediation effect through their behavior with the course material that was posted online. So this is one example of the behavior change that I was talking about earlier, that when they get this message, it can be inspiring, it can inspire a different type of behavior that can lead to better performance in the course.

Rebecca: Those are really strong click rates generally, [LAUGHTER] especially when we’re often complaining that students don’t read our emails or messages. [LAUGHTER]

Elizabeth: Yes. So, we did look at differences in Blackboard in terms of clicking on course material versus going to the page where all their grades were, like the gradebook. And we didn’t find any differences on their engagement with the gradebook. So it wasn’t that they were just tracking it more or anything like that. But it seems to be the case that they’re actually engaging with the substantive course materials more, on average, in the growth- mindset condition.

Rebecca: It’s just incredible. [LAUGHTER] Can you talk a little bit about… and we’re hinting towards this, obviously… what does this study suggest that faculty and institutions can do to improve outcomes of first-gen students, or maybe even, Bill, what you’ve continued or started to do?

Bill:I think, for me, what I took away from it was being very intentional with my students in communicating what I believe about them. And I mean that in the spirit of the old NPR show, or the segment that they used to do, you know, “This I believe,” where people would have those small snippets. I felt like this intervention was something along those lines where I could be authentic in communicating what I truly believed about their abilities. And I did it in a way that, with some small tweaks, obviously had these much deeper and larger payoffs. And I think that’s one of the real nuggets of wisdom here is that when you want to transform, and have better student outcomes, you want to narrow these opportunity gaps for students from certain backgrounds or experiences when they come in. Oftentimes, you don’t have to throw away what you’re already doing. Oftentimes, you already have nuggets of really important things that you’ve built over time. And all you need to do is to work with somebody who sees things just a little differently and can tweak them with you. And I think that partnership with Elizabeth and Makita was really critical for me. They were able to take things that I already had, and show me a different way to just modify what I was already doing. That messaging would go out to every student after that time point. When you see an effect that size, it almost becomes ethical to in fact, it is ethical, in my opinion that you don’t preclude any student from experiencing it. So that should be the way that everyone should be communicating with their students, in my opinion. And you want to make sure that it’s authentic as well, because I started with a set of materials that I had already written and sort of came from my heart, my own experience, that made the authenticity, I think, real whereas if I would have tried to have said something to students that I really truly didn’t believe about their abilities, I think they would have sniffed that out in a heartbeat. And we probably wouldn’t have seen the effect that we did.

Rebecca: It’s amazing what a little bit of intention can do in the design of something, especially because we often don’t realize how big of an impact a simple communication might have. We almost think of it as like a throw away, or a quick thing you might be doing, but you add a little intention to it, and it starts having a really different impact.

Elizabeth: Yeah, I think you mentioned earlier that students might not read the email, you might think that they’re not going to engage with it. And we also thought that, [LAUGHTER] so we think that in designing this intervention, we were really drawing on a lot of the intervention research that had been done previously. So research on wise interventions, mindset theory, all of that was integrated into the design of the message itself. So we started with Bill’s original emails that he was sending out to students and the messages that he had already curated for his class, and timing it in a time in the class in which students are receptive to that information, I think, is also really key. So we don’t know this empirically. But I would guess that if we were to send that email a couple of weeks before the exam, it would not have the same kind of impact. We send it directly when that performance information was posted online. And that’s when students are questioning themselves. They’re saying, “What does this score mean? Does this mean that I’m not good at this? Does this mean that I can continue? Does this mean that I’m smart or not smart? …and that uncertainty, that is the right time to infuse your message as an instructor to counteract those uncertain messages about ability. So I think the timing really matters. I think authenticity, like Bill said, really matters. Makita and I have been working with other institutions to implement a similar type of intervention with instructors. And before we did that Makita did a bunch of focus groups with the students at these institutions and showed them the messages that we used in Bill’s class, and we ended up changing them quite dramatically. The students gave us feedback like “nobody here would ever say something like that,” or “here’s what you need to say to reach students here in my class,” or “here’s what I will be receptive to.” So students in different contexts are going to respond to different messages. And so you really do want it to fit your own class, how you would speak, how you would talk to your students, the needs of the students in that moment might be different from the class they’re in. So it really is not a copy and paste type of situation where you were gonna copy what we did here and put it in your class. I think you do need to kind of go through the process of generating it for yourself and your own students.

Rebecca: I could imagine there would be an element of needing to evolve it over time, too, as your student population changes, and as you change as a teacher.

Elizabeth: Yeah, absolutely.

John: One of the things that I think is particularly nice about this study is that we see higher DFW rates by first-gen students in STEM classes in particular, and so many students end up leaving STEM fields that they intended to major in in their first couple of years. And that’s where the rate of return to education is the highest. And in general, first-gen students come from lower socioeconomic groups and lower family income groups, and if we do want to see more equity in outcomes, getting more first-gen students into the STEM fields could do a lot to help reduce some of those equity gaps in terms of overall outcomes.

Elizabeth: You mentioned the long term nature of these effects. I think it’s a great question. I think it’s something that we’re currently looking at in a project that we’re doing that’s funded by the National Science Foundation, where we’re looking at how these interventions in an introductory course can change career trajectories, that when you get a B instead of a C in a major introductory course, that’s gonna mean something different to that student, that’s going to set them on a path of following that next course sequence, that next step. So that’s the question that we’re asking and the research that we’re doing now, and hopefully, we’ll have some answers in a few years.

John: Peter Arcidiacono at Duke had done a study on the decision to stay in STEM fields. And he did look at some of these questions in terms of the effects on major choice based on the grades that were received. We can share a link to that study in the show notes. In the IRB proposal, did you include the possibility of following up these students in terms of their outcomes a few years later, in terms of retention rates for the students?

Elizabeth: That’s a great question. I believe in terms of IRB, we had students consent to their academic records. So we should be able to go back and look at transcripts from the students who participated in the class. We could look at course taking, we could look at graduation rates, we could look at major. If these students were freshmen/sophomores in 2021, then we’re probably around the time where they would be finishing up so maybe in the next year we could.

Bill:I was just thinking the same. Now’s the perfect time to go back and take a look, because we’ve had enough time since the intervention to see what’s happened. And actually I was gonna go back to one of your earlier points. You were talking about statistics earlier. Basically, this is really the only way that we’re going to increase the number of STEM students going forward. If first-generation students at WSU are 40% of our population today, that means if we don’t improve their outcomes, then we’re going to continue to see the declines in STEM outcomes and STEM graduates that we are trying so desperately to fight against, and also, we’re not going to have the representation in our STEM graduates of the population of the US that we want to see.

Rebecca: I think in addition to the growth-mindset piece, I’m sure that the authentic nature of the messages also just showed deep care. And that can be really important for first-generation students who need that ally or advocate for them in their college journey. So I can imagine those messages coming from Bill in Bill’s voice that make them feel like “Yeah, you can do this, I care about you,” from that perspective is also really important.

John: And there was a study done by a couple of economists, entitled “My Professor Cares,” where they did follow up students four years later, and they did find a significant effect in retention rates four years later by a similar type of intervention, I don’t think it was quite as much growth-mindset focused, but just sending a signal that the professor cared about the students outcomes, a very simple message, made a significant difference in student’s grades as well as longer term outcomes. So it’s nice to see more studies of this, especially focused on issues such as growth mindset.

Elizabeth: Yeah.

Rebecca: So we always wrap up by asking: What’s next?.

Elizabeth: So Makita and I and our lab have been working for the past couple years on following up on these data in different institutions. We used this study as pilot data in a grant application. And now we’re sort of scaling up to other institutions. We have a project with a really large HBCU, a Hispanic-Serving Institution, WSU is continuing to be involved as well, an Asian and Pacific Islander Serving Institution to where we have a number of instructors who vary in different characteristics, different class compositions, and we’re looking at these instructor messages in an experiment in these different contexts to see what matters. Does it help students develop their own growth mindset? Does it change their behavior? Does it work in some contexts, but not others? Do the instructor characteristics matter? So far, in the past two years, we’ve collected data from around 10,000 students who have been involved in this intervention project, and we’re currently in data analysis phase. So stay tuned for future studies and papers that come out of that data set.

John: Makita?

Makita: Well, what’s next for me is Elizabeth and I are actually going to meet right after this to do a debrief and to discuss some of the analyses that I just ran on a study that will be involved in my dissertation.

Rebecca: And how about you, Bill?

Bill:For me, I think about: “How do I proselytize about this work?” Because I think about what little changes make a big difference in an institution. We talk a lot about student retention, we talk a lot about student outcomes, we talk about creating the global citizenry that we want to see emerge from higher education. And the one experience that every student has in common, whether they are a first-generation a continuing-gen, or from some other background, is they have to take classes, and they have to be in classroom environments every day of the week, for the most part, over a course of four years. And I think about what would it look like to have interventions like this, that have evidence that back them, that students get inoculated with repeatedly over multiple time points with different voices, and that faculty engagement in the process of helping our students become better, to find success and to move forward in their careers? To me, that’s the magic of this. I’ve been talking to other people about it and I think about what would a curriculum look like that had small things like this? And how could it transform what we do? And I think that’s a vision that I can buy into, and I hope others are gonna come along for that ride.

Rebecca: I think the scope of the interventions that you’re talking about, that have this kind of impact, are really easy to buy into. Because usually you’re thinking about some drastic, giant investment of time, is what most people are thinking about. And that often is what prevents action from happening. So I’m excited, excited to share this and to hear what comes of your bigger study.

John: And one thing we appreciate is when we see things like this posted on social media so we can hear about it [LAUGHTER] and we can share it out a little bit further. [LAUGHTER] And I hope you’re presenting this at various conferences to reach a larger audience.

Elizabeth: Yeah, we will be presenting some of this work at SABER this summer. So that is the Society for the Advancement of Biology education research, and we’ve presented it in some other psychology conferences throughout the past year as well.

John: Excellent.

Bill:And I gave a plenary talk at the summer educational meeting for the ASBMB, American Society for Biochemistry and Molecular Biology. So I gave a little preview of what we had found. And that generated a lot of discussion and excitement, actually people asking for additional resources and ideas and things. So I think there’s a receptive audience out there.

Rebecca: That’s wonderful.

John: Well, thank you for joining us. It’s always great talking to you and it’s great to hear about this newest study and to meet you, Bill. We’re looking forward to hearing more about how things go in the future.

Elizabeth: Thank you so much. Thanks for having us.

Rebecca: I hope you’ll reach out when your next study is done. [LAUGHTER]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

300. Episode 300 Reflection

Posted in assessment, emotion, engagement, equity, grading, higher education, inclusion, large-class instruction, learning, metacognition, motivation, pedagogy, professional development, reflective practice, teaching
August 2, 2023

This is episode 300 of the Tea for Teaching podcast. Whether you are a new listener or have been with us for all 300 episodes, we are very grateful that you’ve joined us on our podcasting journey. In this episode, we celebrate this milestone by reflecting on what we’ve learned and how the podcast has evolved.

Show Notes

Transcript

John: This is episode 300 of the Tea for Teaching podcast. Whether you are a new listener or have been with us for all 300 episodes, we are very grateful that you’ve joined us on our podcasting journey. In this episode, we celebrate this milestone by reflecting on what we’ve learned and how the podcast has evolved.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Now that we’ve reached Episode 300, we thought we’d take the opportunity to reflect on some of the changes that we’ve seen in higher education since we launched the podcast in 2017.

John: Today’s teas are:…

Rebecca: I have Blue Sapphire.

John: I heard you just stocked up on a trip to your favorite tea store.

Rebecca: Yes, I was really excited to stock up on my favorite and I have a couple new ones too. So maybe in some upcoming episodes, we can try those out.

John: And I have Bing Cherry Black tea from Harry and David’s which is made by the Republic of Tea. When we first got started on the podcast, much of the focus was on specific teaching practices and techniques and interesting projects. Most of our guests were people that we knew or guests who were within our professional networks.

Rebecca: in the spring of 2020, as we know, [LAUGHTER] the focus shifted to the challenges associated with remote and online teaching, and the challenges facing remote learners and instructors.

John: As we became accustomed to pandemic teaching, we focused a bit more on faculty concerns as we transitioned into the transformed higher ed landscape. Historically, higher ed had been designed to serve the elites of society, and while higher ed gradually became more open and students have become much more diverse, many residual practices have worked against serving the students that we have. During the pandemic, faculty became much more aware of the inequities facing our students as well as faculty and staff.

Rebecca: Yeah, so one of the things that we’ve been talking about quite a bit is this more holistic focus on the needs of our students and faculty as humans, and really generating and creating a much more inclusive higher ed environment. How do you see that moving forward, John?

John: One of the things we’ve talked about is addressing the needs caused by the increased demands on time for faculty, staff, and students. As we developed new teaching techniques and tried to build more structure into our courses, it put much more demands on faculty in terms of redesigning their courses, in terms of paying more attention to the needs of students, and providing students with more feedback. And that has led to issues with burnout, which we’ve addressed in a number of podcasts.

Rebecca: And you’ve never experienced that, have you, John?

John: The day is not over yet. [LAUGHTER]

Rebecca: Well, how do we think about supporting faculty as we move forward with all these demands on their time in trying to prevent burnout? We’ve talked about this in a couple of episodes, but as we enjoy summer and have a little bit of more downtime for some of us… maybe not you, John… and move into the fall, what are some things that we need to be thinking about for our own classes to prevent burnout?

John: One of the things that we’re trying to be careful with when we recommend new teaching techniques to faculty at the teaching center here, is that they change approaches gradually, that small changes, incremental changes, are much easier to accommodate than the type of rapid changes that people had to do when they first moved into remote teaching. And so I think we have to be careful in making sure that we maintain a balance and we don’t burn out ourselves, because we’re not going to be very effective in supporting our students if we’re struggling to get through each day ourselves.

Rebecca: Yeah, we need to be present, just like we want our students to be able to be present and have the supports around them to be present in their learning. I think one thing that we’re also talking about in grad studies in our office is really this increased stress on faculty, and how do we support faculty, but also how do we support graduate student populations through things like accountability groups, or ways where there’s another human for accountability, but also for support, and not necessarily a mentor model, where there’s a power dynamic, but really a peer-to-peer approach to connect people together.

John: And we’re running two reading groups this fall to address some of these needs. One of the reading groups is on Sarah Rose Cavanagh’s new book, Mind over Monsters. And the other one is the second edition of Jim Lang’s Small Teaching. We had done that a few years ago, but we’ve had a lot of new faculty since then. And while we try to reach as many faculty as we can in our workshops, there’s a lot of faculty who are still teaching in pretty much the same way as faculty were teaching a century or so ago. And we’re hoping that by encouraging small modifications in teaching approaches, it might encourage more faculty to participate in introducing active learning activities and evidence-based teaching approaches.

Rebecca: It’s really easy to slip back into past practices when we’re tired [LAUGHTER] and overworked. And it’s not surprising that people have kind of slipped back into assignments and stuff that they’re really familiar with to reduce the cognitive load around new stuff and the many stressors around. So having that added support to help faculty re-engage with some of those ideas is, I think, a really great idea at this juncture. And I love that Mind over Monsters is one of the reading groups as well, because mental health is such an increasing concern, not just for students, but also for faculty and staff.

John: And we’re very much looking forward to both of these reading groups. Among the things we’ve talked about more frequently since the start of the pandemic are the challenges faced by underrepresented and contingent faculty.

Rebecca: I think when we’re introducing new techniques, and we’re thinking about supporting students around mental health, or we’re thinking about evidence-based practices and engaging in active learning, we need to remember that contingent faculty or underrepresented faculty have different barriers or different obstacles in implementing these things… or even more pushback from students and implementing some of these techniques. So we really need to be cognizant of supporting each other and realizing that we don’t all have the same kind of supports in place. I think some populations of faculty are just overly criticized. And when they try something new, it’s not accepted in the same way that a more dominant group’s adoption of those same techniques might be.

John: And that’s true both by students as well as by their faculty peers. And one of the things that’s come up in many of the podcasts we’ve discussed are the biases in both student and faculty evaluation of teaching.

Rebecca: Yeah. One of the things that I think is on the minds of our faculty too, is, as we’ve seen increased diversity of our students, we’ve seen diversity in levels of preparation. And I think those inequities have always been there. But again, maybe it’s more visible now than it had been in the past. How do we work through that in our classrooms, especially in these more introductory classes as students transition into college?

John: Well, I think those inequities have always been there, but they certainly grew with remote teaching, because our students face very unequal resources in their school districts and in their households. And when people are physically in the classroom, they’re at least exposed to the same infrastructure within their institutions. But when students were taking classes from home, as we talked about in many, many episodes, during a pandemic, they had very unequal network access, they had very unequal computing facilities, they may have been sharing a computer with multiple family members, they may have been forced to work. And as a result, the inequities in prior education and prior learning became much more dramatic during the period of remote teaching. And that disproportionately affected students from low-income households and low-income school districts. And what we have to do is provide resources, I think, for all students to be successful. And while we always should have been teaching, or providing resources and support, for all students, those needs have become much greater now, because while we are bringing in a much more diverse student body, we’re also losing students who come in with less preparation at some of the highest rates we’ve ever seen before. And we have to make sure that we’re providing the students that we accept with the support they need to be successful. And there’s lots of ways of doing it, you can build in some additional resources, you can connect to YouTube videos, and such things and provide support to students, you can use mastery learning quiz systems, and many other techniques. But we have to work towards having more faculty building that in because while many faculty are doing these types of things, and trying to build more support and more structure into their classes, it’s not a universal phenomenon.

Rebecca: And maybe even acknowledging that some students in the class are quite literally working harder to get to the same level.

John: The last few years when I’ve been teaching my large intro class, that’s something I’ve mentioned explicitly. I said, everyone here has all the resources they need to be successful. But if you had taken an AP introductory microeconomics course, or something close to that, you’re not going to have to work as hard to attain mastery of many of the concepts. If you have not been exposed to these things, or if your background in working with math and using graphs is not as strong, you’re going to have to work a bit harder. And that’s not a message that a lot of students appreciate hearing. But if we want to get all of our students to the same level at the end, the students who do come in with a weaker background need additional support to get there. And using tutoring when available, encouraging students to come in and talk to their professors and use office hours, all of those things can help but we’ve got a ways to go. What are some of the things you do to try to provide support for the increasingly diverse student body?

Rebecca: Yeah, I think the reality is that what we give each student isn’t the same, because what they come in with is not the same. So I often am trying to assess where students are and then pushing them right at that moment where they’re at, rather than expecting everyone to be at the same point. And I can do that a little more efficiently in a small class than you can in a bigger class. But I think we need to use those smaller classroom spaces to be able to do that so that everyone feels challenged, but also has what they need.

John: My introductory course is a prerequisite for all upper-level economics classes, and most of my students will be moving on to upper-level classes, so they have to reach at least a minimum level of proficiency in the discipline in order to be successful in their future classes. In other classes, instructors can be more flexible, and just try to get the most learning gains in their students, no matter what their starting points were. In my introductory class, at least, I have to pretty much take the students where they are, and try to get them all to the same place, while making sure that they’re all challenged. And that’s a very challenging goal to reach.

Rebecca: …and the difference between teaching those introductory classes versus higher level classes within a discipline, for sure, I think one of the most efficient things we can do is making sure that all students know the most effective ways to learn, because they don’t necessarily know those things coming into college, or even into graduate school. What we need to just remember is learning isn’t something that we just magically know how to do, we need to learn strategies and techniques that are effective.

John: We’ve had a number of guests over the last year or two who’ve talked about books that they’ve provided, or resources they’ve created to help students be more successful. Because one of the things that’s been pretty obvious for quite a while is that the study techniques and the learning strategies that students use are not generally consistent with what evidence suggests is most effective. And as a result, students are not using their time as efficiently as they can, by engaging in strategies that they perceive as being useful, that really result in very little increase in long-term recall… strategies such as highlighting, repeated rereading, and so forth. And one of the things that might be helpful is if we all could shift students a little bit in the direction of using evidence-based learning strategies, and some of that could take place through course design, by building infrastructures that incentivize the use of these techniques.

Rebecca: Yeah, and I think the moment that students realize that they don’t have the most efficient way, or the moment that a student begins to struggle is different, depending on some of that background, that they have. Students that come in well prepared may have never really struggled in high school, and maybe eventually, maybe even in the first year of college, you don’t struggle, but maybe it hits a little later on in their education, maybe not until graduate school. And then other students might struggle the second they get to college, because there’s not as much structure in place as there was in high school. So I think we need to be underscoring these techniques at all levels, and not just in their first year.

John: And one other thing that’s been discussed in many podcast episodes, is the importance of making the hidden curriculum of higher ed transparent to students, so that we don’t expect students to know what a syllabus is or how it could be used, that we shouldn’t expect students to know what is expected on a term paper in a class without making those expectations explicit and transparent to students. Because in general, we see a lot of students coming in, and they see it as a game where they’re trying to guess at what instructors are asking. And many of those guesses, especially for students who have not been in college prep classes before, are wrong. And they wasted a lot of time and effort that could have been spent more productively developing their understanding of the subject matter.

Rebecca: And the reality is that there’s differences between disciplines and between courses. And so the more we can be explicit about expectations within our own discipline, and within our own courses, and beyond the classroom experience of higher ed, because there’s expectations in other spaces as well, like student clubs, athletics, and all of the rest of the co-curricular activities that support student learning are incredibly important. And those are also not obvious.

John: One of the things that we’ve talked about much more on the podcast, and higher ed in general has been addressing much more extensively since the pandemic, is alternative grading approaches. Because traditional grading approaches and traditional course structures generally incentivize students to cram and to focus on maximizing their grades, rather than maximizing learning, so that if we really want students to shift to evidence based learning strategies, it would be really helpful if we could shift students emphasis away from grades and faculty emphasis away from high-stakes assessed activity and shift it more to activities that result in deeper learning, more long-term learning. And we’ve talked to many guests who have shifted to using strategy such as specifications grading, mastery learning systems, portfolio assessments, and ungraving, which has become one of the most talked about topics in higher ed in the last few years.

Rebecca: Yeah. And I think one of the things that comes up in a lot of those conversations is concerns over students just wanting the right answer and not learning and not critically thinking about the subject matter and the knowing of why and how, and doing analysis. And I think every instructor [LAUGHTER] has a desire for some of those kinds of conversations to come out in their classes, rather than just regurgitation of things that they’ve said in class.

John: Part of the issue is that when we get students in college, they’ve already had 13 years of experience in K through 12, where grades were the primary area of focus. And as a result, it’s hard to shift that focus from grades to learning. Besides alternative grading, we might use some other strategies such as encouraging students to be more reflective on their work, to spend some time in reflection-based activities and metacognitive development type activities.

Rebecca: Yeah, I know, this is a space where I was maybe a little hesitant at first thinking like, “Oh, these are just quick assignments that have no meaning,” but quickly realizing actually the value in really good well designed reflective activities that challenge students to think through how and why they learned something and what it is that they actually got out of an activity. And I’m often very surprised about how much learning occurs that is not visible, despite the fact that I teach studio classes, so I’m with my students much more than the average instructor. So I actually do observe a lot of learning. But in the reflection activities, I’m hearing a lot about how students are spending their time or things that they really struggled with and worked through that I wasn’t aware of. It also helps me understand where they’re not aware [LAUGHTER] of their own learning, or where they’re using strategies that aren’t as effective and helps with interventions. I know you’ve done a lot around metacognition, especially in your lower-level classes, but also in your upper classes.

John: In at least a couple of my courses. I’ve been using the metacognitive cafe discussion forum, which was actually the topic of our second podcast, Judie Littlejohn and I jointly developed this quite a few years back. And it’s been remarkably effective. It’s basically a low-stakes discussion forum that I’m using in my online classes, where students will reflect on their learning and share their learning strategies and will also read a bit about retrieval practice and spaced practice and the benefits of sleep in learning. And every time I do it, even though it’s only a trivial portion of the grade, it’s 5% of their total grade for participating in that activity, the students report that it was the most valuable learning experience they had in the class. A large proportion of the students at the end of the terms say they wish that they had learned these things back in elementary school, that they had been using practices that were not efficient and they didn’t realize that because they’ve never been taught how to learn. And it’s something that students have found really valuable. And the other nice thing about it is because, in this particular case, it’s done in a discussion forum, it helps them build community and helps them get to know each other, because they’ll often talk about the challenges they face. In online classes, many of the students have families where they’re taking care of young children, they may be working different shifts, they may be faced with other challenges that normally wouldn’t come up in a content discussion forum in an online class. But when they share that, and they share those challenges, and they share their career expectations, and they talk about how what they’re learning might be useful in their expected careers, besides the sense of connection, it also helps students see the relevance of what they’re doing and sharing that with other students helps build a little bit more intrinsic motivation in learning.

Rebecca: It also seems like there’s a bit of an immediacy in that context as well, because the information can immediately be put into action in a real lived experience and not something that may feel abstract, which sometimes happens within a discipline when it feels like maybe it’s not a thing I’m going to do anytime soon, professionally. So I think this really highlights the reason why we need to help students hook into everything that we’re doing to make it feel like they have a personal, professional, or educational connection to their own goals.

John: One of the topics that I use in each class where I’ve done this, at a point where students face the first really challenging material in the class, is just asking them to discuss how they deal with challenges. They share useful strategies, but one of the main benefits of that is it normalizes the sense of struggle, that when students are struggling with concepts, they often feel that they’re alone on this, but when they hear that other people are struggling with exactly the same issues and exactly the same concepts, it normalizes it, and again, it helps them understand that challenge is an important part of learning, which is not the message that they’ve generally received throughout their prior educational experience before coming to college.

Rebecca: It seems to me like this is the same reason why our reading groups work so well for faculty development as well is this connection among peers, but also that the challenges we experience are not in isolation. [LAUGHTER]

John: One of the things I’ve always enjoyed about the reading group is when people will come up with a technique or describe something they’ve done in class and people in completely different disciplines who might never have considered that will make connections and see how they could do something similar in their classes. That understanding that we’re all facing the same challenges makes it much easier to deal with some of the day-to-day stresses that we might have in our classrooms.

Rebecca: I know that one of the things that has come up in our reading groups, and also in our conversations about the future of higher ed and where we’re going to be going over the next few years is student engagement, and then specifically, the role of AI tools like ChatGPT [LAUGHTER] in the conversation. So if students don’t feel motivated, and they’re relying heavily on these tools, how do we get students to re-engage with the idea of learning?

John: Well, going back even just a little before the introduction of ChatGPT, which kind of hit higher ed by storm in late November of 2022, we did see a dramatic increase in the use of sites such as chegg.com, and various other sites out there, where the use of those tools became normalized a little bit, which made it much more challenging to give online assessments. And I think that’s where most people are concerned right now about things like ChatGPT, because with other places, you could at least locate where answers were coming from. And you could address that with the students and attribute it to the specific sites where they got their answers, which was, again, a bit of a challenge. But ChatGPT is raising some challenges for assessment that are going to be difficult to deal with, because it’s much more difficult to determine who is the author of specific items submitted online for assessment. And a lot of people are struggling with that right now. I know I’ve been struggling with it. In my spring 2023 online class, the quality of student writing on essays improved fairly dramatically over the course of the semester. And that seemed to correlate with the spread of the use of ChatGPT a bit. AI tools are really powerful, and they can be really useful. And they have a lot of potential value in education and in providing support for personal and work productivity. Right now, I think, more people are focused on the challenges, but we’re going to have to start thinking about ways in which we can productively integrate this and prepare students for a world in which the availability of AI tools will be ubiquitous.

Rebecca: And you teach in some really challenging contexts, really large classes in person, a number of online sections, and I know ChatGPT is keeping you up at night. What are some things that you’re thinking about… maybe haven’t resolved… but that you’re really thinking about redesigning or rethinking or retooling in the fall to just respond to the moment that we are currently living in.

John: As of 2020, I had shifted all the quizzing to online quizzes and tests and midterm exams and so forth. I’m seriously thinking about in my large face-to-face class, moving back to at least in in-class midterm and an in-class final exam. I really appreciated the fact that I could let students do it at their own pace, and that it took some of the anxiety and stress away when students did not have this two-hour time limit to complete an exam in the classroom. But with the size of the class, a large proportion of the testing is done with multiple choice exams, or algorithmic questions, and those are types of things that ChatGPT answers really, really well. Not too long ago, someone posted that ChatGPT 4 received a score of a 99th percentile on the Test of Understanding in College Economics (the microeconomics version of that), and those are the same types of questions that I’d be giving students on these quizzes. And while I had 1000s of questions that I had created that students were selecting from, all of those questions now are vulnerable to the use of AI tools, which makes it much more difficult to assess in that large class. Right now, the only thing I’ve really thought about doing differently in my large class is moving back to at least a couple of in-class exams. Now some of the things I was doing, such as polling questions embedded in the class activities and working on problems in class, where students submit that in real time, are generally much less subject to that type of issue. I know there are tools where students can scan the questions and so forth, they get responses back a bit more quickly, but it wouldn’t be as easy for them to do in real time when they’re in a polling environment. One of the main benefits of that is when I use polling, it was always tied with peer-to-peer discussions. And those peer-to-peer discussions is where most of the learning actually occurred from those in-class problem-solving exercises. For my online class, I’m not sure what I’m going to do. One thing I have done in the past is I’ve had students do podcast projects. And again, it’s pretty easy for chat GPT to generate scripts, but these projects are pretty heavily scaffolded. Students submit a proposal and they go through a number of steps to get there. And projects that are scaffolded like that, are probably a little bit less sensitive to the use of AI tools to generate the entire project. What are you thinking about in terms of your classes, or in terms of the graduate program?

Rebecca: I think we’ve talked a lot about the concern over the validity of our assessments and wanting there to be accuracy, not just for our sakes, but for students’ stakes in the value of their degrees moving forward. Part of it, I think, is really engaging in conversations around ethics around these tools, and not necessarily discouraging the use of the tools entirely, or banning the tools. I think that just motivates people to want to use them more, but rather to use them in ways that are productive, or interesting, but are also well documented… [LAUGHTER] like students are disclosing what they’re doing. And we can analyze the use of the tools in particular ways because maybe it could save time in particular places and not take away from certain kinds of learning, as long as we keep the learning objectives [LAUGHTER] up front. And then we assess when we’re using particular tools to determine whether or not it’s taking away from the learning. But I think these are hard conversations to have, and certainly not things that I want to be policing.

John: And I’d much rather not be policing these things. Sometimes students haven’t given me much choice in that. One example that I’ve seen recently is students submitting exam responses that asked him to analyze recent data, where the response said something to the effect: “as an AI tool, I do not have access to this data.” And when a student submits work like that, it’s pretty clear that they haven’t even read the essay responses they’re submitting on that graded assessment activity. And we want to make sure that students do actually interact and engage with their learning materials. Perhaps we can also design assessments that are not as vulnerable to AI-generated text. This semester, with my online classes, one thing I have shifted to, instead of having them discuss general debates or issues in economics, I have them focus on interpreting videos online, for example, where economists are debating certain topics, or doing readings that are not in the training database for ChatGPT, which means it’s much harder for AI tools to generate responses when they don’t have access to the underlying content that’s the focus of the assessment activity.

Rebecca: Would hyperlocal situations or examples also be a strategy because there’d be less widely available information on something like that.

John: Definitely. Information on the local community or the campus community or other local things, information that would just not be part of the training database is a good place in which we can ask students to connect the materials their learning to real-world events so that you maintain that sense of relevance while ensuring that the students are actively engaging with the work themselves rather than using a tool.

Rebecca: One of the things that I’ve used historically in my design classes, and it’s a little easier again, because I teach studio classes and see students more often so I tend to have a hunch as to what they’re working on, because I’m seeing them working on things, is really documenting process and not just using language, but showing through a video and showing steps along the way that might not be as easy to capture as an end product using an AI tool.

John: In general, open pedagogy projects, too, could be less vulnerable to having work being done entirely by AI tools. So videos would be an example, wikis perhaps might be.

Rebecca: I think that things that combine text and image are more challenging to have an AI tool create, at this moment.

John: That may very well change…

Rebecca: …by the time this episode comes out. [LAUGHTER] I think one of the things that I’m hearing us say actually, is that a lot of the strategies to reduce intellectual integrity or academic integrity issues around ChatGPT are also the things that are more likely to engage students and foster their learning anyways because they’re more authentic assessments, they’re probably more project based, they’re probably more long term with milestones along the way. And these are things that students often deeply engage in. And I think when they can connect to their local community, whether that’s the campus or the community that campus is situated in, or even their own hometown, in different ways around the discipline, those are all ways that students get a hunger to want to learn more.

John: And going back to our earlier discussion of the importance of shifting students’ focus from grades to learning, students are using tools like ChatGPT to raise their grades, even though they recognize it does not support their learning. If we can shift students’ focus to recognize the value of learning as improving skills that they’re going to need later in life, that should reduce the incentive for students to use shortcuts to avoid learning material.

Rebecca: If we’re not just looking for the right answer, but the journey to an answer, and even if it’s an incorrect answer, being able to understand why it’s not correct, and allowing that to be the learning is a really different way than our education has historically worked. The future of higher ed seems really stressful, John. [LAUGHTER]

John: It does, but it always has. That’s nothing new. But certainly the last few years have seen a lot of rapid change that… I hate to use the word unprecedented… but that have been relatively unprecedented.

Rebecca: And I think it really does speak to this need to connect with other colleagues, where we can share some of the challenges that we’re facing and brainstorm together to improve our teaching, but also to improve the level of stress we’re experiencing. [LAUGHTER]

John: One of the other things that we’ve talked about, especially within the last year or so is growing faculty concerns over student engagement. When students first came back to the classroom, there was a lot of excitement about being back. But since then, faculty generally seem to be noting that the level of engagement of students has shifted or has changed somewhat. More students are not completing assignments. Students in some classes have been disappearing from class as the semester progresses. And there’s a lot of concern that students are not as fully engaged with their coursework as they had been prior to the pandemic. So Rebecca, how are we going to solve this?

Rebecca: That’s a good question, John. I think one of the things that this aligns with is the higher incidences of loneliness, and mental health. And so finding ways to connect students to each other, and establishing those peer networks, I think, is one of the most important things that we can do in our classes. And it’s something that I’ve maybe always done in some way. But I’m being much more intentional about moving forward, because I’m feeling like even if students are in the same room, they’re still feeling really isolated. And so we have to be intentional about creating those opportunities for students to experience connection and feeling like they want to show up for each other and for themselves,

John: …using more group activities in class where the work of each student depends on the contribution of the other members does help create that sort of pressure on students to be there for their peers, to be there for the rest of their team. And that could be very useful.

Rebecca: Yeah. And I think the key to that, though, is not just assuming that students know how to interact with one another, or how to depend on one another in a team context, but really scaffolding those learning opportunities that really start with making connections and establishing relationships, because it’s the relationship that’s going to cause the pressure to show up for someone. And I think when we’re seeing high incidence of like ghosting, for example, it’s because the individuals don’t feel connected to the people that they’re ghosting.

John: And there have been a lot of studies done recently that show the importance of a sense of belonging in student persistence. So helping students form those connections is really important, because we have so many students who go to college, build up a huge volume of debt, and then disappear without getting the degree which does serve as a signal that they’ve actually accomplished something as a result of their education. And they end up with more financial struggles than they would have had had they not started. So we do want to help students form those connections for their own sakes, for their own future success. And one of the books we used in a past reading group was Relationship-Rich Education by Leo Lambert and Peter Felton. And that summarizes a lot of the research on the importance of building community and building connections, and also provides some really nice examples of ways in which institutions can transform to help facilitate those connections.

Rebecca: As instructors, we have a lot of power in that space to help students feel a sense of belonging. We can do really simple things to make someone feel seen and if they feel seen, they’re more likely to feel like they belong. So personalized messages, getting to know your students a little bit, being approachable, calling students by name, all of those things help students feel like they’re a part of a particular community. There’s so much to still learn and to come together around. And so I know that we’re looking forward to having many more guests and many more conversations to help work through many of the questions and concerns and things that we’ve raised today and have been raised by our colleagues. Now, John, we always wrap up by asking what’s next?

John: Well, what’s next for me is I’m heading down to North Carolina to teach at Duke again next week. And I’m looking forward to this. And this time, I’m going to try avoiding getting run over by a car. So I can actually teach my classes down there and spend some time away from the hospital this summer,

Rebecca: #life_goals. [LAUGHTER]

John: Small goals are sometimes more achievable. And Rebecca, what’s next for you?

Rebecca: This summer, I’m looking forward to doing some more work on our graduate student online orientation, which we put together as we transition to our new course management system in the fall and also working with some colleagues on an accessibility online module.

John: And we’re looking forward to talking to more of our wonderful guests. I’ve really enjoyed the experience of interacting with so many great people doing some really good research and doing such good work in higher ed.

Rebecca: We’re grateful for all of our guests and all of our listeners. So thanks for listening

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

299. My Professor Cares

Posted in equity, evidence-based practices, feedback, first-generation, higher education, inclusion, large-class instruction, learning, pedagogy, teaching
July 26, 2023

Students from marginalized groups often question whether or not they should be in our classes and disciplines. In this episode, Michal Kurlaender joins us to discuss an easy to implement intervention that faculty can use to improve retention and student success. Michal is a Chancellor’s Leadership Professor in the School of Education at UC Davis and is a co-Director of the California Education Lab. She is a co-author with Scott Carrell of a National Bureau of Economic Research working paper entitled “My Professor Cares: Experimental Evidence on the Role of Faculty Engagement.” (This article is forthcoming in the American Economic Association journal, Economic Policy.)

Show Notes

Transcript

John: Students from marginalized groups often question whether or not they should be in our classes and disciplines. In this episode, we discuss an easy to implement intervention that faculty can use to improve retention and student success.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Michal Kurlaender. Michal is a Chancellor’s Leadership Professor in the School of Education at UC Davis and is a co-Director of the California Education Lab. She is a co-author with Scott Carrell of a National Bureau of Economic Research working paper entitled “My Professor Cares: Experimental Evidence on the Role of Faculty Engagement.” This article is forthcoming in the American Economic Association Journal, Economic Policy. Welcome, Michal.

Michal: Thank you for having me.

John: Our teas today are:… Michal, are you drinking any tea?

Michal: I am not drinking tea, but I did have some not too long ago today.

Rebecca: Do you have a favorite?

Michal: I’ve come back to Earl Grey. I used to be an Earl Grey person. I left it for a while, and it’s just made a comeback for me.

Rebecca: Nice. It’s a classic. I have Christmas tea today, despite the fact that it’s June.

John: And I’m drinking a ginger peach black tea today.

Rebecca: I have my Christmas tea because we had our presidential announcement today. And it was like celebration tea.

John: White smoke has come out of the towers [LAUGHTER] and we have a new college president here.

Michal: Congratulations.

Rebecca: So we invited you here today to discuss the study of the impact of specific faculty behaviors on historically underrepresented minority student success. How did you decide on this specific intervention?

Michal: My colleague and collaborator Scott Carrell and I do a lot of work to try to understand College Access and Success. And in particular, we’re interested in understanding inequalities in graduation rates at more open access institutions, like the California State University system, which has, across its system, some more selective campuses and some more open access institutions. But in particular, what we’ve noticed for years is that the graduation rates of students of color, particularly male students of color, black and LatinX men, were really much lower than other groups. And this was a puzzle to us, largely because the eligibility to get into four-year colleges, including the CSUs is quite substantial. These are primarily B-plus students who have finished a comprehensive set of courses required to be eligible for the CSU. And so to see their graduation rates lag so much behind other students was really troubling to us. And so that’s why we decided to focus particularly on the CSU system. And we focused on one campus in particular, that’s a less selective CSU campus.

John: What was the intervention that you used?

Michal: We didn’t go in knowing what intervention to use, we actually started with a focus group with particularly men of color at this campus and asked them what their challenges were. In particular, what we learned was that their challenges were not necessarily social or more broadly campus level, they were primarily in the classroom, and that is they felt disconnected from their instructors and from what to do to be successful. These were all students who reported feeling quite successful in their high school, feeling quite connected to their high school instructors who encouraged them to go on to college. Many of them were in competitive fields like STEM and engineering. And then they felt like they really struggled in college, and in particular, how to seek help and how to understand what instructors wanted from them. And so we came in quite agnostic, I would say, about what could work, what is helpful here? Is it more writing centers, more coaching, more nudging? We didn’t know. And what we came out with is feeling quite sure that we needed to tackle the classroom. And in particular, I think what we wanted to think about was this untapped source of potential support or hindrance in that is faculty, and think historically, we know that many times we just think of the classroom as kind of untouchable, and we put other support centers, writing centers, and tutoring centers, and other supports for students. And we kind of leave the classroom alone and leave faculty, including ourselves… we’re both faculty… to do what they will. And instead, here we wanted to really think about, could we intervene with faculty to provide more support for students?

Rebecca: Yeah, it’s funny how we often overlook that particular option given that’s a key touchpoint with our students, right?

Michal: Exactly. So we came out deciding that we’re going to do an intervention that was classroom based, and that was going to try to work with faculty to give students more information about what it takes to be successful in the classroom, how to seek help. And then we decided to pilot that to see if the proof of concept worked. And we piloted it at a large, more selective institution on a small-scale pilot, and found some promising results and then launched it at full scale. This article describes that whole kind of research process as well, which we think is also an important contribution to the literature… to not just immediately do something, but to actually think about the way in which it might function and just really to understand from students what they tell us they need.

John: Your initial pilot used a light-touch intervention, could you describe that intervention?

Michal: So initially, in the pilot, what we did was a slightly underpowered pilot in the sense that we took students who didn’t complete their first homework assignment in a classroom where you have to complete a set number of homework assignments. And you could miss one, but historically, we knew that students who missed the first one often struggled in the class. And so that’s the point of randomization that we took for the pilot. That is we took those who didn’t submit their first homework assignment and to half of them we sent an email saying, “Hey, we noticed you did and submit your homework assignment. Just to remind you to do well on this course, here are some things that might be helpful.” We also provided some information on what’s coming up and reminding of the office hours and how to seek help. And then we did two others. Importantly, the two other emails provided information that showed the students that we knew how they were doing in the class so far. So after the midterm, and before the final, and we found again, it was underpowered, but we found positive effects among the treatment group. And then that was conditional on some other ex post characteristics that we added to the pilot, but then we decided to launch it at the CSU campus that we worked with at full scale across a random sample of introductory courses.

John: For those listeners who are not familiar with statistics, when you mentioned that this is an underpowered test, could you just explain that in terms that…

Rebecca: …Rebecca can understand? [LAUGHTER]

Michal: Absolutely. So they were underpowered statistically, to detect a statistically significant difference between the treated students, those who got the emails, and those who didn’t. And so for that, we need a large enough sample size of treated versus control students, particularly if we’re going to add other kinds of observations about them, like their gender, or their race, or their prior academic achievement. And so when we say something’s underpowered, we might see the positive effect that is better achievement in terms of the final or in satisfactory progress in the course. But that difference may not be large enough in statistical terms to consider it statistically significant, even if the mean differences are actually in the direction that you expect. So to get that, to be able to actually detect significant effects, you need a large enough sample size. And that’s when we launched into the population that we were specifically focused on, which was the population at this less selective campus.

Rebecca: At that less selective campus when you scaled things up, did you keep the intervention the same? Or did it change? What did that scale up look like?

Michal: Great question. So the first thing we did was we really focused on introductory courses. This was also piloted in an introductory course. But we wanted to focus on particularly large classes, especially because the information was going to come from the instructor and we were doing a randomized control trial, that is some students are going to get this treatment and others are not. So the class had to be large enough for it to not be weird that some students were getting it and others were not. If you’re in a classroom of 30, that might be strange if you’re talking to someone next to you, and they get this email from an instructor, but basically what we did is we recruited faculty, we randomly chose 30 large undergraduate courses. And then we recruited those faculty and said, “Will you be part of an experiment with us? …and here’s what you need to do.” And the important thing here is that we’re not trying to dramatically change faculty style in the classroom, we all have our own style, the own way we write a syllabus and what we expect from students. What we wanted, we had several key principles. The key principle first was that faculty need to directly communicate with the students showing them that they know who they are. So it very much said in an email, Dear Rebecca, or Dear John. They needed to provide information that was specific to their class, it couldn’t be quite generic. We provided them some templates, but the goal was for them to personalize them and say, “Here’s this upcoming unit, here’s what to look out for, here’s how I would study for it, Here are my office hours, and so it provided information. So the way we changed it is… it was a semester-long courses. and so we requested that faculty sent a minimum of three emails to students, one after the first assignment or exam, sometimes earlier if they didn’t have an exam, and two later, we wanted it within the first three weeks of the semester, one after the midterm, and one before the final. And the important thing is, in the second two emails, those were further personalized to sort of say, again, “Dear Rebecca, I see that you’ve gotten a 72 on your midterm, it’s not too late to improve your grade in this class. Here are the things that you could do. And so it was personalized also just showing that we know how you fared in this class. And so again, the goal was to let faculty, in their own words and in their own course formats, personalize these emails, with the principal being information to students, personalization to students, and help seeking behavior advice.

John: And this process is a personalization, was this done in a mail-merge type format? Because I would think to scale this intervention would be a lot easier if you did do it either using the tools within the LMS or using some type of a mail merge.

Michal: Great question. So again, this was a grant-funded study and where we could provide some support to faculty, some faculty didn’t need additional RA support from us and either knew how to do a mail merge, it really worked with their course management system, like Canvas or Blackboard and found it very efficient to work on the own. Others you may or may not be surprised, did request our help from our graduate students. And we did provide support including one actually helping a faculty member directly write individual emails for students to support. You’ll probably ask me how the faculty feel about this. And I will say we actually asked them how long it took. It didn’t take more than a minute an email and so we do kind of try to guesstimate the investment on the faculty’s time to do this, and it very much varies on their comfort level with the course management system.

Rebecca: In the scaled up version of this study, did you continue only interacting with students who had struggled and missed their first assignment or is that a shift from the pilot to the big study?

Michal: Yeah, thank you for catching that. No, it is a shift. We did open it up. We believed actually, theoretically, our priors were that anyone could benefit from this. So if you were a B or on the cusp, we have lots of researchers suggest students, especially in introductory courses, some students, particularly first gen students might take a B or a C as a signal that they shouldn’t be in a particular major. We really did want to encourage across the achievement distribution for everyone. As to John’s earlier question, as you scale this up, or as people have talked to me since this experiment said, what if I want to do this, but I teach 400 people, you could one year, one term, try it with your lower achievers, another term, try it with those at that C range, or others. And so we did in this initial intervention want to do it across the board.

John: How large was the sample on the scaled up version?

Michal: It was 20 faculty, some were in multiple classes, we had 22 classrooms overall, and roughly 3000 students.

John: Excellent. How large was the estimated effect in the scaled up version?

Michal: First, it’s important to note, as we’re talking about findings, that our findings are really concentrated on first- and second-year students or new students, and who are from underrepresented minority backgrounds, so URM students, and we find that their treatments are about five percentage points more likely to earn an A or a B in the course by comparison to control students. So again, just important to note, we find overall positive effects for the whole of treated students. But they’re only statistically significant for the URM students that we target, that our intervention aimed to focus on.

Rebecca: Was the impact limited to just the classes the students were in, or was there an effect beyond that individual class?

Michal: Yeah, so that’s a great question, and we do find what are called spillover effects. And that is that those same students, those URM students, had a positive effect of being in the treatment group, even in courses that were not part of the experiment. That positive effect was much smaller in magnitude, it was like three quarters the size of the effect of the actual treated course. But still statistically significant at the ,10 level.

Rebecca: it seems so easy, just three emails.

Michal: Yeah, it does. It takes an investment. But yeah, I think it does beg the question, I think, for me, and maybe this is something that you want to talk about a little bit later. But we do so many things to introduce first-gen students or to get students in the classroom and again to provide support externally, but we do tend to sort of assume that they’ll just survive or just be okay in the classroom without training faculty on how people might experience their classroom differently. And so, again, we do test for other subgroups. I focus on first-gen, because it’s a concept that is helpful for people to think about students who don’t have at home, at least, people to tell them what to expect in the college environment and how you might go to someone’s faculty office hours, and they’re not there, or they’re there, but they’re sort of like, “Yes, did you have a question about the material?” Whereas to know, you could go just to review the material or you don’t have to have a question, you can just show up, things like that. Being able to feel comfortable asking questions in class and others who don’t or just step in at the end of class and sort of say, I found this part of the lecture really complicated, will you be reviewing it again the next day? So things of that nature. And so I don’t think these emails did those, but they sort of remind us that there are things that faculty can do to remind students that they see them, that they see that they’re in a classroom, and that they know that they may be enjoying parts or struggling in parts, and that there are some actions that they can take to be more successful in their classroom.

John: Did the effect size vary with class size? Was there a more substantive effect in larger classes than smaller classes? Because I would think it might be easier for students to feel more lost in a larger section, especially as a first-gen student?

Michal: It’s a great question. We aren’t able to test that, our sizes were all pretty similar, and we didn’t have enough. We actually chose the largest classes that exist in this campus, which don’t get much bigger than 150. I think it is worth testing. Absolutely. I can tell you from the pilot, that, in particular, that was a class of 400 students, and both for the pilot and for the full scale up the types of emails we got, that faculty got back. So that’s one thing we could talk about is how did students respond more generally, and many students emailed instructors back, and in particular, in the pilot, but also in the scale up, we got those emails back from the faculty who were in our experiment, and they very much appreciated the email and said, “No faculty members ever emailed me before,” or “especially in a class of this size,” or “I so appreciate the email. I’m working really hard.” So the first and sort of overwhelming finding from these emails is just gratitude from students that a faculty member emailed them, particularly we noted that in the pilot when the class was quite so large.

Rebecca: Yeah, I can imagine that just something that feels personalized, whether or not it’s super personalized or not, just feels personalized, really helps students feel seen.

Michal: Exactly that. That’s right. I should also say in addition to just grades and you asked me about graduation outcomes, we also included a survey, both in our pilot and in the scale up, to try to get at some of the mechanisms and in particular, we asked questions like, “Did you feel this instructor supports your learning? Did you feel you could reach out to your instructor?” And we do find consistent with our intervention that students in the treatment group reported more positive outcomes on these dimensions.

Rebecca: That’s so fascinating, because it’s so easy. Like it just isn’t that complicated.

John: In your study, you also examined how this effect persists over time, which certainly relates to the graduation thing. What did you find in terms of the persistence of this effect over time,

Michal: We did look at long-run treatment effects. That is, we waited to see what happened several years later, we presented on this paper in the shorter term outcomes. And we tracked these students and worked with this institution closely, we really wanted to know did it affect the outcome we care most about, which is graduation? In other words, we care about student success in a particular classroom, and maybe they’re slightly better grade, or not dropping out of the course. But we really care about their longer term outcome of finishing. Again, for this specific group of interest. We note that the treatment results in that 7.3 percentage point increase in persistence, one semester later, and then a higher four percentage points difference in graduation. So we do find positive effects on the likelihood of graduation.

Rebecca: We’ve talked a bit about the impact on students and how students have responded, how did faculty respond to participating in this intervention?

Michal: First, I’ll just say, again, we had to recruit faculty to do this. And so we do track the faculty who said no, and we did as much reconnaissance work, if you will, to understand that we did need to self select faculty, keeping in mind that if we did self select faculty who just had a proclivity to help students or being this intervention, if anything, we perhaps under reported some of our findings, but we do as much effort to understand how representative our faculty are, which we did determine they are, and in the paper, there’s some details about that. And they represent faculty from a real diversity of disciplines, from music to engineering, across the board, humanities and social sciences, we had the whole range, we had the range of faculty. So you know, “I do this a little bit sometimes. And the two of you are, I would have said, I’d do the same with a student who doesn’t show up or doesn’t complete to sort of track them down. But I’ve never done it systematically. And I’ve always wondered if it even matters.” We had everything from that kind of faculty member to a faculty member that’s like, “Well, I’ll do this, but I don’t think it matters. I mean, at the end of the day, the students who want to put in the work put in the work.” And so we had a whole range in their attitudes. We did offer a stipend to do this, because we did believe it takes time. And we wanted to sort of show that faculty who do have a lot of demands, especially at teaching institutions, that this was going to take some time. And so we haven’t done it again without an incentive, or with an institutional incentive, that’s part of like performance evaluations or something. So that’s yet another thing that in terms of where to take this in terms of where institutions might take this, for our perspective, it was externally funded, they were only accountable to us in their efforts to do this. And so we talked to them multiple times in the term, again, some were in both waves of the study, because we did it over two terms. And then we surveyed them at the end and really got some details from them. Some of this is in the paper around how they felt. And I would say most expressed similar to what you expressed, Rebecca, which is like “Wow, this some emails and I made this difference, especially in underrepresented minority students lives and in their classroom, and it felt really good about the impact.” …keeping in mind, we talked to them also, before we knew the results. And so at that point, they just were sort of documenting how much time it took to do the emails, and what kind of emails they got in response from students and most felt, I would say, humbled by the thank yous that something so small, like an email, got so much gratitude back from students. We did have some faculty that sort of said, this takes a lot of time, and I’m not sure it’s much of a help. In our last survey with faculty, we actually provided them the full scale results and said, “Here’s, by the way, what we learned from the study,” and then asked them to respond or to reflect on that. And many said, “Wow,” like, again, similar to your reaction, “a few emails could make such a big difference, I will be sure to continue.” We ask them directly if they will and we report this in the paper. Most do say that now that they know the positive impacts of this study, they’re likely to continue with these emails.

Rebecca: I imagine the workload for a faculty member isn’t necessarily in drafting those initial emails, but maybe the responses to the emails [LAUGHTER] you might receive back.

Michal: It’s a good question. I don’t know how many continued the conversation once an email was sent, the standard emails, part of the experiment and the student wrote back and said, “Thank you so much.” I’m not sure they continued. We did a lot of qualitative coding, which we don’t report in the paper, we report some but then we did a lot more internally. And there definitely were a lot of hardships described among students who did reply, the extent to which faculty replied with those hardships offering extensions or any other kind of augmentation to their requirements, we don’t know.

John: In an introductory microeconomics course I use the Lumen Learning Waymaker package which actually does automate emails to students based on their performance on weekly quizzes and so forth. And even though the students in the class realized it was automated, they’d still write back to me and I’d respond to it. And they’d often apologize, saying, “I had a bad week and I know I need to work harder.” But it did often start a dialogue that might not have occurred otherwise. And I’ve often wondered how large that effect was, but because it’s automated for the whole class, it’s hard to measure the differential effect of that. So it’s nice to see this result, that that type of approach works.

Michal: Yeah, I think that what you’re describing is exactly right, this sort of feedback. Our whole intervention is built around theories, not just from kind of behavioral economics or nudging or information source, but on the education literature on feedback, and the important role of feedback, and the timing of it. And it’s most useful if it’s not just performative like feedback, like your grade on an assessment like a “C,” but that actually gives you more information about how you’re doing or what to do to improve. And so this kind of thing you’re describing John is exactly right. I think we know that more touch points with students through assessments, as opposed to all hanging on a midterm and final also support students to get more feedback about how they’re doing.

Rebecca:I think sometimes students know that they’re struggling, you get a grade back, you know, if you’re doing well or not, but I think a lot of students need more coaching around what to do to improve or to better understand how the grade is calculated, to just take the time and attention. It’s there. It’s in the syllabus. But sometimes they don’t realize what they should prioritize. And including some of that kind of messaging makes a lot of sense. I know that when I’ve done that with my students, they’ve been really appreciative because they didn’t realize that they were putting all their energy into something that didn’t really matter as much.

Michal: Yeah, that’s right. And I think coaching is a great choice of words around what to do with it. I also think many students come to our universities with really uneven or unequal preparation for those courses. And so I think a lot about students who came from a high school where they took an AP course in economics or chemistry that might as well have been a college level course. Many of these questions are great on a grade curve. And so that C might be an excellent grade for them, given the type of preparation that they had, but they don’t know that necessarily, and they might, to them, signal that maybe this isn’t the right major for them. And so I also think coaching around what to do with the grade when you’re kind of passionate about a subject and not to give up on yourself too quickly. Many are juggling jobs, we know for some, it’s their only work is to get through this term, and others are doing this while working and taking care of family members or whatever. And so that grade that they got often conveys information that we as faculty don’t necessarily know anything about how they’re interpreting,

John: I would think just a signal, as in the title of your paper, “my professor cares” might create a sense of connection and belonging that might otherwise be missing for someone who is a first-gen student who might not feel that they belong in the institution.

Michal: Yes, I agree. And I think there are more and more studies coming out, particularly in social psychology, but elsewhere about the importance of belonging. We know from the K-12 literature that it’s having a teacher who cares about you matters, actually. And again, nothing dramatic happens once you get to college. But we assume it’s completely different, where in fact, I think having an adult or particularly your instructor care, you feel a sense that that instructor cares about your learning, or how you’re doing in their class, irrespective of the grade per se, just that you’re making progress in the class or feeling comfortable in the class, I think is really important. And I think it’s hard to test. And most of the belonging literature has been on survey type research, “I feel like I belong here.” And it’s not as much in the classroom, although there’s increasingly more studies about belonging in the college classroom beyond just a university at large.

John: A while back, we interviewed Peter Arcidiacono, on a paper that looked at the impact of differential grading between STEM and non-STEM courses. One of the things you just said reminded me a little bit of that, because one of the things that was noted in that paper is that many of the people, particularly female and underrepresented minority students who switch their majors out of the STEM fields had some of the higher grades in the class, but it was below their expectations. And I’m wondering if this type of intervention might have an effect of letting them know that that performance in that discipline may not be all that bad. Since we’re probably not going to eliminate grading differentials between STEM and non-STEM disciplines, perhaps some type of personal communication might help preserve some of those connections so we don’t lose as many people in the STEM fields where the returns to education are the highest.

Michal: Absolutely, I think well said. That’s exactly right. And I think that is among the reasons we wanted to get across the grade distribution, not just those who are really, really struggling. And also because we do think that students might give too much meaning from a signal of a grade early on in their academic pursuits where they can get through a certain amount of courses and then maybe where the fun stuff of their major where they really see that utility of a particular course for a future career path matters. And so I think that’s right and I do think Peter’s work and other people’s work looking at the impacts of particular kind of grades and grading distributions or signals of grades, I think, are really important. I think that’s an area that’s blossoming in economics and in other fields to sort of better understand heterogeneity or differences between subgroups around how particular information like an assessment, grade or a test score.

John: One of the things we’ve been seeing in a lot of studies is that many changes in education, such as using more active learning techniques, providing more course structure, benefits all students but disproportionately benefits those students who are historically underrepresented. And it seems like this study just provides more evidence of that, that good teaching practices benefits everyone, but especially benefits the people who are most at risk in higher ed. And those are often the people who can get the greatest benefit by persisting in higher ed.

Michal: Yes, I think that’s exactly right, if we’re really going to address disparities in college outcomes, and I think one really important source to go to is the importance of information gaps. And that would be particularly for first-generation students, for students of color, but also for students who come from unequal K-12 backgrounds. Colleges and universities often know and often are recipients of systematically particular high schools in their states, especially public flagships, community colleges, others, and so they are aware, they offer relationships with those K-12 high schools that are feeders to their institutions. And that is an important source of information that they can provide to high school students as they enter college for a kind of a warmer handoff, if you will, but that also faculty teaching introductory courses can provide. And so I think, again, if our goal is to address inequalities that we see in college outcomes, then I think information, particularly for those for whom their information gaps, is particularly key. Students want to be seen and not necessarily yes, there’s the anonymity of a large lecture hall, that maybe don’t want to be called on. But that doesn’t mean you don’t want to know that your faculty member sees you and knows whether you’re doing well or struggling, or how you feel about the class or how to succeed on the next exam or in the next course in that sequence.

Rebecca: So if there’s other faculty who think, “Hmm, three emails, that seems easy,” What recommendations would you make to those faculty?

Michal: Yeah, I mean, I think what the first recommendation I would make is to try it, to do it. I think thinking about how you communicate in your syllabus that about your forms of communication are important. So if you’re going to do an email, I think one thing that we would have loved to test and if we were to continue further is the format for the information. And so I think letting also students know that you want to hear from them over email or through other means, I think is useful. So first, deciding on the kind of medium like how you’re going to communicate this, I think email makes sense. When faculty start texting students, maybe we’d move to a text them information. But that’s not the case for most of us, so it’s through the course management system or email. I would say focus on again, what we know from the literature on feedback is that for it to be as specific as possible about what students can do with this information, and so that is looking at your syllabus closely, knowing… we typically do as faculty… where students trip up in the material, what’s complex, what’s up ahead. And so giving that kind of feedback as well about how to prepare for the next assignment or exam, what has tripped people up in the past, what you know, might work for them is really helpful. Again, other research has suggested the importance of going to office hours might matter. But that means you need it to show up, you have to think about how you structure your office hours. Incidentally, we did try to track that… quite hard to do whether it actually promoted more office hours in the pilot, we believe it did promote more office hour usage. More broadly, it’s something we’d like to test, the actual help-seeking behavior of students. So I would say faculty should do it if you’re teaching a 400-person class and you can’t imagine doing this for 200 students or even 100 students, maybe start with students who you see as struggling based on that first assignment, as we did in the pilot and see what you can learn from that… maybe do as John suggests, which is kind of get savvy on a mail merge and think about ways to do this so it’s more efficiently done, so that you can reach as many students as possible.

John: We just switched recently to Desire to Learn’s Brightspace platform, and it does have intelligent agents and it does have replace strings so you can automate an email conditioned on the grade on either your overall course grade or on a specific item. And if you do it on overall course grade, (which I just set up, by the way, for my summer class last week), students get an email saying, “I see that you’re struggling, there are some things you might want to try.” It would be nice if I could put in their grade without having to go to mail merge, but I don’t think that would be possible. And I’d like to scale this up. In any case, I’d encourage them to contact me during my office hours or to make an appointment to talk to me. The first iteration of that went out last week. None of the students responded, but it’s a small summer class. So I’m curious to see how this might work. And your paper helped encourage me to try this. I had other reminders out there, but this was one that I thought might be useful, using a specific grade trigger.

Michal: That’s great.

John: We always end with the question. What’s next?

Michal: Well, I think our lab and Scott Carrell and I continue to do this work, and in particular, we’re also spending a lot of our time these days doing work at community colleges in California, which serves one in four community college students nationwide. So also persistence outcomes are quite weak at community colleges, historically, and we’ve seen real declines in enrollment at community colleges since the pandemic. And so we are definitely doing some work at community colleges. We continue to track and follow graduation rates, particularly inequality in graduation rates at CSU. And we’d love to launch another intervention. So stay tuned on that. I can tell you, we are quite committed to understanding the college classroom beyond college settings more generally, and so hope that the college classroom continues to be a source of important information for the field about how to better support student success,

John: You’re doing some really wonderful research. And it’s really nice to see some of the attention that this got because your article has been mentioned in The Chronicle. It’s been mentioned in Inside Higher Ed, and I’ve seen people tweeting about it ever since it came out. It’s good to see this research becoming popularized.

Michal: Well, we appreciate it, especially since Scott and I did not succeed on that front, that is becoming those people who do good social media. Other people are better at that than I am. And I’m always a little troubled when I talk to more junior faculty around “Do you need to do all that?” …and 10 years ago or so I would say “No, just do good work and it doesn’t matter.” And now I confess the sort of buzz that some people are better able to develop around their findings in their papers seems to matter and so it’s really nice when it happens to you because we didn’t do it ourselves. [LAUGHTER] So I appreciate my friends Sue Dynarski and others who’ve done a really nice job promoting this paper.

Rebecca: Well, thank you so much for joining us and for sharing your work with our audience.

Michal: My pleasure. Thanks for having me.

John: And we will include a link to your study in the show notes and we encourage our audience to read it.

Michal: Wonderful.

John: Thank you.

Michal: Thank you so much. I really appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

233. Guided Notetaking

Posted in equity, higher education, large-class instruction, learning, lecture, pedagogy, teaching
March 30, 2022

Many college classes contain a substantial lecture component, but our students arrive at college with little or no training in taking effective notes. In this episode, Tanya Martini joins us to discuss how guided note taking can be used to promote equity and student success. Tanya is a Professor of Psychology at Brock University in Ontario.

Show Notes

Transcript

John: Many college classes contain a substantial lecture component, but our students arrive at college with little or no training in taking effective notes. In this episode, we examine how guided note taking can be used to promote equity and student success.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Tanya Martini. Tanya is a Professor of Psychology at Brock University in Ontario. Welcome, Tanya.

Tanya: Thanks very much for having me.

John: Our teas today are… Tanya, are you drinking tea?

Tanya: I am drinking tea, yes. So, my tea is a pomegranate white.

Rebecca: Oh that sounds nice. Nice and light.

Tanya: Yeah. My friend was joking with me because I said I was doing this podcast about tea and teaching, and he said, “So, like, basically two of your favorite things in the world.” I said, “Pretty much!”

Rebecca: You’re in excellent company then. [LAUGHTER]

Tanya: Are you also drinking tea?

Rebecca: Always, always.

Tanya: Always.

Rebecca: Today I have English afternoon. I was in a rush, so I couldn’t get a fancy pot going.

John: And I’m on my fourth cup of tea for the day, and it’s a peppermint spearmint blend.

Tanya: Oh, that’s nice, that’s really nice. My mum was English, so we always drank tea when I was growing up. And then I met my husband when we were hiking in Wales. So we basically have a household where the kettle is never cold.

Rebecca: Yeah, my house is like that, too.

Tanya: Is it?

Rebecca: When John’s like, “Fourth cup?” I’m like… “Fourth pot? What are you talking about?” [LAUGHTER]

John: We have had three kettles today, because we’ve had a number of people come into the office earlier today.

Rebecca: We’ve invited you here today to discuss how you’ve been using guided note taking in your introductory psychology course. Can you give us a little background on the course first?

Tanya: Sure, it’s at Brock. So there’s 1500 students in the intro psych class, and about 220 or 230 of them would be psych majors in their first year. And the rest of them are taking it either as a requirement for another degree program like education or nursing, or they take it as a social sciences elective. So I would say about 85 or 90% of them are genuinely in their first year, they’ve just arrived from high school. We don’t have a lot of non-traditional students in the class, a few, but the majority of them are 17, 18 year olds, and they’re just making their transition from high school to university.

John: What’s the modality for this class? And how is it taught in terms of dealing with that many students in one class?

Tanya: Yeah, it’s a good question, because different people will handle that sort of volume of students in different ways. We made a really intentional choice even before I came on board that we wanted all psychology students to have the same experience. So rather than trying to split the students up into different groups taught by different instructors, what the instructors do is they split the weeks. So our course runs 12 and 12, like 12 weeks in the fall and then again in the winter. So it’s a 24 week, full-year course for us from September to April. And in the past I would say decade or so it’s three instructors, but it’s usually kind of two primary instructors and what we call a junior partner. And so the two, kind of, main instructors would teach 10 weeks each and then the junior partner would teach four. And it’s been kind of a good model. I started as the junior partner and it gives people a bit of a chance to decide… Is this a course that I could see myself moving into? Or is this really not something for me? Because it’s not for everybody to be in a course that’s that large. So in terms of structuring it, that’s how we do it. And it gives everybody a very consistent kind of experience. And then, like most Canadian universities the last couple of years, the big, big courses have been asynchronous online. So that’s what we did last year, what we’re doing currently this year. In normal times, it’s run as a three hour per week course, where two hours is allocated to a lecture. We don’t have enough seats in any space for all of those people. So what we do is we run the same lecture three times a week to 500 people, and we would do a two-hour lecture. And then we run 75 small group seminars every week, so 18 to 20 people usually. And those seminars are run by third and fourth year undergraduates. So we have, like, a peer mentorship model in the seminars. And it works actually pretty well, because what we find is that the first-year students, our impression, anyway, from the evaluations, is they like having other students leading those discussions. It makes them feel comfortable, and not just with the content, but I think they feel comfortable if they’re struggling with the transition. They feel as though this is kind of a senior mentor who can help them to find the resources that they need. But I really like the fact that our department and our faculty kind of puts the resources behind it because it’s quite resource intensive to run that many seminars in a course. It’s nice because it helps students to make the integration into university and help them cross that bridge from high school. Because they’ve got this one person who knows who they are, knows them by name, knows that if they go missing somebody will be checking in on them to make sure that they’re okay. And it also just helps them to meet people and make new friends in the first year. And then a few years ago, we got permission to have the teaching assistants, those third and fourth year students also take a course in facilitating good discussions. So that they’re not walking into the seminars feeling ill prepared. And it’s a great space for us to teach them some transferable skills that are very applicable to the workplace. So it’s kind of a nice model. Very resource intensive, and I’m grateful for that, but it works really well.

Rebecca: Can you talk a little bit more about the seminar for your TAs?

Tanya: Yeah, sure. It’s a full-year course. So it runs basically in parallel to the first-year course. And what we do is, I really try to model it as… it’s not a kind of “how to TA intro psych course,” I really try to sell it and emphasize its applicability to the world more broadly. So we tackle a number of different topics that are relevant to those seminars, but are also relevant to the working world, too. So we talk about diversity in groups. And so if you’re facilitating conversations in groups that are diverse, what are some of the advantages to that? And what are some of the challenges? And how do you overcome those kinds of challenges? How do you manage the balance of the discussion so that you don’t have decisions being made based on the people who talk the most or who talk the loudest? How do you get over those kinds of challenges? We talk about active listening and what that means, and we talk about cultivating a level of self awareness so that if you start to feel kind of anxious or upset, you recognize that in yourself, and you know what to do to bring it down. We talk about… How do you facilitate things when it starts to get challenging and heated? What kinds of things can you do? And then in the second semester, we move into talking about things, like, How do you give a good presentation? So if we move away from the facilitation skills, we talk about good multimedia presentations, the students do the equivalent of a 15 minute TED Talk. So it’s all meant to be very applicable to a broad array of settings. And most students find that it’s a great community, because we hire 18 students every year to fulfill the seminar leader role. And we tend to find that they coalesce into a really nice, tight community, they’re very supportive of one another. So you get a great exchange of ideas and lots of support, if you’re feeling anxious, or things aren’t going well, or whatever. So that course runs separately, but it’s a really nice counterpoint to the very, very large first year course. So I love having the big, big class on the one hand, but then also this smaller class where I really get to know those 18 TAs really well.

Rebecca: In our notes for this episode related to this course, one of the things that you talk about is how diversity doesn’t necessarily promote inclusion, or the other way around. Can you expand on that a little bit more? It totally caught my attention. I was, like, I want to know more about that!

Tanya: Yeah, it’s such a great point. So I have the luxury of being on sabbatical this year for the first time in eight years, and it’s such a great gift. And one of the things that I really wanted to do was, to do more reading about diversity and inclusion and thinking a little bit about… How do we incorporate that into the syllabus? And what does it mean for our seminars? Are we doing the best job that we can be doing? And I have been taking, like, a number of workshops, and I went to this great workshop being run by an organization that promotes diverse viewpoints. And the woman who was running it is from an organization in the States called The Village Square, where they purposefully bring people together who are on both sides of the political divide, I think, in that case, left and right. And I had a conversation with her, and she’s, like, taking this really great course, this… it’s a MOOC basically, Massive Open Online Course, and she said, “I think you would really like it. It’s all about bridging differences.” And so that comment about you can have diversity without any inclusion comes from this course that I’m taking. And they just talk about how you can have diversity, like, you can purposefully hire or take on people who look different from one another, or different religious backgrounds, political backgrounds. But unless everybody feels comfortable articulating their opinions and talking about where they’re at, unless they all feel comfortable with one another, you don’t really get true inclusion. Because inclusion sort of necessitates that not just you have a roomful of diverse people, but that all of those people are contributing to the conversation. And I thought that that was a really valuable and important point to take away. Because although I think, of course, you have to make some effort to bring those diverse people into the room, the job doesn’t stop there. Because at that point then, you have the sometimes challenging job of getting all those people to a place where they all feel as though their opinion matters, and they’re comfortable articulating it. And they go on to talk a little bit about how in some cultures, for example, there’s much greater value placed on listening rather than talking all the time. And so you have to work with that and work around that and try to draw people in so that the discussion is somewhat balanced. And at that point you’ve kind of done a better job of reaching inclusivity, as opposed to just diversity.

John: This is a really great set of skills you’re providing the students, who no matter what they do in the future are going to be working with diverse groups, as well as having to do presentations in some form or another. You mentioned hiring these students, what type of compensation do they receive? Are they paid for this work? Or are they receiving credit for it as a result of the courses they take?

Tanya: Yeah, so that’s another great question. And it’s a model that has changed over time. So at Brock, and at a number of schools in Ontario, teaching assistants are unionized. And for a long time this was a unionized position, they would get paid for something like 300 hours a year. And then, like this won’t surprise you at all, we went through financially difficult times as budgets were shrinking. And there was a period in the last decade where enrollments were shrinking just because of the demographics of Ontario. There was a much smaller pool of 17-year-olds in high school, and we were all competing for those people. And I was really only the junior partner at that time. But at that time, the Dean really started to make noises about the amount of money that it cost to mount this course. And so what we decided to do was to implement this strategy where we would make it a hybrid. And what we did was we rolled some of the things that used to be paid tasks into a course. So on the one hand, it would be disingenuous of me to say this was all driven by pedagogy, because it was driven to a large extent by budget. So what we did was we created this course, and about one third of the hours, we rolled into course related stuff. So a lot of the seminar preparation material, where we would pay them for that before it kind of became part of the course and talking through the seminar preparation with your colleagues and me giving some scaffolding around that. So now they get paid for about 200 hours. And the rest of it is rolled into this full-year course from September to April. So they get course credit for that. What it does mean, though, just from a logistical point of view is we used to have a model where people could do the course a number of times, they could TA the course a number of times. In fact, they were more likely to because they would get seniority in the union. But now you can only TA when you’re taking this course, and you can only take it once. So we don’t have the continuity we used to have, but we get a fresh group of 18 people. And I try to look at it as the upside of that is we can deliver these skills to more people because it’s a unique cohort every single year.

Rebecca: Are most of the courses in your college the two semesters back-to-back, like full-year courses?

Tanya: No. In fact, it’s kind of a rarity. Almost all of them are a half-year course, so 12 weeks, from either September to December or January to April. Intro psych is one and then we have a research methods course that’s a full year. But, in general, the guiding principle has been one of breadth. But I think what we have always felt is that a full year for first year is a good idea. Not so much because it’s such an important year for us and there’s so much to deliver. I mean, there is, but I think just in terms of integrating students into the department and to the university, and again, helping them to make that transition, giving them the full year course is really helpful in terms of allowing time and space for that to happen.

John: We saw an article in The Chronicle recently describing your use of guided note taking and it seemed really effective. Could you describe how guided note taking has been used in your classes and how your use of it has evolved?

Tanya: Basically in terms of historical context, what we find, and this won’t surprise anybody, in a class of 1500 people, you get everything from A to Zed. So we have students who are amazingly well prepared for university, and they know how to take notes, and they know how to read dense text, and they’re very skilled and savvy about taking exams, and they have great study strategies. And then on the other end, we have students who are quite ill prepared. And sometimes not even ill prepared, but maybe they’ve been out of school for a long time, and so those skills are kind of rusty. And so retention is an issue for us. We’re always trying to make sure that we’re a course where we’re bringing everybody up to the same place. By the end of the 24 weeks we’re trying to get people ready so that everybody is kind of on a level playing field. And when I joined the course, laptops were just starting to be a big thing. And what we were doing at that time, and this predates my involvement in the course, is we were supplying notes to students. But they were the PowerPoint notes. So you print the PowerPoint notes, you’ve got the slide, and they’ve got a bunch of lines on the side, and students would make these notes. And we would supply these paper notes for a fee, and they would buy them at the bookstore bundled up. And then they could just put them in a binder, and bring them to the course. And so some of the students, as I was coming into the course, were just kind of ignoring them and they were taking notes on a laptop, other people were taking notes using these paper copies. So it was kind of quite a mix at that time. But one of the things that really started to become clear to me was students really weren’t sure what they should be writing down. And my slides are somewhat sparse. Yeah, I’ll have the main points on there. And students would come up during the break, and they’d say, “Well, I kind of missed this point.” And sometimes I would be looking and it was just full of dense text, like, they were trying to write down verbatim everything I said. And then some of them would come and they hadn’t written down anything because they had the picture of the slide there, and they assumed that if it wasn’t on the slide it probably wasn’t worth talking about or remembering. So it really struck me how there was so much diversity in what people interpreted to be good note taking. And I was talking to my colleague about that, and I said, “Sure,” that this is really a good thing. And so that was my first sense that maybe we needed to do something and talk more explicitly about note taking. And could we do something that would scaffold their note taking? And so that was where the first iteration of our guided notes came from. And basically, the first iteration was just, like, a hierarchical overview of, here’s what the lecture looks like from the top, here’s the three main points or the four main points, and here are the subpoints. Just like on a PowerPoint slide, you get the main points and the subpoints. So that they could kind of see at a glance, these are the big things. And then what the guided notes looked like was almost like a series of small exam questions, because that’s really what the lecture is, it’s the answers to a number of different questions. And so if I was talking about, say, four things that determine whether you will pay attention to something in the environment, the question in the guided notes would be: “Name and describe four things that determine whether you pay attention to something in the environment.” And might even give them a table to fill in. And so they started to see the lecture as providing answers to a number of questions, some of which are connected, and giving them clearer space in which to put it in. And then we had students in that first iteration commenting on the fact that they didn’t want to take notes on a computer, or they didn’t have a computer. And so we started creating a digital copy and a PDF copy with spaces, and they could either write with their pen or they could type on a computer. And that’s what we did for four or five years, that’s kind of what it looked like. And then one of the things that I wanted to do during this sabbatical was just think a little bit more about them, because I started to think it would be a good idea to have something similar for the text. So I co-authored the textbook that we use, and I started to think, you know, it’d be really good to have something comparable to the text. Because in as much as they struggle to take notes during lectures, sometimes I think they struggle just knowing what to extract from the textbook. There’s so much material there, and sometimes they come in with all the highlighting. Okay but, [LAUGHTER] like, it’s the whole page.

Rebecca: It’s a rainbow page. [LAUGHTER]

Tanya: I know, the rainbow page! And so it was sort of clear that sometimes we’re struggling to extract what the central stuff was. And rarely did I see people making marginal notes or anything like that. And so I started to think that would be helpful. So that was one of my sabbatical tasks. So I went into the note taking literature a little more deeply, and I started to create them, and they’re a little more in keeping with the Cornell Notes. So they take the form of the Cornell Notes, and they are a little more sophisticated. And that was what prompted the rework of the lecture notes that they commented on in The Chronicle. So they look more like Cornell Notes now with the features of Cornell Notes. And the other thing, it’s such a great blessing that a sabbatical is, I was reading about diversity and inclusion, and went back into some texts that I hadn’t read for a while, like Dan Willingham’s book, Why Don’t Students Like School? And he talks about the fact that, one of the things that you could do and might be advantageous to do is to sort of frame the lecture around a series of questions and make them explicit. And so it really pushed me… this whole reformulation of the lecture notes has really pressed me to think about my lectures a lot more carefully. And so what I started to do is think… Okay, well, if I was going to distill this section on attention into, like, six or eight main questions, what are they? Because Willingham’s contention is, instead of it being like this steady stream of information just being thrown at you, if you can kind of organize it into a series of questions that are somewhat interesting, that makes things, first of all, organized a little bit better in your head, but also just a little more interesting to listen to. So the guided notes now kind of look like Cornell Notes. And they’ve got this accompanying, what I call “the roadmap,” which is basically just a graphic organizer. I did a lot of reading about graphic organizers. It sort of shows, these are the main points, these are the main questions. And then as we’re answering this question, we’re tackling this study and this study. And you’re not meant to write on the roadmap, it just gives you a visual of… here’s where we are in the grand scheme of things, here’s where we are in this particular unit.

Rebecca: For those that aren’t familiar, can you talk a little bit about what Cornell Notes are?

Tanya: Sure, yeah, I would love to talk about Cornell Notes. And there are obviously slight variations, but the way Cornell Notes are set up is that if you’re thinking about a page, there’s two main columns, usually. And in the left-hand column, in my notes, I list the main questions or the main points for my textbook notes. It’s like, here are the learning objectives for this chapter. And then the students’ written notes are on the right-hand side. And I scaffold those by putting in the sort of smaller questions that were always in the guided notes. Like, name and describe the four things that guide whether you’ll pay attention to something in the environment, is there. But what I do is also to include, and sometimes it’s structured differently, but I’ll section on, like, What are your questions about this? So that students have a space to write down if something’s going by, and they know that they haven’t understood it, it’s a space where they can very clearly plant a flag and say: “I need to go and check on this,” or “I need to ask the instructor, or I need to go to office hours, I need to talk to my TA.” So there’s a very clear space there for… What questions do you have? And I always frame it that way, too, because I think it’s a subtle difference. But when you say, “What questions do you have?” You’re basically saying, “It’s totally normal to have questions. So what are they?” So our Cornell Notes look like that. They’ve got a section for here are the main questions over here on the left-hand side, and as we’re answering that question, here are the things we’re going to cover. And you can make your notes on that right-hand column. And then before we move on to another question… What questions do you have about this? What do you need to check on?

John: You mentioned at the start of this, the wide diversity in students’ note taking skills, and so the students who are going to do well would probably do well, anyway. But I would think that this would benefit some of the students who would come in with weaker skills, and they are now able to focus on the things that are much more relevant and important for learning the course content. As well as it’s providing them with some clues about how it all fits together with the questions and with the structure you provide. Has this reduced some of the equity gaps in your classes?

Tanya: Yeah, I think that’s a great question. And the answer is, I don’t know, because I don’t have data. But one of the things that I found really interesting, and I was saying I did a lot of reading about note taking and so on, and I hadn’t really thought about guided notes as being an equity issue. But Viji Sathy and Kelly Hogan, who are both in North Carolina, have written extensively about how providing structure of this kind is incredibly beneficial in terms of students who have come from usually some disadvantage and have less preparation, or their preparation for university hasn’t been as strong. And so I don’t have any data, but certainly Kelly Hogan in her intro biology class, and I really admired the fact that she actually dug around in the data. And what she found was that, quite unbeknownst to her, the number of Ds and Fs was not spread equally across different groups. And so she found that people who were Caucasian ancestry, like they were getting a relatively small proportion of Ds and Fs. It was slightly higher among her Latino and Latina students. And then it was really quite high among the African American students. And I found that quite surprising too, but certainly she’s got data to support the idea that providing this kind of structure is very beneficial in terms of diminishing those kinds of gaps. But what I will say, and I’m not sure if this would resonate with other people, one of the things that has always concerned me a little is when you provide this kind of structure in a first year course, and then you send them off into the world in second year, with me knowing that none of my colleagues are providing these kinds of support… Are you in fact just pushing off the retention issue and all the problems into second year? And so one of the things that I think is really important is not just creating these guided notes and the scaffolding, but really talking very explicitly about: If you find this helpful, let’s talk about why it’s helpful. And let’s talk about how you can transfer that skill into another course where they’re not necessarily going to give you guided notes for your textbook and for your lecture. So what are these things doing for you? How are they scaffolding? And I think having that explicit conversation is really important. Otherwise, I feel like we run the risk of just pulling the rug from under them, but doing it a little bit later on. So one of the things we talk about is, How do you decide what’s really important? So if it’s in a textbook, being cued into what’s in bold type, and what’s in italics, and so on. But I also try to kind of just get back to the structure issue. Because it’s interesting how you teach for a long time, and it seems so obvious, but it’s not obvious to them. So it’s like, if I give you this chapter, why don’t you just try mapping out the headers and the subheaders? So that you get some sense of how this material is organized. And you’ll see in chapter eight, there’s like four big topics, that’s the big text, the main A headers, and then there might be two subheaders under this one. And then if you kind of create that map for yourself, it gives you the big overall picture. And one of the benefits of teaching intro psych is we talk a lot about memory. And how is memory structured? And what do you do to facilitate memory? And we talk about how memory is organized in this kind of way, and hierarchies are important and useful. And so I’m really trying hard not just to provide the scaffolding, but to help them to understand why the scaffolding works. So that in second year, if somebody has dismantled it and the scaffolding isn’t there, at least you have some sense of, “Okay, how can I recreate that effect for myself?” And I think that that’s something that is still kind of a work in progress. I don’t know if I do that well enough yet, or enough yet. But I think that that’s a really important component of providing the support in the first place.

John: And that’s something I think that most students have not been exposed to along the way. So helping them develop those types of skills would be really helpful, I think. And we should probably mention that Viji Sathy and Kelly Hogan have a book coming out at some point in the near future, and I’m very much looking forward to reading that.

Tanya: Me too.

Rebecca: I’m sure many of our listeners will be. We have it on our radar, like the second it comes out, we’re going to know. [LAUGHTER] Have your students responded to this scaffolding that you’re providing through note taking? Have they given you some informal feedback that you could share?

Tanya: Yeah, so, so we’ve had informal feedback verbally, and sometimes it’s solicited and sometimes it’s not. So if students come to my office hours, and they bring their notes, I don’t hesitate to ask them, “How’s it working for you? Does it matter to you?” Occasionally students will tell me they don’t use the notes, but they will refer to the notes before a test. And sometimes they tell me that it feels like it’s the only thing that is keeping them going through the lectures, which sometimes does feel like it’s quite a lot to take on board. But on the course evaluations, we see a number of people… because sometimes I asked about it explicitly on the course evals, and what we get is uniformly positive feedback. Sometimes it’s quite effusive, and sometimes it’s not, but we don’t get people saying, “This is terrible, don’t do this again.” Even if they’re not using them, they genuinely appreciate the fact that we think about the fact that this is sometimes a skill that’s challenging, and that we’re trying to support them as best we can. So yeah, the informal feedback and the formal feedback has been uniformly positive. The question of the data is an important one, and I don’t have any data to supply. But I kind of lean on the fact that Kelly Hogan’s work, to me, is pretty compelling if you can move students in that way. That’s a big deal to me, because, as I said, we have a lot of traditional students. They’re 17 and 18-years-old, but they come from such different places, and the extent to which high school has prepared them well varies a lot. And I’m not sure what your experience has been like, but I think the pandemic has just amplified that a lot. The kinds of experiences they’ve had, the extent to which they’ve been in the class versus not in the class, the extent to which teachers have had the resources to rise to the occasion, or not, has really created significant inequities, I think, in high schools in Ontario. I don’t doubt that everybody’s doing the best job they can, but they don’t always have the same resources to work with. So I think we’ll only see more of that in the future as they move through high school and arrive on our doorstep.

Rebecca: I think tools like the note taking that you’re talking about, are just really great ways to help students filter out the mass amount of noise in an already very noisy world. And it seems particularly noisy these days. [LAUGHTER]

Tanya: Yes.

Rebecca: There’s so much going on, that helping provide that structure to really narrow focus seems really helpful. I know that my students have been certainly reporting that there’s just so many distractions, and things that are occupying their minds, that the ability to have structure like this in place can be really helpful to getting through a class, or being successful in a class in ways that maybe if that structure wasn’t there wouldn’t happen at this moment in time. Maybe it would happen in a different moment in time, but not at this moment in time.

Tanya: No, I totally agree with you. I think that that’s true. And I also think that, I’m not sure about your experience, but my experience is, even in the normal times, many of them are not accustomed to sort of sitting through it, our lecture. I do try to break it up quite a bit, actually, so that we’re trying different class activities. Or I’ll do something where, in a class of 500, I can divide them into groups, and we try to replicate an experiment. And like, “You’re going to be the group that has these instructions, and you close your eyes now, and you guys will have these instructions, and then…” So we’re always trying to mix it up a little bit. But sitting for two hours and trying to take stuff on board, it’s a lot. And I’ve been doing different things on this sabbatical, some of which have involved me sitting for hours, and struggling by the end of it.

Rebecca: I love that you’re taking classes on sabbatical, it was something that I did when I was on sabbatical as well. It’s a nice reminder of what it’s like to be a beginner in things again.

Tanya: It’s totally true. Especially, I think, the last couple of years. Doesn’t it feel like there hasn’t been two minutes to spare to do anything that elevates your teaching? I was trying to really struggle just to get through, and you know we were pivoting this really big course online, and suddenly all the assignments are different, and no in-person exams. It just felt like… it was so intense. And so this has been just such a great gap to kind of go, “I have time to do some reading,” and “I have time to take this course that I think will be beneficial to me, and it might be beneficial to my students.” So yeah, it’s been so amazing [LAUGHTER] just to be able to remember what it’s like to go learn some new things and get excited about that. I also started taking karate lessons, right? Which I thought, this is the best example of why you have to really have a lot of patience with beginners. Because I’m now at that stage where I’m a complete beginner myself, and hopelessly uncoordinated, and there’s all these black belts who are incredibly patient with me. But it’s sort of like, “Oh yeah,” everybody who teaches first year students should actually go do something brand new every now and again and remember what it was like to feel like you don’t know anything, and you’re not really very good at it. So that’s been a good reminder, too.

Rebecca: I love it.

John: When you were talking about the course for the teaching assistants, you emphasized how the skills they’re learning are going to be useful in the rest of their lives. Since most of the students in your psychology class are not going to become psych majors, and I think many of us have experienced challenges in getting students engaged in content in gen-ed classes. What type of strategies do you use to help emphasize the relevance of the skills they’re acquiring or the knowledge they’re acquiring in your class?

Tanya: Yeah. So that’s been kind of the focus of our research for a number of years now, and it was born of just some of those lightbulb moments. I’m sure you guys have had them too, where you have these experiences, like, “Ohhhh!” So I’m just going to describe one of them to you. So it started to become clear to me… now I’m going back probably eight or nine years, where I was having a good discussion with a student who had worked as a TA for me, she had worked as a research assistant for me. She was one of these young women who just had it all going on, like really smart, and really thoughtful, and a good thinker. And we were having this conversation as she was getting ready to apply to graduate school, and she was putting her application stuff together, her letter of intent. And we got to talking and, just to paraphrase, she was basically communicating that she really didn’t know, she spent four years and tens of thousands of dollars, and she really didn’t know how to articulate or leverage what she had been learning. So even though I had zero doubt that she had all these skills, she didn’t know how to talk about them, and she didn’t know how to talk to somebody else and persuade them that she had skills. And I suddenly started to think, that’s a huge problem. [LAUGHTER] That’s a really, really big problem. And that led to some of my early research, where I started to look at that question among our site undergrads at Brock, and it wasn’t an anomaly at all. And what I found was that students, when you ask them, “What do you think you’re getting out of these assignments that they’re giving you? Most of them would say, “Well, the instructor wants me to learn about topic X. They want me to learn about the content.” And very rarely did they ever say, “They want me to improve my communication skills,” or, “They want me to improve my teamwork skills,” or whatever. And so for me, that was really telling. And so I started just trying to be a lot more intentional about being explicit with students about the skills. And so in PSYC 1F90, what the main project that they do is, they would gather some data as a group, usually on some really famous study that works pretty well consistently, and they would gather data from themselves as participants. And I would get the data from the TAs, and I would combine it, and I would analyze, it’s very simple analysis. And then they would take the findings, and they would write it up like a mini psychological paper with an introduction, method, results, and discussion. And I would talk a little bit about that, and I would talk about the fact that it builds communication skills and writing skills. And so I think that I’m doing well, and then I had my second lightbulb moment when I had a history major come in. And we were talking about the assignment, and she was very polite, like really polite, but kind of candid. And she basically said she thought the assignment was pointless. And I said, “I was talking in lecture, it builds communication skills.” And she said, and again I’m paraphrasing, “Well, maybe that’s the way psychologists communicate, but that’s not the way historians communicate.” And I thought, that’s really interesting. [LAUGHTER] Because suddenly it became clear that she was kind of focused on the superficial aspects, like, it’s got our introduction, method, results, and discussion, and that’s not what historians do. And I suddenly realized it’s not actually enough to say, “Well it builds communication skills.” You actually have to talk about how there’s some transferability of communication from one discipline to another, right? So, absolutely true, historians probably write papers that look quite different. But let’s talk about the fact that all communication—written, verbal, whatever—you have to think, first of all that, What’s the main thing you want to talk about? So what are the main points you want to make? And what are the subpoints? And how are you going to order those points? What’s the logical flow of information that you want to present? And then you need to think about your audience and think about their characteristics. Do you need to persuade them? Or are you going to be preaching to the choir? What do they know in advance? What background do you need? All these things, it doesn’t matter, I said in an essay that I wrote for the Times Higher Education, it doesn’t matter whether you’re like a cop giving testimony to a jury, or whether you’re writing a psychology paper, or whether you’re given a TED talk, or whether you’re writing a history paper, or whether you’re writing an auditing report for Price Waterhouse. Those things matter cutting across different disciplines. And I never realized that it was really important to actually get to that level of detail in explaining the skills to get buy-in, but that’s what I do now. We’re doing a different project going forward, and I’ve got a whole section of the instruction sheet. It’s like, How can you leverage the skills from this assignment in a job interview? And I don’t know if they’ll read it, but I’ll be talking about it in class anyway. And so this is where I’ve arrived in terms of students and skills. And I know some people will probably say, “Wow, you spend a lot of time in class talking about skills,” but I actually think it’s really an important element of what we’re doing. And I think, left to their own devices, it’s not always obvious to students that this is going on, that you’re acquiring this, as well as some knowledge of “what are the factors that influence whether you pay attention to something in the environment.”

John: Faculty often complain about the silos that we created within our discipline, and we forget that students create their own silos where they think of each thing as something entirely separate from the rest of their academic career. And if we can make it clear to students that the material they’re learning is going to be useful for them, they’re going to be much more engaged and much more interested and enjoy it a lot more, I would think.

Tanya: Yeah, I think it’s really important in terms of buy-in, I really do. And you’re absolutely right about the silos, and I think I had been quite naive to that. I think I really didn’t realize it until this wonderful young woman took the time to articulate it to me, and that was incredibly helpful. So you’re right.

Rebecca: While you’re talking about this, Tanya, I’m also thinking about how often I experience students focusing in on the thing that seems newest to them, the thing that’s most unfamiliar. And that’s what they get hung up on, and they think the entire class is that one thing that they don’t understand. And then the things that seem familiar—like writing seems familiar, we’ve written before—that’s the thing they pay the least attention to, and it’s often some of the things that we want to pay the most attention to. And so it’s funny how the newness, or our fear of not knowing how to do something, can put our attention there, and forget about the details of other things that also have a learning value to them that aren’t always recognized.

Tanya: Yeah, and I think you’re right about that. I have to constantly remind myself that they might not see that. It’s very obvious, but they might not see that. And to make it explicit, and to take time to talk about it, and answer questions about it. And that was something that I didn’t do nearly enough of in the early days of my teaching. Like, I really just thought, I don’t know, I thought if it was obvious to me it should be obvious to you. [LAUGHTER] But students over the years in articulating some of these things have really crystallized for me how important it is to be explicit. And for sure, some of them absolutely do get it, they don’t need me to explain it. But I come back to the variability in a class of 1500, and even really, I mean, I don’t even think that’s totally wrong for upper-year courses. Sometimes students don’t necessarily always see things that we think that they will. It helps me to think to myself… Okay, what’s the most important thing here, in terms of content but also in terms of skills? And then let’s make sure that I talk about that, and talk about how it moves from this particular course and this particular assignment. Like, can you see how you might use it out there in the world? And I usually try to talk in terms of multiple examples, like my students are going into law enforcement, they’re going into business, they’re going into nursing. So I try to draw from a few different disciplines, just to give this sense of… Yep, it transfers over here, and it transfers over there, and it transfers over here, just so they get in the habit of thinking about that.

John: We always end with the question, What’s next?

Tanya: What’s next for me is, I’m still working on the diversity and inclusion piece, and so that’s taking up lots of the space between my ears. Trying to think more about… How, in a class that is so diverse, how do you create the kind of inclusivity that we’re looking for? Recognizing that students don’t participate for lots of different reasons, and we want to kind of take them gently into the water in some cases. So I was mentioning that we’re moving away from that old assignment where they would write kind of a mini psychological paper. And then what I’m trying to get them to do is to work on a persuasive message, and they’re going to present it in a couple of different ways, so they choose a topic they agree or disagree with. And then what I’m working on is… It’s got an element where they have to find some sources to support them, and there’s an information literacy component. So I’m doing a lot of research on that, and how do you promote good information literacy at a time when information is just boundless, and some of it’s good and some of it’s not good? And then they have to present it in a screencast. So we talk to them about good multimedia, and they do a very short screencast with their preliminary ideas. And then they have to write an op-ed. And so one of the things I’ve been dialoguing with my co-instructors about is, just lots of students are incredibly nervous about putting a screencast together. I try to talk a little bit about how this is kind of a gentle introduction to presentations. So we’re not going to ask you to give a flawless presentation in front of 20 people or more, and if you don’t like your first recording, you can try re-recording. So I acknowledge that there are people who have significant anxiety, for example, and so on that diversity, on a side of that kind of diversity, trying to create an assignment that will gently lead them into improving their sense of confidence about their ability to present. And looking at the seminars, and asking myself, How can I draw on this Massive Open Online Course? Which is all about, How do you facilitate good communication among people who don’t agree? How can I bring that into the TA course and help them to inspire really good, inclusive conversations among the first-year students? So I’m thinking about inclusivity in a really broad kind of way. I’m thinking about it on the side of my assignments, I’m thinking about reworking my syllabus. I’m thinking about… How do we bring it into the seminars? In more than just, “Well I’m going to introduce you to some African Canadian scholars,” or “I’m going to draw in people who are diverse into the course.” It’s more about, how do we get diverse participation? And the sense from students that we’re all part of this community, and we’re all going to contribute to this community that is the intro psych.

Rebecca: Sounds like you’ve got a lot on the horizon for sure.

Tanya: Yeah, it’s an exciting time, it really is. It feels like it’s a time where you just have an opportunity to do some really good work, and I’m going to really try to capitalize on it.

John: Well, thank you. It was really enjoyable talking to you and you’ve given us a lot to think about.

Tanya: Thank you very much. It’s been such a pleasure to speak with you today.

Rebecca: Thanks, Tanya.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle, Annalyn Smith, and Joshua Vega.

[MUSIC]