84. Barriers to Active Learning

Despite research demonstrating the efficacy of active learning approaches, observations of classroom instruction show limited use. In this episode, Lindsay Wheeler and Hannah Sturtevant join us to explore potential interventions to overcome the barriers to the adoption of effective teaching practices.

Lindsay is the Assistant Director of STEM education initiatives at the UVA Center for Teaching Excellence and an assistant professor. Lindsay’s background is in chemistry and she has a PhD in science education. Hannah’s a postdoctoral research associate at the center. Her PhD is in chemistry with an emphasis on chemical education.

Show Notes

  • Henderson, C., & Dancy, M. H. (2007). Barriers to the use of research-based instructional strategies: The influence of both individual and situational characteristics. Physical Review Special Topics-Physics Education Research, 3(2), 020102.
  • University of Virginia programs
  • Teach Better Podcast Episode 80
  • Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE Life Sciences Education, 12(4), 618–627.
  • POGIL- Process Oriented Guided Inquiry Learning
  • PODLive! Webinar
  • Meghan Bathgate — Postdoctoral associate at Yale University
  • Emily Walter — Assistant professor of Biology at California State University, Fresno

Transcript

Rebecca: Despite research demonstrating the efficacy of active learning approaches, observations of classroom instruction show limited use. In this episode, we explore potential interventions to overcome the barriers to the adoption of effective teaching practices.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guests today are Doctors Lindsay Wheeler and Hannah Sturtevant. Lindsay is the Assistant Director of STEM Education Initiatives at the UVA Center for Teaching Excellence and an assistant professor. Lindsay’s background is in chemistry and she has a PhD in science education. Hannah’s a postdoctoral research associate at the center. Her PhD is in chemistry with an emphasis in chemical education. Welcome, Lindsay and Hannah.

John: Welcome.

Hannah: Thank you.

Lindsay: Thank you.

John:   Our teas today are…

Hannah:  I have a lemon filled Earl Grey tea. [LAUGHTER]

Lindsay: I have my water.

Rebecca: I’m drinking English Afternoon.

John: And I have Blueberry Green tea. We’ve invited you here to discuss the study you’ve done on why STEM faculty are reluctant to try new teaching techniques. What prompted the study?

Lindsay: One of the big things that we try to focus on in our center is how we use local data to drive faculty development to help improve teaching and learning on our campus. As part of that back in 2016, 2017, we did a large-scale observation project where we observed over 200 STEM undergraduate courses. And we wanted to look for differences in the different instructional practices that our faculty were using based on whether they were engaging in our center or whether they were not. And this was sort of the beginning piece of driving everything that we’ve done since then, because we did see differences in their instructional practices between faculty who have and have not engaged in our center, but we didn’t see as much as we thought we would. And so we really wanted to further explore that and understand what things were hindering faculty from doing what they wanted, using evidence-based practices, particularly those that had gone through our Course Design Institute and had done other programs with us. And these are things that we heard anecdotally but we really wanted to better systematically measure this. That’s where Hannah comes in as a postdoctoral research associate and I’ll let her talk about what we did to further explore this idea of what the barriers were.

Hannah: So I came into the project when Lindsay was wanting to develop this barrier survey of some kind. And so I started by going through the literature and I found a lot of work that was of a qualitative nature that people had done in various fields, looking at barriers to implementing evidence-based practices and research-based practices. A lot of different terms are used so you have to know which ones to search depending on which field… in which journal… you’re in, so I got introduced to that, which was a bit of a challenge, but was able to kind of sort out all these different areas and find work that had been done both in DBER in specific fields and then more of the faculty development field. So I pulled on all of those different sources, but I did not find any survey instrument that was of a quantitative nature that delineated all of these different barriers that had been found in the qualitative papers. I found a couple surveys that had little sections of barriers and then I found a survey that looked at institutional climate, but I didn’t find any that delineated lots and lots of barriers that I’d seen in the qualitative work. So I drew on all that qualitative work to develop a survey instrument that we then piloted, so that’s kind of where all that came from.

Lindsay: And to add into that, there are benefits for doing interviews and qualitative work, but we wanted to really be able to find a way to quickly but systematically capture these barriers. Because as I mentioned, we are really interested in using that locally driven data and there’s only so many people in our center that can be able to do that work. That was part of the driving force behind developing the survey itself.

John: Just backing up a step, Hannah mentioned DBER. For our listeners, could you define that just so there is some clarity there?

Hannah: Yes, DBER is Discipline-Based Education Research. So I am a chemical educator, I’m a DBER researcher. Biology educators, astronomy educators, those are all DBER researchers.

John: What did you find in the survey?

Hannah: That survey instrument was not just barriers, but also some related ideas, so it included a section on teaching-research identity because that was something that came out of looking at the literature and seeing that this tension between teacher-researcher identity seemed to be something that might be a part of the barriers. So we added a section on that because we’d also not found any survey instruments that delineated those in a quantitative way. So moving on to the study. We piloted with 86 and that was a subset of the 150 instructors that were observed in the study that Lindsay mentioned earlier, that was kind of a rationale for the current research. So we were able to get 86 complete datasets out of that from the 150 that we sent it to. So first of all, we had 46 Likert scale questions— different statements about barriers that faculty participants could rate on a scale of one-to-five of, “This is not at all a barrier for me,” to “This is a barrier for me all the time.” And when we looked at the results of all of those Likert questions, the top five were number one—and that’s 65 percent said this was at least a moderate barrier for them so they rated that at three-out-of five at least—was lack of time. The second was tenure and promotion guidelines. The third was fixed seats or infrastructure constraints at 61 percent of faculty mentioning that. Number four was that students don’t come prepared at 59 percent. And then five was that too much prep time in particular was required to implement these evidence-based practices, that was at 50 percent of people mentioning. And we investigated those also qualitatively and the qualitative question that we asked—the open-ended question that we asked—was simply, “What barriers are most significant to you in your own teaching and why?” That question was a bit different. So we had all 46 of those Likert scale statements that faculty rated, but this one was getting at, “Okay, so now thinking about your work, what is the most significant barrier for you?” so it was a slightly different question than what we asked to the quantitative, and it produced some very interesting results. So what these 86 respondents said is, number one, aligned with the quantitative at lack of time, but that was only 57 percent that were saying that. Second was classroom space and lack of needed technology at 22 percent. Third was the lack of institutional support, so there’s a lot wrapped up into that question. And then number four was a variety of student-related issues and student resistance and not doing what they’re needing to do at 12 percent. And then finally, the lack of TA support and classes being too large coming in at nine and eight percent. So that gave us a greater understanding of what’s the number one issue for our particular faculty, as well as the overall landscape of all of these different Likert scale barriers. So that was interesting and drove what we were doing in our research. So one of the other results that came out of this work had to do with satisfaction and dissatisfaction with evidence-based practices. We asked the faculty who responded to the survey to go through a list of evidence-based practices and say which ones that they used. And looking at one of those practices—for instance—collaborative learning, we asked them if they were satisfied or dissatisfied with that practice—or both—and that was the practice that people were most dissatisfied with. And when we looked at that, and we compared it with their barriers results, we found descriptively that those faculty had higher barriers across all of the different barrier groupings on the survey. The ones that were dissatisfied with collaborative learning had higher barriers across all the different barrier groupings and we ended up grouping those into five. They had higher barriers across the board and we had been investigating, “What does that mean?” and as we’ve been expanding the study, wanting to get more data to really understand that and look into the policy responses on why they’re dissatisfied… things like that. But what came out of that was what Lindsay referred to, was the need to support faculty, not just before they implement an evidence-based practice, but when they’re implementing it. And we found this excellent study from Henderson and Dancy back in 2007. They did a qualitative study of physics faculty looking at supporting them and what they found is for those faculty that weren’t supported, once they came across these, what they called “situation barriers,” when they were implementing a practice, that made them stop using the practice. And so we think that our results really back up what Henderson and Dancy found and the need to support faculty once they start using a practice, helping them understand what barriers are going to be when they implement that practice and then supporting them throughout the time that they’re implementing. Because otherwise, if they’re not aware of the barriers that they’re going to face, then they may stop using that practice altogether. So that was one of the tentative results that came out of this pilot study was showing us… demonstrating the need to support those faculty.

Rebecca: I was also going to say that a lot of times faculty don’t give themselves a break. The first time you do something, you’re not perfect at it, just like our students, they’re not perfect at it the first time. You have to practice and do it over and over again to get good at it. So I think reminding faculty when they’re doing something new that will also happen for them, doesn’t hurt. [LAUGHTER]

Hannah: Exactly. There was a study that came out recently, it was over five years of implementation. And the first year went horribly, and they adjusted. It wasn’t until like the third implementation that things started to go much better, student resistance started to go down, and just recognizing the first time you implement, there will be a lot of barriers… there will be a lot of problems and that’s okay… to keep going, that this is a normal thing.

Lindsay: I think that’s part of, really, the importance of this. Other people are struggling too. Helping to normalize the fact that when you try something new in the classroom, and it doesn’t go well, it’s par for the course and that other faculty are going through that as well.

Rebecca: Those are some interesting results, but not entirely surprising. I think those are some similar things that we’ve heard and seen in other research. But interesting that it’s at your specific institution from your specific faculty, and that the qualitative and quantitative pieces somewhat align. So what have you been doing with that data?

Lindsay: We have a few different programs that we are working on refining, aligning, expanding to what we’ve found systematically in these surveys with our faculty. Some of these include our Ignite program. Our Ignite program is something that we’ve been running with new faculty for the last few years. This is a program meant to support faculty as they implement a newly redesigned course. So these new faculty go through a week-long Course Design Institute with us and then they spend the next semester whenever they implement their new course, either in Fall or Spring, they meet biweekly with one of our faculty developers, and anywhere from five to 10 other new faculty in a learning community and they build on some of the things that they’ve been learning about course design and implementation. So they’re really getting that support throughout the semester. And one of the things that came out of our barriers survey was that the other work that we’ve been doing—particularly around these observations—is that the implementation is really important and that we really need to support faculty through that. We have some studies that, particularly around Ignite and new faculty, that demonstrate how important this learning community is, not just for the implementation, but the success of students. And so now we are expanding our Ignite program to all faculty, not just new faculty at our institution. We’re doing that for the first time this Fall semester. So that’s one of the programs that we have refined based on some of the data that we’ve been finding.

John: I think one of the benefits of that is if one of the barriers is departmental culture, that prevents people from trying new techniques, bringing in more senior faculty might break that down.

Lindsay: Yes, and one of the places that we’re beginning to expand to as well are learning communities, particularly for mid-career faculty. Many of our Ignite faculty are now moving into being tenured and so they are now becoming leaders in their departments and how do we foster and continue to help support them around teaching and learning?

Rebecca: Does your Ignite program come with course releases or does it come with time?

Lindsay: That’s a good question. We do not have course release at our institution, but they do receive a $1500 professional development fund, which helps support them in being able to continue to develop, they may be able to go to conferences, they do get supported in that way. Another one of our programs that we are developing and as Hannah mentioned one of the barriers are around class size and TAs. And so we have developed over the last few years a program called Spark. Spark is intended to be a program to support teaching assistants in the STEM departments. And over the last three years, we have had over 250 TAs enroll in our one-credit teaching methods course where they actually learn about different pedagogical techniques, learning theory, and they’re able to apply that every week as they are TAs in lab courses, discussion sessions, and even in co-instructor type roles. And that has been a really important piece to help support transformation in the STEM departments because our TAs are really the primary point person in many of our first- year courses and so providing them the support has been really transformative. One of the third things that we are doing in the center is around curriculum redesign. So one of the things that we found in the study that I think you alluded to was the differences between departments and the importance of the departmental culture and departmental support in helping faculty be able to utilize and implement evidence-based practices. And so we are actually working with departments to think about not just individual courses, but what is the curriculum look like for an actual major? What do we want our students to be able to know, value, and do at the end of four years—or five years—within different departments? And so we’re really working to develop this. This is something that we’re doing this year and really working to refine our programming around curriculum development and redesign.

Rebecca: One of the themes of all three programs is curriculum development. What are some things specifically that you’ve implemented or changed consistently to help with some of the issues that you’ve identified?

Lindsay: As part of the redesign process, we don’t necessarily recommend a single type of redesign or curriculum. We really strive to use evidence-based practices, whether that’s at the course level or curriculum level, to allow faculty to think about what best aligns with what they want students to be getting out of the course, or the curriculum. For example, if one of their learning objectives has to do with being able to collaborate and communicate, we might recommend some sort of collaborative learning design as implemented in their course. If they’re more interested in students engaging with the community, that might look a little different in terms of the actual design of the course. So I don’t know if that answers your question, but we don’t necessarily recommend one particular approach.

Rebecca: If faculty are resistant to evidence-based practices, and you were already introducing faculty to evidence-based practices in these programs, is there a different way that you’re presenting this information now to faculty to get them to buy in more to these practices, especially considering time concerns and student resistance and that kind of thing?

Lindsay: Interestingly enough, there are a handful of faculty that I think are resistant to the idea of active learning. The way that we’ve set up at least our Course Design Institute is in such a way that we attend to motivation first, and so we really get very little resistance to the idea of active learning or evidence-based practices. They want to do it. Some of them do do it. They either feel like they can’t do it as much as they want to or they do it and they’re not satisfied with their practice. We really don’t run up into the barrier of, “I don’t believe in active learning,” with the exception of a handful of faculty.

John: And there’s probably not much you can do with those. But I would think working with entire departments might help reduce some of the resistance because when you have that sort of collaboration with the department, it becomes part of the department culture, I would think. How has that been working?

Lindsay: We have had a cohort of faculty within a department go through our Course Design Institute and then another program that paralleled Ignite that was specifically for STEM faculty. And this department really has transformed, so this is about five years ago that they went through as a cohort. The department itself, the culture there is focused on teaching and learning, they continue to engage with our center, we have a recently started SoTL Scholars Program, so Scholarship of Teaching and Learning. We had five of the faculty from that department actually go through this together this past year. They’ve started their own reading group. We’ve been looking at data from the department and we see that student failure rates are going down in their department, particularly for underrepresented students. So working with the departments I think are really, really important and we’re seeing the fruits of that.

John: Earlier Hannah mentioned something about looking at issues of identity in terms of teachers and scholars and so forth. And I would think that perhaps the work you’re doing with SoTL might help unify that. Could you tell us a little bit more about the results you found and how you’ve been addressing those?

Hannah: What we expected to find was that there would be a correlation between teaching and research identities and that if you were high in teaching, you might be lower in research. If you were high in research, you might be lower in teaching. And what we found was that there was no correlation, that you could have both. You could be both an excellent teacher and researcher, you can be really strong in both of those identities, or you could not be. It was all over the place. And part of that is the sample size, and we have since expanded and haven’t analyzed that data yet, but we’re looking into that more.

Lindsay: And to add on to that, so the way that we looked at identity was the idea of how connected you feel with that particular profession. So if you feel connected to the teaching community versus feeling connected to the research community. And we also had a third aspect to that, which was the work identity… so how connected do they feel to the university? What we found was that faculty who had a strong work identity—meaning that they felt connected to the institution—they felt that the department was less of a barrier for implementing evidence-based practices, and they didn’t perceive that they had barriers related to supports. So things like having TAs, classroom space, and things like that.

John: Going back a little bit, you mentioned that one of the barriers that some faculty mentioned was the size of their classes. How have you helped faculty get past that?

Lindsay: We’ve actually had conflicting results around that. So faculty perceived class size as being a barrier to implementing evidence-based practices. But when we look at the actual observations of those faculty teaching, we see that faculty who have engaged in our center use more evidence-based practices, even when controlling for class size. And so what we need to further investigate is how our center plays a role in reducing barriers for faculty. The sample size that we have with our survey results is much smaller, and we can’t really disaggregate. There is something interesting that has to do with class size, and we’re not exactly sure what it is. Whether it’s a perceived barrier or an actual barrier, we’re not quite sure. But I might guess it’s a perceived barrier because we do see more active learning even when classes are large. So faculty are able to do these things, but sometimes they may not think they can.

Rebecca: Or they might not know what practices work at a large scale, because there’s different ways to implement… and so the more we expose them…

Hannah: Exactly. Yeah, because they’re trying to use approaches that require a studio. You can’t do that with a 500-student lecture. So obviously, that particular evidence-based practice is not going to be useful in that case. You can bring in some of these perhaps smaller practices but that are still powerful to get students actively working and collaborating with one another. Think-pair-shares, things like that, that you can still do and then there’s all sorts of work—great work—that’s going on now talking about what you can do with large classes.

Lindsay: And those are the things that we talk about in our Course Design Institute. How do you design your course, knowing that you have particular limitations because of things like class size? Or maybe it’s a required entry-level course, or maybe it’s an upper-level course, or a graduate course. All of those things are really important in thinking about the design.

Rebecca: Or the chairs don’t move.

Lindsay: But we do talk with them about how to deal with that. So in the lecture hall that the seats are fixed and you want to do group work, we have recommended to faculty—if they have space—leave every third row empty, and that way you can actually access students and students can turn backwards to work with people behind that. So we definitely try to help them think about ways to go beyond what they think are perceived barriers.

Rebecca: How to hack your classroom 101.

Lindsay: How to hack your classroom, I like it.

John: And actually, let me put a plug in for one of the Teach Better podcast episodes, which came out in April on the importance of classroom design. We’ll include a link to that in the show notes. The research they were citing finds that active learning helps, but classroom design helps even controlling for the use of active learning. So some of that flexibility is useful. This has been implemented in STEM fields—I think many of those topics that you found would work in other disciplines. Has the teaching center more broadly started to roll out some of these techniques throughout the institution?

Lindsay: I’m going to answer this from a much more broad perspective, thinking about what we’re doing in terms of our programming and supporting faculty. And I think Hannah can talk about the more specific piece around what we’re doing to better gather data around faculty barriers beyond our STEM faculty. So one of the biggest things that I think I mentioned at the beginning that we really are striving to do is use our own local data in addition to the literature to really drive what we do. For us, this goes beyond just doing a needs assessment. This is really doing research around teaching and learning at our institution. One of the pieces of evidence that we found around our prior work is the ways that students engage with each other in class, and how the instructor sets up that group work in class is really important to student success. And so what we are now doing this past year, we are collecting data to better understand not just faculty perceptions of how they design group work, how they assign students to groups, what do they do to assess group work, but we’re also looking at the student perspective. We are actually following students that are working in groups over time, having them reflect on their practice, share audio files and share working documents, to better understand what’s going on in group work. All of that data now we’re using to develop a advanced collaborative institute for faculty that’s going to use not just the literature that’s already published around group work, but also locally derived data that’s both STEM and non-STEM faculty in classrooms. And it’s been interesting because we think about our disciplines being very distinct in terms of  “Oh well,  STEM classrooms are very special, and they need to do these particular things.” As we’ve interviewed faculty, the reasons why they use group work—regardless of their discipline—is very similar. They want students to develop professional skills. I think it’s really important to gather that data to understand this perspective so that when we develop these programs and supports for faculty, we can actually talk about what the faculty are saying and how we use that to improve. So that’s just one example of how we’re broadening this idea of data-driven faculty professional development.

Rebecca: How are you gathering that data about group work?

Lindsay: In our center, I am 50 percent research and assessment and so a lot of my work is around being able to assess our programs, but also be able to gather the data to drive programming. As we said in the beginning, my PhD is in science education. So this is my formal training, being able to do this type of work. So I actually have a group of three graduate students—as well as Hannah and another postdoc—that helps support the research and assessment and center. So for example, as part of that group work study, I had one graduate student who over the course of two weeks, interviewed 19 faculty and over 1000 minutes of interviews that had to be transcribed. I really have a committed group of graduate students and postdocs that help support this work, because they’re really interested in helping make the improvements as well. I don’t think if this was something that was very abstract and not related to helping improve instruction that we would have such buy-in from the people that are helping support this work. So we’re doing interviews with faculty, students are submitting reflections, audio files and documents. So those are the data sources we have right now. We also have syllabi and course documents that the faculty have developed that articulate how they are setting up these group work or group projects.

John: That’s a great resource, I think, for all teaching centers because most of us don’t do that, and it’s nice to see this sort of research. We often talk to faculty about the importance of doing SoTL research, how the classes are working, but teaching centers don’t always do quite as much assessment of how their programs work, and how things are working on their own campuses in this way. So it’s a nice example, I think.

Hannah: Right, and I can talk to the real specific research that we’re trying to do to expand from STEM into non-STEM fields to kind of get more of that research across the university going. So the survey that I developed that has the barriers, that has the identity, that has some qualitative background questions to try to understand where their beliefs come from, all of that. I have been working with STEM faculty and non-STEM faculty now, to expand into the humanities, the arts, the social sciences. And what we’ve been doing is working with humanities faculty at the Center and then I had a focus group this week with several scholars in those areas to talk about the language that we use in the survey. So what I quickly found when we were trying to expand the survey across the university, is that the language that you use is really important. Now STEM faculty, they are fine with the use of the term evidence-based practices. And discipline-based faculty and researchers, we want to see the evidence. We want to know if something works, we want to know that there was a rigorous study that backs up that particular practice, and once we see that, we’re ready to kind of go for it. But when you try to expand that wording into the humanities, that’s not so much a crucial thing for them, they’re wanting to see that things work. The type of research that they do is very different and when we use the term evidence-based practices, the way that they think about that is very different from STEM faculty. So we had to change the wording, we’re modifying the survey, how the questions are asked, the types of words that we use, the assumptions that we’re making. So that’s been my job the past few weeks and will continue because it’s been proven it can be quite challenging to make sure that we’re not alienating a lot of the people that are taking the survey to the point where they see certain words and are like, “This doesn’t apply to me, I don’t want to take the survey anymore.” So that’s been the challenge with this, expanding this from STEM, is the language can be a barrier to people taking the survey and then we don’t get the data that we need. I’ve been working to figure out, “How do we talk about this in a way that we can compare across all of these groups, but still get useful data and not alienate groups within those different departments.”

Rebecca: I think sharing a summary of that information would actually be useful for a lot of centers and researchers too because teaching and learning centers probably also suffer from their advertisements and stuff, perhaps alienating groups of people and not realizing it for the same reason, potentially.

Hannah: Definitely, definitely. And one of the humanities faculty members here at the center and I have been talking about that and may be coming out with a paper once we gather more data on this, on the language that we use. What is useful and what is not useful by discipline?

John: That’s something I wouldn’t have thought of, because we use a lot of evidence-based practices here all the time.

Hannah: Yeah, I didn’t think of it either, and so I was in for quite the shock when I started talking with humanities faculty.

Lindsay: And I think another thing to add in terms of how we’re broadening this work, one of the places that I’ve begun to explore is how do we set up infrastructure at our institution so that we can actually systematically gather data, connect data sources, and then help faculty use that individually to improve instruction. It doesn’t do anybody any good if we gather evidence or research—we do research on our own—and then we don’t do anything with it. And so, we’re developing as I mentioned, our SoTL Scholars program so we can help faculty learn how to do this research on their own. So we are developing a set of tools that we can use to, for example, go out and observe faculty teaching in their classrooms and then from that data, create some sort of visualization that can be used in a consultation. We have a consultation program—many institutions in our centers have consultation programs—but what we really want to begin to do is gather that data in a way that we can begin to represent it on some sort of timeline, where the faculty can see, “Okay, the first 15 minutes of class we did lecture, I asked a few questions here and there, students didn’t answer those questions,” or, “I answered them myself or I moved on too quickly,” and so really honing in on some of those small details that can really help them make tweaks and improvements to their own instruction. So we’re really working at that infrastructural-level now to think about how do we create these tools and set up databases so that we can gather data and share that with faculty.

Rebecca: A follow up question to the qualitative research that you did at the very beginning… What kind of observations you were making for that qualitative research and what you were focused on? What you were looking for specifically…

Lindsay: Good questions. So the original observation study that we did a few years ago, we ended up using COPUS, the Classroom Observation Protocol for Undergraduate STEM. If you’re not familiar with that, COPUS measures the presence or absence of various different types of student and instructor behaviors over two-minute time increments. I was able to train 35 undergrads on how to use COPUS reliably and we were able to gather… for each individual course we observed twice. And we were able to then calculate the percent of time the instructor spent lecturing, or spent doing quicker questions, group work, administrative tasks. And we were recently co-authors on a science publication where the COPUS data were then transformed into profiles and so we were able to then categorize these different classes as primarily lecture—which was greater than 80-percent lecture using COPUS—interactive lecture—which was lecture but it had some clicker questions or some other group work interspersed throughout— and then the third set of categories was around student-centered instruction, so it could be POGIL type classes—so Process Oriented Guided Inquiry Learning type classes—or primarily group work, working on worksheets, doing problem solving, or a variety of different group activities. And so of those, we had 239 classes that we observed. Of those, we were able to classify those classes into those three categories—lecture, interactive lecture, and student-centered—and then we took those classes and organized them based on the intervention that the faculty have gone through. So whether they’ve engaged in our Course Design Institute, whether they’ve done our Ignite program, and we actually had a fair amount of faculty that we observed that have never engaged in our center at all. And so that’s where we were beginning to see differences… that our Ignite faculty, we saw much more student-centered instruction than faculty who had never engaged our center. We also gathered grade data on those classes. Do you want to know about that?

Rebecca: Yeah.

John: Sure.

Lindsay: This is actually a paper that’s currently in review, but the grade data was the thing that was really interesting to me. What we ended up doing is we calculated a DFW rate. That’s D, F, and withdrawal. So basically, failure rate for students in those classes that we observed, those 239 classes. We also were able to calculate failure rates for underrepresented minority students. So those were black, African-American students, and Hispanic students combined together compared to white students in the class. And even when we gathered observations of 239 courses, when you started to look at the courses taught by faculty at the different types of interventions—so that was Ignite, Course Design Institute—and then when you broke it down even further by, “Let’s look at those courses taught by Ignite faculty that did active learning, or lecture, or interactive lecture,” the numbers got very small very quickly. But one of the most interesting pieces that we found descriptively was, when you looked at just courses that were categorized as having student-centered instruction—so active learning, group work, those types of things—the faculty that have gone through things like our Ignite program, and another program called Nucleus—which is similar for STEM faculty—the failure rates between white students and underrepresented students were nonexistent. When you looked at student-centered courses where the instructors had not gone through our Course Design Institute or gone through any of our communities, the failure rates for underrepresented minority students were four times that of white students. Now this is descriptive, this is not anything that’s inferential, but that was one of the driving forces for me that made me realize that we need to look more at group work and what was going on in group work because it’s suggested that when you implement group work or student-centered instruction in your courses and you’re not supported in doing so, you are doing a disservice to your students, and that seems to differentially impact underrepresented students more so than white students. And that was really disturbing to me that we saw those differences on average. This was not the max, this was a mean value. And so that was so important for us to further explore, and we would not have known not had we not done such a large-scale study, and had we not used our own data.

Rebecca: That’s really interesting.

Lindsay: Thank you.

John: You’re making a big difference there, clearly.

Lindsay: We are, and it’s so exciting.

Rebecca: Yeah, I think sometimes we don’t always realize those other kinds of impacts. Or that there could be a difference in the kind of impact that one makes. So I think that’s a really interesting initial discovery to explore, so I’m really interested to see what else you find out.

Lindsay: So we wouldn’t have been able to make those findings had we not been able to connect to institutional data, and so that’s another reason why this developing infrastructure is so, so important, that we’re not going to be able to find meaning if we’re not connecting all of the pieces.

Rebecca: I think one of the things that is really interesting is that you’ve been able to do such robust research at your own institution and have the support to do that. Even how you structured that and how you’ve gathered that would be of interest to many other centers, I think. Sometimes the details of how you arrange that and organize it and how one thing led to another can help other organizations do something similar.

Lindsay: Thank you. I will put a plug in. So in terms of helping other centers be able to do this type of systematic research assessment work, we had a PODLive! webinar on Friday, April 26. If you’re a POD member, you should be able to access this through their website to see what we talked about and what questions we ask ourselves as we go through the process of thinking through measuring impact.

Rebecca: Great. We will make sure we link to that in the show notes and let people know how to access that.

John: We always end with the question, what are you doing next? You’ve already described some things, but we’ll still ask anyway.

Lindsay: So if you can’t tell already, I’m really passionate about data… using data to help drive what we do to improve teaching and learning. And so the two sort of big things that are next for me are really trying to build the infrastructure so that we can liberate data and be able to use data meaningfully, respectfully, and purposefully to help improve instruction. And also being able to help empower our faculty to be able to do research on teaching and learning in their classrooms… so trying to expand our SoTL Scholars Program, and developing further supports in that area. So that’s what’s next for me.

Hannah: And for me, I am working on a couple projects related to the barriers work. So we talked earlier about the humanities expansion, so developing a survey instrument that can be given across departments. So I’m continuing to work on that, work on the language that we’re using, making it relevant to them. And then we’ve got a national study that we’re trying to work on. So we have implemented the pilot—which is what we talked about today, the results of that—and then we implemented a second one also at UVA, but much larger. And then we’re wanting to now expand this and do a national study because the real beauty of this instrument is that it’s not just for us at UVA, it is meant to be a tool for any university, any department to be able to use. And one of the findings that came out of our study was that the barriers are different by department. The barriers, the use of evidence-based practices differs by department… it’s not just the university being different from another university. It’s the department being different from another department at a different university. And so this tool allows any department, any university, to give this to their faculty and see contextually, what are the barriers for these faculty? Now you look across the board, time is usually the highest barrier, but what comes after that differs by department. If there’s particular issues with one department, one university with the teaching-research balance at a given university, all of that’s going to be different. And so the beauty of this instrument is let’s look at a variety of types of universities, types of departments, let’s try to understand what is useful, what are supports, what are barriers across different institutions, across different departments. Try to look for where are there trends and where are there not trends. Where is it just entirely dependent on a given context and where do we see maybe some trends in tenure-track faculty versus non-tenure-track faculty, general faculty, things like that. So we’re really hoping to dig into a much larger sample in the coming year and investigate this further, and I will say that there are a couple of other researchers who are also working on this. So this is an up-and-coming area of research that you’ve got Megan Bathgate at Yale, you’ve got Emily Walter at Cal State Fresno, they’re both doing studies along this idea of barriers and supports for faculty using evidence-based practices. So, I just wanted to put a plug in that we’re not the only researchers doing this. There’s a lot of great work that’s going on and I think this is an up-and-coming area to really help support moving higher education forward and transforming higher education, ultimately, by understanding how can we help our faculty implement more of these practices that we know are going to support our students better?

Rebecca: Great, sounds like a lot of exciting things coming down the road for us to take in soon.

Hannah: Definitely.

Lindsay: Yes.

John: Thank you for joining us. This was a really interesting discussion, and I think many of us will reflect on it in our teaching centers.

Rebecca: Yeah, thank you so much.

Lindsay: Well, thank you, appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

83. ACUE

Faculty are often excited after attending professional development workshops and plan to implement new techniques, but often don’t follow through. In this episode Dr. Penny MacCormack joins us to talk about one program that provides scaffolding and structure to help faculty improve their teaching using evidence-based practices.

Penny is the Chief Academic Officer of the Association of College and University Educators (ACUE). Before joining ACUE, Penny had served as the Chief Academic Officer for the New Jersey State Department of Education and as an adjunct professor at Southern Connecticut State University, and Montclair State University. She began her career in education as a science teacher.

Show Notes

Transcript

Rebecca: Faculty are often excited after attending professional development workshops and plan to implement new techniques, but often don’t follow through. In this episode we talk about one program that provides scaffolding and structure to help faculty improve their teaching using evidence-based practices.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Dr. Penny MacCormack, the Chief Academic Officer of the Association of College and University Educators, or ACUE. Before joining ACUE, Penny had served as the Chief Academic Officer for the New Jersey State Department of Education and as an adjunct professor at Southern Connecticut State University, and Montclair State University. She began her career in education as a science teacher. Welcome, Penny.

Penny: Thank you. I’m happy to be here.

Rebecca: Today’s teas are…

Penny: Green tea.

John: I have Bing Cherry Black tea.

Rebecca: And I have Lady Grey.

John: We’ve invited you here to join us to discuss ACUE’s effective practice framework and the associated professional development program. How did this program come about?

Penny: So I think, like many ideas, initially with a conversation among leaders in higher education, some very respected leaders, talking about some of the challenges and changes happening in higher ed. An increasingly diverse student body, certainly more attention being paid to retention and graduation rates, and increasing contingent faculty, as well as the public starting to question the quality and the value of a degree in higher ed. And as we looked at the student success agenda, with many strategies that made good sense, really paying attention to maybe more nuanced financial supports, guided pathways with better advisement, data analytics, instructional supports, et cetera. We felt that there was a missing element and we felt like that element was more foundational than just one of the strategies that folks should be thinking of. For example, guided pathways or advisement make really good sense to us…that a student would have a clear path to a meaningful degree. But what we thought attention needed to be paid to was the quality of instruction in those courses along the pathway, and then across an entire institution, the quality of teaching. And we were very aware of the fact that faculty—including contingent faculty—are experts in their discipline, in their subject area, and they’re experts in the research processes. But most have little—sometimes no—training in evidence-based teaching practices in teaching. So we felt like that missing foundation needed to be addressed and set about to develop a comprehensive…we wanted something that would give folks a foundational base of the evidence-based teaching practices we know to be effective in the college classroom. So we wanted to be comprehensive, we wanted it to be research based, we wanted it to be high quality, and we wanted to be scalable. Recognizing that while it’s important for small groups of instructors to become better teachers, the reality is, all of our students, and all of our faculty deserve to be interacting with the evidence-based teaching practices we know actually improve engagement and deepen learning. So we set about to do that.

Rebecca: It’s a pretty big undertaking. It sounds like you probably had a lot of people involved in that process. Can you talk a little bit about how did the design of the program happen and who was involved?

Penny: So you’ll notice here one of the things I said was comprehensive, that we wanted faculty to gain a foundation in evidence-based practices. And so we needed to identify, what are the core set of knowledge and skills you need to be effective in the college classroom? And to be very honest, we had hoped perhaps that already existed somewhere. [LAUGHTER] But lo and behold, that was not the case. And so we reached out to scholars in teaching and learning across the country and worked with them, did a deep dive into the literature, and worked through an iterative process to identify that core set of knowledge and skills. And once we had that, we also worked with the American Council on Education, to endorse our courses and our framework. And they brought to bear their own set of experts across the country in the Scholarship of Teaching and Learning to review the framework. And then eventually, ACE endorsed the framework and so we feel pretty confident at this point through the processes we used and ACE used to say that our framework and effective practice does outline the core set of knowledge and skills you need to be effective in the college classroom. So in that case, the folks who really informed that work are the experts in the scholarship of teaching and learning across the country, folks like Linda Nilson, Tom Angelo, Elizabeth Barkley, Saundra McGuire, really making sure again, to involve those folks that teaching centers across the country know really have done the majority of scholarship in that area.

Rebecca: Of course, once you came up with the framework and that comprehensive knowledge, you had to figure out how to deliver it. Can you talk a little bit about how that decision was made?

Penny: Absolutely. You point out something that is quite important. It’s one thing to develop a list, right? “Here’s the core set of knowledge and skills.” It’s yet another thing to do that those other three describers, right? Research based—that was kind of easy, because the list was research based—but high quality. And for me, when I’m talking with folks, high quality really means that faculty will love it. Because if faculty are not going to be engaged in this course and engaged enough to actually change the practices that they’re using in the classroom, then we’re not going to realize that student level impact that is our mission. So in order to design the course now—to your point, got to do that part—we did a couple of things. So one, we paid a lot of attention to the research on how people learn, how does the brain work, and specifically, how do adults learn. The course needed to be scalable. It needed to be offered online, so a lot of attention to online practices. But then we did something really important. And that was to talk to faculty focus groups across the country and do a couple of things. One, put some materials in front of them. Some questions, some video, some text, and ask them to critique, which they did happily, because faculty are quite good at critiquing. [LAUGHTER] The second thing we did was we asked them, “What would you need to consider changing the practices you use in the classroom?” And so they were crystal clear. One, they wanted to see those evidence-based practices in action, in authentic classrooms, by their peers…peers teaching…people that they could see would be instructors in the classroom. Two, they wanted to hear from those instructors why they were using those practices. Icing on the cake would be to hear from students as well, how those practices were working for them. Three, they wanted to hear from researchers. They wanted to hear from the folks who demonstrated that these practices are effective in the classroom. Makes sense, they’re higher ed folks, they want to hear from the folks that did the research. And four, they wanted opportunities to learn, discuss with their colleagues as they were learning, to learn with and from their colleagues. And so just as we paid attention to the research on how people learn, how adults learn, online practices, we paid really careful attention to what faculty asked for, and we delivered it. We made sure that those four things that I heard over and over and over again—from faculty across the country—we delivered on. We listened to them.

John: Maybe it would help if you sketch out the process of a typical module, because it incorporates all those things. And we’re new to ACUE, but our faculty so far have really been enjoying it and they really appreciate the design of the program. But it might help for our listeners who aren’t as familiar to know how a typical module is structured.

Penny: I’m happy to discuss the learning design because we spend a lot of time and a lot of attention to it. Each module includes 12 components. I can divide those 12 components into four groups of three. So the first three components are really designed to pique somebody’s interest and to activate prior knowledge. So we show an introduction video, where that includes clips from our classroom demonstration, kind of like how 60 Minutes gets you interested in the rest of the show, we’re showing little clips to get folks interested in the topic. We outline very clearly the learning objectives and the rationale for the module, so we connect the practices that they’re going to learn to the research that demonstrates it does impact students, and then we offer a group of questions to activate that prior knowledge because what we know about that is if you activate prior knowledge, you’re more ready for new knowledge. So that’s the first three components. The second three are designed to build that foundational knowledge. We decided to show before tell first. And so we have a classroom demonstration video, where you see faculty utilizing the evidence-based practices being recommended in that module. You hear from those faculty why they’re using those practices and you hear from students about how those practices are impacting their learning. Next component, you hear from the researchers about the research behind that component. We actually utilize speed drawing there, so that it’s not just a talking head, but there’s a little bit more interaction going on and then finally, we offer resources to faculty so that when they implement any one of the practices that they’ve just seen in that classroom demo, they have all the resources they would need to implement. The next three components are about deepening learning, and allowing for that collaboration to happen with their colleagues. And so the first component is some text. We wanted faculty to read a little bit deeper about the practices and the way we do that is to address some of the common misconceptions, common challenges that faculty might think of, and we address those with the research. And so a common challenge or a common misconception will include a couple of paragraphs from the research about why that’s a challenge and how to overcome it or why that misconception exists in the information that kind of helps you see it differently. We follow that by two sections of what we call observe and analyze. Up to this point in any module, faculty would be able to do all of those components on their own online when it’s most convenient for them. With the observe and analyze, oftentimes faculty will schedule a particular day that they’re all going to engage in watching these videos, and the videos are of what I call developing practice. So you’ll remember that faculty would have seen effective practice, they would have heard from the researchers, but now we show them developing practice—somebody doing some things well and some things that could be adjusted some—and that is the conversation that faculty have. So they watch this video, and then they engage in an online conversation—some of our partners will sometimes bring folks together face to face—but they engage in a rich conversation about what that person is doing well, and what they might adjust or tweak.

John: We should note that no actual students were harmed during these demonstration component videos.

Penny: Absolutely. As a matter of fact, during the demonstration videos where we were doing developing practice, students knew what we were doing, and it’s completely scripted. So I think what was interesting about students is they understood when a practice was really effective, because remember, it’s developing. So it’s not like a train wreck, it’s some things being done well, and some things that could be tweaked. And when you think about it, the faculty watching the video are in the same shoes as the person trying it for the first time. So they’re watching somebody try something for the first time making some mistakes but doing some things that are quite good. And they’re able, they have that opportunity, before they’re asked to implement one of those practices in their classroom so it’s a really rich learning opportunity that they get to do with their cohort to collaborate with their colleagues. The last set of components, faculty are asked to practice and reflect and then we do a closing video. So we indicate to faculty, “Here are the learning objectives for the module and here are the practices.” And there’s always between five to 10 practices offered in every single module. And we say to faculty, “Choose one,” and that’s important. In adult learning you don’t want to say, “This is the one thing you have to do and you have to do it now,” because faculty are teaching different classes, have different students that they’re working with, we want to give them a choice. So they choose one of those practices and they implement it in their classroom. And then what we require is they reflect on that experience in writing. And that written reflection is submitted to us to be scored. We do present to faculty a rubric for how we’re going to score that reflection. So those requirements are up front, we try to practice what we preach, as far as teaching and learning goes. Faculty submit the reflection, we have national readers that score it using the rubric, and if a faculty’s reflection isn’t quite up to our meets category, we get it back to them with specific feedback and they can resubmit. Now we finish every module with a closing summary—again, practicing what we preach, good teaching and learning—close with a summary of the learning objectives and some more commentary from the researchers.

John: A lot of our faculty have commented how they appreciate the fact that the course itself uses all the practices that are implemented—as you mentioned—and they really enjoy the skeletal outlines, they like the ability to go in and critique these demonstrations. And one of the things that we as working with our teaching center appreciate is that we’ve done workshops on many of these topics and some people have attended them two or three years in a row without actually implementing them. And what we really appreciate is the fact that now people have to get past that barrier of actually trying it in the classroom. And a lot of people who have been coming to our gatherings have said they did this for the course and now they’re doing it in every class. So it’s already making some big changes in people’s teaching practice. So it’s been working really well.

Rebecca: I think another real strength is the external reviewers is really important so that as teaching and learning center staff, we can support our colleagues and not feel like there’s some sort of punitive relationship where we’re judging.

Penny: Yeah, we are a learning organization and so actually when we first piloted a smaller number of the modules, we had the facilitators—our course facilitators, often folks from an institution’s teaching and learning center—scoring their reflections, and they were crystal clear with us that that didn’t feel right. And so we took that on, so that they could really be the coaches that we want them to be with the cohorts.

Rebecca: I think that works really well and I think that really encourages faculty to follow through and to do them and to actually take the actions in the classroom. So I think we really benefited from that particular feature.

Penny: Yeah. I know our mission has been to realize student outcomes— better retention, graduation rates, better learning— through quality instruction. And so in order to impact students, we knew faculty had to go beyond learning these evidence-based practices, but actually using them and so the requirement to complete a module became the implementing of one of the practices. And then what we know to be true in professional development is reflection is such a strong way to not only implement but actually to continue thinking about what went well, what didn’t go well, what might I refine, et cetera. That’s really putting you on the trajectory to becoming a better and better instructor.

Rebecca: I think one of the other interesting advantages of this particular online course is that a lot of our faculty may never have taken an online course but may be asked to teach online courses, so having the experience of a well designed online course is an important experience, especially as faculty move more and more into teaching online and having an idea of how to implement some of these practices, not just in face-to-face situations, but also in online or hybrid situations.

John: And we should also note that in each module, the options that people have could be either for a face-to-face class, or there’s a set of options for people who are teaching online, so it facilitates both types of instruction directly for people with different teaching schedules.

Penny: And we have actually even brought that to a more sophisticated level. So we will be offering our course in online essentials coming up in the next few months, where if we had a cohort of online instructors, they would be doing an observe and analyze about online instruction versus face-to-face so that they would really have that full experience of, “How do I do this core set of skills needed to be an effective instructor online?” So we’ve gone beyond just offering the online resources, to making sure we offer some real high quality learning experiences for them.

Rebecca: That’s great.

John: You mentioned the goal of improving instruction and improving all these outcomes. I know that there’s been some research that has been done at some campuses in terms of what sort of impact this has had. Could you tell us a little bit about what’s been found in terms of the effectiveness of this program in improving student outcomes?

Penny: Absolutely. We’re really, really proud of the work that we’ve done with regards to efficacy. And I think it’s important to recognize that when we partner with any institution, we partner to assist and support implementation. So when you partner with ACUE, we don’t say, “You can click on here and get to our courses, and good luck!” [LAUGHTER] When we partner, every institution has an academic director who will work with the campus lead—oftentimes the teaching and learning center folks as well—to design the course sequence and cadence and make sure that it makes sense for that particular group of faculty. And then in addition to assisting with implementation, we actually study efficacy. And we are very proud of multiple studies now demonstrating student impact. But I always like to indicate that the first set of data that we collected was around faculty, because as I was mentioning before, if faculty aren’t engaged with the course, faculty aren’t learning, and faculty aren’t changing their practices, then you have no hopes of seeing student impact. And we’re particularly proud of what we have with regards to faculty data across over 2,000 faculty members. Ninety-seven percent on average report that the course is relevant. On average, faculty report learning 55 new practices and learning more about 71. And then on average, faculty report implementing 28 new practices as they engage with the 25 modules and a plan to implement 28 more. So we’ve got that faculty data that says to us, “Hey, you know what, you’ll likely have student impact data,” because again, all of the practices in the course are evidence based, they’re already research based. And we’re, again, really proud to share some of the findings we have at Delta State, we have a study where we were able to show an increase in A’s, B’s and C’s, and a decrease in DFW’s. At Miami Dade College, we were able to show an all of these results are statistically significant. In fact, I invite anyone to go on our website, look at the impact page, if they’re particularly interested in the statistical analyses. At Miami Dade, we saw increased student engagement, comparing faculty to themselves before and after they engaged in the course as well as to a matched cohort. We saw an increase in grades. At Texas Women’s University we saw an elimination of course completion gap, a rracial course completion gap. And at Broward, we actually gave students surveys where they indicated that they had engaged regularly in evidence-based teaching practices. And we’ve got a number of studies currently going on so we have been able to show and realize the student level impact that you might expect as faculty start to regularly use evidence-based teaching practices. It’s really, pretty quite amazing.

John: How many schools have participated in this program?

Penny: So currently, we are partnering with over 100 colleges and universities across 37 states. And again, as we partner with any university, we work with them to design the course offering for that particular set of faculty at that particular institution.

John: We appreciated the fact that since we started in late January that the structure was able to accommodate teaching schedules of our faculty, so that people were doing things that were relevant at that portion of the year.

Penny: Yeah, I am particularly proud of the fact that this is not just some lockstep set of courses we ask you to follow, but rather thoughtfully sequenced, dependent on when faculty are starting to engage in the course, and we sequence in a way so that faculty pretty early on—as they implement in their classrooms—start to have some positive feedback from students because that itself is pretty motivating.

Rebecca: Yeah, I think one thing to point out is that we often think about when you teach someone how to teach, you start with the syllabus or you start at the beginning, and we started in the middle, because we were in the middle of the semester, and it made perfect sense for our faculty. I think that it was really effective and I think that the faculty really appreciated that they were able to do stuff right away and not plan things for a semester out.

Penny: Yeah, what we found essentially is as much as I love to think about learning outcomes, and aligning my assessments and aligning my activities, that’s not what everybody enjoys doing. And it’s best to put that towards the end of a sequence. So that faculty really can utilize practices that connect with their students, motivate their students, really embrace the diversity in their classroom, and have those kinds of interactions and then get to, “Okay, so how do I structure this? How do I write a learning outcome that really helps students learn more? How do I make sure my assessments are aligned,” et cetera. That’s work that’s best after they’ve had some of those other experiences.

John: And after the toolkits have been developed, so they have activities they can plug into those learning objectives.

Penny: I do think that when an institution feels like, “Gosh, we need to do something about courses,” they’ll often go to course design as their strategy and leave out the how the course is taught all together and just think the redesign is going to do it, but it really is the combination.

Rebecca: So we always wrap up by asking what’s next?

John: For either you or ACUE?

Penny: For both me and ACUE—I’m happy to say—as I described before, we’re a learning organization. So we are constantly listening to our partners, seeing what’s happening in higher ed where we think we might be able to have some positive impact. But one of the key areas—no surprise—is continuing education. So, we’re helping faculty have this strong foundation, but we know it takes a lifetime to become an effective instructor. And so we want to support faculty in continuing to build on that strong foundation. As well as looking at what are some other areas in higher education where we might be able to offer some courses and some learning that would assist with, again, realizing student success.

John: We’ve really enjoyed talking to you and we’re really enjoying the program here.

Rebecca: Yeah, thank you so much for joining us today.

Penny: I’m so happy that folks are enjoying the program. When we hear from faculty and we hear the kinds of appreciation and even as they talk about how their students are more engaged or learning at deeper levels, there’s simply nothing better than that, and so we’re excited to be working with you folks and with folks across the country.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

74. Uncoverage

Introductory textbooks in most college disciplines tend to become thicker over time as new topics are steadily added while old topics remain. Classes designed to “cover” all of these topics necessarily sacrifice depth of coverage. In this episode, Dr. David Voelker joins us to examine how some faculty are changing their focus from “coverage” to providing students with an opportunity to actively engage in the discipline and uncover its power to help explain their world.

David is an Associate Professor of History at the University of Wisconsin Green Bay. He is also the Co-Director of the Wisconsin Teaching Fellows and Scholars Program and the co-author with Joel Sipress of “The End of the History Survey Course: The Rise and Fall of the Coverage Model,” which was published in the Journal of American History in March 2011.

Show Notes

  • Voelker, D. J. (2008). Assessing student understanding in introductory courses: A sample strategy. The History Teacher, 41(4), 505-518
  • Sipress, J. M., & Voelker, D. J. (2009). From learning history to doing history. Exploring signature pedagogies: Approaches to teaching disciplinary habits of mind, 19-35.
  • Sipress, J. M., & Voelker, D. J. (2011). The end of the history survey course: The rise and fall of the coverage model. The Journal of American History, 97(4), 1050-1066.
  • Voelker, D. J., & Armstrong, A. (2013). Designing a question-driven US history course. OAH Magazine of History, 27(3), 19-24.
  • Wiggins, G., Wiggins, G. P., & McTighe, J. (2005). Understanding by design. Ascd.
  • Gordon Wood – Author and History Professor
  • Gary Nash – American historian
  • Angela Bauer – Professor and Chair of Biology at High Point University
  • Ryan Martin – Psychology Professor and Associate Dean of Recruitment, Outreach, and Communications at University of Wisconsin at Green Bay
  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • 61. A Motivational Syllabus. Tea for Teaching podcast (with Christine Harrington)
  • Christine Harrington – Associate professor in the Department of Educational Leadership at New Jersey City University and the author of Dynamic Lecturing
  • Lendol Calder – Professor of History at Augustana College
  • University of Wisconsin’s Teaching Fellows and Scholars Program
  • UW Faculty College
  • Spring Conference on Teaching and Learning
  • Regan Gurung – Professor of Human Development and Psychology at UW Green Bay
  • Scholarship of Teaching and Learning
  • Gurung, R. A., Chick, N. L., & Haynie, A. (2009). Exploring signature pedagogies: Approaches to teaching disciplinary habits of mind. Stylus Publishing, LLC.
  • Ciccone, A. A. (2012). Exploring more signature pedagogies: Approaches to teaching disciplinary habits of mind. Stylus Publishing, LLC..
  • Gurung, R. A., & Voelker, D. J. (Eds. Gurung, R. A., & Landrum, R. E. (2013). Assessment and the scholarship of teaching and learning..). (2017). Big Picture Pedagogy: Finding Interdisciplinary Solutions to Common Learning Problems: New Directions for Teaching and Learning, Number 151. John Wiley & Sons.
  • 54. SOTL. Tea for Teaching podcast
  • International Society for the Scholarship of Teaching and Learning
  • American Historical Association
  • Tuning Project

Transcript

John: Introductory textbooks in most college disciplines tend to become thicker over time as new topics are steadily added while old topics remain. Classes designed to “cover” all of these topics necessarily sacrifice depth of coverage. In this episode, we examine how some faculty are changing their focus from “coverage” to providing students with an opportunity to actively engage in the discipline and uncover its power to help explain their world.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Dr. David J. Voelker, an Associate Professor of History at the University of Wisconsin Green Bay. He is also the Co-Director of the Wisconsin Teaching Fellows and Scholars Program. He is the co-author with Joel Sipress of “The End of the History Survey Course: The Rise and Fall of the Coverage Model,” which was published in the Journal of American History in March 2011. Welcome, David.

David: It’s nice to join you.

Rebecca: Today’s teas are…

John:…are you drinking tea?

David: Oh of course, always. Yes. I’m having a Moroccan Mint Green tea.

Rebecca: So you’re a tea drinker so you can come anytime. [LAUGHTER]

John: I’m drinking Black Raspberry Green tea.

David: Oh that sounds nice.

Rebecca: And I have Bombay Chai today.

John: We’ve invited you here primarily to talk about your work on redesigning the introductory history course. Could you tell us a little bit about the problems that you observed in the traditional approach to teaching the survey course?

David: I started teaching history at UW-Green Bay in 2003 and I was trying to create a class that would be very engaging for students. We would look at primary documents, I would really encourage them to think critically about the materials, and so forth. But I was still doing a lot of coverage, I went through the standard list of topics that you would follow to teach American History and I had a textbook. Well, after doing that a couple of years, I had two really wonderful students in my office. These were very engaged, hardworking, curious students, and they were talking to me about an upcoming exam and I found that they were just trying to memorize the content as I had taught it. Now, they were clearly capable of so much more, and they were doing more than that. But when it came time to the hard work they were putting in to study, they saw it as a memorization exercise. And I realized, “Wow, if these are the most engaged students in the class, then something’s not going the way I want it to go,” because that’s where their attention was…on memorization. That’s when I began a long journey to rebuild the way I was teaching—or reconceived how I was teaching—the intro history course. I ended up actually developing not only a new way of teaching the course, but entirely new ways of assessing student learning. So that when students were studying, when they were really putting in the effort and so forth, they would be focusing on the things that I really wanted them to learn, which was: how to interpret a primary source, how to find the argument made by another historian, how to put all that together into historical analysis. So that’s a quick summary of how I got on the path of reworking my introductory history course.

John: I don’t think this is unique to history. I see the same thing in economics. And in general, when students come to college, they often learn somewhere along the way that memorizing lists of things is what they’re expected to be able to do in college, and what you’re addressing is a problem that we see in all of our disciplines to some extent. So what would you recommend as an alternative? How have you approached this issue, or how have you tried to resolve this?

David: Well, first I should say I was really inspired by the book Understanding by Design. And I just looked at my copy and apparently I acquired that and started reading it in 2006. So it’s been a while ago, and it’s quite marked up. But the basic premise there is, instead of starting with, “Here’s a list of all the things I want to cover,” or “Here’s a list of all the things I want students to read,” you really start with “What is it that I want students to be able to do as a result of taking this class?” Now, students taking an intro history class, I’m not going to say, “Well, I want them to be able to build a house,” or something like that. What they’re going to be able to do is think in particular ways. You know, not necessarily the kind of skills that you would be able to observe in some product other than their thought. So I really tried to develop a class that would help students develop their ability to think historically. Now I realize they’re doing that for an intro course at a beginner kind of level, but the emphasis is still on historical thinking.

Rebecca: When you’re talking about the ways that we want students to think and work, we often get frustrated as faculty when students aren’t doing that but we don’t always take the time to articulate that, so I think it’s important to highlight that as a good starting point.

David: One thing I tried to do here is… well, just to be really intentional. What is it that I want them to learn? And then I think the crux of it is to make sure that I’m actually assessing them on the things that I really want them to learn. And I would guess that the overwhelming majority of college level history instructors really do want their students to come away with the ability to think historically, or at least to have some growth in that area with the ability to know what to do with a primary source, you know, historical documents, or perhaps an artifact or something like that. So I would say our hearts are in the right place but the question is, how do we actually get there? How do we align our highest learning outcomes with the actual assessments that we’re doing? In Understanding by Design—I don’t think I mentioned the authors earlier but Grant Wiggins and Jay McTighe—they really put a lot of emphasis on assessment. Now I know in higher education there are folks who are wary of assessment and they’re thinking of institutional assessment, which I would say is important, but here we’re thinking about how are you grading the students? How do you know what they’re learning? So I decided to rethink my assessment and make sure that my exams were really requiring students to do some historical thinking on the spot, also drawing on and demonstrating that they have knowledge about the history we’ve been studying. So rather than seeing if they memorize some content, and you know they can somehow show that on an objective exam, what I do is on exams, they encounter a historical claim. So a really simple one would be “Christopher Columbus discovered a new world in 1492,” and I ask them to think about that critically and to think about that the way a historian thinks about it. And I asked them to argue both for and against the statement like that. So they’re arguing, they’re not simply regurgitating information. That means they’re using evidence and, in order to use evidence, you have to have content knowledge, right? You have to have knowledge of historical context, you have to know some of the sources. And so I see all that stuff that we “cover,” so to speak, really is the raw materials that students are working with as they develop their ability to think historically.

John: You’ve mentioned using in general, a backwards design approach, and what you’ve just described is part of it, where you start with the ultimate learning objective and then you design assessments that would measure that. How do you prepare students to reach that level?

David: Well, I guess that’s another big question, right? One thing, I try to be very transparent about this. I mean it’s interesting how sometimes in higher education we’re not very transparent. And that means that I talk with students about what it means to think historically, what that looks like. I try to model that for them, of course, but I do it in a way that’s explicit. I mean, here again, I would say most history faculty across the nation model historical thinking all the time, but they may not be very explicit about what they’re doing. So I tr y to be explicit about what it means to think historically and then also to give students a chance to practice that in class. So it happens my classes are 80 minutes long and well, I’m very grateful that I’m not lecturing for 80 minutes straight. I would just find that very difficult to believe I’m engaging anybody… I’m covering stuff for 80 minutes. So, I really try to break the class session up and have students not only talking about primary documents—which is pretty common for history classes—but we also look at different perspectives from historians. Now, I don’t introduce them to the vast historiography on any topic that we’re looking at, but I do make sure that they know about at least two different perspectives on major issues of the colonization of North America, or the American Revolution, or the coming of the American Civil War. So they can see that there’s actually a debate out there, rather than assuming that, “Well, history is pretty cut and dry. Like, we know what happened, we’re just describing it.” So, I really try to introduce to them the ways in which the study of history is an ongoing dialogue among historians, that we’re always turning up new sources, and we have new ways of thinking about old sources. And so I give them examples of how historical thinking is working in the field and really ask them to wrestle with some big questions. Again, it’s at an introductory level, so they’re not going off and reading everything that Gordon Wood wrote about the American Revolution and then comparing that to Gary Nash’s interpretation or something like that, but they are getting little snippets of that sort of thing so they can really get a sense of the debate and they talk about that in class. I don’t actually hold formal debates but I’ll have small groups discussing a particular issue and then I make a sort of matrix on the whiteboard and they go up to the whiteboard and sort of vote for where they think this historian stands, and where they think a different historian stands on the issue, and then where their group is, and so they’re really playing around with those different positions. And then, of course, on the exams, they’re actually making their own individual arguments using the primary sources and the secondary sources.

Rebecca: One of the things that you said was “modeling, but making that explicit.” Could you talk a little bit about how you shift from just modeling modeling explicitly or being explicit about doing that?

David: Yeah, sure. Well, one thing I do is I have a kind of graphic that I show that I created that tries to describe in a basic way what historical thinking looks like because I do think a lot of students come in with a really simplistic understanding of history. Again, this notion that it’s pretty cut and dry, it’s a description of what happened in the past. And so I try to help them understand both, through my own comments but then through practice that, “Okay, yeah, we have descriptions about the past, but those are based on evidence that has to be interpreted.” And those descriptions, we assemble those into a narrative and built into those narratives is an actual explanation and analysis of what happened. So we’re really thinking about why things turned out the way they did, how different things are interconnected, and then finally—and maybe the most importantly—is significance. Historians are also wrestling with the significance of the past, different events, and developments. Not only the significance in a narrow historical sense like, “Okay, this event was significant because it led to another event,” but also a longer term significance, and then I think even a kind of moral significance. What can we learn from the past? How can we use the past to understand American national identity? Those sorts of questions. So we really kind of build up, there’s multiple levels of complexity there. And I try to get my students at least beginning to climb that ladder to some of those higher levels, even if they’re just doing it in a really rudimentary way. But then I hope they can really see the value of history that goes beyond just kind of rote recollection of something that happened in the past.

John: Do students sometimes resist moving into this more appropriate view of history? They’ve learned through elementary and secondary schools what history is. How do you break their expectations and get them into this more active role?

David: I’m kind of smiling, or maybe even grimacing right now, because I usually am able to win students over. I mean, there’s so much about the past that’s exciting, that’s interesting, that’s unexpected. When we start to dig deeply into some of these topics, they’re interested and they’re curious and sometimes they’re also pretty frustrated or even angry because they feel like they’ve been misled in the past. In other words, that maybe the basic information they had from high school—of course the quality of that instruction varies—but I do have a lot of students who come in who are upset. They feel like they’ve been misled. And I want to rush to add that I have many students who have had excellent teachers in the past and are already really interested in history in part for that reason. So I think there’s a lot for students to be curious about. Occasionally I do run into a student who is very resistant. Most students find that, “Hey, turns out this is a lot more interesting and engaging,” than the history they had been exposed to in the past, or it’s deepening and complicating the pretty good understanding that they already had.

Rebecca: Do you find that they’re surprised by the first assessment or exam because they have these prior expectations and then your methodology in your class is different from their past experience?

David: Well that’s something that I anticipated and so we actually have a practice exam, so to speak, very early in the semester. In fact, that example I gave a moment ago of Christopher Columbus discovered a new world and 1492 is a sort of maybe second week of class practice exam prompt. And so I have them do that outside of class and then bring it in and we essentially go over that together. They share their thoughts with their classmates, we talk it over, I give them a number of examples. So for a student who’s coming to class and whose keeping up, there aren’t really going to be any big surprises and I give them some samples of what the exams are going to look like from other classes. I used to teach a lot more modern American history and so I can easily just share some sample exams that students told me it was fine if I shared and I think that’s really helpful, especially if there’s a big change in expectations, then I think to be fair to students, you should really be clear with them about how the expectations might be different than what they’ve encountered in the past. And a lot of students actually like the assessments I’m using. I think they find them more interesting. They’re perhaps a little harder to study for in the sense that it’s a little less clear cut but they have a pretty good idea of what the major topics are and I give them some suggestions for what they should be reviewing in terms of sources. So they really come in thinking, “Well, I’m going to need to refer to least a couple of these primary documents and the perspective or argument of this historian.” We practice all that in class because again, that’s what I want them to be able to do. So I make sure that we spend a lot of class time essentially practicing the very thing that they’re going to be assessed on.

John: If someone were to come into your class, what would it typically look like? Are the students broken up into really small groups? Bigger groups? How do you have them engage with the material that way?

David: Well I typically have about 65 students in my introductory US history class. I have another class that I teach in a similar fashion that’s actually a writing e mphasis class though it’s a little smaller at 45. So an intro level class and environmental history. But in both of those classes, we will typically start out with the whole group and I make some introductory comments. Sometimes I give a brief lecture on background context. I mean, historians really do value context. This is part of how you make sense of the primary documents and so forth. So I do all of that and you would expect to see that in a history classroom, but then before too long, I’m giving the students some kind of a series of questions and they’re working with primary documents or they’re working with some short essays by historians and talking those through in their small groups and then we have various ways of reporting out. Like I said, sometimes I will send them to the board so they’re kind of recording their analysis on the whiteboard. Other times it’s just more like I’ll choose particular groups to respond and sometimes I just open it up. So, I think that’s all pretty typical except that I’m constantly going back and forth from maybe a brief lecture to small group discussions to a large group discussion. I suspect there’s more back and forth than you would typically see in a more traditional coverage oriented history classroom. Again, many instructors do use primary documents and they would have students talking about them. I think maybe where I’m going further is just that they’re going to have to write about those sources on an exam.

Rebecca: I think sometimes demonstrating that you really value a particular practice by spending so much class time on it really helps students understand how that’s a part of the discipline or a way a discipline works.

David: Yeah I think so, and many students really seem to appreciate having quite a bit of discussion time. I mean they find it productive, they have a clear task, they have something they’re working with. So at the end of the semester I do have a lot of students who say, “Wow that was so valuable to hear different perspectives.” And I think especially for first-year students, many of their classes are very large, many of their classes are lecture based, and so they don’t actually have very many opportunities to talk with other students in a meaningful way about the course material. And I realized that maybe at a small liberal arts college it would be a whole different story, but I’m at a regional comprehensive public university and I know that a lot of my students are in that kind of situation.

Rebecca: I think sometimes there’s a mismatch between the assessments that we assign and what our real objectives are…because it’s easier to assess other things than what we really want to know, and that tends to lead faculty to be resistant because of workload concerns. Can you talk a little bit about the grading and how you might manage that?

David: I have to agree with everything you’re saying there. That’s a real conundrum. The most meaningful assessments are also usually the most labor intensive and so I’ve had to be very deliberate in my case about what compromises I’m going to make. And one thing that I’ve had to do with the for and against essays, students write maybe six of these essays during the semester and they’re fairly brief. I really emphasize with the students that there’s not any room for BS in these essays. So there aren’t a lot of preliminaries and so forth, they just dive right in and make their arguments for and against. So that’s one thing that I’m doing. It’s not a formal essay because I don’t have time to go over all that extra content when I’m grading. I think the other thing that I do is most of the feedback I give on those essays I do with the whole class. So I give a kind of collective feedback where I show some examples. I actually fabricate a kind of weak response [LAUGHTER] and put that up on the screen and ask the students to assess it and then I show them a stronger response and then we talk about that. So they’re getting some examples of work that could use improvement and that’s generally a composite. I mean, I don’t ever show bad student work. But then often the strong examples…I’ve asked the students, “Hey can I share this with the class?” and we’ll take a look at that as well. So what I’m getting at here is one way I save time is a lot of the feedback I’m giving is collective feedback. If I were writing extensive comments on every one of these essays for 65 students, it just wouldn’t be feasible. I would have to do something else. So I’ve decided that’s a compromise that’s worth it.

John: When they’re looking at someone else’s work, the work that you fabricated, it’s probably a lot easier for them to recognize problems than when they try to diagnose it in their own, because they’ve taken ownership of theirs and they become committed. But when you prompt them by giving them this type of thing, it’s a whole lot easier for them to see mistakes and recognize them and perhaps avoid them in their own work.

David: Yeah, I think so, or at least that’s my hope. And, you know, they talk with their classmates about it. So they’re getting multiple perspectives on the shortcomings of a weak historical interpretation.

Rebecca: Have you or your colleagues seen a difference in the upper-level classes that build upon this like introductory class where the move has gone from coverage to uncoverage? Have they been more successful at the upper level or do you have any evidence related to that?

David: That’s a really good question and I certainly don’t have systematically collected evidence to respond to that. Now I do see a lot of these students again in upper-level classes and the students I’ve worked with before do seem well prepared to jump into deeper conversations, whether that’s using more primary documents or more extensive secondary sources. So I find that they’re well prepared for upper-level classes as far as what I’m able to observe among my own students. Now, we do still have courses that are closer to the coverage model on my campus, for sure. I mean, different colleagues of mine, they all have their own approaches. And just to be clear, I would never say my approach is the one right way to do anything. It’s something that works for me and that I think really serves the learning outcomes that are important in my discipline and in my department, but also to me. But I think there’s a lot of different ways to get to that. I guess I would say I don’t have any regrets when it comes to upper level classes. I don’t feel like, “Well, these students aren’t well prepared because we didn’t cover everything at the intro level.” I am always reminding myself, “Well, just because I’ve covered something doesn’t mean that anyone has learned it.”

Rebecca: We’ve been talking about this a lot. We have these first-year signature classes on our campus that are for first-year students that may or may not be continuing in a particular discipline but are really meant to help students integrate into our college and acting like a college student, et cetera. So they don’t have as many coverage concerns in terms of topics that we’re historically thinking about. So we’ve been talking a lot about how that frees you up a lot to really focus on some of these ways of thinking. So we’ve been having this conversation a lot, we’ve had a couple of recent podcast episodes related to that but we’ve also had these conversations on campus that have really made me, and I think some of my colleagues, think about, what is it that we really want students to learn even at the upper levels? We think about, “Oh we got to make sure we get through the whole textbook,” or whatever but really at the end of the day, I think sometimes we place value on things that we don’t actually value.

David: Well what you said really resonates for me. I’ve been teaching a first-year seminar for I think five years now and that’s actually in the Environmental Humanities where there is no canon. There is no textbook. And I’ve certainly had to build on scholarship that’s out there. I mean, I’m not just creating this out of thin air, but I didn’t have to worry about the pressure to cover some specific body of material. Nobody was counting on me to be sure to cover these 10 things. And I learned a lot from teaching in that first-year seminar context where the pressure for coverage is largely eliminated and I think it’s made me a better teacher in all of my classes. I really do believe that, because I’ve taken some of that spirit into some more traditional history courses.

John: I think this is an issue that comes up in many disciplines. I know in economics there have been discussions for years. The economics textbooks—just like history textbooks—keep getting bigger, there’s new chapters added. Since I was in the history class, there’s been a whole lot of time included, [LAUGHTER] you know, in terms of the chronological study. But more generally, in economics, there’s a lot of discussion about how difficult it is to cover all of this and how little students seem to come out learning. But there’s a trade-off when sometimes an introductory economics class is the only thing required for perhaps all upper-level courses, so there is some pressure to make sure you cover it. It’s a very active dialogue right now and there’s many attempts to reach a consensus, but I think this is happening in many disciplines, isn’t it?

David: I think so. Several years ago, I was reading some workshops on this very topic for the UW system and when I was doing that, I interviewed some colleagues in other disciplines. I talked with Angela Bauer, who’s now at High Point University in biology and I talked with my colleague Brian Martin in psychology here at UW Green Bay. And what I found was that they were really trying to go down this road in disciplines where the knowledge is more structured. As you go through you really are building on stuff that you learned earlier, maybe in more linear fashion. And in the history major, we’re looking at the history of so many different times and places that no one person can master all of that. So you don’t have quite the same expectation. But in any case, what they were doing was starting out with some big questions and this is something that Wiggins and McTighe recommend in Understanding by Design as well. So you can structure a course around big questions. And those questions should be interesting to most students who are engaging in the discipline particularly if they want to go on. And I find that really helpful. So I do that in my history courses but I think that you can do that in any discipline. Like what are the big issues here? What are the big questions in the discipline? And to answer those questions, you’re going to be using specific disciplinary ways of thinking and disciplinary tools, so you can really start to align things there. Here’s kind of the raw material you’re working with, here are the skills and other kinds of thinking tools you’re going to be using, and here’s what you need to be able to do in the end. If you can really line those things up, I think students walk away with so much more than if they are simply approaching it as, “Okay, we’re covering this, I’m going to take a test on this, and then probably I can forget most of it.”

John: Ken Bain has written about this extensively in terms of picking those big goals and developing it in What the Best College Teachers Do.

David: Yeah.

John: And we just recently had a podcast with Christine Harrington, where she talked about that in terms of building a syllabus, starting with those big questions and building your course around that. It’s a really important topic to help build student motivation and interests and tie everything together.

David: I really admire the work of both Ken Bain and Christine Harrington. Back when I got started with all of this, in addition to Understanding by Design and history, Lendol Calder was doing a lot of work in this area—and he still is today—and I think he pulled this word out of Understanding by Design, but “uncoverage.” So what can we uncover about discipline and disciplinary thinking? Because if we’re using the coverage model, yeah, we’re covering lot of things, but we’re also covering up many of the fundamental aspects of the discipline. If what you have in front of you is a history textbook, it’s written in a single authoritative voice. This is what happened, this is how it was, this is how it is. There’s something really misleading about that, because that’s not how the discipline of history really works. Where did all that knowledge come from? What are the debates? And so forth. So I really liked that idea of uncoverage and I wanted to acknowledge both Understanding by Design and Lendol Calder for sharing that concept with me. I think it’s a really helpful way of thinking about what we should be doing even in our introductory courses, or maybe especially in our introductory courses. What can we do to uncover the key concepts and ways of thinking in our disciplines?

Rebecca: When you decided to shift from a coverage model to more of an uncoverage model, did you decide to just jump in with two feet or did you take more of an iterative approach? I’m sure other faculty who are interested in this idea would like some guidance on that.

David: Well, yeah, so do as I say, not as I did. I really jumped in with two feet and maybe even got in over my head for a while. And you know, that’s something you can do from time to time. I don’t regret doing that. At the time I was a participant in the Wisconsin Teaching Fellows and Scholars Program—which I’ve now been co-director of for six years. It’s actually my final year—but back then I was just getting my feet wet in the scholarship of teaching and learning. And I believe I had a one-course reassignment that year and so I had just a little bit more time to try to implement a pretty radical makeover for my intro history class and there have been many iterations since then, including some pretty significant changes because, of course, you always learn a lot when you make big changes in a classroom. There are always things you would do differently. That said, I think you really can just choose some part of the class and really start working on getting to some of that deeper learning and assessing in new ways. You can try out new ways to assess with new ways to get students working more directly toward your learning outcomes.

Rebecca: You just mentioned the Teaching Fellows and Scholars Program at your institution. Can you tell us a little bit about what that is?

David: This is actually a UW System program so it’s statewide and it’s been around for probably 30 years, although it has evolved quite a bit ―I think by the late 90s, it was taking a turn toward the scholarship of teaching and learning—so it’s a year-long professional development opportunity for instructors in the UW system and we have many campuses here across the state and each of those campuses sends a couple of representatives to participate in this program. We have something we call Faculty College that meets every year and that’s several days, usually in late May, so kind of right after the academic year has ended. You have these really dedicated teachers going right back at it, like, “Hey, we’re going to spend almost a week working on improving teaching and learning.” So that’s actually a broader program that the Wisconsin Teaching Fellows and Scholars participate in that and that’s sort of where we launch the cohort each year. Then a little later in the summer we have a week-long summer institute and the fellows and scholars are actually designing scholarship of teaching and learning projects at that point. So they’re figuring out what kind of research questions would they like to look at in their classes, what kind of evidence of student learning are they going to gather, and so forth. Then we come back together again in January and do some troubleshooting on the project which they’ve been working on through the fall semester and they typically are gathering more data in the spring semester or more evidence to analyze. And then they actually do poster presentations in April at a statewide teaching and learning conference. So it’s a pretty substantive program. A lot of folks do end up continuing to work on their project and ultimately publishing something. It’s not necessarily a requirement, but many of the participants do go on and remain active in the scholarship of teaching and learning. For me, I think it’s really important because it provides a community in the UW system around teaching and learning, and so while you’re actually working on your projects for that year, you have some support. You have a support network to help you figure out how to handle your data analysis or how to get through the IRB process or whatever it might be. But then that community continues on afterwards and so we’ve had quite a few…thinking of three or four different books…published on the scholarship of teaching and learning where many of the participants that were in the book project came out of the Wisconsin Teaching Fellows and Scholars program so there’s the signature pedagogies book, it’s called Exploring Signature Pedagogies and then Exploring More Signature Pedagogies, two wonderful books and Regan Gurung and I recently published collection of essays on Big Picture Pedagogy, and that’s taking a kind of interdisciplinary approach to the scholarship of teaching and learning. And most of the folks in there came out of the UW system, and many of them were former teaching fellows and scholars. So it’s a really interesting program that I think really enriches what we’re doing here in Wisconsin.

John: That sounds like a wonderful program. I wish we had more of that here.

Rebecca: Yeah. Do you have a little bit of advice for people who are interested in starting in the scholarship of teaching and learning? That’s one of the things that we’ve talked about quite a bit on this podcast and anytime we can get someone to provide a little insight into getting that started…

John: In fact, Regan was on a few months back.

David: It can be intimidating to get started in the scholarship of teaching and learning, especially if you’re coming from a discipline that doesn’t normally do research on or with living people. Like, you know, for me as a historian, my research subjects are typically dead and gone. [LAUGHTER] So it’s quite different to think about doing research on my student learning or with my students and their learning. So I think it’s helpful to start small and just to try to think of the question about student learning that you can answer using methodologies that you’re already pretty familiar with. So you don’t want to tackle a project that’s going to require you to learn how to do advanced statistical analysis if you don’t have experience with that already. So I think it’s important to realize there are a lot of different ways to collect and analyze evidence of student learning and if you continue to develop as a SOTL scholar you will explore more and more of those ways of gathering and analyzing evidence. But it’s completely fine to start out with something that you’re already comfortable with and familiar with and you’re really asking a question—or you can start with a simple question about—what are my students learning in this particular area? I know there’s a struggle here, I know there’s something difficult here, and I’m going to pay some extra attention. I’m going to look at this in a more systematic way than I normally would just in the regular course of grading some papers or exams.

John: In your paper on the rise and fall of the coverage model, I think you noted that there’s a growing emphasis on the scholarship of teaching and learning within history and that seems to be resulting in a growing network of scholars who are working in this. Could you tell us just a little bit about that?

David: Sure. Let’s see, back in 2006 a group came together called the International Society for the Scholarship of Teaching and Learning in History and I don’t know exactly what the membership or the participation in that group is but it’s a substantial group of history scholars who are really interested in promoting the scholarship of teaching and learning in history. Along with that, the American Historical Association now maybe for a decade or so has been working on something called the Tuning Project and that’s focusing on learning outcomes for history majors and that project has involved many, many people across the country in all different subfields of history thinking about what is it that we really expect and want history majors to learn and that transcends any particular content area. In other words, if you’re asking that for about history, and we’re thinking about historians coming in who are, not only United States historians, but looking at many different periods of time and in places around the world, you can’t just come up with, “Well here’s a list of twenty thousand facts [LAUGHTER] that we expect all history majors to know.” It was apparent that wasn’t going to work. And so what the Tuning Project has done, it’s allowed our discipline here in the United States to really think carefully about historical thinking and the basic skills and practices of being a historian and that at this point, it’s all very well spelled out. And history departments can then use that to improve the learning outcomes at their program level and you can take that down to the level of individual courses as well. So I think between the scholarship of teaching and learning and this assessment oriented Tuning Project, that we’ve had a real increase in interest and commitment here when it comes to thinking about what does it mean to learn history and to learn how to do history?

John: The last question we always ask is, what are you doing next?

David: I’m happy to say I have a sabbatical a year from now, just a one-semester sabbatical in the spring of 2020, and I’ve really become interested in environmental history and more broadly environmental humanities (I mentioned that earlier in terms of the first-year seminar I’ve been teaching). And my project is going to be to try to articulate a pedagogy for environmental history and environmental humanities and what I’m thinking of there is that there’s more to this field of study than the usual kind of content and skill mastery. And I think there are a lot of areas where this is true, but there seems to me to be a kind of affective component here, an emotional component, as we’re facing up to the environmental degradation and numerous environmental crises in our time. I say facing up to—sometimes we’re not facing up to those problems—and it can just be overwhelming from the standpoint of a learner—especially a young person—to really come to terms with the content that we would have to look at to the areas of environmental studies more generally. So I’m looking at contemplative pedagogies to think about one way to help students come to terms with and to deeply process what it is we’re studying when we look at problems with sustainability. For example, climate change. I’m really excited to dig into that next year.

John: Those sound like fascinating topics, and they should be fun to look at. And it’ll be an interesting challenge working with those issues, especially now.

Rebecca: Yeah it sounds really intense, but really needed.

David: Yeah, I think it is going to be necessary to grapple with those issues going forward.

John: Thank you for joining us. This has been really interesting.

Rebecca: Yeah, thank you so much.

David: Oh, thank you. I really enjoyed the conversation and my tea cup is empty, I don’t know about yours but I think I’m going to have to go get another cup.

John: Yeah, mine’s empty too.

Rebecca: Yeah mine’s getting really close.

John: I don’t know if you can see it, but we have a whole table covered with tea back there in the back of the room.

David: Oh I see that now.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

71. Small Teaching Online

Face-to-face classes have been offered for centuries. Online instruction, though, is relatively recent and many faculty that teach online have little prior experience or training in online instruction. In today’s episode, Flower Darby joins us to explore some easy-to-implement teaching techniques that can be used to help improve the learning experiences of our online students.

Show Notes

Transcript

John: Face-to-face classes have been offered for centuries. Online instruction, though, is relatively recent and many faculty that teach online have little prior experience or training in online instruction. In today’s episode, we explore some easy-to-implement teaching techniques that can be used to help improve the learning experiences of our online students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Flower Darby an Instructional Designer an Adjunct Instructor in several disciplines and the author (with James Lang) of Small Teaching Online, which is scheduled for release in June 2019. Welcome Flower.

Flower: Hi John. Hi Rebecca. Thank you for having me. I appreciate that. It’s good to be here.

Rebecca: We’re really glad that you’re joining us as well. Today our teas are:

Flower: I am drinking Builders tea. Good, strong cuppa here.

Rebecca: Sounds yummy.

John: We have some of that next door. I am drinking ginger peach gree n tea.

Rebecca: I have my Golden Monkey again today.

John: We ran a faculty reading group here in the Fall semester of 2017 based on Small Teaching. Many faculty found that to be highly inspirational and we had over 100 people participate in that. One of the things that came up quite a bit is how this might be applied online. So there’s a lot of people interested in your forthcoming book. Could you tell us a little bit about how this book came about?

Flower: Sure. So Jim Lang came to my campus—Northern Arizona University—in January 2018 and delivered a talk on Small Teaching and as we know, the book has been very impactful for faculty around the country and around the world. And while he was at our campus, when it came time for the question and answers, somebody raised their hand and said, “Sure, but how do you do this online?” And Jim’s immediate response was, “That’s the first question I always get at every talk that I give,” and he said, “I don’t know. I would need a co-author because I don’t know how to do this online, but that would be a great book.” So I thought about that for a few days and then I approached him and I said, “Pick me. I would love write that book with you. I can see the value of it, I can see the need for it.” So that’s how the conversation began.

Rebecca: How does this extend the approach that was used in Small Teaching?

Flower: Well, it follows the same principles for certain that there is learning science that we can draw on to help us make the everyday decisions in our teaching and learning that have really an outsized impact on student learning and outcomes. So there are little things that we do on a day-to-day basis and we can draw from the research to discover what will have the most impact. Again, understanding that in order for faculty to really be able to implement something new, it’s got to be feasible. It must be doable. The daunting overhaul of a major course redesign is so off-putting that most faculty won’t get around to it, myself included. When I have gone to multiple workshops and conferences and sessions or read about an approach. And I, “That is a great idea,” and I spend about five minutes thinking about how I might incorporate that into my class and then I say, “Too much work. Too much time. I don’t have that time available. I don’t want to implement something that’s only half baked,” and the idea gets left out. So in our online classes, there are so many things that we can do that are on that small scale but will have that outsized impact on our students’ engagement and their learning. And so that’s what this book sets out to do, is to explain a lot of those principles and draw on the research that we have to show faculty how they can make these changes in their online classes.

Rebecca: We’ve talked many times on our podcast about the lack of preparation for faculty teaching in general and that’s certainly true for online teaching. You might have taught a face-to-face class, and then all of a sudden, now you’re teaching an online class and boom, you have to figure it out. Can you help us think through what are some things that faculty can do as they’re new or getting used to being an online teacher?

Flower: Sure, and I think that’s really the point here. Centuries, millennia, compared to the way that we teach and we coach and we mentor face to face, or even as we’re doing here using video conferencing software, but it’s a real-time interaction. Well, online teaching is very, very recent, say 20 years or so. And faculty don’t have the experience that they bring into the physical classroom. You may have heard of the phrase of the apprenticeship of observation coined by Dan Lortie. And this is the idea that by the time a teacher steps into a classroom to teach, he or she has had years and years of experience in a physical classroom being a student and observing what happens and how things go and thinking, if somebody chooses to be a teacher, then they’ve clearly put a lot of thought into how they want to teach. Well, we simply don’t have that for online. I do expect this to change in coming years. But the fact is right now that most of our faculty have either never taken an online class or if they have, it may be a very limited experience, not the years and years that they came out of K-12 with. And the same, quite frankly, is true for our students. They’re also pretty inexperienced at an online classroom. And the way this pans out is that literally faculty and students both don’t know what’s supposed to happen in an online class. They don’t have the social norms, they don’t know what the classroom looks like. If you think about it, when you walk into a physical classroom to start teaching, you know what’s in the room and you know what’s supposed to happen. You see the desks or the tables, you see a lectern at the front, you see a whiteboard or a projection screen, and students and faculty understand what is supposed to happen. Students go and sit in the desks, they face front, they wait for the faculty to come to the podium. It’s rare that a student would walk into a classroom and at the beginning of the hour, just step into the lectern. Students know that’s not what they do. But my argument is we don’t have that kind of social norming convention for online classes…yet. I think it’s coming, but right now many of the people who find themselves in our online learning environments go into that space, and they don’t know what things should look like, they don’t know where the light switch is, they don’t know where the desks are, where the whiteboard is. So just that whole lack of experience is rather disconcerting. And it’s hard to know what to do. Faculty don’t have experience—they have haven’t seen models, students are equally unprepared—so there’s a lot of work to be done here just to understand what should happen in an online class, what the furniture is, where it should be to facilitate learning. That’s where those gaps happen for faculty. You ask, “How can faculty prepare themselves?” I could talk for days about that question. It’s a growing need and some institutions are beginning to recognize the importance of doing a much more thorough job of preparing faculty to teach online. But I will argue that those institutions are still pretty few and far between. I would say, based on my research and my experience, the vast majority of faculty who are teaching online have not had specific development in that area. They have not observed peers’ classes. In fact, what can happen is a negative effect. Very commonly, when faculty begin teaching online, they are handed somebody else’s content. We’ve seen that happen and that’s a mercy in a way because that way faculty who are new to teaching online don’t also have to develop the course. But what can easily happen is that the content that might be given to a new faculty member might not actually be exemplary in the design and the delivery of that material. So then what happens is the only experience that faculty get is observing the content and the structure of a less than ideal example and then that’s the model that they have and they think, “Oh, I guess this is how it is”. So work can be done on developing better exemplars, better development programs. I believe, as faculty are coming out of online graduate programs down the line a little way, I believe, will have better experienced faculty and students. A lot of research going on in this area, but that work is all to be done.

Rebecca: As you were talking, I thought, you provide a nice model, it’s a nice way of thinking about it, you don’t know where the furniture is.

Flower: Mmm-hmm

Rebecca: It sticks with me. I was thinking about that the experience that a lot of faculty and students have is more in the realm of social media and so they’re looking for cues that are similar to those kinds of environments. The activity that’s happening in those environments is really different than the kinds of activities we would expect to happen in an online platform for learning.

Flower: Right, that’s a great point. We interact with other people so much online and on our devices using social media and what’s interesting to me is that we can really engage with people in those online spaces. Somebody tweets something that’s a little bit incendiary or provocative and you get all kinds of people jumping in and commenting and you know, sometimes things get heated, or a really heartwarming moment is tweeted or shared on Instagram, and people are all over that post. But the opposite is kind of true in our online classes. Indeed, I feel like we could bring in some of the techniques from social media into online classes. I’m not saying that faculty should all have a component of Twitter or Instagram in their online classes, but what I’m saying is that it’s possible to deeply engage people in online interactions. And that’s not a feature that, I would say, generally characterizes online classes—we usually hear the opposite, that it’s not engaging, it’s difficult to drum up those discussion posts—and I feel like if we could draw some of those principles from how we interact with people online, in social media, of using our devices, if we could bring those into the online classes right away, we’ll see more engagement and engagement precedes learning. Students have to want to be there in order to learn when we’re engaging them and if you could imagine posting a discussion post and then you can’t wait to see what people are responding. We do that all the time on Facebook or Twitter sending something out and then, “Oh let me see! Did people like that? Did people say anything?” And we just naturally are drawn into those spaces to check and see what are people’s reactions? Well, if we could design that kind of a discussion board for online classes, where it’s so interesting and engaging that people want to rush back and see who’s talking to them, who’s replying to them, that would go away way to improving the online learning experience for both faculty and students.

John: That’s not an experience though that many people teaching online find in their discussion forums. Are there any hints or tips that you can give people to make their discussion forums a bit more engaging so that students don’t wait until the last minute to do the standard three posts or whatever is required in that course?

Flower: Great question, John, and a big one. And again, thinking about Small Teaching ways of making small changes, I heard of an example recently where faculty asked students to reply to their peers posts using a GIF that just represented—one of those funny moving little images that sort of expressed—their reaction. And that’s an example of bringing in new ways of engaging and it’s not rocket science. It’s also perhaps a little more fun, which is important to bring into an online class. A great way of sort of getting students to think differently. But if that idea doesn’t resonate with you, maybe you might want to try offering options in your discussion board questions. I’ve supported over 100 faculty, I might even say, hundreds of faculty in the design and development of their online courses and what I see sometimes is one question for students to answer and oftentimes it’s kind of black and white. It’s hard to discuss a question like that. So first of all, craft questions that are discussable, that there’s some debate around that you can make different arguments or points of view. Tie those questions to students’ experiences. How is the content impacting them personally? Where do they see these concepts in their own life and experience? And, even better, provide three or four different questions that students could choose to respond to and then ideally, everyone isn’t all talking about the same question, so that’s more of a natural way of fostering some conversation in an online discussion.

John: One of the nice things about tying it to personal things, I would think that that would also help build more of a sense of community within the group because the students get to know each other a little bit better, which may affect their engagement in other activities,

Flower: Right. Anything that we can do to increase the value and the relevance of what we’re asking students to do online is hugely impactful, and it doesn’t have to take much. I have a colleague who teaches an online First-Year Seminar course, which in a way is a bit of an oxymoron because First-Year Seminar courses are often designed to really hook in our first-year students who are transitioning to university life, but she was tasked with developing and teaching a really highly engaging and supportive Freshman First-Year Seminar class. And one of the things that she does is she brings in a discussion board and one of the prompts is, “If you could be a superhero, what would your superpowers be?” And again, maybe on the surface some people might think that’s a bit trivial, but what she’s doing is she’s getting students to talk about character traits and hero qualities and concepts that rely and relate to the material that they’re engaging with… yet in a fun and a more personal way. And it certainly does a lot to foster those relationships that are so important for online classes to build that community. Absolutely.

Rebecca: I think one of the methods that we hear a lot about in terms of online learning is the ability to do quizzing and retrieval practice and interleaving through quizzing. But are there some other ways that we can integrate some of these evidence-based practices that aren’t maybe the typical solutions that we tend to think of online?

Flower: I think one of the most underutilized functions of the Learning Management System is what we call adaptive release or conditional release. And I actually want to pause here and say that these Learning Management Systems have come a long way in recent years, and they still have a long way to go. [LAUGHTER]

Rebecca and John: Yes. [LAUGHTER]

Flower: For many faculty and students, the functionality is lacking, the furniture is stark, they’re not attractive places to be and as I said earlier, engagement precedes learning. If you were to ask yourself for online faculty who are listening, “Do you want to be in your online class?” I suspect many faculty would struggle to answer with a resounding “Yes.” And so here’s a shoutout to our LMS developers to think about space design and the experience of students and faculty in these spaces. Having said that, there is some very interesting functionality that is oftentimes underutilized and I would argue that’s because, again, faculty may not have the preparation and the exemplars to begin teaching online. With adaptive or conditional release—it’s called different things in different systems—you can set a task that then opens up the rest of the content in that module. And I love to use this. You can use it equally effectively at the end of an online module or at the very beginning to open the next module. Now what you can do with this is you can embed retrieval practice exercise. Or, you know, drawing from Jim Lang’s book, Small Teaching, a predicting exercise works equally well. A curiosity provoking exercise, and all it really has to be is an assignment where students submit whatever it might be. A two sentence summary of what their big takeaways were from the previous module, or predicting what might be in the coming module or posing some questions… this can be written, it could be a recorded submission for students who might find it easier to talk through their ideas. Once students submit that element, then the rest of the module opens. Before they do that they can’t access any of the content. These things don’t even have to be graded, you can just set them to be worth zero points, but they are mandatory because the students can’t proceed with the content until they submit them. So when you think about that feature, there’s a lot of creative things that you can do that don’t impact faculty grading time. That’s a big tenet of the new book is we can’t overburden faculty with grading and yet tie into those practices that we know from the research are effective.

John: One of the chapters of your forthcoming book is on fostering student persistence and success. Could you give us perhaps a strategy or two that might be useful in encouraging student persistence? Because I know one of the problems in online classes is they often have higher drop, fail, and withdrawal rates than face-to-face classes. What are some techniques that faculty can do to help improve student persistence in the class and the program?

Flower: That’s right, great question. As you point out, the attrition rates in online classes are remarkably higher. And we also find that for students who are less prepared for higher education, if they don’t succeed in that class, then the odds increase dramatically that they won’t actually persist and attain a college degree, and that’s a problem. But as I was saying earlier, a global concern is that online classes are not nice places to be. And if your listeners have any pushback on that, please feel free to reach out and engage with me on that assertion. But what can we do to just make the place a little more pleasant? How can we be warm and friendly and supportive and encouraging? How can we allow our humanity—even our personality—to show through? I was speaking with a good friend and a colleague of mine just a few months ago when I was delivering a little talk about this book and he was telling me though he’s been teaching online for 10 years, that he’d never thought of just being himself in his online class. And he explained to me that he loves teaching in person—he’s quite a character, super dynamic, very engaging, funny, loves to interact with his students in the classroom—and yet, he told me when he goes into his online class, it’s like a robot. There is no trace of his personality. And other people are saying this too, just be yourself in those online classes and make a deliberate effort to infuse warmth. But a specific strategy that people might want to try is to assign a goals contract as one of the items that are due in the opening module—or the orientation module—and a goals contract, you’ll see different kinds of variations around, but here’s the two pieces that I really like. A lot of people are talking about assigning sort of a memo of understanding or a contract where students agree that in this online class, they should schedule set times, they should plan on X number of hours per week, they should reach out immediately if they have questions. People are doing that. I like to embed a different element as well, which is to require students to set a couple of goals and it can be literally two. What are two goals that you have for your learning, or your success, your ability to earn an A in this class? And then an interesting twist is to ask students to identify one potential challenge. It’s still the case, I have my students all the time saying, “Well, my computer is in the shop, [LAUGHTER] it’s sort of all of a sudden, it busted and now it’s at the technician and I can’t do my online tasks.” So helping students to think in advance about a scenario such as that and of course, in that particular case, many campuses have computer labs or libraries where students can go and access another way to get into the course but maybe they haven’t thought about it in advance. So in the goals contract, ask students to set two goals, identify one potential challenge that might come up, and identify a strategy for how they can address that particular challenge. And certainly, identifying one challenge is not going to cover the range of things that happen in life during the course of an online class, but I think it sets the tone to get students thinking that one little hiccup doesn’t mean that we’re all done with this online class and we just have to sort of fade away and stop participating. And then what you can also do, periodically throughout the class, is you can ask students to revisit those goals that they set for themselves. How are they doing with that? What kind of progress are they making? Are there some strategies that aren’t working for them? Do they need to recommit to the intentional and deliberate scheduling of their class time? Just helping students be very explicit about what their plan is to succeed and finish the course.

Rebecca: What I like about what you’re saying is, it switches from really having the faculty member impose everything, and have the students be co-authors of the class to some extent, and they have some ownership over the space, which generally means that they’ll probably commit more.

Flower: What we know about online learning is that students must have a higher degree of self regulation, self direction, they must be more motivated, and be able to manage their time well. And if students don’t have those things, it’s much less likely that they’ll persist and finish an online class. And yet, when you think about it, online classes work directly against a student’s ability to do those things and here’s what I mean. When you are teaching in person, when you’re a student in an in-person class, you know that every Tuesday and Thursday at 9:30 you’re supposed to be in the classroom and it’s a natural way to help students hold themselves accountable for doing the work. Now I know sometimes students come to classes and they’re not fully prepared, but there’s still that built in mechanism where they’re going to be in the same room with their faculty member with other students. There’s a social element of accountability that’s like, “Well I know I’m supposed to show up and I should have my stuff done,” or “There’s a test next week and I need to be ready.” Well those real time interactions and those interactions with physical people don’t tend to happen in the typical asynchronous online course. Very often—I would say 99% of the time, probably—an online student is sitting at home by himself or at the coffee shop by herself. If she has a quick question about something, she can’t do what she does in the classroom and say, “Hey, did you understand what we’re supposed to do on that particular assignment?” or “Hey, faculty member, can you just re-explain that? I’m not quite there yet.” There’s no way to get that immediate response, that immediate quick guidance that might take two minutes in a physical classroom. So, students don’t have the accountability, they don’t have the physical presence of the instructor or the student, and so we have to go above and beyond in our efforts to build in structures that help students develop the kinds of self-regulated skills, the kinds of self-directed learning skills. Many of our students are not coming in with those skills already but we know students who do have those skills will be much more successful, so let’s build that into our curriculum. Let’s help them develop some of those, let’s talk to them about the importance of monitoring their own learning, and let’s structure exercises that will help them to do this. I’m pulling a lot of this material from Linda Nilson’s book—It’s called Creating Self-Regulated Learners—Although that book is not necessarily focused for an online environment, I think it can be hugely helpful to our online students to be very transparent with them about the importance of developing these habits, these behaviors for success. And as I said, structuring exercises and graded assignments that help them to do that, to hold them accountable.

Rebecca: Following up with what you just said, there’s a chapter in your forthcoming book called “Creating Autonomy.” Can you talk a little bit about small ways that we can give students autonomy in the classroom and in an online space?

Flower: Sure. And again, let’s be sure to keep that focus on small, doable, feasible changes, things that you could do in maybe a 15-minute work session and have it rolled out for your online class. One thing that we could do is to develop a self-enroll group structure. Many online faculty like to bring in collaborative learning tasks to, again, foster that community and the peer to peer instruction and learning that is so important as we know, but I think oftentimes we sort of assume that what we should do is purposefully group students, and there’s certainly value to be found in designing purposeful groups. But what can also be very interesting is to allow students to enroll themselves in groups that might cover a range of different topics. For example, sometimes I teach Educational Technology online classes. And if I were teaching that class today, I might offer five different groups that students can sign up for on a first-come first-serve basis. And one might be virtual reality, and one might be mobile learning, and one might be writing in digital spaces. So students could naturally choose a topic that they’re more interested in pursuing and when students have that level of autonomy, to make that choice of what their going to focus on, that’s one way of embedding just an opportunity for students to exercise that autonomy. Another even easier way I’ve already mentioned here is to offer students a choice between whether they want to submit a written task or whether they prefer to record on video or audio. Students carry these amazing devices in their pockets all the time with high- tech recording equipment embedded right in them. And students love the freedom of just being able to talk through their ideas, their responses, you can get a much more authentic response from students. Teach them how to use the recording software, or how to upload the video or the audio clip into the LMS and now you’ve got an easy choice that you can give students. If you prefer to write this, go ahead. If you prefer to record it, do it that way.

John: One thing that struck me is I used VoiceThread last year in an online class and I expected they’d actually use the video option with it very often. I gave them the choice of whether they use just voice or voice and video or use a video recording and yet none of them ever presented on video, which surprised me, given how common that is in social media. Why might that be?

Flower: Well, sure. I also require video discussions in some of my classes. And what I have learned is that people are nervous, especially in an academic setting about how they come across on camera. I feel like audio is a little bit less threatening, but sometimes people don’t like the way they look. And, you know, faculty too. [LAUGHTER] A lot of faculty are uncomfortable with recording on a video, and yet, it’s the way of the future. So right now, currently, I’m teaching my graduate level class on technological fluency and leadership. I require those video discussions and I say to them, “Are you nervous about doing this? Well, I want you to do it anyway,” because video interviews right, over Skype, or Zoom, as we’re doing here today, or video resumes. These are a thing that are happening and helping people to get more comfortable with showing their face on camera. I also talked to them a lot about the importance of seeing their peers faces and how much we can learn just from that. In the program that I teach now, students tend to take classes with the same people. But in my class, they always say, “I’ve never put a face to a name, how nice it is to see you,” and it makes a huge impact in terms of that community element. But I talk to my students very explicitly. Now, it’s also really important to think about situations where a student may not want to represent their face and there can be very good reasons. I had a tragic situation just last year where a student did not post the video, she posted a static picture of herself. I came to find out at the end of the semester. When she did that I was like, “Oh well, dock a few points, whatever. That was weird. Why’d she do that?” I later found out that she had been in a domestic abuse situation and she was ashamed of the way that her face looked because it was still very visible—the damage—and it just struck me to the core. An arbitrary decision that I made that it’s so important to talk to each other and look each other in the eye and she had a really, really strong reason for not wanting to do that. So back to that topic of offering a choice, what I do now is I tell my students, “If there’s a really good reason that you don’t want to show your face on the video, please send me a quick note. You don’t even have to tell me details, but just explain that you’re going to choose to do this other thing instead,” and posting a static picture is still pretty effective. So I think it’s very important to remember that our online students are people. They have lives and we need to be thinking about the decisions that we’re making in our teaching and how that might come across to a student… how it might induce anxiety in ways that we never anticipated.

Rebecca: One of the things that I wanted to follow up on and see discussed, the self-enrolling groups and collaborative work online. I think we have clear ideas about how that might work in a physical classroom, but not always a good clear way of how we can coach students through collaborative learning online. So even small, quick things that came up in Small Teaching, like Think-Pair-Share, you can envision how to do that in a classroom, but maybe have no idea how to do that in an online classroom.

Flower: It’s a great question, Rebecca. And I know that there’s actually a lot of pushback from online students and sometimes online faculty about the value of collaborative learning activities. It just so happens that my husband is in an online Master’s program right now and so I’m living with the student experience. And it’s frustrating to our online students—many of whom are not traditional 18 to 24 year olds, they might be returning adults—and one of the reasons (in fact, a primary reason) that our online students choose that modality is because they have busy lives. A big percentage of our students have jobs, families, obligations, and they need to do their work when they have time. That might be 8 pm, that might be 11 pm after all the kids are in bed. It might be 6 am, I like to do my online class at 6 am. When you require students to work in groups in an online setting, you’re removing that degree of scheduling flexibility that students value in an online class. So if you choose to require online activities, I have certainly moved towards lower stakes and opportunities that don’t require real-time meetings between students online. And you mentioned a great one. Think-Pair-Share can be set up in an online class. So there’s lots of ways that you can do this but the first thing that came to my mind is you could set up groups of two, and you could auto-enroll students in a group of two, and then they have their own individual discussion boards. In most Learning Management Systems when you have groups you can have kind of a private discussion board where students can interact with each other there, or I’m a big fan of letting some of the learning come outside of the Learning Management System. So let students know who their buddy is, have them exchange phone numbers, and they can just talk on the phone. Sometimes we forget those simple solutions. But a Think-Pair-Share—and so many ways that you could set this up, you could change it from module to module so people are always working with somebody else—just share an idea, discuss something, take it offline, come back and just write or record a quick summary of how that interaction went. When it’s not such a high stakes assignment, students can better engage in those opportunities. It’s so much easier to find 15 minutes to talk with one person than it is to find an hour with four working adults who all have family obligations. So I love the idea of lowering the stakes and embedding lots of little opportunities for students to work in pairs or in groups of three where it’s easier to coordinate. There’s less pressure about the online group member who never does the work—sorry, but that’s a thing—and just help students see other ways of interacting. Now, with my instructional designer hat on I want to remind us of the importance of making sure that online collaborative work aligns with the outcomes of the course. Very important to think about why you’re asking students to work together. Does this actually relate to what you want them to learn and get out of the course? Very important to pause, ask yourself some of those questions before you randomly assign group work because we should have group work, which I’m guilty of doing. [LAUGHTER] It’s an easy thing to do, “I guess we should have group work,” but really pausing to think carefully about the purpose of that. And then again, maybe thinking creatively about those lower stake ways of connecting students and facilitating some more authentic interactions. Maybe they’re going to text each other. That’s fine, they’re talking. We do a lot of talking on text these days. Help students connect in ways that are not so stilted, which is often what we see in the use of the discussion board and the LMS.

Rebecca: I found that too, I use Slack a lot in my classes, because it’s a common platform for designers and people in that realm to communicate professionally, and they love it. It’s convenient, it’s on their phones, takes it away from a clunky interface…

Flower: Sure.

Rebecca: .. some of the LMS’s have and it’s really productive. And they’re able to do that midnight chat with each other.

Flower: Yes, absolutely. Again, let’s think creatively about tools that students already have. I honestly believe that a lot of Learning Management Systems actually raise barriers to student learning because most of them—although this is getting better—most of them don’t have a super robust mobile app and so a student, really to engage with coursework, has to find a place where they can sit down and log into the computer and access the course and jump through a million hoops before they can even get to where the learning is. Whereas if we take some of that learning into apps that they’re already using or things that they’re doing on their phone anyway where it’s in their pocket, we can communicate in real time. Now I need to exercise caution here because many faculty think, “Oh great. I’ll do Slack, and I’ll do VoiceThread, and I’ll do Flipgrid, and I’ll do Twitter, and I’ll do Pinterest, and it’s just going to be so interesting and fun.” Well, if there’s a reason for using some of those tools, absolutely. If those tools are just shiny entertainment—bells and whistles—then you may want to think again. Another important consideration if you’re asking students to use tools that are not in the Learning Management System is whether those tools are fully accessible for students, whether there’s any fee that’s involved, whether students might have to set up a new account with a new password, that might just be a hassle. So really you want to think carefully about what you’re asking students to do. Are the tools fully accessible and usable and cost friendly? Do they support your learning outcomes? And yet, if a tool that you’re thinking about using passes all those tests, then by all means jump right in. This semester I’m using Remind which is the simplest tool on the planet and the most effective. [LAUGHTER] It’s more in use in K-12 currently than in higher ed. It’s simply a text app that anonymizes people’s phone numbers. So I invite my students to sign up for my Remind list. I don’t require it. But then I can easily send a quick little 140 character reminder, “Don’t forget this assessment is due on this particular day,” or “New content has just been released. Login when you get a chance.” The message goes right to where the students are and because I make it optional, nobody is required to have the annoying instructor on their phone all the time. But students who want some additional support with managing deadlines and the class experience really appreciate the use of the simple tool called Remind.

Rebecca: I think one of the things that we know from a lot of evidence-based practices and books that have come out—including Small Teaching—is that frequent feedback is useful. But we also know that frequent feedback can seem really daunting to a faculty member, and time consuming. So are there ways that you would suggest managing some feedback opportunities online, but keeping it easy, quick, and reasonable?

Flower: Sure. Another great question. Another underutilized approach—at least in my experience supporting the faculty that I work with—is the ability to embed feedback into auto-graded multiple choice or true- false types of quizzes within the Learning Management System. So in most of these systems you can design feedback that will show up for students as soon as they submit the quiz. You can set those quizzes to show students which questions they got right or wrong and in the wrong answers you can embed feedback that says, “Please review pages 32 to 35 of this chapter. That is where you’ll find this information.” Similarly, you could encourage or embed challenging feedback and by that I mean, “Great, you totally know this material. If you’re interested in learning more, you may want to check out this website or this resource,” to offer students a range of experiences and engage students at their different levels of experience with the content. To be fair, setting up that kind of embedded feedback takes a little bit of time in the first place, but many of us teach those online courses over and over again, and once you’ve done that work, you can benefit from it time and time again. If you’re not sure how to do that in your Learning Management System, just about every institution has a Learning Management System support team with instructional designers or system admins, help desk folks who can walk you through the creation of that kind of embedded feedback. And it’s timely, it’s right there when the students are thinking about that problem in the first place, it’s relevant, and it’s a great way to automate some useful feedback for student learning.

John: You have a chapter in this forthcoming book on developing as an online instructor. Are there some general suggestions that you can give to faculty who’d like to improve and develop new skills or improve skills as an online instructor? Besides buying the book, of course. [LAUGHTER]

Flower: Right. Great question. Again, what this comes down to for me is that it’s just new. It’s just new for a lot of people. And to be honest, I suspect that many online faculty didn’t really set out to be great online faculty and many faculty are not finding the experience quite as rewarding as they might find the classroom experience. In fact, I have some data to back me up on that. The 2017 survey of faculty and information technology from EDUCAUSE Center of Analysis and Research found that of over 13,000 faculty respondents, 91% said that they don’t prefer to teach online. 9% said, “Great, I love to teach online.” That’s 91% of us who would rather teach anywhere else. [LAUGHTER] So how can we cultivate that joy, that buzz that we get in the classroom? We love teaching. If we didn’t, we wouldn’t be doing it because we don’t get paid enough. How can we cultivate that for ourselves? Now a barrier or a common challenge is time. Who has time to go and learn how to do a whole new skill? It’s different than teaching in person. But there are, again, small things that we can do to increase our awareness. One of the most effective things that you can do as an instructor is to seek an experienced and a thriving online instructor and ask to shadow that class. Ask to be added into that class shell and just observe. How does that person interact with students? What are the structures? What is the teaching? What happens while the class is in session? That can be hugely impactful, it’s usually free, [LAUGHTER] and faculty can invest the amount of time that they have. In fact, this is how I first got started with online teaching over 10 years ago, is before I was going to teach a class. Luckily, this offer was made to me the semester prior to my first online class, just to observe another class and see what happens in there. Simple structure could easily be set up for faculty who are scheduled to teach a new class in the Fall, have them observe or shadow a class in the Spring or the Summer, and yet an often overlooked solution. Certainly there’s lots of online resources. There’s podcasts like this, there are blogs that people are writing about innovative things that they’re doing, but sometimes just finding a thriving online faculty to interact with, shadow, observe, be mentored by, can be the most effective way to learn how to do this better.

John: I even sometimes encourage faculty to join a MOOC, because often you can find some interesting practices there that scale without necessarily requiring much effort on the part of the instructor.

Flower: That’s right. That’s one of my other recommendations and I hope I haven’t given all of the book away here. [LAUGHTER] But one of the other recommendations in that chapter is just to take an online course in whatever form that you can. Whether it’s a MOOC, a lot of organizations like the Online Learning Consortium, Quality Matters, offer online professional development opportunities for faculty. Even if it’s not about teaching online, just go take a class that is online. Or maybe personal interest. Sign up and take Spanish online. And having the experience as an online student is hugely impactful to help you understand what your students are going through. Even as a faculty if you’re taking a course and you’re reading the instructions going: “Now, what am I supposed to do with that?” Immediately, you have much more clarity about what your students might be experiencing and then you can take steps to address those kinds of gaps or areas of concern that might be in your own class… that you may not have previously seen before.

Rebecca: I think the recommendation of taking a course outside of your normal domain or area of expertise is key because you’ve got students who are in an environment they’re not familiar with, with a topic they’re not familiar with. And so to kind of simulate that, I think is key.

Flower: Right?

Rebecca: I know I’ve done that in the past and it’s like, “Oh yes, I forgot what it was like to be a beginner.”

Flower: Absolutely. In fact, I had a really interesting process or experience this past fall semester where I was supporting a redesign in a large cap biology class of liberal studies—or general education biology class—large enrollment. My background is in English literature… the humanities. I don’t think I ever took a hard science class in college because I did an honors program where we could do more sort of ethical concerns related to science. But I went to that class frequently throughout that semester and I clearly remember the first day. 240 students and me and I was sitting in the lecture hall with the students and it was just very, very impactful. Putting me in a situation that was foreign to me—I don’t teach large cap classes, I don’t know a thing about biology—I do now, I know a little more [LAUGHTER]—But I was a novice learner in a very foreign environment and that’s what our students are in our online classes, which is really quite anxiety producing if you think about it. Going into an unknown space, not knowing what’s expected, you don’t know how to get ahold of your faculty member a lot of the time. So just being intentional about helping students be more comfortable and more at ease in our online classes—be more available to them—can make a big difference. And again, you get that insight differently when you choose to place yourself in a situation where you’re a novice, and you’re not really sure what to expect. That’s a great point.

John: Are there any other topics that we should address that we haven’t raised yet? Anything else you’d like to emphasize?

Flower: You know, really only one thing comes to mind and that is an insight that I had literally this past week, which is that I feel like sometimes online faculty—myself included—have somehow developed the notion that we don’t really need to talk with our students. And let me explain what I mean by that. Again, I’m teaching an eight week—it’s an accelerated graduate level course right now—I’m busy. My students are busy. And on a whim a couple of weeks ago, I said, “Well, I know you have this assignment coming up by Sunday night, I’ll be available on Saturday between the hours of 1 to 5pm.” I don’t like to work on Sundays. I tell my students that if you want to just pick up the phone and call me on Saturday, go ahead. So that weekend, I did. I had a student who called me and she was a chatty Cathy, and we stayed on the phone for quite some time, but she got a better understanding of the assignment and how to be successful. Well two weeks later, which was this past weekend, it was my daughter’s 11th birthday and I was right in the middle of finalizing all the food preparation and everything else. And lo and behold, there’s my phone ringing and I can tell that it’s not a connection of mine. And I went, “Uh-oh, it’s one of my students,” [LAUGHTER] because I had said Saturdays 1 to five and that same student who had called me a couple of weeks prior called and we had a great conversation. 15 minutes, I was able to keep chopping the carrots while I was talking with her. And it just occurred to me, that wasn’t really a convenient time for me personally because I was doing that final party prep, but so what? The student needed help in that moment and just taking the time to answer the phone and talking through a couple of quick questions, it was helpful for her, and it just got me thinking about how, you know what, I don’t think a lot of us really talk to our online students, like, literally talk on the phone. I know some faculty have the online office hours, I know people are using video conferencing systems, I’m available, but one of the things I’ve started doing is just saying, “Hey, if you have a quick question, just call me. We’ll talk it through.” And sometimes a five-minute conversation can ease that student’s anxiety and answer a few questions. This happened to me again yesterday where a student was like, “Before I submit tonight, can I please just check in with you?” I talked with her while I was commuting to campus and it’s just a way of talking person-to-person, humanizing the online learning experience. But like I said, I think somewhere along the line personally I had formed this opinion that we don’t actually talk to our online students. And I don’t know why that’s a perception because if you’re teaching in person you talk with your students. If there’s somebody who has a question after class, you stay a few minutes after and answer those questions. But I think for online faculty somehow we’ve missed that connection and it can be a powerful and so simple solution to helping our students thrive and succeed. I think faculty and students both overlook some of those simple solutions. It doesn’t have to be a long, tedious, written interaction in a discussion forum. It could be a phone call, and so much can be conveyed through the tone of voice and emphasis, just as I’m doing here today. And as we all do, when we’re teaching live. Just picking up the phone and calling the students or inviting them to call you. Simple, powerful.

Rebecca: I think you’re pointing to something that I know I’ve experienced even though I don’t teach online regularly. It’s just online communication is always written and it feels daunting and it feels really time consuming. And it feels like, “Oh I got to sit down and dedicate time to do this.” So it’s nice to be reminded that there’s other ways to respond.

Flower: Just in my own work somewhere along the line, I forgot about the phone in my day-to-day job. My full-time job is as an instructional designer and it seems like we never just pick up the phone anymore. It’s always email. And as you said, it just takes longer, especially if you have a little bit of confusion and you’re going back and forth on email. I literally in the past few months, I’ve just remembered how to pick up the phone and call somebody. Have a five-minute conversation, you get your questions answered. And just reminding ourselves of the importance of real- time interactions sometimes, and moving away from the requirement that everything needs to be written all the time. I’m a big fan of video announcements, I do that all the time in my online classes and again, the reason I do it is because tone of voice, inflection, emphasis, and funny faces sometimes, or just emphasis where I might just kind of widen my eyes a little bit to explain that, you know, “This is really important. Pay attention and focus.” Just finding these other forms of communication apart from writing can make a big difference in the online learning experience as well.

John: We always end with the question, what are you doing next?

Flower: Well, I’m not quite done with this project. [LAUGHTER] So I’m wrapping up this book development. But what’s really making me passionate now is to really focus on being a crusader for online education. It’s undervalued. It’s under-supported. I know that faculty don’t see the joy of teaching online and I know that students approach it the same way like, “Well, I have to get this degree and I guess this is a convenient way to do it.” I just want to advocate for how online learning and teaching can be impactful, can be rewarding, and joy giving, and you don’t see that reflected even in the coverage of teaching in higher education. Most of the time, the focus is on what we’re doing in the classroom and that’s so important, but there’s a big gap. What are we doing in our online classrooms? I just want to move into that space and encourage people to think about how they teach in person, and how to do those things in their online classes in ways that are not so daunting that they never get around to it.

Rebecca: This has been really great. I’m looking forward to picking up your book and maybe thinking about teaching online. [LAUGHTER]

Flower: That’s right. And if you don’t mind, Rebecca, I’ll just pick up on that, which is that again, I think a lot of faculty don’t say, “Hey, wow, what a cool opportunity. I totally want to teach online.” For many faculty it’s a daunting prospect, “I don’t know how to do this.” But it can be a really great way to reinvigorate your teaching—to find new ways of finding and addressing those challenges. Keep in mind institutions have the support professionals, instructional designers and such, who can help if you’re thinking about moving into online teaching. Talk with some of those faculty support folks, talk with your colleagues, and jump right in. It’s more fun than a lot of people think.

Rebecca: Well thank you so much for joining us today.

John: Thank you.

Flower: Thank you. What an absolute privilege and honor to be here. Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, and Jacob Alverson.

62. 2018 Reflections

We’ve had over a year of inspiring guests and great information on the Tea for Teaching podcast. We thought it would be fun to spend our time today discussing the tools and techniques that we’ve put into practice.

Show Notes

Tea for Teaching podcast episodes referred to in this podcast:

Other citations:

  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
  • Sue, D. W. (2016). Race talk and the conspiracy of silence: Understanding and facilitating difficult dialogues on race. John Wiley & Sons.
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc..

Transcript

Rebecca: We’ve had over a year of inspiring guests and great information on the Tea for Teaching podcast. We thought it would be fun to spend our time today discussing the tools and techniques that we’ve put into practice.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today’s guests are John and Rebecca.

John: And today’s teas are…

Rebecca: Christmas Tea.

John: And I am having Christmas Tea.

Rebecca: Really?

John: I am.

Rebecca: We didn’t plan that. Well it’s the end of your reflections—guess it’s in the season. So, John, what are some of the things that you’ve tried that made a big impact on your class this year?

John: One thing that was really influential was the metacognitive cafe low-stakes online discussion forum that Judie Littlejohn developed and developed and presented in our second episode. This discussion forum allows students to collaboratively work together to improve their metacognition and to become more effective in learning. The student response has been so positive in the online classes where Judie and I have both introduced it that I’m going to try to introduce it into my large introductory face-t-face introductory classes. It’s going to be a little more challenging trying to come up with a scalable way of doing that that would work with classes of 3 to 4 hundred students, but I think the benefits make it worth the attempt.

Rebecca: That episode inspired me a little bit too and although I don’t have that specific set up in our capstone class, we started asking students about the workspaces that are most effective for them and in design there’s a lot of open-space studios and then those that might have more private space, and so we’ve been talking a lot about that and so things that they might intuit to be more comfortable or more productive environment for them or making them be far more reflective and precise about… and so that same practice of being intentional and thinking through that has been really effective and useful.

John: Another thing that had a large impact on me was Jeffrey Riman’s discussion in episode 10 on VoiceThread. Up until then I had been playing with VoiceThread every few years, occasionally giving workshops to demonstrate it to faculty but he convinced me to actually try it and I used it last spring and it worked very well in my online labor economics class. I was very pleased with how it worked. I actually used this in the metacognitive cafe discussions in my online labor economics class, but I used a text based discussion for the weekly content discussions. One of the interesting things is that after hearing all of the students’ voices in the metacognitive cafe discussions, when I was reading their other discussions, I could hear their voices and it created a little bit more sense of presence and connection, and the students responded the same way… that getting to hear each other’s voices made them feel a little bit more connected and gave them more of a sense of community.

Rebecca: I can imagine that it might also help students hear the inflection and the intention of some of the words and written content a little more clearly so they know that maybe a student had a good intention about what they said rather than assuming a bad intention just based on hearing a student’s participation throughout the semester.

John: As we’ve noted in a reading group—I think it was something that Michelle Miller talked about in Minds Online—everyone is much more likely to misinterpret meanings in text or to read negative connotations into things even when they’re not meant. What are some of the other episodes that influenced you?

Rebecca: I go back to Writing Better Writing Assignments, episode 31 with Allison Rank and Heather Pool as a reminder when I start writing new assignments. They are so intentional about making sure that you’re phrasing questions effectively and efficiently and you’re really asking students what you actually want them to deliver and I think that serves as a good reminder and I feel like every time I start making a new assignment I go back to some of those ideas. I don’t always necessarily listen to the episode again but I certainly skim through that transcript again.

John: I do the same thing sometimes. Either the transcript or sometimes the resources when there’s something I need to look up.

Rebecca: Their paper is really great too, so I’ve referred back to that a number of times as well. One of the other episodes that I found myself going back to a lot in the throes of this particular semester was episode 46, Creative Risk-Taking with Wendy Watson. In my department, historically we’ve had a lot of creative play between our classes and sometimes creative competitions and things and for some reason we have a lot going on with new renovations and stuff on our campus that have been keeping us extra busy and I think we had to cut something and we had started to cut the play and we’re all just feeling really dragged down and tired and what-have-you. So we brought some play back. So this semester my students invented a food truck for the campus and we created a website around it, created a menu so it was really creative and fun, and then some other classes created commercials and other resources and things to extend the brand and so the students all had a great time and it was really kind of funny because when we handed stuff off to the class that did little mini commercials they didn’t realize it wasn’t real. [LAUGHTER] They were like, “this will be really awesome.” They were really looking forward to it, but…

John: We were hoping for taco trucks on every corner and it didn’t turn out that way.

Rebecca: Yeah, actually it’s Space Buds; it’s a potato truck coming soon. [LAUGHTER]

John: Like the ice cream of the future, yes. [LAUGHTER] Which is now here; that’s no longer the ice cream of the future.

Rebecca: Yeah, but I think that’s a good reminder that learning can be fun and sometimes when you bring in these fun, creative opportunities for students to explore the class material everybody has more fun, including the faculty member and I think it just lightens the mood and makes everyone move forward faster.

John: That’s a nice reminder for all of us.

Rebecca: Yeah.

John: And also her discussion of just the importance of being willing to take risks in the classroom and not just go through routines and go through the motions the same way every semester was really refreshing.

Rebecca: Yeah, it can be really scary to take on a new project. When I decided that I’m just gonna change my project and it’s gonna be this after having engaged in that particular episode it’s like, “I don’t know how this is gonna go, fingers crossed…” but it worked out great.

John: One other thing that’s been affecting my class this semester was the podcast with Marela Fiacco, episode 34 on Flex Courses. In most of my classes I’ve been recording the sessions and posting them for students and if I knew a student was going to be out of town I would also livestream the class, but based on what she was doing with her class I just told my students that I’m live streaming each of my large classes and I’ve been amazed at the number of people who have been just joining in from wherever they happen to be. I’ve had up to 10 to 12 percent of the class and sometimes it’s because they’re not feeling well—we had a number of people out sick for a while—and while they’re lying in bed half dead or whatever they can still join in with the class… they can pose questions that pop up on the screen for a very few seconds… and they can also participate in all the clicker questions if they have the remote app… and that’s been working pretty well; it’s provided access for students who are away… who are out of town visiting family and traveling… and it’s just become routine where some people just enjoy it—they’d rather be by themselves. Some of the students who were sitting in the back have found that they can find a much quieter environment than some of the environments in the classroom— while we try to keep the noise level in the class down at times, some people are intimidated by having 300 to 400 students around them actively discussing things during some of the peer-to-peer things and they feel more comfortable in a quieter environment and they can do it that way.

Rebecca: That’s great. I think sometimes we expect that when we provide that kind of access that all of a sudden students aren’t going to get what they need or they’re gonna slack off or not take advantage but you can see that they’re there… they’re present… they’re participating, so that’s exciting.

John: Another thing that really influenced me quite a bit last spring was episode 12 with Doug McKee where he was talking about active learning. Two of the things he was doing I implemented right away, particularly the two-stage exam. I implemented it that semester—I talked about it first with the students to see if they were interested; they were and it just worked beautifully. We talked about that in an earlier episode too, and another thing I introduced was the student poster session that he discussed using in place of the end-of-term student presentations where students get up with their PowerPoint displays and those who are not presenting are sitting there anxiously worried about their presentation and most students were just not that actively engaged. When they got to create posters and post them around the room and have some of their friends and some other faculty from the department, and actually even the Dean came in and viewed them, they were so much more excited and instead of presenting for 8 to 10 minutes for each student they got to stand up there for the whole class period and explain it to all their colleagues and because I broke it up a little bit so that half of the students could go and visit the posters created by the other students for a period of time and then the other half could visit the others, they weren’t just standing by their posters and we had other people come in and talk about it and visit with him and they found it much more interesting and they were much more actively engaged in the presentations and much more enthused about it than they were otherwise, and several of them said that they wish all their presentations were done in that way.

Rebecca: I think that was a theme that came up in a number of episodes this idea of presenting your material out to a bigger audience than just the class. I wonder if some of the success is not just because it was this particular format with their students but because you were also inviting others in and it raised the stakes a little bit which might have just professionalized the whole experience. You mentioned the two stage exams—so I know we talked extensively about that on another episode, but it’s still really interesting to me too. I haven’t quite figured out how to implement it in the kinds of things that I teach but it’s still tumbling around in my mind; I like that idea and I do some things that are like that where I have students solve a problem on their own and then they come together and try to solve it together and in general that methodology seems to work just really well.

John: Peer instruction in general…

Rebecca: Yeah.

John: …works really well, which is when we talked about that more extensively.. What are some of the themes from our past podcasts that have influence your practice as a teacher the most?

Rebecca: I think this year seems to be my focus on diversity and inclusion and it’s for a few reasons: one is all of our accessibility initiatives that I’m highly involved with on campus, but also our reading group this year is on Race Talk by Derald Wing Sue… so I’m completely immersed in that particular subject… so I seem to be latching on to anything that’s about that and trying to digest that… and I also teach accessibility and things in my classes so I’m constantly thinking through how to teach empathy, how to get students to think about different experiences that are very different from their own and how to communicate that and how to get students to engage in that practice, so the episodes that really focused on diversity and inclusion include episode 41. Instructional Communication with Jennifer Knapp, one of our colleagues here on campus; episode 50. Diversity and Inclusion with Rodman King, who is our Diversity and Inclusion Officer on our campus; episode 49. Closing the Gap with Angela Bauer; and episode 58. Role Play with Jill Peterfeso, and the combination of those episodes runs the gamut of thinking through your interpersonal communications with students and in between students to how to present information to students to thinking about how to include students who traditionally may have been excluded from the discipline or from the community and then also thinking about historical context with the roleplay and how there’s a lot of ways that you can use roleplay to explore a wide variety of ideas in a safer space because it’s in a performative space rather than “reality” and that allows for some discussions and things to unfold in a way that it wouldn’t otherwise.

John: Jen Knapp’s discussion of the importance of creating a comfortable learning environment, which came through in the other episodes as well, really struck me in reducing the barriers between instructor and students and that’s particularly important in a large class because it’s really easy for there to be this big divide where it’s you and them and being more informal and more relaxed in the classroom can break down some of those barriers that are even more common or more difficult to break down when you’re in a large class setting than in a smaller group, I think.

Rebecca: Quantity alone can be intimidating.

John: It can be, but if you can break that down a little bit it helps, and just walking around, which I’ve always been doing, but I’m doing even more now… and that’s been helpful in getting to know some of the students a little bit more and making them more comfortable so they’re more likely to come by outside of class as well.

Rebecca: I’ve been a lot more aware of the way that I phrase things… the way that I address particular issues… being more positive in how we might address bigger patterns of struggle in the class… framing it in a way that’s much more positive promotes a growth mindset; these are things that I’ve been focusing on more and I know that I plan over winter break to revisit these particular episodes to pull out more of those details and to refresh my memory on some of those things so that when I go into the spring I can be a lot more on point on some of those things that I really care deeply about.

John: For many years now I’ve been interested in trying to build a growth mindset in students since reading Carol Dweck’s work, but Angela Bauer’s results with that in terms of how weekly growth mindset messaging in their introductory classes made a significant difference in narrowing the performance gap. I’ve been trying to do more growth mindset messaging in my own classes—more consciously doing it; I had been doing it a little bit but I’m trying to do it much more regularly whenever I send emails out to students, or whenever we’re talking about some of the more challenging things, I remind them that most students find this material challenging and that they get better at it by doing it. In economics I face a lot of students, and I know you do too, who claim they just are not math people and they just can’t understand graphs and it’s tough getting past that but reminding them that that’s a struggle for very many students and that it won’t be a struggle if they just continue to work and become more comfortable with it.

Rebecca: That raises a lot of the issues and things that came out in Marcia Burrell’s episode as well when we’re thinking about math people and this gatekeeping that happens. So it’s interesting that that particular episode ties really nicely with Angela Bauer’s work in really breaking down these barriers and helping people have access, which I think is really powerful and important work to be doing. I know that I’m catching myself saying things in a way that’s like, “Oh, why did I just do that” and undoing some things that I’m so used to doing that’s so embedded in how we work—come to find out I’m a much more of a negative person than I thought I was. [LAUGHTER] …and really.. It’s really a struggle. How about some of the themes that you’ve pulled out?

John: Some of the things that I think I’ve been most interested in and have been trying to work on the most is those episodes dealing with evidence-based teaching methods, especially because of the scale of the classes I teach; I’d like to do that as efficiently and as effectively as possible. Bill Goffe was one of the first people to discuss that very extensively and since he was coming from a background in economics it was very directly relevant to what I do and Bill and I have worked together for quite a few years here and it was nice to talk to him about some of the things he’s been doing and how he’s been evolving as a teacher. I’ve learned a lot from Bill over the years and it’s been… it’s been very productive and that episode was really useful. Episode 37 with Michelle Miller was very interesting in terms of where she sees the future of education going and where she sees some interesting possibilities for growth. Michelle Miller has been a major influence on me for quite a while from the first time I saw her present on low-stakes testing a number of years ago.

Rebecca: …and on our community in general because she’s been here as a guest speaker and we’ve done book clubs and things around her work.

John: …and she ran workshops here on two occasions and it’s been very, very productive. One of the things that influenced me this year was Dom Casadonte’s episode 42 on the flipped classroom… and I had been doing a mostly flipped classroom for quite a few years now, but I was still including a little bit more short lectures in it—five, ten minute lectures and he suggested that if you’re going to do it you should do it all the way because one of the problems is students in a flipped classroom environment will claim that you’re not teaching them and if you do a little bit of teaching them they come to expect that and then they feel cheated out of the rest so I’ve been much more explicit. I’ve always tried to prep students and to frame it in terms of the use of evidence-based practices but I’ve restructured my class a little bit to make it more obvious what we’re doing and I remind students more regularly that the basic stuff they need to learn on their own and it’s set up where they do some reading, they take some quizzes on it and then they reflect back on it and they give me a report on what things that they’re still struggling with and then we focus our class time on that. So the lecturing has been cut out except for the times when they’re stuck on something and then I’ll give short lectures to go over the things that they’re really puzzled by, but it’s much more focused and much more of the time is spent working on problem solving in the classroom.

Rebecca: It’s amazing how reminding students can be such a powerful tool for any of these evidence-based practices because if we’ve talked extensively about how it doesn’t always feel good to learn… it’s tricky… you feel challenged and so it doesn’t always feel good to learn—but reminding students why we’re doing things I certainly have also found helps a lot; it helps immensely; students buy in much more quickly when they realize why you’re doing something and when they can see that you’re customizing the content for them, even though you probably can guess often what you might need to spend time on, they feel like it’s customized and I think that means a lot to them too, probably.

John: In fact, most of the material I have I do prepare in advance, but I ask them to submit their list of concerns every day at noon and then I leave for class at 2:00 so I don’t have much time to customize it, but as you said, they feel that it’s very personalized because I refer back to the comments that they’ve been making and saying 60 percent of you said you’re struggling with this, let’s focus on some problems with this, which I’ve always known students struggle with and I’ve always been focusing on that but making it more obvious that I’m responding to their needs, I think, gives them a bit more buy-in, generally.

Rebecca: Definitely. I think some of the other themes that really have bubbled up for me is the revisiting of Open; we had a big series more recently about Open Pedagogy, Open resources; we started a long time ago in episode 8 with Kris Munger on Creating an Open textbook, but then episode 52, 55, 56 and 57 all focused on concepts of “Open.” So, I’m really excited about Open again. I forgot and it was actually Robin DeRosa that reminded me that I had been doing Open for so long and that it’s the culture of the field that I’m in to be open, so open-source software and things is something that I’ve been heavily engaged with since I’ve been a professional in the field but I kind of forgot that that’s what it was all about; I just needed that like little reminder and that little kick to get back into that mindset. So I’m really excited about focusing on that and making that more explicit in what I do and the reasons why I do it.

John: Robin DeRosa’s visit was very inspiring to many of us here when she both talked on campus and joined us in episode 55 and I’m planning to finally release my textbook as an open textbook for this spring, which is more OER than Open Pedagogy, but she also convinced me to try having students work on an open pedagogy project in the capstone course that I’ve been doing and so I’m working on putting together plans for that for the spring. I do hope that that works out, but I’m really excited about it.

Rebecca: Yeah, I’m excited about Open not just in the classroom but also as a scholar and having more open scholarship and publishing and open access journals and things like that as well. I also have to say that episode 57 with Fiona Coll was really exciting about Scalar. I haven’t had a chance to really experiment but my mind was blown by the possibility and I was really excited about the power of that particular platform—I’m not sure how it might be integrated into what I do quite yet but it’s something that I’m excited to experiment with. Are there any other themes that bubbled up for you?

John: The scholarship of teaching and learning was a theme that came up in quite a few of them where we were talking about research, but in particular two of the episodes: episode 26 with David Eubanks and episode 54 on the scholarship of teaching and learning with Regan Gurung were both really inspiring as well in terms of ways that perhaps we could do a little bit more research in our classrooms about what works.

Rebecca: I’m also really excited to do more scholarship in this area and build it into and integrate it more closely into my teaching in general so that it becomes just an overlapped area and everything’s more integrated. I’ve been slowly working in my personal research, my creative work, the service that I do on campus and my teaching have all been thematically related but I’m working really hard to integrate them more closely so that I can do more publishing on some of the ways that this influences my teaching and learning. Related to the scholarship of teaching and learning and actually Open is the idea of engaged scholarship where the work that you’re doing is really integrated into the community; you’re really working with the community, so I was really excited by episode 51 with Khuram Hussain and the ideas of having conversations with community… community really having a mutual relationship with the campus, and working on projects together…maybe outside of a semester framework. I’ve been doing a lot of community projects historically and I found some of the same struggles and some of the concerns he raised were things that I had certainly experienced in the work that I had done previously, so I think he offered some new ideas and some old ideas that I was familiar with as well that I needed some reminders of ways to approach some problems and do some creative projects. So I’m excited to start building on that work as well.

John: I thought that was a fascinating discussion and a so much more productive way of doing community based learning.

Rebecca: Yeah, I think he raised the issue of charitable work versus engaged work and I totally buy into that model already, but I like that he provided some really clear ways of being more engaged as a scholar in the communities that we work in.

John: It seems like a much better experience for the students, for the community, and for college community relations.

Rebecca: Definitely.

John: One other episode that really influenced me was episode 30 with Charles Dziuban on adaptive learning; it influenced me so much that I proposed a SUNY wide task group on adaptive learning and we have now people from quite a few campuses are exploring this and looking at a variety of adaptive learning platforms and we’re also going to be preparing a report for all of SUNY, at least in a preliminary form by the end of the spring 2019 semester.

Rebecca: That’s a pretty exciting endeavor and a big one, too. Interesting how some of these episodes have sparked really big changes, big work that we’re doing—not just ourselves but collaborating with our colleagues on some interesting projects.

John: One of the challenges I faced is that virtually every episode has suggested some things that I’d like to try and it’s a challenge to try to keep from trying to do them all at once.

Rebecca: Yeah, I think we struggled at coming up with a list of some of the things that inspired us the most but that’s because everything inspired us and so… [LAUGHTER] It was a challenge to come up with that particular list, but I think that we have some themes that we’re working on as a campus and individually and we’re in a really interesting and enjoyable position to get to talk to all these wonderful colleagues about the work that they’re doing and to learn from our colleagues. So at the end of the year it’s always good to be thankful for things and I’m certainly thankful for that opportunity to talk to all of the great guests that we’ve had and to learn so much in this past year.

John: And we’d like to thank all of our listeners for joining with us, and if you have any suggestions for topics that you’d like to hear discussed on a future episode, please let us know, and we’re happy to try to add them.

Rebecca: Happy New Year, everyone.

John: Happy New Years. [Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

61. A Motivational Syllabus

Do you wish your students knew what was on the syllabus? In this episode, Dr. Christine Harrington joins us to explore how we can design a syllabus that helps us improve our course design, motivates students, and  provides a cognitive map of the course that students will find useful. Christine is a Professor of History and Social Science at Middlesex College, and is the author of Designing a Motivational Syllabus (and several other books related to teaching, learning, and student success). Christine has been the Executive Director of the Student Success Center at the NJ County of Community Colleges.

Show Notes

  • Harrington, C., & Thomas, M. (2018).  Designing a motivational syllabus:  Creating a learning path for student engagement.  Sterling, VA:  Stylus Publishing.
  • Bain, K. (n.d.). The promising syllabus.  The Center for Teaching Excellence at New York University. Retrieved from: http://www.bestteachersinstitute.org/promisingsyllabus.pdf
  • Listeners to this podcast can purchase Designing a Motivational Syllabus at a 20% discount by visiting the Stylus Publishing order page and using the offer code: DAMS20. This offer applies to the paperback, hardcover, and ebook versions and is valid through 6/30/2019.
  • www.scholarlyteaching.org  – Christine’s website.

Transcript

John: Do you wish your students knew what was on the syllabus? In this episode we’ll explore how we can design a syllabus that helps us improve our course design, motivates students, and provides a cognitive map of the course that students will find useful.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Dr. Christine Harrington, a Professor of History and Social Science at Middlesex County College, and the author of Designing a Motivational Syllabus—and several other books related to teaching, learning, and student success. Christine has been the Executive Director of the Student Success Center at the New Jersey Council of Community Colleges.

John: Welcome, Christine.

Christine: Thank you, it’s a pleasure to be here.

Rebecca: Today our teas are…

Christine: I am not drinking tea; I’m not a tea drinker, I just do water and I will do iced tea once in a while, but not at the moment.

[LAUGHTER]

John: I’m drinking Tea Forte black currant tea.

Rebecca: And I have Prince of Wales today—mixing it up, you know?

John: We invited you here to talk a bit about your book, Designing a Motivational Syllabus, released just this past May, and the syllabus of many faculty tends to read sort of like a legal document and it often tends to be a bit off-putting and some people just provide a list of topics. You have a much different approach and it seems really productive. Could you tell us a little bit about that approach?

Christine: Sure, I’d love to. Thanks so much, John. I really believe that the syllabus is an underutilized resource. As we’re beginning our semester as faculty we always are required to put together a syllabus that explains to students what the expectations of the course are. But as you mentioned, many faculty treat it as a list of do’s and don’ts, making sure that we’re communicating what the classroom rules and expectations are and maybe the course topics, but it often kind of starts and stops there. So, I really think that it’s an opportunity for us to invite students to our course and Ken Bain is actually someone who’s written a lot about this as well, you know, inviting them to the feast. I think that’s what we want to do: we want to give them the excitement and passion that we feel as faculty and get them really excited about the course too. So one of the areas I saw as a gap or missing in the literature was the motivational angle of the syllabus. In addition to providing some really good resources and providing a course map to students, I think we can motivate our students through communicating our passion, telling them a little bit more about what to expect in the course in a more conversational style… by the words that we use… by the images that we include on the syllabus… and then also providing them really helpful information so that they view this as a course that they’re excited about and they will feel supported in.

John: We actually had Ken Bain here about 12 years ago, I believe it was, and he gave an all-day workshop on building a syllabus and I attended that and it was wonderful; much of your book reminds me of that, but you also go quite a bit further and provide a lot more suggestions in detail, so I like your approach. Could you tell us a little bit more about how the syllabus serves as an entry point to course design or redesign?

Christine: Sure. I think that for many of us the idea of redesigning or designing a course for the first time even is quite daunting and overwhelming to really think about how to engage our students and achieve all of our course learning outcomes. So, I’ve used the syllabus as a vehicle for that, as an entry point that I find that faculty find it a little easier if you’re working with a more concrete practical document to help them understand course design. Now, I have been accused by some of my colleagues of doing the old switch and bait, you know, I did a syllabus workshop that really was a course design workshop and they’re like, “Wait a minute, I think that you tricked me here,” and I said, “No, I didn’t, I thought you were talking about the syllabus, and if you’re going to talk about and think about what kinds of assignments and assessments you’re going to use—this is course design.” So you need to have a larger conversation. So, I said it’s an easier way for faculty to begin the dialogue and really take a good deep look at, well, what is it that I’m asking students to do and how does this fit into the overall course as well as the overall program that the course fits within… so, seeing the larger picture in terms of the course and program learning outcomes and revisiting the assignments and assessments and perhaps moving away from some of the “always have done this” kinds of assignments… you know, traditional exa…, paper… presentation… that almost all of us have in our course in one way or another. It doesn’t mean you have to abandon ship, but it is a great opportunity to step back and say, “Wait a minute, are these the assessment tools that are really going to help support student learning and help ensure that they’re going to achieve the learning outcomes that we set forth in the syllabus?” and then the syllabus really becomes the map for students. So it is the document that communicates the design of your course. As you are crafting your syllabus, you’re really thinking about: “What is it that I’m asking students to do? Why am I asking them to do that? and what kinds of supports am I going to put in to place so that they can accomplish those tasks successfully?” So, I really believe that it is a course design tool and that if you do it well, a well-designed syllabus really will show students exactly how they get from point A to B and the kinds of supports that are available to, you know, really enhance their learning journey along the way.

Rebecca: [If] faculty were to use the syllabus as a way to redesign, where in this syllabus should they start?

Christine: Well, that is not an easy question. You know, I think it depends on—I tell faculty they have two choices when they’re thinking about redesigning their syllabus: you can take the big approach, which is the course design approach, which is going to be looking at your course learning outcomes and then looking at what kinds of assessments or assignments are aligned to those learning outcomes… and then what I usually do for this big approach is ask them to think about what are the key summative assessments that you’re looking for and then work backwards from there using the backwards design process and determine what kind of formative assessments they need to use… and then as you start to craft the way your course is going to be developed, I say take your course outline your schedule of what you’re going to do—week 1, week 2, week 3 and start to plug in those summative assessments and then start to plug in the formative assessments that you’ve identified… and then that will help you determine what needs to happen in week 1, 2 or 3 to help them be successful on those tasks. So, it’s really kind of using that backwards design. I like to say it starts with the learning outcomes, it shifts over to the assessments that you’re going to use and then it starts to move into the course outline, you know, or sequence of topics that would be really important. But I said to you there’s kind of two ways that faculty can begin to, you know, redesign their syllabus. This is the big way and the way that I would really love faculty to do it, but if someone’s saying to me “The semester’s starting next week, that’s just too monumental of a task…” You cannot engage in this process in a day or two. It takes a significant chunk of time for you to really re-evaluate what it is that you’re trying to accomplish. So there’s also a lot of takeaways in this book about how you could do things literally in five minutes or less that would enhance the motivation and engage students in learning. For example, I did a workshop on my campus where we had a syllabus redesign summer camp and at the beginning of the semester we had faculty submit their current syllabi and at the end we had them submit their final syllabi… and the transformation just from a visual perspective alone was really incredible. So, if you didn’t even look deeply at the design piece… for instance, having a nice photo to draw students in (that’s related to your course content)… there was a biology faculty member who put this amazing, great engaging skeleton on the front page… and that really was much more effective than having her first syllabi, which had all rules and regulations…you know: “do this, don’t do that…” and a welcome statement and a picture of yourself… really just some of those kinds of elements really can make a difference. There’s some research studies out there that support adding a few additional words like “please come and talk to me” makes it much more likely that a student will come and talk to you, and it communicates that you want them to come and talk to you. There are some very easy fixes; changing it from formal language such as “the professor will” and “the student will” to “I am going to” and “you will do this…” Just using that more personal language can really help. So those fixes are literally… you could do it in five minutes or less if you want to make a couple of minor changes to increase motivation, but the overall course design is obviously a much bigger process, I’m not going to pretend it’s not.

Rebecca: I think it’s always a good reminder that you can always do small things before you can jump into a big thing and that the big thing is, you know, valuable. We were laughing at the beginning of what you were talking about a minute ago because I did the same exact thing here where I did a syllabus workshop that was a complete course redesigned workshop.

John: …and I suggested we rename it in the future as a course redesign or course design or redesign, but maybe leaving it as a syllabus workshop might work.

Christine: Yeah, I think you’ll get more people to participate. It’s less scary. [LAUGHTER]

Rebecca: It’s sneaky.

John: It is. It’s a sneaky way of getting in.

Christine: …and it also really allows faculty to walk away with something very practical… tangible… that they actually have done as evidence of participating in that workshop, and that’s great for administrators to see as well.

Rebecca: That’s a good point. So you talked a little bit about the syllabus as a tool for faculty to help think about organizing their class and redesigning it, making sure that students are going to learn what we’re hoping that they’re gonna learn. Can you talk a little bit about how the syllabus is a tool for students?

Christine: Sure. I think it’s really critical for students to not just take this document and put it aside but to recognize the value that it has… and I will tell you that students who see a more in-depth, comprehensive syllabi have a much more positive perception of the faculty member and also of the student experience of being in that class. From the student perspective, it’s motivational for them to know that they have a faculty member that cares enough to put together this really comprehensive package. Having a long syllabus that does not have any visual tools in it and is overwhelming… whether it’s legalese… that is something that students are not going to use much. But when you create a syllabus that’s motivational and engaging and visually effective, students will use that document and they really will appreciate it. Now, they do need reminders about how to use that. I think that it is a document that all of us quite frankly emphasize in the first day or first week of the semester and then often don’t revisit except for to say “in the syllabus…” which may or may not help a student if they’re having trouble navigating it. So, I’m a very big fan of making sure that we make it a document that is student friendly. If it is a longer document, including a table of contents, so that they know they don’t need to read all of this. Maybe the last half of the syllabus is the rubric section with specifics on the assignment… that they just need to know it’s there when the time comes for them to look at it. So I will often encourage students to view the syllabus very much like their textbook. You don’t need to read the entire textbook during the first week of class, but you certainly need to know what’s in the textbook so you’re not just focused on chapter one. You need to acclimate yourself to all of the information that’s in the text and what kinds of topics and resources are included. Well, the same goes for the syllabus… so really helping them use it as a resource and not feeling like they need to read it and memorize it, but instead use it as a tool to help them be successful.

John: One of the things you suggest is doing a screencast with the syllabus perhaps to make sure that students do look at it and to make it a bit more welcoming. Could you talk to us a little bit about that?

Christine: The screencast, I think, is a very valuable strategy, especially in the online class, but it can also be helpful in an in-person class. We all know that sometimes students are adding and dropping in the beginning of the semester and might miss an important conversation, and this really allows you to communicate about the syllabus to students. In addition, I will tell you that I have had students… I tend to have a fairly lengthy syllabi, as you can imagine, based on my textbook, I like to include a lot of resources… and I have had students say to me, when I got the syllabus via email from you, I really was overwhelmed and I was ready to run away from this class; I thought it was going to be a lot of work because it was a long syllabus, and once you explained it and we started to see the resources in it, I discovered that’s not the case at all. So I think that having that personal touch and the the nonverbals that you can really communicate through a screencast with a web video as well as the audio really does help students understand the value of the syllabus and we have so many great online tools now, like screencast-o-matic, that are free… things like that… that you can really easily do that in a short period of time to do an introduction, and students can refer back to that as they need to throughout the semester.

Rebecca: Can you talk a little bit about the kinds of things you would recommend faculty highlight or introduce about the syllabus on the first day? You mentioned identifying some of the resources and things in it. But, as you know, there’s a lot of faculty that call the first couple class days syllabus days and some of them actually read the syllabus to the students. What would you recommend?

Christine: Well, I certainly would not recommend reading the syllabus to the students. [LAUGHTER] That is not engaging. I think that the part of the syllabus that doesn’t get as much attention as it should is the “Why are we together?” The syllabus communicates: the purpose of the class, the goals of the class, and “What are the takeaways that they’re going to get as a result of being in this class?” Students, what they’re going to immediately want to know is about the grade, right? They go directly to the page that has information on the grade and the assignments that they need to do. But if they go there first, they’re missing the big picture. So, I think that we as faculty have a wonderful opportunity, whether it’s through a screencast or live in a regular classroom setting, to emphasize the learning outcomes of the course in a user-friendly way… not necessarily reading the learning outcomes, but to passionately explain why this course matters so much and the value of the course and the skills, and not just the knowledge that they’re going to get, but really the experience and the confidence that they’re going to get as a result of being in this course. So I really find that to be the most important piece to emphasize, and then helping them see the direct correlation and connection between what it is that they’re going to achieve and those learning experiences. So whether they’re assignments or assessments… the why behind all of those… so they don’t just view it as a big long checklist of “this is what I have to do because it’s a college course,” but they understand that that’s the roadmap that’s going to help them accomplish all those tasks. So, for instance, if I can give you one example, quizzing is something that not every faculty member does, sometimes it’s more of a more high-stakes midterm/final kind of situation… but faculty who really want to provide that opportunity for students to have formative assessments along the way would also include quizzing… and when you do that what happens is is that you’re helping students learn those skills along the way and help them self-regulate whether or not they’re on task to achieve the learning outcomes. But students may view them as busy work or that you don’t believe they’re going to read without being held accountable. By explaining the why in the rationale and bringing some of the research in on the testing effect and explaining to them that the reason for me doing this is because the research shows that if you test yourself you are much more likely to learn that content and it will stick with you throughout much longer periods of time. So providing the why, I think, is probably the most important part that I would bring their attention to and I think that we don’t do that enough as faculty.

John: Just as a plug for a future podcast, Michelle Miller will be a guest in a few weeks where she’ll be talking about the testing effect and retrieval practice.

Christine: Terrific.

John: But that is an issue. Students see testing as something negative; it’s not something they find quite as enjoyable… so providing that rationale is really useful and students don’t always buy it but the more you can convince them and the more evidence you can provide, I think the more likely it is that they’ll see the benefits.

Rebecca: Yeah, John and I have talked about this before that when I started doing that in my design classes, which is a place where testing is not as common, I had students actually asking for more, which I found to be very bizarre initially. You don’t generally have students asking for more tests or quizzes, but when they started realizing how it was helping keep them on track they actually found them really valuable.

John: In helping them assess their learning and to help improve their own metacognition of what they know and what they don’t know, it can be really useful.

Rebecca: One of the things that you have in your syllabus is a teaching statement. Can you talk a little bit about why you include that and why you recommend including that? Because that’s not something you commonly see in a syllabus.

Christine: Absolutely, in fact, there are a couple elements that I think are essential if you want to use the syllabus as a motivational tool, and I see the teaching statement as being one of the key elements; it’s an opportunity for you to start to build a relationship with your students, and it gives you a chance to share some background about who you are and why you’re passionate about the subject and what they can expect to happen during class. As we all know, the professor-student rapport is probably one of the most important predictors of success. Students who have professors who they believe care about them and are interested and engaged are much more likely to be successful than students who have faculty who are much not engaged and maybe not as connected to them. So, I believe that we could use the syllabus to begin developing that relationship, because we often send this out prior to even meeting a student for the very first time, and it also might be something that is shared on some kind of management system within the university or college setting for students to decide which classes to take. So it can invite them to why they should be taking your class –it’s really a wonderful way for you to share a little bit about yourself and your professional background expertise and passion.

John: You also suggest in your book that the syllabus can serve as a communication tool and it also makes it easier to be transparent in terms of how you grade and letting students know this, and that can increase equity, or at least a perception of equity. Could you talk a little bit about that?

Christine: Sure, I think it’s really critical that we are being as explicit and as transparent as we can be. There are going to be some students who can more easily connect those dots than others and when we make the dots connected for them we’re equaling the playing field to ensure that all of our students know what it is that they need to do in order to get to the finish line and how the different tasks relate to one another. So the more you can communicate and ensure that some of the students who may not naturally see those connections can see those connections, I think that really does improve learning and the academic experience for all students.

Rebecca: You mentioned earlier about referencing the syllabus and having students use the syllabus as a tool throughout the semester; you also mentioned early on that faculty have a tendency to say “it’s on the syllabus” without really providing much more guidance than that. Can you talk a little bit about ways that you recommend using the syllabus at farther points in the semester to help support students and continue to motivate students beyond just the beginning of the class?

Christine: Sure. I think that is critical. You know, many of us do activities on the very first day of class. I’m hoping that many of us are not reading the syllabus anymore and we’re starting to get more engaging strategies at the start of the semester. I know folks do a syllabus quiz and things of that nature. I actually think that having a group quiz format,, something that’s more interactive, is great. I do jigsaw classroom exercises at the beginning of the semester on the syllabus. They’re diving into that resource and understanding it and reporting back and teaching their classmates about the different section that they were assigned to. I think setting the stage at the beginning of the semester is really important, but we can’t stop there; we need to then follow through and revisit the syllabus throughout the semester. So what I typically do is I will often ask students to, at the beginning of class, (I always ask them to have their syllabus with them)… and I might give them a few minutes and I do this activity called dusting off the cobwebs… where they have to recall what we talked about last class and maybe from the readings and then we can look forward. So after we clean up our house in terms of where we were then what’s coming next, so what are we talking about today? How does this link up with the concept that we talked about last class, and then what what’s coming up in terms of what’s due? So instead of me putting on the board or on a PowerPoint slide, “Next week, don’t forget you have to submit the first part of your project” or whatever it might be. I’m having students give those daily reminders. So you can literally spend five minutes or less in a class, and maybe once a week; it doesn’t have to be necessarily every class… but maybe Monday’s will be your dusting day and looking forward opportunity. So I think that’s really helpful. The other time where I spend a little bit more time on it is when there is a big project that’s coming up. So at this point of the semester I will often have students work in either a partner group or a small group and in that situation I’m asking them to look at pages 12 to 14 that outline the details related to assignment 1 and the rubric of how you’ll get assessed on assignment 1. I want you to review that. I want you to put that in your own words… tell your classmate about what it is… and then you have an opportunity to ask me any questions about it. So, I’m basically training them to engage in that process. Again, this doesn’t need to eat up a tremendous amount of class time; it can be a few minutes. But by doing that you end up often getting better products to grade which makes your life much happier when it’s time for all the papers to be handed in because even though you put it in the syllabus, it doesn’t mean that they’ve looked in the syllabus… or they knew where to look… or maybe something didn’t make sense to them and they were not comfortable asking without the opportunity given to them in that very explicit way. So, I find that that really as a very helpful process. I also like to do an activity kind of mid-semester looking at the learning outcomes… so, going back to saying “Okay, so here’s what we said we were going to be able to learn and be able to do at the end of the semester. We’re about halfway done. I want you to look at the learning outcomes and do a self assessment. Where are you at on a scale of one to five? What do you need to do in order to get to the level five at the end of the semester? …and some of that’s going to happen obviously in classes or through the assignments. But, what else can you do to ensure that you’ll achieve all of those learning outcomes?” So, I like to use it in a self-regulatory way as well.

John: One of the things related to that is you suggest the use of an assignment grade tracking form. I’ve always kept my gradebook in Blackboard so students can see where they are but students don’t always seem to pay much attention to that. Having them create their own assignment grade tracking might be useful. Could you talk a little bit about what the form is and why you recommend that?

Christine: Sure. I do think that with our current technologies {Blackboard, Canvas, thinks of that nature), the LMS systems really do have a pretty robust gradebook feature where students can easily track their progress. Because in order for them to self-regulate they need to know whether or not they need some external data to see if they’re on track or not. To me, as long as they’re engaged in that checking and self-regulatory behavior, I don’t think it matters whether it’s definitely in the syllabus or in Canvas or Blackboard. But unfortunately, not every faculty member is using the gradebook to its full capacity, so sometimes students are left wondering about their grade and I want them to feel in charge of knowing how to do it. I also think that they have a hard time sometimes seeing the weighting of assignments so that they might view a smaller assignment as being equal to that of a larger one and not recognizing the significance that can have on your grade. In the absence of some of the technology tools… and there are great apps for this too so if your student has a faculty member that is not using an LMS gradebook, they can go ahead and download an app… and I think that’s a great way to track it as well. But just including something like that on the syllabus helps them see the breakdown on the weighting of the different grades so they can see how that final grade is determined.. Because I think you’re right. In the LMS’s I see that students are often looking only at the current calculated grade rather than looking at all of the pieces of how that grade came to be. So anything we can do to help them better understand the grading process and how those elements go into the final grade, I think, is useful.

Rebecca: In your book and also in the example syllabi that you’ve provided (both on your website and also in your book) you talk a little bit about your assignment sheets with rubrics and things completely spelled out… so not something that’s more generalized but something incredibly specific. Can you talk a little bit about the choice to do that and the advantages of doing something like that?

Christine: Sure. So I think that we all provide students with details about our assignments; it’s about where does that happen. For some of us, we think that that should happen outside of the syllabus in the LMS in a different place… under assignments or some other tab rather than being in the syllabus itself. I think it’s really helpful for students to have a complete package in the syllabus. Now, just because I think that doesn’t mean that it’s true, right? Actually I did a really neat study with a colleague of mine Crystal Quillen at Middlesex County College where we examined the student perception of syllabi length and we shared different syllabi. There was a 6-page a 9-page and a 15-page syllabus and they were randomly assigned to different groups. What we discovered was that students who were reviewing a medium or long (which was 9 or 15 page) syllabus actually found that syllabus to be more positive. So they had a more positive perception of the faculty member in terms of being motivated and things of that nature. In addition, we asked the specific question of the students “Would you rather have all of the details about your assignment in one place in the syllabus or is that not what you want? Would you rather just know ‘I have to write a paper’ and then have those details about the paper be provided at the point that you need them and in a different place within your learning management system?” …and 66% of the students said we want it all in one place. I think one of the challenges that our students have is that every faculty member sets up their LMS page a little differently… and I know colleges really work hard to try to have some consistency across the different course shells that exist… but students really do struggle with trying to find that information. If we can guarantee to students that all of the essential information you need about your assignments and your learning path are in the syllabus, then I think that makes a lot of sense. I really think it’s important for faculty and students to understand that it’s almost like an addendum to the syllabus, but it is in that document. So that they don’t need to get overwhelmed by it on day one but they know that it’s a resource… very much, like I had said before, like the textbook is.

Rebecca: I think that’s an interesting point right, a lot of our students are in five classes and if there’s five different ways of doing things and it’s organized five different ways with five different evaluation systems it can get a little overwhelming after a while. It’s a lot to keep track of. We often complain that students don’t keep track of things, but we certainly don’t help it.

John: We’d like to reduce the cognitive load.

Christine: Yeah, that’s for sure. Unfortunately, it’s the case and sometimes we need to get ourselves back in from the student lens to see what does life look like from their perspective. …and if I could just say one other thing about the “It’s in the syllabus” comment… I mean, believe me, I pour my heart and soul into creating my syllabus. My husband often laughs at me because he’s like “Haven’t you taught this course before? You look like you haven’t taught this before…” because I’m spending hours and hours and hours and I just had it last semester but I know it could be so much better. I’m trying to find ways to communicate it better. So, I know that the information is in the syllabus because I put it there. I spent many hours doing that. But if I give if a student is active enough and engaged enough to ask me a question about an assignment or the course and I just say “It’s in the syllabus” my syllabus is a long document. I do need to navigate them to which part of the syllabus it’s in, because my syllabus is probably a little different than other syllabi that they have looked at. So, I feel like it’s so easy and frustrating for us when the students may not have looked carefully before asking us. So that’s a skill we need to help them learn. But maybe they did look and they didn’t find it as quickly as they would like to. Let’s be honest. You and I also are not going to spend an enormous amount of time looking at a website if we can’t find what we’re looking for right away. We’re going to ask someone. So, we want to make it as easy to navigate as possible, and having consistency I think across different courses does help… not that you need to have a rigid standardized syllabus that looks exactly the same in every course. I think you need to have a little bit of room for the flavor and the personality of the course to show.

Rebecca: Those are things that we just forget about. We forget that it can be really overwhelming to look at documents. Yet we all complain about the same thing when someone else makes a document that we have to look at and we can’t find something. So, it’s good to double-check ourselves. So, I appreciate the reminder.

Christine: Absolutely.

Rebecca: One of the things that needs to be in a syllabus to some extent is the policies… there’s college policies that might have a particularly language that you have to include, but then also maybe what some of your own policies are as an instructor. How do you suggest including those in a motivational, inspiring, and supportive way? Because sometimes they don’t feel very inspiring or supportive.

Christine: That’s a great question. In fact, I have noticed that probably one of the biggest demotivators of a syllabus is the policy section… and many times faculty add more and more policies based on negative experiences that they’ve had. So something happens in a classroom setting and they’re like “I need a policy on that, so I’m going to add another policy about that…” and it starts to become this really big long laundry list. And clearly we have to have policies… I’m not saying we shouldn’t but I think the way in which we communicate our policies really do matter. When we have a list of “don’t do this: don’t cheat, don’t plagiarize…” all these kinds of rules and regulations… we’re kind of communicating to students that we think you’re going to do this, so I’m going to set you straight right now… rather than using more positive language. Instead we could communicate the same policy… I like to use the academic integrity one. So instead of saying “don’t plagiarize” instead… if you have a policy about academic integrity and the importance of why that matters so much and how everyone is expected to uphold academic integrity and engage in honest actions… That I think sets a very different beginning to that policy. The other piece is that we sometimes create policies that promote, I think, more achievement gaps… and actually gets back to that question you asked me earlier about equity, because many of our policies do not promote equity. I’ll take a late work policy, for instance… and I recognize the fact that we all need to be timely with our tasks. I mean in the world of work people are going to expect you to complete tasks on time and I recognize that that’s a very important skill. However, I also know that we are all human beings with a life on the side, you know, so that life happens sometimes that may prevent us from being on time with a task. I know I personally have not always been on time. I’m a very timely person, but there have been times when I have missed a deadline and haven’t been exactly where I needed to be at the time I should have been there. It’s not a pattern, but it does happen. So I think we need to have policies that are building in some of the flexibility that communicates to students that we respect them… we recognize that they have a life outside of school or at least outside of our class… sometimes our policies don’t even seem to recognize that they have other classes… like our class is the only one. So, students complain about that quite a bit… thinking that you’re looking at this only from your angle and not recognizing that this is one of many classes that I’m taking. When you think about policies such as that, it’s important to communicate it in a way that isn’t taxing on your time so that you’re taking late work every minute of every day… but is respectful. So, a very simple way to do that is “Here’s the policy. I expect you to be on time with tasks, especially if you’re doing a group task and your classmates are dependent on you.” I tend to be a little bit more rigid with my policies when it’s a group related task versus an individual task. But I also know that life happens and if you are in a situation where you’re not able to meet a deadline, please come and talk to me.” Because, if you put in a policy that says no late work is accepted… everything must be handed in on time. Well, certainly you won’t have to deal with any late work… that helps you with your time management, but it really is inequitable because the student who comes from a culture where it’s fine to challenge authority might come to you and say “My grandmother passed away last week. I have this really horrific thing happened in my life…” and, many of us… I know I myself… I had at some point a no makeup policy. It wasn’t a real policy… if you came to me and it was a good reason then I gave you an extension. But I only did that if you came to me. I did have no makeup policy on the syllabus. So, the problem with that is that there are certain groups of individuals that are not going to challenge authority and take your word at face value. So now you’ve put them at a distinct disadvantage in the class. So I think it’s important for our policy to do a couple of things: one is first of all they should be accurate… so I did not really have a no makeup policy… I had a “makeup if you have a good reason.” So, it wasn’t accurate. So now my policy is much more reflective of my current practices. I expect you to be on time but if something happens, communicate with me and we will see what we can do. It’s not promising them the world but it’s certainly promising them at least the conversation… and second in addition to having it be clearly communicated, it really needs to be equitable so that everyone gets an opportunity and it’s not a case-by-case situation where if they’re more willing to challenge authority they’re going to be more likely to get a positive outcome.

Rebecca: One of the things that you mentioned earlier is shifting the language in the syllabus to something that’s more personal from something that feels more like legalese or something. In those circumstances where an institution might impose a particular policy that’s written in a particular way that doesn’t match the voice of the rest of the document, how do you suggest dealing with that in a syllabus, when it might be required of you?

Christine: I think this is one of the big challenges that faculty face, is when there’s required elements that are not very motivational in nature. So first of all I would say try to start a conversation on your campus about revising that language, not necessarily the policy… that’s what the policy is going to be… but can we introduce it in a different way? So, I would say if you can do that that would be ideal because then that would be benefiting all of the students in all of the classes across your entire campus if they’re required. So I think that’s probably the point of intervention that I would encourage you to take and you could go back and refer to some of the ideas in the text or talking with colleagues about other ways to better word some of that policy language. If you’re not able to switch the language, or you want a quicker fix while that conversation is happening, I think it’s appropriate (and again you need to find out on your campus if it is) to maybe have an introductory statement: “The next section of the syllabus is going to be the institutional policies that every faculty member needs to include.” So, not saying they’re badly worded, but you’re saying that they’re different… like you can definitely see that I need to include these and I certainly wouldn’t have them (unless you’re required to) on the first page or two. Let the more positive motivational pieces be front and center and then have the policies be later on in the syllabus. So almost like you have a cover page or some kind of introduction before getting into the more typical policy language, I guess, if you need to include it I think can be helpful.

Rebecca: I’ve done things where, for example around intellectual integrity, there’s a campus statement (and I label it as such) and then my policy which kind of interprets that and puts it into context and is in my language… so the same idea that you have about introductory or it’s you’re kind of separating the two and making it clear like whose is whose.

Christine: mm-hmm

Rebecca: …and I think that sometimes has helped students too… but I’ve always found that to be jarring.

Christine: Yes. I would agree I think that definitely is, and I like your approach really of summarizing it because sometimes those policies that we get handed down are very lengthy and students probably aren’t going to read them. So, even if you gave the one- or two-sentence summary of what that meant…translation is… you know this is what you need to do… Be honest. you know, engage in honest action. It really matters. We all want a good reputation and we all want to learn. So in order to do that, these are the kinds of strategies that you need to engage in… and going to the integrity topic again, I think so many times students are unintentionally plagiarizing… not always necessarily doing it on purpose. So maybe helping them understand how they can better learn how to cite sources appropriately or how to paraphrase more effectively… So, pointing them to resources that are going to help them with all the tasks.

John: In your book, in addition to providing a lot of great resources about the syllabus and a lot of great recommendations and the evidence behind them you also provide quite a few some very nice sample syllabi at the end of the book and it’s a great resource and your publisher has very gracefully provided us with a discount code to any of our listeners which is DAMS20 and you can do that by going to the Stylus Publishing website. We’ll include a link to that in the show notes.

Christine: John, if you don’t mind I’d like to also share my website which is just www.scholarlyteaching.org. If you go to that website you will see several other teaching and learning resources,including several sample syllabi… and the syllabi that we used in the research study that I mentioned earlier on the length of the syllabus are provided there as well.

Rebecca: There’s also really good videos… a syllabi checklist… There’s some really great resources on the website. So definitely I recommend checking that out…

John: …and also information about your other books.

Christine: Yes, thanks for that John… appreciate it.

John: We always end our podcast with the question what are you doing next?

Christine: Well it’s interesting that you asked that. I am looking at options right now but I am very much interested in staying connected to the teaching and learning space and how we can improve what we do in the classroom. I’m spending some time thinking about moving it up to a higher level and engaging administration in some of the conversations that we’re having about teaching and learning and putting the teaching and learning centers kind of front and center really in conversations about student learning and engagement on campuses. So, for instance I work in the community college system as you know. There’s a national movement called Guided Pathways and this national movement is all about improving the success outcomes of students and it happens to a variety of ways. They talk about making sure our programs are clear, so that they’re defined and students know how to get from point A to point B. They talk about helping students choose a pathway and stay on a pathway, and they also talk about ensuring learning… but, having been a part of this national conversation, the “insuring learning” really is very much an afterthought, I think, unfortunately. I feel like we’re dancing around the classroom. So I’d like to take some of this work that I’ve been doing and work that’s been very directly helpful to the faculty and try to shift it to being helpful to the community college leadership as well as leadership at the four-year universities as well… to emphasize the importance of good and effective teaching practices. So we’ll see where that takes me. I’m not really sure how that’s going to transpire, but I did just present on that topic at the POD conference. I’m putting teaching and learning centers right in front and center in the Guided Pathways movement, getting them at the table of these conversations. So, I’m very interested in further pursuing that at this point.

Rebecca: That sounds like that could be really valuable to a lot of the faculty because translating that information to the administration is always really useful in finding support and all working together to have these really stronger outcomes for students.

Christine: Absolutely.

John: Well, thank you. This has been a fascinating conversation and this is a book I’m going to recommend to all of our faculty.

Rebecca: Yeah, so glad you were able to join us.

Christine: Well, thank you. I really appreciate the invitation and I hope that everyone listening is able to design motivational syllabi and if that happens our students are the ones who will benefit at the end of the day. So thank all of you for listening and for supporting students in their learning journey.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

54. SOTL

As faculty, we face a tradeoff between spending time on  teaching and on research activities. In this episode, Dr. Regan Gurung joins us to explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.  Regan is the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research.

Show Notes

Show Notes

John: As faculty, we face a tradeoff between spending time on teaching and on research activities. In this episode, we explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.

[MUSIC]

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Regan Gurung, the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research. Welcome.

John: Welcome.

Regan: Thanks a lot, Rebecca and John.

John: Our teas today are…

Rebecca: I’m drinking Prince of Wales today.

Regan: Alright.

John: I’m drinking ginger tea.

Regan: Ooh, now you’re making me want to. [LAUGHTER]

John: We’ve invited you here today to talk about research in the scholarship of teaching and learning, or SOTL. You’ve conducted a lot of research on teaching and learning as well as research within your discipline. In most disciplines there has been an increase in the journals devoted to teaching and learning and an increase in research in teaching and learning, but it hasn’t reached everywhere yet. SOTL research is often not discussed in graduate programs and is sometimes devalued by campus colleagues. Why does that occur?

Regan: So. I think there are multiple reasons why the—and I’m going to start with the devaluing. I think there’s a lot of uncertainty about what it exactly it is, so on one hand, when people say a scholarship of teaching and learning… very often if it’s somebody who hasn’t really read up on it recently the sense is, oh, you know, that’s research on teaching; that’s not as good as your regular research. Now, I think that’s a misperception and once upon a time, and here I mean maybe even 15 years ago, there was some scholarship of teaching and learning that wasn’t done very well and I think people have heard about that in the past and that’s why there’s that knee-jerk reaction. Far too often it’s seen as something where it’s not as rigorous, perhaps, or it’s not done in the same way and most of that is wrong. What I like to tell folks who see that is, if you think the scholarship of teaching and learning is not rigorous, well, you haven’t tried to submit something to a journal recently. I co-edit a journal on the scholarship of teaching and learning in psychology and I can actually see some people submit poor work and I send it right back; I do the classic desk rejection and I say, look, this is just not good enough. So my favorite tip for “How do you write for a scholarship of teaching journal?” is very simple: just like you write anything else. There’s a lot of baggage, but I think that as you alluded to, John, it has changed more recently and I think part of what you notice now or what I’ve been seeing is that this kind of work, this kind of examination is being called different things. For example, a term that I’m hearing more and more often is DBER: disciplinary based educational research. And I’m hearing this come out of medical schools and engineering schools and social work schools and many professional programs where they’re doing DBER, which is essentially what the scholarship of teaching and learning is. So, I think because of that baggage with the term, people are calling it different things but in general the work is getting much more rigorous.

John: Excellent, and if changing the name is sufficient to do that, it’s a valuable step.

Regan: I think that’s why, when I talk about it I like to talk about it as: “Do you want to know if your students are learning? Do you want to know if your teaching is effective?” Well, then you should do some research on it. You can call it what you want. I started really calling it pedagogical research because that’s what it was, but it’s truly a rose by any name.

John: And that’s something that Carl Wieman has emphasized.

Regan: Absolutely, yup.

John: In the sciences, you test hypotheses and there’s no reason we couldn’t do the same thing in our teaching.

Regan: Exactly.

John: And that’s starting to happen, or it’s happening more and more.

Rebecca: In some disciplines, the scholarship of teaching and learning is not accepted as being part of their tenure and promotion file, for example. What would you recommend faculty do in a department like that if they really want to get started in SOTL?

Regan: Well, so, Rebecca, let me take you a half step back.

Rebecca: Yeah.

Regan: When you say “in some disciplines it isn’t as accepted.” What has surprised me is that most disciplines have actually been doing the scholarship of teaching and learning and publishing it for the longest time. I mean, if you take a look at chemistry, it goes back, gosh, seventy years or so. Almost every discipline out there has a journal that publishes the scholarship of teaching and learning, but, and here’s the big but: most of us in our normal training never run into it. So, I’ll take my own case. In psychology, the Teaching of Psychology Journal has been around for 46 years, yet all through grad school, all through my post-doc I never even knew the journal existed. Why? Because the programs that I went through weren’t focused on teaching the individuals—wonderful as they may be—who I worked with didn’t do that kind of work, so they didn’t know about it. So I think that’s a really important fine-tune there: there is a journal in almost every discipline—almost every discipline—for the scholarship of teaching and learning. So, it’s just a question of discovering it… it’s a question of finding it. Now, that said, where can they start? I think I can answer your question from a conceptual level and from a practical level, so I’ll start with the practical. The easiest place to start, there are lots of compilations of how to do it. For example, I think both of you have my website. On my website I have a simple tab called SOTL. On that tab is a list of places to get going, and I’ve organized it so that there’s a brief introduction to SOTL, there are journals, there are resources, there are little handouts. So, if a faculty member has even ten minutes, go to my website, hit SOTL, scroll through. That’s the more practical, that’s the easiest way to get started. From a conceptual standpoint it really starts with the question, what aspect of your teaching or your student learning are you curious about? John, I know you do some work in large-class instruction in economics. Why is this assignment not working? Can I get my students to remember certain concepts better if I change how I present information? It starts with a question. And you don’t have to read anything, you don’t have to look at any manual. If you look at your class and you go, “Hmmm, why isn’t this working, or why isn’t that working?” That’s where it begins, and from there you follow the same route that we always do: go look at what’s been published in it, fine-tune your question, design, think about what do you want to change and so on and so forth. I think it’ll help if I give you my working definition of the scholarship of teaching and learning, and when I think about it I think of SOTL as encompassing those theoretical underpinnings of how we learn. And more specifically, I see it as the intentional and systematic modifications of pedagogy and here’s the important part: the assessment of the resulting changes in learning. So that’s the key: you intentionally, you systematically, modify what you’re doing and then you measure whether it worked or not. That’s it. I could say that nonchalantly. There’s a technique , there’s a robustness to it, but at the heart, where do you start? You start by asking the question.

Rebecca: I think one of the things that I hear you saying is not much different than someone has a really reflective teaching practice—they’re doing it but not in that systematic way?

Regan:Yeah, absolutely right. There’s a term called scholarly teaching and in this kind of literature there’s a distinction made between scholarly teaching and the scholarship of teaching and learning, and all the distinction is is that scholarly teacher is reflecting on their work and then you’re right, you’re absolutely right; making those intentional systemic changes. That’s scholarly teaching. When it becomes the scholarship of teaching and learning is when you present it or you publish it, preferably through peer-reviewed ways, but you’re absolutely right; at the heart of it it’s scholarly teaching. It’s reflective intentional systematic changes.

John: One of the barriers, that people who are considering doing research in the scholarship of teaching and learning, is going through IRB approval, and in many disciplines that’s something they haven’t experienced before. It’s common in psychology. It’s less common in economics and perhaps in art.

Rebecca: It doesn’t exist in design. [LAUGHTER]

John: Could you tell us a little bit about that process?

Regan: Sure. Every university has an institutional review board and essentially what that board does is it’s in place to make sure that any research that’s being done isn’t harmful. Now, normally when we think about harmful we think about a drug or a food substance being tested, but here it just means any research that’s being done, and so when you do the scholarship of teaching and learning or when you’re examining your classes, yes, you could just look at your exams and see if exam scores are changing, but, if you do want to publish that, if you do want to share that, you really should go through institutional review board review. Now, the key thing here: it does sound like this whole new world, and it is, but at the heart of it is a very simple process. Now, there are three levels of review and I think knowing about the levels helps. For example, the first level is called an exempt review. The next level is called an expedited review, and the third level is called a full board review. I don’t think I’ve run into scholarship of teaching and learning that has gone through a full board review, because we’re not doing things that are more than minimum level of stress. Now when you say, hey, hang on, I didn’t know they were stress involved. Well, anytime you ask anybody to fill out a survey, there’s a minimal level of stress. And when you’re asking your students to reflect on their learning, well that’s a minimum level of stress. Every university has its own procedure. SUNY Oswego probably has a forum online. It’s a short forum; you’re basically telling this board what you plan on doing, what you plan on doing with the information, and most importantly, in these kind of cases, you are letting the board know whether or not students will be put under duress. What the IRB is going to look for is are you the instructor in some way forcing your students to do things that normally wouldn’t be done in the normal course of the educational process. But at the heart of it, all you’re doing is you’re sharing with this board whether or not you can do it and most scholarship of teaching and learning is at that exempt level. That exempt level essentially translates to exempt from further review. It doesn’t mean exempt from being reviewed; it just means this is mundane and low stress enough that it’s exempt from further review. Now that second level, expedited. If you do want to measure or keep track of names, if you want to look at how certain names relate to scores down the line—and that’s actually some really key research—well that’s expedited review. Now, even there it’s reviewed by one person. Both the expedited and the exempt review are reviewed by one person, often the chair. It often takes no longer than a week, and by doing that you just know that all your t’s are crossed and your i’s are dotted and it’s the ethical thing to do. So, whenever people say: “Oh, this is really mundane and I’m not really doing much more than just measuring student learning,” I still sa y if there’s any chance you want to present it or publish it make sure you go through the IRB.

John: And many journals will require evidence of completion of the IRB process.

Regan: Oh, absolutely. The moment you want to publish it you have to sign off saying that you got IRB review..

John: We do use an expedited review process on our campus. I was going to say, though, that we’re recording this a bit early because we’ve recorded a few things in advance, so we’re recording this in late October, but just yesterday I read that Rice University has introduced a streamlined expedited review process or IRB and apparently that’s something that’s been happening at more and more campuses. Are you familiar with that?

Regan: You know, not as much, because right now there’s so much up in the air with the IRB because national guidelines are changing. They were supposed to have changed in January, then it was moved to July. The latest I heard is it’s moved to next January. So, for the most part actual regulations are changing. Even on our own campus we switch from one form of human subjects training to another form, but this so called short-form expedited process will definitely help. That said, even the regular expedited, it’s a very easy process and I think the neat thing about this—and I tell students this when I’m teaching research methods, too—as the instructor or the researcher, just going through that IRB form really reminds you of some key things that you may have otherwise forgotten about, so, yes.

Rebecca: Do you talk a little bit about your own research to give people an overview of what project might look like from the beginning to the end?

Regan: Sure. What really got me interested in this is I teach large introductory psychology classes, the class is 250 individuals and I was struck by how when publisher reps come into my office and try to convince me to adopt one book over the other they would talk about the pedagogical aids in the textbook; “oh, look, our book has this and our book has that.” And that really got me started studying textbooks and how students use textbooks. So the umbrella under which I do research is student studying: What’s the optimal way for students to study? …and I use both a social psychology and a cognitive psychology lens or approach to it and it really started with looking at how they use textbook pedagogical aids. So, for example, in one of my really first studies I measured which of the different aids in a textbook the student uses and then I used their usage to predict their exam scores. Now, what I found, and this is what really surprised me and got me doing this even more, is that even those students were using and focusing on key terms a lot. Now, mind you, I’ll take a half step back—you may not be surprised to know that students use bold terms, they use italics, that’s what they focus a lot on. But students in my study also said that they use key terms a lot. Now if you’re studying key terms that should be good. If you’re making flashcards and studying those key terms that should be good, but what I found is that the more students use key terms the worse their exam scores. There was this negative correlation and that’s completely counterintuitive. Why would they go the opposite direction? So, I dug into it some more and I realized that students spend so much time on key terms or so much time on flashcards that they’re not studying in any other way. So even though they’re using flashcards, they’re so intent on memorizing and surface-level processing that they’re not doing deeper level processing. So, that was some years ago and I’ve been building on that, trying to unpack how students study. My most recent study… that’s actually under review right now… a colleague, Kate Burns, and I took two of the most recommended cognitive psychology study techniques, which is repeated practice or testing yourself frequently and spacing out your practice or spacing out your studying, and we took both of these and across nine different campuses divided up classes such that the students in those classes were either using high or low levels of each of these. So, in one study across multiple campuses we tested is there a main effect of one of these types of studying or is there an interaction? And what we found is that there is an interaction and the critical component seems to be spacing out your studying. Not so much even repeating your studying, but really spacing out your studying, and I think what’s interesting here is the reason this is happening is the students who said that they were testing themselves repeatedly, that sounds great, and if you’re a cognitive psychologist you say, hey, the lab says repeat testing is great; the problem is in the classroom a lot of students who were repeatedly testing themselves were repeatedly testing themselves during a really short period of time.

John: Right, I’ve seen that myself.

Regan: And I think that’s the issue, but because we had both these factors in the study, we could actually tease that out. So that’s the kind of work that I do… is take a look at what the cognitive lab says is important; let’s see how it works in the actual classroom.

John: Now was this a controlled experiment? Or was this based on the students’ behavior?

Regan: So, yes and no, okay. [LAUGHTER] I love this study because of a number of reasons. Number one, we tested two different techniques in the same thing. Number two, we did it at multiple institutions, so it’s not just my classroom. A lot of SOTL is one class. So, here we went beyond to try and generalize. But, to get to your question, we actually used a true experimental design. So we recruited these different campuses and we assigned a classroom. So, for example, I’d say, “Hey John, thanks for taking part. If you can have your students do high repetition and high spacing?” “Hey Rebecca, thanks for taking part. Could you have your students do high repetition and low spacing?” And that’s how we spread it out. We had about two campuses in each of these cells. That’s the true experiment on paper. To get to the other part of what you said… in reality, that’s not exactly what students always did. And you know students; we can tell them to do something but a whole bunch of things gets in the way. Fortunately, of course, we measured self reports of what students said they actually did and it was relatively close to the study cells, but even though it varied a little bit we could still control for it. So, yes, it was close to a controlled study as much as you could control something in the real world across nine campuses.

John: That brings us to the general question of how you construct controls. Suppose that you make a change in your class; how do you get the counterfactual?

Regan: Right.

John: What would be some examples for people designing an experiment?

Regan: The word control, especially in research, has the true connotation of the word control group and that’s controlling for factors as different from having a control group. Optimally we’d love a control group. The problem with the control group is that it means no treatment. So, very often a true control group means this group of students is not getting something. From a philosophical and an ethical standpoint, I don’t like the notion of one group not getting something. So, the word I like to use is comparison group. So, your question still holds, but what’s the comparison group? I think here’s where if you’re fortunate enough to teach multiple sections, well one of the sections can be the comparison group. If you’re not fortunate enough to have multiple sections, you compare the students this semester with the students the last semester when you weren’t doing that new, funky innovation. So, there are a bunch of different ways to gather the comparison group, but you’re absolutely right: having a comparison group is important. Most commonly in scholarship of teaching and learning, the comparison is the students before that intervention, so it’s a classic pre- and post- measure. I’ll give you this quiz before I’ve introduced the material, I give you an equivalent quiz after, let’s see if there are changes in learning. And that’s the most common comparison; you’re comparing them with them before but optimally again you want a different section, you want a group of students, a different semester, or so on, and so on.

John: And it’s best if you have some other controls…

Regan: Absolutely.

John: for student ability and characteristics.

Regan: You nailed one of the key—my two favorite are effort and ability. As much as possible, measure their GPA. If they’re first-year students, measure their high school ACT scores or their high school GPA and then you have to measure ability, and I think those two are probably the usual suspects for control. And again, a lot of SOTL doesn’t do that and it should.

Rebecca: I think one thing that comes up a lot for me (and maybe some others who are in disciplines maybe more similar to my own) is that the kind of research that we do is not this kind of research generally, but we’re really interested in what’s happening in our classrooms. So, for faculty who might be in the arts or some other area where we’re doing really different kinds of research, how would you recommend being able to partner or do this kind of work without that background?

Regan: And I think implicit in your question is the “Do I need to have a certain methodological tool bag?” and I remember I was at a conference once and somebody accosted me and said “Hey, is it true that you have to be a social scientist to do this work?” And the answer is no, and I wrote a pretty funky essay called “Get Foxy,” which is how social scientists can benefit from the methodologies of the humanists and vice versa. But, you’re right; you can collaborate if you need to do that kind of work, but there are a lot of questions even within your discipline… and when I think about SOTL I think about answering questions about teaching and learning with the tools of your discipline. Now, I’ll give you an example: a good friend of mine was an art and her project, or something that she wanted to dig into, was to improve student critiques in an art class. Here we have students learning how to do art (and I think it was drawing or jewelry making) and across the course of the semester everybody had to present their work and then critique each other’s work… and those critiques, they just didn’t have the teeth that she wanted them to, so she was giving them skills and how to do it. So here’s a case of how did she know whether or not the critiquing tools were increasing? Well, she came up with a simple rubric and to score them against and look at if the scores changed. Now, you may say, well, we very often in the arts and theater you don’t get skills to do that, which is true, but that’s where I think collaboration comes in and that’s why what’s really neat about scholarship of teaching and learning is very often there are class collaborations. I have a historian on my campus who wanted to change the quality of his essays and he and David Voelker changed how he was teaching and wanted to see it roll out and had students on their essays use teams in a different way. Well, he compared, and John this goes back to your point, he compared essays from before the change with essays from after the change, counted up the number of teams students had and then, Rebecca, to your point went over to my colleague in psychology and said, hey, can you tell me if this is statistically different. So, he didn’t even bother with doing the stats; he just said, “Hey look, I don’t need to do the stats.” But you can, in a click, and literally within minutes my colleague in psychology had done the stats for him. I think that’s the kind of stuff that can happen to truly get at those answers if you go, “You know, I don’t know how to do that.” But, you’d be surprised… the basic skills for SOTL can give you enough to test questions pretty well.

Rebecca: I think John and I have also found in the teaching center that it’s really exciting when faculty from different disciplines start talking about their research when they’re looking at learning because there’s things that we can learn from each other and the more that we’re talking across disciplines can be really valuable as well.

Regan: Right, and I think this is where reading the rich literature that exists in your discipline or even across disciplines on scholarship on teaching and learning really gives you the leg up, because I find now when I do workshops and somebody says, “You know, I’ve got this question; I don’t know how to start.” More often than not it’ll remind me of a study that I can say, hey, here’s what you can do. And it’s just because I read a lot and I’ve got all that in my head and I just matched to that question and it’s pretty easy. I mean, very rarely do we have to invent something from scratch. We go, “Hey, yeah, you know what? Here’s the study that’s pretty close to the question you have, let’s use that methodology.”

Rebecca: So, how do we build a culture of the scholarship of teaching and learning—the departments who might have faculty who are resistant to the idea of their colleagues spending their time doing that? How do we start changing minds and really building a culture that embraces the idea of the scholarship of teaching and learning?

Regan: Well, I think you’ve got to attack it from two different levels. You definitely want a champion in the administration who is educated enough about the scholarship of teaching and learning and how it can be done robustly. If you can convince somebody of it’s worth and then if you go “How do you do that?” …well that’s where you need to make sure you have at your fingertips, as a teaching and learning center, the exemplars of really robust work… and I think if you have that really robust work at your fingertips, that’s definitely a key place to start. One of my favorite examples along those lines of trying to convince (especially administrators) about the worth of scholarship of teaching and learning, I recommend a 2011 publication by Hutchings, Huber, and Ciccone, it’s called A Scholarship of Teaching and Learning Reconsidered and this 2011 publication is a great collection. It does your homework for you. That one book pulls together evidence for why scholarship of teaching and learning helps students, helps faculty, helps institutions. So that’s where the top down—get your administrators to check that book out and go, “Oh yeah, look, there is actually some good research.” Coming at it from the other angle—I know this for a fact—there are people on your campus doing some of that work, but often they may be isolated, they may be a small group. You want to strengthen them so that they can spread that to their circles, and that’s really how it starts. On my campus, when Scott was the Dean at Green Bay, we did a lot to develop scholarship of teaching and learning through the teaching center. There was one year where we had 14 faculty who got together every month and talked about their projects. Now you may say, well, that’s 14 and you had 160 faculty. You know what, you do 10 of working every year and colleagues see the value of the work those 10 or 14 are doing, pretty soon you’re gonna have a culture where people recognize it more and appreciate it more. So I think that’s how it goes… you put your efforts on those people who are already doing it to make them stronger and that’s gonna spill over and pretty soon you’re gonna win over folks.

John: We generally had support from the upper administration and there’s often been a lot of faculty who are new, interested in doing it; it’s usually the promotions and tenure committees that have served as a barrier in some departments, but we’ll work on that and we need to keep working on that.

Regan: Well, just along those lines on our campus we felt so strongly about the scholarship of teaching and learning that the Faculty Senate actually passed a resolution recognizing the importance of scholarship of teaching and learning. Now again, it still gave department chairs some leeway, but at least the faculty voted on it as something that the university values and that goes a really long way to having especially junior faculty say, you know, I can do this.

Rebecca: Certainly makes faculty, especially junior faculty, feel supported when the Senate is saying, “Yes, we believe in this” and it’s not just one person saying we don’t.

Regan: Absolutely. And they’ll be naysayers. We started off this conversation with “There are people out there who think it’s not good enough” and there are people out there but I’ve had conversations with such people on my campus where sharing some information, sharing things about how it’s done goes a long way towards changing minds.

John: In my department, it’s helped that I’ve been the chair of our search committee for a few decades now. We’ve generally hired people who are interested in this, but that’s not the case in all of our departments yet, but we’re hoping that’ll change. For those who have small classes or may not be interested in doing research in their own classes, one other option is meta-analysis. Could you talk a little bit about that?

Regan: So meta-analysis, where one study is taking a look at a lot of different studies, there is the mother of all meta analyses… is one that we should talk about because I think the interested person can run to it. John Hattie, now at the University of Melbourne, did a meta-analysis where actually he did a meta-meta-analysis; took 900 meta analyses and then synthesized the data from those 900 studies that had already synthesized data, and the reason I like talking about that is the sample size when you take all those 900 meta analyses is a quarter of a billion with a “b”; that’s a lot of data points, it’s a lot of students. And what’s neat about meta analyses is that instead of just being one study at one place it’s now multiple studies over multiple contexts, and if you can find an effect over multiple contexts, that’s really saying something because a lot of single studies are so geared into the local context of where that place is that if you run into a meta analysis, so even if anybody listening pulls up an educational journal or an SOTL journal and sees meta analysis in the title, I would spend more time reading that one because it’s gonna be more likely to generalize from that. So, I think it’s statistical and methodological advances now mean that there are more meta analyses around and more meta, meta analyses around as well.

Rebecca: As an advocate for the scholarship of teaching and learning, where do you hope the scholarship of teaching and learning goes in the next five years?

Regan: Honestly, I think it should be a part of every teacher’s repertoire. When I think about a model teacher, and it’s not just when I think about it—I’ve published on evidence-based college and university teaching and when my co-authors and I looked at all the evidence out there and what makes a successful university teacher… one of those components, and we found six… I mean, it wasn’t just student evaluations, no, it was your syllabi, it was your course design, but one big element was doing the scholarship of teaching and learning… and to answer your question, I think if in five years from now we can see it be part of teacher training to look at your class with that intentional systematic lens, I think that’s where the field needs to get to.

John: At the very least it would get people to start considering evidence-based teaching practices instead of just replicating whatever was done to them in graduate school.

Regan: Absolutely. People would be surprised at how much good SOTL there is out there, and I always like sending folks to the Kennesaw State Center for Teaching and Learning where they have a list of journals in SOTL in essentially every field. You will scroll through that list for ages and it is just mind-boggling to realize that, “Wow, SOTL has been going on for a very long time.” And Rebecca, you mentioned art and performance arts and theater and music… not as much, but even there there is a fair amount and I think it’s just a question of getting folks making those resources more available to individuals and that’s why whenever I interact with teaching and learning centers I have a short list of key resources to look at. And again, that’s on my SOTL link. But, even that small list is an eye-opener to most people who never knew this existed, and I think once they realize it’s there they will start seeing it everywhere and once you start doing it it really energizes you. For those of us who’ve been teaching for 20-plus years to look at our classes with that new eye of how can I change something, how can I make it better and then seeing the positive effects of those changes, that’s invigorating.

Rebecca: I’m energized after having this conversation.

Regan: It is good stuff.

Rebecca: Yeah.

Regan: I just got back from a three-day conference and all we did was sit around and talk about cool SOTL. And you’re right …came back and sitting on the plane I was texting people with study ideas to collaborate on. It was that exciting.

Rebecca: The more you talk… collaborate… the more it happens.

Regan: There you go.

Rebecca: So, we always wrap up by asking, what’s next?

Regan: You know, I think I like getting the bang for my buck and you mentioned this in the intro: right now I’m working on the American Psych Association’s Introductory Psychology Initiative and what’s next is basically two years of really focusing on the introductory psychology course. It’s taken by close to a 1.5 million students a year and I’d like to make sure we can make that course the best learning experience for our students as possible, so that’s where my energy is gonna be for the next little bit.

John: That’s a big task and a very useful one.

Rebecca: And definitely worthwhile. Well, thank you so much for spending some time with us this afternoon. it’s been eye-opening and exciting… energizing. I can’t wait to look through some of the resources.

Regan: You know, is there anything else that you’d like, get in touch and I welcome anybody listening to get in touch as well.

John: Thank you, and we’ll share links to the resources you mentioned in the show notes.

Regan: Sounds good.

[MUSIC]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen, and Dante Perez.

[MUSIC]

38. Reflective practice

Now that we have been on summer vacation for a while, we thought it would be useful to take a break from our usual interview format to reflect on the previous semester and our plans for the fall. We also provide some recommendations on summer reading related to professional development.

Show Notes

  • Sue, D. W. (2016). Race talk and the conspiracy of silence: Understanding and facilitating difficult dialogues on race. John Wiley & Sons.
  • Guffey, E. (2017). Designing Disability: Symbols, Space, and Society. Bloomsbury Publishing.
  • Evans, N. J., Broido, E. M., Brown, K. R., & Wilke, A. K. (2017). Disability in higher education: A social justice approach. John Wiley & Sons.
  • Hogan, Lara (2016). Demystifying Public Speaking. A Book Apart (https://abookapart.com/products/demystifying-public-speaking)
  • Hoffman, Kevin H. (2018). Meeting Design: For Managers, Makers and Everyone.  Rosenfeld Media.
  • Openpedagogy.org
  • Schwartz, D. L., Tsang, J. M., & Blair, K. P. (2016). The ABCs of how we learn: 26 scientifically proven approaches, how they work, and when to use them. WW Norton & Company.
  • Cavanagh, S. R. (2016). The Spark of Learning: Energizing the College Classroom with the Science of Emotion. West Virginia University Press.
  • McGuire, S. Y. (2015). Teach students how to learn: strategies you can incorporate into any course to improve student metacognition, study skills, and motivation. Stylus Publishing, LLC.
  • Parkes, J., & Zimmaro, D. (2016). Learning and assessing with multiple-choice questions in college classrooms. Routledge.
  • Lewis, M. (2016). The undoing project: A friendship that changed our minds. WW Norton & Company.The Undoing Project – Michael Lewis
  • Tea for Teaching podcast: 15. Civic Engagement – a discussion with Allison Rank about the Vote Oswego project.
  • DeRosa, Robin (2017). “OER Bigger than Affordability” Inside Higher Ed. November 1.
  • Tea for Teaching podcast: 30. Adaptive Learning
  • Videoscribe
  • Flipgrid
  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
  • Learning How to Learn MOOC
  • Oakley, B. A. (2014). A mind for numbers: How to excel at math and science (even if you flunked algebra). TarcherPerigree.
  • Oakley, B. (2017). Mindshift: Break through obstacles to learning and discover your hidden potential. Penguin.
  • Oakley, B. (2018). Learning How to Learn: How to Succeed in School without Spending all your Time Studying; a Guide for Kids and Teens. Penguin.
  • Brown, P. C., Roediger III, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc..
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Teaching in Higher Ed – Bonni Stachoviak
  • Teach Better – Doug McKee and Edward O’Neill
  • Email addresses: john.kane@oswego.edu and rebecca.mushtare@oswego.edu

John: Now that we have been on summer vacation for a while, we thought it would be useful to take a break from our usual interview format to reflect on the previous semester and our plans for the fall.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Today our teas are:

Rebecca: …a mix of seven different kinds of tea, and it’s not really describable at this point.

John: After I’ve had many different types of tea today, I have Twinings’ Wild Berries herbal tea.

Rebecca: Finally dropping the caffeine after a long day?

John: …after many teas earlier in the day, yes.

Rebecca: So, I start my reflective practice while grading during finals week and for me it’s a really effective and productive procrastination technique. As I’m reading assignments or looking at projects and making notes about things that clearly did not work or “Wow, I really should cover these skills better” or “This really worked…” and I have a running dialogue with myself while I’m grading them and I use that for planning for the fall. What are your practices like, John?

John: I’d like to do that a bit during grading week but during grading week I’m generally busy working on the workshop schedule for our workshops here…

Rebecca: What?

John: … and also working on plans for various presentations at the SUNY Conference on Instructional Technology and so forth… and then getting ready for my trip down to North Carolina for the summer. So, I try to do it as I’m going during the semester so that I keep in my blackboard folder for each course a hidden folder where I list any problems… and I’ll do that for the course overall, as well as within individual modules. That way, when I go to refresh the course in the future I’ll have a list of things in general I want to do differently as well as specific recommendations in specific components of the course.

Rebecca: Have you ever accidentally made one of those hidden files not hidden?

John: I have not, no. [LAUGHTER]. I’m much more likely to leave something hidden that the students have as an assignment, but they’re usually pretty good at reminding me of that as we go through.

Rebecca: I think my greatest fear of having notes like that would be that I would make them really public and then probably have some sort of snarky comment in my hidden files. [LAUGHTER]

John: So, we thought maybe we talked a little bit about our lists of plans and then make some general recommendations of things that we found useful. So, Rebecca would you like to go first?

John: Sure, I think both of us have a fairly aggressive reading reading dream list. I don’t know how much either of us will get through that list, but my list includes Race Talk and [the] Conspiracy of Silence by Derald Wing Sue… which jDerald Wing Sue’s coming to our campus in the fall to give a talk based on this book… and we’re gonna have a reading group again. So, I want to make sure I’m on top of that.

John: That’s also on my list. I started reading it earlier, but I got buried in the semester, so it’s on the top of my summer reading list.

Rebecca: Yeah, I read the first chapter but then that’s as far as I got. I’m also planning to read… I started reading but I didn’t have time to finish a book called Designing Disability: Symbols, Space, and Society by Elizabeth Guthrie. It’s a really interesting book about the history of the wheelchair symbol. So, it’s related to design, obviously, which is my area of teaching… but also my interest in accessibility, which I’ve been working on a lot on campus. Related to that, I also am planning to read Disability in Higher Education: a Social Justice Approach by Nancy Evans. I started reading that during this semester and read a few chapters here and there but didn’t get all the way through. It’s a pretty hefty read. So, I’m hoping to get through a lot of that this summer… and then I have two other books that are not so much teaching related but come out of the design field. One of them is Demystifying Public Speaking, by Laura Hogen, which is from a series called A Book Apart… it’s made for designers, so I’m hoping to read that book and pull out some nuggets that might be helpful for students who get a little nervous about public speaking… or see whether or not it’s a good recommendation for our advanced students in our program… and then the other one that comes from a designer is Meeting Design for Managers, Makers, and Everyone by Kevin H. Hoffman. I’ve seen Kevin speak and have had some conversations with him in the past about designing meetings, so that meetings are actually productive and useful rather than unproductive and something that could maybe have happened in an email. So, I’m looking forward to reading a fuller version of his process. What are you hoping to read, John?

John: Well, several these I’ve already started again but haven’t gotten too far but they’re enough so that they’re on my Kindle or I have the books very handy… and I plan to read them as soon as I can. One is The ABCs of How We Learn by Daniel Schwartz. I actually made it, I believe, through letter L before I had to put it down to get caught up on some other things.

Rebecca: Yeah, I remember getting some updates in the various letters and it did kind of fizzle out.

John: So, I will finish that fairly soon, I believe. The Spark of Learning is a book I’ve heard wonderful things about from Sarah Rose Cavanagh. I’m hoping to read that this summer. It’s also on my Kindle app. The Teach Students How to Learn book by Saundra Yancy McGuire and Thomas Angelo is a really good book that talks about ways of improving student metacognition. Again, I’ve read a little bit of that just to see that it is something I really want to continue with. Another thing I’d like to look at, since I teach large classes where I use a lot of multiple-choice questions, is a book that I heard about on a couple of other podcasts on teaching and learning… in particular, the Teaching in Higher Ed podcast, which is Learning and Assessing with Multiple-Choice Questions in College Classrooms by Jay Parkes and Dawn Zimmaro. That’s something I haven’t started yet, but I do have a copy of that and I’m looking forward to reading it. Another book that somewhat on the border between teaching and learning and my work in economics is The Undoing Project by Michael Lewis. It’s a book on the early development of behavioral economics by Kahneman and Tversky, and the reason why it’s on the border of economics and teaching is that behavioral economics explains why people don’t always behave as rational agents… and certainly that’s important in trying to understand how people work from an economics perspective… but when we’re dealing with students and faculty we observe that people don’t always behave, perhaps,in an optimal fashion. We don’t see people engaging in activities that are in their long-run self-interest, and they often will prefer short-run benefits over long term benefits, even though they know they’d be better off doing their work a bit earlier and so forth. So, it overlaps between those two interests. I’m looking forward to that I guess that’s it for my books.
So, what are your plans for redeveloping or redesigning some of your courses?

Rebecca: Well, I have a new class that I’ll be offering in the fall that’s related to some other special topics I’ve taught before on experience design… and in that class we’re gonna do two community projects: one is called “recollections storytelling through mementos“ which is the design of an interactive exhibit that will travel to multiple adult care facilities in central New York. It’s the second exhibition in a series. The last one we did was a couple of years ago… and so the design and development of that will happen partially through the summer and then in my class in the fall… and then the exhibit will go up and travel next year in 2019… and then the other project that we’re gonna work on is our very famous [LAUGHTER] regular guest Allison Rank, who’s talked about her project Vote Oswego. My students will be working on that project as well, doing some design work with her class. We scheduled our two classes so that they would be at the same time slot, so that they could collaborate a little bit easier this time. so I’m looking forward to working with Allison a little bit this summer to make some specific plans for that for the Fall. So, I’m doing that and then revisiting my web design courses like I do every year: a) the content generally changes because standards and things and web change but I’m also… I had my little list, as I was grading, of things that I want to make sure that I’m doing and some of that means integrating more reflective practice opportunities I think it’s really important and I always plan on doing that and then somehow it gets cut. So, I decided I really need to just actively decide to cut something else out, so that there is actually that room and that’s not what gets cut in the future.

I’m also working on some new accessibility modules and I’m also really thinking of… I’ve been doing a lot of quizzes based on our reading groups and things that we’ve been talking about for retrieval practice… but I’m really thinking about switching to trying some in-class polls even though my class is relatively small and mixing in some practical exercises and I was doing both of those kinds of things in the quizzes and I think spreading those out a little bit will actually help with engagement, and also make it so it doesn’t take up as much class time.

John: In terms of the use of polling in small classes… for the last five or six years now I’ve been using polling in classes that I teach at Duke where generally there are between sixteen and twenty students, and it works just as well in small classes as it does in large ones. In some ways it works a little bit better.

Rebecca: Yeah, I can imagine that and I know that you’ve talked about that in the past, so you’re wearing on me. [LAUGHTER]

John: It’s a good practice.

Rebecca: Yeah, how about you?

John: Well, I’ve got a number of things planned. One is, I’ve been wanting to adopt an OER for a long time, but I’ve been somewhat tied to the adaptive learning tools and so forth provided by publishers, as well as the array of materials they provide… but, I want to explore some OER options for my large introductory class.

Rebecca: For those that aren’t familiar, what’s an OER.

John: Open educational resources… basically things that are released under Creative Commons licenses… and there’s two major advantages of that: one is that it would be free for students… students would also have access from the first day of class, and we’ll be talking about that more in future episodes… and another thing I’d like to do more is explore some alternatives to publisher provided adaptive learning tools so that it might be possible to find some ways of integrating OER with it, or to investigate ways in which OER materials can be used with adaptive learning systems that can work in classes where you want to have enough variety in the question so students can’t just look them up on the internet…

Rebecca: …and if you’re a little more interested in OER and the kind of big impact that that can have on students, you may want to check out Robin Derosa’s article in Higher Ed “OER Bigger than Affordability.” …and then we also have a previous episode that’s about adaptive learning that people might want to check out if they’re curious about that.

John: I believe was episode 30. Another thing I’d like to do, along the same lines, is I had written an econometrics text that I’ve been using in class for a while. I’d like to rewrite that as an OER text, and one of the things I need to do is update some of the old videos I’ve created. Last winter, when I was at the OLC conference in Orlando (at Disney World) I saw a presentation on Videoscribe and I had seen some videos created by that and it just looked really really cool and so I purchased a subscription to that and now I actually have to actually learn how to use it… and it does involve a bit of work… and there’s a bit of start-up costs in that, but it’s a very powerful tool and it looks like a really good way of presenting technical material.

I’d also like to explore a little bit of Flipgrid just because i’ve used voicethread now and I keep hearing really good things about Flipgrid, so I’d like to look at that and compare the benefits of the two systems.

Rebecca: What’s a Flipgrid?

John: Flipgrid is very much like Voicethread except the videos are provided in a grid. In many ways, it’s very similar to Voicethread except your class shows up as an array on the screen. You can click on any of the boxes for the students and hear or see their responses.

Rebecca: So, it sounds like the interfaces may be the big benefit there.

John: I believe so. I need to explore it more. It’s something I’ve been hearing a lot about from a lot of people who do some really good work, so I’d like to see how it compares.

Rebecca: You know all your talk of OERs and open education resources reminded me that one of the key things I have on my to-do list is to explore all the available resources that are available on openpedagogy.org. After hearing Robin DeRosa talk about it at CIT, the conference that John and I were at in late May, I got really excited about some of her teaching techniques and I just really want to see what else is out there and what’s available. So, who knows, it might really overhaul something.

John: I was at the same talk and we were both so impressed by it we went down and we talked to Robin at the end and we’ve invited her to come back to Oswego in the fall to give a presentation here, and there’s a good chance that she will appear as a guest on a future episode of the podcast. So, there’s also some things we’d like to recommend to others: books and tools that we found really useful. So, would you like to start?

Rebecca: Alright, so most of our recommendations are publications that have highly influenced our show. So one of those is Minds Online by Michelle Miller, a great cognitive psychologist. The book is about being online, but all the things she talks about works in in-person classes too, so I highly recommend that book.

John: Michelle Miller, after I had read her book, so impressed me that I invited her to come up to campus to give a workshop here… and people were so impressed by that that we created our reading group series here. Our first one was Michelle Miller’s Minds Online and participants were so enthused about that they insisted that we bring her back again at the end of the reading group and she was a wonderful speaker as well as a very good author.

Rebecca: Yeah, and a great facilitator too. We also want to recommend Barbara Oakley’s Learning How to Learn MOOC. It’s a great way to learn the basic cognitive science behind the evidence-based practices. So, if you’re not familiar, that’s a great way to follow along and get involved and her videos are fantastic.

John: It’s also the most popular MOOC in the world…

Rebecca: …and it’s the biggest one too, right?

John: and it’s the biggest one and she’s got hundreds of thousands of students taking it. It’s a four-week experience and I encourage all my students to take it.

Rebecca: …and if you’ve never done a MOOC, what a great experience to take one of the best MOOCs in the world.

John: It also provides very good examples of effective practice for online teaching that are very scalable. So, there’s a lot of good reasons to do it.

Rebecca: She also has some other great books including: A Mind for Numbers, Mind Shift, and Learning How to Learn: How to Succeed in School without Spending all your Time Studying; a Guide for Kids and Teens. That last one is a new one that’s directed specifically at middle school and high school students.

John: Another book, I think, that we’d both strongly recommend is Make it Stick. We used that as our second reading group here at Oswego a couple years ago, and Peter Brown came up and presented on that. but it’s by Brown, Roediger, and McDaniel. Peter Brown is a novelist and Roediger and McDaniel have done a tremendous amount of work in studying how people learn.

Rebecca: We can’t go without mentioning Carol Dweck’s Mindset book as well. We often see who we might traditionally think of as being quote unquote good students, “A” students maybe who hit something in college where they realize that they have to struggle a little bit and they don’t know what to do, because everything’s always come easily to them… but they struggle because they don’t have a growth mindset. So, this is a great way to learn more about the differences between fixed and growth mindsets and maybe put some strategies in place to help all of our students move more towards a growth mindset in the courses we teach.

John: The next thing we recommend is Jim Lang’s Small Teaching. it covers much of the same material as Minds Online and Make it Stick but it does it in a somewhat different way. It focuses on small techniques that you can change in your classroom that pay off very substantially. So, for people who don’t want to substantially revise their courses, it’s a very effective way of making small modifications… activities that take five to ten minutes in a class… that have a very large impact without requiring a dramatic overhaul or restructuring of your course.

Rebecca: Yeah, and the faculty here have responded very well to this book and have made a lot of small changes to their classes in the last year and had big success.

John: Another thing we’d like to mention are some podcasts that we listen to that have some really good coverage of topics related to higher education. The first one is Teaching in Higher Ed by Bonnie Stachoviak. The other one we want to recommend is Teach Better by Doug McKee and Edward O’Neill and you might remember Doug McKee from a previous episode.

Rebecca: So, we usually conclude by asking what’s next, but if you really want to know you could just listen to this episode again. We made a lot of references during this episode to a lot of great material and I can’t imagine that you wrote it all down, especially if you’re driving in your car, right? So, remember to check the show notes will have specific links and details so that you can find all these resources so that you can also enjoy some of these during your summer.

John: If any of you have any recommendations for topics for the show, please write to either of us. Our email addresses will be in the show notes.

Rebecca: We also wanted to take a couple minutes and just reflect on the podcast itself. We really appreciate the community of listeners that we’ve gained. We never expected this to even go on this long. It was a little experiment that we had that we wanted to try out in the fall and now we’re on Episode… oh, I don’t know what episode we’ll be on.

John: We’ve been really impressed by how many listeners we’ve reached across the U.S. and throughout the world. We were expecting we’d mostly get people listening from our institution and perhaps some of our colleagues in other places. So, we very much appreciate all the support you shown.

Rebecca: …and please let us know if there’s other things that we can cover that you’re really interested in or really need some professional development in.

John: We hope you’re enjoying your summer vacation. Enjoy the rest of your summer!

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

24. Gender bias in course evaluations

Have you ever received comments in student evaluations that focus on your appearance, your personality, or competence? Do students refer to you as teacher or an inappropriate title, like Mr. or Mrs., rather than professor? For some, this may sound all too familiar. In this episode, Kristina Mitchell, a Political Science Professor from Texas Tech University, joins us to discuss her research exploring gender bias in student course evaluations.

Show Notes

  • Fox, R. L., & Lawless, J. L. (2010). If only they’d ask: Gender, recruitment, and political ambition. The Journal of Politics, 72(2), 310-326.
  • MacNell, L., Driscoll, A., & Hunt, A. N. (2015). What’s in a name: Exposing gender bias in student ratings of teaching. Innovative Higher Education, 40(4), 291-303.
  • Miller, Michelle (2018). “Forget Mentors — What We Really Need are Fans.” Chronicle of Higher Education. February 22, 2018..
  • Mitchell, Kristina (2018). “Student Evaluations Can’t Be Used to Assess Professors.Salon. March 19, 2018.
  • Mitchell, Kristina (2017). “It’s a Dangerous Business, Being a Female Professor.Chronicle of Higher Education. June 15, 2017.
  • Mitchell, Kristina M.W. and Jonathan Martin. “Gender Bias in Student Evaluations.” Forthcoming at PS: Political Science & Politics.

Transcript

Rebecca: Have you ever received comments in student evaluations that focus on your appearance, your personality, or competence? Do students refer to you as teacher or an inappropriate title, like Mr. or Mrs., rather than Professor? For some, this may sound all too familiar. In this episode, we’ll discuss one study that explores bias in course evaluations.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.
Today our guest is Kristina Mitchell, a faculty member and director of the online education program for the Political Science Department at Texas Tech. In addition to research in international trade and globalization, Kristina has been investigating bias in student evaluations, motherhood and academia, women in leadership and academia, among other teaching and learning subjects. Welcome Kristina.

Kristina: Thank you.

John: Today our teas are?

Kristina: Diet coke. Yes, I’ve got a diet coke today.

[LAUGHTER]

Rebecca: At least you have something to drink. I have Prince of Wales tea.

John: …and I have pineapple ginger green tea.

John: Could you tell us a little bit about your instructional role at Texas Tech?

Kristina: Sure, so when I started at Texas Tech six years ago, I was just a Visiting Assistant Professor teaching a standard 2-2 load… so, two face-to-face courses in every semester, but our department was struggling with some issues in making sure that we could address the need for general education courses. So in the state of Texas every student graduating from a public university is required to take two semesters of government (we lovingly call it the “Political Science Professor Full Employment Act”) and so what ends up happening at a university like Texas Tech with pushing forty thousand students almost, is that we have about five thousand students every semester that need to take these courses… and, unless we’re going to teach them in the football stadium, it became really challenging to try and meet this demand. Students were struggling to even graduate on time, because they weren’t able to get into these courses. So, I was brought in and my role was to oversee an online program in which students would take their courses online asynchronously. They log in, complete the coursework on their own time (provided they meet the deadlines), and I’m in a supervisory role. My first semester doing this I was the instructor of record, I was managing all of the TAs, I was writing all the content, so I stayed really busy with that many students working all by myself. But now we have a team of people: a co-instructor, two course assistants, and lots of graduate students. So, I just kind of sit at the top of the umbrella, if you will, and handle the high level supervisory issues in these big courses.

John: Is it self-paced?

Kristina: It’s self-paced with deadlines, so the students can complete their work in the middle of the night, or in the daytime or whenever is most convenient for them, provided they meet the deadlines.

Rebecca: So, you’ve been working on some research on bias in faculty evaluations. What prompted this interest?

Kristina: What prompted this was my co-instructor, a couple of years ago, was a PhD student here at Texas Tech University and he was helping instruct these courses and handle some of those five thousand students… and as we were just anecdotally discussing our experiences in interacting with the students, we were just noticing that the kinds of emails he received were different. The kinds of things that students said or asked of him were different. They seemed to be a lot more likely to ask me for exceptions… to ask me to be sympathetic…. to be understanding of the student situation… and he just didn’t really seem to find that to be the case. So of course, as political scientists, our initial thought was: “we could test this.” We could actually look and see if this stands up to some more rigorous empirical evaluation, and so that’s what made us decide to dig into this a little deeper.

John: …and you had a nice sized sample there.

Kristina: We did. Right now, we have about 5000 students this semester. We looked at a set of those courses. We tried to choose the course sections that wouldn’t be characteristically different than the others. So, not the first one, and not the last one, because we thought maybe students who register first might be characteristically different than the students who register later. So, we took we chose a pretty good-sized sample out of our 5,000 students.

John: …and what did you find?

Kristina: So, we did our research in two parts. The first thing we looked at was the comments that we received. As I said, our anecdotal evidence really stemmed from the way students interacted with us and the way they talked to us. We wanted to be able to measure and do some content analysis of what the students said about us in their course evaluations. So, we looked at the formal in-class university-sponsored evaluation, where the students are asked to give a comment on their professors… and we looked at this for both our face-to-face courses that we teach and the online courses as well. And what we were looking for wasn’t whether they think he’s a good professor or a bad professor, because obviously if we were teaching different courses, there’s not really a way to compare a stats course that I was teaching to a comparative Western Europe course that he was teaching. All we were looking at was what are the themes? What kinds of things do they talk about when they’re talking about him versus talking about me? What kind of language do they use and we also did the same thing for informal comments and evaluation? So, you have probably heard of the website “Rate My Professors”?

John: Yes.

[LAUGHTER]

Kristina: Yes, everyone’s heard of that website and none of us like it very much… and let me tell you, reading through my “Rate My Professors” comments was probably one of the worst experiences that I’ve had as a faculty member, but it was really enlightening in the sense of seeing what kinds of things they were saying about me… and the way they were talking about me versus the way they were talking about him. So again, maybe he’s just a better professor than I am… so we weren’t looking for positive or negative. We were just looking at the content theme… and so the kinds of themes we looked at were: Does the student mention the professor’s personality? Do they say nice… or rude… or funny? Do they mention the professor’s appearance? Do they say ugly… pretty? Do they comment on what he or she is wearing? Do they talk about the competence, like how how well-qualified their professor is to teach this course and how do they refer to their professor? Do they call their professor a teacher? Or do they call their professor rightfully a professor? And these are the categories that we really noticed some statistically significant differences. So we found that my male co-author was more likely to get comments that talked about his competence and his qualification and he was much more likely to be called professor… which is interesting because at the time he was a graduate student. So, he didn’t have a doctorate yet… he wouldn’t really technically be considered a professor… and on the other hand when we looked at comments that students wrote about me, whether they were positive or negative… nice or mean comments… they talked about my personality. They talked about my appearance and they called me a teacher. So whether they were saying she’s a good teacher or a bad teacher… that’s how they chose to describe me.

Rebecca: That’s really fascinating. I also noticed, not just students having these conversations, but in the Chronicle article that you published, there was quite a discussion that followed up related to this topic as well, and in that there was a number of comments where women responded with empathetic responses and also encouraged some strategies to deal with the issues. But, then there was at least one very persistent person, who kept saying things like: “males also are victimized.” How do we make these conversations more productive and is there something about the anonymity of these environments that makes these comments more prevalent?

Kristina: I think that’s a really great question. I wish I had a full answer for you on how we could make conversations like this more productive. I definitely think that there’s a temptation for men who hear these experiences to almost take it personally… as though when I write this article, I’m telling men: “You have done something wrong…” when that’s not really the case… and, my co-author, as we were looking at these results about the comments and as we were reading each other’s comments, so we could code them for what kinds of themes we were observing… he was almost apologetic. He was like: “Wow, I haven’t done anything to deserve these different kinds of comments that I’m getting. You’re a perfectly nice woman, I don’t know why they’re saying things like this about you.” So, I think framing the conversation in terms of what steps can we take to help, because if I’m just talking about how terrible it is to get mean reviews on Rate My Professors, that’s not really giving a positive: “Here’s a thing that you can do to help me…” or “Here’s something that you can do to advocate for me.” So, I think a lot of times what men who are listening need… maybe they’re feeling helpless… maybe they’re feeling defensive…. What they need is a strategy. Something they can do going forward to help women who are experiencing these things.

Rebecca: I noticed that some of the comments in relationship to your Chronicle article indicated ways that minimize your authoritative role to avoid certain kinds of comments and I wonder if you had a response to that… and I think we don’t want to diminish our authoritative roles as faculty members, but I think that sometimes those are the strategies that we’re often encouraged to take.

Kristina: I agree, I definitely noticed that a lot of the response to how can we prevent this from happening got into “How can we shelter me from these students,” as opposed to “How can we teach these students to behave differently.” I definitely think the anonymous nature of student evaluation comments and Rate My Professors and internet comments in general. You definitely notice when you go to an internet comment section that anonymous comments tend to be the worst one. …and so the idea that what we’re observing, it’s not that an anonymous platform causes people to behave in sexist ways, It’s that there’s underlying sexism and the anonymous nature of these platforms just gives us a way to observe the underlying sexism that was already there. So the important thing is not to take away my role as the person in charge. The important thing is to teach students, and both men and women, that women are in positions of authority and that there’s a certain way to communicate professionally. Student evaluations can be helpful. I’ve had helpful comments that help me restructure my course. So, it’s a way to practice engaging professionally and learning to work with women. My students are going to work for women and with women for the rest of their lives. They need to learn, as college students, how to go about doing that.

John: Do you have any suggestions on how we could encourage that they’re part of the culture and in individual courses the impact we have is somewhat limited. What can we do to try to improve this?

Kristina: Well, I’ve definitely made the case previously to others on my campus and at other campuses that the sort of lip service approach to compliance with things like Title 9 isn’t enough. So, I don’t know if there at your institution there’s some sort of online Title 9 training, where you know…

John: Oh, yeah…

Kristina: …you watch a video

Rebecca: Yeah…

Kristina: … you watch a video… you click through the answers… it tells you: “are you a mandatory reporter?” and “what should you do in this situation?” …and I think a lot of people don’t really take that very seriously; it’s just viewed as something to get through so that the university cannot be sued in the case that something happens. So, I don’t think that that’s enough. I think that more cultural changes and widespread buy-in are a lot more important than making sure everyone takes their Title 9 training. So, in our work I mentioned that we did this in two parts, and the second part just looked at the ordinal evaluations. The 1 to 5 scale, 5 being the best… rank your professor how effective he or she is… and not only are students perhaps not very well qualified to evaluate pedagogical practices, but once again we found that even in these identical online courses, a man received higher ordinal evaluations than a woman did. And so what this tells me is in a campus culture we should stop focusing on using student evaluations in promotion and tenure, because they’re biased against women… and we should stop encouraging students to write anonymous comments on their evaluations. We should either make them non-anonymous or we should eliminate the comment section all together. Just because if we’re providing a platform it’s almost sanctioning this behavior. If we’re saying, “we value what you write in this comment,” then we’re almost telling students your sexist comment is okay and it’s valued and we’re going to read it… and that’s not a culture that’s going to foster positive environment for women.

John: Especially when the administration and department review committees use those evaluations as part of the promotion and tenure review process.

Kristina: Exactly. I mean when I think about the prospect of my department chair or my Dean reading through all the comments that I had to read through when I did this research, I’m pretty sure that he would get an idea of who I am as a faculty member that, to me…maybe I’m biased… but to me, is not very consistent with actually what happens in my classroom.

Rebecca: It’s interesting that anonymity.. right, we talk about anonymity providing more of a platform for this become present. But I’ve also had a number of colleagues share their own examples of hate speech and inappropriate sexual language when anonymity wasn’t a veil that they could hide behind, increasingly more recently. So I wonder, if your research shows any increase in this behavior and why?

Kristina: We haven’t really looked at this phenomenon over time. That’s just not something that we’ve been able to look at in our data, but I would like to continue to update this study. I definitely think that… current political climate is creating an atmosphere where perhaps people don’t feel that saying things that are racist or sexist are as shameful as they once perceived them to be. So there’s definitely a big stigma against identifying yourself as Nazi or even Nazi adjacent and that stigma, while it’s still there, the stigma against it seems to be lessening a little bit. I don’t know necessarily that I’ve seen an increase in what kinds of behavior I’m observing from my students, but I definitely will say that a student… an undergraduate student… gave me his number on his final exam this last semester like I was going to call him over the summer. So, it definitely happens in non-anonymous settings too.

John: Now there have been a lot of studies that have looked at the effect of gender on course evaluations, and all that I’ve seen so far find exactly the same type of results. That there’s a significant penalty for being female. One of those, if I remember correctly (and I think you referred to it in your paper), was a study where… it was a large online collection of online classes, where they changed the gender identity of the presenters randomly in different sections of the course, and they found very different types of responses and evaluations.

Kristina: Yes, that was definitely a study that that… I hate to say we tried to emulate because we were limited in what we could do in terms of manipulating the gender identity of the professor… but I think that their model is just one of the most airtight ways to test this. I agree, this is definitely something that’s been tested before. We’re not the first ones to come to this conclusion… I think our research design is really strong in terms of the identical nature of the online courses. At some point, I find myself… when I when I was talking about this research with a woman in political science who’s a colleague of mine… the question is how many times do we have to publish this before people are going to just believe us… that it’s the case. The response tends to be: “Well, maybe women are just worse professors or maybe there’s some artifacts in the data that is causing this statistically significant difference.” I don’t know how many times we have to publish it before before administrations and universities at large take notice… that this is a real phenomenon… that’s not just a random artifact of one institution or one discipline.

John: It seems to be remarkably robust across studies. So, what could institutions do to get around this problem? You mentioned the problem with relying on these for review. Would peer evaluation be better, or might there even be a similar bias there?

Kristina: I definitely think peer evaluation is an alternative that’s often presented, when we’re thinking of alternative ways to evaluate teaching effectiveness. Peer evaluation may be subject to the same biases. So, I don’t know that literature well enough off the top of my head, but I imagine that it could suffer from the same problems in terms of faculty members who are women… faculty members of color… faculty members with thick accents, with English that’s difficult to understand… might still be dinged on their peer evaluations. Although we would hope that people who are trained in pedagogy who’ve been teaching would be less subject to those biases. We could also think about self evaluation. Faculty members can generate portfolios that highlight their own experiences, and say here’s what I’m doing the classroom that makes me a good teacher… here are the undergraduate research projects I’ve sponsored… here the graduate students who’ve completed their doctoral degrees under my supervision… and that’s a way to let the faculty member take the lead in describing his or her own teaching. We could also just weight student evaluations. We know that women receive 0.4 points lower on a five-point scale, then we could just bump them up by 0.4. None of these solutions are ideal. But, I think some of the really sexist and misogynist problems in terms of receiving commentary, that is truly sexually objectifying female professors… that could be eliminated with almost any of these solutions. Peer evaluation… removing anonymous comments… self-evaluation…. and that’s really the piece that is the most dramatically effective in women being able to experience higher education in the same way that men do.

Rebecca: So, obviously if there’s this bias in evaluations then there’s likely to be the same bias within the classroom experience as well. We just don’t necessarily have an easy way of measuring that. But if you’re using teaching strategies that use dialogue and interactions with students rather than a “sage on the stage” methodology, I think that in some cases we make ourselves vulnerable and that does help teaching and learning, because it helps our students understand that we’re not you perfectly experts in everything… that we have to ask questions and investigate and learn things too… and that can be really valuable for students to see. But we also want to make sure that we don’t undermine our own authority in the classroom either. Do you have any strategies or ideas around around like that kind of in-class issue?

Kristina: Yeah, I think that the bias against women continues to exist just in a standard face-to-face class. One time, when I was teaching a game theory course, I was writing an equation on the board and it was the last three minutes of class and we’re trying to rush through you the first-order conditions and all sorts of things… and I had written the equation wrong, and as soon as my students left the classroom I looked at it and I went, “oh my gosh, I’ve written that incorrectly,” and so the next day when they came back to class, I I felt like I had two choices: we could either just move on and I could pretend like it never happened, or I could admit to them, that I taught this wrong… I wrote this wrong. So I did. I told them “Rip out the page from yesterday’s notes because that formula is wrong,” and I rewrote it on the board… and I got a specific comment in my evaluation, saying she doesn’t know what she’s talking about.. that she got that she got this thing wrong… and it was definitely something that, while I don’t have an experimental evidence that says that if a man does the same thing you won’t get penalized in the same way, to me it very much wrapped into that idea that women are are perceived as less qualified as men. So whether it’s because we’ll refer to as teachers or whether it’s because the student evaluations focused more on men’s competence, women are just seen as less likely to be qualified. How many times have you had a male TA and the students go up to the TA to ask questions about the course instead of you. So, I definitely think it’s difficult for women in the classroom to maintain that authority, while still acknowledging that they don’t know everything about everything No professor could. I mean we all think we do of course…. So, I think owning some of the fact that there are things you don’t know is important, no matter what your gender is, but I also try to prime my students I tell them about the research that I do. I tell them about the consistent studies in the literature that exists that shows that students are more likely to perceive and talk about women differently, because I hope that just making them aware that this is a potential issue, might adjust their thinking. So that if they start thinking “wow, my professor doesn’t know what she’s talking about” they might take a moment, and think “would I feel the same way if my professor were a man.”

Rebecca: I think that’s an interesting strategy. We found the similar kind of priming of students about evidence-based practices in the classroom works really well… and getting students to think differently about things that they might be resistant to… So, I could see how that that might work, but I wonder how often men do the same kind of priming on this particular topic.

Kristina: I don’t know. That would be an interesting next experiment to run if I were to do a treatment in two classes face-to-face classes and and you know do have a priming effect for a woman teaching a course versus a man and seeing if it had any kind of different effect. I think a lot of times men perhaps aren’t even aware that these issues exist. So, talking about the way that women experience teaching college in a different way… if men aren’t having this conversation in their classroom, it’s probably not because they’re thinking, “oh man, I really hope my female colleagues get bad evaluations so that they don’t get tenure.” It’s probably just because they aren’t really thinking about this as an issue… just because as a sort of white man in higher education you very much look like what professors have looked like for hundreds of years… and so it’s just a different experience, and perhaps something that men aren’t thinking about… and that’s why I’m getting the message out there so important because so many men want to help. They want to make things more equitable for women and I think when they’re made aware of it, and given some strategies to overcome it, they will. I’ve definitely found a lot of support in a lot of areas in my discipline.

John: …and things like your Chronicle article there’s a good place to start too… just making this more visible more frequently and making it harder for people to ignore.

Kristina: I agree. I think being able to speak out is really important, and I know sometimes women don’t want to speak out, either because they’re not in a position where they can or because they’re fearing backlash from speaking out. So, I think it’s on those of us who are in positions where we can speak up. I think it falls on us to try and say these things out loud, so that women who can’t… their voices are still heard.

John: Going back to the issue of creating teaching portfolios for faculty… that’s a good solution. Might it help if they can document the achievement of learning outcomes and so forth, so that that would free you from the potential of both student bias and perhaps peer bias. So that if you can show that your students are doing well compared to national norms or compared to others in the department, might that be a way of perhaps getting past some of these issues?

Kristina: I definitely think that’s a great place to start, especially in demonstrating what your strategies are to try and help your students achieve these learning outcomes. I always still worry about student level characteristics that are going to affect whether students can achieve learning outcomes or not. Students from disadvantaged backgrounds… students from underrepresented groups… students who don’t come to class or who don’t really care about being in class… these are all students who aren’t going to achieve the learning outcomes at the same rate as students who come to class… who are from privileged backgrounds… and so putting it on a professor alone to make sure students achieve those learning outcomes, still can suffer from some things that aren’t attributable to the professor’s behavior.

John: As long as that’s not correlated across sections, though, that should get swept out. As long as the classes are large enough to get reasonable power.

Kristina: Yeah, absolutely. I think it’s definitely it’s time for more evaluation into into how these measures are useful. I know there’s been a lot of articles in the New York Times op-ed, I think there was one in Inside Higher Ed, really questioning some of these assessment metrics. So, I think the time is now to really dig into these and figure out what they’re really measuring.

Rebecca: You’ve also been studying bias related to race and language, can you talk a little bit about this research?

Kristina: Yes, so this is a piggyback project after after I got finished with the gender bias paper, what I really wanted to do was get into race, gender, and accented English. Because I think not only women are suffering when we rely on student evaluations, it’s people of different racial and ethnic groups… it’s people whose English might be more difficult to understand. What we were able to do in this work is control for everything. So, we taught completely identical online courses the only difference we didn’t even I didn’t even allow the professors to interact with the students via email. I told them to make sure I… like Cyrano de Bergerac…writing all of their emails for them over a summer course and so they were handling the course level stuff just not the student facing things. They were teaching their online course but they weren’t directly interacting with the students in a way that wasn’t controlled… and the the faculty members recorded these welcome videos, which had their face… it had their English, whether it was accented or not… and I’m I asked some students who weren’t enrolled in the course to identify whether these faculty members were minorities and what their gender was. Because what’s important isn’t necessarily how the faculty member identifies – as a minority or not – as whether the students perceive them as minority… and even after controlling for all of that… controlling for everything… when everything was identical, I thought there was no way I was going to get any statistically significant results, and yet we did. So, we controlled even for the final grades in the course… even we controlled for how well students performed… the only significant predictor for those ordinal evaluation scores with whether the professor was a woman and whether the professor was a minority. We didn’t see accented English come up as significant, probably because it’s an online course. They’re just not listening to the faculty members more often than these introductory welcome videos. But we did when we asked students to identify the gender and the race of the professor’s based on a picture. We asked the student: “Do you think you would have a difficult time understanding this person’s English” and we found that Asian faculty members, without even hearing them speak, students very much thought that they would have difficulty understanding their English… and then we have a faculty member here who… blonde hair and blue eyes… but speaks with a very thick Hispanic accent, and the students who looked at his picture… none of them perceived that they would have a difficult time understanding his English. So, I think there’s a lot of biases on the part of students just based on what their professors look like and how they sound.

John: Can you think of any ways of redesigning course evaluations to get around this? Would it help if the evaluations were focused more on the specific activities that were done in class… in terms of providing frequent feedback… in terms of giving students multiple opportunities for expression? My guess is it prob ably wouldn’t make much of a difference.

Kristina: I think, as of now, the way our course evaluations here at Texas Tech University look is that they’re asked to rate their professors you know in a 1 to 5 on things like “did the professor provide adequate feedback?” and “was this course a valuable experience?” and” “was the professor effective?” and that gives an opportunity for a lot of: “I’m going to give five to this professor, but only fours to this professor” even when the behaviors in class might not have been dramatically different. Now this is also speculation, but maybe if there was more of a “yes/no,” “Did the professor provide feedback?” “Were there different kinds of assignment?” “Was class valuable?” Maybe that would be a way to get rid of those small nuances. Like I said, when we did our study, the difference was .4 out of a five-point scale, and so these differences aren’t maybe substantively hugely different. Maybe it’s a difference between you know a 4 and a 4.5. Substantively, that’s not very different. So, maybe if we offered students just a “yes/no,” “Were these basic expectations satisfied?” maybe that could help and that might be something that’s worth exploring. I definitely think that either removing the comment section altogether, or providing some very specific how-to guidelines on what kinds of comments should be provided. I think that that’s the way to address these open-ended say whatever you want… “are you mad? “…are you trying to ask your professor out? …trying to eliminate those comments would be the best way to make evaluations more useful.

John: You’re also working on a study of women in academic leadership. What are you finding?

Kristina: A very famous political science study, done by a woman named Jennifer Lawless, looked at the reasons why women choose not to run for office. So we know that women are underrepresented in elective office, you know the country’s over half women but, we’re definitely not seeing half of our legislative bodies filled with women. What the Lawless and Fox study finds, is not that women can’t win when they run, it’s just that women don’t perceive that they’re qualified to run at all. So, when you ask men, do you think you’re qualified to run for office, men are a lot more likely to say: “oh yeah, totally… I could I could be a Congressman,” whereas women, even with the same kind of qualifications, they’re less likely to perceive themselves as qualified. So, what my co-author Jared Perkins at Cal State Long Beach and I decided to do, is see whether this phenomenon is the same in higher education leadership positions. So one thing that’s often stated is that the best way to ensure that women are treated equally in higher education, is just to put more women in positions of leadership… that we can do all the Title 9 trainings in the world, but until more women are in positions of leadership, we’re not going to see real change…. and we wanted to find out why we haven’t seen that. So you know 56 percent of college students right now are women, but when we’re looking at R1 institutions only about 25% of those university presidents are women, and then the numbers can definitely get worse depending on what subset of universities you’re looking at. We did a very small pilot study of three different institutions across the country. We looked at an R1 and R2 and an R3 Carnegie classification institution. Our pilot study was small, but our initial findings seem to show that that women are not being encouraged to hold these offices at the same rate as men are. So what we saw was that… we asked men “have you ever held an administrative position at a university?” About 60% of the men reported that they had, and about 27% of women reported that they had, and we also asked “Did you ever apply for an administrative position? …and only 21% of the men said that they had applied for an administrative position, while 27% of women said they had applied. Off course it could be that they misunderstood the question… that maybe they thought we meant “Did you apply and not get it?” but we also think that there may be something to explore when it comes to when women apply for these positions they get them. There are qualified women ready to go and ready to apply, but men may be asked to take positions… encouraged to take positions… or appointed to positions where there might be opportunities to say: “There’s a qualified woman. Let’s ask her to serve in this position instead.”

John: That’s not an uncommon result. I know in studies and labor markets starting salaries are often comparable, but women are less likely to be promoted and some studies have suggested that one factor is that women are less likely to apply for higher level positions. Actually, there’s even more evidence that suggests that women are less likely to apply for promotions, higher pay, etc. and that may be at least a common factor that we’re seeing in lots of areas.

Kristina: Absolutely. I definitely think that University administrations need to place a priority on encouraging women to apply for grants, awards, positions, and leadership because there are plenty of qualified women out there, we just need to make sure that they’re actively being encouraged to take these roles.

Rebecca: Which leads us nicely to the motherhood penalty. I know you’re also doing some research in this area about being a mother and in academia, can you talk a little bit about how this impacts some of the other things that you’ve been looking at?

Kristina: Absolutely. The idea to study the motherhood penalty in academia stemmed from reading some of those “Rate My Professor” comments. Because at my institution, we didn’t have a maternity leave policy in place… so I came back to work after two weeks of having my child and I brought him to work. So my department was supportive. I just brought him into my office and worked with the baby for the whole semester… and it was difficult, it was definitely a challenge to try and do any kind of work while a baby is, in the sling, in front of your chest… but one of my “Rate My Professor” evaluations from the semester that I had my son, mentioned that I was on pregnancy leave the whole semester and I was no help. And so this offended me to my core, having been a woman who took two weeks of maternity leave before coming back to work… because I didn’t… I wasn’t on maternity leave the whole semester, and in addition… if I had been, what kind of reason is that to ding a professor on her evaluation? Like she birthed a human child and is having to take care of that child… that shouldn’t ever be something that comes up in a student comment about whether the professor was effective or not.

So what we want to look at are just the ways in which women are penalized when they have children. Even just anecdotally, and our data collection is very much in its initial stages on this project… but as we think through our anecdotal experiences, when department schedule meetings at 3:30 or 4:00 p.m., if women are acting as the primary caregiver for their children (which they often are) this disadvantages them because they’re not able to be there. You have to choose whether to meet your child at the bus stop or to go to this department meeting… or networking opportunities, are often difficult for women to attend if they’re responsible for childcare. Conferences have explored the idea of having childcare available for parents because, a lot of times, new mothers are just not able to attend these academic conferences… which are an important part of networking and most disciplines… because they can’t get childcare. So at the Southern Political Science Association meeting that I went to in January, a woman brought her baby and was on a panel with her baby. So, I think we’re making good strides in making sure mothers are included, but what we want to explore is whether student evaluations will reflect differences in whether they know that their professor is a mother or whether they don’t. So, how would students react if in one class I just said I was cancelling office hours without giving a reason and then in another class, I said it was because I had a sick child or I had to take my child to an event. That’s kind of where we’re going with this project and we really, really hope to dig into what’s the relationship between the motherhood penalty and student evaluation.

Rebecca: Given all of the research that you’re doing and the things that you’re looking at, how do we start to change the culture of institutions?

Kristina: Well, I’m thinking that we’re on the right direction. Like I said, I see a lot more opportunities at conferences for childcare and for women to just bring their children. I see a lot of men who are standing up and saying, “hey, I can help, I’m in a position of power and I can help with this” and what, you know, without our male allies helping us, I mean, men had to give women the right to vote, we didn’t just get that on our own. So, we really count on allies to put us forward for awards. One thing, I think, that’s an important distinction that I learned about from a keynote speaker is the difference between mentoring and sponsoring. So, mentoring is a great activity, we all need a mentor, someone we can go to for advice, someone we can ask for help, someone who can guide us through our professional lives. But what women really need is a sponsor, someone who will publicly advocate for a woman whether that’s putting her in front of the Dean and saying, “Look at the great work she’s doing” or whether it’s writing a letter of recommendation saying, “This woman needs to be considered for this promotion or for this grant.” Sponsorship, I think, is the next step in making sure that women are supported. A mentor might advise a woman on whether she should miss that meeting or that networking opportunity to be with her child. A sponsor would email and say, “we need to change the time because the women in our department can’t come. because they have events that they need to be with their children.”

John: A similar article appeared in a Chronicle post in late February or maybe the first week in March by Michelle Miller where she made a slightly different version. Mentoring is really good… and we need mentors, but she suggested that sometimes having fans would be helpful. People who would just help share information… so when you do something good… people who will post it on social networks and share it widely in addition to the usual mentoring role. So, having those types of connections can be helpful and certainly sponsors would be a good way of doing this.

Rebecca: I’ve been seeing the same kind of research and strategies being promoted in the tech industry, which I’m a part of as well. So, I think it’s a strategy that a lot of women are advocating for and their allies are advocating for it as well. So hopefully we’ll see more of that.

Kristina: I think the idea of fans and someone to just share your work is hugely important. I have to put in a plug for the amazing group: “Women Also Know Stuff.”

Rebecca: Awesome.

Kristina: It’s a political science specific website, but there are many offshoots in many different disciplines and really it’s just the chance that, if you say, “I need to figure out somebody who knows something about international trade wars.” Well, you can go to this website and find a woman who knows something about this, so that you’re not stuck with the same faces… the same male faces,,, that are telling you about current events. So “Women Also Know Stuff” is a great place. They share all kinds of research and they just provide a place that you can look for an expert in a field who is a woman. I promise they exist.

Rebecca: I’ve been using Twitter to do some of the same kind of collection. There might be topics that I teach that I’m not necessarily familiar with… scholars who are not white men… And so, put a plug out like, “hey, I need information on this particular subject. Who are the people you turn to who are not?”

John: You just did that not too long ago.

Rebecca: Yeah, and it, you know, I got a giant list and it was really helpful.

John: One thing that may help alleviate this a little bit is now we have so many better tools for virtual participation. So, if there are events in departments that have to be later, there’s no reason why someone couldn’t participate virtually from home while taking care of a child, whether it’s a male or female. Disproportionately, it tends to be females doing that but you could be sitting there with a child on your lap, participating in the meeting, turning a microphone on and off, depending on the noise level at home, and that should help… or at least potentially, it offers a capability of reducing this.

Rebecca: I know someone who did a workshop like that this winter.

John: Just this winter, Rebecca was doing some workshops where she had to be home with her daughter who wasn’t feeling well and she still came in, virtually, and gave the workshops and it worked really well.

Kristina: Yeah, I definitely think that that’s a great way to make sure that that everyone’s included, whether it’s because they’re mothers or fathers or just unavailable… and I think that’s where we look to sponsors… the department chairs… department leadership to say, “This is how we’re going to include this person in thid activity” rather than it being left up to the woman herself to try and find a way to be included. We need to look to put people in positions of leadership to actively find ways to include people regardless of their family status or their gender.

Rebecca: This has been a really great discussion, some really helpful resources and great information to share with our colleagues across all the places that…

John: …everywhere that people happen to listen… and you’re doing some fascinating research and I’m going to keep following it as these things come out.

Rebecca: …and, of course, we always end asking what are you gonna do next. You have so many things already on the agenda but what’s next?

Kristina: So next up on my list is an article that’s currently under review that looks at the “leaky pipeline.” So the leaky pipeline is a phenomenon in which women, like we were saying, start at the same position as men do, but then they fall out of the tenure track, they fall out of academia more generally… they end up with lower salaries and lower position. So, we’re looking at what factors, what administrative responsibilities, might lead women to fall off the tenure track. We already know that women do a lot more service work and a lot more committee work than men do, so we’re specifically looking at some other administrative responsibilities that we think might contribute to that leaky pipeline.

Rebecca: Sounds great. Keep everyone posted when that comes out and we’ll share it out when it’s available.

Kristina: Thanks.

John: …and we will share in the show notes links to papers that you published and working papers and anything else you’d like us to share related to this. Okay, well thank you.

Kristina: Thank you.
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

20. New faculty transition

New faculty often come out of graduate programs that have trained them to be researchers but not teachers. The transition into full time teaching can be stressful and overwhelming for these colleagues. Maggie Schmuhl, a new faculty member in the Public Justice Department at SUNY-Oswego joins us to discuss how she has embraced evidence-based methods in her practice as a teacher.

Show Notes

  • Dweck, C. S. (2006). Mindset: The new psychology of success. Random House Incorporated.
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • LePore, Jill (2014). “The Surprising Origin Story of Wonder Woman.” The Smithsonian Magazine. October
  • 12. The Active Learning Initiative at Cornell.” the 1/17/2018 Tea for Teaching podcast discussion with Doug McKee in which two-stage exams were discussed.

Transcript

Rebecca: Today, our guest is Dr. Maggie Schmuhl, a first-year faculty member in Public Justice at SUNY Oswego. Her research focuses on structural inequality, violence against women, and punishment. At SUNY Oswego, Maggie has taught American Criminal Courts and Judicial Process and Women in Crime. Welcome.

John: Welcome.

Maggie: Hello.

John: Today our teas are…

Maggie: I have a green tea that’s mint.

Rebecca: I’m also drinking green tea today, but mine is jasmine green tea.

John: I’m drinking a custom blend of peppermint, spearmint, and tarragon.

Rebecca: Ooh, yum.

Maggie: Fancy.

[LAUGHTER]

Rebecca: So, today we have a slightly different setup. We invited Maggie to come in and talk a little bit about the experience of a first-year faculty member and that transition from graduate school to teaching. Can you describe a little bit about what that transition’s been like? You’ve completed one full semester; you’re into your second semester. So, what’s that transition like? And what are some of the biggest hurdles?

Maggie: Yeah, the first year has been a lot of trial and error, a lot of learning curves and really just getting to know the university… getting to know the students… the department… and all of the intricacies of balancing research, balancing teaching, getting better at both of those things, and you know making time to explore a new place, a new city, it’s been good so far.

Rebecca: So in graduate school, was there a focus on teaching and developing curriculum, or was it more focused on research?

Maggie: Grad school was certainly more focused on research. It was about developing our research styles, our methodologies, our research interests. Teaching was not a major focus for a lot of reasons… but often teaching was a responsibility that we had, but not one that was explored as in-depth as our research. Since my first year in the program they have implemented third-year development seminars to talk about teaching, but for most of us, we had to really find our own way, we had to rely on upper cohort members to help guide us through our first time teaching, and we really had to spend our own time thinking about what kind of teachers we wanted to be, and how much effort we wanted to put into our teaching.

John: This is not uncommon in graduate programs. The faculty are focused on their research because they have to be if they want to keep their jobs. There are some programs that do more professional development, but they’re relatively rare… at least in my discipline.

Rebecca: I went to graduate school at Syracuse University and they actually had a development program for graduate students.

Maggie: Oh, that’s interesting.

Rebecca: So there was in the beginning, but then I also was in a fellowship program, where you actually put together teaching portfolios, and things, like you would if you were applying for teaching positions and things. That was part of the development and there was ongoing workshops.

Maggie: Yeah, so I think, for me, when I realized that I had a passion for teaching, I spent a lot of time seeking out professors that were engaged in wanting to make all of us into effective teachers… and so that drive, I think, that perhaps came from my desire to be a better teacher, helped me find better resources… within the program… within faculty… and I also served as the president of our doctoral association, and so when it came time to go on the job market, we sought out those faculty that were interested in helping us develop our teaching portfolios, and so we’d hold programs but a lot of it was student driven.

Rebecca: Did you find it challenging, when you were in graduate school. to balance that? When you have this interest and desire to explore teaching almost as a secondary research interest, right?

Maggie: Right.

Rebecca: How did that work? and what challenges did you face because of that? Because I’m sure you had colleagues in the same position who weren’t as interested in teaching and just didn’t spend that much time on it.

Maggie: Sure…. and I think the demands of grad school really keeps any particular person from excelling at teaching… and spending the time that it takes to implement and learn about effective practices for most of us. We’re trying to finish our dissertations. We’re trying to publish on research, and while all of that’s important, there’s kind of a piece of the puzzle that gets neglected, and often it was teaching.

John: …and so, you’re now at a four-year school and you’ve been really active in some of our workshops. We had a reading group last semester that focused on Small Teaching and you attended that workshop regularly… and read through that… and then you implemented some things. How did that go?

Maggie: Yes, I really enjoyed those Small Teaching reading group that we did,0 mostly because it gave me the time and in place to really explore what I wanted to be in the classroom and how I wanted my students to interact with me in the classroom. In that group, I really enjoyed the opportunity to talk with other faculty members who have this experience and have the kind of classrooms that I want to emulate and to get to learn. I really enjoyed the Small Teaching reading group. It gave me a place and a regular time to work into my schedule to sit down, talk about the kind of teacher I wanted to be, to listen to teachers with a lot more experience and how they develop their classroom, and implement these effective learning strategies to create a more productive learning environment, and to teach students and to challenge them to think critically about the world.

Rebecca: I think sometimes carving out time is one of the most difficult things, right? To think about teaching.

Maggie: Yeah.

Rebecca: So the fact that you said that the reading group provides this regular way of holding you accountable to think about these things…

Maggie: Yeah, absolutely. So, I think being a first semester faculty member I wanted to get involved in the campus community… to get to know other faculty members… to see how they’ve been successful in and outside of the classroom… and for me to try to broaden my perspectives on teaching… try to learn about the techniques that are important to facilitate learning… and to carry on to help students become valuable members in the justice system and whatever career paths they they choose.

John: Now a little bit of background on the reading group, we had a hundred and two faculty and staff members who participated in it, a large proportion were faculty. We met multiple times a week, and one of the things that happened there is people from different disciplines got a chance to talk about issues they’ve had in the classrooms, and how they’ve worked on it, and they got suggestions from other people in different departments. How did you find that experience getting to work with faculty from the sciences, from the humanities, from art and so forth?

Maggie: What I really liked about the diversity of the faculty there was… especially the math teachers. Every time they would talk about their experience in the classroom, I remember my own struggles and successes at learning something like math. And to think of a discipline that we wouldn’t normally consider has (or can) benefit (maybe) from a variety of teaching methods. I think that hearing their experiences throughout their teaching careers gives some important insight I can carry on to my own classes.

John: Could you tell us a little bit about the courses that you teach, how big they are and what the subjects are?

Maggie: Yes, so currently I teach… well, and last semester I also taught American Criminal Courts and Judicial Process. These classes ranged anywhere from fifteen students to about forty students, depending on the time of day and the enrollment. The other class that I currently teach this semester is Women and Crime, and that is about thirty-five students.

John: …and what techniques have you tried that were new to you?

Maggie: So, one of the challenges I had in my first semester here was teaching at an 8 a.m. section, and trying to…

[LAUGHTER]

John: There are no good solutions…

Maggie: Yeah…

Rebecca: Was it trying to teaching at 8 am or was it the students trying to take a class at 8 am?

Maggie: I think it was the students trying to take a class at 8 am… for sure. It’s hard…

John: It’s hard to stay up that late…
MAGIE: It is.

John: …they get tired and they need to get some sleep.

Maggie: Yes, that’s the end of their day, as opposed to the beginning of most of ours… and so, the 8:00 a.m. class… it was like pulling teeth trying to get them engaged and participating. In the first semester, I think I carried a lot of the same methods and practices that I had developed in grad school, and some of them… through the Small Teaching reading group, I found that I have names for all of that practice… like retrieval practices… and summarizing and recapping at the beginning of courses and at the end of a lecture… and holding small discussion groups. But, somehow none of that was quite enough to bring the 8 a.m. students back to thinking critically about the judicial system. Recently, in my current semester, I’ve started pairing up the students… and I can’t remember exactly what we called this, but pairing up the students to… after they’ve completed a mini pop quiz in class (which they all freak out about, but eventually I tell them that it’s not being graded). So, I’ve paired them up and they discussed their answers and then as a group they present their new answers and…

John: …a think-pair-share method.

Maggie: Yeah, a think-pair-share method. Yeah, so I’ve been implementing think-pair-share and a lot of the 8AM students, especially the ones that were falling asleep, they’re now forced to you know really think about this material right off the bat, and it’s helped keep them engaged throughout the rest of the course. I’ve really enjoyed taking that method and seeing them wake up a little bit more.

Rebecca: That’s great…

Maggie: Yeah.

Rebecca: …it’s not a super difficult thing to employ…

Maggie: It’s really easy.

Rebecca: …but it makes a huge difference.

Maggie: Right, Absolutely.

Rebecca: …and something that may work across all class periods, but sometimes you just have that particular class that’s got a slightly different personality…

Maggie: Right, yeah.

Rebecca: Sometimes due to the time or sometimes just the makeup of the group, that …employing different things in that situation… sometimes you have to troubleshoot like that.

Maggie: Yeah, absolutely and I think…often when you do have big personalities in the classroom, they’re so much fun… and they really they bring the other students into the discussion. But when you don’t have that, if you have students who are maybe a little more timid in the classroom, I think that think-pair-share is a good way to bring each and every one of our students into the discussion.

John: What were some of the other techniques that you may have tried?

Maggie: I’ve also implemented some low-stakes testing for my American Criminal Courts class and that has been going well so far. We’ll see how everything continues in this semester. I’m hoping that this will leave them more prepared for their midterm and their final exams by continually asking them… and then asking them questions from prior lectures… a lot of interleaving… also give them bonus questions on those quizzes to help them predict what we’re talking about next week, so I think there’s going to be a tangible difference in their grades when midterms roll around.

Rebecca: I was really surprised when I implemented more testing. You always hear conversations about test anxiety and nobody likes tests… nobody wants to take tests… and I’m in a discipline where tests are not that common. But I’ve been surprised historically, that the students maybe grumble at the beginning about it…

Maggie: Um-hmm.

Rebecca: …but over time they actually really appreciate it… and if you didn’t have one, like “what’s going on, why don’t we have one today?” They find it helpful and useful to keep them on point. How are students responding to this regular testing?

Maggie: I was actually really surprised when I got an evaluation last semester where a student asked if they could have more tests in the class. Because the format of the exams was four exams a semester, they were longer and they were, I think, looking for something that kept them accountable for the readings… something that kept them accountable for paying attention in class… and so far everyone has been… I don’t think they’re thrilled with it, but I think they understand the reason for having the tests… because I took time at the beginning of our class to talk about why having these low-stakes testings are important for their learning, but important also as they prepare for exams… and to really get this foundational information to build on in future classes.

John: …just simply reminding them that, making a mistake on a quiz that an infinitesimally small part of their grade is much better than making that mistake on a major exam…. and reminding them that this is, in large part, for formative purposes can really help, I think.

Maggie: Yeah, and for some students – who are perhaps a little more on the perfectionist side – I’ve had a couple of them pretty concerned when they miss a quiz but I tell them that I’ve dropped the lowest quiz score…maybe it’ll be the lowest two quiz scores… we’ll see how the semester goes… but to keep them accountable, but also to remind them that they’re human and things happens… they miss a quiz… they forget… and they have the opportunity to learn from that mistake, but to have not such a detrimental effect on their on their grade.

Rebecca: One thing that came up in our reading group frequently, was that faculty had much more success in their classes with this particular technique, if you took the time in class to talk to your students about why you’re doing it and how that’s helping their learning. I think that most faculty who’ve implemented it like you, and have spent the time to share that information with students, have found far more success than faculty who have just implemented the technique without explaining it .

Maggie: Yeah, and that’s one thing I’d really took from the reading group too, is that if we explain to students why the things that they typically hate are actually important and are beneficial to them, there’s a lot more buy-in from them.

John: It’s helpful in general, because students have habits of learning that we know aren’t as effective as they could be, and they tend to resist things like testing for learning. They much prefer rereading things or highlighting things, and after the second re-reading the evidence is pretty clear that there is virtually no increase in the amount that they’ll remember later… but it doesn’t feel that way. When they take a test on something and they get things wrong, it doesn’t feel as good. So, it’s important to help set it up and prime them so that they understand that this is really useful for them.

Rebecca: …and I think when you take class time to talk about evidence-based practices, not just on the first day, but a few times throughout the semester, those same students who are struggling with other parts of learning will speak up and ask more questions. I had a student today who just called me over and was asking like “I’m having a really hard time figuring out how to structure my files, so I don’t lose stuff.” Just a basic organizational thing… but in my field that’s quite detrimental, actually, if you can’t figure out how to do it. …and so, it’s kind of funny. Now they see you as a resource of someone who can help me learn better, not just in this class, but in other classes too…

Maggie: Yeah.

Rebecca: …which is kind of a nice feeling sometimes.

Maggie: Yeah, and I’ve also found that even beyond pedagogical discussions with students, that some of these concepts actually can apply to a lot of the content that we’ve learned. So, in my Women in Crime classroom we’re talking about labeling theory, and what it does to a person when they think they’ve reached the limit of their identity, if they’ve….

John: …issues of stereotype threat.

Maggie: Yeah, right. Exactly. If they fit the stereotype, how do they… can they learn to move beyond that… and so I talk to them about how, when they’re journaling I write all of my comments to reflect the work that they’ve done, and not the person that they are. …and so, I had them to think back about some of those comments that I do make and I tell them that this was purposeful on my end… Because I want them to know that they can do better in some cases… and in other cases that work has reflected some of their best effort… and that’s a good thing, right? And that their effort is just as important as, perhaps more important as, who they are as a person.

John: So it helps build a growth mindset ..

Maggie: Yeah.

John: …of the sort that Carol Dweck talks about. and that’s really helpful, because if they can learn that they can learn and improve their work, they become much more effective.

Rebecca: One of the things that I’ve found, implementing some of these techniques (especially in the first couple of tries), is that, when it doesn’t feel good to learn, right? [LAUGHTER]… we sometimes have this illusion that we know something, but we don’t actually know it. The students can get a little downtrodden, right? … a little down on themselves… and so then you have to remember to monitor for that a little bit…. which I’ve learned over time, and then you got to kind of just stop and allow for an opportunity to show success. So, for example, in my classes, I started breaking my first project assignment into very small pieces, so everything was low stakes. But, I could see that at some point they just maxed out, and they’re “I can’t do any of this, like I don’t know any of this.” I was like “well, actually you do know most of it, you’re just panicking for no reason.” So, we stopped one day, and we just did we just did a brand new little thing that demonstrated to them that they could actually do the entire project that we’d been doing… in two hours, despite the fact that we’ve spent three weeks on it… and yes, there was a couple little hurdles that they had, but the hurdles they had were minor, and they could do it. I think to allow for that growth mindset… yes, you need them to fail and realize that they can do better, but then also allow for some opportunities where they get some real success too. It seems like you’re interested in this growth mindset idea, so have you been experimenting with any of these sorts of things in your classes?

Maggie: When I started doing the mini pop quizzes, because not only do I throw in a mini pop quiz occasionally, but I also have them doing low stakes quizzing over the weekends on Blackboard. So, I think they start to get a little overwhelmed. We were able to do this on our quiz, when we had everything in front of us, but now that we don’t have anything in front of us, you know, I can see it in their faces. They’re freaking out. They’re upset with themselves, because they knew that they had this information somewhere, they just hadn’t had the time to actually recall it without their notes. I think that, hopefully, the more we do this, the more they’ll take their quizzes maybe a little more seriously and try to really push themselves to do it without without their notes.

Rebecca: So, actually trying to recall the information as opposed to look it up.

Maggie: Right, yeah, right.

Rebecca: Yeah, sometimes students do things out of convenience and meeting a deadline as opposed to valuing the …

Maggie: Sure.

Rebecca: …learning and the more we demonstrate to them that we value that they’ve learned, the better. This semester, I just… for some reason my class is just full of anxiety in a way that maybe even last semester it wasn’t… I didn’t change anything… it’s the same thing… but I spent the time, I felt it…. this big ball of anxiety is not going to move forward… because the anxiety is getting in the way of learning now…. to just stop and recognize that you’re observing something… and then make a change…. this is what the syllabus says, but I’m feeling this and I see this. Do you guys agree? Yeah, great! How about we do this instead? Does that make sense to everyone? And then all of a sudden… buy-in again.

Maggie: Right, yeah. They feel like they have the ability to structure the class themselves right? ..that it’s not just you sitting there saying “This is what we’re doing.” You’re asking them: “If we change this, will this be better for our class?” I think that’s cool, and does keep them invested in the class itself.

Rebecca: What other kinds of supports do you feel a new faculty member needs in place to be successful as a teacher?

Maggie: So, I think that for me, because in grad school I sought out different faculty members and helped create some of the programs that I thought were missing… I think for those faculty members who haven’t put as much emphasis on their teaching, perhaps to reach out to them and maybe have a friend bring them along to activities and in workshops at the Center for Excellence in Learning and Teaching holds.

John: …and it’s not surprising though that some people, when they first start teaching would not put that much weight on teaching because they’ve just come from an R1 institution, where their focus is often entirely, or nearly entirely, on research … and most people start by teaching the way they’ve learned …and the way they’ve learned is often just simply lectures and exams…

Maggie: Yeah.

John: One of the things I’ve observed as a chair of our recruitment committee (for the last couple of decades) is that, in economics at least, a very large proportion of faculty have no background in teaching while they’re in grad school. In our last search, we had, I think, three or four people who had actually some knowledge of effective teaching practices… and actually three of them made it to our top five list of candidates, but they were by far the exception… and it’s a tough adjustment …and it takes a while, especially when you have to start your research very quickly in order to meet tenure requirements …it’s a difficult adjustment.

Maggie: Yeah.

Rebecca: I think the first semester, in general, is a tough adjustment. You’re at a new institution… there’s institutional memory that you’re not privy to… there’s all kinds of acronyms that you don’t understand. It takes a lot of time to figure out who this student population is… and you might not think that between institutions the students change very much, but man they’re really different… and you have to adjust your teaching to the population that you’re dealing with. As a designer, I would always jump on my soapbox to say you have to design for your audience, and I don’t think designing your classroom experiences is any different.

Maggie: I’d say in grad school, most of the courses I took were very heavy with reading…. and a lot of discussion based classes. But a lot of students don’t have the time to do the kind of reading we did… and that’s why it was grad school and this is undergrad…. and for them who are just learning… and I think we talked about this in the Small Teaching reading group… that we have the ability to make connections across different concepts and how they interrelate to each other, but the students aren’t there yet …and so, when you’re going from one institution where you have gauged where these students are and what kind of connections they’re able to make, because you know a little bit more about their experience, and then you move to a new university and those experiences… some of them are similar, but a lot of them are vastly different… and to gauge where they’re able to make these connections and how much I have to draw out those connections certainly changes on that university.

Rebecca: It isn’t just university, it’s sometimes like semester to semester….

Maggie: Sure.

Rebecca: …or group to group. You almost need to build in a way in your classes to figure out… some sort of little survey or something… like, who are these people? what do they already know?

Maggie: Yeah, yeah.

Rebecca: So that you can make those connections… so that you have a better understanding of what your class’s mental model looks like versus your own.

John: …and there’s a number of ways of doing it… some people will give a pretest at the beginning… just asking what people had, or some general questions about the discipline or their prior knowledge. Others will use clickers and other things throughout the term to assess knowledge before moving on. …and, as you said, just asking students to reflect on what they know and perhaps write it down or at least bring it to the discussion, is another good way to help determine the level, so you can do more just-in-time teaching and deal with what students come in with rather than what you think they should come in with.

Rebecca: What’s your favorite way of handling that John, in your classes?

John: It varies a lot by class. I use clickers regularly in actually nearly all of my classes. I don’t use it in the seminar class I’m teaching, but I’m using them in my econometrics class which is, I think, about 35 to 40 students, somewhere in that range… and I use it in my large class where I have somewhere between 360 and 420 students every fall… and it’s a good way of getting that sort of information. I’ve also sometimes used pre-tests on basic math skills or other things in my large intro class. Sometimes in a smaller class I’ll just ask them what classes they’ve had in the past. When I’m teaching econometrics, for example, I have some students who are math majors who’ve already had multivariable calculus and three or four stat classes. Other students come in who took, sometime in the last three or four years, a very basic stat class… and they come in with very different backgrounds and very different information. Some of the people in the class are math majors who just want to pick up a course and they haven’t had that many economic classes and so finding out what they know helps me determine how much emphasis I need to focus. One of the problems though is that students have such diverse that it’s difficult, but the more we can get them working together with peer instruction, the more they can help each other fill in those gaps.

Rebecca: I think one of the things that I like about what you were saying is that students, in some of those methods, it’s revealed what other students know too, so that they can see that “I have a different mental model, but it matches up with some other people, and here’s some other students and they have this other kind of information.” I find it helpful, and I think the students find it helpful to recognize that people come with different expertise and that we can lean on those expertise at different times. Do you have any strategies that you’ve been using to figure out who’s in the room yet or is that something that you’re still kind of experimenting with?

Maggie: It’s something that I’m still experimenting with. Last semester, I did a pretest and it was helpful for gauging what they knew about criminal justice, but not necessarily what they knew about courts and how those concepts in criminal justice relate to the court system… and this semester I did not do a pretest, but another thing that, I think has been an interesting way to see what they know when they know it is I’ve had them write down for a minute the important things we’ve talked about last class and then I go around the room and every person has to say at least one thing… and everything they know about that thing… but it has to be different from the other person and so that challenges them… and of course the people in the very back corner of the class are freaking out because they’re afraid everything’s gonna be said… but then it turns out that we only get to half of the topics that we actually discussed in the class. So, I think that giving them the opportunity to actually see what they know, I think is important for them in the classroom moving forward.

Rebecca: Yeah, a lot of students don’t have good metacognitive skills, in general. They have no idea what they know. So, if you take the time to get them to even stop and think about what they know, it’s more time than they probably spent on it…

Maggie: Right. Yeah.

Rebecca: …giving them the time and demonstrating that you think it’s valuable for them to spend time thinking about what they know… by you even spending two or three minutes in class on it… all of a sudden lets them know that that’s something that’s valuable.

John: …and actually I’m going to be trying something new this semester to help build on those skills with something that Doug McKee talked about in an earlier podcast, which is the use of two-stage exams. I’m giving an exam in my econometrics class next week next Wednesday and then they’ll all take it individually. Then I will grade them, and then the next class they’re going to work on subset of the questions in small groups… and then they’ll submit group responses which will be weighted as a portion of their overall score. So, basically, they get the opportunity to improve their scores on some of the more challenging questions by sharing their knowledge and in places where that’s been done they found some fairly significant learning gains from leveraging the knowledge of their peers.

Rebecca: Sounds very similar to Maggie’s strategy for some of her quizzes.

Maggie: Yeah. that’s exactly what the mini pop quiz that I’ve been implementing… that they can try to figure out their answers on their own, but then they can really talk with another student… see where their strengths are where… perhaps the other students strengths are and builds from there.

Rebecca: I think what’s nice about pointing out those two examples is that both emphasize peer instruction… but ones in a high-stakes situation… ones in a low-stakes situation. The fact that there’s kind of two graded parts in the in the two-stage exam is a way to demonstrate a way of doing it in a higher stakes situation and then a nice low stakes situation where they try it on their own but then they can collaborate before they’re ever graded is a different scenario and a different level of pressure, etc.

John: …and adding one more level complexity to mine… it’s actually a little lower stakes than it might sound because, while they have three exams (I’ve told them this at the beginning of the term but I’ll have to remind them after they get their test scores back), I have a series of exams that are progressively cumulative, and if they do better on the second exam it will replace the first; if they do better on the third exam it will replace either of the first two… because they’re tested on all the material again. So, they get another chance. It does make the subsequent exams a little bit higher stakes if they didn’t do well, but they have the opportunity to make mistakes, learn from that, and improve their scores.

Rebecca: Great.

John: …and thanks to Doug for the suggestion about the the two-stage exams!
Now, you’re also going to be trying something new next year. Oswego is introducing some new signature courses for students in their first year. You, very graciously, were one of the new faculty who chose to participate in that. Could you tell us a little bit about the course that you’re going to be doing?

Maggie: There’s a group of faculty who have been asked to teach very small seminar like courses that are really aimed at engaging students in their very first year… their very first semester at Oswego and with the intent to get them connected to the University and to get them really excited about their coursework and uncover some of their interests. The class that I’ll be teaching is called the Injustice League. It’s on crime, inequalities, and injustice in comic books… and so, in the class we’re going to read a lot of comic books and some graphic novels… and we will have the chance to talk about how those comic books reflect inequalities that exist in society… how they reinforce some of those inequalities… and how comic books are used today to deliver how comic books are used today to facilitate discussions on race, gender, class inequalities.

John: That sounds like a lot of fun

Maggie:Yeah, it’s gonna be…

Rebecca: Yeah, can I sign up?

John: I wish I could take many of these class but that one in particular sounded really interesting.

Maggie: Yeah, it’s gonna be fun, I think, for students who perhaps don’t have an interest in comic books… or those who do, they can share their interests and we can learn a lot of things about the worlds on our way.

Rebecca: What I like about that is using something that’s more popular media and then using it as a tool to apply critical lenses…

Maggie: Yes

Rebecca: …and really getting students to think critically. We don’t have to always think so abstractly to think critically…

Maggie: Sure.

Rebecca: …a comic book is a nice tangible thing that you can look at and analyze and evaluate.

Maggie: Yeah, and apply to other discussions that we have going on in society. So if it’s examining gender and equality, and the way female superheroes are portrayed in their dress, and how we, as a society, have developed gendered expectations of women and of women of color and all of those intersecting identities.

John: …and Wonder Woman was developed in large part to help correct some of those gender imbalances, right? …it was developed by a psychologist who wanted to help provide a better gender…

Rebecca: Have you seen what she wears?

John: Well, there was that, too… and and there were some other issues there, but that was the rationale…

Rebecca: Yeah, of course, it was a male psychologist.

John: It was, but…
[LAUGHTER]
…in any case…. Okay, never mind.

Rebecca: I was just having a conversation with another one of our colleagues the other day about some of the topics that are really intangible like race, gender, and inequality. There are things that we talk about, but they’re abstract or conceptual, and so sometimes students have a really hard time getting their finger on it and finding a doorway in. So, I like the idea of the comic book as a really specific physical object… a really tangible space to enter into those discussions rather than thinking up in the air….

Maggie: Yeah.

Rebecca: …where it’s not always easy to digest that if you don’t have a good mental model of what you’re talking about.

John: …and it may also make it easier for students to separate themselves from those issues, because of course they don’t have any biased views themselves… but when they start seeing it perhaps in the comic books it might be help them identify it more generally in society and in themselves and in the world around them.

Maggie: Yeah, it helps them uncover where their biases do lie.

John: Right, but they’re implicit biases in part because they’re not aware of them so…

Maggie: Right. Yeah, they discover where these gendered expectations that they have been surrounded by throughout their growth as children and adolescents… and in why those expectations are problematic, and through comic books they can actually see it. We can actually point to why wealth inequality can create a superhero or it can create a villain.

John: …and it’s also perhaps less threatening that way, because it’s a comic book. It’s not their life directly, but it’s a nice lens by which they can start seeing these issues better.

Maggie: Absolutely.

Rebecca: Sounds like you had a lot of exciting teaching things on the horizon, what else are you gonna do next?

Maggie: I would like to develop a class that looks at punishment and the historical development of punishment and how sexism and racism in society have influenced that development in our society. …and I think for classes, we’re talking about really tough subjects that it’s important for them to be engaged and to feel comfortable having these conversations, because they are so necessary… and through these techniques, I think it gives them a level of comfort in the classroom to be themselves and to understand their positionality in society and how their experiences have impacted the way they view these social issues, and how they can resist against some of those preconceived ideas.

John: Excellent.

Rebecca: Right.

John: …and I think this could be a great topic for a future podcast…

Rebecca: Yeah, we’ll follow up on that

John: …when your class is underway.

Rebecca: Absolutely. Definitely. Well, thank you so much for joining us Maggie and sharing your perspective. I think a lot of faculty can relate and have had similar struggles and also similar successes, but it’s really nice to see your journey. So, thanks for sharing it with us today.

Maggie: Thanks.

John: Thanks a lot. It’s great having you here at Oswego.

Maggie: Thank you. It’s good being here.