188. Student-Ready Courses

College faculty sometimes complain that many of the first-year students who enter their courses are not “college ready.” In this episode, Natalie Hurley joins us to examine strategies that can be used to ease this transition and help ensure that our courses are “student ready.” Natalie is a New York State Master Teacher and a 2018 NNSTOY STEM Fellow who teaches high school mathematics in the Indian River Central School District in Watertown, NY.

Shownotes

Transcript

John: College faculty sometimes complain that many of the first-year students who enter their courses are not “college ready.” In this episode, we examine strategies that can be used to ease this transition and help ensure that our courses are “student ready.”

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Natalie Hurley. Natalie is a New York State Master Teacher and a 2018 NNSTOY STEM Fellow who teaches high school mathematics in the Indian River Central School District in Watertown, NY. Natalie was also one of our students here at SUNY Oswego. Welcome, Natalie,

Natalie: T hank you so much for having me. It’s a pleasure to be here.

Rebecca: Today’s teas are… Natalie, are you drinking any tea?

Natalie: I am, I’m drinking a peach tea because it’s warming up outside and I want to feel that warmth.

Rebecca: That sounds right up John’s alley,

John: …and I am drinking a ginger peach green tea.

Rebecca: I knew there’d be peach. It’s almost like you guys coordinated. I have Irish breakfast today.

John: No Scottish?

Rebecca: I am almost out of the Irish breakfast. And then we will move on to a different container.

John: Move down the Empire [LAUGHTER] and head back to your English breakfast and afternoon.

Rebecca: Yeah. [LAUGHTER]

John: Okay, we’ve invited you here to talk about the transition between high school and college. Could you first tell us a little bit about the courses that you teach?

Natalie: I teach high school math. And I have a pretty nice spectrum. I teach algebra I, calculus, and precalculus. I’m in my fifth year of teaching precalculus and calculus, and I have taught algebra I, eight years.

John: Excellent. What were your majors in college?

Natalie: So, I was a math major from the get go. And then I had this fantastic professor who helped me find my love of economics. His name is Professor John Kane, and he’s here with us too. [LAUGHTER] And then, once I started to find my love of economics, I picked up a minor in it. Unfortunately, it was too late in the game to pick up a double major, but the minor was great too.

John: And we did talk a little bit about you going on to grad school in economics. And I was a little disappointed that you didn’t, but then I thought you could do a lot of good in the school system. So, one of the things we want to talk to you about is the differences between your experiences in college and the way you teach your classes now. Could you talk a little bit about some of the differences between what you observed as an undergraduate student and the way in which you teach now in secondary school?

Natalie: Yes. So there are definitely a lot of differences. Although I tried to definitely keep that idea of college and career readiness, being wholesome in my classroom. I remember in college it was a lot of lecture style, a combination between very, very large classes of hundreds. And my math classes were usually pretty small, less than 20, generally. In my classroom, in a non pandemic year, I have somewhere between 15 and 20 students usually. Of course, now we’re doing cohorts. So I have a class as small as two and another class as large as 13… is my largest class… on Thursdays and Fridays. So I just remember, in college, a lot of lecture style: the teacher talks, you write notes, you can ask questions, but it’s very much like you raise your hand, if a teacher calls on you, you can ask a question. In my classroom, it’s more a conversation about math, whereas I’m leading them to discover the math, hopefully on their own, with a little bit of questioning from me to lead them there. There’s a lot of… in a non-pandemic year, think-pair-shares, or working in groups, or talking it out with their partner, or somebody that they sit near… things like that, that’s a little bit harder to do these days, especially when I only have a class of two in there sitting six feet away from each other or more.

Rebecca: Everything’s a think-pair-share. [LAUGHTER]

Natalie: Right. So yeah, and as I grow, and I become more seasoned, it becomes more student centered and less about me than it had been before. So I want to put the students in the center, whereas when I was there, it was very much the teachers up there telling you how to do it. And then this is how you do the calculus. And then this is how you do calc 2, this is how you do calc 3, and they just show you and you find your study groups, and then you work with each other after class as opposed to in class. So that seems to be some of the differences. I remember in college is a lot of rote memorization: know these words, be able to regurgitate this proof, things like that, where just now, in my classroom, and really the focus of high school math, and really all math through the Common Core standards, is that deep understanding, getting to the core and understanding vertically, what number sense and number theory is throughout all the grade levels.

Rebecca: It sounds like you’ve incorporated a lot of evidence-based practices into your own practice, which we encourage all of our faculty, even at the college level, to do as well. We hear a lot of faculty complaining about students not being “college ready.” And maybe that’s because there’s still a lot of lecture and memorization, and we want to keep changing that. What are some ways that we can become more “student ready” and bring students in?

Natalie: I think it’s very important that colleges understand what it’s like in the high school, how teachers are teaching, what they’re teaching. I’m in my ninth year, and the next year, we’ll start to move into the next gen math standards. And then that’s going to be the third set of standards. And I think it’s really important for college professors at all levels to be able to understand what did these standards look like for these kids? And how do I need to change what I’m teaching to adapt to that? I recently had a very informal discussion with the master teacher program and SUNY professors at SUNY Delhi about their math and science curriculum. And there was just a lot of surprises on both ends about what’s being taught, what isn’t being taught any more… emphasis on the calculator… standards now are being written so that there’s a lot of calculator use and students are losing a lot of their number sense, because you don’t need to know how to add and subtract, multiply and divide fractions when you get into high school, because you have a calculator that’s going to do it all, which turns into a lot of button pushing. But that’s definitely going to have a trickle down effect into… or trickle up effect, I guess… in college, when these students get there and they’re so used to being fully dependent on calculator and technology use… which is great if you’re a college that also promotes that. But some colleges may also still say, “Hey, we want to limit technology.”

John: Since you were in college, I think much of our teaching has changed. I know in my own department, most of us were doing a lot of lecturing when you were a student. And there’s been a pretty steady transition away from that. And that’s been true in many departments, including our math department.

Natalie: That’s great to hear.

John: But we still see a lot of lecture. And one of the things we’re hoping is that perhaps the experience of the transition to remote instruction has encouraged more people to try some new approaches to teaching and learning.

Natalie: We have definitely seen that in the high school, a lot of teachers who were behind the ball, as far as utilizing technology in their classrooms, it’s been forced to become great at it… literally overnight. So I guess if there’s a silver lining to all of this, that is definitely key right there. I think something that’s extremely important in high school is making connections to real world, making everything very real life, how am I ever going to use this in real life and making sure that that is evident in the student learning. And I don’t teach in a college, I haven’t been in a college in 10-15 years. But I think that if you wanted to be more student ready is also connecting all of your curricula to career readiness. Students want to know why am I taking this class if I’m going to be a librarian, or if I’m going to be a social worker. So just being able to bring those connections to them, could also help colleges be more student ready.

Rebecca: Just that relevance alone is more motivating to get students excited about topics. So finding ways to connect to students, no matter the level, high school or college, is a really great way to bring them in and bring them along.

John: One of the challenges I know in economics we face, is with students saying, “I’m just not very good at math,” or “I have trouble with graphs.” And I suspect maybe you might have seen a little bit of that, too. How do you address that issue of a fixed mindset concerning student’s ability to engage in mathematics?

Natalie: In my calculus and precalculus classes, and you’ll be even surprised to hear it that I do hear that there. But it’s extremely prevalent in algebra I, that idea of “My mom wasn’t good at math, so I’m not going to be good at math.” “I’ve always had math support, AIS,” or “I’ve been able to get through because I’ve always stayed after with the teacher.” I definitely still hear that, but the idea is to kind of break that mold by saying, “Hey, listen, this is a new year. This is a new teacher, this is a new curriculum, maybe this is going to be the one for you.” I find a lot of students struggle through geometry. So as soon as I bring up geometry, it’s like, “Oh, no, I don’t want to.” And I say, “Listen, let’s connect it to algebra, then. Do you feel more comfortable making the connections to algebra?” …or I don’t even know. I don’t know how to tell them that you can be better. Maybe you had a bad teacher, maybe you had a teacher who didn’t make things relevant for you, or just some bad experiences, but every year’s a new year. I’ve seen all sorts of success stories of students who truly find their niche late in high school math. So in my class, I’ve definitely adopted the idea of a growth mindset. And I’ve been studying on best practices to help do this. And part of that has been through standards-based grading. I’m kind of a beginner, a novice, of standards-based grading… more of a cafeteria, I’m choosing which aspects of it that I’d like to implement versus which ones don’t work for me, yet, as I learn and grow, which is definitely not the high school or college that I went to, to implement things like re-quizzes and retests. And it was hard for me to even start with this idea. Nobody else in my department was doing anything like this. And to be honest, as I was starting my career, I started my career right as common core was being implemented. So there was a ton of work in developing all the new curricula. I started in an eighth grade and moved all the way up and I’m like, “Wait, you want me to make two tests? four tests? two quizzes? three quizzes? four? And you want me to give them to these kids and grade them and change their grades and all of the clerical work. But now that I’m nine years in, I’m ready for it. And I find that the students have such a better… I guess they feel better about themselves when they go in to take a quiz when they know that there’s going to be redemption. One thing that I do is, if their test grade beats their quiz grade, I let the test grade replace the quiz grade in the gradebook. If you can show growth, by the time you get to the test… and that idea actually comes from college. If your grade on the final is better than your grade in the class, some professors will do something like that. Or if your grade is high enough, you don’t even need to take the final or I don’t know if anybody’s still doing things like that. But that kind of way, it takes the stress of testing and quizzing, and it takes it off of the grade and puts it more onto the learning. And, however, I do understand that that practice might not be making my students “college ready” if their college professors are not following that, which probably they are not. However, it’s something that I feel I need to do in order to focus on the learning that needs to happen.

Rebecca: I think one of the things that we’ve been working really hard to do at the college level is to incorporate some of those practices as well, to encourage learning, and that it’s about test taking and the testing effect as a way of remembering and learning and practicing rather than some moment in time that somehow reflects all learning that has ever occurred, which is not really relevant or accurate.

John: And you would be happy to know that many people in our math department are doing mastery quizzing and mastery learning approaches where they allow students multiple attempts to demonstrate mastery.

Rebecca: One of the things that I think is somewhat problematic for the transition of students from high school to college is that we don’t have these kinds of conversations very often. They’re not in place unless we go out of our way to talk to local high school teachers about what’s going on in local schools and vice versa, where high school teachers are asking us what’s going on at the college level. Can you talk a little bit about how your Delhi experience evolved in ways that you could encourage others to have those same kinds of conversations?

Natalie: I believe it was somebody from SUNY Delhi reached out to the Executive Director of the New York State Master Teacher program, if you haven’t heard about it, it’s a STEM initiative to retain teachers in math and science. And it’s a four-year fellowship that you have to apply and interview and be selected to join. But, it ends up creating a very strong network of teachers statewide that are actually affiliated with SUNY, every region has a SUNY campus that they are affiliated with, Up here in the North Country, mine is Plattsburgh. And since our program has moved more to an online format, we were able to all participate with SUNY Delhi through this conversation. And I think that just leads to more ideas for the future or to somebody who has an in with some SUNY people. I think we all do professional development. I do it as a high school teacher, and you guys do it as professors. But then, where can we find the middle ground that can maybe mesh some professional development and share some best practices between our two groups?

Rebecca: Sounds like an excellent idea.

John: That is a practice that I think we should do more of. One of the issues that we have is that in general, high schools provide a lot of support for students. And often some of that support comes from parents who help encourage students to be successful in school, and then all of a sudden, people go away to college. And sometimes that doesn’t work quite as well, and we lose a lot of people. Do you have any suggestions on what perhaps colleges could do better to help retain more students?

Natalie: That’s a great question. And I think as we grow through the years, you’re noticing a lot more of, shall I say, helicopter parents or parents who are very, very involved in their children’s lives, and so much to the fact that they actually can become bulldozer parents and literally take roadblocks out of their children’s way so that they never experienced any kind of struggle. And that, of course, is not anything that me as a high school teacher, or you guys as college faculty can do, other than just encouraging parents to step back. Students should be very autonomous. I think that, if anything, we’re going to see out of this pandemic. Another I guess silver lining is the students are becoming more autonomous. They have to when their parents are working and they’re only going to school two days a week and they’re responsible for their own learning the other three days a week. They’re also learning skills, like do I care for online learning? So when I get to college, should I steer towards classes that are online or steer away from classes that are online. So these are definitely going to be skills. So you guys might want to look out for when you’re screening students or as advisors, “Hey, how did you do when you were fully remote? Were you able to get all of your work done? Did your mom have to come home and sit down with you and work with you? Or were you able to get up in the morning, eat your breakfast, and then get right started on your schoolwork?” I make a joke to my seniors about this time of year that if they are still being woken up by their parents, they are not ready for college. You will be home by Christmas if your mother is still waking you up to get you in the shower and get to school on time. Students are going to have a lot of free time when they get to college. Their lives are managed for 40 hours a week, an hour bus ride in and back, somebody tells them when it’s time to eat lunch, then they go right to sports practice and come back. They’re so busy when they’re in high school. And then when they get to college, and they have class that are for 15 hours a week, that’s a lot of extra time for these kids to have to manage. And then when they start to find out, “Oh, nobody’s gonna call my mom, if I don’t go to school?” That’s the kind of mentality that these kids are going to. And I ask my students often, “What do you think is going to be one of your biggest struggles?” or “What do you think is going to be a biggest struggle of your peers? …and it is time management. The students are used to doing a lot of their work in school, either in the class or given class time to start assignments, to do assignments, or they have a study hall where somebody says, “Okay, get your books out, get to work.” And now they’re just going to go to class, they’re going to learn for an hour, hour and a half, and then go back to their dorm and have to start studying on their own. And even that, just the idea of needing to study, a lot of students can very easily get through high school without studying, without ever learning how to study. They just are good students, they know how to sit and behave, they learn well, they do their assignments, and that’s enough. I was a victim of that. I was always above average just by doing exactly what is expected of me. And then when you get to the bigger pond and you become the smaller fish, that can cause a lot of struggle for students, that idea of time that’s not managed, that they need to learn how to manage on their own.

Rebecca: I think you’re highlighting a lot of great themes here. And we certainly experience as college faculty, but your students are right: time management is the number one struggle of our college students. [LAUGHTER] They already know that it’s a problem. We know that it’s a problem. High school teachers know it’s a problem. But we don’t actively necessarily all collaborate on finding solutions to that problem. We just expect overnight students are somehow magically going to have autonomy and know what to do with it. [LAUGHTER]

John: And that’s especially true for first-gen students who haven’t had family members talk to them about those challenges and those issues, and who don’t have that sort of support. And suddenly, they have all this free time. And they don’t have any tests until weeks away or months away. And they just had these big assignments due so they don’t have anything they have to do right away. But one of the things that we know makes that worse is online instruction. Because there’s a lot of research. I did some about 17 or 18 years ago that found that freshmen and sophomores just did dramatically worse with online courses. And we had a guest on our podcast several months ago now, who found the same thing in a much larger study… that juniors and seniors and older individuals, if they’ve been successful in making it to that stage of college, they tend to be relatively successful at managing their time effectively. So online instruction is much more challenging for freshmen. And one of the things that I know many faculty are concerned about is the fact that all of our students coming in next year will have spent at least a year in some sort of remote or online instruction. And the quality of that varies quite a bit across school districts. And I think that is going to be a challenge we’re going to need to address. Do you have any thoughts on how we can help students be better at this?

Natalie: You are definitely correct, there are going to be a lot of gaps to fill, based on how students have received instruction, in how much instruction, we were given guidance that perhaps we’d only get through 80% of our material. And as you know, at the end of the 2020 school year, some of the learning immediately stopped in mid March with the “do no harm” idea of you can’t do harm to any of their grades, whatever their grade is when they left is what their grade needs to be. So we saw students who had very high averages that said, “You know what, it’s good enough. So I know you can’t give me anything less than a 98. So I’m not doing anything for the rest of the year.” And that was a very real situation that’s not going to be reflected in a transcript that you receive as a college advisor or as a college applicant when these kids are applying to school. I’m hopeful that students could self assess where they’re at, that if a college is going to have a lot of remote opportunities, and they are good at online learning, absolutely, go there, that’s for you. And if not, then that might not be for you.

John: I think we can also refer back to an earlier podcast we did with Viji Sathy and Kelly Hogan, who talked about the importance of structure in all of our classes, that providing students with clear directions, with giving them more expectations, and a couple of other podcasts that we had with Betsy Barre, she discusses the importance of sharing information about the time that’s required for various tasks. So giving students more detailed instructions and giving them more guidance on how much time they should be expected to work on things, perhaps, may help.

Natalie: I completely agree, I remember the rule of thumb was a three-credit level class would have about nine hours, the total amount of work, including class time for that class. And I try to drill this into my students heads as a pre-calc and calc teacher that you’re going to be doing a lot of work outside of this class, which is not what they’re used to. They’re used to, “I’m going to go home, I’m going to do 15 minutes worth of homework, and it’s going to be graded based on effort. So I get a free 100 in the gradebook if I did it, and nobody’s gonna care if I did it correctly, I could do the whole thing wrong, I could copy from my friend, and there we go, I get the credit.” And that’s just not college readiness, or career readiness, for hopefully most careers. But I would love to see some kind of, even in high schools and then have it trickle right up to college, some kind of bridging from each course… ninth grade, these are our expectations, 10th grade, you get a little bit more strict restrictions, and then all of that being an idea of we’re bridging the gap from 12th grade to college. And as well as college professors and faculty looking back and saying, “Okay, this is where they’re at when they’re coming to us. And so we can pick it up from here.”

Rebecca: Yeah, that ramping is so important. And just acknowledging where students are at, and meeting them where they’re at, and not having some false expectation of where we wish they were, which is very different. And I think what John was talking about, like providing some time allotments and things, but just even providing students with a sample of what their outside of class time should just generally be looking like, maybe you are spending three hours outside of class studying or six hours outside of class studying. But what are those six hours look like? What does studying look like for this class? What are the kinds of exercises that would be really helpful. And faculty have a vision of what that should be, we just often don’t communicate it.

John: And often when faculty do, though, they communicate what worked for them, which is what had worked for their professors before them, which is not always what would work for a typical student.

Rebecca: What do you mean? Highlighting, John?

John: …and repeated rereading, and focusing on learning styles, but there’s a lot of things out there that faculty may encourage students to do that is not really always consistent with evidence. And going back to your point earlier, Natalie, about providing the relevance of things, one of the problems I think that faculty have is we got into these things, because we’re just really interested in the questions of the discipline. And the things that interest us now are based on having studied the discipline for a long time. And it’s a little harder, often, to connect with the types of things that would interest a student who’s just coming out of high school or is in a sophomore or junior position in college, because they don’t have that same network of concepts to make the topics that we find interesting as interesting to them. That’s something I think we definitely need to work on.

Rebecca: I know one thing that I always struggle with in the design classes that I teach is I’m constantly trying to get students to think about audiences other than themselves. But then when you start to put things in perspective, you realize they don’t really know much beyond themselves. They know maybe what an audience younger than them might experience or have expectations of, but they have a really hard time envisioning what a professional world might be like, because you just don’t have any experience in it. So we often come to the table with so many assumptions that we think that they have, and they just don’t have the life experience to match it.

John: We’ve talked a little bit about how instruction was affected during the pandemic, how did your classes transition?

Natalie: I saw this, the pandemic, as a way to be able to spread my wings and try and implement some different teaching strategies that I had thought about, but I wasn’t totally sure if I wanted to dive right in in one given year. And one of those ideas was the idea of a flipped classroom. I had never felt confident to be able to create a flipped classroom that I felt students would be responsible for. That being, watching the videos at home and then us just working on problems together in class. And to be honest, that’s pretty much what has happened. I have students who will have to watch videos on the two days, three days that they’re not here, and they have to, and I’ve figured out now how to assess that they’ve watched the videos, and to make sure that they are responsible for that learning. It’s not totally flipped. But I do have to provide instruction to these students three days of the week that they’re not around. And I did that in a couple of different ways for each one of my preps, because I didn’t want to get so solid in just one way was “the way,” and I wanted to try some things out. So for calculus, I teach calculus fully synchronous. My students who are at home Google meet live with the students that I have in class in front of me. We’re able to talk about problems as a bigger class instead of just the five who are in front of me. And that’s worked out quite well for us. Granted, class is at 10 o’clock, so it’s not too super early for these students. So, precalculus, I do kind of more HyFlex. They have the opportunity to come to the live session if they want to, or I do make videos for them to watch on their own. One thing that we had to keep in mind this year was that some of these students are responsible for younger siblings while parents are at work. So we needed to be a little bit more flexible in how we set up our classes. And when I had the conversation with my calculus students, they said, “Nope, we could totally do synchronous.” And if they needed me to make a video, I can make them a video. And then algebraI, I do completely asynchronous, the videos are already set up. There’s no live session. When they come in the next day, or whatever day follows watching the videos, I check and make sure that they watch their videos, give them a little bit of credit, just for doing what they were supposed to do. And, to be honest, I have become very, very graceful compared to how I ever was in the first seven and a half, eight years of teaching. My department very rarely ever accepts late work. Math is a sequential subject, you need to do the work today so that you know what’s going on tomorrow in class. This year, I’m just happy it’s getting done and getting done with integrity. So I’ve become so extremely graceful with students who are getting work done after due dates, right before the quarter ends. It’s not great, but they are also learning a very important lesson in “it’s a lot easier if I just do a little bit every day instead of trying to cram it all in at the end of the semester, or at the end of the quarter.” So, hopefully, that idea can trickle up to college as well. It’s not very easy to dig yourself out of a hole, especially in college.

John: You mentioned with your recordings that you were verifying that they actually watched the videos. How do you do that?

Natalie: With my algebra I, the videos obviously match their note packet, they have guided notes that matches very nicely. So the next day I just come in, check, they’re going to show me a front and back with all of the notes filled in. For precalculus, I feel like I shouldn’t have to check their notes, that they should be responsible enough for that level to actually watch the videos, I did get a sense that they weren’t watching the videos, mostly because I was bringing things up when they came to class, and I had a lot of blank stares looking at me. So I started using a program called Edpuzzle. They have to sign into it with their Google account. Lots of teachers in my school use this. And I was able to see who was actually watching. And it was funny, I had a student come to me and he goes, “I’m going to make a guess that 1/3 of the kids are actually watching the videos.” And as soon as I started counting up the number of students who actually watched the videos, it was about 1/3. So that gave a very stern talking to about what my expectations were in that they’re not going to have the best understanding and knowledge if they’re just practicing the homework assignments, that it needs to be the full package, coming to class in whatever shape or form that looks like, be it coming to the live session or watching the videos. But all of it needs to be done in order to be successful in precalculus.

John: Do you embed questions in the videos?

Natalie: I am just getting my feet wet with it. I just started using it. So no, I haven’t gotten that far. But I did hear that you can and you can do a little checkup and it will not let you get through unless you do the question. So is that something you guys have used?

John: I haven’t used EdPuzzle. But I have been using PlayPosit this year. And I embed questions in it and it is required as part of the grade in my econometrics class and in my introductory microeconomics class, and I’m probably incorporating it in more classes as I go forward. But students actually have responded really positively. They discovered that when they watch the videos, it’s really helpful, because when there was no grading involved or no questions, the level of use of the videos was dramatically lower. So this provides just a small incentive, it’s a trivial part of their grade, but it’s enough to induce them to watch them, though. I’ve been really pleased with it. One advantage of Edpuzzle though, is that It’s free, as far as I understand, whilel, I do have to pay for PlayPosit.

Natalie: Yep, it’s free. And we’re a Google school. So they are able to sign right in. And it was easy for me to use, I could see all the students, I could see how much of it they watched. So that was pretty cool.

Rebecca: That’s great. As I’m hearing you talk about how your teaching has transitioned, it largely matches how faculty at the college level have also had to transition. So in some ways, we may have been brought together unexpectedly. [LAUGHTER] So there’s a lot more college faculty also using a flipped classroom model, and using more class time to solve problems and work on the more complicated things. Because often the video lectures or that kind of part of the material is foundational. Some of it might be memorization things, it might be terms and things like this, and then you put it into action in the class, and you can have guidance and coaching. So what’s really interesting is that the pandemic may have brought these two experiences together, and that bridge might be more present than it has ever been before.

Natalie: Exactly, I used to think that there was no way for my students to learn, if I wasn’t the one who was telling them and showing them. If I wasn’t up at the front of the board, doing it with them, doing it for them, then they weren’t going to learn… there was no way they can learn this on their own, and boy, was I ever wrong. Even though I could do 10 problems with them up at the board, whereas they could do three working together in a small group or in a pair, it’s so much more valuable to listen to them have those conversations, and to hear them explain these things, in their own words, or what’s very powerful too, is when I hear my words come out of their mouth, as they’re explaining it to another student. So that’s totally cool to hear that. And I think that’s the direction of education. The prep work is more important than what the teacher role is in the classroom. So the role of the teacher plays planning and prepping for a lesson is so much more powerful than what the teacher is doing in the classroom at that time as far as leading instruction.

Rebecca: That design of experience is so powerful, just generally. I think one of the things that I know I have always struggled with using a flipped classroom model, but I’m definitely getting better, the more and more I do it, is really scaling back on how much can actually be done in class, when students are working through problems and they’re not having quite as much coaching. You’re really letting them struggle and fail and try again. That takes time. It’s not like something that can happen automatically. So we might be used to going through more examples, but less examples, but more depth can be really powerful. And your right, students explaining to other students is an amazing thing to see. But it also helps them articulate or realize where they don’t understand, which is something that you don’t necessarily recognize when someone’s explaining something to you. Because when someone explains it to you, of course it seems obvious, until you have to do it yourself.

Natalie: Absolutely.

John: One of the things I was thinking about is I have to create a video on log transformations in econometrics tonight, and that reminded me of all that.

Natalie: Do you want one of mine? I can grab it right off Edpuzzle. [LAUGHTER]

Rebecca: Can you imagine that, though, like sharing back and forth between high school and college and sharing resources? We should do more of this.

Natalie: Oh, absolutely. Why are we always reinventing the wheel for ourselves, when there’s so much already out there?

John: When you first create a flipped classroom model, it’s a lot of work, creating those videos, maybe embedding some questions in them. And doing that is a tremendous amount of work upfront. But it’s taking this stuff that’s relatively easy for students to learn and shifting it outside of the classroom. So that when students are in class, they’re able to focus on the things they have the most trouble with. Because, as Rebecca said, you can provide a really good lecture on how to do something, but if students haven’t wrestled with it, they’re not going to learn it as well. I know for many years, I was doing an awful lot of lecturing, and I’ve cut back to very, very little now other than when I’m recording lectures, and I’m seeing students struggle with materials, but then when they are assessed on it, there’s much more balance in their performance. Because before there were always some students who would pick up on everything you did… people like you, Natalie, and people like those who are faculty. We were able to learn the stuff by listening to people tell us how to do it and figure it out on our own. But that doesn’t work well for a lot of students. And when they’re there explaining it to each other, they learn it much more deeply than they ever would by trying to do the more difficult parts on their own. Because in the traditional model, we’d give them the basics and show them how to do things. But the only time when they really got to apply that was either when they were working alone, doing homework, trying to struggle to solve some problems. or on a high-stakes exam. And those are times when perhaps they need the most support and the flipped classroom model can really address that really nicely.

Natalie: Yep, I absolutely agree.

John: We always end with the question: “What’s next?” …which is a question I think everyone in education is wondering.

Natalie: I think what’s next is filling in the gaps. There is going to be a lot of gaps for a lot of years. And we keep saying I can’t wait till next year, I can’t wait till it’s back to normal. But pandemic aside, I don’t think it’s going to be normal for a long time, we’re going to have a lot of students who need a lot of support for a lot of years, and it’s going to affect me. And then it’s going to affect college faculty, as the students get up there, addressing where the gaps are, how can we fill them? How can we give these students the support that they need, even under just budgets that were being given that were cut in this past year or could be getting cut? We’re going to need a lot of support from each other, from our colleagues, from our administrators from the community… and I think a lot of understanding… these students are going to need understanding, we’re going to need understanding.

Rebecca: I really like the underscoring of this empathy towards one another, both between faculty and teachers, as well as between students and teachers. I think that’s a really nice note to end on. Thanks so much for sharing your insights, Natalie. I think many faculty will want to take the time and effort to reach out to some high school teachers and make some connections and start to figure out ways to bridge those gaps.

Natalie: Thank you so much for having me, this has been such a valuable experience. And I’m always grateful when anybody wants to listen to my opinion and share conversation with me.

John: Well, it’s great talking to you again, I very much enjoyed working with you when you were a student here, and it’s nice to see just how successful you’ve been. And I suppose I should mention that this came about because one of my current students had listened to the podcast, had shared it with people on Facebook, and then she said, “And you know, you really should invite one of my former teachers,” and so we invited you.

Natalie: That is so awesome. I feel very honored that that student thought that I had something great to share. And it’s been so great to be back in SUNY Oswego, among other Lakers, and of course, a professor who was a great mentor and led me down a path of success.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

168. Synchronous Online Learning

The pandemic forced many faculty to experiment in different modalities in 2020. In this episode, we reflect on our own teaching experiences with synchronous online courses this year.

Show Notes

Transcript

John: As we approach the end of a really challenging year, we’d like to thank all of our guests who provided so much help and support to us and all of our listeners, and we’d like to thank you, our listeners, for hanging in there with us. We’ve all learned a lot in 2020 and we’re looking forward to a chance to apply what we’ve learned in circumstances in which there are fewer external threats.

…and now we return to our regularly scheduled podcast.

The pandemic forced many faculty to experiment in different modalities in 2020. In this episode, we reflect on our own teaching experiences with synchronous online courses this year.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Over the past few months, we’ve talked a lot about the pandemic and ways to adjust our teaching. And we’ve talked a lot about online learning, but we haven’t really focused on synchronous learning. John and I both taught synchronously this semester. So we decided that in this episode, we would focus a little bit more on synchronous learning and what we’ve learned about it in our own experiments in our classes.

John: Our teas today are:

Rebecca: I have Scottish breakfast once again.

John: …and I have a blend of spearmint and peppermint tea.

Rebecca: That sounds much healthier than my choice.

John: It’s not my first tea of the day.

Rebecca: This is not mine, either.

John: This is my first herbal tea of the day.

Rebecca: This is my second pot of the day. [LAUGHTER]

John: So, Rebecca, what classes were you teaching this fall?

Rebecca: I was teaching two design classes that are smaller. So I had one web design course that was stacked. So, it had beginning, intermediate and advanced students in it, 25 students, and we met synchronously, but also had asynchronous classes, it’s considered a studio course. So for a three credit course, we have six hours of class time, and three hours of outside work, which is a different balance than maybe other folks. And then the other class I was teaching was a special topics design course, which was smaller, it was about 10 students. And that class was also synchronous, but it was a project-based class, and we worked on two community design projects: one for a project called Vote Oswego, which was a get-out-the-vote initiative on campus, and the second is a project called “Recollection,” which is a storytelling project with adult care facilities.

John: And we do have an earlier podcast on an earlier iteration of Vote Oswego. So we’ll include a link to that in the show notes.

Rebecca: So John, what kind of classes are you teaching? We obviously don’t teach the same thing.

John: I was teaching two classes this fall. One was a large synchronous session with 288 students. And the other was a fully asynchronous section with 60 students this semester,

Rebecca: At what level were the students in both of your classes?

John: These were both introductory economics classes. So most students in the class were freshmen, and it was their first economics course.

Rebecca: So, your classes are much larger than mine, you’re teaching much more younger students or newer students, and my classes are smaller, project-based, and usually junior or senior students.

John: Yes. And there’s certainly some differences in the disciplines as well.

Rebecca: No, that they’re the same. [LAUGHTER]

John: Why did you choose a synchronous mode of delivery rather than an asynchronous mode, or a face-to-face option this fall?

Rebecca: So I chose not to do face-to-face delivery for my own health reasons, I chose to not be on campus for my own safety, because I have a chronic illness. So I chose specifically to have strong synchronous components, because a lot of our students are used to working in a studio together and having a community around each other and kind of feed off of each other’s work and work collaboratively. And I wanted, because of the classes I was teaching, to continue to have collaboration as a key part of my class. And I was really concerned that if I didn’t have a strong synchronous component, my students wouldn’t be able to effectively collaborate with each other, because there would be too much scheduling issues and what have you. So it’s a little bit of a carryover from the way that I would run my classes in person in that I give a lot of class time to project-based learning and team-based work and do a lot of lectures and things like that asynchronously in like a flipped classroom style. How about you, John?

John: Basically, I tried to preserve something as close as possible to what was originally scheduled or what was originally planned. And my large class is typically about 400 to 420 students, and I just couldn’t imagine taking that class and doing it in a completely asynchronous manner, because when I teach a class asynchronously, I give students lots of individual feedback, and it would be really challenging providing individual feedback to several hundred students. I just didn’t really have the time to do it in that sort of mode. So I thought it was better to work in a mode where I could give students feedback in a group setting using some interactive tools, where they’re all getting feedback at once. It was the only way I could see handling a group that large. If I was trying to do it as an online class, it would be effectively more in the form of a MOOC with very little interaction, either among the students or between me and the students.

Rebecca: One of the things that we both talked about before we started recording was how we both used a flipped classroom model to help with our synchronous session. So can you talk a little bit about how you did that and what students were doing outside of class.

John: This is actually, in many ways, similar to what I had done in a face-to-face class. Before each class, students would have some readings to work through. And I use the Lumen Learning Waymaker package, which is basically taking materials from a textbook, combining it with interactive multimedia, where they got to shift demand and supply curves and other curves around and see how they responded when they change parameters. And they read a bit in that online text and then they would work through some problems on it where they were allowed multiple attempts at those problems. I also created some videos with embedded questions that were at a somewhat higher level than the textbook readings, which was a little bit more challenging. And they were given unlimited attempts to work through those videos with the questions. And they also, outside of class, participated in discussion boards, where I asked them to relate what they were learning to things in the world around them, in their own lives and their own experiences.

Rebecca: …nice little inclusive teaching practice right there, right? …connecting students with their experience and making it relevant to them.

John: Right, because we know that students learn things most effectively when it has some salience, when they see the relevance to their life.

Rebecca: And the waymaker package, if I remember correctly, had some quizzing and stuff associated with that, and unlimited attempts, a version of retrieval practice there.

John: It’s a mix of things with unlimited attempts and limited attempts. So, the microeconomics Waymaker package is designed, and all of their Waymaker packages, for that matter, are designed, is that they start with a list of broad learning objectives. And they break it down at each module level to sub objectives. And they break those down into sub modules. So, in what would have been the equivalent of a chapter of a textbook, there’s usually two to four sub modules on particular aspects of that. And students work through that. And embedded in it, they have some review questions, some practice questions. And those they can take an unlimited number of times at any point in the course. Once they complete the module, they have a module quiz where they are limited to only two attempts at it. But they’re getting feedback on what they did well. And what they didn’t do well. It’s automatically color coded to indicate whether they mastered the material in one of the blocks of content in there. And then, if they take the module quiz, it will give them feedback on what areas they did well, and what areas they need to work at more. And they’re being directed back to the areas that they need to review. And there, they do have unlimited practice opportunities. And the other thing I did is I created my own videos that focused primarily on the topics that students generally find the most challenging. And in economics, that’s generally with either applications involving math or involving graphs. It was one module a week, and I would take the topics that I know, from past experience, they were likely to have the most problems with, I’d create my own videos with that. And I was using PlayPosit, which allows you to embed questions in there. Most of those videos I created were between five and 12 minutes in length. They would watch the videos and answer questions as they were going. And if they got one of the questions wrong, they could go back and replay that portion of the video and then try it again. And they were given unlimited opportunities for that.

Rebecca: I think you mentioned students really loved those opportunities.

John: At the end of the class, I gave them a Jamboard, which I know is something you’ve used more regularly, asking them what worked well. And there was very much universal agreement on the PlayPosit, as well as on the Waymaker aspects of the course. They really liked the fact that there were practice activities embedded right in their textbook, and that they could go back and try things over and over again until they mastered it. And it was giving them feedback on whether they had, in fact, attained mastery at every step. And it was a nice visual indication of what they’ve learned and what they still needed to work on more.

Rebecca: Excellent.

John: What did you do in your asynchronous components of your class?

Rebecca: Well, the balance of my classes, as I mentioned before, is a little wonky in that we’re supposed to spend more time in class and less time out. So asynchronously, I did a lot of independent stuff that students were not necessarily doing collaboratively. So this is where I had lecture videos that are recorded that were about the topics that they were going to be working on or introduce the component of the project that they were going to be doing. And then they also were completing things like LinkedIn Learning tutorials. And we also have access to D-Q University, which is a set of tutorials for accessibility, and teaches accessibility. So I took advantage of that package as well. And largely they were completing those kinds of tutorials, both of those have exercise files and that kind of thing that they can follow along with. They get little certificates. When they’re done completing there’s little quiz questions and stuff. So they were doing a lot of that kind of work asynchronously. They were also using Slack to communicate with their teams for independent things that they were working on that they needed to communicate out to teams when they were working on projects together. And I also use Slack as a place to have discussion. So like you, I had discussion questions that tried to make what we were talking about relevant. We were exploring design, specifically like web design and how they interacted as a consumer versus how they would interact as a maker and did a lot of observational studies. We also did some discussion boards that were really about design activities and things that got students off the computer. So they were just documenting what they did off-screen, offline. So things like listening to a podcast so that they didn’t have to be staring at a screen and what their takeaways were. They attended virtual conferences, which I guess was still on-screen, and did some sketching, like paper prototyping and some other methods that we like to encourage our students to do, just to kind of help balance the screen time a little bit for students. So that’s largely what they were doing asynchronously.

John: Were you having them submit some copies of that work in some way, or were they just reflecting on the work that they had done?

Rebecca: The little non screen activities were documented in a discussion, essentially, that we were holding on Slack, and then tutorials and things, they were just submitting their completed certificates. And so I broke down those LinkedIn Learning courses and things over multiple weeks. So they didn’t really submit those certificates until they were completed. But they were doing a little bit by little bit, but if they didn’t do the tutorials, they wouldn’t be able to do the projects or the actual work that we were doing of the class. So it was pretty important that they were doing those components outside of class.

John: Once your students were in class, what did you have them do in a typical class session?

Rebecca: The two classes I was teaching I handled a bit differently because of just the sheer volume of students in the bigger class, which was 25 students that were working on projects. They were working on collaborative projects, in teams of three, for the most part. And so what we would often do is show-and-tell’s or critiques in small groups. So let’s say there was two or three teams together that we would do a little critique with in a breakout room, while other teams were meeting and collaborating. We would also do things like come together to answer questions about things that they were working on, troubleshoot or whatever, and then go work on projects in breakouts for a bit. And then we’ll come back at a scheduled time. I also did one-on-one meetings with students during class time. So I’d set up things like a quiet work breakout room or the chatty breakout room. And students would pick the place that they wanted to go while they were working on projects. And then I would meet with them individually for critique, and often a lot of code troubleshooting is a lot of what I spend synchronous time doing. And students sometimes met with my TA to do the same thing, and with our small groups. I also did a lot of design challenges. And students really liked those and would like to do more to hold them accountable for the kind of material that they were learning outside of class or being introduced to outside of class in a low-stakes environment to test it out with some peers and troubleshoot. So I would pose a little design problem. And then they’d work in a small group to work on that problem in a very tight amount of time. They might spend 30 minutes… my classes are three hours long… or an hour, and then we’d come back and show them off or talk about different things. And I tried to make those design challenges fun and entertaining. So one of the first things we did, which worked really well to start gelling their teams that they were with the whole semester was designing an emoji for Slack that they used for their team. And they loved that assignment. It was partly about working at a small size, and so it was tied to some of the curriculum that we were doing, but it was fun. So they did that in a small team and then had to implement it. Later on in the semester, we did things like a 404 error page for their projects, which were just kind of entertaining. We tried to make them amusing, so that if you landed on a page, it was a good user experience. So things that maybe wouldn’t typically work on in one of my classes that were a little bit more fun, but really were emphasizing the technical and conceptual things that we were working on. The other thing that we use synchronous time for is I took advantage of our virtual platform, and I brought in alumni multiple times, and local designers multiple times and did little Q and A’s with them. Not every week, but every few weeks, or every couple weeks, I would bring in a designer for a 30-minute session. They’d introduce their work. And then students did a Q&A with them, which students really loved. And it broke up our time a bit and really gave them something special that maybe we didn’t always do in a face-to-face class that made the synchronous environment kind of special.

John: Excellent. That is a nice opportunity provided by Zoom that actually could work in the classroom too. But I think many of us just hadn’t really considered it so much. It doesn’t really matter where you are when you’re teaching in this sort of synchronous environment. So it’s very easy to bring in guest speakers and it’s something we’ve probably should have been doing more of in the past, but I think many of us will be doing more in the future.

Rebecca: So John, how did you use your synchronous time?

John: I had told students before each class session, what specific topics we’d be working on. And then most of the class time was spent asking him a series of problems of progressively higher levels of challenge. I basically adopted Eric Mazur’s clicker strategy of trying to find challenging questions where roughly half the class will get it wrong the first time and then letting them meet (in this case, I had the meet in breakout rooms), discussing it and coming back and voting again on it. And generally, you’d see a fairly significant increase in the performance after they’ve had that chance to engage in peer discussion. And that’s where a lot of the learning seems to happen when clickers are being used. I used iClicker. The only difference is students could not use a physical radio frequency clicker because they have a range of a couple 100 meters and students were spread out all over the world, I had one student in Egypt, I had students in South America and students spread throughout the country this time. So they needed to use either their laptop or a mobile device in order to do that. We discussed it as a whole class after they come back from the breakout rooms. And then I’d asked them to explain their choices. I generally have them use chat, and then I’d go through and correct any misperceptions they’d have. And I try to guide them to the correct answer by asking them questions, and letting them see for themselves why some of the answers were right, and some of them were wrong. And generally, that’s how we spent many of our classes. Initially, I was also using Kahoot! from time to time. They enjoyed Kahoot!, but I noticed a bit of a drop off when we were doing the Kahoot! sessions, because those were not graded. And with the clicker questions, they were being graded, and that tended to receive a somewhat higher level of interest. It was very low stakes, they got a certain number of points for an incorrect answer on either attempt, and they got a bit more points when they answered the question correctly. And initially, I was giving him three points for an incorrect answer, and five for a correct one. And they asked it perhaps that could be bumped up, because some of the questions were so challenging. And I did raise it. So they ended up getting four points for any answer, and five points for a correct answer. So it is extremely low stakes. So I tried to do a lot of retrieval practice in the class, where it started from essentially no stakes with the embedded questions in the reading, then it ramped up to in class applications of this, where they still get 80%, even if they got it wrong, but they had another chance to get it correct. And then they took that module quiz, and even there, they had two attempts at it. So if they made mistakes, they had lots of resources they could go back to and work on it. So I tried to set it up and provide them with many pathways to attain mastery of the content, and to encourage a growth mindset and to encourage them to recognize that people make mistakes when they’re learning and that there’s a lot of benefit from having those mistakes as part of your learning process. There’s a lot of research that shows that we learn things much more deepl if we get them wrong, when we first try it, we’re much more likely to remember it later on, then if we happen to get it correct, initially. In that case, we’re much more likely to forget it a bit later. And that was a bit of a challenge for students. But I think they eventually appreciated the fact that everything was fairly low stakes.

Rebecca: I think I’m seeing some themes in the things that, although we’re teaching very different classes in very different contexts, there’s some real big themes about how we’re using our synchronous time, and even how we’re using our asynchronous time. And so there’s an emphasis on peer interaction and establishing those peer networks, really enforcing or reinforcing things and dealing with muddy points. And then also just providing the encouragement and support like that low-stakes environment or trying to foster a growth mindset. So in my classes, I did the same thing. I was doing peer group work and trying to really get them to collaborate and troubleshoot together and they love that that… that was really valuable. I spent time doing live demos and troubleshooting, when there was a really troublesome technical component or something that they were trying to do that a lot of them were having trouble with, that they could ask me live questions. So that same muddy point kind of thing that you were getting to in what you were discussing. And then, finally, the growth mindset that you started bringing up, I’d also tried to do and, although I didn’t have a lot of low-stakes testing, or something like that, I set my projects up so they were done in sprints. So a long full-semester project was broken into multiple two-week sprints, where they would work on something, get feedback, and then could revisit whatever they did, and then add a new component to it. And so I did that throughout the whole semester. So there was a bit of retrieval practice, a bit of spaced practice in there, and certainly some fostering a growth mindset and the idea that you make mistakes and that’s how you learn. And I spent a lot of time… I don’t know if you experienced this too, John… but I experienced a lot of time in synchronous and saying like, “You can do this. It’ll be okay. And this is how the learning experience works.”

John: And I did have to do a lot of that, especially in the first few weeks of the semester, because they were not used to a flipped class environment. And they were not used to this notion of making mistakes and learning from mistakes as part of your learning process. Because most of them have come up through their elementary and secondary school system thinking that they need to memorize some things and reproduce it on exams. And they do well if they get high scores, and they don’t make mistakes. And that’s just not how we learn in general. And it was important, I think, to help remind them of that. Another aspect of the flipped class environment that we’re both using is that we let students learn some of the basic skills, the easy things that they can learn pretty easily on their own, from other resources. And we’re trying to focus our class time using essentially a just-in-time teaching approach where you focus on the things that students always have trouble. In a traditional classroom environment, what normally happens is students will learn the easy stuff in class where faculty will lecture them on basic definitions and basic concepts. And then it all makes a lot of sense until students try to apply it. And they try to apply it typically in assignments outside of class, or in high-stakes exams. And it’s much more productive if the students use the time outside of class to master those basic concepts. And then we hold them accountable for having done that somehow in class. And then we give them assistance on the things that they find challenging when they need it. Not after they’ve had that experience of a more high-stakes assessment in some way.

Rebecca: Yeah, I think what I found or that students really shared with me that was something that they really appreciated was that there was a lot of structured time to work on those difficult problems in class. This is true of my face-to-face classes too, but even maybe more so in this online environment where students were having a really hard time managing their time. I would allow time to work on a project during class… it was scheduled, but then there was a check in point later on in the day. You wouldn’t want to spend three hours staring at a screen on Zoom, like this makes no sense. So I certainly did not do that. And I don’t want anyone to think that I did that. But, I would do things like “Okay, we’re going to check in at 9:30. And then we’re going to do a little activity together. And then you’re going to have some work time to work on X. And then we’re going to come back at 11. And you’re going to show me what you did. And then we’re going to have a little discussion or do another little activity, and then we’re gonna come back again at 12.” And we would have a schedule where there was time to kind of come back. What I found is, over time, students often wouldn’t actually get off of Zoom. They would just turn their cameras off and their microphones off. And I would do the same if it was like a work time. And then when we all came back on, a lot of students would turn the media back on. That said, I, of course did not require that depending on where students were, I certainly had students that were in environments where they couldn’t turn their cameras on, or had really poor internet connections, we adjusted as necessary there, and we had a way to communicate in a much more low-tech fashion using Slack during class time. So if something happened with someone’s internet connection, or whatever, they could still stay connected with us and what we’re doing.

John: How did you assess student learning in your class?

Rebecca: My classes are all project based. So the majority of grades are built on projects, not entirely, we had discussion boards, and I had some collaboration things that they were doing, and they were evaluated on those things as well. But projects were the significant piece of the puzzle. And the way that I graded them was really just providing feedback about the kinds of things I was going to ultimately grade very regularly throughout the semester. So every couple of weeks, they were getting feedback on their code for my web class, for example, feedback on their design, feedback on their writing, not a specific grade, necessarily, but feedback on all of those elements that were going to go into the final project. And then the ability to revise all of those again and again and again and continue to get feedback on those.

John: Did you have your students engaged in any reflective tasks?

Rebecca: Yeah, that’s a really great question, John. I had reflection built in two ways. So at the end of each sprint, or kind of module in my class, they were working on two projects at the same time throughout the semester, so they’d work kind of two weeks on one project, two weeks on the next project and cycle back… that was so that I had time to give them feedback regularly. So that was part of my structure. But at the end of one of those modules, I had a reflection activity that I implemented using a Google form. So a few different prompts to think about what they got out of that sprint, goals for their next sprint, that kind of thing. And then I also had some big group reflections at different moments during the semester, I had one at the beginning, and a couple in the middle and one at the end. And I use Jamboard for that, which is a Google suite tool that has sticky notes, and is the same kind of way that you might brainstorm. So I use it as a way to collect reflections in sticky note form, essentially, virtually. And I would have a reflection question for folks to respond to or a couple of different boards with different kinds of questions. In the beginning, we did something called “hopes and fears,” which is something I’ve talked about before… setting up the class like, what are they hopeful that they’re going to get out of a collaborative project? What are they scared about? We find out that like, all the teams have the same hopes and fears. During the middle of the semester, what are some of the big takeaways that you’ve had? What are some things that you want to work on? What are some things that you’d like to see changed about the class and various themes bubble up on that. And then at the end of the semester, I asked questions like, “What was your biggest takeaway? What was the thing you were surprised that you learned? What is one recommendation of something you would change in this semester?” and “What is something that you want to continue learning?” and I got really useful feedback on what to change about the class but also, some really great themes bubbled up across the class, which really results in like kind of three or four things for each of those questions, which was a nice way to wrap up the end of the class and summarize for students after they completed that task. And one thing that I like about the Jamboard is that it actually ends up being anonymous. You can see people while they’re working on it, but it doesn’t keep a name with a sticky, ultimately.

John: So you can see who’s active in the board, but you don’t see who is writing which note.

Rebecca: Yeah, exactly. So that worked really well for me. How about you, John, were you able to build in reflection? I know you have such a big class. So it can be tricky.

John: I wasn’t able to do as much of that with my large section. But I did have them do that, to some extent in their discussions. For one discussion forum in both classes, I had them use a tool called Packback, which uses artificial intelligence to give students some feedback as they’re writing their prompts. And each week, students had to post a question related to that week’s material and they had to respond to at least a couple other people. But one of the nice things about Packback is it will check the cognitive level of the posts, it will give them some feedback in terms of grammar, it will also do a little bit of checking to see if the material has previously been posted. And it gives students some feedback, encouraging them to say more than “I agree.” And it also encourages them to document sources and to provide resources or references for the arguments they provide. And they get a score on that. So it takes a lot of the evaluation of that away. And so I monitored all of that. But it was something that seemed to function pretty well, just by the interaction between the users and that system. I haven’t really mentioned much about what I did in my online class. My online class uses many of the tools, but obviously, I couldn’t do synchronous, because that class is fully asynchronous. I couldn’t do the same type of instruction. But I had students do two other things in that class that provided opportunities for reflection, one of which was I had them work in a metacognitive cafe, low-stakes discussion forum, where they reflected on what they were learning and the learning process. And that gave them another way of making connections to their learning and reflecting on how well they were learning materials and what barriers they were facing, and also sharing effective learning strategies with each other. They were given some readings each week, generally on research-based learning practices as a primer for many of those discussions… others, they were just reflecting on what they’ve learned and how it might be useful in their life to tie it back to themselves. But the other thing I had students do is work on two podcast projects in that class. And in those they were taking what they were learning and making reflections about how that connected to the world around them. Many of them ended up being related to COVID and pandemics, but they were making some really good connections, and they were getting a chance to see how the material they were learning had some relevance in their own lives. And a lot of that came out in some of the things they were discussing in their podcasts. And they also did use Jamboard once at the very end of the term. But I also use Google forms a few times to have them reflect on the process of what was working, what wasn’t working in the class and what was working and what was not working in their own learning processes and what I can do and what they could do to help them learn more effectively.

Rebecca: I don’t know about you, but I was really surprised at how well synchronous learning actually went for me. I had some technical difficulties early on with my internet connection. It took me a while, but I got around to fixing that problem by hardwiring my internet and resolving some of those things. But I felt just as connected to my students as I would normally. I had a lot of interactions. And in some ways, I was able to facilitate those interactions a little more equitably online, because it wasn’t just the person who came to nudge me and stand in line and be the next person. Instead, I could really coordinate using waiting rooms and breakout rooms and really give everybody a chance to have one-on-one interactions with me, which I really appreciated. And I really did get to know all of my students quite well, which I was a little bit surprised about. And then, in an area where it’s really technicalaAnd we’re doing a lot of coding and things on screen, being able to share screens and take control of another person’s computer to fix things or show them how to do something was incredibly valuable. We use some of those kinds of tools in person. But it actually was, I think, in some cases more effective using this particular tool. So I was kind of surprised at how well some things worked. And I think that even when things go back to face to face, there’s definitely some components here that I would keep.

John: I’d agree. And I think students were amazed at how well some of those tools work. When in breakout rooms, they would be using the whiteboard features, they would be sharing screens, they’d be making the case, they’d be drawing on the screens, and that was something that would be much harder to do in a face-to-face environment. Initially, at the beginning of the class, I had some issues with chat being kind of flooded with irrelevant material, and I had to clamp down on that a little bit. But within a couple of weeks, they started actually using it very productively, and it provided a voice for all students, even those quiet students who would have otherwise sat in the back of this large lecture hall. They were able to type something in chat, after thinking about what they wanted to say before doing it, without being concerned about interrupting the discussion that was going on. And I think that was really helpful. And when I taught large classes with three to 400 students, there’s almost always 3 to 10 Students who have trouble not having side conversations when there’s other activities going on. And that mute option is kind of a nice feature and the ability to set their microphones so they’re all muted unless they choose to unmute… to have the default being muted until people click the unmute option… made it really easy. And I was amazed at how quickly they adjusted to muting and unmuting. By the end of the term there was maybe only once or twice a class where a family member or someone else would walk into the room and start talking. And then they’d remember, they had to mute their mics, and it was very rare. In a class that large, I was impressed by it… and working with students one on one, during office hours, it was so much easier to have students just share the screen and show you exactly what their problems were then to correspond with them with email, or even have them boot up their computer or you try to find what they were talking about when they came to your office. It was just much more efficient.

Rebecca: Yeah, I could actually see it. You can Zoom in, you can see what they’re talking about. I also found, and I was really floored, in this last week of classes, students were doing their final presentations,at how well they did develop facility with these tools. They’ve developed a lot of fluency in the kinds of tools that are actually very relevant to my particular discipline. It’s relevant to many disciplines. But designers use these tools all the time when they’re working with clients. And so it was amazing to me that we got through 15 presentations so efficiently. We didn’t wait for anybody to share their screen. They just knew what they needed to prepare, had it ready, they started developing slide decks really effectively, and could just do the things that they needed to do really efficiently. One of the last things I said to my class was like “I’m so proud of you just being able to do that. We didn’t have to wait for anybody today. That was amazing.” And so maybe a little bit of a blessing in disguise, you hate saying like, “Oh, the plague is such a great thing.” But they really did develop some useful skills and tools and they became more effective communicators. That was something that a lot of students reflected on and things that they didn’t expect to learn is how much better they became collaborators and just communicators generally… not just in person, like through Zoom or in text… like through chat in Slack.

John: Video conferencing is likely to be a part of their lives in the foreseeable future, especially now that everyone has adapted tp this mode, it’s very useful for them to learn how to use that efficiently. The one thing I do miss though, is seeing their faces in person and recognizing them. One concern that I have is, I’m hoping to be back on campus in the fall, there may be students that I work with who interact with me regularly, whose voice I would recognize or whose name I would recognize on the screen, but whose face I just wouldn’t recognize because a very large proportion of students just didn’t feel comfortable having their cameras on regularly, and I understand that. We’ve got a lot of students living in crowded living quarters or working with very poor network connections. But I do miss actually physically seeing them. And I had my last class session earlier today. And I encouraged them to stop by in the fall and just say hello.

Rebecca: Yeah, I mean, I agree that the physicality is certainly something that’s missing. But it was amazing to me how connected I still felt to all of the students at the end of the semester. And I think that they felt connected to each other too and they verbalized that, and also wrote that in their Jamboard reflection. So although there’s much to be improved, given this was the first time out and an experiment in many ways. I’m really thankful that I read Flower Darby’s book about Small Teaching Online because that actually informed a lot of my practices, even though it was synchronous, and a lot of her material was about asynchronous learning. It really did help me remind myself of things that I already knew that I needed to do, but to kind of make a checklist of things that I definitely needed to do as I was rethinking my classes for the fall. So thanks for chatting with me, John.

John: We always end with the question: “What’s next?”

Rebecca: I am sitting down to reflect on what worked and what didn’t work, to try to troubleshoot some things for the spring. And I’m teaching a class that’s brand new to our curriculum for the first time in the spring. And so we’re developing it for online synchronous, although ultimately, it’ll probably be a face-to-face class. We’ve had to re-conceptualize some of the things that we were going to do because of the technology limitations that students may have. If they’re online, we’re expecting that we might have a lot of students who are relying on their phones versus software and having access to high-end software packages or computers that can run them. So we’ve had to rethink things. But I’m pretty excited about being able to experiment with my students with all kinds of technology in the spring, but it’s definitely a puzzle that I’m currently starting to work on. How about you, John?

John: Well, I still have a lot of grading to do. But once that is done, one of the things I’m going to be doing is converting a textbook I had written in econometrics to a Pressbooks site, which will be a lot of conversion because it’s originally in LaTex, a typesetting language used for mathematical typing and I’m planning to create a lot of videos, I’m hoping to get many of them done over the break so that I’m not spending 15 or 20 hours a week creating videos as I was all fall. And I’m hoping to get a little bit further ahead of the semester this time, so I’m not doing as much preparation at the last moment. And we’re both going to be working on putting together a series of workshops in January for our faculty to help people prepare for whatever comes at them this spring

Rebecca: We’re just going to be really busy.[LAUGHTER]

John: I’ve never spent as many hours working on my classes as I have this semester.

Rebecca: I agree. There was a lot of startup costs converting to this modality, but I’m hoping a lot of that stuff I’ll be able to keep and reuse moving forward. Thanks, John. Always nice talking to you, John. We chat all the time. But it’s nice to sometimes hear about some of the thought process and things behind some of the decisions that you’ve made in your classes. So it was really nice to actually hear about how you did some of that stuff this semester. So thanks.

John: And I also appreciate hearing more about what you’ve been doing in your classes. We spend most of our time on podcasts talking to our guests and only mentioning little snippets of what we’ve been doing ourselves.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

110. Fostering a Growth Mindset

Some students with fixed mindsets enter our classes expecting to be unsuccessful while others believe that they have a natural talent in the discipline. In either case, these students often get discouraged when they experience challenging tasks. In this episode, Sarah Hanusch and John Myers join us to discuss how they have revised their classes and used metacognitive exercises to help students develop a growth mindset and to recognize the benefit of learning from mistakes. Sarah and John are both Assistant Professors in the Department of Mathematics at SUNY Oswego.

Show Notes

Transcript

John K.: Some students with fixed mindsets enter our classes expecting to be unsuccessful while others believe that they have a natural talent in the discipline. In either case, these students often get discouraged when they experience challenging tasks. In this episode, we examine how two faculty members have revised their classes and used metacognitive exercises to help students develop a growth mindset and to recognize the benefit of learning from mistakes.

[MUSIC]

John K.: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John K.: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Sarah Hanusch and John Myers. Sarah and John are both Assistant Professors in the Department of Mathematics at SUNY-Oswego. Welcome, John and welcome back, Sarah.

Sarah: Thank you.

John M.: Thank you.

John K.: Our teas today are?

Sarah: None today

John M.: Yeah, imaginary tea. No tea for me.

Rebecca: The imaginary tea…that’s what my daughter likes to drink. That kind.

John M.: Yeah, I’m in good company there&hellp;

Rebecca: I have English afternoon.

John K.: And I have a ginger tea.

Rebecca: We invited you here today to talk a little bit about how you’ve introduced a project on metacognition in some of your mathematics courses. Can you tell us a little bit about the project?

John M.: Sure, this began, I believe, in the spring of 2018 in a Calculus I course. And the idea was that, Calculus I is known across, basically the entire country…every school in the country…as being a very difficult course. So, you have a lot of students who are coming in, especially in the spring semester, who had bad experiences with calculus in the past. And in particular, I’ve been told by some colleagues that there’s going to be some students in there that more support than I suppose you would imagine. The situation was that on the very first day of class, I had students coming in who have had bad experiences with it in the past. And then at the same time, I have the students that are typically high performing. And they have difficult times also with perfection, you know, being obsessed with 4.0s and grades and that type of stuff. So the idea was that I wanted to simultaneously address failure with the students and perfection at the same time. And I was sort of led to think about this metacognition project, actually, funnily enough, on a flight back from San Diego. I was at what are called the joint meetings for mathematicians, and a lot of progressive newer teaching techniques are talked about at this conference. And I’m flying back from the conference on the airplane and I’m getting really introspective and I’m thinking like, I really need to do something to talk to my kids about failure and perfection. And then it occurred to me that there was this blog post that I had just read a couple weeks before by a mathematician by the name of Matt Boelkins at Grand Valley State University. And he had this idea for a metacognitive project that addressed all sorts of things like growth mindset, fixed mindset, productive failure, and all these different things. And I decided about a week before classes started that this is what I was going to do.

Rebecca: That’s when all the best ideas happen.

John M.: I know…right before class and on an airplane. I get really introspective when I’m on airplanes and staring out the window and thinking of all the big things in life and stuff.

Sarah: And essentially, John came to me and said, “I’m thinking about doing this project.” And I said “Well, that sounds cool. And let’s see if we can measure if it has any positive effect or not.” So, I sort of came in on the research side of it…of “let’s see if this is effective for changing attitudes towards mathematics.” And since then, I’ve stolen the project to use in my own classes. But, it really started as I came in sort of more on the research side of things

John M.: I think stolen might have been a strong word, but…

Sarah: I didn’t ask…I just took it. [LAUGHTER]

John K.: For the research project did you do pre- and post-tests on attitudes?

Sarah: We did a pre- and post-test, we use an assessment called MAPS which is the Mathematics Attitudes and Perceptions Survey. It’s a 31-item survey. It assesses, I think, it’s seven different dimensions. Some of them are growth mindset. Do they view mathematics as being answer focused or process focused? The categories were growth mindset, the applicability of mathematics to the real world, their confidence in mathematics, their interest in mathematics, their persistence in mathematics, their ability to make sense of mathematics, and do they view mathematics as being answer focused or process focused?

John K.: Sounds like a good instrument. Before we talk about the results, let’s talk a little bit more about how you implemented it. How was the project structured in terms of what activities did the students do during the class?

John M.: So the idea was that over the entire semester, they would have a selection of articles online to read, they would have a selection of YouTube videos to watch and it was essentially experts that are addressing these various topics. So, like for example, there is a clip by Carol Dweck, one of the originators of the theory of growth and fixed mindsets, and they were to watch these clips and read these articles across the semester. And then I think it was probably with two weeks or three weeks left in the semester, they’d have to write a reflective essay. It was an attempt to sort of shift the culture in the classroom towards viewing mistakes and failure as productive and as opportunities for learning. Because I think in wider culture, everybody believes that math is just about the right answer. And that if you can’t get the right answer, then there’s no worth in whatever effort it was that you put in to get to that point. And I wanted to provide sort of a counterpoint to that, so a counter narrative. Being honest about how many times per day mathematicians actually do fail, you know, that type of thing. So yeah, the main component was this essay that was reflecting on the stuff that they read and watched over the semester, and then there was sort of like daily conversations.

John K.: Were the conversations online or were they in class conversations?

John M.: In class…in office hours, just kind of whenever they popped up. I remember a couple conversations that happened after I gave back exams, for example, or rather right before I gave back exams. So for example, I would say, you know, I’m about to hand back exams. And I want you when you see the score, when you put the paper over and see your score, I want you to immediately think how are you going to frame this result in your mind. Are you going to look at that score and be happy with it and chalk it up to just your natural talents? Or are you going to say, “Oh, this is a result of hard work?” And then if you’re not happy with your score, are you going to put it away and never look at again, or are you going to engage with your mistakes and make them productive mistakes? It was sort of intervention through conversation that happened on an almost daily basis.

Rebecca: Did you notice a difference in the kinds of conversations you were having in class because they were doing these readings and watching these videos, maybe conversations you hadn’t experienced before in the classroom?

John M.: Yes. In particular, I had students come into office hours and they were relentless with trying to understand the material because they knew that they were going to have another shot to get it right. And I had never experienced that before. In fact, in one of my student’s essays, I had a student tell me that when she’s not done well on exams in the past, she would just take the exam and stuff it into her book bag and never look at it again. And she told me that just because of because of how I was structuring the course that she doesn’t do that anymore. She actually pulls it out and engages with the mistakes and the comments that I put on the exam and comes and talks to me about the exam and everything. So I did see a change in the students.

John K.: Was some of it based on the reflections or was it also partly based on a restructuring of a course to give students more opportunities to redo things or to try things again?

John M.: I believe the latter had something to do with it. Because the idea was that I could say these things out loud to them. But I wanted to actually build components into the course in addition to the essay that sort of reflect the themes that I’m trying to communicate to them.

John K.: Telling them that they can learn from mistakes, if you don’t give them the opportunity…

John M.: Right.

John K.: …to learn from mistakes might not be as productive. I think both components are really valuable. I just want to make sure we were clear on that, too.

John M.: I think that you risk sounding like a cliche motivational poster, if you don’t actually put some meat on the bones with it.

Rebecca: Can you talk about some ways that you actually built that into the course?

John M.: I did test corrections. I don’t remember exactly, I think it was get back half the credit they missed or something like that. So, the idea was that they had to engage with the mistakes on their exams and correct them. And it had to be perfect. So they had a week to turn in their test corrections, and then I would re-grade them. This was very time consuming, as you might imagine, but the students I believe, really responded to it. It really sort of hooked in with the theme that I was trying to send.

Sarah: And since then, we’ve both moved to more mastery based grading. John before I did, but a system where students keep trying things until they get it right. And that really helps sort of drive that “learn from your mistakes” message home.

John K.: Are you able to do some of that in an automated way? Or is this all involving more grading on your part?

Sarah: The way I’m doing it, unfortunately, it’s more grading on my part. Although I will say this semester I’m doing these mastery based quizzes, but I’m not collecting homework. So, it’s kind of a toss up in terms of how much…it isn’t really extra grading. I’m just grading more things in another category.

John M.: Right, I would not do test corrections again. Not only was it a lot of time to grade, but then I had issues with academic honesty. The mastery based thing I have found is, I believe, much more effective.

John K.: Another thing you may want to consider that we’ve talked about in a couple of past podcasts is having a two-stage exam, where in the first stage, they do it themselves. And then you have them break up into groups and do either all the questions or a subset of those as a group. So, you’ve got some peer instruction going on as well…and that way it’s done right in class and it can be done, if the exam is short enough or the class period is long enough you can do both of it. A common practice is to do two-thirds say individual and then one-third for the group activity, which has many of the same things. They don’t know what they’ve gotten wrong, but when they’re sharing with their peers, they’re talking it over and it means you only have to grade the group exams on the second stage, which makes it a whole lot easier than individual ones.

John M.: Right. Yeah, I have a friend I believe he has done that stuff like that. So yeah,

John K.: The Carl Wieman Science Education Institute, I believe, has a lot of information on that. I’ve been doing it the last couple of years, and it’s been working really well. Doug Mckee was a guest on an earlier podcast, we talked about that as well. Are there other things we want to talk about in terms of what you’ve done in the courses?

Sarah: One thing that we’ve both done since this initial project is we’ve taken some of the ideas of this project, but interspersed it more throughout the course. One thing I know at the time that John observed was that he felt like a lot of the students started the projects in the last week, right? And so what I’ve done instead of doing a big project of these topics is I’ve taken these articles and done the second week of class, you have to read one of them and respond on it. And then the fourth week, you have to do another one, and so on. So it’s a little bit of it throughout the whole course instead of all loaded at the end. I think it helps having some of those conversations with the students as well because they’re not just seeing the ideas in the conversations. They’re not just seeing the ideas in the paper. They’re kind of seeing both and it just helps intersperse it a little bit throughout the semester. I know I’ve done that a couple times now. I think you’ve done that since as well.

John M.: I did a pre-semester sort of essay and then I did a post-semester essay. But it was in response to the first time we did that, which is referred into the paper, and one of my students actually told me in their essay, he was like, ‘Hey, I wish I had this at the beginning of the semester.” So yeah, it’s definitely like a “duh” moment. Like, I probably should have done something earlier in the semester, instead of waiting all until the end. But, you learn as you do these things, so. But the essays that the students wrote… I provided them with prompts just to alleviate any sort of writer’s block that they may have. But, the students who basically ignored my prompts and told me their personal stories were the essays essentially that I still remember. I had students that were straight A students that were telling me exactly what I thought was going to happen: that they’ve been the smart person their entire life, and they kind of feel trapped by being a smart person. They don’t want to take any risks because if they risk something and fail, then that’s their identity as a smart person, right? They’re not smart anymore. I’ve had students from the other end of the grading spectrum who basically told me that the first day they walked into the class before I even said anything, they were already convinced that they were going to fail the class. I had students tell me about mental health problems. I had adult learners talking about balancing life and school issues. I mean, it’s just absolutely amazing what they told me, they opened up basically. That made a big impression on me.

John K.: Tying into an earlier podcast, Judie Littlejohn and I had introduced something really similar where we have weekly discussion forums. And I also noticed the same sort of thing, that I got to know the students much better because when they were talking about some of the barriers or the issues they face, they were sharing a lot of details about their life. And you get to know them better and they also seem to form a little bit more of a tighter classroom community because they also got to know each other a little bit more.

Rebecca: It is kind of interesting how when students are talking about their process or who they are as learners, is very different than talking about the subject matter. And it does get them to open up and may be engaged with faculty in a way that they wouldn’t otherwise.

John M.: And I have found being honest about my own failures in the past has been a catalyst for conversation, right? Because they view us as professors, they view us as the authority figures, the experts in that we never fail. And basically telling them how many times I fail on a daily basis in my own mathematical research. It goes a long way, I think… finding common ground with them. And acknowledging how difficult the subject material is. I mean, there’s a reason that calculus has a high failure rate because it’s a hard course, among other reasons. Yeah, just having the humility with the students and kind of stepping down off of the pedestal in front of them, I think that it helps.

Rebecca: So do you want to share some of the results that you got from your study?

Sarah: We saw some very significant quantitative results. I mentioned the MAPS instrument is what we use. It’s a 31-point scale. Its reliability and validity has been established pretty well, especially in calculus classes. One of the things that they did was they looked to see if the items were consistent with expert consensus…. So, with how mathematicians view it and all of the items were valid with the attitudes of mathematicians except some of the growth mindset scales. Research says that that’s an important scale as well. And on this 31-point scale, we saw an almost 4-point improvement from pre-test to post-test…of the students becoming more aligned with the expert opinions, which is a really significant amount…I mean, almost 10% improvement, which is even more remarkable, because when this assessment was first validated, they found that there was usually a negative result from taking a Calculus I class. So, the attitudes get worse pre-post in a calculus class and ours had statistically significant improvement. In addition, we saw statistically significant improvement among all of the sub scales. Now some of them were better than others. Some were just barely below .05 in terms of significance and others were much more significant. I mean, we really saw that over the course of this semester, they really did change their attitudes. We also had some evidence, as John’s already talked about, from their essays…where they said how they started to view mistakes as productive, and they started to feel like there was value in making mistakes and learning from them.

John K.: You mentioned alignment with an expert scale, can you explain that for our listeners?

Sarah: Essentially, what the original authors and it was Code et. al. that did this paper and develop this instrument. They gave this survey to students and they gave it to mathematicians and looked for alignment. Particularly they were looking for whether or not the mathematicians agreed on the items. And the idea was our goal is to get math students to have attitudes more like mathematicians, because that’s our goal, right? …is to develop future mathematicians. And so we would like those attitudes to get closer to how mathematicians view mathematics. They had high agreement among the mathematicians on every item, like I said, except one or two of the growth mindset questions. So, in other words, this survey reflects how mathematicians view mathematics. And that was how they determined the right answers on the survey, whether a particular item is something you should agree with or something you should disagree with. They went with the expert consensus.

John K.: So now, I may be misconstruing this, but are you suggesting that perhaps a lot of mathematicians had adopted a fixed mindset? So, there was a bit more variance there on that?

Sarah: I will say that was what the results of their validation showed.

John K.: Okay.

Sarah: And leave it at that. [LAUGHTER]

John K.: It does remind me of that study a few months ago, that found that when instructors had a growth mindset, the achievement gap narrowed and the drop-fail-withdrawal rate was much lower in courses, then for those instructors who had a fixed mindset. I think that maybe even more of an issue in the STEM fields than it is in humanities and social sciences, but I think it’s not uncommon everywhere.

Rebecca: I say it’s a common problem everywhere.

John M.: I’ll say it…mathematicians suffer from fixed mindsets. I’ll just say it, right? [LAUGHTER]

John K.: Many academics do.

Sarah: Yeah.

John M.: Yes, of course.

Sarah: I mean, the people who choose to become academics are often the people that were successful in school and they decide to continue with it. I mean, it is less likely that people who felt unsuccessful decide to keep going and to go into academia.

John K.: Selectivity bias there and that reinforces a belief in a fixed mindset, perhaps.

Sarah: Precisely.

Rebecca: What kind of response have you seen from students from…I mean, it sounds to me like this one study lead to good results, and then that changed many classes in that you’ve taught or the way that you’re teaching, how have students responded?

Sarah: Generally positively. I think doing the projects at the end of the semester wasn’t the best idea because they just feel so overwhelmed at the end of the semester with exams and projects and everything coming due. So, I did get some responses of “W hy do I have to do this now.” But generally, I think they appreciated learning about learning.

John M.: I think that given the opportunity to talk about their past experiences, I think they appreciated that. For the most part, I’ll agree with Sarah. I think that the message landed with an awful lot of students like I wanted it to. Some of my favorite essays were students who told me that they thought I was crazy on the first day. I mean, you go into a math class to learn math, you don’t go into a math class to study metacognition, or whatever it may be. I had one student the first time around, who basically told me it was all a load of crap, like why this is not working at all. And I had a student the last time that I did this, she was very skeptical towards the end even. Basically, aliken it to just some cheesy self-help stuff. I think that most students responded positively.

Rebecca: Have you seen the response impact other faculty in your area? For example, if they really liked having those techniques and things introduced in your class, have they asked other math faculty to do that in future classes or are you finding that its not many math students who were actually in that particular class?

Sarah: We haven’t done any tracking, so I don’t know where his students have gone. I mean, I’m sure some of them went on to Calc II…I’m sure some of them did not. Right. I mean, I guess most of them would have had Jess the following semester, right? Did she say anything?

John M.: No, she didn’t say anything. I’m teaching Calc III right now, and I have some of my former calculus students that were in this and they’re doing well.[LAUGHTER] Small sample size, but yeah, they’re doing well.

John K.: That could be an interesting follow up though to see how successful they were in the subsequent classes.

Sarah: Yeah.

Rebecca: Sometimes we’ve heard anecdotes, of departments and things when there’s been change that if students really respond well to whatever the techniques are, that they will demand it of other faculty members, and John’s talked about this before in economics.

John K.: Yeah, when you can show results…

Rebecca: Yeah.

John K.: …that there’s been some gain, and especially if it comes from students at the same time, it often puts pressure on other people in the department because if you’re able to show people that your technique has been successful and students are coming in and saying, “G ee, I wish you would consider doing this. I did this in my intro classes, and it was really helpful.” That sometimes helps make change much easier.

Sarah: Yeah, so one of the things that we did look at was we compared the final exam scores of John’s sections to the other sections of calculus that semester. Now, there was some other issues that clouded that data a little bit. His scores were a little bit lower than the other instructors. But what was really surprising, essentially, if you look at, I don’t remember if it were just the final exams or the semester grades. The DF rates were the same among the sections, but the withdrawal rates were significantly different. And that almost no one withdrew from John’s sections. I think there were two if I remember the data correctly, whereas there was like five or six on average from the other sections. And so the DFW rates were different, but the DF rates weren’t. So I just thought that was an unusual circumstance. So, it seems like the students were sticking with his class… and pushing through.

John K.: And if there is a larger portion of students staying with the class, then perhaps a slightly lower average grade is not necessarily a bad sign…

Sarah: Exactly.

John K.: …because student success is partly measured for persistence to completing the course.

Sarah: Exactly. I think because there were more students who stuck it through to the final exam, then his final exam scores ended up being a little bit lower. But again, if you looked at like overall course grades, they ended up being pretty consistent, other than the W rates. I wanted to make sure that there weren’t significant differences in the rates and I think it was just shy of being statistically significant. Like, if you had one more student that would’ve been significant. But just to make sure that, especially like adding the test corrections in wasn’t substantially making the class too easy, right? Because that’s often a critique that, you know, “Well you make these changes, but is that just making the class too easy and people who aren’t really prepared, are they passing?” And so I just did this analysis of the, like I said, it was really just a t-test analysis, but just to see whether or not it was significantly lower and it wasn’t significant. It was lower, right, just not significantly. And then like I said, I looked at retention rates just more as an explanation for why the average was lower.

John K.: In a lot of studies of interventions, the dependent variable is the drop-fail-withdrawal rates, because that’s a measure of success in completing the course. That by itself could be an interesting focus of a study. I’ve been running this metacognitive cafe in my online classes for a while and I did have a student in the class who wrote a few times about the metacognitive development that was introduced in one of your classes. They didn’t specify who but they said, we’re also doing some work on metacognition in the math class, and they said it was really useful and it was nice to see it in two classes.

Sarah: Yay!!

John M.: Good.

John K.: So there’s at least one positive data point there or one additional data point there. So are you going to continue this in the future? And if so, what might you do differently?

Sarah: Well, I think we’ve mentioned already that we’ve worked on including some of the ideas at the beginning of the semester and throughout the semester, rather than one project at the end. For the reason that it really benefits them most at the beginning of the semester when things are getting started. I think we’ve also both changed different things about our grading systems to incorporate more opportunities for growth.

John M.: The last time I did this, I introduced some articles that were a little bit more rigorous with the data and the science, because I sort of wanted to counter that kind of criticism that all this “Oh this is just a bunch of TED Talks…” that kind of thing. So, I really wanted the students to see some of the science behind it, the science of learning, because I really wanted to send that message that “No, this is not me just standing up here saying, ‘Oh, this is going to help you or anything, right?’ This is actually stuff that researchers have thought about before.”

John K.: I had a very similar response the first time I did this. I had a video I posted which was a TED talk by a cognitive scientist who talked about research that showed that learning styles were a myth. And some students had come to believe in the existence of learning styles because they’ve heard of them and often been tested, multiple times in multiple years, on their learning styles. Sometimes even through college and that’s rather troubling. The students said, “Well, this is just one researcher, I’m sure there’s lots of other studies. I don’t believe it because it’s not consistent with what I’ve always been told or what I’ve heard.” So I decided to modify it then and I added to that discussion, five or six research studies. In case you don’t believe this TED talk by someone who’s done a lot of research on this, here’s a number of studies, including some meta analyses of several hundred studies of this issue, and that has cut much of that discussion. They’re less likely to argue against it when it’s not just a talking head or not just a video when they can actually see a study even if they don’t understand all the aspects of it.

Sarah: Yeah. So I think that’s one thing we’ve tweaked what articles and what videos are we showing. I know the semester I gave my students a article that had just come out this September, that students perceive active learning as being less efficient, even when they’re learning more. In some physics classes at Harvard, they gave two weeks at each thing… two weeks of active and two weeks of lecture, and then they had them switch. And the students learned more with the active learning, but felt they learned less. And my students have been feeling frustrated because they feel like they’re not learning enough and that I’m not telling them what to do.

Rebecca: You’re not “teaching” them.

Sarah: I’m not teaching them. And we spend the class period, letting them vent. So all their feelings were out in the open. But, then I sort of countered with this article saying, “Look, I promise you really are learning things. You just don’t feel like you are. But you really, really are. And you’re actually learning it better than if I were using a different style.” So, that’s one way that we’re tweaking the articles because sometimes the research comes out that’s pertinent.

John K.: We refer to that Harvard study in a few past podcasts. We touched on it in a podcast that will release on October 9th. I haven’t shared it with my class yet, but I’ve been tempted to.

Rebecca: What was the discussion like talking about that particular article? Given that they were frustrated?

Sarah: I mostly was just trying to acknowledge that I understand their frustrations…and that, yes, the way I’m teaching this class can be frustrating. I agree. Sometimes I get frustrated about it. But I know that ultimately, they are learning things and that they are going to be stronger writers and stronger students of mathematics by using this structure. And so I kind of use it as evidence of I’m not changing.

Rebecca: So I hear you…

Sarah: Yeah.

Rebecca: …nut…

Sarah: I hear you, but…

John K.: I had this very conversation with my class today. They’re coming up for an exam very shortly. And I asked them, how did they review before an exam and the most common answer was they like to reread the material over and over again. And I mentioned some of the research on that. And I said, the best way to review is to work on problems with this. And I gave them several ways in which they could do that, that are built into the course structure. And I said, “But that doesn’t feel as effective. Why?” And one of the students said, “Well, I get things wrong.” And I said, “And when would you rather get things wrong, when you’re reviewing for an exam, or when you’re taking exams?” And I think some of them got that message. So I’m hoping we’ll see when they take the test next week.

John M.: Right? It seems like anytime you do anything that’s just not a standard straight lecture, there’s a certain amount of buy in that you need to get from the students. And sometimes that can be very difficult. There’s almost a salesmanship that you have to do throughout the semester to make sure that everybody’s on the same page and to kind of fight those feelings where the students give you a lot of pushback. Yeah, that’s the great fear is that when you innovate or you experiment that’s going to go horribly wrong. And sometimes it does, but, you know, we still keep going.

John K.: Because students are creatures of habit. They’ve learned certain things and they want to keep doing things the same way. And anything new can seem troubling, especially if they’re getting feedback along the way that says they need to work more on things…that’s not as pleasant as rereading things and having everything look familiar.

John M.: Right

Rebecca: Passively sitting in a lecture when things all seem like it makes perfect sense to you, because an expert is describing it who knows what they’re talking about, right? Always feels easier than trying to apply it yourself. And I think that students, even though the lecture might feel better, and learning is hard…over time…at the end, when they’ve seen how much they’ve accomplished, and you do have them reflect…many of them appreciate or come around. Sometimes, it’s not in that same semester, sometimes it’s emails, months or years later.

John K.: Yes.

John M.: Right. Right, right.

Sarah: If only if we could do course evals, you know, a whole year later,

John K.: Or five years later. That may not work too well in my tenure process, though.

Rebecca: We always wrap up asking what’s next?

Sarah: Well, the first thing is we’re hoping our article gets published. It’s been submitted. We’re waiting for reviewers. I’m going on maternity leave next semester…that’s really what’s next.

Rebecca: Sounds like a new adventure.

Sarah: It is a brand new adventure.

John M.: Wow, I don’t think that far ahead, I guess. Yeah, I guess I’m that unoriginal, huh. But, yeah, no I’m just trying to…

Sarah: We’re moving to a new building.

John M.: Yeah, moving to a new building, and getting a new department chair. Yeah, that’s right.

John K.: A new desk to go with the chair?

John M.: No. Ah… Yeah, funny, funny, funny.

Sarah: if only…

Rebecca: Well, thanks so much for joining us, this has been really interesting.

[MUSIC]

John K.: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John K.: Editing assistance provided by Brittany Jones and Kiara Montero.