20. New faculty transition

New faculty often come out of graduate programs that have trained them to be researchers but not teachers. The transition into full time teaching can be stressful and overwhelming for these colleagues. Maggie Schmuhl, a new faculty member in the Public Justice Department at SUNY-Oswego joins us to discuss how she has embraced evidence-based methods in her practice as a teacher.

Show Notes

  • Dweck, C. S. (2006). Mindset: The new psychology of success. Random House Incorporated.
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • LePore, Jill (2014). “The Surprising Origin Story of Wonder Woman.” The Smithsonian Magazine. October
  • 12. The Active Learning Initiative at Cornell.” the 1/17/2018 Tea for Teaching podcast discussion with Doug McKee in which two-stage exams were discussed.


Rebecca: Today, our guest is Dr. Maggie Schmuhl, a first-year faculty member in Public Justice at SUNY Oswego. Her research focuses on structural inequality, violence against women, and punishment. At SUNY Oswego, Maggie has taught American Criminal Courts and Judicial Process and Women in Crime. Welcome.

John: Welcome.

Maggie: Hello.

John: Today our teas are…

Maggie: I have a green tea that’s mint.

Rebecca: I’m also drinking green tea today, but mine is jasmine green tea.

John: I’m drinking a custom blend of peppermint, spearmint, and tarragon.

Rebecca: Ooh, yum.

Maggie: Fancy.


Rebecca: So, today we have a slightly different setup. We invited Maggie to come in and talk a little bit about the experience of a first-year faculty member and that transition from graduate school to teaching. Can you describe a little bit about what that transition’s been like? You’ve completed one full semester; you’re into your second semester. So, what’s that transition like? And what are some of the biggest hurdles?

Maggie: Yeah, the first year has been a lot of trial and error, a lot of learning curves and really just getting to know the university… getting to know the students… the department… and all of the intricacies of balancing research, balancing teaching, getting better at both of those things, and you know making time to explore a new place, a new city, it’s been good so far.

Rebecca: So in graduate school, was there a focus on teaching and developing curriculum, or was it more focused on research?

Maggie: Grad school was certainly more focused on research. It was about developing our research styles, our methodologies, our research interests. Teaching was not a major focus for a lot of reasons… but often teaching was a responsibility that we had, but not one that was explored as in-depth as our research. Since my first year in the program they have implemented third-year development seminars to talk about teaching, but for most of us, we had to really find our own way, we had to rely on upper cohort members to help guide us through our first time teaching, and we really had to spend our own time thinking about what kind of teachers we wanted to be, and how much effort we wanted to put into our teaching.

John: This is not uncommon in graduate programs. The faculty are focused on their research because they have to be if they want to keep their jobs. There are some programs that do more professional development, but they’re relatively rare… at least in my discipline.

Rebecca: I went to graduate school at Syracuse University and they actually had a development program for graduate students.

Maggie: Oh, that’s interesting.

Rebecca: So there was in the beginning, but then I also was in a fellowship program, where you actually put together teaching portfolios, and things, like you would if you were applying for teaching positions and things. That was part of the development and there was ongoing workshops.

Maggie: Yeah, so I think, for me, when I realized that I had a passion for teaching, I spent a lot of time seeking out professors that were engaged in wanting to make all of us into effective teachers… and so that drive, I think, that perhaps came from my desire to be a better teacher, helped me find better resources… within the program… within faculty… and I also served as the president of our doctoral association, and so when it came time to go on the job market, we sought out those faculty that were interested in helping us develop our teaching portfolios, and so we’d hold programs but a lot of it was student driven.

Rebecca: Did you find it challenging, when you were in graduate school. to balance that? When you have this interest and desire to explore teaching almost as a secondary research interest, right?

Maggie: Right.

Rebecca: How did that work? and what challenges did you face because of that? Because I’m sure you had colleagues in the same position who weren’t as interested in teaching and just didn’t spend that much time on it.

Maggie: Sure…. and I think the demands of grad school really keeps any particular person from excelling at teaching… and spending the time that it takes to implement and learn about effective practices for most of us. We’re trying to finish our dissertations. We’re trying to publish on research, and while all of that’s important, there’s kind of a piece of the puzzle that gets neglected, and often it was teaching.

John: …and so, you’re now at a four-year school and you’ve been really active in some of our workshops. We had a reading group last semester that focused on Small Teaching and you attended that workshop regularly… and read through that… and then you implemented some things. How did that go?

Maggie: Yes, I really enjoyed those Small Teaching reading group that we did,0 mostly because it gave me the time and in place to really explore what I wanted to be in the classroom and how I wanted my students to interact with me in the classroom. In that group, I really enjoyed the opportunity to talk with other faculty members who have this experience and have the kind of classrooms that I want to emulate and to get to learn. I really enjoyed the Small Teaching reading group. It gave me a place and a regular time to work into my schedule to sit down, talk about the kind of teacher I wanted to be, to listen to teachers with a lot more experience and how they develop their classroom, and implement these effective learning strategies to create a more productive learning environment, and to teach students and to challenge them to think critically about the world.

Rebecca: I think sometimes carving out time is one of the most difficult things, right? To think about teaching.

Maggie: Yeah.

Rebecca: So the fact that you said that the reading group provides this regular way of holding you accountable to think about these things…

Maggie: Yeah, absolutely. So, I think being a first semester faculty member I wanted to get involved in the campus community… to get to know other faculty members… to see how they’ve been successful in and outside of the classroom… and for me to try to broaden my perspectives on teaching… try to learn about the techniques that are important to facilitate learning… and to carry on to help students become valuable members in the justice system and whatever career paths they they choose.

John: Now a little bit of background on the reading group, we had a hundred and two faculty and staff members who participated in it, a large proportion were faculty. We met multiple times a week, and one of the things that happened there is people from different disciplines got a chance to talk about issues they’ve had in the classrooms, and how they’ve worked on it, and they got suggestions from other people in different departments. How did you find that experience getting to work with faculty from the sciences, from the humanities, from art and so forth?

Maggie: What I really liked about the diversity of the faculty there was… especially the math teachers. Every time they would talk about their experience in the classroom, I remember my own struggles and successes at learning something like math. And to think of a discipline that we wouldn’t normally consider has (or can) benefit (maybe) from a variety of teaching methods. I think that hearing their experiences throughout their teaching careers gives some important insight I can carry on to my own classes.

John: Could you tell us a little bit about the courses that you teach, how big they are and what the subjects are?

Maggie: Yes, so currently I teach… well, and last semester I also taught American Criminal Courts and Judicial Process. These classes ranged anywhere from fifteen students to about forty students, depending on the time of day and the enrollment. The other class that I currently teach this semester is Women and Crime, and that is about thirty-five students.

John: …and what techniques have you tried that were new to you?

Maggie: So, one of the challenges I had in my first semester here was teaching at an 8 a.m. section, and trying to…


John: There are no good solutions…

Maggie: Yeah…

Rebecca: Was it trying to teaching at 8 am or was it the students trying to take a class at 8 am?

Maggie: I think it was the students trying to take a class at 8 am… for sure. It’s hard…

John: It’s hard to stay up that late…
MAGIE: It is.

John: …they get tired and they need to get some sleep.

Maggie: Yes, that’s the end of their day, as opposed to the beginning of most of ours… and so, the 8:00 a.m. class… it was like pulling teeth trying to get them engaged and participating. In the first semester, I think I carried a lot of the same methods and practices that I had developed in grad school, and some of them… through the Small Teaching reading group, I found that I have names for all of that practice… like retrieval practices… and summarizing and recapping at the beginning of courses and at the end of a lecture… and holding small discussion groups. But, somehow none of that was quite enough to bring the 8 a.m. students back to thinking critically about the judicial system. Recently, in my current semester, I’ve started pairing up the students… and I can’t remember exactly what we called this, but pairing up the students to… after they’ve completed a mini pop quiz in class (which they all freak out about, but eventually I tell them that it’s not being graded). So, I’ve paired them up and they discussed their answers and then as a group they present their new answers and…

John: …a think-pair-share method.

Maggie: Yeah, a think-pair-share method. Yeah, so I’ve been implementing think-pair-share and a lot of the 8AM students, especially the ones that were falling asleep, they’re now forced to you know really think about this material right off the bat, and it’s helped keep them engaged throughout the rest of the course. I’ve really enjoyed taking that method and seeing them wake up a little bit more.

Rebecca: That’s great…

Maggie: Yeah.

Rebecca: …it’s not a super difficult thing to employ…

Maggie: It’s really easy.

Rebecca: …but it makes a huge difference.

Maggie: Right, Absolutely.

Rebecca: …and something that may work across all class periods, but sometimes you just have that particular class that’s got a slightly different personality…

Maggie: Right, yeah.

Rebecca: Sometimes due to the time or sometimes just the makeup of the group, that …employing different things in that situation… sometimes you have to troubleshoot like that.

Maggie: Yeah, absolutely and I think…often when you do have big personalities in the classroom, they’re so much fun… and they really they bring the other students into the discussion. But when you don’t have that, if you have students who are maybe a little more timid in the classroom, I think that think-pair-share is a good way to bring each and every one of our students into the discussion.

John: What were some of the other techniques that you may have tried?

Maggie: I’ve also implemented some low-stakes testing for my American Criminal Courts class and that has been going well so far. We’ll see how everything continues in this semester. I’m hoping that this will leave them more prepared for their midterm and their final exams by continually asking them… and then asking them questions from prior lectures… a lot of interleaving… also give them bonus questions on those quizzes to help them predict what we’re talking about next week, so I think there’s going to be a tangible difference in their grades when midterms roll around.

Rebecca: I was really surprised when I implemented more testing. You always hear conversations about test anxiety and nobody likes tests… nobody wants to take tests… and I’m in a discipline where tests are not that common. But I’ve been surprised historically, that the students maybe grumble at the beginning about it…

Maggie: Um-hmm.

Rebecca: …but over time they actually really appreciate it… and if you didn’t have one, like “what’s going on, why don’t we have one today?” They find it helpful and useful to keep them on point. How are students responding to this regular testing?

Maggie: I was actually really surprised when I got an evaluation last semester where a student asked if they could have more tests in the class. Because the format of the exams was four exams a semester, they were longer and they were, I think, looking for something that kept them accountable for the readings… something that kept them accountable for paying attention in class… and so far everyone has been… I don’t think they’re thrilled with it, but I think they understand the reason for having the tests… because I took time at the beginning of our class to talk about why having these low-stakes testings are important for their learning, but important also as they prepare for exams… and to really get this foundational information to build on in future classes.

John: …just simply reminding them that, making a mistake on a quiz that an infinitesimally small part of their grade is much better than making that mistake on a major exam…. and reminding them that this is, in large part, for formative purposes can really help, I think.

Maggie: Yeah, and for some students – who are perhaps a little more on the perfectionist side – I’ve had a couple of them pretty concerned when they miss a quiz but I tell them that I’ve dropped the lowest quiz score…maybe it’ll be the lowest two quiz scores… we’ll see how the semester goes… but to keep them accountable, but also to remind them that they’re human and things happens… they miss a quiz… they forget… and they have the opportunity to learn from that mistake, but to have not such a detrimental effect on their on their grade.

Rebecca: One thing that came up in our reading group frequently, was that faculty had much more success in their classes with this particular technique, if you took the time in class to talk to your students about why you’re doing it and how that’s helping their learning. I think that most faculty who’ve implemented it like you, and have spent the time to share that information with students, have found far more success than faculty who have just implemented the technique without explaining it .

Maggie: Yeah, and that’s one thing I’d really took from the reading group too, is that if we explain to students why the things that they typically hate are actually important and are beneficial to them, there’s a lot more buy-in from them.

John: It’s helpful in general, because students have habits of learning that we know aren’t as effective as they could be, and they tend to resist things like testing for learning. They much prefer rereading things or highlighting things, and after the second re-reading the evidence is pretty clear that there is virtually no increase in the amount that they’ll remember later… but it doesn’t feel that way. When they take a test on something and they get things wrong, it doesn’t feel as good. So, it’s important to help set it up and prime them so that they understand that this is really useful for them.

Rebecca: …and I think when you take class time to talk about evidence-based practices, not just on the first day, but a few times throughout the semester, those same students who are struggling with other parts of learning will speak up and ask more questions. I had a student today who just called me over and was asking like “I’m having a really hard time figuring out how to structure my files, so I don’t lose stuff.” Just a basic organizational thing… but in my field that’s quite detrimental, actually, if you can’t figure out how to do it. …and so, it’s kind of funny. Now they see you as a resource of someone who can help me learn better, not just in this class, but in other classes too…

Maggie: Yeah.

Rebecca: …which is kind of a nice feeling sometimes.

Maggie: Yeah, and I’ve also found that even beyond pedagogical discussions with students, that some of these concepts actually can apply to a lot of the content that we’ve learned. So, in my Women in Crime classroom we’re talking about labeling theory, and what it does to a person when they think they’ve reached the limit of their identity, if they’ve….

John: …issues of stereotype threat.

Maggie: Yeah, right. Exactly. If they fit the stereotype, how do they… can they learn to move beyond that… and so I talk to them about how, when they’re journaling I write all of my comments to reflect the work that they’ve done, and not the person that they are. …and so, I had them to think back about some of those comments that I do make and I tell them that this was purposeful on my end… Because I want them to know that they can do better in some cases… and in other cases that work has reflected some of their best effort… and that’s a good thing, right? And that their effort is just as important as, perhaps more important as, who they are as a person.

John: So it helps build a growth mindset ..

Maggie: Yeah.

John: …of the sort that Carol Dweck talks about. and that’s really helpful, because if they can learn that they can learn and improve their work, they become much more effective.

Rebecca: One of the things that I’ve found, implementing some of these techniques (especially in the first couple of tries), is that, when it doesn’t feel good to learn, right? [LAUGHTER]… we sometimes have this illusion that we know something, but we don’t actually know it. The students can get a little downtrodden, right? … a little down on themselves… and so then you have to remember to monitor for that a little bit…. which I’ve learned over time, and then you got to kind of just stop and allow for an opportunity to show success. So, for example, in my classes, I started breaking my first project assignment into very small pieces, so everything was low stakes. But, I could see that at some point they just maxed out, and they’re “I can’t do any of this, like I don’t know any of this.” I was like “well, actually you do know most of it, you’re just panicking for no reason.” So, we stopped one day, and we just did we just did a brand new little thing that demonstrated to them that they could actually do the entire project that we’d been doing… in two hours, despite the fact that we’ve spent three weeks on it… and yes, there was a couple little hurdles that they had, but the hurdles they had were minor, and they could do it. I think to allow for that growth mindset… yes, you need them to fail and realize that they can do better, but then also allow for some opportunities where they get some real success too. It seems like you’re interested in this growth mindset idea, so have you been experimenting with any of these sorts of things in your classes?

Maggie: When I started doing the mini pop quizzes, because not only do I throw in a mini pop quiz occasionally, but I also have them doing low stakes quizzing over the weekends on Blackboard. So, I think they start to get a little overwhelmed. We were able to do this on our quiz, when we had everything in front of us, but now that we don’t have anything in front of us, you know, I can see it in their faces. They’re freaking out. They’re upset with themselves, because they knew that they had this information somewhere, they just hadn’t had the time to actually recall it without their notes. I think that, hopefully, the more we do this, the more they’ll take their quizzes maybe a little more seriously and try to really push themselves to do it without without their notes.

Rebecca: So, actually trying to recall the information as opposed to look it up.

Maggie: Right, yeah, right.

Rebecca: Yeah, sometimes students do things out of convenience and meeting a deadline as opposed to valuing the …

Maggie: Sure.

Rebecca: …learning and the more we demonstrate to them that we value that they’ve learned, the better. This semester, I just… for some reason my class is just full of anxiety in a way that maybe even last semester it wasn’t… I didn’t change anything… it’s the same thing… but I spent the time, I felt it…. this big ball of anxiety is not going to move forward… because the anxiety is getting in the way of learning now…. to just stop and recognize that you’re observing something… and then make a change…. this is what the syllabus says, but I’m feeling this and I see this. Do you guys agree? Yeah, great! How about we do this instead? Does that make sense to everyone? And then all of a sudden… buy-in again.

Maggie: Right, yeah. They feel like they have the ability to structure the class themselves right? ..that it’s not just you sitting there saying “This is what we’re doing.” You’re asking them: “If we change this, will this be better for our class?” I think that’s cool, and does keep them invested in the class itself.

Rebecca: What other kinds of supports do you feel a new faculty member needs in place to be successful as a teacher?

Maggie: So, I think that for me, because in grad school I sought out different faculty members and helped create some of the programs that I thought were missing… I think for those faculty members who haven’t put as much emphasis on their teaching, perhaps to reach out to them and maybe have a friend bring them along to activities and in workshops at the Center for Excellence in Learning and Teaching holds.

John: …and it’s not surprising though that some people, when they first start teaching would not put that much weight on teaching because they’ve just come from an R1 institution, where their focus is often entirely, or nearly entirely, on research … and most people start by teaching the way they’ve learned …and the way they’ve learned is often just simply lectures and exams…

Maggie: Yeah.

John: One of the things I’ve observed as a chair of our recruitment committee (for the last couple of decades) is that, in economics at least, a very large proportion of faculty have no background in teaching while they’re in grad school. In our last search, we had, I think, three or four people who had actually some knowledge of effective teaching practices… and actually three of them made it to our top five list of candidates, but they were by far the exception… and it’s a tough adjustment …and it takes a while, especially when you have to start your research very quickly in order to meet tenure requirements …it’s a difficult adjustment.

Maggie: Yeah.

Rebecca: I think the first semester, in general, is a tough adjustment. You’re at a new institution… there’s institutional memory that you’re not privy to… there’s all kinds of acronyms that you don’t understand. It takes a lot of time to figure out who this student population is… and you might not think that between institutions the students change very much, but man they’re really different… and you have to adjust your teaching to the population that you’re dealing with. As a designer, I would always jump on my soapbox to say you have to design for your audience, and I don’t think designing your classroom experiences is any different.

Maggie: I’d say in grad school, most of the courses I took were very heavy with reading…. and a lot of discussion based classes. But a lot of students don’t have the time to do the kind of reading we did… and that’s why it was grad school and this is undergrad…. and for them who are just learning… and I think we talked about this in the Small Teaching reading group… that we have the ability to make connections across different concepts and how they interrelate to each other, but the students aren’t there yet …and so, when you’re going from one institution where you have gauged where these students are and what kind of connections they’re able to make, because you know a little bit more about their experience, and then you move to a new university and those experiences… some of them are similar, but a lot of them are vastly different… and to gauge where they’re able to make these connections and how much I have to draw out those connections certainly changes on that university.

Rebecca: It isn’t just university, it’s sometimes like semester to semester….

Maggie: Sure.

Rebecca: …or group to group. You almost need to build in a way in your classes to figure out… some sort of little survey or something… like, who are these people? what do they already know?

Maggie: Yeah, yeah.

Rebecca: So that you can make those connections… so that you have a better understanding of what your class’s mental model looks like versus your own.

John: …and there’s a number of ways of doing it… some people will give a pretest at the beginning… just asking what people had, or some general questions about the discipline or their prior knowledge. Others will use clickers and other things throughout the term to assess knowledge before moving on. …and, as you said, just asking students to reflect on what they know and perhaps write it down or at least bring it to the discussion, is another good way to help determine the level, so you can do more just-in-time teaching and deal with what students come in with rather than what you think they should come in with.

Rebecca: What’s your favorite way of handling that John, in your classes?

John: It varies a lot by class. I use clickers regularly in actually nearly all of my classes. I don’t use it in the seminar class I’m teaching, but I’m using them in my econometrics class which is, I think, about 35 to 40 students, somewhere in that range… and I use it in my large class where I have somewhere between 360 and 420 students every fall… and it’s a good way of getting that sort of information. I’ve also sometimes used pre-tests on basic math skills or other things in my large intro class. Sometimes in a smaller class I’ll just ask them what classes they’ve had in the past. When I’m teaching econometrics, for example, I have some students who are math majors who’ve already had multivariable calculus and three or four stat classes. Other students come in who took, sometime in the last three or four years, a very basic stat class… and they come in with very different backgrounds and very different information. Some of the people in the class are math majors who just want to pick up a course and they haven’t had that many economic classes and so finding out what they know helps me determine how much emphasis I need to focus. One of the problems though is that students have such diverse that it’s difficult, but the more we can get them working together with peer instruction, the more they can help each other fill in those gaps.

Rebecca: I think one of the things that I like about what you were saying is that students, in some of those methods, it’s revealed what other students know too, so that they can see that “I have a different mental model, but it matches up with some other people, and here’s some other students and they have this other kind of information.” I find it helpful, and I think the students find it helpful to recognize that people come with different expertise and that we can lean on those expertise at different times. Do you have any strategies that you’ve been using to figure out who’s in the room yet or is that something that you’re still kind of experimenting with?

Maggie: It’s something that I’m still experimenting with. Last semester, I did a pretest and it was helpful for gauging what they knew about criminal justice, but not necessarily what they knew about courts and how those concepts in criminal justice relate to the court system… and this semester I did not do a pretest, but another thing that, I think has been an interesting way to see what they know when they know it is I’ve had them write down for a minute the important things we’ve talked about last class and then I go around the room and every person has to say at least one thing… and everything they know about that thing… but it has to be different from the other person and so that challenges them… and of course the people in the very back corner of the class are freaking out because they’re afraid everything’s gonna be said… but then it turns out that we only get to half of the topics that we actually discussed in the class. So, I think that giving them the opportunity to actually see what they know, I think is important for them in the classroom moving forward.

Rebecca: Yeah, a lot of students don’t have good metacognitive skills, in general. They have no idea what they know. So, if you take the time to get them to even stop and think about what they know, it’s more time than they probably spent on it…

Maggie: Right. Yeah.

Rebecca: …giving them the time and demonstrating that you think it’s valuable for them to spend time thinking about what they know… by you even spending two or three minutes in class on it… all of a sudden lets them know that that’s something that’s valuable.

John: …and actually I’m going to be trying something new this semester to help build on those skills with something that Doug McKee talked about in an earlier podcast, which is the use of two-stage exams. I’m giving an exam in my econometrics class next week next Wednesday and then they’ll all take it individually. Then I will grade them, and then the next class they’re going to work on subset of the questions in small groups… and then they’ll submit group responses which will be weighted as a portion of their overall score. So, basically, they get the opportunity to improve their scores on some of the more challenging questions by sharing their knowledge and in places where that’s been done they found some fairly significant learning gains from leveraging the knowledge of their peers.

Rebecca: Sounds very similar to Maggie’s strategy for some of her quizzes.

Maggie: Yeah. that’s exactly what the mini pop quiz that I’ve been implementing… that they can try to figure out their answers on their own, but then they can really talk with another student… see where their strengths are where… perhaps the other students strengths are and builds from there.

Rebecca: I think what’s nice about pointing out those two examples is that both emphasize peer instruction… but ones in a high-stakes situation… ones in a low-stakes situation. The fact that there’s kind of two graded parts in the in the two-stage exam is a way to demonstrate a way of doing it in a higher stakes situation and then a nice low stakes situation where they try it on their own but then they can collaborate before they’re ever graded is a different scenario and a different level of pressure, etc.

John: …and adding one more level complexity to mine… it’s actually a little lower stakes than it might sound because, while they have three exams (I’ve told them this at the beginning of the term but I’ll have to remind them after they get their test scores back), I have a series of exams that are progressively cumulative, and if they do better on the second exam it will replace the first; if they do better on the third exam it will replace either of the first two… because they’re tested on all the material again. So, they get another chance. It does make the subsequent exams a little bit higher stakes if they didn’t do well, but they have the opportunity to make mistakes, learn from that, and improve their scores.

Rebecca: Great.

John: …and thanks to Doug for the suggestion about the the two-stage exams!
Now, you’re also going to be trying something new next year. Oswego is introducing some new signature courses for students in their first year. You, very graciously, were one of the new faculty who chose to participate in that. Could you tell us a little bit about the course that you’re going to be doing?

Maggie: There’s a group of faculty who have been asked to teach very small seminar like courses that are really aimed at engaging students in their very first year… their very first semester at Oswego and with the intent to get them connected to the University and to get them really excited about their coursework and uncover some of their interests. The class that I’ll be teaching is called the Injustice League. It’s on crime, inequalities, and injustice in comic books… and so, in the class we’re going to read a lot of comic books and some graphic novels… and we will have the chance to talk about how those comic books reflect inequalities that exist in society… how they reinforce some of those inequalities… and how comic books are used today to deliver how comic books are used today to facilitate discussions on race, gender, class inequalities.

John: That sounds like a lot of fun

Maggie:Yeah, it’s gonna be…

Rebecca: Yeah, can I sign up?

John: I wish I could take many of these class but that one in particular sounded really interesting.

Maggie: Yeah, it’s gonna be fun, I think, for students who perhaps don’t have an interest in comic books… or those who do, they can share their interests and we can learn a lot of things about the worlds on our way.

Rebecca: What I like about that is using something that’s more popular media and then using it as a tool to apply critical lenses…

Maggie: Yes

Rebecca: …and really getting students to think critically. We don’t have to always think so abstractly to think critically…

Maggie: Sure.

Rebecca: …a comic book is a nice tangible thing that you can look at and analyze and evaluate.

Maggie: Yeah, and apply to other discussions that we have going on in society. So if it’s examining gender and equality, and the way female superheroes are portrayed in their dress, and how we, as a society, have developed gendered expectations of women and of women of color and all of those intersecting identities.

John: …and Wonder Woman was developed in large part to help correct some of those gender imbalances, right? …it was developed by a psychologist who wanted to help provide a better gender…

Rebecca: Have you seen what she wears?

John: Well, there was that, too… and and there were some other issues there, but that was the rationale…

Rebecca: Yeah, of course, it was a male psychologist.

John: It was, but…
…in any case…. Okay, never mind.

Rebecca: I was just having a conversation with another one of our colleagues the other day about some of the topics that are really intangible like race, gender, and inequality. There are things that we talk about, but they’re abstract or conceptual, and so sometimes students have a really hard time getting their finger on it and finding a doorway in. So, I like the idea of the comic book as a really specific physical object… a really tangible space to enter into those discussions rather than thinking up in the air….

Maggie: Yeah.

Rebecca: …where it’s not always easy to digest that if you don’t have a good mental model of what you’re talking about.

John: …and it may also make it easier for students to separate themselves from those issues, because of course they don’t have any biased views themselves… but when they start seeing it perhaps in the comic books it might be help them identify it more generally in society and in themselves and in the world around them.

Maggie: Yeah, it helps them uncover where their biases do lie.

John: Right, but they’re implicit biases in part because they’re not aware of them so…

Maggie: Right. Yeah, they discover where these gendered expectations that they have been surrounded by throughout their growth as children and adolescents… and in why those expectations are problematic, and through comic books they can actually see it. We can actually point to why wealth inequality can create a superhero or it can create a villain.

John: …and it’s also perhaps less threatening that way, because it’s a comic book. It’s not their life directly, but it’s a nice lens by which they can start seeing these issues better.

Maggie: Absolutely.

Rebecca: Sounds like you had a lot of exciting teaching things on the horizon, what else are you gonna do next?

Maggie: I would like to develop a class that looks at punishment and the historical development of punishment and how sexism and racism in society have influenced that development in our society. …and I think for classes, we’re talking about really tough subjects that it’s important for them to be engaged and to feel comfortable having these conversations, because they are so necessary… and through these techniques, I think it gives them a level of comfort in the classroom to be themselves and to understand their positionality in society and how their experiences have impacted the way they view these social issues, and how they can resist against some of those preconceived ideas.

John: Excellent.

Rebecca: Right.

John: …and I think this could be a great topic for a future podcast…

Rebecca: Yeah, we’ll follow up on that

John: …when your class is underway.

Rebecca: Absolutely. Definitely. Well, thank you so much for joining us Maggie and sharing your perspective. I think a lot of faculty can relate and have had similar struggles and also similar successes, but it’s really nice to see your journey. So, thanks for sharing it with us today.

Maggie: Thanks.

John: Thanks a lot. It’s great having you here at Oswego.

Maggie: Thank you. It’s good being here.

19. Common Problem Pedagogy

Most colleges are organized as a collection of academic silos. Many challenging problems facing society, though, are multifaceted. In this episode, Leigh Allison Wilson joins us to discuss the use of common problem pedagogy, an approach that allows students to address a problem from multiple disciplinary perspectives.

Leigh is the Director of the Interdisciplinary Program and Activities Center at SUNY-Oswego. She is also the author of two collections of stories, one of which won the Flannery O’Connor Award for Short Fiction. Her stories have appeared in the Georgia Review, Grand Street, Harper’s, The Kenyon Review, Smokelong Quarterly, The Southern Review, and elsewhere. Leigh teaches creative writing at SUNY Oswego. In addition to the Flannery O’Connor award, she has received the Saltonstall Award for Creative Nonfiction, and a Pulitzer nomination by William Morrow for her collection Wind. Leigh is a Michener Fellow of the Copernicus Society and is a Henry Hoyns fellow of the University of Virginia.

Show Notes


John: Most colleges are organized as a collection of academic silos. Many challenging problems facing society, though, are multifaceted. In this episode, we explore common problem pedagogy, an approach that allows students to address a problem from multiple disciplinary perspectives.

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Leigh Allison Wilson. She is the author of two collections of stories, one of which won the Flannery O’Connor Award for Short Fiction. Her stories have appeared in the Georgia Review, Grand Street, Harper’s, The Kenyon Review, Smokelong Quarterly, The Southern Review, and elsewhere. She teaches creative writing at SUNY Oswego. In addition to the Flannery O’Connor award, she has received the Saltonstall Award for Creative Nonfiction, and a Pulitzer nomination by William Morrow for her collection Wind. She is a Michener Fellow of the Copernicus Society and is a Henry Hoyns fellow of the University of Virginia.

Rebecca: Welcome, Leigh.

John: Welcome, Leigh.

Leigh: Thank you, John. It’s very nice of you to have me

Rebecca: Today, our teas are:

John: Ginger peach green tea.

Leigh: Mine is Constant Comment, a southern favorite.

Rebecca: …which Leigh brought for me to try, so that’s what I’m drinking, too.

So, Leigh you helped organize a number of community-based projects that bring faculty together across campus. What got you involved in this kind of work in the first place?

Leigh: Well, you know what? If I went back to the roots of it all, I have to say Amy Bartell in the art department. I have a flash fiction class that is my advanced writing class, and one semester she just suddenly said: “Why don’t your students write a very short piece? My students can illustrate it, and we’ll frame both things and put them side-by-side… and then we’ll have a show.” …which was so much fun, but it wasn’t just fun, it was my first taste of having a collaborative common problem project. Because, it turned out to be a common problem. We didn’t know it… we thought we were just writing our fiction… or, I thought they were just gonna be writing their fiction. But we’ve discovered that if there was going to be an illustrator paying attention to it… all of a sudden, it got more serious. The game got more serious. There was an audience who was really going to be checking it out, and there was also an audience that was going to be looking at the illustration and looking at their work at the same time, and all of a sudden the students were much more professional about their attitudes to their work. So, that’s the beginning of it. That was called Graphic Flash… and we’re still doing it, but now it’s expanded into a film class that’s taking the stories and making short films out of it… and a music class that’s taking the short films and scoring them… and ‘cause now I like working with local partners… local high schools have been making movie posters.

Rebecca: Great.

Leigh: …for the stories. So, that expanded… and because of that expansion, I started getting interested in – not just common projects that involved a common problem – but also collaborative projects in general… and the ease with which they could be expanded… which I think is one big factor in project-based learning.

Rebecca: The first big project was the Smart Neighbors project which is still ongoing.

Leigh: What happened was… I was doing Smart Neighbors and there was a notice from the Provost office and there was a call for participants in a SUNY wide grant. They wanted four SUNY schools to be involved in a common problem pedagogy grant… and at the time they were trying to get a Teagle grant which is an ExxonMobil grant. But the point of the Teagle grant was to get humanities to work with another discipline, usually a professional discipline, so that’s why it began in that way. I wrote in and SUNY Cortland and Oneonta and Plattsburgh were all involved in it. We all have different projects going on but ours became the Smart Neighbors project.

Rebecca: Please describe what that is for those that don’t know?

Leigh: Basically I have always….Well, I love Oswego as a town, and I’ve loved living here and I’ve always wanted to do something that could give back. But, I’m a creative writer and, short of putting it as a setting in a lot of short stories… which I have done… that’s not really giving back… I have always worried since I’ve been here about the economic difficulties facing any new business. This is a stat from a few years ago, but one statistic is that a new business in Oswego has a lifespan of about 13 months… and that’s a terrible statistic. I don’t think it’s true anymore… I think there are great changes going on in town now… but, I wanted to do something with the town. My concept for Smart Neighbors was to have a lot of different disciplines collaborate in the promotion of a downtown independent business. It was a simple concept, because I didn’t have elaborate blueprints for what they should be doing or what we should be doing. I had no elaborate plans for what each individual discipline should be doing. It should be promoting the business. Period. ….and that’s sort of continued to be how it is. People take it as they can imagine it… and so a lot of very imaginative things have come out of that… the things that are not traditionally considered promotional materials… which, in fact, really are promotional materials.

John: What are some examples?

Leigh: A literary citizenship class that Donna Steiner is working with, because they’re mostly creative writers, they tend to do digital essays… but they’re digital essays that often have a fanciful story involved in them. So, if it’s a bookstore… one digital essay took a book that the bookstore was selling… talked about the author ….did graphics about the plot of it… and then ended up back at the bookstore… and so you basically you were interested in the book… and then it began to talk about how the imagination could be served by the bookstore. Another one in the same class followed someone who bought a book to their home, took film clips and photographs of the person sitting where they liked to read with all of their books around them… and just talking about what it meant to be able to walk downtown and buy a book and take it home and start reading it. So, that was a nice little piece too. …but not things that you necessarily are expecting, or what an advertising agency would have put out.

John: …and how have the businesses responded to this? Have they been using these materials in their marketing?

Leigh: They have. One of the things that is a centerpiece is the banner… and the art students… the photography students have been at the heart of that… and all of the businesses end up displaying it. There are huge banners… they fill a whole wall… but all of the businesses have been using the banners. They love those… Also, every business nowadays… and this is one thing that we’ve been working with the businesses on… having an online presence… but that’s one of the reasons there’s so many digital projects involved. Because we want the businesses to be able to use them online. So, the digital essays do get used online as part of their presentation to the public.

John: …and how have the students reacted to doing something where their work is going to be more public? They’re not just submitting something read by their instructor and their peers, but it actually may have an impact on some business in the community.

Leigh: The impacts on our students are the impacts that I think they’ve found across the country when dealing with applied learning, civic engagement, volunteerism… well, basically best practices in general…. but, number one (this is the thing that I’m most proud of) is that the students leave that program, even though it’s one assignment in one course (for most of them… it’s not the whole course) but they leave having experienced that assignment with a sort of sense of social responsibility that I don’t think they had before… or a notion of philanthropy. One of the things I tell them…. All of the classes (this year we had 11 classes from different disciplines) and we all meet at the beginning of the semester in Marano Auditorium… and one thing I told them this year is that we think of social responsibility as as one thing and philanthropy as another thing… but really, I think, what we should be doing in these places we love (like I love Oswego) is actually contributing our talents… not just our money… but we should be spending our money locally too – but but also contributing our talents – to these businesses… even if we’re a business owner contributing to another person’s business is something that I think we’re obliged to do too – because the local success really is our own success… and we tend to think of businesses as competitive, but I think that’s a mistake. I think smarter neighbors…

John: …hence the name…

Leigh: …work together in these collaborative ways.

Rebecca: What are the biggest lessons that you’ve learned from doing some of these projects?

Leigh: Well, I should tell more of what the students got out of it, but…

Rebecca: Yes.

Leigh: I think, other than just that sense of social responsibility and what notions of philanthropy, they leave knowing much more clearly what they know and have learned in their disciplines… meta-knowledge of what they’re capable of… which is huge for our creative writers. Because, I don’t think they’re clear on the fact that… they know they’re probably not going to immediately get the Pulitzer, but what can they do with this? …and it’s important for them to to learn that…. that they can write for multiple audiences in multiple ways. But, they also learn what other disciplines know and can do… which they haven’t thought about that deeply. It’s a mystery to them what, for instance, the marketing students do. They market things… maybe it’s advertising… something like that… but then they see them come in and actually take the business that they’ve been working with and figure out a plan for them… and how the college itself can be moved into that plan… and suddenly: “Oh, I can work with that…” and they start thinking of digital essays they could work with… and imaginary stories that take that marketing plan and actually enact it with characters (which they’re good at imagining)… And professional skills… just getting somewhere on time… being late or on time to a class seems less important, but if the interview that you needed to have… and you’re late for and you can’t now have it… that makes an impact forever. You tend to be on time for an interview… and they do have to interview the local partners. Preparations… They get there. Nobody’s going to be telling them what to do. They have to figure out what they need to know, and they need to find it out. So, they need to plan before they get there. I personally am very happy with my students learning what it means to write for a particular audience, as opposed to whoever they want to. It’s very good for them to try to please a certain person with a certain product.

Rebecca: Because it’s usually an audience that they wouldn’t have picked or imagined on their own.

Leigh: That’s right… that’s right. …and my point would be that, even when they’re writing their Great American Novel, they should be expanding their notion of what audiences they’re hitting, instead of just “this is what I want to read.” They need to think about what their vision of the world is and how to can pull as many people into it as possible. I just think it’s memorable to them. I think it’s life-changing to them to work, however briefly, donating their time to a place at least for a while they’re calling home.

John: Excellent.

Leigh: But, I think they’re things that the faculty learned too… not just the students, there are faculty outcomes, I think, as well. My whole idea in Smart Neighbors was to just get faculty’s feet wet with one assignment in one class… and if you can do that… once they see the effect on students… because, that’s one thing I really do believe about the faculty here… they really are committed teachers. Now sometimes you worry about how time-consuming is it going to be to work with another class as other disciplines… how time-consuming is this or that? Because we’re already putting a huge amount of time into our teaching. So, it seemed smart to get faculty accustomed, or introduced to, collaborative, or civic engagement, or applied learning kinds of pedagogy in the easiest possible way. So, one assignment… and not an assignment that necessarily requires interactions with a lot of other faculty to figure out how to do it. Now, I will say, for Smart Neighbors anyway, the faculty do have to connect with the local partners. But, they don’t necessarily have to figure out what everybody’s doing in all of the classes to make it work. They have their piece of the puzzle and they’re contributing it.

John: How many classes work with a particular business? Are there multiple businesses that they’re working with? or is it just one business each year?

Leigh: Well, it’s grown. The first year, we had four classes and they were working on the bookstore. The River’s End Bookstore.

Leigh: Tell me your question again.

Rebecca: Really asking whether or not there is more than one community partner at any given time.

John: Yes.

Leigh: Yes. I think what you’re asking is a good question because, once you get to a certain number of people… of courses… not people, but courses… you’re overwhelming a local partner and we got to that quickly last year. We worked with a candy store (and I think there were seven different classes involved) and an unbelievable generosity of time from that owner… but it was clear that we were gonna have to figure out other ways of doing this. So, last year we did the Farmers Market, which worked out, We had eleven courses involved too – and that worked out much better because there are multiple farmers bringing their goods to the Farmers Market and there are they’re in different groups with different farms. So that worked out a little bit better. Also, because the Chamber of Commerce is ultimately responsible for the Farmers Market, we were able to do some projects just for the Chamber. For instance, they needed a new logo and we sort of pulled that into the Smart Neighbors project as well. So, I’m trying to define what we’re doing a little wider. …and you’re right, have more local partners…. if we’re gonna have this many continue.

John: It sounds like it’s grown really quickly.

Leigh: It really has… and I will just say again I think the faculty discovered that there’s a certain ease of practice in getting used to this… and once you see the students and the effect on the students, then I think you’re hooked. And the reason it’s grown is that the courses who have done it in the past continue to do it; they want to keep doing it. And that is how I got the idea for Grand Challenges.

Rebecca: That seems like a nice segue right into it, right?

John: Yeah.

Rebecca: So, we’re launching the Grand Challenges. Can you talk a little bit about what the Grand Challenges are and what the goals are?

Leigh: There’s a line in our strategic plan that that’s my favorite line…. and I think it’s the most memorable line… and it talks about how we, as a community, are going to tackle the Grand Challenges…. find solutions to the Grand Challenges of our time… and I love it… because it’s aspirational for one thing. I really do want to believe that our students and our faculty can tackle the Grand Challenges of our time, and I think we can, frankly… but it’s also that notion of “tackling a challenge” is very project oriented. You get your hands dirty. You figure out something, and then you try to come up with solutions because of it… and, so it appealed to me just in terms of having a common problem. But, those Grand Challenges have to be tackled together. I mean, I don’t think there’s any challenge of any size in the complexity of our world today that can be done by a single person just sitting in their garage thinking. I think almost everything we do in the future is going to have to be collaborative and probably cross-disciplinary in some way. So, it just seemed to me a natural segue from Smart Neighbors to getting the whole campus to work on a single… it’s not really a single issue either…. it’s more… we were talking about this, Rebecca, I imagine the topics for Grand Challenges to be very concrete things, because I think, as academics, we tend toward a more abstract way of looking at things…

Rebecca: …which is particularly hard for our students to get their heads around. They need something tangible.

Leigh: Right, I think so too… and to come up with projects… actual projects that are going to take place in the world with local partners… or involving civic engagement or volunteerism… require a certain concreteness. So, at any rate, the Grand Challenges project was just something I began to think. The notion of having multiple disciplines work on the same thing… it’s just a short step to getting the entire campus to work as much as much together as possible on the same topic… One of the things I didn’t say about Smart Neighbors is that Oswego is already a very collaborative culture… and that we’re very far along in terms of faculty tipping into these kinds of projects very easily…. and I’ve found just talking across campus, the way for instance when I spoke to Faculty Assembly, and the reception there was so astonishing. People aren’t resisting it out of hand. It’s just such a pleasure to work with people who are willing to take on these new things without immediate misgiving. At any rate, as you know, the topic that we’ve that we’re working with this year is fresh water which is concrete, but also can involve a lot of sustainability.

John: …but fluid, too.


Leigh: Very funny, John…
But, one of the things that I like about that particular topic is that you can look out any window on campus and fresh water is exactly what you’re looking at… and that it should matter to us makes sense to me. But, to go back to the teaching culture here, I have found when I talk about this to any group of faculty, immediately ideas are popping. They’re thinking about it. They’re talking about it. They clearly already thought about it. The Grand Challenge doesn’t really even begin until the fall of this year… and I’ve got a list …I brought with me a whole list of like couple of dozen projects that people are already doing right now…. this semester…

John: In preparation?

Leigh: Just because they can. Not only in preparation, just… let’s begin… Why wait till the fall? I’ve spent the last week finalizing touches to a micro grant the Provost office has, thank goodness, very gallantly is going to put some money in a pot to give some grants to people to do these collaborative works. Well, let’s just put it this way… even if you’re just doing an assignment in your class, you can put in for one of these grants. But, I think we’re going to privilege, probably, the collaborative civic engagement projects… or they’ll get the higher money amounts, just because there are more people involved. The administration on campus has just been so supportive. The provost office is doing the micro grants. The Student Affairs has, I can’t talk about it because the contracts haven’t been signed, but they’ve got people who are well-known coming to speak on campus.

John: So, there’s going to be some other programming throughout the college.

Leigh: That’s right. Artswego has a special category for its grants this year that are going to privilege some Grand Challenge proposals.

Rebecca: What I like about that concept is that the learning doesn’t just happen in a classroom on a college campus. It’s happening from multiple perspectives and it’s happening in and out. It’s happening formally and informally.

Leigh: That’s right.

Rebecca: That’s nice that there’s a lot of systems in place to help support that and that idea because, if students are experiencing the topic of water, in a lot of different disciplines on and outside of class right then they’re gonna start seeing how all these things connect together…

Leigh: That’s right.

Rebecca: …and we have general education as a part of our curriculum, as many colleges do, and the students tend to not have any idea how that is relevant or important or what that does for them. and I think this might be a really great way for them to start seeing that all these things are actually connected and it’s important to know different points of view and the different disciplinary perspectives on things… so that there is that idea that we can’t tackle these really big problems…

Leigh: …by ourselves.

Rebecca: …without looking from multiple perspectives.

Leigh: Yeah.

John: …and faculty are often in their own silos and students see the classes as separate islands that are not connected in any way… and showing that we can look at the same issues broadly from a number of different perspectives might help them form better connections and deepen their learning.

Rebecca: …and even continue to update the curriculum to reflect this change in practice. It’s a move away from silos to things being a little more messy, and so how do you allow for your curriculum to embrace that messiness.

Leigh: I think you’re exactly right, Rebecca. …and I, for one, think the future (it might not be in our generation) but the future really will be a future that doesn’t necessarily have departments… doesn’t necessarily have disciplines separated in this way…. that in fact encourages cross-disciplinary activity. I think the School of Communications, Media and the Arts [SCMA] is already sort of moving toward that. They’re a very collaborative school and work very well… that just in my experience doing these projects, they work very well across campus with any discipline.

Rebecca: Go SCMA.


Leigh: I am on the board. It’s because of that that I asked to be on their advisory board, frankly.

Rebecca: Yeah.

Leigh: But, yeah. I think the beauty of the grant, of Grand Challenges, is that we’re already a collaborative school and this just puts the name on it. It puts a focus for that and it’s something I think we really ought to be celebrating here. …and to get back to the administration being supportive, the President from the beginning has been behind this and I think that, really more than anything, has been one reason for this to be a successful rollout.

Rebecca: Are there plans to research or study the outcomes of the initiative to measure what impact doing something like this has on our learning community and/or on the community at large?

Leigh: Well, one of the things that I hope from these micro grants is, because they have to give the proposal at the project proposal… and give what they hope the outcomes will be… and then when they do their final reports, what they think the outcomes really were. I’m hoping that that will be the first step toward being able to assess some of the things going on. It’s more difficult in the general population, One of the things I’m reluctant to do is add a layer that makes people hesitant to get their feet wet with these pedagogies. But, I think, just once this gets going… I think it will become easier and easier to get people to assess for what the outcomes are. To be honest, I think it’s so night and day what the students get out of these best practices that the faculty will want to start assessing and seeing what these outcomes are and what it means in their classroom.

John: In an earlier discussion, you mentioned that your work with the Digital Oz project grew out of your work with the Smart Neighbors project. Could you tell us a little bit about the Digital Oz project and how it relates to your work with Smart Neighbors.

Leigh: Digital Oz is a presentation… online presentation site… for SUNY Oswego students’ digital work.One of the things that occurred to me after doing Smart Neighbors is that these collaborative efforts on campus are here and gone tomorrow …because there’s no place to archive or curate the materials that the students produce… and so Digital Oz has become a space where the collaborative work can actually be presented. The students are doing such amazing work. It’s great that Digital Oz exists so that the students can have some sort of public presentation.

John: Could you describe Digital Oz a little bit for listeners who may not be familiar with it?

Leigh: One of the things that I’ve always liked about Oswego students is that they have authenticity that is almost indescribable… but once you see them tell a story, you feel it instantly… and so I think because our students all have these stories it’d be nice if we had a site that had them tell them. So, we created Digital Oz and it has different categories. One category… the students talk about how they ended up being passionate about what they’re passionate about here (whether it’s their discipline or some sort of co-curricular activity that they do) and what’s the story behind that. How do they become passionate about it? …and there’s some amazing stories there. Students who, for instance, work as EMTs on the ambulance service on campus have some unbelievably touching stories about why they care… about being able to go to somebody and help them. But, there’s another category that’s called “moments that change their lives” …the students lives, and they talk about them in very moving ways as well. But one of the categories, as I said is “Collaborate” and students who have worked together on projects put artifacts that they’ve created for those projects online… and those two are… I guess you don’t realize the range and creativity and professionalism of our student work until you start seeing it put together in the same place…. And Digital Oz, since we’re talking about it… I’ll just say it’s… digitaloz.oswego.edu is the website if you want to look at it. But, it’s a place, I think, high school students look at and find feel like they can have a home here.

John: Excellent, and we will share that link in the show notes.

Leigh: Thank you.

Rebecca: So usually we like to end with, “What are you going to do next?” So, you’ve got this big giant project.

John: It’s still under way…

Rebecca: You’ve got this big giant project. What’s down the road a little bit for you?

Leigh: Well, I really do think that the Grand Challenges is as grand as I’ll probably get.


Because I don’t know how I can get grander.

John: The very Grand Challenge.

Rebecca: Super Grand Challenges.


Leigh: I know… it will be like Mario. But, one of the things… I’m talking to the woman who’s in charge of applied learning at SUNY Central, and I’m gonna talk up the Grand Challenges just because I think it really is a harbinger of what the future is going to be, not only in terms of what you do in collaborative ways, or best practices but also in what it’s going to ultimately mean for what the shape of the university is. So, I guess I’m not going to become a traveling advocate across the campuses across SUNY, but I do think I really do think this is where the future is headed for higher ed. I hope so anyway. I do.

Rebecca: Great. Well, thank you so much for sharing what you’ve been working on, Leigh. I think everyone will continue to be inspired.

John:Thank you. It’s a great series of project.

Leigh: Thanks, you guys.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

18. Faculty Development

We all want to be more effective teachers, but face increased demands on our time. What can colleges and universities do to efficiently support faculty development? In this episode, we discuss these issues with Chris Price, the Academic Program Manager at the Center for Professional Development at the State University of New York. Before joining the Center for Professional Development, Chris was the Director of the Center for Excellence in Learning and Teaching at SUNY Brockport. While at Brockport, Chris also taught classes in Political Science and in the online Masters of Arts in Liberal Studies program.

Show Notes


Rebecca: We all want to be more effective teachers, but face increased demands on our time. What can colleges and universities do to efficiently support faculty development? In this episode, we’ll discuss how teaching and learning centers in the State University of New York system are tackling these issues.

John: Our guest today is Chris Price. Chris is the Academic Program Manager at the Center for Professional Development at the State University of New York. Before joining the Center for Professional Development, Chris was the Director of the Center for Excellence in Learning and Teaching at SUNY Brockport. While at Brockport, he also taught classes in Political Science and in the online Masters of Arts in Liberal Studies program. Welcome, Chris.

Chris: Thanks, John. Good to be here.

Rebecca: Today our teas are:

Chris: I just finished my coffee.


Rebecca: It’s really an epidemic.

Chris: …but. if I was having tea, I’d be having Earl Grey.

John: Okay. Rebecca?

Rebecca: Jasmine green tea.

John: …and I am drinking Harry and David’s Bing Cherry Black Tea.

Chris: Mmmm. That sounds good, I’ll have to check it out.

Rebecca: So Chris, your role at SUNY is somewhat unique. Can you talk a little bit about what you do?

Chris: Yeah, sure. My role is kind of twofold. First thing that I try to do is keep abreast of what is going on in faculty professional development across the system and I do that primarily through talking with and networking with all the people in the system who do faculty professional development… and that runs the gamut from people who are directors of teaching and learning centers to those who work in instructional design and support online courses and hybrid courses and that sort of thing. So, that’s part of my job… and just keeping that group together. I like to say it’s like herding feral cats, because people do faculty development tend to wear a lot of hats and they are usually doing a million things at once. So trying to keep up to date with what they’re doing is kind of a challenge. The other thing I do is develop and deliver our academic programs… there’s the title: academic programs manager. Our academic programs really just service everybody who is involved with teaching and learning in the system. Our programs aren’t really meant to replace… or be the only thing a campus will utilize for their professional development for their faculty. They’re more supplemental. We have online certificate programs for new instructors in teaching and learning. We have programs that are for those to learn about assessment of student learning and institutional effectiveness… So, again a lot of them are really meant to supplement what campuses are doing. I just started in this role about three and a half years ago. I joined the staff of CPD permanently in July, leaving my position at Brockport. So, we’re really starting to ramp up the number of programs that we are offering.

John: You’ve been conducting focus group meetings at a variety of SUNY campuses on professional development needs of faculty. What have you found to be some of the most pressing concerns and needs?

Chris: Yeah, so let me just back up a second… just explain why I’m doing it. Back in July, as part of our program development, we sent out a survey to people work in faculty professional development and asked them what their interests and needs and concerns were… and we learned a little bit from that survey. We learned that the most pressing concerns are in three areas. Number one, helping faculty adopt innovative teaching and learning practices. Over sixty percent of the people who answered the survey indicated that that was something they were very concerned with. And then coming in number two, with just over fifty percent of the respondents, they were looking to help faculty instructors better design courses using sort of data and best practices to improve teaching and learning, and then third, we found that faculty developing folks across the system are really interested in increasing faculty and staff knowledge and participation in their professional development programs. So, those are the concerns but how can we address those concerns, and what is behind those concern? And so we came up with the idea of doing some focus groups across the system. We have 64 campuses, so we didn’t think we could do them at every campus, that would take it a little too long. We looked at the regions where people answered the survey so we wanted to first go out to the places where a number of people had answered the survey and clustered around there. So, so far I’ve done five of them in Plattsburgh, Purchase, Stony Brook, Genesee Community College and Buff State. I have another one scheduled in March at the CPD in Syracuse and I might do another one in Albany later in the spring, and so far I’ve had about a third of the 64 campuses represented in these focus groups. Most of them have been anywhere between five and 12 people and they all been pretty much nice and balanced between those at the community colleges, those at the comprehensives and those at the university centers. So, like I said I’ve been doing them over the last few months and I’ve learned a number of things.

We were joking before we started talking about this that resources are a top concern for many who do faculty professional development, but you’ll find though that it’s not the case…. and those who don’t have a lot of resources probably don’t want to hear this necessarily…. but there are some campuses where faculty professional development is really well resourced and there are a lot of funds that campuses are extending towards faculty professional development. The one thing I found consistent across the campuses, while some campuses don’t have a lot of resources for faculty professional development, you’ll find that the resources kind of followed trends. Many campuses, especially those that have online teaching and learning as a strategic priority, are investing money in supporting faculty professional development and online teaching. Lately we’ve seen a lot of resources follow other trends. Innovation… innovative teaching and learning… so you are seeing a lot of funds now expanded towards that. A lot of the campuses do follow what the system is doing and system does provide money for professional development for innovative things. So, one example would be the open educational resource initiative that’s going on now, where there is some money for that.

The second concern that faculty development folks seem to have is the fact that the folks they’re working with, the faculty, the instructors, teachers, professors… their needs and motivations for participating in faculty professional development vary… widely. Many that they work with are ahead of the curve in terms of adopting innovative teaching and learning practices and highly motivated to participate in their programs. These are the folks that come to all our programs, come to all our workshops, come to our professional development days. They’re the folks that are excited about what we do… and they’re the students that sit in the front row of the classroom, right? and they’re great and we love to work with them, and they give us great feedback on our programs and help inspire us to do what we do. However others, and I would say the majority of faculty, need incentives to participate in our programs like this… and so those are the folks that I think we spend a lot of our time thinking about.

What sort of incentives should we provide folks? Obviously, where resources are somewhat limited, we can’t just pay people to participate in faculty professional development. We have to be a little bit more creative in the incentives that we provide to folks.

The other thing about instructors is that, and again because they kind of are a heterogeneous group, some are very skeptical about what we do, and about innovation in general in teaching and learning. I think most professors tend to teach as they were taught. Typically, if you get a PhD or you advanced to this level of your career, you will be probably been successful no matter what others did to you… but, we have to face the fact that our students are not us. Most of our students will not end up as faculty as professors, and so the things that work for us aren’t necessarily going to work for them, and so the skeptical folk…. I mean again some of them are skeptical for a very good reason. There are a lot of innovative teaching learning practices that don’t bear fruit in the end. I won’t point anything out specifically. I don’t to alienate anybody from their favorite pedagogical technique… but if you look at the literature, there are certain things that just don’t work out in the end. So some of the skepticism is worthwhile listening to, while other skeptical attitude, you just gonna have to ignore, and figuring out which is which is a tough thing for these folks to do.

The other concern that faculty development folks have is that… and this is a concern I’m sure for everyone is that there’s limited time for professional development. One of the we’ve seen in higher education over the last couple decades, is the decreased number of tenured and tenure-track faculty, and what that essentially has done is increase the burden on those who are tenured and tenure track faculty in terms of service, especially. They are required to do more and more than their colleagues 20-30 years had to do, because they’re just fewer of them to share the load. There’s also just more going on on colleges and universities than 20 or 30 years ago. I mean there are just the programs and things that we have to keep afloat, are just increasing year after year… and then, of course, the fact that there are fewer tenured and tenure-track faculty means that more classes are taught on many campuses by adjunct part-time contingent faculty… and they don’t necessarily have time for professional development, because they’re either teaching five… six… seven… eight… classes or they’re working full-time in addition to the classes they teach. So all the folks that we would like to participate in faculty professional development, just don’t necessarily have the time to do it…. and so, trying to figure out ways to do just-in-time kind of professional development preoccupies a lot of folks who do faculty development. So, we had folks on campuses talk about going to departments and framing the professional development activities in the ways they think in their disciplines is one successful strategy.

Another challenge, and I have three more that I’ll mention, is we have folks who do this work who are directors of centers for teaching and learning….but not all campuses have centers for teaching and learning. Only about half have them ….about 30 to roughly 32 of 64 campuses, so those folks kind of do their work in the guise of a center… some of this work happens where faculty get release time to do professional development work or to lead faculty development sessions and that sort of thing and so that’s another population. A lot of this work is being done by instructional designers who are initially hired to just support online faculty, but they’re also not going to turn away faculty who aren’t teaching online if they ask for help…. and, in many cases, librarians are helping faculty with their teaching and so we have a heterogeneous group, and that kind of makes it hard to sort of say, okay… this practice will work for you… because you’re in a totally different department reporting to a different line than someone else… and then another challenge is the choice of whether or not to go deep with professional development… so… go and meet people one-on-one and do consultations, or go wide and schedule workshops… come up with online resources for folks to take advantage of on their own time…. and so this is a choice that folks have to make all the time, and the challenge behind this…. and the thing that I think compounds this challenge… is that we don’t really have a lot of good assessment data, impact data, about what works in faculty development. There’s some out there, but on the campuses, what most people look to to judge success are the numbers of people served. So, I know, for years at Brockport, when I would do my annual report, I would just count how many people we helped or we served through events or consultations, and of course the big events so that lots of people came, were the ones that Jack our numbers up… but that didn’t really say whether or not the folks who came to those events got anything out of them necessarily. I think we have a lot of work to do around assessing what works and what doesn’t in faculty professional development, and that’s going to help us in the long run hopefully improve the type of things that we do in the campuses… and the last thing… the last kind of concern… the thing that a lot of folks said that they tried to do and they struggle to do, was to look at faculty on a continuum, that faculty are a heterogeneous group. Some folks come and they are ahead of the curve… early adopters… on top of the literature of teaching and learning… and they are the ones that like I said are easy to serve and are very eager to learn from us and and to participate in our workshops and activities and that sort of thing. But many faculty, because they’ve got so many other things to do… their own research… many courses to teach…. lots of service… aren’t really on top of what works in the classroom, and it’s not always the best approach to pitch programs that are way ahead of where they’re at. You have to meet faculty where they’re at… so what’s innovative for someone is what you kind of have to define as innovative to that person. It not be innovative to you to say… I don’t know use clickers or something like that… but it may be innovative that person… and you alienate them if you try to make them think that they have to adopt techniques that are beyond where they’re at.

The problem is, this isn’t the most efficient approach and when you’re looking at every individual as a unique case, you can get bogged down and not really be able to create these programs that you could pitch to a general audience.

So the other challenge is that there are a lot of administrative and organizational churn. At Brockport, when I was there we had five Provosts in ten years… and with each Provost comes a new set of priorities and a new organizational structure to report to, and other people that we need to work with…. and so that also compounds the challenges. So I’ve been talking for a while, I bet you have other questions, but as you could see there are a lot of concerns that faculty development folks have… and they’ve been very generous and sharing with me so far.

Rebecca: So when you’re looking at the concerns and in these discussions, did some of these folks provide some information about innovative things that they’re doing to address some of the concerns that you just summarized for us?

Chris: Yes. Definitely, and I think a lot of them fall into a few categories. So I’ve talked before about either going deep or going wide. So most of them all do, and value, the one-to-one consultations they do with faculty, and so they are all still doing those. That’s where they really are able to have an impact I think, but others are trying to reach you know wider audiences through a variety of methods. For example, in Suffolk, they subscribe to this program called Monday Morning Mentors, and so all faculty, regardless of status…part-time, tenured, tenure track… they get an email on Monday morning with some kind of teaching tip directed at maybe something at that point of the semester… and they all get that, and it’s kind of low-hanging fruit for them. I think it’s easy enough for them to do it, and I think they subscribe to a service that gives them those tips. Other campuses are doing a lot with…. those are lucky to have some resources…are doing a lot around grants. So we talked about incentives and the need for incentives… and some faculty need that, and so some campuses are providing incentives to their faculty for… adopt a technique or redesign a course around something. For example, Farmingdale does this. They’ve had one around hybrid teaching and learning and other programs. Buff State has a program where they give instructors some money and some support to redesign a course around an innovative teaching technique.

Campuses are still doing day-long traditional professional development events with speakers and facilitators. Those are still popular… maybe not the most innovative things in the world, but I think they work really well… Some of the more kind of unique approaches I heard about related to regional collaboration. So, I know you guys at Oswego had that book club in which you invited folks from other campuses to participate. That to me is something that I hope to facilitate a little bit more within the system… because there’s something to be said for working with others on other campuses, to get your juices flowing and to hear about other things that others are doing. I think especially for those that are on campuses for a long time, you start to hear the same things over and over… and it’s helpful to sometimes sort of talk to folks outside of your campus… especially for faculty that teach the same discipline on other campuses….and that’s sometimes that’s a good way to think differently about your teaching…. and then the last thing I’ll say I’d like to see more of that I don’t see as much of… although Buff State just recently has been doing this.. and are planning on doing more of that is supporting the scholarship of teaching and learning through very informal ways, so providing incentives for faculty to take the time to do scholarship around their teaching… and supporting that… and rewarding that on the campus.

John: How have you seen the needs for professional development change since you’ve been involved with professional development?

Chris: Yeah, so it’s interesting, so I’ve been doing this work for about 13… 14 years… and when I started, there was a real hard boundary between those technology training and then those who do sort of teaching and learning professional development…. and so I’ve seen that slowly break down. The people that have attended the focus groups I’ve conducted so far have been both reporting to CIOs on campuses but also those who report to Provosts and so those boundaries…. they are slowly breaking down in which those who are using technology in their teaching… that’s not considered really separate from classroom based teaching. It’s all kind of mixing together and I see that as a very positive development because I think that we can’t avoid technology. now in teaching and learning…. and not that everybody has to utilize every single piece of technology that’s out there, but considering how it impacts learning… everyone should consider how it impacts learning, and I think we need to have all our tools in our toolbox to help folks do that. The other thing I’ve seen change is that innovation narrative, as I spoke about earlier, is really driving professional development conversation, and everybody seems like innovation and being innovative in higher ed is what everybody wants. I don’t think everybody knows exactly what they mean by that, I think some campuses using that term and not really I think defining what it means for that campus to be innovative… I think it’s a good thing that we’re considering what’s the latest and greatest out there in micro-credentialing or think of all these sort of innovative practices, I think campuses really need to think about what that means for them and teaching and learning centers are actually ideally positioned to help facilitate that conversation. Another change over the last 15 years or so is that active learning is no longer openly questioned by folks, When I first started I heard faculty say all the time: “Well, I learned great when I was in lots of lecture based classes and why can’t our students learn through lecture?” and now I think folks are recognizing the value of active learning techniques and we don’t have to throw away lecture… and I think there are a lot of good articles and research out there that shows that some lecture is valuable, but I think everybody, for the most part at least, now doesn’t openly question the value of active learning techniques. One thing I haven’t seen as much of, that I was hoping at this point we would see more of is assessment of student learning, scholarship of teaching and learning in classroom research being more widely adopted. I still don’t see a lot of that going on. There’s lots of educational research out there by people who are in the fields of Education, but for faculty to be doing their own research is something that I would like to see more of and I was hoping to see more of.

Rebecca: Over time, have you seen any changes in faculty doing more evidence-based practices?

Chris: This kind of goes back to the idea of innovation driving the narrative now. I think faculty do care that what they’re doing maybe has some basis in literature, but I don’t know that they’re actually diving in and doing the research themselves and sort of: “oh, this I saw this, I read this article… and there’s some research on this, and this worked… and I’m gonna adopt it.” I don’t necessarily see that that much, because I think once you start doing that I think you’re gonna start down that road of maybe doing some basic research yourself in your classes. I’d like to think that that’s the case…. but this is just an opinion, I don’t have any research to back this up… but my guess is that most fact that he still choose to do what they do in the class because their colleagues are doing it or they hear about it offhand. They’re not doing exhaustive literature reviews to make those decisions about what they do in the classroom.

John: But that does open up a bit of a lever for introducing new techniques, so that professional development centers don’t necessarily need to reach all the faculty… if they can reach some influential faculty member in the department and help them introduce more effective practices… quite often other people will adopt it… especially with the growing culture of assessment. If they see the results are a bit stronger or sometimes if someone introduces something more effective and students can see that it’s made the class more effective, they’ll often ask other faculty in the department to perhaps try something similar. I know that’s happened quite a bit here in a number of departments.

Chris: Yeah, I didn’t mean to suggest that hearing it from colleagues was not a good way to go. We have to leverage that, as you’re saying, John. I think we have to figure out ways to make those connections that faculty make with others a little bit more… almost intentional… and leverage that. I think it’s great when that goes well…. when someone hears something from a colleague and then they adopt it and they improve upon it and then they talk to their colleague about: “how here’s what you did… here’s how I did it… and then… oh…” and you listen…. It’s this iterative process… where that good expands out like that, but, as we all know, I mean we can also have bad expend that way as well, right?

John: Yes.

Chris: So if they’re not being critically reflective about what they’re doing….. you could imagine a scenario where folks latch onto something and it doesn’t really work well, but everybody’s doing it, so I guess I should too. So I’d like to see us leverage that and then there’s different ways we could do that.

John: …and those conversations don’t always take place in departments. I know here, periodically…. at least in my division in arts and sciences, the Deans have sometimes encouraged departments to have retreats where they discuss effective practices, and have these discussions… but they don’t always take place.

Chris: Yeah, and that was actually something that I would like to see more of campuses doing…. to put these discussions at the center of all their initiatives when something comes down, especially…. say…for a general education program reform. Finger Lakes Community College recently went through a general education program reform and they put faculty development at the center of that, and so… not only did they rethink how to deliver their curriculum, but they also thought simultaneously how can we help faculty improve their practice to meet the needs of this new curriculum… and I heard that again and again in the focus groups… that, the programs that faculty got the most excited about were the ones that were really tied very directly to what the college was doing strategically, and so they kind of went hand in hand. This is where administration plays a big role… and developers recommending this to their administrators is, I think, a good thing to do…. because… sometimes, they understand that faculty development is important… but they’re not faculty developers… they don’t do that… so they don’t know exactly how to build that into initiatives on the campus so that it works well. A lot of times, it’s just tacked on like: “okay, we’re gonna have this initiative to do this… and… oh yeah, we’ll have to have some professional development, so we’ll figure that out later. That’s not the way to do it. You kind of have to think about at the beginning for it to work well.

Rebecca: In addition to teaching and learning centers or professional development centers on campuses, what are some other ways faculty members can expand their professional development?

Chris: Yeah, so I think networking is probably the best way to do that. Try to find others… not necessarily on your campus, but in your discipline. All disciplines, for the most part, have either conferences or journals that relate to teaching in that discipline… and this is the way faculty think, right? They think first as whatever they teach….you’re a political scientist… you’re a psychologist… you’re a biologist… that’s how you think first. So, trying to find others who think in those terms… and think about teaching in those terms… I think is the first step for everybody to make. But, obviously, I think teaching learning centers are important, and I think all campuses should have them… because it’s nice to have that place where everybody, can congregate around teaching and learning… and have something for these initiatives to revolve around… Beyond those two things, I think teaching conferences are great… But, I think teaching and learning conferences are great mostly as venues for faculty to present their work… giving them an opportunity to present scholarship on their teaching. They can be good for learning as well, but I think learning in this area happens best when you’re really, again doing the research in your classes themselves first.

John: We’ve been doing reading groups here for the last three years… and one of the things that really surprised a lot of the participants…. because we had people coming in from all across campus… in very diverse areas…. is how common the problems that they were having were… and how many solutions people in very different disciplines had… because they wouldn’t have thought to look at that first… and your point about working first within the discipline is a very good one. Because, people are more comfortable if they hear it from other people teaching the same courses or very similar classes.

Chris: It’s funny… because that advice that I would give… going to your discipline first… it does run contrary to the idea that, as you just said, John, It is true that teaching and learning is teaching and learning… and the obstacles that people face are very similar even in very diverse subjects… but, like you said, you have to bring people along where they’re at… first.. and it’s good to see something a little bit familiar first…. and then kind of move on… and then maybe learn from others and other disciplines. So, yeah… it’s one of those things I always get pushback from folks a little bit… we know that you could learn from folks in other disciplines….and, in fact, it’s good to do that… but I think, for many people, it’s good to maybe start off looking in their discipline.

John: …Because when people try something new they have to move outside their comfort zone…. and making it a little more comfortable initially can often help.

Rebecca: I think we saw that a little bit in our reading groups as well… when you’re reading an example or something and it’s not in your discipline and you can’t quite envision how it might apply to your content area. So that’s where I think it’s really valuable to find people in your discipline who can bring some expertise to the table.

John: …and even just the comfort of knowing that other people have done these things and it’s worked… and they can provide you with examples and sometimes a more packaged solution that directly applies to your discipline.

Chris:Yeah, that’s actually one of the innovative practices that… actually they’re talking about this at Buff State…. and they have folks get together and and share resources and repositories, They’ll create some kind of learning activity and then they’ll put it in a repository and then you could adopt that… you could amend that… and I think having those resources in your discipline leads to that culture of sharing first…. and once you start down that path you can, like I said, look to other disciplines to learn from and adopt practices from. It’s like I said a way of moving along a continuum. You have to start where you’re most comfortable and then push yourself gradually to other areas where you’re not as familiar.

Rebecca: What about faculty who are those early adopters? …or who might be a little ahead of the curve? How do we make sure that we continue to engage them?

Chris: That’s a great point. I think those are the folks that you need to have on your advisory board for your Center. Those are the folks that you’re going to bring in to do workshops… but you also have to mentor them a little bit… because I think, like in a class where you have those advanced learners, they can sometimes turn off those who are more novice… because they appear to be know-it-alls… or “I never can know what that person knows.” So, with some careful mentoring, I think those are your folks that you could turn to to help deliver programs and workshops…. maybe facilitate learning communities or reading groups…. really try to harness their enthusiasm. They’re also the folks that you turn to when you want to have folks meet with administrators, and broker or at least advocate for your programs because administrators always like the examples of faculty who are doing innovative things… and so those are folks that we want to send to meetings or advocate for resources for your Center.

John: You mentioned learning communities. One of the things I’ve heard from many people at Brockport was how effective the learning communities were at Brockport. Could you tell us a little bit about how you arrange them? …how they work? …and what sort of incentives were provided to faculty to participate in those?

Chris: Yeah, sure. The way the faculty learning community program worked at Brockport was, it was very faculty centered. We would solicit applications from those who are interested in facilitating a learning community on a topic. We did not restrict the topics to just teaching and learning… We opened it up to, say, research methods types of learning community topics. So we had a couple run on qualitative research methods… quantitative research methods… so we didn’t provide a lot of guidelines around what topic we were interested in learning about. Which is a little bit unique. There are many colleges and universities that organize them around themes and they come up with the themes in advance. So we decided, well… let’s see what the faculty want to learn about… let’s get that full list to put out there.
After that step, we would advertise the proposed topics to the faculty, and even before they ran, we asked them to sign up, so that it would help us decide which ones we would select and run based on how many folks would sign up for them. In an average year, we’d usually have maybe six to ten proposals…. and of those six to ten maybe only five to eight had enough people for them to run. We would usually require at least like eight people to sign up for them to be considered. Then we would look at them and then we would make the decisions. Part of my advisory groups role was to make those decisions about which ones we would fund.

The person who proposed it would be paid as a facilitator to make sure that FLC [faculty learning community] would accomplish what it had proposed to accomplish throughout the year… and mostly that just they would meet every couple of weeks… and then I would meet with those facilitators as a kind of mini-learning community once a month, just to check in and see what they needed… and basically just talk together collectively about their progress during the year.

They would run for an academic year. We’d actually start working over the summer. I’d have a full-day orientation for the new facilitators usually in June, before the year they would run… and then the FLCs would meet every couple weeks. One of the things I always stressed with the FLCs, and I think one of the things that made them work well, is that the goal of FLCs was the professional development of the members. They weren’t really required to come up with a collective deliverable… that’s the job of a committee… and I want to stress that FLCs, if they work well, can’t be seen as committees. They have to be seen as a group of folks who are looking to learn together… and really learn something… and a benefit to their own practice… At the end, some of the FLCs would pursue group projects, but that was their decision. It wasn’t something we would impose upon them… and my other concern with that is I didn’t want the FLC program to be seen as a vehicle to accomplish initiatives on the campus…that say, the administration wanted to accomplish and sort of co-opt them. …and there were occasions where we would get applications for FLCs, where a clearly a Dean had put the bug in the ear of a faculty member and said: “Hey, propose an FLC on this topic.” …but faculty would never sign up for those.


So they would never run anyway… but, yeah…. it was a great program. They did have a little bit of a budget… a few hundred dollars per member… that they would pool collectively… sometimes to send someone to a conference, or a couple people to a conference or buy us some materials or buy a piece of hardware or software to help what they do.

If you go to the Brockport website and search Brockport faculty learning communities, you’ll find all the ones that we did… and their end-of-the-year reports we would put on our website, so that we would share their learning with the entire community. My favorite part of the program was at the end of the academic year, we would have a end-of-year luncheon. We invited all the FLC members… usually the Provost would come… and then we would invite the people that were going to facilitate and/or be part of the next year’s FLCs to hand the baton off to them, and so they can hear about what those FLC accomplished and what those folks accomplished and then inspire them to do their work the following year.

John: That’s a very nice structure.

Chris: Yeah, it worked really well. Thing that was great about it was that it kind of helped me do my work. There were topics that faculty were concerned about,… say: “Hey, propose a FLC on that topic.” ….and it would really start to generate this momentum around faculty developments in those areas… that they would take the ball and run with that. I was a center essentially of one, we had one other full-time staff member, an administrative support person, so there was only so much I could do, by myself and with only a couple of people, so these were ways for faculty to own their professional development… faculty centered professional development is the way I would always look at.

John: Excellent.

Rebecca: So what are you gonna do next?

Chris: One of the things that I would really like to do… on the top of my wish list for the system, is to take almost that FLC model… and apply it in the system. Now, I don’t think it makes sense to do it exactly the way we did at Brockport and just sort of say: “Hey, we’ll have system-wide FLCs.” I don’t think that necessarily would work, but I would like to have some kind of a network of faculty who are involved in teaching and learning projects… maybe scholarship of teaching and learning projects… or action research projects. Something where they’re doing a investigation of a teaching and learning method…almost like SUNY teaching and learning scholars… or something like SUNY teaching scholars…. or something like that… where they collectively work together… most of its going to have to be virtually… online… where they’re working together… supporting one another… getting support from the system somehow…. and at the end presenting their work maybe at CIT [a SUNY Conference on Instruction and Technology] or another sort of system-wide program, and then we can gradually build this cohort of SUNY teaching scholars and have maybe them be recognized by their campus somehow. I haven’t figured how we’re gonna do that yet, but I’ve been putting bugs in the ear of anybody they will listen about that. I think that we should do something like this.

Rebecca: Sounds pretty exciting. I think there’s desires on individual campuses for a cohort like that, but maybe there’s not always enough of a cohort on an individual campus… so having something system-wide could be really beneficial.

Chris: Yeah, exactly and again I think just that benefit of hearing from people on different campuses… just these focus groups I’ve been having… they’re just one shot, two hour deals, but having folks come together regionally, and really facilitating that regional conversation… I think they’ve been sort of saying: “Oh, this is great… we should do this more often.” So, like I said, I’m hoping this to spur those connections and I see this as just another opportunity to do that.

John: Excellent.

Rebecca: Great. Well, thanks for spending some time with us today and sharing your expertise and getting us all thinking a little bit more about our own professional development.

Chris: My pleasure, yeah.

John: Thank you, Chris.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

17. Online learning

Enrollment in online classes has grown steadily over the last few decades. Today, over 30% of college students enroll in at least one online course. In this episode, we discuss the evolution of  and possible future directions of online learning with Greg Ketcham, the Assistant Dean of the Division of Extended Learning at SUNY Oswego.

Show Notes

Books used for SUNY-Oswego reading groups (referenced by Greg):

  • Brown, P. C., Roediger, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press. (one of the books used in a reading group at SUNY-Oswego)


John: Enrollment in online classes has grown steadily over the last few decades. Today, over thirty percent of college students enroll in at least one online course. In this episode, we examine how online learning has evolved, and is continuing to evolve, to better serve student needs.

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Today our guest is Greg Ketcham, the Assistant Dean of the Division of Extended Learning at SUNY Oswego. Greg focuses primarily on programs serving adult learners. Greg is actively involved in Educause, the Online Learning Consortium, and the University Professional Continuing Education Association. Welcome, Greg.

John: Welcome.

Greg: Thank you, Rebecca. Thank you, John.

John: So today, our teas are…

Rebecca: English Afternoon.

Greg: Actually we were just joking about this before we started that I think we have to change the name of the show to “Coffee for Cognition.” So, I’m drinking coffee.

John: …and I am drinking Harry and David’s Bing Cherry tea.

Rebecca: It sounds like a mouthful.

Greg: Well, if you’re going to be giving a plug, I have to give a plug then to Recess Coffee because it’s my favorite coffee roasting vendor in Syracuse.

Rebecca: I say it’s a nice, local, upstate…right?

Greg: Absolutely.

John: So you’ve been involved in online education for quite a while… since sometime last century…

Greg: You’re making me feel really old, thank you.

John: …both as a student and as an administrator. From your perspective, how has online education evolved over that time period?

Greg: Sure. That’s a really good question. Now granted, I really was not there at the dawn of time of online learning, I want to be clear about that. But, actually, as an outside casual observer… well, maybe I was…at least in terms of the late twentieth century…. because if we really think about what distance learning looked like before the advent of the internet, it was really what I refer to as ITV, Instructional Television. Sunrise Semester being probably the most famous example of that, that I think is still in production and out there for years and years and years out of New York City. And obviously, at the advent of time based on the technology, there was a one-way transmission of knowledge. To say that it was really education, It was not a bidirectional experience in any way. Moving forward into the 90s, with the advent of the Internet and shortly thereon, the World Wide Web, suddenly we begin to move into something that becomes much more bidirectional and I love these acronyms because nobody remembers them anymore: MUSHes, MUDs, multi-user domains that were largely text based, that kind of go back to the Adventures of Zelda, in a way, if we go back and remember that far back in terms of what an interactive game experience was like. So those were text based and as we begin to scroll forward, really to where we are today, the changes in the underlying technology change the kinds of interactions that we can have.

So, largely speaking, when I started, which was thirteen years ago here, we were really looking at online learning in a purely asynchronous form and really, of course, we said that was for learner convenience… which I think is still true. I think that’s the reason why it is asynchronous, because the typical online learner is an adult and is a part-time student… is juggling many things in his or her life and learning is a part of that… so, it has to be based on when it fits into their time budget. So where we are, compared to again let’s say 2005. 2005, we had limitations in our learning technology. It was largely text based so the forms and interaction between instructor and student were predominantly text based. Today, we’re at a point where those kinds of interactions can truly be multimedia. We can incorporate audio… we can incorporate video… we can have online chat sessions together… video sessions together. So we’ve really broadened our palette in the ways in which we can interact and communicate and create those learning spaces. That to me is a really big deal and I think I’ve become much more of a convert to these sort of multimodal means, being a student again.[Laughter]

Yeah, eating my own dog food as they say, right?….being the continuing adult learner… I’m actually a part-time learner in a doctoral program and it’s always good, actually, to flip the equation, if you can, right? To go from instructor to student and remember what the student experience is like, first of all. To be in an online course and go, “Oh God, really? This is what it’s like?” Now, that’s not true of all my online courses so far, I just want to be clear about that. But what I did experience was the fact that we injected group work into one course and, of course, I’ve never personally loved the idea of group work, period. But leveraging Google Hangouts, leveraging Google Docs, it was transformative in the sense that it really was creating a sense of community. That’s something we’ve always strived for in online learning and moving outside the bounds of that purely asynchronous construct– it was just truly transformative to me in my thinking about it. And as we move forward, I think we have these opportunities to leverage the technology and not be so trapped in the box of the tools.

Rebecca: What I’m hearing is… early on, even when it was online, it was still traditional correspondence classes, right? So, you’re corresponding… but now, it’s community…

Greg: RIght.

Rebecca: …and I think the difference in those two words is really powerful… and even just thinking about what they mean.

Greg: I think that’s true, and it’s interesting because you’re seeing this right now in our media. I want to say “the media,” about our media that we consume about higher ed. Western Governors University, being on the leading edge of doing competency based education, it tends to subvert our notions of what online learning is…. maybe not necessarily true for Western Governors, but in some models, you’ve got kind of a pay-one-price, consume-all-you-can… it’s not bounded by the conventions of a semester. It’s not in the normal control functions of an instructor helping to manage the students ‘learning. So conversely then, the Department of Education at the federal level is saying, “You know what? This looks like a correspondence course, and if this is a correspondence course, then it doesn’t qualify for financial aid.” So there’s a lot of implications in there for us, in terms of thinking about the design of learning…. unfortunately, secondary effects that impact the student in terms of their financial ability to take the course. But, to your point Rebecca, I think the difference again between correspondence and community…. because traditional, fully asynchronous learning is always sort of time modulated and time delayed. It does look more like the conversation between John Adams and Thomas Jefferson, right? There’s a lot of thought going on and there’s a lot of reflection going on, based upon reading which you’ve said, “I’m going to come back with a response,” but it isn’t necessarily dialogue…. and dialogue, I think, is part of a community because it’s really of the moment. Things build… things are reactive. I think that’s a potential change. Yes.

Rebecca: Seems like a way that you might be able to build mental models more effectively because the learners are interacting with each other much earlier on and they probably have more similar mental models then an instructor….

Greg: Right.

Rebecca: There’s a different relationship there that I could see how that could be really beneficial to help overcome some of the misconceptions and things that students might have because they come out a lot sooner.

Greg: Exactly.

John: But also with the introduction of more group work in online courses, even if the course is designed to be fully asynchronous, the work within the groups does not have to be, so that the student, as she said, could be using Google Hangouts soon or other tools and working collaboratively at times that they arrange within the group.

Greg: Right, and again, I think what you’re looking at is you’re beginning to model work flows in collaborative strategies that our students would use out in the world once they graduate. All of us do things across time and space that are outside the university with other colleagues. In how we approach that are those very same tools. So, why not expose them to your students early on? …and say “This is how it works. You don’t have to get together in the library every day… You really don’t have to.” The challenge tends to be when you’re blending a group of learners…. between the target audience, those adults who aren’t here, and our traditional students who are here…. because when you do propose an assignment to the students… it’s group work… what do they do? They say, “OK, let’s get together in the library at three.” ….and the student who’s not here has to speak up and say: “Excuse me, I’m a hundred miles away. I really can’t do that. Let’s think about some other method to make this work.”

Rebecca: I’m definitely a strong proponent of capitalizing on the idea that these are professional skills to develop. So, even in my non-online classes, I use tools like that for group work: Slack, and things for group discussions and things… and I think that when you frame that for students and help them realize that it is a professional tool and a professional opportunity, they do buy in… and so I think there’s a real positive in this online environment to encourage small groups and things to start using these tools because then they’re seeing how it’s going to benefit them in the future. Obviously, non-traditional students would catch on to that much sooner. We have to be probably a little more explicit with more traditional-age college students about the benefits, and that it is a professional skill.

Greg: I think so. I do think we have to move beyond the culture of the familiar uses of social media, which most of our traditional age students are very proficient in, into more professional uses of those kinds of technologies. So, it’s not just dropping “LOLs” all over the place [LAUGHTER], it’s communicating in a more meaningful and deep way…. and I think those are the skills that we’re helping them to learn in these kinds of online environments.

John: A lot of faculty, when they first start teaching online — and I started teaching online a couple decades ago… sometime last century, too… when they first do it, they tend to try to replicate what they were doing in the classroom, and then they discover that doesn’t work very well… and there are these rich tools out there that can make interaction much more effective. How do you work with faculty to try to transition them to alternative teaching methods, and things that work better online?

Greg: Right, when I did faculty development, in working with instructors who were new to online learning, I would pose a scenario something like this to them: “Okay, you’re very comfortable with standing up in front of your class and delivering the content and doing a lecture. Now imagine your students are next door behind a wall… they can’t see you…. they can’t hear you…. How are you going to teach them?” What you’re doing is you’re imposing those barriers of time and space right away to begin to reframe the conversation. and in reframing the conversation, one of the things that I found initially to be very helpful… It was pointed out to me by one of our colleagues here that I was actually espousing a very well-known design theory which is called backwards design (from Wiggins and McTighe, I believe)….and the notion is that we begin the conversation by talking about: “What are your students supposed to do at the end of this course? What is it that they’re able to do?” I rarely use the phrase learning outcomes because that sounds rather abstract, but if you put it in a concrete observable frame of “How do you know that they’ve learned what they’re supposed to learn?” “What are those artifacts?” is my favorite phrase. If we can start with: “What are they supposed to be able to do? How do they show me that they can do it? What are the artifacts? What are the outputs? Oh, those are the outputs?” Those are the actual learning activities. How do we scaffold them to those learning activities? What are the instructional materials and activities that precede that?

You deconstruct. You deconstruct what faculty think they understand about their teaching process. Faculty come in, many times, with the frame of thinking… truly about content delivery, which is perhaps not doing yourself justice in terms of your skills and what you really bring to the classroom, but if you think of yourself as simply as an amplifier and conveyor of content, then one of the things we do is begin to change that around… and discuss what a facilitator does versus a pure instructor. I think that’s part of it too.

Rebecca:I would imagine that you also have many conversations about audience… that we don’t necessarily have in more traditional face-to-face classrooms…. thinking about who are your learners? what do they come to the table with? and what are their other life responsibilities and things? To find that balance and how things might work.

Greg: Right. You do try to interject… to talk about what the audience looks like… and I think also Rebecca, to your point about audience behaviors and audience constraints we can discuss things that we are now gleaning from research. So, for instance, you may say to me: “Well, you know, Greg, I’m just gonna record my 55-minute lecture that I do every Monday, Wednesday, and Friday… and we’re good to go.”

John:…and a lot of faculty come in planning to do that.

Greg: Sure… sure… There’s my favorite animation out there… which still exists and I forget the acronym of that that animation program… there was like these little furry creatures and the professor comes in talks to the instructional designer and says: “Hello (in this very cultured British accent), I am going to teach my course online. I have recorded all of my lectures.” …and you have to break that down and you have to say: “Well, current research shows us that the attention span of what people are looking at, it’s probably about in you know six to ten minutes.”

John: Actually, we just did a podcast on that. We recorded it just a couple days ago, and it will be released just a week before yours. That research turns out not to really exist or not to really show that.

Greg: Can I retract that statement?

John: …but people have been saying that for decades. So, it’s one of those myths, like learning styles…

Greg: Right.

John: …and that Dale’s Cone of Learning….

Greg: Oh, yes…

John: ….that people just keep recreating.

Greg: …and you know how much I love Dale’s Cone of Learning. That’s just a fantastic fabrication… two things that are mashed together…

John: …including citations that don’t exist…

Greg: Mm-hmm. Exactly.

John: …and articles that were never created in journals that weren’t there.

Greg: Exactly.

Rebecca: Yeah. So, apparently the attention span stuff is too.

Greg: This is something, that I think, we have to all be aware of… that we’d like to refer to these things… and obviously I’m guilty of this too…

John: …and I have too.

Greg: ….without going out and stopping and asking ourselves: “Well, is this still true? Is this still current?” ….and we all know many people who will talk to us about learning styles and you just want to go: “Okay. Stop right there. Don’t say another word. Let me hand you this article from Daniel Willingham.” We do have to be careful about that, but I also think you have to look at it in the construct of, for instance… now I am doing this all the time at a personal level all the time…. I find this… and it probably says something about my mental state and how it’s devolving… but if I am watching even a Netflix video I will often throw it into 1.5 speed or 2.0 speed. I will do this with podcasts, and if anyone out there is listening right now, feel free… speed me up. I won’t sound any more articulate, I’ll just go faster… but we do this compression, because potentially we’re time challenged… potentially our attention spans are impacted. So, I do think we need to look at those kinds of behaviors. You can look at the log behaviors on the learning management system, and students are popping in and out… popping in and out…. popping in and out…

John: Which is good if they’re engaging in spaced practice.

REG: Right.

John: …but that’s not always the case.

Greg: …may not be.

John: and there are issues. Even if there’s no magic attention span issue, there are issues with cognitive loads.

Greg: Right.

John: …and that chunking things into smaller more manageable chunks, especially for beginning students is really effective, and that’s where a lot of the online classes tend to be focused.

Greg: Exactly. Let’s go back to the typical statement about multitasking… multi-processing… A lot of the research on that is really much more granular in terms of the kinds of parallel tasks that can be effectively executed with multiple inputs versus this notion of “I’m facebooking and I’m in my course and I’m listening to a podcast….” because there’s obviously multiple inputs and we process all those things differently. So you can’t just crunch that and make a blanket statement about it. But, you can find fascinating things coming out there… out of cognitive science research… that says, for instance (I won’t get this right because I would have to go back and listen to this story)… but they measured the effects of retention of white text on a black video vs. black text on a white video, and there were significant differences. So, looking at that as an input and informing design-based practices when you’re creating this media is incredibly important….

John: …and part of the issue is when things are harder to read, students have to process it more and they end up recalling more of it. There’s even studies that show that if you use, and…

Rebecca: Stop!


Rebecca: Just stop!

John: …if you use a really hard to read font, students will recall more of it later.

Greg: That’s fascinating.

John: It may not be a desirable difficulty, but it does result in more retention.

Greg: OK. So….

Rebecca: so but they might not read the whole thing, because it’s difficult to read….

Greg: So, conversely, are there any studies measuring the effect of…. let’s say, an easy friendly format that’s easier to read… and I’m thinking Comic Sans, obviously. What’s the impact there? Do we know?


Rebecca: Well, actually…

Greg: Can we do a study on that, ‘cause I would love to. [LAUGHTER] I would love to build a whole course in just Comic Sans.

Rebecca: There are some studies about Comic Sans, but they’re always in these isolated situations.

Greg: Yeah.

Rebecca: So, yes, the letter forms are more identifiable.

Greg: Certainly.

Rebecca: So, it actually does help some students for certain kinds of cognitive disability, because…

John: …including dyslexia.

Rebecca: Yeah, because the shapes and letters are really quick to identify so there’s a legibility that can come out…. However, does that help most students? Not necessarily… and does it help with a lot of content? Not necessarily… right because it might be okay for a small amount. As a designer, I just want to die.

Greg: Yeah. Well, it’s interesting…


Rebecca: None of those studies are done with design in mind at all… and so they’re really in these isolated situations. So, I would really love to see some of these cognitive science studies related to visual design with an actual visual designer… to see whether or not some of the things are actually beneficial.

Greg: I think that would be really interesting… and so to loop back on this… what we are trying to do is, in essence, create a learning environment for students that is: 1. easy to navigate… again, to the point of design, easy to find your way… find your way back again.. Will you re-enter that particular unit or module… and package it in such a way that it provides sort of a continuity of experience for the student? Beginning, with as we used to refer to him, high father Robert Gagne… Gagne’s “Nine Events of Instruction”… One of the things you do is, of course, you state what the learner is going to do… the advance organizer.. and the advance organizer helps focus the students’ attention as to what is about to occur in this learning module. This week, we’ll be covering this particular topic. You’re going to read this. You’re going to do these learning activities. By the way, this builds upon what we did last week by adding this. That last part is often what’s missing in an advance organizer.

John: …to create those connections.

Greg: Right. ….and that’s one of the things we sort of point out… because if you go back, and I recently read this… my world is shattered… people are just calling names… and saying Gagne was just an out-and-out behaviorist. Well, of course he was. I mean… in the context of the times, much of what instructional design theory looked like was based on the principles of behaviorism, because instructional design kind of goes back to World War II. It really emerged, I think, immediately after World War II, but given the needs of having to train multiple thousands of soldiers in a brief timeframe, it became obvious that there had to become a systematized way to develop and produce instruction.. . and that sort of behaviorist mindset carried through…really, I think, up until about the time when I was in graduate school somewhere in the 90s. There was this revolutionary… shocking… special journal that came out that posited that we should really be incorporating social constructivism… not just at the learning theory level but at the instructional design level too… and I think you sort of see that today, still, in this sense of community… in the sense that we now think truly of learning as socially mediated. It occurs within a group… so we think about what that looks like… and how we support that group of learners…. and that tends to be, I think, a transition point… a pivot point for us. Somebody was slamming poor Ben Bloom the other day. …again, because we’re really quantifying and proscribing what learning looks like in terms of those domains. The struggle, of course, is that we need to somehow define learning such that we have uniform measures of evaluation. If you took a purely constructivist standpoint… yeah, I think you would look at it and say: “Well, whatever you did was great, and if it doesn’t work… so.. well when we’ve got to put somebody on a continuum of a grading scale… and so that’s that’s the challenge. I think we always try to mediate.

Rebecca: We’ve been spending a lot of time talking about the role of instructional designers. Could you just take a couple minutes to explain what an instructional designer is, and maybe explain a little bit about what faculty could learn from instructional designers?

Greg: Sure. Instructional designers are somewhere between a unicorn and potentially a dodo. One of the fun things in life, if you actually are an instructional designer, is to go out into any kind of social setting and do the cocktail party meet and greet: “Hi, what do you do? I’m an instructional designer….” and people just look at you very very blankly and then you have to find ways to elaborate out in some way that makes sense to them. What does that mean? …and I had this problem with my parents… I had this problem with my children…. My children would go into school and they would say: “My dad works on computers, and he helps teachers learn how to use computers.” As an abstraction, it’s pretty close… pretty close…. but but not quite. My daughter… now actually, being a teacher, we now speak the same language. Yeah, she now goes: “Oh my God, that’s what you were talking about all those years…” like, yes, now you see… now you see… When I talk to you about Bloom’s taxonomy, you know exactly what I’m talking about… and…. so, really, instructional designers are, I would say…. one way to describe this… I’m not super fond of this description… is to say that they are learning technologists. We could say they’re learning specialists… we could say they’re learning engineers… we could say they’re learning designers… The focus, I think, is the fact that they are knowledgeable about the science and theory of learning. So, what an instructional designer brings to the equation, in working with a faculty member, is that perspective on evidence-based practice in learning. This is what the research tells us…. Oh, that research about video we’ll throw that out, okay… because you’re not up to date… but largely we try to stay up-to-date on the research… to say to faculty: “You know these are really the best practices if you’re going to engage in, let’s say, online discussion.” Because, back to what you said earlier, John, we may not be able to replicate the classroom, but let’s create an equivalent learning activity, right? Exactly.

Rebecca: …or a learning community.

Greg: …or a learning community, Yeah.

John: …and we should note that most faculty, especially those in older cohorts, were never trained in learning theory. They picked up what they saw their faculty do, and they come out of grad schools often where there’s very little or no emphasis on teaching or effective teaching… and they’re often told: “Don’t waste your time worrying about your teaching, focus on your research.” ….and having that sort of support can be really useful for faculty.

Greg: Right, and to be honest, it’s a very delicate conversation to have when you’re looking at someone who is, as we always say, you are the subject matter expert. I really know next to nothing about your discipline… but what I do know is I do know how people learn and I do know how to create effective learning experiences online and that’s what I’m here to help you understand. So we move away from any fears, any concerns that we’re here to challenge your notions of what you do in your discipline, because I don’t what you do in your discipline. By the time we’re done working together, I will know substantially more, which is the tremendously fun part of the job. You get to learn everything that everyone does here. Who gets to do that? We get to do that!

John:…Or if you’re doing podcast you can… and it’s fascinating….

Rebecca: It is.

Greg: It is fascinating. It’s so much fun. I just ran across, last week, this great study from a group called Intentional Futures, and they were really kinda trying to quantify what an instructional designer is… and they broke it down, I think, into four quadrants which I thought was incredibly useful to just share this out. So, the four primary roles of an instructional designer are: to help design learning experiences; to actually manage that production process of creating online courses or units of learning; to actually train the faculty, whether it’s discussion around pedagogy as we’re talking about here today or whether it’s the specifics of a tool set that the faculty wants to use; and most importantly, there’s the sort of Maytag repairman element that there honestly is ongoing support for faculty, continuing weekly… not just at the beginning of the semester… not just at the end when everybody’s trying to figure out why their grades don’t look right in Blackboard. [LAUGHTER] …although that happens. That’s right. We know that. We know when the peak calling times are, based on faculty work, but…. an ongoing effort to continue to help faculty throughout the semester and their teaching practice. So, that’s what happens. There’s a lot of one-on-one consultation with faculty, which again, is just fantastic in terms of creating relationships and getting to know people and getting to understand that subject matter…
JOHN :…and one other thing I think we could talk a little bit about is… we’ve been focusing mostly on online instruction and the role of instructional designers and learning new tools there, but there’s often a feedback effect that works to affect how people teach their face-to-face classes. The division between face-to-face and online is no longer quite as clear as it was thirty years or so ago.

Greg: That’s very true, John. In fact, I think, as we attempt to define the spectrum of technology enhanced learning, or technology supported learning, even those initial divisions that we created are really arbitrary today. Because, we would say… “Well, there’s ‘web-enhanced learning.’” I don’t even really know what that means…. but it means that somehow you’re doing something other just having students read out of a textbook, right? …and you’re doing something beyond just lecturing that somehow incorporates some instructional technology into that mix… and in the middle between web enhancing and fully online there’s this idea of blended ….which we like to call hybrid here, because we just want to be different… I don’t know… but most of the world refers to it as blended learning and in the K-12 domain they like to call it “flipping the classroom” because it sounds… I don’t know… you’ve got a psychomotor thing going on in there… it’s kinetic… I don’t know… but it’s the same thing… it is finding the correct balance between what occurs in the classroom and what occurs online and what we’ve seen… John, you’ve probably said this to me over the years… many faculty have said this to me over the years, unprompted… that they bring these things back into the classroom… and it isn’t necessarily just the technology of the affordances of the learning management system, but how you think about constructing that learning experience.

John: When I first taught online, I was using many of the same things that were very common at the time: these text mini-lectures and tests, and so forth… with weekly quizzes and discussion forums and because it was fairly new, there were a group of people in economics who decided maybe we should do a research project on that and we did… and we found that students really didn’t learn very much. In fact, they did a bit worse in the online courses than they did face-to-face. So, that forced all of us… but.. well a couple of the people there stopped teaching online in response, but others went and looked a little bit further into perhaps what might work. I attended some workshops. it was one actually given by Michelle Miller down in Orlando. I think you may have been there, too… at one of Carol Twigg’s workshops…

Greg: Right… right… right… I was in with you. Yeah.

John: She was using low stakes quizzing… and I started doing more research on that… and I introduced that in my online class and it worked really well, and student performance went up dramatically… and I’ve been doing it in my face-to-face classes ever since… and there’s a wide variety of things that I first tried in some of my online classes that have moved their way over…. and there’s not that much of a difference between the way I teach my face-to-face classes and my online classes.

Greg: Right, I mean it really begins to break it down and you begin to hopefully ask yourself the question: “Well, why am i standing up here spending the first half class restating the readings that my students should have read? Why shouldn’t I put up a quiz ahead of time to confirm that they read it?” ….but more importantly, and more valuably… use that as a diagnostic to find those fuzzy points in the reading and then let’s talk about that in class.

John: So you can do some just-in-time teaching. You don’t waste time on things that they do understand and you can spend more time on the things they don’t.

Greg: Right.

John: …and along those lines, one of the reasons for that issue that you mentioned about… going over things that they should have learned in the reading is that faculty who lecture primarily, often get into this situation where they tell students to do the reading… students come to class and they ask them questions about the reading and they find students haven’t done the reading… and in response they end up going over the reading… and then students realize they don’t have to do the reading, because it’s going to be gone over in class anyway… and then the faculty realize that they’re never doing the reading so they have to do it in class….

Greg: Yes.

John: …and we get this vicious downward spiral in terms of expectations of both students and faculty — where students end up not learning as much as they could be if that time outside of class was more productively used.

Greg: Right. You see this, as much as you can take rate my professor with a salt mine, where the salt… one of the themes that you can find in there, is that many times students will say “you don’t need to buy the book, because the professor will tell you everything you need to know in class.” It’s exactly what you just described, John.

John: …which is generally a much smaller subset of the content that we’d like them to learn.

Greg: Right.

Rebecca: I wanted to follow up a little bit. We’ve been talking a lot about evidence-based teaching and the role of instructional designers, but how about the role of administrators? What role do they play in helping advocate for evidence-based teaching on campuses?

Greg: Well, I do think… given in our particular frame… in our particular world… we’re a comprehensive college, so presumably our primary focus is teaching and learning. Presumably, we are creating culture here that really values and places teaching and learning first… and I do think, honestly, what you both have done here… in terms of creating reading groups… in terms of bringing in outside, evidence-based, yet eminently readable texts for a faculty to examine together…. to go out and try those strategies together… I think that’s incredibly valuable that we are doing that… and then we have a culture that actually supports us doing that. I do think if we could shift the frame a little bit more in terms of faculty activity… not just simply publishing to publish within your domain, but perhaps publishing to show the effects of teaching and learning strategies in your domain connects it all together better in a way.

John: ….the scholarship of learning and teaching.

Greg: Exactly… and I just say that somewhat selfishly because I think we need more of that within the disciplines. Because we need to recognize where there are disciplinary differences and where certain strategies may be more effective than another is… and I think reframing the conversation at the administrative level about expectations for faculty in terms of publication could help us in that. That’s very easy for me to say since I’m staff and not faculty, so I can come up with all kinds of crazy ideas that exist outside of the culture… but recognizing that providing incentives locally to create actual research opportunities in the scholarship of teaching and learning, as John says. For instance, doing things with open educational resources… if we can then turn around and measure the impact in terms of learning… and we’ve actually seen quite a bit on that happening out there at the community college level so far. Can we replicate that? The challenge for us to replicate it, frankly, is that we have to create materials and learning experiences at the upper division level. It’s really super easy, I shouldn’t say that, it’s easier. It’s easier to create foundational course materials because they can be more widely shared and it’s much more difficult when you move into money and banking or other specialized economics topics, because… what’s the audience that you’re constructing it for beyond your own local audience? You have to assess the cost-benefit analysis of doing that… but I mean those are ways we can engage in that… if I ran the world.


John: Where do you see online education going in the next five years?

Greg: We know it’s not going away. I think that’s the easiest thing to say. We know it continues to grow. I think one of the things that we’ll see change more rapidly, at least I hope so, is to bridge the gap between how we are engaging learners in this construct… this horrible walled garden of the learning management system versus the learners’ world — which is mobile, and its social — and given the fact that our students are always on… always connected… always mobile… how do you move from the learning exchanges that are best played out on a big screen and the keyboard, If I want to break it down to the technological problems? I had this conversation a week or so ago. A learning management system vendor was asking me: “What do you want to see? What do you want to see in the learning management system in the future?” …and I said I would really like to see a way for you to think about this problem of mobile first… and if we can still think of discussion as a valid construct of engaging in critical thinking, then I want a way for students to easily do voice-to-text in that environment. It’s things like that. It’s things like thinking about how do we situate that, and how do you situate it if I’m in my car an hour every day each way? ….and that’s lost time for me in a sense. I can listen to our podcast which is really cool, and I can listen to other podcasts, but what if I could actually be interacting with my course while I’m in the car in some way? or if I was on the train? We have to really rethink what that delivery looks like and how we interact with things… and are there ways that augmented reality can be brought into this mix… again, through our phones… they’re with us…. they can do this overlay…. What can we do with that? Those are like the new frontiers.

Rebecca: In design, it’s actually an old frontier…. but it’s just applying it to this context. It’s user centered design.

Greg: It is and it’s totally understanding your user behaviors… the environment in which they live in and interact in… and it’s not a new concept.

John: But the technology has changed quite a bit… because now both iOS and Android operating systems have AR kits built in to make it easier to record and to implement AR.

Rebecca: …and really new. The AR kit on the iPhones just came out in the last few months.

John: …and similarly the Android one has just come out.

Greg: Yeah, and you know overlays like that… your voice assistance that we all have on our phones… are there other ways that those can be integrated in too? I think these are more interesting things… how you look at the challenge. The fact that you can create adaptive learning that works pretty well in an app. Duolingo is a pretty good and often cited example of an adaptive learning app for language acquisition. The challenge there, of course, is that that is one silo.

John: …and we don’t see as much of that. Carnegie Mellon was doing some great work on this over the last few decades.

Greg: Yeah.

John: …but there aren’t that many implementations. There’s cog books and one other publisher who started bundling some of these in packages but there’s not that many courses developed yet, and there’s still a ways to go.

Greg: Right.

John: …but that offers a possibility of having customized learning paths for students where they work on the things they don’t understand as well… and they build in all the best practices of learning… and we can get people to learn more efficiently once those tools are there. But, it’s still an early time for that.

Greg: It is an early time, and it’s somewhat beyond my skill set and your skill set to go out and just create that. It’s like saying I’m gonna go build an airplane. I have a friend who built an airplane, but before he retired he was an engineer, so he had the skill set to do that.

John: Cogbooks… and Acrobatiq (I believe is the other company)… have provided a framework for instructors to build that. But there’s still a lot of time… because you have to think about what area students might have problems with… and then build materials to get them past those problems… and there’s some pretty high fixed costs for doing that.

Greg: There are. That understanding a particular domain and understanding how you remediate in those weak areas. Whether or not AI can really, through machine learning, get there with us, I think is another thing. Because I tend to be a person who doesn’t subscribe to the model of the brain in a purely computational model. I think there’s a lot more, in the sense of mind, than just thinking about storage and retrieval. So, it is, I think, one of those great challenges to get through… and I do think, while you can build specialized apps that do that. the problem then becomes… as we know, what we deal with a lot is command and control as instructors… and by command and control, I really mean being able to understand and manage the learning… but having visibility into the learning and being able to assess the learning, where human judgment combined with some rubric development is necessary. So, centralizing all that together… one colleague argues that the worst thing they ever did in growing the learning management system was to add a grade book. Because you and I evaluate things differently, even if we live in the same world… even if we’re in the same domain… So, my grading schema doesn’t look like your grading schema… and you build this horrible, horrible, horrible, layer of complexity into the grade book to try to accommodate everyone’s variations… and so there’s one argument that says maybe we just need an app that’s just a grade book… and it just sucks in the data from all these apps that do what they do really well. That potentially… maybe… is part of what’s known as the next generation digital learning environment.

John: We’ve been hearing about that for decades.

Greg: Right… right?

John: Yeah.

Greg: …and it’s been another forever… I had conversations somewhere 8…10 years ago… very parallel to that… that was being framed as the learning management operating system at that time… and we were thinking about this sort of decentralized approach, loosely coupled, that through other structures and other communication methods like LTI… blah, blah, blah ….I won’t go down the rabbit hole with all these acronyms… but ways in which you could move the data around and share it across these varying systems… and we’re back to that conversation… and what the learning management system developers do with these inputs is the big question.

Rebecca: So, you have some pretty interesting visions for the future. What are you gonna do next?

Greg: Well, I really hope to complete my doctorate before I retire… that’s my major life goal.

John: That’s a great program, by the way. You’re in the program in Buffalo?

Greg: Yes. University of Buffalo… The acronym is CISL [pronounced sizzle]. Yes. Curriculum, Instruction and the Science of Learning… and I think that’s really fascinating that we’re now seeing programmatic titles that put the words learning and science together. So we’re really emphasizing that indeed you can draw upon evidence-based practice… you can examine the research and inform practice. It happens to be the only fully online doctoral program that SUNY offers. So, as a matter of convenience for somebody like me, it’s fantastic…. and it’s been a great experience… and it’s kind of brought me around to, again, examining some of these logical fallacies that we continue to carry around and when we’re done, I’ve got to go back and I’ve got to read up again on video and attention.

John: Neil Bradbury, by the way, is the person who did that, and I believe that will be released in mid-February.

Greg: Okay. Sometimes I get questions from on high in administration: “What is the most appropriate size for an online course in terms of seats?”

John: Five?

Greg: Yeah, well… it depends… in a graduate course in a seminar… yeah… probably five is right [LAUGHTER]… frankly… because it depends upon…. the context is everything.

John:… what they’re doing.

Greg: Right. When you break down and you try to do a literature analysis on class sizing, context is everything. So you can’t provide a universal rule or even a sliding scale to Deans and Provosts and say: “Well, it looks like this.” Not necessarily… but you take other constructs that again we tend to look at and know when our received assumed wisdom: effective discussion is three posts… the student engages in the question and they engage with two other learners. Well, why? What makes that effective? What does that have to do with anything? That’s an arbitrary number that somebody invented to generate activity. So, looking at what are the constructs that have been defined to actually promote critical thinking… and if we break critical thinking down into elements… into certain specific responses…. Couldn’t we create a better grading rubric that supports the evidence of that? ….and that’s kind of where I am Rebecca. I’m finding these new things to kind of come back and shake the tree with everybody… and where it’s most fun is shaking the tree with fellow instructional designers… who also teach…. and what do they say to me? They go: “I know what critical thinking looks like in my course.” I’m like… Really? Really? You’re saying that? Aren’t we beyond that? Actually, I had the discussion once here… way back when…. I won’t say what department… I won’t say who… but we were discussing the utility of rubrics in grading and the response I received was: “I don’t need a rubric. I know what learning looks like in my students.” Yeah, that’s fantastic…. really objective…. not subjective at all. That’s great…

John: …as long as students share that vision….however it may happen to exist at that time in that person’s head.

Greg: Exactly… Exactly… so I think what is next for me… I think it’s continuing to look at these things… continuing to examine what’s happening in research and bringing that back into our practice so that we continue to evolve as a community here.

John: It’s an exciting time. There’s so much going on out there.

Greg: It is. Definitely.

Rebecca: Well, thanks so much for joining us today, and taking some time out to talk instructional design with us.

Greg: Thanks for having me. Thanks for the coffee

John: That’s right…. we did have coffee here. But that’s because we had an early meeting prior to this. We normally don’t have coffee in our office.

Greg: You might want to think about a name change for the podcast… I’m just saying… [LAUGHTER]

John: We’ve been getting that sorta feedback recently.

Rebecca: Yeah.

John: Well, thank you.

Greg: Thanks again.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

16. Student attention span

Have you ever been told that to keep students engaged you should chunk lectures into ten minute segments? Neil Bradbury, a Professor of Physiology and Biophysics at the School of Graduate and Postdoctoral studies at the Rosalind Franklin University of Science and Medicine, investigated the origins of this myth. In this episode, Neil joins us to discuss his review of the research on student attention spans.

Show Notes

  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • Bradbury, N. A. (2016). Attention span during lectures: 8 seconds, 10 minutes, or more?. Adv Physiol Educ, 40, 509-513.


John: Have you ever been told that to keep students engaged you should chunk lectures into ten minute segments? In this episode, we examine the origins of this myth about student attention.

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist,

John: and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Neil Bradbury. Neil is a Professor of Physiology and Biophysics at the School of Graduate and Postdoctoral studies at the Rosalind Franklin University of Science and Medicine. Welcome, Neil.

Neil: Thank you. Nice to meet you.

John: Nice meeting you.

Rebecca: Yeah, thank you for joining us. Today our teas are

Neil: Today I’m going to go with Lapsang Souchong.

Rebecca: Sounds like a great choice.

John: Very nice, I got some of that, too. I have to keep it separate though because it has a smoky smell and it blends with the rest.

Neil: Yes it does have a strong, smoky smell.

Rebecca: And today I have vanilla chai.

John: I have Yorkshire Gold tea.
Your two thousand and sixteen paper on student attention span has gotten quite a bit of interest on a number of email lists and professional development groups and that’s where I first saw it. What prompted your interest in this topic? It’s a bit different than your usual research.

Neil: It is a little different. I have gone to various teaching institutional days and I had been told of this ten-minute rule, and naturally I accepted it because that was what the elite was telling me. And then shortly before I started looking into this, our medical school, like many medical schools in the country, was revising its curriculum entirely over the entire four years… and it was in a meeting with the Dean going through the new curriculum where he mentioned again, “I now have to stop because I’ve been talking for 10 minutes and so I need to do something different…. and I thought to myself “Well, that’s interesting, I wonder where this notion comes from?” …and I decided I was going to find out, because everyone told me ten minutes is the rule, ten minutes is the rule, but no one knew where it came from… and so I decided that I would have a look at this and if indeed it was the case I should be taking notice of this when I’m teaching… if it’s not the case, where did it come from? Where did this apparent educational myth arise? So that was really what started my interest in this… looking for where the source of this myth was.

John: I have the same sort of thing… I just took it as given, I’ve seen that said so many times and seen it in so many books and papers and recommendations, that I also took this as given. One of the things I’ve always used to judge the quality of someone’s presentation on teaching and learning is whether they start mentioning learning styles, or they put up a picture of Dale’s Cone of Learning, because those are myths that are pretty well known and pretty well established, but this one I think a lot of us had taken as a given…. there must be some research on… or we wouldn’t keep hearing it so much, and we should have known better. So what did you find when you began to investigate this? Where did this rule come from?

Neil: When I was looking through this, as you say, it’s often repeated, and people make the statements and then refer to a paper that was published many years ago… and when I eventually tracked down all the references, I did eventually come to a paper that was published in the fifties… and much to my surprise, the paper actually didn’t mention attention span at all in any of the words, which was a little curious since this is the basis that everyone uses for the ten-minute attention span. And what I found was that, in actual fact, the paper does not describe attention at all, but rather note taking…. and even more curious is that a subsequent publication by these authors even stated that they felt that note taking was no basis for discerning attention span anyway. So I think the whole propagation… and to be fair it’s the original authors, they did not say that this was looking at attention span… but somehow it’s got changed over the years… saying that they were looking at attention span of ten minutes, and that got propagated through the literature, and propagated by people who were looking not at the primary literature, but at someone else’s interpretation of the literature and it just propagates without anyone actually going back to the primary literature.

John: It’s nice that you did. It’s about time someone did and thank you for doing that.

Rebecca: I think it’s so important in the day we’re really focused on the idea of evidence-based teaching that we do remind ourselves that we should be looking at the evidence and not just taking things for granted.

John: And in your paper you also mention that there were some potential flaws in some of the early studies on note taking. What were the major flaws in some of that early work?

Neil: I think that some of the major flaws were really in experimental design. As a scientist, I spend a lot of effort pre-planning my experiments, often taking more time to plan the experiment than to do the experiment, and I found that a lot of these lacked really rigorous planning. So, for example, one study involved two lecturers that went in to observe a class, and in the paper, they stated that more often than not at least one person turned up to investigate attention. Well, if you’ve only got two people looking at student attention and only one of them occasionally turns up, it’s hard to take anything seriously that comes out of that study. So, a lot of things like that. One of the things that, as a scientist, I particularly have to take notice of is statistical rigor and we have to take care to provide instruction in what statistics we’re using and whether or not our statistics are valid. Many of the studies just state categorically, “our results were statistically significant” with no indication as to what those statistics were but then that gets repeated by the next person who just blithely states “oh, well this person states that it was significant” and just the takes that for granted, and so it gets propagated.

John: …and it might have been significant at the sixty percent level or something similar, if they don’t specify…

Neil: We don’t know because, it’s never mentioned in the papers.

John: …and one of the things you mention, in one of those early studies of note taking was that it may have been perhaps more of a measure, I think, of the content of the presentation rather than student attention.

Neil: Yes, I think it’s important when we’re considering lectures as to really fundamentally what the purpose of a lecture is, and I clearly don’t think that the purpose of a lecture is to have students take notes as the end result of a lecture. The notes should be something that the student refers to at the end of the lecture to remind them of what was covered. Now certainly note taking is important, but we don’t take notes on everything. So, for example, when I’m giving lectures I’m trying to convey a certain concept that may be difficult or it may be easy, but I try and give some illustrations of how that concept can be applied to real-life situations. Often, as it turns out, one of the things I like to do is discuss how my concepts can be applied to understanding how people use different drugs to murder people.



Now I don’t really expect students to take copious notes on how to murder someone they don’t like.


Neil:So, that may be… the students would not take notes on, that’s just a little bit of fun and interest. The physiology underlying that? Then yes, that’s important to take notes on. So, you need to think about what’s being discussed. Is it important to take notes on that? Other things? No, it’s not important to take notes on. So I think there’s a balance there… that you can’t just take notes across the entire lecture, it’s really what’s being conveyed by the teacher at that particular time as to whether the notes are worth taking or not.

John: …and students could be very engaged, but not taking notes, because they are actively processing the information and making connections. And there may be no need for notes if that part of the presentation is very clear and doesn’t give them new information they need to transcribe somehow.

Neil: Absolutely agree… and just staring at a teacher or writing notes doesn’t necessarily mean that they’re focused on what they’re doing.

Rebecca: …especially if the content of the notes aren’t really being evaluated, right? It could be notes on anything!

Neil: Well, I think we’ve all done that at some point. I think we’re all guilty of that, but you’re absolutely right, just taking notes for the sake of taking note has no intrinsic value either.

John: …and in one of the studies, you mention that the students were keeping track of their level of attention where they were periodically polled and they had to indicate their level of engagement, but that was done with two different classes: one with first-year students and the other with fifth-year students, but they didn’t hold constant the instructor. So, the results were very different across instructors.

Neil: I think that that’s true broadly ,that we have all experienced instructors in our own life that have been really wonderful and engaging, and we really enjoy going to. We’ve also had instructors that we really did not want to go and listen to. So, I think there is certainly a large component of the instruction that makes a big difference in how they present. I think the other difference is that when comparing a first year with a fifth year is a huge difference. A first year really does not have any basis for knowledge, they have very little understanding of what’s going on, and so any new knowledge that they get is a huge increase. Compared to someone in the fifth year, where they may be really not learning new things, but they’re adding to what they already know, and it’s a lot easier to learn when you already know things.

John: You have more connections there.

Neil: Trying to fit things into a model that you already have to further strengthen that, that’s really not that hard to do, compared to learning a concept from the start. So, I think there’s a big difference there, between first and fifth years on a lot of levels.

Rebecca: Why do you think note taking was conflated with attention in the first place?

Neil: I think it’s conflated because it is really hard to assess attention, and we all know that when we go to lectures, we’re supposed to take notes. I think that was conflated because note taking is something obvious that we can see. We think of attention and people paying attention to what’s going on… that goes on inside the brain and you can’t really see what’s going on inside the brain; whereas you can see people writing notes. I think that was just used as a surrogate because it was something that could be measured, not necessarily because it was a valid measure of something that was going on.

Rebecca: What were your biggest takeaway is from the survey of literature that you did?

Neil: I think the biggest thing that I’ve taken away is that, as a teacher, I don’t need to worry about keeping things as ten minutes. I can focus on a concept. I was thinking about this as I went with my family to the latest Star Wars movie. I was thinking “What if I watch Star Wars only in ten-minute segments? and that those segments were not connected with each other?” I don’t think anybody would go and see that movie. So the ten minutes attention span really doesn’t hold up for that… no other experience in life do we have or do anything that’s only ten minutes. Why would it be any different from a lecture? And so I have freed myself from having to worry about going in ten-minute blocks, and I can focus more on providing a conceptual framework for what I’m trying to cover.

John: Students don’t have any trouble watching an hour and a half or two and a half or three hour movie or sitting at an engaging video game for hours at a time, and they often require lots of learning and so forth… and we don’t really worry about the amount of time it takes on those activities. When I’ve gone to talks, there have been some talks I’ve been at that, within thirty seconds, I’d like to bolt for the door (or fall asleep) and there’s been others where I’ve been fascinated for an hour or more at a time.

Neil: I think it’s the latter that we’re trying to go for… and it would be nice if a student suddenly looks at that watch and says “Oh I’ve been here for two hours …where’s the time gone? It’s been so engaging.” That’s what I, as an instructor, am really trying to aim for… is not to have a student worrying about what they’re going to be doing in the next two minutes but to realize Wow, the lecture is over. It’s been worthwhile. I’ve learned something and I’m not really bothered what the time is.”

John: That’s always great when that happens, I wish it happened more often, but it’s great when students want to stick around and find out more and they’re not ready to leave at the end of the class.

Rebecca: Sounds to me like a lot of faculty probably need to spend more time figuring out how to make their lectures more engaging and to captivate their audience, rather than simply trying to break things up into smaller pieces, right? So you might not take communication style or things like that as seriously, but perhaps something that we need to invest more time in.

Neil: I agree, I think one of the questions that I always try and ask myself is “Why is what I’m teaching important? Am I teaching it because it’s there in the textbook or am I teaching it because it really is important?” …and if it’s important, I should be able to describe why it’s important… why you need to learn this…. and it shouldn’t be abstract, it should have application to what the students are to be doing. You need to learn this because it means that you can understand what’s coming next. Since I’m teaching a lot of medical students, why it’s going to make an impact upon health care of the patients you are going to be seeing. If I’m talking to chemistry students, you need to know this is what’s important because it’s going to dictate how you design your chemistry experiments. So, no matter what you’re teaching, I think you have to come up with reasons for why what you’re teaching is important, convince yourself it’s important, and then try to convince the students it’s important.

John: That’s a point that Ken Bain made in his book What The Best College Teachers Do. He suggests that you should start with the key concepts in each class, explain to students why those are really important… you explain to them they need to be able to do these things to be able to answer those big important questions that matter to them in some way. You mentioned before that first-year students often don’t have as rich of a network of concepts, and as a result there’s perhaps a bit more cognitive load that they have to deal with. Might there be some advantage of breaking up a presentation though into small chunks and then having them actively engage with the material before moving on to the next concept — in terms of keeping the cognitive load manageable?

Neil: I think there is some merit to that. I think it should be dictated by the material, rather than a clock, and so we can look at things and see where are the boundaries that make a unit of knowledge a reasonable unit… whether you can cover that in six minutes or fifteen minutes, I don’t think that matters, but can you have a coherent unit that can stand on its own that you can put together with other things. So, content is more important than the time allotted to it.

Rebecca: I think what you said about content is really important and as a designer there’s a methodology called “content first,” and it makes sense in a classroom setting too… where you decide what the content is and then design around that, right? There’s some things are going to make more sense to do hands on, some things that are in going to make more sense to provide a lecture on… and if you let the content dictate what it is that you do, it makes a lot of sense.

Neil: …and certainly, when you’re thinking about how you’re going to organize a lecture, you have certain content that you need to cover. But the order of that content is really important. You need to start out with the foundations before you can go to the building, and to start with those foundations, that may be small chunks. I agree that can be time limited, and that you could work on, and then as you getting more knowledge more content, you start building that up into a logical, coherent molecule. But, you’ve got to start with the basics first and then you can build onto larger structures.

Rebecca: It gets funny that, as experts in a particular topic, we forget that there’s building blocks and foundations because our mental models are so much more complex. So, I think although sometimes these things seem obvious, we need the reminders to take a step back and remember what it’s like to be a novice at something.

Neil: I agree. Most of my research is focused on cystic fibrosis and I’ve been doing that for many years, but I realize when I’m teaching that to students, they don’t have the decades of experience that I have. They are not going to become experts in an hour. What I’m trying to convey to them is the broad concepts and if they get those broad concepts, they don’t need to know the minutiae that intrigues me on a daily basis. But, they have to have the broader picture and so I have to put myself in a student’s position and understand from their position what they need to learn and even what they’re capable of learning at an early stage

John: Now, have you shared these results with your colleagues and how have they responded? Has this affected how they’re teaching?

Neil: As you might expect, it’s been somewhat of a mixed reaction. Some have really liked it, some have not really been that embracing of it. I remember we had a speaker from another institution who were discussing their new curriculum and they had designed their entire curriculum around 10 minutes… and I had the temerity to ask why that was the case and the response I got was “Well, everybody knows the attention span of students is only 10 minutes.” …and so… it happens, and it’s propagated, and I think most people really appreciate the fact that there really isn’t a lot of basis for this 10 minutes, but they don’t know that. They’ve heard and been told over and over again… it’s ten minutes. It turns out that’s not the case, so we don’t worry about, this is not something that you need to worry about. Plenty of other things to worry about when you’re teaching, but this is not one of them… So, don’t worry.

John: About ten years ago here, we had a guest speaker, whose name I won’t mention, but he gave a brilliant hour and fifteen minute talk on how a lecture is ineffective…. and later at a reception I went up to him and said “that was an incredibly good lecture… it was really engaging and dynamic and everyone seemed really interested,” and he did appreciate a little bit, I think, the irony of that comment.

Neil: Yes, I agree. I think the question is, “What is the point of a lecture?’ And I don’t think a lecture is really the place for student learning. That’s not where students learn. The point of a lecture is to convey information the students can learn later, but I think there also is an important point of inspiration. It should give students an understanding of why this is important and an appreciation for students to think “this is exciting, this is really fun stuff to learn.” If I can help students make their own decision that this is fun to learn, then I don’t have to worry about making it fun to learn. They’ve decided it’s fun and they’re going to invest effort in learning it themselves. And so part of my role as an instructor and a lecturer, is to get the students to appreciate something, that I think is true, is that learning is a lot of fun… it’s a fun thing to do and that it doesn’t matter what I’m learning… whether it’s the material I’m interested in or something completely different… it’s fun to learn… and if we can convey to students it’s fun to learn, they’ll be more than happy to learn things no matter what the instructor does.

Rebecca: Such a great point.

John: It is, and I also feel the same way about faculty. That we all got into this because we were among those students who found it fun to learn, but sometimes people forget that once they’ve been teaching for a while and it would be nice if we could also encourage each other to share that enthusiasm for learning.

Neil: Yeah, I think it’s important for faculty to go and visit each other’s lectures… how we can learn from each other. I’ve been working on my lecture style for many years, I doubt very much whether is the best lecture style going. It seems to be appreciated by the students, but I can always learn and improve. I only see it from one perspective, having faculty and colleagues come to my lectures, me going to their lectures can be of huge benefit in improving everybody’s teaching.

Rebecca: Is there something specific that you’re working on right now as a lecturer to improve?

Neil: With the new curriculum that we’re bringing in, there has been a large change in how we teach, and so that’s reflecting a lot on the content that we teach, and so I’m trying to come up with ways that integrate a lot of material across a lot of different disciplines. Which is proving to be a little bit of a challenge. As a physiologist, I’ve been teaching physiology. Trying to bring in other disciplines into the classes is proving to be a little bit of a challenge, but it’s also exciting and has a lot of opportunity that I can bring a lot of different aspects in. And I think that I’m going to learn a lot and hopefully that will be conveyed to the students… that we can all learn together something that we may not have covered before.

Rebecca: Sounds exciting but also very challenging.

Neil: That’s why we’re educators, we like challenge and those go together as educators.

Rebecca: Do you think after studying attention span in this way that it’s worth more study and to have others investigate attention or is it something that we’re spending too much time thinking about?

Neil: I think there is some basis for looking at other aspects. Most of what we’re focused on so far obviously is the lecture, but as we know now, the lecture is not the only teaching modality that we use. We have small groups, we also have lab practicals, we have discussions, and so far this attention span has only really focused on lectures. I think it would be informative to also look at other ways in which we teach, look at small group learning, look at peer learning, look at practical learning. I think that is going to be an interesting avenue to explore… is what is the attention span, by whatever criteria the define attention. Is that different from those modalities than a lecture? Is it the same? Is it different for each modality that we look at?

John: …and might it also vary by the topic you’re looking at? That some topics for many students would just be more interesting than others and that, I would assume, would vary quite a bit across students as well.

Neil: I think you’re right. Not everything that we cover is exciting and interesting. There are some things we just have to do because you need that knowledge, not because it’s exciting. But, that it allows you to get the exciting parts. So again coming back to knowing fundamentals and getting that basic groundwork that allows you to do the fun stuff later.

John: …and explaining to students why they need to know those basic, less fun, things helps provide them with a bit more motivation to work through it because they see where it’s going and how it’s connected.

Neil: Yes, you always have to have an endpoint… why this is important. It’s important because we’ve got to get to this position and this provides a pathway to get there, and once we’ve got there, a whole bunch of things will open up to you that you didn’t even realize.

Rebecca: Are you planning to do any more work on attention span or are you attention spanned-out?

Neil: Well I was thinking about it, but I got bored.


Neil:I’m really interested in looking at these different teaching modalities to see where that applies to different avenues, because we’ve only focused really on the lecture and that certainly has been a dominant component of education at the institution. We’re moving away from that… lectures are still important, but we’re also incorporating small group learning… peer learning…. and I think it’s going to be instructive to discover whether or not attention is really critical there. How can we get students, when they’re doing peer learning, to take this into account to make sure no one’s just falling off the edge and not learning anything. So I think it’s going to be exciting to increase our understanding of how students learn, how they’re attentive, how they’re focused on what they’re doing.

John: In your review of the literature, did you think of any good ways of addressing the question of attention span? Or is it, by its nature, impossible to measure well?

Neil: I think it’s a really difficult thing to measure because it is something that is going on inside people’s brains and that’s always a hard thing to measure. Certainly, we can put people into CAT machines and MRIs. But that’s probably not a good learning environment for anything. So I think it’s nebulous by its very nature. I think the important point is: “Are the students learning anything.” I don’t think necessarily what we’re covering in the lecture is the be all and end all. Some of the experiments that were performed, that I discussed in my paper, were evaluations of what students learned that were taken immediately after the lecture. But as I pointed out, no one ever does an examination immediately after the lecture… and so those kind of studies are really, to my mind, fairly meaningless. The question is, downstream a couple of weeks later, when we examine the students on the content of that material, have they learnt it then? And, I think, that’s when we really get to assess whether they were paying attention… not by looking at whether a student is taking notes during a particular lecture, not by asking them questions immediately after the lecture, but whether they’ve really spent time going over that material, and again, and again so that they can adequately answer questions two or three weeks later when we examine them on the material.

Rebecca: So it almost might be whether or not they’re engaged and motivated enough to want to continue pursuing that information, so that they can pass those exams and things a couple weeks later… and not really whether or not they’re paying attention in the moment that it’s introduced.

John: …and that ties back to the inspirational role of lectures that you suggested earlier.

Neil: Yes, we should inspire students to want to learn. We can never just force feed students information… it’s just gonna bounce back. We have to inspire students to want to learn for themselves, and that’s what effective teachers do. They don’t teach, they get the students to learn themselves because they’re excited about learning.

John: Very good. Well thank you, this was fascinating.

Rebecca: Yeah, thank you so much for spending some time with us today.

Neil: Well, I enjoyed it and thank you for this opportunity to discuss the paper, I really enjoyed it.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

15. Civic Engagement

Real-world learning experiences come in a variety of flavors. In this episode, Allison Rank, a political scientist at SUNY-Oswego, joins us to discuss how she has built a course in which students organize and run a non-partisan voter registration and get-out-the-vote campaign. This project combines many of the best features of service learning and simulation.

Show Notes


Rebecca: Real-world learning experiences come in a variety of flavors. In this episode, we’ll explore ways to combine the best features of simulation and service learning to increase learning in a campus-wide voter registration and get-out-the-vote campaign.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist,

John: and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.
Today, our guest is Dr. Allison Rank, an assistant professor of political science at SUNY Oswego. Allison is an expert in the role American youth play in the electorate and the founder of voter registration initiative called Vote Oswego. Welcome, Allison.

John: Welcome.

Allison: Thank you.

Rebecca: Today our teas are:

John: Yorkshire gold.

Allison: I don’t drink tea.

Rebecca: It’s an epidemic…. like… this is the third one, John.

John: I know, three in a row

Rebecca: Three strikes you’re out. No more non-tea drinkers… All right… I’m drinking English Afternoon despite the fact that it’s still morning… because I need it.

John: Just barely morning, though.

Rebecca: Good, okay.

John: So, what is Vote Oswego?

Allison: Vote Oswego is a student-run, nonpartisan voter registration and voter mobilization drive on the SUNY Oswego campus in the Fall of 2016.

Rebecca: What led you to start Vote Oswego?

Allison: So, prior to earning my PhD in Political Science, I actually spent three years working as a political organizer. I’d worked for a presidential campaign in the state of Ohio and then had spent a couple of years working with college students on a variety of non-partisan campaigns. One of the things I learned from doing that work on college campuses, is that when students have an interest in doing political work, there’s a lot of skills that they can get out of doing that work but that they don’t necessarily think about. And so once I was here…. I was here first in the Fall of 2014… and saw what the voter registration drive looked like, it was clear that students were doing some volunteering for it, and some students were really excited about it, but I didn’t think they were really getting any organizing skills out of doing it. They were sort of more just sitting at tables and sitting out with voter registration forms the same way they would sit out with cupcakes at a bake sale… it wasn’t really about organizing skills. And so I wanted to start a course or something here where the voter registration drive would become more about students learning to organize rather than just being treated as widgets to be organized by other people.

John: Did you do this as part of a class or was it a set of classes or a separate activity?

Allison: I actually was able to get permission to run it as a special topics class in Political Science. I’ve since gotten it approved as an official course, but initially I actually just pitched it as a practical political skills class where the students would come in and learn about grassroots organizing techniques and then get to implement those techniques through a voter registration drive.

Rebecca: I would imagine that a course like that would be particularly helpful to campuses that are more rural than urban.

Allison: Yeah… for me, after a couple of years here, I had a lot of students that wanted to get involved in politics but I’d end up in conversations with them about how hard it was to figure out transportation to Syracuse, figuring out logistics or the cost of doing it– we have so many students not only in terms of being a rural campus, but also in terms of the student population that’s also trying to juggle working and paid work that they need in order to be here. And so then, taking time out to do a political internship, especially with the schedule around an election, can be really challenging. And so being able to offer that opportunity on campus, and also around something that can give them course credit or internship credit without leaving the campus, and for something that the campus is already gonna put energy and attention towards, I think, is really helpful.

Rebecca: Is running something like this as a class common on other campuses?

Allison: I don’t know of other places where a full grassroots campaign has been run out of a class. It’s fairly common in Political Science to have some type of activity based around voter registration, right? So, for us in Political Science, coming up with civic engagement projects where you can avoid partisanship and partisan issues, is a really big deal. And so non-partisan voter registration drives around elections are a great place to do that, but often it’s asking students to go out and volunteer as poll workers, or do exit polls, or maybe helping set up a campus debate with a couple of candidates rather than really digging into an on-campus, full-fledged grassroots mobilization campaign.

John: Was it easy to keep it non-partisan in the classroom?

Allison: Oddly, it really was. The very first day of class, I ran an non-partisanship training with the students. So what does it mean to behave in a non-partisan fashion? What does it mean to keep your social media non-partisan through this event? What are the conditions under which you need to be non-partisan, right? Students signed up for this class because they’re people who care about politics, right? So many of them, I am certain, had very deep feelings about what they wanted to happen in this election. They weren’t allowed to talk about them if they were at a Vote Oswego event. I think there was a guideline around however many Vote Oswego students were hanging out together, that they could be recognized as a group of Vote Oswego students. If they were wearing their Vote Oswego t-shirt, they could not both talk about something partisan and have any reference to Vote Oswego in, for instance, a online social media “about themselves” section. And I would actually, essentially, run pop quizzes with them where I would try to get them to do something partisan, right? I would come up to them and say some incredibly partisan statement and they would actually have to practice what the non-partisan response would be.

John: That’s a useful skill, today.

Allison: Yeah!

Rebecca: Probably one th at a lot of faculty could use some training on, too, because politics come up a lot in classes. Can you give us an example of something that you would do?

Allison: Sure. So one of the things that is often defined as partisanship is if you endorse an issue that is so clearly identified with one political party over another…

John: …like science….

Allison: …even if you don’t, we would use things like building the wall, right? If you say something like “you should register to vote because it’s really important that we build the wall,” that would be considered a partisan statement from the last election, regardless of not mentioning a candidate or a political party. And you would get individuals coming up to the table who wanted to register that would say things like, “it’s really important to me that I register because I really care about maintaining woman’s right to choose” or “I really care about building the wall…” something that you could clearly align. And so I would do that to students, and they essentially had a set of responses they were allowed to give. So things like, “I’m happy to hear that you’re excited about what’s happening in this election, it’s really important that you get registered to vote” or they’re allowed to not and say, “I acknowledge that that’s something that you’re really passionate about.” Vote Oswego is non-partisan, we just care that you’re able to express whatever you care about, right? It’s sort of acknowledging that that individual has something that they really care about, but not endorsing it yourself.

John: How did they do in those pop quizzes?

Allison: They generally did really well. The first day, they would get really awkward and nervous and not know what to do, but after sort of half an hour of drills, they got incredibly good at it. It also helped that it was in the syllabus and they signed a contract with me that if I caught you violating the non-partisan mandate after one warning, you automatically got fired from the campaign, which meant you failed the course. So they took it seriously.

John: So it was somewhat high-stakes.

Allison: Yes.

Rebecca: A little arm-twisting there?

Allison: Yes. But they did really well, and actually after the election, a student made a comment in class where he basically said it’s really weird to me…. I feel like I know people in this class so well and we’re really good friends and we’ve worked so hard and I have no idea how anyone in here voted and I said, “that’s great, that’s exactly what should have happened, please don’t talk about it.”

Rebecca: Can you talk about some of the other results that you saw in your class?

Allison: Sure. There’s sort of two different sets of results, right? We talked about the students as having two identities in this class. They were both students and they were staffers and they needed to be concerned about themselves in both of those roles. So as staffers, they had fantastic success. The campaign registered over a thousand students on the SUNY Oswego campus, they helped over 1500 students request absentee ballots, they came up with some really great campaign strategies in terms of helping students with absentee ballots… get those mailed in… get those stamped… helping students get to the polls… building a great coalition with other folks on campus. As students, I think what was great is that because the students were out in the field and they were known as Vote Oswego students on campus. Their friends all knew that they were doing it, they were in those t-shirts all the time, they were visible. They took a real ownership over this project, in a way that I have a hard time envisioning getting students to do about short-term volunteer work, or sort of asking them to go volunteer with another campaign, or even the type of simulations that political science professors can get really good results with, in terms of learning outcomes, the type of ownership that these students felt and how seriously they took it, I’d be hard-pressed to get that result in another way. Because they took it so seriously, and because they took such ownership of it, I think their critical thinking and analytical skills really, really improved. You could sort of watch as we went through the campaign, students go from looking to me and looking to the couple of interns that we had in leadership roles of the campaign, to figure out sort of like “all this thing happened and what am I supposed to do” and “please answer this question for me,” to like, “well, this thing happened while I was standing at a table and here’s what I did,” and I would hear about it three or four days later as opposed to getting a sort of frantic, “help me figure it out.”

John: they started taking more responsibility–

Allison: Exactly.

John: –and making more of the decisions then just reporting back.

Allison: Exactly. And then also being able to constructively critique each other’s decisions once we– we called classroom meetings campaign meetings, right? So in campaign meetings, being able to say, “Hey, I know this is what happened last week, actually I think we need to fix it in X, Y, & Z ways.” Which, for those of us who have tried to get students to give critical constructive feedback on each other’s papers, it’s really hard to get them to engage each other that way, and the students really took to that sort of analytical and critical work with each other in really constructive ways by the end of the campaign.

Rebecca: So in addition to students finding that kind of personal ownership over the experience, what are some of the other factors that you think made this particular project, in this particular situation, really successful?

Allison: I think there are a couple of things that made this project work really well. I think that, one, is that a non-partisan voter registration drive is something that students can get excited about, even if they’re really uncomfortable with the idea of the conflict around politics. So students that are interested in politics, but don’t really want to be in the debates around politics, can latch on to this as a project that they can get excited about. So, for instance, we had a number of students from PR who took this class because they saw it as that they didn’t really want to get into politics, but they want to know how to run something big, and so this provides that type of opportunity. The second thing is that the calendar just works. So I think it can be really hard to get students excited on a project if they can’t actually take it through the finish line. And what works about a non-partisan voter registration and voter mobilization drive is that in most states, the voter registration deadline is around four to six weeks after school starts, and then you get about another four to six weeks before the election itself, and then you’ve still got another four to six weeks before the end of the semester. And so it perfectly stages itself, provided that the faculty or some other set of students have done some of the set-up, for students to come in learn a set of skills, build skills, execute, get a couple of do-overs, and then still have time to reflect on the project before the semester is out.

Rebecca: I think that’s one thing that’s really unique about the timeline, is that a lot of kind of activity-based learning or community based learning projects, they go straight to the end of this semester and it really is hard to build in that reflection piece, so it’s nice to have substantial time to do that, and really think through that, and do post mortems and plan for the next time around so that the next set of students can learn from the previous set.

Allison: Yeah, it worked really well. I think that space allowed for a couple of assignments, both in terms of a post-mortem and having them really think critically about what they would have done differently in what advice they want to give the next group, but also for those students who want to go into this type of work, a lot of it is contract consulting work. So you’d run a campaign, and then that campaigns over, and then, what do you do next? And so one of the assignments for the class was actually to apply… mock apply for many of them. Though, a few students who are graduating did really apply for different types of political jobs. And so actually learning how to translate this real experience into a cover letter, and into a resume, and being able to pitch that what they had done was not just work for a class, but was actually work for a campaign.

John: Excellent. Did any of them end up working on campaigns?

Allison: A number of them have had internships. Someone received an internship, I believe, in Senator Schumer’s office off of the experience in her application, for that was actually what she submitted for the final project in that class.

John: Excellent. How did students, in general, respond to it? What sort of feedback have you had from students?

Allison: From the student population on campus or from…?

John: Or… well, actually from both within the class and also more broadly.

Allison: So students within the class thought that it was an immense amount of work, but also seemed very satisfied with the experience themselves. The sort of anonymous feedback sheets that I did with students over the course of the semester, students repeatedly talked about how much they were getting out of the experience in terms of learning what went on, quote unquote, behind the scenes of campaigns and how much harder it is then it looks like it is on television, comments like that. For this student population more broadly, it’s been interesting. There were definitely a set of students for whom having the voter registration and voter mobilization drive become something bigger on campus. I think it felt a little bit intrusive, though I’d argue that that’s what grassroots campaigning look like, you’re just gonna get asked if you’re registered to vote four times a day, in the days leading up to the voter registration deadline, and not for… even the students in my class who said, “I think we’re bothering people.” I said, “you are bothering people, you want them to register to vote.” So there was a little bit of that. On the other hand, students were really excited and I’ve actually had a number of students ask me if I’m running the class again, when the class is running again. Sort of having seen it happen, are really interested in getting that experience.

John: Very good. If someone were to stop in on your classroom, what would it look like?

Allison: I suspect it would initially look like chaos. [laughter]
The campaign classroom, I think, is a very different feel than a lot of other classrooms. After the first couple of weeks, I basically demoted myself to note-taker. I was technically the campaign manager, but I was really there to act as a check if I thought they were straying into something that potentially– this never happened, but I essentially was there to see do we stray into something that potentially smacks of a real problem, right? …in terms of their regularly… like election law regulations or guidelines for the campus, keeping track of the money that we still had, and what we could spend money on in the overall campaign calendar. But I would most frequently in that classroom, whoever was in charge of running a particular campaign team that was working on a strategy, would be running the meeting and I’d be at the front of the room essentially taking notes on the giant whiteboard in order to track the conversation and basically remind people of what decisions needed to be made before we left that campaign meeting. There’d also be a number of classes where you would have come to the classroom and no one would have been in it because there were either students out phone-banking, students were running a voter education program in one of our dorms, students were out running a training for other volunteers… sort of really being out in the field as much as possible and I was just running around trying to see what was happening in all of those locations and troubleshooting when it was needed.

John: So how many volunteers did they bring in from outside their class?

Allison: We ended up having over 250 unique volunteers from outside of the class that did work with Vote Oswego.

John: That’s impressive.

Rebecca: So you mentioned money and finances and so I think, I think that’s usually a big question for any sort of community project, campus project, etc. So how did you work the money side of things?

Allison: I think, one of the real benefits of the voter registration drive, is not only does it match the calendar, but it matches a place where campuses are already inclined to spend some money. So, I was able to put together money from a couple different places. One was actually from our Student Association. The student government here at SUNY Oswego put in it, ended up being close to $2,000, ultimately, that helped cover– it initially helped cover a bulk of the t-shirts actually, in visibility materials. I also put together money from our Community Services office, which is who had been coordinating the voter registration drive before. So instead of the money that they had spent running their own project, they were willing to put it towards this project. Which again, giving students access over how those dollars are spent I think was really important. I also was able to get resources from a couple different places as a faculty member, so I pitched Vote Oswego originally at a faculty academic affairs retreat at the start of a year and received $1000. The idea was voted as a best new innovation for teaching and learning at SUNY Oswego’s campus. And then I also received a Curriculum Innovation Grant here at SUNY Oswego that helped cover for me traveling to grassroots organizing training with the new voters project, to essentially get a refresher. It had been… let’s just say I was not text messaging…. that did not exist when I was last organizing… so getting a nice refresher on what sort of the the modern techniques were and best practices was really helpful.

Rebecca: How can others get involved like this particular project on this campus or run similar projects?

Allison: Yeah, so on this campus, faculty or students or staff that are interested should just shoot me an email. Definitely trying as soon as possible to start ramping up the plans for the 2018 midterm version and really starting to lay the groundwork for something big in 2020. Folks on other campuses that are interested in figuring out how this project worked, I actually just had a co-authored piece come out in the Journal of Political Science Education, it’s available as of yesterday online entitled “Vote Oswego: Developing and Assessing the Campaign-as-Course Model” that does quite a bit to outline how this project can run, where it fits pedagogically in sort of that space of taking some of the best parts of both simulations and service-learning. That article actually includes quite a bit from the course calendar, assessment strategies, as well as some student outcomes. And I want to point out that that piece was co-authored with Angela Tylock, who was one of the lead interns for the project. She graduated from SUNY Oswego in Spring of 2017.

John: Very good. We’ll include a link to that reference in the show notes.
What specific guidance might you give to other campuses trying to do similar projects?

Allison: I would just really encourage faculty and campuses generally, even if you don’t want to run it as a credit bearing course, to figure out how students can take the lead as organizers. I think, too often, students become the volunteers, right? There’s sort of a whole apparatus with lots of different nonprofits that are doing really good work to get students to vote and that’s really important, but I think on campuses, we’re missing really big opportunities if we treat elections as an opportunity to get students to vote, but not as an opportunity to get students the skills that they are gonna want and need for a whole variety of things. If you want to go work in a non-profit, you’ve got to know how to build a coalition. If you want to work for your kids’ PTA and make sure that they’re getting the resources they need, the ability to run a meeting and get petition signatures, is actually really important. And all of those types of civic skills are things that students can and, I think, should be getting by volunteering or helping to run one of these drives.

John: So it’s very much an active learning exercise…

Allison: Absolutely, absolutely.

John: …where students played an important role in building it.

Rebecca: How did you get students to take that active role? I mean, it’s easy to assign tasks and be the leader, but how did that feel?

Allison: There was… definitely, I had to be fine with a level of loss of control that I am often not fine with in my classes. What I did is actually work to set up the first two weeks of the semester, I had the calendar planned out, so I had worked ahead of time to set up tables and events that were happening for orientation, had coordinated with faculty around campus to have individuals come in and give announcements and register students in that first 10 minutes that really, the first week of school, you can almost always, the first day of class, give up 10 minutes, after you review the syllabus, to get some students registered to vote. After that first two weeks, the students who were enrolled in the course, had had an opportunity to be trained in those skills, to get their feet wet in the skills, to get feedback on the skills and then I didn’t plan anything else. I basically said, “now it’s up to you, what are we gonna do?” And goals have been set for the campaign so the students knew, “here’s where we want to get, here are what we think our rates are gonna be. So if we want to register 500 people from tabling, here’s how many table hours we need scheduled, how are we gonna make that happen?” And I think two things then happened. One, I stepped back and basically told students, “you’re the expert on where students on this campus are.” I come here, I go to work, sometimes I go to events, and then I go home. I don’t actually know what dining halls are packed on what days. Turns out chicken sandwiches, big deal, chicken sandwich day at late night, right? There are all of these things that happen on campus that, as a faculty member, I don’t know about. So students basically learned that I wasn’t gonna tell them not to register students at 11 p.m. at night if that’s where they thought students were, they ran with it. The other thing that happened is they realized that I would let things fail. If students scheduled events and those events went poorly, they went poorly. And I wasn’t gonna fix those events for them– with the exception of confirming that registration forms were filled out correctly. We had an entire process for making sure that voter registration forms were correctly done. But in terms of the grassroots apparatus around that, if students didn’t plan well, they didn’t plan well, and they were the ones that had to stand there while the event went poorly. And I think between those two things, the students really became engaged around sort of their responsibility and taking ownership over the campaign.

Rebecca: Was most of the learning then taking place by “let’s try this, let’s fail, let’s try again,” an iteration rather than like doing readings or other kinds of … ?

Allison: Yes, there were minimal readings while the campaign was actually happening. There was quite a bit of reading and reflecting once election day happened, but prior to that, it was much more “these are the tried-and-true tactics, what do you want to get out and do? How do you think these tactics will best adjust to the environment that you’re in and the student population were working with?”

John: And students learn a lot more by making mistakes and recovering from them, and it sounds like you set up a mechanism where there was lots of feedback from each other.

Allison: Yes.

John: That’s excellent.

Rebecca: So, usually we wrap up these conversations by asking what are you gonna do next?

Allison: Next for Vote Oswego is an effort to improve the connections between the voter registration drive as a grassroots campaign to the voter registration drive as an overall campaign that involves lots of different components. So actually, Rebecca, had a class that worked on the website for Vote Oswego, it was a project for one of her classes we’re hoping to do, I think, much more of that for the 2016 campaign as well as trying to figure out what other faculty or other classes could also use a voter registration drive… benefit from that timing… benefit from the fact that it can be student driven and student owned in a lot of ways, to really get their classes involved with this as a project.

John: Very good.

Rebecca: Thanks for taking the time out and sharing your project. I think it probably has encouraged a lot of people to start thinking about those midterm elections and how they might be able to get students tapped into it.

John: And will you be doing this every other year now?

Allison: The goal is to do it every other year. I haven’t done it for a midterm yet, I think that will be different. I think there will be more actual campaign literature there just because it will be difficult to get it sort of as ramped up as a presidential election, but the goal is to do it every other year.

John: Very good. Okay, well thank you.

Allison: Thank you.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

14. Microcredentials

In this episode, we discuss the growing role of microcredentials in higher education with Jill Pippin (Dean of Extended Learning at SUNY-Oswego), Nan Travers (Director of the Center for Leadership in Credentialling Learning at Empire State College), and Ken Lindblom (Dean of the School of Professional Development at the State University of New York at Stony Brook). Jill, Nan, and Ken are members of a State University of New York task force on microcredentials.


Rebecca: Our guests today are: Jill Pippin, the Dean of Extended Learning at SUNY-Oswego; Nan Travers, the Director of the Center for Leadership and Credentialing Learning at Empire State College; and Ken Lindblom, the Dean of the School of Professional Development at the State University of New York at Stony Brook.

John: Welcome, everyone!

Nan: Thank you. Hello.

Jill: Thank you.

Ken: It’s good to be here.

John: Our teas today are:

Rebecca: Jasmine green tea.

John: Jill?

Jill: I actually don’t drink tea.

John: Oh… here we go again…. Okay, Nan?

Nan: I’m drinking Celestial Seasons Bengal Spice.

John: …and, Ken?

Ken: My tea today is coffee.

John: …we get a lot of that…. Ok.
…and I have black raspberry green tea from Tea Republic.
So, today we’re going to be talking about microcredentials. Would someone like to tell us a little bit about what microcredentials are?

Ken: Sure, I’d be happy to tell you a bit about what microcredentials are. So there are traditional microcredentials that most people know all about, such as certificates, minors, or just either credit or non-credit certificates. So they’re pieces of larger degrees, but there are now new digital microcredentials that are having a bigger impact on the field, and that internet technology has allowed us to take more advantage of. So there are internet certificates and there are also digital badges, which are icons that can be put on a LinkedIn resume or shared through somebody’s website or on a Twitter feed… and they indicate that the earner of the microcredential has developed particular skills or abilities that will be useful in the workplace.

Nan: …and just to add to what Ken has said, with the open digital badges that are out there, they actually hold on to all of the information around the assessed learning…. the different competencies that an individual has, and the ways in which they’ve assessed it. So if they’re used, let’s say, in the workplace, an employer could actually click into the badge and be able to see exactly how the person has been assessed… which gives a lot of information that a traditional transcript does not give, because it does have that background information in there.

John: Who can issue microcredentials? or who does issue microcredentials?

Jill: …really industry, colleges, various and sundry types of organizations.

Ken: Yeah, in fact, Jill’s right. There’s no real regulation of microcredentials right now. So they can be given by any group that simply creates a microcredential and awards it to someone… and then they say what it is. So the microcredential’s value is really based on the reputation of the issuer.
Honestly, universities and colleges are pretty slow to get to this kind of technology, as we often are. So it’s new for us, but there are private companies that have been issuing them, and there been individual instructors at the college, and especially at the k-12 level, who have been using badge technology to motivate and to assess student work for quite a few years… but for the university level, this is exciting new territory that we’re really jumping into now.

Jill: Yeah, microcredentials are shorter… they’re more flexible…. and they’re very skill based… and so they’re new for colleges, I think in a lot of ways….. maybe not so much for our non-credit side of the house… those that have been doing training programs and things are very practical… skill-based pieces… but in terms of having ladders to credit and having credit courses seen through the lens of a smaller chunk of time, and of topic area, and focus… I think that’s the real change or the real difference in micro-credentialing than from a traditional environment…

Nan: …and what’s really important here is that the demand for these really, in many ways, is coming from industry where they really need better signals as to what people know and what they can do, and as Jill just mentioned, that they’re very skills based. This enables somebody to be able to get a good idea about what a potential employee is able to do. So the demand for microcredentials is really increasing, as industry are using them more and more and there’s many different groups that are really focused on using either the microcredentials, or specifically the badges (which is really a type of microcredential). There are some projects right now where there are whole cities that have come together and have been developing microcredentials and badging systems to make sure that all people in the community have the ability to show those skills as they go for employment. There are also some companies that are starting to come out. For example, there’s a company called “Degreed,” which is degreed.com. It’s a company that enables people to get their skills assessed and microcredentialed, and at the same time working with companies… there’s some big companies such as Bank of America… there’s many other ones that are on their website listed… and they work with the companies and identify the different skills that people need… and then credential the people who are trying to apply with those…. so that there’s a real matching. It becomes a competency-based employment matching system in many ways.

Ken: Some of the ways that badges have been useful are exactly what Nan and Jill are saying, that it’s come from the employers who are asking for specific information about what students will come to them with. We are also able to develop badges in concert with specific employers, if there’s particular training or education or sets of skills or abilities that they’d like their applicants to have… but there’s also another great advantage to microcredentials, particularly badges, that allow us to show the in-depth learning that goes on in classes. My other hat, other than Dean, is that I’m a Professor of English, and so in a lot of humanities courses the direct connection to skills isn’t as obvious to people as it is in an area say like teacher education. So what we can do with a badge is we can point out the specific skills that students are developing in a class on rhetorical theory, or on Shakespearean plays, or whatever. We can point out the analytical learning that they’re doing, the kind of critical thinking, the kind of communicative writing, so that those courses translate into the kind of skills that people are looking for… and of course, our students are picking those things up, but now we can make it more visible as a result of the technology of digital badges.

Jill: It’s an exciting time in higher education. I mean it really is, in terms of microcredentials, because higher ed has the opportunity to validate those credentials. A lot of them, as we said before, have been out there… non-credit skill-based smaller chunks of learning… but the idea of having them all kind of on the same playing field… and almost apples-to-apples in terms of validating learning outcomes… and making sure they’re part of a longer pathway toward higher education. It’s really exciting.

John: When someone sees a transcript and sees English 101 or English 373 or Eco 101, it doesn’t really tell the employer that much about what the students actually learned, but the microcredentials provide information about specific skills that would be relevant. Is there much evidence of the impact this has on employability or in terms of career placement?

Nan: There has been some work that is being done on that, and as I mentioned there are some companies that are even starting to get in the field because there is such a high demand for companies to be able to do competency-based hiring. There’s an initiative that the Lumina Foundation has been funding called Connecting Credentials and, in that initiative, they’ve been looking at microcredentials as a piece of that. That initiative has brought together many different businesses, organizations, and higher education together at the table to really discuss ways in which credentials can better serve all of those different sectors… and so some of the work that they have been working on and that can be viewed at connectingcredentials.org has really been looking at some of the impact of microcredentials on employability.

John: Based on that, I would think, that when colleges are coming up with microcredential programs, it might be useful to work with businesses and to get feedback from businesses on what types of skills they’re looking for… for guidance or some help in designing microcredential programs?

Jill: Absolutely.

Ken: Yeah. I can talk a little bit about some experience we’ve had at Stony Brook on that. We’ve been working with an organization called FREE which is Family Residences and Essential Enterprises. They’re a large agency that supports students, children, and adults with disabilities… and we worked with them to create several badges that align directly with their national standards and the certification needs of their employees. So now we’ve got a system where one of the things that their employees need is food literacy. If they’re running a house for people with disabilities, people who need assistance, they have to be able to demonstrate that they’re able to produce healthy nutritious meals… and so once they’ve gone through this training, which is specifically aligned with their curriculum, having earned the badge will demonstrate that the employee has developed that set of skills. We’ve also got one for them on leadership among their managers and we’re developing more… and the fact that we’ve developed that with the employer… and now the employer is actually contracting with us to deliver that instruction to their employees. We’ve done really well and we’ve issued well over a hundred badges to that agency in just about a year.

John: Excellent.

Nan: There’s also, as we think about it from an employability perspective… there is also another important area that’s happening with the microcredentials and the badges in higher education…is to really be looking at some of those more liberal arts kinds of skills: being able to be a good communicator… to have good resiliency… these are also very important pieces that go into being a good worker… and so there are many institutions as we look across the United States that are really looking at some of these broader skills. There’s also some work that’s being done on the student services side which is really looking at how students have been engaging and being involved within the institution. So, there are these other pieces that also help to build that whole person… how somebody really is involved in higher education… what they know… what they can do… and the kinds of different volunteer pieces… as well as the different kinds of things that they have engaged while they are they are there: working in teams, doing different projects. So, there’s lots of different ways of using those badges. There are also some institutions who are using these badges as a beginning point for students. For some people, it’s scary to start at higher ed again, and to be able to take a little bit of a program that’s a smaller program that actually has a credential at the end of it, is a really motivating thing. Students come away saying: “Well I did that. I can do more…” and so it becomes a really good recruitment tool… but it also is a real good student support tool in order to help people start the path of education as well.

Ken: …and you know, Nan, that’s an important point too… and it works the other way for people who are in, let’s say a master’s degree program…. they don’t not learn anything new until the very end when they’re issued the degree… they’re actually building skills and developing abilities all along the way. So, what the digital badge or a microcredential can do is make visible the learning that they’re doing along the way. So after three or four courses, they’ve earned a credential that demonstrates that value. So they don’t have to wait until they finish 10 or 11 courses.

John: So, it lets them have small goals along the way, and they’re able to achieve success, and perhaps help build a growth mindset for those students who might not have done that otherwise.

Ken: Yes.

Nan: Yes.

Ken: Well put, John.

John: How does this integrate with traditional courses? Are there badges that are offered… or a given badge might be offered by multiple courses? or do individual courses offer multiple badges or microcredentials?

Ken: It can go in lots of different ways. There are instructors who build badging into their own classes. Those aren’t really microcredentials the way we’re talking about them. We’re talking about microcredentials that are somewhere between a course and a degree. So, at Stony Brook, for example, we have what we call a university badge program, and in order for a University badge to exist, it must require between 2 and 4 4-credit courses. So a total of 6 to 12 credits, that’s the point at which students can earn a university badge at Stony Brook University. Those courses work together. So, for example, we have a badge in design thinking, and in order to earn that badge students must get at least a “B” on two courses that we have on design thinking. We also have a badge in employer-employee relations within our Human Resources program… and in order to earn that badge, there are three specific classes that students have to take and earn at least a B on each of those classes.

Nan: So, there is also another approach in terms of thinking about how the microcredentials can intersect and kind of interface with the traditional credentials, the traditional degrees, and that’s through different forms of prior learning assessment. So, what we also see is that students come with licenses, certifications, different kinds of these smaller credentials that represent verifiable college-level learning… and through either an individualized portfolio assessment process or, at our institution at SUNY Empire State College, we also have a process called professional learning evaluations… where we go in and evaluate training, licenses, certifications, and those are evaluated for college credit . Those are then also integrated within the curriculum, and treated as… really transfer credit… they’re advanced standing credit. So, students also have the ability to bring knowledge with them through the microcredentials… they’ve been verified by another organization, and then we re-verify that learning at a college level to make sure that it is valid learning for a degree… and then integrate it within the curriculum.

John: In Ken’s case, it sounds like the microcredential is more than a course, in other cases it might be roughly equivalent to a course… or might it sometimes be less than a course? Where a course might provide individuals with specific skills, some which they might have in other courses? or is that less common?

Jill: You’re right, there’s a spectrum. So, for instance if you look at it from a traditional standpoint, a technology course might already have an embedded microcredential in the form of OSHA training, for example. That’s a microcredential, in that particular example, and so we have the opportunity to look at the skill based smaller chunks that may be very specific to an occupation or employers need for someone to have those skills and be able to put some framework around it so that it can be understood and communicated to an employer.

Ken: One of the exciting things about badging and microcredentials right now which Jill alluded to earlier is that there really isn’t any regulation regarding them yet. So when you say a college degree, that has a standardized meaning but when you say a microcredential or a digital badge, there’s no standardized meaning whatsoever, so what we’re doing is we’re creating different versions of microcredentials and the meaning of them is dependent on that specific situation. So one of the things that’s exciting about being a University in a College is we can really bring academic rigor to these no matter how many skills and what level of learning of the digital badge represents… you know because it comes from a university particularly a SUNY it’s going to be a high quality badge. But it’s incumbent upon the one who’s reading the badge to understand what that badge actually means, and depending where it comes from, depending on the size of the badge, and what the number of skills and abilities aligned to it are, the badge means different things and that’s why it’s so important that the badge includes the metadata – all that in depth and formation that you get when you click on the digital badge icon and all of that information pops up.

Nan: In addition, nationally the IMS global learning community has been developing standards and hopefully there’ll be national standards around the data, how that’s reported, and being able to allow people to really understand and compare the attributes of the criteria of how it’s been assessed, and so there’s a great deal of work that’s being done at a national level to really be thinking about how we can have some good standardization and guidelines around what we mean by certain things in the digital badging. So I think that’s something to pay attention to in terms of what’s coming about.

Ken: Yes it’s exciting space before the standardization has been done, because there’s a lot of innovative potential there, but as we standardize there’ll be more comparability and that’ll be easier to do. So, we may lose some of that innovation later but we’ll just have to see. It’s very interesting to be at the beginning of this process like this because degrees were really kind of finalized at the end of the 19th century, and now at the beginning of the 21st century we’re reinventing that kind of work.

John: Now earlier, it was suggested that other groups have been creating micro-credentials in industry and private firms. One of the advantages, I would think, perhaps that colleges and universities would have is a reputation for certifying skills. Does a reputation of colleges perhaps in universities give us a bit of an edge in creating microcredentials compared to industry?
JILL : One would hope, however there are examples of all sorts of industry entities out there that are offering microcredentials – think of the coding academies that are prolific and they’re very skill based, very specific to an industry, in the industries needs the employers understand what that outcome is from that training and they’re able to therefore value it, and the employee is able to communicate it very effectively. But where I think the colleges have an opportunity and universities have an opportunity to really shine here is that this is where we have the experts, we have people who are very well-versed and researched in their area of scholarship, and they’re able to really look at curriculum and validate it, and make sure that it is expressed in terms of college-level learning outcomes.

Nan: In addition, I think that higher ed has the opportunity to really integrate the industry certifications with curriculum and the stacking process bringing in those microcredentials from industry or having them right within the higher ed curriculum and then being able to roll that in and build it into the curriculum, so that a degree, I can imagine, as we evolve higher education over the next decade or so, that people as they graduate… they’re graduating with a college degree, they’re graduating also with microcredentials, and together they’re able to really indicate what a student knows and what a student can do which really can help the student a great deal more than when it’s just a degree that doesn’t really spell out what some of the details about what somebody knows.

Rebecca: I’m curious whether or not there’s any conversations happening with accreditation organizations about micro credentialing and how they might be involved in the conversation.

Nan: So at this point there are conversations that are happening at the accreditation level and for example, every regional accreditation agency has policy around the assessment of learning. Sometimes specifically around prior learning assessment, sometimes around transfer credit, which within those policies they’re really starting to look at how those learning pieces can come in. When it’s on the for-credit side, then there really needs to be a demonstration by the institution that those microcredentials are meeting the same academic standards as the courses are also. So using the accreditation standards and making sure that all policies and procedures are of the same quality and integrity ensures that it all fits together.

Ken: I think it’s not only an opportunity for universities that we’re developing micro-credentials, but I think it’s our responsibility to do so, because the idea of digital badges for example was popularized in the corporate sector before universities got on board and they ran the gamut in terms of quality and value and frankly there are some predatory institutions that award badges that may not have much value at all to students, and yet they can be quite costly. So I think it was very incumbent upon the university to create valuable microcredentials that would have real academic rigor and support behind them. In addition to that, some of these institutions were also using their badge programs to undercut the value of the degree and say “Well, you don’t actually need a college degree with all that fluff, you just need to get the skills training that you’ll get from a badge.” And we know that a college degree delivers far more than just a set of discrete skills, it gives better ways of seeing the fuller world, of understanding the integration of knowledge, of being able to employ social skills along with technical ability, and digital badges at the university level allow us to make those connections more visible. But it also can help us prevent attacks against the university, which are done purely from profiteering perspective sometimes.

Jill: We can provide some validity and some academic integrity to the smaller microcredential world, then I think higher ed as Ken says has a responsibility to do so.

Nan: It also shows a shift in some of the role of higher education where it becomes even more important that we take the lead in helping to integrate people’s skills and their knowledge and then how that relates to work and life. In many ways, the older higher ed… we had a much more of a role of just delivering information and making sure people had information. Now I think our role has really shifted, where we need to take the leadership in the integration of knowledge and learning.

Rebecca: I’m hearing a lot of conversation focusing on skills and lower levels of the Bloom’s taxonomy, so it would be interesting to hear of examples at higher levels of thinking and working.

Ken: Well, Bloom’s taxonomy actually is a taxonomy of skills and domains of knowledge and abilities so that there are certainly skills involved with synthesis and evaluation, which are at the top of Bloom’s taxonomy. So digital badges can work with that. Digital badges… the skills can involve being able to examine a great deal of knowledge and solve specific problems in an industry, and these are the highest levels of application of knowledge and learning.

Nan: In higher ed they’re also being looked at both at the undergraduate and graduate level, and so it’s not just that entry-level piece. Again, we keep talking about licenses and certifications as a type of microcredential, and there are many out there that you cannot acquire until you have reached certain levels of knowledge and abilities. I know we have focused a great deal of this conversation in terms of being skills-based, but in industry they’re really talking about it more as competencies, and the definition of competencies is what you know and what you can do, so it’s both knowledge and skill space, it is not just skill space.

Ken: In fact, one of the issues that some faculty have with microcredentials, particularly digital badges, is that they have a sense that it’s focused too heavily on utilitarian skill, and not focused heavily enough on the larger and the higher levels of learning that Rebecca is talking about. So I think Nan’s bringing in the idea of competency-based learning is really very helpful that way.

John: So, basically those skills could be at any level.
What are some of the other concerns that faculty might have that might lead to some resistance to adopting microcredentials at a given institution?

Nan: So one of the areas that they may talk about is the concern of the integrity. The academic integrity of the microcredential, or of the badge. And what’s important is that each institution really look at their own process for reviewing microcredentials and improving them, especially if they are on the credit side and they’re going to be integrated within the curriculum. So they need to follow the same standards that any course will follow, and that should really help relieve that concern about academic integrity.

Ken: Yeah, in fact the SUNY microcredentials group, which all of us on this podcast are involved with, specifically points out that faculty governance has to be heavily involved in the creation of any digital badge or micro credential program. That’s the whole point of bringing the university level to this. Is that faculty governance that academic input is going to be behind every microcredential that we create. One of the other things that my faculty colleagues have had trouble with, is the very name of digital badges, and they think it sounds a little silly, a little juvenile. They always say, “oh, well, this is just Boy Scouts and Girl Scouts” and so to them it can feel a little silly. It actually doesn’t come from Boy Scouts and Girl Scouts. Digital badges come from gamification and motivational psychologists looked at why people were willing to do so many rote tasks in an online game. Even though they weren’t being paid to do so, and didn’t seem very exciting on its own and what they found is that people were willing to do that because they would earn a badge or they would level up or earn special privileges along the way, and that was very motivating for people. That’s where this technology really came from and then we built more academic rigor into it. The metaphor that I like to use with my faculty colleagues, which was suggested to me by one of my English department colleagues, Peter Manning. He pointed out that in the medieval period in England archers would learn different skills and when they developed a new skill, they would be given a feather of a different color, and then that feather would be put in the cap. So literally a badge is like a feather in the cap, and when you see somebody coming with 8 or 10 feathers of different colors, this is going to be a formidable adversary. Just like people with a few did badges from the SUNY system, they’re gonna be formidable employees.

Jill: The other thing I like to jump in and say too is – the Girl Scout in the Boy Scout badging system if you really know what the badges represent – you know that there are very strident rules learning outcomes and so on involved in attaining the badge. The badge is a way of just demarcing that they attained it. The quality is inherent in the group that’s setting up the equation by which you earn the badge.

John: So it’s still certifying skill.

Jill: It’s still certifying something and again the institution has the ability to determine what that something is, and to make sure that it is of quality.

John: Now one other thing I was thinking is that if an institution instituted a badging system, it might actually force faculty to reflect a little bit on what types of skills they’re teaching in the class, and that could be an interesting part of a curriculum redesign process in a department, because we haven’t always used backwards design where we thought about our learning objectives. Quite often faculty will say, I’d like to teach a course in this because it’s really interesting to me, but perhaps more focus on skills development in our regular curriculum would be a useful process in general.

Jill: I agree.

Ken: I think that’s a great idea, John. We haven’t used the badging system in my school that way yet, but I think it’s a great idea and honestly there are faculty who bristle at the notion that their teaching skills, and digital badging really strikes at the heart of that, in my perspective, elitist attitude about education. We do want to open up students Minds, we do want to expose them to more of the aesthetic pleasures of life, but we also want to help students improve their own lives in material ways as well, and badging can help us make visible, and strengthen the ways in which we do that in higher education. I think we should be very proud of that.

Nan: So again one of the reasons I like to use the word competency, is because it brings the knowledge and skills together, and we’re actually talking about skills as though they are isolated away from the knowledge pieces, and you can’t have skill without knowledge. To develop good knowledge, you need certain skills, and so I think it’s important to really think about this not as two different things that are separated and somehow we all of a sudden are going to be just skills based, but much rather that we’re developing people’s competencies to be highly educated people.

Jill: Very symbiotic really, and I think this is also where you get at the idea of how can non-credit and credit work together. If you’re thinking about them, in terms of the outcomes and developing your class in that way, and if one of those by itself would be something that’s non-credit, and then if you build them all together then you get a course. Or then your couple of graduate courses together, then you get a credential that is something on the way to a graduate degree.

John: This brings us to the concept of stackable credentials or some microcredentials designed to be stackable to build towards higher level credentials.

Ken: Really a micro-credentialing systems, should always be stackable. That’s one of the bedrocks of the whole idea of it. So it’s not required that a student go beyond one microcredential, but microcredentials should always be applicable to some larger credential of some sort. So, for example, all of the university badges at Stony Brook University stacked toward a master’s degree. And in fact we’ve tried to create what’s called a constellation of badges, so that students can wind their way to a master’s degree by using badges… or on their way to a master’s degree they can pick particular badges to help highlight specialties among electives that they can choose. So it’s a way for them to say, yes I have a Master of Arts in Liberal Studies, and as part of that I have a particular specialization in financial literacy, or in teacher leadership, or an area such as that. But yeah, microcredentials should always be able to stack to something larger. And if we do it right, eventually we’ll have a system that works really from the first… from high school to really into retirement, because there can be lifelong learning. That’s involved in microcredentials as well. There’s always more to learn, so there should always be new microcredentials to earn.

Nan: I totally agree with Ken and if we provide different microcredentials and don’t provide how they do stack and build a pathway, then we really have not helped our students. In many ways we have left it, traditionally, historically, left it up to the individual to figure out how their bits and pieces of learning all fit together and we kind of expect that they’ve got the ability to kind of put it all together and apply it in many different ways, and I think that the role that microcredentials is really playing here, is a way of helping us start to talk about these discrete pieces, and then also how they build together and stack, which gives the person the ability to think about how it fits into the whole. I think what microcredentials is doing is opening up higher education, in a way to really be thinking about how to better serve our students, and give them those abilities to take what they know, package it in different ways, be able to apply it in many different ways, and be able to build that lifelong goals, and seeing how it all fits together.

Rebecca: Just thought I’d follow up a little bit. I think a lot of examples that we see are often in tech or in business and those are the ones that seem very concrete to many of us, but for those of you that have instituted some of these microcredentials already, how does it fit into a liberal arts context, which might not be so obvious to some folks?

Nan: There’s actually quite a few examples of microcredentials and badges that are more on the liberal arts side. There’s been some initiatives across the United States where different institutions have been developing, what we can think of as the 21st century skills: communication, problem-solving, applying learning, being resilient. These are some of the kinds of badges that are starting to really evolve out of higher education, which really brings in those different pieces of a liberal arts education, and being able to lift that up and give the students the ability to say, “I’ve got some good problem solving skills and here’s some examples and I can show it through this badge.” When we look at the research in terms of what employers need for the 21st century employee, we’re really looking at very strong liberal arts education that is then integrated into a workplace situation. So I’m seeing a lot more badges being grown in that liberal arts arena.

Ken: Yeah, at Stony Brook University, we have a number of badges that are in the liberal arts. So for example, we have a badge in diverse literatures. So there may be people who wish to earn that just for personal enrichment, but it’s something that might be really interesting to English teachers as well, because by earning a badge in diverse literatures, which requires a minimum of three classes in different areas, different nationalities of literature, teachers will be able to go on to select pieces of literature more appropriate for diverse audiences. They’ll be able to explore greater world literatures because of the background that they’ve had in exploring different literatures in their classes. So, that’s just one example, but of our about 30 badges, about third of them are in those humanities areas. That said, I will acknowledge that they are not anywhere near as popular as the more business oriented and professional oriented badges, where the link to skills simply seems more obvious. So I think that the liberal studie… the liberal arts… the humanities badges.. the connection is not quite as clear and so there’s still a lot of potential there.

Jill: It’s so important for the employers and for the students themselves, but I think almost most importantly the employers to understand what that means. They have to understand you have a microcredential or a badge and problem solving. They have to have some kind of trust, that it’s truly a skill that equates to their workplace situation, and that’s where the online systems where you can actually delve into what’s behind the my credential, is so important. You can really sit there and look at it, and verify that what the competencies and the skills that the individual has attained through earning this badge.

John: So the definition in the metadata is really important and establishing exactly what sort. Now that brings us to another question. At this point each institution that’s using badges is developing its own set of badges and competencies. Has there been any effort at trying to get some standardization and portability of this across institutions or is it too early for that, or do you see it going in that direction at some point?

Ken: John, it certainly hasn’t happened yet, but I do know that the SUNY Board of Trustees at their last meeting started to consider developing working groups to do just what you’re saying. So it’s not so much to standardize what badges are, but rather to standardize reporting and explore ways to help badge earners to explain and demonstrate their badges to employers, and to other schools more easily. So I know that’s where the SUNY system is headed.

Nan: And if it is for credit, then it falls within transfer credit anyway. So really, if it has gone through the appropriate academic curriculum development processes, the governance processes, then it has the same rigor and therefore is very transferable through our policies on transfer. So really what we need to be doing is doing some good work around the non-credit side,…that really helps the transfer of non-credit learning.

Jill: And one way we can do that is by reinvigorating and breathing new life into a 1973 policy that SUNY has on the books for the awarding of CEUs )continuing education units). It has a recommendation in a process by which campuses can take non-credit curriculum and send it up through a faculty expert and it has a certain guideline about how do you come up with an approval process and how many CEUs could be granted for such work. So, there are some skeleton pieces to how SUNY may codify that moving forward, at this point there is not a rule about how to move forward with non-credit. In fact, SUNY I think trying to be responsive to the emergent nature of this very concept, it has not tried to come in and be too prescriptive yet.

John: On the other hand, when students do receive microcredentials at multiple institutions. Let’s say they start at a community college. They move perhaps to Empire State, maybe they move to a four-year college for university, if they don’t finish and get a degree, they still would have some microcredentials that they could use when they go on the market, because many of them perhaps might use Credly or some other system where they can put it on the LinkedIn profile and they still have that certification, which if they just don’t get the degree it just shows them as not being a degree recipient, which actually seems to hurt people in the job market, but perhaps if they could establish that they have been acquiring skills a long way, maybe it might be helpful for students.

Nan: John, that is a really good point. In many ways, our degrees set up a system where if anyone who steps out of a degree has nothing to show for it and therefore is at a disadvantage, and the microcredentials can help demonstrate their progress, and the competencies that they already have, and so it can play a very important role in people’s lives, when students do need to step in and out of higher education.

John: So where do you see microcredentials going in the future? How do you see this evolving?

Ken: It’s in such an amorphous space right now, it’s hard to imagine what it’s going to undulate into. A big part of what’s happening now is what Nan has talked about. An attempt to try to put some boundaries on this and bring some common definitions to bear on the technology and and the idea of a microcredential, but I think it’s going to still expand. What it’ll do is it’s going to increase partnerships among interesting groups. I think in a lot of these, the universities will be at the center of the partnership, but we’ll be bringing in many more student groups, industry partners, government groups, nonprofits. I think it’s going to increase the amount of communication dramatically, and that’s very exciting because for too many years universities have fulfilled that stereotype of the ivory tower, and this is really breaking that down in some very productive ways.

Nan: And when we look at it from a national perspective, and looking at it to see where some of the direction is going with groups such as IMS global, with connecting credentials and other groups, but what we’re really seeing is the prediction that every student would have a comprehensive digital student record, that they would take with them. It becomes a digital portfolio and the badges would be in their microcredentials, any degrees, they’d have an ability to be able to transport themselves in many different directions, because all of that information would be there, and that digital student record would allow anybody to click in and see the metadata behind it, to know what those competencies that people have, and how it was assessed, what it really means so that there’s a real description of that. That would also enable students to have, again the prediction is that students would be able to transfer from institution to institution. They’ll be able to stack up and build their degrees in ways that would really support the student in their whole life pathway. Ken has just mentioned about partnerships. I think that what we would see is a great deal of partnerships across institutions and with institutions in industry, that really start to build these pathways that people can move along with their comprehensive digital student record.

Ken: Nan, can I ask you a question?

Nan: Yes.

Ken: So a few years ago, there was a lot of talk about they termed co-curricular transcripts, which would be the kind of transcript that would include club membership, informal learning, not credited learning, but it sounds like we may be getting beyond that in a really positive way, and that just the idea of a transcript is becoming a little transformed, so that those other kinds of learning will actually be transcripted in the same digital format. Am I reading that right? Do you think that’s where we’re going?

Nan: Yes, I do think that’s where we’re going, Ken. We’re right at the end of a multi-year, multi-institutional project that Lumina funded, looking at these comprehensive digital student records, that go way beyond… also capturing things like clubs and other kinds of things that students engage, but really, they’re competency-based they start to record those competencies, the data behind the competencies, and when students are in a club or when they’re doing other kinds of activities, the kinds of competencies that they’re gaining from those pieces are also being recorded. So it’s not just: “You are a member of a club, what did you really learn and what can you do because of that?” and so I think that we’re gonna see that evolving more and more over the next decade or so.

Ken: That’s great, thank you.

Jill: If I could add to the question about the role of microcredentials evolving. One of the things that I think is going to be happening, and part of why I’m so excited about microcredentials is, I see this as having a nice connection for the non-credit side of the house of colleges and universities to the credit side, because for so many years, non-credit has been connecting with, and trying to serve business and industry, in ways that really have been limited, and so this really opens up the ability to connect and collaborate with credit expertise within the institution, to be able to create those true pathways, from start to finish from the smallest first step along that pathway all the way through, and that’s really exciting, and I think… and I hope… that’s part of this overall discussion we’re having about micro-credentials moving forward. In a lot of ways this is cyclically. We talked about the CEU policy of 1973. There has been this two sides of the house as they say, as I said a number times today, and really we’re all about education and trying to help people to learn things and be able to apply them to their jobs and their lives and having that connection be that much more seamless and clear. I think that’s one of the most exciting things, from my seat at the table.

John: Well, thank you all for joining us.

Nan: Thank you

John: Look forward to hearing more.

Jill: Thanks for having us.

Ken: Pleasure to be here.

Nan: Take care everybody, bye bye.

Jill: Bye, guys.

Rebecca: Thank you.

13. Authentic Learning

In this episode, Rebecca Mushtare discusses how she has used community-based learning and simulation projects to provide authentic learning experiences in her design courses.

Show Notes


John: Today, our guest is… Wait, there’s no one in the guest chair. Who’s our guest today?

Rebecca: It’s me! It’s me!

John: Oh, yeah. Okay. Today, our guest is Rebecca Mushtare, who will be talking about how she uses authentic learning techniques in some of her classes.
Today, our teas are:

Rebecca: Comfort and Joy.

John: Peppermint Bark. So we’ve got some holiday tea left over from the holidays. So when people talk about authentic learning, what do they mean?

Rebecca: It really means something, it’s like a real world problem of some sort, or where students are gaining experience as a professional or in something that’s very similar to a professional. A lot of times, authentic learning experiences include ill-structured problems. So not like the kind of question-and-answer things that we might have in a very structured classroom context, but where it gets messy. There’s variables that we can’t necessarily plan for in advance. That often happens and then a lot of times they’re also project-based exercises or experiments too.

John: So, one of the main reasons for using these authentic learning exercises, besides providing students with training that’s relevant for their field, it also provides them with learning experiences where there is quite a bit of intrinsic motivation, right?

Rebecca: Yeah, I think that students respond really positively to authentic learning experiences, because they can see how it’s relevant to them, and relevant to their professional careers. So even when I do small exercises in class, like writing an email as a professional, students latch on to that writing opportunity more so than other kinds of writing opportunities, because they understand that that’s important, relevant and necessary.

John: What types of activities have you used in your classes?

Rebecca: Well, at first it’s probably important to understand what kinds of classes I teach, to kind of get some context.

John: So, what types of classes have you used these in?

Rebecca: Yeah, so I predominantly teach studio- based classes, mostly web design courses. So, I’ll focus on those, because those are the ones of my regular load. So those are the things that I teach most frequently and I’ve done the most experimentation in. I do community-based learning or a form of service-learning, and I also do simulations, and it depends on whether it’s a beginning or an advanced class, which one I do. Community-based learning, or these service-learning, opportunities are generally working with a community client… generally a nonprofit organization, who doesn’t have the capacity or the budget to hire a professional design agency to do something. So we’re providing a service in a way that builds their capacity. In my advanced classes, I’ve done a lot of community-based learning. Locally, we’ve done the Children’s Museum of Oswego website, the Childrens’ Board of Oswego website and students are also wrapping up a project for the Oswego County Airport. So all of these are possibilities where they get to design a real website… they work as a team and I serve as the creative director, so this is different than an internship or other opportunity where they might get real-world experience because they’re getting a lot of coaching throughout the entire process, that they may or may not actually get in some of those other contexts like a volunteer or an intern.

John: When you serve as a creative director… could you provide a little bit more detail on that role for those of us who don’t work in those areas?

Rebecca: Yeah, it’s a step up from maybe a project manager and that I oversee all of the creative decisions, including what research methods are gonna be used to learn more about the audience, and the client… making sure that whatever meetings and things we have are all scheduled through me, so that I’m highly informed and participating in the project. It’s not something that students are doing without me being involved. It’s not an “outside-of-class assignment,” where they’re just doing a group project and they do whatever with the community and then show up and it gets done. Rather I’m heavily engaging with that community partner as well, and this is really important because of the longevity of the project. At some point the students are gonna go away. The semester ends… and the project may or may not be done. I either need to have another class that finishes up the project… I might finish it up…. or whatever needs to be done, I need to make sure that that continues, because the timeline of a client or community organization is quite different from our semester schedules. My involvement is really important in that role.

John: …and that helps with buy-in from the community partners, it provides assurance that the tests will actually be completed…

Rebecca: Right, and to actually to be able to do a project like this in a semester requires some significant planning on my part with a community partner in advance of the semester. So, I have to really understand their needs ahead of time to make sure that they are not far beyond what my students are capable of with my help. I also have to make sure that they know, as the community partner, what they’re gonna need to have ready so that the students can actually get to the part that they need to do. With web design something that most people don’t realize is that there’s a lot of writing content… and designers don’t write the content. The community partner or the client does. They need some coaching through that, and so I help facilitate some of that. Some of my scholarship as a professional is in that area where I’m working with these community partners, as a professional and as a consultant.

John: How much of the interaction with a partner is done by you, and how much is done by the students?

Rebecca: It’s a little both. At the beginning, well before the semester starts, I’m the one that makes contact with the community partner. We figure out how the semester is gonna be organized, establish roles and responsibilities, usually some sort of agreement. I usually make sure we negotiate some sort of copyright agreement that favor students using stuff in their portfolios, and set all those things up upfront, then we usually set a project launch date, and the client will come to campus. I make sure they’re available during my class time, and that they’re available pretty regularly through the semester and can come in a week’s notice, so they block out that time slot, so that they can come. So they come… I help the students prepare for that meeting. They ask questions… Q&A… so that the students learn about the community organization and what needs to happen. If it’s possible we usually schedule a trip to the community organizations, so we can see firsthand what they do, and so the students are interacting directly with a client in those circumstances… and then it depends what else needs to be done. So, for example, with the Oswego County Airport project that we’re finishing up, some of the students did some photography and things on the premise so they coordinated directly with a client to make arrangements for what time and that kind of thing. So sometimes it’s easier for them to do that communication, but largely if it’s about approvals and things like that, that all goes through me, which is in keeping with how it would be in a professional environment, where the creative director or an art director or someone above entry-level designers would be the ones having that contact.

John: From the students perspective, what are some of the benefits of this sort of project?

Rebecca: They’re really excited because they end up having portfolio work, which is important. They can put a line on the resume, essentially saying that they worked on a real project, that’s really being used, and then they also get to see an entire project all the way through. So in these cases, what we’re doing… community based learning or these community projects…they are able to participate in the research, development, design, the whole shebang, but usually they pick one rule that they do in depth which is something different than I would be able to do in other contexts. For example, someone might be the developer, or one of many developers, or someone might be a researcher primarily… even though they’re working on all the different parts of the project. So, they like the fact that they can do some work in depth. Usually in these classes I’m doing two big projects. So they’re doing this one and then they’re doing some sort of other individual project that complements it in some way.

John: So how much of this is done with teams of students working on the project and and how much of it is done by individual students working on individual components?

Rebecca: Well, the whole thing is usually a whole class project, which means that I really need to make sure that all the moving parts are working together and coordinating and what-have-you. We use Slack which is a team chat that we use outside of class to keep in contact about different things… and this last project we did something called “playbacks.” So, one day a week we did little playbacks about what everybody was doing and what they’re up to and what they needed from other individuals to keep the lines of communication open… and then certain roles and things are maybe small groups that need to work together to get particular pieces done.

John: You mentioned the portfolio piece for students. If they’re part of this big group, how do they identify the components that they worked on?

Rebecca: Yeah, we talked a lot about portfolio documentation, because working in a team is pretty standard protocol in the field that I’m in. What students do is they document the entire project, but they specify in that documentation what their role was… and so they always credit all the other people that worked on the project.

John: Excellent. What are some of the challenges that you face in working on an authentic learning project? ….with standard projects where you have a very finite well-structured problem, it’s fairly easy….well, at least you control the environment much more. When you’re working with someone in the community and you’re working with real-world development, what are some of the challenges unique to that type of framework?

Rebecca: Yeah, there’s many… [Laughter] One of the key issues is timeline. The timelines never match up, and so you always need to have a back-up plan for how something is gonna get finished… because it’s almost never totally finished during this semester. So, sometimes that means some people in the class are doing an independent study to finish stuff up…sometimes it means I’m gonna do something… sometimes it means another class is gonna pick up the pieces… or whatever… but that that needs to be in place, and that needs to be in place from the beginning. It’s really important for it to be in a learning environment that students can fail safely. They need to be able to screw up and that be okay.

John: It’s certainly safer for them to do that on this project than on their first job.

Rebecca: Right, exactly… and so you know part of my negotiations at the beginning of a project like this with a client is letting them clearly understand that this is a learning experience and learning comes first from my perspective, but that their needs will be met, but it might be met on a longer timeline than they really want.

John: …or perhaps a more iterative journey than they expected.

Rebecca: Yeah, exactly… and in most cases the community partner is more than happy to participate and especially because I always see the community partner as a co-teacher… they’re there to teach certain lessons, too – and that might mean letting us know that a student’s gonna fall flat… and letting them do that… and then help them figure out how to do it better next time… and give them that next time as part of the project. So, there’s been many times where some someone maybe provided a deliverable that wasn’t quite up to snuff and then deadlines had to shift so that that person could revise and meet the standard that needed to be met. So the students are generally working at a much higher level because essentially they can’t really fail. They can fail and revise… and revise… and revise, but eventually they get to a minimal standard… which I find to be helpful… and then the other thing is you really have to be flexible. All kinds of things happen… an organization’s budget can totally become a disaster and they have to refocus their attention on something else… and so you might feel abandoned.

John: While it’s the main focus of your class…

Rebecca: Yeah…

John: …it may not be the organization’s main focus.

Rebecca: Right. …like any of these things can happen without you expecting it even if you have you think all you have you have all your t’s crossed and i’s dotted at the beginning of the agreement. So that happens. Sometimes, students just don’t follow through in the way that you think that they’re going to or it or that you know they can… and so like what do you do in those situations? You kind of have to have those kind of failsafes in place. This is one of the reasons why, to provide an authentic learning experience for beginning students, I moved away from community-based learning. I used to do community-based learning in my beginning class. I do it now, but in a very different way than doing an actual website project, because there’s too much at stake there. So, I say that I save those experiences for my advanced students.

John: Going back to that… in your beginning classes what do you do differently to create the same sort of environment, but perhaps with a little less risk?

Rebecca: Yeah, I do two things. One that is a community-based project… and that’s what I call a consultation report. What they end up doing, in that respect, is, instead of doing a full design project for somebody, they do some of the research and analysis and do some proposals… some ideas… that we then hand over to the client that they can then use to either hire my advanced class or to hire a designer to take on but they understand more where they’re situated and so as part of that we do some accessibility testing… we do user testing… and things like that…. and so we’ve done that for a couple of different organizations, and that’s worked out pretty well. That gives students an opportunity to communicate with a client a bit and also do some formal presentations, which is nice…. and then the one that I use probably more frequently in my beginning class is a simulated client project. I have established a few scenarios that our clients… they have specific goals and needs… they have personas…. they have email addresses, etc…. and then students will work in small groups and then they communicate directly with the client all through written communication, although they can schedule an appointment…. I do have heads on popsicle sticks in which case they can meet with their puppets…. [laughter] which is always surprising to them because I don’t tell them upfront that I do that. So they come to my office and my door is always shut… for that situation I’ve reorganized my office. They knock on the door and there’s a head on a stick… and if they laugh I shut the door… and they have to start over. They have to take it seriously.

John: So, for the artificial clients, you create the email addresses and it will go to you?

Rebecca: Yup.

John: …and then you will respond as if in the role of the client.

Rebecca: Yeah, they each have a personality. So there’s four or five different clients. They all have very different personalities… and students start talking about their clients and the different kinds of ways that they behave. They have certain ways that they open and close their emails. One’s very curt and aggressive. One is very grandmotherly… very caring and kind.

John: Do you ever get them mixed up?

Rebecca: I have little notes when I start doing it that I keep on my laptop… a sticky note that just reminds me… a couple key words like who is who, so I don’t get confused.

John: Yes, that could cause some problems if you went from the very curt person to the grandmotherly person…

Rebecca: Yeah, and then if a student emails their client and they’re out of bounds or something then I email back as the professor from my school email address… and it says “This is a note from your professor” and then I indicate what’s wrong… and I make them redo it.

John: So, how do they react to the puppet?

Rebecca: They’re usually surprised but then they find it amusing… and they take it seriously… especially if I shut the door on them ‘cause they laughed at me… and they started over and I keep a straight face and whatever ‘cause you just know you never know who you’re gonna interact… and so the first time you meet someone you could be surprised, right?

John: It could be someone who’s a puppet.

Rebecca: It could be a puppet… you just never know… so, yeah, they generally respond pretty well to that… and usually if they meet with me in person as the client, then after that meeting I make them stay for a couple minutes and we just talk about how it went and things that they could have done differently.

John: Excellent. In an earlier podcast interview with Stephanie Pritchard, we talked about the Voices of Oswego Veterans project and that also seems to fit in as another type of authentic learning experience. Could you just recall that for people who may not have yet listened to that earlier podcast?

Rebecca: Sure. That project, in particular, The Voice of Oswego Veterans, was a collaboration between Stephanie Pritchard’s writing class, Peter Cardone’s photography class, two of Kelli DiRisio’s design classes, and my web design class. So, instead of doing my standard simulated client project with my beginning students, that group did the Voices of Oswego Veterans website. So that was somewhere between a simulation and a client because they didn’t have a direct client to talk to, but it was a real project and they had real content and real goals that they needed to meet… and that was taken really seriously by students and I think that was in part because it was going to be published. So, they didn’t get as much of the client interaction, which I think a lot of times the students value a lot from my classes, but it was still a very authentic experience and the students got a lot out of it and they were really committed to the goal of the project which was to dispel stereotypes about veterans. There’s a lot of assumptions that we identified early in the project… that people assumed that veterans are old… they associate it with World War II, and to think that “oh, wait, we have students on campus who are veterans, that just boggled some of their minds and we wanted to make sure that those students are seen as students as well.

John: How have students responded in general to the project?

Rebecca: I think, in general, students respond to any of these authentic learning experiences fairly positively. I think they all think it’s a lot of work, especially because the revision is taken a lot more seriously… and you think that that maybe wouldn’t be true of the simulation, but they get into it and they continue to revise and they want to meet and satisfy the client….that’s the goal at the end of the day. They need the thumbs-up from the client at the end… and so I think that is motivating and it seems realistic enough that they want to give it their all…. and that definitely is true on community projects. The one that we’re finishing up now, I have a student who graduated who’s finishing up a couple things that she couldn’t quite get to work the way she wanted to and she’s finishing that up right now

John: Okay, so I guess the next question is: “What are you going to do next?”

Rebecca: That’s a good question…. [Laughter] I should have known that was coming.

John: You usually ask that question.

Rebecca: Yeah, I know, right? So, I guess it’s only fair that it’s asked of me.
So, the next thing that I’m planning to do related to authentic learning is to emphasize thinking about audience empathy and stereotypes a little bit more. That Voices of Oswego Veterans project, I think, was particularly successful in helping students actively design to dispel certain stereotypes and I’ve really been trying to get students to think about audiences who are different from themselves… which is a challenge…. and that seem to work really well, so I’m trying to find a way to embed that more so in both my beginning and advanced classes.

John: Excellent. Well, thank you. This was an interesting discussion.

Rebecca: Thanks, John.

John: Looking forward to hearing more about it as the next semester progresses.

12. The Active Learning Initiative at Cornell

In this episode, we discuss Cornell’s Active Learning Initiative with Doug McKee, an economist at Cornell and a co-host of the Teach Better podcast. This initiative, designed to increase the use of active learning in instruction at Cornell, provides funding to departments to redesign courses to employ evidence-based active learning techniques. Doug provides an overview of the program and a discussion of how this program is being implemented to transform economics classes.

Show Notes

John: Our guest today is Doug McKee. He will be discussing the Active Learning Initiative at Cornell. Doug is an economist at Cornell and also a co-host of the Teach Better Podcast, which is one of our favorite podcasts on teaching and learning.

Rebecca: Today our teas are:

John: Ginger peach white tea.

Doug: I’ll be honest. I can’t stand tea.


Rebecca: Absurd.


Doug: It’s not that I don’t need the caffeine, but my preferred method of caffeine injection is coffee.

Rebecca: Yeah, that’s a bummer. It’s not “The coffee podcast.” There’s lots of coffee here..

Doug: Maybe next week I can be on the coffee for teaching podcast.


John: Or we could have a special episode.

Rebecca: Yeah.

Doug: Oh… a special coffee episode.

Rebecca: That would be kind of fun.

Doug: You’d need guest hosts.

Rebecca: Yeah.

John: The Starbucks next door is closed over the break, so we’re…

Rebecca: Anyways… I’m drinking a golden English breakfast.

Doug: Do you change the teas every single week?

Rebecca: We try to.

John: We generally do, but we got a couple hundred of them….

Doug: But, don’t you just have a few that you like?

Rebecca: I have a favorite, yeah.

Doug: Don’t you just want to drink that one all the time?

Rebecca: Sometimes.

John: Sometimes we do. I’ve used the same ones occasionally.The last couple of episodes, I had the ginger peach black tea….

Rebecca: …that’s his favorite.
JOHNL… and the ginger peach green tea… and this is a ginger peach white tea, which I drink later in the day because it has a little less caffeine.

Doug: uh-huh…

Rebecca: I like the English afternoon tea.

Doug: I would think in the afternoon you’d want the one with more caffeine.

John: I’m generally better in the afternoons and evenings. I have more trouble getting energetic in the morning.

Rebecca: John loves 8 a.m. meetings.

John: Could you describe the Active Learning Initiative at Cornell? How did it originate and how is it working?

Doug: The first thing I want to say is I’m a relative newcomer to the Active Learning Initiative. I joined Cornell about a year and a half ago, in large part because I was excited about joining the Active Learning Initiative. But it’s a project or really a program that started back in 2013. It was the brainchild of Peter LePage, who was actually a guest on episode 50 of the Teach Better Podcast and he is a former Dean of the College of Arts and Sciences. He’s a physicist and he had read a fair amount of the literature… the big literature on active learning in physics… and wanted to come up with a program that would actually instill this way of teaching across the college. He also happened to be a friend of Carl Wieman, and so he knew all about the Science Education Initiative as it was happening… and the Active Learning Initiative is a program modeled on the same principles as the Science Education Initiative… and the ideas that departments compete for grants, and once they get that money, they use it to hire people with disciplinary knowledge (usually postdocs into their own Department) who can co-teach with faculty and train faculty in how to use these methods. Because what I find, is, when you ask faculty why they don’t teach actively or why they depend on the pure lecture, even though there’s a fair amount of evidence that active learning works better on a variety of dimensions, “we don’t have the time”… well, the department education specialist or the postdoc can help with that…. they say “we don’t know how”…. and so the department education specialist comes in with that knowledge of how to develop clicker questions and what a good small group activity looks like, and then finally when they say ”well, I don’t believe it,” it’s the department education specialist can both reference literature… talk in the language of the the discipline… and…. and this is a big part of both the Science Education Initiative and the Active Learning Initiative, creates knowledge and actually try things in that discipline and evaluate what the effects are on student learning of teaching in new ways…
JOHN…and the STEM fields have been a bit of a leader in that. Physics, in particular, has been very active in development of this.

Doug: That’s right. So, the first round of the the grants funded by the Active Learning Initiative at Cornell were given to the Physics department and the Biology… actually it was a co-proposal by their two biology departments. Why? I don’t understand why universities can’t have one biology department anymore? They don’t.

John: Out of curiosity, what are the two areas?

Doug: Ecology and evolutionary biology… and then there’s the, I think, molecular biology and like these kinds of biologists…. I don’t know… I’m not a biologist, but biologists have a hard time talking to each other, it turns out. But these biologists could talk to each other even though they’re in different departments, and…

John: … maybe because they were in different departments it was easier.

Doug: …it was easier… You’re right, it was easier…

[LAUGHTER] as long as they just had to talk about teaching and not about who the appropriate person to hire was… and so those were the first two departments… and then since then, in 2017 there was another round of grants granted and one of those was economics and the idea there was to branch out beyond the sciences and the STEM fields into the social sciences and even the humanities.

John:… and this was funded by a donation from alumni, right?

Doug: So, the funding comes from an alumnus who had a passion for active learning. I think what happened was he and Peter were at a dinner… and Peter was talking about how excited he was about changing how he taught his classes, and how he wished that it would happen more often, and this person came up to him and said “I heard what you were talking about and I’d be really interested in funding that.”

Rebecca: That’s exciting.

Doug: On the other hand, the Science Education Initiative, as far as I know, was funded by internal money. So it was Carl Wieman was very persuasive in convincing the administration at the University of Colorado and the University of British Columbia, and now he’s at Stanford, and he has internal money at Stanford, to do a lot of the the same thing.

Rebecca: Funding is great to support some of these things, but I would say that our Writing Fellows program on our campus is not that different from hiring these postdocs, because we have writing fellows that are assigned to each of our schools. We have a previous episode on that topic with Stephanie Pritchard, and we have writing experts who meet with faculty in different departments to help them develop writing assignments to meet some of our standards for writing across the curriculum, and help faculty understand how to teach writing….

Doug: Right.

Rebecca: … in their disciplines. So, in some ways, it’s a very similar sort of model. It’s not about active learning, but it’s about how to do this writing infusion.

Doug: So in many ways, it does sound similar, but I’m gonna point out some of the differences.

Rebecca: Great!

Doug: These postdocs live in the department, and they have PhDs in that discipline, and so in some ways they get way more cred that way… and respect from the faculty, because they’re thought of as part of the department. Another difference is that the departments have to write proposals, so they’ve committed and they’ve said in the proposal how excited they are about this, whether they are or are not.

John: They’ve at least made a commitment.

Doug: They’ve at least made a commitment and said… stated on paper that they’re excited and they want this to happen and this should be a success. It’s not some external entity, and I think that makes a difference.

Rebecca: How long do the postdocs stay in the department?

Doug: So, it varies from department to department, but in our proposal we felt strongly that in economics the postdocs are generally two years, and so our postdocs are 2-year postdocs. But in physics, I think, the postdocs tend to be longer, and so their postdocs, I think, are three-year postdocs. One year is just not enough, because they arrive and they immediately have to start preparing to go back out on the job market, and try to get jobs elsewhere.

John: But I think you had mentioned that while they were there for two years, there were going to be a number of postdocs that were staggered over a four-year period.

Doug: Right. So our project is a five-year project and what we’re doing is we’re hiring four two-year postdocs starting, with one in the first year and then every year after that, and so during the middle three years of the program we’ll have two in residence and then in the first and the last we’ll have one.

John: You mentioned the Economics Department as one of the recipients of this. How many other departments were there?

Doug: Five. So the physics department has another grant to overhaul how they teach their lab courses, and this is actually… Natasha Holmes, I think, is arguably the world’s expert on really modernizing and changing how we teach lab courses; taking them away from following recipes to creating the recipes and answering interesting questions using experiments, and creating those experiments, and she recently joined the Cornell faculty. The music department is using a grant from the Active Learning Initiative to integrate active learning into a composition course where students all have keyboards in the classroom and the keyboards act a little bit like music clickers, so the…

John: with MIDI controllers, probably?

Doug: Right… and the instructor can then listen and select different things that different students have played and then play them for the whole class. It looks fantastic. Sociology is doing fairly straightforward integration of active learning into their large lecture classes, using group activities and clickers. They’re also standardizing, and making much more active, the discussion sections for those classes. The math department is overhauling their introductory calculus courses. Let’s see, that’s math, physics, economics, sociology, music, and then the sixth is classics…. and Classics is creating a brand new course that’s not a pure lecture course, that has students, I think, doing projects during the semester.

John: …and your department is doing how many classes?

Doug: We are treating, or transforming, eight classes?

John: Which classes are you doing?

Doug: So, we’re basically doing the entire core curriculum, or the required courses for the major. It’s not exactly that right now. We’re really doing seven plus a popular elective course called behavioral economics… and why aren’t we doing all eight of our required classes? Well, one of them is a class that we don’t actually own, but we’re hoping to somehow, over the next five years, find the money where we can treat that class too, because it would be a shame to just do seven out of eight.

Rebecca: How are your faculty in your department responding to the initiative and getting involved?

Doug: So we have a small core that’s very excited and they’ll be involved in the program early on in the first couple years. Then we have another set of faculty that are very excited in theory, as long as it’s far enough in the future…. And I think we have faculty that are not that excited, but aren’t actually being affected by it, and so they’ve been fairly passive. I’ve been thrilled not to have anyone in the department that’s actively opposed, and so it’s been great, actually, and the vast majority of the work happens when a course is actually being transformed. Right now we’re making it very easy on ourselves. The first course that we’re transforming is one of my own courses.

John: Which makes it easier.

Doug: Yeah.

Rebecca: Yeah.

Doug: So we can get the progress. We can just get the whole procedure right… get that stuff in shape as well as we can. So… okay, we’re gonna do a control course in the fall. We’re gonna do a treatment course in the spring. So we’re gonna teach it fairly plain vanilla in the fall, and we’re gonna treat it in the spring and we want really good measurements of learning. How do we go about creating those measurements of learning? Because unlike physics which has like 80 standard concept inventories, economics has one. It’s the Test of Understanding of College Economics, and it’s pretty good, but it’s only for principles courses, and so we have to build from scratch… and so next year we’ll actually be bringing more faculty into the fold, and we’ve scheduled it such that the fact that we bring in our faculty that are excited about the project.

Rebecca: What have you learned from the process so far?

Doug: Measurement is hard… and it’s not even that it’s just difficult… it’s a lot of work… and so we had a meeting with some of the folks in biology and where we talked about what we were doing and Ron Harris-Warrick said something. He said “It seems like you’re spending 90% of your time on measuring and assessing, whereas we spent 90% of our time actually changing the teaching” and I said “That’s exactly right” because we’re not changing the teaching yet and we want to know if, in fact, what we do when we change the teaching works …and so we need really good measures, and so that’s where all our investments been. Now, is that changing? Yes. In two weeks we start classes up again, and we start teaching the transformed class and so we have actually done a fair bit of work, and we have big picture. We know what we’re going to do, and we’re very excited about it, but we have a lot of work to do this spring on actually where we focus on changing the teaching.

Rebecca: So, we’re taking this week by week?


Doug: Well, it’s gonna be a lot of just-in-time curriculum development.


John: I’m very familiar with that process.

Doug: Right… right… right. I got to say, like after a semester of focusing assessment, which has been really fun, we’ve created what we think could be the start of two standard…. like two… we’ve tripled the number of standard assessments in the field in one semester. I mean… we have work to do still and we’re the only ones using it right now, but that’s changing. I am pretty excited to actually change how I teach this class and we have a lot of ideas about things we can do.

John: You’ve already been doing a lot of active learning in your classes before, so what are you going to be doing differently?

Doug: I divide it up into two chunks. So, we’ve got some high bang for the buck things… things like two-stage exams. One problem that we’ve had in the past, and I think it’s a very common problem, which is: you give an exam… the students take the exam… and then there’s a whole bunch they can learn after the exam about the mistakes that they made, and more than half my students don’t even come and pick up their exams. They see the number and then they… if it’s bad they get sad, and if that’s good they’re like I’m fine… and neither group is learning from their mistakes, and so what we’re doing is we’re saying you’re gonna take the exam… and then you’re gonna take it again in groups and then your grade for the exam will be a combination: 80 percent is your individual grade and 20% the group grade, and so at that point you’re actually discussing it. It forces students to actually discuss and talk about these problems in another step, and I think that’ll make a big difference.

John: You had a podcast episode on that not too long ago.

Doug: Yes, with Teddys Svonoros, where he’s been giving two-stage exams for quite a long time at the Kennedy School at Harvard.

John: So listen to that podcast. It’s a great way to learn more about that. We’ll put a link in the show notes.

Doug: It’s a great podcast.
Another high bang for the buck thing, we think, is assigning points in their grade to whether or not they participate in answering clicker questions in class. I have been, for my entire teaching career, adamantly opposed to giving points for class participation. I’ve always felt that it cheapens it, and I want students to be there and participate because it improves learning… not because they’re getting points toward their grade… and over the summer I was convinced that that is actually the wrong attitude. Students look at how you assign points for their final grade as a signal of how important things are.

Rebecca: …of what you value.

Doug: …of what I value…. and so if I say there’s no points for showing up at class and it’s not required, then they’ll be like: “Oh, he doesn’t think it’s important” and I can say it’s really important over and over and it doesn’t have nearly the…

John: Well, clicker questions offer some other things. It gives you some feedback too.

Doug: Right.

John: …in addition to other things, because if one person asks a question or answers a question or raises a point you know how that person responds or how they understand the concept.

Doug: Right.

John: But, with clickers you can get a feel for how all of your students are doing.

Doug: Right.

John: …and you can adjust what you’re doing in class to compensate for that somewhat. So there’s some good merits for clickers whether they are graded or not.

Doug: Right, but you’re not gonna get anything out of the clickers if the students aren’t in the classroom.

John: Right.

Doug: ….and so I assign points to get them into the classroom and I think that’s just gonna bring up a big chunk, because at the end of the semester you look at the grade distribution and you look to see who got the highest grades in the class… and guess what? You know all those people because they show up in class.

John: Yes.

Doug: ….and then you look at the bottom and your like, how did I miss these people? They did terribly the whole semester….

John: …but you didn’t see them as much.

Doug: I don’t even recognize them. I look at the pictures and they don’t even look familiar… and so pulling those people in. Those are a few of the little things… little in terms of effort… but we hope big in terms of impacts. But the class has clicker questions… the class already has… I like do short lectures… and they do problems to practice… and so we’re we’re hoping to really gain in the spring is by adding something called “invention activities” and so these are an idea that Dan Schwartz and his students have been writing papers about for a little while… and so probably the easiest thing to read if you’re interested in invention activities is the J chapter for “Just-in-Time Telling” in Dan Schwartz’s and co-authors book called The ABCs of How We Learn, which is an amazing book. I highly recommend it. If you read any book on teaching at all The ABCs of How We Learn is a great one in terms of telling you practical things you can do and giving you the evidence… and Dan Schwartz does great work… and so the idea is, in a nutshell, let students grapple with a problem before you actually teach them the solution… because it primes their brain… it sets up those knowledge structures that you can then hang the expert methods on.

John: So it activates prior knowledge….

Doug: Exactly.

John: …and gets them ready to form more complex….

Doug: It even create creates prior knowledge… and so he has these two papers. One is called “The Time for Telling” and the other is called “Inventing to Prepare for Future Learning” where he does these really amazing experiments. The one that I really like in “Time for Telling” is about casework. He has these three groups and one of the groups he gives a bunch of data on classic psychology experiments and just has them graph it and talk about it and try to figure out why the patterns are there and then he explains what we actually learn from these psychology experiments.. and then they play around with it and then he gives them a test… where they have to predict what the outcome would be of some new psychology experiments. That’s group A and they do great. But then group B… he skips that first stage and instead they read a chapter about what we can learn from these psychology experts and they summarize it… which doesn’t sound so bad… and then he gives the lecture and they do far worse… and so I’m really hoping that we can get a big bang from these priming activities and I think this class.. it’s a second semester class on statistical methods for economists, also known as econometrics… ‘cause god forbid economists use a word that other people already know…

[LAUGHTER] but there’s a whole bunch of methods there and so I look at that class as a really great match for this method…. where I can show them data and put them in this situation where a new method would be really valuable… and the old methods work poorly… and have them play around with it and then teach them the method.

John: That sounds really promising.

Doug: I think it’ll be super fun.

John: We just did a reading group here on Small Teaching and one of the chapters in that that people were pretty excited about was on prediction and it…

Doug: Exactly.

John: …summarized a lot of the research on that (which is another good reference in addition to what you just mentioned)

Doug: Right.

John: … to see a summary of the literature on that and it sounds like it’s quite effective in a wide variety of studies now.

Doug: I mean, I do think you have to be careful with students in explaining why you’re doing it upfront. I think it can be very frustrating to give students problems that you know they’re going to fail at. Students don’t like failing.

Rebecca: No.

John: But the research shows that when they have wrong predictions they actually have larger learning gains.

Doug: Right.

John: …as a result of the prediction activity.
DOUG. Right.

John: … and as long as you they know that, and as long as it’s low- or no-stakes in the prediction….

Doug: Right.

John: ….it shouldn’t harm them… but it doesn’t feel as good, so there’s a metacognitive issue there.

Doug: Well, I try to model making lots of mistakes in the classroom

John: I do that too, but I’ve never called it a strategy., but…

Doug: Exactly.

Rebecca: I think students respond really well to seeing their faculty be vulnerable and human…

Doug: Oh, I agree with that.

Rebecca: …and I know we all make mistakes and I think you’re right that students are… they’re afraid to fail…

Doug: Right.

Rebecca: …and they’re afraid to be wrong. So having an environment that’s setup that it feels safe to make those mistakes.

Doug: Right.

Rebecca: …and then get better… and the goal is to get better… really sets up an atmosphere that really supports learning because everyone feels safe about learning… and I think that that’s not always the way that students think about learning.

Doug: Not at all. Not at all. They think “if I get the answer right I’ve learned and if I get the answer wrong I haven’t” and it’s actually the opposite. If you get the answer right you haven’t learned it…like, you knew it.

Rebecca: Yeah.

Doug: Yeah, if you get the answer wrong… oh my god, there’s an opportunity to learn something…

John: …and it’s hard to convince some of that sometimes.

Doug: It is. It is. I tell them I want you to get things wrong, because if you just get everything right you’re wasting time…. then I’m wasting my time.

John: I tell them the same thing.

Rebecca: Yeah.
REBECA: So, can I circle back to something that we talked about earlier? I heard you use the word fun and assessment in the same sentence.


John: It’s not a common juxtaposition.

Rebecca: Exactly. I was hoping that you could talk a little bit more about what was fun about this development.

Doug: Oh my god, I’d love to.
Yes. So first of all, I admit that I like writing tests. I think it’s really fun to come up with new scenarios that get at concepts that I’ve taught before… and so I can take knowledge in some completely different area and then frame it. So I remember one time I had to give a makeup exam, but I didn’t really want to write a whole makeup exam and so all I did was change the wording… so a question about hospitals became a question about pet stores then a question about restaurants became a question about something completely different… but the methods were all identical and so that I think that’s kind of fun… but most people wouldn’t think that’s fun. So the part of the process that we use that I think more people would find fun in that was a key part and it was something that Carl Wieman and Wendy Adams call “think-alouds” where we draft the assessments and we bring a student in and we have them do it. So these assessments are 20 to 25 multiple-choice questions and if all you see are the sequence of multiple-choice answers, it’s hard to tell what’s going on in a student’s head… okay …and so what we do is we say sit down and we say vocalize everything you’re thinking as you take this…and we don’t say anything…. we don’t say no don’t explain what you’re doing, because that changes how you actually answer it. Just vocalize what you’re doing as if you’re taking it and the results are so eye opening. I learned so much about how students actually think about these things… the amount of just pure pattern matching, it would blow anybody’s mind. They say things like “You’re asking about a confidence interval… well, I remember that confidence intervals had something to do with standard errors, and so I’m gonna choose the answer that says something about a standard error.” It’s kind of frightening, to be honest… and then pretty often you ask a question where you say… you show them a picture, and then you ask a question that’s related to the picture… and they ignore the picture. They don’t even look at the picture. Yeah, ok, that’s not good… and so we did five of these during the fall with this the big assessment of learning that we wanted to use both in the fall and in the spring… so then the control course in the fall and the treatment course in the spring. We’re gonna compare results, and each time we did these “think-alouds” we would have like ten changes. We would add options because we’d find out that students…. it was really common for them to make this one kind of mistake that we hadn’t foreseen… and get an answer that wasn’t one of the choices. Sometimes they would pick the right answer for completely wrong reason like “oh well, it’s always the one that’s a yes.” Oh…well maybe we should make the right answer the one with the “no.”

John: So, to develop assessments or concept tests, it’s really helpful to know the common misperceptions so that you can break them down.

Doug: Absolutely.

Rebecca: What I’m hearing is what we call in design a user test.

Doug: Right.

Rebecca: So you’ve designed something, and now you’re testing it out on a test audience

Doug: Exactly.

Rebecca: …and making revisions.

Doug: …and no one does that.

Rebecca: Yeah, it’s kind of interesting, right? Like we don’t think of students as being users of a class, but essentially they are.

Doug: Right. I think when we teach, usually we have so little insight into what’s actually going on in students heads… and I think in a pure lecture it’s the extreme case. You get zero insight because you teach and they listen… and the only time you ever see what they’re learning is when you see their test results, or what they’ve written and so a big part of why clickers are useful is not only because it activates their brains and they’re practicing things… but the feedback it gives the faculty and so doing these think-alouds… it’s like an extreme version of insight into what students are actually thinking.

Rebecca: How do you recruit these students for the think-alouds?

Doug: What we did is we have the rosters from the previous year… students who had taken the class… and we invited a broad range, so we didn’t just get the A students… we got the A students… the A- students… the B students… even a couple C students, and then I think we paid them twenty dollars each… but you pay them a little bit and they show up.

Rebecca: That’s great.

Doug: Yeah… yeah, yeah, yeah.

Rebecca: Have you thought about using the think-aloud method at all as part of your class? I’m almost wondering if there’s a way to kind of integrate that more into, probably not a large class, but a smaller class.

Doug: That’s a really interesting idea. I think when you give students kind of meaty problems to do in class, and they’re working on them in small groups, you can go around and you can just listen. So there are these incredible transcripts of what students said to each other during the invention activities in the “Inventing to Prepare for Future Learning” paper. So the subjects were ninth grade statistics students, but it’s remarkable just how similar teaching is. There’s a lot more in common across grade levels than there is difference. I mean there are differences, third graders are not exactly the same. My classes look far more like a good third grade class then they do a pure lecture class.

Rebecca: it makes a lot of sense, right? I mean active learning works and they figured that out in elementary school.

Doug: Right, can we please not forget that?


John: Another way of getting the same sort of thing was suggested by Eric Mazur when he was here during a visit, and I’m sure he’s mentioned this in other places as well. When he develops clicker quizzes, he tries to aim for about half of the students getting them correct and the other half wrong, so that you’ve got a good base when he does the peer instruction component of it. What he does to develop the questions is pose a question and leave free response questions and he’ll just let students write their responses and then he’ll use that to pick the most common misconceptions that he’ll build into the clicker questions. Which gives you a little more scale, but not quite as much information as a think-aloud, perhaps.

Doug: …and the ideas that you’re iteratively refining your class semester over semester. I wish that that was a more common attitude toward teaching… but what I find is… I meet a lot of faculty that… they developed the class… they fix the things that are obviously broken… and then it’s done… and then they come in… they give their lecture… they walk out… and there’s very little investment after the fact.

John: We’re creatures of habit.

Doug: Right.

John: We tend to resist change. That’s one of the things behavioral economics tells us.

Doug: I was gonna say that classical economics tells us that people respond to incentives.

John: Right

Doug: If you’re above the bar… like you’ve responded to all the incentives… that are all the professional incentives that are there.

Rebecca: As a designer I just can’t help myself from redesigning and redesigning and redesigning… it’s never done.


John: I’m never satisfied with the way my course goes and I keep wanting to make it better.

Doug: Me too.

Rebecca: Yeah, but that’s why you two run a teaching center and have the teaching podcasts. You’re not the problem.

John: Well, my students might disagree at times… but that’s another issue.

So, are the other departments involved doing as much with the assessment component or is this something that you’re perhaps focusing on a bit more? or is it built into all of it?

Doug: No, I would say… I believe, no. It’s kind of a pet issue of mine… like I think assessment is super important. I think math is leaning on existing assessments… which is fine…

John: ….for departments or for majors where you have existing assessments that works well, but as you mentioned, we just have the micro and macro TUCE exam…

Doug: That’s all we have.

Rebecca: I mean, you have something.

John: We have something and we use it.

Rebecca: There’s nothing in my field.

Doug: So, in sociology there’s nothing also… and so sociology has actually been investing in assessments there. It’s tough… it’s tough. Sociologists are allergic to multiple-choice and so they will never use multiple choice… and I’d like to see a graphic design assessment that’s multiple choice.

Rebecca: I made one.

It’s hard, though.

Doug: So, Natasha Holmes and I have this ongoing back and forth where I say “Here’s something that’s tough to teach and evaluate with multiple choice” and she says “No, you could do it” and the line moves. So we agree that there are plenty of things that can’t… we agree on a lot that can …and we agree that fluency in playing the piano… you can’t… but where the line is in between, we go really back and forth… and I find that over time I believe more and more can.

Rebecca: I think there’s a lot of conceptual things that you can test that way.

Doug: Right.

Rebecca: …and measure certain kinds of understanding.

Doug: Right.

Rebecca: …but maybe not always… you can’t do a practical application with the multiple choice.

Doug: Right.

Rebecca: Like in graphic design, for example.

Doug: RIGHT. So, not everything, but a lot.

Rebecca: Yeah.

John: But you could devise, as we talked about in an earlier meeting today…. you could use rubrics for those areas.

Doug: Right… right.

John: We had a number of faculty a few years ago, who were new faculty in creative fields, and they were very concerned about how to evaluate creative work… so we put together a panel of people from art, from music, from screenwriting, a playwright, and from creative writing, and what was remarkable is they all said exactly the same things: that they use rubrics very heavily; that while people perceive this as creative work, there’s very specific things that they’re looking for in the writing; and as long as they develop good rubrics that capture what they’re looking, for it lets students know what they should be striving for what they view as important. Just as you said to give them points as students, matters but giving students rubrics helps them see what you think is important in their work… and it makes it easier for them to try to meet those standards.

Doug: Do you have a book that you recommend?

John: I don’t.

Doug: Well, wouldn’t it be great?

John: It would be. I’m not sure if there is one.

Rebecca: I don’t know of one, but I think it’s not a lot different. Revising and rewriting a rubric is no different than revising and writing multiple-choice questions

Doug: Right, right.

Rebecca: It takes time to refine that and get it to measure exactly what you want it to measure.

Doug: Right.

John: It just impressed me, though, that they all had exactly the same thing in fields that were so diverse.

Doug: Right.

Rebecca: Yeah, there’s still values to the discipline, right? There’s still things that we value. So, it might be innovation… so, maybe that’s an item on your rubric.

Doug: It’s pretty easy to get inter-rater reliability with a multiple-choice test. I think if the correct answer is A and you give a C, John and I are both gonna say it’s wrong; whereas, with a rubric it’s not obvious.

Rebecca: Right.

Doug: But I think… I mean… like so many things… like it’s not that rubrics are good and multiple choice exams are bad. There are plenty of really good multiple-choice assessments and bad ones and there’s a big difference between a bad rubric and a good rubric.

Rebecca: Um-hm

Doug: This is where people say “I tried active learning… it didn’t work …and so I don’t believe in active learning anymore.”

John: But by building the results and by working on the development of tests and assessments that could be used across time and across disciplines…

Doug: Right.

John: ….that makes it easier to build a case for the efficacy of active learning…

Doug: Right.

John: ….and that’s one of the reasons why I think why this has been so effective in so many of the STEM fields, particularly in physics.

Rebecca: I think another thing to think about is, if you’ve never done active learning as a teaching method before, then like you are a beginner, you’re not an expert…

Doug: Right.

Rebecca: …and so just like our students, the first time out of the gate….

Doug: Right.

Rebecca: ….we’re not going to be an expert, and it’s not going to be perfect, so we have to be vulnerable as learners as well… and so I think sometimes reminding folks that… wait a second, right now you’re a learner, and it’s ok… it’s ok not to be perfect…. and it’s ok that it failed. We can learn a lot from it….

Doug: Right.

Rebecca: …it can be really useful.

Doug: Right. Boy, that attitude… I wish it was more common.

John: New faculty, while they may have been exposed to more active learning techniques, they’re also sometimes reluctant to go against department standards of teaching because they know that there’s some chance that new things they try may fail… and it’s certainly safer and easier to do it after tenure, although by then people often get into habits that are hard to break.

Doug: So, a strategy that my department is planning to apply, and their current chair is highly supportive of, is bringing brand new faculty into the Active Learning Initiative right away… and so giving them classes that have already been transformed… so it becomes the new norm… so it’s not that you’re taking a risk or trying these new things and if it doesn’t work the rest of the faculty are going to shake their head… it’s “This is how you’re supposed to do it.”

John: Yes, having that as a prepackaged method of teaching…

Doug: Exactly.
JOHN… certainly would make it easier to disseminate that.

Doug: That’s the hope.

John: Excellent.

Doug: ….and they also won’t have the… I guess… the institutional power to fight against it as hard… as an established faculty member.

John: That’s true.

Rebecca: Yeah. How have students responded to some of the initiatives. I know in your department you haven’t done that active learning kind of stage but I know that you’ve had some other programs

Doug: Ok.

John: You’ve been doing active learning techniques in other classes for a while.

Doug: So, I can say three things. First, is in physics course evaluation. It improved and the students seem to really like it, and feel like they’re getting more out of the class than they are [in other classes]… because what they’re doing is they’re taking these highly flipped physics classes along with other classes outside of physics that are not… and so they see the contrasts… and I think both physics and biology have done a good job in explaining why they’re doing what they’re doing…. so that those students that say “ Why are you making us struggle? Why don’t you just tell us the answers, and then we’ll know how to do it?” and they’re explaining why that’s actually less effective… and to be honest in my experience, I do a fair amount of active learning in my classes already… and you have to explain why you’re doing it.

Rebecca: I agree. Students respond really well to that.

Doug: Right.

Rebecca: Yeah.

John: We’re much better in any case. They still sometimes resist because, one of the problems with a lot of the approaches, is that they don’t get as much positive feedback right away…

Doug: Right.

John: …because they do make mistakes and fail and that doesn’t feel quite as good as…

Doug: Right.

John: …doing well on the one or two or three exams that they might have happened to have had otherwise after cramming the night before.

Doug: In all three cases, I think, explaining what you’re doing, and why, before you actually do it… it’s pretty effective… and again you’re always going to have these students that fight against it. They’ve succeeded their whole… and it’s usually the top… and it’s those students that have succeeded in memorizing, and taking notes, and doing well on exams their whole life… and now you’re teaching them in this new way… and it’s harder. They’re gonna resist… not that surprising…

John: But it can work the other way, too. That the students who find that it’s more helpful to go through active learning techniques might encourage other faculty perhaps to adopt it.

Doug: That’s the hope… and I think the great majority, I think, get a pretty to have a pretty positive experience… and even more by the end of the semester. I think most of the students are going to have bought in if it’s done well. I mean. I’m not gonna say that every single class that did active learning the students loved it, because there’s some crappy active learning classes out there. I knew someone… they decided to flip their class and they admitted all of this… like I heard this from them, not their students… they said I turn all my lectures into videos and it was great and I really invested in the videos and then we would show up the class, I’d be like ok and I had no idea what to do in the class …They hadn’t invested in the class part.

Rebecca: Right.

John:: …and that’s the big part of active learning.

Rebecca: …that’s the active part.

Doug: That’s right. Right.

John: In most disciplies, you don’t need to create videos. There’s a lot already on YouTube. The real work needs to be on what you’re going to do in class to give you the most value added there.

Doug: That’s right. That’s right.

Rebecca: Our last question usually is what are you gonna do next?

Doug: So, we have a whole transformation plan. I mean at this point, which I talked a little bit about, but my spring will be heavily invested in creating and trying these invention activities. I’m crossing my fingers that they work. It’s the first course we transform and the department is part of the Active Learning Initiative. It’s a kind of a big ask… that we try something completely new and actually see some big results. It could be, at the end of the semester, we said why didn’t we just take that lousy pure lecture class… because that would be a lot easier to improve. I don’t know. Well, it’ll be exciting to find out. But the other big thing on my plate that I haven’t talked about already is taking these assessments that we’ve drafted in the fall and piloting them with partners at other institutions… and so if any of your listeners teach either introductory statistics or a second semester econometrics course, please contact me, douglas.mckee@cornell.edu, because we would love to have you pilot this thing…. and I’m hoping to give a big talk on it… on both of these assessments with our postdoc George Orlov who’s amazing… and will be on the academic market next fall, so if you’re looking to hire an economist that really loves teaching, and is great at it, and does really super interesting research, he’s your guy.

John: But not yet… not until he finishes…

Doug: Oh no… don’t hire him yet…

[LAUGHTER] …and so, what we’re hoping to do is make these published standards that can be used across the discipline as ways of evaluating teaching… and that can be big picture… we did this big intervention… or I think a lot of people use these standard assessments as ways to identify where the soft spots are in what their students are learning… and so people do that too… and so the vision is, for over the next five years, to take what we’re doing now and multiply it by eight… and so there’s there’s a lot to do.

Rebecca: … big shoes to fill.

Doug: …we bit off a lot. We promised alot. Now, we just have to execute. So far, so good.

Rebecca: I can’t wait to hear about the results and hopefully we have follow up and find out what happened.

Doug: We’ve submitted an abstract to the Conference on Teaching and Research and Economic Education which will be this June, where we report on the results of the transformed class, and so we won’t know… we will be getting the data in…

Rebecca: …moments before it’s due…

Doug: …moments before presenting the work. Hopefully, we’ll have a lot of incentive to put that together.

Rebecca: Great.

John: Excellent.

Doug: Oh, thank you both.

Rebecca: Well, thank you!

John: This was fascinating. Thank you for coming down here… or coming up here to Oswego.

Rebecca: Yeah, we’ve really enjoyed your visit.

Doug: It’s been a lot of fun you guys are doing great work here.

John: …and we really enjoy your podcast!

Doug: Thank you. More podcasts! I’m your number one fan.

John: OK, Thank you.

Rebecca: Thank you.

11. Mobile technology in the classroom

Smartphones, laptops and tablets can be useful learning tools in the classroom; they can also be a source of distraction. In this episode, we discuss alternative policies that faculty and students might adopt to facilitate learning. Recent research on the relative effectiveness of handwritten vs. digital note taking is also examined.

Show Notes

Coming Soon!

  • Aguilar-Roca, N. M., Williams, A. E., & O’Dowd, D. K. (2012). The impact of laptop-free zones on student performance and attitudes in large lectures. Computers & Education, 59(4), 1300-1308.
  • Artz, Benjamin and Johnson, Marianne and Robson, Denise and Taengnoi, Sarinda, Note-Taking in the Digital Age: Evidence from Classroom Random Control Trials (September 13, 2017).
  • Bain, Ken (2011). What the best college teachers do. Harvard University Press.
  • Bui, D. C., Myerson, J., & Hale, S. (2013). Note-taking with computers: Exploring alternative strategies for improved recall. Journal of Educational Psychology, 105(2), 299.
  • Carter, S. P., Greenberg, K., & Walker, M. S. (2017). The impact of computer usage on academic performance: Evidence from a randomized trial at the United States Military Academy. Economics of Education Review, 56, 118-132.
  • Brooks, D. Christopher and Pomerantz, Jeffrey (2017). ECAR Study of Undergraduate Students and Information Technology, 2017.
  • Hembrooke, H., & Gay, G. (2003). The laptop and the lecture: The effects of multitasking in learning environments. Journal of computing in higher education, 15(1), 46-64.
  • Lang, James M. “The Distracted Classroom.” The Chronicle of Higher Education. March 13, 2017.
  • Miller, Michelle (2017). Addiction, Accommodation, and Better Solutions to the Laptop Problem. Dec. 8, 2017 blog post.
  • Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological science, 25(6), 1159-1168.
  • Patterson, R. W., & Patterson, R. M. (2017). Computers and productivity: Evidence from laptop use in the college classroom. Economics of Education Review, 57, 66-79.
  • Ravizza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2017). Logged in and zoned out: How laptop internet use relates to classroom learning. Psychological science, 28(2), 171-180.
  • Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24-31.


John: Today in Oswego, it is approximately six degrees and we’ve had about four feet of snow in the last couple of days. We’re going to talk about a topic that comes up often in many of our workshops at the Center for Excellence in Learning and Teaching here, which is student mobile device use in the classroom.

Rebecca: Today our teas are:

John: Blueberry green tea

Rebecca: …. and Jasmine green.

Rebecca: So, in 2017 an ECAR Study of Undergraduate Students and Information Technology report came out and indicated that laptop ownership is up to 95 percent amongst undergraduate students and smartphones device use is up to 97 percent. I’m sure Chromebooks and other inexpensive devices have certainly increased the amount of technology… but they’re everywhere… and I definitely feel that in my classroom I don’t think I’ve had a semester yet where a student doesn’t have a smartphone or a laptop with them.

John: Another thing that the study noted is that tablet use is actually declining a little bit over the last couple of years. Still, though, about 50 percent of students that have a tablet computer as well… and most students report owning multiple mobile devices. I’ve noticed the same thing, that over the last three or four years more and more of my students in my large intro class (with about 360 or 400 students) are coming with computers. I’d say probably about 60 to 70% of them show up in class with them now.

Rebecca: Do you have any sort of policy on using these sorts of devices in your classroom?

John: I actually do encourage the use of them for specific purposes. One of the main concerns that faculty express is that computers provide a very convenient source of distractions, and I do observe that…. because I wander around the classroom quite a bit and it’s not uncommon to see students doing things other than what we’re working on in class. So that’s an issue, I think, which is at the heart of most people’s concern.

Rebecca: How do you discourage that distraction in such a large class?

John: I remind them that their time in class is limited and that it perhaps might be more effective for them if they were to use them for class related activities, which could include note-taking… or it also includes frequent use of polling. I use i>clickers in class and many of the students, probably a third to a half, are now doing the polling using either a laptop or their smartphones. What’s your policy on computers in class?

Rebecca: I teach web design courses, so obviously having multiple devices on any given student is convenient because we do need to test our work on multiple devices. So we certainly use them, in that respect. When we’re going over new material, I encourage students that want to use their devices to sit in the periphery of the room, so that they don’t necessarily distract other people who aren’t using devices… and then a lot of times we’re doing things that require diagramming and what-have-you… and so it’s just more convenient to take notes… not using a computer… unless they have a device where they can draw really easily, maybe a tablet that has a drawing application or something on it. So that’s usually my policy and most students are fine with that… and I also include in my syllabus that if you are distracting people around you then I might ask you to put it away… which I generally don’t have to do… but I have had to occasionally.

John: One of the concerns is that there’s a number of studies out there that find that when students are using computers in class, in a manner unrelated to the class, it not only harms their learning… but it also harms the learning of those around them. One of the studies on that was done by Sana, Weston, and Cepeda in a 2013 article in Computers & Education and there have been other studies that have found the same sort of results. So I use it, actually, as a learning experience because we talk about externalities in my introductory economics class, where externalities involve side effects of your actions that either provide benefits or harm to others…. and I know that when you have something that’s distracting others around you, that’s going to harm their learning as well… and perhaps we should discourage such negative externalities.

Rebecca: I think one of the things that also comes up is… distraction is one thing…. so if you’re distracted, how do you change from being distracted to having focused attention, right? ….and so… obviously the polling that you do in your classes is a great way to get students to refocus and regain attention. Are there other things that you do in your other classes that aren’t so large?

John: In my econometrics class, I will often have students work on problems in class where I’ll give them them a problem… they’ll pull up their regression software… run some results… and then we’ll talk about it in class… and generally there’s much less of an issue in upper-level classes, because the students tend to have more intrinsic motivation. They tend to be more focused on the topic in those classes as well.

Rebecca: I’d say my one class is an upper-level class but it’s still an introductory class. It’s an introduction to web design and I have a wide variety of students from first year students to graduate students that are all in the same room at the same time, but they’re all beginners and it’s an elective, so there’s a tendency to want to be there in the first place. They have some sort of motivation for wanting to learn that, so I don’t feel those same kinds of strains about focused attention that other faculty do in a general education class. But there is a lot of information out there, or strategies that you can use to start thinking about how those general education classes connect to individual majors, right? So the more you can help students find how it’s relevant to them, the more likely they will be to engage in the topic. So I’d recommend for faculty to really think about that at the start of the semester, and make that a good sell at the beginning… and connect back to that over and over again throughout the semester, so that students recognize that the general education courses really do matter to whatever it is that they’re gonna do as a professional.

John: …. and I do the same sort of thing the first day of class. I’ll generally ask my students why they’re taking this course and I’ll take responses from them sometimes I’ll do it on a poll and the results will come up… other times I’ll just have them say why they’re there and very quickly someone will say “Well, I’m here because I have to be here.” I said “ Well, why do you have to be here?” and they’ll respond “It’s required for my major.” So then the next logical question is “Why is it required for your major? What about this course is it that people think is so essential for your major that you all have to take it?” … and then that helps them think about why perhaps the things they’re learning here are relevant for their future careers…. and then I’ll tie it to that a little bit. I’ll ask them “So, what are you going to learn in this class that will help you in your future jobs that come out of this major?” and I remind them of that periodically as well… and try to make connections between what we’re doing and what their educational and career aspirations are.

Rebecca: ….This is really related to the idea of the “promising syllabus” that both Ken Bain and James Lang talk and write about quite a bit, where you’re really defining what are the big problems that the class is going to address…. and coming back to them over and over again… and that might be a strategy to really provide that focused attention.

John: mm-hmm. So, more broadly, in terms of policies, there’s three broad sets of policies that people seem to be using: one is to completely ban all mobile device use in the classroom (and we have quite a few faculty here that choose to do that), others have more of a laissez-faire policy, where students are encouraged to use computers productively… and that’s the approach I’ve generally tried to use. and others where mobile technology is is required for specific tasks… and that’s also true in my classes to a large extent.

Rebecca: Although that can be challenging, if you happen to have equity issues, right, at your university? ….and students, that three or five percent that don’t have devices, you need a way for those students to have devices.

John: Well, actually, I think the three to five percent would be with any one device that they don’t have. Virtually all students have at least one device. It’s been many years since I’ve had a student who didn’t have either a laptop or a smartphone. Most of the students come with at least one, and many of them come with both. When I teach in a summer program at Duke, for example, there are some students who may not have it. I generally bring some spare fifty-dollar Android tablets and pass them out if they forgot to bring their device. It’s not that they don’t have it, but sometimes they forget to bring it with them…. And you can do lots of activities where people share their device, as well. So the equity issue is much less of an issue than it would have been a decade or more ago. It may be in some other institutions. In lower-income areas or in some community colleges, device ownership is a bit lower, but we don’t really have much of that issue here.

Rebecca: I usually discover that the first day of class when I do a survey of my students to see what devices they have and then knowing that information, I can design activities and things that take advantage of those devices or avoid using certain devices if students really don’t have them. But I’ve also noticed, in keeping with the recent study, that my students have had more and more of those devices available to them in every class period. People who decide to have a ban on technology in the classroom… if they phrase it like that in the syllabus, and talk about that, I sometimes fear that that just sets up like rules and regulations sort of philosophy for the classroom. That sometimes doesn’t set up a classroom environment or climate that is welcoming and open. So I think if you want to discourage that use then you have to be really careful about the language that you use around that… so it doesn’t feel like a penalizing system.

John: One strategy that some faculty have used is…. you could invite students perhaps to refrain from distractive use of technology, because it could be in their own interest.

Rebecca: …and explaining that, and explaining why you might want them to put it away, I think, it can be helpful. So, rather than having a strict policy, maybe having those minor discussions here and there about when it becomes distracting, or not useful, it could be helpful.

John: … and they’re going to be living in a world where mobile technology is not going away. They’re going to be in a workplace using mobile technology and perhaps learning how to use it more productively might be useful.
One of the reasons faculty give for banning devices is that there have been some studies suggesting that students who take notes by hand perform better, or recall more information, than students who take notes on computers. One of the problems, though, with those studies (and there have been many of them) is that generally it was based on situations where students self selected, in terms of whether to take notes on computers or to take notes by hand… and those studies generally didn’t control for self selection. The most common explanation of the finding that note-taking by hand was more effective than note-taking by computers was that when students take notes by hand they don’t write as much… so when they looked at notes taken by hand and by computers, students who took notes by hand tended to write much more condensed notes, and the argument is there’s more cognitive processing going on in the note-taking… because they have to think about what the key concepts are and the additional learning is really coming from that part. The people who took notes by computers generally type much more and it appeared when you looked at the number of words typed, for example, that they were attempting to transcribe everything that was being said… or everything that was projected onto a screen (for people who were using PowerPoints, or Prezis, or Google slides, or Beamer).

Rebecca: That was one of the issues that I had with some of the studies… Specifically was that the way that they were measuring whether or not the notes were effective was whether or not they had everything that was provided copied down basically verbatim plus some as the highest score… and I would argue that that’s not very good note-taking in the first place.

John: Well, I think most of them… they actually focused on performance on some type of tests. In terms of recall, some of them were controlled experiments where there was an experimental lecture provided, and then students were tested on it immediately after, or a day or two later. But the analysis seemed to suggest that simply transcribing everything that was said is not very helpful, as you suggested there.

Rebecca: Yeah, and we know from evidence-based practices, that simply copying down, without doing any cognitive processing about what is important or what’s not important about that content…. and how does that content connect to other things, isn’t very useful. …and just reviewing your notes, in general, is also not a useful studying technique. So, if you’re only looking at those kinds of features, you’re not necessarily observing whether or not the laptop or device is inhibiting learning.

John: So, the argument for banning it is that it will reduce the number of distractions and it will force students to take notes perhaps in a more effective manner. The only problem with that logic, from my perspective, is that I remember when I was a student… I didn’t have a laptop or a mobile device, and I was perfectly able to distract myself when I found a class to be boring. There was always a notebook that I brought (I actually had only one notebook throughout my undergraduate career, I was not very good at taking notes back then) and I used it to play hangman, to write down notes to make plans for going out after class, and other things with friends around me, and I really didn’t need mobile technology to provide distractions.

Rebecca: The equivalent of texting, right? You’re still communicating with your friends by a notebook instead of through mobile devices, right? I had perfectly colored biology diagrams, I recall.
JOHN… hence, the art major thing…
….but one thing that these studies did not do is…. they found these very strong results… but they didn’t control for self selection. They observed that students who chose to take notes on computers did less well in remembering things than students who chose to take notes by hand.
There was an interesting study, though, released on September 13, 2017, by Artz, Johnson, Robson and Taengnoi (not sure of the pronunciation on those) in which there was a randomized controlled experiment conducted in, I believe, five economics classes… and they sorted students by their ID numbers. Students who had odd numbered student ID numbers were placed in one group, students with even numbers were placed in the other. A guest instructor was brought in who provided a very structured presentation in each of the classes. Half of the students, roughly, were required in the first sequence to take notes by computer and the other half were required to take notes by hand…. and then they reversed that a month later…. where they had the same groups but they switched the odd and even numbers. When they were tested on the material two days later, there was no significant difference. What this result seems to suggest is that the earlier studies were subject to sample selection bias, and the basic problem is that students who were more likely to choose to take notes on computers were students, on average, who would do less well whether they took notes by hand or by computer.

Rebecca: Students who weren’t as strong necessarily academically would be the ones who are more likely to choose to take notes on a computer… maybe that’s because they weren’t as interested in the subject matter…, maybe they didn’t really understand the subject matter and they were choosing that device, in particular, because they could “multitask.”

John: …and one of the things we do know, and this this has been found in many studies, is that none of us is very good at multitasking. The students dramatically overestimate their ability to multitask and they underestimate the amount of time they actually do multitask… and pulling away your attention to look at a text… to respond to a text… to jump to a website… to respond to a Facebook, or an Instagram ,or other notification… has a fairly strong adverse effect on your recall… on your ability to remember things… and students significantly underestimate how often they do that, as measured by studies where they’ve they were using a proxy server to actually track students’ use of mobile devices.
The main conclusion of the study is that it really doesn’t matter whether students take notes by hand or whether they take notes on a computer.

Rebecca: That leads me to kind of question whether or not students are just not very good note takers, right? We don’t spend a lot of time, especially at the college and university level, teaching students how to take good notes… and they think that copying verbatim whatever they see on the board, or what have you, is how to take notes, rather than taking the time to reflect, process, and figure out what things mean and putting it in their own words. So, I wonder how many faculty actually take any time to discuss note-taking or what might be effective in a particular class. I remember in Small Teaching that James Lang suggested using an outline and providing students with outlines that the students could fill in with blanks to follow along and focus their attention but also structure their notes in a way that might be useful. So, that’s one technique faculty could use.

John: Right, it could be an outline, or it could even be distributing… for those who use PowerPoint slides or other types of presentations …distributing those with some gaps, because if you provide students with too much detail then they tend not to use it very much… they tend not to annotate it quite as much. One of my former colleagues, Bill Goffe (who was on the show a few weeks ago), when he shares PowerPoint slides (or at least in the past when he shared PowerPoint slides) he’d leave the graphs off of it. So, he’d have some text there and some basic outlines, but then students would have to fill in part of it. So he’d share the notes in advance, but there would be things they’d have to fill in… which fills that same sort of purpose, of having a broad outline but then requiring students to make some connections on that as well.

Rebecca: Similarly, I think students generally don’t have very good metacognition, right? ….and so I find that using quizzes in class and going over those quizzes is a good way for students to fill in their notes. What I do is a series of review questions most class periods at the start of class that interleave different topics that we’ve had throughout the semester. Students take those quizzes and then I have them self grade it in a red pen… in a different color… and we go over it… and I require them to take notes about why their answer is incorrect and what the correct answer is. So I’m forcing them to do that reflective piece about what the thing means, and put it in their own words, essentially. If they just regurgitate something that I had said, and they can’t actually explain what that means, then we take that opportunity during the review questions… time to figure that out or take the time to do that.

John: In my class, probably more than half the time… sometimes 60 or 70% of the time, I’m giving students problems where they have to solve them, and I use Eric Mazur’s think-pair-share type of methodology with clickers, where they first answer the questions individually and they give their responses… and then if about half of them get it incorrect (I try to aim for questions where about half of them will get it correct the first time) then I have them discuss it and work on it and then they vote again… and generally there is some significant improvement and it’s that retrieval practice that’s probably much more effective than just simply lecturing to students. I do give some short lectures in class for 5 or 10 minutes at a time and students will often comment that they need more time to write things down and I’ll remind them that perhaps what they should be writing down is not every word that I say or every word that they may happen to see but the things that would be most helpful for them and making those connections… because I share any slides that I use with my class, so they don’t really need to try to transcribe everything. If they really want a transcription of everything, I record the class. They could watch it. We use Panopto here. They could watch it later if they need to go back and review something. They don’t really need to use the notes for that and they should be focusing on trying to use their time more effectively to help them understand the things that they don’t understand as well… and the retrieval practice that’s being done helps them… or at least the goal is, to help them understand what they don’t know… to improve that metacognition as you mentioned.

Rebecca: One of the things that I started doing after we’ve had our reading groups on campus, is spending more time in class talking about evidence-based practices, so that students are more aware of how they learn so that they can be more effective… But one of the things that this study is encouraging me to think more about, is to maybe spend a little bit of time in class to go over note-taking strategies… and when I’m finding that students are struggling with that, to kind of review those again. I do the evidence-based stuff frequently when we’re doing review questions because I often ask “Now, why do we do review questions?” before we do them to encourage students to take advantage of that opportunity… and remember that it’s not really a penalizing thing, but really about learning. So I think the note-taking is something that I want to take on maybe this spring and think about how to help students get better at that because every time I’ve asked a student who’s asked for help on something… and I say ”Well, where is that in your notes?” …their notes are like a disaster and they’re not organized. They’re in multiple notebooks, they’re in random scraps of paper or whatever. So I think finding methods to help students do that more effectively is worth the time and effort and I’d encourage other faculty to think about taking that on.

John: Normally, ask our guests what they’re going to do next, but now we’ll ask each other. What are you going to do next, Rebecca?

Rebecca: So, related to laptops and things in the classroom, I think that I’m going to be a little more cognizant about actively having students use their devices. So if they have them, that we’re taking advantage of the fact that they’re in class, and I’m definitely gonna find a way to spend some instruction time talking about note-taking. How about you, John?

John: I’ve been expanding, gradually, in talking more about why I’m doing things and to help motivate the methods that I’ve been using in class with retrieval practice, spaced practice and interleaved practice. It doesn’t always get through to students, but I’m gonna keep working on that… but I’m also going to be trying a couple of new things this spring semester. One is following up with the discussion we had with Jeffrey Riman, which came out in episode 10, I’m going to be using VoiceThread in at least one of my classes this spring, and I’m also going to flip my econometrics class a bit more than it has been flipped in the past… and I’m also going to write a couple new chapters for the book that we’re using. So I’ve got a lot of plans for this semester. I hope I manage to get them all together before the semester starts.
Thanks for joining us.

Rebecca: Catch you next time.