Category: collaboration

19. Common Problem Pedagogy

Posted in active learning, authentic learning, collaboration, common problem pedagogy, higher education, interdisciplinary projects, learning, pedagogy, project based learning, service-learning
March 7, 2018

Most colleges are organized as a collection of academic silos. Many challenging problems facing society, though, are multifaceted. In this episode, Leigh Allison Wilson joins us to discuss the use of common problem pedagogy, an approach that allows students to address a problem from multiple disciplinary perspectives.

Leigh is the Director of the Interdisciplinary Program and Activities Center at SUNY-Oswego. She is also the author of two collections of stories, one of which won the Flannery O’Connor Award for Short Fiction. Her stories have appeared in the Georgia Review, Grand Street, Harper’s, The Kenyon Review, Smokelong Quarterly, The Southern Review, and elsewhere. Leigh teaches creative writing at SUNY Oswego. In addition to the Flannery O’Connor award, she has received the Saltonstall Award for Creative Nonfiction, and a Pulitzer nomination by William Morrow for her collection Wind. Leigh is a Michener Fellow of the Copernicus Society and is a Henry Hoyns fellow of the University of Virginia.

Show Notes

Transcript

John: Most colleges are organized as a collection of academic silos. Many challenging problems facing society, though, are multifaceted. In this episode, we explore common problem pedagogy, an approach that allows students to address a problem from multiple disciplinary perspectives.

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Leigh Allison Wilson. She is the author of two collections of stories, one of which won the Flannery O’Connor Award for Short Fiction. Her stories have appeared in the Georgia Review, Grand Street, Harper’s, The Kenyon Review, Smokelong Quarterly, The Southern Review, and elsewhere. She teaches creative writing at SUNY Oswego. In addition to the Flannery O’Connor award, she has received the Saltonstall Award for Creative Nonfiction, and a Pulitzer nomination by William Morrow for her collection Wind. She is a Michener Fellow of the Copernicus Society and is a Henry Hoyns fellow of the University of Virginia.

Rebecca: Welcome, Leigh.

John: Welcome, Leigh.

Leigh: Thank you, John. It’s very nice of you to have me

Rebecca: Today, our teas are:

John: Ginger peach green tea.

Leigh: Mine is Constant Comment, a southern favorite.

Rebecca: …which Leigh brought for me to try, so that’s what I’m drinking, too.

So, Leigh you helped organize a number of community-based projects that bring faculty together across campus. What got you involved in this kind of work in the first place?

Leigh: Well, you know what? If I went back to the roots of it all, I have to say Amy Bartell in the art department. I have a flash fiction class that is my advanced writing class, and one semester she just suddenly said: “Why don’t your students write a very short piece? My students can illustrate it, and we’ll frame both things and put them side-by-side… and then we’ll have a show.” …which was so much fun, but it wasn’t just fun, it was my first taste of having a collaborative common problem project. Because, it turned out to be a common problem. We didn’t know it… we thought we were just writing our fiction… or, I thought they were just gonna be writing their fiction. But we’ve discovered that if there was going to be an illustrator paying attention to it… all of a sudden, it got more serious. The game got more serious. There was an audience who was really going to be checking it out, and there was also an audience that was going to be looking at the illustration and looking at their work at the same time, and all of a sudden the students were much more professional about their attitudes to their work. So, that’s the beginning of it. That was called Graphic Flash… and we’re still doing it, but now it’s expanded into a film class that’s taking the stories and making short films out of it… and a music class that’s taking the short films and scoring them… and ‘cause now I like working with local partners… local high schools have been making movie posters.

Rebecca: Great.

Leigh: …for the stories. So, that expanded… and because of that expansion, I started getting interested in – not just common projects that involved a common problem – but also collaborative projects in general… and the ease with which they could be expanded… which I think is one big factor in project-based learning.

Rebecca: The first big project was the Smart Neighbors project which is still ongoing.

Leigh: What happened was… I was doing Smart Neighbors and there was a notice from the Provost office and there was a call for participants in a SUNY wide grant. They wanted four SUNY schools to be involved in a common problem pedagogy grant… and at the time they were trying to get a Teagle grant which is an ExxonMobil grant. But the point of the Teagle grant was to get humanities to work with another discipline, usually a professional discipline, so that’s why it began in that way. I wrote in and SUNY Cortland and Oneonta and Plattsburgh were all involved in it. We all have different projects going on but ours became the Smart Neighbors project.

Rebecca: Please describe what that is for those that don’t know?

Leigh: Basically I have always….Well, I love Oswego as a town, and I’ve loved living here and I’ve always wanted to do something that could give back. But, I’m a creative writer and, short of putting it as a setting in a lot of short stories… which I have done… that’s not really giving back… I have always worried since I’ve been here about the economic difficulties facing any new business. This is a stat from a few years ago, but one statistic is that a new business in Oswego has a lifespan of about 13 months… and that’s a terrible statistic. I don’t think it’s true anymore… I think there are great changes going on in town now… but, I wanted to do something with the town. My concept for Smart Neighbors was to have a lot of different disciplines collaborate in the promotion of a downtown independent business. It was a simple concept, because I didn’t have elaborate blueprints for what they should be doing or what we should be doing. I had no elaborate plans for what each individual discipline should be doing. It should be promoting the business. Period. ….and that’s sort of continued to be how it is. People take it as they can imagine it… and so a lot of very imaginative things have come out of that… the things that are not traditionally considered promotional materials… which, in fact, really are promotional materials.

John: What are some examples?

Leigh: A literary citizenship class that Donna Steiner is working with, because they’re mostly creative writers, they tend to do digital essays… but they’re digital essays that often have a fanciful story involved in them. So, if it’s a bookstore… one digital essay took a book that the bookstore was selling… talked about the author ….did graphics about the plot of it… and then ended up back at the bookstore… and so you basically you were interested in the book… and then it began to talk about how the imagination could be served by the bookstore. Another one in the same class followed someone who bought a book to their home, took film clips and photographs of the person sitting where they liked to read with all of their books around them… and just talking about what it meant to be able to walk downtown and buy a book and take it home and start reading it. So, that was a nice little piece too. …but not things that you necessarily are expecting, or what an advertising agency would have put out.

John: …and how have the businesses responded to this? Have they been using these materials in their marketing?

Leigh: They have. One of the things that is a centerpiece is the banner… and the art students… the photography students have been at the heart of that… and all of the businesses end up displaying it. There are huge banners… they fill a whole wall… but all of the businesses have been using the banners. They love those… Also, every business nowadays… and this is one thing that we’ve been working with the businesses on… having an online presence… but that’s one of the reasons there’s so many digital projects involved. Because we want the businesses to be able to use them online. So, the digital essays do get used online as part of their presentation to the public.

John: …and how have the students reacted to doing something where their work is going to be more public? They’re not just submitting something read by their instructor and their peers, but it actually may have an impact on some business in the community.

Leigh: The impacts on our students are the impacts that I think they’ve found across the country when dealing with applied learning, civic engagement, volunteerism… well, basically best practices in general…. but, number one (this is the thing that I’m most proud of) is that the students leave that program, even though it’s one assignment in one course (for most of them… it’s not the whole course) but they leave having experienced that assignment with a sort of sense of social responsibility that I don’t think they had before… or a notion of philanthropy. One of the things I tell them…. All of the classes (this year we had 11 classes from different disciplines) and we all meet at the beginning of the semester in Marano Auditorium… and one thing I told them this year is that we think of social responsibility as as one thing and philanthropy as another thing… but really, I think, what we should be doing in these places we love (like I love Oswego) is actually contributing our talents… not just our money… but we should be spending our money locally too – but but also contributing our talents – to these businesses… even if we’re a business owner contributing to another person’s business is something that I think we’re obliged to do too – because the local success really is our own success… and we tend to think of businesses as competitive, but I think that’s a mistake. I think smarter neighbors…

John: …hence the name…

Leigh: …work together in these collaborative ways.

Rebecca: What are the biggest lessons that you’ve learned from doing some of these projects?

Leigh: Well, I should tell more of what the students got out of it, but…

Rebecca: Yes.

Leigh: I think, other than just that sense of social responsibility and what notions of philanthropy, they leave knowing much more clearly what they know and have learned in their disciplines… meta-knowledge of what they’re capable of… which is huge for our creative writers. Because, I don’t think they’re clear on the fact that… they know they’re probably not going to immediately get the Pulitzer, but what can they do with this? …and it’s important for them to to learn that…. that they can write for multiple audiences in multiple ways. But, they also learn what other disciplines know and can do… which they haven’t thought about that deeply. It’s a mystery to them what, for instance, the marketing students do. They market things… maybe it’s advertising… something like that… but then they see them come in and actually take the business that they’ve been working with and figure out a plan for them… and how the college itself can be moved into that plan… and suddenly: “Oh, I can work with that…” and they start thinking of digital essays they could work with… and imaginary stories that take that marketing plan and actually enact it with characters (which they’re good at imagining)… And professional skills… just getting somewhere on time… being late or on time to a class seems less important, but if the interview that you needed to have… and you’re late for and you can’t now have it… that makes an impact forever. You tend to be on time for an interview… and they do have to interview the local partners. Preparations… They get there. Nobody’s going to be telling them what to do. They have to figure out what they need to know, and they need to find it out. So, they need to plan before they get there. I personally am very happy with my students learning what it means to write for a particular audience, as opposed to whoever they want to. It’s very good for them to try to please a certain person with a certain product.

Rebecca: Because it’s usually an audience that they wouldn’t have picked or imagined on their own.

Leigh: That’s right… that’s right. …and my point would be that, even when they’re writing their Great American Novel, they should be expanding their notion of what audiences they’re hitting, instead of just “this is what I want to read.” They need to think about what their vision of the world is and how to can pull as many people into it as possible. I just think it’s memorable to them. I think it’s life-changing to them to work, however briefly, donating their time to a place at least for a while they’re calling home.

John: Excellent.

Leigh: But, I think they’re things that the faculty learned too… not just the students, there are faculty outcomes, I think, as well. My whole idea in Smart Neighbors was to just get faculty’s feet wet with one assignment in one class… and if you can do that… once they see the effect on students… because, that’s one thing I really do believe about the faculty here… they really are committed teachers. Now sometimes you worry about how time-consuming is it going to be to work with another class as other disciplines… how time-consuming is this or that? Because we’re already putting a huge amount of time into our teaching. So, it seemed smart to get faculty accustomed, or introduced to, collaborative, or civic engagement, or applied learning kinds of pedagogy in the easiest possible way. So, one assignment… and not an assignment that necessarily requires interactions with a lot of other faculty to figure out how to do it. Now, I will say, for Smart Neighbors anyway, the faculty do have to connect with the local partners. But, they don’t necessarily have to figure out what everybody’s doing in all of the classes to make it work. They have their piece of the puzzle and they’re contributing it.

John: How many classes work with a particular business? Are there multiple businesses that they’re working with? or is it just one business each year?

Leigh: Well, it’s grown. The first year, we had four classes and they were working on the bookstore. The River’s End Bookstore.

Leigh: Tell me your question again.

Rebecca: Really asking whether or not there is more than one community partner at any given time.

John: Yes.

Leigh: Yes. I think what you’re asking is a good question because, once you get to a certain number of people… of courses… not people, but courses… you’re overwhelming a local partner and we got to that quickly last year. We worked with a candy store (and I think there were seven different classes involved) and an unbelievable generosity of time from that owner… but it was clear that we were gonna have to figure out other ways of doing this. So, last year we did the Farmers Market, which worked out, We had eleven courses involved too – and that worked out much better because there are multiple farmers bringing their goods to the Farmers Market and there are they’re in different groups with different farms. So that worked out a little bit better. Also, because the Chamber of Commerce is ultimately responsible for the Farmers Market, we were able to do some projects just for the Chamber. For instance, they needed a new logo and we sort of pulled that into the Smart Neighbors project as well. So, I’m trying to define what we’re doing a little wider. …and you’re right, have more local partners…. if we’re gonna have this many continue.

John: It sounds like it’s grown really quickly.

Leigh: It really has… and I will just say again I think the faculty discovered that there’s a certain ease of practice in getting used to this… and once you see the students and the effect on the students, then I think you’re hooked. And the reason it’s grown is that the courses who have done it in the past continue to do it; they want to keep doing it. And that is how I got the idea for Grand Challenges.

Rebecca: That seems like a nice segue right into it, right?

John: Yeah.

Rebecca: So, we’re launching the Grand Challenges. Can you talk a little bit about what the Grand Challenges are and what the goals are?

Leigh: There’s a line in our strategic plan that that’s my favorite line…. and I think it’s the most memorable line… and it talks about how we, as a community, are going to tackle the Grand Challenges…. find solutions to the Grand Challenges of our time… and I love it… because it’s aspirational for one thing. I really do want to believe that our students and our faculty can tackle the Grand Challenges of our time, and I think we can, frankly… but it’s also that notion of “tackling a challenge” is very project oriented. You get your hands dirty. You figure out something, and then you try to come up with solutions because of it… and, so it appealed to me just in terms of having a common problem. But, those Grand Challenges have to be tackled together. I mean, I don’t think there’s any challenge of any size in the complexity of our world today that can be done by a single person just sitting in their garage thinking. I think almost everything we do in the future is going to have to be collaborative and probably cross-disciplinary in some way. So, it just seemed to me a natural segue from Smart Neighbors to getting the whole campus to work on a single… it’s not really a single issue either…. it’s more… we were talking about this, Rebecca, I imagine the topics for Grand Challenges to be very concrete things, because I think, as academics, we tend toward a more abstract way of looking at things…

Rebecca: …which is particularly hard for our students to get their heads around. They need something tangible.

Leigh: Right, I think so too… and to come up with projects… actual projects that are going to take place in the world with local partners… or involving civic engagement or volunteerism… require a certain concreteness. So, at any rate, the Grand Challenges project was just something I began to think. The notion of having multiple disciplines work on the same thing… it’s just a short step to getting the entire campus to work as much as much together as possible on the same topic… One of the things I didn’t say about Smart Neighbors is that Oswego is already a very collaborative culture… and that we’re very far along in terms of faculty tipping into these kinds of projects very easily…. and I’ve found just talking across campus, the way for instance when I spoke to Faculty Assembly, and the reception there was so astonishing. People aren’t resisting it out of hand. It’s just such a pleasure to work with people who are willing to take on these new things without immediate misgiving. At any rate, as you know, the topic that we’ve that we’re working with this year is fresh water which is concrete, but also can involve a lot of sustainability.

John: …but fluid, too.

[LAUGHTER]

Leigh: Very funny, John…
But, one of the things that I like about that particular topic is that you can look out any window on campus and fresh water is exactly what you’re looking at… and that it should matter to us makes sense to me. But, to go back to the teaching culture here, I have found when I talk about this to any group of faculty, immediately ideas are popping. They’re thinking about it. They’re talking about it. They clearly already thought about it. The Grand Challenge doesn’t really even begin until the fall of this year… and I’ve got a list …I brought with me a whole list of like couple of dozen projects that people are already doing right now…. this semester…

John: In preparation?

Leigh: Just because they can. Not only in preparation, just… let’s begin… Why wait till the fall? I’ve spent the last week finalizing touches to a micro grant the Provost office has, thank goodness, very gallantly is going to put some money in a pot to give some grants to people to do these collaborative works. Well, let’s just put it this way… even if you’re just doing an assignment in your class, you can put in for one of these grants. But, I think we’re going to privilege, probably, the collaborative civic engagement projects… or they’ll get the higher money amounts, just because there are more people involved. The administration on campus has just been so supportive. The provost office is doing the micro grants. The Student Affairs has, I can’t talk about it because the contracts haven’t been signed, but they’ve got people who are well-known coming to speak on campus.

John: So, there’s going to be some other programming throughout the college.

Leigh: That’s right. Artswego has a special category for its grants this year that are going to privilege some Grand Challenge proposals.

Rebecca: What I like about that concept is that the learning doesn’t just happen in a classroom on a college campus. It’s happening from multiple perspectives and it’s happening in and out. It’s happening formally and informally.

Leigh: That’s right.

Rebecca: That’s nice that there’s a lot of systems in place to help support that and that idea because, if students are experiencing the topic of water, in a lot of different disciplines on and outside of class right then they’re gonna start seeing how all these things connect together…

Leigh: That’s right.

Rebecca: …and we have general education as a part of our curriculum, as many colleges do, and the students tend to not have any idea how that is relevant or important or what that does for them. and I think this might be a really great way for them to start seeing that all these things are actually connected and it’s important to know different points of view and the different disciplinary perspectives on things… so that there is that idea that we can’t tackle these really big problems…

Leigh: …by ourselves.

Rebecca: …without looking from multiple perspectives.

Leigh: Yeah.

John: …and faculty are often in their own silos and students see the classes as separate islands that are not connected in any way… and showing that we can look at the same issues broadly from a number of different perspectives might help them form better connections and deepen their learning.

Rebecca: …and even continue to update the curriculum to reflect this change in practice. It’s a move away from silos to things being a little more messy, and so how do you allow for your curriculum to embrace that messiness.

Leigh: I think you’re exactly right, Rebecca. …and I, for one, think the future (it might not be in our generation) but the future really will be a future that doesn’t necessarily have departments… doesn’t necessarily have disciplines separated in this way…. that in fact encourages cross-disciplinary activity. I think the School of Communications, Media and the Arts [SCMA] is already sort of moving toward that. They’re a very collaborative school and work very well… that just in my experience doing these projects, they work very well across campus with any discipline.

Rebecca: Go SCMA.

[LAUGHTER]

Leigh: I am on the board. It’s because of that that I asked to be on their advisory board, frankly.

Rebecca: Yeah.

Leigh: But, yeah. I think the beauty of the grant, of Grand Challenges, is that we’re already a collaborative school and this just puts the name on it. It puts a focus for that and it’s something I think we really ought to be celebrating here. …and to get back to the administration being supportive, the President from the beginning has been behind this and I think that, really more than anything, has been one reason for this to be a successful rollout.

Rebecca: Are there plans to research or study the outcomes of the initiative to measure what impact doing something like this has on our learning community and/or on the community at large?

Leigh: Well, one of the things that I hope from these micro grants is, because they have to give the proposal at the project proposal… and give what they hope the outcomes will be… and then when they do their final reports, what they think the outcomes really were. I’m hoping that that will be the first step toward being able to assess some of the things going on. It’s more difficult in the general population, One of the things I’m reluctant to do is add a layer that makes people hesitant to get their feet wet with these pedagogies. But, I think, just once this gets going… I think it will become easier and easier to get people to assess for what the outcomes are. To be honest, I think it’s so night and day what the students get out of these best practices that the faculty will want to start assessing and seeing what these outcomes are and what it means in their classroom.

John: In an earlier discussion, you mentioned that your work with the Digital Oz project grew out of your work with the Smart Neighbors project. Could you tell us a little bit about the Digital Oz project and how it relates to your work with Smart Neighbors.

Leigh: Digital Oz is a presentation… online presentation site… for SUNY Oswego students’ digital work.One of the things that occurred to me after doing Smart Neighbors is that these collaborative efforts on campus are here and gone tomorrow …because there’s no place to archive or curate the materials that the students produce… and so Digital Oz has become a space where the collaborative work can actually be presented. The students are doing such amazing work. It’s great that Digital Oz exists so that the students can have some sort of public presentation.

John: Could you describe Digital Oz a little bit for listeners who may not be familiar with it?

Leigh: One of the things that I’ve always liked about Oswego students is that they have authenticity that is almost indescribable… but once you see them tell a story, you feel it instantly… and so I think because our students all have these stories it’d be nice if we had a site that had them tell them. So, we created Digital Oz and it has different categories. One category… the students talk about how they ended up being passionate about what they’re passionate about here (whether it’s their discipline or some sort of co-curricular activity that they do) and what’s the story behind that. How do they become passionate about it? …and there’s some amazing stories there. Students who, for instance, work as EMTs on the ambulance service on campus have some unbelievably touching stories about why they care… about being able to go to somebody and help them. But, there’s another category that’s called “moments that change their lives” …the students lives, and they talk about them in very moving ways as well. But one of the categories, as I said is “Collaborate” and students who have worked together on projects put artifacts that they’ve created for those projects online… and those two are… I guess you don’t realize the range and creativity and professionalism of our student work until you start seeing it put together in the same place…. And Digital Oz, since we’re talking about it… I’ll just say it’s… digitaloz.oswego.edu is the website if you want to look at it. But, it’s a place, I think, high school students look at and find feel like they can have a home here.

John: Excellent, and we will share that link in the show notes.

Leigh: Thank you.

Rebecca: So usually we like to end with, “What are you going to do next?” So, you’ve got this big giant project.

John: It’s still under way…

Rebecca: You’ve got this big giant project. What’s down the road a little bit for you?

Leigh: Well, I really do think that the Grand Challenges is as grand as I’ll probably get.

[LAUGHTER]

Because I don’t know how I can get grander.

John: The very Grand Challenge.

Rebecca: Super Grand Challenges.

[LAUGHTER]

Leigh: I know… it will be like Mario. But, one of the things… I’m talking to the woman who’s in charge of applied learning at SUNY Central, and I’m gonna talk up the Grand Challenges just because I think it really is a harbinger of what the future is going to be, not only in terms of what you do in collaborative ways, or best practices but also in what it’s going to ultimately mean for what the shape of the university is. So, I guess I’m not going to become a traveling advocate across the campuses across SUNY, but I do think I really do think this is where the future is headed for higher ed. I hope so anyway. I do.

Rebecca: Great. Well, thank you so much for sharing what you’ve been working on, Leigh. I think everyone will continue to be inspired.

John:Thank you. It’s a great series of project.

Leigh: Thanks, you guys.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

15. Civic Engagement

Posted in active learning, authentic learning, collaboration, higher education, interdisciplinary projects, learning, pedagogy, project based learning
February 7, 2018

Real-world learning experiences come in a variety of flavors. In this episode, Allison Rank, a political scientist at SUNY-Oswego, joins us to discuss how she has built a course in which students organize and run a non-partisan voter registration and get-out-the-vote campaign. This project combines many of the best features of service learning and simulation.

Show Notes

Transcript

Rebecca: Real-world learning experiences come in a variety of flavors. In this episode, we’ll explore ways to combine the best features of simulation and service learning to increase learning in a campus-wide voter registration and get-out-the-vote campaign.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist,

John: and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.
Today, our guest is Dr. Allison Rank, an assistant professor of political science at SUNY Oswego. Allison is an expert in the role American youth play in the electorate and the founder of voter registration initiative called Vote Oswego. Welcome, Allison.

John: Welcome.

Allison: Thank you.

Rebecca: Today our teas are:

John: Yorkshire gold.

Allison: I don’t drink tea.

Rebecca: It’s an epidemic…. like… this is the third one, John.

John: I know, three in a row

Rebecca: Three strikes you’re out. No more non-tea drinkers… All right… I’m drinking English Afternoon despite the fact that it’s still morning… because I need it.

John: Just barely morning, though.

Rebecca: Good, okay.

John: So, what is Vote Oswego?

Allison: Vote Oswego is a student-run, nonpartisan voter registration and voter mobilization drive on the SUNY Oswego campus in the Fall of 2016.

Rebecca: What led you to start Vote Oswego?

Allison: So, prior to earning my PhD in Political Science, I actually spent three years working as a political organizer. I’d worked for a presidential campaign in the state of Ohio and then had spent a couple of years working with college students on a variety of non-partisan campaigns. One of the things I learned from doing that work on college campuses, is that when students have an interest in doing political work, there’s a lot of skills that they can get out of doing that work but that they don’t necessarily think about. And so once I was here…. I was here first in the Fall of 2014… and saw what the voter registration drive looked like, it was clear that students were doing some volunteering for it, and some students were really excited about it, but I didn’t think they were really getting any organizing skills out of doing it. They were sort of more just sitting at tables and sitting out with voter registration forms the same way they would sit out with cupcakes at a bake sale… it wasn’t really about organizing skills. And so I wanted to start a course or something here where the voter registration drive would become more about students learning to organize rather than just being treated as widgets to be organized by other people.

John: Did you do this as part of a class or was it a set of classes or a separate activity?

Allison: I actually was able to get permission to run it as a special topics class in Political Science. I’ve since gotten it approved as an official course, but initially I actually just pitched it as a practical political skills class where the students would come in and learn about grassroots organizing techniques and then get to implement those techniques through a voter registration drive.

Rebecca: I would imagine that a course like that would be particularly helpful to campuses that are more rural than urban.

Allison: Yeah… for me, after a couple of years here, I had a lot of students that wanted to get involved in politics but I’d end up in conversations with them about how hard it was to figure out transportation to Syracuse, figuring out logistics or the cost of doing it– we have so many students not only in terms of being a rural campus, but also in terms of the student population that’s also trying to juggle working and paid work that they need in order to be here. And so then, taking time out to do a political internship, especially with the schedule around an election, can be really challenging. And so being able to offer that opportunity on campus, and also around something that can give them course credit or internship credit without leaving the campus, and for something that the campus is already gonna put energy and attention towards, I think, is really helpful.

Rebecca: Is running something like this as a class common on other campuses?

Allison: I don’t know of other places where a full grassroots campaign has been run out of a class. It’s fairly common in Political Science to have some type of activity based around voter registration, right? So, for us in Political Science, coming up with civic engagement projects where you can avoid partisanship and partisan issues, is a really big deal. And so non-partisan voter registration drives around elections are a great place to do that, but often it’s asking students to go out and volunteer as poll workers, or do exit polls, or maybe helping set up a campus debate with a couple of candidates rather than really digging into an on-campus, full-fledged grassroots mobilization campaign.

John: Was it easy to keep it non-partisan in the classroom?

Allison: Oddly, it really was. The very first day of class, I ran an non-partisanship training with the students. So what does it mean to behave in a non-partisan fashion? What does it mean to keep your social media non-partisan through this event? What are the conditions under which you need to be non-partisan, right? Students signed up for this class because they’re people who care about politics, right? So many of them, I am certain, had very deep feelings about what they wanted to happen in this election. They weren’t allowed to talk about them if they were at a Vote Oswego event. I think there was a guideline around however many Vote Oswego students were hanging out together, that they could be recognized as a group of Vote Oswego students. If they were wearing their Vote Oswego t-shirt, they could not both talk about something partisan and have any reference to Vote Oswego in, for instance, a online social media “about themselves” section. And I would actually, essentially, run pop quizzes with them where I would try to get them to do something partisan, right? I would come up to them and say some incredibly partisan statement and they would actually have to practice what the non-partisan response would be.

John: That’s a useful skill, today.

Allison: Yeah!

Rebecca: Probably one th at a lot of faculty could use some training on, too, because politics come up a lot in classes. Can you give us an example of something that you would do?

Allison: Sure. So one of the things that is often defined as partisanship is if you endorse an issue that is so clearly identified with one political party over another…

John: …like science….

Allison: …even if you don’t, we would use things like building the wall, right? If you say something like “you should register to vote because it’s really important that we build the wall,” that would be considered a partisan statement from the last election, regardless of not mentioning a candidate or a political party. And you would get individuals coming up to the table who wanted to register that would say things like, “it’s really important to me that I register because I really care about maintaining woman’s right to choose” or “I really care about building the wall…” something that you could clearly align. And so I would do that to students, and they essentially had a set of responses they were allowed to give. So things like, “I’m happy to hear that you’re excited about what’s happening in this election, it’s really important that you get registered to vote” or they’re allowed to not and say, “I acknowledge that that’s something that you’re really passionate about.” Vote Oswego is non-partisan, we just care that you’re able to express whatever you care about, right? It’s sort of acknowledging that that individual has something that they really care about, but not endorsing it yourself.

John: How did they do in those pop quizzes?

Allison: They generally did really well. The first day, they would get really awkward and nervous and not know what to do, but after sort of half an hour of drills, they got incredibly good at it. It also helped that it was in the syllabus and they signed a contract with me that if I caught you violating the non-partisan mandate after one warning, you automatically got fired from the campaign, which meant you failed the course. So they took it seriously.

John: So it was somewhat high-stakes.

Allison: Yes.

Rebecca: A little arm-twisting there?

Allison: Yes. But they did really well, and actually after the election, a student made a comment in class where he basically said it’s really weird to me…. I feel like I know people in this class so well and we’re really good friends and we’ve worked so hard and I have no idea how anyone in here voted and I said, “that’s great, that’s exactly what should have happened, please don’t talk about it.”

Rebecca: Can you talk about some of the other results that you saw in your class?

Allison: Sure. There’s sort of two different sets of results, right? We talked about the students as having two identities in this class. They were both students and they were staffers and they needed to be concerned about themselves in both of those roles. So as staffers, they had fantastic success. The campaign registered over a thousand students on the SUNY Oswego campus, they helped over 1500 students request absentee ballots, they came up with some really great campaign strategies in terms of helping students with absentee ballots… get those mailed in… get those stamped… helping students get to the polls… building a great coalition with other folks on campus. As students, I think what was great is that because the students were out in the field and they were known as Vote Oswego students on campus. Their friends all knew that they were doing it, they were in those t-shirts all the time, they were visible. They took a real ownership over this project, in a way that I have a hard time envisioning getting students to do about short-term volunteer work, or sort of asking them to go volunteer with another campaign, or even the type of simulations that political science professors can get really good results with, in terms of learning outcomes, the type of ownership that these students felt and how seriously they took it, I’d be hard-pressed to get that result in another way. Because they took it so seriously, and because they took such ownership of it, I think their critical thinking and analytical skills really, really improved. You could sort of watch as we went through the campaign, students go from looking to me and looking to the couple of interns that we had in leadership roles of the campaign, to figure out sort of like “all this thing happened and what am I supposed to do” and “please answer this question for me,” to like, “well, this thing happened while I was standing at a table and here’s what I did,” and I would hear about it three or four days later as opposed to getting a sort of frantic, “help me figure it out.”

John: they started taking more responsibility–

Allison: Exactly.

John: –and making more of the decisions then just reporting back.

Allison: Exactly. And then also being able to constructively critique each other’s decisions once we– we called classroom meetings campaign meetings, right? So in campaign meetings, being able to say, “Hey, I know this is what happened last week, actually I think we need to fix it in X, Y, & Z ways.” Which, for those of us who have tried to get students to give critical constructive feedback on each other’s papers, it’s really hard to get them to engage each other that way, and the students really took to that sort of analytical and critical work with each other in really constructive ways by the end of the campaign.

Rebecca: So in addition to students finding that kind of personal ownership over the experience, what are some of the other factors that you think made this particular project, in this particular situation, really successful?

Allison: I think there are a couple of things that made this project work really well. I think that, one, is that a non-partisan voter registration drive is something that students can get excited about, even if they’re really uncomfortable with the idea of the conflict around politics. So students that are interested in politics, but don’t really want to be in the debates around politics, can latch on to this as a project that they can get excited about. So, for instance, we had a number of students from PR who took this class because they saw it as that they didn’t really want to get into politics, but they want to know how to run something big, and so this provides that type of opportunity. The second thing is that the calendar just works. So I think it can be really hard to get students excited on a project if they can’t actually take it through the finish line. And what works about a non-partisan voter registration and voter mobilization drive is that in most states, the voter registration deadline is around four to six weeks after school starts, and then you get about another four to six weeks before the election itself, and then you’ve still got another four to six weeks before the end of the semester. And so it perfectly stages itself, provided that the faculty or some other set of students have done some of the set-up, for students to come in learn a set of skills, build skills, execute, get a couple of do-overs, and then still have time to reflect on the project before the semester is out.

Rebecca: I think that’s one thing that’s really unique about the timeline, is that a lot of kind of activity-based learning or community based learning projects, they go straight to the end of this semester and it really is hard to build in that reflection piece, so it’s nice to have substantial time to do that, and really think through that, and do post mortems and plan for the next time around so that the next set of students can learn from the previous set.

Allison: Yeah, it worked really well. I think that space allowed for a couple of assignments, both in terms of a post-mortem and having them really think critically about what they would have done differently in what advice they want to give the next group, but also for those students who want to go into this type of work, a lot of it is contract consulting work. So you’d run a campaign, and then that campaigns over, and then, what do you do next? And so one of the assignments for the class was actually to apply… mock apply for many of them. Though, a few students who are graduating did really apply for different types of political jobs. And so actually learning how to translate this real experience into a cover letter, and into a resume, and being able to pitch that what they had done was not just work for a class, but was actually work for a campaign.

John: Excellent. Did any of them end up working on campaigns?

Allison: A number of them have had internships. Someone received an internship, I believe, in Senator Schumer’s office off of the experience in her application, for that was actually what she submitted for the final project in that class.

John: Excellent. How did students, in general, respond to it? What sort of feedback have you had from students?

Allison: From the student population on campus or from…?

John: Or… well, actually from both within the class and also more broadly.

Allison: So students within the class thought that it was an immense amount of work, but also seemed very satisfied with the experience themselves. The sort of anonymous feedback sheets that I did with students over the course of the semester, students repeatedly talked about how much they were getting out of the experience in terms of learning what went on, quote unquote, behind the scenes of campaigns and how much harder it is then it looks like it is on television, comments like that. For this student population more broadly, it’s been interesting. There were definitely a set of students for whom having the voter registration and voter mobilization drive become something bigger on campus. I think it felt a little bit intrusive, though I’d argue that that’s what grassroots campaigning look like, you’re just gonna get asked if you’re registered to vote four times a day, in the days leading up to the voter registration deadline, and not for… even the students in my class who said, “I think we’re bothering people.” I said, “you are bothering people, you want them to register to vote.” So there was a little bit of that. On the other hand, students were really excited and I’ve actually had a number of students ask me if I’m running the class again, when the class is running again. Sort of having seen it happen, are really interested in getting that experience.

John: Very good. If someone were to stop in on your classroom, what would it look like?

Allison: I suspect it would initially look like chaos. [laughter]
The campaign classroom, I think, is a very different feel than a lot of other classrooms. After the first couple of weeks, I basically demoted myself to note-taker. I was technically the campaign manager, but I was really there to act as a check if I thought they were straying into something that potentially– this never happened, but I essentially was there to see do we stray into something that potentially smacks of a real problem, right? …in terms of their regularly… like election law regulations or guidelines for the campus, keeping track of the money that we still had, and what we could spend money on in the overall campaign calendar. But I would most frequently in that classroom, whoever was in charge of running a particular campaign team that was working on a strategy, would be running the meeting and I’d be at the front of the room essentially taking notes on the giant whiteboard in order to track the conversation and basically remind people of what decisions needed to be made before we left that campaign meeting. There’d also be a number of classes where you would have come to the classroom and no one would have been in it because there were either students out phone-banking, students were running a voter education program in one of our dorms, students were out running a training for other volunteers… sort of really being out in the field as much as possible and I was just running around trying to see what was happening in all of those locations and troubleshooting when it was needed.

John: So how many volunteers did they bring in from outside their class?

Allison: We ended up having over 250 unique volunteers from outside of the class that did work with Vote Oswego.

John: That’s impressive.

Rebecca: So you mentioned money and finances and so I think, I think that’s usually a big question for any sort of community project, campus project, etc. So how did you work the money side of things?

Allison: I think, one of the real benefits of the voter registration drive, is not only does it match the calendar, but it matches a place where campuses are already inclined to spend some money. So, I was able to put together money from a couple different places. One was actually from our Student Association. The student government here at SUNY Oswego put in it, ended up being close to $2,000, ultimately, that helped cover– it initially helped cover a bulk of the t-shirts actually, in visibility materials. I also put together money from our Community Services office, which is who had been coordinating the voter registration drive before. So instead of the money that they had spent running their own project, they were willing to put it towards this project. Which again, giving students access over how those dollars are spent I think was really important. I also was able to get resources from a couple different places as a faculty member, so I pitched Vote Oswego originally at a faculty academic affairs retreat at the start of a year and received $1000. The idea was voted as a best new innovation for teaching and learning at SUNY Oswego’s campus. And then I also received a Curriculum Innovation Grant here at SUNY Oswego that helped cover for me traveling to grassroots organizing training with the new voters project, to essentially get a refresher. It had been… let’s just say I was not text messaging…. that did not exist when I was last organizing… so getting a nice refresher on what sort of the the modern techniques were and best practices was really helpful.

Rebecca: How can others get involved like this particular project on this campus or run similar projects?

Allison: Yeah, so on this campus, faculty or students or staff that are interested should just shoot me an email. Definitely trying as soon as possible to start ramping up the plans for the 2018 midterm version and really starting to lay the groundwork for something big in 2020. Folks on other campuses that are interested in figuring out how this project worked, I actually just had a co-authored piece come out in the Journal of Political Science Education, it’s available as of yesterday online entitled “Vote Oswego: Developing and Assessing the Campaign-as-Course Model” that does quite a bit to outline how this project can run, where it fits pedagogically in sort of that space of taking some of the best parts of both simulations and service-learning. That article actually includes quite a bit from the course calendar, assessment strategies, as well as some student outcomes. And I want to point out that that piece was co-authored with Angela Tylock, who was one of the lead interns for the project. She graduated from SUNY Oswego in Spring of 2017.

John: Very good. We’ll include a link to that reference in the show notes.
What specific guidance might you give to other campuses trying to do similar projects?

Allison: I would just really encourage faculty and campuses generally, even if you don’t want to run it as a credit bearing course, to figure out how students can take the lead as organizers. I think, too often, students become the volunteers, right? There’s sort of a whole apparatus with lots of different nonprofits that are doing really good work to get students to vote and that’s really important, but I think on campuses, we’re missing really big opportunities if we treat elections as an opportunity to get students to vote, but not as an opportunity to get students the skills that they are gonna want and need for a whole variety of things. If you want to go work in a non-profit, you’ve got to know how to build a coalition. If you want to work for your kids’ PTA and make sure that they’re getting the resources they need, the ability to run a meeting and get petition signatures, is actually really important. And all of those types of civic skills are things that students can and, I think, should be getting by volunteering or helping to run one of these drives.

John: So it’s very much an active learning exercise…

Allison: Absolutely, absolutely.

John: …where students played an important role in building it.

Rebecca: How did you get students to take that active role? I mean, it’s easy to assign tasks and be the leader, but how did that feel?

Allison: There was… definitely, I had to be fine with a level of loss of control that I am often not fine with in my classes. What I did is actually work to set up the first two weeks of the semester, I had the calendar planned out, so I had worked ahead of time to set up tables and events that were happening for orientation, had coordinated with faculty around campus to have individuals come in and give announcements and register students in that first 10 minutes that really, the first week of school, you can almost always, the first day of class, give up 10 minutes, after you review the syllabus, to get some students registered to vote. After that first two weeks, the students who were enrolled in the course, had had an opportunity to be trained in those skills, to get their feet wet in the skills, to get feedback on the skills and then I didn’t plan anything else. I basically said, “now it’s up to you, what are we gonna do?” And goals have been set for the campaign so the students knew, “here’s where we want to get, here are what we think our rates are gonna be. So if we want to register 500 people from tabling, here’s how many table hours we need scheduled, how are we gonna make that happen?” And I think two things then happened. One, I stepped back and basically told students, “you’re the expert on where students on this campus are.” I come here, I go to work, sometimes I go to events, and then I go home. I don’t actually know what dining halls are packed on what days. Turns out chicken sandwiches, big deal, chicken sandwich day at late night, right? There are all of these things that happen on campus that, as a faculty member, I don’t know about. So students basically learned that I wasn’t gonna tell them not to register students at 11 p.m. at night if that’s where they thought students were, they ran with it. The other thing that happened is they realized that I would let things fail. If students scheduled events and those events went poorly, they went poorly. And I wasn’t gonna fix those events for them– with the exception of confirming that registration forms were filled out correctly. We had an entire process for making sure that voter registration forms were correctly done. But in terms of the grassroots apparatus around that, if students didn’t plan well, they didn’t plan well, and they were the ones that had to stand there while the event went poorly. And I think between those two things, the students really became engaged around sort of their responsibility and taking ownership over the campaign.

Rebecca: Was most of the learning then taking place by “let’s try this, let’s fail, let’s try again,” an iteration rather than like doing readings or other kinds of … ?

Allison: Yes, there were minimal readings while the campaign was actually happening. There was quite a bit of reading and reflecting once election day happened, but prior to that, it was much more “these are the tried-and-true tactics, what do you want to get out and do? How do you think these tactics will best adjust to the environment that you’re in and the student population were working with?”

John: And students learn a lot more by making mistakes and recovering from them, and it sounds like you set up a mechanism where there was lots of feedback from each other.

Allison: Yes.

John: That’s excellent.

Rebecca: So, usually we wrap up these conversations by asking what are you gonna do next?

Allison: Next for Vote Oswego is an effort to improve the connections between the voter registration drive as a grassroots campaign to the voter registration drive as an overall campaign that involves lots of different components. So actually, Rebecca, had a class that worked on the website for Vote Oswego, it was a project for one of her classes we’re hoping to do, I think, much more of that for the 2016 campaign as well as trying to figure out what other faculty or other classes could also use a voter registration drive… benefit from that timing… benefit from the fact that it can be student driven and student owned in a lot of ways, to really get their classes involved with this as a project.

John: Very good.

Rebecca: Thanks for taking the time out and sharing your project. I think it probably has encouraged a lot of people to start thinking about those midterm elections and how they might be able to get students tapped into it.

John: And will you be doing this every other year now?

Allison: The goal is to do it every other year. I haven’t done it for a midterm yet, I think that will be different. I think there will be more actual campaign literature there just because it will be difficult to get it sort of as ramped up as a presidential election, but the goal is to do it every other year.

John: Very good. Okay, well thank you.

Allison: Thank you.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

13. Authentic Learning

Posted in active learning, authentic learning, collaboration, higher education, learning, project based learning
January 24, 2018

In this episode, Rebecca Mushtare discusses how she has used community-based learning and simulation projects to provide authentic learning experiences in her design courses.

Show Notes

Transcript

John: Today, our guest is… Wait, there’s no one in the guest chair. Who’s our guest today?

Rebecca: It’s me! It’s me!

John: Oh, yeah. Okay. Today, our guest is Rebecca Mushtare, who will be talking about how she uses authentic learning techniques in some of her classes.
Today, our teas are:

Rebecca: Comfort and Joy.

John: Peppermint Bark. So we’ve got some holiday tea left over from the holidays. So when people talk about authentic learning, what do they mean?

Rebecca: It really means something, it’s like a real world problem of some sort, or where students are gaining experience as a professional or in something that’s very similar to a professional. A lot of times, authentic learning experiences include ill-structured problems. So not like the kind of question-and-answer things that we might have in a very structured classroom context, but where it gets messy. There’s variables that we can’t necessarily plan for in advance. That often happens and then a lot of times they’re also project-based exercises or experiments too.

John: So, one of the main reasons for using these authentic learning exercises, besides providing students with training that’s relevant for their field, it also provides them with learning experiences where there is quite a bit of intrinsic motivation, right?

Rebecca: Yeah, I think that students respond really positively to authentic learning experiences, because they can see how it’s relevant to them, and relevant to their professional careers. So even when I do small exercises in class, like writing an email as a professional, students latch on to that writing opportunity more so than other kinds of writing opportunities, because they understand that that’s important, relevant and necessary.

John: What types of activities have you used in your classes?

Rebecca: Well, at first it’s probably important to understand what kinds of classes I teach, to kind of get some context.

John: So, what types of classes have you used these in?

Rebecca: Yeah, so I predominantly teach studio- based classes, mostly web design courses. So, I’ll focus on those, because those are the ones of my regular load. So those are the things that I teach most frequently and I’ve done the most experimentation in. I do community-based learning or a form of service-learning, and I also do simulations, and it depends on whether it’s a beginning or an advanced class, which one I do. Community-based learning, or these service-learning, opportunities are generally working with a community client… generally a nonprofit organization, who doesn’t have the capacity or the budget to hire a professional design agency to do something. So we’re providing a service in a way that builds their capacity. In my advanced classes, I’ve done a lot of community-based learning. Locally, we’ve done the Children’s Museum of Oswego website, the Childrens’ Board of Oswego website and students are also wrapping up a project for the Oswego County Airport. So all of these are possibilities where they get to design a real website… they work as a team and I serve as the creative director, so this is different than an internship or other opportunity where they might get real-world experience because they’re getting a lot of coaching throughout the entire process, that they may or may not actually get in some of those other contexts like a volunteer or an intern.

John: When you serve as a creative director… could you provide a little bit more detail on that role for those of us who don’t work in those areas?

Rebecca: Yeah, it’s a step up from maybe a project manager and that I oversee all of the creative decisions, including what research methods are gonna be used to learn more about the audience, and the client… making sure that whatever meetings and things we have are all scheduled through me, so that I’m highly informed and participating in the project. It’s not something that students are doing without me being involved. It’s not an “outside-of-class assignment,” where they’re just doing a group project and they do whatever with the community and then show up and it gets done. Rather I’m heavily engaging with that community partner as well, and this is really important because of the longevity of the project. At some point the students are gonna go away. The semester ends… and the project may or may not be done. I either need to have another class that finishes up the project… I might finish it up…. or whatever needs to be done, I need to make sure that that continues, because the timeline of a client or community organization is quite different from our semester schedules. My involvement is really important in that role.

John: …and that helps with buy-in from the community partners, it provides assurance that the tests will actually be completed…

Rebecca: Right, and to actually to be able to do a project like this in a semester requires some significant planning on my part with a community partner in advance of the semester. So, I have to really understand their needs ahead of time to make sure that they are not far beyond what my students are capable of with my help. I also have to make sure that they know, as the community partner, what they’re gonna need to have ready so that the students can actually get to the part that they need to do. With web design something that most people don’t realize is that there’s a lot of writing content… and designers don’t write the content. The community partner or the client does. They need some coaching through that, and so I help facilitate some of that. Some of my scholarship as a professional is in that area where I’m working with these community partners, as a professional and as a consultant.

John: How much of the interaction with a partner is done by you, and how much is done by the students?

Rebecca: It’s a little both. At the beginning, well before the semester starts, I’m the one that makes contact with the community partner. We figure out how the semester is gonna be organized, establish roles and responsibilities, usually some sort of agreement. I usually make sure we negotiate some sort of copyright agreement that favor students using stuff in their portfolios, and set all those things up upfront, then we usually set a project launch date, and the client will come to campus. I make sure they’re available during my class time, and that they’re available pretty regularly through the semester and can come in a week’s notice, so they block out that time slot, so that they can come. So they come… I help the students prepare for that meeting. They ask questions… Q&A… so that the students learn about the community organization and what needs to happen. If it’s possible we usually schedule a trip to the community organizations, so we can see firsthand what they do, and so the students are interacting directly with a client in those circumstances… and then it depends what else needs to be done. So, for example, with the Oswego County Airport project that we’re finishing up, some of the students did some photography and things on the premise so they coordinated directly with a client to make arrangements for what time and that kind of thing. So sometimes it’s easier for them to do that communication, but largely if it’s about approvals and things like that, that all goes through me, which is in keeping with how it would be in a professional environment, where the creative director or an art director or someone above entry-level designers would be the ones having that contact.

John: From the students perspective, what are some of the benefits of this sort of project?

Rebecca: They’re really excited because they end up having portfolio work, which is important. They can put a line on the resume, essentially saying that they worked on a real project, that’s really being used, and then they also get to see an entire project all the way through. So in these cases, what we’re doing… community based learning or these community projects…they are able to participate in the research, development, design, the whole shebang, but usually they pick one rule that they do in depth which is something different than I would be able to do in other contexts. For example, someone might be the developer, or one of many developers, or someone might be a researcher primarily… even though they’re working on all the different parts of the project. So, they like the fact that they can do some work in depth. Usually in these classes I’m doing two big projects. So they’re doing this one and then they’re doing some sort of other individual project that complements it in some way.

John: So how much of this is done with teams of students working on the project and and how much of it is done by individual students working on individual components?

Rebecca: Well, the whole thing is usually a whole class project, which means that I really need to make sure that all the moving parts are working together and coordinating and what-have-you. We use Slack which is a team chat that we use outside of class to keep in contact about different things… and this last project we did something called “playbacks.” So, one day a week we did little playbacks about what everybody was doing and what they’re up to and what they needed from other individuals to keep the lines of communication open… and then certain roles and things are maybe small groups that need to work together to get particular pieces done.

John: You mentioned the portfolio piece for students. If they’re part of this big group, how do they identify the components that they worked on?

Rebecca: Yeah, we talked a lot about portfolio documentation, because working in a team is pretty standard protocol in the field that I’m in. What students do is they document the entire project, but they specify in that documentation what their role was… and so they always credit all the other people that worked on the project.

John: Excellent. What are some of the challenges that you face in working on an authentic learning project? ….with standard projects where you have a very finite well-structured problem, it’s fairly easy….well, at least you control the environment much more. When you’re working with someone in the community and you’re working with real-world development, what are some of the challenges unique to that type of framework?

Rebecca: Yeah, there’s many… [Laughter] One of the key issues is timeline. The timelines never match up, and so you always need to have a back-up plan for how something is gonna get finished… because it’s almost never totally finished during this semester. So, sometimes that means some people in the class are doing an independent study to finish stuff up…sometimes it means I’m gonna do something… sometimes it means another class is gonna pick up the pieces… or whatever… but that that needs to be in place, and that needs to be in place from the beginning. It’s really important for it to be in a learning environment that students can fail safely. They need to be able to screw up and that be okay.

John: It’s certainly safer for them to do that on this project than on their first job.

Rebecca: Right, exactly… and so you know part of my negotiations at the beginning of a project like this with a client is letting them clearly understand that this is a learning experience and learning comes first from my perspective, but that their needs will be met, but it might be met on a longer timeline than they really want.

John: …or perhaps a more iterative journey than they expected.

Rebecca: Yeah, exactly… and in most cases the community partner is more than happy to participate and especially because I always see the community partner as a co-teacher… they’re there to teach certain lessons, too – and that might mean letting us know that a student’s gonna fall flat… and letting them do that… and then help them figure out how to do it better next time… and give them that next time as part of the project. So, there’s been many times where some someone maybe provided a deliverable that wasn’t quite up to snuff and then deadlines had to shift so that that person could revise and meet the standard that needed to be met. So the students are generally working at a much higher level because essentially they can’t really fail. They can fail and revise… and revise… and revise, but eventually they get to a minimal standard… which I find to be helpful… and then the other thing is you really have to be flexible. All kinds of things happen… an organization’s budget can totally become a disaster and they have to refocus their attention on something else… and so you might feel abandoned.

John: While it’s the main focus of your class…

Rebecca: Yeah…

John: …it may not be the organization’s main focus.

Rebecca: Right. …like any of these things can happen without you expecting it even if you have you think all you have you have all your t’s crossed and i’s dotted at the beginning of the agreement. So that happens. Sometimes, students just don’t follow through in the way that you think that they’re going to or it or that you know they can… and so like what do you do in those situations? You kind of have to have those kind of failsafes in place. This is one of the reasons why, to provide an authentic learning experience for beginning students, I moved away from community-based learning. I used to do community-based learning in my beginning class. I do it now, but in a very different way than doing an actual website project, because there’s too much at stake there. So, I say that I save those experiences for my advanced students.

John: Going back to that… in your beginning classes what do you do differently to create the same sort of environment, but perhaps with a little less risk?

Rebecca: Yeah, I do two things. One that is a community-based project… and that’s what I call a consultation report. What they end up doing, in that respect, is, instead of doing a full design project for somebody, they do some of the research and analysis and do some proposals… some ideas… that we then hand over to the client that they can then use to either hire my advanced class or to hire a designer to take on but they understand more where they’re situated and so as part of that we do some accessibility testing… we do user testing… and things like that…. and so we’ve done that for a couple of different organizations, and that’s worked out pretty well. That gives students an opportunity to communicate with a client a bit and also do some formal presentations, which is nice…. and then the one that I use probably more frequently in my beginning class is a simulated client project. I have established a few scenarios that our clients… they have specific goals and needs… they have personas…. they have email addresses, etc…. and then students will work in small groups and then they communicate directly with the client all through written communication, although they can schedule an appointment…. I do have heads on popsicle sticks in which case they can meet with their puppets…. [laughter] which is always surprising to them because I don’t tell them upfront that I do that. So they come to my office and my door is always shut… for that situation I’ve reorganized my office. They knock on the door and there’s a head on a stick… and if they laugh I shut the door… and they have to start over. They have to take it seriously.
[Laughter]

John: So, for the artificial clients, you create the email addresses and it will go to you?

Rebecca: Yup.

John: …and then you will respond as if in the role of the client.

Rebecca: Yeah, they each have a personality. So there’s four or five different clients. They all have very different personalities… and students start talking about their clients and the different kinds of ways that they behave. They have certain ways that they open and close their emails. One’s very curt and aggressive. One is very grandmotherly… very caring and kind.

John: Do you ever get them mixed up?

Rebecca: I have little notes when I start doing it that I keep on my laptop… a sticky note that just reminds me… a couple key words like who is who, so I don’t get confused.

John: Yes, that could cause some problems if you went from the very curt person to the grandmotherly person…

Rebecca: Yeah, and then if a student emails their client and they’re out of bounds or something then I email back as the professor from my school email address… and it says “This is a note from your professor” and then I indicate what’s wrong… and I make them redo it.

John: So, how do they react to the puppet?
[Laughter]

Rebecca: They’re usually surprised but then they find it amusing… and they take it seriously… especially if I shut the door on them ‘cause they laughed at me… and they started over and I keep a straight face and whatever ‘cause you just know you never know who you’re gonna interact… and so the first time you meet someone you could be surprised, right?

John: It could be someone who’s a puppet.

Rebecca: It could be a puppet… you just never know… so, yeah, they generally respond pretty well to that… and usually if they meet with me in person as the client, then after that meeting I make them stay for a couple minutes and we just talk about how it went and things that they could have done differently.

John: Excellent. In an earlier podcast interview with Stephanie Pritchard, we talked about the Voices of Oswego Veterans project and that also seems to fit in as another type of authentic learning experience. Could you just recall that for people who may not have yet listened to that earlier podcast?

Rebecca: Sure. That project, in particular, The Voice of Oswego Veterans, was a collaboration between Stephanie Pritchard’s writing class, Peter Cardone’s photography class, two of Kelli DiRisio’s design classes, and my web design class. So, instead of doing my standard simulated client project with my beginning students, that group did the Voices of Oswego Veterans website. So that was somewhere between a simulation and a client because they didn’t have a direct client to talk to, but it was a real project and they had real content and real goals that they needed to meet… and that was taken really seriously by students and I think that was in part because it was going to be published. So, they didn’t get as much of the client interaction, which I think a lot of times the students value a lot from my classes, but it was still a very authentic experience and the students got a lot out of it and they were really committed to the goal of the project which was to dispel stereotypes about veterans. There’s a lot of assumptions that we identified early in the project… that people assumed that veterans are old… they associate it with World War II, and to think that “oh, wait, we have students on campus who are veterans, that just boggled some of their minds and we wanted to make sure that those students are seen as students as well.

John: How have students responded in general to the project?

Rebecca: I think, in general, students respond to any of these authentic learning experiences fairly positively. I think they all think it’s a lot of work, especially because the revision is taken a lot more seriously… and you think that that maybe wouldn’t be true of the simulation, but they get into it and they continue to revise and they want to meet and satisfy the client….that’s the goal at the end of the day. They need the thumbs-up from the client at the end… and so I think that is motivating and it seems realistic enough that they want to give it their all…. and that definitely is true on community projects. The one that we’re finishing up now, I have a student who graduated who’s finishing up a couple things that she couldn’t quite get to work the way she wanted to and she’s finishing that up right now

John: Okay, so I guess the next question is: “What are you going to do next?”

Rebecca: That’s a good question…. [Laughter] I should have known that was coming.

John: You usually ask that question.

Rebecca: Yeah, I know, right? So, I guess it’s only fair that it’s asked of me.
So, the next thing that I’m planning to do related to authentic learning is to emphasize thinking about audience empathy and stereotypes a little bit more. That Voices of Oswego Veterans project, I think, was particularly successful in helping students actively design to dispel certain stereotypes and I’ve really been trying to get students to think about audiences who are different from themselves… which is a challenge…. and that seem to work really well, so I’m trying to find a way to embed that more so in both my beginning and advanced classes.

John: Excellent. Well, thank you. This was an interesting discussion.

Rebecca: Thanks, John.

John: Looking forward to hearing more about it as the next semester progresses.

12. The Active Learning Initiative at Cornell

Posted in active learning, collaboration, higher education, learning, pedagogy, teaching
January 17, 2018

In this episode, we discuss Cornell’s Active Learning Initiative with Doug McKee, an economist at Cornell and a co-host of the Teach Better podcast. This initiative, designed to increase the use of active learning in instruction at Cornell, provides funding to departments to redesign courses to employ evidence-based active learning techniques. Doug provides an overview of the program and a discussion of how this program is being implemented to transform economics classes.

Show Notes

John: Our guest today is Doug McKee. He will be discussing the Active Learning Initiative at Cornell. Doug is an economist at Cornell and also a co-host of the Teach Better Podcast, which is one of our favorite podcasts on teaching and learning.

Rebecca: Today our teas are:

John: Ginger peach white tea.

Doug: I’ll be honest. I can’t stand tea.

[LAUGHTER]

Rebecca: Absurd.

[LAUGHTER]

Doug: It’s not that I don’t need the caffeine, but my preferred method of caffeine injection is coffee.

Rebecca: Yeah, that’s a bummer. It’s not “The coffee podcast.” There’s lots of coffee here..

Doug: Maybe next week I can be on the coffee for teaching podcast.

[LAUGHTER]

John: Or we could have a special episode.

Rebecca: Yeah.

Doug: Oh… a special coffee episode.

Rebecca: That would be kind of fun.

Doug: You’d need guest hosts.

Rebecca: Yeah.

John: The Starbucks next door is closed over the break, so we’re…

Rebecca: Anyways… I’m drinking a golden English breakfast.

Doug: Do you change the teas every single week?

Rebecca: We try to.

John: We generally do, but we got a couple hundred of them….

Doug: But, don’t you just have a few that you like?

Rebecca: I have a favorite, yeah.

Doug: Don’t you just want to drink that one all the time?

Rebecca: Sometimes.

John: Sometimes we do. I’ve used the same ones occasionally.The last couple of episodes, I had the ginger peach black tea….

Rebecca: …that’s his favorite.
JOHNL… and the ginger peach green tea… and this is a ginger peach white tea, which I drink later in the day because it has a little less caffeine.

Doug: uh-huh…

Rebecca: I like the English afternoon tea.

Doug: I would think in the afternoon you’d want the one with more caffeine.

John: I’m generally better in the afternoons and evenings. I have more trouble getting energetic in the morning.

Rebecca: John loves 8 a.m. meetings.

John: Could you describe the Active Learning Initiative at Cornell? How did it originate and how is it working?

Doug: The first thing I want to say is I’m a relative newcomer to the Active Learning Initiative. I joined Cornell about a year and a half ago, in large part because I was excited about joining the Active Learning Initiative. But it’s a project or really a program that started back in 2013. It was the brainchild of Peter LePage, who was actually a guest on episode 50 of the Teach Better Podcast and he is a former Dean of the College of Arts and Sciences. He’s a physicist and he had read a fair amount of the literature… the big literature on active learning in physics… and wanted to come up with a program that would actually instill this way of teaching across the college. He also happened to be a friend of Carl Wieman, and so he knew all about the Science Education Initiative as it was happening… and the Active Learning Initiative is a program modeled on the same principles as the Science Education Initiative… and the ideas that departments compete for grants, and once they get that money, they use it to hire people with disciplinary knowledge (usually postdocs into their own Department) who can co-teach with faculty and train faculty in how to use these methods. Because what I find, is, when you ask faculty why they don’t teach actively or why they depend on the pure lecture, even though there’s a fair amount of evidence that active learning works better on a variety of dimensions, “we don’t have the time”… well, the department education specialist or the postdoc can help with that…. they say “we don’t know how”…. and so the department education specialist comes in with that knowledge of how to develop clicker questions and what a good small group activity looks like, and then finally when they say ”well, I don’t believe it,” it’s the department education specialist can both reference literature… talk in the language of the the discipline… and…. and this is a big part of both the Science Education Initiative and the Active Learning Initiative, creates knowledge and actually try things in that discipline and evaluate what the effects are on student learning of teaching in new ways…
JOHN…and the STEM fields have been a bit of a leader in that. Physics, in particular, has been very active in development of this.

Doug: That’s right. So, the first round of the the grants funded by the Active Learning Initiative at Cornell were given to the Physics department and the Biology… actually it was a co-proposal by their two biology departments. Why? I don’t understand why universities can’t have one biology department anymore? They don’t.

John: Out of curiosity, what are the two areas?

Doug: Ecology and evolutionary biology… and then there’s the, I think, molecular biology and like these kinds of biologists…. I don’t know… I’m not a biologist, but biologists have a hard time talking to each other, it turns out. But these biologists could talk to each other even though they’re in different departments, and…

John: … maybe because they were in different departments it was easier.

Doug: …it was easier… You’re right, it was easier…

[LAUGHTER] as long as they just had to talk about teaching and not about who the appropriate person to hire was… and so those were the first two departments… and then since then, in 2017 there was another round of grants granted and one of those was economics and the idea there was to branch out beyond the sciences and the STEM fields into the social sciences and even the humanities.

John:… and this was funded by a donation from alumni, right?

Doug: So, the funding comes from an alumnus who had a passion for active learning. I think what happened was he and Peter were at a dinner… and Peter was talking about how excited he was about changing how he taught his classes, and how he wished that it would happen more often, and this person came up to him and said “I heard what you were talking about and I’d be really interested in funding that.”

Rebecca: That’s exciting.

Doug: On the other hand, the Science Education Initiative, as far as I know, was funded by internal money. So it was Carl Wieman was very persuasive in convincing the administration at the University of Colorado and the University of British Columbia, and now he’s at Stanford, and he has internal money at Stanford, to do a lot of the the same thing.

Rebecca: Funding is great to support some of these things, but I would say that our Writing Fellows program on our campus is not that different from hiring these postdocs, because we have writing fellows that are assigned to each of our schools. We have a previous episode on that topic with Stephanie Pritchard, and we have writing experts who meet with faculty in different departments to help them develop writing assignments to meet some of our standards for writing across the curriculum, and help faculty understand how to teach writing….

Doug: Right.

Rebecca: … in their disciplines. So, in some ways, it’s a very similar sort of model. It’s not about active learning, but it’s about how to do this writing infusion.

Doug: So in many ways, it does sound similar, but I’m gonna point out some of the differences.

Rebecca: Great!

Doug: These postdocs live in the department, and they have PhDs in that discipline, and so in some ways they get way more cred that way… and respect from the faculty, because they’re thought of as part of the department. Another difference is that the departments have to write proposals, so they’ve committed and they’ve said in the proposal how excited they are about this, whether they are or are not.

John: They’ve at least made a commitment.

Doug: They’ve at least made a commitment and said… stated on paper that they’re excited and they want this to happen and this should be a success. It’s not some external entity, and I think that makes a difference.

Rebecca: How long do the postdocs stay in the department?

Doug: So, it varies from department to department, but in our proposal we felt strongly that in economics the postdocs are generally two years, and so our postdocs are 2-year postdocs. But in physics, I think, the postdocs tend to be longer, and so their postdocs, I think, are three-year postdocs. One year is just not enough, because they arrive and they immediately have to start preparing to go back out on the job market, and try to get jobs elsewhere.

John: But I think you had mentioned that while they were there for two years, there were going to be a number of postdocs that were staggered over a four-year period.

Doug: Right. So our project is a five-year project and what we’re doing is we’re hiring four two-year postdocs starting, with one in the first year and then every year after that, and so during the middle three years of the program we’ll have two in residence and then in the first and the last we’ll have one.

John: You mentioned the Economics Department as one of the recipients of this. How many other departments were there?

Doug: Five. So the physics department has another grant to overhaul how they teach their lab courses, and this is actually… Natasha Holmes, I think, is arguably the world’s expert on really modernizing and changing how we teach lab courses; taking them away from following recipes to creating the recipes and answering interesting questions using experiments, and creating those experiments, and she recently joined the Cornell faculty. The music department is using a grant from the Active Learning Initiative to integrate active learning into a composition course where students all have keyboards in the classroom and the keyboards act a little bit like music clickers, so the…

John: with MIDI controllers, probably?

Doug: Right… and the instructor can then listen and select different things that different students have played and then play them for the whole class. It looks fantastic. Sociology is doing fairly straightforward integration of active learning into their large lecture classes, using group activities and clickers. They’re also standardizing, and making much more active, the discussion sections for those classes. The math department is overhauling their introductory calculus courses. Let’s see, that’s math, physics, economics, sociology, music, and then the sixth is classics…. and Classics is creating a brand new course that’s not a pure lecture course, that has students, I think, doing projects during the semester.

John: …and your department is doing how many classes?

Doug: We are treating, or transforming, eight classes?

John: Which classes are you doing?

Doug: So, we’re basically doing the entire core curriculum, or the required courses for the major. It’s not exactly that right now. We’re really doing seven plus a popular elective course called behavioral economics… and why aren’t we doing all eight of our required classes? Well, one of them is a class that we don’t actually own, but we’re hoping to somehow, over the next five years, find the money where we can treat that class too, because it would be a shame to just do seven out of eight.

Rebecca: How are your faculty in your department responding to the initiative and getting involved?

Doug: So we have a small core that’s very excited and they’ll be involved in the program early on in the first couple years. Then we have another set of faculty that are very excited in theory, as long as it’s far enough in the future…. And I think we have faculty that are not that excited, but aren’t actually being affected by it, and so they’ve been fairly passive. I’ve been thrilled not to have anyone in the department that’s actively opposed, and so it’s been great, actually, and the vast majority of the work happens when a course is actually being transformed. Right now we’re making it very easy on ourselves. The first course that we’re transforming is one of my own courses.

John: Which makes it easier.

Doug: Yeah.

Rebecca: Yeah.

Doug: So we can get the progress. We can just get the whole procedure right… get that stuff in shape as well as we can. So… okay, we’re gonna do a control course in the fall. We’re gonna do a treatment course in the spring. So we’re gonna teach it fairly plain vanilla in the fall, and we’re gonna treat it in the spring and we want really good measurements of learning. How do we go about creating those measurements of learning? Because unlike physics which has like 80 standard concept inventories, economics has one. It’s the Test of Understanding of College Economics, and it’s pretty good, but it’s only for principles courses, and so we have to build from scratch… and so next year we’ll actually be bringing more faculty into the fold, and we’ve scheduled it such that the fact that we bring in our faculty that are excited about the project.

Rebecca: What have you learned from the process so far?

Doug: Measurement is hard… and it’s not even that it’s just difficult… it’s a lot of work… and so we had a meeting with some of the folks in biology and where we talked about what we were doing and Ron Harris-Warrick said something. He said “It seems like you’re spending 90% of your time on measuring and assessing, whereas we spent 90% of our time actually changing the teaching” and I said “That’s exactly right” because we’re not changing the teaching yet and we want to know if, in fact, what we do when we change the teaching works …and so we need really good measures, and so that’s where all our investments been. Now, is that changing? Yes. In two weeks we start classes up again, and we start teaching the transformed class and so we have actually done a fair bit of work, and we have big picture. We know what we’re going to do, and we’re very excited about it, but we have a lot of work to do this spring on actually where we focus on changing the teaching.

Rebecca: So, we’re taking this week by week?

[LAUGHTER]

Doug: Well, it’s gonna be a lot of just-in-time curriculum development.

[LAUGHTER]

John: I’m very familiar with that process.

Doug: Right… right… right. I got to say, like after a semester of focusing assessment, which has been really fun, we’ve created what we think could be the start of two standard…. like two… we’ve tripled the number of standard assessments in the field in one semester. I mean… we have work to do still and we’re the only ones using it right now, but that’s changing. I am pretty excited to actually change how I teach this class and we have a lot of ideas about things we can do.

John: You’ve already been doing a lot of active learning in your classes before, so what are you going to be doing differently?

Doug: I divide it up into two chunks. So, we’ve got some high bang for the buck things… things like two-stage exams. One problem that we’ve had in the past, and I think it’s a very common problem, which is: you give an exam… the students take the exam… and then there’s a whole bunch they can learn after the exam about the mistakes that they made, and more than half my students don’t even come and pick up their exams. They see the number and then they… if it’s bad they get sad, and if that’s good they’re like I’m fine… and neither group is learning from their mistakes, and so what we’re doing is we’re saying you’re gonna take the exam… and then you’re gonna take it again in groups and then your grade for the exam will be a combination: 80 percent is your individual grade and 20% the group grade, and so at that point you’re actually discussing it. It forces students to actually discuss and talk about these problems in another step, and I think that’ll make a big difference.

John: You had a podcast episode on that not too long ago.

Doug: Yes, with Teddys Svonoros, where he’s been giving two-stage exams for quite a long time at the Kennedy School at Harvard.

John: So listen to that podcast. It’s a great way to learn more about that. We’ll put a link in the show notes.

Doug: It’s a great podcast.
Another high bang for the buck thing, we think, is assigning points in their grade to whether or not they participate in answering clicker questions in class. I have been, for my entire teaching career, adamantly opposed to giving points for class participation. I’ve always felt that it cheapens it, and I want students to be there and participate because it improves learning… not because they’re getting points toward their grade… and over the summer I was convinced that that is actually the wrong attitude. Students look at how you assign points for their final grade as a signal of how important things are.

Rebecca: …of what you value.

Doug: …of what I value…. and so if I say there’s no points for showing up at class and it’s not required, then they’ll be like: “Oh, he doesn’t think it’s important” and I can say it’s really important over and over and it doesn’t have nearly the…

John: Well, clicker questions offer some other things. It gives you some feedback too.

Doug: Right.

John: …in addition to other things, because if one person asks a question or answers a question or raises a point you know how that person responds or how they understand the concept.

Doug: Right.

John: But, with clickers you can get a feel for how all of your students are doing.

Doug: Right.

John: …and you can adjust what you’re doing in class to compensate for that somewhat. So there’s some good merits for clickers whether they are graded or not.

Doug: Right, but you’re not gonna get anything out of the clickers if the students aren’t in the classroom.

John: Right.

Doug: ….and so I assign points to get them into the classroom and I think that’s just gonna bring up a big chunk, because at the end of the semester you look at the grade distribution and you look to see who got the highest grades in the class… and guess what? You know all those people because they show up in class.

John: Yes.

Doug: ….and then you look at the bottom and your like, how did I miss these people? They did terribly the whole semester….

John: …but you didn’t see them as much.

Doug: I don’t even recognize them. I look at the pictures and they don’t even look familiar… and so pulling those people in. Those are a few of the little things… little in terms of effort… but we hope big in terms of impacts. But the class has clicker questions… the class already has… I like do short lectures… and they do problems to practice… and so we’re we’re hoping to really gain in the spring is by adding something called “invention activities” and so these are an idea that Dan Schwartz and his students have been writing papers about for a little while… and so probably the easiest thing to read if you’re interested in invention activities is the J chapter for “Just-in-Time Telling” in Dan Schwartz’s and co-authors book called The ABCs of How We Learn, which is an amazing book. I highly recommend it. If you read any book on teaching at all The ABCs of How We Learn is a great one in terms of telling you practical things you can do and giving you the evidence… and Dan Schwartz does great work… and so the idea is, in a nutshell, let students grapple with a problem before you actually teach them the solution… because it primes their brain… it sets up those knowledge structures that you can then hang the expert methods on.

John: So it activates prior knowledge….

Doug: Exactly.

John: …and gets them ready to form more complex….

Doug: It even create creates prior knowledge… and so he has these two papers. One is called “The Time for Telling” and the other is called “Inventing to Prepare for Future Learning” where he does these really amazing experiments. The one that I really like in “Time for Telling” is about casework. He has these three groups and one of the groups he gives a bunch of data on classic psychology experiments and just has them graph it and talk about it and try to figure out why the patterns are there and then he explains what we actually learn from these psychology experiments.. and then they play around with it and then he gives them a test… where they have to predict what the outcome would be of some new psychology experiments. That’s group A and they do great. But then group B… he skips that first stage and instead they read a chapter about what we can learn from these psychology experts and they summarize it… which doesn’t sound so bad… and then he gives the lecture and they do far worse… and so I’m really hoping that we can get a big bang from these priming activities and I think this class.. it’s a second semester class on statistical methods for economists, also known as econometrics… ‘cause god forbid economists use a word that other people already know…

[LAUGHTER] but there’s a whole bunch of methods there and so I look at that class as a really great match for this method…. where I can show them data and put them in this situation where a new method would be really valuable… and the old methods work poorly… and have them play around with it and then teach them the method.

John: That sounds really promising.

Doug: I think it’ll be super fun.

John: We just did a reading group here on Small Teaching and one of the chapters in that that people were pretty excited about was on prediction and it…

Doug: Exactly.

John: …summarized a lot of the research on that (which is another good reference in addition to what you just mentioned)

Doug: Right.

John: … to see a summary of the literature on that and it sounds like it’s quite effective in a wide variety of studies now.

Doug: I mean, I do think you have to be careful with students in explaining why you’re doing it upfront. I think it can be very frustrating to give students problems that you know they’re going to fail at. Students don’t like failing.

Rebecca: No.

John: But the research shows that when they have wrong predictions they actually have larger learning gains.

Doug: Right.

John: …as a result of the prediction activity.
DOUG. Right.

John: … and as long as you they know that, and as long as it’s low- or no-stakes in the prediction….

Doug: Right.

John: ….it shouldn’t harm them… but it doesn’t feel as good, so there’s a metacognitive issue there.

Doug: Well, I try to model making lots of mistakes in the classroom

John: I do that too, but I’ve never called it a strategy., but…

Doug: Exactly.

Rebecca: I think students respond really well to seeing their faculty be vulnerable and human…

Doug: Oh, I agree with that.

Rebecca: …and I know we all make mistakes and I think you’re right that students are… they’re afraid to fail…

Doug: Right.

Rebecca: …and they’re afraid to be wrong. So having an environment that’s setup that it feels safe to make those mistakes.

Doug: Right.

Rebecca: …and then get better… and the goal is to get better… really sets up an atmosphere that really supports learning because everyone feels safe about learning… and I think that that’s not always the way that students think about learning.

Doug: Not at all. Not at all. They think “if I get the answer right I’ve learned and if I get the answer wrong I haven’t” and it’s actually the opposite. If you get the answer right you haven’t learned it…like, you knew it.

Rebecca: Yeah.

Doug: Yeah, if you get the answer wrong… oh my god, there’s an opportunity to learn something…

John: …and it’s hard to convince some of that sometimes.

Doug: It is. It is. I tell them I want you to get things wrong, because if you just get everything right you’re wasting time…. then I’m wasting my time.

John: I tell them the same thing.

Rebecca: Yeah.
REBECA: So, can I circle back to something that we talked about earlier? I heard you use the word fun and assessment in the same sentence.

[LAUGHTER]

John: It’s not a common juxtaposition.

Rebecca: Exactly. I was hoping that you could talk a little bit more about what was fun about this development.

Doug: Oh my god, I’d love to.
Yes. So first of all, I admit that I like writing tests. I think it’s really fun to come up with new scenarios that get at concepts that I’ve taught before… and so I can take knowledge in some completely different area and then frame it. So I remember one time I had to give a makeup exam, but I didn’t really want to write a whole makeup exam and so all I did was change the wording… so a question about hospitals became a question about pet stores then a question about restaurants became a question about something completely different… but the methods were all identical and so that I think that’s kind of fun… but most people wouldn’t think that’s fun. So the part of the process that we use that I think more people would find fun in that was a key part and it was something that Carl Wieman and Wendy Adams call “think-alouds” where we draft the assessments and we bring a student in and we have them do it. So these assessments are 20 to 25 multiple-choice questions and if all you see are the sequence of multiple-choice answers, it’s hard to tell what’s going on in a student’s head… okay …and so what we do is we say sit down and we say vocalize everything you’re thinking as you take this…and we don’t say anything…. we don’t say no don’t explain what you’re doing, because that changes how you actually answer it. Just vocalize what you’re doing as if you’re taking it and the results are so eye opening. I learned so much about how students actually think about these things… the amount of just pure pattern matching, it would blow anybody’s mind. They say things like “You’re asking about a confidence interval… well, I remember that confidence intervals had something to do with standard errors, and so I’m gonna choose the answer that says something about a standard error.” It’s kind of frightening, to be honest… and then pretty often you ask a question where you say… you show them a picture, and then you ask a question that’s related to the picture… and they ignore the picture. They don’t even look at the picture. Yeah, ok, that’s not good… and so we did five of these during the fall with this the big assessment of learning that we wanted to use both in the fall and in the spring… so then the control course in the fall and the treatment course in the spring. We’re gonna compare results, and each time we did these “think-alouds” we would have like ten changes. We would add options because we’d find out that students…. it was really common for them to make this one kind of mistake that we hadn’t foreseen… and get an answer that wasn’t one of the choices. Sometimes they would pick the right answer for completely wrong reason like “oh well, it’s always the one that’s a yes.” Oh…well maybe we should make the right answer the one with the “no.”

John: So, to develop assessments or concept tests, it’s really helpful to know the common misperceptions so that you can break them down.

Doug: Absolutely.

Rebecca: What I’m hearing is what we call in design a user test.

Doug: Right.

Rebecca: So you’ve designed something, and now you’re testing it out on a test audience

Doug: Exactly.

Rebecca: …and making revisions.

Doug: …and no one does that.

Rebecca: Yeah, it’s kind of interesting, right? Like we don’t think of students as being users of a class, but essentially they are.

Doug: Right. I think when we teach, usually we have so little insight into what’s actually going on in students heads… and I think in a pure lecture it’s the extreme case. You get zero insight because you teach and they listen… and the only time you ever see what they’re learning is when you see their test results, or what they’ve written and so a big part of why clickers are useful is not only because it activates their brains and they’re practicing things… but the feedback it gives the faculty and so doing these think-alouds… it’s like an extreme version of insight into what students are actually thinking.

Rebecca: How do you recruit these students for the think-alouds?

Doug: What we did is we have the rosters from the previous year… students who had taken the class… and we invited a broad range, so we didn’t just get the A students… we got the A students… the A- students… the B students… even a couple C students, and then I think we paid them twenty dollars each… but you pay them a little bit and they show up.

Rebecca: That’s great.

Doug: Yeah… yeah, yeah, yeah.

Rebecca: Have you thought about using the think-aloud method at all as part of your class? I’m almost wondering if there’s a way to kind of integrate that more into, probably not a large class, but a smaller class.

Doug: That’s a really interesting idea. I think when you give students kind of meaty problems to do in class, and they’re working on them in small groups, you can go around and you can just listen. So there are these incredible transcripts of what students said to each other during the invention activities in the “Inventing to Prepare for Future Learning” paper. So the subjects were ninth grade statistics students, but it’s remarkable just how similar teaching is. There’s a lot more in common across grade levels than there is difference. I mean there are differences, third graders are not exactly the same. My classes look far more like a good third grade class then they do a pure lecture class.

Rebecca: it makes a lot of sense, right? I mean active learning works and they figured that out in elementary school.

Doug: Right, can we please not forget that?

[LAUGHTER]

John: Another way of getting the same sort of thing was suggested by Eric Mazur when he was here during a visit, and I’m sure he’s mentioned this in other places as well. When he develops clicker quizzes, he tries to aim for about half of the students getting them correct and the other half wrong, so that you’ve got a good base when he does the peer instruction component of it. What he does to develop the questions is pose a question and leave free response questions and he’ll just let students write their responses and then he’ll use that to pick the most common misconceptions that he’ll build into the clicker questions. Which gives you a little more scale, but not quite as much information as a think-aloud, perhaps.

Doug: …and the ideas that you’re iteratively refining your class semester over semester. I wish that that was a more common attitude toward teaching… but what I find is… I meet a lot of faculty that… they developed the class… they fix the things that are obviously broken… and then it’s done… and then they come in… they give their lecture… they walk out… and there’s very little investment after the fact.

John: We’re creatures of habit.

Doug: Right.

John: We tend to resist change. That’s one of the things behavioral economics tells us.

Doug: I was gonna say that classical economics tells us that people respond to incentives.

John: Right

Doug: If you’re above the bar… like you’ve responded to all the incentives… that are all the professional incentives that are there.

Rebecca: As a designer I just can’t help myself from redesigning and redesigning and redesigning… it’s never done.

[LAUGHTER]

John: I’m never satisfied with the way my course goes and I keep wanting to make it better.

Doug: Me too.

Rebecca: Yeah, but that’s why you two run a teaching center and have the teaching podcasts. You’re not the problem.

John: Well, my students might disagree at times… but that’s another issue.

[LAUGHTER]
So, are the other departments involved doing as much with the assessment component or is this something that you’re perhaps focusing on a bit more? or is it built into all of it?

Doug: No, I would say… I believe, no. It’s kind of a pet issue of mine… like I think assessment is super important. I think math is leaning on existing assessments… which is fine…

John: ….for departments or for majors where you have existing assessments that works well, but as you mentioned, we just have the micro and macro TUCE exam…

Doug: That’s all we have.

Rebecca: I mean, you have something.

John: We have something and we use it.

Rebecca: There’s nothing in my field.

Doug: So, in sociology there’s nothing also… and so sociology has actually been investing in assessments there. It’s tough… it’s tough. Sociologists are allergic to multiple-choice and so they will never use multiple choice… and I’d like to see a graphic design assessment that’s multiple choice.

Rebecca: I made one.

[LAUGHTER]
It’s hard, though.

Doug: So, Natasha Holmes and I have this ongoing back and forth where I say “Here’s something that’s tough to teach and evaluate with multiple choice” and she says “No, you could do it” and the line moves. So we agree that there are plenty of things that can’t… we agree on a lot that can …and we agree that fluency in playing the piano… you can’t… but where the line is in between, we go really back and forth… and I find that over time I believe more and more can.

Rebecca: I think there’s a lot of conceptual things that you can test that way.

Doug: Right.

Rebecca: …and measure certain kinds of understanding.

Doug: Right.

Rebecca: …but maybe not always… you can’t do a practical application with the multiple choice.

Doug: Right.

Rebecca: Like in graphic design, for example.

Doug: RIGHT. So, not everything, but a lot.

Rebecca: Yeah.

John: But you could devise, as we talked about in an earlier meeting today…. you could use rubrics for those areas.

Doug: Right… right.

John: We had a number of faculty a few years ago, who were new faculty in creative fields, and they were very concerned about how to evaluate creative work… so we put together a panel of people from art, from music, from screenwriting, a playwright, and from creative writing, and what was remarkable is they all said exactly the same things: that they use rubrics very heavily; that while people perceive this as creative work, there’s very specific things that they’re looking for in the writing; and as long as they develop good rubrics that capture what they’re looking, for it lets students know what they should be striving for what they view as important. Just as you said to give them points as students, matters but giving students rubrics helps them see what you think is important in their work… and it makes it easier for them to try to meet those standards.

Doug: Do you have a book that you recommend?

John: I don’t.

Doug: Well, wouldn’t it be great?

John: It would be. I’m not sure if there is one.

Rebecca: I don’t know of one, but I think it’s not a lot different. Revising and rewriting a rubric is no different than revising and writing multiple-choice questions

Doug: Right, right.

Rebecca: It takes time to refine that and get it to measure exactly what you want it to measure.

Doug: Right.

John: It just impressed me, though, that they all had exactly the same thing in fields that were so diverse.

Doug: Right.

Rebecca: Yeah, there’s still values to the discipline, right? There’s still things that we value. So, it might be innovation… so, maybe that’s an item on your rubric.

Doug: It’s pretty easy to get inter-rater reliability with a multiple-choice test. I think if the correct answer is A and you give a C, John and I are both gonna say it’s wrong; whereas, with a rubric it’s not obvious.

Rebecca: Right.

Doug: But I think… I mean… like so many things… like it’s not that rubrics are good and multiple choice exams are bad. There are plenty of really good multiple-choice assessments and bad ones and there’s a big difference between a bad rubric and a good rubric.

Rebecca: Um-hm

Doug: This is where people say “I tried active learning… it didn’t work …and so I don’t believe in active learning anymore.”

John: But by building the results and by working on the development of tests and assessments that could be used across time and across disciplines…

Doug: Right.

John: ….that makes it easier to build a case for the efficacy of active learning…

Doug: Right.

John: ….and that’s one of the reasons why I think why this has been so effective in so many of the STEM fields, particularly in physics.

Rebecca: I think another thing to think about is, if you’ve never done active learning as a teaching method before, then like you are a beginner, you’re not an expert…

Doug: Right.

Rebecca: …and so just like our students, the first time out of the gate….

Doug: Right.

Rebecca: ….we’re not going to be an expert, and it’s not going to be perfect, so we have to be vulnerable as learners as well… and so I think sometimes reminding folks that… wait a second, right now you’re a learner, and it’s ok… it’s ok not to be perfect…. and it’s ok that it failed. We can learn a lot from it….

Doug: Right.

Rebecca: …it can be really useful.

Doug: Right. Boy, that attitude… I wish it was more common.

John: New faculty, while they may have been exposed to more active learning techniques, they’re also sometimes reluctant to go against department standards of teaching because they know that there’s some chance that new things they try may fail… and it’s certainly safer and easier to do it after tenure, although by then people often get into habits that are hard to break.

Doug: So, a strategy that my department is planning to apply, and their current chair is highly supportive of, is bringing brand new faculty into the Active Learning Initiative right away… and so giving them classes that have already been transformed… so it becomes the new norm… so it’s not that you’re taking a risk or trying these new things and if it doesn’t work the rest of the faculty are going to shake their head… it’s “This is how you’re supposed to do it.”

John: Yes, having that as a prepackaged method of teaching…

Doug: Exactly.
JOHN… certainly would make it easier to disseminate that.

Doug: That’s the hope.

John: Excellent.

Doug: ….and they also won’t have the… I guess… the institutional power to fight against it as hard… as an established faculty member.

John: That’s true.

Rebecca: Yeah. How have students responded to some of the initiatives. I know in your department you haven’t done that active learning kind of stage but I know that you’ve had some other programs

Doug: Ok.

John: You’ve been doing active learning techniques in other classes for a while.

Doug: So, I can say three things. First, is in physics course evaluation. It improved and the students seem to really like it, and feel like they’re getting more out of the class than they are [in other classes]… because what they’re doing is they’re taking these highly flipped physics classes along with other classes outside of physics that are not… and so they see the contrasts… and I think both physics and biology have done a good job in explaining why they’re doing what they’re doing…. so that those students that say “ Why are you making us struggle? Why don’t you just tell us the answers, and then we’ll know how to do it?” and they’re explaining why that’s actually less effective… and to be honest in my experience, I do a fair amount of active learning in my classes already… and you have to explain why you’re doing it.

Rebecca: I agree. Students respond really well to that.

Doug: Right.

Rebecca: Yeah.

John: We’re much better in any case. They still sometimes resist because, one of the problems with a lot of the approaches, is that they don’t get as much positive feedback right away…

Doug: Right.

John: …because they do make mistakes and fail and that doesn’t feel quite as good as…

Doug: Right.

John: …doing well on the one or two or three exams that they might have happened to have had otherwise after cramming the night before.

Doug: In all three cases, I think, explaining what you’re doing, and why, before you actually do it… it’s pretty effective… and again you’re always going to have these students that fight against it. They’ve succeeded their whole… and it’s usually the top… and it’s those students that have succeeded in memorizing, and taking notes, and doing well on exams their whole life… and now you’re teaching them in this new way… and it’s harder. They’re gonna resist… not that surprising…

John: But it can work the other way, too. That the students who find that it’s more helpful to go through active learning techniques might encourage other faculty perhaps to adopt it.

Doug: That’s the hope… and I think the great majority, I think, get a pretty to have a pretty positive experience… and even more by the end of the semester. I think most of the students are going to have bought in if it’s done well. I mean. I’m not gonna say that every single class that did active learning the students loved it, because there’s some crappy active learning classes out there. I knew someone… they decided to flip their class and they admitted all of this… like I heard this from them, not their students… they said I turn all my lectures into videos and it was great and I really invested in the videos and then we would show up the class, I’d be like ok and I had no idea what to do in the class …They hadn’t invested in the class part.

Rebecca: Right.

John:: …and that’s the big part of active learning.

Rebecca: …that’s the active part.

Doug: That’s right. Right.

John: In most disciplies, you don’t need to create videos. There’s a lot already on YouTube. The real work needs to be on what you’re going to do in class to give you the most value added there.

Doug: That’s right. That’s right.

Rebecca: Our last question usually is what are you gonna do next?

Doug: So, we have a whole transformation plan. I mean at this point, which I talked a little bit about, but my spring will be heavily invested in creating and trying these invention activities. I’m crossing my fingers that they work. It’s the first course we transform and the department is part of the Active Learning Initiative. It’s a kind of a big ask… that we try something completely new and actually see some big results. It could be, at the end of the semester, we said why didn’t we just take that lousy pure lecture class… because that would be a lot easier to improve. I don’t know. Well, it’ll be exciting to find out. But the other big thing on my plate that I haven’t talked about already is taking these assessments that we’ve drafted in the fall and piloting them with partners at other institutions… and so if any of your listeners teach either introductory statistics or a second semester econometrics course, please contact me, douglas.mckee@cornell.edu, because we would love to have you pilot this thing…. and I’m hoping to give a big talk on it… on both of these assessments with our postdoc George Orlov who’s amazing… and will be on the academic market next fall, so if you’re looking to hire an economist that really loves teaching, and is great at it, and does really super interesting research, he’s your guy.

John: But not yet… not until he finishes…

Doug: Oh no… don’t hire him yet…

[LAUGHTER] …and so, what we’re hoping to do is make these published standards that can be used across the discipline as ways of evaluating teaching… and that can be big picture… we did this big intervention… or I think a lot of people use these standard assessments as ways to identify where the soft spots are in what their students are learning… and so people do that too… and so the vision is, for over the next five years, to take what we’re doing now and multiply it by eight… and so there’s there’s a lot to do.

Rebecca: … big shoes to fill.

Doug: …we bit off a lot. We promised alot. Now, we just have to execute. So far, so good.

Rebecca: I can’t wait to hear about the results and hopefully we have follow up and find out what happened.

Doug: We’ve submitted an abstract to the Conference on Teaching and Research and Economic Education which will be this June, where we report on the results of the transformed class, and so we won’t know… we will be getting the data in…

Rebecca: …moments before it’s due…

Doug: …moments before presenting the work. Hopefully, we’ll have a lot of incentive to put that together.

Rebecca: Great.

John: Excellent.

Doug: Oh, thank you both.

Rebecca: Well, thank you!

John: This was fascinating. Thank you for coming down here… or coming up here to Oswego.

Rebecca: Yeah, we’ve really enjoyed your visit.

Doug: It’s been a lot of fun you guys are doing great work here.

John: …and we really enjoy your podcast!

Doug: Thank you. More podcasts! I’m your number one fan.

John: OK, Thank you.

Rebecca: Thank you.