Category: active learning

36. Peer instruction

Posted in active learning, classroom management, evidence-based practices, higher education, large classes, learning, mental models, metacognition, pedagogy, peer instruction, teaching
July 4, 2018

Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know (and don’t know). Did you picture a lecture class, students taking a test, or students writing? In this episode, John discusses three ways in which he has been using peer-instruction in his classes: classroom polling, calibrated peer review writing assignments, and two-stage exams.

Show Notes

Transcript

Rebecca: Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know. Did you picture a lecture class, students taking a test, or students writing? If not, stay tuned, this episode explores ways to use peer-instruction to transform the learning experience.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today’s guest is my co-host John Kane. John is the director of the Center for Excellence in Teaching—that’s not even right…

[LAUGHTER]

John: …Center for Excellence in Learning and Teaching at SUNY Oswego.

Rebecca: Yeah, woops! Welcome to your own show, John!

John: Thanks, Rebecca.

Rebecca: Today our teas are…

John: Prince of Wales.

Rebecca: Oh, that’s a good one.

Rebecca: I have Golden Tipped English Breakfast today.

John: Excellent.

Rebecca: One of the areas you’ve been teaching experimenting in, and that I’m fascinated in, is peer instruction. Can you tell us a little bit about what peer instruction is and why you’re drawn to using this methodology in your courses?

John: Peer instruction involves using peers to assist with instruction, where students explain….

Rebecca: Thanks John.

[LAUGHTER]

John: …where students explain things to each other. One of the issues that we have is that, once we become experts in the field, it’s very hard for us to express things in terms that are easily understood by students. There’s a “curse of knowledge;” once you become adept at something, it’s really hard to explain things at a level that’s appropriate to the level of understanding that students may have. There was a classic study done in which a researcher gave people a list of songs, very well-known popular songs, and asked them to tap out the beats from that song.

Rebecca: Oh, I would fail…

John: …and then before actually seeing if people would recognize it (who had the same list), she asked them to make a prediction of what proportion of people would understand it based on their tapping… and they overestimated that by a factor of I believe, somewhere around 20 times. Basically, it was purely random if people happen to guess it. But the issue is, once you hear something in your own mind, it’s clear to you, but it may not always be clear to the people who don’t have the same rich net of connections. When students are explaining things to each other, they benefit from taking a position, arguing that position, trying to filling gaps and they’re also explaining in terms that are appropriate for people at their level of cognitive development for people who have a similar background in terms of what they know and their prior knowledge.

Rebecca: Sounds like a really good way to expand and refine mental models and also just develop better metacognition. Because, as soon as you go to explain it, you realize what you don’t understand.

John: …and if you don’t understand it yourself, your peers will often help you understand. they’ll say: “Well, you haven’t considered this…” and that sort of interaction is one that doesn’t work as well when it’s instructor to a large group of students. But, it does work very well one-on-one.

Rebecca: You’re known on our campus for teaching really large lecture sections. Implementing peer instruction in a large setting can seem pretty daunting, especially to someone who teaches smaller classes like I do. What strategies do you use?

John: The most commonly used one is to use clicker quizzes… and I use a methodology that Eric Mazur developed slightly over 20 years ago, where you ask the students a challenging question… you try to find questions that about half of them will get wrong… and over time you can develop that, you can come up with a pool of questions that fit somewhere in that range… and you let students first vote on the response themselves after they’ve had a little bit of time to process it, and then you look at the results. If you see that 90% or more of them got it correct or even 80% or more, you can just go over it and move on to the next topic, because most students understand it. But, if you see that somewhere around half of them get it right and somewhere around half of them get it wrong (plus or minus 20% or so), then the next stage is to let them explain it to each other, and that’s where the peer instruction comes in. When you have students argue it and take a stand and a position on it, we get a very significant gain and improvement when we then let them vote on it a second time… and the usual practice is not to reveal the poll results or the answer until after they’ve had that opportunity to engage in that discussion.

Rebecca: Just make sure, to make sure I understand correctly: you do the poll, you see the results as students don’t see the results…

John: Right.

Rebecca: …based on their answers or their responses when you decide whether or not they do the peer instruction piece. How long do they usually talk to each other about the topic?

John: It depends on the problem and normally I will have some undergraduate TAs and I’ll wander around the class and see what they’re talking about, listen in, answer some questions from them and the TAs will be doing the same thing…. and it’s usually pretty clear when they’re coming to a consensus. You can see them reaching for their clickers or their phones and getting ready to vote, so generally it may only be a minute or two, it could be longer… it depends on the complexity of the problems. Some of the problems require a bit of effort and require some calculations, but normally they’ve already done that… so, the second stage, where there’s a discussion, you can hear the volume build-up and then as they’re approaching solutions and consensus, it tends to drop back down again. It’s fairly easy to get a pretty good read on where they are and when they’re ready to vote again.

Rebecca: I imagine that you would really need to keep your ear to the ground, otherwise chaos could ensue. Because now, if they’re finished talking about the problem and there’s still time, then they could easily derail if you’re not quick to get back to the clicker question.

John: Right, and normally the time is generally held fairly tight. I suspect sometimes it’s only 30 seconds to a minute, other times it may go up to a couple minutes, but if I see them getting distracted and doing other things, the polling starts immediately.

Rebecca: Obviously technology is your friend in this particular situation. Can you talk a little bit about the technology you’re using to manage this many students all at once?

John: Here, we’ve adopted iClicker as a campus standard, so we use that in pretty much all of the classes where we’re doing polling and there’s both a physical radio frequency clicker that students may buy or they can buy an app and pay by the semester or over four years for the use of the app.

Rebecca: How do you make sure that the cost doesn’t get prohibitive to students?

John: That’s an issue, and it’s been a major source of concern…

Rebecca: They’re not very expensive, right?

John: Well, they can be expensive. A new clicker costs somewhere around $40. A used one can often be purchased for $15 to $20, sometimes less… and the apps I think, are somewhere around $12 to $15 for a semester and I think about $35 for four years.

Rebecca: …and you can use the clickers in all of the classes, right? So if multiple faculty member(s) are using all the same system, then the investment is a good one for students.

John: …and that’s why we have a campus adoption because in places where you don’t have that, students might have to buy two or three or four different clicker systems in different classes. So, once they buy the clicker for one as long as they hold on to it, they can use it in classes for the rest of their career. Almost everyone in the economics department, for example, now uses clickers, so if they’re economics majors or business majors, it’s very likely they’ll use them in multiple courses. The cost is much more tolerable when it’s spread out over multiple classes.

Rebecca: The other area where you do some peer instruction in these large classes is in writing. Which seems kind of crazy. You have all these students in this big classroom and somehow you manage to do writing assignments.

John: Yeah, my large class generally is somewhere between 350 and 420 students. At one time, for actually about a decade or so, I was giving weekly online discussion forums. But grading that or evaluating that and providing feedback was taking an awful lot of time…probably 30 to 40 hours a week. So, I pretty much…

Rebecca: A full-time job in and of itself…

John: I stopped that a few years ago and, a few years back, I replaced that with calibrated peer review assignments. The calibrated peer review system is something that Eric Mazur talked about while he was here… a visit in 2014… and when he mentioned it, a lot of people got excited. The way the system works is that you create an assignment, you store it on a central server at UCLA, and then it’s something that other people can adapt and use and modify—it’s released under a license, which is similar to a creative common license within the system… and you create the assignment… you create an evaluation rubric for the assignment… and you have to be really careful in designing that to make sure it’s one that students will be able to apply, because other ones that do that… and then you create three sample assignments yourself: a low-quality one, a medium quality one, and a high quality assignment… and you have students submit their own assignments first (according to the rubric and guidelines you provide to them)… then they go in and they evaluate the three that you’ve done. They’re given in random order, and they’re assessed in terms of how closely their evaluations match yours. That’s the calibration part. Students receive a calibration score based on how similar their evaluations are to the ones that you assigned to the sample responses. Then after they complete that stage, they evaluate each other, using the same rubric, and a weighted average of those scores is assigned as a component of the grade. They’re graded in a number of dimensions. One is based on the weighted average of the peers, where students who had a high calibration score will have evaluations that rate more highly in evaluating other students. They’re also rated in terms of how closely their evaluations match the others during that stage. So, if their evaluation is an outlier… much higher or much lower than other students… they lose some points on that… and then after they evaluate the other three students, they rate their own work… and one of the goals of that is so that they have improved metacognition. That, by the time they go back and look at their work again, they’ve rated three works by the instructor and three assignments done by their peers and then they’re asked to evaluate their own using the same criteria. What’s really interesting about the calibrated peer review process is their grade on this is tied not to whether they give themselves a high or low score on this, but it’s how close their self evaluation comes to the weighted average of their peer evaluations. So, they have an incentive to try to look at their work more objectively, and not try to game the system… because if they score their work too high or too low, they could end up with no points on the self-evaluation stage. So, the closer they get to the weighted average of their peer evaluations, the higher the score will be on that component.

Rebecca: I think that’s an area that we often see students struggling, is being able to effectively evaluate their work or other work. So, really training them to use a rubric and understand and think about what’s important or what’s not important about particular kinds of assignments or particular kinds of work could be really valuable to students in a way that we don’t really have other systems to do that.

John: The nice thing about this is it scales really easily. There’s a lot of upfront work in creating the assignments, creating the rubric, and a really good practice is to test them thoroughly before you give them out the first time. What I normally had done is asked some of my peers to look at that, some of my colleagues to look at it, and sometimes I’d have some upper-level students were…. and this does give students a little bit more reflective practice, where they get to look at their own work a bit more critically, perhaps, and reflect on it and see how they’re doing compared to how other students are doing in the course… and I think that’s helpful.

Rebecca: I think that the rubric would probably be a challenge to make but I think what would be more challenging is putting up those different assignments that are scored at different levels at the very beginning as your calibration tool. What strategies have you developed to make those in a way that it doesn’t take forever?

John: Well, I only do this three times in a semester, and once you’ve done it once, if you design it in a way so that it won’t go stale… and I generally have students, for example, find some articles in the news in the last six months that relate to a topic that we’ve talked about, or I ask students to find some examples in their own life to illustrate behavioral economics concepts in one of the assignments, for example…. where it’s not something that they could easily copy and paste from other people’s work. Because, there is always a concern with academic dishonesty and so forth. You don’t want these things showing up on Chegg or any of those other systems, where it would be easy to copy and paste good responses. So, I’ve tried to design assignments where once they’re done, they can be used for multiple years in one form or another. I modify them each year based on how they work. But perhaps a more serious problem is what happens when students really don’t like the evaluations. One of the things I’ve done when I’ve used this is to have three of these assignments, but I drop the lowest score… because, sometimes people will get some scores back that they didn’t expect or they may have neglected to look at the rubric I sent them and they may have omitted a major part of the assignment and ended up losing quite a bit of points all the way through that. But, as long as one of the scores is dropped, they have the opportunity to learn from their mistakes and do a little bit better. But, there are procedures built-in that make it easier to catch any outliers when you have someone who is just rating everyone extremely highly or rating everyone really poorly—inappropriately highly or poorly. There are tools in it which will give you a list of all the cases where there’s a high variance across reviewers or where someone happened to be evaluated by people who had very low calibration scores… so, if you end up with two out of the three peer reviews with low scores, that’s something that’s flagged by the system. I check all the cases where it’s flagged and I tell the students if they’re unhappy with their score or if they have any questions about it, to contact me, explain why they’re dissatisfied with their score, and then I’ll go in and look at it. In nearly all cases, it’s been an issue with the students submission and not with the peer reviews. Because, while some people tend to overrate things and some tend to underestimate some of it, compared to where I would evaluate the wok… on average, it’s been very close, typically, to what I would have scored or what I would have assigned as a score. But I do make, in rare cases, some adjustments when I see that something went wrong in the process.

Rebecca: Do you prevent students from seeing the score then, until you’ve reviewed all of the scores to make sure that you’re okay with what has happened before they have access or…?

John: In this system, that really can’t be done easily…

Rebecca: ok.

John:… because what happens is they get the results as soon as the last stage is completed. I’ll send a note out saying, “Now that the stage is completed, you can review your scores, you can read all the comments that your peers have provided, and you can see what your grade is at each component…” and we have gone over that in class so they know what they’ll be seeing.

Rebecca: What kind of workload do you end up with, dealing with problems?

John: In general, when I’ve used this in the class of 360 to 420 students, there’s usually 3 to 5 students who find their grade unreasonable, and sometimes, I found the grades perfectly fine. Occasionally one or two of those, I’ll make some minor adjustments to—if something went wrong where one of their peer reviewers didn’t show up, for example, one or two of them didn’t complete that stage of the assignment, and someone was overly harsh or perhaps overly harsh in their grading, but it’s rare.

Rebecca: Can that system be used for things other than writing? Like other kinds of documents?

John: It could be used for any type of document because basically students will either write something up or they’ll submit something and it could be an image, it could be used for peer review, or calibrated peer review, on pretty much anything as long as it can be disseminated in digital format. It could be used for websites, for example.

Rebecca: Well, that’s what I was getting at when I was asking.
You also teach some upper-level seminar courses with 30 or so students. This semester, you tried a two-stage exam after talking with Doug McKee when he was on campus about it. What is a two-stage exam and how did it work?

John: Backing up a bit, I was considering it even before Doug came here because I heard the episode of the Teach Better podcast where they discussed a two-stage exam and then when we were talking here and he was in one of our earlier podcasts and we discussed this very issue, I became more interested after we talked with Doug. A two-stage exam is one where in the first stage of the process, students take the exam by themselves and then in the second stage, they do some group work– either on a subset of the questions or on some very closely related questions. It’s being used quite a bit in the sciences and there’s a growing amount of research indicating that it has been successful. Some studies have found weak results, others are finding stronger results, but it’s still fairly early in the exploration of this. The Carl Wieman Science Education Initiative has quite a few resources associated with two-stage exams. This leverages peer instruction in the second stage.

The usual process, or the most common practice, is to take the exam period and have students work on this for the first two-thirds or so of the exam time slot and then they work in a group in the last third. I did it a little bit differently than this. In my case, I gave the exam on a Wednesday and I graded the exam but didn’t get them back to the students and then I selected a subset of the questions and I had them work on them in groups on that Friday… and that worked pretty well too, they had a little chance to review in between, they didn’t get to keep the exams, but there were only seven questions on it. They could go back and review things. I didn’t tell them which questions would be on the second stage in large part because I didn’t know. I told them that two of the questions would definitely be on it, but it would depend on how they did on the other part. So, I was able to look at the exam, find the parts where they had the most trouble, and assigned those as ones for the second stage… and in general, it was a remarkable experience. It was really nice to be giving an exam and to see students working in groups of three or four, actively discussing the issues, arguing over them, trying to explain things to each other and it was a really fun experience. It was very energizing to see that much effort being devoted to try to understand concepts that students had some difficulty with.

Rebecca: I remember seeing an image of your class being really actively engaged, really talking about the core class material that you shared during your test and I think the caption was: “This is during a test!”
[LAUGHTER]

John: Yes, I took a picture of it from my phone and I think I sent that to you during the exam because it was just so exciting to see that… and it was also a reminder for myself just how well this was working. I wandered around the room and listened in on the discussions and they were all very focused and coming up with much better explanations of these things then they would have likely been able to see if it was a whole class discussion… because they were very focused, they were arguing over what was the best approach to deal with some of these problems. I could see people making connections and suddenly understanding how things they had done before fit in and pulling together a lot of concepts that they might not have done as effectively if it had not been for those small group discussions.

Rebecca: Were you tempted to join in on those conversations because they were so lively?

John: I was, but I mostly just listened in and let them work it out themselves… and in general, they did quite a bit better… and what I should have mentioned before is that the overall grade for the exam is a weighted average of the first part and the second with most of the weight being on the individual part. One of the things that really appealed to me is that typically, when we give an exam and then grade it and return it, the students who did well generally just put it away and are happy with the results and they may glance at some of the things they got wrong (if they got many things wrong), but they’re not going to spend a lot of time actively processing it. The students who did poorly tend to get discouraged, some of them may give up a bit, but rarely are they likely to go back and try to put in the effort to correct their mistakes and to see where they went wrong. It was really nice to see that processing taking place by both groups. The students who did really well the first time deepen their understanding by explaining it to others and I suspect that should increase their long-term recall of this. The act of explaining it to others in some studies seems to be really helpful in encouraging transfer, where you can take concepts and apply them to other circumstances and when you’re in a course like econometrics, you have to be able to apply the same concepts in a wide variety of topics and areas. I think it was a very useful experience.

Rebecca: I think it’s a great method to allow some time and space for a reflective practice, because students tend not to do that on their own unless they’re asked to do it and if you do it as a homework assignment, I suspect that students don’t really spend that much time doing it, but this time they spent the whole class period doing the reflection. So, that seems really valuable.

John: Because I know a lot of people will do that. They’ll have an exam, they’ll give it back to students, and they’ll tell them they can make up part of the grade if they turn it in with corrections… and many students would do that, but I don’t think that would be as effective as having the group discussion on this. Some of them were able to make very clear what they didn’t understand and then they were able to get explanations from others and sometimes the explanations were right, sometimes they were wrong, but they had to process it much more actively and that’s always helpful, I think.

Rebecca: The grade weights is what seems most compelling to me in this situation because I’ve offered quizzes in my classes, more low-stakes assignments where I let students work on it for a while and I don’t tell them that they’re gonna get to do some peer instruction as part of it, but then they’re struggling with what they’re doing and then I say, “Oh, well, you have five minutes to work with your peers to revise anything you want to do before you turn it in.” And those generally result in some pretty active conversations as well, but there still are those few students who just copy down the answer and don’t engage in the conversation… but I think if there was that wait between before and after, that would really change that dynamic. So, I think that that’s a really compelling opportunity.

John: I thought it was useful and another reason why I didn’t do it all at one stage in one day is because I’m teaching on a Monday-Wednesday-Friday schedule and we only have 55 minutes and I have quite a few students in the class who are not native English speakers and they always take more time or they need more time to process and write information in a second language. So, I didn’t want to constrain the time and make both parts of it much shorter.

Rebecca: If you encourage people to practice and retrieve that information in extra time outside of class, there’s nothing wrong with that either.

John: Exactly.

Rebecca: I’d rather the students learn the material rather than just panic about a test. What do you recommend to our listeners to read to learn more about this evidence-based practice?

John: In terms of peer instruction, Derek Bruff has a really good book on using clickers. Eric Mazur’s original book on this, which is now slightly over 20 years old, is still very good… where he describes a process of developing this peer instruction technique. Eric Mazur also gave a talk here a few years ago and we have a recording of his presentation on this. There’s a really great example in there where he used peer instruction and what was most compelling about it, and Rebecca’s heard this before, but…

Rebecca: I was there!

John: …and Rebecca was there, was he used this example where he gave a really short presentation on what happens to the hole in a plate of metal if you heat it up… and people were asked to vote on that and then they had a chance to discuss it.

Rebecca: …and he never told us the answer!

John: …and then he noted how energized people were and he said, “You were so actively discussing these things…” When he tried to go on after making a point about how they suddenly were interested in something they normally wouldn’t have been interested in… he started to go on to the next topic. People were really upset, because they wanted the answer and he finally gave the answer, but he did that deliberately to show that this sort of thing… where the students don’t know the answer but they committed to a position and they want to know if they’re right… builds a sort of interest in learning that might not intrinsically be there otherwise.

…and that’s exactly what I saw, by the way, in my exam. They were so actively discussing things that normally they’d be bored out of their minds with. So, that environment can be very supportive of learning.

Rebecca: Yeah, it really gets people curious. I remember being in that room… dying to know what version was right? People had such compelling arguments.

[LAUGHTER]

John: Exactly, and that’s why it’s really good to pick questions, with any of these things, where it’s not going to be clearly obvious, where they have to process it, and they have to make connections, and you could build a case, correctly or wrongly, for different answers, and people want to know what the answers are.

Rebecca: I mean it was key that he finally gave the answer, right? So there was some corrective feedback there, so that people didn’t continue to mislearn the information.

John: And that was nearly four years ago, and we remember that very vividly. If that was just a point in a class that was given… say, four years ago, we probably wouldn’t be talking about that now.

Rebecca: I can’t believe it was that long ago.

John: I think it was.

Rebecca: It was a while ago.

John: Yeah.

Rebecca: …and so I’m dying to know, what are you gonna do next?

John: One of the next things I’m going to do is a follow-up to something we talked about in an earlier episode, when we talked to Judie Littlejohn about the metacognitive cafe. One of the things I’ve been observing is that the use of this process by having students work to improve their metacognition about how they learn and what they’re learning… Students, at least, perceive there is being some significant learning gains from that. That’s convinced me that I’d like to do something similar in a large class, but an online discussion forum for 400 students again doesn’t scale quite as well. So, I’m going to be doing some weekly activities and I’m working with Liz Dunne Schmitt who teaches our large macro class in the spring semester, and a couple of other people: Kris Munger, and Michelle Miller, who also who’s the author of Minds Online: Teaching Effectively with Technology (and was a guest here a while back). We’re going to try to put together an experiment where we use some evidence-based methods as weekly assignments, say for ten weeks in a semester…. that’s our current plan at least)… and students will be exposed to this… and they’ll engage in some sort of reflection or some practice with one of these activities… and then in terms of evidence-based methods of learning, such as retrieval practice, spaced practice, and interleaved practice, and similar things… and then we’re going to see how that exposure along with some reasonably easily assessed activity, which could be just some short responses in a forum or it could be perhaps some online quizzes, evaluating whether that impacts their actual behavior in the class, and their actual performance in the class. One-half of the group will be exposed to those types of interventions, and the other half will be exposed to some form of standard study skills module, because most of the students in this class of freshmen and basically what we’re looking at is, if we present students with evidence on what really increases our ability to learn, whether that will result in significant change in either their behavior, or in their performance. So, we’re going to try, at least the plan, is to try to see whether that affects the number of times they take quizzes that can be taken repeatedly, whether it affects the number of times they log in and view other materials, and whether it changes a perception of how we learn. so right now we’re at the…

Rebecca: And performance too, right?

John: …and their performance.

Rebecca: And is the plan to start collecting that data in the fall?

John: The plan is to put all this together the spring, I’m hoping and then to submit a proposal to the IRB, and then to conduct the study and the fall and the spring, at least for a first stage and then we’re hoping to be able to follow these students up, to see if this has a significant effect later in terms of their grades or their persistence.

Rebecca: Sounds pretty exciting. I’m looking forward to hearing how that goes.

John: It is. I’m looking forward to it being all together and actually being implemented. I think it’s an interesting study.

Rebecca: We’ll have to have you back, John.

[LAUGHTER]

John: I think we can manage that.

Rebecca: Well, thanks so much for sharing all this information about peer instruction. I know it’s something that I’m always kind of asking you about and like to hear about, and I’m sure others will too.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

32. The Three Little Pigs

Posted in accessibility, active learning, authentic learning, collaboration, higher education, inclusion, learning, pedagogy, project based learning, teaching
June 6, 2018

What do the three little pigs, the big bad wolf, and dragons have to do with web design? More than you would think. Rebecca Mushtare discusses how a trip through fairy tales may open up the opportunity to develop empathy skills and conversations about race, disability and identity.

Allison Rank joins us again this week, this time as a guest host.

Show Notes

Transcript

John: What do the three little pigs, the big bad wolf, and dragons have to do with web design? More than you would think. In this episode, we’ll explore how a trip through fairy tales opens up the opportunity to develop empathy skills in conversations about race, disability, and identity.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer. Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Allison Rank, a frequent guest on this podcast, joins us today as guest host. Our guest today is Rebecca Mushtare who, until this episode, had been the co-host of this podcast.

Allison: Nobody panic. She’ll be back in this chair next week.

John: Today our teas are:

Allison: English breakfast tea.

Rebecca: What?!?

Allison: …under duress. I’m highly under caffeinated.

Rebecca: I’m drinking my normal English afternoon tea.

John: …and I’m drinking pineapple ginger green tea.
We invited you here today… because you’re always here… but we’re asking you…

Rebecca: …it’s a matter of convenience….

[LAUGHTER]

John: A year ago your daughter was born… now the three little pigs have invaded your class. Could you tell us a little bit about how the three little pigs made their way into your web design class?

Rebecca: I’ve been looking for ways to help students develop more empathy for their audiences, and it’s been a struggle. Students (or anybody who’s new to anything) will immediately try to make things for themselves, because it’s the audience they know best. So, it’s the easiest way. If you’re working on technical things or other concepts you don’t have to worry about audience too, because you have that part figured out. But, I’ve been really wanting to challenge students to dive into audience and also deal with accessibility issues which doesn’t come intuitively to them. So, the three little pigs actually offers a really great opportunity to have different audience members to think about (and audience members that don’t really exist); it becomes a safe zone. In this scenario, I’m using three titles as ethnographies for the students to read to get to know their audience better. I spent some time reading about ten different versions of the “three little pigs” and I’ve identified the best three. They are: The True Story of the Three Little Pigs by John Scieszka and Lane Smith, and The Three Pigs by David Wiesner, and There’s a Wolf at the Door: Five Classic Tales Retold by Z.B. Alley and R.W. Alley.
They read those books and then we come into class and I ask them to help me understand who all the characters are, what’s important to them, and some of their characteristics or qualities that we need to think about in terms of design… and then (from the perspective of the characters) what’s going on in the community that they live in… and the frame that I’m giving my students is that they’re in this community called Dragon Town. Dragon Town has a mayor named Mayor Melanie McDonald, and she’s human, but there are talking animals and dragons and other creatures that live in this community together and there’s a clear creature divide going on. So, the humans seem to value themselves more than the other critters in town. The poor pigs, they’ve got houses that are falling down. They don’t even up stand the Wolf’s breath. So, we’ve got some issues going on here.
The students read the stories, came to class, brainstormed about these characters, and helped identify some really big issues that were happening in Dragon Town… and then my challenge to them was, in teams of three or four, to identify one of those 10 that we identified as a class…choose one that they were gonna use a web design to help raise awareness of or to start to tackle. Obviously they’re not gonna solve these big problems, but they could make a dent into it.

John: The purpose then is to have students look at a problem from another perspective, from the perspective of the intended audience of the webpage, rather than using their own biases.

Rebecca: Yeah, exactly, and it’s something that they really need to practice… and so, yeah, this is a good opportunity to do that. They’re characters that their familiar with, but the books actually challenged a lot of their initial remembrances of some of the stories. So, it’s a nice way to get them to revisit that in a different way.

Allison: How was this different than how you’ve tried to approach the same topic in earlier iterations of the class?

Rebecca: In a previous episode, I think I talked about my simulated client project where I had these big company scenarios with the audience members being Oswego (the community that we live in) and they worked okay… but the students had trouble aligning themselves with older adults or middle-aged individuals who they just don’t seem to find relevant to themselves and even though these are individuals that are readily available in our community that you could interview and get to know, it was a struggle. We did a project in the fall, “The Voices of Oswego Veterans” project that we had a guest (Stephanie Pritchard) on who talked about that project… and we did a web project with that as well… and that was another way to deal with the audience. This time the audience was members of the Oswego community (the SUNY Oswego community), so they had a little bit easier access to that community… but the community that they were representing was different from themselves. These were students, so the population that they were addressing or talking about was student veterans, which was an identity that nobody in the class happened to identify with. That got us closest to solving the problem… but it wasn’t quite where I wanted them to be yet. What’s nice about this is that you don’t have to worry about offending anybody, because they’re not real.

Allison: I can imagine how the fictional characters are really helpful in terms of giving students a lot of space to play and a lot of leverage, but I have to imagine that there are some real challenges associated with giving them that amount of space as well. I guess I sort of have a gut reaction that thinks that they will make up things that cause problems in and of themselves. They’ve got enough rope to get in some dangerous positions. What are some of the challenges that you faced?

Rebecca: That’s a really good question. What I found was, they were willing to talk about things that they were never willing to talk about before. That, first of all, was a good space to be in. That was things like: “oh, there’s species profiling going on…,” “oh, there’s accessibility issues because pigs have hooves so they can’t type and tap on the computer screen…” …the accessibility issues that just bubble up. There was also the concern that critters were eating other neighbors, so we needed to start a campaign to be vegetarian, for example. So, there’s a lot of different things that came up…. a lot of social issues… another one was stranger danger… and then they did these presentations to the mayor, and it was important because we brought someone from outside in and I think that helped prevent some of the issues that you were identifying could bubble up as being a problem, but there was someone that wasn’t me who was the audience but I didn’t tell him who it was gonna be (it was just a grad student I bribed) who came in and just sat and played the part and asked questions and what have you…. and they were taking notes and then we went away and had a meeting and I came back with notes to the students about what the client was concerned about. So, that helped resolve some issues. But, you know, in the presentations there were some crazy things that happened… like the one on stranger danger, for example, the students had still indicated that the stranger, the bad character, was the wolf and the whole point was that all of the animals, and all of the creatures, and all of the humans, also have children and they all need to be concerned about strangers. That we shouldn’t associate one population as the bad actor. We ended up having to have a conversation about that. You can’t perpetuate these stereotypes, but what happened was we could have that conversation safely.

Allison: The familiarity played in the same way that a stereotype would traditionally function in class, but in a much safer space to have the conversation that resolves it.

Rebecca: Exactly. We were having crazy conversations about racial bias, and all these sorts of things, but under this guise of “it’s about the species” and the species problem that’s going on. And now all of a sudden it became safe. When that one group was having issues getting their head around it, I said to them: “You realize that this is the exact same thing as racial bias, right?” and they just looked at me with deer-in-the-headlights look. The next time they came back, the whole project was fixed.

Allison: That was actually gonna be my next question. At what point did you pull out from playing in the sort of allegorical space to say: “Hey, here’s what we actually just did” or did you let the experience and the skill building stand on its own?

Rebecca: I let things unfold organically, and I prodded and probed as necessary. I didn’t want any projects to perpetuate stereotypes or to perpetuate lack of accessibility… those two key issues. I probed and invaded their team time a lot with those particular things to push them on that, but you know they’re not perfect. But, I think they did a lot more growing in that area than they would have otherwise. What I think is missing, that I want to do next time is allow for more of that reflection at the end, so that they could apply it to some other projects. What I’m thinking about doing is have them present the work as if they were in an interview, and so how would you explain this project and what you learned from this project to a potential employer who has no idea what Dragon Town is, so that it becomes something that’s valid and useful… and I think that’s going to take some effort on their part to make that leap. But I think it’s actually a really good project for them to talk about in an interview and most employers would see the value in that.
I already have them do portfolio documentation. I already have them thinking about that, but I need to coach them through that process a little bit more…. and maybe actually make them present that.

John: Yeah, I could see an employer looking at a webpage making a case on avoiding inter-species consumption and being perhaps a little bit puzzled….

Rebecca: The tagline was “don’t eat your neighbor.”

John: Yes.

Rebecca:… which I thought was right on.

Allison: Yeah.

Rebecca: Well, and that group actually was interesting too because they wanted to do something that was: “Don’t eat your neighbor.” They wanted to be vegetarian but I was like, “Well, dragons have a big appetite. What are you gonna do for them?” So they came up with this tree salad or whatever that has just bigger things. They had to adapt the recipes and things like don’t forget there’s small kids. You got to think about these different populations, and they adjusted their content accordingly, to rise to that occasion. I also found this really great article about whether or not pigs are colorblind that I used as a doorway into thinking about accessibility issues. Apparently, I learned, pigs don’t perceive color the same way that humans do. They can’t perceive as many colors, so we have to really be concerned about the spectrum of colors and the kind of contrast that colors have… so that they would be accessible to pigs,,, but that led into conversations about maybe the pigs have to use voice activation because their hooves won’t let them type on their devices… and then we also had to talk about a mobile device for a dragon is pretty large…. so we had certainly some fun playful conversations, but they were really meaningful. We started talking about those issues pretty deeply in a way that I’ve never had in my class before.

John: Were the students more open to addressing these issues when it was in this safe zone or this safe space?

Rebecca: Yeah, even when I called that one group out on being stereotypical and perpetuating bias, they just received… and were like: “Oh, okay” and then you try it again… “is this better?” “My god, could you push it a little bit more?” and gave them some ideas about how they could push it… and our first solution wasn’t great after that…. It was to put in a separate monster that didn’t exist in this world as being the stranger, and then I identified that like when someone the other, we shouldn’t just assume that they’re the bad person or the bad creature. We had to be careful. I tried to call them out on whether or not we were using the word person, because it didn’t apply to dragons. So, it was funny [in] their presentations they were really conscious about things like that and trying to be inclusive in their language. So, yeah we ended up trying to tackle some of those things, and I was pretty impressed with how far they got… but it took some pushing. That one group took four or five tries before they had something that was gonna work.

John: How did students respond when you first gave them the assignment?

Rebecca: Well, I should probably provide a little setup in that my class includes design students, marketing students, and graduate students in HCI. So, it’s a fairly diverse population in and of itself in terms of disciplinary background. So there’s that. There are a number of people in the class who may not be traditionally artsy or creative, so it’s a little risky, right? I think I’m also known for being very serious. Which if you know me personally, that might not be true, but in the classroom students perceive me as being very serious… and the semester just was not going great, to be honest. It’s like something’s got to give, the students were struggling with a lot of the technical things, and so I basically threw the syllabus out or revised it significantly. stopped and did just technical exercises so students get comfortable with some of the things that they were really struggling with… and then one day I just showed up and said this is what we’re doing… and they had a ton of fun…. and were shocked… they’re just like “Is she serious? She lost it?” There was definitely those looks, but then there was a couple of key students who just jumped in and ran with it… and I think that really helped. So, I’m hoping that that will happen again. I think if everyone in the class is a little too serious, I don’t know that it would work.

Allison: Would you plan on sticking with, in the future, the three little pigs as sort of the through line story or it sounds like the story with the five different ways that the wolf is at your door? Does that give you some entree into some other storytelling avenues?

Rebecca: There is some entree into some other avenues and I maybe need to read some more fairy tales to be up on that, but the reason why I stuck with the Three Little Pigs is actually the wolf is the character that carries through all of them. So, that the five stories that are connected are all based on the wolf and different stories. So there’s Little Red Riding Hood, the Boy who Called Wolf, those are some of the stories in that other one. So, maybe there’d be some versions? I also happen to know that there was like the version of the Three Little Pigs told from the wolf’s point of view, so I really like that because it’s in direct conflict with the Three Little Pigs version of the story. I liked that the ethnographies that they were collecting were realistic in that they conflicted with one another, that they had to deal with the fact that there was conflicting information, and that they had to resolve that or deal with the fact that a wolf’s perspective was different than the pigs perspective of what the wolves perspective was… and I think that was a healthy messiness about it that worked pretty well… and the particular version of the Three Little Pigs that I used pigs escaped getting eaten by the wolf because they jump out of the storybook. So, there’s some plot twists in there that the students wouldn’t necessarily expect. It’s not a traditional version of the story… plus, they all have really great illustrations and they’re beautifully designed.

Allison: Are there other classes where you’d be interested in trying the same type of fictional ethnography technique?

Rebecca: I think it could work in some other scenarios, but I like this because it’s in my intro class. It’s a nice doorway in. What I’m really interested in seeing is, when I have a couple of these students in the advanced class next time, if that impacts their ability to do some actual real audience research and use that research in context. I think I want to monitor that first before doing some of this other work. I like it in particular because it’s a beginning class even though it’s at the 300 level.

John: It sounds like a really fun project, and there’s nothing really wrong with making learning fun.

Rebecca: Yeah, I had a good time and we had some moments where you had to really practice the deadpan look, you know, be really serious about what it is that we’re doing… and that part was really fun.

Allison: …and that seems like an amazing turnaround on a class where you have to scrap the syllabus halfway through a semester.

Rebecca: Yeah, it was amazing… the community that was formed around the project… and the way that they were exchanging with one another and coming together was incredible, and I was so thankful.

[LAUGHTER]
There’s nothing worse than an off semester and you just want out. I think everybody wanted out and so I just said “We’re out. We’re gonna try something new” and it worked, so that was good.

John: I guess the next question is: “what are you going to do next?”

Rebecca: That’s a good question… I think that with this project I’m hoping to expand it a little bit… so I’m currently thinking through “are there things that I can eliminate that I was doing before that I could embed in this project or I just allow them to have the time and space to fully build things out?” They have really good ideas and pretty good plans and the execution is almost there and I’d like to be able to have them have that time for the “almost there” to be “there” and then also to do that reflection piece that I kind of half-assed.

John: Okay, well thank you for joining us and I guess we’ll see you again on our next episode… and back as a host.

Rebecca: I mean, that is, if you’ll have me back.

[LAUGHTER]

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[Music]

30. Adaptive Learning

Posted in active learning, adaptive learning, assessment, higher education, learning, OER, online learning, pedagogy, teaching
May 23, 2018

Do your students arrive in your classes with diverse educational backgrounds? Does a one-size-fits-all instructional strategy leave some students struggling and others bored? Charles Dziuban joins us in this episode to discuss how adaptive learning systems can help provide all of our students with a personalized educational path that is based on their own individual needs.

Show Notes

In order of appearance:

Transcript

John: Do your students arrive in your classes with diverse educational needs? Does a one-size-fits-all instructional strategy leave some students struggling and others bored? In this episode, we examine how adaptive learning systems can help provide all of your students with a personalized educational path that is based on their own individual needs.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Charles Dziuban. Chuck is the Director of the Research Initiative for Teaching Effectiveness at the University of Central Florida, where he has been a faculty member since 1970 teaching research design and statistics. He’s also a founding Director of the University’s Faculty Center for Teaching and Learning. Welcome, Chuck.

Chuck: Oh, thank you both. I’m just so pleased to be here. The good feelings of Oswego continue… you know my growing up upstate New York and my attending Oswego. I’m really honored to do this… makes me feel good.

Rebecca: So glad to have you.

John: We’re very glad to have you.

Rebecca: Today our teas are:

Chuck: I have water from the water fountain.

John: My tea is Yorkshire Gold.

Rebecca: I have Lady Grey today.

John: We’ve invited you to join us to talk about adaptive learning. So, what is adaptive learning? Is it something new?

Chuck: Sure I’d be happy to talk about that. Let me answer that question in reverse order. It’s anything but new. I think adaptive learning has been around since we’re thinking about teaching and learning. I think if we went back to the Middle Ages we have an idea of this notion of adaptive learning. It is not new. In fact, where I came from, and what I’m in my career, I’m looking at a paper written by John Carroll called “A Model of School Learning…” where John laid out this set of equations that basically answers this kind of a proposition: if how much time a student spends learning is constant, what they will learn will be the variable; if what they learn is the constant, how much time they spend in the educational enterprise will be the variable. So that’s the basic sort of notion of that. When we define that, it kind of makes sense. If we give students a limited amount of time and anything… a course… a university… a semester… whatever you call it. It makes sense that what they acquire in that activity will be the variable. If what they learn is a constant, how long they’re going to spend in this enterprise will be the variable and we’ve all experienced that, but we’ve all butted our heads up against something for years and years and years until we finally understood it… and this is the basic notion of adaptive learning…. whatever time it takes, it takes for you to learn something… and he went further to break that down. When we say time, it’s just not how much time in the Augustan sense, it’s how much time do I spend actually in the learning enterprise and what is your perseverance… and what is your aptitude for this… so kind of modified for that. So, that makes eminent sense. That’s what adaptive learning is… basically it’s going to take whatever time you need as a student to learn this material to some sort of satisfaction level. That’s what it is. It’s a very simple concept.

John: …because students come in with very different backgrounds. Some have very rich backgrounds in some areas and weaker in others… and that’s going to vary quite a bit depending on what they’ve learned in prior courses or in their life experiences.

Chuck: Oh, absolutely. I mean I can remember my own experience walking in new courses at Oswego being virtually lost because I had no background and no experience in the course and having to really ramp up my energy and effort and use of time in order to: 1. reach the baseline in the course and go on in other courses. In adaptive learning we have students who enter courses who… don’t take this the wrong way… but fundamentally don’t need to be there. They’ve mastered the material… they can move on… and I’ll give you examples of that later. So, this gives a great deal of flexibility to the learning enterprise.

Rebecca: From the learning perspective, it sounds really wonderful. From a teaching perspective, it seems like it can be a little bit challenging – especially before some of these technologies were available.

Chuck: Nightmare. Yeah, it is. Of course, the way we’ve conceptualized this whole notion of how students learn, it really does… We’ve organized learning around this entity called the class… it’s a unit… yeah, and we call it a class… and if you think about it in terms of a course sequence, say in mathematics, the students who don’t pass their placement exam so they take intermediate algebra, and college algebra, and trig, then precalculus. You can conceive of that as these discrete units, but if you think about it, what we’re doing in our math department… we’re designing that as a series of skills across those things we’ve called classes and that makes the whole notion of teaching quite different. Really what you have to do at adaptive learning… if you begin to plan a course you really have to look at what you have taught, what the granularity of the course is, and your role as an instructor in all of that. It’s very very different and it really forces our instructors to think about what it is I’m teaching… it is much more than a syllabus. The granularity of what you teach becomes critically important.

Rebecca: How did UCF get involved in adaptive learning?

Chuck: It’s an interesting question. That was organic. You may or may not know that UCF is the fastest growing university in the country. It is one of the largest universities in the country. The good news is we’re growing; the bad news is we’re growing. [LAUGHTER] We cannot possibly build an infrastructure to house our students. So, besides adaptive learning, we have online courses, we have blended courses, we have lecture capture courses, and adaptive learning is a natural outgrowth of that sort of structure. 43% of our full-time credit hours are produced online so we’re almost a half online university… and one of the things is we became involved in this simply because one of the things that John said. Our students are very diverse. They come with very different backgrounds and even putting them in this thing called a course may not make sense for some of them in terms of their pre assessment of where they are… and the notion of adaptive learning pre-assessing them and then putting them in the proper learning sequence within a set of objectives is why we got involved in adaptive learning… to doing this… and what we did is we simply recruited our best faculty, say “Look, see if you can do this. See if you can, then kind of accommodate what students are, where they are, and how they’re learning, in a much more flexible environment. It just simply make sense for students… and given the fact that we have a diverse background…. students coming from underrepresented populations… it just makes sense at so many levels.

John: One case where we’ve done that institutionally here at Oswego is, beginning about three or four years ago, we adopted Aleks for the math requirement. It’s used for initial placement and then it’s also used for students to get up to the level needed to meet basic course requirements. So, students come in and then if they don’t meet the requirements for the courses they’d like to take, they can spend the summer working on that to move up to the level… and then test themselves again to basically get to the level they need to be to make good progress… and it’s been working very well from everything I’ve heard, but we haven’t done much more yet at an institutional level.

Chuck: Yeah, well there are, as you probably know, with technologies like this there’s a probably a new platform coming out every day. I mean this as a metaphor. Our math department uses Aleks and if you go into an Aleks class… and Aleks is a pre-designed course, there is some flexibility but it’s a course that’s pre-designed. These platforms come in basically two varieties: where courses are embedded in them like stuff with McGraw-Hill or Knewton or something like this, or content agnostic like Realizeit where you have to build the course. That’s two kinds of things. We have an Aleks course in college algebra, and if you look at it, it kind of looks like chaos. Students are coming in, there are people lecturing on the boards, there are student assistants helping them, they’re on the thing. When they master it, they master it and they leave. So, any concept of a lecture course is completely out the window, but what’s happening is: students are being assessed, they’re being reassigned to different areas within the Aleks platform to master the skills, and then moving on. So, some of them can complete a course in half a semester… and I assume you’re experienced at seeing this at Oswego, but one of the things we’re facing is where with adaptive learning, we’re trying to get over the teach then test model for a course. We’re trying to embed the learning assessment within the adaptive learning platform. So, we have a test-free course where essentially assessment becomes part of the learning platform assessment… actually becomes part of the teaching… and it’s really quite exciting, but it is very daunting to do. We’re kind of wedded to tests in our environment.

John: I believe that you’ve been working with Colorado Technical University.

Chuck: Yeah!

John: How did that partnership come about?

Chuck: Organically.

[LAUGHTER]

When we came to realize that adaptive learning may offer some advantages for us, we asked several vendors to come in and make presentations to faculty… and this was basically a faculty decision. They saw several platforms and they said “Realizeit is the one. This is the one… because we want the flexibility to build our own courses.” But, of course, there’s a Chinese proverb: “Be careful what you ask for….” and they got Realizeit… but now they had to build the courses. That’s daunting when you start building your courses. Realizeit’s been very helpful. They have a process called ingestion where they’ll try to take whatever you have in a course… and I’ll say this and I say this with love and kindness… all of these platforms have a bit of clunk associated with them. They all work, but they are all some problems associated with them and problems in quotes. I’ll say challenges, or in Provost speak, they have opportunities associated with them. [LAUGHTER] …and we work with them… but in doing this kind of thing, but we began presenting this with our partner (and I’ll talk about our research relationship with RealizeIt in a little bit)… but working with RealizeIt in some research that we began presenting and Colorado Technical University has been doing something that we’re not able to do yet, that is scale it. Scaling it is an issue. What they have done is, they’re a private for-profit institution, and they’ve scaled it at a remarkable kind of thing and thrown a lot of resources at it. So, we’re very good at research, they’re very good at scaling, and Realizeit is very good at research. So, when we saw them presenting and they heard us presenting, it was sort of a speed date. They said “hey, you’re doing pretty cool stuff” and we said to them “hey, you’re doing pretty cool stuff, you want to play?” …and that’s how it started… working with them and it’s led to several publications and several presentations, and then with RealizeIt looped into this, working in the background, we’ve been able to do this kind of thing where we think partnerships are really important in studying this… and of course I’m gonna recruit Oswego today. You should be joining us.

John: I’ve been looking at adaptive learning platforms for a few years.

Chuck: Yeah.

John: I’ve worked with some from the publishers but it would really be nice to build something from the ground up. So, we’ll talk about that perhaps a little bit more later.

Chuck: Alright. Well, I will say this. Our philosophy is we give away everything. You can have anything we have. We’ve published several papers. We have several projects underway now with adaptive learning. Any of your audience, and certainly my brothers and sisters at Oswego can have anything UCF has done. The idea is we need to do it for partnership. I did a presentation on, if you go to Google and search “the Grand Cafe.” The Grand Cafe was the first coffee house in England, and what it was was a result when the British discovered coffee houses ideas really blossomed. The Brits didn’t drink water for a long time because they were afraid of the water, and so for a couple of hundred years they basically started having a beer for breakfast.

John: Right.

Chuck: A couple of beers at mid break, and then a beer and a gin. They spent a couple hundred years drunk.

John: ….and so did the founding fathers.

Chuck: Yeah, absolutely. But when they discovered coffee houses… a coffee and tea (which you’re drinking)… they were in this space they were in this learning space and when you switch from a depressant to a stimulant there are many more ideas and that’s the grand Enlightenment… but the point being that it was a partnership. When we talk about these grand ideas they are not Eureka moments, they are not lightbulb moments, they are people working together creating ideas and, if you will, letting ideas have sex… in the idea that these ideas can grow and be developed and that’s what happened…. and that’s what happened with our partnership and now we’ve added another partner. We’ve added a Petroleum Geo-Services in Oslo, Norway. It’s an exploration company looking for oil all through the world, and suddenly they have to train these people aboard ship and they looked at this and said: “Whoa, it costs us a fortune to bring these geophysicists back to Oslo to train them. I wonder if we can do it aboard ship with adaptive learning.” So, we’re looking at that. So, there are a lot of possibilities and I’ll say we worried about the students adapting to adaptive learning. No problem. Of course, it’s embedded in our LMS so they don’t even know they’re not in Canvas. They still think they’re in Canvas.

Rebecca: So, you alluded a little bit to Realizeit’s potential, because you mentioned that it’s one of the few platforms where you can build courses from the ground up. Can you expand a little bit more about its strengths relative to some other platforms and maybe also some of the stumbling blocks associated with building something from the ground up?

Chuck: I’m not sure I know the characteristics of all the platforms. I know that Realizeit is content agnostic, and I’m sure there are other platforms that allow you to do that… and probably there are other platforms available too, as you begin to explore this where courses are assembled but they have some flexibility for faculty members to take components in and out. I would think that would be a real advantage. It’s sort of the continuum… building of course from ground up is daunting and one of the problems is the granularity of the course. Getting this level of granularity so the course flows evenly…. it’s always a rough start in terms of doing that and we’ve provided as much support we can for faculty.

John: How can faculty get started using adaptive learning? Is this something that’s best done at an individual level, or at a departmental or an institutional level?

Chuck: It’s a big task and whatever support you could provide… especially instructional design support… in terms of looking at a course, redesigning it, and putting it together, and that’s very important… and run-throughs are very important, test beds for courses before you roll them out. I, in my role having to evaluate our technology, I taught online, blended, lecture capture, adaptive, and each time I rolled it out… it’s been terrible. It has a lot of bumps along the road and our working two years with adaptive learning…. they’ve had some issues and by our work with them for two years, we have helped them clean up a lot of issues… and they would be the first to admit that. They help out on the research side of this. They’re very very strong. They can do research things we can’t. We can do research things they can’t do. It’s the perfect yin and yang… and this is a wonderful kind of relationship to have…. and Michael Feldstein and Phil Hill and the MindWires e-literate have built this project now they call the Empirical Educator where they’re trying to get vendors and academics together to work together… to begin to look at this.

John: We will include links to the articles that you’ve mentioned in the show notes.

Chuck: Absolutely. Okay.

John: The question I had about that (in terms of individual development or other forms of development) was basically whether people are developing their own variants of the course or is it done at the departmental level? Or the program level?

Chuck: Both.

John: Okay.

Chuck: How do we begin an adaptive learning? One of the things you have to do to demonstrate to faculty that it can be successful. So what we have done is we cherry-picked our best faculty, the kind of faculty that if you show them this platform, they’d say, “hey, we’d like to try this.” So our initial pilot study was done in Psychology, was done in College Algebra and Pathophysiology. And all of these faculty are the ones who would say, “I would try to do this anyway,” and we built it that way and then began demonstrating it with individual faculty. And now we have a pilot project in our College of Business to make as many courses as we can adaptive learning. So it’s now up to the college level and we are now making presentations to all of the individual departments saying, “look, this is the possibility— is this a strategic initiative for your department? Do you think this would be something that would be a value to a department? A value add to your department versus one-off courses?” Because, for instance, in Psychology, the Psychology professor developed it over three semesters and faculty really do have the option of how much they can lock down this course. You can lock down an adaptive learning course so it looks like a regular course or you could make it go completely adaptive. It’s pretty scary stuff and what Jeff did in Psychology is his third go-around, he said, “it’s adaptive, go!” That’s all there was to it. They could go through at any pace they wanted to. In a cohort of say 20% finished the course in three weeks and that sounds like heresy, okay. But they did it and they were finished. Their next thing they said, “okay, we want Psychology too, we’re ready for it” and wasn’t ready. Okay, it wasn’t ready. So what do you do with them— your students who have completed a course in three weeks. They’re done, they’re verified, they’re certified that they’ve completed the course and they have nowhere to go. So it has great consequences for building this out in terms of doing this and we work very hard. Intermediate Algebra (they didn’t make it into College Algebra) so Tammy Muse, who will be featured on 60 Minutes in a couple of weeks, has allowed them to start Intermediate Algebra when they complete [it], go directly to the Adaptive Learning College Algebra course and that cohort now is finishing the course on time and it’s really quite amazing. And what’s the objective is, well, make that cohort larger. Just the things that you were saying about Aleks, (that’s a bad word in Florida, “remediate”), but let them acquire the skills they don’t have and then go directly to the course they need. And what Realizeit will do is sequencing them back and Tammy has done an amazing thing. She has made all of her assessment items reflect the diversity of our campus with names and the diversity of the disciplines she made them have the disciplines reflect whatever their major is, whether it be Engineering or Physics or whatever. And though the problem sets “they work” all reflect those kinds of things. A lot of work, but it’s working beautifully.

Rebecca: Sounds really exciting.

Chuck: Well, on certain days, it is.

Rebecca: [Laughter] Exciting can be both scary and…

Chuck: Oh yeah, yeah yeah, yeah.

Rebecca: We talked a little bit about whether or not the platform allows for interleaved practice. How does it work, what does that look like? Can you describe one of these courses?

Chuck: As best I can.

Rebecca: Yeah.

Chuck: It looks like you’re in an LMS (Learning Management System) and you have exercises. RealizeIt has a decision engine built in and I know a good deal about the decision engine. I don’t know everything about it because these platforms are proprietary. These vendors do not like to give away their trade secrets but Realizeit is Bayesian-based. It gets prior knowledge assessment of a student, then based on that prior knowledge, it assigns a student to a location in a course and begins and then begins to assess them. It looks like an LMS, the learning can be anything from videos, to simulations, to reading, to practice exercises, to discussion boards— it can be any format that you would normally have in any particular course. Students can participate in any number of ways they are assessed and then based on the assessment, which can be (I’ll talk about that in a second), can be anything. It can be a simulation, it can be a practice exercise, it can be a performance— it can be anything. It evaluates them and based on the evaluation, it reassesses their learning path and then their sequence back into that and they can have any number of ways to go through the learning factor. So this platform is always thinking about them but it is re-assigning them and it is constantly re-assigning them to learning passes that goes all the way through. It’s kind of scary to look at in terms of what’s happening and in it then, it has for them, “you’ve mastered these, you have not mastered these kinds of things,” I wish I can tell you it works as smoothly as I just described it…

John: [LAUGHTER] But it’s getting there.

Chuck: It’s getting there and what we tend to do is overestimate the short-term impact of these things and underestimate the long-term impact— that’s something we do with all technologies. If you look at the latest MIT 10 technologies that are gonna revolutionize them, one of them is Babel Fish earbuds that instantly translate languages for you. Well, they can’t do that exactly yet, they will eventually maybe get very good at that, but that’s what it looks like. It looks like a regular LMS and if you experience it, you have this sort of seamless feeling that you’re moving through this with no real impediments to this so you kind of go through your own pace. And what it also does it learns how you best learn. If you do it best with reading, it’ll do it that way. If your best with simulations, it’ll steer you up to simulations, but it’ll steer you to them. But faculty have to prepare all of those things so “therein lies the rub” as a friend of ours in the round theatres in England would have said.

John: I looked at the Acrobatiq platform a couple years ago, I met with some of the representatives from the gardening. I haven’t looked at Realizeit very much yet, but these platforms are really good at giving students lots of retrieval practice and assessing where they have weaknesses and doing that type of [a]daptation. But one of the things we know is that interleague practice, as Rebecca mentioned, is really helpful in increasing recall when you ask students to go back and test them on things that they learned earlier in the course is really helpful and encouraging deeper long-term learning. And I asked about Acrobatiq’s ability to do that and there wasn’t any and that’s why we were a little bit curious about whether perhaps Realizeit had that ability to go back and bring in questions earlier. I know Aleks does that a little bit but most of the platform’s I’ve looked at so far haven’t.

Chuck: Well with Realizeit, students have that option. I guess if we could make interleave a verb, they had the option to interleave, I’ll give you an example of that. Pathophysiology— in the state of Florida there’s a requirement, now, most hospitals require their RN’s to become BSN’s and that’s causing some angst. Nursing is stressful enough but now nurses are given a certain timeframe to achieve their BSN’s so nurses are coming back, some of them unhappily. RNs to get their BSN’s and you’re doing it in adaptive learning platforms and online platforms in the Pathophysiology was adaptive so Julie Hinkle, who taught the course, said, “I’ve got 30-year RN’s who worked in Cardiology their whole career coming into my Pathophysiology course, taking the Cardio unit.” Well, they know more about cardio than I ever will and so within the adaptive learning platform, they simply go to the Cardio unit and test out. They’re done, ok, because they know everything but nurses are funny. If they don’t get quote “a hundred,” they’re not satisfied. So that’s their interleaving, they will go back and test themselves again and again and again and again until they get the satisfaction. High pressure, high pressure in that field. It’s not like beginning Psychology— if I pass, I pass in terms of this. And what Julia’s done is created incredible adaptive learning measurement devices. She’ll give them a series of bloodwork in blood gases that they all have to look at. I mean it’s hard and they all get different values and they all have to assess this patient and the protocol for this patient based on different values associated with and then she’s got them in discussion boards and if the values don’t make sense, the nurses, because of who they are, will go back until they get it. They had the option to do that. In this case that even you talked about her student-driven, you don’t have to force them— yeah, they have the option to do it, some will, some won’t.

Rebecca: Have you been using any open education resources as part of the content for the adaptive courses or are these all closed system faculty you’ve created? Materials…

Chuck: Our faculty, when they begin looking at it, you had this sort of adoption curve, we got the early adopters, who will do anything. When we look at our online courses— we’re very good at online teaching, we are very very good at it. However, we probably have 2,000 courses in our vault and faculty will ask me, “will I be any good teaching online?” Well, my response is, “are you any good face-to-face and if you’re not, you’re not gonna be very good online.” But we range from faculty who do things that are very text-based, faculty members who will not stop putting bells and whistles in their courses. And we say to them, “stop it, you’re being annoying. Stop with the gizmos in the course.” Somewhere, there’s a balance in this kind of stuff and obviously what you have for this preparation of course is a lot of work. I don’t know about you and Oswego, I assume everybody teaches one course and has a lot of free time [LAUGHTER]. We’re not that way at UCF. Our courses are very large, heavy teaching loads and right away, faculty say, “this is the too daunting.” So what I realize is doing in a lot of platforms are doing that are like us. We’re looking at OER (Open Educational Resources) and saying, “what of this can we ingest into this and make it available for a course.” In terms of, “can we take some of the right stuff, put it in there, and load it up so it’s ready to go” and then you can adjust it as we go along. Obviously those courses are good and if we can do that for faculty, it’s a great service because building a course ground up is daunting no matter what happened. And unless your institution provide faculty support, it’s probably too daunting for an individual faculty member to do this by themselves in my estimation.

John: Does Realizeit provide any package materials to help get people started, for example, including OER.

Chuck: Yeah they do, they provides a great deal of resources for it. They have said representatives down when things were not working as well as we’d hope. Basically all of these vendors and I’ll say they have the Veg-O-Matic, we have the platform for them to try it and it doesn’t chop vegetables equally well. That’s the partnership that’s so critical and there’s vendors are not just vendors anymore. They have to be active partners in this educational enterprise. They’re not just selling us stuff— they have to come and help us and Realizeit is very good about that. And others are, too. I’m not pitching Realizeit by any stretch of the imagination.

John: But it sounds like it’s worked pretty well for you guys.

Chuck: Yeah, it’s worked pretty well for us. Our issues are, “how do we scale it, how do we scale it?”

John: What sort of resistance has ever been from faculty? Is it mostly just a time issue or there are other issues that faculty are concerned about?

Chuck: I can probably give you a less of them. One: it’s a lot of work. For all faculty, I think one of the questions are, I’m not sure of the culture and Oswego, but at UCF, it still remains get-ahead equals teaching service and research and we know what of those three carries the great weight.

John: Yes, that’s an issue that’s discussed all the time.

Chuck: So if I’m gonna spend all of my time in preparation of this kind of enterprise, where’s the reward system within the culture of the University for this? And I don’t mean to be crass but what’s in it for me? Why should I do this? And we have to provide some sort of reward perfectly. We’ve done a lot, we have sowed awards that rewards faculty thing, but we tried by giving them course releases and all of the help we can have. We have a large staff that support faculty in the online environment and we have a bank of instructional designers that help faculty members load up their adaptive learning course. And basically our philosophy is if faculty can feel better about their teaching, they’ll follow you anywhere. I mean that’s yet— you know we all want to be the best teachers we can and it feels so good when it goes well and it feels so awful when it goes poorly and we’ve all had that experience.

John: Oh yeah.

Chuck: I am personally— I’m very fond of the quotes attributed to Augustine. I thought I understood it until I tried to teach it. Not one of us has not experienced that at one time in our position. [LAUGHTER] I’ve been talking about things that I’ve had no business talking about. I knew them, but I didn’t know how to teach them.

John: But you learn that as you go. [LAUGHTER]

Chuck: Yeah, oh yeah yeah, yeah.

John: In your recent EDUCAUSE paper on Adaptive Learning, you describe different types of student interaction with a platform. Could you tell us a little bit about that? I remember tortoise and hares and a few other animals in there. [LAUGHTER]

Chuck: Yes tortoise, hares…

Rebecca: Frogs and kangaroos.

Chuck: Frogs and kangaroos [LAUGHTER]— I’ve got to tell you, Realizeit is headquartered in Dublin, Ireland so one, it’s fun to listen to them talk. And two, they’re very bright so in doing this, what we decided, and then we’ve done this a lot, we had a research partnership and you’ve probably known this. If you begin presenting some of this stuff in tabular form and tables, your audience glazes over. In about two minutes, they stopped listening. So the idea there is, “how can we begin to portray this data in a way that’s more engaging to audiences?” And working with Realizeit and Colm Howlin, who was their Director of Research— is brilliant. We’re both big fans of Hans Rosling in Sweden, passed recently, but his approach to showing data in motion…

John: Oh, it’s wonderful, yes.

Chuck: It’s just absolutely wonderful. So we thought, “can we look at this” and what we weren’t able to do is portray the Hans Rosling Gapminder way, is looking at students traversing through an adaptive learning course of Psychology that is showing them in motion two things: what they’re achieving versus the number of activities. And that’s available from us in an animated form and then looking at them as they traverse through there, we have found that they produce certain behavioral styles and we found originally four. And the people in Ireland named this— they’re metaphors. Turned out to be animal[s]. The first one was the hare, and the hare— and remember what we’re plotting: number of learning activities against achievement level. The hare just goes zip. They start the course and they finish it in a very few days. That’s what they do— they’re finished. When I described in Psychology, that’s what happened in Psychology— they’re finished. And it sounds like heresy, but they’re finished. They’re done, ‘kay? And then the second one was the tortoise. The tortoise is the one who goes step by step by step by step by step through the course. They just progressed through the course in little increments and do this. And they get there, but they get there slowly. And then the Frog. The Frog did exactly what they would do in a regular course. There were eight modules in the Psychology course— they completed one a week. They did it week by week by week by week the way they would do it in a course. And interesting enough, when Jeff turned the cord, Jeff Cassisi, he turned the course open. There are a lot of students who did that— they went week by week by week because that’s what they’ve been taught. For them, that’s what of course was: you do it in these increments. And then the kangaroo— didn’t do a damn thing until the end of the course and then did it all in three weeks. Did it all— just zipped to the end of it and it’s fun to watch him in animation. He’s just dead in the water until the end and then he goes zip! And he’s at the end of the course. So there are four different ways that they approach it in terms of doing this and to see that in motion is very compelling. What it tells you is, at least in this course, there are various behavioral styles to the course and you have to be comfortable with them because they’ve all reached mastery and they’ve all reached mastery in some ways we wouldn’t approve of it. We wouldn’t approve of that kangaroo, would we? Doing nothing until the end of the course. That’s not right, but they finished so we have to learn how to deal with that in terms of doing this.

Rebecca: Despite the fact that many academics exhibit that exact behavior… [LAUGHTER]

John: That’s why we have deadlines. [LAUGHTER]

Chuck: Yeah, we have deadlines. Yeah, nothing is quite as motivating as fear, right? [LAUGHTER] yeah yeah yeah, but one of the things that I’ll say (is) there’s a lot of research that’s affecting us. And a really compelling book is called Scarcity, I don’t know if you’ve ever seen it, by Mullainathan, where they talk about students completely compelling to us at UCF for students coming from underrepresented populations in terms of what you think of students living close to the poverty line, what are they dealing with. They’re dealing with so many things in their life. Money, time, danger, single parenthood, finances, family, two part-time jobs, borrowing money from college. Fundamentally, when these students come to us, they’re exhibiting and expending so much cognitive energy just living life. Then when they come to us on campus, they burned up most of their cognitive bandwidth and then we put them in these lockstep courses where for some reason, they miss a class, they’re behind the power curve. We don’t design courses for these students. We firmly believe that they have as much intellectual capacity as any other students but we have not designed our University to accommodate. Adaptive learning is perfect for these students, it really is in terms of doing it so we’re trying to do that and accommodate these kinds of things. No wonder they drop out. In some cases, dropping out of a course becomes the optimal decision and what it is is their superstructure of life is so tenuous that if any one thing fails, this whole house of cards come tumbling down for these students. So they get behind a couple of classes and they’re done and we have to find a way to accommodate this. We’ve got to begin adjusting the way we organize ourselves.

John: What implication does adaptive learning have for the structure of the University?

Chuck: It has a lot. Our students are, now that they’re learning, asking really interesting questions like, “why do we need semesters” and the only answer we have is because we have semesters. [LAUGHTER] We don’t really have good answer for that and we’ve done this an adaptive learning we designed it. Now if you finish early, well you go on to the other course. The other end of adaptive learning is so what if you need an extra three weeks to complete the material and the semester is over? But that’s a nightmare for Financial Aid, it’s a nightmare for the Registrar, but we’re working on that in terms of doing this. But it hasn’t potentially turned on University structure on its head and I’m not sure we’re ready for that at the moment.

Rebecca: When a faculty member creates one of these courses, obviously they’re very involved in that content structure, but what’s their role in facilitating the course?

Chuck: Mm-hmm.

Rebecca: So if the class was to extend three weeks past for a particular student, does that mean then that faculty member is also engaged for that three weeks or what does that look like?

Chuck: Absolutely. In for a penny, in for a pound. If you’re involved with this kind of thing, if a student is still working within the platform and we’ve made agreements with the Registrar that they would get credit for the semester in which they did this, although they went past all of the due dates, a faculty member is still involved. So it changes the role of the faculty member immensely in terms of doing it. Some students (this is gonna come out wrong but), don’t need the faculty member. I mean, they need the faculty member, of course, I need the faculty member. This by no means abrogates the role of a faculty member, but it certainly does change it. Faculty members have to know when to intervene, when not to intervene— it changes us. It’s like teaching online: you have difficulty adapting to this. When I taught an online course, I did have difficulty adapting to my role, not being the center of attention all of the time. It was hard I have a big ego and it was difficult for me but but I got over it.

Rebecca: Are the assessments related to the adaptive learning stuff that’s like automated or something that the faculty member is manually grading using rubrics or things like that?

Chuck: Both, they are both. They get up to the faculty member we have a department who is now just getting over the notion of letting the assessments run within the adaptive learning platform and believing the assessments that it is as competency-based and students exhibited competency. That’s a hard sell for departments. Say a math department who say, fine, but they still have to take the test. You know what I’m saying? It’s a slow-moving thing and what we have done now is an a/b study in terms of demonstrating the fact that students who are assessed within the adaptive prep won’t do as well as students who were in other courses and they took the departmental exam and did equally well. We have to demonstrate that. It’s a hard sell. It’s very scary. It is a very scary phenomenon. Yes, you can design all kinds of platforms and you can build all kinds of intermediate testing devices, you can give tests within the platform and if students do well on the test, it can be, automated is the wrong word, it can be Bayesian decision to cycle them back to where they need to exhibit their skills and you can retest them. So it’s this continual cycling kind of thing.

Rebecca: I have a really easy time imagining how this would work for a knowledge gaining sort of class, maybe an a lower level of course, but a harder time envisioning when it might be like in an upper division class or one that might be more project-based or application based. What’s the experience been on your campus in terms of introductory level classes versus upper level classes giving adaptive learning a chance?

Chuck: That’s a good question. Question posted versus how would you teach Macbeth in adaptive learning? How would you do that? How would you teach clinical psychology in an adaptive learning course? The answer is that probably adaptive learning is not equally well suited for all disciplines or all levels. One is, adaptive learning is really really suited very well for hierarchical structured courses where achieving something at one level depends on achieving something at a slightly lower level. Like sequencing in math, or chemistry, or physics, or computer science. You can do it and we get it beginning psychology but is it really necessary? When you have eight modules in beginning psychology and there is a natural organic order but they don’t necessarily depend on each other. Now, Young came after Freud but, is it really dependent? That’s the kind of thing, I think there are some areas where it’s much better suited, and your question is well taken. I think we have to do a lot of exploration in terms of where adaptive learning is most suited and fits into our curriculum and it may not well fit equally well across all disciplines, all ecology. It’s a question we’ve got to do a lot of work on. Hopefully SUNY Oswego will answer most of those questions. [LAUGHTER]

John: Next semester. It’ll take a little while.

Chuck: Yeah, right. Right.

Rebecca: Do any of the classes involve, that you’ve been highlighting on your campus, have writing as a key component? I’m just curious.

Chuck: Yeah. I think probably in psychology there would be some writing involved in it. They would have to do some reaction papers and do that, yeah. So I think I can comfortably answer that question as, yes, there have been. It’s equally possible. The question is, can you teach creative writing in an adaptive learning course? In some ways it’s equally suited to it because I can imagine you could build a pretty good workshop in create a writing course, with a lot of work, but you can do it.

John: I looked at RealizeIt’s website and it said they create unique formative assessment items based on instructor provided question templates. How does that work?

Chuck: It works very well. In terms of doing it, I’ll tell you one of the things, yeah Ryan Baker wrote a really good paper. The technology is developed really well but the assessment in general is still kind of heuristic, if you know what I mean by that, we’re gonna assess your competency by whether you get four or five items right. That’s the heuristic part of that. You have to design better assessment devices. What we are doing now is we have to transform the assessment paradigm in terms of what they look like, in terms of are they authentic, are they reflective, and are they contextually relevant? Students respond much better to questions that are related to the disciplines that they’re going through, and we have to develop that. We’re nowhere near that, but we’re working on that. They’re very good about helping them. You give them a template, we want to do this, we want to do this in a simulation platform, they’ll help you work with it. They’re very good about doing that, but they can’t do it all. It is a partnership.

Rebecca: We usually hear about adaptive learning in online context. Have you had any experience in a hybrid environment or an in-person environment?

Chuck: Oh absolutely. We’ve taught some blended, adaptive learning courses, and it makes absolute sense. What’s a blended course? What do faculty members, the first time they think about blended, and they think about it incorrectly. What am I face-to-face, can I offload to the online environment? That’s about the worst way to go teaching a blended course, right? The thing is you look at what are the appropriate kinds of things for these two different formats? We have this all of the time where essentially we’ve had a blended nursing courses where they do material content offline and come in to the class and essentially do problem solving. It’s basically is very, very appropriate for this notion of a flipped blended course. We’re actually having a faculty member do it in statics, in engineering, which is really exciting. You know, it’s a whole notion of some of the courses are flipped some of the courses are flopped. [LAUGHTER]. But we sort of look at that and it makes kind of sense. But again, back to your question, is where is this modality most appropriate? Where does it fit? That’s the kind of question, and that’s really a departmental discipline sort of thing. Yeah, every discipline believes they are unique, right? Yeah, their pedagogical issues are unique to it. Yeah right [LAUGHTER].

John: Actually, along those lines we’ve had a number of reading groups here and we’ve had faculty from different departments get together and the faculty who are new to this type of thing expressed surprise at how common the concerns that they had were. To find out that people in other disciplines, very different disciplines, faced exactly the same problems and sometimes had some really good solutions.

Chuck: Yeah, we do this thing, I’m very fond of a concept by the sociologist, Susan Lee Star, she called it a boundary object. A boundary object is something like this, we do it all the time at UCF; critical thinking. We’re really into active learning now at the moment and active learning is another boundary object. We’ll do this say with the Faculty Senate, who’s in favor of critical thinking? Of course every hand goes up [LAUGHTER]. You wouldn’t dare say you’re not in favor of critical thinking. But when you get in a large community of practice nobody agrees what it is, right?

John: Exactly.

Rebecca: Yeah.

Chuck: Exactly. So, Susan Lee Star, this boundary object, is something like that. That it holds a community of practice together, but it’s very weak. It’s not strong enough to be really functional in a large community of practice but you go back to individual constituency, go back to physics, or rhetoric, or creative writing, or education, and they damn well know what critical thinking is in their discipline, right?

John: Right.

Chuck: They do. They absolutely can do it and they’re very powerful. When you bring it back to the community you’re back into the same dog fight and that’s a very powerful concept. I can name literally dozens of them: active learning, critical thinking, online learning, you go on. Very, very powerful.

Rebecca: I find writing is one that bubbles up. What do we mean by writing? What does that look like?

Chuck: Oh yeah. What is writing look like when you’re tweeting— are you writing?

Rebecca: Yeah, exactly.

Chuck: When you’re blogging, are you writing? How do you workshop writing now? Are workshops necessary? It’s all fascinating to me, but I guess that’s because we’re academics. If it weren’t for boundary objects we wouldn’t have anything to do. [LAUGHTER].

Rebecca: You start talking a little bit earlier about the implications of adaptive learning at a university. Can you expand upon that a little bit? [LAUGHTER]

Chuck: Sure. I mean the implications is; what does this say for the structure of the university? What does this say for the way when we’ve organized this enterprise called learning? There’s another great boundary object: student learning, student learning outcome, one of my favorites [LAUGHTER]. Go ahead, define that. My friend, Anders Norberg, from Sweden, we’ve written a paper called, a Time Based Model of Blended Learning. Where time becomes a fundamental design structure of a university and it’s virtually very very different from the way we organize learning at the moment and how we have organized learning is in the sense of discrete units called classes, called units, called semesters, called years and called matriculation period time. It has tremendous implications for that kind of structure. We are employed by all of these kinds of things and we’re organized by all of that so sooner or later we’re going to have to re-examine all of that, if we’re going to adopt these things and adapt these things and be environmental like that. There are three things I think associated with good ideas. One is was the adjacent possible, what’s the next reasonable step, it’s what’s next, what can we reasonably accomplish next, that’s the adjacent possible. Outside of that is the adjacent impossible, you can’t do it, you can’t do it. Secondly, you have to have the slow hunch, you have to stay the course. How many things have we done this but we’re gonna try it, it didn’t work right away and then we dumped it? I’m sure nobody at SUNY Oswego has done that, but we certainly have done that at UCF. You got to stay the course! If you read Darwin’s autobiography, he said “Yet a Eureka moment about natural selection when true”. If you carefully looked at his notes he founded months and years for it was in his notes he just didn’t know it was no Eureka moment, that’s stay the course. If you know this, you got to stay the course because it’ll be bad times, everything’s not gonna work, it’s just not going to be the way. Again, you over expect short term, you under expect long term. Then the third thing is you gotta have this liquid network, you’ve got to work with the vendors, you got to work with faculty, you got to work with administrators. SUNY has to work with UCF that has to work with CTU, that has to work with Carnegie Mellon, that has to work with Oleum Geo Services. You have to have this liquid network where we can share ideas. We can’t do it alone, frankly you can’t do it alone, we’ve got to work that way and if we work that way it’s going to change the whole way we do with the business. I guess the question for us is how much do we want to change? It’s way above my pay grade to change a University and I expect for you too, but sooner or later we’re gonna have to accomplish that there’s a lot of implications for the University.

Rebecca: There’s a lot that I’m hearing you talk about, that reminds me of agile design practice. That’s made a big boom in technology and design. The idea of small little sprints break a big problem into smaller problems that you can work towards. And then it’s iterative, you keep going back and it’s circular it’s not a straight line, that’s what I’m hearing you say. That’s the way that it needs to be tackled.

Chuck: I think that’s a good metaphor, we really be in an agile scrum and keep doing it until we get it right. That’s frankly not been our history right, part of our history is let’s declare a victory and move on to something else. I mean that in the kindest kind of way but we have some serious issues in my judgment. We have tremendous educational inequality in this country, unless we crack that, we’ve got some issues that we’re never going to solve.

John: And you guys are doing quite a bit on all those fronts and I hope we’ll have you back for some future podcasts to talk about some of those things.

Chuck: We’ve made some amazing breakthroughs in communities that you think wouldn’t achieve. The talent pool is as deep as anywhere else, but we’ve got to figure out how to let it up.

Rebecca: I think the other thing that I was hearing and what you’re talking about is the idea of these micro credentials is also surfacing that we had a prior podcast on. I really can see the adaptive learning move at the same pace as micro-credentialing because I think they’re directly related to one another and how the system might have to shift.

Chuck: Kahneman wrote that great book “Thinking Fast and Slow” and kind of admitted in terms of one of our habits is we attempt to solve a hard problem; we try to measure outcomes, we can’t figure out what to measure so we measure something else that we know how to measure. I’ll give you a great example “student learning outcomes”. We’re not too good at that, so we measure grades we measure course success, course success is not learning outcomes, we do an easier thing because we don’t know how to do the hard thing. I do that all the time! [LAUGHTER]

John: We all do that… I think.

Chuck: Yeah, I really love Oswego.

John: We enjoy it too, I’ve been here since 1983.

Rebecca: I came back.

John: That’s right! Rebecca was a student as well.

Chuck: Rebecca we knew you’d come crawling back…[LAUGHTER]… Where does the time go John? What happened?

John: I know, I just got here it feels like.

Chuck: Remember John Lennon? That song “My Boy My Beautiful Boy” that’s what happens when you’re busy making other plans.

John: Yes.

Rebecca: All the talk of time leads to the question of, Mike what does the future hold?

John: What are you doing next?

Chuck: What are we doing next? We’re reinventing the University, that’s what we’re doing. We have a new president, president designate Dale Whitaker who’s been our Provost. Dale is a very big thinker and he’s developing ideas like zero probation, all services, the hub for faculty, the faculty center and all services faculty be located in one kind of place that students become an active part of the instructional process. They’re no longer receptacles, if they become teachers as well we have to do that. They have a lot to teach us, it’s a different kind of world to do this. We have to understand better how our students acquire knowledge, we have to better understand the many generations that are existing on our campuses. I kind of like to get my news from “The Onion”.

Rebecca: hahaha !

John: That’s the best place these days and it’s not that far off.

Chuck: My favorite tagline in the onion is eccentric student reads entire book, we have to begin to accommodate the way students learn and use their learning devices. My graduate students at times will send me off to get a cup of coffee when they fix my technology problem. Technology for them is what Machan of the Reedy called “Living now in the InfoSphere” where information communication technologies talk to each other where we’re no longer in the loop, we’re on the loop. And I think you can understand that in terms of when you look at your Facebook page and what you were looking at on Amazon pops up on your Facebook page, you know that these technologies are talking to each other.

Rebecca: I thought they were reading my mind [Laughter]

Chuck: But the cover of last week’s Economist was “Epic Fail” and it was the Facebook F falling off the fail, lying on its back. The covered this week is AI spy, artificial intelligence spying you on the workplace, we have lots going on. I think the recent things that have happened over the weeks with Facebook give us the fact that we have some serious examination of our culture and our information to confront in the decades to come. I think the future is being defined for us by forces outside of our realm. This is pretty scary stuff for me. Thank you both, you guys are great. I really enjoyed it, thanks for having me

John: We’ve enjoyed this tremendously.

Rebecca: Yeah, it’s really interesting to hear what you’re doing.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

25. Service learning

Posted in active learning, authentic learning, collaboration, higher education, learning, online learning, pedagogy, project based learning, service-learning, teaching
April 18, 2018

Applied learning at the graduate level generally takes the form of traditional research projects, but other models can be successful. Linley Melhem, the Director of the International Teaching Assistant Program at Texas Tech University, joins us to discuss how service learning can challenge graduate students academically while building the capacity of an organization or department to take on a project or tackle a problem. The particular project discussed in this episode involves small teams of graduate students working with faculty and instructional designers to assist language faculty in transitioning existing face-to-face courses to a hybrid format.

Show Notes

Transcript

John: Applied learning at the graduate level generally takes the form of traditional research projects, but other models can be successful. In this episode, we’ll explore how service learning can challenge graduate students academically while building the capacity of an organization or department to take on a project or tackle a problem.

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Today our guest is Linley Melhem, the Director of the International Teaching Assistant Program at Texas Tech University. Her background is in applied linguistics and teaching English as a second language. Welcome, Linley.

Linley: Thank you so much for having me.

John: Today, our teas are…

Linley: I am drinking a beverage that starts with “T,” but it’s Turkish coffee.

Rebecca: Alright.

John: Okay.

Rebecca: I like how you answered that. I’m with you.

John: That works.

Linley: I know, I know it’s important. I know it’s been an issue on your podcast in the past so I tried to meet you halfway.

Rebecca: We appreciate it.

John: So… your tea, Rebecca.

Rebecca: My tea today is Paris tea.

John: My tea is pomegranate green tea.

Rebecca: Although the Turkish coffee does sound good.

Linley: Well, it’s delicious.

John: Could you tell us a little bit about the Masters level applied linguistics course that you co-teach?

Linley: Yes, this semester at Texas Tech, we are offering a course called “Technology in Teaching Second Languages.” We have a group of about 15 masters-level applied linguistics students that are taking this course, and the applied linguistics program focuses on developing pedagogical skills for teaching a second or a foreign language. And this course specifically is looking at how we can integrate technology into that process. The course has been offered for some time, but this is the first semester that we’ve offered it as a service-learning course, and the course has always had some type of applied component and probably would have satisfied the service-learning requirements even beforehand, but we’ve just officially transitioned it into that space. And basically what we have going on in the course is these graduate students working in teams, and each of them have been assigned to a faculty member in our department who teaches a lower-level foreign language course… and they are helping develop some online tools and materials with those faculty members to help them transition those lower-level foreign language courses into a hybrid model. As our graduate students are learning about how to use computer assisted language learning, they’re directly applying that to projects with faculty in our department.

John: Were the faculty originally teaching face-to-face classes or online classes? …the classes that are being converted to the hybrid format.

Linley: Yes, those classes have traditionally been fully face-to-face and in the next year or so, we’re looking at moving them to a hybrid model. Most of those classes are five-hour courses, meeting daily five days a week and we’re looking at transitioning to three hours face-to-face a week, and then two hours online.

John: What prompted the change to a hybrid format?

Linley: Well, I think, like many institutions, the administration is the first to see: “Hey, we think there may be some benefit here not only to making these courses more flexible for students but also there are some other administrative advantages just logistically to that model.” These courses can be really effective and students often have a very positive experience with them. So, in this case, the administration is encouraging all of these basic level language courses to be moved to that format.

John: There was a major study not too long ago that indicated that hybrid classes outperformed both face-to-face and online classes. We can include a link to that in the show notes.

Linley: I’m honestly new to this as well. I’m just learning more and more about the benefits of these types of courses and some of the amazing advantages that they offer especially in the language learning environment and I think that lots of language teachers specifically are resistant to this type of of learning because they feel that while all learning… I think for many teachers… feel deeply relational… language learning especially feels very relational… that you’re creating a culture in your classroom that you’re oftentimes your students’ only connection to the sociolinguistic world that you’re introducing them to… and so there’s a lot of hesitation to remove any of that face-to-face time… and there’s an amazing body of literature that shows that there’s a lot we can do that’s highly effective in an online platform.

John: What are some of the changes that are being implemented in the hybrid format?

Linley: It will look a little bit different for every language in our case, because it depends a little bit on the text that different languages are using. So for instance, in the Spanish classroom, where they have already been using hybrid courses for some time at our institution, there is a wealth of options in terms of materials that publishers make available to instructors, whereas in some other languages like in Arabic, there are not quite so many materials available. So exactly what those changes look like will be slightly different for each language and of course, there’s some choice there for each instructor about exactly what they want to do. But we’re looking at making sure that our instructors are comfortable implementing a flipped model for these hybrid courses so that students are coming into class having already reviewed material that they can use in communicative activities in that face-to-face environment. And I think that’s what’s really exciting about a second language classroom or a foreign language classroom…. that we are always looking to increase the interactivity between students, so when we have the majority of rote-learning that is necessary for vocabulary building and things like that… when that’s taking place outside of the classroom, we can preserve a culture or a feel in the classroom that’s highly interactive from the first minute to the last, every time students show up in that face-to-face environment.

John: What type of assistance are your students providing to those instructors?

Linley: Some of the content in the course that they’re taking is introducing them to specific technology mediums that may be useful for language teaching and language learning. And then they are also working directly with the instructional designers that are available to all faculty in our e-learning program. That’s sort of a unique component… that some of what they’re doing is just introducing faculty to resources that already existed for them but that faculty weren’t sure how to access or maybe they felt they didn’t have time to work with those instructional designers. So, some of what our students are doing in this class… they’re sitting down with faculty, and the lingo that we’re using in this environment is that these teams of students are working with a client. So they’re referring to their community partner who is a faculty member, as a client. So they sit down with their client, and they say, “what are your concerns about moving to a hybrid model? What do you feel like you can do? What do you feel like you can’t do? What would you like to see accomplished by the end of this semester?” And each of those projects look slightly different, which is really exciting and lots of fun, but also certainly challenging because there are lots of different things in the works, but these students are meeting with those instructional designers… and then, in many of the courses, what they’re doing for the faculty is saying, “okay, let me take your existing syllabus and let’s transition this into modules that could be used in a hybrid course and let’s figure out what aspects of your content could be moved to an online format and what needs to stay face-to-face.”

Rebecca: Can you give a couple of examples of some specific things that the students are doing or the specific deliverables for reference?

Linley: Yes, for instance, our students right now, they actually have a case study that’s due on Saturday. So, I’m looking forward to reading those in full, but I’ve just started to look over some of them. So, the chapter that they read and their textbook was about listening comprehension, and some of what they worked on were designing listening comprehension activities using some sort of computer assisted language learning technology. So, for instance I believe students that were working on an Arabic course, they were taking some content that was based around learning terminology related to the weather, and so they took a video that was available online that was a weather forecast in Arabic… and so they developed audio recordings of the instructor who is describing this terminology in Arabic so that the students can get an ear for it in that simplified format before they then went and listened to an authentic weather forecast. So, material created for native Arabic speakers… not necessarily for Arabic learners… and then the students designed a quiz where the language learning students would be asked to identify which of the vocabulary that they had already learned were present in that weather forecast. So this would be a listening activity where they were listening for vocabulary that they had already learned the meaning of in an authentic setting. So that would be an example of an activity that an instructor could have students complete before they come to class where they did something interactive talking about the weather… they would first maybe do a listening activity like that online.

Rebecca: You can see how valuable it is to have these masters level students helping fill some of those gaps for your faculty just because it takes a long time to sift through the materials, find good examples, so that they have those good authentic experiences.

John: Has that eased the transition for some of the faculty who might have been apprehensive about moving to a hybrid format? Does the support that your grad students are providing make it a bit easier for them?

Linley: I think it has. I think also because faculty many times feel “oh, just by the nature of being a little bit older than the graduate students or even the students that I’m teaching. I’m inherently at a disadvantage. I’m not familiar with this type of technology.” But, we we know that actually graduates, and many undergraduate students, even if they’re interacting with technology on a regular basis… they may not be so savvy for using it for educational purposes. So, I think even that… lowering that barrier a little bit to show that actually these graduate students are having to learn how to use this technology as well so it can be done. So just watching someone else learn in front of them makes the whole thing a little bit more approachable and then certainly having some support, even just in someone else saying, “hey, I’m already dedicating some time, so I’ve developed a few activities.” And I think oftentimes instructors see that kind of gets the wheels turning to them and they say well I can do that, that’s not that complicated and I could replicate the same style of activity for number of content areas and so it makes the whole process much more approachable.

Rebecca: Sounds like a really sneaky way to do professional development to me.

Linley: Yes, that’s a really exciting thing… and that is one of the great benefits of service learning in general is that our graduate students are developing some wonderful skills in working with a client. So they are essentially material designers for a client and they are required to communicate with the client, to organize their schedules, and coordinate time. And one of the first things that we did in class was even talk about how to have a meeting with someone and how to deal with faculty that may have a lot of resistance to developing these types of materials or have great concern. And even some professional communication techniques about how to approach those meetings. So there are so many wonderful things happening at the same time.

Rebecca: Sounds really great… it also sounds like there’s a lot of moving parts. Having taught classes where there’s a lot of clients in the past, I know that that can be really complicated to manage and oversee. Do you have some strategies that you’re using to help everyone stay organized and to keep yourself organized. What’s your role in this project?

Linley: Yes, so as I mentioned this is the first time that we’re offering the course in exactly this format. So you’re hearing a very live perspective on how we’re figuring out how to manage this. But one advantage of the course is that I am co-teaching this course with Dr. Stephanie Borst and she has taught this course for years and has had great success with a number of different practical projects that they’ve taken on. I have been working on developing service-learning courses in our department, so that’s how I became involved when we decided to move it to a full service-learning model. And the advantage is that because there are two of us, that we can manage some of these projects. There are a lot of moving parts. We also probably would not have had so many students… we have 15 students in this course… we probably would not have taken on so many if there weren’t two of us. But, in this way we can serve a greater number of faculty members. But I think one thing that has been crucial is helping students develop an action plan at the beginning of this semester that they continually update. And because they’re all using a relatively standard format for an action plan… we provided a template but actually all the groups ended up developing a slightly different format, but because the format is mostly similar, we can sit down in class– and our class is actually a hybrid model as well, so we’re only meeting half-time face-to-face and then the rest of the time online. So when we do sit down face-to-face with our with our students, we can look at their action plans and get a sense of where they’re at and how they’re moving forward. And so having the ability to get a really quick snapshot of how they’re progressing I think has been key to providing feedback to them and helping them manage their relationships with their clients.

Rebecca: Is your action plan format something you’d be willing to share with our listeners?

Linley: Oh yeah, certainly.

John: Okay, we can put that in the show notes.

Rebecca: I think sometimes starting projects like this can be really challenging because you don’t quite have an idea of how to get organized and seeing models of how to stay organized is always really helpful.

John: And that relate back to the teachers making a transition to teaching hybrid, that there’s this big psychological barrier to trying something new but once they get started it’s not so bad… but you have to get through that initial thing. And it sounds like what you’re doing there is making it a little bit easier in the same way that this document perhaps might help other people thinking of doing the same thing.

Linley: Well we certainly hope so.

John: How have the faculty been reacting in terms of the tools they’re seeing. Do they see the usefulness of some of these online tools? And what are some examples of the online tools that might be used? You mentioned the project in terms of the weather reports and so forth, but are there any particular online learning approaches that you’re using that the faculty might not have considered before?

Linley: Yes, we started from a very theoretical standpoint in this course so we’re just now getting into some of the nitty-gritty of the actual tools that can be used in this environment. The faculty are reacting well. I think they are encouraged that they are receiving some support and getting some help and just because these students are kind of helping them get started, and even introducing them to (like I mentioned) some of the tools that already existed for faculty through instructional designers at our university, they’re saying “oh, okay there actually are templates to help me throughout this process, I can even online find something like a course design plan that helps me develop my material into a set of modules.” And it’s not that different from developing a syllabus, which most of them have done in the past. And so then they’re seeing some things like students may be introducing something. Most faculty are familiar with a discussion board for instance in an online course. However, they’re not sure how students will be able to practice maybe speaking in the target language, and then may see something like Flip Grid where students could essentially post a video of themselves and they say “oh, okay… so students can do speaking practice outside of the classroom…” that’s not something that we would lose in using a hybrid model.

Rebecca: VoiceThread would be another really great tool if you haven’t explored that one yet.

Linley: So I recently heard about that on other podcasts but I have yet to check it out myself.

John: FlipGrid is very similar, I believe, to Voicethread.

John: Yeah, I was thinking something like VoiceThread or FlipGrid would be a really good online approach. Have they done any other direct interactions online — with other native speakers, for example?

Linley: Well that’s an interesting idea because that is actually something that many faculty members are already facilitating in their face-to-face courses. They are connecting learners to native speakers in various countries across the world, but that’s typically on a at-choice basis, so maybe for extra credit or just for students that are highly motivated. So I think instructors are seeing that they’re actually already using some techniques that could be more fully integrated into a hybrid course in a way that would be really beneficial for all students. So, there are some really interesting literature about the benefits of that type of approach. Obviously you run into issues especially because we’re talking about at this level, lower-level language courses, so these are students that would really be struggling to communicate at a very basic level. But there are some opportunities for them to connect to native speakers in the countries that speak those languages that are really exciting and that tend to really motivate students to learn and engage in more extensive language learning like study abroad.

John: One of the things we do in SUNY is… we have something called COIL which is Cooperative Online International Learning program. Where courses in the U.S. pair up with courses in other countries. In the U.S. most of the course end up being taught in English because most SUNY students don’t have as much of a background in foreign languages, but many of the partner schools are doing it primarily to help the students acquire English skills. And I was thinking if you were doing some upper-level courses something similar could work in the other direction; where if you had more advanced language students working with students on projects dealing with culture or cross-cultural comparisons… might be an interesting sort of pairing.

Linley: That would be phenomenal.

Rebecca: So it’s really unusual to hear about service learning at the Masters level and you mentioned that this was the first semester that you were doing the service-learning component with this course. Can you talk a little bit about that decision and what motivated you to use this particular methodology?

Linley: Yes, as I mentioned I had the opportunity to participate in a service-learning fellowship about a year ago. And I was initially looking at developing some service-learning courses in our department for undergraduate students. And honestly the idea came about as I was listening to my colleagues discuss some of their concerns about implementing a hybrid model in their courses, and so I knew that this technology in teaching second language course existed and I knew that many actually of the teaching assistants in those foreign language classes were enrolled in the applied linguistics program. And so many of them took that course and I thought well, we have this group of students that’s developing this knowledge… we have these faculty members who are needing some support and this type of knowledge… why couldn’t we just put these together. And so there were obvious gains, like you mentioned Rebecca, in terms of the professional development… for the Master’s level students to get some practical experience, so it seemed like a no-brainer to try and put those together.

Rebecca: Related to that in terms of a professional skill for graduate students… I can imagine that it would be really easy for their clients to want this project to just get bigger and bigger and bigger and have crazy scope creep.How are you making sure that these projects don’t get too big?

Linley: That’s exactly right, and we are facing that issue… and part of the problem is that because the faculty are not familiar with exactly what’s involved in transitioning to a hybrid model, they don’t sometimes know what they’re asking for or how time-consuming certain tasks would be for the graduate students. I think that is one of the great outcomes of the course… that the students are having to learn how to negotiate that with a client. These are our faculty members in our department they are clients but the students are having to say, “Wow, that sounds like a great idea. I think what we could definitely do for you this semester might look a little bit more like this which is a bit more narrow in scope, our goal would be to provide something that’s really helpful to you but we may not be able to accomplish all of that this semester…” which is challenging in terms of professional communication. But I think one of the really important aspects of that is making sure that our students know how much we expect of them in terms of that they are well-informed about how much time they should be spending on this type of task, and that is something that we’re having to continually negotiate. And we have had some students take on too much and they have had to go back and say “okay, we may not be able to do quite that much….” or they’ve met with an e-learning course designer who’s accustomed to working with faculty on a really tight deadline and so they said “okay, why don’t you go work on this piece let’s meet again next week,” and you have this big chunk of work done and the students aren’t only doing the service-learning project, they also have coursework related to this course and so they’ve had to say to the instructional designer: “Actually, could we meet in two weeks instead?” So they’re figuring out some of those professional communication and time management issues in managing the scope of their own projects, which has been highly beneficial. But there has there has been a lot of back-and-forth negotiation and that is something that my co-instructor and I are observing and as we look at those action plans that’s something we’re talking about… are you biting off more than you can chew? And how can we figure out how to integrate what you’re already doing in the course into the deliverables for your client to make sure that we’re not overwhelming our students with too much.

Rebecca: I can imagine that in this situation having a co-teacher could be really helpful to bounce ideas off of each other, but that also is another layer of complexity. I’m wondering how you’re also managing that… to make sure that your collaboration with Stephanie is also running smoothly?

Linley: Yes. So she’s at a disadvantage because she’s not here to see the results so I’ll speak for both of us. But I think it’s going quite smoothly, I wasn’t sure what that would look like initially. We’ve never worked together in this capacity before and I’ve never co-taught a course before, so I had no idea what that would look like. However, because this is a hybrid course and a lot of what we do face-to-face is more in a workshop type setting, I think the co-instructor model works quite well because we’re not really lecturing to the students or there’s not a concern about making sure that we’re on the same page because she and I can have lots of those discussions between the two of us as we prepare content that will be put online… things like that… developing rubrics… those kinds of issues. So I would say, one issue for instance that came up is even ensuring that we’re both interpreting a rubrics that we’re using the same way because we take turns may be grading certain types of assignments, so wanting to be consistent in the implementation of those rubrics. But because a lot of that communication is happening via email or over Blackboard, then we can see how the other person is responding to those types of issues and so anything where it seems like we’re not on the same page, it’s been pretty simple to iron out, outside of that face-to-face environment. But it’s honestly been much smoother than I thought it might be. Stephanie is fantastic to work with but I really thought “I’m not sure what this will look like,” but it’s been easier than I thought it might be it.

Rebecca: Sounds to me like it in some ways you end up learning a lot more about your colleagues and how they grade and what they value by co-teaching with them and then at the same time in this particular situation you’ve got two people to put out fires.

Linley: Exactly, and I think that at first maybe the students weren’t sure what to make of having two instructors… that they weren’t sure whom to go with with concerns and things like that. But as I mentioned, if we’re having these conversations over email then they just copy both of us and whoever responds first then the students I think seem to like that model because they probably tend to get a response a bit quicker than if it were just one of us. And then also I do think we develop our own areas of focus, so I am more leaning towards management of the service-learning project and Stephanie is most familiar with the content of the course. So while we both speak into both of those things we kind of have our areas of expertise.

JONE: How many students are working with each instructor? How big are the groups?

Linley: So the groups are different sizes, our smallest is two people… so, actually we have two groups of two that are working in different environments. I will say one other unique thing about this course of that our group of students is highly diverse. So we have lots of international students in the applied linguistics program, so they speak lots of different languages. That’s a great advantage because as they work on the materials for these different foreign language classes, they may have a great deal of knowledge about that language. That’s also kind of spoken to how we divided those groups up. We do have a couple of groups… for instance, we have two students who are helping develop materials for a German class and neither one of those students speak German, but they’ve had great success in the instructional design component. So, that’s another challenge that has arisen in this particular context. But then we have another group of four students who’s working on a project. And so you asked earlier about scope, the size of the group, and how many people are contributing also influences how great the scope of what they can take on is.

John: What benefits do your students get from this type of class format… the service-learning and the hybrid nature… that they might not have received in a more traditional class setting?

Linley: I think one of the greatest benefits that they are getting out of this setting is in working directly with a faculty member who intends to actually implement these materials with students, is that they are giving a sense of material design that’s not only evidence-based but constrained by the real-world environment. The students are applied linguistics programs tend to get lots of wonderful information and lots of great ideas about best practices for teaching a language, but they may struggle with gaining a sense of how to implement that in only a 50 minute face-to-face class. So, those are some of the real-world constraints that that are ironed out as they work with a faculty member who has tons of experience working with real students in the real classroom. So, if the student designs this activity that’s elaborate and meaningful and evidence-based and wonderful, but it would be way too time-consuming for students to actually accomplish, or maybe it would be too advanced for students at this level, which graduate students may not have a clear sense of exactly what that would look like. Then the faculty member is saying, “ I don’t think my students could do that or this would take way too much time.” So it’s building in an awareness of some real-world constraints that may not be so evident to our graduate students otherwise. And then additionally, as we mentioned earlier, they’re developing some of those professional skills that they would never otherwise be able to develop. They’re working on communicating with a client, they’re working together in a group, they are negotiating roles… all different kinds of things that we tend to face when we enter the workforce in general.

Rebecca: Great.

Linley: One thing that I really love about service-learning is the emphasis on civic engagement and the awareness of diversity and different types of issues that come up in the real world. And I think that it’s interesting to see how our students are becoming more sensitive to the different types of students that we have at Texas Tech University and their different experiences of the college classroom. There are different experiences of technology, there are different aspect of resources, so I am excited to see how in this service learning environment students are becoming more aware of who student populations really are and to some of the diverse challenges that face those two populations. I think that sort of awareness raising is really exciting. And then additionally, I like the idea that students will be graduating and entering the workforce with this idea of cooperation, because they’re working together as a group and they’re working with all of these faculty members as opposed to moving into an educational environment, where we often have a tendency to work in a silo. They’re having some experience bridging those gaps and reaching across the aisle and saying “O kay, what are you doing here? How can we use those strategy” in the areas that we’re trying to operate. So I think they are walking away with a greater sense of cooperation, but I hope they will carry into the institutions where they either continue their graduate work or are working as professionals.

John: One thing we have to ask is about your podcast. What started you on the podcast? I see you’ve got a pretty big audience there in terms of the number of downloads for the podcast. Could you tell us a little about it?

Linley: Yes. I wanted to start a podcast because I love podcasts, I really enjoy listening to them, they are a big part of my personal learning and they’re one of those things that I find the more I listen the more creative I feel…. that I’m just exposed to lots of different ideas. And I started looking around for English content that would be useful to some of my other students. I also teach English as a second language mostly for graduate international students who will be teaching in their various content areas but using English as the mode of instruction. And so, what I realized is that there are obviously tons of podcasts in English but some them are pretty… well they’re definitely designed for native speakers, so there’s no support for language learning or they’re designed for people who are very early language learners… so, just focusing on lots of vocabulary building. So I noticed that there was a bit of a gap there in terms of something that was designed for intermediate or advanced speakers of English, but with just a little support for language learning. So I thought let’s just create it… let’s try it out. So that’s what we did and I think like these projects that we’re describing the exciting thing about something like a podcast is that you really can dive in with not a lot of experience or complicated resources. So most of the episodes that we have on the podcast are recorded on my iPhone, and I’ve had family members on the podcast, we’ve had different individuals from around the university, and the students in my classroom have responded well. I’ve been able to take some of the content that we were developing for that podcast and use it in my classroom, which is always exciting when you can get double use out of any project that you’re working on. And we did have we have seen a positive response internationally where it seems like people all over the globe are excited to have this type of content. So at its height, we had a good number of people listening in Benin, in Africa, and I have no idea how they found out about it, but we had quite a following there for a while. And I’ve taken a bit of a break in producing content as I’ve focused on some other projects, but I have been looking into how I can make use of some other resources on campus in terms of maybe having an intern or developing some type of service-learning course where students could help me, especially on the technical side, because I don’t mind talking, I don’t mind conducting an interview, but the editing is more time-consuming than I would like.

John: We have noticed that too.

Linley: It’s remarkable.

John: Your podcast seems like a great resource for graduate students because you deal with a lot of topics like how to understand slang or Texas accents, for example, or in similar topics. For grad students who’ve learned English formally in their countries… coming to a new institution… coming to a new country… it might be helpful for them to fill in some of the gaps that might not otherwise have been done in their instruction. I was I was really impressed by it.

Linley: Well thank you.

Rebecca: What also seems nice about a podcast is that if it’s a gap in their knowledge, but they don’t want people to know that it’s a gap in their knowledge, you can listen to a podcast without anyone really knowing. So, you can fill those gaps easily.

Linley: Yeah exactly.

John: You could be listening to it at the gym… while driving… while walking…. or when you’re sitting at home.

Linley: That’s exactly right.

John: We normally ask as the last question: what are you going to do next?

Linley: Oh, well, that’s a great question. So I’ve mentioned that on a personal level I’m expecting a baby soon, so that has taken up lots of head space in terms of what I’ll be doing next. I’m not sure how my personal life will be changing but professionally, I am definitely interested in continuing to examine ways that service learning can be used in the classroom. So I would love to see me in my ESL courses (English as a second language courses) see ways that international graduate students can be contributing meaningful service to our community while learning English. And I could see lots of amazing ways that could take place. Our international students on campus are usually here because they are so bright. They have a lot to contribute to scholarship and research… In general. But, oftentimes, as they struggle to communicate in English at the same level as a native speaker they’re often underestimated. So I think if we could look at ways of incorporating service-learning courses where students were learning English and then contributing some of the things they’re really great at doing, it would have a wonderful impact on our university, our community, and international students. So that’s one thing I would like to look at developing and certainly getting back into the podcast game. So as I mentioned, I haven’t produced new content in a while so I would really like to to get back into that, to come up with some new ideas for how we can contribute to English learners all across the globe.

Rebecca: Well sounds like we have two different we have you back to talk about later.

Linley: I would love it.

John: Well thank you very much, it’s been a pleasure talking to you

Rebecca: Yeah it’s really great hearing about what you’re up to and and how it’s coming along.

Linley: Well thank you so much for having me and I have really enjoyed listening to your podcast. I found the episode on online teaching especially relevant to things that I’m working on and thinking about these days. So, thank you so much for for all that you’re doing.

John: It’s been a lot of fun.

Rebecca: Yeah, thank you.

John: Thank you.

Linley: All right, thank you.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

23. Teaching with comics

Posted in active learning, collaboration, comics, inclusion, interdisciplinary projects, learning, mental models, OER, pedagogy, teaching
April 4, 2018

Looking for ways to increase student confidence in their ability to learn? Or their ability to see themselves as professionals in the field? In this episode, Carly Tribulli, a Biology Professor at SUNY-Farmingdale, joins us to discuss how comics may be created and used to meet students where they’re at, draw them in, and help them develop mental models of complicated processes and concepts.

We discuss Carly’s plans to create an OER biology textbook in which biological processes are represented using comic strips, her planned research on the effectiveness of instructional use of comics, as well the positive role model that she provides in Carly’s Adventures in Waspland, an instructional comic that Carly created for the American Museum of Natural History during her graduate study there.

Show Notes

Carly’s Work

Topics mentioned in the podcast (in order of their appearance):

Economics comic books:

STEM web comics recommended by Carly:

Transcript

Rebecca: Looking for ways to increase student confidence in their ability to learn? Or their ability to see themselves as professionals in the field? In this episode, we’ll explore how one faculty member uses comics to meet students where they’re at, draw them in, and help them develop mental models of complicated processes and concepts.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Carly Tribull, an assistant professor at Farmingdale State College, where she mostly teaches general biology for non major students in entomology. Her interests include bugs, biology, and of course, comics. Welcome, Carly.

Carly: Hi, nice to meet you guys.

John: Welcome. Today our teas are…

Carly: I’m actually drinking… a kind of cold coffee. But, but it’s good. I like it.

John: Yeah.

Rebecca: …and it used to be warm.

Carly: It used to be warm. I got it about an hour ago, so I knew this was going to happen, but I was like “You know, this is my only opportunity to get coffee, and I know you guys like to talk about what we’re drinking…”, and I was like “ooh, yeah a coffee, cool… I could have lied…”

Rebecca: That’s true. I have a Paris tea.

John: and I have blueberry green tea.

Rebecca: So, Carly, can you tell us a little bit about how you’ve been able to combine your interests in art and biology in your educational and career paths?

Carly: So, I’ve always been interested in both art and biology ever since I was a little kid. I grew up in a very science-forward family. There was a lot of interest in me becoming a biologist and my parents were both very encouraging, and my dad always sat and watched those sort of Wild Animal channel, Discovery Channel shows when I was a kid, with all like this farming animals and stuff like that. So, I was always interested in the animals and eventually that led to drawing animals. By the time that I was in high school, I was taking formal training in art and doing AP art and things like that, but also very much maintaining my biology education. By the time I was later in high school, I was drawing comics. I had discovered comics around early high school. I read a lot of manga, and then I started reading more graphic novels, never a lot of the superhero comics, but more of the weird offbeat stuff like the Sandman, and a bunch of manga series. So I started drawing comics, and I drew a bunch of weird comics and then I entered college at UC Berkeley, and I was a double major in art and biology, and I just continued that path all the way through. And I was really stubborn about not giving up art, despite the fact that I had chosen not to go to a traditional art school. I knew at that point I was going to go into biology, but I was very much stubbornly holding on to art, and so what happened when I was at Berkeley, is that I was actually able to do biological illustration as an undergraduate researcher. And that was the very first research experience I ever had, doing biological illustration for a paleontology lab. This has always made sense to me as a biologist, because there’s a really, really huge history of biology and art meeting together. Especially in entomology, when you consider the work of Maria Sybilla Marian, who is one the famous female entomologists of her time (probably the only major female entomologist of her time) and she was really the first person to study metamorphosis. And much of the way she shared that information, since this was obviously way before photography, was by these really elaborate illustrations that were shared with other entomologists at the time. So to me, it’s always made sense that there is some sort of crossover between biology and art, and I think while I was in college I was very stubbornly imagining myself as becoming that type of natural historian. And then when I was in graduate school there was a lot of encouragement for me to continue doing comics, weirdly enough.

John: Could you tell us a little bit about your graduate program?

Carly: I went to the Richard Gilder graduate school at the American Museum of Natural History, and that’s a pretty long name, but historically the museum has always funded graduate students from the City University of New York, and from the Bronx Botanical Gardens, and from NYU and Columbia, but only within the past seven or eight years or so, did they decide to start their own in-house PhD program. So, we still have all of those students that are coming from other institutions, but only recently where we like, we’re going to create our own graduate program. It was very, very, very different from your standard evolutionary biology PhD program. Usually the big state public schools, and a few of the private schools that are strong in the sciences, have an evolution in ecology, biology grad program that you spend five to six years and that you TA undergraduates to support your stipend. But at the AMNH, because it’s a museum, there are no undergraduates for you to TA. and you also have to finish in four years. So, because you had no formal TAships, and the funding was very good so you didn’t really need them, you were very much encouraged to do these informal teaching assistantships, and to find your way into the outreach education side of the museum, or working on exhibits and making yourself part of the contributing community to the museum. That is basically how the grad school ran, and I did my PhD in the evolutionary systematics of these parasitoid wasps that I study.

John: It sounded like a really natural blend of your interest and a superb educational path for you, in terms of giving you a way of continuing your earlier interest.

Rebecca: Before we jump forward I’m really curious, Carly, as an art faculty member, if you could talk a little bit about that first project, that first opportunity you had as a student and how you got that opportunity to combine your interests. Was it something that you pursued or was it something that your faculty helped to nurture?

Carly: Kind of a combination of both. My freshman year at Berkeley, I took an undergraduate symposium with Kevin Padian, who is a vertebrate paleontologist, and it was very much your standard freshman seminar. It was actually very small, it was only about 10 students. We did some readings, we did some talking, and around that time I think I was looking for research opportunities, and so I started talking with him and I started trying to get myself into the lab as an undergraduate researcher for future semesters, and it came up that I’m a biological illustrator, or that I was interested in biological illustration, and I think at some point he was like “okay, show me what you got, go draw the T-Rex,” because there’s a big T-Rex in the center floor of the Valley Life Sciences Building at Berkeley. And I went down and I drew it as best as I could and apparently he was pretty satisfied with my work. So, I joined the lab, and I was assigned to a current PhD student at the time named Katie Brakora, and I actually drew some of the images that were used in her dissertation. And that was excellent. I didn’t become Kevin Padian’s biological illustrator, but I was working with grad students that were going through grad student life, finishing their work… and at the same time I was taking the core art classes, because I was a double major and I knew I was going to be a double major for my freshman year. So, I was doing all of your standard intro to drawing, intro to painting, techniques classes, and things like that and it actually worked out really well for me to be a biological illustrator, as sort of a side biology undergraduate researcher, because Berkeley’s art program isn’t really focused on illustration or comics. It’s actually much more of a fine arts program. So, sometimes I was actually butting heads with the other art faculty, because I was very illustration focused and they’re very studio fine arts, and I was like not all of us are going to become studio painters. So, illustration seems like a skill that I should be investing in.

Rebecca: What a great story. Thanks, Carly.

John: While you’re in grad school, one of your projects was developing Carly’s Adventures in Wasp Land, and we’ve looked through that and it’s superbly drawn and fascinating. Could you tell us a little bit more about some of your work with illustrations and developing comics while you were at the American Museum?

Carly: I guess this goes back to how Carly’s Adventures in Wasp Land started, which by the way is not the title I came up with it, that was the title that the museum folks came up with it, I was just like, “okay.”

John: Did you have a title?

Carly: No, I did not have a title, that was probably an error on my part. I was opening myself up there, I think it’s a fine title. It’s a little bit goofy that it has my own name in it, but, whatever. In my interview to get into grad school, I had actually brought my portfolio in biological illustration, which was very unusual. Of course, evolutionary biology does attract people who can draw, but I think I was the first person who had come to that relatively new program with a portfolio. [LAUGHTER] I was kind of a scrappy undergraduate. I didn’t do that great in my courses. I’m a terrible memorizer, which allows me to sympathize with other students that aren’t doing that great in intro biology, especially my own students, because I actually didn’t do all that well for the first two years. And part of making myself an attractive student to graduate schools, was actually building up my research curriculum. I did a lot of research with Marvelee Wake at Berkeley after the Padian lab, and then also building up this biological illustration thing early on. I interviewed with Jim Carpenter, he accepted me to his lab, and I think he was very impressed with the fact that I did illustration and apparently it stuck with him enough that when he got a grant from the NSF, he came to me about helping him out with the broader impact section of that grant, and broader impacts is where you actually have to make your grant meaningful outside of academia. So, it’s where you would have outreach education. He remembered from my interview that I like to draw, he came to me and he was like “do you want to work with the digital outreach education side of the museum, and create a project with them? “And I was like “yeah, sure,” and as long as it was about teaching kids about wasps, and the different types of wasps, I pretty much had free rein. I started working with Ology, which is the digital outreach section of the museum, and a lot of what would happen is collaboration between me, Jim, and the Ology folks, especially when it came to writing the script for that comic, because the Ology folks have way more experience in writing for middle school readers than I did. So there was a lot of modification of my script but mostly I had free reign when it came to the illustration side of things, and I also mostly had free reign when it came to the creative decisions, like the decisions to make the wasps anthropomorphic and have them talking with you, that was something I decided on, even though it isn’t truly a hundred percent scientifically accurate. It was something that both the Ology folks and Jim signed off on.

John: I liked it.

Rebecca: I thought it worked well for adults too, I don’t think it’s just for middle schoolers. I’m just saying… I know way more about wasps now than I did before I read it.

John: Me too, and it was much more engaging than reading a textbook description of those things.

Carly: Thank you so much!

Rebecca: I also just really love that you’re like a superhero in the story. What a great way for little girls and boys to see a strong female scientist… taking on the wasp. I just thought it was a really great way to frame the story.

Carly: Yeah, and I think the first chapter in Carly’s Adventures in Wasp Land doesn’t actually talk about wasps but it sort of talks about me and how I became an entomologist. That wasn’t part of the original plan, but me and the folks at Ology, and eventually Jim was totally on board with this, felt that it was important that part of the broader impacts, should be showing young girls that they too could be an entomologist, this field that is commonly associated (at least by other people who are outside of entomology) as being male-dominated and being a career for boys… showing them that, that’s not necessarily the case. So, that’s when the strengths of comics especially when it comes to showing girls and underrepresented minority students that they can envision themselves also as scientists. That’s one of the things you can do with comics that I find really engaging… is that, in your choice of narrator, you can make those decisions.

John: I believe you’re releasing some of your materials under an OER license. Is that correct?

Carly: Yes, not Carly’s Adventures in Wasp Land. That is an OER in that it’s freely available, but it’s going to stick with the museum’s website for the time being (as far as I know). What I’m putting on an OER license is actually the comic textbook that I’m going to be eventually making for the Farmingdale State general biology students, but it’s certainly going to be available to any SUNY professor or any professor anywhere.

John: Have you requested an grant for that or are you doing this on your own?

Carly: So, I’ve requested a summer stipend and I should be finding out about that soon. As you might guess drawing and writing comics takes a lot of time, much longer than say a written textbook would take, and there are certainly many professors that are working on written OERs for their class. So, I’ve requested a summer stipend and SUNY Farmingdale has recently announced that there’s going to be an OER incentive grant, so I’ll be applying for that too.

John: Excellent.

Rebecca: You’ve also done some writing about using comics for science specifically, can you talk a little bit about the research that you’ve done in this area?

Carly: Yeah, so I think when I accepted the job at Farmingdale, I knew that I was going to be very, very, interested in making comics and researching the impact of comics… part of the research that I do for my tenure decision… and luckily the faculty here have been very supportive of that. Farmingdale is a primarily undergraduate institution, so there’s actually lots of professors that are also not only researching their scientific field or their artistic field, but are also researching educational techniques in their field. Part of preparing for that was actually some work that I did last year. I was actually invited to an open access issue from the Entomological Society of America, on educational communication in the sciences …and they had known over the years, because I kept presenting on comics, that my interest really lied in the use of comics as outreach education. So, I began actually searching through the literature because this was something I wanted to continue doing as a professor once I moved to Farmingdale, and it was also something that I just wanted to continue just as someone who was going to keep making educational comics regardless. And so what I found in doing this big review paper called “Sequential Science” is that there is much research in how comics impact the interest and attitudes towards the material, at students at a variety of levels, but there isn’t so much research in actually measuring their gains in content knowledge. So there’s lots of research to show that comics makes students at all levels more interested in the material, but not a lot showing and quantifying how much more they’re learning and retaining. So, I think that’s an area that I actually want to put more research into myself… but yeah I spent a lot of time for that paper reading a bunch of other papers about studies that had been conducted.

John: Have you started this research or is this a plan for future research?

Carly:This is definitely a plan for future. So the development of the OER textbook for gen bio is just happening right now, and anecdotally I’ve certainly seen students are more interested, so I do incorporate comics into my slides right now. They’re not my comics necessarily, they’re comics from a lot of different sources like Beatrice the Biologist or Your Wildlife, those are popular webcomics that are biology focused. I also make some drawings for them for the slides as well. In reality, any comic is just a set of sequential images. So, I can draw a set of sequential images that are explaining mitosis and meiosis. My students might not necessarily read those as comics or recognize them as comics, but they’re still comics because they’re telling an ordered set of events. So when I do that, anecdotally, I can tell you that the students are more interested… especially if there’s just been a slide with the textbook image and some complicated information, if I can show them that slide and then be like “oh let me break it down into these steps that I’ve drawn out” it seems to help them. But have I actually started measuring the impacts? No, not yet.

John: So do you have a research plan on that?

Carly: Yeah, so as the OER textbook is going to take some time to make. It’s probably going to take a couple of years to finish in its entirety, but there’s no reason that I can’t start exposing the students to the chapters as I complete them. So, until the OER is finished in its entirety, and given that I usually teach multiple sections of gen bio, I’m going to start setting up testing control groups just looking at small chapters, as I complete them. So, one class will receive the comics, the other class won’t receive the comics, and since both classes have the same test, I can actually see if there’s any improvement. Now, once the comic is finished in its entirety, that’s when I’ll actually begin the full-scale research… and what’s going to happen there is… again I teach multiple sections of gen bio… I can set up a test group and a control group. The test group will get the comic textbook and then the control group would get a traditional OER (probably the OpenStax gen bio textbook) and I can give them the whole textbook at that point and measure what their differences are in terms of performance using their midterms and their quizzes and their homework assignments. But I also plan on surveying them on interest, because although the interest and the attitudes might not seem as strong a topic as actual performance, I think when you’re teaching non major biology students (many of them who feel like they’re just there to check off a box), many of them who have prevailing biases against science… many of them who don’t feel like they can connect to science… I think it’s so important to measure those attitude changes.

Rebecca: Why does a sequential format work so well for a topic like biology? What do you see the benefit of being sequential in that way? This sequential art form.

Carly: So even in general biology, intro biology for non-majors, there’s still lots of processes that are multiple steps. So, I don’t know if either of you remember learning the Krebs cycle or photosynthesis. These are very complicated multi-step processes where something has to happen and then there’s a result… and then another thing happens and then there’s some sort of result. So, there’s plenty of stuff, even in the entry level biology classes, that lend themselves really well to a narrative. Comics really are any progression of images that build a narrative… now, that narrative doesn’t have to be fiction. The point is that there’s an order of events and together that order of events makes sense. You actually don’t have to add words for it to be considered a comic, but obviously the words help in the context of a biology class. I think given that there are so many multi-step processes whether you’re studying the Krebs cycle… or photosynthesis… or mitosis… or meiosis… or even natural selection or ecology… sequential comics… so these images, where you have processes that are laid out in order and broken down into steps, really help intro students.

John: Do you have an anticipated timespan on your textbook project?

Carly: I suspect that it’s going to be this summer. I probably have two and a half months that I’m not actually teaching, but I’ll also be doing research on my scientific stuff (on my wasp studies) at the same time. I suspect that I’ll be able to draft out the first half of the textbook and probably be able to complete about three to four chapters of it. So, I’ll have those chapters ready for the fall semester and then I’ll try to get some work done during the fall semester and keep building that project. I suspect in total it’s going to take me at least two summers and also the semesters between, where I’m actually doing much more work on sort of my regular school requirements to actually finish it.

John: Do you have any people who’ll be working with you on reviewing this and giving feedback?

Carly: Not yet, but I recognize the need for that. I want to have this textbook be one of the contributions that I have for getting tenure. Making a textbook is a common contribution for the tenure package, but to make a textbook you actually have to have some form of peer review if you’re going to go through a publisher. So, when you’re making your own OER and you’re publishing it on your own website, you might lose some of that aspect of peer review. The plan right now is to actually enlist a set of beta readers who are also science educators in their own field and have criticism from them. This isn’t quite the same as having peer review, but I think for now it’s the very best that I can do, but I’m certainly open to suggestion and open to constructive criticism and changing things up. One of the challenges of creating your own OER is that at some point you might lose the more rigorous aspects of submitting a textbook to a standard publishing company.

Rebecca: Will you have an editor working with you for this project?

Carly: Currently no one is lined up, but that’s a valid suggestion, to actually pay an editor… probably someone who works in science textbooks. But, I think before I can even get to that point, I actually have to have a fairly large body of material to show them in the first place.

John: I would think that one thing that would be useful is, once you have this material, adoptions and response from adopters could be used in place of the peer review.

Carly: Oh yes, certainly. And when I put it up on the website there’s definitely going to be a forum for educators to be like “You know what, this didn’t make a lot of sense. Can you change the wording on this?” So, treating this as a living body of work instead of: “oh, I published that, it’s done…” because there’s no cost associated with changing and the material outside of my own time cost because there’s no physical version. So, it actually wouldn’t be all that difficult for me to have those changes be something that’s constantly happening, especially as we find out better ways to teach say homologous chromosomes, or mitosis, or things like that. But even before I launch it, I still want to have beta readers that can give it a read-through even before that, but having the ability for educators to constantly give me comments would be something that’s on the main website.

Rebecca: What software are you using to manage the process?

Carly: The website build itself is through SquareSpace and that is because I have absolutely no training in making a website, whatsoever. So that’s the actual platform that I’m building the website through. In terms of drawing, I start a lot of stuff out by hand and then I usually draw it in Photoshop on a tablet. Certainly, there are times when my tablet is down and I have to draw it by hand, and then scan it… that’s also a possibility. There is something else I’m interested in and this is more of a conversation about OER versus publishers. On the major publishers textbooks right now… so, McGraw-Hill, Pearson, Cengage, stuff like that… I think they’ve recognized that students can get OERs for free, professors can get OERs for free. So, what these publishers are doing now is that they’re offering adaptive learning systems, where you have assignments that get harder or easier as the student does better or worse, where the grades go automatically into the professor’s LMS which (if you’re at a school that doesn’t have grad students) is great because you don’t have a TA to do your grading. The publishers are offering these adaptive learning systems that go seamlessly into your Blackboard or your Angel (or whatever you’re using), but if you’re developing an OER you don’t have that capability. You can make standard multiple choice quizzes on Blackboard and give them to your students, but that’s not the same thing as an adaptive learning system that tracks your students progress. So, I would also be interested in working with someone (or maybe even SUNY at large) to develop platforms that actually make these adaptive learning systems… because then I think they’ll actually be able to convince more professors to adopt OERs.

John: Some of the publishers do have that. I know that Cengage, Pearson, and McGraw-Hill have been putting together packages of OER materials, where they add other resources to them (including some adaptive learning tools) that they release under a fairly inexpensive license. Another option might be to investigate Lumen Learning. Lumen Learning works with OpenStax and they package OER materials with some other materials they’ve created through a variety of grant-funded activities. But that might be worth doing and SUNY does have a contract with Lumen Learning on these things.

Carly: Yeah, I would like to work with someone that is not just SUNY…

John: Right..

Carly: I’m a SUNY professor but I would like people at the University of California system to be able to use my comics.

John: Lumen Learning is not restricted to SUNY.

Carly: OK.

John: SUNY happens to have a contract where they get a discounted price on the bundles when colleges adopt the Lumen Learning platform, but it’s basically a bundling platform that works with OpenStax and other OER materials.

Carly: Yeah, so that’s worth considering, because not only do I want to make the comic, I also want to make assessment tools… so that whenever professors are using my comic they also have a test bank… a way to create these adaptive learning assignments and things like that. So, this is something I’ve talked about before in my presentations at the Entomological Society of America… that you can’t just make a comic and put it out there for educators, you actually have to provide study tools, study guides, teaching plans, teaching lessons, to actually make it useful for educators.
I really like the idea of there being a platform where a professor could create their own test bank and then assign levels and topics to those questions and then just be able to import those into something that is automatically going to make adaptive learning assignments.

John: I don’t think we’ve got that yet, but there are a couple platforms out there: CogBooks and Acrobatiq. Both are do-it-yourself platforms for creating adaptive learning solutions and based on the Carnegie Mellon system,… which they’ve been doing for quite a while there. But it’s a lot of work, and it automates some of the process so you don’t actually have to do the programming, but you still have to work through most of the structure yourself. I noticed that you give students the option of making their own comics for extra credit. Could you tell us about that? how have students responded? and how has that worked?

Carly: Sure, so this has really come out of a desire to actually start generating and using comics in my class while getting the OER ready… because I have people who are asking me “What results do you have already? How have students responded?” And I’m like, “I haven’t finished the comic yet.” So, I’m aware of that and so that’s where incorporating comics into the classroom right now, while I’m preparing, comes from. General biology is a very difficult course for most incoming freshmen (which is the vast majority of the students I have). What it feels like to me is that I give all of my students the benefit of the doubt… I assume that they’re all studying… and when they do poorly on their first test I don’t say to them “Oh, it’s cause you guys didn’t study enough.” I say to them “No one has taught you how to study.” So a lot of my students, when they do poorly on their exams and they come to me during office hours, I ask them how did you study? And inevitably the answer I get is “I reread the PowerPoint notes, I reread the slides,” and so I’m like “No, no that’s not how you study, that’s just reading”. I try to emphasize that studying is the active reorganization and recontextualization of all of the information sources I’m giving you, not just my PowerPoint slides, but the lecture notes your hopefully taking in class, the textbook itself, the homework assignments. There are all these different forms of information that I’m giving you, and what I’m hoping you’re doing is actively reorganizing it. So, we talk about rewriting your notes. We talk about how to actually make flashcards that are effective. We talk about making flowcharts… and really from that last one… making flowcharts… that’s kind of like making a comic already. With the making comics as an extra credit, I’m really just encouraging to do another form of studying, where they have to take all this material for a midterm and they have to draw their own comic. So, usually what I do is I start the first couple of pages for them. So, on my Twitter right now I can actually send you an image of this first page I’ve made to kickstart their own process. So, spring break is coming up and they have a midterm, not the day after spring break that would be cruel, but the Thursday after spring break. That midterm is going to cover mitosis, meiosis, inheritance, and DNA transcription and translation. And these all seem like different topics but in reality they’re all very interconnected topics. You really can’t talk about mitosis until you can talk about alleles, and genes, and Mendelian inheritance and things like that. So I’m trying to encourage the students to conceptualize that these are all interrelated things because I think that I’ll actually help them memorize things better than just treating them as separate slides that they’re just reading through. At the end of next week’s Thursday lecture, the one right before spring break I’m going to introduce this project and hopefully I get some results from it. Previously I had done this at my last teaching position, which was at Sam Houston State University. I was a visiting assistant professor there, and for extra credit, I offered students the opportunity to make a comic on the same set of materials and I get responses… but the problem is that I get responses usually from the students who don’t need the extra credit. I think this is something that’s a common problem with offering extra credit… that inevitably many, many, many times it’s the students that don’t actually need the extra credit that turn in the extra credit assignments. Now. I still enjoy reading them and they still say that it was helpful and it’s a new study technique that they’re going to do, but reaching out to the other students is one of the challenges I’m facing as a young professor.

John: We all do that, it’s not just related to age. I know in my class I give them lots of chances to retake tests… the people who do it the most are the students who are already doing best in class. So, it increases the variance in the outcomes quite a bit when that’s not entirely the goal, you’d like to have everyone rise up but not necessarily spread out further on that continuum.

Rebecca: So, I’m curious with a project like this, do you use the opportunity as being a scientist who also as an artist to sneak in some art teaching as well? Do you use things like Scott McCloud’s Understanding Comics or anything as a tool to help students understand how to put together a comic and the medium of a comic?

Carly: I love Understanding Comics, it was like one of my foundational books when I was an undergraduate taking my first comic drawing class. I actually tried to avoid situations like that because I don’t want to discourage the students that feel like they don’t know how to draw. Which is a silly thing because everyone can draw… drawing well is a different thing. I don’t want them to get hung up on how good their drawings are. I want them to get hung up on how much conceptual sense that it makes. So certainly Scott McCloud talks about this, about how you can still have a comic that’s just stick figures. And so for me, I don’t want them to freak out about the fact that I’m an artist, and that I’m pretty decent at drawing, and that I expect them also to be pretty decent at drawing. But the funny thing about teaching non-majors is that inevitably some of them are art majors. So, that’s that’s always fun, they’re always surprised to find out when they come to my office and they see that I have paintings that I made as an undergraduate up on my walls and things like that. I would love to refer classic comic making literature, but it’s just something that I don’t have enough time when I’m just spending five minutes to introduce something. But, certainly… the students that come to office hours… we do talk about you know what makes a comic because I also have students that read a lot of comics. I have lots of students that are going on the Manga reading websites and a lot of students that talk about superhero comics with me when they find out that I like comics. So it does come up, but it’s usually not something I have time to make part of my already jam-packed lecture.

John: Students often have this perspective that they’re either creative or they’re good at quantitative skills in STEM fields, and it’s really nice that you’re modeling the possibility that you can be both.. That they’re not mutually exclusive.

Rebecca: That’s also why I like McCloud as a reference book too, because it’s not really about fine art in the traditional sense but rather about how to tell a story. Which is interesting and helpful and doesn’t really necessarily emphasize being able to draw.

Carly: Yeah, I think he has that… what does an expression look like, and it’s just like two dots for eyes, and then eyebrows, and then a line for a mouth, and you can get the full range of human emotion. And then I show students comics like XKCD, that is just stick figures and it’s really effective so, yeah. I try to avoid things where they feel like they have to be a professional artist, not to say that’s what McCloud does, you just pointed out that it doesn’t do that. But I try to focus more on the conceptual – like how does this help you study, you’re not just making this to impress me. And you get that a lot with extra credits, sometimes you feel like students are just doing those projects to get extra credit. Instead I’m trying to be like “Mo, no this is a study tool. This benefits you.”

Rebecca: Have you had any students follow in your footsteps and develop a love of both art and science and pursued you as a mentor?

Carly: At Sam Houston State, I certainly had students that like to come and chat with me and sort of explore those topics. But unfortunately, I had to leave there to start the position I have at Farmingdale, and unfortunately I just haven’t been here long enough to build those connections. One of the things I want to do, as I’m at Farmingdale a bit longer, and I get settled in, is actually propose a biological illustration class. So we have the ability as biology faculty to offer these topics in biology courses, and one of the ones I really want to do is biological illustration… especially since we share our building with art, or rather… I think it’s design communications… whatever the technical college…

Rebecca: Communications design… probably.

Carly: Yeah… but they’re still students taking drawing and watercolor and painting so..

Rebecca: How cool. That would be so fun.

Carly: Yeah, and you know what I actually kind of taught that course at Berkeley. When I was an undergrad at Berkeley, there was this thing where students could actually teach one-credit non-graded courses. So, I actually offered a biological illustration course. Sort of one of those things to build my resume and make up for my not-so-great GPA, but I actually really loved doing it and it seemed like as long as you can get some specimens, and you can sit down, and you have a studio space, you can come up with some amazing work, and luckily I’m still a research associate at the Museum of Natural History, so hopefully they’ll let me borrow some animal mounts. But there’s also insects. Insects are great… they’re cheap and I’m also the entomology professor so it could just become entomological illustration and then of course Farmingdale also has a huge Horticulture Department and botanical illustration has always historically… much like art has been a big part of biology… art has been a big part of botany for a long time. So I think we have the ability to do this, and that there would be interest, and it’d be a cool collaboration with these two departments that are both in Hale Hall.

John: How have your colleagues responded?

Carly: I would say positively… extremely positively. I’ve been thinking more about transitioning into… not fully being a pedagogy researcher… but having it be a large part of what I do on the research side. So, I still plan on doing my usual wasp entomology taxonomy research, but I also want to do a lot of research that’s in comics and the use of comics. That was something that came up in my interview and I think it overall was a helpful thing, and even while I’ve been here I’ve talked about it a lot with my chair and she’s been extremely supportive, and my other colleagues have also been supportive. I haven’t received any negative pushback… which I think was something that I was expecting… because when you look at the literature about educators… whether they or not they want to use comics, there’s this fear… that comics have this bias against them. And so a lot of educators at the primary and secondary levels are kind of afraid of assigning them, and they’re afraid they’re going to be looked down upon by parents and by other educators. But I’ve been extremely fortunate, and I have faced none of that and largely the faculty have been very supportive.

Rebecca: I wonder if some people maybe perceive comics as just being not very rigorous. Which is crazy,because you can provide so much more information… because there’s a visual element as well as a text element. So they might actually be more rigorous.

Carly: Yeah. We talk about lack of rigor and lack of detail in textbooks anyways. If you look at a non major biology textbook it’s obviously not going to be as detailed as a major’s introductory biology textbook, and there’s a reason for that. You’re not teaching people who are going to continue in biology for the most part, so there’s less detail. But, still people harp on the lack of information and the lack of rigor. So, I feel like that’s going to be an argument that comes up no matter what assigned reading you’re going to use. Certainly with comics there’s another bias and that there’s a bunch of superhero comics… but comics are actually a lot more diverse these days.

Rebecca: Comics are probably a really great way to help students understand those basic concepts so that they can build their mental model because they probably come with all sorts of assumptions and things that are not correct, and I could see how demonstrating visually could help overcome some of that.

Carly: Yeah, certainly, and for me it really comes down to what is the point of general biology? What am I aiming to do? I still want my students to learn about photosynthesis, and the Krebs cycle, and mitosis, and meiosis. But I also want them to come away with an appreciation and a sense that they are able to understand it. I want them to walk away from the class with positive feelings towards science and not just- it’s a collection of facts I had to memorize.

John: I wish I had had a class like this when I was in college. it seems like a fascinating way of addressing this

Rebecca: It goes back to what we were talking about earlier. I didn’t know I wanted to know about wasps, but maybe I want to know more now after reading your comic.

Carly: Yeah. So these are all like my lofty aspirations as an educator, but I’m pretty sure I’m still making common mistakes and it’s still a bunch of facts that they have to memorize, sometimes. But I feel an awareness of these of these issues is helping and hopefully I only get better at that process.

John: …and there’s nothing wrong with it being fun for them to learn those facts. ..

Carly: Yeah.

John: … they do need to learn facts but there’s nothing saying it can’t be engaging.

Rebecca: Well, providing those sequences might make it easier to remember, because you have a clearer understanding of how the things connect. The visual representation can help provide those connections that words don’t always help because it’s too abstract.

Carly: I think with biology, especially at the introductory level, especially when you’re a professor that doesn’t have graduate student instructors or TAs, you don’t have a lot of time. So we always talk about wanting to have critical thinking questions and essays, but inevitably just because of time constrictions it does largely become scantron multiple-choice questions, and in that way it does become a lot of memorization. Now I still think that memorization is valid. I still think it’s important to know the steps and the processes and be able to call up that knowledge. But for me, the struggle is making that memorization easier. And if comics make that easier then I’m accomplishing my goal…

John: One of the things that really impressed me, though just following you on Twitter recently since I saw your work, is how engaged you are in the scholarship of learning and teaching in your discipline. It’s nice to see people starting their careers doing that. What got you interested in doing research on teaching and learning?

Carly: I think it actually comes down to who professors are. Professors tend to have PhDs, and in my case, I didn’t take any classes about how to teach. So I think most of us are just kind of thrown into this process and we learned slowly along the way. I was like “Well, there’s a whole body of research out there…” and I started reading some papers about how to be a more effective teacher. We have our own center here for teaching that has workshops and stuff like that, and I think recognizing my lack of formal training, I have no teachers certification or anything like that, made me more interested

Carly: I’ve got the list of questions.

Rebecca: One of the things that I think is kind of interesting is the way that we started this whole conversation… and it ties nicely back to the scholarship of teaching and learning… is that your first research position was doing illustrations. And I think that in academia, we don’t often see those sorts of actions as being research. So I really love that that role was called researcher and brings all this sort of together. It doesn’t have to be traditional to be effective or useful.

Carly: Now certainly that first position as an illustrator in the Padian lab… I still wanted to do traditional types of research, but that experience (as someone who is already sort of hanging around on the graduate student level and hanging around the research labs) made me a person that was visible in a crowd of something like 2,000 undergraduate biology students. So from the Padian lab, I was actually able to transition into a more traditional research role that actually led me to parasitism, to studying parasitism, and that was in the Wake lab with Marvalee Wake, who is one of my most important mentors as an undergrad. But yes, my first research position I was called an undergraduate researcher was actually just doing illustration. And I learned a lot about vertebrate anatomy because that was what Katie Brakora studied.

Rebecca: People don’t realize that when you’re doing that kind of illustration work, what kind of attention to detail you need to pay, and how much you can actually learn by just looking at something very carefully.

Carly: Oh yeah, being able to measure something… getting proportions down correctly. There’s a lot of math that goes into biological illustration and serve a lot of rigor. And then you just spend hours stippling, and that was my life.

Carly: Yeah, I would just say if this sounds like something that a faculty member is listening to this podcast and they’re like, “Ah I want to either start making comics or I want to incorporate comics even into a STEM class, I have lots of resources and I can sort of talk ad nauseam about that. You know like, “What are some good comics if you’re teaching biochemistry? What are some good comics if you’re teaching literature?” So certainly if there’s anyone who’s interested in either making comics or choosing comics for their classroom, I’d be happy to talk to folks.
I think unless you’re a comic book reader you probably don’t realize just how much comics have grown outside of what you might have imagined they were twenty years ago, and you’d be surprised by the amount of some relatable materials… especially in the social studies classes… especially in history, there’s a lot of memoirs… a lot of historical memoirs right now in comics.

John: Actually right now, I can think of at least a couple of examples in economics of comic book series that were created for instructional purposes. The Federal Reserve Bank of New York created a series of comic books to help provide middle- and high-school students with information about the monetary system and the role of the Federal Reserve Board; the other, a series of comic books created featuring Captain Euro… this was originally created to provide support for the introduction of the Euro and for the European Union in general.

Carly: Medicine has really moved with this, especially when they’re thinking about “How do we make information that transcends language barriers?” I follow a Twitter that is just medical graphics and there are conferences on medical comics as well. So I think that’s a field that’s really sort of latched on to making comics as a way to share information with patients, and there’s actually been some research showing that it’s more effective.

Rebecca: It’s used a lot in areas where there might be outreach for really low income or people in poverty who need important information about health or resources and things, and that’s where literacy might be an issue, and so sequential images are often used in those contexts as well. When I was doing a project in India, I discovered all of these really interesting graphics that were used… sequential graphics… to get people to do all sorts of things because there’s so many different languages… to kind of overcome that barrier. It was really interesting.
So we usually wrap up our interviews with the question of what are you gonna do next, you’ve already talked about a number of things that are on the horizon, but is there anything specific you want to share as your your next step, whatever it is that you want to research or do?

Carly: Yeah, so we’ve talked a lot about comics, but I can tell you a couple of other things that are on the horizon for me. My field season… the actual going out and studying wasps that I do that’s going to start up in the summer… and I’m hopefully going to bring an undergraduate or two with me, and then hopefully bring that undergraduate to present at the Entomological Society of America. So, that’s sort of the science side of my life, but sort of the swing back I’ve been talking a lot at the Entomological Society of America about using comics in entomology research… and sort of more in line with what you guys do generally, my next thing is actually proposing a symposium on education for undergraduates. Since most entomologists that are at a university don’t just teach entomology, we also generally teach any biology courses. So, kind of swinging more strictly into undergraduate education instead of the broader community outreach education that I’ve been doing with comics outside of academia. So, that’s exactly next on the horizon for me outside of just keeping working on comics.

Rebecca: So, where do your wasps take you this summer?

Carly: They’re going to take me hopefully to Puerto Rico for about a week, down to Florida for probably a week or two, and also local collecting. There hasn’t been a lot done around the Northeast, so going out to the Pine Barrens on Long Island and then probably making it up as far up as you guys and things like that and further up and down the East Coast.

John: Well if you do get up here let us know

Rebecca: Yeah.

John: … and we’ll take you out to lunch or dinner.

Carly: Oh, thank you. Yeah. This has been great guys, thanks for having me and inviting me to this.

Rebecca: Yeah, thanks for sharing all that you’re doing it sounds really exciting, I can’t wait to see it all happen.

John: It’s great to have you here, and you’re doing some wonderful work.

Rebecca: And you have two fans here and two advocates here.

Carly: Oh thank you, that’s important. I want to like tour all of the centers for teaching and learning excellence, however it’s called at every university, and you know be like “Comics, comics, comics, comics!”

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

21. Project-based learning

Posted in active learning, authentic learning, collaboration, higher education, pedagogy, project based learning
March 21, 2018

Big, complex, and messy problems provide rich learning experiences for students, but can be overwhelming if not properly scaffolded. In this episode, Jeff Bradbury joins us to discuss a semester-long sound-replacement project that his students complete in a course on Sound for Television and Film.

Show Notes

  • BRC 308 – Sound for Television and Film – course description
  • BRC 3089 – Sound for Television and Film II – course description

Transcript

Rebecca: Big, complex, and messy problems are rich learning experiences for students but can be overwhelming if not properly scaffolded. In this episode, we’ll focus on how to organize a class around a single big project.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Jeff Bradbury. Jeff is a Visiting Assistant Professor at SUNY Oswego. He teaches classes in audio production.

Rebecca: Welcome, Jeff.

John: Welcome, Jeff.

Jeff: Well, thank you. Glad to be here.

Rebecca: Today our teas are:

John: Ginger peach whole leaf tea.

Jeff: I missed that one, but I got some really delicious black raspberry green tea that I’m really digging..

Rebecca: I’m drinking exotic mango and ginger green tea.

Jeff: That’s impressive.

Rebecca: From Twinings.

John: Oh! Twinings, okay, okay.

Jeff: These people are serious about their tea here. It’s…

John: I forgot that we had that.

Jeff: …impressive.

John: What classes do you normally teach?

Jeff: Well, I teach a number of different audio classes here at SUNY Oswego. The most basic class I teach is a class in radio production, and that I treat more or less like a basic audio production class. Just the real basics of which end of a microphone to speak into… just basic broadcast performance stuff. I’m not an on-air broadcaster myself, but just basic things like how to sit up straight, project your voice… this kind of stuff makes a difference when you’re recording your voice, and then I teach my signature class here at Oswego… a course called BRC 308, and that is Sound for Television and Film. I also teach a successor to that one, which is BRC 309, which is very creatively called Sound for Television and Film II.

[LAUGHTER]

Rebecca: Yeah, we have similar classes with Graphic Design I and II.

Jeff: Yes, yeah.

John: So when we talked about your work in these classes earlier, one of the things you mentioned is that you have a project where students rescore cartoons. Could you tell us a little bit about that?

Jeff: Yeah, definitely. The course I teach… BRC 308… some students do cartoons… more or less these days you have these all the animated films… the 3D animated films that are so popular… and a lot of times those work really good for… basically what the project is, is a sound replacement project. So, they take a scene probably anywhere from a minute and 30 seconds to three minutes, and we dump all of the sound, which includes dialogue, sound effects, music, ambiance, all that kind of stuff… They dump all that, and they recreate everything themselves from scratch… and it isn’t actually just cartoons, or animated stuff — they can do live-action TV shows or films as well. I always tell the students probably one of the biggest things that you’ll take away from this class is how much goes in to the sound side of any…even just dramatic television shows, episodic television, especially films… I mean there’s there’s so much to it… If you can imagine in your mind’s eye, when they mix the sound for a major motion picture, they’re sitting in what looks like a movie theater, and they’re sitting in front of a console that is generally about… probably anywhere from fifteen to thirty feet long… and there are anywhere from two to three to maybe even four people sitting at literally hundreds of faders mixing every single little element… perhaps different instruments from from an orchestra… from a score… or you have… every time somebody sets a coffee cup on a table… and every little sound and dialogue and clothing movement and stuff like that that they have to reproduce. So, when you say a semester project replacing the sound for one minute and 30 second clip from a film sounds like: “Oh, that’s gonna be a snap.” …actually it’s not. When there’s so when their sound is that dense, I have students who who end up with projects with anywhere from 25 or 30 to up to 60 or more tracks of audio that they’re mixing all simultaneously with different processing and all this kind of stuff to make this work. So, there’s there’s a lot to it, and they learn really quickly. They’re like, “Wow” …because I always tell them… I say: “Can you imagine doing this for the full two-hour film?“ …and they just shake their heads. Because they spend about about ten weeks working on their final project.

Rebecca: Is the goal of the project to respond to what they’re seeing, and make the soundtrack? Or is it to replicate what was there before?

Jeff: Since this is their first stab at this, I have them bring into their project the original audio from the film, that’s what I refer to as the reference audio. Usually what they try to do is replicate what they hear and what was done before, but I always try to tell them “That is an interpretation of what the sound should sound like, that doesn’t necessarily mean that it has to be that way.” …and I show them examples… that you can take something …an explosion or something… and you can have two completely different sounds and it’ll still work. The viewer, or the the audience member, will accept what you give them as long as it’s within a certain realm… and you can really, really affect the feel and the outcome of a scene… even just what sound effects you choose. But yeah, I always tell them: “Don’t be afraid to try something different than what you hear.” What’s amazing about major motion pictures and television shows and whatnot is that they don’t always represent everything… so, I tell them: “Look what’s happening there.” …or even better yet… a lot of times there’s things that probably are happening but you don’t see it in the plane of the screen. For example, on a city street… and you see a certain view of a car parked on a city street and there’s dialogue happening or something… there could be a truck backing up behind you and if you put that sound in the scene, it’s completely believable that that’s there. That’s one of the things about sound that I teach them and show them… it’s so awesome… it’s 360 degrees wherever you believe you are in this motion picture there’s also any sound from anywhere even if you can’t see it is acceptable… and a lot of times you need that. When you’re in city in the distance… honking cars and sirens and that kind of stuff. You can’t see them, but hearing those make that seem believable.

Rebecca: That’s a big project, you’re talking about all the layers and the complexity of it. How do you help students scaffold something like that, and guide them through that process? At first it seems like: “Oh, it’s a small project… we have all semester to do it,” but then I’m sure very quickly it becomes very apparent to the students that: “Oh crap, this is a gigantic project and I have no idea how I’m gonna get this done.”

Jeff: There’s perhaps two answers to that question. Oftentimes, students when they pick their projects, they immediately go for the most climactic scene of the movie. And so I tell them… and I hold the veto power and I always jokingly say… and it’s a joke now but I had a student present… wanted to do… I haven’t seen the movie in some time but there’s the movie Gladiator, and I think an opening battle scene of Gladiator… “No… No. You’re not gonna do that… at the end of this class you’ll understand why, but that is way too complex. There’s too many layers of things going on. You can’t do that. I try to tell them so that they don’t bite off more than they can chew. I say: “Okay, if you want to do a scene from your favorite movie or something like that, that’s great. That’s a great place to start.” I tell them: “Try to find those moments in the film that are in between the most climactic parts of it, because you may watch it… and you may listen to it… and you may say: “Well, this is kind of simple.” Trust me, there’s a lot more going on there sonically then you know about… So, that’s one way… I try to coach them so they don’t bite off more than they can chew. Then, from there what we do is I have them fill out what’s called spotting sheets. So, let’s say they choose a two minute piece of a film. I capture that film piece and then give them what’s called a window dub… and the window dub has, and you may have seen this… if you’ve ever seen behind the scenes videos and stuff… it basically has the timecode – hours, minutes, seconds, frames – burned into the actual picture. This gives them a reference. I have a copy of this… they have a copy of this. They work in teams or sometimes even groups of three on these projects. They all have copies of this. They carry them around on their phone. They can communicate with each other and say: “Well, there’s that punch at one minute thirty:two seconds and four frames and I’m not really sure what to do with that.” Everybody else can look at the same video and know exactly what they’re talking about. But what they start off with is… they go through their entire project and they list according to that timecode the sound, then what the timecode is, and then some notes on how they think they need to acquire that asset. Do they need to go out in the real world and record it with a portable field recorder? Do they need to do that in our Foley room – which is a quiet space that we have to record? And a last resort… because I prefer that the students record everything themselves… when they do that, I think it’s something that they’re very, very proud of when they do it that way… but sometimes they have sounds that it’s not feasible for them to acquire themselves, for instance, any sort of firearms. So we have an extensive sound-effects library that they can go in to capture some sound effects. Some people think that’s cheating. It’s not really because the sound effects never fit… just plug and play… There’s always some sort of editing, manipulation, chopping, multiplying, that has to be done to make them fit.

Rebecca: Encouraging students to pick those in-between spaces also seems like a good opportunity for them to be a little more creative than they would be able to otherwise. Those climatic scenes people have specific expectations because they’ve seen the movie, but those in-between spaces are probably ones where they can be a little more experimental and try some different things and feel safe doing so.

Jeff: I think so, yeah. What’s interesting to me is that I’ve found so many cool TV shows and films that I never heard of or seen before when students bring in some of their ideas… and I think you’re right. At first, they want to do something really big… and more is more… but when they do search and find these these moments in between the climactic scenes, I think that… yeah, it does give them a chance to… actually what I think works best is that… because there’s not so much layering going on as when you’re doing a climactic scene… there’s always tons of sound happening all at the same time. You have more individual sounds happening, and so they can focus on this.
One of the best projects that a student has done yet was a student… if you’re familiar with the film The Breakfast Club… there’s a great scene in The Breakfast Club where they’re all at their first day of Saturday detention or whatever it is there… I guess the whole film is one day… and they’re eating their lunch. There’s this great moment where they all sit down and one student brings out all this different stuff… one girl making her lunch… and there and there’s very little dialogue. It’s perfect for this class… it’s like a three-minute thing… and this student who did this… it was an immense amount of work, but it was so great because each sound that happens is… one thing happens… then the next thing… then the next thing… then the next thing. So it really allows them to focus on the individual sounds… how they capture them and stuff like that.

Rebecca: Hmmm.

Jeff: Yeah.

Rebecca: That’s interesting.

John: Do you have the students record the dialogue as well?

Jeff: I do. One thing that a lot of people don’t know about major motion pictures is that a lot of these big-budget major motion pictures, even television shows… most of the dialogue is typically replaced… The principal actor shoots a scene on location and unless it’s an environment where they can get a boom mic or something… or a well-hidden lavalier microphone… unless they can get it recorded really clean on set… they take that same actor and they go in to a quiet studio later on… and that same actor re-does their lines to get them clean. They refer to that as ADR. And ADR stands for Automated Dialogue Replacement.

Rebecca: Doesn’t sound that automated…

Jeff: I know.

[LAUGHTER]

There’s parts of it that can be automated, as far as they the start and stop of when they do it… and a lot of times it’s also referred to as looping, and so what they would do is the actor gets several tapes and I think that’s where they get the automated from, I’m not really sure. So that’s a part of it… replacing dialogue. So, I have them do the dialogue. I also try to coach them to not pick scenes that are really dialogue heavy because that can be a challenge, because you have to find somebody who can say those lines convincingly.

John: When you first started teaching this class, was this something you introduced right away, or did this evolve out of some earlier work?

Jeff: The first time that I taught Sound for Television and Film I did teach it this way… with this project. When I was in graduate school at Syracuse University, I was one of two students who were teaching assistants that helped in the audio classes. My main background with audio is on the music recording side. So I started off recording bands and all that kind of stuff… and I was the the TA for the music recording class. The other class that was also offered was this sound design kind of a class and that’s basically the project that they did… so that’s where I got the ideas. I knew that they did this sound replacement, but I had never taken that class. I just figured well that’s probably the best way to go about it. So, I just kind of started off…I just… “Let’s do this.” It’s been far too many years that I’ve been teaching this…. I’ve taught this class here at Oswego, I have taught the class at Syracuse University, I also taught a version of the same class at Ithaca College. So I’ve been honing, and perfecting, and altering this for many years.

Rebecca: What are some of the biggest changes that you’ve made to the class from this trial and error revision process?

Jeff: I would say the big things are the guidelines that I impose on how they choose a project because when I first started doing it, I had no idea… and then what I noticed was happening is students were choosing the most climactic moments in the films. I think the lightbulb went off for me when a student tried to do the scene from Titanic where the ship hits the iceberg, and unfortunately their project went the way of the Titanic…

[LAUGHTER]

…and I thought to myself “Okay, did this student really fail or did they just bite off more than they could chew?” And so I think that’s probably the big one there. I’m trying to think other things that…

Rebecca: Have you added more check-in points or things like that?

Jeff: Yeah, I have. Because one thing that I’ve found over the years is that if you just give the students an open-ended: “Okay, start now and it’s eight weeks from now or ten weeks from now the project is due…” You know where this is going… they’re gonna wait until a week before it’s due and try to finish it all… And that just will not work with something like this… they’ll die. About four weeks before the end of the of the project… before the end of the semester… I say “Okay, give us a progress report.” That’s worth twenty percent of their grade. The final project is worth forty percent… this is worth twenty percent… and I say these are the following things that you have to show if you want the twenty percent in your progress. There’s a laundry list… so, you have to have all your sound effects at least basically recorded… you have to have all your dialogue done… So, that helps immensely… because it forces them to get started on it rather than just waiting until the end.

John: So if someone were to stop in on one of your classes what would it look like, or what would it sound like?

Jeff: So, that’s really interesting… and it varies every single day. The first six weeks of the class, of the course is really a boot camp… for recording… using Pro Tools… I mean… Pro Tools, the application we use is an extremely deep robust program. I’ve gotten pretty good at sort of boiling it down to the need-to-know basis… like “Look, this will get you started, anything else you need to know, just ask me and I’ll show you on a need-to-know basis.”… and that seems to work pretty good.
So, gosh, what would somebody see if they walked in my class? The first six weeks when we’re just sort of in bootcamp mode, it looks like a regular class… I’m teaching them stuff, I have structured lectures… I have them try things like, “Okay now I’ve showed you” or “I’ve given an example, now you try it” …then I walk around and make sure everybody’s getting the basic set of skills. Once we get to the midterm, the second half of the class is basically class time…. it’s just work time. And so usually students work in pairs on a project. So, you might see one group of students sitting there making notes, looking at the computer, sort of figuring out what their game plan is… another group of students working on headphones across the hall from the classroom… we have our basic production studio where they can go in there where they where they manage recording. Then there’s a quiet space on the other side where they have a screen. It looks very professional… It looks like it’s done in the industry… and they’re in there recording clothing movements… or dialogue… or all this kind of stuff to the moving image… and me, I’m generally just running around answering questions… a lot of times putting out fires. Thank God, my previous existence as far as career-wise, I was a computer consultant, because I’m really adept and knowledgeable of computers… hard drives… when things go wrong… because that’s the space we live in. The students have to buy their own hard drives. They’re constantly having file management issues…

Rebecca: …sounds really familiar.

Jeff: Exactly, right?

[LAUGHTER]

When you work in anything creative these days, you know the more about computers and hardware and especially file management… And honestly, can I just say on a side note, I’ve noticed that it’s so much more difficult nowadays to teach that stuff and I think the main reason is… look at what most technology the students have nowadays. Phones, there’s no file management to them, right? Basic… like what folder… what directory… and if you don’t know that stuff cold… in the world I live, anyway, it’s like you die. I swear to god, that’s one of the hardest things I deal with… is having to teach them that basic stuff and when they screw that up, it screws everything up. They spend a whole day working and they didn’t import something the right way, then they come back the next day… it’s all missing.

John: Smartphones are getting a bit better than that. Android had a file structure from the beginning that was somewhat transparent although not easily accessible, and now there is a files app on iOS 11.

Rebecca: Yeah.

John: You can access things but it’s not a very easy-to-use file…

Rebecca: Yeah.

John: …it’s not very hierarchical.

Rebecca: It’s not like when you were using older operating systems where you were actively getting into directories and out of directories, right?

Jeff: Right, yeah.

Rebecca: So, if you come from that perspective, it’s a lot easier to know how to file manage, right? Whereas a lot of our students I think, rely on searching… Like, “I don’t know where it went, it just went somewhere magically and I don’t remember what I called it… so, can you wave your magic wand and find it for me?”
[LAUGHTER]

Jeff: Right. Right, one of the things that I do out of necessity is I spend a number of classes in the beginning of the semester, and I apologized to all them up front. I say, “Okay, look, I don’t want to treat anybody like a kindergartener, but we’re going to kindergarten for a couple days, okay? Lesson number one: this is a file, this is a folder, this is how you look at your folder and file management in a hierarchy way.” All this kind of stuff …and ‘cause some of the kids know it and they get it, but others have just not. For us too, because we’re working on Macs, there’s a certain number of students who are PC-based in their experience and so they’re trying to wrap their head around using a Mac. But, I wish they could all come in just knowing all that stuff cold and we could just jump right into it. But I’ve found, and that’s another thing perhaps in the question you asked earlier “what have you changed,” is I found, that if I don’t spend a certain amount of time back in kindergarten (I know that’s probably not the best way to talk about it) we all really pay for it later.

Rebecca: Yeah.

John: But that’s a realistic learning experience because these are issues they’d face when they’re out there doing this type of job.

Jeff: Absolutely.

John: …and it sounds like they’re getting us some background in all aspects of the job that they might have if they were going to work in this field from recording to mixing.

Jeff: Yeah.

Rebecca: The reality is you got to meet students where they’re at.

Jeff: Absolutely

Rebecca: …and so, if they have no idea how to structure their files, and that’s key to what you’re doing, you got to start there.

Jeff: Yeah well, and I do explain to them too, I say, “This may be very simplistic to you,” but I’d give them sort of the story of like, “okay, let’s say that you’re gonna go and actually break into this field and do this for a living. Generally speaking, the way people start off, if you’re gonna be working in post-production sound, is you start off as an assistant.” …and assistants, basically what they do is they work in 100% file management. They take recordings that were done in the field… they do like a basic chop and an edit… and they’re given what’s usually referred to in the industry as a “Bible,” and they have naming conventions: this is how you name it, this is where you put it, this is the metadata information you’re to put with that. Because on a typical major motion picture, when the sound team is going out and collecting sound effects in the fields like that, they’ll have, a lot of times, a database of over a hundred thousand distinct audio files that the senior audio designers are gonna try to call from to build these scenes and stuff. So, I tell them that, “that’s where you start… so, even though you might look at it as a little bit boring, you’re gonna be doing that for a while before you’re ever gonna become the sound designer guy.

John: I know you got a license for a set of royalty-free sounds some that have been cleared of copyright. Are there any copyright issues with the project?

Jeff: That’s an interesting question. Basically, the students are grabbing a scene from a film or a television show, something that’s copyrighted, but for learning purposes… So, I think it’s really a fair use example. We’re stripping the sound and then they’re doing the best job they can to recreate the sound for that scene. There have been a couple of times the student has tried… they’ve been so proud of their work, they’ve tried to upload their project to YouTube and of course, I think automatic filters that they have… they say, “uh-uh, you can’t put a scene from such-and-such a film” which actually surprises me because I never realized this until the students do it… When I have them look and suggest to see possible scenes that they might want to pick for their project, you can basically just go on YouTube and you can search for like, oh the “dinner scene of Shrek…” it’ll come up. Somebody put it up there… it’s there. It’s kind of interesting that all this stuff is out there. But I’ve never had any issues with copyright and I tell the students, I say, “look, the final product that you’re gonna end up with here is a portfolio piece… and so in other words, it’s not something that you’re going to be trying to sell or trying to put out into any sort of public domain in any way… any shape, anyway. If you want to get a job in this field, you’re gonna show this to somebody in an intimate setting.” Say, “I did the sound replacement for this” and they’re hopefully gonna be impressed and so it’s not going to be a copyright issue I think per se.

Rebecca: So, it might be like a demo reel or a demo behind a password or something like that anyways.

Jeff: Yeah, exactly.

John: So, how have students responded to this type of project?

Jeff: Students absolutely love doing this. I mean, it’s the one thing that gets me out of bed and keeps me doing this kind of stuff. Because I witness a transformation. Now I sort of enjoy the fact that there’s a bit of a reputation that the class is cool and it’s fun and maybe even a little bit that the professor is kind of cool, I don’t know. It’s a difficult class to get into… it fills up really really quickly. But I see so many times like students I think, were kind of interested in the topic… But by the end of the semester, I think every semester, every section I teach, I have at least one student is who’s like, “this is what I want to do with my life.” …and I think that’s really cool… when they get into it. When you go out… a lot of times, what they have to do is they have to get like one of the portable recorders… I see that you have a zoom recorded on the table. We have something similar to that. We have some Tascam recorders… and then, of course, they have to go out in the field and… a lot of times… record sounds in the real world that they’re gonna bring in and edit and then put in. I cannot explain to you how rewarding it is when you go out and you spend a lot of time getting just the right sound that you capture… you bring it in… and you spot the sound …which basically means you just put the sound on the timeline and you line it up in the right timing and you play it and it works. It is so satisfying, it is so satisfying… and I see them the first time they’ve done that, the first time they’ve recorded their own sound and they put it into the project… they’re giddy… For a lot of students, once they do it once, they’re hooked and so they can’t wait to go grab the recorder, go record some more sounds, and put them back in. …and yeah, it’s one of those things where it is a lot of work, but the pride they feel in the finished product… I don’t know if it’s maybe because students are just getting better… or I’d like to think that I’m getting better teaching the class but I think semester after semester after semester and the projects keep getting better and better and better.

I actually had a student who did a scene from one of the Pirates of the Caribbean movies and I initially told this particular student: “I don’t know, that’s some dense sound.” It wasn’t the most climactic scene, but it was there was a lot happening. He said, “no, I can do it” and I always tried to defer to their judgment when that happens. I’m like, “if you really think you can do it, I’ll let you do it.” He did such an exceptional job. This was the my favorite story and I always tell this to the students. He took the project home after the end of the semester and he said he played it for his older brother. He said, “hey, check this out, I want to play something.” So he did the right thing, he didn’t tell him what it was, he just played the scene with his sound replacement and it was the ultimate compliment when he said, “yeah so it’s the scene from… that scene from Pirates of the Caribbean.” He’s like, “you liked it?” He’s like, “well yeah, I mean I like the movie” and he’s like, “I dumped all the sound and replaced everything” and he’s like, “No way.” He couldn’t distinguish it from the original film and I thought that’s an accomplishment because there was some complex sounds that were going on in there and this student just really busted their you-know-what to do this. One of the things that a lot of students want to do is… if I had a nickel for every time they want to change the little girl in a scene’s voice to a big burly man, right? Isn’t that funny? The big burly man sounds like a little girl and I tell them “no, I’m not gonna…” because they think it would be funny… I say, “well here’s the deal, ultimately I want you to try to make something that will not fool, but just like that last example I played, you want to play this for somebody, you don’t want them to to know about it.” As soon as you do something in the scene that doesn’t fit, it immediately calls the listener or the viewer to the fact that something’s been done with the sound so you break the…

Rebecca: The smoke and mirrors, right?

Jeff: Right. Right, you ruin it for them. So I tell them, “I absolutely forbid that from happening. If you can’t find a little girl to replace that dialogue with, don’t do the scene.” Just a little side note.

Rebecca: So, you’ve talked about revising how you teach, so what are you gonna do next?

Jeff: One of the things that, in the successor to this class, it’s Sound for Television and Film II… Basically, that course is… students take what they’ve learned from doing it the first time… choose another scene… and do it again. But this time, they choose a little bit longer scene, and they get to start right out from the beginning and have a lot more time to work on it. I’ve had, a few times… students actually were able to pick actual short animations from real creators and have a chance to do it for real. I’d like to be able to find ways to do more of that, but it’s really challenging to try to find original work that needs this level of work… that’s not too much… it’s really kind of a hard space to grab.
What’s next for me is to really try to figure out how to get students to do it for real. I’ve got one of my students from years ago who is now working out in Los Angeles and doing this for real… and I keep threatening him that I’m gonna send him a student or two too as an intern… and where he works, they actually they have a formalized internship process… but it’s funny, it’s hard to get students who are willing to go out there… be able to live…and be able to start at the bottom. Because basically you have to start as an unpaid intern at a place like this… and start from the bottom… and move their way up.

John: Okay, well thank you, this was fascinating.

Rebecca: Yeah, thanks for sharing your story.

Jeff: Yeah, thank you for having me.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

19. Common Problem Pedagogy

Posted in active learning, authentic learning, collaboration, common problem pedagogy, higher education, interdisciplinary projects, learning, pedagogy, project based learning, service-learning
March 7, 2018

Most colleges are organized as a collection of academic silos. Many challenging problems facing society, though, are multifaceted. In this episode, Leigh Allison Wilson joins us to discuss the use of common problem pedagogy, an approach that allows students to address a problem from multiple disciplinary perspectives.

Leigh is the Director of the Interdisciplinary Program and Activities Center at SUNY-Oswego. She is also the author of two collections of stories, one of which won the Flannery O’Connor Award for Short Fiction. Her stories have appeared in the Georgia Review, Grand Street, Harper’s, The Kenyon Review, Smokelong Quarterly, The Southern Review, and elsewhere. Leigh teaches creative writing at SUNY Oswego. In addition to the Flannery O’Connor award, she has received the Saltonstall Award for Creative Nonfiction, and a Pulitzer nomination by William Morrow for her collection Wind. Leigh is a Michener Fellow of the Copernicus Society and is a Henry Hoyns fellow of the University of Virginia.

Show Notes

Transcript

John: Most colleges are organized as a collection of academic silos. Many challenging problems facing society, though, are multifaceted. In this episode, we explore common problem pedagogy, an approach that allows students to address a problem from multiple disciplinary perspectives.

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Leigh Allison Wilson. She is the author of two collections of stories, one of which won the Flannery O’Connor Award for Short Fiction. Her stories have appeared in the Georgia Review, Grand Street, Harper’s, The Kenyon Review, Smokelong Quarterly, The Southern Review, and elsewhere. She teaches creative writing at SUNY Oswego. In addition to the Flannery O’Connor award, she has received the Saltonstall Award for Creative Nonfiction, and a Pulitzer nomination by William Morrow for her collection Wind. She is a Michener Fellow of the Copernicus Society and is a Henry Hoyns fellow of the University of Virginia.

Rebecca: Welcome, Leigh.

John: Welcome, Leigh.

Leigh: Thank you, John. It’s very nice of you to have me

Rebecca: Today, our teas are:

John: Ginger peach green tea.

Leigh: Mine is Constant Comment, a southern favorite.

Rebecca: …which Leigh brought for me to try, so that’s what I’m drinking, too.

So, Leigh you helped organize a number of community-based projects that bring faculty together across campus. What got you involved in this kind of work in the first place?

Leigh: Well, you know what? If I went back to the roots of it all, I have to say Amy Bartell in the art department. I have a flash fiction class that is my advanced writing class, and one semester she just suddenly said: “Why don’t your students write a very short piece? My students can illustrate it, and we’ll frame both things and put them side-by-side… and then we’ll have a show.” …which was so much fun, but it wasn’t just fun, it was my first taste of having a collaborative common problem project. Because, it turned out to be a common problem. We didn’t know it… we thought we were just writing our fiction… or, I thought they were just gonna be writing their fiction. But we’ve discovered that if there was going to be an illustrator paying attention to it… all of a sudden, it got more serious. The game got more serious. There was an audience who was really going to be checking it out, and there was also an audience that was going to be looking at the illustration and looking at their work at the same time, and all of a sudden the students were much more professional about their attitudes to their work. So, that’s the beginning of it. That was called Graphic Flash… and we’re still doing it, but now it’s expanded into a film class that’s taking the stories and making short films out of it… and a music class that’s taking the short films and scoring them… and ‘cause now I like working with local partners… local high schools have been making movie posters.

Rebecca: Great.

Leigh: …for the stories. So, that expanded… and because of that expansion, I started getting interested in – not just common projects that involved a common problem – but also collaborative projects in general… and the ease with which they could be expanded… which I think is one big factor in project-based learning.

Rebecca: The first big project was the Smart Neighbors project which is still ongoing.

Leigh: What happened was… I was doing Smart Neighbors and there was a notice from the Provost office and there was a call for participants in a SUNY wide grant. They wanted four SUNY schools to be involved in a common problem pedagogy grant… and at the time they were trying to get a Teagle grant which is an ExxonMobil grant. But the point of the Teagle grant was to get humanities to work with another discipline, usually a professional discipline, so that’s why it began in that way. I wrote in and SUNY Cortland and Oneonta and Plattsburgh were all involved in it. We all have different projects going on but ours became the Smart Neighbors project.

Rebecca: Please describe what that is for those that don’t know?

Leigh: Basically I have always….Well, I love Oswego as a town, and I’ve loved living here and I’ve always wanted to do something that could give back. But, I’m a creative writer and, short of putting it as a setting in a lot of short stories… which I have done… that’s not really giving back… I have always worried since I’ve been here about the economic difficulties facing any new business. This is a stat from a few years ago, but one statistic is that a new business in Oswego has a lifespan of about 13 months… and that’s a terrible statistic. I don’t think it’s true anymore… I think there are great changes going on in town now… but, I wanted to do something with the town. My concept for Smart Neighbors was to have a lot of different disciplines collaborate in the promotion of a downtown independent business. It was a simple concept, because I didn’t have elaborate blueprints for what they should be doing or what we should be doing. I had no elaborate plans for what each individual discipline should be doing. It should be promoting the business. Period. ….and that’s sort of continued to be how it is. People take it as they can imagine it… and so a lot of very imaginative things have come out of that… the things that are not traditionally considered promotional materials… which, in fact, really are promotional materials.

John: What are some examples?

Leigh: A literary citizenship class that Donna Steiner is working with, because they’re mostly creative writers, they tend to do digital essays… but they’re digital essays that often have a fanciful story involved in them. So, if it’s a bookstore… one digital essay took a book that the bookstore was selling… talked about the author ….did graphics about the plot of it… and then ended up back at the bookstore… and so you basically you were interested in the book… and then it began to talk about how the imagination could be served by the bookstore. Another one in the same class followed someone who bought a book to their home, took film clips and photographs of the person sitting where they liked to read with all of their books around them… and just talking about what it meant to be able to walk downtown and buy a book and take it home and start reading it. So, that was a nice little piece too. …but not things that you necessarily are expecting, or what an advertising agency would have put out.

John: …and how have the businesses responded to this? Have they been using these materials in their marketing?

Leigh: They have. One of the things that is a centerpiece is the banner… and the art students… the photography students have been at the heart of that… and all of the businesses end up displaying it. There are huge banners… they fill a whole wall… but all of the businesses have been using the banners. They love those… Also, every business nowadays… and this is one thing that we’ve been working with the businesses on… having an online presence… but that’s one of the reasons there’s so many digital projects involved. Because we want the businesses to be able to use them online. So, the digital essays do get used online as part of their presentation to the public.

John: …and how have the students reacted to doing something where their work is going to be more public? They’re not just submitting something read by their instructor and their peers, but it actually may have an impact on some business in the community.

Leigh: The impacts on our students are the impacts that I think they’ve found across the country when dealing with applied learning, civic engagement, volunteerism… well, basically best practices in general…. but, number one (this is the thing that I’m most proud of) is that the students leave that program, even though it’s one assignment in one course (for most of them… it’s not the whole course) but they leave having experienced that assignment with a sort of sense of social responsibility that I don’t think they had before… or a notion of philanthropy. One of the things I tell them…. All of the classes (this year we had 11 classes from different disciplines) and we all meet at the beginning of the semester in Marano Auditorium… and one thing I told them this year is that we think of social responsibility as as one thing and philanthropy as another thing… but really, I think, what we should be doing in these places we love (like I love Oswego) is actually contributing our talents… not just our money… but we should be spending our money locally too – but but also contributing our talents – to these businesses… even if we’re a business owner contributing to another person’s business is something that I think we’re obliged to do too – because the local success really is our own success… and we tend to think of businesses as competitive, but I think that’s a mistake. I think smarter neighbors…

John: …hence the name…

Leigh: …work together in these collaborative ways.

Rebecca: What are the biggest lessons that you’ve learned from doing some of these projects?

Leigh: Well, I should tell more of what the students got out of it, but…

Rebecca: Yes.

Leigh: I think, other than just that sense of social responsibility and what notions of philanthropy, they leave knowing much more clearly what they know and have learned in their disciplines… meta-knowledge of what they’re capable of… which is huge for our creative writers. Because, I don’t think they’re clear on the fact that… they know they’re probably not going to immediately get the Pulitzer, but what can they do with this? …and it’s important for them to to learn that…. that they can write for multiple audiences in multiple ways. But, they also learn what other disciplines know and can do… which they haven’t thought about that deeply. It’s a mystery to them what, for instance, the marketing students do. They market things… maybe it’s advertising… something like that… but then they see them come in and actually take the business that they’ve been working with and figure out a plan for them… and how the college itself can be moved into that plan… and suddenly: “Oh, I can work with that…” and they start thinking of digital essays they could work with… and imaginary stories that take that marketing plan and actually enact it with characters (which they’re good at imagining)… And professional skills… just getting somewhere on time… being late or on time to a class seems less important, but if the interview that you needed to have… and you’re late for and you can’t now have it… that makes an impact forever. You tend to be on time for an interview… and they do have to interview the local partners. Preparations… They get there. Nobody’s going to be telling them what to do. They have to figure out what they need to know, and they need to find it out. So, they need to plan before they get there. I personally am very happy with my students learning what it means to write for a particular audience, as opposed to whoever they want to. It’s very good for them to try to please a certain person with a certain product.

Rebecca: Because it’s usually an audience that they wouldn’t have picked or imagined on their own.

Leigh: That’s right… that’s right. …and my point would be that, even when they’re writing their Great American Novel, they should be expanding their notion of what audiences they’re hitting, instead of just “this is what I want to read.” They need to think about what their vision of the world is and how to can pull as many people into it as possible. I just think it’s memorable to them. I think it’s life-changing to them to work, however briefly, donating their time to a place at least for a while they’re calling home.

John: Excellent.

Leigh: But, I think they’re things that the faculty learned too… not just the students, there are faculty outcomes, I think, as well. My whole idea in Smart Neighbors was to just get faculty’s feet wet with one assignment in one class… and if you can do that… once they see the effect on students… because, that’s one thing I really do believe about the faculty here… they really are committed teachers. Now sometimes you worry about how time-consuming is it going to be to work with another class as other disciplines… how time-consuming is this or that? Because we’re already putting a huge amount of time into our teaching. So, it seemed smart to get faculty accustomed, or introduced to, collaborative, or civic engagement, or applied learning kinds of pedagogy in the easiest possible way. So, one assignment… and not an assignment that necessarily requires interactions with a lot of other faculty to figure out how to do it. Now, I will say, for Smart Neighbors anyway, the faculty do have to connect with the local partners. But, they don’t necessarily have to figure out what everybody’s doing in all of the classes to make it work. They have their piece of the puzzle and they’re contributing it.

John: How many classes work with a particular business? Are there multiple businesses that they’re working with? or is it just one business each year?

Leigh: Well, it’s grown. The first year, we had four classes and they were working on the bookstore. The River’s End Bookstore.

Leigh: Tell me your question again.

Rebecca: Really asking whether or not there is more than one community partner at any given time.

John: Yes.

Leigh: Yes. I think what you’re asking is a good question because, once you get to a certain number of people… of courses… not people, but courses… you’re overwhelming a local partner and we got to that quickly last year. We worked with a candy store (and I think there were seven different classes involved) and an unbelievable generosity of time from that owner… but it was clear that we were gonna have to figure out other ways of doing this. So, last year we did the Farmers Market, which worked out, We had eleven courses involved too – and that worked out much better because there are multiple farmers bringing their goods to the Farmers Market and there are they’re in different groups with different farms. So that worked out a little bit better. Also, because the Chamber of Commerce is ultimately responsible for the Farmers Market, we were able to do some projects just for the Chamber. For instance, they needed a new logo and we sort of pulled that into the Smart Neighbors project as well. So, I’m trying to define what we’re doing a little wider. …and you’re right, have more local partners…. if we’re gonna have this many continue.

John: It sounds like it’s grown really quickly.

Leigh: It really has… and I will just say again I think the faculty discovered that there’s a certain ease of practice in getting used to this… and once you see the students and the effect on the students, then I think you’re hooked. And the reason it’s grown is that the courses who have done it in the past continue to do it; they want to keep doing it. And that is how I got the idea for Grand Challenges.

Rebecca: That seems like a nice segue right into it, right?

John: Yeah.

Rebecca: So, we’re launching the Grand Challenges. Can you talk a little bit about what the Grand Challenges are and what the goals are?

Leigh: There’s a line in our strategic plan that that’s my favorite line…. and I think it’s the most memorable line… and it talks about how we, as a community, are going to tackle the Grand Challenges…. find solutions to the Grand Challenges of our time… and I love it… because it’s aspirational for one thing. I really do want to believe that our students and our faculty can tackle the Grand Challenges of our time, and I think we can, frankly… but it’s also that notion of “tackling a challenge” is very project oriented. You get your hands dirty. You figure out something, and then you try to come up with solutions because of it… and, so it appealed to me just in terms of having a common problem. But, those Grand Challenges have to be tackled together. I mean, I don’t think there’s any challenge of any size in the complexity of our world today that can be done by a single person just sitting in their garage thinking. I think almost everything we do in the future is going to have to be collaborative and probably cross-disciplinary in some way. So, it just seemed to me a natural segue from Smart Neighbors to getting the whole campus to work on a single… it’s not really a single issue either…. it’s more… we were talking about this, Rebecca, I imagine the topics for Grand Challenges to be very concrete things, because I think, as academics, we tend toward a more abstract way of looking at things…

Rebecca: …which is particularly hard for our students to get their heads around. They need something tangible.

Leigh: Right, I think so too… and to come up with projects… actual projects that are going to take place in the world with local partners… or involving civic engagement or volunteerism… require a certain concreteness. So, at any rate, the Grand Challenges project was just something I began to think. The notion of having multiple disciplines work on the same thing… it’s just a short step to getting the entire campus to work as much as much together as possible on the same topic… One of the things I didn’t say about Smart Neighbors is that Oswego is already a very collaborative culture… and that we’re very far along in terms of faculty tipping into these kinds of projects very easily…. and I’ve found just talking across campus, the way for instance when I spoke to Faculty Assembly, and the reception there was so astonishing. People aren’t resisting it out of hand. It’s just such a pleasure to work with people who are willing to take on these new things without immediate misgiving. At any rate, as you know, the topic that we’ve that we’re working with this year is fresh water which is concrete, but also can involve a lot of sustainability.

John: …but fluid, too.

[LAUGHTER]

Leigh: Very funny, John…
But, one of the things that I like about that particular topic is that you can look out any window on campus and fresh water is exactly what you’re looking at… and that it should matter to us makes sense to me. But, to go back to the teaching culture here, I have found when I talk about this to any group of faculty, immediately ideas are popping. They’re thinking about it. They’re talking about it. They clearly already thought about it. The Grand Challenge doesn’t really even begin until the fall of this year… and I’ve got a list …I brought with me a whole list of like couple of dozen projects that people are already doing right now…. this semester…

John: In preparation?

Leigh: Just because they can. Not only in preparation, just… let’s begin… Why wait till the fall? I’ve spent the last week finalizing touches to a micro grant the Provost office has, thank goodness, very gallantly is going to put some money in a pot to give some grants to people to do these collaborative works. Well, let’s just put it this way… even if you’re just doing an assignment in your class, you can put in for one of these grants. But, I think we’re going to privilege, probably, the collaborative civic engagement projects… or they’ll get the higher money amounts, just because there are more people involved. The administration on campus has just been so supportive. The provost office is doing the micro grants. The Student Affairs has, I can’t talk about it because the contracts haven’t been signed, but they’ve got people who are well-known coming to speak on campus.

John: So, there’s going to be some other programming throughout the college.

Leigh: That’s right. Artswego has a special category for its grants this year that are going to privilege some Grand Challenge proposals.

Rebecca: What I like about that concept is that the learning doesn’t just happen in a classroom on a college campus. It’s happening from multiple perspectives and it’s happening in and out. It’s happening formally and informally.

Leigh: That’s right.

Rebecca: That’s nice that there’s a lot of systems in place to help support that and that idea because, if students are experiencing the topic of water, in a lot of different disciplines on and outside of class right then they’re gonna start seeing how all these things connect together…

Leigh: That’s right.

Rebecca: …and we have general education as a part of our curriculum, as many colleges do, and the students tend to not have any idea how that is relevant or important or what that does for them. and I think this might be a really great way for them to start seeing that all these things are actually connected and it’s important to know different points of view and the different disciplinary perspectives on things… so that there is that idea that we can’t tackle these really big problems…

Leigh: …by ourselves.

Rebecca: …without looking from multiple perspectives.

Leigh: Yeah.

John: …and faculty are often in their own silos and students see the classes as separate islands that are not connected in any way… and showing that we can look at the same issues broadly from a number of different perspectives might help them form better connections and deepen their learning.

Rebecca: …and even continue to update the curriculum to reflect this change in practice. It’s a move away from silos to things being a little more messy, and so how do you allow for your curriculum to embrace that messiness.

Leigh: I think you’re exactly right, Rebecca. …and I, for one, think the future (it might not be in our generation) but the future really will be a future that doesn’t necessarily have departments… doesn’t necessarily have disciplines separated in this way…. that in fact encourages cross-disciplinary activity. I think the School of Communications, Media and the Arts [SCMA] is already sort of moving toward that. They’re a very collaborative school and work very well… that just in my experience doing these projects, they work very well across campus with any discipline.

Rebecca: Go SCMA.

[LAUGHTER]

Leigh: I am on the board. It’s because of that that I asked to be on their advisory board, frankly.

Rebecca: Yeah.

Leigh: But, yeah. I think the beauty of the grant, of Grand Challenges, is that we’re already a collaborative school and this just puts the name on it. It puts a focus for that and it’s something I think we really ought to be celebrating here. …and to get back to the administration being supportive, the President from the beginning has been behind this and I think that, really more than anything, has been one reason for this to be a successful rollout.

Rebecca: Are there plans to research or study the outcomes of the initiative to measure what impact doing something like this has on our learning community and/or on the community at large?

Leigh: Well, one of the things that I hope from these micro grants is, because they have to give the proposal at the project proposal… and give what they hope the outcomes will be… and then when they do their final reports, what they think the outcomes really were. I’m hoping that that will be the first step toward being able to assess some of the things going on. It’s more difficult in the general population, One of the things I’m reluctant to do is add a layer that makes people hesitant to get their feet wet with these pedagogies. But, I think, just once this gets going… I think it will become easier and easier to get people to assess for what the outcomes are. To be honest, I think it’s so night and day what the students get out of these best practices that the faculty will want to start assessing and seeing what these outcomes are and what it means in their classroom.

John: In an earlier discussion, you mentioned that your work with the Digital Oz project grew out of your work with the Smart Neighbors project. Could you tell us a little bit about the Digital Oz project and how it relates to your work with Smart Neighbors.

Leigh: Digital Oz is a presentation… online presentation site… for SUNY Oswego students’ digital work.One of the things that occurred to me after doing Smart Neighbors is that these collaborative efforts on campus are here and gone tomorrow …because there’s no place to archive or curate the materials that the students produce… and so Digital Oz has become a space where the collaborative work can actually be presented. The students are doing such amazing work. It’s great that Digital Oz exists so that the students can have some sort of public presentation.

John: Could you describe Digital Oz a little bit for listeners who may not be familiar with it?

Leigh: One of the things that I’ve always liked about Oswego students is that they have authenticity that is almost indescribable… but once you see them tell a story, you feel it instantly… and so I think because our students all have these stories it’d be nice if we had a site that had them tell them. So, we created Digital Oz and it has different categories. One category… the students talk about how they ended up being passionate about what they’re passionate about here (whether it’s their discipline or some sort of co-curricular activity that they do) and what’s the story behind that. How do they become passionate about it? …and there’s some amazing stories there. Students who, for instance, work as EMTs on the ambulance service on campus have some unbelievably touching stories about why they care… about being able to go to somebody and help them. But, there’s another category that’s called “moments that change their lives” …the students lives, and they talk about them in very moving ways as well. But one of the categories, as I said is “Collaborate” and students who have worked together on projects put artifacts that they’ve created for those projects online… and those two are… I guess you don’t realize the range and creativity and professionalism of our student work until you start seeing it put together in the same place…. And Digital Oz, since we’re talking about it… I’ll just say it’s… digitaloz.oswego.edu is the website if you want to look at it. But, it’s a place, I think, high school students look at and find feel like they can have a home here.

John: Excellent, and we will share that link in the show notes.

Leigh: Thank you.

Rebecca: So usually we like to end with, “What are you going to do next?” So, you’ve got this big giant project.

John: It’s still under way…

Rebecca: You’ve got this big giant project. What’s down the road a little bit for you?

Leigh: Well, I really do think that the Grand Challenges is as grand as I’ll probably get.

[LAUGHTER]

Because I don’t know how I can get grander.

John: The very Grand Challenge.

Rebecca: Super Grand Challenges.

[LAUGHTER]

Leigh: I know… it will be like Mario. But, one of the things… I’m talking to the woman who’s in charge of applied learning at SUNY Central, and I’m gonna talk up the Grand Challenges just because I think it really is a harbinger of what the future is going to be, not only in terms of what you do in collaborative ways, or best practices but also in what it’s going to ultimately mean for what the shape of the university is. So, I guess I’m not going to become a traveling advocate across the campuses across SUNY, but I do think I really do think this is where the future is headed for higher ed. I hope so anyway. I do.

Rebecca: Great. Well, thank you so much for sharing what you’ve been working on, Leigh. I think everyone will continue to be inspired.

John:Thank you. It’s a great series of project.

Leigh: Thanks, you guys.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

15. Civic Engagement

Posted in active learning, authentic learning, collaboration, higher education, interdisciplinary projects, learning, pedagogy, project based learning
February 7, 2018

Real-world learning experiences come in a variety of flavors. In this episode, Allison Rank, a political scientist at SUNY-Oswego, joins us to discuss how she has built a course in which students organize and run a non-partisan voter registration and get-out-the-vote campaign. This project combines many of the best features of service learning and simulation.

Show Notes

Transcript

Rebecca: Real-world learning experiences come in a variety of flavors. In this episode, we’ll explore ways to combine the best features of simulation and service learning to increase learning in a campus-wide voter registration and get-out-the-vote campaign.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist,

John: and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.
Today, our guest is Dr. Allison Rank, an assistant professor of political science at SUNY Oswego. Allison is an expert in the role American youth play in the electorate and the founder of voter registration initiative called Vote Oswego. Welcome, Allison.

John: Welcome.

Allison: Thank you.

Rebecca: Today our teas are:

John: Yorkshire gold.

Allison: I don’t drink tea.

Rebecca: It’s an epidemic…. like… this is the third one, John.

John: I know, three in a row

Rebecca: Three strikes you’re out. No more non-tea drinkers… All right… I’m drinking English Afternoon despite the fact that it’s still morning… because I need it.

John: Just barely morning, though.

Rebecca: Good, okay.

John: So, what is Vote Oswego?

Allison: Vote Oswego is a student-run, nonpartisan voter registration and voter mobilization drive on the SUNY Oswego campus in the Fall of 2016.

Rebecca: What led you to start Vote Oswego?

Allison: So, prior to earning my PhD in Political Science, I actually spent three years working as a political organizer. I’d worked for a presidential campaign in the state of Ohio and then had spent a couple of years working with college students on a variety of non-partisan campaigns. One of the things I learned from doing that work on college campuses, is that when students have an interest in doing political work, there’s a lot of skills that they can get out of doing that work but that they don’t necessarily think about. And so once I was here…. I was here first in the Fall of 2014… and saw what the voter registration drive looked like, it was clear that students were doing some volunteering for it, and some students were really excited about it, but I didn’t think they were really getting any organizing skills out of doing it. They were sort of more just sitting at tables and sitting out with voter registration forms the same way they would sit out with cupcakes at a bake sale… it wasn’t really about organizing skills. And so I wanted to start a course or something here where the voter registration drive would become more about students learning to organize rather than just being treated as widgets to be organized by other people.

John: Did you do this as part of a class or was it a set of classes or a separate activity?

Allison: I actually was able to get permission to run it as a special topics class in Political Science. I’ve since gotten it approved as an official course, but initially I actually just pitched it as a practical political skills class where the students would come in and learn about grassroots organizing techniques and then get to implement those techniques through a voter registration drive.

Rebecca: I would imagine that a course like that would be particularly helpful to campuses that are more rural than urban.

Allison: Yeah… for me, after a couple of years here, I had a lot of students that wanted to get involved in politics but I’d end up in conversations with them about how hard it was to figure out transportation to Syracuse, figuring out logistics or the cost of doing it– we have so many students not only in terms of being a rural campus, but also in terms of the student population that’s also trying to juggle working and paid work that they need in order to be here. And so then, taking time out to do a political internship, especially with the schedule around an election, can be really challenging. And so being able to offer that opportunity on campus, and also around something that can give them course credit or internship credit without leaving the campus, and for something that the campus is already gonna put energy and attention towards, I think, is really helpful.

Rebecca: Is running something like this as a class common on other campuses?

Allison: I don’t know of other places where a full grassroots campaign has been run out of a class. It’s fairly common in Political Science to have some type of activity based around voter registration, right? So, for us in Political Science, coming up with civic engagement projects where you can avoid partisanship and partisan issues, is a really big deal. And so non-partisan voter registration drives around elections are a great place to do that, but often it’s asking students to go out and volunteer as poll workers, or do exit polls, or maybe helping set up a campus debate with a couple of candidates rather than really digging into an on-campus, full-fledged grassroots mobilization campaign.

John: Was it easy to keep it non-partisan in the classroom?

Allison: Oddly, it really was. The very first day of class, I ran an non-partisanship training with the students. So what does it mean to behave in a non-partisan fashion? What does it mean to keep your social media non-partisan through this event? What are the conditions under which you need to be non-partisan, right? Students signed up for this class because they’re people who care about politics, right? So many of them, I am certain, had very deep feelings about what they wanted to happen in this election. They weren’t allowed to talk about them if they were at a Vote Oswego event. I think there was a guideline around however many Vote Oswego students were hanging out together, that they could be recognized as a group of Vote Oswego students. If they were wearing their Vote Oswego t-shirt, they could not both talk about something partisan and have any reference to Vote Oswego in, for instance, a online social media “about themselves” section. And I would actually, essentially, run pop quizzes with them where I would try to get them to do something partisan, right? I would come up to them and say some incredibly partisan statement and they would actually have to practice what the non-partisan response would be.

John: That’s a useful skill, today.

Allison: Yeah!

Rebecca: Probably one th at a lot of faculty could use some training on, too, because politics come up a lot in classes. Can you give us an example of something that you would do?

Allison: Sure. So one of the things that is often defined as partisanship is if you endorse an issue that is so clearly identified with one political party over another…

John: …like science….

Allison: …even if you don’t, we would use things like building the wall, right? If you say something like “you should register to vote because it’s really important that we build the wall,” that would be considered a partisan statement from the last election, regardless of not mentioning a candidate or a political party. And you would get individuals coming up to the table who wanted to register that would say things like, “it’s really important to me that I register because I really care about maintaining woman’s right to choose” or “I really care about building the wall…” something that you could clearly align. And so I would do that to students, and they essentially had a set of responses they were allowed to give. So things like, “I’m happy to hear that you’re excited about what’s happening in this election, it’s really important that you get registered to vote” or they’re allowed to not and say, “I acknowledge that that’s something that you’re really passionate about.” Vote Oswego is non-partisan, we just care that you’re able to express whatever you care about, right? It’s sort of acknowledging that that individual has something that they really care about, but not endorsing it yourself.

John: How did they do in those pop quizzes?

Allison: They generally did really well. The first day, they would get really awkward and nervous and not know what to do, but after sort of half an hour of drills, they got incredibly good at it. It also helped that it was in the syllabus and they signed a contract with me that if I caught you violating the non-partisan mandate after one warning, you automatically got fired from the campaign, which meant you failed the course. So they took it seriously.

John: So it was somewhat high-stakes.

Allison: Yes.

Rebecca: A little arm-twisting there?

Allison: Yes. But they did really well, and actually after the election, a student made a comment in class where he basically said it’s really weird to me…. I feel like I know people in this class so well and we’re really good friends and we’ve worked so hard and I have no idea how anyone in here voted and I said, “that’s great, that’s exactly what should have happened, please don’t talk about it.”

Rebecca: Can you talk about some of the other results that you saw in your class?

Allison: Sure. There’s sort of two different sets of results, right? We talked about the students as having two identities in this class. They were both students and they were staffers and they needed to be concerned about themselves in both of those roles. So as staffers, they had fantastic success. The campaign registered over a thousand students on the SUNY Oswego campus, they helped over 1500 students request absentee ballots, they came up with some really great campaign strategies in terms of helping students with absentee ballots… get those mailed in… get those stamped… helping students get to the polls… building a great coalition with other folks on campus. As students, I think what was great is that because the students were out in the field and they were known as Vote Oswego students on campus. Their friends all knew that they were doing it, they were in those t-shirts all the time, they were visible. They took a real ownership over this project, in a way that I have a hard time envisioning getting students to do about short-term volunteer work, or sort of asking them to go volunteer with another campaign, or even the type of simulations that political science professors can get really good results with, in terms of learning outcomes, the type of ownership that these students felt and how seriously they took it, I’d be hard-pressed to get that result in another way. Because they took it so seriously, and because they took such ownership of it, I think their critical thinking and analytical skills really, really improved. You could sort of watch as we went through the campaign, students go from looking to me and looking to the couple of interns that we had in leadership roles of the campaign, to figure out sort of like “all this thing happened and what am I supposed to do” and “please answer this question for me,” to like, “well, this thing happened while I was standing at a table and here’s what I did,” and I would hear about it three or four days later as opposed to getting a sort of frantic, “help me figure it out.”

John: they started taking more responsibility–

Allison: Exactly.

John: –and making more of the decisions then just reporting back.

Allison: Exactly. And then also being able to constructively critique each other’s decisions once we– we called classroom meetings campaign meetings, right? So in campaign meetings, being able to say, “Hey, I know this is what happened last week, actually I think we need to fix it in X, Y, & Z ways.” Which, for those of us who have tried to get students to give critical constructive feedback on each other’s papers, it’s really hard to get them to engage each other that way, and the students really took to that sort of analytical and critical work with each other in really constructive ways by the end of the campaign.

Rebecca: So in addition to students finding that kind of personal ownership over the experience, what are some of the other factors that you think made this particular project, in this particular situation, really successful?

Allison: I think there are a couple of things that made this project work really well. I think that, one, is that a non-partisan voter registration drive is something that students can get excited about, even if they’re really uncomfortable with the idea of the conflict around politics. So students that are interested in politics, but don’t really want to be in the debates around politics, can latch on to this as a project that they can get excited about. So, for instance, we had a number of students from PR who took this class because they saw it as that they didn’t really want to get into politics, but they want to know how to run something big, and so this provides that type of opportunity. The second thing is that the calendar just works. So I think it can be really hard to get students excited on a project if they can’t actually take it through the finish line. And what works about a non-partisan voter registration and voter mobilization drive is that in most states, the voter registration deadline is around four to six weeks after school starts, and then you get about another four to six weeks before the election itself, and then you’ve still got another four to six weeks before the end of the semester. And so it perfectly stages itself, provided that the faculty or some other set of students have done some of the set-up, for students to come in learn a set of skills, build skills, execute, get a couple of do-overs, and then still have time to reflect on the project before the semester is out.

Rebecca: I think that’s one thing that’s really unique about the timeline, is that a lot of kind of activity-based learning or community based learning projects, they go straight to the end of this semester and it really is hard to build in that reflection piece, so it’s nice to have substantial time to do that, and really think through that, and do post mortems and plan for the next time around so that the next set of students can learn from the previous set.

Allison: Yeah, it worked really well. I think that space allowed for a couple of assignments, both in terms of a post-mortem and having them really think critically about what they would have done differently in what advice they want to give the next group, but also for those students who want to go into this type of work, a lot of it is contract consulting work. So you’d run a campaign, and then that campaigns over, and then, what do you do next? And so one of the assignments for the class was actually to apply… mock apply for many of them. Though, a few students who are graduating did really apply for different types of political jobs. And so actually learning how to translate this real experience into a cover letter, and into a resume, and being able to pitch that what they had done was not just work for a class, but was actually work for a campaign.

John: Excellent. Did any of them end up working on campaigns?

Allison: A number of them have had internships. Someone received an internship, I believe, in Senator Schumer’s office off of the experience in her application, for that was actually what she submitted for the final project in that class.

John: Excellent. How did students, in general, respond to it? What sort of feedback have you had from students?

Allison: From the student population on campus or from…?

John: Or… well, actually from both within the class and also more broadly.

Allison: So students within the class thought that it was an immense amount of work, but also seemed very satisfied with the experience themselves. The sort of anonymous feedback sheets that I did with students over the course of the semester, students repeatedly talked about how much they were getting out of the experience in terms of learning what went on, quote unquote, behind the scenes of campaigns and how much harder it is then it looks like it is on television, comments like that. For this student population more broadly, it’s been interesting. There were definitely a set of students for whom having the voter registration and voter mobilization drive become something bigger on campus. I think it felt a little bit intrusive, though I’d argue that that’s what grassroots campaigning look like, you’re just gonna get asked if you’re registered to vote four times a day, in the days leading up to the voter registration deadline, and not for… even the students in my class who said, “I think we’re bothering people.” I said, “you are bothering people, you want them to register to vote.” So there was a little bit of that. On the other hand, students were really excited and I’ve actually had a number of students ask me if I’m running the class again, when the class is running again. Sort of having seen it happen, are really interested in getting that experience.

John: Very good. If someone were to stop in on your classroom, what would it look like?

Allison: I suspect it would initially look like chaos. [laughter]
The campaign classroom, I think, is a very different feel than a lot of other classrooms. After the first couple of weeks, I basically demoted myself to note-taker. I was technically the campaign manager, but I was really there to act as a check if I thought they were straying into something that potentially– this never happened, but I essentially was there to see do we stray into something that potentially smacks of a real problem, right? …in terms of their regularly… like election law regulations or guidelines for the campus, keeping track of the money that we still had, and what we could spend money on in the overall campaign calendar. But I would most frequently in that classroom, whoever was in charge of running a particular campaign team that was working on a strategy, would be running the meeting and I’d be at the front of the room essentially taking notes on the giant whiteboard in order to track the conversation and basically remind people of what decisions needed to be made before we left that campaign meeting. There’d also be a number of classes where you would have come to the classroom and no one would have been in it because there were either students out phone-banking, students were running a voter education program in one of our dorms, students were out running a training for other volunteers… sort of really being out in the field as much as possible and I was just running around trying to see what was happening in all of those locations and troubleshooting when it was needed.

John: So how many volunteers did they bring in from outside their class?

Allison: We ended up having over 250 unique volunteers from outside of the class that did work with Vote Oswego.

John: That’s impressive.

Rebecca: So you mentioned money and finances and so I think, I think that’s usually a big question for any sort of community project, campus project, etc. So how did you work the money side of things?

Allison: I think, one of the real benefits of the voter registration drive, is not only does it match the calendar, but it matches a place where campuses are already inclined to spend some money. So, I was able to put together money from a couple different places. One was actually from our Student Association. The student government here at SUNY Oswego put in it, ended up being close to $2,000, ultimately, that helped cover– it initially helped cover a bulk of the t-shirts actually, in visibility materials. I also put together money from our Community Services office, which is who had been coordinating the voter registration drive before. So instead of the money that they had spent running their own project, they were willing to put it towards this project. Which again, giving students access over how those dollars are spent I think was really important. I also was able to get resources from a couple different places as a faculty member, so I pitched Vote Oswego originally at a faculty academic affairs retreat at the start of a year and received $1000. The idea was voted as a best new innovation for teaching and learning at SUNY Oswego’s campus. And then I also received a Curriculum Innovation Grant here at SUNY Oswego that helped cover for me traveling to grassroots organizing training with the new voters project, to essentially get a refresher. It had been… let’s just say I was not text messaging…. that did not exist when I was last organizing… so getting a nice refresher on what sort of the the modern techniques were and best practices was really helpful.

Rebecca: How can others get involved like this particular project on this campus or run similar projects?

Allison: Yeah, so on this campus, faculty or students or staff that are interested should just shoot me an email. Definitely trying as soon as possible to start ramping up the plans for the 2018 midterm version and really starting to lay the groundwork for something big in 2020. Folks on other campuses that are interested in figuring out how this project worked, I actually just had a co-authored piece come out in the Journal of Political Science Education, it’s available as of yesterday online entitled “Vote Oswego: Developing and Assessing the Campaign-as-Course Model” that does quite a bit to outline how this project can run, where it fits pedagogically in sort of that space of taking some of the best parts of both simulations and service-learning. That article actually includes quite a bit from the course calendar, assessment strategies, as well as some student outcomes. And I want to point out that that piece was co-authored with Angela Tylock, who was one of the lead interns for the project. She graduated from SUNY Oswego in Spring of 2017.

John: Very good. We’ll include a link to that reference in the show notes.
What specific guidance might you give to other campuses trying to do similar projects?

Allison: I would just really encourage faculty and campuses generally, even if you don’t want to run it as a credit bearing course, to figure out how students can take the lead as organizers. I think, too often, students become the volunteers, right? There’s sort of a whole apparatus with lots of different nonprofits that are doing really good work to get students to vote and that’s really important, but I think on campuses, we’re missing really big opportunities if we treat elections as an opportunity to get students to vote, but not as an opportunity to get students the skills that they are gonna want and need for a whole variety of things. If you want to go work in a non-profit, you’ve got to know how to build a coalition. If you want to work for your kids’ PTA and make sure that they’re getting the resources they need, the ability to run a meeting and get petition signatures, is actually really important. And all of those types of civic skills are things that students can and, I think, should be getting by volunteering or helping to run one of these drives.

John: So it’s very much an active learning exercise…

Allison: Absolutely, absolutely.

John: …where students played an important role in building it.

Rebecca: How did you get students to take that active role? I mean, it’s easy to assign tasks and be the leader, but how did that feel?

Allison: There was… definitely, I had to be fine with a level of loss of control that I am often not fine with in my classes. What I did is actually work to set up the first two weeks of the semester, I had the calendar planned out, so I had worked ahead of time to set up tables and events that were happening for orientation, had coordinated with faculty around campus to have individuals come in and give announcements and register students in that first 10 minutes that really, the first week of school, you can almost always, the first day of class, give up 10 minutes, after you review the syllabus, to get some students registered to vote. After that first two weeks, the students who were enrolled in the course, had had an opportunity to be trained in those skills, to get their feet wet in the skills, to get feedback on the skills and then I didn’t plan anything else. I basically said, “now it’s up to you, what are we gonna do?” And goals have been set for the campaign so the students knew, “here’s where we want to get, here are what we think our rates are gonna be. So if we want to register 500 people from tabling, here’s how many table hours we need scheduled, how are we gonna make that happen?” And I think two things then happened. One, I stepped back and basically told students, “you’re the expert on where students on this campus are.” I come here, I go to work, sometimes I go to events, and then I go home. I don’t actually know what dining halls are packed on what days. Turns out chicken sandwiches, big deal, chicken sandwich day at late night, right? There are all of these things that happen on campus that, as a faculty member, I don’t know about. So students basically learned that I wasn’t gonna tell them not to register students at 11 p.m. at night if that’s where they thought students were, they ran with it. The other thing that happened is they realized that I would let things fail. If students scheduled events and those events went poorly, they went poorly. And I wasn’t gonna fix those events for them– with the exception of confirming that registration forms were filled out correctly. We had an entire process for making sure that voter registration forms were correctly done. But in terms of the grassroots apparatus around that, if students didn’t plan well, they didn’t plan well, and they were the ones that had to stand there while the event went poorly. And I think between those two things, the students really became engaged around sort of their responsibility and taking ownership over the campaign.

Rebecca: Was most of the learning then taking place by “let’s try this, let’s fail, let’s try again,” an iteration rather than like doing readings or other kinds of … ?

Allison: Yes, there were minimal readings while the campaign was actually happening. There was quite a bit of reading and reflecting once election day happened, but prior to that, it was much more “these are the tried-and-true tactics, what do you want to get out and do? How do you think these tactics will best adjust to the environment that you’re in and the student population were working with?”

John: And students learn a lot more by making mistakes and recovering from them, and it sounds like you set up a mechanism where there was lots of feedback from each other.

Allison: Yes.

John: That’s excellent.

Rebecca: So, usually we wrap up these conversations by asking what are you gonna do next?

Allison: Next for Vote Oswego is an effort to improve the connections between the voter registration drive as a grassroots campaign to the voter registration drive as an overall campaign that involves lots of different components. So actually, Rebecca, had a class that worked on the website for Vote Oswego, it was a project for one of her classes we’re hoping to do, I think, much more of that for the 2016 campaign as well as trying to figure out what other faculty or other classes could also use a voter registration drive… benefit from that timing… benefit from the fact that it can be student driven and student owned in a lot of ways, to really get their classes involved with this as a project.

John: Very good.

Rebecca: Thanks for taking the time out and sharing your project. I think it probably has encouraged a lot of people to start thinking about those midterm elections and how they might be able to get students tapped into it.

John: And will you be doing this every other year now?

Allison: The goal is to do it every other year. I haven’t done it for a midterm yet, I think that will be different. I think there will be more actual campaign literature there just because it will be difficult to get it sort of as ramped up as a presidential election, but the goal is to do it every other year.

John: Very good. Okay, well thank you.

Allison: Thank you.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

13. Authentic Learning

Posted in active learning, authentic learning, collaboration, higher education, learning, project based learning
January 24, 2018

In this episode, Rebecca Mushtare discusses how she has used community-based learning and simulation projects to provide authentic learning experiences in her design courses.

Show Notes

Transcript

John: Today, our guest is… Wait, there’s no one in the guest chair. Who’s our guest today?

Rebecca: It’s me! It’s me!

John: Oh, yeah. Okay. Today, our guest is Rebecca Mushtare, who will be talking about how she uses authentic learning techniques in some of her classes.
Today, our teas are:

Rebecca: Comfort and Joy.

John: Peppermint Bark. So we’ve got some holiday tea left over from the holidays. So when people talk about authentic learning, what do they mean?

Rebecca: It really means something, it’s like a real world problem of some sort, or where students are gaining experience as a professional or in something that’s very similar to a professional. A lot of times, authentic learning experiences include ill-structured problems. So not like the kind of question-and-answer things that we might have in a very structured classroom context, but where it gets messy. There’s variables that we can’t necessarily plan for in advance. That often happens and then a lot of times they’re also project-based exercises or experiments too.

John: So, one of the main reasons for using these authentic learning exercises, besides providing students with training that’s relevant for their field, it also provides them with learning experiences where there is quite a bit of intrinsic motivation, right?

Rebecca: Yeah, I think that students respond really positively to authentic learning experiences, because they can see how it’s relevant to them, and relevant to their professional careers. So even when I do small exercises in class, like writing an email as a professional, students latch on to that writing opportunity more so than other kinds of writing opportunities, because they understand that that’s important, relevant and necessary.

John: What types of activities have you used in your classes?

Rebecca: Well, at first it’s probably important to understand what kinds of classes I teach, to kind of get some context.

John: So, what types of classes have you used these in?

Rebecca: Yeah, so I predominantly teach studio- based classes, mostly web design courses. So, I’ll focus on those, because those are the ones of my regular load. So those are the things that I teach most frequently and I’ve done the most experimentation in. I do community-based learning or a form of service-learning, and I also do simulations, and it depends on whether it’s a beginning or an advanced class, which one I do. Community-based learning, or these service-learning, opportunities are generally working with a community client… generally a nonprofit organization, who doesn’t have the capacity or the budget to hire a professional design agency to do something. So we’re providing a service in a way that builds their capacity. In my advanced classes, I’ve done a lot of community-based learning. Locally, we’ve done the Children’s Museum of Oswego website, the Childrens’ Board of Oswego website and students are also wrapping up a project for the Oswego County Airport. So all of these are possibilities where they get to design a real website… they work as a team and I serve as the creative director, so this is different than an internship or other opportunity where they might get real-world experience because they’re getting a lot of coaching throughout the entire process, that they may or may not actually get in some of those other contexts like a volunteer or an intern.

John: When you serve as a creative director… could you provide a little bit more detail on that role for those of us who don’t work in those areas?

Rebecca: Yeah, it’s a step up from maybe a project manager and that I oversee all of the creative decisions, including what research methods are gonna be used to learn more about the audience, and the client… making sure that whatever meetings and things we have are all scheduled through me, so that I’m highly informed and participating in the project. It’s not something that students are doing without me being involved. It’s not an “outside-of-class assignment,” where they’re just doing a group project and they do whatever with the community and then show up and it gets done. Rather I’m heavily engaging with that community partner as well, and this is really important because of the longevity of the project. At some point the students are gonna go away. The semester ends… and the project may or may not be done. I either need to have another class that finishes up the project… I might finish it up…. or whatever needs to be done, I need to make sure that that continues, because the timeline of a client or community organization is quite different from our semester schedules. My involvement is really important in that role.

John: …and that helps with buy-in from the community partners, it provides assurance that the tests will actually be completed…

Rebecca: Right, and to actually to be able to do a project like this in a semester requires some significant planning on my part with a community partner in advance of the semester. So, I have to really understand their needs ahead of time to make sure that they are not far beyond what my students are capable of with my help. I also have to make sure that they know, as the community partner, what they’re gonna need to have ready so that the students can actually get to the part that they need to do. With web design something that most people don’t realize is that there’s a lot of writing content… and designers don’t write the content. The community partner or the client does. They need some coaching through that, and so I help facilitate some of that. Some of my scholarship as a professional is in that area where I’m working with these community partners, as a professional and as a consultant.

John: How much of the interaction with a partner is done by you, and how much is done by the students?

Rebecca: It’s a little both. At the beginning, well before the semester starts, I’m the one that makes contact with the community partner. We figure out how the semester is gonna be organized, establish roles and responsibilities, usually some sort of agreement. I usually make sure we negotiate some sort of copyright agreement that favor students using stuff in their portfolios, and set all those things up upfront, then we usually set a project launch date, and the client will come to campus. I make sure they’re available during my class time, and that they’re available pretty regularly through the semester and can come in a week’s notice, so they block out that time slot, so that they can come. So they come… I help the students prepare for that meeting. They ask questions… Q&A… so that the students learn about the community organization and what needs to happen. If it’s possible we usually schedule a trip to the community organizations, so we can see firsthand what they do, and so the students are interacting directly with a client in those circumstances… and then it depends what else needs to be done. So, for example, with the Oswego County Airport project that we’re finishing up, some of the students did some photography and things on the premise so they coordinated directly with a client to make arrangements for what time and that kind of thing. So sometimes it’s easier for them to do that communication, but largely if it’s about approvals and things like that, that all goes through me, which is in keeping with how it would be in a professional environment, where the creative director or an art director or someone above entry-level designers would be the ones having that contact.

John: From the students perspective, what are some of the benefits of this sort of project?

Rebecca: They’re really excited because they end up having portfolio work, which is important. They can put a line on the resume, essentially saying that they worked on a real project, that’s really being used, and then they also get to see an entire project all the way through. So in these cases, what we’re doing… community based learning or these community projects…they are able to participate in the research, development, design, the whole shebang, but usually they pick one rule that they do in depth which is something different than I would be able to do in other contexts. For example, someone might be the developer, or one of many developers, or someone might be a researcher primarily… even though they’re working on all the different parts of the project. So, they like the fact that they can do some work in depth. Usually in these classes I’m doing two big projects. So they’re doing this one and then they’re doing some sort of other individual project that complements it in some way.

John: So how much of this is done with teams of students working on the project and and how much of it is done by individual students working on individual components?

Rebecca: Well, the whole thing is usually a whole class project, which means that I really need to make sure that all the moving parts are working together and coordinating and what-have-you. We use Slack which is a team chat that we use outside of class to keep in contact about different things… and this last project we did something called “playbacks.” So, one day a week we did little playbacks about what everybody was doing and what they’re up to and what they needed from other individuals to keep the lines of communication open… and then certain roles and things are maybe small groups that need to work together to get particular pieces done.

John: You mentioned the portfolio piece for students. If they’re part of this big group, how do they identify the components that they worked on?

Rebecca: Yeah, we talked a lot about portfolio documentation, because working in a team is pretty standard protocol in the field that I’m in. What students do is they document the entire project, but they specify in that documentation what their role was… and so they always credit all the other people that worked on the project.

John: Excellent. What are some of the challenges that you face in working on an authentic learning project? ….with standard projects where you have a very finite well-structured problem, it’s fairly easy….well, at least you control the environment much more. When you’re working with someone in the community and you’re working with real-world development, what are some of the challenges unique to that type of framework?

Rebecca: Yeah, there’s many… [Laughter] One of the key issues is timeline. The timelines never match up, and so you always need to have a back-up plan for how something is gonna get finished… because it’s almost never totally finished during this semester. So, sometimes that means some people in the class are doing an independent study to finish stuff up…sometimes it means I’m gonna do something… sometimes it means another class is gonna pick up the pieces… or whatever… but that that needs to be in place, and that needs to be in place from the beginning. It’s really important for it to be in a learning environment that students can fail safely. They need to be able to screw up and that be okay.

John: It’s certainly safer for them to do that on this project than on their first job.

Rebecca: Right, exactly… and so you know part of my negotiations at the beginning of a project like this with a client is letting them clearly understand that this is a learning experience and learning comes first from my perspective, but that their needs will be met, but it might be met on a longer timeline than they really want.

John: …or perhaps a more iterative journey than they expected.

Rebecca: Yeah, exactly… and in most cases the community partner is more than happy to participate and especially because I always see the community partner as a co-teacher… they’re there to teach certain lessons, too – and that might mean letting us know that a student’s gonna fall flat… and letting them do that… and then help them figure out how to do it better next time… and give them that next time as part of the project. So, there’s been many times where some someone maybe provided a deliverable that wasn’t quite up to snuff and then deadlines had to shift so that that person could revise and meet the standard that needed to be met. So the students are generally working at a much higher level because essentially they can’t really fail. They can fail and revise… and revise… and revise, but eventually they get to a minimal standard… which I find to be helpful… and then the other thing is you really have to be flexible. All kinds of things happen… an organization’s budget can totally become a disaster and they have to refocus their attention on something else… and so you might feel abandoned.

John: While it’s the main focus of your class…

Rebecca: Yeah…

John: …it may not be the organization’s main focus.

Rebecca: Right. …like any of these things can happen without you expecting it even if you have you think all you have you have all your t’s crossed and i’s dotted at the beginning of the agreement. So that happens. Sometimes, students just don’t follow through in the way that you think that they’re going to or it or that you know they can… and so like what do you do in those situations? You kind of have to have those kind of failsafes in place. This is one of the reasons why, to provide an authentic learning experience for beginning students, I moved away from community-based learning. I used to do community-based learning in my beginning class. I do it now, but in a very different way than doing an actual website project, because there’s too much at stake there. So, I say that I save those experiences for my advanced students.

John: Going back to that… in your beginning classes what do you do differently to create the same sort of environment, but perhaps with a little less risk?

Rebecca: Yeah, I do two things. One that is a community-based project… and that’s what I call a consultation report. What they end up doing, in that respect, is, instead of doing a full design project for somebody, they do some of the research and analysis and do some proposals… some ideas… that we then hand over to the client that they can then use to either hire my advanced class or to hire a designer to take on but they understand more where they’re situated and so as part of that we do some accessibility testing… we do user testing… and things like that…. and so we’ve done that for a couple of different organizations, and that’s worked out pretty well. That gives students an opportunity to communicate with a client a bit and also do some formal presentations, which is nice…. and then the one that I use probably more frequently in my beginning class is a simulated client project. I have established a few scenarios that our clients… they have specific goals and needs… they have personas…. they have email addresses, etc…. and then students will work in small groups and then they communicate directly with the client all through written communication, although they can schedule an appointment…. I do have heads on popsicle sticks in which case they can meet with their puppets…. [laughter] which is always surprising to them because I don’t tell them upfront that I do that. So they come to my office and my door is always shut… for that situation I’ve reorganized my office. They knock on the door and there’s a head on a stick… and if they laugh I shut the door… and they have to start over. They have to take it seriously.
[Laughter]

John: So, for the artificial clients, you create the email addresses and it will go to you?

Rebecca: Yup.

John: …and then you will respond as if in the role of the client.

Rebecca: Yeah, they each have a personality. So there’s four or five different clients. They all have very different personalities… and students start talking about their clients and the different kinds of ways that they behave. They have certain ways that they open and close their emails. One’s very curt and aggressive. One is very grandmotherly… very caring and kind.

John: Do you ever get them mixed up?

Rebecca: I have little notes when I start doing it that I keep on my laptop… a sticky note that just reminds me… a couple key words like who is who, so I don’t get confused.

John: Yes, that could cause some problems if you went from the very curt person to the grandmotherly person…

Rebecca: Yeah, and then if a student emails their client and they’re out of bounds or something then I email back as the professor from my school email address… and it says “This is a note from your professor” and then I indicate what’s wrong… and I make them redo it.

John: So, how do they react to the puppet?
[Laughter]

Rebecca: They’re usually surprised but then they find it amusing… and they take it seriously… especially if I shut the door on them ‘cause they laughed at me… and they started over and I keep a straight face and whatever ‘cause you just know you never know who you’re gonna interact… and so the first time you meet someone you could be surprised, right?

John: It could be someone who’s a puppet.

Rebecca: It could be a puppet… you just never know… so, yeah, they generally respond pretty well to that… and usually if they meet with me in person as the client, then after that meeting I make them stay for a couple minutes and we just talk about how it went and things that they could have done differently.

John: Excellent. In an earlier podcast interview with Stephanie Pritchard, we talked about the Voices of Oswego Veterans project and that also seems to fit in as another type of authentic learning experience. Could you just recall that for people who may not have yet listened to that earlier podcast?

Rebecca: Sure. That project, in particular, The Voice of Oswego Veterans, was a collaboration between Stephanie Pritchard’s writing class, Peter Cardone’s photography class, two of Kelli DiRisio’s design classes, and my web design class. So, instead of doing my standard simulated client project with my beginning students, that group did the Voices of Oswego Veterans website. So that was somewhere between a simulation and a client because they didn’t have a direct client to talk to, but it was a real project and they had real content and real goals that they needed to meet… and that was taken really seriously by students and I think that was in part because it was going to be published. So, they didn’t get as much of the client interaction, which I think a lot of times the students value a lot from my classes, but it was still a very authentic experience and the students got a lot out of it and they were really committed to the goal of the project which was to dispel stereotypes about veterans. There’s a lot of assumptions that we identified early in the project… that people assumed that veterans are old… they associate it with World War II, and to think that “oh, wait, we have students on campus who are veterans, that just boggled some of their minds and we wanted to make sure that those students are seen as students as well.

John: How have students responded in general to the project?

Rebecca: I think, in general, students respond to any of these authentic learning experiences fairly positively. I think they all think it’s a lot of work, especially because the revision is taken a lot more seriously… and you think that that maybe wouldn’t be true of the simulation, but they get into it and they continue to revise and they want to meet and satisfy the client….that’s the goal at the end of the day. They need the thumbs-up from the client at the end… and so I think that is motivating and it seems realistic enough that they want to give it their all…. and that definitely is true on community projects. The one that we’re finishing up now, I have a student who graduated who’s finishing up a couple things that she couldn’t quite get to work the way she wanted to and she’s finishing that up right now

John: Okay, so I guess the next question is: “What are you going to do next?”

Rebecca: That’s a good question…. [Laughter] I should have known that was coming.

John: You usually ask that question.

Rebecca: Yeah, I know, right? So, I guess it’s only fair that it’s asked of me.
So, the next thing that I’m planning to do related to authentic learning is to emphasize thinking about audience empathy and stereotypes a little bit more. That Voices of Oswego Veterans project, I think, was particularly successful in helping students actively design to dispel certain stereotypes and I’ve really been trying to get students to think about audiences who are different from themselves… which is a challenge…. and that seem to work really well, so I’m trying to find a way to embed that more so in both my beginning and advanced classes.

John: Excellent. Well, thank you. This was an interesting discussion.

Rebecca: Thanks, John.

John: Looking forward to hearing more about it as the next semester progresses.

12. The Active Learning Initiative at Cornell

Posted in active learning, collaboration, higher education, learning, pedagogy, teaching
January 17, 2018

In this episode, we discuss Cornell’s Active Learning Initiative with Doug McKee, an economist at Cornell and a co-host of the Teach Better podcast. This initiative, designed to increase the use of active learning in instruction at Cornell, provides funding to departments to redesign courses to employ evidence-based active learning techniques. Doug provides an overview of the program and a discussion of how this program is being implemented to transform economics classes.

Show Notes

John: Our guest today is Doug McKee. He will be discussing the Active Learning Initiative at Cornell. Doug is an economist at Cornell and also a co-host of the Teach Better Podcast, which is one of our favorite podcasts on teaching and learning.

Rebecca: Today our teas are:

John: Ginger peach white tea.

Doug: I’ll be honest. I can’t stand tea.

[LAUGHTER]

Rebecca: Absurd.

[LAUGHTER]

Doug: It’s not that I don’t need the caffeine, but my preferred method of caffeine injection is coffee.

Rebecca: Yeah, that’s a bummer. It’s not “The coffee podcast.” There’s lots of coffee here..

Doug: Maybe next week I can be on the coffee for teaching podcast.

[LAUGHTER]

John: Or we could have a special episode.

Rebecca: Yeah.

Doug: Oh… a special coffee episode.

Rebecca: That would be kind of fun.

Doug: You’d need guest hosts.

Rebecca: Yeah.

John: The Starbucks next door is closed over the break, so we’re…

Rebecca: Anyways… I’m drinking a golden English breakfast.

Doug: Do you change the teas every single week?

Rebecca: We try to.

John: We generally do, but we got a couple hundred of them….

Doug: But, don’t you just have a few that you like?

Rebecca: I have a favorite, yeah.

Doug: Don’t you just want to drink that one all the time?

Rebecca: Sometimes.

John: Sometimes we do. I’ve used the same ones occasionally.The last couple of episodes, I had the ginger peach black tea….

Rebecca: …that’s his favorite.
JOHNL… and the ginger peach green tea… and this is a ginger peach white tea, which I drink later in the day because it has a little less caffeine.

Doug: uh-huh…

Rebecca: I like the English afternoon tea.

Doug: I would think in the afternoon you’d want the one with more caffeine.

John: I’m generally better in the afternoons and evenings. I have more trouble getting energetic in the morning.

Rebecca: John loves 8 a.m. meetings.

John: Could you describe the Active Learning Initiative at Cornell? How did it originate and how is it working?

Doug: The first thing I want to say is I’m a relative newcomer to the Active Learning Initiative. I joined Cornell about a year and a half ago, in large part because I was excited about joining the Active Learning Initiative. But it’s a project or really a program that started back in 2013. It was the brainchild of Peter LePage, who was actually a guest on episode 50 of the Teach Better Podcast and he is a former Dean of the College of Arts and Sciences. He’s a physicist and he had read a fair amount of the literature… the big literature on active learning in physics… and wanted to come up with a program that would actually instill this way of teaching across the college. He also happened to be a friend of Carl Wieman, and so he knew all about the Science Education Initiative as it was happening… and the Active Learning Initiative is a program modeled on the same principles as the Science Education Initiative… and the ideas that departments compete for grants, and once they get that money, they use it to hire people with disciplinary knowledge (usually postdocs into their own Department) who can co-teach with faculty and train faculty in how to use these methods. Because what I find, is, when you ask faculty why they don’t teach actively or why they depend on the pure lecture, even though there’s a fair amount of evidence that active learning works better on a variety of dimensions, “we don’t have the time”… well, the department education specialist or the postdoc can help with that…. they say “we don’t know how”…. and so the department education specialist comes in with that knowledge of how to develop clicker questions and what a good small group activity looks like, and then finally when they say ”well, I don’t believe it,” it’s the department education specialist can both reference literature… talk in the language of the the discipline… and…. and this is a big part of both the Science Education Initiative and the Active Learning Initiative, creates knowledge and actually try things in that discipline and evaluate what the effects are on student learning of teaching in new ways…
JOHN…and the STEM fields have been a bit of a leader in that. Physics, in particular, has been very active in development of this.

Doug: That’s right. So, the first round of the the grants funded by the Active Learning Initiative at Cornell were given to the Physics department and the Biology… actually it was a co-proposal by their two biology departments. Why? I don’t understand why universities can’t have one biology department anymore? They don’t.

John: Out of curiosity, what are the two areas?

Doug: Ecology and evolutionary biology… and then there’s the, I think, molecular biology and like these kinds of biologists…. I don’t know… I’m not a biologist, but biologists have a hard time talking to each other, it turns out. But these biologists could talk to each other even though they’re in different departments, and…

John: … maybe because they were in different departments it was easier.

Doug: …it was easier… You’re right, it was easier…

[LAUGHTER] as long as they just had to talk about teaching and not about who the appropriate person to hire was… and so those were the first two departments… and then since then, in 2017 there was another round of grants granted and one of those was economics and the idea there was to branch out beyond the sciences and the STEM fields into the social sciences and even the humanities.

John:… and this was funded by a donation from alumni, right?

Doug: So, the funding comes from an alumnus who had a passion for active learning. I think what happened was he and Peter were at a dinner… and Peter was talking about how excited he was about changing how he taught his classes, and how he wished that it would happen more often, and this person came up to him and said “I heard what you were talking about and I’d be really interested in funding that.”

Rebecca: That’s exciting.

Doug: On the other hand, the Science Education Initiative, as far as I know, was funded by internal money. So it was Carl Wieman was very persuasive in convincing the administration at the University of Colorado and the University of British Columbia, and now he’s at Stanford, and he has internal money at Stanford, to do a lot of the the same thing.

Rebecca: Funding is great to support some of these things, but I would say that our Writing Fellows program on our campus is not that different from hiring these postdocs, because we have writing fellows that are assigned to each of our schools. We have a previous episode on that topic with Stephanie Pritchard, and we have writing experts who meet with faculty in different departments to help them develop writing assignments to meet some of our standards for writing across the curriculum, and help faculty understand how to teach writing….

Doug: Right.

Rebecca: … in their disciplines. So, in some ways, it’s a very similar sort of model. It’s not about active learning, but it’s about how to do this writing infusion.

Doug: So in many ways, it does sound similar, but I’m gonna point out some of the differences.

Rebecca: Great!

Doug: These postdocs live in the department, and they have PhDs in that discipline, and so in some ways they get way more cred that way… and respect from the faculty, because they’re thought of as part of the department. Another difference is that the departments have to write proposals, so they’ve committed and they’ve said in the proposal how excited they are about this, whether they are or are not.

John: They’ve at least made a commitment.

Doug: They’ve at least made a commitment and said… stated on paper that they’re excited and they want this to happen and this should be a success. It’s not some external entity, and I think that makes a difference.

Rebecca: How long do the postdocs stay in the department?

Doug: So, it varies from department to department, but in our proposal we felt strongly that in economics the postdocs are generally two years, and so our postdocs are 2-year postdocs. But in physics, I think, the postdocs tend to be longer, and so their postdocs, I think, are three-year postdocs. One year is just not enough, because they arrive and they immediately have to start preparing to go back out on the job market, and try to get jobs elsewhere.

John: But I think you had mentioned that while they were there for two years, there were going to be a number of postdocs that were staggered over a four-year period.

Doug: Right. So our project is a five-year project and what we’re doing is we’re hiring four two-year postdocs starting, with one in the first year and then every year after that, and so during the middle three years of the program we’ll have two in residence and then in the first and the last we’ll have one.

John: You mentioned the Economics Department as one of the recipients of this. How many other departments were there?

Doug: Five. So the physics department has another grant to overhaul how they teach their lab courses, and this is actually… Natasha Holmes, I think, is arguably the world’s expert on really modernizing and changing how we teach lab courses; taking them away from following recipes to creating the recipes and answering interesting questions using experiments, and creating those experiments, and she recently joined the Cornell faculty. The music department is using a grant from the Active Learning Initiative to integrate active learning into a composition course where students all have keyboards in the classroom and the keyboards act a little bit like music clickers, so the…

John: with MIDI controllers, probably?

Doug: Right… and the instructor can then listen and select different things that different students have played and then play them for the whole class. It looks fantastic. Sociology is doing fairly straightforward integration of active learning into their large lecture classes, using group activities and clickers. They’re also standardizing, and making much more active, the discussion sections for those classes. The math department is overhauling their introductory calculus courses. Let’s see, that’s math, physics, economics, sociology, music, and then the sixth is classics…. and Classics is creating a brand new course that’s not a pure lecture course, that has students, I think, doing projects during the semester.

John: …and your department is doing how many classes?

Doug: We are treating, or transforming, eight classes?

John: Which classes are you doing?

Doug: So, we’re basically doing the entire core curriculum, or the required courses for the major. It’s not exactly that right now. We’re really doing seven plus a popular elective course called behavioral economics… and why aren’t we doing all eight of our required classes? Well, one of them is a class that we don’t actually own, but we’re hoping to somehow, over the next five years, find the money where we can treat that class too, because it would be a shame to just do seven out of eight.

Rebecca: How are your faculty in your department responding to the initiative and getting involved?

Doug: So we have a small core that’s very excited and they’ll be involved in the program early on in the first couple years. Then we have another set of faculty that are very excited in theory, as long as it’s far enough in the future…. And I think we have faculty that are not that excited, but aren’t actually being affected by it, and so they’ve been fairly passive. I’ve been thrilled not to have anyone in the department that’s actively opposed, and so it’s been great, actually, and the vast majority of the work happens when a course is actually being transformed. Right now we’re making it very easy on ourselves. The first course that we’re transforming is one of my own courses.

John: Which makes it easier.

Doug: Yeah.

Rebecca: Yeah.

Doug: So we can get the progress. We can just get the whole procedure right… get that stuff in shape as well as we can. So… okay, we’re gonna do a control course in the fall. We’re gonna do a treatment course in the spring. So we’re gonna teach it fairly plain vanilla in the fall, and we’re gonna treat it in the spring and we want really good measurements of learning. How do we go about creating those measurements of learning? Because unlike physics which has like 80 standard concept inventories, economics has one. It’s the Test of Understanding of College Economics, and it’s pretty good, but it’s only for principles courses, and so we have to build from scratch… and so next year we’ll actually be bringing more faculty into the fold, and we’ve scheduled it such that the fact that we bring in our faculty that are excited about the project.

Rebecca: What have you learned from the process so far?

Doug: Measurement is hard… and it’s not even that it’s just difficult… it’s a lot of work… and so we had a meeting with some of the folks in biology and where we talked about what we were doing and Ron Harris-Warrick said something. He said “It seems like you’re spending 90% of your time on measuring and assessing, whereas we spent 90% of our time actually changing the teaching” and I said “That’s exactly right” because we’re not changing the teaching yet and we want to know if, in fact, what we do when we change the teaching works …and so we need really good measures, and so that’s where all our investments been. Now, is that changing? Yes. In two weeks we start classes up again, and we start teaching the transformed class and so we have actually done a fair bit of work, and we have big picture. We know what we’re going to do, and we’re very excited about it, but we have a lot of work to do this spring on actually where we focus on changing the teaching.

Rebecca: So, we’re taking this week by week?

[LAUGHTER]

Doug: Well, it’s gonna be a lot of just-in-time curriculum development.

[LAUGHTER]

John: I’m very familiar with that process.

Doug: Right… right… right. I got to say, like after a semester of focusing assessment, which has been really fun, we’ve created what we think could be the start of two standard…. like two… we’ve tripled the number of standard assessments in the field in one semester. I mean… we have work to do still and we’re the only ones using it right now, but that’s changing. I am pretty excited to actually change how I teach this class and we have a lot of ideas about things we can do.

John: You’ve already been doing a lot of active learning in your classes before, so what are you going to be doing differently?

Doug: I divide it up into two chunks. So, we’ve got some high bang for the buck things… things like two-stage exams. One problem that we’ve had in the past, and I think it’s a very common problem, which is: you give an exam… the students take the exam… and then there’s a whole bunch they can learn after the exam about the mistakes that they made, and more than half my students don’t even come and pick up their exams. They see the number and then they… if it’s bad they get sad, and if that’s good they’re like I’m fine… and neither group is learning from their mistakes, and so what we’re doing is we’re saying you’re gonna take the exam… and then you’re gonna take it again in groups and then your grade for the exam will be a combination: 80 percent is your individual grade and 20% the group grade, and so at that point you’re actually discussing it. It forces students to actually discuss and talk about these problems in another step, and I think that’ll make a big difference.

John: You had a podcast episode on that not too long ago.

Doug: Yes, with Teddys Svonoros, where he’s been giving two-stage exams for quite a long time at the Kennedy School at Harvard.

John: So listen to that podcast. It’s a great way to learn more about that. We’ll put a link in the show notes.

Doug: It’s a great podcast.
Another high bang for the buck thing, we think, is assigning points in their grade to whether or not they participate in answering clicker questions in class. I have been, for my entire teaching career, adamantly opposed to giving points for class participation. I’ve always felt that it cheapens it, and I want students to be there and participate because it improves learning… not because they’re getting points toward their grade… and over the summer I was convinced that that is actually the wrong attitude. Students look at how you assign points for their final grade as a signal of how important things are.

Rebecca: …of what you value.

Doug: …of what I value…. and so if I say there’s no points for showing up at class and it’s not required, then they’ll be like: “Oh, he doesn’t think it’s important” and I can say it’s really important over and over and it doesn’t have nearly the…

John: Well, clicker questions offer some other things. It gives you some feedback too.

Doug: Right.

John: …in addition to other things, because if one person asks a question or answers a question or raises a point you know how that person responds or how they understand the concept.

Doug: Right.

John: But, with clickers you can get a feel for how all of your students are doing.

Doug: Right.

John: …and you can adjust what you’re doing in class to compensate for that somewhat. So there’s some good merits for clickers whether they are graded or not.

Doug: Right, but you’re not gonna get anything out of the clickers if the students aren’t in the classroom.

John: Right.

Doug: ….and so I assign points to get them into the classroom and I think that’s just gonna bring up a big chunk, because at the end of the semester you look at the grade distribution and you look to see who got the highest grades in the class… and guess what? You know all those people because they show up in class.

John: Yes.

Doug: ….and then you look at the bottom and your like, how did I miss these people? They did terribly the whole semester….

John: …but you didn’t see them as much.

Doug: I don’t even recognize them. I look at the pictures and they don’t even look familiar… and so pulling those people in. Those are a few of the little things… little in terms of effort… but we hope big in terms of impacts. But the class has clicker questions… the class already has… I like do short lectures… and they do problems to practice… and so we’re we’re hoping to really gain in the spring is by adding something called “invention activities” and so these are an idea that Dan Schwartz and his students have been writing papers about for a little while… and so probably the easiest thing to read if you’re interested in invention activities is the J chapter for “Just-in-Time Telling” in Dan Schwartz’s and co-authors book called The ABCs of How We Learn, which is an amazing book. I highly recommend it. If you read any book on teaching at all The ABCs of How We Learn is a great one in terms of telling you practical things you can do and giving you the evidence… and Dan Schwartz does great work… and so the idea is, in a nutshell, let students grapple with a problem before you actually teach them the solution… because it primes their brain… it sets up those knowledge structures that you can then hang the expert methods on.

John: So it activates prior knowledge….

Doug: Exactly.

John: …and gets them ready to form more complex….

Doug: It even create creates prior knowledge… and so he has these two papers. One is called “The Time for Telling” and the other is called “Inventing to Prepare for Future Learning” where he does these really amazing experiments. The one that I really like in “Time for Telling” is about casework. He has these three groups and one of the groups he gives a bunch of data on classic psychology experiments and just has them graph it and talk about it and try to figure out why the patterns are there and then he explains what we actually learn from these psychology experiments.. and then they play around with it and then he gives them a test… where they have to predict what the outcome would be of some new psychology experiments. That’s group A and they do great. But then group B… he skips that first stage and instead they read a chapter about what we can learn from these psychology experts and they summarize it… which doesn’t sound so bad… and then he gives the lecture and they do far worse… and so I’m really hoping that we can get a big bang from these priming activities and I think this class.. it’s a second semester class on statistical methods for economists, also known as econometrics… ‘cause god forbid economists use a word that other people already know…

[LAUGHTER] but there’s a whole bunch of methods there and so I look at that class as a really great match for this method…. where I can show them data and put them in this situation where a new method would be really valuable… and the old methods work poorly… and have them play around with it and then teach them the method.

John: That sounds really promising.

Doug: I think it’ll be super fun.

John: We just did a reading group here on Small Teaching and one of the chapters in that that people were pretty excited about was on prediction and it…

Doug: Exactly.

John: …summarized a lot of the research on that (which is another good reference in addition to what you just mentioned)

Doug: Right.

John: … to see a summary of the literature on that and it sounds like it’s quite effective in a wide variety of studies now.

Doug: I mean, I do think you have to be careful with students in explaining why you’re doing it upfront. I think it can be very frustrating to give students problems that you know they’re going to fail at. Students don’t like failing.

Rebecca: No.

John: But the research shows that when they have wrong predictions they actually have larger learning gains.

Doug: Right.

John: …as a result of the prediction activity.
DOUG. Right.

John: … and as long as you they know that, and as long as it’s low- or no-stakes in the prediction….

Doug: Right.

John: ….it shouldn’t harm them… but it doesn’t feel as good, so there’s a metacognitive issue there.

Doug: Well, I try to model making lots of mistakes in the classroom

John: I do that too, but I’ve never called it a strategy., but…

Doug: Exactly.

Rebecca: I think students respond really well to seeing their faculty be vulnerable and human…

Doug: Oh, I agree with that.

Rebecca: …and I know we all make mistakes and I think you’re right that students are… they’re afraid to fail…

Doug: Right.

Rebecca: …and they’re afraid to be wrong. So having an environment that’s setup that it feels safe to make those mistakes.

Doug: Right.

Rebecca: …and then get better… and the goal is to get better… really sets up an atmosphere that really supports learning because everyone feels safe about learning… and I think that that’s not always the way that students think about learning.

Doug: Not at all. Not at all. They think “if I get the answer right I’ve learned and if I get the answer wrong I haven’t” and it’s actually the opposite. If you get the answer right you haven’t learned it…like, you knew it.

Rebecca: Yeah.

Doug: Yeah, if you get the answer wrong… oh my god, there’s an opportunity to learn something…

John: …and it’s hard to convince some of that sometimes.

Doug: It is. It is. I tell them I want you to get things wrong, because if you just get everything right you’re wasting time…. then I’m wasting my time.

John: I tell them the same thing.

Rebecca: Yeah.
REBECA: So, can I circle back to something that we talked about earlier? I heard you use the word fun and assessment in the same sentence.

[LAUGHTER]

John: It’s not a common juxtaposition.

Rebecca: Exactly. I was hoping that you could talk a little bit more about what was fun about this development.

Doug: Oh my god, I’d love to.
Yes. So first of all, I admit that I like writing tests. I think it’s really fun to come up with new scenarios that get at concepts that I’ve taught before… and so I can take knowledge in some completely different area and then frame it. So I remember one time I had to give a makeup exam, but I didn’t really want to write a whole makeup exam and so all I did was change the wording… so a question about hospitals became a question about pet stores then a question about restaurants became a question about something completely different… but the methods were all identical and so that I think that’s kind of fun… but most people wouldn’t think that’s fun. So the part of the process that we use that I think more people would find fun in that was a key part and it was something that Carl Wieman and Wendy Adams call “think-alouds” where we draft the assessments and we bring a student in and we have them do it. So these assessments are 20 to 25 multiple-choice questions and if all you see are the sequence of multiple-choice answers, it’s hard to tell what’s going on in a student’s head… okay …and so what we do is we say sit down and we say vocalize everything you’re thinking as you take this…and we don’t say anything…. we don’t say no don’t explain what you’re doing, because that changes how you actually answer it. Just vocalize what you’re doing as if you’re taking it and the results are so eye opening. I learned so much about how students actually think about these things… the amount of just pure pattern matching, it would blow anybody’s mind. They say things like “You’re asking about a confidence interval… well, I remember that confidence intervals had something to do with standard errors, and so I’m gonna choose the answer that says something about a standard error.” It’s kind of frightening, to be honest… and then pretty often you ask a question where you say… you show them a picture, and then you ask a question that’s related to the picture… and they ignore the picture. They don’t even look at the picture. Yeah, ok, that’s not good… and so we did five of these during the fall with this the big assessment of learning that we wanted to use both in the fall and in the spring… so then the control course in the fall and the treatment course in the spring. We’re gonna compare results, and each time we did these “think-alouds” we would have like ten changes. We would add options because we’d find out that students…. it was really common for them to make this one kind of mistake that we hadn’t foreseen… and get an answer that wasn’t one of the choices. Sometimes they would pick the right answer for completely wrong reason like “oh well, it’s always the one that’s a yes.” Oh…well maybe we should make the right answer the one with the “no.”

John: So, to develop assessments or concept tests, it’s really helpful to know the common misperceptions so that you can break them down.

Doug: Absolutely.

Rebecca: What I’m hearing is what we call in design a user test.

Doug: Right.

Rebecca: So you’ve designed something, and now you’re testing it out on a test audience

Doug: Exactly.

Rebecca: …and making revisions.

Doug: …and no one does that.

Rebecca: Yeah, it’s kind of interesting, right? Like we don’t think of students as being users of a class, but essentially they are.

Doug: Right. I think when we teach, usually we have so little insight into what’s actually going on in students heads… and I think in a pure lecture it’s the extreme case. You get zero insight because you teach and they listen… and the only time you ever see what they’re learning is when you see their test results, or what they’ve written and so a big part of why clickers are useful is not only because it activates their brains and they’re practicing things… but the feedback it gives the faculty and so doing these think-alouds… it’s like an extreme version of insight into what students are actually thinking.

Rebecca: How do you recruit these students for the think-alouds?

Doug: What we did is we have the rosters from the previous year… students who had taken the class… and we invited a broad range, so we didn’t just get the A students… we got the A students… the A- students… the B students… even a couple C students, and then I think we paid them twenty dollars each… but you pay them a little bit and they show up.

Rebecca: That’s great.

Doug: Yeah… yeah, yeah, yeah.

Rebecca: Have you thought about using the think-aloud method at all as part of your class? I’m almost wondering if there’s a way to kind of integrate that more into, probably not a large class, but a smaller class.

Doug: That’s a really interesting idea. I think when you give students kind of meaty problems to do in class, and they’re working on them in small groups, you can go around and you can just listen. So there are these incredible transcripts of what students said to each other during the invention activities in the “Inventing to Prepare for Future Learning” paper. So the subjects were ninth grade statistics students, but it’s remarkable just how similar teaching is. There’s a lot more in common across grade levels than there is difference. I mean there are differences, third graders are not exactly the same. My classes look far more like a good third grade class then they do a pure lecture class.

Rebecca: it makes a lot of sense, right? I mean active learning works and they figured that out in elementary school.

Doug: Right, can we please not forget that?

[LAUGHTER]

John: Another way of getting the same sort of thing was suggested by Eric Mazur when he was here during a visit, and I’m sure he’s mentioned this in other places as well. When he develops clicker quizzes, he tries to aim for about half of the students getting them correct and the other half wrong, so that you’ve got a good base when he does the peer instruction component of it. What he does to develop the questions is pose a question and leave free response questions and he’ll just let students write their responses and then he’ll use that to pick the most common misconceptions that he’ll build into the clicker questions. Which gives you a little more scale, but not quite as much information as a think-aloud, perhaps.

Doug: …and the ideas that you’re iteratively refining your class semester over semester. I wish that that was a more common attitude toward teaching… but what I find is… I meet a lot of faculty that… they developed the class… they fix the things that are obviously broken… and then it’s done… and then they come in… they give their lecture… they walk out… and there’s very little investment after the fact.

John: We’re creatures of habit.

Doug: Right.

John: We tend to resist change. That’s one of the things behavioral economics tells us.

Doug: I was gonna say that classical economics tells us that people respond to incentives.

John: Right

Doug: If you’re above the bar… like you’ve responded to all the incentives… that are all the professional incentives that are there.

Rebecca: As a designer I just can’t help myself from redesigning and redesigning and redesigning… it’s never done.

[LAUGHTER]

John: I’m never satisfied with the way my course goes and I keep wanting to make it better.

Doug: Me too.

Rebecca: Yeah, but that’s why you two run a teaching center and have the teaching podcasts. You’re not the problem.

John: Well, my students might disagree at times… but that’s another issue.

[LAUGHTER]
So, are the other departments involved doing as much with the assessment component or is this something that you’re perhaps focusing on a bit more? or is it built into all of it?

Doug: No, I would say… I believe, no. It’s kind of a pet issue of mine… like I think assessment is super important. I think math is leaning on existing assessments… which is fine…

John: ….for departments or for majors where you have existing assessments that works well, but as you mentioned, we just have the micro and macro TUCE exam…

Doug: That’s all we have.

Rebecca: I mean, you have something.

John: We have something and we use it.

Rebecca: There’s nothing in my field.

Doug: So, in sociology there’s nothing also… and so sociology has actually been investing in assessments there. It’s tough… it’s tough. Sociologists are allergic to multiple-choice and so they will never use multiple choice… and I’d like to see a graphic design assessment that’s multiple choice.

Rebecca: I made one.

[LAUGHTER]
It’s hard, though.

Doug: So, Natasha Holmes and I have this ongoing back and forth where I say “Here’s something that’s tough to teach and evaluate with multiple choice” and she says “No, you could do it” and the line moves. So we agree that there are plenty of things that can’t… we agree on a lot that can …and we agree that fluency in playing the piano… you can’t… but where the line is in between, we go really back and forth… and I find that over time I believe more and more can.

Rebecca: I think there’s a lot of conceptual things that you can test that way.

Doug: Right.

Rebecca: …and measure certain kinds of understanding.

Doug: Right.

Rebecca: …but maybe not always… you can’t do a practical application with the multiple choice.

Doug: Right.

Rebecca: Like in graphic design, for example.

Doug: RIGHT. So, not everything, but a lot.

Rebecca: Yeah.

John: But you could devise, as we talked about in an earlier meeting today…. you could use rubrics for those areas.

Doug: Right… right.

John: We had a number of faculty a few years ago, who were new faculty in creative fields, and they were very concerned about how to evaluate creative work… so we put together a panel of people from art, from music, from screenwriting, a playwright, and from creative writing, and what was remarkable is they all said exactly the same things: that they use rubrics very heavily; that while people perceive this as creative work, there’s very specific things that they’re looking for in the writing; and as long as they develop good rubrics that capture what they’re looking, for it lets students know what they should be striving for what they view as important. Just as you said to give them points as students, matters but giving students rubrics helps them see what you think is important in their work… and it makes it easier for them to try to meet those standards.

Doug: Do you have a book that you recommend?

John: I don’t.

Doug: Well, wouldn’t it be great?

John: It would be. I’m not sure if there is one.

Rebecca: I don’t know of one, but I think it’s not a lot different. Revising and rewriting a rubric is no different than revising and writing multiple-choice questions

Doug: Right, right.

Rebecca: It takes time to refine that and get it to measure exactly what you want it to measure.

Doug: Right.

John: It just impressed me, though, that they all had exactly the same thing in fields that were so diverse.

Doug: Right.

Rebecca: Yeah, there’s still values to the discipline, right? There’s still things that we value. So, it might be innovation… so, maybe that’s an item on your rubric.

Doug: It’s pretty easy to get inter-rater reliability with a multiple-choice test. I think if the correct answer is A and you give a C, John and I are both gonna say it’s wrong; whereas, with a rubric it’s not obvious.

Rebecca: Right.

Doug: But I think… I mean… like so many things… like it’s not that rubrics are good and multiple choice exams are bad. There are plenty of really good multiple-choice assessments and bad ones and there’s a big difference between a bad rubric and a good rubric.

Rebecca: Um-hm

Doug: This is where people say “I tried active learning… it didn’t work …and so I don’t believe in active learning anymore.”

John: But by building the results and by working on the development of tests and assessments that could be used across time and across disciplines…

Doug: Right.

John: ….that makes it easier to build a case for the efficacy of active learning…

Doug: Right.

John: ….and that’s one of the reasons why I think why this has been so effective in so many of the STEM fields, particularly in physics.

Rebecca: I think another thing to think about is, if you’ve never done active learning as a teaching method before, then like you are a beginner, you’re not an expert…

Doug: Right.

Rebecca: …and so just like our students, the first time out of the gate….

Doug: Right.

Rebecca: ….we’re not going to be an expert, and it’s not going to be perfect, so we have to be vulnerable as learners as well… and so I think sometimes reminding folks that… wait a second, right now you’re a learner, and it’s ok… it’s ok not to be perfect…. and it’s ok that it failed. We can learn a lot from it….

Doug: Right.

Rebecca: …it can be really useful.

Doug: Right. Boy, that attitude… I wish it was more common.

John: New faculty, while they may have been exposed to more active learning techniques, they’re also sometimes reluctant to go against department standards of teaching because they know that there’s some chance that new things they try may fail… and it’s certainly safer and easier to do it after tenure, although by then people often get into habits that are hard to break.

Doug: So, a strategy that my department is planning to apply, and their current chair is highly supportive of, is bringing brand new faculty into the Active Learning Initiative right away… and so giving them classes that have already been transformed… so it becomes the new norm… so it’s not that you’re taking a risk or trying these new things and if it doesn’t work the rest of the faculty are going to shake their head… it’s “This is how you’re supposed to do it.”

John: Yes, having that as a prepackaged method of teaching…

Doug: Exactly.
JOHN… certainly would make it easier to disseminate that.

Doug: That’s the hope.

John: Excellent.

Doug: ….and they also won’t have the… I guess… the institutional power to fight against it as hard… as an established faculty member.

John: That’s true.

Rebecca: Yeah. How have students responded to some of the initiatives. I know in your department you haven’t done that active learning kind of stage but I know that you’ve had some other programs

Doug: Ok.

John: You’ve been doing active learning techniques in other classes for a while.

Doug: So, I can say three things. First, is in physics course evaluation. It improved and the students seem to really like it, and feel like they’re getting more out of the class than they are [in other classes]… because what they’re doing is they’re taking these highly flipped physics classes along with other classes outside of physics that are not… and so they see the contrasts… and I think both physics and biology have done a good job in explaining why they’re doing what they’re doing…. so that those students that say “ Why are you making us struggle? Why don’t you just tell us the answers, and then we’ll know how to do it?” and they’re explaining why that’s actually less effective… and to be honest in my experience, I do a fair amount of active learning in my classes already… and you have to explain why you’re doing it.

Rebecca: I agree. Students respond really well to that.

Doug: Right.

Rebecca: Yeah.

John: We’re much better in any case. They still sometimes resist because, one of the problems with a lot of the approaches, is that they don’t get as much positive feedback right away…

Doug: Right.

John: …because they do make mistakes and fail and that doesn’t feel quite as good as…

Doug: Right.

John: …doing well on the one or two or three exams that they might have happened to have had otherwise after cramming the night before.

Doug: In all three cases, I think, explaining what you’re doing, and why, before you actually do it… it’s pretty effective… and again you’re always going to have these students that fight against it. They’ve succeeded their whole… and it’s usually the top… and it’s those students that have succeeded in memorizing, and taking notes, and doing well on exams their whole life… and now you’re teaching them in this new way… and it’s harder. They’re gonna resist… not that surprising…

John: But it can work the other way, too. That the students who find that it’s more helpful to go through active learning techniques might encourage other faculty perhaps to adopt it.

Doug: That’s the hope… and I think the great majority, I think, get a pretty to have a pretty positive experience… and even more by the end of the semester. I think most of the students are going to have bought in if it’s done well. I mean. I’m not gonna say that every single class that did active learning the students loved it, because there’s some crappy active learning classes out there. I knew someone… they decided to flip their class and they admitted all of this… like I heard this from them, not their students… they said I turn all my lectures into videos and it was great and I really invested in the videos and then we would show up the class, I’d be like ok and I had no idea what to do in the class …They hadn’t invested in the class part.

Rebecca: Right.

John:: …and that’s the big part of active learning.

Rebecca: …that’s the active part.

Doug: That’s right. Right.

John: In most disciplies, you don’t need to create videos. There’s a lot already on YouTube. The real work needs to be on what you’re going to do in class to give you the most value added there.

Doug: That’s right. That’s right.

Rebecca: Our last question usually is what are you gonna do next?

Doug: So, we have a whole transformation plan. I mean at this point, which I talked a little bit about, but my spring will be heavily invested in creating and trying these invention activities. I’m crossing my fingers that they work. It’s the first course we transform and the department is part of the Active Learning Initiative. It’s a kind of a big ask… that we try something completely new and actually see some big results. It could be, at the end of the semester, we said why didn’t we just take that lousy pure lecture class… because that would be a lot easier to improve. I don’t know. Well, it’ll be exciting to find out. But the other big thing on my plate that I haven’t talked about already is taking these assessments that we’ve drafted in the fall and piloting them with partners at other institutions… and so if any of your listeners teach either introductory statistics or a second semester econometrics course, please contact me, douglas.mckee@cornell.edu, because we would love to have you pilot this thing…. and I’m hoping to give a big talk on it… on both of these assessments with our postdoc George Orlov who’s amazing… and will be on the academic market next fall, so if you’re looking to hire an economist that really loves teaching, and is great at it, and does really super interesting research, he’s your guy.

John: But not yet… not until he finishes…

Doug: Oh no… don’t hire him yet…

[LAUGHTER] …and so, what we’re hoping to do is make these published standards that can be used across the discipline as ways of evaluating teaching… and that can be big picture… we did this big intervention… or I think a lot of people use these standard assessments as ways to identify where the soft spots are in what their students are learning… and so people do that too… and so the vision is, for over the next five years, to take what we’re doing now and multiply it by eight… and so there’s there’s a lot to do.

Rebecca: … big shoes to fill.

Doug: …we bit off a lot. We promised alot. Now, we just have to execute. So far, so good.

Rebecca: I can’t wait to hear about the results and hopefully we have follow up and find out what happened.

Doug: We’ve submitted an abstract to the Conference on Teaching and Research and Economic Education which will be this June, where we report on the results of the transformed class, and so we won’t know… we will be getting the data in…

Rebecca: …moments before it’s due…

Doug: …moments before presenting the work. Hopefully, we’ll have a lot of incentive to put that together.

Rebecca: Great.

John: Excellent.

Doug: Oh, thank you both.

Rebecca: Well, thank you!

John: This was fascinating. Thank you for coming down here… or coming up here to Oswego.

Rebecca: Yeah, we’ve really enjoyed your visit.

Doug: It’s been a lot of fun you guys are doing great work here.

John: …and we really enjoy your podcast!

Doug: Thank you. More podcasts! I’m your number one fan.

John: OK, Thank you.

Rebecca: Thank you.