94. Open Reflection

Students can provide useful feedback on instructional practices and class design when they are asked, In this episode, three students from John’s spring economics capstone class join us to provide their reflections on the class’s experiment in developing an open pedagogy project. Our guests in today’s episode are Maria Aldrich, Victoria Heist and Charlie Tararzona.

Show Notes

Transcript

Rebecca: Students can provide useful feedback on instructional practices and class design when they are asked. In this episode, students join us to provide an open reflection on one class’s experiment in developing an open pedagogy project.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guests today are Maria Aldrich, Victoria Heist and Charlie Tarazona, three students who participated in the creation of an open pedagogy project in one of my economics classes this spring semester. Welcome.

Victoria: Thanks for having us.

Maria: Thank you.

Charlie: Yep, excited to be here.

Rebecca: Today’s teas are…

John: None of you are drinking tea, are you?

Maria: No.

Victoria: No tea.

Charlie: No tea today.

Rebecca: How regretful. [LAUGHTER]

John: It happens with many of our guests. I’m having ginger peach black tea.

Rebecca: And I’m drinking Lady Grey. The issue is our tea selection is no longer close to our recording studio. It’s a problem. It’s an epidemic now with our tea choices.

John: …now that we’re recording in this little closet in a building next door, where at least we don’t have toilets flushing every 30 seconds or so that we have to edit out.

Rebecca: So John, can you start first by explaining what open pedagogy is, to kind of frame our discussion?

John: Going back a step further. Last year, I saw a presentation by Robin DeRosa who presented on this at the SUNY Conference on Instruction and Technology and she made a really compelling case for some of the advantages that open pedagogy projects have. And open pedagogy just involves having students create content that is open and shared publicly with the world.

Rebecca: So what class did you do an open pedagogy project in?

John: This class is a capstone course in the economics department here. It’s taken mostly by seniors and a few juniors. And it’s a seminar course in economic theory and policy. It’s one of our smaller classes. And we had only 27 students in this capstone, this semester.

Rebecca: So why this class?

John: Because the students were ready for it. The course builds on all the courses that they’ve had up to this point and it allows them to pull together material they’ve learned in all of their prior courses, as well as the cognate classes in statistics and math and so on.

Rebecca: So what kind of project exactly did you propose to these students?

John: I originally proposed two options. One was to do something on behavioral economics, because past classes have found that to be a lot of fun, and another one I suggested was they could just pick current topics and work in small groups and create papers on that. Turned out that they really didn’t like any of those ideas and given the nature of open pedagogy, I left it up to the class to decide what their topics would be. And I think it was actually Charlie, who came up with the idea. And would you like to tell us what that was?

Charlie: I know you had mentioned in the beginning of the class the idea of open pedagogy. And I found that pretty interesting because it seemed like a good opportunity for us as seniors and juniors to really put what we had learned out there. And also, in terms of topic selection, you gave us the opportunity to really choose which topics we wanted to talk about. We ended up choosing the topic of intergenerational mobility and economic inequality. We focused more on the economic inequality aspect of it in the end. But yeah, like I said, it was just a good opportunity for everybody to really finish their college careers with something that they can show.

Rebecca: Dr. Kane is going to close his ears now and you guys are going to tell us exactly what you thought when he said, “Hey, you’re going to write a book.”

Victoria: I was hesitant at first, just because group projects are kind of daunting, especially in economics. However, a collaborative group project was exciting to do… to see all of our work put together. As economic students it isn’t something you really see, it’s usually individual work.

Maria: Oh, yeah, I would agree with Victoria. I was kind of hesitant at first, especially because it was something new for our class so I figured there are probably a lot of kinks that needed to be fixed. So I was a little worried about not having everything fully figured out at first. I thought it was something interesting. It appealed to me because I like the thought that other people could read what we had written and we could have control of what we would want to talk about.

Charlie: And the topic and the idea of a book project really intrigued me… that it just let us put out there what we had learned over these past few years and gave us something that we can show in the end of it.

Rebecca: Were any of you scared?

Victoria: Not scared. I wouldn’t use that word.

John: Were you concerned?

Victoria: A bit concerned, just because I like doing my individual work. I feel stronger in that.

Maria: Yeah, I would agree. I think group work can sometimes be difficult to have for every class because everyone has a different writing style and everyone works on their projects at different times. So I think at first, you’re a little bit worried that not everyone will be able to work well together. But I found that in my group, we were able to work very well and we’re able to meet once a week to go over what we needed to work on for the week.

Charlie: Yeah, I found something similar to that experience. Whereas my group, after the first few weeks, figured out what we wanted to do, and when we could meet, and what was the most effective way for us to put the book project together? And I think it turned out really well.

Rebecca: So you’ve all mentioned groups, can you talk a little bit about what the groups were, how they were determined, and how that worked?

Victoria: The groups were groups of three from the class, because there’s 27 people. And then we’re able to email Professor Kane and ask if we wanted to work with anyone specific, like if we had friends in the class, we could work with them. But if not, or if we didn’t want to work with someone we knew, we’re able to randomize it.

Maria: I was put into a group of two other random people that I didn’t know, but we were able to set up a group chat immediately and communicate very well through that.

Charlie: I actually emailed Professor Kane about being a group in Victoria and we also included another student in that. I think it worked out pretty well and I was happy with how it turned out.

John: Before the groups were formed, though, the class decided on what the topics would be. So we had kind of a free-form planning session where we narrowed it down to nine topics you wanted to address. And then at that point, we knew how big the groups were going to be. And it worked out nicely with three people per group.

Rebecca: How did each group get assigned a topic?

Charlie: So the way we assigned topics was, we had created a list of the nine topics, and then each individual group could choose their top three, and then we divided them that way based on everybody’s top choice. If they didn’t happen to get their top choice, they usually got their second or third, I think that only happened for maybe two groups, and they seem to be fine with what they ended up with.

John: And going back a little bit further, it was a weighted voting scheme that you didn’t just rank them… that you assigned points, if I remember was it 10 points I gave you? And so if you really wanted to chapter you could bid all 10 points on that. And if you were indifferent, you could have assigned weight to your top three preferences and so forth. And it did work out really nicely where I think most groups got their top choice, but two of them ended up with their second or third choice, but it seemed to work.

Rebecca: How did you find collaborating in the end?

Charlie: I found that it worked really well meeting every week. We also had presentations every week that we gave on specific topics that we’re talking about during that week. So that set the initial schedule for us to meet every week and talk about what we were doing and what was going on. Also with the book project at the time, in terms of organization, I found it very laid out and simple.

Rebecca: That sounds like you had a writing group that met that frequently, but it also would be more of like a study group as well?

Charlie: Yeah, I would definitely say it was a mix between a writing group… a study group. Your group members ended up being the way, if you wanted to succeed in the class, like that was the way to do it was to work cohesively with your group members.

John: And it should be noted that they had other tasks in the class as well, where they selected topics that were presented each week and each group was responsible for presenting an article or a research paper on a topic, some of which were related to the book and others were completely different. The groups were persistent across all the assignments and involved more than just writing the book.

Rebecca: How’d you get feedback to make sure whatever you’re putting out in the public was good enough?

Maria: Well, we mainly used Google Docs. At first, we tried to use hypothesis. But that wasn’t really working out well. So we ended up just going back to Google Docs. And each group would be given a couple of chapters to review each week. And they would write a couple comments in that chapter as well as some comments made by our professor and we used that and we also used each other’s feedback to make those edits.

John: How did that work? Where the comments helpful?

Victoria: No. [LAUGHTER] Just because I’m very protective over my work, which I know I should be open to criticisms. However, I got some comments sometimes I was just like questioning, like instead of “what about this article that you might want to look at” it would be “change this word,” where I know we emphasized that often in class many, many times, but still people would persistently do that.

John: I hope that wasn’t from me. Was it from me?

Victoria: Oh, no. Well, if you wanted to do that, you’re the professor. You can do that. But you also give us feedback that’s helpful. Whereas, students I feel like if they’re rushed or doing it, like 20 minutes before the class, they’re not going to look at me like, “Oh, what about this topic that might be interesting to consider.” Instead, they’re like, “Switch this word.” That just might be the students in the class too, just because we did have a lot of work in the class. And I found that because our group would meet to practice our presentation before the class, a lot of groups are doing their final work 15 minutes before the class.

John: Yes, the quality of the work did vary a little bit across the groups and across the individuals within the groups. Overall, there was some really high quality work, and all three of you did really well. But the quality of the feedback varied quite a bit.

Rebecca: So the feedback was generally done outside of class? Like not during class time?

John: Primarily, except for the presentations on the work where there was some feedback during the presentations.

Victoria: Yes, but I found that your feedback was most helpful, rather than the students.

Maria: Yeah, I would say I paid a lot more attention to Professor Kane’s feedback than some of my fellow students. Luckily, we didn’t have that experience. We had a lot of people give a sincere, really constructive feedback, but sometimes I’d be hesitant to take that feedback because I didn’t know that was the direction that I should be going in. But I definitely think our experience was a little better and our comments were more substantial, I would say.

Victoria: And I think next time maybe switching the groups that review the feedback might be helpful, because if you have one group that gives worse feedback, and you keep getting that, it’s not as helpful.

John: The way it was structured was each group reviewed and provided comments on three other groups, and we did that on three stages. And the class decided to maintain persistent groups there. I did give them the option, but I think it does make much more sense to vary it so you’re getting a wider range of feedback.

Charlie: I think the idea to keep persistent groups stem from the fact that we wanted to have somebody read the paper and then continue to read the paper throughout the weeks when we were supposed to be improving it or making it better. So then they could also see the changes we were making. And I agree with my classmates where I think we can say that it didn’t work out too well. There’s some groups just didn’t happen to give feedback that was too good.

Victoria: I also think part of it was the length of the papers because each of us had to review three full papers for the weeks that we did that, and three 20-page papers is a lot of reading to do on student written economics. And I think maybe in the beginning it was helpful to read all three, but maybe as time went on to scale that back a little bit, so we don’t get burnt out.

John: More detailed feedback on a smaller number of papers.

Victoria: Yeah because at first, I find myself doing it too. Like the first paper, I’ll take the time to read every single word and provide helpful feedback. But I can see myself not doing as much on the third.

John: I gave feedback in three different ways. The first time I gave video feedback, and while I’ve heard that that can be really efficient, I was taking about two hours or so per paper. And that was really slow and people really didn’t like the feedback that much because some of the feedback was fairly long in terms of the suggestions. So, I probably gave a little too much feedback. The second was with comments embedded in Adobe. And the third time I just basically went along with everyone else and provided the feedback directly in Google Docs. And the nice thing about that is I was able to see some other suggestions and sometimes I’d say, “Well, maybe that’s not such a good idea.” Because in many cases, the original draft actually made more sense than the feedback.

Rebecca: What way did you all like feedback better?

Charlie: I think the best feedback I received was actually in class feedback when I would go to Professor Kane and ask him, “Hey, you know, this is what’s going on with my paper. Is there something else I could look at? Is there another source I can find?” I found that to be the most effective in helping me write the paper. I was also a fan of the comments in Google Docs, they were pretty helpful.

Maria: Yeah, I think the most helpful feedback were the comments from Google Docs because, for that last draft, I was able to go through and resolve any comments that I had made the changes to and that just helped motivate me to make my draft a really good copy. And then I would say that I was really against the video feedback because I personally like to review feedback multiple times. I like to go through it and make changes to it. And I found myself just typing up his comments at the end of the doc so that other teammates could see it as well. So I was definitely against the video feedback and prefer the Google Doc comments.

Victoria: Yeah, I prefer Google Docs as well, just because I could see where exactly you wanted the changes done. It gave more specific feedback and then it also gave the students validity I guess, like this should be changed. Like I made a comment in one paper and said, “I think you mean a different word.” And they just resolved it and moved on. But then Professor Kane came through and said, “Yes, I agree.” And I think that you agreed, and they now are aware that yes, those changes need to be made.

Rebecca: I think sometimes when a faculty member responds to student comments in a way that it also helps students know how to make better comments. So it would be interesting to see how another round of that would have gone after Dr. Kane had responded to some of those right? To see if the comments were better the next time around.

John: Yeah, I think I should have done that from the beginning. And I’m sorry, I didn’t. But in the future, I’ll probably use Hypothesis. Now that we have Hypothesis in Blackboard it will be much easier. Among the problems we had is that people had some trouble making comments on Google Docs because they also had edit access to those and they couldn’t mark up specific text. And with PDFs, that was a bit of a problem given the way the browsers were set up that they had to change a program in order to make comments on PDF documents. So now that we have that in our learning management system, it’s going to be much easier to do that and the comments will be a little more persistent, because one of the issues was people were, as you mentioned, resolving comments sometimes before anyone else had a chance to see them. And the strategy was to have the draft documents with the comments copied over to another folder, and they were only supposed to make changes in their working document, not in the documents used for comments. But there were three or four people who through three drafts, just didn’t quite get that notion and I’d see the email saying that comments were resolved, and I would go back in and unresolve them. But in any case, there were some problems with those. That’s an issue that I think has to be worked out a little bit more efficiently.

Rebecca: Beside some of the technical issues that we mentioned, what were some of the biggest challenges of working on a project like this?

Charlie: I think one of the bigger challenges was keeping the cohesive idea behind the whole book where the topic we had chose was income inequality and we also had talked about intergenerational mobility. But as the book progressed, we kind of saw that portion of the book fall off a little bit where chapters were really focusing on the income and economic inequality topic.

Rebecca: So is that something you discussed in class to keep everybody on track?

Charlie: I think we mentioned it at one point towards the end, we’re just like, “Okay, are we going to keep this? Are we going to not keep this?” And I think we agreed, we could talk about it but we won’t make it a major portion of the book.

John: There was also some scaffolding on the project… that it didn’t just start with people starting to write, groups were first asked to put together a bibliography, and then an annotated bibliography, and then an outline of the chapter, and then the actual writing started after they had feedback on each of those steps.

Maria: I would agree with Charlie, I was definitely worried about the cohesiveness of the entire book. But for my group, specifically, we did a very broad topic, the global trends of economic inequality, and for myself, it was really hard to find relevant subjects to talk about because it was just such a broad topic. It was really hard for each of us to find something that we could spend a large amount of time writing about. So I’m not sure how the other groups felt. But for us, it was definitely hard narrowing down what we specifically wanted to talk about, and then to find resources that were recent enough to include.

Victoria: Yeah, I agree with you on that. I think one change I would make after we figure out the specific topics, you can go deeper in that because it’s hard as a group to form a thesis statement or very cohesive argument because we ended up doing more of a timeline than like an argumentative paper because you have to split it up.

John: Your topic specifically was on what?

Victoria: Tax-structure and income inequality. So basically, we looked at early 20th century, later 20th century, and the 21st century, and how the changing tax structures led to increasing income inequality over time. So that’s kind of how we split it up. But I think if I was to do it again, I would take a different approach to it, because I did the first section and finding information on World War One income inequality is much harder than it seems. So I struggled a lot with that too.

Charlie: Yeah, in terms of how we wrote our chapter of the book, I’m usually a fan of writing papers that follow a timeline as an explanation but that’s just a personal preference. It doesn’t work for everybody. So I can definitely see how making the cohesive argument along with following that timeline can be pretty difficult.

John: In your chapter, I think the timeline made a bit of sense. We were talking about the evolution of it and the transitions in your chapter were pretty smooth. I don’t think that worked as well in all the chapters, quite often it looked like they were three essays…

Victoria: Yes.

John: …chopped and pasted together.

Victoria: There was this one paper with a bunch of sub topics, but it wasn’t cohesive. And I was reading it and it just did not make any sense to me how it was organized. So that was one of the suggestions I made… maybe taking a step further in class and presenting maybe our papers a little earlier.

John: In more stages…

VICTORIA. I was just trying to read it and I just could not make sense of the organization of it, where maybe if we caught that earlier we maybe could have made better paper.

John: I was giving them feedback in several groups… that sort of feedback… that they need to smooth out the transitions and have a more logical structure. But some groups responded really well and did a nice job with that, other groups were a little more reluctant to do that.

Rebecca: Perhaps some groups will respond really well to some peer pressure. [LAUGHTER]

John: And having the presentations in class would have helped do that. When people in the class were saying, “This is just too disorganized.” And most of them got better by the end, but it was a stretch getting there.

Rebecca: So you’ve talked a little bit about some of the challenges, but what was really rewarding about working on this project?

Victoria: I found it really helpful to work with the group. I had Charlie and then another student, Junweii, in my group and we all read each other’s parts. I know I went through the document and made comments for my own group too. And we were all able to bring it together, make comments for each other, ask each other questions about like what sources to use especially too. And it was easier in that regard than an individual paper. Because if you make a mistake and you don’t realize it, no one’s there to help you, it’s just you. But here we have people to help each other.

Charlie: Yeah, I always find it beneficial to complete a task with other students also trying to complete the same task as you. It just makes the learning more interesting. And you’re more willing to go and spend the extra hour looking at the document to just make sure you understand what you’re writing, but also that it fits with whomever else you’re working with. I found it really beneficial or satisfying just the fact that we, like I said, could create something that any ordinary person could probably read and understand what was happening.

Maria: Yeah, I think the most rewarding part for me was just seeing that finished product and getting you know, positive feedback from Professor Kane and from my other group members. I think working in that group setting helps to motivate me to do the best of my ability. And I think it was just rewarding at the end to see everything come together well.

Victoria: I think it gave us all a deeper understanding of the material too because, instead of writing it yourself… because you can write something and not understand it. I’ve done it many times. [LAUGHTER] But when you’re sitting in a group, getting a presentation ready, you each need to understand the material. So you’re explaining what you learn to each other. And that’s something you don’t get by yourself. I found that really rewarding.

John: What about the public nature of the project? The fact that this will be out there, it will have your names on it, and it could be out there indefinitely.

Charlie: I found that portion of the project pretty intriguing and exciting. Just like I said, you can go out there, and obviously we’re college students, we’re looking for employment after this. So just showing an employer, “Hey, I’ve written something that’s been published. It’s out there, you can go read it for yourself and see what you think.” It gives something for the students to show.

Victoria: Yeah, it made it exciting because we knew what was at the end of the project. Rather than just a finished paper, we actually had something to like prove ourselves, like we did this.

Maria: I think for me, it was cool to know because at the end of the semester, I’m able to go to my family and say, “Oh, here you go. This is something that I worked on all semester long. Here’s something that you can read and you can better understand what I’ve studied for the past four years.” So I think it was helpful that I was able to show my family I’ve worked hard on this. This is something that is to show for that.

Charlie: I would definitely concur with that. Economics as a topic isn’t really discussed when you’re talking just with family members, so many of them don’t understand what you’re talking about. And you’ll try, but it’s hard sometimes. So to put something together that they’d be able to read and understand, I found that pretty satisfying.

John: And how did the class select the audience for this? What level was it written for?

Victoria: Students with a background in economics I think we decided on. But we came together as a class and decided on that. But you need economic background to understand some of the things we wrote.

John: But at an introductory level, so it wasn’t written at an advanced level. It was written for people who’ve had an economics course somewhere along the way.

Victoria: Or just no background. You don’t have to go to college to read the book.

Rebecca: How would you change this project in the future? We touched on a couple of things here and there, but do you have any other key things that, if the same exact project were presented to another group of students, how would you change the structure? Or the way it’s organized? Or the way that it’s presented the first day?

Victoria: Thinking about the class as a whole rather than just the book project, we did weekly presentations which was a lot of work in itself. So I would probably minimize those and focus on the book. Because we were sitting there reading 20+ page economic journals every week and making a presentation on it and doing the book project. So I think having more time dedicated to the book project and presenting on that material, rather than just economic journals that people have written, like it gives background, which is helpful, but maybe a little less, or maybe shorter ones, or ones that are just easier to understand. Because I know a lot of times you would say, like, “I know you guys don’t understand this, it’s challenging. But we still need to know it.” Like you would explain it in class, which would be helpful, but reading something you don’t understand is really difficult for students… in economics specifically. That’s challenging.

Maria: Yeah, I would agree. I think, at the beginning of the semester, it was a lot of work to have to juggle both the presentations and the book at the same time. So I kind of like the idea, I’m not sure if it was you Victoria, who mentioned it in class, of doing the presentation one week and then the next week working on the book and having class time devoted to the book in the week after. I think that would have been very helpful too because we did meet as groups, but if we were able to meet in a class setting than I think other classmates will be able to make comments on your chapter and offer advice. I think it would just help overall with the workload that we have.

Charlie: I also agree with that. I think the improvement can be made where we’d work on maybe a random topic every other week, and do a presentation on that, and then also incorporate the book project into that. I think it would help with the cohesiveness of the book along with just feedback and all the other problems that we had discussed.

John: One of the things I had suggested at the very beginning, you may recall, is I suggested one option is to spend the whole class focused on this. Another option is just to do it the way it was done in the past, or something else. And the class actually voted for the something else. Now having had the experience, the something else didn’t work quite as well, and that more class time should have been devoted, I think, to this and I saw that too.

Victoria: I think we’re just looking for something exciting. Like yeah, it’s a book project like we know what we’re going to do with that. But the presentations just added something else, but if I went back to a book project because then we could have taken the steps at a slower pace too, like the annotated bibliography, like the topics, we could have taken way more time with that than we did. Because once we did that very quickly, and then went into presentations, and then we just had due dates instead of meetings in class.

Maria: Yeah, I think for us, what appealed to us with this combination of the book and the presentations was that the presentations offered structure for us when we knew what we were getting with those presentations. We knew each week that we’d come in with the presentation. And I think with the book, we were excited because it was something new and different and I think we were a little too hesitant to go fully and choose the book, because we weren’t sure what we would be doing in class. We weren’t sure how we would be tested on that. So I think the combination of fields lost because we were able to have that structure, but we were also able to try something new.

Charlie: I know for some of the students in the class they had mentioned to me… they were hesitant to get rid of the presentations weekly because they were a fan of learning something new every week and learning a different topic, not just focusing on the book project. They really wanted to increase their knowledge base by just learning about multiple fields of economics. So I think that’s why we ended up going with what we went with in the end. But I think we all could all agree that if we had done that every other week, it would have been more efficient.

John: I agree. And I think some combination might be good for the reason you mentioned, but more class time devoted to it would be helpful.

Victoria: Maybe at first too, do a presentation. Like the first presentation, I don’t know what week that was, but maybe keep that one because when our group really met each other, we worked together, and then we planned a time every week where we would meet.

John: And if this is done again, and that will be if the class wants to do this in the future, perhaps that first topic for the readings could be related to whatever they choose to do so they’re actually doing some scaffolding with the presentations then.

Rebecca: I had something similar in my classes before where a team formed early on. We did something small, low stakes, to figure out how to work with each other and what doesn’t go well. So that when we did something a little more high stakes, you already knew what the wrinkles were going to be so that you could plan for that moving on. So it sounds like your presentations served that purpose, whether or not you intended that to happen or not.

John: But it became a lot of work when it was done every week, in addition to writing a book.

Victoria: That was difficult.

Maria: Yeah, I think it just helped to make us all more comfortable with each other and more comfortable speaking in front of the class.

Rebecca: So the big question is, of course, should other faculty do this?

Victoria: Yes, I’m working on my honors thesis right now, which is kind of what you would do in a traditional seminar. And it’s very difficult. So just having people there… write it with you… know what you’re talking about… You can ask them questions. In our group chat, we often ask, “What would you recommend for this part of the paper? Or what articles do you think are appropriate for this?” If you’re doing it by yourself, it’s very difficult. And the overarching topic… I feel like in a lot of seminars, they have that. It’s a topic for the seminar, but it doesn’t really filter through as well as the book project does, because we are all cohesive, all of us together working as a class of 27 people, which you never see. So, I found it really helpful and I liked it a lot. And it wasn’t like a crazy amount of work. You did the work, and you study, you did the presentations, and you wrote a paper, but it didn’t take you hours every day to work on. I feel like I learned more in this class than I have in other classes that I write individual papers for.

Maria: Well, I think I would partially agree and partially disagree with that. I think as a class, we all appreciated that Professor Kane was willing to change like the class structure and was willing to try something new. And I think that was definitely intriguing for us and provided something different as our last economic course. But I think if I had done my own topic paper, I think I probably would have learned a little bit more, I think just I would preferred that. But I think it was still important to get this experience and try something new.

Charlie: I think I would definitely suggest it to some other faculty members to maybe try this out. Like Victoria was saying, working with a group is pretty beneficial. And I feel like, from a personal standpoint, I learn more when I’m working with other people who I can ask questions to, get feedback from. Really, it helps your understanding of the class. In terms of incentive, I find that I wanted to work on the book project because you had that end goal of, “This is something that I can put out there and show to somebody.”

Victoria: Yeah, but at the same time, group work can sometimes be the worst thing that ever happens to you. Like we got really, really lucky because I know Charlie, we’re friends so we were like, “Okay, let’s work together. We’ll just get one random person.” Junwei was like such a blessing. We just work together so beautifully, but I feel like if we had someone that didn’t want to do the work… wasn’t willing to put in the work… didn’t show up to meetings… that would ruin the project for us. So I don’t know how you could fix that. But just if there’s a good group, it works. If there isn’t, I feel like it wouldn’t work as well.

Rebecca: So good to write one book during your time here, but maybe not many books. [LAUGHTER]

John: But there could be other things. For example, they could have been podcasts that were created. They could be collections of essays.They could be video projects that are put together by groups. So there’s a lot of different things that could be done.

VICTORIAL: Yeah, I would throw that out there. If you did this again with another book, like, yeah, you can write a book, but you can also do that… a different kind of form of the same kind of structure. That would be interesting.

Maria: I would be interested in doing some type of podcast because I know some of my friends in their classes have been required to do podcasts. And I feel like you have to prepare really well for that. So I think maybe that would have forced your teammates, if they weren’t doing the work, to do the work so that they wouldn’t get to the studio and not have anything to say. So I think that would have been another really cool option.

Charlie: I think it would be a good option for capstone classes, just because I know for a lot of majors, you hear what the capstone is about for the three years before you even get there. And I know personally for me, I’m also trying to get a political science degree, my capstone is next semester, like I’m already dreading the 25-page paper I’m gonna have to write. So to switch it up and have the students maybe not know exactly what they’re in for, I think it gives a little bit of an intrigue and like, “Okay, this isn’t just the I’m going to go and write a paper all year. It’s something else that I’m going to do.”

Victoria: Yeah, it’s more fun. I’m more willing to write a paper that my group members are in. Like we can all see each other too in the Google doc and talk to each other in the chat… be like, “What do you think about this part?” Or like Charlie can watch me while I’m writing my part of the paper and say, “This is good. Maybe change this. Or bring this sentence up.” You don’t do that in individual papers and even if you write an individual paper and have peer feedback, it’s not the same as having it right there, real time, or just people caring more because it’s theirs too.

John: We did have some issues with that early on though, in the first draft or two, because there were some people who really didn’t want to try using Google docs for writing. And were any of you involved in that?

Charlie: So, I’m not opposed to Google Docs. [LAUGHTER] I had just always used Word documents before. So it took a little bit of getting used to but once you commit to it, it’s a really nice thing to have in your repertoire. Google Docs, I feel like, is used by countless numbers of people, companies, places, businesses, the college. So honestly, as a student, you should just take the incentive to try to get to learn it. And once you learn it, it’s really beneficial to you.

John: One of the problems was that some people were writing in Word and then uploading it to the drive and that made it really hard for other people to edit. And eventually everyone switched over, but it did take a few iterations with some people.

Maria: So yeah, I think there were a couple of challenges with having different drafts because people made comments on separate drafts. So I think just sending out a reminder email would be helpful and letting people know because I know I think I made my changes on the wrong draft the first time and we had to send an email right away to have him fix that. So I think just having it set up all before the due dates like before you mention it in class would be really helpful too.

John: Yeah, there were some rough spots. This was new for me too.

Rebecca: So we always wrap up our podcast by asking what’s next for each of you?

Charlie: This December, I’m looking to graduate from Oswego, which is exciting for me. And after that, I’m not really sure what’s going to go on. We’ll see.

Maria: Well, I’m graduating this Saturday, and I’m going to be moving down to Florida for a little bit and doing an internship there.

Victoria: I’m also graduating Saturday and I’ll be working at HSBC this July in their graduate development program.

Rebecca: Sounds like exciting futures for each of you.

John: What are you doing in Florida? An internship where?

Maria: I’m doing the college program, the Disney College Program.

John: Oh, wonderful. Maybe I’ll see you there at the OLC conference. Well, thank you. It’s been a pleasure working with you all semester. And thank you for joining us.

Charlie: Thank you for having us.

Victoria: Thank you.

Maria: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

88. School Partnerships

What does it mean to have a collaborative learning community inclusive of faculty, professionals in the field, and current students? In this episode Dr. Christine Walsh and Kara Shore join us to explore one such partnership that is rich in mentorship, professional development, and mutual respect that could serve as a model for other schools and programs.

Christine is a visiting assistant professor and professional development liaison in the curriculum and instruction department at SUNY Oswego. Kara is a Principal at Leighton elementary school here in Oswego.

Show Notes

Transcript

John: What does it mean to have a collaborative learning community inclusive of faculty, professionals in the field, and current students? In this episode we explore one such partnership that is rich in mentorship, professional development, and mutual respect that could serve as a model for other schools and programs.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guests today are Dr. Christine Walsh and Kara Shore. Christine is a visiting assistant professor and professional development liaison in the curriculum and instruction department at SUNY Oswego. Kara is a Principal at Leighton elementary school here in Oswego. Welcome.

Kara: Thank you.

Christine: Thank you. It’s good to be here.

Rebecca: Today’s teas are…

Kara: Sweet tea…

Christine: …and Jasmine tea.

Rebecca: Those sound good.

John: Ginger Peach Black tea.

Rebecca: And I have Christmas tea in July.

John: So we’ve invited you here to discuss the partnership between the Curriculum and Instruction department at SUNY Oswego and Leighton Elementary School. Tell us a little bit about that program and how it got started.

Christine: Sure, I’ll start. SUNY Oswego’s School of Education has a long standing relationship with Oswego City School District. I came to the college in 1990 and we had already been working together in preparation of high quality teachers, both elementary teachers and secondary teachers…. teachers in the school district except our in-service students for practicum for student teaching placements. And so in the 90s, we began a PDS—Professional Development School—partnership across Oswego County, and Oswego City School District has really been at the forefront of that since the 90s. I’ve been the PDS liaison here for about 10 years and so it just makes sense to continue enriching that partnership in many different ways. And this is our third year now in the Leighton-Fitzhugh learning community and it really is reaching its richest quality at this point, and in part because of Kara coming in as principal there.

Kara: Thank you, Chris, for saying that—for me when I came in three years ago, really got off the ground running as far as starting this partnership. And we did some planning in the first summer that I came. And really what we talked about was, and these are kind of Chris’s words I’ll use—how can we make it clinically rich—was the term that she used and, kind of thinking about that as we go forward, how can we make it so that our student teachers, or rather the student teachers that come to us from SUNY Oswego, how can we make it so that they are really getting all the experiences that they would have once they’re hired as a teacher? And so we know that from being teachers ourselves that six to eight weeks of student teaching and maybe some practicum hours is certainly helpful in that goal, but it’s really not seeing the whole picture of really what happens in a school day to day and so that’s really kind of where we started from. And then it was all the details that we had to get situated so that we can make sure that it was clinically enriched for those students that were coming into the program.

Christine: The superintendent in the Oswego district now, Dean Goewey, actually approached people in our President’s office here at the college and he said, “What can we do to really cement this relationship to go beyond what other districts are doing with SUNY Oswego School of Ed, to honor a clinically rich experience for undergrads for pre-service teachers, and bring professional development in for in-service teachers?” And so he kind of has a vision of this very strong collaborative learning community. And he said, “I’m going to give a classroom in Leighton elementary school to SUNY Oswego. This is going to be a dedicated room. The technology belongs to SUNY, the equipment, the furniture belongs to SUNY, faculty from SUNY will teach their courses there.” And so our students now take courses right at Leighton—their three education courses in the fall are right at Leighton—so we bring their faculty in to meet Kara’s faculty and staff. They’re an integral part of the professional development we do with teachers, our pre-service candidates are a part of our professional development now which in other districts, pre-service teachers really don’t become a part of professional development—they’re just taking their coursework—but we like to see the two populations together, send the same messages to both groups, and it is a true learning community. We sit down every month, and all the planning is collaborative. And in those ways, it’s really become so much richer than we expected.

Kara: And really, by the students being part of that professional development, they are able to have that professional development and their classes right on our campus at Leighton and then they’re able to take that learning and go right into the classroom. So it’s not removed by a few days or a few weeks, it can happen right away. So, as we know with all learning, you can put it into practice right away, you have a better chance of solidifying what it is that you’ll be doing when you’re working with the children.

Rebecca: What do our students say about that experience of taking classes at Leighton and then being able to have that direct experience in the classroom?

Christine: I do want to start off by saying that we’ve morphed from the Leighton learning community into the Leighton-Fitzhugh learning community, because Leighton is a relatively small building now that the district office is housed there and we had so many pre-service candidates interested in being in the program, we now rely on the Fitzhugh elementary school right down the road, and the principal and teachers there are very much a part of this learning community too. And so our candidates take their classes and go right into the classrooms at Leighton or they jump in their car and they go right into classrooms at Fitzhugh and it’s seamless for them. I think they appreciate that they’re not just on campus. They know that they need to learn as much from people in schools as they’re learning from people at the college and without one of those partners, they’re not getting a really true learning experience and a realistic learning experience. We need the K-12 setting for teacher preparation, and we feel they need us in many ways as well. And so it’s not an either-or situation, I think we respect the whole package and our students now, we can see the light bulbs going off for the pre-service teachers. And they go right from class where they hear about this particular theory or method of instruction, and then they go right into their host teachers classroom and they work with children for so many more hours than what our state ed requires for teacher prep and they see it happening and they say “No, I really don’t like how that’s working,” and they question it and they really are more critical thinkers because they’re in the schools more. So they’ve got that theory-practice connection down pat.

Kara: And I would say that just my own experience as a student teacher way back when, I would have never thought to go into the principal’s office. I don’t think I remember who the principals were in the places that I was put into as a practicum student and/or student teacher. And really, I have connections with those students. So not only are they working with us day to day, they really become part of our staff in everything that they do. They’re eating lunch in the same places the teachers are eating their lunch, often. Sometimes they’re in their own classroom, so the college classroom rather so that they can have their privacy but a lot of times they’re right with our teachers even down to eating their lunch. I have parent meetings and when I have parent meetings with students, they are part of those meetings. We have CSE meetings which are special education meetings, we have open house, all those things that invite our parents in to speak with us about their children, and now these pre-service teachers, these student teachers from SUNY Oswego, they are all a part of that process. So I really get to know them as well as they get to know me so I think that’s a big distinction between what we would normally see if students are just doing those six weeks.

Rebecca: I can imagine that most students don’t think of going to the principal’s office because that would be a bad thing. [LAUGHTER]

Kara: That’s right. That’s right and we’ve got to change that, right? That paradigm shift on that. So it’s very true, it’s very true.

John: It seems like a much richer experience than they typically would receive in in-service teaching where they’re just there for a few days or portion of days each week with much more immersion in a much more realistic environment.

Christine: Absolutely. Right from the beginning, we know that the college culture and climate is so different from what we live in the schools. Our schedules are different, our calendars are different, the whole energy is different in these two settings. And so it’s so interesting to work with one foot in both places, and our candidates too, they need to be flexible because things don’t always go as planned when they’re out in the schools or when they’re at the college and they have to juggle more things on a regular basis than a typical practicum student or student teacher, but we think that’s a good thing because they have the support there. They have the support from more college people in that same location, they have support from the building principal, the host teachers in that building. It is a real learning community because there’s no hierarchy and that’s a model that I think is so important for new teachers to grasp… that it doesn’t have to be that we have to have a boss or a boss of a boss and that teachers are leaders and they need to be able to connect and communicate with administrators, teachers, it doesn’t matter what your title is. And I’m finding in our learning community, we really have that communication without the fear of hierarchical constraints, which happens in a lot of places.

Kara: Yeah, and I’m really glad you mentioned that Chris—to kind of backtrack a little bit what you said a few minutes ago—it’s that professionalism. It’s understanding what it is you need to do when you walk into a school building and how you need to carry yourself. And sometimes that’s not something we might learn in a college class. But it just becomes natural because they see everyone around them and they experience what everyone else is doing. And so because of that, it just sort of happens on its own, which is, I think that and of itself, if I’m going to interview some candidates in the summer, and I’m interviewing candidates that really had those experiences and they can talk about those experiences, that interview is going to look a lot different than just someone that’s kind of talking to me about maybe theory that they have learned in a classroom. Not that that’s a bad thing—that’s a really good thing and an important thing—but if they can actually talk about how they put that into practice, that learning that happened in the classroom, that’s going to be a real strong candidate that I know is ready to go and is ready to work with whatever students come in front of them.

Rebecca: I can imagine that in a lot of disciplines, not just education, that students have a mental model of whatever the discipline or whatever the job is going to be that’s very different from what it actually is and in part because their experience of it may be from a consumer point of view or as a student rather than as a faculty member. It’s the different side of the coin. Or maybe they have pictures of what that might be from media, which doesn’t include all of the nuance that we actually experience in our jobs. So I can really imagine how much being immersed in that way can really help them understand the interconnectedness and how all these pieces work together rather than thinking, “Here’s my little hole that I’m going to exist in.” rather than realizing that everything’s connected and that you do have to adjust based on other people, bigger picture things, strategies that are being used within the entire school rather than just in a particular classroom, et cetera.

Kara: Yeah, and I think you find out very quickly if this is what you want to do. There’s lots of articles out there, lots of data, that shows that there’s a lot of teacher burnout, and so in trying to be proactive around that, I think this is one of the ways that we do that because I think students come out and they really know, “Is this for me, is this what I have passion for? Is this what I want to be doing for the next 20 years?” So I think it really gives them that guidance as well.

Christine: It’s not an easy job, not at all. Sometimes when you’re sitting on campus in a college class and you’re studying, you’re reading out of a book, you’re reading articles, you’re reading current literature, you’re talking theories, you’re talking methods, without the practical context to connect it to, and not just a short time that you’re in this context, but you’re really—like you were saying—you’re immersed in this context over and over and over, that’s when connections are going to be made. And so those practices inform both what we do at the college, and then we reflect on what’s happening, and that informs hopefully what the public schools are doing and how they can change.

John: One of the things you mentioned was the professional development aspect of this for teachers in the school. Could you tell us more about that program and how that works?

Christine: This fall, for example, we start out with a cohort of practical students. It is the semester before they student teach. We bring them out. We start in August, the schools don’t start until September, so we have a little bit of time to meet them, work with them. We’ve already recruited host teachers that we’d like to match them with, and we have an orientation at the beginning of that semester because hearing expectations right from the beginning in the school, that they are expected to do this work in has been found to be super valuable. So host teachers hear what the expectations are for their work with our candidates. Candidates hear expectations, not only from our principal, but the PDS liaisons and their professors that semester so everyone’s on the same page for this whole semester. This is what we expect our experience to be like. This is what our requirements are. This is what professionalism looks like in a public school versus walking around a college campus in terms of behavior, dress, social media. I love this work because we take the elephant right out of the room right from day one. There are no questions about what is expected in a public school classroom with children. And in this day and age, you have to be extra, extra cautious, careful, explicit. And it’s different from hanging around a college campus for four years.

Kara: Right, and we’ve been fortunate the last couple of years—maybe even three—but I think it’s been the last couple of years, we’ve been able to invite those pre-service teachers when we have opening day for staff. They’ve been a part of that. So we’ve done some team building exercises and just really get to know each other and that’s what we kind of do when we come back as a staff just to say hi to everyone, and “Welcome back, and how was your summer? And how did things go? And what’s something you’d like to talk about that you’d like to celebrate? What are some goals for the beginning of the school year? What are you thinking?” And they’re all a part of that. So not only are they getting to know our staff,as far as pedagogy goes, but they’re also getting to know our staff as, “What are your interests? What are our interests? What do we have in common?” And I think that’s critically important. As we work with students—no matter what grade level you work with students—making connections with students, we know how important that is. We know that that’s always been important, but we know that in 2019, it’s extra important that we are making relationships with kids. And so the teachers themselves are learning how to do that with these pre-service teachers and they’re learning how to do it back with their host teachers so that when students come into the room when school starts, they’re ready to do that. They’re ready to make those relationships from day one because they’ve already practiced that in the summer.

Rebecca: What a great way to have everyone feel included. I think that sometimes the internships, pre-service teachers, kind of drop-in drop-out like they don’t ever feel fully integrated or included and it sounds really great that when your staff come back, they’re all a part of the same thing.

Kara: Yes. And a perfect example of that is that when our student teachers are out sometimes—because we all are out sometimes, we all get sick sometimes—the students are asking where they are. They asked me were those pre-service teachers are. That would have never happened in the past so I think that’s a great concrete example of how much the kids really start to depend on them being in the classroom.

Rebecca: Can you elaborate a little bit more on what your students get from our college students being present so frequently?

Kara: Sure, absolutely. So we sort of know as teachers and buildings that the more that we can differentiate what students are learning, meaning the more that we can give them experiences and they can actually work with and be concrete… let me give you an example. Let’s say we’re getting ready for our science fair. And so for our science fair, typically, we would have one classroom teacher, we might have a teaching assistant in a room, and we might have anywhere between 20 and 25 students. So you can imagine that the teacher kind of goes through, “This is what needs to be on your poster board.” But then the students have to work independently. They usually will have a rubric and they can go through that rubric and they can look at all the things that should be on the poster board. And then when they’re all done with the finished product, the teacher might rotate around the room, they’re finished with the product. The teacher sort of goes over with them what that looks like. That’s fine, except for you are an end product and you hope it all went well. Okay. But with other student teachers in the room from SUNY Oswego, they are working with kids, two and three kids at a time, and they’re really helping them through that process. So by the time they have a finished product—for example, a science fair project—those students are really able to talk about what it is that they went through when they were learning it. And the student teachers—pre-service teachers—are able to really talk about where students started, and where that growth came from and as they went along, what that looked like. And that’s very different than just saying, “I’m the teacher standing in front of the room, this is what you’re going to learn, and then I’m going to grade you on this product of what I think you should have learned,” versus actually doing it and being a part of the process. So certainly they are doing that every single day and that’s across all disciplines. That’s in social studies, that’s in math, that’s in science, that’s in ELA. Also, we’re able to really take our reading groups, we’re really able to look at data and say, “These are the two or three students that really need this extra support. Now we have that person to give them that extra support.” So great to look at data—very important—but if you don’t have the staffing to then support that, when those students need that extra help, that what happens is kids get into groups, and so you might have a group of six or seven students and they’re still this high and low. That all goes away because we have those extra students that are able to do that and able to teach that reading just like alongside with the supervision of the teacher, of course, but they’re able to really work independently with those students and give them what they really need.

Rebecca: So, much more personalized learning is happening.

Kara: Absolutely.

Christine: We hear stories all the time from the host teachers at Leighton and Fitzhugh, about how much more they can accomplish in a lesson or in a given day. Some of our students even before student teaching, our college students are there three full days a week and taking courses. And so they get to see the children from when they get off the bus until when they get back on the bus at the end of the day, up to three full days a week. And so we watch them go from full-time college student to semi-professional, and then through student teaching into a full professional life—and it’s a really beautiful transformation within a year, their last year of college. But without this setting and without the collaboration, those stories wouldn’t be coming out and the richness really wouldn’t be there. But the professional development is a big part of that. We have a list of PD offerings every semester for host teachers and candidates. It begins with the orientation that we talked about, the opening day for teachers that Kara talked about that our candidates are invited to every year, and then we do something called instructional rounds where our candidates and classroom teachers are invited to do a lesson study. Two of Kara’s teachers had volunteered to do demonstration lessons for their colleagues and our candidates. And so we structure a data collection tool where we’re looking for specific pieces of instruction and elements of classroom learning and teaching and we literally go in and observe the teacher and then we debrief with the teacher afterwards, and it’s a really great form of professional development. Our candidates learn a lot, the in-service teachers, the practicing teachers learn a lot about their own teaching, “What am I doing? What am I not doing? How could I do that better?” And then they can start using their colleagues as resources. Many say, “Gee, I didn’t know you knew how to do that. How did you learn how to do that? Can you teach me how to do that?” So the learning community really is just bolstered by all the PD that we offer to both schools.

Kara: YEAH, And I’m really glad you said that, Chris, because that’s something that I have found to be just really, really an important piece of all this is that often, once we become practitioners out there in the field, we kind of go with what we learn and go with what we think we do well and that’s how that works. And so having that growth mindset, that growth model, is something that we know we should be as teachers. We should be lifelong learners, but how do we actually do that? And so by having that PD, instead of being told, “This is going to be the flavor of the week that we’re going to do for this month,” or “This school year, this is what we’re going to do, and we’re all going to jump on board, and this is how we’re going to teach reading,” let’s say for example. And we do it and certainly we’re good about following through and being good soldiers, but we don’t really know why we do it. And we don’t really know if we’ve grown because we don’t have that time to really reflect. This really gives us that opportunity to do that. An example I have of that is one of the professors Dr. Duffy, who is a professor here at SUNY Oswego. She did some PD around spelling and she did it with the adults—including myself—and there were things that we didn’t know. So we know as adults that we know how to read, but we didn’t really know why we knew how to read or how to read, and so the students really almost knew more than we knew, because they had been learning it and for them, it wasn’t anything that had to be retaught or relearned. And so we actually were reaching out to them for them to help us so that we could be working with the students. And that’s magical. That dynamic is not going to happen in any other setting, that we as the practitioners would be reaching out to the pre-service teachers. So I think that’s a good example of something that really, what we learn is going right into the classroom and how it’s a partnership, not, “I’m the supervisor and you’re sort of the student.” It’s really that partnership. That’s just I think a good example of that.

Rebecca: It sounds like really powerful interdependence. That doesn’t always happen.

Kara: Absolutely.

Christine: It is now. I think it has grown to be that.

Rebecca: Yeah, I can imagine things don’t always start that way. You have to really get to know and trust.

Christine: Trust is a huge part. If we go back three years, I remember walking into Kara’s office and introducing myself. “I’m your PS liaison!” “Oh, okay. Nice to meet you.” It was her very first month on the Leighton campus and, “I have a classroom in your building,” and “Let’s go see my classroom,” and it’s very awkward. It is awkward because it’s brand new for both of us, we don’t know each other, we think that we understand the vision, but it hasn’t really been created yet. All the pieces haven’t been thought through and it’s up to us to create whatever it is. And so it’s exciting and a little scary and weird all at the same time.

Kara: I would agree. We all come from a different place and so we all prioritize differently and I think what we had to do is we had to get in sync with that and have an understanding of the other person’s role and perspective. And I think that’s where we’ve all shown growth so that we can really provide the best model possible for those students that are coming in to learn from us.

Rebecca: It already sounds a lot, like really rich and deep and full of trust so I can imagine that it will continue getting even more rich as your partnership grows over time.

John: And it’s really convenient how close Leighton is to the college. It’s less than two miles away, so students can even walk there and back.

Kara: Yes, absolutely. In fact, I have—this is aside—but we have two students from SUNY Oswego that are part of our AmeriCorps program, and one of the students actually walks from campus so that makes a big difference that students have that accessibility.

Rebecca: So you’ve talked a little bit about the professional development aspect and the relationship that the campus has with providing some professional development opportunities for existing teachers at Leighton and Fitzhugh. Can you talk a little bit more about how that works?

Christine: Sure. We have ongoing professional development based on what our planning committee has decided the teachers would like and what our candidates like and need, and so the planning is always collaborative and then we have a semester long—or year long plan even—but it’s always grounded in what the district has set as their strategic plan, their initiatives. And so because we’ve been a part of Oswego City School District for so many years, we have relationships with people in the district office, in the buildings, we know that they have had two initiatives going on really for the last several years: explicit direct instruction and trauma-based teaching. And then recently they brought in an early literacy initiative that’s across the county. But one great thing about the Leighton-Fitzhugh learning community is that we really zero in on those initiatives. We don’t want our candidates learning things that aren’t going to be useful once they come into their practicum and student teaching. So for example, we have right now, mindfulness classes being offered—not only at Leighton and Fitzhugh but we’ve extended beyond to other buildings in the district. Oswego High School and Oswego Middle School had been involved in those courses for a number of years. We have yoga being taught in three of the buildings in Oswego City School District at no cost to the teachers here, these are all college professional development opportunities that we would like to provide and continue providing to help the district meet their goals. We do PD usually once a semester on giving and receiving quality feedback. So we know one of the sticky points of being in a relationship with a pre-service teacher, for the classroom teacher, is they’ve been dealing with children for many, many years. They haven’t necessarily been communicating with adults in an evaluative or critical thinking kind of way, and so we know the host teachers really are in a position to help our candidates in constructive ways. We don’t want them to be overly critical, but they have to be able to say when they see something going on, “I’d like to sit down and talk about this,” and really hit the nail on the head with that. And at the same time, our candidates—as they mature and become professionals—they have to have the language and the courage to go to the principal or go to the host teacher and say, “I’m really struggling with such and such, can you help me with this?” So giving and receiving quality feedback is a topic for PD that we’ve done a number of times. Co-teaching is a PD that we offer that’s very successful too.

Kara: I think just to add to that, Chris, I think that when the students and the teachers are working together to problem solve through what’s going on when they’re in the classroom, they can always refer back to those experiences that they’ve had during those PD sessions. So it’s not only that it works well when they’re working with students, but it also helps them work together as a team because truly, once the student has been there—I would say after their first or second practicum experience and they’re really part of that pre-service teaching mode—they really are doing that planning with the teacher. And so to be able to have those skills of feedback like Chris had said, is really important because often there isn’t enough time in the day to do that once you’ve started teaching. Once you’re live, you’re live. So to be able to do that ahead of time and even know what questions to ask, or what feedback to give, or why that would even be important, I don’t think is something we would have done before, and now it’s just part of our routine.

Rebecca: That just sounds really great.

John: It does, and one of the things I really like about it.. you mentioned the growth mindset idea. But when our students are there working with teachers and seeing that they’re going through professional development with them, I would think that would help build a growth mindset and help encourage them to become lifelong learners and realize that this is an ongoing process. That’s a really nice aspect of the program.

Christine: Absolutely. For too long we’ve seen such a division between what we experience in a teacher ed program on campus and what the real job looks like, feels like, demands of us, and really we have broken down a lot of that. We’re not completely there yet—we have a lot of work still to do—but for public school people to respect the contributions of teacher educators and for us to respect the jobs, the intense super-demanding jobs of classroom teachers and principals and then to bring all of that together, I think that’s where the power is.

Kara: I think it really forces us to reflect as practitioners because you have these folks around that are really depending on you and looking up to you and watching and we are modeling for them. And so really being able to talk about that, it’s one thing to be doing the job, but after you’ve done it for a while, you don’t so much really talk about it with anyone anymore. But really, that conversation has to happen so that it is rich for those students when they come into our building. So, it helps us be better I think, too, because we want to make sure that we’re doing right by our students that come in.

Christine: It heightens the professionalism just by having us in the building. And it helps us question how and why we do what we do. And we are watching them in action—it forces them to do the same. What are they seeing right now? And what are they thinking about what they’re seeing? And then we come together and talk about what we’re all seeing.

Kara: You have to be willing to be vulnerable to grow and I think that’s a big piece. And I can’t say enough for my staff that really has taken students and really, that’s the word I would use would to be vulnerable, that they really kind of put themselves out there so that the students will be able to go and teach thousands of students for years to come, which is really the ultimate goal… to be able to do that and to be able to give back to their community. Often many of them stay right here in Oswego and that’s really another one of the initiatives that the superintendent is looking at is, “How do we keep our community vibrant? And how do we keep students going?” And I think that’s definitely a piece of that.

Christine: In one of our PDs we invite the HR, the personnel director from Oswego City Schools in for a few minutes so that she can show our candidates how to apply for substitute teaching positions in the district. And it is quite a process, to go through the online application to come in for the interview, to become Board of Education approved. And so our candidates have to want to substitute teach to go through that whole process. But there’s such a shortage right now of high-quality substitute teachers everywhere we look. And so we feel at the college that we want to help address that problem by encouraging our candidates to apply to sub, get board approved. They’re very happy that they can then make some money and then be present in the school more if they could substitute teach and be present in their classrooms more than what they’re required to be. That’s the best marriage of all. We’re really helping both institutions with it. And we do have several board approved candidates in both buildings right now getting great subbing experience.

Kara: I would agree and I think that it really gives them a sense of value. Often they come in and out of fairness to the student teachers—the pre-service teachers, I know I keep using those words interchangeably—but I think that it’s a big commitment for them, and Chris kind of alluded to that. They really have to set their own lives aside to make this commitment because they are spending so much time with us. And I think it validates all of their hard work that we would trust that they could sub and they could be with those students. I think that gives them a sense of confidence and a sense of competency that the work that they have been doing is certainly the same kind of work that they’ll be doing when they’re out in their profession,—hopefully—a few months down the road once they graduate and get a position. So it’s about can you do the job, but also we know in teaching that you psychologically you have to be present all the time and you have to give 100 percent to the kids all the time. They expect that, they need that, they deserve that. And I think for our pre-service teachers to be able to actually do that, and to develop their own style, that’s another piece that you don’t necessarily get with the six weeks. But with us, they have learned what their own style is and how they’re going to go about managing a classroom and teaching the students in front of them.

Rebecca: I can imagine, especially in teaching teachers, but also in other areas that you’re teaching professionals. I’m a graphic designer, I teach graphic designers, which is also a professional degree, that the more you interact and integrate with the profession and know what’s going on and know what the challenges are, the better you can instruct your students and adjust the curriculum in higher ed to better serve what students are actually going to need in the field. So I can imagine, Chris, that being so embedded in the district right now in the way that this program is working, that you’ve learned a ton about how we should be educating future teachers, and have you had any adjustments to the curriculum as a result?

Christine: Well, I think that I am in a unique position being at the college full time and part of my load being out in schools. And so I do bring a lot of information to both groups as I learn it. I bring observations to both groups. I think that’s the only way good change can happen is if we keep those lines open and keep watching and learning from each other. We do have a ways to go, I think. Ideas are kind of popping in my head right now about ways in the future that we could really start bringing college folks and public school people together. Years and years ago I wrote a grant so that half of my load at the college could be covered and I taught a half day every day in a sixth grade ELA classroom in Oswego County with an ELA teacher. We co-taught every day and then on Fridays, I brought my literacy students out to that building to watch us co-teach and then debrief our literacy lesson afterwards. And it was ages ago that that happened, but I still think “Wow, how could we really start learning from each other in very practical ways, and then bring that back to our respective roles? So has our curriculum changed? I think it is starting to. We have a strong link with state education (as do public schools), our standards are changing, state ed regs are changing, what they require of for certification for our in-service teachers it’s constantly changing, and so we have to be in communication with CiTi BOCES, with public schools, with state ed, we can’t be isolated. And we have to keep reaching out and seeing that the schools are continually reaching out to us to be partners in that. So, taking a look at a syllabus, for example, and let’s sit around the table and we’re all looking at a copy of the same syllabus for a methods of instruction course. And all the eyes looking at that document are coming at it with a different lens and wow, what a conversation that would be. “Well, I think the new teacher should have this and this and this in there,” and other people think, “Oh, no, we don’t need as much of this as we have. Let’s take it out,” and just getting into those deep, professional discussions about what’s the most important thing for new teachers to know. I hope that we can keep going in that direction.

Kara: And I think as students go back to their professors, and talk about their assignments and what it is that they’re doing and give their experiences, I think that plants some seeds, and I think that’s what we can hope for going forward.

Christine: One of our methods professors said to me recently, “After I taught this course the first time, I looked at it and said, ‘You know what, they don’t need two research projects. They’re out in the field, they’re out with children all the time. I’m going to cut one of those out. I’m just going to do one research project and get rid of the other one and let them do some action research in the classroom.” Teachers are collecting data all the time on many different things. They’re observing kids in so many different ways and so that’s the research that is valuable, that we can learn so much from. We need books, we need articles, we need current research studies on teaching and learning. But we need action research that’s going on every day with kids in classrooms, too.

John: I noticed in an article on your arrival here that you had done some work at NORAD, before moving into teaching. Could you tell us a little bit about that?

Kara: Sure. Yes, I was in the Air Force and I actually was stationed in Colorado Springs, Colorado, it was about 1990, 1991, and I actually got to work in NORAD. And so that’s where we tracked Santa Claus. So, when I first came to Oswego and they asked the questions around what makes you unique and so we always kind of talk about, “Yeah, I worked inside of a mountain and we track Santa Claus.” And certainly, the United States Air Force does other things besides track Santa Claus there, but certainly it’s all about that problem solving. So when I was in the Air Force, very much there is always an end result. And we don’t give up and we have to figure out a way. There is no “Oh, it didn’t work out. We’ll try better next time.” It’s “We’ll keep working at it till it does work out.” And I think there’s some real same sort of ideas here when we talk about this partnership, that we keep growing and we keep learning, we keep problem solving, and that we don’t give up. Because think about how sad the children would be if Santa Claus didn’t come, right? and NORAD failed… So we want to do the same, think about how our children would fail if we weren’t doing our very best for them every day in a school setting. So, I think they definitely are the same in that way and I think the other thing is that when I was certainly working there, really it’s about how can we do things smarter, how can we do things differently, so that we can still get the same result but we’re not getting “stuck in the weeds” as they say, and I think that we did that at NORAD and I think we certainly are doing that with this program. What are those things that are critical and key to making it—like Chris has always said—that clinically rich environment for our students, for the students of the campus, for all the practitioners that are working with them? So, I would say those are the two things that are alike. No Santa Claus that Leighton though, but while I’m still working on it. [LAUGHTER]

Rebecca: Sightings coming soon.

Kara: Yes, right, sightings coming soon. That’s right.

John: Although apparently there’s Christmas Tea in July.

Rebecca: Yeah, well, you know… hey…

Kara: Yes, absolutely. Absolutely.

Rebecca: We always wrap up by asking what’s next?

Christine: Oh my goodness, we have a wonderful cohort coming in in the Fall, I can’t wait to meet them. I’m just excited to keep going into classrooms and seeing the work that our candidates are able to do. We did not have as high enough expectations of them until we began rich partnerships in schools. These candidates are able to do so much more before they even come student teaching than we ever imagined that they could and so capturing that, capturing concrete ways that they are growing in ways that we’re affecting the children in the elementary school—Kara says we’re not going to stop until we figure this out—we need tangible evidence that this is powerful and that it’s working. We know that it is, it’s not just anecdotal, so we want to look at it through a research lens.

Kara: Right. And I think that the way that we do that is that trust that Chris talked about earlier. I think the more we and/or the way we continue to have that trust with each other, the more we’re going to be able to talk about what’s working well, what are some things that we might want to do differently, and what does that look like? And then let’s actually try it, let’s not just talk about it, but let’s really put it into practice and then see what happens. If we have to take a step back, then we do. But if we don’t, then we know that this is something going forward that we can kind of put in our toolbox.

Rebecca: Sounds really exciting. Thank you so much for spending some time with us and telling us about this partnership.

Christine: You’re welcome.

Kara: Thank you for having us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

67. Iterative OER Development

Imagine course materials that are always up to date and evolve continually to become better at supporting student learning. In this, Dr. Steven Greenlaw joins us to discuss how some publishers of open educational resources are trying to set up sustainable practices to achieve these goals. Steve is a Professor of Economics at the University of Mary Washington and the author of the OpenStax Economics textbooks. He has also developed the materials for Lumen Learning’s Waymaker Introductory Economics texts.

Show Notes

Additional Resources

Transcript

John: Imagine course materials that are always up to date and evolve continually to become better at supporting student learning. In this episode, we discuss how some publishers of open educational resources are trying to set up sustainable practices to achieve these goals.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Steven Greenlaw, a Professor of Economics at the University of Mary Washington and the author of the OpenStax OER Economics textbooks. He has also developed the materials for Lumen Learning’s Waymaker Introductory Economics texts. Welcome, Steve.

Steve: Thank you.

Rebecca: Today teas are:

Steve: I’m drinking coffee. Thank you.

John: …and I have Enchanted Forest Fruits black tea from Epcot which I picked up while I was out there for the OLC conference where I last saw you, Steve.

Rebecca: You’ll never guess what I’m drinking.

John: English afternoon?

Rebecca: Yeah, it’s my favorite.

Steve: Well, honestly, I switched to tea in the afternoon.

Rebecca: See…

Steve: But in the mornings, I tend to drink coffee.

Rebecca: Yeah, you and many other people.

John: What prompted your interest in using and developing OER materials?

Steve: I have to say the developing came first. For a long time, I’ve experimented with textbooks going back into the 1980s, which at least John can remember. And I came to the conclusion that that it didn’t really seem to matter what principles book you used. Students needed a book, particularly for the analytical parts of the course: the models and things like that. But whichever book I used, they seem to learn just as well. And more recently, I’ve come to the conclusion that intro textbooks are commodities, that where companies are going to make their money is in the aftermarket products. But we’re not there yet. At least, the majority of the textbook industry is not there yet. So I had that and I didn’t really pay much attention in the 2000s about what textbook I was using, because I didn’t really think it mattered. But I did notice how high textbook prices were going and it was around that point that I became aware of and interested in OER. Again, this is dating myself, but when I was in college during the mid-1970s, I remember a teacher in my intermediate macro class—John, that’s for you—saying he would never assign texts for a course that collectively cost more than $10. [LAUGHTER] And so that’s sort of my base year. So, I sort of had this in the back of my head, I basically tried to choose around the least expensive textbook that I thought would work. And then out of the blue, OpenStax contacted me and said they had funding to create a principles of micro-macro text, and would I be interested in helping them out. I actually jumped at the opportunity, it sounded like a lot of fun. At that point I had already published one textbook commercially for an upper level course and I knew something about the commercial publishing process. I knew that I didn’t really want to go through that again, but I did want to get my ideas out there. One of the things about commercial publishing is they ask you, “What are all the innovative things you want to do?” and then once they have you on contract, they say, “Oh, but you have to do it like everybody else’s.” So that was the start. A year after the OpenStax book got published, I got contacted by Lumen Learning who said essentially the same thing. They said, “We’re building this digital platform, and we wondered if you would like to be the principals subject matter expert.” That’s the term of art that I become a SME.

John: So could you tell us a little bit about Lumen Learning’s project and the Waymaker version of this?

Steve: Sure.

John: What does it add?

Steve: It adds a lot. So, just to be clear, I wrote the OpenStax principles book. And we can talk about that process later if you want to, especially about peer review and things like that. And then I wrote the Lumen Learning Waymaker version, which was essentially an improved version. When we did the OpenStax principles book, we did it in an incredibly short period of time, I think it was nine months. So, when I did Waymaker for the first time, it allowed me to flesh out some of the things that weren’t ideal in the OpenStax book. And then OpenStax came back to me maybe three years ago and said, “We have funding for a new edition. Would you like to do that?” so I wrote a second formal edition for the OpenStax principles book. And then right after that, I did the same thing for Lumen. So, in my mind, I’ve gone through four versions of this now. And it’s not done and that’s part of the beauty of OER… at least the OER business. So to get back to your question, the OpenStax principles book is a textbook, it’s available in print and a variety of online options. My particular favorite is the phone app. So if I’m in class and a student asked me a question about something, I could literally look it up on my phone. Waymaker is a very different animal. It’s digital courseware so it’s a more immersive, interactive experience for students. And it’s not available in print. For example, how would you show a video or do a simulation in a print textbook? You can’t. The most you could do was provide a URL or something and have the student go out to that. In Waymaker, it’s all in one. So Waymaker, aside from text, it includes video, it includes animations, it includes simulations. Just to give you a specific example, instead of students looking at a graph of supply and demand, they actually get to climb in and take it for a test drive. Students really liked that. Many students seem to get it in a way that’s just looking at a two-dimensional graph, or reading text it is much harder for them.

John: I saw you present on this at the OLC conference…

Steve: Yep.

John: …And you demonstrate this. What software did you use to create those interactive graphs?

Steve: Those little interactives are H5C… maybe… it’s called? [It is H5P]

John: Okay.

Steve: It’s a European company, and it’s open source, and it’s really easy to do. I can say that even though I didn’t create the interactives. That’s the joy of working with a company… they actually have people to do the stuff that you don’t know how to do… unlike my earlier career, when I was the programmer, I was the graphic designer and all of those other things. Talking a little more about Waymaker, it’s more than a source of course content. It’s designed to teach students to study more deeply and more effectively. I don’t know about your students, but my students don’t seem to have learned how to study well. They’re very good at the game of school, but they’re not so good at learning. And I don’t mean that as a criticism. It’s just sort of a fact. They think study means read, highlight, read again, highlight again. When we know a lot from cognitive science now, that learning comes from working with the material. As I like to say, “the best way for students to learn economics is to do economics.” So Waymaker emphasizes mastery learning and personalized tutoring. The tutoring comes both from the software and also from the instructor. It’s designed to give students actionable feedback so that they can make their own decisions about how to allocate their study time. This is a really different way of learning, so I’m going to say it again a little bit differently. Assessment is integral to the learning process, it’s not just or even primarily about the grades. Rather, the assessment is designed to make students interact more deeply with the content and interact in a more intelligent meta-cognitive way. I can go into more detail about what it looks like from the students perspective, if you want.

John: Sure. Could you talk a little bit more about that? It’s a great approach. I tried to do that myself, but it’s always an add-on. Having it integrated is a nice feature, and one of the reasons why I’m planning to adopt your package in the fall.

Steve: This is really different for students, but also for the instructor. I’ve been working on this product for three years. When it finally came out in beta, I thought I knew what was going on, and I was really surprised at how little I knew about how it actually worked. Waymaker is organized into modules, which are analogous to chapters in the text. Students begin each module with the “show what you know,” which is basically a formative assessment. The purpose of that is to identify what content they already know. So, it gives them feedback on how they can efficiently use their study time. So, if there’s stuff that they absolutely already know, they don’t need to read about it again, they can just go into the stuff that they don’t know.

John: And even if they don’t, it activates prior knowledge. And it helps them make connections so that they can learn more effectively…

Steve: …Yes.

John: So there’s a lot of benefits, even for the areas they don’t know.

Steve: Yes. And I’m actually adding a little exercise for my first day of class next week, where I put my students in small groups. Some of whom who’ve had the first semester, and some of them who have not. And I’m going to give them a basically a problem to work with, knowing that some of them won’t really know what to do with it. But I want the groups to start working together. But anyway, I digress. So, as students progress through the content, there are a series of learning activities. The original one is called a “self-check.” It’s basically a short formative quiz. The purpose of the quizzes is not summative assessment. But as I said before, it’s to help students think more about their learning. Think about the idea of a Socratic tutor. The tutor doesn’t ask questions to assess the students’ knowledge, but rather to help them work through the content, help them really understand it. So what happens in Waymaker is: the student reads a page a text, or watches a video, or plays a simulation. And then they’re posed a very short quiz, like one or two questions. If they pass the quiz, the “gate” opens and they move to the next section. If they don’t pass the quiz—and on a one-question quiz, either you get it or you don’t—Waymaker suggests that they review the content before attempting the quiz again. They can take those quizzes as many times as they want to. So they can really build some expertise. There are other sorts of learning activities, but I want to focus on the quizzes today. At the end of the modules, students take a module quiz—essentially a chapter test—which is summative. Again, if they fail to achieve mastery—and the default mastery level is 80%, so it’s pretty high level. As an instructor, you can change that to whatever you want. But I like 80%. So if they don’t achieve 80%, they’re encouraged to study again and they’re given information about what areas to study. And then they can take the module quiz one more time. They’re only allowed to take the module quizzes twice. Now, here’s where it starts to get really interesting from the teacher’s point-of-view. The instructor receives reports from the module quizzes whenever a student fails. So for me, the first really good thing about Waymaker was that I don’t have to go to some website and look at some spreadsheet and see which students are struggling. Rather, anytime a student fails, I get pinged from the software. So it says, “so and so…” Well it’s a little boilerplate language… but basically it says they worked through the module, and they scored a 46 on the module quiz. You might want to reach out to them at that point. So the software is flexible. So you can get these things in real time, you can get them once a day, you can get them once a week, if you want to. I get them once a day. That seems reasonably quick for me. If the students taken the quiz at three in the morning, I’m not up anyway, so it hardly matters. It’s not like I’m going to give them that fast feedback. But what happens is I get that information, and then I get to decide, “What am I going to do about it?” If someone gets a 76 on their first attempt, I generally figure, “Okay, they’re gonna figure this out.” And so I don’t worry about it. If someone gets a 46, then I immediately want to reach out to them and say, “Hey, I see that you’re struggling with this. You know you can take it again. Go back and review the material. And if you’re not sure that you understand it, let me know and I will work with you on this. Because the goal here is mastery. It’s not anything else. Anyway, Waymaker helps me, the instructor, make better, more efficient use of my time. In any given week, Waymaker allows me to know two important things. It allows me to reach out only to those students that need my help. And it lets me know what topics the class is struggling with, so that I can tailor my in-class time to the material where the students need help and not spend it on material where they already know this stuff. Basically, it gives me a better feel for the effectiveness of my teaching and student learning. And that’s really, really important I think as a teacher. I’m embarrassed to think of my early years and teaching, when if I got all the way through the 50 minutes, I counted that as a successful day.

[LAUGHTER]

John: I think many of us started like that.

Rebecca: It ties really nicely to your blog post series that you’ve just recently published. The first one being the critical importance of instructional design…

Steve:Yup…

Rebecca: …where you talk a lot about the instructor’s role is designing the experiences, rather than delivering content. Can you talk a little bit more about how Waymaker helps you do that as an instructor?

Steve: There’s a “just-in-time teaching” element to this. I have a course outline, I know what I’m supposed to be doing on a week-to-week basis. But what happens on any given day depends on the stuff that came before it. I’m absolutely not wedded to the calendar. If the students haven’t figured out what we did on Monday, I’m going to start by spending a little more time on that. But also because of the feedback that I’m getting from Waymaker, there are times when I spend 90% of the class on 10% of the material. Because that’s what I know students are having trouble with. I know that if it’s something analytical, probably what I’m going to want to do is instead of talking to them about it—I mean, certainly I’m going to talk to them about it—I put together some group activities. I do a lot of group activities, small groups, generally two to three people. And then I essentially turn the classroom into a lab experience for that day. They seem to enjoy it more, they seem to get more out of it than me just lecturing over the content. After all the content is in the book. I don’t need to just repeat that stuff. So I guess that’s my short answer to your question.

Rebecca: Can you give an example of the kinds of activities that you’re doing with your students?

Steve: Oh, sure. Supply and demand is the first real model that the students work with. And so one of my learning goals is that they ought to be able to take a scenario… something happens… use supply and demand to analyze the effect on the market for x, gasoline or something like that. Typically what happens is, hopefully they will have read the material in Waymaker. Typically, I spend a day talking about “here’s how you would do it” and then generally what’s going to happen is, I spend a day where I have a couple of problems, like three is all that we’re going to have time to do. And I say, “Get in groups of two or three.” Basically, I count the number of students that showed up that day, because my classes are pretty small. And if it’s divisible by three, I put them in groups of three, if it’s divisible by two, I put them in groups of two. And then I say, “Okay, here’s a problem,” I show them the problem. And I say, “Take 10-minutes to work through this, draw the graphs.” And then they know that I’m going to call some of the groups up to present the results to everyone else. So there’s a little bit of competition. It’s not very stressful. It’s a little stressful for people that don’t like to speak in class, but you’re not there by yourself. You’re there with your group, so it works better that way. So I do a couple of those problems until I’m convinced that most people know what they’re doing. So that would be an example.

John: You also mentioned—when I saw you present at the Online Learning Consortium—how you use some of that feedback to improve the text in your current edition. Could you talk a little bit about that process of revision and creation of the text?

Steve: Sure. While I can’t take all the credit. From the beginning of Waymaker, at least from when I began to get involved… once I realized how integral the assessment process was to Waymaker, I pressed Lumen to make sure that the assessment questions were good. One of the things that I’ve noticed over the years is, test banks seem to be the lowest priority of textbook publishers. Because after all, they’re selling the text but they’re giving away the test bank. So what I want, I guess what we all want, is that the questions in the test bank that Waymaker uses, are discriminating correctly. And that’s harder than you might imagine. To their credit, Lumens put a tremendous amount of effort into this. And more generally, into the design the courseware. This has resulted in a process of continuous improvement. Now, continuous improvement is not a term that excites most faculty. I think that’s a fair statement John? [LAUGHTER]

John: Yes.

Steve: But what it really means is that, Lumen has an ongoing process for improving OER, making it more effective every single semester. And they’ve done this, and we’re now in your five and a half. So how does it work? I have a short answer and a long answer. The short answer is, after every semester Lumen downloads the data from every student who’s given them permission at every school using Waymaker across the country. And then they analyze the data. The analysis identifies where the students are having problems. At that point, we go in and either revise the content to make it clearer, or add some learning activities. Or else we revise the assessments to better capture student learning. We do this a little bit in a panicky way over the winter break, because we only have a month. But we do it intensely every summer. Here’s the longer answer. Over time, we’ve gotten better at doing this more efficiently. Lumen has developed something called “RISE Analysis.” RISE is an acronym. I don’t remember what the letters mean. [LAUGHTER] But basically it asked the question, “Which course materials would benefit the most from improvement?” Or to put it differently, “Which changes would have the greatest impact on learning?” So what we’ve done—and this is all programmed now. So Lumen has dozens of Waymaker courses, not just an economics. Though, I like to think that some of the most interesting stuff is started in the econ Waymaker platform. I’m not just making that up, it’s actually true. [LAUGHTER] So, instead of just doing the aggregate sweep on the data, we particularly look at student learning outcomes. And everything in Waymaker is driven by the student learning outcomes. This is out of order, but let me just throw this in for a minute. The way that Waymaker started is they brought together—I want to say 50 principles instructors from everything from community colleges up to R1s. And we spent four days together. And we asked the question, “What do you have to have in your principles courses?’ And so from that we created a list of primary learning outcomes. And then we drilled down and we now have secondary and tertiary outcomes. So the assessment questions in the test banks are coded down to the third level. So everything is really granular, if you want to think about it in those terms. What we look at is not just which student learning outcomes are students struggling with. But rather, which student learning outcomes where students are doing relatively poorly, are they putting a lot of time and effort into. Because that’s where we’re going to get the biggest bang for the buck in terms of fixing things. So what we do is we look at three things. We look at, “Are the questions badly worded?” We’re mostly done with that at this point. “Are the questions testing what they’re supposed to be testing?” There are some psychometric tests that allow you to do that. And then finally, what we do is—after we’ve exhausted all those—we look at the content and we create new content, or different types of learning activities, and we integrate those into the course. So, the interactives that you saw at OLC John, they were the big new innovation from last summer. So we do this, and then we teach the courses again, and then we start the cycle all over again. So, the process just goes on. It’s not continuous, as in every day, but it’s continuous, as in regular. I’ve used the courseware since the first year, and the courseware has gotten noticeably better. Fewer students are failing to achieve mastery on the module quizzes. And fewer of them are crashing and burning. More of them are in the 60 to 70% range when they fail. But what’s really cool is Lumen has shown no sign that they’re ready to quit, that they’re done with this. As long as they’re willing to do this, I think I’m willing to do this.

Rebecca: I like the iterative process.

Steve: Yeah.

Rebecca: That’s something that, as a designer, I’m very comfortable with… that I do all the time, especially designing online. But one of the things that’s really interesting about this model is that, as the author of the textbook, you don’t just have this finished thing. It’s an ongoing…

Steve: …It is.

Rebecca: … thing. So that’s a really different model of authorship.

Steve: Yes, it is. I think it’s fair to say that we make small changes all the time. And then every summer, we make larger changes. And that’s pretty interesting. Because as a user—as you pointed out—I can see that this is helping.

Rebecca: Yeah, that’s really exciting.

Steve: Right now, the hardest part is getting students to trust the process. Because it’s a very different model of learning. And so one of the things that I’m going to do this semester is, build in opportunities for me to remind them that this is a different process, and that they need to trust the process. One of the things that I did last year, which seemed to help with that was I started using exam wrappers after the midterm exams. And ask them to think about how they were studying, and what they would do differently, and what I could do to help them. It’s real easy to see in 30 seconds, I can tell if they’re taking it seriously or not. And if they’re taking it seriously, I learn a whole lot from what they say. So, anyway, just another little wrinkle.

Rebecca: So we’ve talked a lot about the students and the different learning process for students. You talked a little bit about the different processes being the expert, or the writer of the book. And you also mentioned earlier about the peer-review process for an OER being a bit different. Can you talk a little bit about that?

Steve: Sure, and that’s really important. First of all, people have a wrong idea about how OER is produced. The OER that I have experience with is working with OER publishers. It’s not the loan faculty member working in their spare time in their basement, or something like that. Both the OpenStax and the Lumen experience for me, have been very much a team effort. There have been a lot of people involved. So this is really important because one of the concerns about OER textbooks is their presumed lack of quality. There was an article in The Chronicle about that, today in fact. I have to tell you that the peer-review process that I went through with OpenStax was extensive. The way we did this is, OpenStax purchased a manuscript from Tim Taylor—a prominent economist—as the basis for the first edition. They sent copies of the manuscript out to about two-dozen reviewers all over the country, asking them to identify strengths and weaknesses. Based on those review comments, I rewrote each chapter. Each chapter was then sent out to half a dozen new reviewers. And again, the reviewers were from a range of schools, from community colleges through research universities. I took that feedback and I revised each chapter again before it went through the editorial review and production process. I have to say, this was much more detailed and extensive then when I worked with a commercial publisher. The review process for Lumen was similar, there was a lot of peer review involved. And as I said before, I’ve now written two formal editions of both texts. We’ve gotten lots of feedback from users. I’m pretty happy with that.

Rebecca: Do you find that the difference between OER and a commercial publisher is that you keep getting this feedback from users? And that you’re able to revise based on the use of other faculty, rather than working in a silo?

Steve: If I’d written the principles book for a commercial publisher, I would be better able to answer that. I got no formal feedback on my commercial book. I got a lot of comments from people at conferences and things like that. But we have gotten tons of feedback on the OER books, and that is interesting. You can’t satisfy everybody. Somebody says, “This chapter is too long.” Somebody says the same chapter is too short. But, in general, the feedback has been really, really helpful. And we’ve tried to incorporate it as soon as possible. And with these digital text, it’s really easy to do. I can literally go in and edit if I have five-minutes on the fly. And then it’s out there.

John: While with regular publishers, there’s usually a three-year cycle on intro textbooks.

Steve: Yes. And that’s the other thing that—now I’m not a typical user, but I know that if I want to make a change, it’s going to be done by the next semester. The same thing is generally true of other people who give us feedback. Though, they don’t necessarily know that. W e take that feedback very seriously. And there is no three-year review process. So that’s wonderful.

Rebecca: I love the user-centered design process, like that’s clearly what’s being used.

Steve: Yep, we try.

John: And that iterative process is what we should all be doing with our courses, all the time…

Steve: …Yes.

John: …But the fact that you’re doing it makes it easier for instructors who perhaps, don’t have to do as much of that.

Steve: Yeah. But again, let me just say one thing; Waymaker is not my course, Waymaker is my text. So there’s whole levels to my course that go beyond Waymaker. That’s just one element of it. Not that I’m disagreeing with what you said.

John: I’ve seen you present at conferences on teaching principles, for decades now. And I know you’re constantly changing how you’re teaching your courses and trying new things there. And you’ve been doing a lot of great work for quite a while.

Steve: Thank you.

John: Going back a little bit though, to the question of mastery quizzing. When students take the quizzes at the end of a block, you said there’s one or two questions. When they do it a second time, do they get the same question or different questions?

Steve: No. We are adding questions fairly regularly, and so the test banks are getting larger. From the beginning, I think we started with 2000 questions. But again, that’s across the whole book. The questions are randomly chosen, so the odds are that students would get different questions at the self-check level, at the section level. There’s a different test bank for the self checks than there is for the module quizzes. But there are similar questions. In fact, we wrote two at a time basically when we did that.

John: This is a question more generally about Waymaker. Does it do any type of interleaved practice, where later in the course, does it call back earlier sections? Or is it just based on the current module?

Steve: No, it’s just based on the current module. But my more nuanced response to that is, economics is sort of cumulative. But I have thought about that, we just haven’t thought of a way to build it in yet.

John: In my classes, I’ve been adding that the last couple years where I just randomly pull in questions and the module quizzes from earlier modules. Maybe 10 to 15%, building up to about 20% at the end, just to help do a little bit more spaced practice as well.

Steve: I think I know how you could do that pretty easily. Because instructors have access to the test bank that their students are using, so that you can edit your own questions. But what that also means is that you could move questions from earlier into the course to later in the course. So I think there’s a way to do that.

John: Excellent.

Steve: So John, we learned all this in our graduate training, right?

John: [LAUGHTER] You know, it’s getting a little bit better. Some people are learning these things. We have someone in my department who actually came out of Kentucky where he had a lot of training and teaching and learning. But it’s still pretty uncommon.

Steve: Yep.

John: You mentioned two ways in which, OER materials are developed. Some by primary developers, such as the OpenStax and Lumen. And others, with people working in their basements…

Steve: …Yes.

John: …or working in a dark room somewhere. Which is how I often do a lot of my work. Is that process sustainable? And what role do for-profit publishers such as Lumen play in providing these services, or in continuing the development of OER materials?

Steve: There are a couple questions here. One is, is the development process for published OER materials, or OER materials created by publishers. Is that sustainable? And then the second one is, is the individual scholar model sustainable? And those are very different questions. The individual scholar model, I don’t know if sustainable is the right word. I have a colleague who did this, she did it all on her own. I’m so impressed. She didn’t have any support from the school other than a small summer grant. And she did it without any sort of extrinsic motivators. I think that over time, at least at schools like yours and mine, faculty are going to get credit in tenure and promotion, for creating OER, especially open textbooks. I think that’s really important. I think that people will eventually be able to get sabbatical leaves to create these materials. And I think that’s really important to keep that side of the OER creation process going. As far as revision, I don’t know enough about that to really answer that. But I’m curious. I may have to go talk to my colleague Katie now. As far as the publishers go, and I don’t mean the traditional publishers, every publisher has a plan for how they’re going to do this. Some work better than others. I know something about OpenStax and I know a lot about Lumen, about what their sustainability plan is. OpenStax have develop partnerships with a variety of ancillary publishers like Sapling Learning or Knewton. These people provide aftermarket functionality for the OpenStax books, and in return, they get kickbacks from these ancillary publishers. And by kickbacks, I don’t mean anything pejorative about that. I just mean that they contribute financially. I don’t know any more about how sustainable that model is. I know that that’s what OpenStax has been using. Lumen from the beginning, has been a commercial publisher. It took me two years to figure out how a commercial publisher could make money giving their content away. Maybe others haven’t thought about that, but I sure did. So, the short answer is, Lumen gives the content away, but charges a very modest amount, $25, for the intelligent backend. All the feedback that goes both to the students, and the instructors. Today, you personally, either of you, could go and get a copy of the Lumen Principles and Micro book, or the Principles of Macro book, and it’s yours forever, you can do with it what you want. But if you want to take the full Waymaker course, they charge $25. The idea is, that amount of money is both affordable to students, but also enough to maintain revisions and corrections, and keep the servers running and all of those things. So that’s the answer to that question. And I will say that every semester, I try to be completely transparent, and say, “If you don’t want to pay the $25, you can get all the content for free. But here’s what you lose.” In five years, I’ve never had a student who didn’t pay the $25, because they thought it was like beer money for the weekend, or something. Compared to spending 300 bucks on a traditional text that was nothing to them.

John: What are some of the barriers that you see to faculty adopting OER? You mentioned that people may have this perception of lower quality…

Steve: …Yes .

John: …but there’s quite a bit of evidence that the quality is not weaker in any way. And I think you had done some studies on that a while back, didn’t you?

STEVE. Yes. The number one problem I think is misinformation. The majority of faculty today don’t know what’s available in their discipline. Many of my colleagues have told me, “Yeah, OER sounds like a great idea, but there’s nothing available in my field.” Now, that’s flat out wrong. For your listeners, there is OER available for nearly every Gen-Ed course taught today. So that’s number one, is lack of knowledge of what’s available. Number two is, as you mentioned before, the belief that OER is inferior, that there’s no peer review. And that’s just not true. There’s a couple things here. One is that OER publishers don’t have a sales force, and so it’s going to take longer to get the word out. There’s been a lot of progress over the last few years. But at my school, we’re only in the second-year of our formal OER initiative. So we’ll see how it goes. The other thing that I think gets in the way of adoption of OER is path dependence, and the unwillingness of many faculty to change their textbooks because of the fixed costs involved. “I’m going to have to go through my lecture notes and make sure that I’m using all the same terms as the textbook does,” and that sort of thing. I don’t know the answer to that question. I know that some schools have used financial incentives, fairly modest financial incentives, to get faculty to try to make the switch. As far as my own assessment goes, every summer, I do statistical analysis of the effectiveness of the texts that I’m using. I looked at both the OpenStax Principles book, and also most recently, the Waymaker package. What I’ve looked at is, textbook alone, textbook with ancillary website, digital courseware, and because I used to teach a writing intensive version of the principles courses, I also looked at writing intensive. And what I found is pretty predictable, at least from somebody who has done this for a while. What I found is that there is no significant difference between student learning using OER, with commercial textbooks. I found that using either courseware or an ancillary website improves student learning outcomes, regardless of what the text is that you’re using. And I’ve also found that writing intensive courses seem to work better than non-writing intensive courses, because the students are getting into it in more detail. Over the last two years, I’ve been doing a randomized control trial, where I can really drill down and see what’s going on. And what I found is that using the full Waymaker package seems to have a statistically significant positive impact on student learning. So I’m going to rerun the analysis using last semester data, which I haven’t had a chance to get yet, but I’m anxious to see how that goes too. I believe this stuff works. And so I think sooner or later, more and more publishers—the commercial publishers too—are going to move towards digital courseware type products.

John: I think most of them have started to at least.

Steve: Yes, but it’s like turning the Titanic. Their base is so large that it’s going to take a while before even all of those people get on-board with this.

John: One thing I was wondering is whether you see more collaboration or competition in OER textbooks?

Steve: Initially, there was more collaboration in the early years. And the reason why is because anybody who was doing OER, was increasing the interest in users for everybody’s OER. Now, I think we’re going to see more competition between the users. Especially as more publishers are going to adaptive and personalized learning type courseware. I think that’s a way that publishers are going to be able to say, “Well, yeah, we’re doing that. But we’re doing better in our own particular way.” So I think there’s going to be a fair amount of product differentiation. And it will be harder for faculty, it’s going to take more work to dig in and see exactly what’s going on. I would love to see more published assessment of efficacy on the part of the commercial publishers. They’re only now starting to do that, and the studies that they publish are heavily controlled by them. So it’s not clear that they’re telling us about all of their things, just the ones that work. But at least it’s a start.

John: One of the things I see in most of those studies is comment to the effect that, “Students who use our adaptive learning platform have letter grades on average, one letter grade higher or point eight points higher.”

Steve: Yes, that’s right.

John: And there’s no evidence that they’ve done any control for the students who chose to use it versus those who didn’t.

Steve: That’s right.

John: But it would be nice if we could see more research on that.

Steve: And I think we will. At least I’m hopeful.

John: Earlier you told us a little bit about how your course is structured with some “just-in-time teaching,” and some activities there where you have students work on problems. Could you tell us a little bit more about how you structure your course so that it’s not duplicating the textbook?

Steve: The first thing that I would say is that, my intro course looks like almost anyone else’s Principles of Micro or Macro course. If you look at the course outline, it has all the normal topics in it. A very slight difference is, instead of assigning students chapters to read and problem sets to do, students have modules with content and learning activities to complete. There is some difference between my face-to-face sections and my online sections, because I teach both. My face-to-face sections are pretty much the way I described them to you earlier. My general approach is to do Socratic lecturing with a lot of in-class activities, like the supply and demand problems that I mentioned. I also like to have formal in-class discussions on interesting questions that don’t have a right answer. In the macro class, I spend a day talking about what is money. And I spend the day talking about what is government. And those are things that aren’t done in the same way and the same degree with a textbook, whether it’s Waymaker or something else. My online course is roughly similar. But what I do is I add group and individual activities to the online course to mimic what I do in class. I also have a weekly Google Hangout, a synchronous Google Hangout, where I can give students guidance about what I think they should be doing. And I can give little mini lectures on things that I know students have trouble with. But it also gives them a chance to ask me individual questions in a real time basis, one on one. Not a lot of students come to those Hangouts. I usually have between five and ten, and my classes are about 35. But more than 90% of the students watch the recordings. Google Hangouts are automatically recorded and archived in YouTube. So the students seem to like that a lot.

John: You mentioned that a number of people at Mary Washington have switched over, what proportion, would you say, of the faculty at Mary Washington has moved to using OER?

Steve: Single digits, a handful, probably less than ten at this point. But this semester, I have two new people. So I’m excited about that. And we haven’t yet given them any money or anything to do this. I’ve just been talking to people. I was invited to the College of Business’s summer retreat, and I gave a little talk about OER. And I got two people who expressed an interest in following up. One of whom has already done it. So I think we’re getting there. We just have to be patient.

Rebecca: So we normally wrap up by asking, well, what’s next?

Steve: What’s next for me is I’m continuing to iterate to improve Waymaker. I’m going to continue doing my own statistical analysis. So I get access to the aggregate analysis that Lumen does, but I also have my own analysis. So I can tailor that to my particular students. I also want to do something this semester that I’ve wanted to do for a long time, but have never done it. And that is to write a new non-traditional chapter for the micro book, which is relatively easy to do. It’s just really a question of me sitting down and doing it. So I know it’s doable, but I do want to actually make my version of Waymaker different from the standard version. In part, because it’ll better match the way I teach. But also because I want to see that it’s relatively easy to do so that I can talk about that to faculty.

John: Very good.

Steve: I’m going to the CTREE conference this summer to talk about Waymaker. And this is the first time we’ve actually reached out to a disciplinary conference. So I think that’ll be fun.

John: You know, I always want to go to the CTREE conference, but I teach at Duke in the summer and it runs right into that. So I haven’t been able to go. And we should note that the CTREE conference is a Conference on Teaching and Research and Economic Education.

Steve: I love to talk about this stuff, because I believe it.

Rebecca: Yeah, it was really interesting.

John: Thank you.

Steve: Oh, you’re very welcome. Thanks for the opportunity.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kelly Knight, Kim Fischer, and Jacob Alverson.

[MUSIC]

59. Gatekeeping in Math Ed

Teachers at all levels often play an important role in influencing the educational and career paths of our students. In this episode, Dr. Marcia Burrell joins us to discuss how math teachers play a critical role as gatekeepers who may either welcome students to or provide a barrier to student success in all STEM fields. Marcia is the Chair of the Curriculum and Instruction Department at SUNY Oswego.

Show Notes

  • National Council of Teachers of Math (NCTM)
  • Budapest Semesters in Math Education
  • Polya, G. (1973). How to solve it: A new aspect of mathematical method. Princeton: Princeton University Press.
  • The Polya Approach Used at the University of Idaho
  • Bjork, R.A. (1994). “Institutional Impediments to Effective Training”. Learning, remembering, believing: Enhancing human performance.
  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • Brown, P. C., Roediger III, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
  • Miller, L. & Spiegel, A. (Hosts). (2015, January 23).Invisibilia: How to become Batman pt. 1 [Radio broadcast episode].
  • National Research Council, & Mathematics Learning Study Committee. (2001). Adding it up: Helping children learn mathematics. National Academies Press.
  • Brandsford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. National Academy Press.

Other resources:

  • Larson, M. (2016). The Need to Make Homework Comprehensible. National Council of Teachers of Mathematics.
  • Stinson, D.W (2004). Mathematics as gate-keeper: Three theoretical perspectives that aim toward empowering all children with a Key to the Gate, The Mathematics Educator14 (1), 8–18.
  • Burrell, Marcia (2016) Gatekeeping in Mathematics TEDx talk at OCC. January 29, 2016.

Transcript

John: Teachers at all levels often play an important role in influencing the educational and career paths of our students. In this episode, we examine how math teachers play a critical role as gatekeepers who may either welcome students to or provide a barrier to student success in all STEM fields.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Dr. Marcia Burrell, the Chair of the Curriculum and Instruction Department at SUNY Oswego. Welcome, Marcia.

Marcia: Thank you.

Rebecca: Welcome. Today our teas are…

Marcia: Earl Grey with caffeine.

Rebecca: Extra caffeine. [LAUGHTER]

John: Mine is just a pure peppermint tea today.

Rebecca: And I have a jasmine green tea.

John: We’ve invited you here to talk a little bit about the work you’ve done on math instructors as gatekeepers. What does it mean to be a gatekeeper?

Marcia: Well, I like to use the word gatekeeping because sometimes gatekeeping has to do with an open gate, where you can just slide right through, or someone gives you the key, or they’ve given you the secret password, or it’s a barrier, where if you don’t really know what the hidden curriculum is about passing through the gate then you could stay there and be turned away. And in mathematics a lot of times people are afraid of math or they’ve been socialized to think they cannot do math and it’s really a gate that’s been created either by themselves through socialization or it’s been created by a math person or by someone like a parent who said, “oh, don’t worry, I wasn’t good at math either.” So, when I think about gatekeeping and mathematics it’s really about barriers that are created by us or barriers that are created by others, or for people who are really successful in mathematics, they have an opportunity to open the gate; there are certain things that they can do that will make people pass through the gate more easily.

Rebecca: I think our students can empathize with the idea of gatekeeping when it comes to mathematics—you hear them talking about these stories of certain situations where the barriers have been in place for them, or sometimes that’s faculty. For example, I’ve heard many times in creative fields where the creative faculty might say, “yeah, we know you’re not great at math but you have to take math,” or I had a situation when I was a kid in middle school—I remember distinctly middle school teachers saying “the women in this class aren’t going to do as well” and then I remember the few of us banding together and then we got really good grades on this final exam that we were told that we wouldn’t do well in. I think that those narratives are certainly there and it’s interesting to think about it not only from the person coming to the gate but also from the gatekeeper perspective, which leads me to the question of, what are some things that gatekeepers do that keep people out?

Marcia: I’m gonna focus on math people mostly, where sometimes they say things like maybe in a beginning level math course, “Why didn’t you know that? You should’ve learned that before. I don’t understand why you can’t do fractions.” So, there’s vocabulary built into a lot of us where we send out messages which get people to realize, “Oh, there’s something wrong with me; I should know how to do this.” So, they start imposing those same messages on themselves. The other thing that I think is important is in mathematics there’s always been a stratification about who can do math or who should do math and who can be successful in math. Often, as you just said, women have stories about fighting to get into a advanced math class because they didn’t do very well on some class but they were willing to work hard. So, certain populations are harmed because they’re socialized that way that when women have trouble in mathematics we say, “Oh, we should make it easier; you should do a group of courses that are not gonna lead you to calculus in high school,” but sometimes when men struggle we go, “Oh, struggles perfectly fine.” In the U.S., teachers make it easier for students to learn; they give them answers, they work out all the details. When I say give answers, I mean they work out all of the problems so that it’s really just rote, as opposed to in other countries, struggle is actually honored—hard work and struggle is part of the mathematics learning process, where in the U.S. sometimes we don’t allow people to struggle. If you got a B in Algebra I, well, you don’t really need to take Algebra II because the minimum requirement in New York state is Algebra I, and the fact is struggle is a part of the learning process. Historically, we’ve always stratified who is successful in math or who can take math and the level of courses that people can take. Plato 2,300 years ago believed that everybody needed arithmetic, but the advanced math was relegated to philosopher guardians, and in the 1920s the National Council of Teachers of Mathematics argued to have mathematics part of the curriculum, and between 1890 and the 1940s there was a growth in public schools and the perception was that sometimes they weren’t sure that students had the intellectual capability of doing some of the mathematics that NCTM thought was important. But remember in the 1950s the business world and industry said, “What are you guys doing in schools? The people that you’re putting out there can’t do mathematics.” Well, that was mathematics for a purpose and then Sputnik happened and all of a sudden math became this subject that we wanted to make sure people had. But think about how many English classes do people take—one or two in high school, but in high school students often take four or five math courses. I’m not saying they’re not important, but it really forces us to think about mathematics as an elite subject when gatekeeping from my perspective is it’s not about an elite subject, it’s everyone can do math; people are born mathematical and everyone should have an opportunity to do the subject and not fail at it, but struggle and make movements towards whatever learning they need to do.

John: So a lot of this sounds like our society is creating or emphasizing or encouraging the development of fixed mindsets in math where many messages coming through (as you both have mentioned) in early childhood discourage people from thinking that they’re able to do math and only the elite can get through. Is that common in other cultures?

Marcia: I mentioned earlier that in other cultures hard work and struggle are honored and I witnessed in Budapest, when I was visiting there as part of my sabbatical, that students were asked to go up to the board and struggle through a problem, even if they had no idea. And we do a lot less of that because either you know the answer or you don’t; that doesn’t really work that way—it is an iterative process. I used to work on problems and maybe get a little frustrated, put it away and the next day I’d look at it and I go, “Oh, now I get it.” It’s really about process. The NCTM standards talk about process and product, and if you want people to learn mathematics then you really have to emphasize process, working in teams, giving people a chance to try things and fail but also collaborate with others to ensure that maybe there are multiple ways of approaching a problem, but if you’re not allowing students to talk with one another and work it through, then sometimes they think there’s only one way to do it and it really doesn’t improve their mathematical abilities. Mathematicians are about process—there are certain skills that mathematicians use. Good mathematicians persevere through problem-solving. They check their answers using different methods, they plan how to solve a problem versus jumping into a solution, and they justify the answers and communicate with others. Good mathematicians don’t just know the answer; it’s a process, and there’s even collaboration between mathematicians, but when we teach it on the K-12 level, we say, “This is what you need to learn and you need to learn it in a specified amount of time,” and so a lot of times students are turned off by the way we teach mathematics. Opening the gate is really about helping teachers rethink how they actually teach mathematics. We have a lot of data about how to successfully teach math, and it’s about problem-solving, reasoning, communication, connections and representations, but if you’re just gonna stand at the board and write the answer to a problem, that doesn’t help people really connect to how you came to that problem. So, gatekeeping is about getting teachers to rethink how they’re teaching mathematics and what they think is important. Process and products are important, but process is actually more important.

Rebecca: You mentioned mathematics as a collaborative process, but in my experience in K-12 I don’t think I ever worked with another person once.

Marcia: It’s funny you mention that. Again, the stratification stuff is huge. I attended a program called Budapest Semesters in Math Education and it’s geared for Americans, Canadians to come to this program. They’re interested in both juniors and seniors to come and learn about the problem-solving approach to mathematics. These are students who are mostly math majors, but they could be math ed majors, and they are sent to these schools where they’ve selected the top students in mathematics to use a problem-solving approach and what happens is they give them a problem with no background and they ask them to work out these problems. They can use their textbooks, they can use calculators, but the fact is our students—Americans and Canadians—get to witness students almost trying anything to work out these high-level mathematics problems—sometimes they’re theoretical, sometimes they’re applied. But what the students say who are in this particular program—and I got to be in these classes with them—was, “Why can’t we have all students use some of these processes?” And the processes are really just the things we already know that good mathematicians are supposed to do, sort of George Pólya, you know: analyze the problem, look at all the facts, try something, test your answer. But you actually get to witness that. So, when you asked me “None of the classes I ever went to that were collaborative and problem-based and working in teams,” well we seem to have an idea that only the gifted and talented or special programs will allow kids who already show aptitude to do mathematics in that particular way, and the fact is I visited a school in Budapest where this teacher who’s been working with the gifted and talented students got permission from the parents to try this problem-solving approach for a ninth grade through 12th grade. They had to get sign-offs by parents, because of course, in our system, if kids don’t know certain things by the end of certain grades then their opportunities—another gate—for getting into the university and going through the career path are cut off. So, these parents had to sign off that they were going to risk that what she was gonna to do over the next four years was gonna be helpful to their students and that they wouldn’t be harmed by doing this problem-solving approach. I witnessed several math classes where this teacher had been working as part of her dissertation to have students go through this problem-solving approach—it’s not just Pólya; there are other… Pósa there’s a Pósa method—I met this gentleman who, he was in his 80s and he invented the Pósa method and he’s one of the top mathematicians in his age… in his day, but he devoted his life to teaching problem-solving to kindergarten through grade 12. But the point that I’m making is, I witness students who had been through this process, and they were explaining problems to their peers on the board in ways that I haven’t seen good math teachers explain. But they built these kids up from start to finish to be confident about what they knew, to work in groups, not be afraid to make mistakes, and I think that we can do more of allowing students to learn not just at their own pace, but learn what mathematicians do—the process of engaging with one another if we weren’t so afraid of the whole accountability—what do kids know at the end of 12th grade? What do they know at the end of 11th grade? It’s recursive. Some things they learned in ninth grade in Algebra I will come back in Algebra II and when they’re college students they’re gonna pull the algebra and geometry together, if we allow it, as opposed to looking at these areas as completely separate things. One of the things about gatekeeping is that teachers have to think about students as already being competent; they’ve got to provide students with scaffolding so that students that are in different places have an opportunity to demonstrate what they know. I also think that we have to have high expectations, but we have to let students understand that they can extend the learning if they take some risks; that’s what good mathematicians do, and then we have to exhibit in depth knowledge as well as subject matter knowledge. So there are certain things that gatekeepers—math teachers—can do, but they’ve got to trust that students can learn, and we’ve got to keep the expectations high, but also scaffold for them so that they’re successful.

Rebecca: …a lot of evidence-based practices.

John: Yes, I was just going to say a lot of what you’re talking about, there’s a tremendous amount of research supporting that, not just in math instruction but across the board. In terms of providing students with challenging problems—you have the desirable difficulties of Bjork and Bjork, for example, and in terms of learning from mistakes, that shows up in all of the research on teaching and learning and it’s something that Ken Bain talked about when he summarized some of this research in What the Best College Teachers Do, and it’s also shown up in several of the books we’ve used in our reading groups, Make It Stick and Minds Online, for example: that retrieval practice, low stakes testing, where students can make mistakes and learn from mistakes, is effective in all types of instruction. So, these are really good practices that seem to be mostly neglected in math instruction.

Rebecca: I was expecting John to also mention something about growth mindset. [LAUGHTER]

John: I think I already did a while back, but treating math as something you’re either good at or not good at by teachers and by families and by our culture discourages the development of a growth mindset, and that’s really important. This year I’ve completely flipped my large microeconomics class and one of the things I had them do is before each class I asked them to do some readings and then I asked them to work through some problems in the readings; I have students submit a short Google form, where I ask them just two questions before each class. The first question is: “What have they learned from this reading assignment before that day’s class?” And also, “What are they still struggling with or what don’t they fully understand?” And half to two-thirds of them before each and every class list, “I have trouble interpreting graphs;” “I have trouble understanding graphs;” or that “I have trouble computing these things,” and that’s all basic math, and of course they have trouble doing it when it’s the first time they see it, but they see it as a barrier— “I’m just not good at it,” and every day in class I’ve been trying to encourage them to say, “Well, you may not do it now, but you can get better at this;” “You haven’t yet mastered this;” “You’re not yet good at this, but the more you do it the easier it gets,” and we’re not always seeing that happen, and we see that in lots of areas.

Marcia: Yeah, I think that students are more willing to say “I’m not good at math; I don’t have any experience with math,” but they would never say, “I can’t read; I’m not good at reading.” They might say it, but it’s socially acceptable to say “I’m not good at mathematics,” and the fact is when you look at a group of kindergarteners and they’re in a classroom, they’re all learning for the first time how to add and subtract and they slowly… I’m sorry, through some of their elementary school teachers who often are afraid of mathematics, and they say little things, “Oh, don’t worry about that, it’s okay to not be able to do that, we’ll work on that later on,” but they say it in a way that sometimes gives students permission to say, “Oh, I don’t have to learn that—I’m a girl, I’m a student of color, I don’t have to learn that because the teacher said she doesn’t know it either,” and so one of the concerns that I have for how we train childhood educators is we force them through, at least on our campus, these two math classes where they go kicking and screaming, but the fact is we almost need to reprogram them to think about the things that they can do mathematically and then build curriculum around them. It’s not always about the fact that the way you learn is the same way that all the kids that you’re teaching learn; it’s more about how do you change your perceptions about mathematics. There’s something on NPR, and I’ll have to find the reference a little bit later on, where this young man who was blind learned how to ride a bike, was sent to school, and people couldn’t even really understand why he was able to do all of these things as a blind person—well, his mother decided to treat him like he was a sighted person and it’s a Batman series, where the fact is, if you convince someone that they can do something and you believe it, then all of the things that you do to work on their perceptions about their capacity will come through. But first the teacher has to believe it and then they have to do all of these things to scaffold it. The fact is that, and again, I’ll have to find the researcher, but he did this study where he told all of his researchers that these mice were smart mice… these mice were everyday the same mice… what happened is the researchers came in and they treated those mice like they were smart—they handled them differently, they had them run through whatever people do in psychology with mice, and then he came back later on and said “All of these mice have exactly the same capabilities.” Well, that works in exactly the same way in the math classroom; students come, and if we believe that they’re capable and we come off and treat them with respect about what they have learned and how to build on that, then we’re gonna see better progress in their learning. I have to come back to the gate because the teacher has a lot of power to make the gate accessible or make the gate a barrier, and the barrier is really just the messages that the teacher says to the students and to herself about success in mathematics, and we lose entire generations of people when the gate is closed to them mainly because of perception.

Rebecca: So much discussion of gates it should be important to note that in front of Marcia is this picture of so many different kinds of gates in our conversation. Can you talk a little bit about the gates that you have in front of you?

Marcia: Yeah, I decided to Google different kinds of gates and when you think about the Brandenburg Gate or you think about gates like this one —remind me what this is called; this is in Cincinnati—the arch; this is really a gate, but this shows an opening to something, so when you think about gatekeeping in mathematics, I want us to think about people being gatekeepers for accessibility. So when you look at those pictures you think of when you’re going through the turnstile to pay with an EZ Pass. That is a barrier. If you don’t have money, you don’t have an EZ Pass, you’re not getting through, but if you look at the door to no return like in Benin it’s an opening to the next world just like certain pictures of gates just have you think differently about openings and closings.

Rebecca: There’s some like the dog pen where there is no way in or out; it looks like that one’s just closed forever.

Marcia: Yeah, which one is it? This one or this one? Right, I mean this has a gate, but often people are closed inside of thinking that they can’t do math and they can’t be successful. The job of a teacher would be to help them jump over that particular gate or find a different way to think about opening that particular gate. If you’re a dog and you’re inside of a pen, I think you’re just gonna need somebody to lift you up over that gate, and I think about that with teachers that what they have to do with each individual student is completely different, but their responsibility is to help them understand that they’re all mathematicians and they all have capacity for success in mathematics.

Rebecca: You’ve talked a little bit about how gatekeepers can open the gate or provide the leg up over the wall, or whatever it is, right, that’s there. Can you talk a little bit more about how to be inclusive and how faculty and teachers can really support this environment that would allow for problem-solving and allowing students to fail and try again and to iterate and eventually succeed?

Marcia: I’ve thought a lot about elementary school students and middle school students, where you’ve probably heard about the Montessori Method. The Montessori Method, you work with individuals to build from what their interests are and it turns out that students without a lot of direct instruction can complete whatever the curriculum is for that grade level by mapping to their interests, their strengths, and projects that they do where they’re learning the mathematics in ways that might be considered non-traditional. In the Montessori Method, they’re not just looking at memorizing times tables; they’re looking at multiplication as repeated addition, they’re looking at visualizations instead of just looking at a text. And the fact is that sometimes, I think, that if we allowed students to individualize their learning, especially in middle school and high school, that there’d be more progress than forcing students through the curriculum where each week they’re expected to learn something but they’re not learning it, they’re sort of just being dragged through the mud. And I have a lot of respect for my peers who are math teachers. I was a math teacher where I felt like I know what that kid needs, I need to take time to help that kid through what they need, but I didn’t have the courage to stop what I was doing and figure out how to individualize or make them work in small groups. I was a successful K-12 teacher, but I feel like I started to figure out what was needed when I made the decision to leave. So, part of my job as a math educator is to help our candidates who are gonna be teachers in schools to have the courage to do what they know is right: think about their love of mathematics and give kids problems that are theoretical and have them try it; give them applied problems, give them things where they have to use visualizations and not just know the procedures, but also understand the concept.

John: And also perhaps to use peer instruction, as you talked about, where students explaining things to each other reinforces learning for each student.

Marcia: Yeah, and sometimes the things that we expect of what we call the gifted and talented are exactly the things that other students can do but we’re afraid to take a risk, and I met earlier this afternoon with one of our adjuncts that’s teaching math methods to our graduate students and she said her job is to teach her candidates how to be good teachers, and sometimes that means forgoing what they think they wanted accomplished on that day and building something fun that’s gonna get students to see that math has many openings, not just following things through rote or through memorization. So, I had a really nice conversation with her because she does work in the school systems, but she’s teaching a course for us and she uses constructivist approaches. I have many peers that are still engaged in this math war that it has to be rote, it has to be step-by-step. In the constructivist approach, you care more about the process that students engage in and there’s a program that I listen to on Sunday morning it’s on NPR where it’s a puzzle and the puzzle is usually related to a vocabulary puzzle as opposed to a math puzzle, but the type of thinking that you have to engage to solve those puzzles really is mathematical thinking, so I love those puzzles, but they’re all couched in word puzzles… but it’s really mathematical thinking… and so I think the teachers need to use more of those word puzzles to bring people in so that they understand that they’re engaged in mathematical thinking—it’s just not called mathematical thinking. One of the other things that I wanted to mention before I run out of time is we are heavily tracking students into particular tracks. Sometimes you’re in the track where you’re just going to do Algebra I, and sometimes you’re in the track where you’re gonna get to do Algebra I and Algebra II, and maybe you’ll get to do Geometry, but some of the best learning occurs when there’s heterogeneous grouping and there’s less tracking. This gate stuff, these gatekeeping, really reinforces tracking, which when students come to SUNY Oswego and they’re in a remedial class and don’t know why they’re in the remedial class, because they may have been tracked in a particular way and cut off many, many job opportunities or majors because they were tracked in a particular way, and that is gatekeeping that occurs in fourth grade. And again our responsibility for our childhood educators is to get kids to think more broadly about what mathematics is; it’s not just arithmetic, it’s not just geometry, it’s not just theoretical problems; there are many types of problems that childhood people could engage students in that wouldn’t shut the door to possibilities 10 or 12 years later when students find out that they were tracked in a way that makes it so that they could never do graphic design or they could never do engineering or something else that they didn’t really understand was possible because somebody closed the gate early on.

John: …and that’s really important because most of the growth in income inequality is due to differences in educational attainment and the returns to education. And the returns to education in the STEM fields is far above the returns in other areas as well. So, keeping people out of those areas means that the people with those areas end up doing really well, but the people without those skills end up in jobs that are perhaps overcrowded with lower job prospects, lower prospects of growth and it helps reduce social mobility and economic mobility. It’s a serious problem in our society; it’s the worst we’ve ever seen it in the U.S.

Marcia: Yeah, I can’t connect it completely to perceptions, but a long time ago I taught a remedial math course at Clinton Community College and I had a student in that class and she was a smart person—I think everyone is smart—but I walked through how to study math, how to approach it: you are capable, work hard, keep asking questions, and about 10 years later I got a postcard from her—this flabbergasted me; she was in a remedial class and she had entered a PhD program in mathematics and she said it was just about the fact that somebody finally showed her how to study math—it was read the textbook, try the problems, come to class, listen maybe to the lecture, don’t be afraid to make mistakes; when you’re tired take a break. There are certain things We know that people can be successful in mathematics but we keep thinking that it’s this magic wand thing; it’s not a magic wand thing. We actually know —there’s research from Adding it Up —where we know exactly how people learn math well. The stuff research from Bransford, which how to study mathematics, how to learn mathematics, it’s written in black and white from large-scale studies, but then we return to the rote memorization, follow these steps and that’s not the beauty of mathematics at all.

Rebecca: One of the things that I think is really interesting about what you’re saying is that societally we might think, “Oh, fourth grade teacher… not really gonna have a big impact,” but you’re really talking about this fourth grade teacher is not a gatekeeper of the little gate around the garden; this is like the gate to the universe.

Marcia: Absolutely, and most of our math candidates who are not math concentrates—they’ve got to take these two four-credit math courses—will say, “I just need to get through this class; I hate math.” If you hate math it comes through loud and clear in your teaching; it’s really difficult to mask that. I taught a math for diverse learners course that the School of Education and Arts and Sciences Math Department and Curriculum and Instruction collaborated on and it was a math for diverse learner, so some of the things that I’ve been talking about here was in a full graduate course, and students would say, “Well, I never really thought about that; I thought everybody was gonna learn math the same way I learned math”—you’re a math person, I shouldn’t even say that. You’re a math person—you came through the system and you were successful in the current system, but if you want to build the next generation you’ve got to think about some of these other factors—you’re gonna be in a system, and as we’ve talked about systems, you are part of the system and you do have power to make changes to it, even if it’s perceptions, even if it’s just giving students the perception that you care about their learning and that they can succeed, and so this is really important to me. There are three principles: teachers must engage student misconceptions, understanding requires factual knowledge and conceptual understanding, and a metacognitive approach enables students self-monitoring. If I think about gatekeeping, if teachers kept those three principles in mind, they’re not mine—it’s in the research. This is sort of revolutionary because we don’t want to restrict people to thinking that only certain people can do mathematics, but if math teachers, whether they’re childhood or adolescence, or university teachers think about what good mathematicians do, they’ll follow these three principles and it might move us forward. I know it’s a big deal because the successful people want to keep what they have to themselves, but I think we miss out on the potential of entire generations if we don’t give them access to opening the gate through mathematics. When the Common Core came out teachers had the perception that they had to give these problems to students and parents would call and complain—“I can’t even do these problems; these aren’t the problems that I did when I was a kid”—well, the fact is we weren’t supposed to be sending these problems home; we were supposed to be doing those problems in class, and so a lot of the Common Core mathematics was supposed to be using manipulatives and getting kids to talk about how they think through the arithmetic problem. They were sending home problems and parents were complaining they were spending two or three hours to work through these problems, and there was an article put out—it was an NCTM—where they said what is your problem? No, don’t send these problems home for kids to fight with their parents, ‘cause that’s just gonna reinforce, “Oh, I couldn’t do math either;” it was supposed to be completely done in the classroom in collaborative groups, but we’re still not interested in teaching in that way. So, we sent home the homework—well, you could have been sending home memorize these timetables just as we did 20 years ago or 30 years ago, so finally NCTM put something out to help math teachers in the K-12 area not to send home these problems that would take parents two to three hours, but to rethink the organization of their classrooms where students could work on problems and have fun with mathematics, and the fact is that there are reforms that mathematicians fight about; there are a whole host of mathematicians that said Common Core was bad; Common Core is not bad, the way it was implemented was bad, so now we’ve done some backtracking to think about the fact that when you carry, when you’re subtracting or you’re adding, why do we do that? And the Common Core got students to make sense out of place value and make sense out of what it means when we carry this is about the tens place or the hundreds place and whenever you have new curriculum, Common Core or what was the curriculum in the ‘50s, I can’t remember… the new math… there’s always new math, it’s just an approach to make it more inclusive, but sometimes the way we roll things out makes it difficult, at least for the next generation of teachers, so I’m pro-reform movements, but we have to take the time and the energy to implement it in a way that’s actually gonna be useful—we just keep going back to the way we taught math a hundred years ago.

Rebecca: It sounds like what happened was faculty who knew how to do things a particular way get handed something that’s different but not a way of demonstrating or doing the different, right, like…

John: …without the professional development needed to allow them to implement it effectively.

Marcia: Correct. That’s correct.

Rebecca: The method doesn’t match the material.

Marcia: Exactly. At the same time they were putting out that students have to take a main assessment in fourth grade and eighth grade, but those assessments didn’t really align to this new Common Core curriculum, and so lots of things have changed over the last, I’d say seven to ten years, and we’re sort of coming out of that. When students come to the university level we still expect them to know mathematics. Do you remember twenty-five years ago they changed the math curriculum to be Math A and B, Course I, II, and III? New York state was the only state that was really thinking more globally about, “Wow, it doesn’t always have to be about algebra—it could be about statistics, it can be about more applied,” but the fact is universities didn’t change and we were still expecting students to know this narrow curriculum but it did broaden what people thought about mathematics, but it didn’t really help a lot of those students because then they were closed out of particular career areas because they might have been in a school that embraced applied math or embraced business math or something that might not connect to what they would do at the university level.

John: You’ve also been involved with Project Smart here at Oswego. Could you you tell us a little bit about that and how it relates to math instruction.

Marcia: Project Smart was a thirty-year project where teachers came to SUNY Oswego for summers to do professional development, math, science, technology. There are some teachers retiring over the last couple of years that came to Project Smart right from the beginning. We brought people in like Damian Schofield in the early days to learn about human-computer interaction. We brought people in from music and from art to help teachers integrate other things into their teaching, so they used to come for three weeks, then they came for two weeks, then they came for one week, then we built it into the department where faculty got released time to go into schools and work with teachers from the bottom up to think about how to improve teaching in their classroom. Project Smart really honored the work that teachers did because we would say, “What do you want to improve in your classroom? Are there particular things that you know students are struggling with?” This past year, funding for Project Smart ended, but the institution is still supporting individual faculty to go into schools and work with teachers to build classrooms that connect with the learners that they have in front of them. It’s more connected to what’s called a professional development school, where at the university we have the latest about how to teach, whether it’s math or English or social studies or modern language, and then we go into schools where they’re dealing with kids every single day and we try to help them figure out how to improve as a teacher; we meet them where they are; we build from there, so Project Smart is over—I’m not gonna say it’s dead, but we have a different system to work on professional development schools but just in a different way.

John: So you’re still doing the same thing even though it’s not under that official title?

Marcia: Correct. Correct.

Rebecca: We always wrap up our episodes by asking, what next?

Marcia: Oh my goodness, thank you for asking what next. After returning from my sabbatical, where I had the opportunity to be part of Budapest Semesters in Math Education where I got to see classrooms where students were using Pólya’s problem-solving approach in addition to something called the Pósa method, I worked with Josh McKeown, who’s from international education to reduce the cost of the Budapest program, so we’re working to recruit math students, both childhood and adolescence teacher candidates, as well as straight math candidates to consider going to Budapest over a winter course for one or two weeks over winter session or during spring break. What would they experience if they went to a short course? They would visit classrooms using the Pósa method, they would sit in on some of the math courses at BSME, where teachers are actually showing how to use a problem-solving approach in mathematics, where sometimes our students say “You talk about problem solving, you talk about the constructive approach, but no one is doing it so we don’t really know what it is.” The next step is to work with international ed to get a group of students to do the BSME program.

Rebecca: That’s really incredible.

Marcia: I’m excited about it too and I hope to also re-institute my math for diverse learners course because through that course I reinforce that I believe students should have access to high-quality, engaging math instruction. I believe all students should have mathematically rich curriculum. I believe all students should have high expectations and strong support, and we’re all gatekeepers— we are change agents and we control the gate. I think it’s ambitious because many people don’t agree with me saying that mathematics needs to be more inclusive, but that’s what I’ve been working for my entire career and I hope to continue that way.

Rebecca: Your work is incredible and we’re really excited that you’re doing that work.

Marcia: Thank you.

Rebecca: I know as someone who’s in a field that you don’t always associate with math—I believe in math and so I hope we can all help support your initiative.

John: It’s a major social justice issue.

Marcia: It’s a huge social justice issue because, again, what happens is often students of color, students that come from poor families may or may not have had the best math instruction. I mean, it’s a big cycle, and when they come here we should be able to help not just convince them, but this is a public institution. We should be able to provide access for them to reach whatever goals they hope to. We should be able to take students where they are and help them achieve whatever their focus is, whether it’s math related or not.

John: Well, thank you.

Rebecca: Thank you so much.
[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

52. Metaliteracy

Do your students create digital media in your courses or just consume it? Does the concept of information literacy seem too limited in this context? In this episode, Tom Mackey (Professor in the Department of Arts and Media at Empire State College) and Trudi Jacobson (Head of the Information Literacy Department and Distinguished Librarian at the State University of New York at Albany) join us to discuss metaliteracy as a framework for improving critical thinking and metacognition while students become active participants in the construction of knowledge in online communities.

Show Notes

Transcript

Rebecca: Do your students create digital media in your courses or just consume it? Does the concept of information literacy seem too limited in this context? In this episode we discuss metaliteracy as a framework for improving critical thinking and metacognition while students become active participants in the construction of knowledge in online communities.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guests today are Tom Mackey, Professor in the Department of Arts and Media at Empire State College and Trudi Jacobson, the Head of the Information Literacy Department and Distinguished Librarian at the State University of New York at Albany. In fact, she is currently the only Distinguished Librarian in the SUNY system. Welcome, Tom and Trudi.

Trudi: Thank you.

Tom: Thank you. Very happy to be here. Thanks for the invitation.

Rebecca: Today our teas are…

Trudi: I am drinking highland blend.

Tom: I’m drinking sweet cinnamon spice.

John: And I am drinking chocolate mint tea, a Harney and Sons blend.

Rebecca: I’m back to my good ole English afternoon.

John: Such a surprise.

Rebecca: Sometimes you just need to have the dependable option.

John: Both of you have written very extensively and done a lot of research and workshops on metaliteracy, with three books, three MOOCs (with a fourth one under development), several articles, a badging system and the metaliteracy.org blog. Could you tell our listeners a bit about what metaliteracy is?

Tom: Sure. Thanks, John, I’ll start. Metaliteracy is a pedagogical framework that empowers learners to be active producers of information in collaborative environments. So that’s our elevator speech right there in terms of what it is. Basically, it is an approach to teaching and learning that prepares individuals to be reflective learners in addition to being critical thinkers, and we’ll talk a bit about how that reflection piece is especially critical for a metaliteracy, which, of course, applies metacognition. By doing so, learners are informed consumers of information, which means they ask good questions about the information they encounter in a variety of environments, and as you know, that’s important today with all the different environments and social media environments and access to different kinds of new sources that we have; it includes those especially mediated by technology. And we’ll talk as the idea was first introduced and developed why that was so important to the concept. When we first introduced it we really argued that because of the emergence of social media, online communities especially, think about web 2.0 and the change from the original web—what a critical moment that was—that what we really needed was a metaliteracy that promoted effective participation in these environments. As we know, these collaborative social environments have an engagement piece that is important; we build that into this metaliteracy framework; we thought there was a real need for that—how we were developing it. We also acknowledge that in addition to acquiring information and looking critically at information that individuals really needed to creatively create and share information in this connected network world. This idea of consuming information versus producing information, it’s an idea that’s been around for some time, but we really thought it was critical to develop it into a metaliteracy that also focused on reflection as a core concept. The idea of a metaliteracy is that we look at some of the common characteristics that unite different forms of literacy—that was the other piece of this. We introduced it as this comprehensive, unifying framework. The idea for that was that in this social media environment what we really needed was to try to better understand different competencies, different characteristics of literacies instead of just coming up with a new literacy every time there’s a new technology. We were trying to look at things in a more comprehensive way. As the idea developed in the first book, especially the meta in metaliteracy, intentionally invokes this idea of metacognition. Or thinking about your own thinking: this is really key to metaliteracy because metaliterate learners are reflective about their own learning experiences and they really take charge of their literacy and learning which is really where the empowerment piece comes in.

Trudi: Meta derived from the Greek… also means “after.” Metaliteracy is what happens after literacy. Basic reading and writing, what comes after that. Also what comes after information literacy, which is g enerally thought of as finding and locating information. The definition of information literacy has expanded since we started work on metaliteracy. In addition to reflecting on their own thinking, the metacognitive aspect of metaliteracy also means that individuals have the capacity to self-regulate their own learning, which means that they identify their own strengths and weaknesses and play a role in preparing themselves to adapt to new learning situations. Metaliteracy prepares learners to adapt new technology and to do so in a critical way, that is asking questions about how technologies are designed and the ways that technologies or platforms may impact how we access and create information as well as how we communicate with information. Originally we developed metaliteracy to emphasize how individuals participate in social media environments. And Tom, would you like to talk a little bit about that?

Tom: That piece is really essential to what we’re doing. We see this framework is relevant to a range of collaborative teaching and learning situations, but it is interesting that we saw a real need for emphasizing the social media aspect, online communities, this post web 2.0 environment that we are in, but we also don’t want it to be limited to that. We really see metaliteracy in all environments, all collaborative environments… communities of practice. This is something we should be thinking about beyond just the technology, but really how we engage with each other, how we participate and perhaps also how we blend the technology, how we mediate technology with those spaces as well.

Trudi: One of my favorite parts of metaliteracy is that it advances the idea that learners are teachers. We see this in collaborative environments where learners support and teach other learners, but what’s really important is that often students, for example, don’t think they have any particular expertise in something, and encouraging them to empower to teach others often leads to really interesting situations.

 Tom: That part is so key and that’s something that we saw in our own teaching experiences that when we had students in collaborative situations… group work… building technology tools together… building collaborative websites, for example, that the students themselves were as much a teacher as I was, and trying to foreground that so that they can see it, is critically important.

Rebecca: This is a really interesting framework and you’ve given us a lot to think about. Can you help us make it a little more concrete by providing an example of how you might emphasize metaliteracy in a class or what you mean by a student who might be metaliterate?

Trudi: One of the things that I would do in my classes is encourage students to be information creators and to explore the technology in doing so. So they don’t have a final paper that they have to write, but they may need to create a video or a tutorial or we’ll be talking about our badging system later, maybe creating content for that and doing it in small groups. If they’re doing something where they have to use a technology; I don’t teach them that technology; they sort of learn together and that “learner as teacher” often comes out in those situations because often there’ll be a student in a group who will have more expertise in that area or be more willing to just jump in and see what happens, and then the rest of the group will learn from that. One of the more interesting teams that I had when I’ve taught is one where none of the students felt they could do anything, but they actually accomplished it and their sense of pride and empowerment in doing that was wonderful.

Tom: I have an example: I’m currently teaching a course at Empire State College called “Digital Storytelling,” and the whole point of the course is that students learn about these resources, they locate them (with some prompts from me in the course), but it’s a fully online course and in many ways they have to figure this out on their own, they have to adapt to these new technologies, and I think that they’re looking at their own use of technology in a different way. So, for example, the very first assignment they have to create a selfie video with their cell phone. So they all have cell phones, they probably all done videos before, they probably all done selfies before, but this assignment is really designed for them to introduce themselves to everyone else in the class in a fully online course. From the very beginning they have to challenge themselves to present themselves a certain way to the class… to be themselves but to also think through that presentation, to really be the active producer of information in a collaborative setting where they’re doing something on their own but they’re sharing something about themselves to the other class. In an online course it allows us to get beyond just the text-based introduction and online discussion and to really seeing the students, to hearing from them. I posted a video of myself and it was great to see their response, so it was very much like a classroom situation but it happened asynchronously and online and it was a great way to get the class started, so from the very beginning they saw themselves as digital storytellers and they know that they now are starting their story and that we’re all going to participate and learn from them.

John: So it’s encouraging students not just to critically analyze information as consumers but to be active participants in social dialogues as producers as well. Is that a reasonable short summary?

Trudi: Yeah.

Tom: Absolutely. And what does that mean? …especially in today’s environment, which is very participatory but were divided and partisan in so many different ways. How do we get across those divides? What does it mean to be a responsible participant of information now? What does it mean to be an ethical contributor to these spaces? The whole idea is to really to get them to reflect on this, and not just to produce and share something, but now especially to think about the implications of that so that the informed consumer part is still important so that they’re thinking about these different sources that they’re encountering but also thinking about what they’re creating themselves and sharing.

Trudi: I think when they’re asked to be information producers in this way they think about themselves differently. They create information and share it on social media, but they don’t really think of themselves as information producers, and so I think it expands their horizon.

Tom: They may not have necessarily been asked to do so in an academic environment. This blurring of boundaries between informal learning and formal learning, I think it helps to push that a little bit. Not to say that they’re not beyond our classes, because they might be, but clearly they’re doing it in their everyday practice with their cell phones and the way they consume information now, but this really foregrounds, I think, in some of what the responsibilities are and what the empowerment of that is as well when they’re asked to construct something, so instead of a research paper maybe that is a collaborative media project with their peers—what kind of learning do you gain from that experience?

Trudi: Just one other point. The projects that I was talking about, they need to create them for public consumption. It’s not something that’s just directed at me as the professor of the course. They have to think about it a bit differently.

Tom: That’s a great point, because in the digital storytelling class they’re not just creating it even for the Moodle environment that we’re in; they have to actually upload their selfie videos to YouTube so that they’re thinking a bit about that public consumption piece even beyond the instructor and even beyond the class itself because now it’s up on YouTube and hopefully that’s having an impact on what they’re thinking about in terms of how they present themselves in the information that they’re producing.

Rebecca: I’m hearing two key things bubble up in what you’re talking about and one is audience and the second is reflection. Are those two key things to move up beyond traditional information literacy to this metaliteracy level?

Trudi: I think that those are two key pieces, but I think, well, there’s the old definition of information literacy and then there’s the newer one, which somewhat influenced by metaliteracy, but I think that often information literacy is thought of primarily as consuming and evaluating information, so not the responsible, creative production of it. It’s also too often, I think, seen in the academic setting as just related to research and not sort of life-wide. I think that that’s another element here.

Tom: In many ways that’s what I think we were really originally working against that original information literacy definition, the ALA definition and also the Association of College and Research Libraries, the original standards, b       which were very prescriptive in the way that they were designed, so that we were as a framework were really just trying to open this up and also take into account the technology piece—not make it all about technology, certainly, but in many ways the advance of web 2.0 and emerging technologies was kind of being, at the time, anyway, sort of avoided. We knew that there’s a real shift happening in our culture and I think that we’re sort of on the other side of that now, but I think that was important to bring that into the learning experience to have students really reflect on those environments and what they’re doing in those environments.

John: You both mentioned the new ACRL information literacy framework. How does metaliteracy relate to that?

Trudi: We developed metaliteracy in part because of a frustration, with this old definition as we were talking about and Tom mentioned the standards really were very prescriptive, very skills based, concentrated on behavioral and cognitive learning domains. Metacognition was not a part of it, so you identified metacognition so that reflection as something new and they didn’t explicitly address the affordances of web 2.0. So I was co-chair of the task force that was convened by the Association of College and Research Libraries and I brought the idea of metaliteracy to the group for consideration. There were a lot of forces at work in developing the structure of the framework and there were like 2000 people weighing in so it’s a very interesting process. Threshold concepts or core concepts was one of the primary features that we used with the framework. I sort of quote from the introduction to the frameworks; there are those ideas in any discipline that are passageways or portals to enlarged understanding or ways of thinking and practicing within that discipline. For example, in biology, evolution would be a threshold concept. That was one element and then the other was metaliteracy. The idea of learners as information creators as well as consumers—which we’ve talked about—definitely has a presence in the framework. There are four learning domains in metaliteracies: behavioral, cognitive, affective and metacognitive. These all have made their way into the framework, so there really is in part a close relationship between the two. For example, the affective domain maps to the whole sections on learner dispositions. I think that there really is a close relationship and I think metaliteracy has gotten additional notice from people because it is explicitly mentioned in the framework.

John: So it’s complementary that they fit well together, they link well together.

Trudi: That’s right.

Tom: I think that’s a good way to put it that they’re complementary, because that also allows each approach to still move forward because we see metaliteracy as this evolving concept and we’ve been working together—we’re working with a team of colleagues called the “Metaliteracy Learning Collaborative” on these ideas, we’re writing together and we’re developing this different MOOC and badging projects. Every time we do something new we’re learning something new and we’re trying to build that into the core ideas here. I think that this idea of complementarity is really important to these two; they’re not mutually exclusive, they work together, and as Trudi mentioned, when we go out and talk to different audiences on this they’re interested in both concepts and working with both. One interesting comment we often hear from people is that with metaliteracy they’ll say you found a way to talk about something that we were trying to do or that we were already doing but you found a name for that really made sense. We really like that: the fact that we were able to name something that really probably was in practice but maybe didn’t have as in-depth of a framework built around it and we like that dialogue with practitioners and something we try to do so this idea of theory and practice for metaliteracy is critically important and allows you to move forward.

Trudi: And the ACRL information literacy framework information literacy is not something that can be taught only by librarians so it’s really directed also towards faculty and administrators. It still seems to have a librarian focus to it, whereas metaliteracy, I think, extends beyond that. Librarians are interested in it but we’re also seeing all sorts of things that are being written or talked about by academics in a really broad range of disciplines.

Tom: And we’ve found that in the books we’ll talk about the two unedited books we’ve done in addition to the first metaliteracy book and we saw evidence of that when we do a call for proposal; it’s really from a wide area of academics. We definitely have librarians, but we also have faculty from many different disciplines, and also instructional designers. That piece of it has been really fascinating as well because we’ve been trying to really open it up to as many people as possible and not seeing it just within one particular discipline.

John: How have faculty and librarians responded to your work?

Trudi: There’s been a lot of interest in it to explore one of the collaborations. Somebody that I’m working with at the University at Albany is a political science professor. This will give you an indication how at least one person has responded to our work. She teaches a 200 level political science course that includes some of the general education competencies, one of which is information literacy, and she was developing this course from Pollock. She came to me to talk about information literacy. We ended up talking about metaliteracy and she was so excited by some of the things we’ve talked about that it would do for her students, so this idea of information creators, the empowerment that she has made metaliteracy sort of a key part of her course. She has the students do about 8 activities connected to metaliteracy. These activities come from a digital badging system that we can talk about a little bit later. She actually has students create an activity that would fit into this digital badging system, which is pretty exciting. This year she asked us to extend what we’re doing and we have been creating questions for the students about what it means to be an information creator, information producer, a teacher, a translator of information and we found this very exciting. It’s not just a collaboration in that she is using some of this material for her students, but her students are creating things for us and she’s giving us ideas. It’s just one example but it’s one where it has become a core part of this course, not only when she teaches it but when others teach it as well.

Tom: Collaboration has been key to what we’ve been doing from the very beginning. The first SUNY IITG we received was really to initiate to launch a metaliteracy learning collaborative and that first project led to the development of our first connectivist MOOC… b eginnings of the digital badging system, although it wasn’t part of the initial grant, but that’s something that we started working on, and also what was most important at the time was the development of the first metaliteracy goals and learning objectives which we’ve recently revised but it was important when we developed that that instead of just Trudi and I working on this together, we really opened it to a SUNY-wide audience that included faculty and librarians. Those goals and learning objectives are available via metaliteracy.org and we recently revised them as well. I think that collaboration with the metaliteracy learning collaborative also led to thinking about metaliteracy in a different way and thinking about those four domains of learning that Trudi mentioned previously; we would look at the metacognitive, which we’ve mentioned is key but also the behavioral, the cognitive and the affective domain so that what we’re really looking at is really the whole person. We’ve also through the metaliteracy learning collaborative we’ve been working on papers together, we’ve been working on these MOOCs; we were lucky enough to have the experience of working on a connectivist MOOC really early on and then I took Coursera MOOC and then a Canvas MOOC and now we’re working on open edX and all those projects involve faculty librarians from Empire State College, the University of Albany and other parts of SUNY, that’s really key. We’re very lucky that we’ve been invited to speak on this which also shows the level of interest and how people are responding to it and many different venues and last year we were lucky enough to present at a conference at the University of Guadalajara in Mexico for this literacy and learning conference and it was just a great experience to be there with international scholars who were talking about literacy in various ways and then we added something by talking about metaliteracy and there’s a lot of interest in what we were talking about. We appreciate those opportunities to have conversations that are both theoretical and practical; the response has just been really positive.

John: We should just clarify the IITG program you mentioned is a SUNY-wide competitive grant program for all of the colleges and universities within SUNY. You were one of the early recipients of that and have received some further funding from that, just to explain that to our listeners who are not as familiar with the SUNY system.

Trudi: John, since you mentioned the innovative instruction technology grant, just to show sort of interest from others, we did get one with School of Education faculty member, actually one from Albany and one from Empire State College because they were really interested in the digital badging, but also the idea of a digital citizen. The plan was and happened that graduate students in education who were going to be teachers would have an opportunity to learn about digital citizenship that’s important for them when they’re teaching, also what digital badging is, so there were a couple of different takeaways. We were able to move metaliteracy or an aspect of metaliteracy into graduate education for educators.

Rebecca: There’s been a lot of mention of metaliteracy badges so maybe we can talk about those?

[LAUGHTER]

Trudi: Yeah, certainly. This was something that developed out of one of those innovative instruction technology grants. We’ve been working on them ever since. What we did was we took the learning goals and objectives for metaliteracy and created open content, very ambitious scheme. There’s four digital badges in the system. Each one of which has anywhere from 12 to 20 activities, starting with lower level quests, moving up to challenges and ultimately you get to these four digital badges. They were written by members of the meta literacy learning collaborative. Tom has written some, I’ve written some. Students have written some, so undergraduate and graduate students they’re being used currently at Albany about 2,500 students have gone through parts of this badging system. The only ones so far who’ve actually earned badges are ones who have taken my courses. It’s content that can be used in classes across a range of disciplines. Also, adaptable to the disciplines. I mentioned earlier the political science professor and sometimes she sort of tweaks the assignments in there so it really relates to what she’s teaching in her political science course. The badge system itself at this point is restricted to University of Albany because there’s a single sign-on process, but we do have a website that has all of the content openly available. People are welcome to use this.

Tom: And from the perspective of someone who has developed some content for this it’s really a fascinating experience because you know that you’re somehow reaching learners that are not in your course but that it’s something that you’re opening and you’re sharing, so this idea of thinking about them as open educational resources that can be then adapted for different contexts. It’s really interesting and exciting to know that something I might create as a learning object could be used by a faculty member here at the University at Albany who’s having their students go through it. Some of them that I developed are based on learning activities I had created in some of my information science courses when I taught here at the university, but I’ve adapted them or updated them. That piece of it from a faculty perspective, as long as you’re open to it, is really engaging and interesting and a way to reach other learners who may not be students in your class but you’re sharing those ideas with them.

Trudi: And I don’t know if it’s ok if I plug a book that I just co-edited with Kelsey O’Brien… Just published this month, September 2018, Teaching with Digital Badges, which was published by Rowman & Littlefield. In that book there is a chapter written by Kelsey O’Brien on the metaliteracy badging system.

Rebecca: Great, you’re both working on a new book together, right?

Trudi: Yes.

Tom: Yes.

Rebecca: Can you talk a little bit about that new book and how it connects to your earlier work?

Tom: Sure, the new book is called “Metaliterate Learning for the Post Truth World.” We’ve shifted somewhat from I think what was a really optimistic view of the connected world and how great it is to be producers of information and be participatory to really trying to further emphasize some pieces that were there but I think needed to be fleshed out a bit more for the new environment we’re in post truth, which is based on confirmation bias and misinformation, false information and questions about new sources and all kinds of misleading facts that are being sent out. We really wanted to take that head-on because we saw metaliteracy in many ways even though it’s an idea that had developed previously as something that is a strong education response to some of the concerns and issues that we’re seeing today. Soon after the 2016 election we wrote a piece about fake news and that term is certainly changed even from the time that we originally wrote it. Wrote a piece for the conversation called “How to Reject Fake News in a Digital World,” so again taking a metaliteracy approach to looking at fake news in a critical way. Since that time even the term fake news, of course, has been weaponized, so we have conflicting thoughts about even using that term based on the research some educators think that it’s important to still keep using it and others want to reject it completely but I think we all generally know the narrative of that. The new book we decided to foreground metaliteracy in this environment and to make it an edited book so that we could engage other educators about this idea. Wasn’t just us but that it was other educators who were dealing with it. About half of the book is very theoretical and the other half of the book is more practical. When we did a call for proposals we tried to intentionally keep that open because we wanted different perspectives on this. I wrote the framing chapter to really talk about post truth, to reframe metaliteracy within this context and to also talk about a new figure that Trudi and I developed together based on the metaliteracy learning characteristics. The new book is going to present a new image, a new figure that further develops the metaliteracy idea from a theoretical perspective and talk about the importance of those characteristics in the post truth world. We’re joined by incredibly prestigious authors who from a theoretical standpoint look at things such as the importance of documentation in metaliteracy, and again, what they’re doing is they’re flushing out pieces of metaliteracy that we have not engaged with yet, so it was really exciting to see that. Another author talks about inoculation theory preparing learners to in many ways be resistant to some of the post truth issues that we’re currently engaged in. Scientific literacy, so there’s a whole chapter on the importance of scientific literacy and looking at it through the lens of metaliteracy. Also, looking at the synergy of word and image and photojournalism, Tom Palmer who teaches here in the journalism program at the University of Albany and it was also a journalist who works for the Times Union wrote that chapter. A few of the chapters do deal with the ACRL framework for information literacy for higher education, so we had that perspective. We were talking previously about both concepts are complementary and we have a few authors who really prove that. We also have a few authors who look at such topics as teaching students to be wrong, genre writing in the first year, writing instruction and the application of poetic ethnography in digital storytelling to create narratives in Philadelphia neighborhoods. I’m very interested in digital storytelling. I mentioned that previously and one of our authors also talks about digital storytelling to empower voices and to encourage students to really raise their voice in the current times that we’re in.

Trudi: And earlier you sort of asked how faculty, other educators, librarians have responded to metaliteracy. I think it’s really interesting. Tom and I did a workshop on metaliteracy at Temple University and a couple of these chapters actually came from people who were in that workshop. It was really sort of exciting to see the immediate impact that that had had.

Rebecca: That’s cool. So this sounds like a really great book; when can I get it?

Trudi: Next spring. [LAUGHTER]

John: Ok. Your current MOOC is a Coursera MOOC but you’re developing a new open edX MOOC. Could you tell us a little bit about that and how that new MOOC will differ from the prior MOOCs, because you’ve had more than one in the past?

Tom: This is part of a continuum of those three MOOCs. We actually wrote a paper in Open Praxis that talked about metaliteracy as a pedagogical framework that was applied in these different MOOCs, so we did a kind of compare and contrast of the different MOOC environments but also talked about our experiences and those different platforms and what it was like, and at the end of that paper one of our conclusions was that what we really needed to do next was create a kind of hybrid MOOC environment because what we had previously was the connectivist MOOC which was our first one and that Stephen Downes type approach. We actually used his grasshopper programming to run that MOOC, then we had the more structured xMOOCS, the Coursera and the Canvas. In many ways the paper was about that but what we decided at the end of the paper was we analyzed it was that we needed a hybrid version and it would it be possible to do that, is there a platform out there that has the learner-centered freeform approach of the connectivist MOOC with some of the structures that were valuable in the video that was really key to the xMOOCS. One of the ideas that propelled this idea forward… We also then, of course, had this shift to this thinking of a transition of kind of a connected world to a post truth world and what does that mean, and because we were working on this book “Metaliterate Learning for the Post Truth World,” we thought that’s a theme for a MOOC. We won’t go out there and call it the metaliteracy MOOC, but it’s a post truth MOOC that’s powered by metaliteracy that really applies the metaliteracy framework to each of the modules within the MOOC, so we’re really excited about that. We did apply for another SUNY IITG and we did receive funding for that, which allows us now to build a team—again it’s another Empire State College, University at Albany team—and we’re really excited about it, we’re developing it now, we’re exploring the open edX environment and as part of that too we’re working with the University of Buffalo because they’ve just launched an instance of open edX for their continuing education program and so they’ve already done a lot of the analysis and a lot of the footwork in terms of creating this instance of open edX on their campus, so they’re letting us experiment with what they’ve done and the idea is that our experience as one of the first two SUNY institutions beyond UV that are using open edX that we will hopefully pave the way for other SUNY faculty librarians that want to develop an open edX MOOC.

Trudi: One of the things that we’d like to do with this—Tom mentioned earlier—we’ve recently revised the metaliteracy learning goals and objectives. We are using those as the framework for this new MOOC. We would like to address issues such as confirmation bias, the role of expertise and authority in today’s environment, issues related to safety, security and personal privacy online, representations of reality in a virtual world and all the while sort of empowering participants to raise and share their voices while rebuilding communities of trust.

Rebecca: Who do you see is the audience for this particular MOOC?

Trudi: I think that we’re really hoping that it’s a very broad audience. We’ve had that, for example, with the Coursera MOOC where there were a lot of international participants everywhere from high school students to non-traditional types of students. We learned about their professions which just ran the gamut and I think that although we do hope to introduce this MOOC as part of courses both at Empire State College and at the University at Albany we’re really hoping that the participants are traditional learners and non-traditional learners. I think that what we’re going to be including in the way of content is something that needs to be broadly disseminated.

Tom: I think because that’s one of the advantages of MOOCs is that they do open up a potentially global audience, so we’re hoping for that international perspective as well, and as Trudi mentioned, we are developing courses so that we could on each of our campuses—I’m calling them wraparound courses—so that the courses that introduce students to the MOOC and they can then earn credit for doing so, because that’s been one of the big questions about MOOCs; can you learn credit, so what we’re doing is creating separate courses and in my version of the course I’m doing a full semester course so that the first half of the course will be introducing students to, well, what is a MOOC? What is post truth? What is metaliteracy? And I have a whole section on how to prepare for success in taking a MOOC, and then that will hopefully prepare them to be a successful learner in a MOOC environment so then they’ll take the six-week course and then there will be reflection piece at the end, which is very metaliteracy, and I actually think that a course about a course is very meta, so we’ve got that piece of it, and that idea to emerge from our very first connectivist experience where we tried to do it for credit and sure, you can talk about this experience at the University at Albany. In particular, in many ways the students were not prepared for the connectivist environment, so what we’re trying to do is in mind, since mine will be a full semester course, is invite students to take it but to really prepare them for being successful in MOOC because we know too that completion rates and MOOCs are not always great, but what if you offer it and prepare students for that environment. I think it is unique enough of an environment where that’s worth exploring.

Trudi: And Tom referred to our connectivist MOOC, which I did use as part of a course, essentially a blended course, and I was amazed when the students actually asked for more in-person class meetings because they couldn’t really grasp the idea of the MOOC and the fact that they were making decisions about their own learning. They were making decisions about which readings would be important. They needed to participate through a personal blog that was sort of elected and shared, and what they essentially did was doubt. I had about a 60% dropout rate in the course and the ones who were left were the ones who just wanted their hands held essentially through the rest of the course and that’s where we really learned that what Tom is going to be doing with his course, which is a full semester course, mine will be a quarter course again, is preparing them for this. This MOOC will be a more directed connectivist MOOC, but it was a very important takeaway.

Tom: And I’m hoping that by doing that it prepares them not only for our MOOC but it opens up the possibility of picking other MOOCs for lifelong learning. So that I think there are potential benefits, even beyond this experience. We’re hoping to launch the MOOC,—we’re developing it now—but we’re hoping to launch it for March 2019. It will be called “Empowering Yourself in a Post Truth World,” which is really important because we really want it to be a positive learning experience and one that provides resources for learners to be successful. You can imagine that talking about the post truth world could be a real downer, but what we really want it to be is a real positive focus of how to address the issues, look at these issues critically, but then to leave with some concrete ways of dealing with it. It also builds on some of the other MOOCs we had. The Coursera MOOC, for example, involved empowering yourself in a connected world and we’re running that now as an on-demand version. So when we first ran it in Coursera we were in the course and it was moving along and we were there in the discussions and following it but then Coursera changed its format a little bit and open up this possibility of on-demand and we actually like that because it allows us to have that content out there and to have learners engage with it in a self-paced way. Up to this point we’ve had, based on the stats we continue to receive from Coursera;—it’s running all the time—we’ve had 1,900 registrants and 900 active learners. We were really happy about that because it really gets some of these concepts out there, and I think it’s probably it’s been out there for a couple years now; it’s probably due for a revision, but that’s one of our projects that we’d like to do eventually, but I think that the post truth MOOC will in many ways build on that as well, so if someone wanted to go back they could look at that on-demand version, but as Trudi mentioned, the post truth MOOC is a six module, six-week learning experience on a very specific topic. I think it will be even more of a clearly-defined focused than even the other one.

John: Would be really nice to have all voters taking in the next couple of years. [LAUGHTER]

Trudi: We would like that.

Tom: Yes, yes.

Rebecca: So you’ve talked a lot about the learner side and some of the tools and materials and MOOCs and things that can help learners become more metaliterate. How do you help faculty coach students through this kind of process? What are the takeaways for faculty? They’ve listened to this episode, they’re really interested in the idea; where do they get started?

Trudi: I think not to just promote our books, but I think that perhaps if they took a look at the two edited volumes they might get a sense of how others are doing it and the range of disciplines is pretty broad, so they might find someone in their own or a related one. I think that that might be a good place to start. I think also taking a look at the learning goals and objectives might provide some ideas of things they’re already doing, but perhaps finding ways to highlight them or frame them slightly differently.

Tom: And not to promote our blog, but metaliteracy.org; everything is in there, including the goals and learning objectives. Summaries of all the books, because we’ve had the blog now for a few years, so it’s interesting even to kind of go back and look at some of the original postings, but it links to the books, it links to all the presentations. The presentations are available, and a few of the keynotes that were recorded are in there. I do think the metaliteracy goals and learning objectives are definitely key because those can be easily applied. Should we mention what we were just invited to write because that would actually address this audience as well?

Trudi: Yeah, we’re going to be writing a piece for higher education jobs. They have a couple of newsletters and going to be talking about the importance of teaching or emphasizing metaliteracy on campus for administrators and also what instructors can do. We think that those are going to be appearing in November.

Tom: Because we’ve had a commitment to making everything open—I know it’s a lot to look for, but we do have the metaliteracy YouTube channel, the blog, of course, the presentations and a lot of these resources were intentionally constructed that way so that other educators could use them, so go to “Empowering Yourself in a Connected World” on Coursera and access the videos, use the learning activities in any way you want. Go to the first module; there’s a PDF in there that has the metaliteracy learner roles and we’ve used them as learning activities in our own classes and it has some reflective questions, so you have this diagram that really explains the different roles a learner could take and then it has questions for learners to really think about those roles. So I think a lot of those resources can be adapted in any way that people want, and it’s really an open concept, so we want people to get involved and apply their own approaches to this.

Rebecca: We wrap up by always asking: what next? You’ve given us so much, but what else? [LAUGHTER]

Tom: That’s a really good question. The next book that we mentioned is coming out in the spring. We’re currently working on the open edX MOOC, “Empowering Yourself in a Post Truth World.” We also, of course, will be launching that in the spring.

Trudi: With the digital badging system we would like to if we can find some more funding have a learning pathway portion to it where instructors can really tailor the information or add components for their own disciplines. We’re also working on a metaliteracy module for another innovative instruction technology grant funded project called “I succeed,” which is being developed in western New York, and they’ve asked us to provide a module on metaliteracy and this is going to be directed to high school students who aspire to college or first year college students and can be used by instructors, so we are putting that together with four units.

Tom: We have a few upcoming panel presentations that OLC accelerate in Florida in November.

John: I may see you there.

Tom: Oh, great! I haven’t been there in a couple years so I’m looking forward to getting back and that’s such a great conference.

John: It is.

Tom: And of course there will be continued research and writing. I’m certain that the open edX experience that we’re currently immersed in will lead to a paper, and we’d like to do a research project that assesses the application of the metaliteracy goals and learning objectives. So much of what we’ve been doing is really theorizing and talking about practice and developing these environments, but we would like to delve into that a bit more. We might have an opportunity to work with an international scholar that we met last year at the University of Guadalajara, but we’re not sure about that if that’s going to happen, but that would allow us to really expand the metaliteracy concept: working with international scholars. So there’s a lot of possibilities. Perhaps a coil courses in our future, and that’s another SUNY resource; it’s a collaborative online international learning environment. I think that’s something that we would love to do with an international scholar, so we’ll see if that happens some day. A lot of ideas, got a lot going on, but we’ll see.

John: You got a nice track record of being really productive with us.

Rebecca: Thanks so much for joining us and spending time and giving us lots of things to think about.

John: Yeah, you’re doing some wonderful work.

Trudi:Thank you.

Tom: Thank you so much, we really enjoyed this.

Trudi: Yeah.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

51. Engaged scholarship

Many of us live and work in communities where there is a strong town and gown divide. Building trust, engaging authentically, and developing deep understanding through intergroup dialogue takes time, patience and the right structure. In this episode, Dr. Khuram Hussain, an Associate Professor of Education and Interim Dean at Hobart College, joins us to explore a model of engaged scholarship that challenges the academy to engage in dialogue with and work alongside the community to address pressing local issues.

Show Notes

Transcript

John:Many of us live and work in communities where there is a strong town and gown divide. Building trust, engaging authentically, and developing deep understanding through intergroup dialogue takes time, patience and the right structure. In this episode, we’ll explore a model of engaged scholarship that challenges the academy to engage in dialogue with and work alongside the community to address pressing local issues.

[Music]

John:Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John:…and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Our guest today is Dr. Khuram Hussain, an Associate Professor of Education and Interim Dean at Hobart College. Welcome, Khuram.

John:Welcome.

Khuram: Thank you for having me.

John:Our teas today are:

Khuram: I’m actually drinking coffee. I hope that’s ok.

Rebecca: You and most other people. [LAUGHTER] We’ll let it go.

Khuram: I will end the day with tea.

Rebecca: Ok, perfect. I think we had a recent guest who also ended the day with tea. Today I have chai.

John:And I have pineapple ginger green tea.

Rebecca: Well, that sounds good. You always are far more adventurous than me.

Khuram: If it’s any consolation, I have a little cardamom in my coffee, which I typically put in my tea, but I really like it in coffee as well.

Rebecca: That sounds good. I should try that.

Khuram: I highly recommend it.

Rebecca: Do you have an advice about how much?

Khuram: One. One is good.

Rebecca: One is good. [LAUGHTER].

Khuram: If you want it a little stronger you can crack it and then let it sit and it’ll be even more cardamom(y). [LAUGHTER]

Rebecca: Perfect. [LAUGHTER]

John:We see you’ve done some work with engaged scholarship and service learning. Could you tell us a little bit about what is meant by engaged scholarship for those who aren’t familiar with it?

Khuram: Engaged scholarship is essentially the integration of community needs with learning and it involves addressing community needs along with whatever respective disciplines and skills a scholar may apply to a particular condition. It could be anything from developing a literacy program that is also being useful and utilized in a community, but drawing from that community in order to make sense of what questions you want to answer. So, you’re not drawing it just from a review of literature or from a body of scholarship that emerges from conferences or a community of scholars, but in fact from a variety of voices within the community itself. It’s a much more community relevant approach to even designing research before you actually do it, and it spills out into community engaged teaching as well.

Rebecca: What got you involved in engaged scholarship?

Khuram: I first had the opportunity to do engaged scholarship as a professor of education at Hobart William Smith. I was teaching a course on the civil rights movement and a colleague approached me about volunteering to serve as a professor at a maximum-security prison, and the program there was run by a Bard Prison Initiative where long term inmates were given the opportunity to enroll in an undergraduate program. And so I taught the exact same course that I was teaching on campus within the educational space that they had created for prisoners (maximum security prison) and that was my first chance to think about the ways in which the needs and realities of communities outside of campus and inform the work in learning on campus and could also inform my notions of scholarship.

John:Your work is a form of service learning in terms of the student involvement in it. How does your approach differ from the more traditional service learning approaches?

Khuram: I think that a lot of what I have seen in traditional or conventional service-learning approaches is that there’s a great focus on the ways in which our students will learn by “doing for” communities. So how can we help children learn how to read? How can we provide food to food-scarce areas? And that becomes such a central narrative and the assumptions that young people have about what service-learning is is that we’re gonna learn through service for, and what I think is unique and special about the kind of work that many folks are doing today and I hope to be a part of that (and I hope I have been a part of that) is to do service with. To move from that model means we are required to collaborate and to take a much more team-based approach to service work and the learning then moves both ways. The service then moves both ways, and that I think is the fundamental difference between what we’ve been trying to do the last few years and what we’ve often seen provided to students.

Rebecca: How does your engaged scholarship relate to the service-learning projects and things that you do with students?

Khuram: In part, the ways in which engaged scholarship works is by providing students and faculty and community members an opportunity to create knowledge out of the questions and concerns that emerge in community related work. So for instance, we started an initiative known as “Tools for Social Change” some years ago, and before we looked at any kind of service project we looked at the ways in which the community saw itself. How did long-term residents see college campus residents? How did college campus residents in the same city see long-term residents of the city? And put them into intentional dialogue, first through interpersonal relationship building and then talking about social and structural issues that have informed their understanding of themselves within the city. And within larger structures of identity, race and class particularly. After they developed that understanding we asked, “Ok, what does this community mean to you? Where do you feel empowered? Where do you feel isolated?” Based on the answers to that, we were able to map out a different kind of geography. Even though we had developed a sense of connection and collectivity as members of a community that had been dialoguing all semester, we were operating within a city that was deeply segregated and divided, and so it was from there that we looked at scholarship. We looked at research that we could pursue, and one of the first things that became really important for us to consider was the way in which the economics of the city and the capacity of some to gain access to jobs opportunity was very different than it was for others. And so we ended up taking that initial group and developing wider groups that would go out into the city and inquire… essentially do a self-study of the city about the economics and economic opportunities that were available. And so essentially it was these two stages: first of engaging in dialogue; coming to an understanding of what shared community work could be and then going out into the city with the same participants and essentially conducting appreciative inquiry and having students and faculty and community members (long-term community members) interviewing members of the community, and we were out at the Salvation Army, we were in barbershops, we were in laundromat, we were in every corner of the city and particularly in corners of the city that didn’t often have a strong voice or were not well represented, I should say, in conversations about economic development. We were able to take those, transcribe them and give them to members of the working group that are trained qualitative researchers. They synthesized that, summarized it, and we were able to present it to the city. So, here we’ve created knowledge and we’ve created it through a certain kind of process, right? You might want to call it bottom-up, but I like to see it as horizontal; it’s relational knowledge, and that, I think, is one of the most powerful things about service-learning with as well as engaged scholarship with.

John:That group that was doing the analysis of the data… Were they faculty? Were they students? Was it some mix?

Khuram: It was some mix, but here you do have kind of a hierarchy of knowledge and skill, I should say, in terms of how to do this, and so students and community members were trained by ethnographers and researchers on how to hold a tape recorder, what kinds of questions, and how to ask questions, the ethics of confidentiality, and then they went out and they conducted (after receiving a few weeks of training) these interviews in the community and it was the researchers, mostly faculty, that then booked and analyzed that data and ultimately synthesized that data, but every turn there was some part of this that was democratic and collaborative. Even the questions themselves were questions that the participants generated in concert with other community members. What is it that we want to know about ourselves? And so those were the questions that were ultimately used when we did the broader interviews.

Rebecca: Sounds like a really powerful way of breaking down the town-gown divide that happens in a lot of communities where there’s an institution of higher education.

Khuram: I think that it was transformational for all of us. I don’t think anyone could truly have appreciated what was going to happen, and I think part of it is because it was an open conversation and we sustained a certain level of openness, curiosity, and vulnerability to each other as well as what we hope would come out of it, and I mean for me it’s transformed the way I think about everything from teaching to service to even social action and the role of institutions of higher education in really engaging in communities, and so the power of it, I think, was also to reveal what’s possible that we are capable of operating on different terms and the institutions of higher education do not need to be paternalistic in their engagement with communities and they do not need to take a charity-based approach in their supportive communities; they can be collaborative, it just requires us to match strength to strength to define the things that are going to be valuable for college students and faculty and staff to learn from communities and what communities will benefit learning with their work with institutions of higher education.

John:It strikes me too that this type of project could be much more sustainable. Many service-learning projects or one-off projects where the students work and do something in the community or to the community or for the community, but when you get the community itself engaged it swould seem that that could, at least for some types of projects, set the stage for continued collaboration, either with later groups of students working with them or with the community itself. Has there been much success in continuing the efforts once the classes ended?

Khuram: I first off want to say that I absolutely agree that service-learning is conventionally structured as a one semester project-based or hour-based experience, and it’s usually focused on alleviating one particular social issue, and what we have found is that it’s necessary to do year-long initiatives and we’ve been very fortunate to see that this initiative has been able to sustain itself for over three years, but that’s required us to allow it to evolve into what it needed to and one of the biggest parts of that has been that it has been untied from any particular course. It used to just be tied to my classes and so students would do service learning project were tied to classes they were taking with me. Now, students are participating as participants in independent studies, they’re participating in different working groups that sustain themselves a little bit more autonomously, and that is also true for a lot of long-term community residents that have joined smaller working groups. There’s a working group on food insecurity, there’s a working group on political representation, there’s a working group on economic empowerment and economic opportunity, and so any one of these working groups becomes its own kind of autonomous community that intersects with long-term residents and college students and faculty and staff and that, I think, is a sign of progress and health, is when the institution of higher ed that’s tied to these projects doesn’t need to own it, control it, and manage every aspect of it. If it can become a little bit more fluid and have its own purpose outside of a predetermined purpose from the institution, it becomes more organic and more impactful often.

Rebecca: The continuity that set up in a structure like that of “community who doesn’t go away” versus students who drop in and out as they go through four years—they’re a member of the community but then they often leave—seems like it’s a really useful model for not only making the learning better but just making the impact better. Can you talk a little bit about the community’s response to these projects.

Khuram: Yes, drive-by service-learning isn’t the way to transform communities or students; it requires a real, authentic engagement, and I think when you put people in real situations you get real outcomes and that’s across the spectrum. So you’re going to get people that are going to collaborate, develop great friendships, but you’re also going to get friction and struggle and honest expressions of frustration with one another. And so that becomes a part of it too, so our students need to learn or end up learning—whether they need to or not—the ways in which their participation is both important but sometimes limited. They are going to sit and be witnesses to long-standing struggles in a community; for instance, long standing struggles between law enforcement and communities of color, and they’re going to find their own footing in those spaces; they’re going to need to make sense of how to be an ally, how to be an advocate for an inclusive community that they now belong to, so the stakes become a little bit more real. But I would be a little bit disingenuous if I was going to imply that it’s neat and tidy. I’ve received pushback at times. I remember we were holding a dialogue and I had said that we’re really starting to build some really empowering opportunities here and someone coughed and said, you’re from the colleges; you have all the power. It was a great check on my own assumptions about how I was being seen in that space… that participating in a community activity while still being associated in some ways representative of a very wealthy, multi-million dollar institution in a post-industrial Rust Belt City is not going to play out in someone else’s mind the way that it might in mine. Now what I’m proud of in that work is that someone felt that they were in a space where they could call out people’s unseen or unacknowledged privilege, and that I thought was really important for other people to see, and for me to experience, but it also means that tension in real relationships is ongoing. Honestly, we are not dealing with a utopian situation where we’re all playing on equal terms; we’re coming with different levels of capital and different levels of support within that community, so even as we do this work, my students are good to remember, as am I, we cannot be tourists in other people’s lives, that if we have certain privileges this is a place to take responsibility for some of them.

Rebecca: In a situation like this where tensions can be high, differences big sometimes, and you’re trying to dialogue, how do you set up that environment so people feel safe, like the situation that you’ve just described.

Khuram: Always sit in a circle. Always begin with some expectations. What do we need from each other to have respectful and productive and meaningful conversations? Let’s create those standards together and revisit them every time we sit in circle together. Have people that are prepared to facilitate, that have training or are getting training in facilitation; that needs to be, I think, a critical piece of that, because while it is important to hear from everyone, there is a lot of value in having someone who can reflect back some of the bigger messages and patterns that are emerging in the conversation, someone that can point to the standards that we’ve set for ourselves and what we expect as our best way of engaging, and to remind people that there are strategies that we’ve identified when things get really heated where we want to go with that. So, I think being very intentional about creating a dialogical space, and for us, the use of intergroup dialogue and a lot of the pedagogical strategies developed by the University of Michigan Intergroup Dialogue were very important and helpful resources to get started.

Rebecca: I think one of the things that I was hearing here that I want to just note, is if you’re having one of these conversations that you should have a facilitator and that the facilitator is not really participating in the conversation but rather facilitating the conversation. I think that can be challenging if we want to be involved in those conversations, but you need to make sure that you’ve picked that person and that person is staying as a third party.

Khuram: Yes, absolutely. And we typically have two people that will facilitate and that way there’s still some opportunity to give feedback or response or to slightly move out of a facilitator role, at least in terms of being able to share some ideas. But yeah, it does require you to pull back a bit. But having two facilitators… and it isn’t something that can’t be learned; I don’t think that people have to be lifelong professional facilitators. Most teachers are facilitators, and most of us have some experience facilitating or mediating conversations between others. As much as it’s important to start with people that have a background in facilitation, I think ultimately you want to end in a place where many of the participants feel comfortable and can contribute to the facilitation process over time, so we would meet every week. Ideally, we wanted to prepare people for their opportunity to do some facilitating. At this point we’ve seen dozens of participants go on to do much more formal facilitation in other spaces. That’s something that I’m very proud of and I’m very proud of them, I should say, for what they’ve accomplished.

John:You had mentioned some broad categories of tasks and working groups. What were some of the specific projects that were undertaken by people working in these projects in the community?

Khuram: All of these emerged dialogically as members of the campus community and long-term residents of the community talk through ways in which they felt connected and disconnected. We had four big ones, I’d say. We had community police relations, economic opportunity, food justice and food insecurity, and political representation. I’ll touch on each of them a little bit and then if you want to know a little bit more about any one of them I can pause. So, for food justice and insecurity, part of the challenge was an immediate one where it was about galvanizing community members to glean food and to increase access to fresh food, so we had volunteers doing gleaning. In the midst of that they were also looking at the president’s food deserts and dialoging along with community members about their access to nutrition and presenting some of those findings to the City Council and the Mayor. Or police community relations, we had two dedicated members who were part of a standing committee known as the Community Compact that met with different members of law enforcement and city government on a regular basis to talk about police-community relations and to develop programs to engage the community as well as to address certain policies. Then we have political representation, and for that what we saw was a wonderful volunteer energy of members of our entire group that went out and facilitated dialogues between political candidates and community members. Unlike conventional town halls where you’d have people sitting behind a table or behind a podium, we chat in circle with political candidates, and we had facilitators asking questions and facilitating dialogue in a pretty different kind of environment than I think a lot of us have when we engage with people that want to be elected, as well as elected officials. So we ran those, along with giving people an opportunity to register to vote. For economic empowerment, we trained facilitators to go out into the community in pairs and to hold circles in different corners of the community… in laundromats… in a variety of public spaces… to ask them what were the ways in which they were experiencing opportunity and what were the ways in which they were limited from economic opportunity. We also explored with them if they could wake up tomorrow to a different city, what would it look like? What opportunities would exist? And we took all of that and made it a final document called the “Big Talk in a Little City,” which has become an important and integral part of the city’s long-term commitment to economic empowerment, and so, not only are those voices and stories included in an official document, those voices and stories are now helping to shape policy and resource distribution in the city.

John:How have students reacted to this? Have any of them considered careers as working with communities and such things?

Khuram: For some of our graduates this has been life-changing. I think that one of the most fundamental things that we did well was simply to put people that would otherwise never have encountered each other in the same room and to ask them to share their stories and to talk about themselves. Developing those personal relationships between people that would otherwise pass each other on the street without a glance. People that had age differences, 40, 50, 60 years, people that had racial and socio-economic differences and geographic differences were suddenly having dinner at each other’s table, knew the names of family members, and knew the smallest things about one another were coming to their respective graduations and ceremonies and really becoming participants in each other’s lives. So, for a lot of our undergraduate students, having an opportunity like that is so deeply transformative because now policy is not just a matter of abstract equity and justice; it’s a matter of empathy and equity. You feel differently for someone who feels like a friend or family when they are in need and that informs your approach to policy and your approach to work in a community differently. So, we’ve had students that have gone on to do some really powerful work in law clinics, AmeriCorps and have stayed in the community to do some of that work because it was so transformational and they committed so much of their learning to this kind of engagement that they want to continue it. We do have a few folks that took a gap year between graduate school and stayed on, or decided to pursue a different kind of professional path because of the work they did.

John:That’s impressive.

Rebecca: It’s really exciting.

Khuram: I wouldn’t believe it if I hadn’t seen it.

John:Could you give us some idea of the scale of this—how many students are involved and how has it grown?

Khuram: We started with a relatively small group of about 20 students and 20 long-term community members, and in terms of active participants, it never really went much bigger than that, but it sustained itself over time and it also engaged a lot of other students and long-term community members for months at a time. What I mean by that, for instance, is a lot of our sustained participants would engage their friends, their roommates, their neighbors to come to our weekly sessions. So, we would oftentimes have topical session that were open to the public and those open sessions we could have up to 60, 70, 100, 200 people at those sessions, and so we had an active presence for quite a long time in the community when the courses were running, and now that we have the working groups there’s smaller numbers, but again, their impact, I think, in some ways is deeper because they’ve sustained some really deep work. One of the most incredible things that I saw the students do was they developed a course that would involve high school and college students learning together; so they essentially wanted to do what we were doing through these community dialogues in the high school. They wrote a course proposal, they submitted the course proposal, and after a few revisions and edits it was approved by both the college and the high school and we had a small group of about a half-dozen college students and a half-dozen high school students that took a course together at the high school. And that’s not a lot of people—but that doesn’t—what an incredible experience that they’re participating in something they helped codesign in order to address an issue that they perceive to be real across these age differences and community differences; that these teenagers and these college students together identified this town-gown divide and saw high school and college as a way to build bridges and constructed a course to do that and then participated in that course together. To me, that’s a kind of deep, transformative, impact that doesn’t quite reflect big numbers, but big experiences.

John:It’s certainly a testament to the impact that it had on those students that they were willing to do this and interested and motivated to do this.

Khuram: Absolutely.

John:How have your colleagues responded?

Khuram: I think that my colleagues have been excited, and I think that for many of them it created a new opportunity for them to engage. So, we’ve had faculty that have come in as participants, we’ve had them lead certain workshops and activities. They’ve come in with their expertise within their respective disciplines and fields. So, we’ve had a really great showing of faculty support. And part of it is we did not host this work on campus. We were very intentional about finding a place and space that was both a place that could be shared as well as a place that was easily accessible for long-term community residents, and so we found ourselves at the oldest black church in the city and a place that many of my colleagues had never been… that many people in the community had never been, and it was in the part of the city that is still segregated across a number of lines of race and class, and yet it was one of the most racially and socioeconomically diverse spaces that you could sit in in the city and here it was in a historically or at least currently segregated space… and so I think the opportunity for faculty and for staff to engage with a community that they’re really caring about in a context that seemed more inclusive was really exciting and affirmed a lot of their values. I think this is something that people really want, it’s just a matter of creating the opportunity so that they can engage in it. I don’t think that most faculty or staff want to engage in these kind of vertical relationships with communities. It’s just how we’ve been doing things for so long.

Rebecca: Seems like your background in teaching about equity and teaching about intersectionality and doing some research in the classroom about these topics set you up really well to do this work. Are there tips or other things that could provide faculty who don’t have that same background that you could share to give us a doorway in?

Khuram: I think that in some ways having a background as a scholar in any kind of social justice or equity field can be a barrier, and here’s why. That work is always in your head and it is disembodied in the institution, and the institution is, by its very nature, disembodied from the communities that it surrounds. And so you can very easily be a deft and prolific scholar of social inequity and convey and facilitate inequity in your actual life. So really it’s not a guarantee of anything. I think the measure of your capacity is in the doing, and I think it’s really about addressing questions. Who am I inviting to the table? Where is the table? Who is not here? What do I need to ask now to get who’s not here, here? Those are the more important questions, and I think if we don’t presume that there’s a certain kind of institutional privilege that comes even with being able to wax philosophical about questions of equity, then we’ve already lost the plot. We’ve got to honestly think about the spaces and places in which we’re doing our work and the kinds of privileges that we need to interrogate about ourselves before we can do any of this work in equitable and meaningful ways, and so I would say this work is for everybody, and this work is for anybody who is willing to really work with community members and to find shared purpose with community members. It’s willing to listen and learn from… and is not just interested in providing to.

Rebecca: Those are such great reminders… and empowering to make sure that we can all find a way to help and work with the communities that we live in.

Khuram: Yeah, and sometimes it does mean maybe rethinking a service-learning project that’s a semester long and seeing if you can map it out over a year. Would you spend a semester just creating relationships between students, yourself and long-term residents of a community just in that exploratory project? and then say, “Ok, out of this what have we identified collectively as a community need that we can address as a class?” …so that you get, of course, that buy-in, which is so important, but there’s a truly transformative possibility that is emerged that simply wasn’t there until you took the time to really connect and build that relationship, so I’m also in practical terms a really big proponent of year-long service-learning initiatives and moving away from the pressures of a semester-long initiative, unless you’re willing to do half a semester of really just relationship building and collective meaning-making and then cut the service piece a little shorter.

John:We usually wrap up the podcast with a question: “What are you going to do next?”

Khuram: What I would like to do next is to start preparing and supporting students to be the initiators of this work. I am currently working with a couple student groups that are creating their own curriculum and their own activities to engage people in the community with. Right now it’s a youth-to-youth, college student and high school student initiative, and the aim there is to just be a guide on the side, to really maximize whatever space and context I can help create for students to develop their own initiatives for engagement. Again, along these principles of working with, but to see our students become the guides that they need that our students can be the leaders that they’re looking for and that they can help develop leadership in their communities, and so for me right now what that involves is again having college students and high school students connect and collaborate and learn from each other with really very little use of faculty and take from us what you need and build what you must.

Rebecca: Sounds really exciting. Thank you so much for all that you shared today; I think it gives us all a lot to think about. Not just think about it; we need to take action too. [LAUGHTER]

Khuram:Thank you.

John:Thank you.
[MUSIC]

John:If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

49. Closing the performance gap

Sometimes, as faculty, we are quick to assume that performance gaps in our courses are due to the level of preparedness of students rather than what we do or do not do in our departments. In this episode, Dr. Angela Bauer, the chair of the Biology Department at High Point University, joins us to discuss how community building activities and growth mindset messaging combined with active learning strategies can help close the gap.

Show Notes

  • “Success for all Students: TOSS workshops” – Inside UW-Green Bay News (This includes a short video clip in which Dr. Bauer describes TOSS workshops)
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc.
  • Barkley, E. F., Cross, K. P., & Major, C. H. (2014). Collaborative learning techniques: A handbook for college faculty. John Wiley & Sons.
  • Life Sciences Education
  • Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of personality and social psychology, 69(5), 797.
  • Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American psychologist, 52(6), 613.
  • The Teaching Lab Podcast – Angela Bauer’s new podcast series. (Coming soon to iTunes and other podcast services)

Transcript

Coming Soon!

46. Creative risk-taking

When you teach the same classes every year, it’s easy to fall into routines. Classes, though, can be much more fun for you and your students if you are willing to take some risk by experimenting with new teaching approaches. In this episode, Dr. Wendy Watson, a a senior lecturer of political science and pre-law advisor at the University of North Texas. joins us to discuss how she has engaged her students by introducing some very creative and fun assignments in her classes.

Show Notes

  • Ishiyama, J., & Watson, W. L. (2014). Using Computer-Based Writing Software to Facilitate Writing Assignments in Large Political Science Classes. Journal of Political Science Education, 10(1), 93-101.
  • Watson, W. L., Hamner, J., Oldmixon, E. A., & King, K. (2015). 14. After the apocalypse: a simulation for Introduction to Politics classes. Handbook on Teaching and Learning in Political Science and International Relations, 157.
  • Wendy Watson (2016) Best and Worst Teaching Moments (Center for Learning Enhancement, Assessment, and Redesign, UNT video) – This contains a description of the zombie apocalypse project.
  • Center for Learning Enhancement, Assessment, and Redesign at UNT
  • Olson, Katie (2017). “Local Author Gets Cozy with Mystery Genre.” The Dentonite. October 3, 2017
  • Wendy Lyn Watson – author website

Transcript

John: When you teach the same classes every year, it’s very easy to fall into routines. Classes, though, can be much more fun for you and your students if you are willing to take some risk by experimenting with new teaching approaches. In this episode, we examine how one professor has engaged her students by introducing some very creative and fun assignments in her classes.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]
Rebecca: Today our guest is Dr. Wendy Watson, a senior lecturer in political science and pre-law advisor at the University of North Texas. Welcome, Wendy.
Wendy: Hi, thank you for having me.

John: We’re glad to have you here.
Our teas today are:
Wendy: I am drinking Paris. It’s a blend from Harney and Sons.

John: We have that next door.
Rebecca: Yeah, a tasty one. I have Irish breakfast tea today.

John: …and I have ginger peach green tea.
We invited you here to talk a little bit about some of the interesting things you’re doing with your classes. Could you tell us first a little bit about the classes that you normally teach.
Wendy: Sure. In the state of Texas there is a requirement that every student take two Introduction to American Politics courses in our department. We refer to that as the full employment plan. So, I teach both of those courses and then, other than that, I teach all of our law related courses. I’m not the only one, but I teach all of the law related courses: our legal systems course, civil rights and civil liberties, the rights of criminal defendants, constitutional law, an LSAT prep class, gay rights in the Constitution, and a seminar on the death penalty, in varying cycles.

John: You do quite a few innovative things in your classes, and one of those is having your students rewrite the Constitution after a zombie apocalypse. Could you tell us a little bit about that activity?
Wendy: Yeah, the idea is that the zombie apocalypse has occurred. This is actually for one of the flavors of Introduction to American politics, and this particular course deals with institutions: the founding of the Constitution, federalism, the legislature, the executive, the judiciary, and civil rights and civil liberties. The idea is that the zombie apocalypse has occurred. Huge portions of the population of the US have been destroyed and the remaining members of the country are required to rebuild the United States and part of that is rewriting the Constitution. Essentially, what they’re doing is building a government from the state of nature, but they don’t know that. They think they’re building a constitution after the zombie apocalypse, and that’s way more fun. It’s a guided exercise; they get worksheets every week making them think about “What is bicameralism? What are the benefits of bicameralism? What are the drawbacks of bicameralism? etc. They don’t just get to go off and write crazy things. They actually have to think about stuff and then they work in groups to create these Constitutions. One of the things that I really love about the course is that they actually do have to grapple with these issues. They sometimes get pretty heated.

John: How large are the groups?
Wendy: Usually these introductory courses are about a hundred and twenty five students and I put them in groups of about five to seven.
Rebecca: And are these things that happened outside of class, in class, online?
Wendy: No. I’ve taught the class as an online class in which case it obviously happens online, but when I’m teaching the class as a face-to-face course I actually do give them class time. Having them do it outside of the class nothing ever happened, giving them the time in class keeps them from hating me and also ensures that they actually do provide some sort of useful product at the end.
Rebecca: What assignments or exercises or things that you would normally do in class does this exercise replace?
Wendy: You know it doesn’t actually replace any exercises because if I weren’t using this activity, all of their homework would be outside of class and they’re still doing all of that. So what it’s really replacing is me lecturing and I’ve got no problem with that and I don’t think they have a problem with that. It’s more exciting or more interesting for them to be doing something, talking to each other than it is to be sitting in a seat listening to me, I think, I’m pretty sure. And I think it’s actually more educational for them to be engaging in the material as opposed to passively sitting and listening to me. Yeah. So although all they’re missing out on is me talking.
Rebecca: How did you how did you decide to go in this direction and develop this particular activity?
Wendy: I was trying to think of a way to create a simulation that would last throughout that semester, so something that kind of continued over the course of a term. And I wasn’t really sure what that would be, and I think we were watching The Walking Dead. But honestly how that all came together I couldn’t tell you, but yeah I’m really happy with that. It’s been adopted by several of my colleagues and by a professor at University of Whitewater. She used it in a summer program for high school students, and yeah, I’m really happy with that how that one turned out.
Rebecca: How did the students respond?
Wendy: You know,of course there are always students who are not going to respond at all. But I’ve never had a student who actively said that they hated it which is, I guess, good. I’ve had a lot of positive feedback and I had you know one of my best student interactions ever over this particular assignment. Again, I’m going to apologize to all of my biology friends out there. One of the features of the assignment is that the zombies fall into two categories. The type one zombies who are traditional brain-eating zombies and then the type two zombies who have developed a lesser mutated form of the virus. And so they have features of zombies, they have the shuffling gait and the slurred speech, but they have higher order cognitive thinking and they don’t eat brains, and they’re just generally safe. But if two type two zombies have a child together there is a probability or a possibility that their child will be a type 1 zombie. Again, this makes no sense at all, since it’s a virus and that just doesn’t make any sense. But, it raises this question of what do you do with type 2 zombies? Do you sterilize them? Do you kill them? What do you do with them? And they were grappling with this issue one day. And this poor student comes in, and he was, I swear to God, he was almost in tears. Because his group had decided collectively to exterminate the type 2 zombies and he said,” what do you do when you encounter people who are terrible?” And so he ended up having this long talk about how do you deal with the notion that there are Hitler’s in the world. I was like “Well, you have to remember that there are Gandhi’s in the world.” It was a long and lovely conversation about the essence of mankind and the balance of good and evil. And I kept emphasizing to him that this wasn’t real and that his friends were not evil, but anyway it was it was a great conversation and I was so touched that he took it so seriously. It’s just a testament to me of the fact that students really are interested in the material if you give them an opportunity to be interested in the material.
Rebecca: It sounds to me too like it allows them to really grapple with the really difficult conversations that are around rights and lack of rights and who gets those rights. That might be really uncomfortable if you talk about it in a in a real situation, but in this safe simulation you can have some of those challenging conversations that you might not be able to have as effectively.
Wendy: Yeah, I think that’s right. If you’re talking about things like race or sexual orientation, you’re always confronted with the fact that there are people in the room whose actual rights are implicated, and that does tend to make people sent to themselves perhaps, and that’s not necessarily what you want in real active discourse. So, when you’re talking about something that is seemingly unreal, it is unreal… they’re zombies… it’s not real. I do think that it gives people the opportunity to think through issues in a way that is safer, but also more honest.

John: The type 2 zombies add to the degree of difficulty or the level of challenge there.
Wendy: Yes, exactly.

John: You’ve also created a 500-person learning community, could you tell us a little bit about that?
Wendy: Yeah, that was nuts! My university decided to try to create a variety of different models of learning communities, sort of all at once, that alone was nuts. But I was going to be involved in a combined course learning community, so without any residential component. And I found this wonderful man in the psychology department who probably had no idea what he was getting into, and we created this community that was 500 students. His Introduction to Psychology course and my Introduction of Political Behavior course, that’s the other half of our introductory American politics duo. And our courses were back-to-back, so there were times when he could have two hours, and times when I could have two hours. And we focused on political psychology, specifically as it related to campaigns. And over the course of the semester, they each had to read three or four articles and write one page papers about them, little summaries, and then they came together and they shared their information, and they had to come up with the campaign strategy for either one or two presumed political presidential candidates. At the time we thought that was going to be Clinton and Rubio… that obviously didn’t happen. But they created these poster presentations and then we picked from among those poster presentations the 10 best, and we took those to UNT on the square which is a little gallery space in downtown Denton. And we invited faculty and university administration and we invited the Denton Record Chronicle which is our local newspaper. And the students really got into it, the ones who won showed up with their little red bow ties if they were representing Rubio and they had candy at their stations. And it was really awesome. It was great.
Rebecca: What do you think one of the biggest learning gains was for students who were in this learning community scenario where you were diving into something in depth from two different points of view?
Wendy: I think one of the things that they gained was an understanding that these two disciplines actually interacted with one another, that psychology and political science weren’t sort of siloed ideas, that they actually were related to one another. And I think one of the other things that they learned is that what they learned in class actually had implications for the real world. That things that we were learning in psychology and political science had implications for how politicians were actually running their campaigns. And that they could take the skills that they were learning at UNT and potentially apply them to a job, which is always a big thing. [LAUGHTER] Getting a job is good.
Rebecca: What level are the students in these classes?
Wendy: In those particular learning communities, most of the students were freshmen, first-year students, because they had to be advised into them, somebody had to sort of point them towards this pair of courses, so they tended to be freshmen. Otherwise these courses actually tend to draw students all the way up to their senior year, because they put them off until they have to graduate. But for these particular communities, they pretty much have an advisor say, “Hey, here’s a good idea. Take both of these courses.” They tended to be freshmen.
Rebecca: Did you find that the learning community method works particularly well with first-year students?
Wendy: I think for a lot of types of innovation it doesn’t necessarily, but I think for this, it did, because I think their desire to please was strong. And I think that they didn’t any preconceived notions of what college classes were supposed to be like, so they were maybe more receptive to the idea of doing something different. For all they knew this is what it was supposed to be like. [LAUGHTER]

John: …and getting that introduction to an interdisciplinary view of the world is probably good to do before they get too deeply into the silos of their major.
Wendy: Yeah, I agree.
Rebecca: So you’re full of brainy ideas and another one that you pulled off was an online Electoral College simulation game, can you tell us about that too?
Wendy: Yeah. So that was a lot of that was a lot of fun. I actually have to give most of the credit for the online component to our office, here it’s called CLEAR the Center for Learning Enhancement Assessment and Redesign. The assessment component has largely gone out of clear, but that’s still what we call them. They do all of our online support, learning management system, redesigned helping us create online courses, all of that sort of work. And I had a sort of a low-tech version of this course. Originally they were working in groups, I always make them work in groups, I don’t know why. But they had groups and the idea was I used the map from 270 to win, which has sort of the baseline Electoral College predictions, and which states are going red, which states are going blue, etc. And then students had campaign money and they could essentially bet their money on individual states. And if you were the Republican Party and you bet fifty dollars here, but then the Democrats get 51, then the Democrats won the state, so whoever bet more money in a state won the state. And so you could see the strategy of betting in different states of spending more campaign money more campaign resources in each state, and as you won a state, the states that were blue moved around or the states that were red moved around and you could see the total – who was winning the electoral college. And it was played in three rounds. But this was a huge pain to implement in the classroom with having to update this Excel spreadsheet every round and get people’s votes every round. It was a nightmare. So CLEAR created an online version for us that allowed students to play against each other online and it was really slick, it was beautiful, I loved it.
Rebecca: So, I’m noticing the “loved” as opposed to “I love it”.
Wendy: You notice that didn’t you.[LAUGHTER]…… Yeah, so I think another point to make here is that if you’re going to launch into one of these grand plans, you really do want to have some long-term commitment from your University. I love my university but long-term commitment is not their forte and for the learning community, for example, Adriel and I (my co-conspirator and I), we put a lot of effort into that course and we ended up offering it twice. It went really well both times but to the extent we needed money it came from a Title III grant that ended. So, we didn’t have the money anymore and then we also depended very much on help from the registrar, from advising, and from admissions to help us coordinate all of the the details. Because it was no small matter, right? It was actually very difficult. It wasn’t just us. There are all sorts of offices that had to help us out with this. And the university basically was like, “Oh, we’re done.” That was difficult and so we just lost the necessary institutional support for maintaining that program. And with the electoral college I went for like a year and a half without teaching that course, so it didn’t get used because nobody else was using it. And so CLEAR stopped supporting it on their website. It just went away and it’s just gone. So, it’s just one of those things. You kind of need to get it in writing, because there’s a tremendous amount of start-up costs associated with these programs and unless you know that that’s going to carry forth and this investment is going to pay out over an extended period of time, tt could be a little bit demoralizing.

John: In one of your other experiments in class, you did something with a mystery room. Could you tell us how that worked?
Wendy: Oh yeah, that was this last year. That was so much fun. Yeah, so the game was actually called Free Lucky. Lucky is UNT’s unofficial mascot. He’s an albino squirrel; he’s actually not lucky at all. We’ve had a series of Lucky’s on campus and the only two that I’m aware of… one got carried off by a red tail hawk and the other one got hit by a car, so they’re not lucky. [LAUGHTER] But we call him Lucky and you can get little lucky dolls. And so I got little Lucky dolls and I shoved them in little cloth pencil cases and I put combination locks on the pencil cases, so he had to get him out by undoing the lock. And I’m put my groups of students… groups again… in various study rooms in the library and they each had a little encased enshrouded Lucky in their room. And then they started the game with a question on their learning management system on Blackboard. This was for an LSAT prep course and the beautiful thing about the LSAT is that you have these questions with very specific answers. No question… here’s the answer… that’s it. The first question, if they got it right, it led them to a webpage with another question; if they got it wrong it led them to a webpage that had nothing and then it sent them back to the original page, and so forth and so on. It sent them to various pages around the web, some of them with clues, some of them with other questions, eventually it would’ve taken them off of the web and sometimes it pointed them to different clues around the room. There were various and sundry things on the table, some of them which mattered… there was a playing card… it actually was a clue, but then there were things like spools of thread that meant nothing. There was envelopes taped under the table that had a whole series of questions. And the questions there, if you answered them all, there were four of them and those gave you letters and then there was a tongue depressor on the table that helped you translate the letters into numbers and that was the code to the combination lock and that allowed you to free Lucky. And the first team that got Lucky to me… I was sitting in the lobby of the library, first team that got Lucky to me won… and they won packets of colored highlighters, which doesn’t sound exciting but they were all pre-law students and that’s like gold in the legal community… is colored highlighters. So it was exciting, they were really thrilled.

John: It sounds like fun.
Rebecca: It sounds like a lot of fun.
Wendy: It was.
Rebecca: What made you decide to do a mystery room?
Wendy: Well, you know, we have one here in Denton, and I think it looks really cool and I want to go, but I can never get people to go with me, and so I decided well I’m just gonna create my own. I wanted to do something, again, that was interesting. As much as the LSAT prep stuff was really interesting and important for my students, it’s not super engaging. We could stand up there and write logic game trees on the board, for hours on end, but that’s not exciting. That’s not even lecture exciting, that’s just really really boring. So I wanted to at least break up the class a little bit by having something that was more engaging, more active, something that was interesting.

John: And it brings in gamification too, where there’s some incentives and competition.
Wendy: Yeah. Oh yeah, the competition was big. I had one group that came down with Lucky after about a minute and a half. I was like, “You did not answer all those questions.” The guy who handed me Lucky, he’s like, “You gave this puzzle to a marine .” [LAUGHTER] I was like, “So, did you just bust the lock?” He’s like “No, I didn’t have to bust the lock. I could get him out without busting the lock.” I was like “You have to open the lock, you can’t cheat.” [LAUGHTER]. So they went back, they did it. But anyway, yeah, it was definitely a game to them. They were serious about it.
Rebecca: That’s hysterical and unexpected, right? [LAUGHTER]
Wendy: Completely.

John: A common theme of all this is that you seem to experiment with your classes and take some risks in trying new things. Could you tell us a little bit about what prompts you to do that?
Wendy: A couple of things. One, is that honestly it keeps me interested in the courses. I can get bored with the material as much as they can. In fact, they sit through it for a single semester, I sit through it for semester, after semester, after semester. And you can only talk about the appointments clause for one or two times before you’re like “Oh my god, I’m gonna dig my eyes out. This is really dull.” And that’s something I actually enjoy, right? I think the appointments clause is interesting. You still want to shake it up a little bit. And the other reason is that I really do believe that students learn better if they are engaged. As much as I love to hear myself speak, I don’t necessarily think that they love to hear me speak. I think that they get more out of my class if they are doing something. If they are seeing some connection between what we’re doing in the real world. If they can see themselves actively engaged. If they have a sense that they have power in the class. Some sense of control over their own education. I think all of those things are really valuable to them. So it’s a little more effort for me, but I think the payoffs are worth it.
Rebecca: So all of these examples that you’ve shared with us today are really different from one another: they use different technologies, different setups. What is your advice to someone who wants to take some risks and try something new, but it’s something that they’ve never done before?
Wendy: Start small. Don’t start with a 500 person learning community, which is what I did. That was dumb. It worked out, but it was dumb. Yeah, start small. Collaborate with somebody so you have somebody to lean on and share ideas. That’s maybe why the learning community worked, is that I had something called the Core Academy, so we were focused on these sorts of things together. And then I had my my co-teacher, Adriel, to work with. I think having a support system and starting small is the way to go. You don’t have to do a semester long simulation, you can devote one class to something. Use a method that lots of people are using, like team-based learning. You don’t have to do that all semester you could do it for one class. There’s nothing wrong with starting small and then getting bigger.
Rebecca: Did you start small?
Wendy: I did not [Laughter].

John: Somehow I suspected that would be the answer.
Wendy: Yeah, that’s not my style. But again, I think that if you’re worried about getting started, if you are less stupid than I am, then don’t hesitate to start small. There’s nothing wrong with that.
Rebecca: Have you had any student resistance to some of the alternative or non-traditional methods that you’ve been using in your classes?
Wendy: I get a little resistance sometimes. For the most part, they actually seem to enjoy it. Every now and then I’ll get a student who seems to think that I’m not doing my job. I mean I’ve had students who flat out on evaluation have said “I expected to come to class and hear you talk and you didn’t.” Like “Really? That was what you expected?” I mean, yeah, I assumed that is the expectation, but like, “You’re disappointed that didn’t happen?” I can’t imagine that. And of course there’s always, as I mentioned, a lot of these things involve group work, and a lot of students have resistance to group work. Even when the group work ultimately works out okay, they still are annoyed that I put them in groups. Just the anxiety associated with group work carries over to the end of the semester. Of course, some groups don’t work out. You’ve always got somebody in some group that either doesn’t pull their weight, or is responsible for a part of the project and fails to turn it in, or somebody in the group who is bossy. You always have some group that’s got a problem and I usually try to mediate that situation, but sometimes they don’t come to me until it’s too late. There are always points of contention. But they’re relatively few, and honestly I’ve always got a few complaints when I lecture too. I’d like to say I never have complaints there, but I do.
Rebecca: I read this really great article about you being a mystery novelist.
Wendy: I am.
Rebecca: Can you talk a little bit about that?
Wendy: Sure, yeah. I am a mystery writer. I started writing a long time ago, right around 2001 actually. A bad year. But I had my first novel published in 2009. I write a type of mystery called a Cozy, which is exactly what it sounds like, it’s cozy. They are light, often funny mysteries. Amateur sleuths, so no cops or private investigators. They can be in the book, but they’re not the primary character. Usually female sleuths, small town, no sex or violence on the page. I mean obviously somebody dies but it happens off the screen. They’re really quite delightful. I said PG-13, I actually included the word “bitch” in my first book, and it wasn’t even calling somebody bitch. It was like son of a bitch. I hope I’m not destroying your podcast by using that word [Laughter]. I actually got nasty emails about using that word. Really? Oh my goodness. I don’t use that word there anymore. Yeah so I started writing I’m working on number seven right now and, that’s that.

John: How do you manage that along with all your innovation in class? It seems like that’s a lot of demands on your time. How do you allocate your time?
Wendy: Not well. Yeah, I was talking about this with a colleague this morning, we were talking about this LSAT prep course (she’s teaching it this semester) about the fact that prelaw students really should be studying a lot for the LSAT. It’s a huge portion of their application. Yet, for some reason, they don’t and instead they focus so much on their GPA, which is important, but honestly, not as important as their LSAT score. They shouldn’t let their GPA slide either, let’s be clear. But in the grand scheme they should be focusing on their LSAT score. We were discussing the fact that the LSAT is way far away but their GPA is right in front of them, and so that just feels like the thing they need to tackle right now. And for me my deadlines are way far away and my courses right in front of me. So I tend to focus on my coursework and I’m not so great about meeting my deadlines, and I apologize deeply to my editor, but that’s just the way it is. I do though have a calendar, a very detailed calendar, that I keep, that has specific time set aside for every single thing that I do. Not always true to that calendar, but I do have a calendar, and it includes time set aside for writing.
Rebecca: Do you find that your writing life and your teaching life influence one another?
Wendy: Yes. Certainly my academic life has influenced my writing life. One of my books was set on college campus and I got to kill off a couple people that I didn’t like so much, which was awesome [Laughter]. Certainly, I think that my tendency toward narrative, toward storytelling, influences my use of hypotheticals in my classes. To the extent that I’m sort of telling stories. Like the zombie apocalypse, I didn’t just write a paragraph: there has been a zombie apocalypse. It’s this, probably too long story, about this has happened, and it’s all dramatic, and that’s definitely a carryover from my writing life.
Rebecca: I imagine that those details though and that spike in the climax to a story, are all the things that get students really engaged and interested and and buy into the simulation and take it seriously. As opposed to something that’s a little more surface level and that it’s a little harder to imagine.
Wendy: Yeah, and I think sometimes one of the things my cozies tend to include is humor, at least I hope it’s humor. I tend to inject that into my hypotheticals a lot and I think that that helps. One of the simulations that I do in my legal systems class is a negotiation divorce case. Each side in the negotiation has information about their client. Some of its common knowledge, that both sides have, and the wife’s attorney has knowledge that only the wife has provided and the husband’s attorney has information that only the husband has provided, and they know that that information is going to come up during the negotiation in a series of PowerPoint slides. They don’t know when that’s going to happen, but the idea is that all the sudden the wife is going to blurt something out during the negotiation. They also don’t know that there’s information that the husband and wife have not told their own attorney and that’s going to come out in the course of the negotiation. So I had great fun crafting the simulation; like the things that the husband and wife have done, and the pieces of information that come out are delicious, and the students have so much fun finding out about these details. And yeah, I think that that makes the whole simulation so much more engaging, instead of just calculating the appropriate alimony. I think it’s a lot more fun.
Rebecca: Can you share a couple of tips from your creative writing self that might help other people come up with hypotheticals or examples that they could use in their classes?
Wendy: Yeah, I think one thing that you want to do is provide detail. If you’re going to create a hypothetical, create a character to go with the hypothetical, and then provide some detail about the character and the setting and those sorts of things. It really enriches the hypothetical. It doesn’t all have to be completely relevant. In fact, sometimes it’s better if it’s not all relevant because then it forces the student to look past the things that aren’t relevant to find the things that are. I think that’s probably the key is to include at least one person in your narrative and then provide some detail. Provide a setting, provide some description of your character, provide some element of detail about what’s happening, so that it’s not sterile or clinical. Because that’s, like you said, that’s really going to draw the student in, in a way that’s sort of, A happened, B happened, C happened, or not.
Rebecca: That’s great advice [LAUGHTER].

John: We always end with the question, what are you going to do next?
Wendy: So this year I’m actually not teaching, which it is really weird for me. Last year this time, I took a position as the director of the university’s core curriculum. So, this year I’m going to be continuing with my pre-law advising but otherwise I’m focused on the university’s core curriculum. I will be engaged in assessment, which is everybody’s favorite thing, but I’m also gonna be developing a lot of programs related to our cores. So some programs related to writing across the curriculum, some programs related to bringing back, I hope, some of our learning community endeavors, and possibly exploring some other options that would allow us to really enrich our university core curriculum for our students. When I talk to students now they talk about them as the basics or the things that they have to check off, and I want them to think of those classes as something more than that. So that’s what’s next for me.
Rebecca: Sounds like the right person might be in that job to help inspire students. [LAUGHTER] I think sometimes that’s a hard sell these days, helping students recognize the value of a liberal education, and get them excited about it and help them find connections.
Wendy: Yeah, I agree. I think I have a tough road ahead of me but I’m going to do my best.
Rebecca: I look forward to hearing more about it.
Wendy: Yeah, thank you. I’d love to come back sometime.

John: We’d love to have you back.
Rebecca: Thank you so much for spending time with us this afternoon and sharing all your great initiatives in your classes, I hope it’ll inspire a lot of our listeners.
Wendy: Thank you, I really enjoyed it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

44. Industry realistic experiences

Student motivation is enhanced when students see that the work they are doing is relevant to their future careers. In this episode, Dr. Bastian Tenbergen, an assistant professor of Computer Science at the State University of New York at Oswego, joins us to discuss how industry realistic projects may be used to enhance learning in software engineering classes.

Show Notes

  • Daun, M., Salmon, A., Tenbergen, B., Weyer, T., & Pohl, K. (2014, April). Industrial case studies in graduate requirements engineering courses: The impact on student motivation. In Software Engineering Education and Training (CSEE&T), 2014 IEEE 27th Conference on (pp. 3-12). IEEE.
  • Daun, M., Salmon, A., Weyer, T., Pohl, K., & Tenbergen, B. (2016, April). Project-based learning with examples from industry in university courses: an experience report from an undergraduate requirements engineering course. In Software Engineering Education and Training (CSEET), 2016 IEEE 29th International Conference on (pp. 184-193). IEEE.
  • Dijkstra, E. W. (1959). “A Note on Two Problems in Connection with Graphs.” Numerische Math. 1, 269-271.

Transcript

John: Student motivation is enhanced when students see that the work they’re doing in their classes is relevant to their future careers. In this episode we examine how industry realistic projects may be used to enhance learning in software engineering classes.
Thanks for joining us for “Tea for Teaching,” an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego [Music]

John: Today our guest is Dr. Bastian Tenbergen, an assistant professor of computer science at the State University of New York at Oswego. Welcome!

Bastian: Thank you, thanks for having me.

Rebecca: Today our teas are…

Bastian: Well, upon John’s recommendation, I’m having the mint herbal mix tea, which is excellent! I’m a peppermint tea drinker, so this is blowing my mind right now.

Rebecca: Excellent!

John: I’m having ginger tea.

Rebecca: I’m having Prince of Wales today.

Bastian: I like the ginger tea, that is my favorite tea.

John: It’s good.

Bastian: Ginger and fennel and peppermint, those are my three.

John: We invited you here to talk a bit about the projects that you have students do in your computer science classes. What classes do you generally teach?

Bastian: I’m teaching in the computer science department, but I’m mostly teaching software engineering courses. We actually have two separate majors: we have computer science majors (Bachelor of Arts and Bachelor of Science) and we also have a software engineering Bachelor of Science program. People usually confuse software engineering and computer science or at the very least don’t really know what the differentiation is. In contrast to computer science where it’s really all about programming and all about finding optimal algorithms to solve problem x for person Y, software engineering is concerned with the process of development from A to Z. So from requirements all the way to programming which is a small part of it, all the way to Quality Assurance and also budgeting. Also, the business aspect of it, so it has a wider focus.

Rebecca: It’s a little more client facing?

Bastian: Very much client facing, yes. By trade I’m a requirements engineer you can say and a very smart person who very recently submitted his PhD dissertation (which I’m very proud of him that he did finally did that). He wants to find requirements engineering as a socio technical process that implements the vision of a system given the time and budget constraints that you have. They usually also call us the context of the system, the developmental context of a system. It’s the budget, the time, the resources you have and such things. Those are considerations during software engineering.

John: In what classes do you have students engage in projects?

Bastian: Well it is very hard to teach computer science without actually using projects. You can teach the skills but at some point the art of making software becomes more than the alignment of skills in a particular way. Legitimately almost all classes we teach have a very heavy focus on projects. I’m teaching a software and safety requirements engineering course which is project-based, at least a quarter to half the students grades depends on the project. I’m also teaching a software quality assurance class where at least a quarter, sometimes half of the grade depends on project performance. I’m also teaching occasionally capstone courses, where the capstone experience in the software engineering program really tries to simulate how an independent developer develops a spoke software for one individual client and one of my favorite things to teach is a class called “Software Design”. The term design implies software architecture but it’s not just that. For those software engineers out there listening, this particular course is called that for historic reasons, but it’s really a design process class. The entire class collaborates together on producing one substantial piece of software, which is usually on the first day of class. I demand like big evil Papa Smurf that this project could be marketable, so the explicit goal is we want to market it, which sometimes works and sometimes doesn’t, but that’s the goal. Then we differentiate the students into teams and have a database team, a GUI team, we have graduate students at our university that specifically focus on usability and human factors so we have those as a team, we have requirements teams, we have Quality Assurance teams. They have to learn not only how to work together, they also have to learn how to apply their skills, have to learn how to best make design decisions, how to communicate them and not only how to communicate them with like-minded peers that are also scientifically or engineering capable but also with a stakeholder. Software engineering in general is very focused on the people who are giving the money for a project. In my classes I really focus on the fact that students should be able to argue their rationales, not to other engineers and not to other technicians, but to their grandmother because if you can explain it to your grandma, you can explain it to the person who gives you money in the project; and that usually worked well.

John: How early in the term do students decide on the project?

Bastian: So, It depends. It depends on the course. In my requirements engineering and software quality assurance class where we also teach skills, we also teach requirements, solicitation, or you teach let’s say data flow based testing, which is a new technique for them to pick up. There, I usually pick the projects for them or if they have a particular good idea we’ll discuss it, but usually it’s in the first week or so that they finalize the project. In capstone classes and in the software design process class, I usually conceive the project ideas and then we make the necessary choices, let’s say the necessary preliminary choices in the first week. What I mean by necessary and preliminary choices it’s this; I basically say “I want a universal all-transfunctionater” and no one has any clue what that is and I say “great it’s your job to ask the stakeholder, who is also me, what I mean by that.” Then the requirements team would differentiate the people into teams and the people who self-select into requirements they say: “Ok, well Bastian, what did you mean by that?” …and I say “Well, I meant… really… whatever… a cow milking device.” So the project kind of takes shape. So, I force them to come up with the requirements and to get them out of me, so that, as an instructor I basically have a dual role… or actually triple role, sometimes quadruple role and I’m project manager for them. I’m also the stakeholder, I’m also the person who gives them advice and the instructor that says “dude you shouldn’t do this because X & Y & Z or whatever. Or, maybe here’s a great idea that someone else just had and maybe try this.” More often than not I’m also the conflict solver and a psychologist that lets them cry on the shoulder because at some point during the semester everyone is just frustrated. This is part of the experience I guess but that’s why I usually tell my students the trick is to be successful despite other humans and once that idea clicks, working together never becomes a problem ever again. So as you lose one conflict earlier in the semester and then it kind of dissolves and this is when you see the students go from students to professionals. It’s my favorite class to teach because you can see how the students go from “professor, how do you want this” to “well Bastian I know you said you want a cow milking device but see we don’t have any cows, so how about we build you this instead”. It’s important in these kinds of projects for them to be able to communicate what the stakeholder wanted versus what we can conceivably give to the stakeholder given the time and the budget and the people that we have on staff.

Rebecca: Or what this stakeholder may actually need and doesn’t realize that they need.

Bastian: That’s right! Two years ago, I co-taught to this class for the first time which was great because then we could literally play good-cop and bad-cop. One stakeholder and one instructor will always be against the ideas, which believe it or not wasn’t necessarily me, and the other one was always in favor and would always say “oh yes that’s fine, that’s fine, Keep going”. But you know even if you have someone who constantly approves of what you do you don’t know whether or not you’re actually making any good progress. So it may feel good to have your ego stroked and be told that yes everything is great but that doesn’t necessarily mean that you’re making useful progress. Really in the end the only way you learn is if you make mistakes. On the other hand of course being told everything is bad or everything is completely horrible and how dare you even propose this doesn’t help either. So the truth is somewhere in the middle and it’s for the students to find out what goes. That’s the tricky part about teaching this kind of class, is to guide the knowledge discovery process such that they find it but they can still be successful despite having to do all the work themselves really.

Rebecca: So you’re describing mostly the setup for your software design class right? Which is a big team right that has small teams on it, but you’re all working on the same project.

Bastian: Yes.

Rebecca: Are your other projects and your other classes also set up so that everyone’s working on the same project or individuals working on a project? How are the setup similar are different?

Bastian: You have teams of students I have a very much focused on that that students would at least together with one other person. And the reason is, four eyes see more than two eyes, that’s why. Plus I encourage them like, hey, you know if you talk to another person, if you vocalize your problems, it helps, it stimulates your thinking. So that’s why I do this for example my requirements class, I give the general theme of the project and then let the students do some of it on their own. For example, a little while ago when I taught this software and safety the requirements class first here in the US, I gave the students the opportunity to I said, “okay, we have these cyber physical Rovers or robots, never mind what cyber physical systems are but it’s a buzzword and they can do certain things something makes them special”. We discussed this in class and I said, “we have these robots, and I want you to do something cool with them.” “They each have individual functionalities, pick one for different sensors, different robots had different sensors, pick one and do something fun with it”. And they pitched the project ideas. For example, one of them said, “I want my robot to exit a maze.” Great idea do it. Another person said, “I want my robot to use the camera and use computer vision to recognize another robot and drive after him”. And it was a cool project. Another team of thing was three students actually said, “no we don’t like the robots we’d much rather do something else and here’s an idea”, and I said “okay”. Soon as the learning objectives that I have to find in my syllabus are roughly aligned, I’ll let them go. My general philosophy is if the student has a better idea than me and can argue it, ok. Because I want to learn something too, right? (laughter). So I let them do it and let him explore it if they have the idea right.

John: The students would have more ownership till when they come up with the idea.

Bastian: True. Usually I’m not sure if it’s me over the project or it’s just those cute little robots that we have, but usually students are quite enthusiastic about projects. For the coming semester believe it or not we bought programmable slot cars. Remember those slot cars that you used to race on the like little tracks, you a little controller in your hand you can push more and less gas and throttle. We bought programmable ones and we’re gonna be using that in a project. I’m super excited about this and can’t wait to play with that. I’m hoping students will be excited about this too. And if they’re not then fine they’re not expensive.(Laughter). Plus we have several other faculty in our department who are quite excited about these. I’m not going to tell you the name of the manufacturer but they have a very cool API, which is an application programming interface, which is really simple and open. I haven’t tried them out yet, so I’m hoping it that’s a needle platform to automotive software engineering projects which would be cool.

Rebecca: So, as your students are working in teams and you’re trying to make sure that they’re prepared for professional life, right? You’ve talked a little bit about thinking about clients and things like that. How do you make sure that the problem that they’re solving is realistic and it’s not pared down so much that it’s unrealistic? Sometimes when students self define a project, it’s in a context that wouldn’t generally exist when they are working on their own unless they’re at a startup.

Bastian: That is so true. I would argue that finding the project not necessarily the scope, but the project domain is probably one of the two hardest things about doing the project. In fact, I’m not sure if I’m allowed to say this and make some advertisement on my own behalf here, but colleagues of mine and I wrote two academic papers and we’ve just submitted the third one on project-based learning in industry-realistic case examples in software engineering to a fairly substantial fairly high ranking conference. The industry realistic examples, they usually reflect one or two aspects that you would commonly find in let’s say industrial development projects. For example, the problem of, let’s say sensor integration. If you have a little robot and you tell the robot to rotate 90 degrees, you can know whether or not that thing actually turned 90 degrees because the one motor if you have two wheels, assuming you have a two-wheeled robot one motor might be have different manufacturing tolerances and maybe a little bit stronger than the other one, so you may be turned 89 degrees, maybe you turn 94 degrees. So how do you fix that? Well you could put a little sensor on it that does that, but the only rotational sensors you have they are going to be inaccurate too. Especially if you have let’s say have the robot run on carpet rather than tile. All of a sudden the physical setting and that the robot is in has a great impact on the software that you’re developing, and that is an industry realistic problem. Let’s say you fly an autonomous aerial vehicle somewhere and try to detect wildfires, which we are currently experiencing a very hot summer with a lot of drought. So they do this, they use drones to detect wildfires. How do you know you’re actually currently flying through smoke as opposed to through humidity or through fog or through a regular cloud? You have to use sensors. It’s a realistic problem. So the domain flying an actual drone is hard, so we use a little robot which however has the same kind of problems. I was very fortunate that earlier in life, I was working with some industrial companies in research projects and so it’s relatively easy for me to figure out what could be a challenge that the software developer or software engineers is going to be facing. So in those two papers that are just described, we focused on how to apply industry realistic case examples and we figured out what kind of properties these have. For example, you want to be sure that the project that you give to your students doesn’t have a bunch of challenges, but just one is usually enough, just to focus on one little challenge. For example, get the little robot to rotate accurately, but you don’t tell them make a project that lets the robot rotate, because that’s boring. Instead you say, “hey, why don’t you write an overtaking algorithm for robots?” And usually you know full well that in order to make those robots actually overtake one another like cars on a highway, a lot of things have to fall into place. First for example, you have to figure out how to make this robot drive straight and that is already a project in an art of itself. So the other important criterion for these industry realistic projects is to have the project scalable. So toward the end of the semester I usually joke with my students and say, “well, if you can’t finish your project in time, it’s either because you didn’t scope the requirements right, or because you bit off more you can chew, development is harder than you initially thought, or maybe because we haven’t redefined success yet.” So if you can’t be successful redefine success. Which when I say that really what I mean is I tell them, listen, you can’t deliver what you wanted to deliver, fine, not a problem happens all the time in reality, instead tell me what we can expect. Given the time that’s left what can we expect. “Well, we can actually make the robot overtake”, they will say, “but we can make it drive straight with a certain level of accuracy.” That seems boring and uninteresting when I say it like this but it’s actually a remarkable feat. At the end of the semester, two kinds of students those that are happy to be done because this was horrible experience, the minority thankfully, or you have the people that say, “oh my god, had no idea how hard it actually is to interface hardware and software.”

Rebecca: Really a big lesson in scoping, it’s like how do you break a big project into small pieces.

Bastian: Absolutely.

Rebecca: Understand that small pieces have to be completed before you can put them together to make a big piece. It’s like modular design.

Bastian: Yeah, absolutely. Modular design is one of those keywords buzzwords almost from the 90s, but they were right. You divide and conquer is a recurring theme in computer science that works everywhere. If you want to sort numbers you divide and you conquer it’s the fastest way to do it and if you want to develop a software project you divide and you conquer. Your first build project one and project two. You can scope this whatever way you want. Very often actually I have students who halfway through the project realize the potential that the project has that they’re working on and say, “hey Bastion, I really would like to bring this project into this direction instead I know you said overtake algorithm, but let’s do a path finding algorithm instead.” Esker Dijkstra in the 1960s wrote basically the silver bullet of shortest path algorithms and, can I implement that and put it in the robot? And why not? Just last semester I had someone interested in that doing it. The third characteristic about these projects is don’t be a stickler too much for what the industry really experiences and let the student figure it out on their own. And the one hand you could simulate what companies develop software to particular degree. So you could say, oh we are all now going to fill out application slips or vacation slips or things like this right, but that this misleading from the art of developing software. On the other hand when you tell the student hey listen or when the student asks, “hey listen, I want to bring this in another direction because I find this really interesting,” usually what comes out is something really rewarding, In my experience at least. So the third concept is don’t overdo it students will by themselves, with enough enthusiasm, drive it into a direction that is going to blow your mind, theirs and yours.

Rebecca: So when students are working together in teams and they’re taking on kind of different roles. How do you help the students divide those rules but then also make sure that they’re learning all of the skills or techniques that you want them to learn.

Bastian: That’s hard it’s really really hard and I would say that there’s no silver bullet of how to do this. It is an unfortunate truth that the larger the project is the more people are working on the same project, the higher the chance that at least one person is simply left out and you can be the kind of person that says okay, let’s try to live this person up to make sure that they learn something, but to be entirely honest, in part, in my opinion it’s a component of the experience to make yourself available to your team. So what I do throughout the semester is encourage students to contribute any way they can and students miss understand sometimes from a grading perspective that contributing means being the natural-born leader. In my experience, every team no matter what has one or maybe two people who are really great at the technology and also really great with people and their form naturally adopt the role of the leader. Assigning a leader doesn’t really work all that often. You can say okay you’re a graduate student so you’ll have more management responsibilities and that usually works. But often there’s one non graduate student who’s also fulfilling this managerial role so part of the experience is to find any way you can possibly be helpful for your team this doesn’t necessarily have to be the leader role. You cannot be a leader and be a rather shy quiet person and still get an A in project based courses, the way I teach him. Simply because what does an A mean? An A means here an excellent outstanding student and when are you excellent outstanding student? Well, in these cases when you’re an excellent outstanding team member for your team. When are you doing that? Well, when you contribute stuff any way you can to your team such that your team can continue. I’ll give you an example, if you are the kind of person that never volunteers presentations in class, that never contacts me as the instructor with questions, that never has an management important role in the team but manages all the background communication, implements all the code, and does all the right things in the simply couldn’t contribute couldn’t do what they’re supposed to do if it wasn’t for your input; you’re an A student, regardless of whether or not you’re very outspoken and outgoing or not. On the other hand, if you are a student who talks a lot and who is volunteering a lot, and who is putting themselves in the limelight a lot, but at the end of the day your team can’t count on you because you didn’t show up for the team meeting or because you promised something but never delivered or because the stuff that you deliver is of poor quality and your team decides to drop it and not use your work. Then you’re clearly on the other end of the grading spectrum. So I have a rubric, a rubric system where I say oh can a student clearly is the backbone of the team any way possible a B student is delivering useful stuff in regular intervals and C student is well useful when being assigned work, right, and a D student is unfortunately not useful even when prompted and an e student is the kind of student where the team said listen we’ve asked you 15 times you haven’t done a darn thing we’re done with you.

John: We should know that as we go for some reason we use E’s instead of F’s.

Bastian: Oh that’s right. I’m sorry.

Rebecca: Its alphabetical.

John: It doesn’t make sense to any of us but it’s been done here for a long time.

Bastian: It’s true. So a student that is failing the course with an E or other universities with an F usually those students know that they are. Usually before they are even assigned a failing grade I’ve had numerous conversations with them not as the manager, not asked stakeholder, but as the Papa Smurf (laughter) who says listen, if you want to pass this class, and for software engineering students in our university this class is a core requirement, so they have to have a passing grade in this class to graduate. I say listen, right now you’re not. We’re also doing peer evaluations so some people could say well if you were the one that subjectively evaluates the students isn’t that unfair and the answer is yes, of course. So I’ve experimented with this, just evaluations by me, and I had some good experiences with it and also some very bad ones, unfortunately. So within disputes, and it happens occasionally. What I like to do is peer evaluations where students within the same team evaluate other team members on a scale of say 25 points and usually, and remarkably, these peer evaluations match my subjective opinion almost all the time, 100 %. Students when they evaluate others are usually little positively biased and they are reluctant to evaluate people really badly, but if you ignore that, the subjective evaluation students have of each other are matching my observations very well.

John: How often do they get feedback in terms of how well they’re doing?

Bastian: Every day, every day. We meet usually in this class, we are meeting three times a week or the university has allotted three meeting times a week. I like to schedule two meetings where I’m there and they are reporting to me in daily scrums, those of you who are software engineers,yes we’re doing AGILE methodology specifically scrum. We do daily scrum so it’s basically, you stand up when and you say this is what we have done from last time until today, this is what we’re currently working on, this is what we’ll do next, these are the roadblocks, these are possible problems, and these are questions that we have. Five minutes, everyone does it and usually takes the entire class period to figure out problems, to resolve roadblocks, and most of the time it’s minor things but gotta get done because it’s the planning for the rest. So, during that is when I provide feedback by saying hey have you done this yet or have you thought about that yet, or John Doe here, was supposed to deliver this and that, did they? On the other hand, I’ve very often we have experiences that students say well, see our friend Jane Doe here foresaw two weeks ago that this is going to be a problem, so she already did this and that in anticipation. That’s how you know you have a really great student at hand, right, when they can anticipate problems in the future but would usually only experienced engineers are able to do. So they get feedback every time. What I do however, is the third class meeting that we have, I usually reserve for project work. Because that is the one day in the academic schedule for all students in the class, and if you have 30 people in the class, that I know they have time. Especially at the beginning of the semester I often hear things like, oh we don’t have class on Friday. I’m like, no, no, no, no no, you have class. I might not be there and the reality is that of course I’m there, I’m just then the next room letting them duke it out, and when the shouting or the crying gets too loud, I walk in. Or they decide on things and they have a question and needed it answered right then right there, so they walk over to the other room, or wherever, I am and they ask me. Or I just sit quietly in the room and let the students plan the work on their own. So, the idea is that the third meeting of the week is usually when they get to make progress when they need other people to be present. We also usually coordinate using online chat functions, we’ve used Discord.

John: This is used in a lot of gaming.

Bastian: Yes yes I use them gaming a lot right? Plus all my students they’re all familiar with it because they’re usually all gamers. And we even have a little Steam community going because, you’re nerds like that. So they coordinate through Discord and sometimes they say, hey Bastian is a fine if we don’t meet in person because John and Jane are out of town because, whatever, wedding or sick or whatever, is it okay if we do this online? I say sure, I don’t care how you get it done, just get it done. That’s all I care about. I care about you make progress any way you can. Next semester I’m actually preparing for having this class for the first time in a sort of hybrid fashion. Hybrid in how a university means a portion of the course is online the other portion is a physical in class meetings and what I want to experiment with is, moving this course to an entirely online fashion. Basically simulate how offshore development works. Let’s say you have a team working in Atlanta, you have a team that works here in upstate New York, and you have another team in India or Poland or Germany, and they work together they have to coordinate somehow. So we’re gonna do this next semester. I’m excited, really excited for that.

John: Interesting. Will there be a synchronous component where you have everyone report?

John: Absolutely. So the reason why I said hybrid is because we’re gonna meet exactly twice in person. It’s going to be at the first class we’re going to actually physically meet. I tell them that from now on we’re not going to meet anymore. Instead, we’re going to meet online using an online meeting tool. The university has a couple of licenses that we’re friendly enough to allow me to use one. So we’re using this tool, we’re doing online meetings where everyone has to be present and has to do the same things we would otherwise do if we had physical, in-class meetings; the daily scrum, this is what I’ve been working on, this is what I’m gonna do next, this is what we as a team have been doing. So we still have the immediate feedback component, we can still plan ahead and we can still do all of this. The second time we meet will be at the end of the semester when we present the final project and when we show the final implementation to the stakeholder. Basically like a sales pitch. Of course that’s gonna be problematic because specially the usability folks, those part of the team who are going to be conducting actual usability tests with human subjects committee approval and everything, so we do it the actual way that a company does it, they of course have to meet. This is for next semester I’m actually thinking about having them fill out mock travel requests just to get them accustomed to this. So we’ll see how this work. I’m quite excited about this prospect. I looked at the class roster the other day and I think I have a really cool crew of really capable people and as things gonna be great.

Rebecca: What are some challenges that you’ve run across teaching project-based classes and some advice that maybe you could give to a faculty who’s newer to this methodology?

Bastian: I would still consider myself new to this. I’m actually junior faculty so I’m only, in quotation marks, an assistant professor at this university for just about three years. But our department usually have four as project involve classes taught by more senior faculty. One of the most significant challenges that have experienced this when you have disruptive students. Every once in a while you have a student who completely hates the idea of projects and frankly I was one of them when I was in grad school, I was I was one of them because at the end of grad school I was like if I hear the word project one more time I’m going to flip out. These days I have a different opinion of this. I understand that some people are just fed up with it and I understand. Especially when they have to work with other people that they don’t know that don’t have the same work ethic that they do, they get frustrated a lot. So a recurring problem is student frustration with other students. That’s why I joke with them and say well this class is not about skill acquisition, I don’t need you to know how to compile code, at this point I expect you know how to do this. I need you to learn how to be successful despite other people in your group. You need to be successful despite the fact that you’re running out of time. That kind of stuff. So it takes a little bit of convincing sometimes but usually you’ll find the trick is to find an amicable solution. Then if there’s conflict between people then talk to both sides and say listen, I’m not your enemy, I’m not here to point fingers, I’m not here to agree with you or disagree with you, I’m here to help you facilitate a compromise. That is sometimes challenging. It happens every single semester, but it’s challenging. My strategy usually is to listen to both sides and say okay and maybe you just used the word, the wrong words, maybe you use the wrong language, maybe there’s cultural differences, you have students from other countries and they might not have the same work ethic that you do they may work 24/7 it feels like and you will really appreciate your weekends off. That is fine that is a fine, thing to do we just need to be upfront about it we just say, listen Jane, I’m not gonna work Sunday nights because Sunday night’s is when I relax. Or hey, I’m sorry Wayne, tomorrow morning 8 o’clock is the only time we can meet, can you somehow make it happen? So it’s really about compromise and it’s the case-by-case thing but my strategy is listen to them all and if they can’t make a decision on their own, then I make one, and they just have to abide by it. Usually it’s not a problem.

John: Which is also a useful job skill because they’re going to be in these environments.

Bastian: Exactly. In fact, when I say we simulate the way a software company develops software, I’m not joking. We really do it. These conflicts that you have in a class like this are literally the same. Most students really appreciate the experience, they may hate going through it but they usually love it at the end. In fact two years ago, I had a graduate student who was a graduate student of human-computer interaction, of which our University has a master’s program, but her background I believe it was art. She came from an art background.

Rebecca: Probably a graphic design student.

Bastian: Um, I’m not certain about that, but probably. The strength of the HCI graduate program is that it has so many people from so many different backgrounds, which is a great asset, and you can draw from really greatly talented people. Unfortunately, the downside is well these people they may have taken exactly one computer science class ,namely introduction to programming, and they have never done anything software, ever, ever again. But this person she hated going through this class she hated every single second of it but now she is working for a rather renown company here in upstate New York and she says I’m really experiencing this every day of my life, and I’m so thankful we went through this. This is the best worst class you’ll ever take in your entire life. It’s not about making students suffer of course it is about making them experience something in a realistic fashion, and tone it down a little bit. I don’t want to be the evil boss, I don’t want to be the guy who okay’s everything, and the truth is somewhere in the middle and usually that kind of pans out. Another really challenging thing though is when you have the disruptive student. Not just someone who’s fed up with projects or fed up with people in the project but actually tries to sabotage it. Not too long ago I had a student who was let’s say, extremely convinced of their own opinion, and this person, they were very sure of their own abilities. They were very keen on arguing they would argue everything until you’re blue in the face. They would misinterpret people stopping to argue because they just fed up with it, with oh they just conceded, I won the argument. So I had this person actually say, what everyone is praising me for my great ideas. I said well, sure, but you’ve done these three components that you’ve developed for this project, and your team has used none of them. Your team is no longer inviting you to team meetings, on my recommendation, because whenever you were at a team meeting they would not get anything done. So what do you think, what do you think this is, this is not okay, this isn’t an okay behavior. So in the end we found a way to help this person become useful after all, for the team, but it was very very challenging. In this particular semester I would think that unfortunately half of my teaching load was probably just taking care of this one person. Later I found out from other faculty that they were difficult in other classes also, so it wasn’t really me or the class, it was just personal issue. Even though this person took a lot of my time, ordinarily this class is the easiest to teach because, I don’t need to prepare anything, I have no preparation some grading afterwards but no preparation. On the other hand, you also have to be ready to face anything. You walk in a classroom and you don’t know what fresh hell awaits you that morning in terms of conflicts, but as I said, it’s only experienced as conflict while you were in it, afterwards you’re laughing it off and everyone is usually happy that it happened this way. So that’s what I’m saying is like a rewarding class to teach, but it’s kind of tough.

Rebecca: I imagine you probably have busy office hours as well with project based learning.

Bastian: Oh yeah. So much so that my faculty website says, office hours by appointment only. In reality it means, if I’m in, I’ll probably have time for you. Because with classes like these problems emerge right then and there, and I don’t mean interpersonal problems I mean, oh snap, we really need to use this one server but the server just went down. What do we do now? Or, we’re using this Google API and Google did what Google loves to do, namely change their API, what do we do now? Or, not too long ago, we were developing Facebook integration and Facebook from one day to the next took away the ability to post across pages on Facebook. So the project was kind of dead in the water, what do you do now? And that’s the problem that emerges immediately and you have to fix it, the students can’t fix it. When the resources that they need vanish, they can’t help themselves, there’s no way they can recover on their own. So that’s when after a short brief moment of panic, where I panic myself, we have to fix it somehow.

Rebecca: And you become the magic wand. [laughter] That’s what my students think when they’re standing in line for project-based learning. It’s like they come in it’s like, please I can’t move forward.

John: Those are all realistic type problems that they will be facing.

Bastian: It happens all the time happens to companies all the time, if you’re in the reality of the situation is Facebook doesn’t just take this away neither does Google. Google as opposed to, for example Oracle, they don’t really change their Java API all that much and if they do they have support for the things that you use to use,it call it deprecated, Google just switches it off. But they don’t do it from one day to the next there is usually a period where they tell you, oh by the way in a year or so we’re gonna switch over this in that server. So technically as a student you could be prepared if you did enough research but realistically, they have to complete this project, and our semesters are 15 weeks long, they have to complete this in 15 weeks so you have to make some concessions. Then we’ll just redefine the scope we just focus on something else. For example, a little while ago Google took away the opportunity of making your own google map, and when I say that is not a google map of let’s say, I don’t know, Oswego New York. Using the Google map engine, make a map of your bedroom, that’s what I meant. So they took away that opportunity or they took away certain functionalities that we wanted in one of our robot projects. I said well, they can’t do that so what I’m gonna do instead? One student suggested, hey, can we use the Unity engine to model a room that robot moves in? I said sure. Unity is a game engine to make video games. I said okay sure, you can do that, but I don’t know unity very well. Actually, I don’t know it at all. So, we have people here on campus who do know this, but I’m having a feeling to become good enough at unity to make this project work we’ll take another semester of itself. So why don’t you do it the easy way? Take a picture of the room that you want to use, and then “restorize” it and just fake it till you make it. So in the end the project was successful despite Google’s API being on.

John: What are some examples of specific topics that are used in design class?

Bastian: So in the software design process class, the first time I taught it here in Oswego, we did a family tree website, like those find your ancestry websites that you can find on the internet. Mainly because my Dad, he now passed away, but my dad was really into that and he wanted a website just to show our own family tree. We did that which was marginally successful. It was a decent family tree some of the features that we initially shot for were not delivered but, you know, we can safely say it was a family tree. A year after that we did an automated clicker system and I know that John here, is very much a proponent of using clickers and classrooms. If you have seen that millionaire quiz show on TV, they have little devices, and you can basically poll the audience in the classroom or in a question or multiple choice type answers. So we implemented it, and I’m of the firm opinion that no student should have to pay money or anything because tuition is already high enough, so we implemented a free one. That was using students own cell phones and wireless network they could poll.

John: You had some classes actually use it as clients for protocall.

Bastian: That’s right. So I used it in my own introduction to programming class. I used that semester, I used them as guinea pigs. They were excited beyond belief. They kind of liked it. It was very buggy of course mainly because doing it over wireless is really bad protocol. Plus if you have a wireless network in a large lecture hall it is an even worse protocol. So there were some problems with it that we couldn’t just solve, that were just unsolvable to us. But in principle, in a small enough audience, let’s say inside of 20 students, it would work great. Last semester was particularly exciting due to a scheduling error by, I’m not gonna say whom, but say by certain administrative forces, I unfortunately and accidentally had twice as many students in this class as I was supposed to. I like to teach this class with like between 15 and let’s say 25. Because we have a lot of students sometimes we have to unfortunately have 30 students in this class. Last semester I had 50, so yeah.

Rebecca: Oops.

Bastian: That was awful. But I decided after I talked to our department head, Doug Lea, and he says well, what you’re gonna do, pick up people and kick them out? We decided that this is a really evil thing to do to students so we just bit in the sour apple and said okay fine, let’s do a red team blue team approach. Where we had the same project and we split the class in half saying you’re team blue, you’re choosing a different design solution than team red. They both implemented a Scrabble clone. Those of you have played Scrabble board game, and we can use words and play words, and the idea is that people would walk by a kiosk system, which is actually running right now and the entrance of our science building here, is a computer in a display case. It’s running a cloned version of Scrabble. People can walk by with their cell phones connect to a little wireless that is emitted there and then they get a hand dealt on their cell phone, then they can play words. Of course they’d have the usual problems like, the first person that walks by plays an unspeakable word, so we made it Oswego themed and say if you play certain words you would get bonuses and such things. I would just mentioned in the coming semester I’m going to teach this class for the first time mainly online and I’m thinking about doing a Productivity type software. Something like it connects to your email account and looks for what your emails are actually about; how much time do you spend in your emails, how much time do you waste? For me, as faculty I always feel like I’m doing 5 % teaching, 3 % research, and 97 million % of miscellaneous administrative stuff, so mostly probably emails.

Rebecca: Mostly email. [laughter]

Bastian: I want to know if that’s true. I want to see what do my email say I am communicating about the most? On the one hand you have to connect to Google’s IMAP account and download emails and then you’d have to some natural language processing to parts of speech in the email and so on. Of course there gonna be privacy issues with this. These days everyone is really concerned about privacy, as they should, so we’re gonna have a little team that is gonna be specifically concerned with making sure that we abide by ISO 27000 privacy regulations. Unless the students have a better idea of course. [laughter]

John: So our last question is, what are you going to do next?

Bastian: I’m really excited. I had a student, I was successful in obtaining funding for a student project over the summer, and this student built an indoor GPS navigation system for robots. Now when I say that I mean mainly the API. So from this grant money we bought a little ultrasonic location beacons, you could say, which can be distributed around the room and the robot gets another location beacon slapped on top of itself, and then the robot knows in relationship to all the other beacons, where it is. Using this little system he implemented a GPS type API that allows us to say, robot go exactly there, and the robot will drive up to two centimeters precise to that position. The robot has obstacle avoidance, it has pathfinding capabilities, and all that stuff. So one of the things that I want to do next is have a fleet of those robots, we have several of those robots, but only one of them is location aware right now. When I put location awareness on several other robots and then simulate let’s say exoplanet exploration, using those little things. Let’s say you have three or four or five or 20 of those robots roaming around in a large room and one of them finds an obstacle and says, hey guys, here’s an obstacle don’t run into. It tells all the other robots where that obstacle is and then the next time when the next robot comes around, to a similar location, and says oh here’s an obstacle, here’s the question; is it the same obstacle? Because if it is, then we don’t have to put two obstacles on the conceptual map, we have to do just one. So it’s something I want to do it also ties into into my research. Like one of the things that I’m really, really focusing on is to make sure that the students just don’t do boring little projects. Every student in computer science has implemented a library system or an ATM, you know boring, been done before. I’ve worked, as I said earlier, in cyber-physical systems and safety-critical requirements and such things, so I use those ideas in my classes and I want them to solve tiny little projects therein. I just mentioned earlier, we bought these programmable slot cars. What I want to do next is do obstacle avoidance and automatic cruise controls with those slot cars and just automotive type software engineering projects. That’s what’s happening. I’m really excited about that too.

Rebecca: Great. Thanks for joining us today.

Bastian: Thank you for having me, I’ve really enjoyed being here.

John: You’re doing some really interesting things there.

Bastian: I’m not doing any of them. [laughter] The students are doing them. I’m just there for the ride, really. [Music]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on Teaforteaching.com. Music by Michael Gary Brewer. [Music]

32. The Three Little Pigs

What do the three little pigs, the big bad wolf, and dragons have to do with web design? More than you would think. Rebecca Mushtare discusses how a trip through fairy tales may open up the opportunity to develop empathy skills and conversations about race, disability and identity.

Allison Rank joins us again this week, this time as a guest host.

Show Notes

Transcript

John: What do the three little pigs, the big bad wolf, and dragons have to do with web design? More than you would think. In this episode, we’ll explore how a trip through fairy tales opens up the opportunity to develop empathy skills in conversations about race, disability, and identity.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer. Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Allison Rank, a frequent guest on this podcast, joins us today as guest host. Our guest today is Rebecca Mushtare who, until this episode, had been the co-host of this podcast.

Allison: Nobody panic. She’ll be back in this chair next week.

John: Today our teas are:

Allison: English breakfast tea.

Rebecca: What?!?

Allison: …under duress. I’m highly under caffeinated.

Rebecca: I’m drinking my normal English afternoon tea.

John: …and I’m drinking pineapple ginger green tea.
We invited you here today… because you’re always here… but we’re asking you…

Rebecca: …it’s a matter of convenience….

[LAUGHTER]

John: A year ago your daughter was born… now the three little pigs have invaded your class. Could you tell us a little bit about how the three little pigs made their way into your web design class?

Rebecca: I’ve been looking for ways to help students develop more empathy for their audiences, and it’s been a struggle. Students (or anybody who’s new to anything) will immediately try to make things for themselves, because it’s the audience they know best. So, it’s the easiest way. If you’re working on technical things or other concepts you don’t have to worry about audience too, because you have that part figured out. But, I’ve been really wanting to challenge students to dive into audience and also deal with accessibility issues which doesn’t come intuitively to them. So, the three little pigs actually offers a really great opportunity to have different audience members to think about (and audience members that don’t really exist); it becomes a safe zone. In this scenario, I’m using three titles as ethnographies for the students to read to get to know their audience better. I spent some time reading about ten different versions of the “three little pigs” and I’ve identified the best three. They are: The True Story of the Three Little Pigs by John Scieszka and Lane Smith, and The Three Pigs by David Wiesner, and There’s a Wolf at the Door: Five Classic Tales Retold by Z.B. Alley and R.W. Alley.
They read those books and then we come into class and I ask them to help me understand who all the characters are, what’s important to them, and some of their characteristics or qualities that we need to think about in terms of design… and then (from the perspective of the characters) what’s going on in the community that they live in… and the frame that I’m giving my students is that they’re in this community called Dragon Town. Dragon Town has a mayor named Mayor Melanie McDonald, and she’s human, but there are talking animals and dragons and other creatures that live in this community together and there’s a clear creature divide going on. So, the humans seem to value themselves more than the other critters in town. The poor pigs, they’ve got houses that are falling down. They don’t even up stand the Wolf’s breath. So, we’ve got some issues going on here.
The students read the stories, came to class, brainstormed about these characters, and helped identify some really big issues that were happening in Dragon Town… and then my challenge to them was, in teams of three or four, to identify one of those 10 that we identified as a class…choose one that they were gonna use a web design to help raise awareness of or to start to tackle. Obviously they’re not gonna solve these big problems, but they could make a dent into it.

John: The purpose then is to have students look at a problem from another perspective, from the perspective of the intended audience of the webpage, rather than using their own biases.

Rebecca: Yeah, exactly, and it’s something that they really need to practice… and so, yeah, this is a good opportunity to do that. They’re characters that their familiar with, but the books actually challenged a lot of their initial remembrances of some of the stories. So, it’s a nice way to get them to revisit that in a different way.

Allison: How was this different than how you’ve tried to approach the same topic in earlier iterations of the class?

Rebecca: In a previous episode, I think I talked about my simulated client project where I had these big company scenarios with the audience members being Oswego (the community that we live in) and they worked okay… but the students had trouble aligning themselves with older adults or middle-aged individuals who they just don’t seem to find relevant to themselves and even though these are individuals that are readily available in our community that you could interview and get to know, it was a struggle. We did a project in the fall, “The Voices of Oswego Veterans” project that we had a guest (Stephanie Pritchard) on who talked about that project… and we did a web project with that as well… and that was another way to deal with the audience. This time the audience was members of the Oswego community (the SUNY Oswego community), so they had a little bit easier access to that community… but the community that they were representing was different from themselves. These were students, so the population that they were addressing or talking about was student veterans, which was an identity that nobody in the class happened to identify with. That got us closest to solving the problem… but it wasn’t quite where I wanted them to be yet. What’s nice about this is that you don’t have to worry about offending anybody, because they’re not real.

Allison: I can imagine how the fictional characters are really helpful in terms of giving students a lot of space to play and a lot of leverage, but I have to imagine that there are some real challenges associated with giving them that amount of space as well. I guess I sort of have a gut reaction that thinks that they will make up things that cause problems in and of themselves. They’ve got enough rope to get in some dangerous positions. What are some of the challenges that you faced?

Rebecca: That’s a really good question. What I found was, they were willing to talk about things that they were never willing to talk about before. That, first of all, was a good space to be in. That was things like: “oh, there’s species profiling going on…,” “oh, there’s accessibility issues because pigs have hooves so they can’t type and tap on the computer screen…” …the accessibility issues that just bubble up. There was also the concern that critters were eating other neighbors, so we needed to start a campaign to be vegetarian, for example. So, there’s a lot of different things that came up…. a lot of social issues… another one was stranger danger… and then they did these presentations to the mayor, and it was important because we brought someone from outside in and I think that helped prevent some of the issues that you were identifying could bubble up as being a problem, but there was someone that wasn’t me who was the audience but I didn’t tell him who it was gonna be (it was just a grad student I bribed) who came in and just sat and played the part and asked questions and what have you…. and they were taking notes and then we went away and had a meeting and I came back with notes to the students about what the client was concerned about. So, that helped resolve some issues. But, you know, in the presentations there were some crazy things that happened… like the one on stranger danger, for example, the students had still indicated that the stranger, the bad character, was the wolf and the whole point was that all of the animals, and all of the creatures, and all of the humans, also have children and they all need to be concerned about strangers. That we shouldn’t associate one population as the bad actor. We ended up having to have a conversation about that. You can’t perpetuate these stereotypes, but what happened was we could have that conversation safely.

Allison: The familiarity played in the same way that a stereotype would traditionally function in class, but in a much safer space to have the conversation that resolves it.

Rebecca: Exactly. We were having crazy conversations about racial bias, and all these sorts of things, but under this guise of “it’s about the species” and the species problem that’s going on. And now all of a sudden it became safe. When that one group was having issues getting their head around it, I said to them: “You realize that this is the exact same thing as racial bias, right?” and they just looked at me with deer-in-the-headlights look. The next time they came back, the whole project was fixed.

Allison: That was actually gonna be my next question. At what point did you pull out from playing in the sort of allegorical space to say: “Hey, here’s what we actually just did” or did you let the experience and the skill building stand on its own?

Rebecca: I let things unfold organically, and I prodded and probed as necessary. I didn’t want any projects to perpetuate stereotypes or to perpetuate lack of accessibility… those two key issues. I probed and invaded their team time a lot with those particular things to push them on that, but you know they’re not perfect. But, I think they did a lot more growing in that area than they would have otherwise. What I think is missing, that I want to do next time is allow for more of that reflection at the end, so that they could apply it to some other projects. What I’m thinking about doing is have them present the work as if they were in an interview, and so how would you explain this project and what you learned from this project to a potential employer who has no idea what Dragon Town is, so that it becomes something that’s valid and useful… and I think that’s going to take some effort on their part to make that leap. But I think it’s actually a really good project for them to talk about in an interview and most employers would see the value in that.
I already have them do portfolio documentation. I already have them thinking about that, but I need to coach them through that process a little bit more…. and maybe actually make them present that.

John: Yeah, I could see an employer looking at a webpage making a case on avoiding inter-species consumption and being perhaps a little bit puzzled….

Rebecca: The tagline was “don’t eat your neighbor.”

John: Yes.

Rebecca:… which I thought was right on.

Allison: Yeah.

Rebecca: Well, and that group actually was interesting too because they wanted to do something that was: “Don’t eat your neighbor.” They wanted to be vegetarian but I was like, “Well, dragons have a big appetite. What are you gonna do for them?” So they came up with this tree salad or whatever that has just bigger things. They had to adapt the recipes and things like don’t forget there’s small kids. You got to think about these different populations, and they adjusted their content accordingly, to rise to that occasion. I also found this really great article about whether or not pigs are colorblind that I used as a doorway into thinking about accessibility issues. Apparently, I learned, pigs don’t perceive color the same way that humans do. They can’t perceive as many colors, so we have to really be concerned about the spectrum of colors and the kind of contrast that colors have… so that they would be accessible to pigs,,, but that led into conversations about maybe the pigs have to use voice activation because their hooves won’t let them type on their devices… and then we also had to talk about a mobile device for a dragon is pretty large…. so we had certainly some fun playful conversations, but they were really meaningful. We started talking about those issues pretty deeply in a way that I’ve never had in my class before.

John: Were the students more open to addressing these issues when it was in this safe zone or this safe space?

Rebecca: Yeah, even when I called that one group out on being stereotypical and perpetuating bias, they just received… and were like: “Oh, okay” and then you try it again… “is this better?” “My god, could you push it a little bit more?” and gave them some ideas about how they could push it… and our first solution wasn’t great after that…. It was to put in a separate monster that didn’t exist in this world as being the stranger, and then I identified that like when someone the other, we shouldn’t just assume that they’re the bad person or the bad creature. We had to be careful. I tried to call them out on whether or not we were using the word person, because it didn’t apply to dragons. So, it was funny [in] their presentations they were really conscious about things like that and trying to be inclusive in their language. So, yeah we ended up trying to tackle some of those things, and I was pretty impressed with how far they got… but it took some pushing. That one group took four or five tries before they had something that was gonna work.

John: How did students respond when you first gave them the assignment?

Rebecca: Well, I should probably provide a little setup in that my class includes design students, marketing students, and graduate students in HCI. So, it’s a fairly diverse population in and of itself in terms of disciplinary background. So there’s that. There are a number of people in the class who may not be traditionally artsy or creative, so it’s a little risky, right? I think I’m also known for being very serious. Which if you know me personally, that might not be true, but in the classroom students perceive me as being very serious… and the semester just was not going great, to be honest. It’s like something’s got to give, the students were struggling with a lot of the technical things, and so I basically threw the syllabus out or revised it significantly. stopped and did just technical exercises so students get comfortable with some of the things that they were really struggling with… and then one day I just showed up and said this is what we’re doing… and they had a ton of fun…. and were shocked… they’re just like “Is she serious? She lost it?” There was definitely those looks, but then there was a couple of key students who just jumped in and ran with it… and I think that really helped. So, I’m hoping that that will happen again. I think if everyone in the class is a little too serious, I don’t know that it would work.

Allison: Would you plan on sticking with, in the future, the three little pigs as sort of the through line story or it sounds like the story with the five different ways that the wolf is at your door? Does that give you some entree into some other storytelling avenues?

Rebecca: There is some entree into some other avenues and I maybe need to read some more fairy tales to be up on that, but the reason why I stuck with the Three Little Pigs is actually the wolf is the character that carries through all of them. So, that the five stories that are connected are all based on the wolf and different stories. So there’s Little Red Riding Hood, the Boy who Called Wolf, those are some of the stories in that other one. So, maybe there’d be some versions? I also happen to know that there was like the version of the Three Little Pigs told from the wolf’s point of view, so I really like that because it’s in direct conflict with the Three Little Pigs version of the story. I liked that the ethnographies that they were collecting were realistic in that they conflicted with one another, that they had to deal with the fact that there was conflicting information, and that they had to resolve that or deal with the fact that a wolf’s perspective was different than the pigs perspective of what the wolves perspective was… and I think that was a healthy messiness about it that worked pretty well… and the particular version of the Three Little Pigs that I used pigs escaped getting eaten by the wolf because they jump out of the storybook. So, there’s some plot twists in there that the students wouldn’t necessarily expect. It’s not a traditional version of the story… plus, they all have really great illustrations and they’re beautifully designed.

Allison: Are there other classes where you’d be interested in trying the same type of fictional ethnography technique?

Rebecca: I think it could work in some other scenarios, but I like this because it’s in my intro class. It’s a nice doorway in. What I’m really interested in seeing is, when I have a couple of these students in the advanced class next time, if that impacts their ability to do some actual real audience research and use that research in context. I think I want to monitor that first before doing some of this other work. I like it in particular because it’s a beginning class even though it’s at the 300 level.

John: It sounds like a really fun project, and there’s nothing really wrong with making learning fun.

Rebecca: Yeah, I had a good time and we had some moments where you had to really practice the deadpan look, you know, be really serious about what it is that we’re doing… and that part was really fun.

Allison: …and that seems like an amazing turnaround on a class where you have to scrap the syllabus halfway through a semester.

Rebecca: Yeah, it was amazing… the community that was formed around the project… and the way that they were exchanging with one another and coming together was incredible, and I was so thankful.

[LAUGHTER]
There’s nothing worse than an off semester and you just want out. I think everybody wanted out and so I just said “We’re out. We’re gonna try something new” and it worked, so that was good.

John: I guess the next question is: “what are you going to do next?”

Rebecca: That’s a good question… I think that with this project I’m hoping to expand it a little bit… so I’m currently thinking through “are there things that I can eliminate that I was doing before that I could embed in this project or I just allow them to have the time and space to fully build things out?” They have really good ideas and pretty good plans and the execution is almost there and I’d like to be able to have them have that time for the “almost there” to be “there” and then also to do that reflection piece that I kind of half-assed.

John: Okay, well thank you for joining us and I guess we’ll see you again on our next episode… and back as a host.

Rebecca: I mean, that is, if you’ll have me back.

[LAUGHTER]

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[Music]