206. U.S. Regulations for Online Classes

To be eligible for U.S. federal financial aid funding, colleges and universities offering distance learning programs must satisfy new federal regulations that went into effect in July 2020 and July 2021.  In this episode, Russell Poulin joins us to discuss how these requirements have changed and what these changes mean for faculty and institutions offering online classes.

Russ is the Executive Director of the WICHE Cooperative for Educational Technologies (WCET), and the Vice President for Technology Enhanced Education at the Western Interstate Commission for Higher Education.

Shownotes

Transcript

John: To be eligible for U.S. federal financial aid funding, colleges and universities offering distance learning programs must satisfy new federal regulations that went into effect in July 2020 and July 2021. In this episode, we examine how these requirements have changed and what these changes mean for faculty and institutions offering online classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners..

[MUSIC]

John: Judie Littlejohn is joining us as a guest host for this episode. Judie is the instructional designer for Genesee Community College, and has been a guest on several of our past episodes.

Judie: And our guest today is Russ Poulin. Russ is the Executive Director of the WICHE Cooperative for Educational Technologies (WCET), and the Vice President for Technology Enhanced Education at the Western Interstate Commission for Higher Education, which, of course is WICHE. Welcome, Russ.

Russell: Oh, it’s so great to be here. Thanks for asking me.

John: Our teas today are:… Russ, are you drinking tea?

Russell: Oh, yes, I’m drinking tea and I live in Longmont, Colorado, which is near Boulder and we have Celestial Seasonings here, and so, I’m drinking Earl Grey that has probiotics in it. That’s a new product for them.

Judie: So you have Earl Grey, but I’ve got Lady Grey black tea from Twinings.

Russell: Oh, very nice!

John: And I have pineapple ginger green tea from the Republic of Tea. We’ve invited you here to discuss the new federal regulations concerning regular and substantive interaction in distance-learning courses. These regulations went into effect in July 2021. We also would like to talk a little bit about the requirements for identity verification that went into effect a year earlier, but before we discuss this, could you tell us a little bit about WCET?

Russell: Oh, I’d love to, and thank you for asking. And so WCET is part of WICHE, the Western Interstate Commission for Higher Education, which is a regional higher education compact focused on the west, but back in 1989, they started WCET to focus on educational technologies, distance ed, online learning – which wasn’t a thing yet then – and, even from the start, we had other states that came in, and now we’ve grown and now we have, members through all 50 states and Canada and even Australia. And our members, are institutions and organizations and corporations interested in the use of distance ed.

Judie: How did WCET become the go-to source for information on regular and substantive interaction?

Russell: Well, we were hearing from members that there was some confusion about exactly what they were supposed to do with this, and that there wasn’t complete clear guidance from the US Department of Education and so Van Davis, who worked at Blackboard at that time, and now works for us here at WCET. And what we did was that we went through all of the guidance, there were some guidance that had been given, and also the findings against several institutions and then tried to put together what is it that they were looking for? What is it that is expected? And then we put together a blog post of our findings, and put that out there, and for like, four or five years, no matter what else we published, that ended up being the top blog post for the year because people were seeing that that was the only place where somebody had compiled this all together and knew what to do. And then in 2019, I was named to the Department of Education’s negotiated rulemaking subcommittee that worked on new rules, and there are people who actually get together and write these rules – you can imagine how exciting that is – and I was part of that and worked on it. And one of the issues that we worked on was this distance education issue and updating this regular and substantive interaction part of it and so you could say I was in the room where it happened! [LAUGHTER]

John: So basically, there was a gap out there that needed to be filled, and there was a lot of concern because of the sanctions that were placed against Western Governors University?

Russell: Yeah, it’d been unclear, and then also yeah, there was these findings against Western Governors University that their courses were all correspondence courses, and that they were expected to pay back – really, when you got to the end of it – it was a billion dollars which they didn’t have in their pockets at the time, and so it really caught the attention of a lot of people and this was a rule that was first written in 1992, had incomplete guidance. It was being administered in different ways and so it was time for a real update and more details about what is it exactly that they’re expecting? And so we worked on that, and we also were able to put in that there’s greater reliance in terms of the relationship between the accrediting agency and the institution, because there were times where it seemed like the auditors were sort of overruling that, and that doesn’t make sense because if you’re meeting what the accreditors want, why is this out of balance?

Judie: So what is needed, then, to differentiate a correspondence course from a credit-bearing online course?

Russell: So, it’s really good to go back and make that distinction and remind people that this is all about federal financial aid and what qualifies for federal financial aid. And they always want to get into, “Well… interaction, the academics, and the pedagogy of it.” Well, okay, put that aside [LAUGHTER] for the moment, we’re talking about, for federal purposes, what qualifies for federal aid and what does not? And the idea was that distance education is something that qualifies for full federal aid, whereas correspondence response education… that you might not get full aid for those courses or if you have enough correspondence students or enough correspondence courses… if over 50% of your students or courses are labeled as correspondence… then your institution is now ineligible for aid, and that’s what happened with Western Governors University. And so, it’s good to be clear, especially as we’re seeing more and more distance ed, hyflex, hybrid sorts of courses about what fits and what does not, and what activities work. We need to know what happens, so that we know that we’re in line, so that we keep our federal financial aid for students.

John: The technology for offering online courses has changed quite a bit since the initial regulations were developed. Under the current revisions, what is meant by interaction in that definition of regular and substantive interaction?

Russell: Yeah, that’s a good point that this rule was written in 1992, prior to online learning [LAUGHTER]. There’s been a lot of changes. But you really do need to parse out the parts of it, and so thanks for asking about the interaction part of it because back in the old definition, prior to July 1, that there’s this really odd thing that, the only thing that counted for interaction was something that was done by the instructor. So, providing a lecture, providing a video, providing an assessment or providing an assignment, any of those sorts of things. Those are the only things that counted, so our interaction here… “Yeah, you said something and I’m the instructor that didn’t count what you said” That’s weird. So, now, there are parts of it (freshly hidden in the regular part of the definition) that if you go down there, that you’ll see that there’s the expectation that you respond to students and student requests. So, at least we’ve made that move that the student being part of the interaction is now a part of what counts, and that’s the great move forward.

Judie: And what is expected for substantive interaction, like how do you define the substantive part?

Russell: So going back again to the old one, that it was all just about content, that if you’re teaching a history course, as one might, that you’re talking about what happened in a particular battle, or what happened in 1792, or those sorts of things, particular things about the course, and interactions about basketball or other extraneous things didn’t count. But under the new rule, we’ve gone to an activity-based notion of it, and I’m going to read these, so stay with me on this, and they’re very short. So one, one of the activities is providing direct instruction. Two, is assessing or providing feedback on coursework. Three is providing information or responding to questions about the content or competencies. Four is facilitating a group discussion. Five is other instructional activities, as approved by the accreditor, that’s a really wide open one. But for these that, if you have any type of online course, I sure hope that you’re assessing what the students are doing, I sure hope that you’re responding to them when they ask questions, and so, you’ve already got two of those right there. The one that’s really in question is the one about direct instruction, and…everyplace else they talk about synchronous or asynchronous, but I know through some things that we learned about the Department of Ed’s thinking of that, and we asked this question directly of them during a webinar about direct instruction. At that time, they said that it was only synchronous education, and sometimes people freak out when I say that, and we are asking for clarification on that in writing, but remember that it’s two of the five, so even if they define it as just synchronous instruction, you’re probably meeting at least three of those five already if you have any type of quality course.

John: One of the issues, I think, that you’re also seeking to resolve is whether synchronous online office hours would count. Has there been any feedback on that yet?

Russell: Yeah, it seemed to us quite clear about the office hours, that that is something that would count and it’s something that did not count in the past. And so, it’s an interesting change for that and one of the reasons that they did that was one of the groups I was representing was competency-based education, and so, we’re going to get into notion of regular here in a moment, you know, how do you define regular for something that, by definition, is irregular in terms of competency based? And it’s based upon student pacing over the faculty pacing, and so, there’s a nod to that in terms of if you have regular office hours and have that posted on the syllabus or somewhere that that would count. And we’ve had some pushback from our financial aid friends, because that’s new to them. They said, “Well, that’s never counted before.” And so again, this is something where we’ve asked the question of the department and hope to get that in writing yet again so that that’s reaffirmed and everybody’s under the same understanding for that one.

Judie: You started to talk about regular, so how do you define “regular” interaction?

Russell: So regular? Yeah, as I was alluding to, that was the hardest one to do, because remember, we’re writing these for, depending on how you count, four to six thousand institutions. Some have short courses, some have competency-based, you think of every variation that you have out there. So saying “meeting once a week,” or doing something once a week just didn’t work because it didn’t fit all those different ones, and once a week would not be enough in a five- or six-week course, that’d be too little. So, we tried a formula and that was a disaster. So anyway, so we have these words, and then there’s a lot more, again, back to the relationship with the accreditor on this and what works. So, these are a little bit vague, but you need to work with your accreditor and how they’re defining these. So there’s really two parts to this. The first is going to be predictable and scheduled, and so this is something where either you have it in your syllabus, and then you have the syllabus at the start and here’s when things are going to happen, and then with that, that you may actually have a two- or three-week break where an instructor is not putting things out or you don’t have interactions, because it makes academic sense that you have the people out doing a paper or group work for two to three weeks and doing that. So that once a week would actually not work there, right? Or the predictable part of it was that maybe it’s not exactly in the syllabus, but what you do is you say, “Every Wednesday, we’re gonna meet or have office hours, or we’ll do something at webinar times.” So something predictable. So, that’s the first part of it. The second part of it was about monitoring the students’ academic engagement. And that was something we really brought in with competency-based education in mind, where what you’re doing is that you are actively following the student and making sure that they’re not out there on their own, and that you’re making sure that the student’s not floating and that you’re seeing that, “Okay, do they need interaction? Do they need some intervention?” Or the second part of that is what I alluded to before, or that the student says, “Okay, I need help with this,” or “I’m ready to move on to the next part of my competencies.” So that’s the other part of it is the bringing that together in terms of monitoring the engagement, so something that’s predictable, scheduled, and then also, outside of that, that you’re actually monitoring and interacting with the student.

John: Are there any requirements concerning the extent to which there should be interactions with individual students as compared to interactions with the class as a whole?

Russell: Well, both of them count. So if you’re giving a lecture, or if you’re doing a group discussion and doing that, that that’s a group thing, and so that counts. And also, remember, the second part of the regular was that you are responding to student requests, and so that’s on an individual basis. And when there is a federal financial aid review, or ”audit” as it’s called, of your institution, what they’ll do is they’ll go and take a sample of classes, and that they’ll look to see what happened in that course? Were there group interactions? Were there individual interactions? That they’ll look to see what happened and then they’ll look to see, does it meet the regulations? And have you developed faculty? Have you let them know that these are the expectations of them? That they’re looking for those sorts of things, and did it actually have an effect in the courses?

Judie: So what would be some examples of regular and substantive interaction that we could build into a course?

Russell: Yeah, that’s a really good question. You’re an instructional designer, you plan the whole course out, right? And you’re going to get more points on the regular side for the predictable and scheduled if you have a syllabus, that… it doesn’t have to be detailed down to every last thing that you’re going to do… but at least you’re showing that what are the expectations along the way? That, when are you going to have assessments? That, if you’re – let’s say that for some courses – that you’ll send out a video with a different lesson every Monday or every Friday, what you’re doing is that you’re hitting the marks on the regular, another is that you are showing that you have some expectations, in terms of the feedback and the feedback loop. And sometimes institutions have this as a policy, sometimes they leave it to the faculty member. But the more classes that you have where you show that the faculty person… you don’t have to respond immediately… but they’re showing that they will respond in 72 hours, not counting a weekend… that they will respond to the students. And there’s that expectation that they will do that because I had a question from one, that they had a faculty member that they put the discussion out there at the start of semester and then graded it at the end of the semester [LAUGHTER]. And first of all, that’s terrible teaching practice, let’s start with that, nevermind the regulations, but that faculty person thought it was good. And the other is, that there’s this expectation that, as you go along that you’re working with the students. So those are some examples of things that you’d want to do there.

Judie: So just a couple of different things that I like to try here is, I really like to try to encourage faculty to give feedback prior to the next due date. That just makes sense, so that if a student is making an error, they’re not going to repeat the same error because they haven’t gotten their feedback yet. So I just think, pedagogically, it makes sense to give the feedback as soon as possible. But I also like to just have faculty create a communication plan when they’re developing their course. I think faculty plan in their heads, you know, “I’m going to send this announcement, I’ll do this feedback, I’ll do X, Y, and Z.” But when they sit down and really map it out in a communication plan for their course, I think that really helps get into that regular schedule. And whether it’s date driven, or day-of-the-week driven or at specific points throughout the course when students reach different milestones, I think that really helps them. My understanding is that those types of things would help people meet the regular and substantive interaction definition. What do you think of that?

Russell: I love both of those ideas, and actually that, really, if you work with the instructional designers, look at what works in terms of good pedagogy, that these are the things that you’re talking about is that having a plan ahead of time and being open with the students so that they know, and then getting back to the students in between assessments so they have the feedback in terms of, they know where they might be falling behind a little bit in some areas and so that they know, “Oh, I don’t quite get that concept. And then, I have a math background and that was so key in mathematics that, if you don’t get this one, you’re not going to do any better on the next test right? And that’s probably true in so many different fields as well. And so, I love both of those ideas in terms of doing things and where you’re informing the student, and then keeping them engaged and then constantly moving them forward.

Judie: It’s great to hear that kind of feedback from you, thanks! [LAUGHTER] I hear a lot from faculty now, especially during COVID, when many are teaching in Zoom. And so they’ll record a live lecture with their students and, assuming that FERPA rules are followed, and there’s no students caught in the video or audio, they want to just show that recording again in the next semester, and want to know if they’re meeting regular and substantive interaction that way. And I tell them that, when they’re giving their lecture, that is regular and substantive interaction when they’re engaged with their live students. But I say that once you make a recording, and put it in the course, it becomes course content, because it’s no longer a unique experience with those individuals talking about their understanding of the course content. And I see like a real fine line there and I wonder what you think of that? Or how that might be interpreted? What do you think of using old recordings versus always expecting some sort of fresh and unique interaction with the students?

Russell: Yeah, I think if that’s all that you did, I think that you’d have a hard time in terms of meeting the regular and substantive interaction, and this is the one where we get back to the direct instruction question on that one, and we did gather that question plus several others and pose them to the Department of Education, because we felt that even since they released the rule that we were hearing different things from them than from the accreditors. And so I’m a little hesitant to give you a yes, that works or not, under the new rules or not until we get a better answer from the Department. My feeling was that, under the old rules that I felt a little bit better about that that probably was problematic. It might still be, but I’m kind of curious to see what they say about the synchronous versus asynchronous going forward. I think that, if that’s all that you relied on, I think that that’d be problematic that you would need to have other sorts of interactions that might make that work.

Judie: Thanks.

John: But videos that were custom created for that week’s activities, or that provide feedback for the class would count, right?

Russell: Yeah, the ones that are custom created.

Judie: I encourage them to make like small targeted videos for clarification. Like, to address a specific topic that they know that students struggle with, as opposed to just making an hour video of you standing at the front of the classroom, talking to people that future students don’t even know.

Russell: Yeah, and I think if you just use the video over and over again, and I certainly saw this in some engineering courses where they’re using the same ones that, what happens is that you have to update your materials every once in a while, too [LAUGHTER].

Judie: Oh, sure

Russell: You need to be doing that. And so I remember witnessing a course where they were falling behind on some facts or raised a lot of questions about advances that had happened after the video had happened, or were quite clearly dated, because they were talking about things that were going on in space as a future thing, instead of a past thing. And so I think, if nothing else, that you’re going beyond whether you meet these rules or not, that you’re diminishing confidence of students in terms of the value of what they’re receiving.

Judie: Sure, that’s a good point.

Russell: One other issue that we didn’t touch on so far had to do with the definition of an instructor, and that was a difficult one for us. And that was another one that Western Governors University got hit on this one. It was a finding against them. And that it seemed like some of the definitions meant like an instructor, that there was one person that was in front, and that a lot of institutions have gone to team teaching or bundled instruction, or using GAs or TAs or there’s several people in the course and with the unbundled instruction that WGU did that they had one content expert providing the content of the course and another one doing the assessment and maybe somebody else doing some of the advising. So you broke it up and that they weren’t counting that even though it was approved by their accrediting agency. So that is one where we have worked on “instructor” and we’re very clear in this, that it is what is approved by the accrediting agency that that is what counts. T hat was sort of alluded to before, now it’s very clear. And so if you have a non-traditional sort of model for your instructors or faculties, you may want to talk to your accrediting agency about how they view that and get something in writing about that.

John: I think that is a pretty common issue where there’s often a master course developed by the content expert and then again, there are instructional teams that work on the whole course, but the division there can vary quite a bit. And I know there’s a lot of interest in institutions in trying to scale online education to make it more efficient, and this is an area that certainly needs to be addressed with the accrediting agencies.

Russell: Definitely, definitely, yeah.

Judie: So these rules that we’ve been talking about also addressed student identity verification. So is the student identity verification related to the regular and substantive interaction? Or is this another area that requires a more precise definition?

Russell: This is actually a whole other area that was in a different part of the regulations, and this is one that actually went into effect in July of 2020, and it’s part of what the accrediting rules are, and there’s a whole list of things that the accrediting agencies are supposed to be looking for when they’re doing your accrediting reviews. One of the things that they’re supposed to be doing is making sure that the institution has, really, policies and processes to make sure that the student who enrolls in the course is the same one who’s completing and submitting the assignments in there. So it really is about academic cheating, and that this is only in the distance-ed world that they have to do this. I have to tell you, in a subcommittee, we tried to expand it and got beat back. So, sorry we lost that fight for you. But it’s still in distance ed where the accreditors are expected to check for that to make sure. And then the big change that happened in that is that, previously, there were some, what were considered ‘“examples” in there. And one was that you had some sort of ID, some sort of login ID for that or that you did proctoring, and that those were meant as examples. And those were taken out, because all too often, what would happen is that an institution would say, “Well, we have an ID!” and they would do nothing else. And so that was clearly insufficient in terms of doing it, and so, they’re raising the bar. The intent is that you have a plan, and that you’re executing the plan, and that you’ve worked that out with your accreditor, and that when that financial aid review happens, that you will be able to demonstrate what you’re doing and if it’s effective. Another part of it, there’s a second section to it, that also talks about that if you have additional costs, and so, let’s say that you’re using a proctoring software, and that that costs so much per student, that you have to notify the student at time of registration, that there’s an extra cost for that. And this is something that a lot of institutions have fallen short on, because what they’ll do is that they’ll notify the student in the syllabus, and so the first day that the student starts they see the syllabus, and all of a sudden they have to pay more money. And the idea is that the student should be able to have a choice at the time they’re picking between which course that they might take or know that they’re going to have an additional cost for participating in that course. And that rule is out there, and it’s a good one, because you’re being clearer to the consumer about what’s going on.

John: So authentication with a password to a course management system is not sufficient, and some type of proctoring software is, but there’s a lot of concerns raised with proctoring. Are there any other ways to authenticate students that meet the requirements without moving to software proctoring solutions?

Russell: Yeah, and I think that over the last year that we’ve seen, the concerns about proctoring software have risen to the fore, and there’s some good ones and that they do some good things, and so you should not throw them all out. But yet, you should pay attention to the concerns about that. But there are ways that we have worked on this in terms of different ways that faculty can work in terms of their assessments. One of the things we talked about is face-to-face proctoring, As distance ed grows, though, that gets to be harder and harder to find enough proctoring sites and the ability to do that, but that is an option. Some of the other things that have been proposed have to do with how you do assessments, and that having more frequent assessments and doing things where it’s easier to take the course than it is to cheat. That if you do like one or two big assessments per term, that it’s a lot easier to get someone to do those for you or the big papers and all that, so that’s one. There’s others where getting people involved in terms of group coursework, or other sorts of authentic assessment type of things, where you get involved in different sorts of things, where you have to stay engaged more and more often through the course, and it’s harder to get somebody else to do that for you. There are people who will take the whole course for you… that’s a problem. But the more barriers that you can put up, and we really do love our instructional designers, but the more that we can do to help faculty with thinking about assessment strategies and effective assessment strategies, the better off that will be with all of this. Aso there are other areas like… oh shoot, I’m blanking on… ICAI, they have a lot of strategies as well out there, and for some reason, it’s early in the morning here and I’m blanking on their name, but there’s an institute for academic integrity that has a lot of good resources on this issue.

Judie: Yeah, I think it’s great to encourage all faculty to work with their instructional designers on authentic assessment.

Russell: Yes, yes, yes! That success will be more sure if you work with your instructional designer.

Judie: So, do you think that this authentication concern is only for assessments? Or is it for day-to-day coursework and interactions, too?

Russell: Yeah, it’s really about any type of quiz or paper or anything that you’re going to be evaluating the student on. That’s really what it’s looking at, because there’s the opportunity for cheating or something bad to happen there. So there are other things that you do in terms of papers, you know, with Turnitin or other sorts of activities that people do or trying to create papers that are more authentic or real: “Write something about your hometown or work on a project in your hometown where there’s not a lot of papers.” [LAUGHTER] So, you can do those sorts of things where it’s harder to plagiarize.

John: It’s really nice to hear that open pedagogy projects, videos that students create, where they’re actively engaged in it, group projects, and all those things can serve the same role without moving to the extreme of proctoring.

Russell: Yes.

John: It is good to note that any courses that require proctoring must list that up front so that students are aware of the cost. And the other issue with that is, as colleges enroll more first-gen students and more students from the lower income quintile, many students won’t have computers or networks that will necessarily support proctoring software. If students are working on their course through their smartphones, most proctoring solutions don’t work with smartphones. But it’s pretty easy for the student to take a video of themselves talking about something, so allowing faculty to have more options for authentication is something that allows for a more inclusive learning environment.

Russell: That was a huge lesson from the move to remote learning due to the pandemic, that you had so many students who did not plan to be in a remote course that uses online tools… that they were using, as you said, cell phones or different types of tablets that were not compatible with some of these proctoring software solutions, or that they didn’t have the adequate bandwidth for taking the test in that way with a full video… that that was a real problem, privacy issues with it. So there were all sorts of things that were problematic with that. And so, being upfront with the students is very good and we do often cite that there wasn’t a federal finding, but it was a student in Nevada, who was in a course and they were not notified that they were going to be using proctoring software. The student was pre-law, decided to flex his pre-law muscles, and got a whole bunch of students behind him and took it to the institution and ultimately went to the Board of Regents there, and all of those students had all of their fees repaid. It wasn’t a federal finding, but what was shown was that they were out of compliance with the federal rule, and the Board of Regents decided to remedy that.

Judie: So looking at regulations like this, we can see that online classes are held to a higher standard than face-to-face classes. Do you think similar requirements should also be implemented for face-to-face classes?

Russell: Well, I think what’s happening is that the vast majority of courses are now digital courses. And whether it is fully online, whether it’s hybrid, hyflex, blended, or just the old term that the OLC used to use of “web enhanced,” that you use the web a lot, even though that you meet face to face Monday-Wednesday-Friday, that we’re seeing all these digital tools going throughout. And what happened with the pandemic? It really took off, right? And so there’s even more of that is gonna happen. And some surveys that one of our organizations, Every Learner Everywhere did, that there was more interest and uptake from faculty in terms of, “Well, I’ve done this now I should do it again.” And so I think what’s happened is that we’re gonna see more and more use of digital technologies through, if not every course, the vast majority of courses. Well, the thing that happens is that these same sorts of problems are in all of these. And you and I know, keep this as our secret between us [LAUGHTER] but, all this stuff was happening even without technologies, right?

Judie: Right.

Russell: A lot of the cheating scandals in some of the service academies in the last few years had very little to do with technologies other than sharing some information. So we know that’s going on. So it’s gonna be interesting. This is one of the things that we’re looking at that I bet we’re going to see a lot of new guidance coming out of this Department of Education that recognizes that and may expand this out quite a bit more, because there are people in the department who have huge concerns about consumer protection issues and the use of online or digital learning regardless of where it’s used, and that they’re seeing that some of these rules need to be applied more broadly. So I wouldn’t be surprised to see that we get guidance that says exactly that coming out and that we have James Kvaal was just finally approved as the Undersecretary there. I think that that staff has been working on these sorts of things, and have been waiting for him to be approved. I would not be surprised that in the next three to six months that we don’t start seeing some new guidance coming out or answers to our questions. I don’t think they wanted to answer our questions until James Kvaal was in. And so I think that we’ll see clarifications and guidance about some of these things where we’ve had questions before, and how do they apply in a hyflex setting? How do they apply in a blended setting?

Judie: That’s good news.

John: And whether the rules are expanded or not, they’re just good practice… that regular and substantive interaction is good pedagogy.

RUSSEL: Yes! [LAUGHTER] Yeah, you really nailed it with that. And that was one of the things that surely the people that run the subcommittee and then the main committee were trying to look at: “What do we do that makes sense in terms of best serving the students?” And we have to remember at the end that these are consumer-protection practices that have to do that, there are also federal-financial-aid protections that aid is going to worthy activities, and so we need to remember that in all this.

John: We always end with the question: “What’s next?”

Russell: Well, I kind of previewed thata bit, that I really do believe that we’re likely to see several more clarifications coming out, or maybe some surprises coming out in the next few months. And so I think it would be good for people to pay attention to what’s going on, and we certainly write about whatever comes out in our WCET Frontiers blogs. So, be watching for that. And on something completely different, that we’re getting together some folks to work on the issue about veterans and their housing allowance. And just quickly on that, that veterans who take all their courses online, get about half or a little bit more of the housing allowance of veterans who take just one course face to face… they could have it all online, but just one course face to face. And it really is antiquated thinking, and it’s something that we need to get fixed. Because, I could be the same student in one term, take all my courses online, and the next term take just one course face to face. I have the same housing cost, I have the same family [LAUGHTER] I still need to eat, but somehow my aid is less. And so we’re working on that one.

John: And you shared many resources with us that we’ll include in the show notes, so those will be available on the website. Well, thank you for joining us, this was really helpful, and I think it’s going to benefit a lot of institutions and a lot of faculty and instructional designers as they plan for future semesters.

Russell: Well, it was a great pleasure being here with you today, and having a little bit of tea in the morning is always good. And so thank you, Judie, thank you, John for inviting me and for having me here.

Judie: Yes, thank you. This was fun. Take care.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

186. Super Courses

Students often see our classes as boxes that they need to check in order to graduate. By reframing our courses around fascinating big questions that students can connect with, we can help our students recognize the value of these learning experiences. In this episode, Ken Bain joins us to explore examples of courses that do this well.

Ken is an award winning teacher, the founder of the teaching centers at Northwestern, New York, and Vanderbilt Universities. He is the author of two very influential prior books, What the Best College Teachers Do and What the Best College Students Do. His newest, Super Courses, was released in March 2021

Shownotes

  • Bain, K. (2004). What the Best College Teachers Do. Harvard University Press.
  • Bain, K. (2012). What the Best College Students Do. Harvard University Press.
  • Bain, K. (2021). Super Courses: The Future of Teaching and Learning. Princeton University Press.
  • Andrew David Kaugman, Books Behind Bars
  • Deci, E. L. (1972). Intrinsic motivation, extrinsic reinforcement, and inequity. Journal of personality and social psychology, 22(1), 113.
  • Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of educational research, 71(1), 1-27.
  • Perusall
  • Hypothes.is

Transcript

John: Students often see our classes as boxes that they need to check in order to graduate. By reframing our courses around fascinating big questions that students can connect with, we can help our students recognize the value of these learning experiences. In this episode, we explore examples of courses that do this well.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Ken Bain. Ken is an award winning teacher, the founder of the teaching centers at Northwestern, New York, and Vanderbilt Universities. He is the author of two very influential prior books, What the Best College Teachers Do and What the Best College Students Do. His newest, Super Courses, was released in March 2021. Welcome, Ken.

Ken: Thank you very much. Glad to be here.

John: We’re really glad to talk to you. You visited Oswego a few years back and people are still talking about your visit.

Ken: Oh, wonderful. I had a wonderful visit.

John: Our teas today are… Are you drinking tea?

Ken: No, my doctor won’t let me do that, and I haven’t had a good cup of tea in… oh my goodness… many, many years.

Rebecca: Oh, that would make me so sad.

Ken: Yes, indeed, it does. Me too. I can’t drink tea… anything that has caffeine in it.

Rebecca: Ah, total bummer.

Ken: Yes, it is. [LAUGHTER]

Rebecca: I have about the strongest caffeinated Irish breakfast tea you can have. [LAUGHTER]

Ken: Oh, my goodness. Well, the last cup of coffee that I had was in 2002. I remember the date. That’s because it….

Rebecca: Oh, no…

Ken: …a traumatic experience, to go cold turkey.

John: Actually, that’s how I started drinking more tea. I had to cut out caffeine, so I started drinking herbal tea.

Ken: Well, I do drink herbal tea from time to time. I just don’t happen to have a cup right now.

John: I am drinking Tea Forte black currant tea today.

Ken: Oh, wonderful.

John: It’s really good.

Rebecca: So we’ve invited you here today to discuss Super Courses. Could you tell us a little bit about the origin of this book project?

Ken: Well, my wife and longtime collaborator, Marsha Marshall Bain, suggested that we do a course around the Invitational syllabus, as we’ve come to call it, what we used to call the promising syllabus. And we began collecting those syllabi from around the world and began looking at them. And in the midst of that endeavor. Peter Dougherty, who is the longtime director of the Princeton University Press, contacted us one day and said, “Would you come down to Princeton, and I’ll buy you lunch?” So that sounded like a great invitation. And we went down. And he asked us what we were working on. And I told him and he said, “Oh, you’re looking for super courses.” And that triggered a whole avalanche Of reconsiderations of what we were doing, and we shifted the Invitational project over to the super course project, and began looking for courses that offered what we had been calling a natural critical learning environment. And we began that project back in, I guess, late 2007.

John: Maybe before we talk about your new book, you can talk just a little bit about the concept of an Invitational syllabus, since that was the origin of this project.

Ken: Oh, sure. It’s the idea of inviting your students into the class, rather than requiring them to come. And rather than focusing upon topics, it focuses upon big and enticing questions, so that the Invitational syllabus begins with an intriguing question, an important question, a beautiful question that students find so enticing, that they say, “I want to be a part of that.” And it becomes a self-motivating experience. So that’s part of what we meant by a natural critical learning environment, is the creation of that self-motivating experience where students would pursue things, not because someone was threatening them with a bad grade, or because they were just looking for credit, but because they became deeply interested in the question.

Rebecca: Can you expand a little bit upon the idea of the natural critical learning environment beyond just the Invitational syllabus?

Ken: Sure, we now have identified, oh, I guess, about 20 some odd elements of what we call a natural critical learning environment. And the first and most foundational of those elements is that it’s organized around those intriguing questions. And its intention is to foster what the literature calls deep learning, that is learning in which students think about implications and applications of what they’re learning and the possibilities of what they’re learning. It’s learning where students look behind the words on a page and think about all of those implications and applications and possibilities and how things are connected to each other. So that’s the foundational element and the chief goal of the natural critical learning environment is to create an atmosphere where students can, and will likely, pursue that deep approach to learning and they develop what we call deep intentions to learn. But, how do they do that? How do we get them to that point? So, what is the natural critical learning environment? Well, it’s an environment where they can try, they can come up short, and get feedback, and try again, without penalty, without any kind of situation where they are punished for coming up short. In other words, if you think about it, it’s the kind of learning environment that we expect, as scientists and as scholars. We try out things, and if they don’t work, if the data doesn’t confirm our hypothesis, we modify it and try again. And we’d be terribly insulted if our first effort out of the box was… and people would say, “That’s nonsense. That’s crazy. Go away.” We try things, get feedback, and try again, and so that’s what the natural critical learning environment does. It’s also an environment where students can work with each other. People learn in community arrangements, where they work with each other to grapple with the problems. And they learn… and this is another key element of the natural critical learning environment… they learn by doing. Sometimes that means learn by teaching. And by teaching, we don’t mean necessarily that they stand in front of a mirror and deliver lectures. In fact, the teaching that they develop often doesn’t even include lectures, it includes a way of fostering very deep learning on the part of other people by creating dialogues, creating exchanges around big questions that move students toward a deeper understanding and a deeper application.

John: Going back to that question of the big questions, because that’s an important part of the approach. I think you talk about that both at the level of the course as a whole in the Invitational syllabus, but also when you’re devising individual components of your course. Could you elaborate a little bit on what faculty should think about when trying to select those questions?

Ken: Yeah. And it’s more than selecting them. It’s framing them, and framing them in a way that will intrigue students. Now, some of the best super courses we came across were questions that sometimes began with questions that were much larger than the course and much larger than the discipline. But in the course of students pursuing those big questions, they discover that “Well, I need to learn chemistry to answer this question,” or “I need to learn history,” or maybe “I need to learn both,” because many of the super courses were multi-disciplinary, built around a big and complex and interesting fascinating question. And then the students would devise ways of trying to answer that question. And the professor would build an environment where they could progressively tackle those questions. They can run from the very simple to the very complex, one of my favorites, and one that I’ve talked about so much, and actually written about, going all the way back 10, 15 years ago, is one that we do mention, briefly, in this new book, but it’s joined by other really exciting examples. First, that old example, it comes from 2006. And there was a professor at Princeton at the time, he was a political historian and political scientist, and who wanted the students to examine the impact of that period we call reconstruction, in period from roughly 1865 to 1877, and to ask themselves, what kind of impact did that period have on subsequent political and social developments and political institutions? Now, as a historian, that’s a very intriguing question to me. [LAUGHTER] But I don’t think you will find many undergraduates who are just dying to pursue that question. So she didn’t ask that question initially. Instead, she built a course around a question that she knew was already on the minds of her students. Now you think about what was on the minds of students in the fall of 20006? …A big question. When I ask American teachers these days, they often can’t remember. [LAUGHTER] I recently did a workshop in China, and in far southwestern China, and they got it immediately. They remembered. But, the question was, basically, “What in the world happened with that disaster we call Katrina?” Now, there’s a lot of evidence that that question became the dominant question in American politics in 2006, 2008, and helped determine the outcome of the election in 2008. You look at Mr. Bush’s numbers of approval, they fell off the cliff after Katrina. So what caused that disaster? So she organized a course, she called “Disaster: Katrina and American Politics.” And students signed up immediately. It became an extraordinarily popular course. Well, how do you get from there to an examination of political history? Well, it happened on the first day. She went into the class, and the first question she asked her students was, “When did the disaster begin? Did it begin in August of 2005, when the storm surge hit New Orleans? Or did it begin in 1866 with the beginning of reconstruction in the Crescent City?” And with that question, she transformed their interest into her interest, and it became the driving push of the whole course. But let me give you another broader example of the book, A guy by the name of Andrew David Kaufman, who teaches at the University of Virginia, about a dozen years ago, organized a program he calls Books Behind Bars. His field is Russian literature, late 19th and early 20th century Russian literature. And that literature is quite famous for asking big questions, questions about: “What’s my purpose in life? What’s my destiny in life?” So what he does in this course, is the help students go into a maximum security correctional facility for young people, people the same age as the UVA students in the course. When they go into that prison, and they help those other young people confront those questions, by reading Tolstoy, by reading Turgenev, raising the questions, and then struggling with them in a class that they do for them once a week, ensure they learn Russian literature, by teaching Russian literature, and not by lecture, but by creating an environment, a natural critical learning environment, where their students, the residents in the correctional facility, will learn just as deeply as they will. And it’s a transformative experience, for both sides, and it changes lives, and it’s self motivating. That makes sense?

Rebecca: Yeah, these are really powerful examples. And I love that both of them have really strong ledes with the course title.

Ken: Yeah. And if you’ll notice also that both have appeal to a sense of altruism. And we discovered that many of these super courses do just that, even in fields like physics and engineering. They do things to help other people. One of our favorites is a course that some high school girls in a high school in northwest Los Angeles, developed for themselves. And the only help they had was they were invited into this program and invited to come up with a project. And they live in a relatively poor neighborhood. They said, “Well, the biggest problem in our neighborhood is homelessness. And we see the homeless out on the street and in the park and under interstate 5 that runs near the high school. And what we want to do is we want to create a portable tent that is solar powered, so they will have heat and the cooking facilities and light and so on and so forth in their tent.” Now to do that, they had to learn engineering. So they organized their own courses, they organized their own sequence of topics that they would pursue. Now they have some guidance. The teachers over there kind of giving them hints or answering questions: “Should we pursue this next?” But they learned everything from electrical engineering to programming, and lots of things in between. But they also learned just the basics of being an engineer. That’s transformative. They created it, and therefore they took ownership of it. Now, the super courses, and the super institutions that we studied, immerse a lot of what they do in the research on human motivation, research pioneered by Edward Deci and Richard Ryan. And they argue, in their well documented research papers, that human beings have three basic needs and that if you meet those psychological needs three basic psychological needs… we have physical needs that go beyond these… the three basic psychological needs. And if you meet those psychological needs, people are just naturally motivated to try to learn. You don’t have to stimulate it, it just occurs naturally. And the problem is often that the way we set up schooling for people doesn’t meet those needs, it actually counters those needs. And so we get classes full of uninterested students, students who are signing off and not really becoming involved. And to address that situation, many of these courses deliberately use Deci and Ryan’s work to build an environment, where, what shall I say, where people are just naturally driven to do what they need to do. Those three needs, by the way, are: a need for autonomy, that is, we like to be in charge of our own lives. We don’t like teachers being in control of our learning. We want teachers to help us with it, that’s different, but not to control that. And beyond autonomy, there’s also a sense of competence. So people, if they feel like if they don’t know something, they can learn it. And they feel that what Carol Dweck calls a growth mindset, that mindset that says, “I may not know this, yet, but I can learn it. I may not know calculus, yet, but I can learn it.” While the person with the fixed mindset says, “I’m not a writing person,” “I’m not a computer person,” “I’m not a whatever person,” and they give up. And they don’t try to learn and to push the envelope. And then finally, it’s the sense of relatedness, people like to be part of a broader effort, and an effort in the super courses that’s often larger than the classes itself, larger than the discipline… that they take on these large projects, because they believe that it can make a difference for themselves, and for other people whose lives they will affect.

John: One of the things that’s challenging, though, for a lot of faculty, is that we do have to assign grades for all of this. So we know ultimately, students are going to get these grades. And that tends to lead to more of a reliance on extrinsic motivation. What can faculty do to provide that sort of encouragement and to help create a growth mindset when students are going to struggle with some of the material at first?

Ken: Yeah, it’s to give them lots of opportunities to try, fail, receive feedback, and try again. Now, that seems really daunting to many faculty members. They say, “Now look, I have to two hundred students, I can’t do that, for all of the students.” But there are ways of doing that. And that’s one of the things that we explore in the book. It is difficult to describe, so I won’t attempt to do so in the conversations here. But, the courses develop ways for students to give feedback to each other. Sometimes they have students make an argument about their own learning, and then have each other to assess that argument and make an argument. And that second part that I mentioned, is really an important part of the natural critical learning environment, that it’s an environment where people deliberately learn to give themselves and each other feedback. So they set up the whole system of marks around that idea that students were going to give each other feedback on how well they’re doing. And they’re going to give themselves feedback. And that they learn to assess their own efforts and work through those. Often, in the course of the term, credit is often given for participating. That is, if you do the work, you get credit for. And only at the end, do you approach anything like a summative judgment that we usually call a final grade. One of the things that we do in the book is to explore the history of grading. And we do that to help people see that grading is, for one, a fairly recent invention in education. The idea of putting a number or a letter on someone else’s thinking, that didn’t emerge until fairly late and really didn’t become entrenched until the late 19th to the early 20th century, and that changed everything. So I want people to see this in that kind of context, that there’s nothing natural or automatic about having the traditional approach to grading. And so what people have done in the super courses, is find ways of saying “Okay, now you’ve joined a community, you’re going to be helping each other to learn and you have responsibilities toward that community and to help each other to assess each other, to give each other feedback… substantive feedback, not scores, substantive feedback to one another. And we’ll try to give you feedback as well, maybe as a group, maybe individually in smaller classes, but to give you that opportunity of trying, coming up short, and being able to try again, without that affecting your overall final grade. And then the final grade is based upon an accumulation of lots of things, and perhaps a final project, a final paper, a final presentation, or something of that sort, rather than just simply accumulating, you get 10% on this and 15% on this and 40% on this aspect of the grading.

John: How can we help students embrace the concept though, of productive failure, that process of trying something, making mistakes, and then learning from that experience. Because that’s something that many of our students don’t naturally come to, because many of them haven’t seen it before up to the point when we have them in class.

Ken: Yeah, exactly. And I think one of the keys is to provide them with very dramatic and enchanting learning environments, at the very beginning of their experience, so that the students say to themselves, “This is going to be different.” Let me give you an example of what I’m talking about… my favorite example of this category. It came from a program that was offered, again, in a secondary school. It is called city term. And it brought students from around the world together right outside of New York City, and they use New York City as their classroom. And the first Saturday in the program, they’re invited to go on a scavenger hunt. And they’re given a list of items that they might look for in New York City. One of them, for example, that they often use was: find the first wooden escalator. So students go off in groups, and there’s a teacher that goes with them, but the teacher doesn’t interfere, and just keep them safe. And beyond that the students go wherever they want to. It’s a wild and exciting adventure. And then at the end of the day, they end up in Central Park on a picnic, and they discuss with each other. “How did you find that escalator at Macy’s? What questions did you begin to ask yourself? Who did you talk to? How did you reason through the process?” And by sharing ideas with each other, what they’re actually doing is learning good research techniques. That’s a wild way of learning good research techniques, to say the least. But it’s something that the students will always remember. And they will latch on to that. And they will latch on to the course now, because that first experience was something that was quite dramatic to them. Now, we don’t all have the opportunity to use New York City as our laboratory or our classroom, and to take students on a scavenger hunt. But we can imagine creating a first assignment, and I’m reluctant to use that word “assignment,” because we found often that these courses don’t talk about assignments, they talk about opportunities, and invitations to students. It’s so exciting that it begins to break down all of their sort of stereotypes in their mind about what’s going to happen in a class. So in Andrew Kaufman’s Books Behind Bars course, they’re first asked to apply for the course. So they have to explain why they want to be in the course, and then helps to begin to break down barriers. And then the first day in the class, he begins to break down the barriers by first telling them about a three minute story about a young man who read a little short story by a guy by the name of Tolstoy, Leo Tolstoy, and as we called him in the West, and about halfway through the story… in other words, about a minute and a half end of the story, the students began to realize that it’s a story about Gandhi. And it’s a story about an important transformation in Gandhi’s life as a result of reading a piece of literature. And so Professor Kaufman says to his student, “We want you to think about a point in your life, when a piece of art, maybe a piece of literature, maybe a painting, maybe a song, but some piece of art had a deep impact upon you and your thinking.” And the students began to discuss with each other. And they began to realize first, that this isn’t going to be a course where the teacher just talks to them and they take notes and then later take a test on whether or not they can recall the notes that they took. But it’s going to be a class that they will dominate, that they will do most of the talking and most of the thinking, and by creating a different kind of environment, you then can move to ultimately getting them to think about such questions as how are you going to assess yourself? How do you know whether or not you’re making progress, and whether or not you’re learning and you’re learning deeply. And the key point here is helping students to learn what it means to learn deeply, that learning deeply is not the traditional strategic learning… “oh, I learn this for the test. I’ll make an A on the test and I’ll make an A in the class.” No. it’s self-driven learning, where you begin to look behind the scenes, where you intend to look for ways in which this course can transform you and transform your thinking.

John: So essentially, I think what you’re saying is we need to help encourage students to develop more reflection on their work and on their learning process.

Ken: Yeah, I think so. I think that’s the key is getting students to think about their own learning, and help them with categories that will enable them to think deeply about their own learning… categories like deep learning, versus strategic learning, or surface learning. The strategic learner just wants to make straight As [LAUGHTER] and they’ll do whatever is necessary to make those straight As. The surface learner just wants to learn enough to pass the course, to be able to perform on an exam or write a paper or whatever it is that’s required of them. But neither one of those two leads to deep intentions, that I deeply want to understand how calculus works, and how it can help me in understanding the world in which I live, and how this applies to me, and my field and how it applies to my major, even though I’m not a mathematician, and how I can change the way in which I think. So developing those deep intentions. and fostering that deep development of intentions, becomes extremely important.

Rebecca: One of the things that you were just mentioning in terms of the strategic learners and the surface learners is how much many of our courses are probably structured with them in mind, rather than a deep learner in mind, and that we perpetuate these kinds of learners rather than deep learners based on our class structures.

Ken: Yeah, exactly. Exactly. And I think we began to break away from that, by the way in which we frame the questions we raise. You think about the example of Kaufman’s course. Or let’s take an example from physics. Eric Mazur, at Harvard, has pioneered a lot of the elements of the natural critical learning environment and super course. And the students in his course do not learn physics, by listening to lecture, boring or otherwise. They learn physics by doing physics. They do three big projects. And each semester of the course… it’s a two-semester course, some students take only one semester… but in either semester, they do three big projects and they’re massive projects. And they work with a team. Each student’s is in a team of about five or six students. And they work together to try to attack a problem. And each of the projects has a back story. For example, you’ve been contacted by a charity that was created in Venezuela, by a well known philanthropist and musician, who became quite convinced that music, and symphonic music in particular (being part of a symphony orchestra), is a transformative experience that can help very poor people rise out of their poverty, and to develop a different mindset that enables them to conquer some of the economic circumstances they face. It’s a program that now has about a million students worldwide who are engaged in it. But it has a problem, namely that some of the students are so poor, that they cannot afford to buy real instruments. Now, you’ve been studying waves and music is made up of waves. So, your team has been invited to create new kinds of instruments that can be made from things that you find in the junkyard. Now that new instrument has to be able to be tuned over to different octaves, has to stay in tune for a specific amount of time, but by creating these new kinds of instruments, you can help the young children. Now, that’s a compelling project. They learn physics and the physics of waves actually doing the project and demonstrating that they can do it. Another project is more fun than anything else. Do you remember the old Rube Goldberg cartoons?

John: I do. I don’t know if Rebecca does. [LAUGHTER]

Rebecca: I know what they are. [LAUGHTER]

Ken: Ok. There was this guy. He was an engineer and he was also a cartoonist from San Francisco. He used to draw these cartoons, where you would attempt to do something very simple, like crack an egg, by a very elaborate piece of machinery, where balls would roll down ramps and would trigger gates that would open up other gates that would cause other balls to fall, and so on, and so forth. So it’s a very elaborate, unnecessary project. So on the Rube Goldberg project, so students are invited to create a Rube Goldberg type device to crack an egg… [LAUGHTER] something you can do very simply by knocking the egg against the edge of your kitchen counter, or your kitchen table. But no, you had to create this very elaborate project. Well, to do that, all of Rube Goldberg drawings and inventions were based on physics. That ball rolling down a ramp at this angle will acquire this speed, and it would open this door and would result in this hammer hitting this hammer that would cause this ball to roll down this ramp… and that would, etc, etc, etc. So you had to know the physics in order to create these absurd projects. But there’s a lot of fun in that. A lot of fun. And the first project they do, I think, is one was just strictly on fun. They build a racing car, and then race each other, like many of them did when they were back in the fifth grade. And they have great fun in doing that. Another one is one where they have to design a lock that other people can’t open. And they get points for 1. being able to keep other people out of their safe, and for being able to crack the lock of other peoples’ safes. So a lot of wild times. Now there’s a textbook that stands behind all of this. And the textbook is written by the professor. And it lays out everything that he might otherwise have said to them in lecture, and more. Now, how do you get students to read that textbook, 1. by creating these enticing projects, but another way you do it is by making reading of the textbook into a social experience. So Mazur and his colleagues created a program which is now available to everyone, called Perusall. And, using Perusall, students can read material together. So in the sense that, “Okay, we’ve got an assignment, we need to read to chapter two. We’re all reading it together, you’re reading it on your computer, I’m reading it on my computer. And as I read along, I may have questions. So I’ll highlight that text, I’ll raise the question.” It’s like writing in the margin, but everybody can see what you wrote. He organizes the groups into groups of 15 or 20 students apiece, and they can read each other’s comments and they can respond to each other’s comments. And to participate in the class, to be a participant in the class, what it means is that they will keep up with offering the comments to each other, and their comments on the text itself, making comments on what’s written there, to raise questions. to answer a question, so on and so forth. And what they found is that reading completion goes from 40 something percent at best, all the way up to 95 to 100% completion.

John: I’ve been using Hypothesis in my classes for the last three years, and nd it’s a very similar type too…

Ken: Yeah, there are several others out there.

John: And students really enjoy that, too. They enjoy seeing what other people are raising questions about. They enjoy answering questions for each other and posing questions to each other, and It seems to make the reading process much more engageing when it becomes this social activity.

Ken: Exactly. Well, that’s the whole idea. And then of course, behind that is this set of really intriguing, interesting, fascinating projects. And there’s a course at an engineering school in Massachussetts that offer a course, for a long time. They no longer offer the course unfortunately, but they offered it for over a decade, I think it’s called the “history of stuff.” [LAUGHTER] And the first day of class, the students go in and they see on the front table stuff. It’s the kind of stuff you might find by going down the aisles of Walmart [LAUGHTER] and picking things off the shelf, just a wild assortment of things. And they’re invited to come down and pick out one of those items. And then to begin to explore it, explore its history. Why was it created? What was it created out of? What materials? What kind of implications does its creation have for society? Does it just clutter up society and create a backlog of unrecyclable material that creates environmental problems of one type or another? Or exactly what is it? Students had a great deal of fun just exploring stuff. And the course ended up by looking at some of the history of technology through the lens of a well known American patriot, Paul Revere. But Paul Revere was also an expert in metals. And so they explore engineering of metals through the eyes of Paul Revere. And it becomes a way of mixing disciplines in a way that makes each discipline more intriguing and more interesting. Rather than “Oh, you study this, then you study that, you make no connection between the two.” Say, one more example?

John: Oh, sure.

Ken: There was a course we looked at in southwest China, and we went to the school, Southwest Jiaotong University, in Chengdu, and it was a course organized by a young woman who teaches physical education. And first day of class, the students are invited to think about what kinds of sports they enjoy doing. is in rock climbing. Is it soccer? Is it basketball? What is it? Now, can you imagine creating an exercise device that will make you a better soccer player, or rock climber, or whatever it is that you want to do… your favorite sport? And then the whole class goes to a sports equipment store, and they began to look at the equipment that’s already there. And then began to think about, “Okay, how can I create something better?” Now, as part of the team, it’s not just this PE teacher, but it’s also other people. Ah, you need someone from biology perhaps, to help them think about… “Well, what kind of exercising do the human muscles need? What kind of social environment do you need to create here?” So, you need other experts. And if you want to make this a product that can go on the market, maybe you need a marketing professor, who can help you devise a marketing plan of the new product that you’re creating. I told this story to my broker several years ago, and his response to me, was: “In Communist China?” And I said, “Yes, they have a market economy just like we do. And they’re interested in marketing and learning marketing. And they have marketing professors, just like we do.” And so they creates this environment where students learn by doing. And they learn by mixing disciplines, rather than keeping them apart. Much more interesting.

Rebecca: I love the move towards more interdisciplinary work….

Ken: Yeah.

Rebecca: …something that I feel really connected to, but it really gets people I think, more excited about different disciplines when they’re more intertwined, because we understand how they’re related to one another.

Ken: Yeah, exactly. So if you’re going to study the human brain, for example, how do you make it interdisciplinary? One of the professors, we studied taught in the medical school and taught medical students about the brain. But she was asked to create a course for undergraduates. This was at Vanderbilt. And so the course on the brain for undergraduates mixed every discipline you can imagine, together, because, as she argued, everything is connected to the brain. So you might be studying music, if that’s your interest, or whatever your interest might be. You might be studying ethics, if that’s your interest, and then you’re encouraged to think about what part of the brain handles ethical questions? What part of the brain helps you to appreciate and understand music? What part of the brain helps you to do this or to do that? So they’re studying all the aspects of the brain, but they’re also studying all these other disciplines, from Holocaust studies to a wide variety of other things, and raising deep ethical questions along the way. And she offered this course for 10 years, and it was a transformative experience for most of her students, the overwhelming majority of them, breaking down stereotypes and prejudices and helping them to also think more deeply about how their brain operates.

Rebecca: Sounds like there’s a lot of classes I should sign up for.

Ken: Exactly. I thought at one point, I was talking to some high school students about where they want to go to school. And I said, “Well, it’d be wonderful if you could go to a school that would mix all these super courses together. Because they’re strung out all over the world.” Maybe there’s a way of doing that virtually I don’t know.

John: Or maybe, as a result of your book, and other similar work, perhaps more faculty will start doing this type of thing and more of their courses.

Ken: Yeah, and perhaps, in designing a curriculum that includes professors from a wide variety of different disciplines, and students from each of those disciplines, working together in small groups, to tackle problems of physics, and then later tackle problems of the brain or tackle problems of history, or tackle problems of well, you name it… and have a opportunity to sort of tour super courses around the world. That would be a wild experience.

John: I still remember examples that you used here when you spoke at Oswego. And I remember examples when I first read your first book on what the best college teachers do, a while back, in large part because you weave in narratives, along with the theory and the reasoning behind these concepts. And I think the use of narrative helps makes the story much more interesting and helps raise curiosity and makes things much more memorable. Is that something faculty should strive to do in their own classes?

Ken: Yes, I think so. And the professor, I was just mentioning at Vanderbilt, I think, did that and created a course that was part history and part neuroanatomy and part philosophy and part literature and part music, but they’re all around narratives of one kind or another. Yeah, I think so. I think creating that narrative. Human beings love stories. And if we began to understand things, in terms of stories, then it becomes much more memorable to us. And we remember what we learn. And if you think about learning, it contains at least these three major aspects: we’ve got to encounter new ideas and procedures, and so forth, but we’ve got to encounter new material, there’s the encountering part. And the second is the making sense of it part where we relate it to other things that we’ve learned. And then the third aspect of it is retaining it long term. So we remember what we remembered, what we learned. And I think encountering all of this in stories, makes it much more memorable. But I think what the super courses do is they have students read stories to learn physics or history or other kinds of things. But they don’t tell them those stories orally, for the most part, they do not use lecture, to do that first aspect, that is of introducing the material. Usually, that’s all that happens in the classes, you’re introduced to the material and lecture, and you never get around, you never have time in class, for those second aspects of the “making sense”part of and the things that you might do to retain it. So super courses are built in a way that they spend their time working on those other aspects. Because the first one, the one of conveying the new information and ideas to the students, that can be done with reading, with films with other ways. But the part of struggling with meaning, with the teacher and with each other, that’s much more complex, and that requires a different kind of approach. And that’s what the super courses offer.

John: This project began with a collection of syllabi, and we should probably note that those syllabi do make it into the book as an appendix. So, not only do you have the stories of how these classes work, but it provides faculty with examples of how these things are implemented. In an Invitational syllabus.

Ken: We took excerpts from some of the syllabi, not all of them. But from a few, to give people illustrations of what we’re doing: one from math, one from the sciences, and one from the humanities.

Rebecca: We always end by asking, what’s next?

Ken: [LAUGHTER] Good question. Well, I have on my agenda, and I’ve been working on, a book aimed at parents. And the working title of the book sort of summarizes the whole idea of the book. Although the working title is 11 words long, andt hat is way too long. But we’ve got to find a way to achieve the same thing with a shorter title, but it’s: How to Help Your Kids Get the Best Out of School. Now we chose those words carefully, because the first task is defining what we mean by the best. And, in part, it means learning to learn deeply. So how do you help your kids to do that? And we chose one of many words we might have used for kids. We said “kids,” ‘cause it’ss short and to the point. We’re trying to shorten bold type as much as possible. And I’m working on that with a colleague, Mindy Maris, and we have a due date with Harvard press of 2022. So, that’s coming up rapidly. So we’ve got a lot of work to do over the next year and a half. But we’ve already done quite a bit of work in organizing that, and so forth. So that’s the next major project.

Rebecca: That sounds like an exciting addition to the collection that you’ve already have out, and rounds out the offerings.

Ken: And then somewhere in the far distant future I play out entirely, I would love to take all the we’ve learned and how to understand the best in any field. And that was a process in itself. How do you define the best and how do you collect evidence that something is better. I’d love to do a book that might be entitled: What do the Best Coaches Do? [LAUGHTER] and describe good coaches in a wide variety of different sports. But that would be my swan song, if I ever get around to it.

Rebecca: Thanks so much for joining us today and sharing some insight into your newest book. I know that a lot of our listeners will be looking forward to reading it soon.

Ken: Well, I look forward to hearing feedback from your listeners. And as we said, toward the end of the book, we hope that at some point, every reader will say “I wouldn’t do it that way. I’d do it this way.” But when they say that, we hope that they will base that judgment on strong evidence that that presents, whatever alternatives they come up, presents a better learning environment than the one we describe in the book. But, we hope this idea of a super course, is something that is organic, it continues to grow. And five years from now, somebody will summarize something about super courses today, meaning the super courses, 2027 or 2030. And may describe a much different book than the one that Marsha and I wrote. But it’s an organic process. And we’re looking forward to the conversation w e hope that the book stimulates.

John: Well, thank you. It was great talking to you again, and we’re looking forward to sharing this with our listeners.

Ken: Thank you.

Rebecca: Thank you so much for your time. We really appreciate it.

Ken: Oh, my pleasure. My pleasure. Anytime.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

181. Capstone Experience

Imagine a course in which the faculty member is a coach who guides students through a real-world project with messy data and the problem-solving that comes with it. In this episode, Dr. Kathryn Berkow joins us to discuss how a course with no content can provide students with a rich learning experience full of analysis and insights. Katy is an Assistant Professor of Management Information Systems at the University of Delaware. She is also the host of the On Cultivating Student Engagement in Higher Ed podcast.

Show Notes

Transcript

John: Imagine a course in which the faculty member is a coach who guides students through a real-world project with messy data and the problem-solving that comes with it. In this episode, we explore how a course with no content can provide students with a rich learning experience full of analysis and insights.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Kathryn Berkow. Katy is an Assistant Professor of Management Information Systems at the University of Delaware. She is also the host of the On Cultivating Student Engagement in Higher Ed podcast. Welcome back, Katy.

Katy: Thanks so much for having me on again.

John: We’re really happy to talk to you again. And our teas today are:

Katy: I’m having a decaf Earl Grey.

Rebecca: …good classic. I’m having a Scottish afternoon tea, which is my new regular.

John: And I am drinking vanilla almond tea. It’s a black tea from the Republic of Tea.

Rebecca: I really like almond teas.

John: This is my first, it was a gift from my son.

Rebecca: So we invited you back Katy to talk about your capstone course that you offer. Can you tell us a little bit about the course?

Katy: Absolutely. I love talking about this class. It’s one of my favorite courses to teach and I teach it every spring. And so having the break in the fall teaching my other courses gives me a lot of renewed energy for this capstone. It’s a senior capstone experience for our business analytics minors in their final semester as they prepare for their next steps after graduation. And the students have a variety of majors and a variety of perspectives in the class. And analytics is only their minor focus. And so in course design I have had, I would say, two influences. I have a colleague who designed a similar capstone for our management information system students that relies on semester long projects with an external sponsor. But also, before coming to the University of Delaware, I worked in the financial services industry as a quant analyst. So I worked through lots of long, larger scale, analytical projects. And so I modeled the course after my colleagues set up quite a bit, but I adapt it from my own professional analytical experiences. So when I first started teaching this course, I shied away from saying what I’m about to say, but now that I’ve seen enough students through the course to know that they learn so much over the course of the semester, I feel very comfortable sharing this: there is no content in this particular course, and I think that’s what makes it so much fun. So we spend the complete semester working on a large scale, real, unwieldy project that is truly representative of the type of projects students will face in their professional careers as data scientists. The students work on the project throughout the semester, they report to me or my co teacher as their manager every week. And we provide feedback on performance, suggestions and resources for how to move forward. When the students are stuck, we’ve usually seen something like that before. And we can brainstorm together how to get unstuck. And sometimes all the students need is confidence that the direction they’re planning to go in is a good one. But because there’s no content, the projects can really unfold and be the focus for all of us throughout the semester, and each student team gets a unique experience. So the two important things that I really want students to know after finishing this course, are: one, that there’s no perfect answer to a complex problem… there are only a degrees of good, better, better than that… when it comes to analytical solutions, which I’m sure is true in so many areas; and number two is that each unique problem needs its own thoughtful solution. We’re not trying to teach students in this class how to think about every problem, just how to think analytically now that they have the analytical tools. Not what to do in every situation, but how to think through each complex new situation they face.

Rebecca: Do students in this class all work on this same project, or do you have small groups working on projects.

Katy: So each student team is working on a project throughout the semester, and this semester, for example, I’ll have 42 students in the class working on nine distinct projects.

Rebecca: Do the students define their own projects, or do you have a predefined project.

Katy: So I create the projects for them with community members. Because there’s no content in the course, the project is the critical design component. So each year, I start getting ready months in advance curating these projects for the semester. In the past, we’ve worked with our own athletics department on a variety of projects, a large retail banking institution, a service provider of home repairs, a few local nonprofit organizations, and lots of others. And so the variety is exciting, because the students all have different interests as well. And I tell these organizations that all I really need from them is a sufficiently challenging research question. I mean, everybody’s got lots of questions, but we have to really hone in on one theme, and then enough data to support finding an answer to that question, perhaps. It’s not been too challenging to get project sponsors interested, because I’m offering free analytics. [LAUGHTER] And so I might contact someone through a friend of a friend and say, “Here’s a few naive questions, I think we might be able to help you answer if you have the data,” and people generally seem excited to have an introductory conversation. So, for example, some of the organizations that we’ve worked with want evidence of program efficacy. They might have survey data or some measurement of before and after metrics on the students or participants in the program. And we can use that data to answer questions. Others have said we want suggestions for how to price the seats in our new stadium which is super open ended and they’ll provide some data about ticket sales, for example, and so it’s a very open ended data-driven question but it’s not standard. And sometimes those non standard questions are even more fun. So I write up a project description after a couple of meetings, discussion, thought with the project sponsor. That might be two to five pages, it’s not a lot of information, and then I get the data into a format that students can work with, which sometimes is me doing nothing to prepare the data. I do want the students to struggle a little bit with formatting the data since data cleaning is a big part of learning to be a successful data analyst, but I provide lightly processed data and a project description to the students as their starting point. So, I ask them what project they’d like to work on after introducing the projects to them on the first day of class and I try to fit students to a project that they’re most interested in, but really sometimes i’m surprised that some projects are more popular than others and it’s not the ones that I expect

John: Paul Handstedt has a book called Creating Wicked Students: Designing Courses for a Complex World and what you’re doing sounds very much like what he’s advocating. Giving students really challenging problems where there’s no clear solutions, is a really good way of helping them pull together all the things they’ve learned. So in a sense the content is really everything they’ve done up to the capstone but you’re giving them an opportunity to apply it in ways that they’ll need to, if they’re going to be successful in their career

Katy: Absolutely. Completely. Because I did read that book and really felt inspired and I think I was already doing this style of course at the time, but it made me feel like I was headed in the right direction, that giving students this opportunity to try solving a problem that has no answer and most of the problems they try to solve in their careers don’t already have answers or we’d just be using those existing solutions. And so it is really good practice I think for whatever’s next especially in the field of analytics. Ee don’t know what the technology looks like or the methods are going to look like as we move forward and so they really need to be able to think critically about ethical concerns, methodologies, how to work with their data in a really honest and skilled way that can be applied to lots of wicked problems.

Rebecca: So, it sounds like in a process like this, which sounds similar to kinds of courses that I also teach, you act more as a coach rather than more of a traditional teacher, kind of coaching them along on how to respond to the data, respond to what’s happening in the moment. Is that correct?

Katy: Absolutely, and that’s something that I love about this class. I know a lot of your guests have talked about removing themselves from the sage on the stage position and becoming the coach or cheerleader in an active environment and that is one of the things I love most. When students are excited and driving the questions, I get even more excited to talk about what those answers might look like with them.

Rebecca: Do students have the opportunity to talk to your community partners or is it always through you?

Katy: So we staged the course in three sections and we have three presentations associated. So, the students will get started, spend maybe three weeks or so working with the data, getting to know it, generating some questions and some initial discovery points, and then they’ll present those results to their sponsor. And actually teaching online has been much easier for the project sponsors because they can easily attend presentations and provide feedback. Usually in that first round, the discovery period, the students present something and the sponsor can say, “Oh, let me explain that.” It’s sort of a back and forth where there’s a lot of sort of correcting any misunderstandings or answering questions. Phase two, the students are working toward what I call an initial solution. This solution might be a basic model that makes some assumptions that are maybe not appropriate, but just to get started. And it underscores what I love most about the class which is the idea of iterative solution. Presentation three is going to give them an opportunity to refine the solution, completely abandon what they did for presentation two, or improve it in a way that makes it more realistic, more robust, more an answer to the project sponsor’s research question. So, absolutely, the students get to interact with their project sponsors during those presentations where they are leading the show, they’re having the conversation, they’re hearing the direct feedback as though the project sponsor is their manager in that moment. And then I can sort of serve as a liaison, saying “I think what you’re saying is this,” “maybe we can put that on our to-do list,” and sort of just offering the support to both sides, to help everybody come together for a solution

John: How do you assess the contributions of individual students in the group project?

Katy: This will be the fourth time we offer this class and this has evolved quite a lot. A few things have remained the same. Those presentations I mentioned are a clear part of assessment in the class. But each week, I require that each team submit a status state. And that will be a highly detailed list of achievements toward answering their research questions, and also a list of highly detailed to-dos for the next week. So, even though it’s not a class in project management, they’re getting that scope of accountability for moving their project forward. And I require that each student’s name appear next to achievements, and next to to- do items. And that gives me an opportunity to really see what’s happening. But we also have weekly meetings. The students, instead of meeting in a classroom setting where the entire class is together, I’ll meet with each group for maybe 30 minutes. If they need more time, certainly, we can have an office hours setup. But 30 minutes is usually plenty for us to discuss anything from the status update, for them to get feedback from me, and for me to say things like, “It doesn’t really seem like we’ve done enough this week, what’s going on team?” …just like their manager would say in real life, maybe they haven’t had a chance to have that really, in-person accountability conversation before. The need for that is very rare. Most of the time, I’m saying things like, “Wow, I’m so blown away by what you’ve achieved this week. How can I best assist this week with your to-dos? What do you want to talk about in this meeting?” but those are the three components. I recently transitioned to specifications grading, which has been a ton of fun in this particular class, because the senior students are so independent and really prepared to graduate, that this gives them a lot of flexibility. So I require excellence in answering the research question as the C-level component. So the team grade is the C, do a great job on this project. And then I can add in individual components that will scale toward a B and an A. So for this semester, for example, the students will earn a B if they complete a data ethics module where they have to think and write about some ethical dilemmas in data science. They write reflections on visiting speakers who are analytical professionals that we have come to class via zoom, and they evaluate their own performance. And then to earn an A, I ask that the students take on a particularly challenging component of answering their team research question. And I don’t give a lot of guidance there except to say, discuss it with your instructor so that everybody’s on the same page. So I can help them determine what is sufficiently challenging to be truly deserving of an A, because every project is so different, I don’t want to spell out what they need to do. But an example would be if you’re working toward a C in the class, you could work with a team member to generate a model that answers a particular, say, sub question of your focus. But if you’re working toward an A, you need to be maybe developing that model on your own. The one thing that I do promise students is that there will be no surprises. I know it sounds like there’s some looseness in the specifications. But we’re talking about it every week in our meetings. And if students are not on track to earn that excellence, C-level grade, then they know about it with plenty of time. And I’ve really never had to give students much more motivation than “Hey, I haven’t really seen much from the team this week. Let’s talk about those to-dos for next week.”

Rebecca: It sounds like those meetings are really important in terms of processing learning, not only just moving the project forward, but also just processing “what are they learning?” …and how might they move that forward in the future, not just in their projects? And you also mentioned some sort of self-evaluation or reflection, can you talk a little bit more about that component?

Katy: Oh, sure. At the end of the semester, I ask them to do this performance evaluation of themselves in much the same way that I have to write one in my role and have done in the past. It gives them a little bit of practice self reflecting. I’m not really judging performance based on their performance assessment. I’ve already seen what they’ve done in the meetings. When they speak with confidence about something complicated, I know that they’ve learned a lot. So I really just ask them to be honest, and say, “What do you feel like were your greatest growth points? What do you feel like you still need to work on as you head into a professional role as a data analyst?” And I also ask for feedback about the course. {What were the elements that you felt contributed most to your growth this semester, and what things didn’t contribute anything?”

Rebecca: What have your student responses been to working on these projects.

Katy: So ever since the first time the course was offered, the students have expressed sincere appreciation for the help in making a transition from student to professional. It seems like by the time this course pops up in their schedules, they are really ready to start becoming more independent, and squeezing into a seat in a classroom just doesn’t feel comfortable for them anymore. So they express that this course and other capstones like it that are problem based really give them an opportunity to be in the driver’s seat and have more independence in their senior spring. Many also have said that they’ve learned the skills of analytical thinking, data cleaning, planning, modeling, but now they’re seeing for the first time how those things go together in a sequence and on a complete scale over much more time. This isn’t just a week long project where everything is abbreviated and if they’re going into an analysis role this is going to be what their career is like. So, I couldn’t be happier than to hear those two things. It makes it feel like a success. But, I’d also like to add something really selfish here. I get so much personal fulfillment from teaching students, at any level, but this class really gives me the opportunity to stand back and coach as you said earlier, Rebecca, rather than be that sage on the stage in front of the class with the slides. And I just get so much enjoyment from seeing students take off, watching them steer the ship for the first time, getting their answers. I also teach them at sophomore levels, the intro course, and so i’m lucky enough to see them again at the senior level and just saying that it makes me feel really proud is such a dramatic understatement because seeing them ready to leave the University of Delaware and become professionals… it’s really really fulfilling in a selfish way.

Rebecca: My experience with classes like that, too, is that students really appreciate the opportunity to try out this pretend career for a moment. [LAUGHTER] It’s a safe place to try that rollout and understand where they fit in the bigger picture or what specific role within an area or field might be a good fit for them in a way that an internship doesn’t, because it’s maybe a little more flexible or you have more of that direct contact with the faculty member during that process instead of just in the work environment. I’ve had students that have said that in those scenarios they’ve really appreciated that opportunity to fail without repercussions.

Katy: Yeah that’s a great point. It’s like we’re the coach but we’re also the bumper in the bowling alley… that we’re just trying to keep them on track and rolling forward. And I think what you said is true, especially as I think about what their roles might look like… different positions as data analysts… maybe they’re the best at doing the analysis, building the models, or maybe they’re the best at communicating the results and being the liaison between people who are super analytical and people who are not super analytical. So, this class, just as you said, gives them an opportunity to try on the different hats that might be available and see where they fit best in a comfortable supportive environment.

John: When you mentioned watching your students grow from seeing them as sophomores to seeing them as seniors, I have very much the same experience. I used to teach a wider variety of classes but in recent years, since i’ve been working in the teaching center, I primarily see them just as freshmen in a large intro economics course and then I see them generally as juniors and seniors in any econometrics class and in a capstone course, and seeing the change in them and seeing how they become confident with their material and seeing the work that they’re able to produce is really impressive. It’s a really nice feeling.

Katy: Absolutely and when you’re talking to people from other universities or if you’re talking to a panel during accreditation, it’s really, really nice to be able to speak to the entire scope of the educational process. When you see them one time as sophomores, or even one time as seniors,you only get that point in time feedback, but when you get to see the whole development process it just makes you so proud of what the students are learning and you have just such appreciation for all that your colleagues do along the way as well. It just makes it feel like there’s a symphony happening here and that you can see it much more clearly.

Rebecca: So, Katy, you’ve talked about this is the fourth time you’ve taught it. So, the first two times I believe were in person, then you had a time that started in person that shift out not in person. I imagine this time is remote.

Katy: Absolutely.

Rebecca: So, can you talk about that transition or transformation?

Katy: Sure. Last spring we were in person for five weeks and then we transitioned online and I was actually having one of my group meetings with students when the email came out. So, the email came out announcing that we were closing maybe at like 4:55 and we were meeting at five. So, I corralled the team into my office and I said “What’s going on? Just read it out loud. Let’s get this out in the open.” And it was so helpful for me to see the students processing the information live because I was processing it too but thinking in the way that they were thinking in that moment was so helpful to me. One student, for example, raised this question of “But i’m supposed to be on spring break when they say we’re coming back to school…” And i’m just like “Uh huh, let’s take it one day at a time. this is obviously new information for all of us.” But that’s how she was processing and the next student said “Are we going to not have a graduation?” …and I don’t know the answer to that. I said “Well I don’t know if this is going to be two weeks oor three months or longer, but we’re just going to take it one day at a time.” it was just so fascinating to hear what was happening in their minds. One of the questions that came up was “What’s going to happen in this class?” …like they felt almost more concerned about this class than some of their other ones transitioning online. And I said, “Well that you certainly don’t need to worry about. I feel less concerned about transitioning this particular class online than I do my other ones. Because all we do is have conversations. We have conversations together in this small group, we have conversations as a class where we’re listening to a speaker, and we have conversations with your project sponsors. And we can do all of that on Zoom.” I was lucky enough to have had some experience with Zoom beforehand. And so I just felt really confident that there might be one or two things I had to think through, but that most of the things that we were doing could easily be achieved via Zoom. So it was the other classes that were more, I’ll say, traditional in delivery format that I was worried about. So this, I feel like the students are getting exactly the same experience, they just don’t get to shake their project sponsors hand, which, you know, is a little bit disappointing. The networking component is really nice in person, but it’s not necessary. And I think meeting in these small groups, I still get to know the students just as well, and really can serve in the capacity of whatever they need, whether it’s mentor, thinking about helping them to find a job, or just project mentor, whatever is needed, I can do via zoom, because it’s in this sort of small group protected setting. And so it has been maybe the greatest challenge to transition other courses, not this one. So I really feel good about how this one has transitioned online,

Rebecca: I was shaking my head up and down the whole time you were talking, Katy, because I felt the same way about my project based courses. And in some ways, some of the logistics got so much easier being online. There’s some classes that I do that were project based, for example, our Vote Oswego project that I do with a political science faculty member in the fall, we do things with our classes together, it was so much easier to find a space we could all fit in on Zoom, we could easily get in and have the space to go into small groups without getting too loud. And some of those logistics actually were really fine online. And then even with some other projects that we were doing, having our community partners join us more easily, in a lot of ways, by being on Zoom, rather than having all the logistics of going to campus in person and finding a time that’s going to work, because there’s all the travel time involved and what have you as well.

Katy: Absolutely. One of our visiting speakers comes from Denver every year, just to be with us. She happens to be a graduate of the University of Delaware and loves spending time with our students and giving back in that way. But when I had to call her and say, “Hey, we don’t need you to come all the way to Denver anymore, we’re going to be virtual,” maybe that was a relief. [LAUGHTER] But now it just opens the door. I’ve heard a lot of your guests say it opens the door for so much flexibility in the future to just bring in more voices into the classroom and have an opportunity to learn from just a variety of different people because the commute time is zero now.

John: Going back to your point about students being a little anxious of how the course was going to proceed, I had a very similar experience with my capstone course compared to another more traditional type class. I met with them on the Monday before the shutdown began, and it wasn’t announced and there had been no discussion, at least that I had heard of on campus. But it was pretty clear that campuses all over the world were shutting down. So it was pretty clear we were going to as well. And I said “We should be prepared that this might be our last time meeting in person.” And they said, “Well, what are we going to do? How are we going to keep doing all these meetings and keep going?” And they wanted to make sure the class would be successful. I said, “Well remember how I told you, if you couldn’t be here sometime, you could just come in on Zoom? Well, that’s what we’ll probably be doing in the event of a shutdown.” Most of them had laptops there, I said, “Go to the Zoom website, create an account, [LAUGHTER] just so you’ll have it there, because you may want to create your own sessions for work within your groups.”And they downloaded Zoom to their smartphones and to their laptops. And I said,”Let’s just try connecting and just make sure you mute your microphone so we don’t get any feedback issues.” And they were pretty relaxed about it. And nothing really about the course changed other than the fact that they were meeting over a computer. They were doing all the presentations, they were doing all of their group work in breakout rooms instead of gathering tables around. And actually in a lot of ways it was easier because when they were meeting in small groups in the physical classroom, they were all looking over the shoulders of the people who were actually doing the writing at the time. And it was just so much easier for them just to share the screen and discuss it from wherever they were and have a much clearer view.

Rebecca: I think it’s really interesting that both of you are talking about how concerned the students were about being able to fulfill that particular class or that particular project and I think it really attests to probably their commitment to that project. And Katy, in your case, it might be just because it’s somewhat high stakes, right? There’s a client or a partner involved that you want to satisfy and it feels really satisfying to do real work for real people. And I’ve had this experience as well with community based projects that students just are all in on those kinds of projects, and just don’t want to see them fail at all.

Katy: That’s true. I’ve never had a problem getting students motivated. I told you once in a while, I’ll say like, “What’s happening this week?” and then you hear there were some tests or something like that. But honestly, the students are highly motivated all semester, because they’re getting to interact with those project sponsors during the presentations. And they’re going to be accountable to that person’s face or group of people during the presentation. And so sometimes I worry, as we get closer to the end of senior spring, that students are going to lose their motivation, and it really doesn’t happen. You know, they’re tired by the end, they’re ready to be done with the project, because this might be the longest project they’ve ever worked on. But they really deliver. And they always really impress me, I don’t feel any stress at the end of the semester with grades, because they, number one, they know what’s happening. We’re very clear all semester long about where their grade is headed, but also because they’re driving it. And this is a group of students who’s elected to have a minor in addition to their major studies. And they’re just highly motivated. Most of the class is earning an A by the end because of the excellent work they produce.

Rebecca: Sounds like a really fun class to teach.

Katy: I have so much fun. The hardest part’s the project. Once the projects are made, the rest is easy. I just show up and ask questions.

Rebecca: Mm hmm. I love my projects classes,

John: Is there any type of artifact that the students create that they can share with potential employers as a demonstration of what they’ve learned.

Katy: I would say I always encourage the students to include their project on their resume. But I do ask students to sign a nondisclosure agreements for their project sponsor. Tthat makes everybody feel more comfortable. Sometimes the data is sensitive if it’s got, for example, young children participating in a program, and other small businesses might not want to share their data. And so I have the students sign something. But I tell them, if you’re interviewing, you can certainly describe your involvement in the project in a loose way, you can talk about the specifics of the modeling you did, and how you contributed to the client’s end goals without saying, “Oh, I worked for this specific company.” And they also do create an executive summary. So that’s an artifact. They share their presentation slides, of course, with their customer, and they create an executive summary. But the goal of that artifact is to deliver information to their project sponsor, and not necessarily to serve as a portfolio. When I offer this class in the graduate level, where I have professional students who might be working, I don’t also ask them to work on a project for another organization. They collect their own project from publicly available data, and they generate a description of what the impact they could have studying this publicly available data might be. They create a digital portfolio using a WordPress blog. And then that’s something that they could really share as an artifact. So if it happened to be publicly available data they were studying, then they could certainly share that with a prospective employer.

Rebecca: It’s also a great opportunity at that capstone level to have conversations about the way that the profession works. Whether it’s nondisclosure agreements or copyright or whatever it might be related to what you’re doing. These are important times to have those conversations before the students graduate and move into their professional lives.

Katy: Absolutely. Because the data in our classes is often either… I’ll call it simulated, which is just made up by me, as an example, or some publicly available data that’s already in a nice format. So, it’s good for them to have the exposure to working with data that is just in a different format, because it comes from a different place. Or to see that not every dataset looks the same as the one your teacher might have curated in your sophomore level introductory course,

John: We always end with the question: “What’s next?”

Katy: So Monday is the start of our semester at the University of Delaware. So what’s next for me is diving into nine really exciting projects this semester with our seniors this year. So I’m really excited about that. But for the course overall, there’s going to be a big change coming up. And that is that I mentioned we have minors in business analytics in the course. We recently added a major in business analytics. So I think the exciting thing to look forward to is that we’ll have an even greater mix of students coming up either next spring or the following spring that will include people who have had even more training in analytics during their time in the University of Delaware, and it’ll increase the variety of solutions that we can provide to our project sponsors. So that’s really exciting, as is being part of a growing program. But in addition to that, I’d like to concur with many of your other recent guests who have said that they’re focused on what the future looks like in their classrooms. Certainly what’s next is going to involve some changes. And it seems like there are lots of opportunities to reimagine our courses when we have the option of being in person, but also using the new tools we’ve learned for engagement and flexibility. So in the broader sense of what’s next, I really don’t know, but I’m thinking about it a lot.

John: Our students got an interesting email. For the first time since I’ve been at Oswego, we had about a foot of snow and it was coming down pretty quickly and instead of getting a notice that classes were canceled, they got a note that all classes will take place remotely today. [LAUGHTER] So one change is, I think, for those students who used to look forward to the occasional snow day in upstate New York, those days are probably gone pretty much everywhere.

Rebecca: It’s not just the students that look forward to those days. [LAUGHTER]

Katy: Absolutely, I agree. But you’re right, that’s a big change. And we’re seeing that at the University of Delaware as well, our winter session courses were not affected by snow.

Rebecca: Well, thanks so much, Katie, for joining us and sharing other great stories from your classroom. There’s so much to learn from your practice and I’m glad that you were able to join us again.

Katy: Well, thank you so much for having me back. I really enjoy talking with both of you. But also I enjoy learning so much from all the guests that you talk with on your show.

John: And I really enjoy your podcast as well and am looking forward to hearing more episodes. And this sounds like a really great project you have there.

Katy: Thanks so much. Yeah, I hope everyone will check out the On Cultivating Student Engagement in Higher Ed podcast and I’m hoping to put out a second season this spring.

John: And we will share a link to that in the show notes.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

177. Blogging in Unexpected Disciplines

Maybe you’ve seen professional development sessions about digital portfolios or blogs and thought, “that is not relevant to my classes.” In this episode, Dr. Kathryn Berkow joins us to discuss how she has used blogging in her Business Analytics class to allow students to share their learning journey. Katy is an Assistant Professor of Management Information Sciences at the University of Delaware. She is also the host of the ON Cultivating Student Engagement in Higher Ed podcast.

Show Notes

Transcript

Rebecca: Maybe you’ve seen professional development sessions about digital portfolios or blogs and thought, “that is not relevant to my classes.” In this episode, we look at one example where blogging has been used to share students’ progress on business analytics projects with an audience.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Kathryn Berkow. Katy is an Assistant Professor of Management Information Sciences at the University of Delaware. She is also the host of the ON Cultivating Student Engagement in Higher Ed podcast. Welcome, Katy.

Katy: Well, thanks so much for having me. I’m a huge fan of your podcast.

John: And we’re fans of yours.

Rebecca: Our teas are:

Katy: I am drinking tea today, I just made a cup of something called Scandinavian detox, which is an herbal caffeine free blend.

John: And I’m drinking ginger peach green tea.

Rebecca: Oh, back to an old favorite. Me too. I’m back to the Scottish afternoon tea… just delivered.

John: We’ve invited here to discuss how you’ve been using blogs in your business analytics class. Could you first describe the class in which you’re doing this?

Katy: Absolutely. So I teach business analytics at the University of Delaware. And the course that we’re talking about today is a reformulation of a traditional second course in statistics, and it’s for sophomores and juniors. So this is a required course in the business school for most business majors. For some, it’s their last statistics class they’ll ever take. And for others, it’s creating the opportunity to introduce perhaps more study in analytics. There’s a wide range of students in the class, lots of different majors, lots of different interests. And the topics that we focus on, are sort of three-pronged instead of a traditional two-pronged approach. So we focus on introducing a programming language, which is always a topic that introduces some fear for many students. We talk about statistical modeling, which also brings some fear. And the third, the part that introduces the reformulation of the class is the communication around the results of building a statistical model. And the potential impact on the research question and what the potential is for using these analytics. We really want to connect the students to the impact that their analytics could have on an organization.

Rebecca: Statistics isn’t necessarily the first topic that comes to mind to meet a blog assignment. So can you talk a little bit about how you decided to introduce blogs into your course?

Katy: Absolutely. And I completely agree with you, Rebecca, this didn’t come on suddenly for me. There were two things that were happening concurrently. The first was, I was experiencing a lot of frustration with my multiple choice homework assignments. When I started teaching the course, I had traditional exams, and these multiple choice questions. And there was always a problem with the multiple choice questions. I’m not putting enough time into writing them well, and generating the multiple questions for each topic that’s going to create that ability to randomize selection for different students. And there was always a problem, you know, a student would email and say, like, “I think I got this wrong, but I don’t know why.” And we would discover that there was actually a problem with the question. But the second thing that was happening as I was becoming more and more frustrated with my homework assignments was that I attended a workshop about digital portfolios. And it was the kind of workshop where you go to several sessions over the day, and you have a great choice. And this was one that I was like, “Well, that sounds interesting, but I’m not sure it’s going to be useful for me,” just like you said, I’m not sure that’s a statistics class was an obvious digital portfolio class. But I just kind of listened, I loved the idea and filed it away. They were giving examples of using it for writing courses or art classes to feature students’ work. But eventually, this moment came where I realized that the two could absolutely go together and highlight the communication in my class that’s missing from the multiple choice homework questions. The students were not getting practice at communicating the impact of their results, or the meaning of their results, in multiple choice questions, really. So the idea to create a space for them to not only report results, but to talk about them in their own words, really made me excited. And I just immediately started developing this new assignment, knowing that I was taking a risk, but I was really excited about it.

John: And so as they’re reporting this, is this something they’re doing over the course of the whole project?

Katy: Yeah, absolutely. So the way the project begins, that first week of the semester, I have students select their own data set of interest from some resources I provide. So an example of that would be like I send them to Kaggle. As a first source. Kaggle.com has lots of publicly available data that people have put up on the site to share. And I’ll have the students select a data set of interest, and work with that particular data set all semester long. So they start by generating an interesting research question to them. And you know, I’m just asking them to put some context around. So some students will select something very serious. Last semester, some students wanted to work with COVID-19 data, which is so relevant and interesting, but others did not want to work with COVID-19 data, they wanted to work with something that was maybe feeling like more of a break from what we were experiencing. So, a lot of students choose sports related data. And whatever the students choose, they’ll work with that data set. The intention is for them to work with that data set all semester long, and apply every skill to that data set and report out. So, there are a variety of different topics, and I also include In my syllabus that we should just be conscious that we’re not all going to choose the same type of data. Some of us are going to choose very serious topics and we should all treat the discussion of those topics in a serious way, and some of us are going to choose some things with a little bit of levity like studying the winners of the bachelorette and what their characteristics are like, and that we should appreciate the levity in that too, and just appreciate each other’s choices.

Rebecca: One of the things that I know that I’ve appreciated in doing projects like this in my class, is the ability for students to practice that professional communication and have an audience. Can you talk a little bit about who the audience of the blog is, and whether or not it’s public facing?

Katy: Oh, sure. So the blogs are intended to be public, because they’ll submit the links to their posts as their homework assignment. And then we use Canvas as an LMS. So Canvas loads those assignments right up, and I can look at them right away, which is great. But because the blogs are public, I also include some notes about that in my syllabus, pointing out that you wouldn’t want to report personal information, for example. You can keep it as private as you’d like to, in that you don’t have to identify yourself as the author if you don’t want to. But I encourage them to keep a professional tone, especially if they want to use this as a potential portfolio to share with future employers as evidence of their study and their work. And I tell them, they can spend as much or as little time as they’d like making it look beautiful, with some really beautiful images. Some students choose not to do that at all and it’s just a plain white background with a title. And that’s fine, too. So I tell them, I am their audience, but that they should consider that this is a public space, and that they might have a broader audience. But I also include in the syllabus, if anyone is uncomfortable with the idea of a public facing blog, they can request a different option. I’ve never actually had to offer that option. But I would offer like, let’s create a Google Doc that is more like with chapters instead of a blog with individual posts.

John: What platform are students using? Do they get to choose our own? Or are they all working in one platform?

Katy: So, I suggest WordPress because it has a free option. And because that’s where I built my example blog, which I’m happy to talk about as well. And I can provide more detailed instructions and troubleshoot with them a little bit more easily. But if they want to choose Wix or something else that they’re more comfortable with, I’m completely fine with that so long as they know that I’m not an expert in any of these other platforms.

Rebecca: Do you want to talk a little bit about that example you just mentioned?

Katy: Absolutely. So the example blog that I built was sort of my first experience with understanding if this could work in the class. As I was trying to formulate what we would do, I was writing my own version of the blog, which later turned into a resource that can help students as an example. So I selected a data set about understanding diabetes, and its relationship to age and blood pressure. And this is a very common data set on Kaggle. So I was writing. The goal is for students to take each topic that we learn in class, and there’s about, say, 15 topics over the course of the semester. So if we learn to build a basic regression model, the students will then apply the basic regression code that we’ve learned in class, and we’ve seen a couple examples and apply it to their own data set. So now I’m thinking about COVID-19 and I want to understand the relationship between, say, rates of cases and age groups, something like that. So they now have my example blog to go to and read and see how I structured what I did. And they’ll say, “Okay, Katy built her model relating blood pressure to age, and I’m going to do something similar.” And one of the reasons why I like creating the example is so that they can see where they’re going. But I also really liked that it serves as sort of a textbook support where we don’t have a textbook in our class. So in the later, more complex topics, I also write a lot about how I’m interpreting my results, and why, so that they have a little more support as they try to make those interpretations. And of course, if they’re having any trouble making those connections, they can always come to office hours and ask questions. But those who have used the example blog as a support really report that it’s useful.

John: You mentioned that you don’t have a textbook, but I’ve seen on your website that you have created videos that serve as a substitute for a textbook.

Katy: Yes, as we’ve transitioned to online remote teaching, I had actually been very lucky to have had experienced teaching online before. And I wasn’t using those videos that I had used in the online version of the class in my on campus class at all. But there have always been students not able to attend class for various reasons. You know, “I’m an athlete and I’m going to be traveling, I’m going to miss class” or “I’m just sick and I’m going to miss class.” So for lots of reasons, students have needed an additional resource and, without a textbook, that’s always been a concern. So I wanted to provide those additional supports but, of course, as I’m sure you’ve considered before as well, does providing the video reduce my attendance in class? There’s this constant struggle. But as I have been doing this online remote teaching, I think that I don’t care, I just want to provide all of the resources, so that every student can find this material in whatever way is easiest, most flexible, most accommodating to their lifestyle. And so that means putting out the videos in what I’m calling a digital textbook. And I can also refer it to students even when we’re on campus as an additional support if they need it after class.

Rebecca: How have your students responded to working through this information in this way? And have they had the opportunity to look at each other’s blogs and comment and do some peer feedback as well?

Katy: That is a great question. I have not incorporated any peer feedback of the blogs into class… except this semester, as we were doing the remote teaching, I was meeting in small groups, so the students would watch the videos from the online class. And then we would come together in conversation. And the only peer feedback that I’ve ever incorporated is having the students present their issues as they work through their blog posts during these small group sessions, and then having others weigh in. And I found that to be so rewarding, it was just so much fun to have them identifying each other’s problems. And there were lots of students in each group studying similar types of data. So one student is studying hockey, “Oh, I’m also studying hockey, did you find this…” and so it really was fun to make those connections. So in that way, we have had some peer oversight and I think I’d like to find ways to continue that kind of small group discussion about the blogs. But overall, the feedback on the blogs has been positive. The way that I know that is I’m looking at course evaluations, I would say, in maybe 40% of my course evaluations, students are mentioning the blogs in general. So, not everyone by a longshot. And when they do mention it, it’s in two ways, in general: 1. I really loved the blog posts, or 2. the blog posts were really hard, but I knew I was learning. And so seeing comments like those, it really makes me feel like I’m achieving the goal of helping the students to get to a place where they’re ready to apply the skill in a new setting. So that’s the beauty of the blog, that it captures what we really want them to do, which is to get into their professional setting, ready to apply it to a new situation, because they have my example blog, they have their example blog, they can remember how to do this given those skills.

John: You provide students with a sample blog, but do you give them any other guidance on what they should include in their blog post?

Katy: Absolutely. So in each problem description, so for each blog post assignment, I’ll give a bulleted list of items that they should include. But in addition to that, because I probably go on a little bit in my bulleted list, I also provide a rubric. And Canvas, as an LMS is really helpful in this way, because I can include the rubric right in the assignment and tell students how I’ll be calculating the points for each assignment. So I can have one point for including an image showing the results, I can have one point for correctly interpreting the intercept, etc. So the students will know as they’re writing, what are the things that I’ll be looking for. But even better than that, they can also see where their blog posts needed improvement. So after it’s been graded, and by the way, it’s very easy to grade in Canvas, because it can take me just a few minutes to read through a post and know whether the student has done the five or six checklist items in the rubric and just click, and I’m done in just a few minutes per student, which is great if you have a large number of students say whether it’s 50 or 150. It’s relatively quick. And I’m very lucky to have the help of a TA with that. But the beauty of the rubric in canvas is that the students can see which buttons I’ve clicked so they can know “Okay, I got a five out of six on this post. And where I wasn’t effective was at describing what the meaning of the intercept was. So now I can go to tutoring or to office hours or talk to the TA about how I could change that for a project or an assessment.”

John: How many posts do they have to make as you’re working through the project.

Katy: So the students will work on a post for every topic over the course of the semester, which ends up being 14 topic related posts. I also have them create a welcome post that’s just “Hey, I’m here blogging as their first assignment…” that just officially sets up their blog and creates their first post walking them through the process. And they create one additional post about “data and impact” I call it so they report which dataset did I select? Why? What’s the potential impact? So I have them create a research question they’ll be focused on through the semester and describe what impact it could have. Now this is a business analytics class, but I think of it more like an organizational analytics. So even if you’re studying the bachelorette, you can position yourself to think about, “Okay, I work for Netflix, and I’m trying to understand, is there a theme here? Do we need to have more diversity and inclusion in our bachelorette or bachelor? So what are the dynamics around the situation we’re studying?” And obviously, some things lend themselves more to a traditional business analytics context than others. But I want the students to know that any organization can have any research question. So you can create that context. And then they do a final post with conclusions at the end of the semester.

John: Do you have a generic rubric that works with all of the posts? Or is there one for each of those stages?

Katy: Great question, John, you’re pointing out that there was quite a bit of setup on the front end of this. So it was good that I was really excited about it when starting the experiment, [LAUGHTER] because I had to build that example blog, I had to build each of the 17 total assignments, and that kind of semi-custom rubric to each one. So each one includes the requirement that an image must be posted, for example. But, if we’re studying linear regression assumptions, I’m going to have one point associated with each assumption that needs to be tested correctly. So there is a lot of customization. However, after creating, because these are so generic or agnostic to the material the student is studying, after creating that first batch of assignments, I’ve been doing this two years now, I have never needed to change it. And I really haven’t had any issues with these assignments when it comes to “this wasn’t clearly enough defined,” for example, because while the students do tend to become frustrated with open-ended assignments, they hear me say, a lot of the time, business analytics is an open-ended kind of structure. It has a lot of art, and a lot of science. And I can’t answer the questions, always in a direct way because your real life is not going to have direct answers to every question. And so over the course of the semester, I do find students tend to get more comfortable with this ill-defined nature. But they also have the support of specific rubrics to give them some sense of structure when possible.

John: Would you recommend this approach to other faculty outside of the more traditional writing fields?

Katy: Absolutely, but I won’t pretend to know how. What I would say is that I encourage just hearing what other people are doing, like I get so much value out of listening to your podcast, and to Bonni Stachoviak’s podcast, just hearing what people are doing in their classrooms that’s engaging their students, because one of the reasons I created my own podcast was to generate more sharing, because what happens in our classrooms is so opaque to others. As I told you, I attended that session a couple of years ago, maybe three or four years ago now. And I didn’t really think it applied. And I know we all say like, “Oh, well that’s not for me,” or “that’s not gonna work,” I just feel like the more ideas we have circulating in our minds that you know “don’t work,” and the more issues we face on our own in our own classes about, especially now, how to engage students under these changing circumstances, I just feel like having a glossary of ideas that you can pull from and maybe adapt to your own class is so helpful. So I think there are lots of potential applications to this, that I could see. In fact, I use this in my other courses. So I teach a capstone course at the senior and graduate level. And when we’re not using proprietary data related to an organization, which I do in my senior capstone, like in the graduate course, I do have students select publicly available data, I also use a blog reporting structure. So I did need to develop a similar framework with rubrics. But it creates this sort of agnostic space where no matter what project you’re working on, you can report out on that topic, and all you need to do is share a link with me and then I can see your status update for the week. And you can really tell a lot by how students are writing about what they’re doing. Now in, say, a finance class where there’s a correct answer to every question, I’m not sure a blog is going to be a perfect fit. But it’s just really nice to have it as one option for reporting something because probably there’s going to be a report or a reflection or writing assignment in some space, where digital reporting is a really nice way to reduce paper, make our classes much more streamlined for remote delivery or flexible delivery as we move into the future. I do think it has a potential for tons of applications that I certainly haven’t thought of yet. But I really like this delivery method in class.

John: And you mentioned reflection, and that’s something that’s useful in any discipline, and that’s one way in which it could be fairly easily integrated. You’ve made my life a lot more complicated this week, because I’m supposed to be preparing my syllabi this week. We’re recording this in late January. We’ll be releasing it a little bit later. And when I saw the description of what you were doing, I was thinking this is something I should try this semester. So I’m probably going to be trying to implement some of these ideas in my econometrics class this semester.

Katy: Well, that’s great to hear. I hope it works.

John: I hope I can get it all together.

Katy: Well, I’d be happy to share anything by the way, like if you want to see my example blog or if you want to see an example rubric or an example assignment description, I’d be happy to share any of those things.

John: That would be very helpful. Thank you.

Rebecca: It is funny that sometimes we get these really great ideas to try out these new things, and John and I certainly experience this frequently. As podcast hosts, we’re exposed to so many great ideas, and then we have so many. But it’s really great, even though that sometimes there’s a lot of time involved in the setup of these things, they often play out in saving time over time….

Katy: Absolutely.

Rebecca: …presuming you stay with it, right? [LAUGHTER]

Katy: Well, absolutely. And that’s one of the things that has been the neatest for me, because one of the issues with the multiple choice questions is that they were so specific, and there was so much opportunity to make a mistake, missing a dollar sign or writing the wrong variable name. With the blog assignments, they’re so open ended. All I’m saying is: “Apply using linear regression with two variables to your dataset.” How can I get that problem description wrong? I can’t. So the beauty of the blog is that the nature of the description of an assignment is so open ended, but so directly flowing from what we did in class that the students really know how to accomplish each step.

Rebecca: I’m sure they also can see the practical application and how all of this skill sets not only from the statistics part of it, but also from the blog component of it, or the digital reporting of it, is practical for their future careers and things. And I think that tends to give students a little bit more buy-in to these kinds of assignments.

Katy: I think you’re right. And it achieves two other critical things. One of them is that it brings the sort of inherent motivation that we know is so important for student learning right up front, the students are choosing what they want to study. So they’re going to have a lot more excitement. And honestly, I see that when I’m talking to them. I don’t know anything about professional hockey, but when my students are working with hockey data, and that’s the area that they have the greatest interest in, I see them light up when I ask questions about like, “Now, how did you make this interpretation? Why do you think this is related to this?” So inherent motivation is a big deal. But the second thing that is a really fun byproduct is that having read what students are doing and having spoken to them in office hours, it brings up a ton of fun examples to talk about in class. So I might remember that Adrian is working on, you know, the COVID-19 data, and I can sort of bring that up casually in class. We found this weird thing that happened in Adrian’s dataset, and why do you think this is happening? Or what do you think is a good solution to this problem that Adrian’s facing and we can talk about things that I had never imagined would come up in class just because of a quirky dataset that was publicly available.

Rebecca: I could see how that also puts students in the position of being an expert or a co- teacher in some ways, which I can see as being another motivating factor.

Katy: Absolutely. That’s such a great point, because they’re also experts in each other’s conversation. So in the small groups that I was doing in the remote teaching in the fall, one student would say, “Oh, well I bought him an idea for your data set based on our conversation last week.” And even if you just watch hockey, and you’re not working on a hockey dataset, but you really like it, you can be an expert in someone else’s space. And clearly I can’t, because I’m upfront with the students about what I know and what I’m interested in and what I’m not. So like, I don’t watch a lot of hockey. so you have to help me understand. And that sort of puts me in a backseat in the same way it puts them in the front seat.

John: Earlier, you mentioned that one of the motivations for doing this was that you wanted to move away from multiple choice tests. Have you eliminated multiple choice tests from this course?

Katy: So yes, I have… sort of. In the remote teaching environment, I have relied more on team projects and multiple choice quizzes. I was trying to model after folks I’ve heard on your podcast and others say that having smaller-stakes quizzes is much more inclusive, gives students more of an opportunity to learn and to course correct, and so I was having these multiple choice quizzes. But I’m finding that the multiple choice quizzes are really an obstacle for students. And I am as frustrated as they are with them. They’re doing great on the blogs, but they’re not doing as well on the multiple choice questions. And I suspect it’s because of the poor design in the multiple choice questions and not because the students don’t know what they’re doing. They couldn’t do a great job on the blog if they didn’t have the skills. So as we head into the spring, where I’ll be teaching remotely again, I am planning to drop the multiple choice quizzes completely. That’s not to say I’ll never use multiple choice questions again. In my in-person class, I have used multiple choice questions or a multiple choice final as part of assessing students. But for now, it seems that the multiple choice quizzes really have not been effective. So this semester, I’m going to be grading students on their blog posts, team project, and participation, which is just an interaction metric

John: Are the team projects the same as a project they’re reporting on the blog?

Katy: Oh, great question. So when I assign a team project to students, it is on a data set that I’ve selected. So at the beginning of the semester, they are downloading a folder of lots of different data sets that they’ll need for examples working through the course skill videos that we’ve talked about. And I’ll choose a data set that they might not have seen yet and write some questions that are somewhat open ended, “build a model,” and I’ll try to randomize across teams like which variables they should use in their model. So they’ll all be predicting sales, for example, but they might have different predictor variables. So team one is going to choose variables, 1, 2, 3, etc. So they’re all working with something a little bit different, but they’re trying to end up in the same place: predicting sales. And these are totally open ended. But they get to work together, and that helps. And I’ve also been using specifications grading in this class over the last semester, which has been a lot of fun. The students have started to appreciate it toward the end of the semester, though, maybe I shouldn’t have selected a semester that was so much in transition to experiment with that. [LAUGHTER] Maybe it created a little stress at the beginning. But by the end of the semester, the students were saying things like, “I really liked that I could choose my level of interaction with the course.” The reason I bring this up is because I would require revisions in team projects. So every team had to score 100%, which of course is alarming to students until you say, “there will be revisions required until you’ve scored 100%.” So I want us to move past these topics, but we are going to make sure that we learn them. And I think that was a really neat experience for me to be able to coach students through finally getting mastery, which I have never done in class before. If a student didn’t master a topic, we just move on and maybe there will be an opportunity down the road. But it really was nice to see students come to a place where they had achieved a particular benchmark.

Rebecca: Were your collaborative assignments done during class time, or were they outside of class?

Katy: So this semester, I allowed students to have an entire day that they could complete the assignment anytime between 12am on that day and 11:59pm. I also encouraged my international students to do it during that 24 hours, but to make sure it was done during that eastern time 24 hours. And they could work on it during the class time, during the scheduled class time. Because it was a day that we would have had class, but they didn’t have to. So they had the flexibility to choose a time that worked for them. If it was late at night, great. If it was early in the morning, great. Or if it was during class time. That’s fine with me too.

John: We’ve really been enjoying your podcast. And you’ve mentioned this a little bit. But can you tell us a little bit more about how you decided to start this podcast?

Katy: Absolutely. First, I want to thank you both for listening. I’m such a fan of your podcast and others that are similar. And just like the digital portfolio idea, the possibility of creating a podcast has been in the back of my mind for a while, listening to some others like yours. So just as I have learned so much from hearing what others are doing via podcasts like yours, I just feel like I can’t get enough examples of creative ideas that people are using to engage their students. And not just big ideas like “Oh, implement a new grading system.” But small ideas like “Oh, I spend the first minute of my class engaging my students in this way,” or “I have this micro assignment that we do only occasionally but builds community.” And as I mentioned before, what we do in our classrooms can feel so opaque. It’s really that I just wanted to contribute in a time when we can all use more sharing of the things that are working for us, and not every idea is perfect for every person. But just more sharing of what’s working so that we have more options to choose from when teaching feels so different and so challenging to so many of us right now. And so the podcast is generating, selfishly, tons of new ideas for me, which even if that’s the only thing that comes out of it, that’s enough. But it’s also doing the service of drawing attention to the great ideas around me at the University of Delaware this semester, and hopefully, in a broader sense in the future. And so I just really appreciate being able to feature great ideas that are happening around me that might not otherwise be heard about except in documentation of excellence, for example.

John: And often those are only seen by a few people on review committees and sharing it more broadly, both within your campus and across the whole academic community, raises the visibility of that work much more extensively.

Katy: That’s my hope because the more people I talk to, I just feel so impressed by the exciting ideas that I’m hearing from different people and to be able to amplify their voices. I couldn’t be happier to be able to do that.

Rebecca: Well, you’ve talked about some really exciting things that you’re working on. But we always like to raise the ante by asking: what’s next?

Katy: So next, for me would be a second season of the podcast. I’d really like to incorporate more ideas from a broader range of places across the University of Delaware, but also bringing in others from outside just to add to the discussion. And also, I’m really thinking a lot, listening a lot to your podcast, and thinking a lot about what the future looks like in my classroom. I will have taught remotely now for…. this will be my second semester in the spring teaching remotely. And I’m learning so many things about how we can make learning more accessible to more students under the current circumstances. But I think a lot of it applies to what our future looks like. So I’m really not sure when we go back in the fall, I’m hopeful we’ll be back in the fall safely, but I’m really not sure what my class is going to look like. I can’t imagine that I’m just gonna slide back into what I used to do without incorporating some new things. And finally, I am expecting a baby in June. So there is a new chapter ahead for me that will bring some other fun changes. And so that is what’s next.

John: Congratulations.

Rebecca: Congratulations. Yeah.

Katy: Thank you very much.

Rebecca: …definitely exciting times on so many fronts.

John: Rebecca had that experience fairly recently.

Katy: Oh, congratulations.

Rebecca: Yeah, my daughter is three and a half now, but yeah, it was a great adventure and continues to be a great adventure. So I know you’ll have a great time.

Katy: Yeah, I’m certainly looking forward to understanding all the things that are ahead and totally different and maybe they’ll also inspire some new ways of thinking about learning from seeing it through a different set of eyes

Rebecca: It definitely did for me. [LAUGHTER]

John: I think it does for everyone who raises children. And actually Josh Eyler, after watching his daughter learn and experience the world…. it inspired him to study more about learning, which is ultimately the source of his book on How Humans Learn.

Katy: I think I heard him talking with you about that on your podcast.

Rebecca: Well, thanks so much, Katy. This has been a great conversation.

Katy: Well, thank you both so much for having me. It’s really been a pleasure chatting with you.

Rebecca: Hopefully we’ll have you back in the future to hear more about the fun things you’re doing in your classroom.

Katy: That sounds fantastic. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

165. Educational Pipeline

A college degree, especially in one of the STEM fields, can provide students with higher incomes, more stable employment prospects, and more pleasant working conditions. Many  students who could benefit from a college degree face a variety of barriers that prevent them from successfully completing their degree. In this episode, Jill Lansing joins us to discuss what colleges and universities can do to help smooth the educational journey from Pre-K to college and to careers for all of our students. Jill is an Assistant Vice Chancellor and Director of Education Pipeline Initiatives at the State University of New York. Before moving to this position in 2009, she had been the Coordinator of P-16 Strategic Planning for the New York State Department of Education.

Transcript

John: A college degree, especially in one of the STEM fields, can provide students with higher incomes, more stable employment prospects, and more pleasant working conditions. Many students who could benefit from a college degree face a variety of barriers that prevent them from successfully completing their degree. In this episode, we discuss what colleges and universities can do to help smooth the educational journey from Pre-K to college and to careers for all of our students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Jill Lansing. She is an Assistant Vice Chancellor and Director of Education Pipeline Initiatives at the State University of New York. Before moving to this position in 2009, Jill had been the Coordinator of P-16 Strategic Planning for the New York State Department of Education. Welcome, Jill.

John: Welcome, Jill.

Jill: Thanks so much for having me today. It’s really truly an honor and a privilege to be here. And

John: Today’s teas are:

Jill: I’m drinking iced tea because of the podcast, and some water.

Rebecca: It’s a little cold outside for me to have iced tea today, but I have Big Red Sun again. And as you can see, I am drinking my way through one whole canister of the same tea. [LAUGHTER]

John: I am drinking Tea Forte’s Earl Grey today.

Rebecca: So Jill, we’ve invited you here today to talk a little bit about your work on educational pathways. Can you tell us a little bit about your role?

Jill: About 10 years ago, almost 11 years ago now, I started at the State University of New York in this role to really try to help to strengthen the alignment between K through 12 education and higher education and adult education, also encouraging adults to return to college. And it has been such an amazing opportunity, I have to say. It’s amazing, It’s been 10 years already, 11 years to think about this pathway. But, I think the idea was, many of the faculty that are listening today have been doing it forever. So they know what works and what doesn’t work. What we did when we started this work about 10 years ago was to really try to create more opportunities to bring resources to the table for faculty at our colleges. So the idea was really, and I’ve been lucky in this regard, was really to figure out what grant opportunities we could apply for, what pathways were available that we could actually say: “This works. This doesn’t” …to try to avoid places where people have had difficulties in the past. So, I spend a lot of my time actually learning from the faculty at SUNY about what does work, and then also try to bring resources to those things that are successful. Before my role at SUNY, I was working as the coordinator of P-16 strategic planning for the State Education Department. And I worked in many different capacities there, one of the places that I worked, that I loved the most, was working in the Office of the Professions and Higher Education. And in the professions, I was so new, I was a graduate student when I started and I got the opportunity to be in the role of the public management internship program at the time, which was to really try to create opportunities for new people to have the opportunity to learn about the government and learn about public administration. And so the idea was to create a public information campaign around career opportunities in the professions like architecture, dentistry, engineering, nursing. And so I was able to learn about so many opportunities there, there are nearly 45 licensed professionals in New York State, all opportunities for students to have career opportunities in. And also from there, I was able to work in the Office of Higher Education, and also the Office of at the time, it was Elementary, Secondary and Continuing Education. And so I really did get to see all of the amazing work that goes on in the State Education Department to support students, and also the pathways that are available for student success. So it really gave me an idea of what we could do at SUNY to really support students beginning at the very early age of pre-kindergarten all the way through elementary school, secondary school, and then into college. So lots of good experience that led to, hopefully, better outcomes for students,

John: What are some of the barriers or the challenges that students face as they move along their educational pathways in the state.

Jill: So I always like to start with opportunities. [LAUGHTER] But I think that barriers are real. In fact, they very much are real. So last year, I had the great opportunity to work with the National Center for Education Statistics as part of the Data Institute team to kind of look at the data and to find out some of those barriers. So, we did some regression analyses, and one of the things that we knew were true, but it really became evident in the numbers, is that economics and race matter. My dissertation was around college choice. And it’s amazing to me that, when the odds are stacked against you when you’re looking for college, that puts you in an unbelievable place where you’re really trying to figure out how to go about college choice, how to go about succeeding in college. So we really do try to look at that in the beginning. But barriers are, you know, your a risk for college. The Education Trust has been an enormous resource for us. They were looking at things like student suspensions in high school. It’s a demoralizer, it also really places students behind where they could be if they didn’t have to experience those kinds of challenges in their high school. So, I would say that the barriers are real, and we’re trying, through our programming, to really address those barriers. I also think another barrier is, and I know a lot of our faculty are interested in this, it has to do with the growth mindset and to what extent is students believe they have the ability to succeed and I think that pervades all economic lines and it’s really an important thing to keep in mind because, as faculty are thinking about how do we work with new students, and they’ve given me so much advice, as we look for money and programming, how do you really try to build a student up and help to empower them as they try to be successful in college? So, I think the barriers are real. And we really need to think about that. And it’s good to have this opportunity to talk about what we can do to really make a difference.

John: What does the data suggest is effective in helping students overcome some of these challenges?

Jill: That’s a great question. And there are so many data points that help to think about what the challenges are, and how to create change around the barriers. So one of the things we’ve been working on right now in collaboration with the Community College Resource Consortium, and also Jobs for the Future is the idea of Guided Pathways. For a long time, students would come into college, particularly in community colleges, and take courses that they were interested in, but didn’t necessarily count toward their major. So, they would take some courses, they weren’t sure what major they were interested in studying, and then they would take some courses, and then they may or may not add up to the total number of credits they need to graduate. So, we’ve been trying to really focus in on college completion and graduation. And that consortium has led us to become part of something called Strong Start to Finish, which is a national initiative to try to get students to complete gateway courses in their first year of college. So, really try to focus in on what matters, also try to avoid remediation through co-requisite coursework, and also opportunities where students don’t have to kind of get behind in their classes and can really kind of get up to speed on what they need to complete their gateway courses. So, that’s been very helpful. And also the Guided Pathways Initiative. If you talk to college presidents across the state, they are head over heels about Guided Pathways. So students enroll in college, they enroll in a meta major, instead of necessarily a particular major, and really try to take the courses in the very beginning of their curriculum that would lead to success in the health sciences professions, Hospitality and Culinary Institute, business professions. So, it’s not we’re asking high school students to make a decision right away about their major, but kind of “tell us what you’re interested in, and then we can connect you with the courses that would actually add up to success.” Because, I think, where the Community College Research Consortium has found many students fall away from college is when it’s not adding up to their degree program or to their area of interest. So, that’s been very powerful. Another area is our P-Tech and Smart Scholars programs that have been guided by faculty, and I’ll say the Faculty Council of Community Colleges, in the very beginning of any early High School initiatives, put in place standards for what quality looks like. And one of the things I was going to talk about later, but I’ll say it now, because it’s so critical, is that rigor matters. So, when faculty put rigorous standards in place, it inspires students to succeed. And I think that they feel a different benchmark about where they’re supposed to be in their collegiate life. So I think rigor matters, and I applaud all those faculty who do this day in and day out, because it’s hard. It is hard in times of COVID-19, it’s hard every day, but just to keep the rigorous standards in place, and then support students on that pathway really matters. So, those are some opportunities, many more. We were able to, a couple of years ago, win a grant from the National Science Foundation, and also from Battelle Institute to help our graduate students work with middle school students to be, in their words, deputized as scientists beginning fourth, fifth and sixth grade in high need school districts. And having the connection between the graduate students and the middle school students was powerful, and all this was under the direction of faculty in our colleges. So, we had our scientists, across SUNY, overseeing this effort. And their ability to really create opportunities for these middle school students, who otherwise would never have had the chance to become scientists in laboratories at our SUNY Colleges, was huge. So I think that there are so many examples that I could share with you, but the ones where I think faculty, graduate students, aspiring faculty, put their passion and their heart into it, as well as their intellect and rigor, those are the ones that really matter for our students.

Rebecca: I wanted to follow up on the picking a major piece a little bit, because I think that is a real struggle for students when they head into colleges with an expectation of like, “Well, what are you going to do when you grow up? What are you going to study?” And the names of majors are so abstract to students, and there’s many careers that students have no idea even exist, and if you’re already in academia, you might have a clearer vision of what those things can lead to, but our students just don’t, and there’s no reason for them to know that

Jill: Exactly. When I was an undergraduate student, physical therapy and occupational therapy were big majors in my school and I didn’t know what they were at all. So the idea of helping students to come into a major in health sciences really will allow them to understand what opportunities are available to them. Another resource I should talk about is a resource called the Empire State STEM Learning Network. And in my career, one of the things that I’ve learned is that often when grants are distributed to an organization, the project period ends when the grant period ends. And one of the lucky things that I’ve been a part of, thanks to the regional leadership across the state, is something called the Empire State STEM Learning Network. And the idea for the STEM Learning Network was to bring together educators and business leaders in regions across the state to really focus on the educational pathway that leads to jobs in healthcare, in manufacturing, in Information Sciences, and architecture, and so many other fields. And I will say that one of the things that made the Empire State STEM Learning Network successful was the commitment regionally to develop partnerships to help lead students into careers. Because, you’re right, students don’t really know what they want, necessarily, in the very beginning of their life. But once they have the experience to be exposed to these opportunities, then all of a sudden, that could open the door for them and really spark an interest in them that we hadn’t thought about before. And even if they don’t go into architecture, maybe they would be interested in something related in engineering, or in teaching architecture. So I think the spark matters. And you’re absolutely right, Rebecca, it’s a really good question, because how would you know what the opportunities are for you? But really trying to create a level of interest with the students and also a level of “I can do it, this is real for me,” that really does matter. And I think our faculty, I mean, I’m talking to our faculty, and I’m so honored, because our faculty are often the drivers of everything that we do, and you know what drives students, what motivates them, and how to really think about what their needs are. And I think young high school students, I am always amazed at what they can do when they see exemplars of success, and also when they understand rigor. But also, I would say, in the way of how to be successful, I think we need to meet students where they are. One of the things that we learned from our colleagues at the Community College Research Consortium was that remediation can be very detrimental to students… the word remediation, the idea behind “I need to still catch up.” But, instead, trying to help to develop the skills that the student is interested in. So astrophysics is a big dream. But really, if you start, piece by piece, level by level, it can be a possibility, or another career option in that same field or discipline could also be possible for students. So really trying to motivate the students to success, I would say, but you’re right, students figure out what they want right off the bat when they’re coming to college is really a challenge.

John: What types of approaches have been effective in overcoming gaps in prior education? Because one of the main issues is that there’s quite a bit of disparity in the amount of training that students receive in their high schools, and that’s tied to a whole host of issues related to funding and housing segregation and other issues. While remedial work can be discouraging, or just even the term “remediation” is discouraging, what strategies have you seen that have been effective in helping students bridge any gaps in their background, to help them get to speed more quickly without being faced with a whole sequence of courses needed to enter, say, the STEM fields?

Jill: Precisely. So, good question. I would say to look at the following programs for good benchmarks and good outcomes data, Educational Opportunity Program (EOP), the EOC, the Liberty Partnerships Program, also very powerful. STEP and CSTEP have been incredible about meeting students where they are and getting the students where they need to be P-Tech has been amazing. And I think the best P-Tech programs and the best early college programs, the best programs that we have across the board, are those that are led by faculty who are determined to help students succeed. For example, in our community colleges, which are open access, and many students come to as the first opportunity they have to enroll in higher education, where there are collaborative opportunities for the students to participate in STEM research opportunities, that is so powerful, it almost closes the gap in the way of letting students have the opportunity to become scientists. I was talking earlier about programs where we worked in our middle schools to help our graduate students come to the middle school students and talk about “Here are opportunities for you to become a deputized scientist.” So, then when that student actually comes to community college, and they’re saying, “Oh, I’m gonna research climate change with my faculty member,” or whatever the research opportunity may be, it becomes a way for the student to see themselves in that career. And then they understand the pathway to the career. So I can’t say enough about the hard work of faculty in this effort, because (and this is my own experience as well), is that faculty in my life have taught me how to think. And I often tell students in high school that college is a place where they can learn to think. And they think that they’re behind the eight ball in some ways, and that they feel like they have to tell the professor about how much they know. And, instead, if you listen to faculty, and I’ve worked with both of you, and you’ve been teaching me about, like, how to think about this, how to think about talking to people, how to think about getting a message out, and I think often new college students feel that they have to prove themselves versus they can learn something, and you’re actually willing to help them. [LAUGHTER] So, I wanted to share an example. A couple years ago, I was interested in the process of statistical process controls as a way of thinking about, “Well, if we are trying to achieve student success at scale, how can we look at numbers and data to help us do that?” So, I was thinking that statistical process control could be helpful in higher education, about seeing trends over time, seeing when there are differences, and why there are differences. Maybe, perhaps, that the spring semester wasn’t working out for students because of X, Y and Z. And the faculty, again, provide such leadership, maybe we could learn from one another. So anyway, I went to this course on statistical process control that was offered by SUNY Polytechnic Institute. And there with me, were three Community College faculty. And they were interested in statistical process control, because they wanted to be able to integrate that curriculum into their coursework. They said, “So many students are coming to us, and they’re interested in manufacturing and machining. And they keep talking about statistical process control.” So they were seeing how can we integrate those into our work, so that the learning opportunities are more aligned with their goals, professionally and personally. And so I thought it was really amazing that these faculties sought out these professional development opportunities to really try to make learning matter for the students and make it something that they were interested in that they could actually apply to their jobs. And I think we talk a lot about workforce, we also are talking about educating a citizenry, and especially with the political climate change, I think we really have an opportunity. We’ve learned a lot in 2020, about COVID-19, about our responsibilities to citizens in this country, African-American citizens in particular. We focus a lot about Black Lives Matter. And after the passing of George Floyd and so many other unfortunate events this year, we’ve learned a lot about why we really have to be focused in on those liberal arts opportunities, and also about governance, and religion. And we need to have active citizens in this country. And I think that is equally important. Often, we talk a lot about STEM and job opportunities, but we have to think about the big picture. And I think that our faculty are leaders in that work. And so, to the extent that we can learn from them, and, again, create opportunities that help to advance our agenda. is really powerful.

John: A couple of times you mentioned P-Tech, for those listeners who may not be familiar with that, could you describe that program and how it works.

Jill: So the P-Tech program has been a program that is being led by IBM, and they’ve been really a front runner in this work. And they’ve put a lot of thought into how to connect high school faculty, high school programs with our colleges, and then also with workforce opportunities in high need fields. And I think that it’s been an awesome opportunity for our students. They are able to enroll in the P-Tech program as high school students, free of charge to students and their families. And I would also add, at this point too, that one of the big assets of SUNY is affordability. And so there are often times where students can actually earn SUNY degrees free of charge. And that is really incredible to graduate debt free. That also reduces a lot of the burdens that we talked about in the very beginning, around economic burdens, challenges around race, all of these opportunities are now readily available to students. So that’s been very powerful through the P-Tech program. And I think the P-Tech program is something that we can model across the board. And we have tried to do that in collaboration with regional BOCES, regional K-12 leaders, to actually develop a scope and sequence of courses beginning in grade 12, that continues on to higher education and then into the workforce. So, what happens is K-12 leaders, community college faculty (also many of our technical colleges also have P-Tech programs), and our business leaders come around the table together. And they think about what would a student need in order to be successful in life and in business in the long term. And I think what it develops is a real true camaraderie that allows successful programming development to happen. What is fortunate about the P-Tech is that it is enabled by grants from the State Education Department. So it does provide opportunities for faculty at K-12, faculty at higher education, business leaders to come together. But that seems to be very powerful when college faculty can also work with K-12 faculty to find out what the needs are, where the gaps are, and how they can work together to try to mitigate those gaps. So, often what I heard when I interview faculty in our community colleges in our technical colleges is that often they are unaware of some of the challenges that K-12 faculty face and then K-12 faculty are so eager to learn more about all of the expertise that community college and our technical colleges and also even graduate programs have to offer. So, when they have the opportunity to speak to one another, then all of a sudden, they’re all on board on changing curriculum and helping students to succeed. So I think the magic in the P-Tech is about this collaborative experience. I often find that regional collaborations are really also very powerful in that our faculty in our regions know what kind of high school students are coming from. And then when they talk to faculty and they can identify, you know, maybe the student didn’t have a fourth year math course in high school, or maybe there are some skills that for some reason the student wasn’t able to develop, and especially this important in COVID-19, because we’re hearing students that are struggling. So I think the connection between high school and college and then regional workforce leaders are very important. And we talk a lot, too, in our work, about regional economic development, and helping to really strengthen New York as a whole through our regional workforce capital. So we’re looking at that. And again, I would say, to create and develop good citizens of the state and of the nation, I think it’s very important. So, I feel like the opportunity for faculty at the K-12 level to connect with our faculty at SUNY, and then also with our workforce and industry leaders (many of whom are SUNY alumni), is a very powerful connection.

John: What are some examples of good collaborations between colleges and P-12 programs? What has worked in terms of helping to bridge the gap in terms of creating an environment of good communication? You mentioned P-tech. Are there any other models that have worked well?

Jill: I can name so many examples of what has been successful. I’m thinking about also the gaps between community colleges and our baccalaureate degree granting programs and our graduate programs. I was thinking offhand about Binghamton University and its collaboration with Broome Community College, really trying to help students who perhaps in the beginning didn’t meet the admission requirements for Binghamton, but had a lot of promise and a lot of opportunity for skill development in STEM. And so they entered into a collaboration with SUNY Broome to help those students to develop the skills at SUNY Broome that would make them eligible for upper-level credits at Binghamton University. So there’s so many examples like that. Our early college high school programs, in addition to our P-Techs are also powerful for making the connection between high school and college. We’ve also done a lot of work with our teacher education programs. We’re really very fortunate to have the opportunity to work with the deans of all of our SUNY teacher education programs, to help them to develop more practical experiences for their teachers to work with the students and try to bridge the gap between high school and college. And I think that that has been so successful, our teacher education programs. And the faculty there have really led the way in terms of helping us to better understand K-12 experiences and create clinical opportunities for prospective teachers. So that has been huge because educator preparation is so essential to really helping to bridge the gap between K-12 and college and they’ve come back to us with so many ideas and opportunities, things that we’ve been able to win grants for and bring resources into the classroom about. So I’ve mentioned the Empire State STEM network before and they have just been amazing. So our colleges will frequently partner with our K-12 leaders to do things like in Buffalo, there’s something called the hands-in-hand program where the students were able to use 3D printing to develop hands for other students internationally, who actually need hands or limbs for some reason in their life. And they’ve been able to actually generate that through 3D printing. The other thing is computer science, cyber security. Our students right now in the P-Tech program are working with SUNY-Orange to develop skills in cybersecurity. So P-Tech is one example. But there are so many, and again, the graduate students that are working across the state with our K-12 students in STEM is also pretty awesome.

Rebecca: The ingredients of successful programs seem to include some collaboration between faculty at different levels to develop curriculum and mentorship, with either faculty at the college level with students, or college students with high school students. Are there other ingredients that we should focus on or highlight as being important to the success of these kinds of programs?

Jill: I think you said it so well, that it’s mostly customized opportunities for students and the places where we find success. And I was just reading the Journal of Higher Education this month, and it was talking about STEM student success and STEM student success at the community college level. And it was really focused around this idea of faculty in the very first year creating an experience for students that gets them engaged in their learning experience across the curriculum… and I think that idea of faculty engagement at the very beginning of learning experiences that are aligned to the student’s personal and professional goals. And again, it comes back to rigor. The students are so inspired by rigor, the students are so inspired by the benchmark is high, and these are the things that I can learn to be successful and to be like faculty. And so I just think faculty role modeling, faculty advising, academic success is so important. We also have the great benefit of SUNY of so many amazing student affairs professionals who also provide coaching and leadership and just a really well rounded approach to student development. Like I’ve mentioned before, the EOP programs are so successful because of this. So I think you’re right, Rebecca, the magic ingredient is collaboration. It’s also about rigor. It’s also about looking at the research and data and talking about what programs can actually generate data to improve success. I also think I’ve learned a lot from faculty about the idea of sort of an interdisciplinary approach to learning I often find that for me in education, if I go and look at that economics literature or the anthropology literature, or the political science literature, I can also learn a lot about how to help to bring these other resources to the table to really think about what success looks like. I recently had the opportunity to work with some medical professionals around the development of guidelines and the rigor to which they looked at guidelines and looked at evidence in the field to make judgments about what they should do, and the way of supporting medical advancements was so powerful that I thought we need to benchmark this in education. So I think looking at what the data tells us, and we’ve had our colleagues, like I mentioned before, at the Community College Research Consortium, at Jobs for the Future, at Achieving the Dream, and so many more places looked at the data for us. But I think that we need to be focused on the long term. And we need to think about what is the data telling us and how that can continue to improve our practice. And then I often think about the importance of qualitative data, what the faculty are telling us in the different disciplines. So that’s why it’s such a great opportunity in this position, because I’m working with faculty, often from economics, from English literature, from science, from political science, from public administration, so many different perspectives. But they all really come to bear when we talk about student success. So that’s been really tremendous for us here at SUNY,

John: Many of the programs you talked about have been at the community college level. And these don’t seem to be quite as common yet at comprehensive institutions. What can traditional liberal arts colleges do to reach out in the ways that community college have been doing for decades? What else should four-year colleges do to emulate the success that many community colleges have had in bridging some of these gaps?

Jill: I really think that you do it well. And I think this is where the SUNY advantage comes in. Because the quality and the dedication of faculty at the four-year colleges and the comprehensive colleges and the University Centers are amazing.

John: What can faculty at four year institutions or University Centers do to reach out to students in high schools to help them see the potential that’s available to them.

Jill: When they have an opportunity to listen to podcasts from faculty or actually engage in their research, it becomes really powerful to these students. And I think part of the challenge with faculty is really in finding a way to kind of create an inroad with these students. And so that’s why I was talking earlier about research opportunities, or really introducing students. I think you also have to help the students learn more about your expertise and try to again meet them where they are, trying to contextualize learning so that the expertise that you have at the comprehensive level and the university center level really can be matched to something that they might see an interest in long term. I’ve worked with an organization called the Army Education Opportunity Program. And this was through an opportunity that we had with Battelle. And the army is trying to recruit the future researchers, scientists, engineers. And what they encourage students to do was to work in groups with a faculty member and a science mentor to address challenges in their own local community. So, what kind of STEM challenges would they need to accomplish in their community? So, they look at things like recycling or safety and parks or child safety. But when the student can see how your expertise at the faculty level applies to their lives, then they are so engaged and faculty will tell me that they’re often outperforming them… that we have students in Long Island, for example, that are applying for patents on downloading data from their computer to a flash drive in instantaneous time, because they actually were listening to a faculty saying, “Ah, this takes so long to download that I actually need some help with this.” So they figured out the magnetic structure behind the thumb drive, and they actually created something that would help them to expedite the time in which they could download data. It was a big win-win, because the faculty was overjoyed. And also the student was able to learn the fundamentals from the faculty member to actually create innovation and to create change. So I think you have it, I was going to talk a little bit about the college choice process, which is so interesting to me, because students in high school have so many options, so many choices, there are 270 plus colleges and universities in New York State alone. And so when you think about the whole universe, and how students go about selecting a place where they can see their baccalaureate success dreams come true or their associate dreams come true. And think about graduate studies, what are they looking for? And based on the literature, after they go through the first predisposition phase, and then the search phase, they actually always talk about student culture in terms of why they make a decision about one college over another. And I also have seen literature that suggests that retention rates are impacted by the choice process around how much the student believes that they fit in. And so it’s hard for faculty at that point to connect with students. But I think that, to the extent that they could appreciate students’ development process and understand that their professional and personal goals and really try to connect them with their own research interests or their own academic interest, I think that can be very powerful in and of itself to really connect the student with the college culture, and with the rigor and academic excellence expectations that faculty have. I think it’s powerful. The other thing I think, is that Chancellor Malatras has been very committed to faculty diversity, because I also really believe that students need to see themselves in their faculty and see themselves in their mentoring experiences. So faculty diversity is also very critical. Our graduate students have been helpful with that as well, to try to make the connection between the faculty ranks and also with our colleges. The Education Trust has some amazing data around faculty diversity at the college level and also at the K-12 level. Because I really do think we need to do more in both arenas to really achieve change for our students, because our students are changing. And the other thing that drives success, and the main thing that drives success is student voices, because we have to hear from our students. And the more that I work with students and work within the framework of student success, I keep going back to the literature to find out what it is that motivates students of various backgrounds, how to really hear their voices, and how to integrate their voices into our activities to really strengthen the outcomes of students in our colleges. So student voices are fundamental to this work.

Rebecca: I think you’ve given a good view of how much has already been done, but also how much there is still to do. So. we usually wrap up by asking what’s next? [LAUGHTER]

Jill: So, so much exciting things are next, I think. In our office, we have an opportunity that was made available to us through the Bill and Melinda Gates Foundation around trying to strengthen communications, advocacy and success tools for students. So the idea would be: how do we really make students understand the value added of SUNY, the value added of the work that we’re doing to try to improve college completion rates, and student success Initiatives. So I would welcome faculty feedback on this. As I said before, faculty have always driven the work that we’ve been leading in our office. And so ideas around how we can better communicate these opportunities that are available to our students and to our parents, also to our guidance counselors, and then policymakers and government leaders about what is the SUNY advantage and about how they can have a part in this, I will say that if you look at SUNY, affordability matters, affordability is huge. The student debt crisis is alarming. And that’s where SUNY has the value and the added advantage. Also, the graduation rates of SUNY often surpass the national average, and are very amazing in terms of what the experience is that post grad and into careers that we’ve been able to achieve for students. And the alumni networks are so strong. And so someone asked me the other day that the SUNY Community College Trustees have been very powerful, great advocates, and they asked, “Can you provide an example of what it looks like to be student of Guided Pathways? So, what is the outcome? And how is it different than it was before? So I guess if faculty could share with us their ideas on first of all, how to continue to improve outcomes around student success, and also how to improve communications, because often faculty are doing this in our classrooms, but they’re saying how do we get the word out that this is so good, that we’re really making a difference, like this student came to us not knowing all the opportunities, and now is head of the class or now is really making huge changes in their field? What are those things that we can highlight, also transfer opportunities, students that might come to community college and then go to our baccalaureate degree institution, or I’m always interested in those that go on to graduate studies, our business and industry partners have been trying to create programs that meet students at the high school or community college level, and then drive them all the way to be our future engineers and scientists and researchers and physicians and faculty and all of the great career opportunities are available students. So what are those pathways look like from your perspective? And I think with this funding, and with this opportunity to really try to communicate these outcomes, we could really have some power that would drive us into the future. So I appreciate the question. And we certainly welcome your input and opportunities.

Rebecca: Well, thank you so much, Jill, for sharing your experiences with us.

Jill: Well, thank you for the opportunity. And I would just say again, our faculty at SUNY are top notch, number one, and really have driven student success as we’ve started this work where we really put an anchor into the ground to try to help to strengthen the alignment between K-12, higher education and workforce, and citizenry. And I would just keep encouraging your great ideas, and we hope to continue to work with you into the future. So thanks for the opportunity to talk to you today. And it’s really an open door. So please keep your ideas coming.

John: Thank you. And we will share your contact information if anyone has any ideas that they’d like to sharee. Thank you very much.

Jill: Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

160. Inclusive Communication

Communication in academia has hidden and unwritten rules that present barriers for students. In this episode, Kristina Ruiz-Mesa joins us to discuss inclusive communication strategies we can use as teachers and mentors to help students feel like they belong in the academy.

Kristina is an Associate Professor of Communication Studies at California State University – Los Angeles. Kristina previously worked in diversity, equity and inclusion research at Villanova University, and as a communication and diversity consultant. Her research on these topics has been published in a variety of academic journals and in book chapters. Her forthcoming textbook Inclusive Public Speaking: Communicating in a Diverse World will be available in late 2020 through Fountainhead Press.

Show Notes

Transcript

John: Communication in academia has hidden and unwritten rules that present barriers for students. In this episode, we explore inclusive communication strategies we can use as teachers and mentors to help students feel like they belong in the academy.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare , a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Kristina Ruiz-Mesa. She is an Associate Professor of Communication Studies at California State University – Los Angeles. Kristina previously worked in diversity, equity and inclusion research at Villanova University, and as a communication and diversity consultant. Her research on these topics has been published in a variety of academic journals and in book chapters. Her forthcoming textbook Inclusive Public Speaking: Communicating in a Diverse World will be available in late 2020 through Fountainhead Press.

John: We can also note that we just saw you recently in ACUE’s webinar on Preparing an Inclusive Online Course, which was released in early October and is available online. We’ll include a link to that in the show notes.

Rebecca: Welcome, Kristina.

John: Welcome, Kristina.

Kristina: Thank you so much for having me.

John: Today’s teas are:…Are you drinking tea, Kristina?

Kristina: I am drinking carbonated water.

Rebecca: …out of a tea cup I might note.

Kristina: I thought it was appropriate. [LAUGHTER]

Rebecca: It’s a beautiful tea cup.

Kristina: Thank you.

John: That’s close enough. [LAUGHTER]

Rebecca: I’m drinking Scottish afternoon tea.

John: And I’m drinking a mix of peppermint and spearmint tea.

Kristina: Lovely.

John: We’ve invited you here to discuss your work on inclusive communication. First, though, could you tell us how you became interested in this area of research?

Kristina: Absolutely. So, the research actually started in my own life, a little more than 30 years ago. And so I grew up in southern New Jersey, in a really diverse town in a Caribbean family. And so my dad’s Cuban, my mom’s Puerto Rican, and lived in this really diverse place. And I went to an inner city Catholic School, where I was one of a few students of color and started noticing differences, differences between how our families communicated, how our teachers communicated with our families. And that sparked an interest in me in saying, “Eell, communication seems to not be one-size-fits-all, we all have different ways of communicating.” And yet, when I was studying communication, and when I was in learning, it was like a one-size-fits-all, like “if you do these communicative practices, you will get the same response.” And that was not the case. I didn’t find that to be the case. And so I wanted to know, how culture, how identities, how intersectional experiences impact the ways that we communicate, the ways that we construct messages, the ways that we analyze our audiences, and think about ways that we can train students to most effectively communicate. So, how they can most effectively communicate in different audiences in different places to achieve their personal and professional goals.

Rebecca: Colleges and Universities have become increasingly diverse, and the composition of faculty, though, not so much so… What sort of challenges does this present for communication between faculty and students?

Kristina: I think this is such an important issue, and one that we are feeling as faculty as well, as in “How can we best serve the needs of all of our students, and recognizing that representation matters in the classroom, and that communication matters in the classroom?” And so when I think about how do we address mentoring? how do we address teaching? And how do we address the practices that we are using in the classroom? What do our materials look like? And so we can’t change our racial identities, we can’t change who our students are, and we wouldn’t want to, right? And so how can we make sure that we are teaching all of the students and so one of the things that I always stress is your course materials. Regardless of subject, you have examples, and you have data sets that you use or readings that you’re using. And so, how are you incorporating more voices, more experiences more identities into the course. And so that can be a way to really show your students’ representation. If you feel like you are not representing all of the identities of your students, which none of us are, no matter what our identities are, we can never fully represent all of our students. So how can we bring in this idea of polyvocality? Lots of different voices, lots of different experiences. And sometimes that means thinking about the datasets that you’re using. Are they representative? Who are they speaking about? Who are they speaking to? Who are the scholars that we’re bringing into conversations? And so I think these are all ways that we can help address diversity, equity, and inclusion in the classroom, and make sure that our students see themselves in the course and see themselves in the materials. And obviously, yes, increasing faculty diversity, staff diversity, making sure that our students feel their experiences and their identities are a part of academia and a part of their institutions. Absolutely. And, there are things that we can do immediately in each of our classrooms to make sure that we are making our classrooms as inclusive as possible.

Rebecca: I like how you’re emphasizing our role or our ability to curate, and not just kind of be everything to everybody, but we can curate experiences that include many points of view.

Kristina: I love that you said curate. So, I always, when I teach my graduate students, I say we have like the coolest museum in the world, right? We get to pick all of these scholars and authors and examples and bring them together into one exhibit, whether that exhibit’s in a face-to-face classroom, in a virtual classroom space, we get to showcase different voices experiences, theories, and applications.

John: That can enrich the conversation by bringing in a diversity of examples and leveraging that diversity in the classroom to provide a richer learning experience.

Kristina: Absolutely. My mantra for teaching and thinking about teaching and what my course materials are, we always start by planning backwards. What do we want our students to know at the end of this course? What do we want them to remember? And I always think about how can I challenge the canon? So the canon that we all learned in graduate school, that we have been reading for decades, some for centuries this material has been going on. How do we challenge and think about ways to expand that knowledge, ways that we can incorporate new voices? And I think that that’s so important.

Rebecca: One of the things that I found really wonderful, and I feel like it’s actually happening more right now because we’re trying extra hard to include students in conversations and make them feel included in a virtual environment to allow them to co-curate with us and to pick sources and to share materials. And my reading list got really long this semester… [LAUGHTER] … ‘cause based on all the things that students have brought to the table, podcasts that they’ve introduced me to, videos that they’ve introduced me to, I have a long list of homework to do.

Kristina: Absolutely. And I love that right. I love that idea of “Okay, we’re co-learners here.” And there’s such a reach. And Rebecca, I love that you say that with podcasts. And my students have introduced me to so many artists and performers and theorists that I was like, “Okay, yes.” And they’re seeing it in social media. They’re seeing up and coming scholars whose work perhaps hasn’t come out and those big journals yet, but that they are releasing blogs, they’re doing podcasts, and I love the perspectives and identities and experiences and new knowledge that’s being incorporated through these venues and avenues.

John: Let’s go back to the mismatch between the diversity of the faculty and the more diverse student body that we’re finally getting in most colleges and universities, now. What’s the impact of that, say, on persistence rates for first-gen students and students from underrepresented groups?

Kristina: Absolutely. So, the research has consistently shown us that mentoring and inclusive pedagogical practices matter. I teach in East Los Angeles. And so, as a Latina scholar teaching a predominately Latino student population, as the only tenured or tenure-track faculty who is Latino, who is Spanish speaking, who can connect with families at graduation and at different ceremonies, I find that I have a very easy time connecting with my students and their experiences, even though our families are from different Latin American countries. I grew up on the East Coast, not the West Coast, I’m Caribbean. And so like all of these differences are still under this umbrella of, I think about, like, cultural norms. And I think about cultural values. And one of them that I stress in my teaching is this idea of familismo, this cultural commitment to family and the family role. And I think about how that influences student persistence. And we’re seeing it very clearly now on our campus. So, my role at Cal State LA is that I’m an associate professor, but I’m also the Director of Oral Communication in Communication, which means we have 4000 students taking a standardized general education oral communication course. And so my instructors see 4000 incoming freshmen every year, and we are hearing consistently this semester that workloads combined with having your classroom now be your living space with your families, how do we negotiate and how do we navigate these spaces? And that is absolutely going to impact persistence and graduation rates. And so I think, for faculty, understanding not only how your students are coming in, what knowledge they are coming in with, but understanding the cultural context in which they’re living, and how that may be impacting the learning experience, the needs of the students in terms of… I always think about applied skills, I teach communication, and so when I came into Cal State, LA, one of the first things I did was say, “How can we get an interview assignment into oral communication?” It’s not part of the general education requirements of the state. And so I went to the chancellor’s office, and I said, here’s my pitch. 80% of our students are first gen. We know that interviewing skills, so much of it is based on these unwritten rules and laws that you learn kind of through family, through friends. But, if you’re your first person in your family who’s gone to college, you might not get those experiences kind of organically. And so we needed to embed it into the general education requirement so that all students benefit from it. And again, the universal design we’re talking about, no one’s going to be disadvantaged from learning interviewing skills and practicing interviewing. And so, I think, thinking about persistence in really applied ways and material realities matter. How are we going to get students to get those internships to get those jobs? And so thinking about how our skills can be taught in a way that is problem posing, and that can be applied to students lives as soon as possible.

Rebecca: What I like about what you’re talking about in terms of the oral communication piece is that it’s such a big part of being professional in every discipline, but we often teach public speaking classes as if it’s a very separate activity. [LAUGHTER] Like, I want to stand up and give speeches. I don’t stand up and give speeches, and most people don’t, the kind of communication you do is different. So, putting it in context like that, and providing a clear application of how those skills can be used somewhere, I think is really helpful, especially for students that don’t have that kind of context to build from.

Kristina: I totally agree.

John: And you mentioned some of the challenges associated with students interacting with families in their homes. One of the issues that faculty keep raising is “Our students won’t turn on their cameras.” And we address that regularly with faculty. But, it’s an issue where faculty are used to seeing faces on the screen. And they’re really upset when people choose not to. How do you respond to that?

Kristina: This is something that I have been hearing in my circles as well. And well meaning faculty are frustrated, because we know that a large percentage of our communication is nonverbal. So, if we are missing those nonverbal cues of understanding, of confusion, it is limiting our ability to be able to connect with our students that way. I get that. And the hard truth is that it’s not about us. And so that’s one of those tough kind of answers. Because, right now, it’s about our students and their success, and whatever we need to do whatever practices that we need to kind of adapt to, it’s about them and about their learning. And so one of the things that I have done is incorporate more of the thumbs up, thumbs down, type in the chat. So you can do a popcorn response by giving an emoji. So offering students various ways of interacting, I think is huge. Also, normalizing the ways that we communicate. So, for a speech, for example, we do want to see them in terms of their nonverbals, we want to see your gesturing, we want to see the ways that you’re connecting. And so we normalized giving speeches in bathtubs, giving them from parking lots, giving them in cars, doing our own mini lectures from like, on the floor in the bathroom, because if we’re doing it, then you can do it. And so kind of modeling, that it’s okay, and that we don’t all have these perfect offices that look like they came off of HGTV, and that there might be a dog barking in the background or someone crying. And that’s okay, this is a global pandemic, there are more important things than whether you can hear a baby crying, or a dog barking, or someone in the background. And so I think also being realistic about our expectations, and as empathetic as we can. And one of the things that I often think about is that many of us teaching at the college level, we’re in the top 5%, top 2% of higher education attainment, how we learned and our experiences and how we are now… We have to remember. We have to remember, what was it like to be an undergrad? And for many of us, that meant “Where are we studying? How could we study, if you don’t have the privilege of going to a library right now or a quiet space?” …then being empathetic enough to know that you don’t understand all of the experiences and lives of your students and give them the benefit of the doubt. that they are trying their best. and they’re doing the best we can… all of us.

John: One of the things I asked my students was to share some of their challenges in a low-stakes discussion forum. And I’ve been amazed at how many students talk about just how difficult it is to find time that’s quiet. They may have a spouse or a partner who’s playing live video games, or more typically, they may have small children or they may have siblings in the rooms or in the dwellings with them. And that makes it very challenging where some of them are saying “I wake up at six in the morning, just so I can find some quiet time in order to do my work.” Or, “I have to wait until everyone’s asleep after midnight or at one in the morning.” And it’s something I think we do need to be a little more cognizant of… even just asking them what sort of challenges they face, perhaps, can help faculty adjust to this somewhat challenging environment we’re all in.

Rebecca: Are you sure those are students talking? Because I feel like you just describe what I’m doing. [LAUGHTER]

John: Faculty have had very similar challenges since last March.

Rebecca: I do think, actually, the struggles that faculty are having with family and things being in the same space as them has actually really, really helped start to connect to some of the real challenges that students face regularly, and not just during a pandemic.

Kristina: Absolutely. And then we compound that with housing insecurity, food insecurity, and the things that our students are experiencing. Just every time my students come into my class, I thank them. That’s the first thing I do. Thank you all so much for being here. I’m excited to have our conversation. And I think that goes a long way. And at the end of every class, acknowledging that, and say, “I know that you’ve got a lot going on, and I am really proud of you.” And I think that that transparency of saying, “This is why I need you to do this assignment. This is why I gave you three readings instead of two.” And I think really explaining the “why” is going even further than it has in the past. And so thinking about the ways that we can make our assignments and our assessments as practical and applied as possible… really helpful right now… as well as checking in with students. I’ve been doing the first kind of 10 minutes of class checking in. Now, I know that’s not possible for all classes, and for all students and for every class, but when it is and when we can or a discussion post, tell me the best thing that’s going on in your week. Just connecting, and having this connection in the classroom, I think, is really important now for maintaining not only community and engagement, but also persistence.

John: Ggiven the challenges you’ve mentioned with communications between faculty and students, one of the issues that may come up is microaggressions. And I know you’ve done some research on that. Could you tell us a little bit about your research on microaggressions in the classroom?

Kristina: Sure. I did a study on microaggressions at a predominantly white institution of higher education and looking at racial microaggressions that students of color were experiencing on campus. And so, just as a quick recap, Wing Sue defines microaggressions as kind of brief commonplace verbal behavior, or environmental indignities. And they can be intentional or unintentional, and they communicate hostile, derogatory, or negative racial slights and insults. But microaggressions can be about sexuality, about social class, about gender. So, they can be across identities. And my research showed that African-American males and Latino males experienced microaggressions at the highest rates of any students. And the experiences oftentimes lead to what we’d call student misbehaviors in the classroom. If students are feeling disrespected by an instructor or by other peers, there was a few paths they would take. One is they would act out in the class. So, they might say things, they might be seemingly disrespectful about the material about the course. They would drop out, and you would never hear from them again. You wouldn’t know why they weren’t engaging the class, they were just gone. And we also saw psychological stressors. So, higher instances of isolation feelings, that they didn’t belong on campus. And again, this was a predominately white institution, and so students of color have these feelings of belonging, questioning of belonging. And so when they experienced microaggressions, these feelings were exacerbated, and they increased experiences of anxiety, depression and social isolation. What my research found was that, if we could inoculate against microaggressions by offering micro-practices and services on campus, that was where we were able to support students in building academic habits that would help support their success. And so this inoculation came in the form of having Diversity, Equity and Inclusion centers, having counseling resources, having safe spaces and inclusive and brave spaces where students could share their experiences. So that it wasn’t just one person saying,”It must be me. It’s something I’m doing.” But, recognizing that these were structural and systemic, and these were problems that were permeating throughout the campus. And so that was something that we found in the research was that primarily African-American males and Latino males were experiencing this more often on campus, and that the ways to minimize the academic impact was to offer services early and often, having male mentoring groups on campus was helpful and having spaces where students could share their counter-narratives and counter-experiences on campus. All were beneficial.

John: And that’s a useful form of remediation, but what can be done, perhaps, in the classroom to address those as they occur?

Kristina: Absolutely, that is my number one piece of advice for faculty is when you see something, when your, like, hairs on the back of your neck are standing up, you’re like, “Oh, this isn’t good,” you need to say something. And that is something that is scary. And for many of us, particularly folks who are not tenured, who are contingent faculty who are hired by the quarter or semester, that can be really scary, because we know that student evaluations matter. Having grievances can affect your job. And so that, and I’m in a privileged position, I’m a tenured state university professor. So I recognize that. And I think that it’s important that if we are going to have inclusive conversations, inclusive learning environments, we have to intervene. Now, knowing how to intervene takes practice, and knowing that you’re not going to get it right every time is humbling, and knowing that we’re always learning and that’s one of the things that I always stress to faculty is that we are literally trained for this we are trained to learn. That is our job, our job is to learn as much as we can, figure out new, innovative, cool ways to apply it, explain it, expand it, that’s the gig. And so this is another area of knowledge that we need to learn, that we need to just say, “Okay, I needed to learn a new computer system, I needed to learn how to teach online, I need to learn what my students are experiencing, so that I can be a better teacher. So that I can learn what has already worked, what practices are embedded.” And so one of the things that I’ve done in the last few years, and that I found to be helpful is to write down what are the specific practices? …not just saying “You need to be an inclusive educator.” Cool. What does that mean? And what does that look like in my classroom. And so, one of my most cited articles is this quick, best practices piece that I can share the link with. It’s a free download. And it’s 10 Best Practices for Facilitating Difficult Dialogues. And it’s tips, for example, like we disagree with ideas, not people. So we focus on the idea not the person, the other is maintaining immediacy, so making sure that we’re talking at the end of class, you don’t leave conversations undone or unsaid. So keeping track of time and recognizing that you might need two or three minutes at the end of class to do relationship repair, to do community check-ins, to do that repair… really important. Also making sure our language is inclusive. So, thinking about the ways that we, from day one, are establishing inclusive language. Are we getting rid of kind of gender binaries and making assumptions about student genders? Are we asking students: “What is your name?” I never read out of rosters. I always have students introduce themselves. Tell me your name. Share your pronouns with me, and modeling that for students. I also include a pronunciation guide because much like we want our students’ names to be honored, we want our names to be honored and said correctly. So, offering tools and resources and normalizing this in communication, whether you’re teaching comm, psychology, math, chemistry, normalizing that this is how effective communication works. And I think that’s really helpful in the classroom. And of course, setting the ground rules, setting the tone, the things that we know as faculty that we ought to do. But those are some of the big ones. And also, the “oops,” and the “ouch” rule is something that we use a lot and saying that, again, in a single 50 minute, hour and 15 minute class, I’m going to say thousands of words. The chances that one or two of them are wrong, or came out too quickly. Or I didn’t mean to say that? Likely. So, recognizing and having the humility to say, “Okay, if I’m going to say an oops, that was my bad. Let’s start over. Let’s take that again.” And, recognizing that if I miss something, having a mechanism in place with the “Ouch,” to say “That was hurtful, I didn’t appreciate that. Can we talk about that for a second?” And pausing and saying, “I’m sorry. How was that hurtful? I’m sorry.” And acknowledging the moment. And I think these are practical things that can feel super awkward if we don’t establish them on day one. But, if it’s just how things are, the beauty of being a college professor, is that every 10 weeks, 16 weeks, quarter semester, we get to start over. And so, re-establish the norms, re-establish how we communicate and how we want to communicate for an inclusive environment.

Rebecca: If you think of it that way, we get so many do overs.

Kristina: Exactly.

John: Eventually, we’ll get things right. I’m still waiting.

Rebecca: That’s empowering. Yeah, I really love the idea of the oops, and the ouch, and really establishing the idea and reminding ourselves that we’re learners too. And we make mistakes, and it does take practice. But just like we want our students to take that first try, we have to do it too. Boy, we should listen to ourselves sometimes,

Kristina: Right, once in a while. [LAUGHTER]

John: Would you recommend that, perhaps, when you have those rules, you give students some say in discussing them and establishing the ground rules?

Kristina: Absolutely. I usually have a few rules that I propose. And then I ask students to add to them, and we do a Google Doc in class, and they can add them in real time. And then I also say from now until next week, review them. If something doesn’t feel right, if you want further explanation, let’s write it out, and let’s talk about it and see how we can come to this together.

Rebecca: One of the things that I really recognize teaching more online than in person is how much more time there really should be to do some of those things at the beginning of the semester, in any semester. But I took the time this semester, and it was really helpful.

Kristina: Love that, that is one of the benefits of teaching online is that I feel like if I miss something, I can make a video, there’s time to kind of fix it. Whereas in face to face, I can send an email, but it’s not the same. Whereas, if everything is built into my learning management system, it’s another opportunity.

Rebecca: So, we talked a little bit about privilege, and how that might impact the kind of experiences you have access to. And one thing that I think we don’t always consider is how our own race, gender, social status and ability status, impact our own social norms. And we don’t necessarily recognize them as being social norms, or that somehow we learned these behaviors, what are some things that we could think about as faculty to better understand what those practices are? And to undo some of them maybe, or at least recognize that there are norms and invite students in to understand that?

Kristina: One of the kind of keys for me is when I hear the word “ought,” like “it ought to be this way,” or “it ought to be…” and I’m like, “Hmm, says who? A really important part of being a good teacher is recognizing that we cannot be all things to all people, and that we have to be critically self reflexive. I read a lot of Bell Hooks work and think about the ways that Hooks asks us to be kind of these self-actualized beings. How do we model the vulnerability and the space? And again, I recognize, I teach communication, I’m humanities professor, I have kind of more flexibility than my spouse who teaches chemistry. And so this idea that it’s going to look different in different classrooms. Absolutely. And, thinking about the ways that we come up with examples, I think, is a way that reflects our own identities. And so one of the ways that I think about that is psychological noise. And so, am I giving an example that is helping students move along in their understanding of a concept? Or have I just put up a giant roadblock because I used an example that’s not clear. And now they’re thinking about the example and they’ve forgotten the concept. So recognizing which examples are from a privileged experience… If you’re giving an example in your like, “So, let’s say you’re in Paris eating a croissant,” and you’re like, “Cool, I saw Emily in Paris, does that count? That was a good show.” And now they’re starting to think about a tangent, that they forgot what you’re teaching. And so, thinking about the ways that our examples can demonstrate our own privileges, and recognizing that talking about more privileged experiences, like, I was thinking about this the other day, when students were talking about having to go to the grocery store, and I was thinking about how many people in my circle were like “Groceries have been delivered since March” and the privilege that that reflects about saying, “Oh, no, I’ve been perfect. I have not had to leave my house.” That’s a privilege. And recognizing that we have paid positions, we still have jobs. And so recognizing that how our examples are privileged, I think, is really important for all of us. And I find the longer that I’m teaching, the more I have to kind of check myself, the more I have to say, “Is this a universal or pretty broad experience? Is this the example resonating?” Is this, as my students would say, “Is that just really boojie?” Like, is this just a really privileged expensive thing, and I’m like, “You caught me.” And I think being humble enough to recognize what our own racial financial gendered positions are, and how our experiences may be tied to those identities and experiences and how that may differ from our students. So, I think that’s something. Examples are one way that I think are really something we can all work on. The other is the ways that we make assumptions about what students ought to know. I’m big on saying that we don’t have underprepared students, we have underprepared teachers, because our students are who our students are on day one. And that’s where we teach them from. What they know is what we know and we’ll build. And I’m very big on understanding that it is my obligation in these 16 weeks to teach them as much as I can. But I have to start where they are. And that’s my job. And if it means that I have to go back in week one, and stay up till midnight, redoing my course schedule, so be it. That’s my job, to make sure that my students are learning and recognizing that where I think they ought to be doesn’t matter. It’s where they are that matters. And that’s our starting place.

Rebecca: So, the way we prevent too much workload at the beginning is we just don’t plan the like last five weeks of the semester, so that if you need to add stuff in the beginning, you can just shift everything.

Kristina: Well, I have my syllabus, and it has the first five weeks, and I always say tentative at the top, and I say this is going to serve the needs of our students and we’ll adjust. And, I think, Rebecca, you hit the nail on the head. Yes, being flexible and adapting and saying, “If we need to take two weeks on this, but you learn it, that’s more important to me than just kind of checking off my boxes, like, Oh, good, we’re in week eight now or week nine.” Absolutely.

Rebecca: I had a conversation with my students this week about projects that they were working on, and they were getting frustrated because they weren’t being as productive as maybe they would be in a non-pandemic situation. Imagine that.

Kristina: Right?

Rebecca: And so they’re like, “But I don’t know how I’m gonna get it done.” It’s like, well, because you’re being unreasonable. Let’s take that back a couple notches, the thing you’re talking about, that’s your next revision. That’s next time. That’s not this time. And I think having those conversations with students about kind of a reality check of what’s even reasonable right now is helpful, because there are these norms of what maybe a normal semester is like, that’s just unrealistic. And maybe it’s unrealistic all the time.

Kristina: Absolutely. And I think for ourselves, too, as faculty, I mean, I have found myself, I don’t know about you all, but I’m working seven days a week. And I’m like, this is not healthy. This is not sustainable. And I’m telling my students, and I’m really open with them. I teach mostly graduate students, but I’m really open with them saying, “Please do as I say, not as I do, because I’m still learning, and I’m still a work in progress, and I’m still trying.” But, I don’t want them to fall into the same patterns that I’m falling into where it’s midnight, and we’re still working. And it’s all the time. And I think that that leads to burnout. And. I know I have been meeting with many more students than in a typical semester. And it’s more one-on-one meetings. And I appreciate that, and I value our time together. And I also am recognizing that I’m making appointments, like from seven, eight in the morning, all the way until late at night. And so our days are kind of blending. And I think that that’s really stressful. And my colleagues who have young children, I feel for them, because they are just working nonstop. And I think we have to be kind to ourselves, we have to show ourselves and our students and our colleagues grace. And to say, Rebecca, I think as you say, this is a pandemic world. So let’s all chill with our expectations, here. [LAUGHTER]

Rebecca: And I think along those lines, emphasizing still how much learning is actually happening.

Kristina: Yes.

Rebecca: …because, what I’ve discovered, is not that students are learning any less. They might be producing less work, but the quality is actually quite good.

Kristina: Absolutely.

Rebecca: And they’re demonstrating that they’re meeting the learning objectives. It’s just maybe there’s some things there that didn’t need to be there.

Kristina: And I don’t know if you all are seeing this, but I’m finding there’s like a decentering of faculty because I’m not lecturing for three hours in a graduate class. I’m, again, curating materials, making mini-lectures, and then using our time together when we have synchronous time, for discussion. And so I’m finding it to be really enriching. Our conversations are great. The chat… students who I have not heard from in previous semesters are now super engaged and participating because they feel more comfortable. Perhaps there’s communication apprehension, and they didn’t want to speak up in front of everyone, but they can chat and they can type in the chat, and that is another avenue. So, I think we’re also seeing opportunities for further engagement and students really taking on the ownership of saying, “I need to do the reading, because I’m not going to get a three-hour lecture, and so I can’t depend on that. I have to depend on myself.” And I think we’re going to see on the other end of this, perhaps, stronger practices of self efficacy and engagement.

Rebecca: I had a whole class of people who read their stuff today. It was amazing.

Kristina: Amazing. [LAUGHTER] Love that. Love it.

John: I haven’t quite gotten there with everyone. But I have somewhat larger classes, too. But yeah, some of the things that we’ve been doing in terms of having people have the chat capability as a backchannel has been really enriching. And I’m hoping that that becomes more widely adopted later. And also, the move to online discussion forums also gives more students a voice than would occur with synchronous communications, because there’s always some people who want to think and process things a little bit more before they jump out there and say something. And I think in that way, at least, we’ve moved to somewhat more inclusive environments. In many ways we haven’t, but at least that’s one area that I think can be useful moving forward.

Kristina: Absolutely. And I think that, John, exactly to your point, I think that we are creating some more opportunities for engagement. And I see the big barrier is getting folks in the classes and making sure they have the WiFi making sure they have a device. I think that’s the big challenge at the beginning of the semester. And so thinking about planning for next semester, for many of us who already know that we are going to stay remote, is thinking about how those first two weeks can be really flexible, because it might take students a while to get access after the holidays and after the New Year. Depending what happens with the election and different things that are happening, they might need a little bit more time to get their financial aid checks. And so thinking about how those first few weeks can be caught u, I think is gonna be really important for the spring

Rebecca: I think that’s a nice lead into how we normally wrap up, which is: What’s next? {LAUGHTER]

Kristina: Who knows? [LAUGHTER]

Rebecca: That’s it, that’s all there is.

Kristina: Who knows? What’s interesting to me is when I think about the possibilities for higher education, I think this is really exciting. So, when I think about the different, you know, 1636 and Harvard’s founding, we have seen really slow change in higher education. And all of the slow change was laughed at in March when they’re like: “Guess what? We are going from moving the battleship to like a jet ski right now. We are going fast, and we are hoping for the best.” And so I think we’re gonna see some rapid and lasting changes in US higher education that would have taken decades had there not been a pandemic. And so my hope is that we are going to increase hybrid offerings, we’re going to increase our capabilities of serving more students by offering more online options. And my hope is that institutions will respond by creating tenured and tenure-track lines or online, totally online, programs and teachings. And we’ve got more than 3000 institutions of higher education in this country, that we can really create more access and engagement and higher education achievement in this country. That’s my hope for what’s next.

Rebecca: I think ending on a hopeful note is a good thing. [LAUGHTER] It’s a time when we need a lot of hope.

John: Certainly.

Kristina: Absolutely.

Rebecca: Thank you so much, Kristina. You’ve given us lots to think about and actions to actually take.

Kristina: Thank you. Thanks for the opportunity. This was super fun. I enjoyed it very much. I enjoyed our conversation

John: We have too and we hope we’ll be talking to you again in the future.

Kristina: Anytime. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

155. Remote Proctoring

Faculty who rely on high-stakes proctored exams in their classrooms often attempt to replicate this approach in online instruction by using remote proctoring services. In this episode, Jessamyn Neuhaus and John Locke join us to discuss some of the issues associated with the use of remote video proctoring and suggest some effective and less problematic alternative methods of assessing student learning.

Jessamyn is the Interim Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at Plattsburgh. She specializes in the study of pop culture, gender studies, and teaching and learning. Jessmyn is the recipient of the State University of New York’s Chancellor’s Award for Teaching Excellence. She is also the author of Geeky Pedagogy: a Guide for Intellectuals, Introverts and Nerds who Want to be Effective Teachers. John is the Coordinator of Technology Enhanced Learning and an adjunct instructor in Communication Studies, also at SUNY Plattsburgh. He recently received his doctorate in interdisciplinary studies with a concentration in humanities and culture, and is currently working on a second historical novel.

Show Notes

Additional Resources/References

Transcript

John: Faculty who rely on high-stakes proctored exams in their classrooms often attempt to replicate this approach in online instruction by using remote proctoring services. In this episode, we discuss some of the issues associated with the use of remote video proctoring and suggest some effective and less problematic alternative methods of assessing student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Jessamyn Neuhaus and John Locke. Jessamyn is the Interim Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at Plattsburgh. She specializes in the study of pop culture, gender studies, and teaching and learning. Jessmyn is the recipient of the State University of New York’s Chancellor’s Award for Teaching Excellence. She is also the author of Geeky Pedagogy: a Guide for Intellectuals, Introverts and Nerds who Want to be Effective Teachers. John is the Coordinator of Technology Enhanced Learning and an adjunct instructor in Communication Studies, also at SUNY Plattsburgh. He recently received his doctorate in interdisciplinary studies with a concentration in humanities and culture, and is currently working on a second historical novel. Welcome, John, and welcome back, Jessamyn.

Jessamyn: Thank you. Thanks for having us.

John L.: Yeah, thanks.

John: Today’s teas are:

Jessamyn: Just plain water for me. Gotta stay hydrated.

John L.: Grande decaf from Starbucks.

John K.: That’s an interesting tea.

Rebecca: Yeah. [LAUGHTER] I have a Scottish afternoon tea

John K.: …and I have ginger peach green tea.

We’ve invited you both here to talk about online proctoring services. As a result of the global pandemic, a lot of people suddenly had to shift from face-to-face instruction to remote instruction or online instruction. And many people who relied on proctored classroom exams are concerned about how to offer tests, and many faculty have been investigating the possibility of using remote proctoring services. What are some of the concerns associated with using online proctoring services?

John L.: Well, to start with, we are all trying to deal with the digital divide. And when you get into online proctored exams, that becomes a pretty big issue in that not all students have the equipment or the bandwidth to be able to participate. It helps to know what the process is. And basically, what we’re dealing with is a test that’s happening while the student is being recorded, both audio and visually being recorded. Usually, it starts out with a little intro section where you have to show an ID to prove who you are, show your space so that everybody can see that you don’t have crib notes on your desk, or there isn’t Albert Einstein in the corner of the room [LAUGHTER] telling you the answers to what you’re working on. And assuming all that goes well, then, of course, you’re taking the tests, usually an online test with a lockdown browser so that you can’t surf for answers anywhere else. It’s a lot of moving parts to make it work in the first place. And the big assumption is, number one, the student has the equipment necessary, and the student has the environment necessary to take a quiz like that. For instance, if you happen to be a student who lives in a very small apartment with a family, and you have brothers and sisters running through the room where you’re taking the test, because you’re at the dining room table, there are so many issues that come into play, not to mention just the fact that you may be embarrassed by your surroundings and don’t feel comfortable showing those surroundings to other people. So for me, that’s probably the first and most critical reason why I always talk to faculty and ask them to think about it before they actually devote themselves to that process. Other issues are, try as you may, there are always ways to get around these sorts of safeguards. And if there’s one thing we know, it’s that somebody who plans to be dishonest will figure out a way to be dishonest. Again, I try to get instructors to be a little more thoughtful with how they’re going to assess that learning is taken place in the first place. And that’s really where my friend Jessamyn has opened my eyes to many of the alternative ways.

Jessamyn: Yeah, there’s a lot of great resources that have been proliferating since the emergency pivot in response to this very question and suggestions, building on research that was already there, for how to assess student learning and in authentic and, as John was mentioning, equitable as possible way. I guess, just what I would add to that in terms of looking at it as a scholar of pedagogy, and taking messages like from James Lang’s book, Cheating Lessons, what do you want to foreground in your message to students in the class climate you’re creating, in the rapport that you’re building with them? The ordeal of the kind of proctoring software that John was describing, and that we were increasingly seeing problems with… the very first message you’re sending to students is: I assume students cheat, I assume students are going to be dishonest. I assume students don’t care about their education enough to try to express their learning as honestly and authentically as possible. And I guess what we, as what John and I both, were inviting faculty to consider when we were doing workshops this summer on this topic is: are there alternatives to this that send a more positive message and create a more productive class client and help you connect to students? Let’s not forget, at a time when everybody is anxious and overextended and fearful, we’re still in the middle of a pandemic. So, what do you want to prioritize as an educator?

John L.: Yeah, and exams are stressful enough as it is. So you add COVID on top of that, and then you add a technology that students aren’t used to. And it’s so much easier to choke under that environment.

Jessamyn: Yeah, an anxious brain is not a brain that can clearly and, to its best ability, express what it knows and show what it knows. All the information about trauma-informed teaching just reminds us that if every chemical and message in your brain is saying, “Run away from the tiger that’s hiding in the jungle,” there’s no room to: “Okay, move your webcam to show behind your ears that you don’t have an earpiece. Now take your laptop over to the door and show that it’s closed.” How is that not creating a prey state of mind with the predator waiting to pounce on you?

John K.: Each of the issues that you both talked about also have a very differential effect in terms of creating an inclusive classroom environment. People from high-income households are more likely to have some nice quiet space, are likely to be able to afford equipment that will work with proctoring software, while Chromebooks and most mobile devices will not work well with proctoring services. And also issues of anxiety and concern about being successful are also probably more likely to be experienced by students who are first- gen students who don’t necessarily have the same expectations of being successful based on their family environment and their social networks. One of the things that concerns me about all this is that the impact would be differentially imposed on students who are already at a disadvantage in terms of the quality of their prior schooling and their resources and their support networks.

John L.: Yeah, I couldn’t agree more. I’m not sure what to add to that, John. [LAUGHTER]

Jessamyn: I can jump in though. I had a thought. I’ve been reflecting… I can’t get it out of my head from a webinar this week that the Chronicle of Higher Education did a panel about the human element in online learning. And one of the panelists, Viji Sathy, mentioned that this crisis has really brought home to a huge new number of educators that we are teaching whole students… that taking into account all aspects of students experiences, their work experiences, family experiences, and these equity issues. So, it’s not that academic inequality is brand new to 2020. But, the awareness of it has really increased and the attention to it has really increased. And I think it’s being highlighted in ways that it’s just impossible to look away from. So this specific issue is touching on, I think, a bigger kind of reckoning that faculty are having on an individual basis, and as institutions. I see a lot more individual instructors really asking, “Wait, am I being inclusive?” The question is way more in people’s minds than I think it’s ever been, in my experience.

Rebecca: Related to that is the idea of accessibility too. With so much delivery in digital formats, the topic of digital accessibility is becoming much more prevalent in the forefront of faculty’s minds, whether they want it to be or not, it becomes something that everyone’s becoming more aware of. This same kind of software also imposes a lot of accessibility issues and barriers for students with disabilities, because a lot of them are not compatible with assistive technology and aren’t built to Web Content Accessibility Guidelines, essentially.

Jessamyn: And related to that, students with anxiety issues, who are struggling with mental health issues… the high-stress, high-stakes examination, in any format, is a challenge. But add to that the technology aspect of it, you are looking at assessment mechanisms that really isn’t being accessible and inclusive, it would not allow all your students to show you what they know.

John K.: One concern that I have about proctoring services is that faculty may see it as a simple solution that will allow them to use tests that they’ve created in the past. Many people have created very elaborate test banks in Blackboard and other places and then they expect that those questions can now be used, if they’re used in a proctored environment, not realizing that most of those questions have already been distributed to multiple sites out there and students would often have access to them, anyway. So I think that proctored systems can provide instructors with a false sense of security and as John mentioned earlier, they can be pretty easily defeated as long as students have devices that will allow them, for example, to do screen shares in the background underneath the proctoring service or perhaps have multiple devices where they can be looking up answers or using some other mechanism that won’t always be easily detected by the proctoring service.

Jessamyn: That’s a good point, and I know John Locke has addressed that issue. I mean, you don’t drill in on it, but when you’re talking to faculty, you often say, “And by the way, this is not a magic bullet, even if you go through all the trouble of setting it up.”

John L.: The idea that somehow having someone else proctor your exam is going to save you time…. That’s not how it works. These proctoring systems just flag potential incidents. You still have to go through and you decide whether or not those are warranted as cheating or if they’re just someone sneezed. So, between setting up the exam and then reviewing the flags, looking for false flags, I don’t know if it saves anybody any time.

Rebecca: I’m team workload reduction.

Jessamyn: Yes.

John L.: Yeah. [LAUGHTER]

Rebecca: So, what do we say to faculty who ask about replicating those high-stakes testing environments in their online environments.

John L.: I say: “Why?” I think that what would be more appropriate is to simulate the environment that somebody needs to perform in where they’ve acquired the knowledge in order to accomplish that performance. For instance, I taught a computer applications course years ago, and for the final exam…. I did have a final exam… but, I told them, “What I hope you get out of this class is to learn how to learn how to use software. So if you haven’t already learned how to learn to use software, now’s your chance. And when you’re out in the real world, you will have the software manual, you will have the person in the cubicle next to you, the only thing you won’t have is me. So, unless you have a question about a specific question on the test, don’t talk to me, as far as anything else that gets you to accomplish the goal, go for it.” If you’re studying to be an ER doctor, perhaps you do need to have the pharmaceutical manuals memorized page by page. [LAUGHTER] But most of us aren’t working in that kind of stressful environment. So, there are better ways, maybe project-oriented ways, to assess that that learning has taken place, that those skills have been received or learned and received.

Jessamyn: I try to assume best intentions on the part of all faculty. And I know that many of my colleagues who expressed that sentiment exactly, like “How can I make sure they’re not cheating?” …they’re not saying that because they’re evil, like “Mwah, hah hah hah, those bad students…” No, they really are concerned about student learning. So, what John and I did was really to frame this as an invitation to faculty, an invitation to think creatively about assessment, authentic assessment, to really be able to measure student learning, but maybe also rethink what you thought and assumed about assessment. And here’s a big bonus, maybe grading it could be less painful. If you are trying something new, something that’s a little bit more creative, that might help you as well in your end. So, that’s been how we’ve been addressing it here at Plattsburgh.

Rebecca: What are some ways to do that assessment, maybe in a class that doesn’t work well for project-based learning. Maybe it’s a bigger section class, or maybe it’s more foundational information that doesn’t lend itself as easily to project-based learning. What are some alternatives?

Jessamyn: There’s always small, lower stakes, regular quizzes. So instead of one big, huge exam, having smaller quizzes along the way. That’s just one off the top of my head… an easy one. John?

John L.: Yeah, well, especially in this environment, discussion forums are really, I think, underutilized. There’s no reason that you can’t build a rubric around a discussion forum and spell out your expectations to students and then hold them to them and grade according to those. Again, it’s taking the student higher up that Bloom’s taxonomy ladder than just memorizing and regurgitating information. It’s causing them to react to other people’s comments within the discussion forum, to assimilate the knowledge that they’ve already accumulated, and to create new and different responses based on that immediate situation. And, the advantage to that for slow thinkers like me, [LAUGHTER] is that you don’t have to be quick on your feet. You’re not the student in the back of the room with his hand up saying, “Well, never mind, you covered that five minutes ago.” It’s kind of an equalizer. I wouldn’t say “Have a discussion forum as a final exam,” but it’s another part of the scaffold to assess that learning is taking place throughout the semester.

Jessamyn: I think there’s a lot of potential for open-book exams as well. In fact, I have used open-book exams for a long time. And, in large part, that is because I really wanted my students to learn, and I wanted to be able to grade an exam very rigorously. So saying, here’s a question you can answer with an open book, and, yeah, you might even talk to someone else about it. But then the final product is an essay question, or it could be a presentation, it could be a sort of annotated bibliography. There’s lots of ways it could go as an open-book exam. But then when I go to assess it, I know that you have the material in front of you. So, I am going to really drill down here, like, “Do you really understand this concept? Can you show me that you understand it?” Because I know you can look at the basic definition in the book that’s open in front of you. So, now you have to show me that you really, really get it, you have to use it, you have to apply it, whatever it is.

Rebecca: What about STEM-oriented examples? A lot of the things that we’ve talked about work really well in the humanities and the arts. How about some things that work well in math and science and other STEM fields?

Jessamyn: So, I’ve been trying to do a little reading in this area. I’ve been hearing from some faculty in this area. So, in an online lab setting, being able to complete the experiment in the correct way, in the scientific-y way… [LAUGHTER] …that could be one way to assess learning… doing something like a fact sheet. So the final product is how you’re assessing the student learning. But again, you could be measuring the application, the correct way to do XYZ in a kind of fact sheet format or a PowerPoint slide or a poster presentation.

John K.: One type of thing we sometimes recommend for people in the STEM fields is that, if they are going to use multiple choice, one way of dealing with this is to use some algorithmically generated questions so that each student gets their own version of the question. Now, the solution procedure may be the same, but for at least low-level skills, that can help to deter some academic integrity issues.

Jessamyn: Student-generated exam questions could be another way to go. If you really understand the material, you’re not just regurgitating memorized material, but if you really understand it, then you should be able to help someone else understand it. And one way you could assess that would be “What are the 10 best exam questions?” …something like that.

Rebecca: Another idea that I’ve heard from people more in the STEM areas is the idea of creating some sort of resource that explains a topic to a non expert audience. So, maybe it’s an experiment or something that you can do with kids, or just kind of generally to someone who’s not in the discipline and get them to grasp whatever it is that you’re trying to assess.

Jessamyn: Yeah.

John L.: This might be going out on a limb for a STEM environment, maybe we could call it STEAM, because there is an artistic bent to it. But, for instance, in an accounting course, if there’s a particular accounting procedure or process that students have to prove that they understand it, they could write a short story, “a day in the life of the accountant to the New York Yankees” or something… and totally fictional, but covering each step in the process that has to be accomplished. And as an instructor, I would love to read something like that rather than checking off right or wrong on a test sheet.

Jessamyn: I’m thinking too about something like following up Rebecca’s suggestion, and increasing accessibility, you could even have students creating resources like that in a variety of formats. It could be a poster, could be a podcast, could be a video, could be a live presentation… You could do something like an oral exam… something like that.

John K.: One of the things I’m doing in my small class of 60 students is having students create podcasts. Unfortunately, that doesn’t scale as well, in my class that’s closer to 300 students. So, I’d really like to do more open pedagogy projects. It’s just, in large intro classes, that’s a bit of a challenge.

Rebecca: John, you have some experience using algorithmic questions, too, as a way of assessment, right?

John K.: Algorithmic questions can work very effectively, in at least making sure that students can use the formulas appropriately, which is a basic skill in many STEM classes.

Jessamyn: What I would like to see is more faculty really having these discussions and swapping these ideas, like on a national scale. I think that the learning curve has been so high for so many instructors in so many ways. Like, not just, “I’ve never even visited the learning management system, and now I have to use it.” Not just that. But, coming to terms with the emotional aspects of teaching and trauma-informed teaching in the midst of, possibly, “I’m at home and I’m supposed to be overseeing my children’s education” or simple childcare issues. All these things are overwhelming so many instructors just day-to-day life. And then on top of that, “Oh, rethink something you’ve used forever. The thing that you relied on from day one, and that you did so well in graduate school… hey, that’s not gonna work.” That’s hard. That’s tough. So, the more sharing of ideas we have, and the more spreading of good possibilities for assessment, the better. And I sent you a list of some of those resources I’ve been providing. They are starting to be generated, especially at university teaching centers and in people’s blogs and essays and such. But, I think the more it just becomes a broad conversation about “What can we do? How can we, in this situation, assess student learning in new ways and recognizing it’s new for us, too.”

Rebecca: Bill Goffe, in our episode 154, Sharing Disciplinary Pedagogies, also offered a way to get people to collaborate across institutions on some of these kinds of things using a simple Google Sheet. So, we’re all kind of forced to be on line in some capacities now, maybe more than before, but maybe that’s also opening some doors for collaboration that haven’t been there before, either.

Jessamyn: I hope so. I mean, John Locke and I, both of our centers had not been collaborating in the past. So, spring of 2020, was like this kind of completely perfect context for us to send a message to the university, the Center for Teaching Excellence, and Technology Enhanced Learning, we work together, and because people needed us both. So, in that sense, I won’t say silver lining, there’s no such thing right now, but it was a unique opportunity for these two very small centers on campus to collaborate.

John L.: Yeah, in fact, I’ve accidentally come up with a tagline that is starting to appear at the bottom of my emails to faculty. And that is, “you are not alone.” They never were, but it’s much more important for them to realize. In fact, I was working with a professor last night who was having some difficulty in the learning management system. And about 10 o’clock, I sent him what I thought was probably the solution. And I didn’t hear back. So, this morning, I sent him an email and said, you know, “How did it work out?” And his response was, “I’m sorry, I haven’t even gotten to it yet. I’m sorry.” And I said, “No, you don’t have to apologize to me, I just want you to know that you’re not alone, that I’m trying to help you. And I’m not going to let go until I know your problem is solved.” And that sort of community approach to learning in general, and what we’re all going through, I think is helpful. If you know that I know I’m struggling with this I’ll bet someone else is too and, maybe between us, we can figure it out. If more people can adopt that thought and not feel that they’re infringing on someone else’s time, I think we’ll all get through this to whatever the other end looks like.

Jessamyn: That was one of the first things that John Locke has said to faculty who wanted to use this remote proctoring system is “Don’t make your life harder than it has to be.” All the student issues aside, and equity and trust and accessibility, but it’s such a pain in the ass. It really is hard to use. And I’m not just talking to the student end is terrible, but from the instructor end. It’s such a pain to set up and he shared with me, sometimes someone will approach him, “Can I set this up,” he said “Okay, but you have to do bla bla bla bla bla, then this and this…” and they’re like, “uh, maybe I’ll rethink this.” LAUGHTER] I mean, let’s try to make our teaching a little bit more joyful, if we can. Let’s try to make it a little bit more creative, for our sake, if nothing else,

John K.: It can be a lot more fun listening to podcasts students create, listening to their videos that they create, looking at documents they create, or infographics and other things, than it is reading a pile of exams, or writing up multiple choice exams.

Jessamyn: For students, too. Conveying their knowledge in a different way. It’s so good for their brain. That’s why I’m always reassuring students, when I’m asking them to do non- traditional assessments, which I mostly use (even before all this). Our students are very traditional in many ways, and they get really nervous when I say, “Okay, so you’re gonna write a short story, you’re gonna do a poster.” And they say: “Wait, what? I’ve never done that before.” Or “ I don’t know, I don’t know if I can do that successfully.” And I’m constantly telling them, “This is you conveying your learning, your skills, your knowledge in a new way, and it feels challenging, but you could do it and it’s great for your brain. It’s like calisthenics for your brain. You’re presenting what you know, just like you would in a traditional research paper or a traditional exam, but it’s in a different format, and that’s great for your thinking in all ways.”

John K.: We always end with the question, what’s next?

John L.: What’s next? I’m waiting for that chip to be implanted in my head so that I won’t have to show you my assessment, you’ll just be able to download it. [LAUGHTER]

Jessamyn: John, what is your next book project?

John L.: My next book project… I’m writing a novel that’s called “Defending Eldorado” and it takes place in South America, about 50 years after Columbus, where a bunch of colonial powers are trying to find Eldorado and the native South, Central and North Americans are doing their best to make sure they don’t find it. And since we never did, obviously, they were successful. Spoiler alert. [LAUGHTER]

John K.: You mentioned that you had just completed a book. What was your most recent book about?

John L.: Ah, my most recent book was actually the prequel to the current book, a nd that was about a group of disillusioned European scholars who left the Academy. They were humanists, they left the academy because it was being run by scholastics. And they decided to find Thomas More’s Utopia, which leads them to the New World, and hilarity ensues. Not really, but… [LAUGHTER]

Rebecca: How about you, Jessamyn?

Jessamyn: I’m headed, coming up, very shortly, I think everybody here is familiar with it, the SUNY Faculty Developers Conference, it’s going to be online and I’m doing a poster there about a series of events that John Locke and I hosted over the spring for faculty. So, that’s coming up next month. I’ve got some speaking things coming up. I’m really excited to be speaking at the Lilly Online Conference in November, and I am reading chapter submissions for an anthology project that’s contracted with West Virginia University Press in their Teaching and Learning Series. It is an anthology of insights into effective teaching and learning from women, marginalized, and underrepresented faculty. I have some fantastic submissions… so many good ones. So, that’s been a really great thing I’ve been working on right now. It’s fun.

Rebecca: Well, thank you both for joining us…

Jessamyn: Thank you.

Rebecca: …and we look forward to your future work, for sure.

John L.: All right, thank you.

John: It’s great talking to both of you.

Jessamyn: Nice to see you both. Hang in there, SUNY Oswego.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Brittany Jones and Savannah Norton.

[MUSIC]

123. New Trends in Science Instruction

Science instruction in K-12 education has long been provided as if science consisted of a body of facts to be memorized. The Next Generation Science Standards, however, rely on an inquiry-based approach in which students learn about science by engaging in scientific exploration. In this episode, Dr. Kristina Mitchell joins us to discuss this approach and its implications for college instruction.

After six years as a director of online education at Texas Tech University, Kristina now works for a science curriculum publishing company and teaches part time at San Jose State University.

Show Notes

Transcript

John: Science instruction in K-12 education has long been provided as if science consisted of a body of facts to be memorized. The Next Generation Science Standards, however, rely on an inquiry-based approach in which students learn about science by engaging in scientific exploration. In this episode, we discuss this approach and its implications for college instruction.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Kristina Mitchell. After six years as a director of online education at Texas Tech University, Kristina now works for a science curriculum publishing company and teaches part-time at San Jose State University. Welcome back, Kristina.

Kristina: Thank you. It’s good to be back.

Rebecca: Today our teas are…

Kristina: Diet Dr. Pepper.

John: The same as last time. [LAUGHTER] I have a green tea today.

Rebecca: And I have a nice pot of brewed English Breakfast tea.

John: We’ve invited you here today to talk about the Next Generation Science Standards. How did you get involved with the Next Generation Science Standards?

Kristina: During my work at Texas Tech University in online education, I was doing a lot of consulting for publishing companies on their online and curriculum offerings, which led to a full-time job offer from a curriculum company that works specifically in K-12 and some higher ed science curriculum, and I learned very quickly that one of the newest trends in science education at the K-12 level are these Next Generation Science Standards that were created by college-level science professors in order to change the way we teach science to K-12 students.

John: How did these standards come about?

Kristina: I have to admit that because I’m relatively new to the science curriculum world, I am not an expert on their creation. But I do know that there were several panels of scientists from various higher education institutions that got together to try and figure out from the perspectives of different disciplines, “What do we want our students who come to university to know about science once they get there?” When I was in middle school, I don’t know how much you remember your middle school science lessons, but I remember dissecting frogs and doing our Punnett squares about genetics, but what I remember most is that we would learn those principles of science like Newton’s laws throughout the week, and then Friday, we would do a lab to prove that they were true. And unfortunately, that’s not how any of us do science. Being a social scientist myself, that’s not how I did science at the college level, as a researcher. So, the panel of experts from various scientific disciplines sought to change the way students think about science to think about it more as a process of investigation, rather than something you sort of memorize and then confirm.

Rebecca: The Next Generation Science Standards have three dimensions. Can you talk about those a little bit?

Kristina: Sure. So, these sort of three dimensions, or the three pillars of the way students are learning in K-12 about science, the first is they’re learning disciplinary core ideas. So these disciplinary core ideas are the core features of disciplines like physics or chemistry and the things that they really have to know about that specific discipline in order to build on that knowledge. The second one is science and engineering practices, so these are the ways we study science. And the great thing about the ways that we study science is that it doesn’t change across disciplines. Whether you’re a physicist or a chemist or a sociologist, you’re going to use the same types of scientific practices to answer your research questions. And then the final dimension, the third dimension, are cross cutting concepts. So, these are things like observing patterns or observing cause and effect, concepts that also span across all disciplines, not necessarily limited only to science. So it’s concepts that students can use to understand the world around them, regardless of what questions they’re asking.

Rebecca: Can you give us an example of how these three dimensions play out in a particular grade level?

Kristina: Sure. So when we’re thinking about what students are doing in science classrooms these days that might be different from the way we learned science when we were in seventh grade, students are doing a lot more investigation centered science. So, you walked into your classroom, your college classroom, and somebody walks in and says to one of your students, “What are you guys doing?” And your student might say, “Oh, we’re reading this chapter on…” Rebecca, I think you’re an art person, right?

Rebecca: Yeah, design.

Kristina: “So we’re reading this chapter on Photoshop,” and you ask them, “Okay, well, why are you doing that?” And they’d say “Well, because Professor Mushtare told us to.” In trying to get them away from that instead, the way they’re trying to implement science in the classrooms now would be to get students, you know, if they came into your classroom to say, “Well what are you guys working on?” And they might say, “Well, we’re trying to make a flyer for this project.” And you’d say “Why?” And they’d say, “Well, because we’re trying to figure out how we can make people more excited in this project that we’re working on.” So, getting students to start from the point of “What are we trying to figure out and why do we have to do this activity in order to figure that out?” …rather than “What are we memorizing, what are we learning because the teacher told us to?”

John: So it’s more of a project-based learning or inquiry-based approach to the discipline.

Kristina: Absolutely. And we’ve seen that a lot emerging in higher ed. So, it’s really great to see it reflected in the K-12 space too.

John: But it’s important probably for faculty to be aware of this, because if students are starting to use this approach, as they move forward, they might expect to see more of that in higher ed, where they might see it as a step backwards if they have to move to something that involves more rote learning, for example.

Kristina: Exactly. We’re seeing a lot less “sage on the stage”-type activities, both in K-12 and in higher ed. So it’s really important, I think, that we all kind of know what both sides of education are doing so we can at least know where our students are coming from.

Rebecca: How do you see this new approach helping students be better prepared for college?

Kristina: Well I definitely think that the idea of empowering students to be in charge of their own knowledge is really important. Sometimes I find that a struggle that my students have is doing this sort of figuring things out on their own. If I don’t give explicit instructions with explicit rubrics, they sometimes feel very lost. And so empowering them to recognize themselves as people who can ask questions and figure out the answers to those questions, that goes far beyond just physics class. That goes into just being a responsible and productive member of society. So, I think hopefully we can see students taking a little bit more ownership in their learning because of these trends at K-12.

John: Does it also perhaps give students a little more motivation when they have a goal in mind, and they know why they’re doing things rather than just doing it because their instructors told them to?

Kristina: Yeah, I would definitely think so. I definitely remember my seventh grade year not being passionate about the science, other than “Wow, cool. It’s a dead frog.” So thinking about empowering students to ask these questions might get them more excited about what they’re learning about.

John: In one of our earlier podcasts, we talked to Josh Eyler, who emphasized the importance of curiosity and nurturing curiosity and this approach seems to be very nicely tied into that.

Kristina: Absolutely. And I think about my own political science classroom. One of the fun activities I’ve done and I swear the first time I went through it, it was one of those days where I thought, “Ah… I’m tired. I don’t have anything that I’m interested in lecturing about. We’re almost through this semester, what am I going to do today?” And so I decided, I’m just going to have the students write questions on index cards related to political science and hand them all in and I’ll answer all of the ones that I can. And that was one of the best days of class. I’ve recreated that many times in semesters. And so seeing that these Next Generation Science Standards where we’re trying to get students to think there are some things you learn about each discipline, but there are some practices and concepts that apply everywhere and you just learn how to ask the right questions and you use these concepts and practices to answer them. It’s something that I’m using in my political science class, even really, before I realized that I was specifically doing so.

Rebecca: Do you have any idea how students are responding to this curricula?

Kristina: So every individual curriculum and program is going to be a little bit different, but they are all going to tie into these main ideas. And one thing that we’ve seen in student performance and student preferences is that the students really like doing things in class, having activities and hands on specifically to science. Of course, that often means labs. But one thing that I often talk to teachers about and get a lot of really good feedback on is the idea that we kind of think of doing science looks like it’s in lab coats with safety goggles, but we all know as researchers ourselves that science looks like talking sometimes. And sometimes it looks like asking questions, and sometimes it looks like taking notes or revising models, and getting the teachers and students to realize that that’s all part of what doing science is. They get really excited at the idea of doing science themselves. Of course, they like to see things explode in class, but sometimes just getting them the opportunity to have discussions with each other. It’s amazing to me what the middle school students are capable of, and our elementary age students, what they’re capable of. I have kids myself and I didn’t know that they were learning such higher level concepts of science in class.

Rebecca: There’s big movements to move STEM to STEAM. Is the humanities represented in this new model?

Kristina: So, when we think about STEM, that’s kind of what everyone’s prioritizing these days: the science, the technology, engineering and mathematics… and the A to add it to STEAM would be the arts. So the idea that the arts are also important, but I think there are some key pieces missing here. When we focus exclusively on STEM and on creation and innovation, that’s really good. We need creation and innovation. But as I feel like sometimes we’re learning right now, without a good background knowledge of history, we might be creating and innovating things that either already exist, or have already been tried, or that don’t have a good basis in what we’ve experienced as a society. It’s like they said on Jurassic Park, you spend so much time wondering whether you can and you don’t stop to think about whether you should.

Rebecca: So you were just raising that question about ethics and things like this. So,how do you see this curriculum evolving so that the humanities are better represented or integrate more into our integrated learning in K through 12?

Kristina: Well, one thing that I think is really great about the NGSS is that it definitely starts to sort of de-silo the disciplines of science. So, when we think about physics and chemistry and earth science and life science, when I was in school, those were all completely separate years, both in middle school and in high school. And the NGSS standards definitely start to recognize that we can’t silo our disciplines like that because they have so much to speak to each other. Some questions you need many disciplines of science to answer. And so I’m hoping that slowly, more and more curriculum writers will apply that same standard to the humanities, to the social sciences, and to the arts, the idea that nothing can truly be siloed because social science speaks to even questions like “What scientific research do we do? What research questions do we ask?” That can’t be explained using natural sciences alone. Because yes, natural sciences might seem really unbiased, but we’re asking certain questions and we’re not asking others. So using social science and humanities to help answer those bigger questions. I’m hoping that we see less siloing across disciplines over time. And even at universities, we’re starting to see increases in things like cluster hires, where multiple people from different disciplines all come at a sort of similar question, kind of like what we’re doing here. We’re three people from very different disciplines. But we’re all interested in the question of, you know, what makes teaching good or bad.

Rebecca: One of the questions that’s risen a lot in higher ed, and I know as a popular conversation on our campus, is about fake news and debunking pseudoscience. Do you see that this curriculum helps with that,? …perpetuates that?

Kristina: I think that’s a really good question, and kind of a difficult one, because I think it’s really important to teach students how to interpret the world around them, and what science is and how it’s done, because that can help us determine the validity of the information that we get. But I also think, like we were just talking about, there is a move away from rote learning, from the idea that an expert in a field tells us what is true about that field and we learn it. The fact that we’re moving toward anyone can be an expert, anyone can do the research themselves, anyone can ask questions. Sometimes, I also worry that that pendulum might swing too far, because that’s sometimes where we get people who go online and find whatever Google can tell them about vaccines not being safe or about climate change not being real. And because they feel empowered to be their own expert, they’re treating themselves and their research as equivalent to people who have devoted their lives to getting advanced degrees and who know the answers to these questions. And so I worry that maybe we’ll see the pendulum swing a little too far in the other direction, but I’m not sure.

Rebecca: Based on knowing what you know about higher ed and what you know about this science curriculum, what do you think faculty should be thinking about in terms of what their students might know or might not know, or how they might know differently than they did before?

Kristina: I think that it’s important to teach students what expertise is to begin with, when we think about what it means to be an expert and know something. So, when we go back to those next generation science standards, and think about those three dimensions, the disciplinary core ideas, to me, those are the things that experts already know and that we can accept as the core ideas of each discipline. So, starting from that point and saying the question of vaccine safety has already been established, we know that the vaccines that are here are safe. That’s a disciplinary core idea. But now let’s use our science and engineering practices and our cross-cutting concepts to think about other types of preventative medicine that maybe don’t have the same consensus or new types of vaccines? How would we know that those are safe or how would we know that those are effective? So, I think it’s important not to forget about those disciplinary core ideas that are things that experts have already figured out questions that have already been settled, and then allowing people to start from that point and ask their own questions and do their own research using the scientific method.

John: Changing the topic just a little bit, we get a fair number of people who are interested in the STEM fields when they’re in middle school and even high school, but then they get to college, and all of a sudden, they have to start taking calculus classes, and Matrix Algebra, and Organic Chemistry. Is there any way that perhaps lessons learned from this approach could be used to make college classes more effective and not having so many people get lost along the way once they hit these courses that have been traditionally a bit of a barrier to progress in those areas?

Kristina: That’s a really great question, and we even see that in political science. I teach our research methods class in political science and it has just the tiniest bit of statistics in it. And students are like, “Why am I doing math in a political science class?” And then of course, in graduate school, we’re seeing increasing amounts of quantitative knowledge needed to finish graduate programs and then to compete on the job market. So, I definitely think that the rise of math as something important is a deterrent to some students who might otherwise be interested in entering STEM or social science fields, but I think that also comes down to the pendulum again. I think that it’s important to realize that quantitative methodology is a method of understanding scientific questions, and you don’t have to use quantitative methodology to answer every scientific question. Some of them can be answered using qualitative methods, and even discussing political theory or other types of theories in other sciences or in the humanities. I think it would be good if we could open up the doors a little bit to people who have really high level intelligence and interest and curiosity, but maybe don’t have an interest in math. I think both by making math more accessible and less required, we might be able to get some really interesting new perspectives in our college level courses. I don’t like the idea of using hard math classes as a way to weed students out. And I think that’s sometimes what they’re used for, because I took Matrix Algebra in grad school, and it was really hard. And I’ve never used it since that class… ever again. So it’s not as though it was a skill I needed to be a PhD in political science. It felt more like a class that was used to weed out students who weren’t as interested or weren’t as good at memorizing how to do Matrix Algebra.

John: The point I was trying to get to, I think, is that perhaps if more of these inquiry-based methods were used to motivate the material, it could help get past that. I actually had a similar experience with Matrix Algebra. And when I was learning, I was able to do the proofs and work through it, but I didn’t really understand it until I started using it in econometrics and other disciplines, when all of a sudden it just made so much sense and why all those theorems were there and useful. But we lost a lot of people along the way there. And perhaps that’s something we could build into all of our classes to help motivate students and not just hit them with a lot of theorems and not necessarily even just rote learning, but a lot of inquiry for which the motivation may not be obvious to people when they take it.

Kristina: Definitely. And I think that when I teach my Statistics and Research Methods class, it always is easier for the students, and they always do better, when I tie the math we’re doing to a real question. So even something as simple as “We’re trying to figure out whether the average American is conservative or liberal,” and this is our question that we’re trying to figure out. Now we need to do a difference of means test to figure it out. And so that would be, going back to what we were saying earlier, the idea of you walking into my class and asking my students what they’re doing. Well, they could answer “We’re memorizing how to do a difference of means test from chapter 12 because Dr. Mitchell told us to,” versus saying, “Well, we’re trying to figure out what the ideology is of Americans from different regions, so we have to use this mathematical test so we can answer that question.” So really about framing it in terms of “What are you doing and why are you doing it?” and making sure that we know that the students would answer in a way that gets them excited about the material.

Rebecca: I think context is everything. The real world problems that you’re talking about in the new science curriculum is not unlike the kinds of things that you’re talking about in your classes or even when you have to use math in design class. So, a lot of visual arts students think math is completely irrelevant to them, but when it’s put in context, when you need to do certain things, all of the sudden it makes more sense. I teach a creative code class and we do fairly complicated geometries there, actually. And if you can see the results, sometimes it makes a lot more sense to some of our students.

Kristina: Exactly. So the fact that they would know why they’re doing this and it’s to accomplish a specific goal, rather than just “We’re memorizing this because the professor told us we had to, it was going to be on the exam.”

Rebecca: Yeah. [LAUGHTER]

Kristina: Lord, help us for how many times we hear “Is this going to be on the exam?” That seems like the motivation for trying to learn something. And it would be great if we could move it away from that being the motivation, toward the motivation being “Well, we’re trying to learn this because it helps us figure out this question or this design problem.”

Rebecca: Or the motivation, like, “Hey, I really need to understand this mathematical principle, because I really want to know the answer to this question.”

Kristina: Exactly.

Rebecca: And that leads to where they actually start asking for the things that we’ve prescribed previously. [LAUGHTER]

Kristina: Yeah, and it becomes just a tool to figure things out. And if they want to figure the answer out, then they’re going to need all the tools.

John: Could you tell us what this would look like if we were to stop by a seventh grade class, and this was being put in practice?

Kristina: Absolutely. So if we think about what a seventh grader might be doing, so if we start from that point of the goal is to figure something out, and maybe they’re learning about Space science. They’re learning about Earth and Space and trying to figure out “Why do some objects in Space orbit other objects?” …like “What does that mean? How does that happen?” So, one really fun activity that I’ve seen middle schoolers doing is they have these embroidery hoops and these marbles. And so what they do is they put the marble in the embroidery hoop, and they sort of spin it around on the table. And so the marble is swirling in circles and then they lift the hoop and marbles start shooting everywhere. So it’s very funny in a seventh grade classroom, because you got kids running everywhere trying to chase their marbles. And so at first, they’re just having fun, tracking which way the marble goes and trying to figure out when they lift the embroidery hoop which direction will the marble head, and then eventually they start realizing, “Oh, well, the marble kind of wants to go straight, but the embroidery hoop is keeping it going in a circle.” So, if we think about the sun, with the planets orbiting it, if the sun suddenly disappeared, Earth would kind of want to go straight out in whatever direction it was headed. But because the sun’s gravity’s in the middle, it’s keeping Earth going in a circle around the sun. And so they start putting these pieces together and eventually they come up with this idea of the relationship between inertia, which keeps the earth going, and gravity, which is what keeps it in that same orbit. When they think about what is the core idea they’re learning about, well, they’re learning about orbit, but the way they’re figuring it out is by looking at this small model of what is actually happening at the planetary level. And that’s really reflective of the kinds of things that scientists do in their labs. We’re running models that try to simulate what’s happening in space, or what’s happening in the political world, or whatever discipline we’re doing. We’re trying to create these models to help us understand it. And so that’s what the students are doing, too.

John: That reminds me of a talk that Eric Mazur has given in several places, including at Oswego, where he talked about the Force Concept Inventory, and how, while students were able to plug numbers into formulas and solve problems very well, when they were asked about questions like that, specifically, if I recall, “Suppose that you’re swinging a rock on a sling, and you release it, what would the motion look like?” And basically it ends up flying in essentially a straight line… well, other than the effect of gravity. But when people were asked that, they would bring up things that they had seen in cartoons where they expected it to loop through the air, perhaps as it’s moving. That’s actually one of those things where people in really good institutions who had taken a college class had some really poor intuition. And this type of practice can help prepare them with better intuition on which to build later.

Kristina: There was also a video at, I think it was Harvard’s graduation, where they asked a bunch of STEM majors who had just graduated if they thought they could make a little circuit that would light up a light bulb. And so they handed them everything they would need to do, and everyone’s like, “Oh, yeah, yeah, I got this no problem.” And they couldn’t, they couldn’t figure out how to light the light bulb up, because as much as they had done the math in class, they just had never really done the practical aspect. And so getting students hands-on to do the things with those science and engineering practices in whatever discipline we’re in. It’s really important to get our students in the driver’s seat to start practicing what they’re going to be doing when they leave the classroom.

Rebecca: It also seems like having multiple pathways to the same information is a good way to play with our memory, and make sure that it sticks and it stays there. So, something that’s embodied, something that you memorize, something that’s a couple different kinds of examples.

Kristina: Exactly. Maybe it can move it from our short-term back to the long-term memory to help us keep that knowledge.

Rebecca: So we always wrap up by asking “What’s next?”

Kristina: So I think I’d really like to start exploring what this does look like in a classroom setting for myself. So rather than just focusing on what it looks like for students in K-12, I’d like to see if I can use a similar style of teaching in my political science classroom, and just see what kinds of evaluations I get and how the students like it. I think the idea of getting rid of these silos and recognizing that we all use the same practices and concepts could help us with keeping our college experience for our students interesting and consistent. And we start realizing that they’re going to learn in each of our disciplinary core ideas, but there are a lot of things that we all have in common. So, I’m really going to try this semester, maybe my student evaluations will suffer greatly, but we all know from my past talks that they’re biased anyway. But I just kind of want to see what it looks like if I let my students take a little bit more of a driver’s seat in asking the question, “What are we going to figure out? What do we want to know about the world?”

Rebecca: Do you have a piece of this semester that you’re most excited about that you already have planned in this domain?

Kristina: Well, I’m doing the research methods class this semester, which of course, involves some math. And I think that I would like to use their ideas about what kind of data we should be looking at and what kind of questions we should be answering. Because I think that’ll make it a lot more meaningful for them, rather than if I say, “We’re going to look at the American national election study,” letting them figure out “What question do you want to answer?” And then I’ll teach them whatever method it is that they need to know in order to answer that question. So, we’re going to try it. We’ll see how it goes.

John: In a past podcast, Doug Mckee talked about a similar situation in his econometrics class where he was using a time-for-telling approach. Where basically you face students with problems and let them wrestle with it a bit before giving them some assistance and helping them resolve things, and there’s a lot of evidence that those techniques can be really effective.

Rebecca: So, it sounds like well wishes on your new adventures. [LAUGHTER]

Kristina: Thank you.

Rebecca: Thank you so much for joining us.

Kristina: Of course, this is great. It’s always great. I think eventually I’ll have to come visit y’all.

John: That would be great. Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Brittany Jones and Savannah Norton.

[MUSIC]

114. Dead But Not Buried

Show Notes

Transcript

John: Travel courses can provide an opportunity to experience a different part of the world through the lens of a particular discipline. In this episode, we discuss the rich interdisciplinary learning opportunities that may occur when faculty and students from two different courses and disciplines travel together.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Today our guest is Dr. Kathleen Blake, a bioarchaeologist, a forensic anthropologist, and an assistant professor in anthropology at the State University of New York at Oswego. Welcome Kat.

Kat: Thank you

John: Our teas today are:

Kat: I’ve got a ginger tea and a mug from the Czech Republic.

Rebecca: The perfect match for today. I’m drinking my English afternoon tea.

John: …and I’m drinking ginger peach black tea. We invited you here today to talk about your collaboration with Rebecca on an international travel experience for students. This collaboration started with your course “Dead but not Buried” with travel to Prague and Brno in 2017. Can you talk a little bit about this course?

Kat: Sure. This course came about because students were interested in bone churches, which are churches that are found all throughout Europe, but there’s one particular beautiful one in the Czech Republic. And I thought “How can I make this into a course, basically?” And so I was looking at to make it into an anthropological framework and the idea was to see how cultures interact with their dead, how they treat their dead and the different cultural aspects of living with the dead, in particular, if the dead are present and active in your culture, as they are in this case. And the class also examines the framework of the dead over time and burials over time, so we went as far back as Neanderthals up through modern times. So, it covered quite a wide range of different time periods.

John: How did this collaboration between the two of you get started?

Kat: I took the class the first time in 2017. And I went on my own. Rebecca and I had initially talked about doing it together. However, we weren’t able to.

Rebecca: Someone decided to have a baby. [LAUGHTER]

Kat: And so we decided that the second time around we would see if we can make this work, and in particular, I was looking for somebody outside of my area. I didn’t want someone in the social sciences, I wanted somebody outside of anthropology and outside the social sciences, and the more we got to talking, the more it seemed like a good fit.

Rebecca: So first, you started by showing me pictures of things that looked really cool, some interesting design things. So she clearly marketed at me, right? [LAUGHTER]

John: Including some dead things, right?

Rebecca: …including some dead things… that I found unusually interesting and got me enticed to figure out what this place was really about and started discovering and doing some research and finding out that Brno actually is quite a design hub… that I wasn’t aware of.

Kat: Through our discussions, I learned a lot more about art than I ever knew. And finding out that the Czech Republic in general is a really big design hub, right?

Rebecca: um-hmm.

John: Could you tell us a little bit about how the collaboration worked in terms of the two different fields?

Kat: We used the existing structure through International Ed (both the Q3 and Q4 structure), but Rebecca was in the Q3 structure.

Rebecca: So, our Q3 and Q4 classes are a half a semester class that then would have travel either during spring break or in May at our institution. So, I taught a Q3 class but we didn’t travel in spring break. We traveled in May all together as a big group.

Kat: And I taught the Q4, which was from spring break until the end of the semester. And then we traveled together. And we tried to encourage the students to sign up for both classes. So, there was some overlap of students, although not all the students signed up for both classes.

Rebecca: And we did that so that students could have up to six credits for their experience, and it would provide interdisciplinary perspective on a place. And so my class focused on design specifically, and designers in the Czech Republic and art forms in the Czech Republic and the histories of that. And then, obviously, I didn’t deal with the dead at all, actually in my class. [LAUGHTER].

Kat: And then I taught my course as normal. And we kind of found there was a little bit of overlap that we didn’t realize there would be through the course.

Rebecca: We did have some assignments that were in common between the two classes. So, that it would help facilitate students making the connections between the course material because all the students ultimately were exposed to at least some of the course material for both courses, even if they weren’t enrolled in both courses because we all traveled as a group. So we were gone for 12 days, and through that time, we visited a number of different sites, some that were more anthropology focused, some that were more design and art focused. But all students went to all of those places. And then there was a couple of things that they could choose to do on their own or explore on their own. And so that worked out, I think, pretty well. And we had some really surprising experiences. For example, we were in one bone church and Kat had set up the history and understanding of the place but then I had a whole bunch of students come up to me and asked me about analyzing the design aspects of the spaces that we were in, which was really interesting. And I found it bizarre because I was learning about the space… It was the first time I had been in it and certainly didn’t feel like an expert… but students were asking me questions about it. But, then I also heard them talking to each other. So, the design students were asking the anthropology students about what they were seeing, like what the bones were, what parts of the body they were from, I was asking the students the same kinds of questions and then they were asking the science students and anthropology students were asking the design students about visual aspects that we were experiencing

John: For those in our audience who are not familiar with bone churches, could you provide a description of what these are?

Kat: The one main bone church that’s probably the most famous is in Kutná Hora in the Czech Republic. And this church was a church that had been consecrated and so it attracted a lot of burials, especially during the black plague. Later on, the graveyard became full and so they dug up all the bones and normally they would put them maybe underneath the church, but in this case in the 1800s, they decided to elaborately decorate the church with the bones creating chandeliers, swag, pyramids of bones, a coat of arms, all kinds of sculptures, so it was a way to elaborately decorate their actual church and they held church services in that location until about eight years ago.

John: Are there are some photos or photo albums that you guys have created that perhaps we could share with our listeners in the show notes.

Kat: Oh, definitely and the students came up with some really good ones, too.

Rebecca: We have a blog the students were contributing to. So, there was some research that happened in my class before we went. The students put together a research guide of contemporary sculptural work that we were going to go to and some building some architecture. But then they also did some reflections about some of the spaces that they had visited once we were there that were more related to Kat’s class.

Kat: And the design students were making comments about how beautiful the bones were… the natural structures, how symmetrical they were… just thinking of the body as more piece of art than it is a piece of anatomy in some way. So it was interesting to hear their comments.

John: You mentioned that students could get up to six hours of credit. Were the two quarter classes three credits or two credits? How did that work?

Kat: Each class was three credits. So a normal quarter class in our institution is three credits,

John: but then did they get additional credits for the travel component?

Kat: that part of the course so we would meet once a week for an hour and then the travel at the end is included in the course.

John: Okay, so that was required it wasn’t an optional component.

Rebecca: Correct. And then the benefit there for students is that it was two different courses, but the travel piece was identical. So, the actual physical going somewhere and the time required for that was only one time. But, there was different coursework for each course. Although we had one reflection journal that was the same across both courses.

John: How did students react to the experience,

Kat: I thought it was positive overall. We got some of the students who had never even heard of anthropology, all of a sudden interested in anthropology. And then we had some of the anthropology students who were interested in things like photography and art. And so were immensely interested in asking Rebecca a lot of questions about both.

Rebecca: I think we have a new photography minor as a result.

Kat: I think so. I think we also have a new art minor.

Rebecca: I think we might, yeah. [LAUGHTER] I think we found it really interesting. And you might think that the art students would band together or the anthropology students would band together, but that didn’t necessarily happen. Partly we were strategic about how we put room assignments and things together. But we had three first-year students that traveled with us that were from different majors. And they bonded really nicely together and were doing all kinds of things together.

Kat: And I think they’re still good friends to this day.

John: Now, we had an earlier podcast in which we talked to two people who had done several trips, and they talked about some of the logistical issues that came up. How did the logistics work with your trip?

Kat:I would say overall, the logistics worked really well. I think the most harried part was the air transport getting there. But once we hit the ground, we had things pretty well organized. We’re very well organized people. I think that’s part of it. We had everything organized down to a tee. And we built in a lot of free time, and that made a big difference.

John: And to be fair, the other people were pretty organized, but there were some airport delays and there were some students who disappeared for a while unexpectedly.

Kat: Thank goodness we didn’t have that.

Rebecca: Yeah. [LAUGHTER] In the past, I have had courses with travel that have had things that went awry. I had a flight that was detoured and had to let fuel out. And there were students sitting next to the window and could see the fuel spewing out. And that was kind of concerning for students who had never flown before. So, certainly I had some harried experiences as well. But, I think overall, not so much. The biggest struggle that we’ve had is one that I also had when I took students to India was actually heat. And then students, although we were trying to prepare them and reminding them about hydrating and things just had a really hard time dealing with warmer weather and walking around a lot.

Kat: And in much of Europe, in the Czech Republic, in particular, things are not air conditioned. So, I think that was an adjustment for many students. Part of the reason that I wanted another faculty along was because I think it’s good to have a second person there for support. In case something goes wrong, you’ve got that second person. I think it really makes a difference. When you’re planning a trip like that.

Rebecca: We had a couple students who weren’t feeling well. We had some sinus congestion stuff that seemed to be spreading amongst the group. There were students that weren’t feeling well but you know, we’re checking in… may have picked one of us to check in with It may not even be the faculty member that they knew the best initially.

Kat: I think that was the case, yes.

Rebecca: So, we were able to tag team that a little bit and help support those students when they weren’t feeling well.

Kat: And that’s the thing. The first time I traveled, I did have a student who was ill and I was busy at the hotel tending to her and I couldn’t be with the other students. So, I think having that backup is important

John: Kat, you had traveled before with this class. But, this time you did it with another faculty member. How was it different this time than in your past experience?

Rebecca: He really wants to know how big of a pain in the butt I am to collaborate with.

John: Oh no, that I know. [LAUGHTER] I know all of that. But, in general, how was the joint experience different from individual experiences?

Kat: Well, Rebecca’s probably the best travel buddy I could have picked, so that worked out really, really well. Not only just having a backup in case students need help but having someone that you can have a break with… the students can go off and do their thing and you can have, shall I say grown-up time on your own with another faculty member, I think, makes a huge difference. Before I was having dinner with the students… I was with them 24-7, and it just sometimes gets to be a little bit too much.

Rebecca: We also found that we were able to do some exploring and find some new potential options for students for future trips. So, we were quite busy. I think the students found it surprising how much walking we did after we walked with them all over the place. We did double the walking on a pretty regular basis.

Kat: Yeah, after dinner, we would just go explore. And we also use that to create challenges for the students. So we were using WhatsApp and we would send pictures of where we were, there was a certain art sculpture or something they were supposed to be finding. And we’d say, Hey, we found it. Have you guys found it yet and give them some challenges each day. So it was a good way for us to get out and about together but also to help students explore.

Rebecca: Yeah, to find different places, but also just to see the environments that they actually were in quite a bit because they were in some of those areas frequently, but not really paying attention to their surroundings as much as maybe we wanted them to. It became a little competitive. And I think there was one time that the students made it out to one of the sculptures that we had trouble finding.

Kat: Yes.

Rebecca: they found it before us…

Kat: Yes.

Rebecca: and then it…

Kat: …the Trabi at the German Embassy.

John: So, you had them do a little scavenger hunts after the regularly scheduled program?

Kat: Yes. And they weren’t hard or rigorous in any way. But we would give them challenges almost every day.

Rebecca: Although we started off with a gamified experience the first day. There are the signs above a lot of the buildings that were before there were numbers or a street sign with numbers, there were little pictures.

Kat: Most people weren’t literate. So…

Rebecca: Yeah. So, it’d be like the house of the Golden Bear or whatever. And so a lot of those exist still in the city of Prague. But there’s one street where there’s a lot of them. So we took them to that street the first day and told them how many that were on the street and gave them the challenge to find them and photograph and document them. But then that became a challenge to find more throughout the city during the week. So, Kat and I would find some like, “Oh, did you find this one yet?”

Kat: Yeah, we were often posting all different ones that they’d never seen, and then they wanted to know and we say, “Ah, but you have to go find it yourself.”

John: How did students do in a different language because I assume most of them were probably not fluent before the trip.

Kat: We tried, we had gotten Duolingo and required them to download it and to try to at least say a few phrases. But Czech is a very difficult language. Surprisingly, some of them picked up a few keywords. And I think they were all commenting that if they made even an attempt that the people were very kind to them, and then would switch to English pretty readily.

Rebecca: Yeah. And we had a particular level that we required students to meet in Duolingo for each of our classes. If you took only one class, it was at a certain level. And if you had taken both classes, it was at a slightly higher level. I’m not sure if we would have picked that same exact app next time, because some of the things that it prioritized in the early levels seemed really not that useful. So, I think we would need to explore a different option. I mean, it certainly had like “Hello,” “thank you,” “bathroom,” all those basic things. Yeah, there’s another Czech app through… I think it’s through the Czech embassy, that we also encouraged them to look up and that seemed to have more user friendly type phrases that would be helpful.

John: We have lots of classes that travel overseas, but not many of them involve this type of collaboration between two disciplines. What are the strengths of this model compared to a single disciplinary approach?

Kat: I would think one of the things is you’re drawing students that you would not normally draw… ones that had never even heard of anthropology were drawn to an anthropology class. I think it’s an opportunity to get students in a different discipline or even a different school that would not have been interested before. \

Rebecca: The classes that we offered were also at a higher level, they were like 300 level. So, at that level, I think students tend to navigate towards classes that there may be more familiar with the faculty. So, as the faculty teaching in the art class, I was then able to get those art students to register for Kat’s class and vice versa. We were able to encourage them to try something different. And we sat in on each class so I attended all of Kat’s classes and she attended all of mine and that also helped the students get familiar with both of us so that they would all be comfortable traveling with both of us. I think it enticed students because of the additional credits that they could get out of the experience. But, I think the real valuable part was traveling to all of those different locations with the mix of students and the mix of faculty that we had… because we had science, anthropology, and art…well, really design… design students.

Kat: …and the science students were geology, zoology, and biology. So, they were not all anthropology students. But, I think those anthropology students, and the bio and zoology all said that they learned so much more about art and design that they just weren’t even aware of before. So, I don’t think that would have happened otherwise. We’ve traveled to the same places, but Rebecca talked about the street with the signs. Well, I’ve taken students down that street before but we never noticed the signs. We never noticed the whole kind of street full of signs. So it brought a whole different level that was not there the first time.

Rebecca: And we ended up going to places that weren’t on your original itinerary either. Students expressed an interest in going to the anthropology museum when we were in Brno. And it was such a great connection between art and anthropology, we had no idea. But, in the first floor was an entire art exhibit. That was the whole focus. And then they had full to scale models of cave paintings up in an upper level that was really fascinating for art students who had seen pictures of these things like teeny tiny in their textbooks and things but kind of got to experience something, although it wasn’t like at the site, you could see him at scale and how it went on the contours of the wall, etc.

Kat: Yeah, they’ve made a 3D model of it. And even anthropology students who’ve seen this in pictures as well, to get a sense of the 3D nature of the cave drawings. And you could see how maybe a bison was created on a bump coming out of the wall. I think for all the students that brought a different dimension to it, whether they’re art or anthropology.

Rebecca: …and I think the students pushed us to do some different things too… like we decided, “Okay, we’re gonna figure out this public transportation in Brno” when we had originally thought about only walking and then gave everyone a challenge to take the train somewhere after we all took the train together somewhere.

Kat: We just gave them a day pass, and they used it. One whole group got lost, but they found their way back. And it was quite the adventure from what we heard. But, I think they all had a great time.

John: In upper-level classes, students are often in very narrow silos. And this got them out of those silos.

Kat: Yeah, definitely. And then we did something that I never in a million years would have done, and that was to go to a puppet show.

Rebecca: Which I thought was a terrible idea when we first sat down.

Kat: It was a terrible idea for the first five minutes of the puppet show. And then it turned into the most hilarious thing that we thought the students would hate and they absolutely loved.

Rebecca: Puppetry is an important art form in the Czech Republic. So, I thought it was important to expose students to this. So, I found a puppet show to go to but then we got there and it was this teeny tiny little theater, really hard chairs. And we sat down we’re like, “We’re only going to make them stay until like intermission right?”

Kat: Like, “Okay, they can leave as soon as we get to that intermission“ and I was surprised they all wanted to stay. They enjoyed it. They thought it was different, but interesting.

Rebecca: Well, because It was in Czech puppetry style, which has some interesting humor in it. But it was Don Giovanni… so in Italian.

Kat: So we couldn’t understand anything.

Rebecca: But, you could, actually, you totally could follow the storyline.

Kat: Sure. I don’t think we had any Italian speaking students to help us out, though.

Rebecca: No.

Kat: And this form of puppetry, you see the puppeteers, which made it even more interesting.

Rebecca: Yeah, it was like a behind-the-scenes tour while you were seeing the show. So I think the students found that really interesting.

John: Are you thinking about doing similar trips in the future?

Kat: We are thinking next time maybe of going to places like Spain or Italy or Portugal, which all have different types of bone churches, so ossuaries or different places like that. Maybe we can take this and modify it for a different location.

Rebecca: And I know that you brought some of the reflections that students had at the end to demonstrate the impact that it had on students. Do you want to share what some of those were?

Kat: Well, one of the students had a really interesting reflection that covered both of our topics. This was an anthropology student. But, one of the things he said was that it was how objects were displayed that was important. And I thought you could talk about bones that way. But you could also talk about the artworks. And that that is how you interpreted them and interacted with them. And I guess the first time I went, I didn’t think about interacting as something we were doing in the bone churches or ossuaries, but we were interacting with that space.

Rebecca: We ended up having a lot of conversations about one of them had music playing, and how that impacted the experience of the space versus if it had been quiet. So we ended up talking about all of those as designed experiences, and not just the visuals or the display of the bone.

Kat: Right, interpreting it on multiple levels. And so that got across to the students because they all seem to comment on that. Other students said that they had no idea what Art Nouveau or Cubism was, or even Soviet architecture and now that they did, and that was from a Zoology and Anthropology major, but they learned quite a bit about that. The Soviet architecture was interestingly blended in with everything else. And I think that’s one of the things I love about Prague and the Czech Republic is that mix of old and new and the students commented on that quite a bit. Another student talked about death and grief in clarifying life. She had just recently lost her grandmother and said that no one in her family would talk about the grieving process. And so she found this class very helpful to help her clarify her thoughts about that whole event in her life.

Rebecca: That was a student that took book classes, right?

Kat: Yeah.

Rebecca: One of the challenges that we gave students was to go into a grocery store and purchase something and then to write about that experience. And we found that they were hesitant to do that on their own. We ended up having to physically take them to the grocery store and go in with them.

Kat: Yes, but where they did open up was the farmers’ market. And every day in Brno, there’s a farmers market in the square and they loved that and were able to navigate that easily. So I don’t know what the difference is. If it’s just less threatening. Those people were less likely to speak English… that makes it interesting, but they seemed to enjoy that more.

Rebecca: We had a number of students that were not from a city and so grocery stores in cities are much smaller. And I think that the students were weirded out by that.

Kat: One of the things that was important through this whole class was that we weren’t just in major cities, so Brno’s the second largest city, Prague’s the largest city in the Czech Republic, but we also went to very small cities. So, students got to see all types of communities within the Czech Republic and I think that was important as well.

Rebecca: And Brno is a second largest city, but not a tourist destination by any means.

Kat: Not a tourist destination, no. And Brno, one of the comments that the students were making was the fact that they could see the homeless people… they were out and about near the train station, things like that. In Prague, they didn’t see that. They were probably there, but they just didn’t see it.

Rebecca: One of the things that we also did in Brno and that we didn’t do in Prague, in part because it’s a smaller place, is that we took students to a university there. It happened to be a design school or art school, but we took students inside and met with a faculty member and got a tour of the spaces and saw some student work. And all the students across all the disciplines found that to be really interesting and useful. So, we noted that next time we take students abroad, we need to make sure that we have some sort of opportunity for them to at least see a university space. We had trouble scheduling interactions between students, our students and their students, because of exam schedules and things. They were in exam time and most of the students weren’t around.

Kat: Yeah, we were not able to schedule with the forensic anthropologist or any of the anthropology faculty for that reason. So that’s the only downside of going this time of year is that that’s typically exam time for European schools.

John: Could you tell us a little bit about your preparation for the trip? I think you went there in advance to investigate places.

Kat: Well, Rebecca had never been there. I have been to Prague several times, I had traveled personally. And before I put the class together, I did take advantage of some funding that was available through our international education program to go and scout out locations. And I had looked at other places I’d looked at to Austria, Austria also has some bone churches. But it just seemed like the Czech Republic was a good fit and to stay in a stronger, more confined area, then to broaden it out, and maybe weaken the impact of it. And I had been to Prague, but I had never been to Brno until I scouted it out, I had not been to all of the small towns that I ended up using the first time around. And so for three of the small towns, it was the first time I’d ever been there when I arrived with students. It worked out okay. I don’t know if I would recommend it, but it did work out okay.

Rebecca: I think in general, I wouldn’t normally do an international class unless I had been there before. But because Kat had gone in this case, it worked out fine. I ended up having to do a lot of extra research on art and design. And I connected with a faculty member in Brno actually about art and design just to make sure we were kind of taking students to the right kinds of places and things because I couldn’t quite get a full sense, especially because that particular city is not tourist oriented. And so some stuff wasn’t even available in English. So, I had a little trouble navigating some of that, but it was a lot easier in Prague.

Kat: And what do you think having gone there the first time, what were your impressions that you wouldn’t have had if you hadn’t gone as a group?

Rebecca: I think it’s always different when you actually visit a place when you have that kind of three dimensional experience of what that feels like, smells like, tastes like. I was very excited that we found some Czech food that was gluten free that I can eat. I really wanted to be able to eat traditional Czech food, but I’m a celiac, so I can’t eat traditional Czech food. So, we were able to find a couple places that took a little extra hike on our part to get to those places.

Kat: But interestingly enough, the Czech Republic, and Brno in particular, are vegan hubs. Who knew? So, we had a couple of vegans with us, we had no problems at all because there were a lot of choices.

Rebecca: We also did all of our logistics, Kat took care of hotel and some transport in country and I think it’s helpful sometimes when it’s the faculty member that does that if you have an orientation as to where things are and how close they are so you can schedule things appropriately.

John: What have you learned? Or what would you like to try that you weren’t able to do this time?

Kat: I know we want to have a hotel with air conditioning next time in Prague. That’s going to be high on the list.

Rebecca: Yeah, I think the students have just such a difficult time dealing with the heat that they need a place where they get a little reprieve, so that they can be more energetic throughout the day. If you can’t get good sleep because you’re too hot at night, then the day becomes challenging. I think we’ll incorporate more challenges from the start. We know what to do challenges on. We tried some out, we know which ones fall flat. [LAUGHTER]

Kat: But, most of them were well received. And so I think we definitely will include those from the first day.

Rebecca: We talked about a different language app… that also seems very important, man. We didn’t have the anthropology museum on the schedule initially, but we will definitely go back there. That was a really good experience.

Kat: We were pretty good about scheduling free time. But I think there were a couple of days that we made notes that we would want to expand the free time that the students had or switch things around the order of when we went to different places. And we did find out that most of the places we wanted to go to in Brno were closed on Monday. That became an issue because we were there on a Monday. [LAUGHTER]

Rebecca: We certainly talked about the possible benefits of combining the class and making it just one class that’s the full semester with the travel. But I think the flexibility of having him be two separate classes is really worthwhile and that we’re still able to give all of the students at least some interdisciplinary experience even if they only end up in one of the courses.

Kat: And we’re going to make it more explicit on International Ed’s website next time that the two classes are connected. That seemed to be absent.

Rebecca: So we also talked about we had to use so many stairs…

Kat: Yeah, and many were outside or very narrow. And so claustrophobia and fear of heights need to be mentioned in our warnings. I had disclosed about stairs, but I didn’t consider the claustrophobia or fear of heights. But, it was something it that was not on my radar. The other thing we were talking about doing differently was instead of using a tour guide at the castle to give them a scavenger hunt at the castle, and then just plan to meet back.

John: We always end our podcast with the question, what are you going to do next?

Kat: I think the next is to plan our next trip to the Czech Republic, but also consider other locations, potentially Spain or Portugal or Italy…

Rebecca: …and get those right on the schedule, right?

Kat: That’s right.

Rebecca: I’m on sabbatical studying accessibility, which was also something that we looked at a little bit while we were in the Czech Republic, because there were a lot of places that were very not accessible. [LAUGHTER]

John: Well, thank you. This was interesting. And it sounds like you’ve got some future interesting trips, just to plan.

Kat: Thank you very much for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

94. Open Reflection

Students can provide useful feedback on instructional practices and class design when they are asked, In this episode, three students from John’s spring economics capstone class join us to provide their reflections on the class’s experiment in developing an open pedagogy project. Our guests in today’s episode are Maria Aldrich, Victoria Heist and Charlie Tararzona.

Show Notes

Transcript

Rebecca: Students can provide useful feedback on instructional practices and class design when they are asked. In this episode, students join us to provide an open reflection on one class’s experiment in developing an open pedagogy project.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guests today are Maria Aldrich, Victoria Heist and Charlie Tarazona, three students who participated in the creation of an open pedagogy project in one of my economics classes this spring semester. Welcome.

Victoria: Thanks for having us.

Maria: Thank you.

Charlie: Yep, excited to be here.

Rebecca: Today’s teas are…

John: None of you are drinking tea, are you?

Maria: No.

Victoria: No tea.

Charlie: No tea today.

Rebecca: How regretful. [LAUGHTER]

John: It happens with many of our guests. I’m having ginger peach black tea.

Rebecca: And I’m drinking Lady Grey. The issue is our tea selection is no longer close to our recording studio. It’s a problem. It’s an epidemic now with our tea choices.

John: …now that we’re recording in this little closet in a building next door, where at least we don’t have toilets flushing every 30 seconds or so that we have to edit out.

Rebecca: So John, can you start first by explaining what open pedagogy is, to kind of frame our discussion?

John: Going back a step further. Last year, I saw a presentation by Robin DeRosa who presented on this at the SUNY Conference on Instruction and Technology and she made a really compelling case for some of the advantages that open pedagogy projects have. And open pedagogy just involves having students create content that is open and shared publicly with the world.

Rebecca: So what class did you do an open pedagogy project in?

John: This class is a capstone course in the economics department here. It’s taken mostly by seniors and a few juniors. And it’s a seminar course in economic theory and policy. It’s one of our smaller classes. And we had only 27 students in this capstone, this semester.

Rebecca: So why this class?

John: Because the students were ready for it. The course builds on all the courses that they’ve had up to this point and it allows them to pull together material they’ve learned in all of their prior courses, as well as the cognate classes in statistics and math and so on.

Rebecca: So what kind of project exactly did you propose to these students?

John: I originally proposed two options. One was to do something on behavioral economics, because past classes have found that to be a lot of fun, and another one I suggested was they could just pick current topics and work in small groups and create papers on that. Turned out that they really didn’t like any of those ideas and given the nature of open pedagogy, I left it up to the class to decide what their topics would be. And I think it was actually Charlie, who came up with the idea. And would you like to tell us what that was?

Charlie: I know you had mentioned in the beginning of the class the idea of open pedagogy. And I found that pretty interesting because it seemed like a good opportunity for us as seniors and juniors to really put what we had learned out there. And also, in terms of topic selection, you gave us the opportunity to really choose which topics we wanted to talk about. We ended up choosing the topic of intergenerational mobility and economic inequality. We focused more on the economic inequality aspect of it in the end. But yeah, like I said, it was just a good opportunity for everybody to really finish their college careers with something that they can show.

Rebecca: Dr. Kane is going to close his ears now and you guys are going to tell us exactly what you thought when he said, “Hey, you’re going to write a book.”

Victoria: I was hesitant at first, just because group projects are kind of daunting, especially in economics. However, a collaborative group project was exciting to do… to see all of our work put together. As economic students it isn’t something you really see, it’s usually individual work.

Maria: Oh, yeah, I would agree with Victoria. I was kind of hesitant at first, especially because it was something new for our class so I figured there are probably a lot of kinks that needed to be fixed. So I was a little worried about not having everything fully figured out at first. I thought it was something interesting. It appealed to me because I like the thought that other people could read what we had written and we could have control of what we would want to talk about.

Charlie: And the topic and the idea of a book project really intrigued me… that it just let us put out there what we had learned over these past few years and gave us something that we can show in the end of it.

Rebecca: Were any of you scared?

Victoria: Not scared. I wouldn’t use that word.

John: Were you concerned?

Victoria: A bit concerned, just because I like doing my individual work. I feel stronger in that.

Maria: Yeah, I would agree. I think group work can sometimes be difficult to have for every class because everyone has a different writing style and everyone works on their projects at different times. So I think at first, you’re a little bit worried that not everyone will be able to work well together. But I found that in my group, we were able to work very well and we’re able to meet once a week to go over what we needed to work on for the week.

Charlie: Yeah, I found something similar to that experience. Whereas my group, after the first few weeks, figured out what we wanted to do, and when we could meet, and what was the most effective way for us to put the book project together? And I think it turned out really well.

Rebecca: So you’ve all mentioned groups, can you talk a little bit about what the groups were, how they were determined, and how that worked?

Victoria: The groups were groups of three from the class, because there’s 27 people. And then we’re able to email Professor Kane and ask if we wanted to work with anyone specific, like if we had friends in the class, we could work with them. But if not, or if we didn’t want to work with someone we knew, we’re able to randomize it.

Maria: I was put into a group of two other random people that I didn’t know, but we were able to set up a group chat immediately and communicate very well through that.

Charlie: I actually emailed Professor Kane about being a group in Victoria and we also included another student in that. I think it worked out pretty well and I was happy with how it turned out.

John: Before the groups were formed, though, the class decided on what the topics would be. So we had kind of a free-form planning session where we narrowed it down to nine topics you wanted to address. And then at that point, we knew how big the groups were going to be. And it worked out nicely with three people per group.

Rebecca: How did each group get assigned a topic?

Charlie: So the way we assigned topics was, we had created a list of the nine topics, and then each individual group could choose their top three, and then we divided them that way based on everybody’s top choice. If they didn’t happen to get their top choice, they usually got their second or third, I think that only happened for maybe two groups, and they seem to be fine with what they ended up with.

John: And going back a little bit further, it was a weighted voting scheme that you didn’t just rank them… that you assigned points, if I remember was it 10 points I gave you? And so if you really wanted to chapter you could bid all 10 points on that. And if you were indifferent, you could have assigned weight to your top three preferences and so forth. And it did work out really nicely where I think most groups got their top choice, but two of them ended up with their second or third choice, but it seemed to work.

Rebecca: How did you find collaborating in the end?

Charlie: I found that it worked really well meeting every week. We also had presentations every week that we gave on specific topics that we’re talking about during that week. So that set the initial schedule for us to meet every week and talk about what we were doing and what was going on. Also with the book project at the time, in terms of organization, I found it very laid out and simple.

Rebecca: That sounds like you had a writing group that met that frequently, but it also would be more of like a study group as well?

Charlie: Yeah, I would definitely say it was a mix between a writing group… a study group. Your group members ended up being the way, if you wanted to succeed in the class, like that was the way to do it was to work cohesively with your group members.

John: And it should be noted that they had other tasks in the class as well, where they selected topics that were presented each week and each group was responsible for presenting an article or a research paper on a topic, some of which were related to the book and others were completely different. The groups were persistent across all the assignments and involved more than just writing the book.

Rebecca: How’d you get feedback to make sure whatever you’re putting out in the public was good enough?

Maria: Well, we mainly used Google Docs. At first, we tried to use hypothesis. But that wasn’t really working out well. So we ended up just going back to Google Docs. And each group would be given a couple of chapters to review each week. And they would write a couple comments in that chapter as well as some comments made by our professor and we used that and we also used each other’s feedback to make those edits.

John: How did that work? Where the comments helpful?

Victoria: No. [LAUGHTER] Just because I’m very protective over my work, which I know I should be open to criticisms. However, I got some comments sometimes I was just like questioning, like instead of “what about this article that you might want to look at” it would be “change this word,” where I know we emphasized that often in class many, many times, but still people would persistently do that.

John: I hope that wasn’t from me. Was it from me?

Victoria: Oh, no. Well, if you wanted to do that, you’re the professor. You can do that. But you also give us feedback that’s helpful. Whereas, students I feel like if they’re rushed or doing it, like 20 minutes before the class, they’re not going to look at me like, “Oh, what about this topic that might be interesting to consider.” Instead, they’re like, “Switch this word.” That just might be the students in the class too, just because we did have a lot of work in the class. And I found that because our group would meet to practice our presentation before the class, a lot of groups are doing their final work 15 minutes before the class.

John: Yes, the quality of the work did vary a little bit across the groups and across the individuals within the groups. Overall, there was some really high quality work, and all three of you did really well. But the quality of the feedback varied quite a bit.

Rebecca: So the feedback was generally done outside of class? Like not during class time?

John: Primarily, except for the presentations on the work where there was some feedback during the presentations.

Victoria: Yes, but I found that your feedback was most helpful, rather than the students.

Maria: Yeah, I would say I paid a lot more attention to Professor Kane’s feedback than some of my fellow students. Luckily, we didn’t have that experience. We had a lot of people give a sincere, really constructive feedback, but sometimes I’d be hesitant to take that feedback because I didn’t know that was the direction that I should be going in. But I definitely think our experience was a little better and our comments were more substantial, I would say.

Victoria: And I think next time maybe switching the groups that review the feedback might be helpful, because if you have one group that gives worse feedback, and you keep getting that, it’s not as helpful.

John: The way it was structured was each group reviewed and provided comments on three other groups, and we did that on three stages. And the class decided to maintain persistent groups there. I did give them the option, but I think it does make much more sense to vary it so you’re getting a wider range of feedback.

Charlie: I think the idea to keep persistent groups stem from the fact that we wanted to have somebody read the paper and then continue to read the paper throughout the weeks when we were supposed to be improving it or making it better. So then they could also see the changes we were making. And I agree with my classmates where I think we can say that it didn’t work out too well. There’s some groups just didn’t happen to give feedback that was too good.

Victoria: I also think part of it was the length of the papers because each of us had to review three full papers for the weeks that we did that, and three 20-page papers is a lot of reading to do on student written economics. And I think maybe in the beginning it was helpful to read all three, but maybe as time went on to scale that back a little bit, so we don’t get burnt out.

John: More detailed feedback on a smaller number of papers.

Victoria: Yeah because at first, I find myself doing it too. Like the first paper, I’ll take the time to read every single word and provide helpful feedback. But I can see myself not doing as much on the third.

John: I gave feedback in three different ways. The first time I gave video feedback, and while I’ve heard that that can be really efficient, I was taking about two hours or so per paper. And that was really slow and people really didn’t like the feedback that much because some of the feedback was fairly long in terms of the suggestions. So, I probably gave a little too much feedback. The second was with comments embedded in Adobe. And the third time I just basically went along with everyone else and provided the feedback directly in Google Docs. And the nice thing about that is I was able to see some other suggestions and sometimes I’d say, “Well, maybe that’s not such a good idea.” Because in many cases, the original draft actually made more sense than the feedback.

Rebecca: What way did you all like feedback better?

Charlie: I think the best feedback I received was actually in class feedback when I would go to Professor Kane and ask him, “Hey, you know, this is what’s going on with my paper. Is there something else I could look at? Is there another source I can find?” I found that to be the most effective in helping me write the paper. I was also a fan of the comments in Google Docs, they were pretty helpful.

Maria: Yeah, I think the most helpful feedback were the comments from Google Docs because, for that last draft, I was able to go through and resolve any comments that I had made the changes to and that just helped motivate me to make my draft a really good copy. And then I would say that I was really against the video feedback because I personally like to review feedback multiple times. I like to go through it and make changes to it. And I found myself just typing up his comments at the end of the doc so that other teammates could see it as well. So I was definitely against the video feedback and prefer the Google Doc comments.

Victoria: Yeah, I prefer Google Docs as well, just because I could see where exactly you wanted the changes done. It gave more specific feedback and then it also gave the students validity I guess, like this should be changed. Like I made a comment in one paper and said, “I think you mean a different word.” And they just resolved it and moved on. But then Professor Kane came through and said, “Yes, I agree.” And I think that you agreed, and they now are aware that yes, those changes need to be made.

Rebecca: I think sometimes when a faculty member responds to student comments in a way that it also helps students know how to make better comments. So it would be interesting to see how another round of that would have gone after Dr. Kane had responded to some of those right? To see if the comments were better the next time around.

John: Yeah, I think I should have done that from the beginning. And I’m sorry, I didn’t. But in the future, I’ll probably use Hypothesis. Now that we have Hypothesis in Blackboard it will be much easier. Among the problems we had is that people had some trouble making comments on Google Docs because they also had edit access to those and they couldn’t mark up specific text. And with PDFs, that was a bit of a problem given the way the browsers were set up that they had to change a program in order to make comments on PDF documents. So now that we have that in our learning management system, it’s going to be much easier to do that and the comments will be a little more persistent, because one of the issues was people were, as you mentioned, resolving comments sometimes before anyone else had a chance to see them. And the strategy was to have the draft documents with the comments copied over to another folder, and they were only supposed to make changes in their working document, not in the documents used for comments. But there were three or four people who through three drafts, just didn’t quite get that notion and I’d see the email saying that comments were resolved, and I would go back in and unresolve them. But in any case, there were some problems with those. That’s an issue that I think has to be worked out a little bit more efficiently.

Rebecca: Beside some of the technical issues that we mentioned, what were some of the biggest challenges of working on a project like this?

Charlie: I think one of the bigger challenges was keeping the cohesive idea behind the whole book where the topic we had chose was income inequality and we also had talked about intergenerational mobility. But as the book progressed, we kind of saw that portion of the book fall off a little bit where chapters were really focusing on the income and economic inequality topic.

Rebecca: So is that something you discussed in class to keep everybody on track?

Charlie: I think we mentioned it at one point towards the end, we’re just like, “Okay, are we going to keep this? Are we going to not keep this?” And I think we agreed, we could talk about it but we won’t make it a major portion of the book.

John: There was also some scaffolding on the project… that it didn’t just start with people starting to write, groups were first asked to put together a bibliography, and then an annotated bibliography, and then an outline of the chapter, and then the actual writing started after they had feedback on each of those steps.

Maria: I would agree with Charlie, I was definitely worried about the cohesiveness of the entire book. But for my group, specifically, we did a very broad topic, the global trends of economic inequality, and for myself, it was really hard to find relevant subjects to talk about because it was just such a broad topic. It was really hard for each of us to find something that we could spend a large amount of time writing about. So I’m not sure how the other groups felt. But for us, it was definitely hard narrowing down what we specifically wanted to talk about, and then to find resources that were recent enough to include.

Victoria: Yeah, I agree with you on that. I think one change I would make after we figure out the specific topics, you can go deeper in that because it’s hard as a group to form a thesis statement or very cohesive argument because we ended up doing more of a timeline than like an argumentative paper because you have to split it up.

John: Your topic specifically was on what?

Victoria: Tax-structure and income inequality. So basically, we looked at early 20th century, later 20th century, and the 21st century, and how the changing tax structures led to increasing income inequality over time. So that’s kind of how we split it up. But I think if I was to do it again, I would take a different approach to it, because I did the first section and finding information on World War One income inequality is much harder than it seems. So I struggled a lot with that too.

Charlie: Yeah, in terms of how we wrote our chapter of the book, I’m usually a fan of writing papers that follow a timeline as an explanation but that’s just a personal preference. It doesn’t work for everybody. So I can definitely see how making the cohesive argument along with following that timeline can be pretty difficult.

John: In your chapter, I think the timeline made a bit of sense. We were talking about the evolution of it and the transitions in your chapter were pretty smooth. I don’t think that worked as well in all the chapters, quite often it looked like they were three essays…

Victoria: Yes.

John: …chopped and pasted together.

Victoria: There was this one paper with a bunch of sub topics, but it wasn’t cohesive. And I was reading it and it just did not make any sense to me how it was organized. So that was one of the suggestions I made… maybe taking a step further in class and presenting maybe our papers a little earlier.

John: In more stages…

VICTORIA. I was just trying to read it and I just could not make sense of the organization of it, where maybe if we caught that earlier we maybe could have made better paper.

John: I was giving them feedback in several groups… that sort of feedback… that they need to smooth out the transitions and have a more logical structure. But some groups responded really well and did a nice job with that, other groups were a little more reluctant to do that.

Rebecca: Perhaps some groups will respond really well to some peer pressure. [LAUGHTER]

John: And having the presentations in class would have helped do that. When people in the class were saying, “This is just too disorganized.” And most of them got better by the end, but it was a stretch getting there.

Rebecca: So you’ve talked a little bit about some of the challenges, but what was really rewarding about working on this project?

Victoria: I found it really helpful to work with the group. I had Charlie and then another student, Junweii, in my group and we all read each other’s parts. I know I went through the document and made comments for my own group too. And we were all able to bring it together, make comments for each other, ask each other questions about like what sources to use especially too. And it was easier in that regard than an individual paper. Because if you make a mistake and you don’t realize it, no one’s there to help you, it’s just you. But here we have people to help each other.

Charlie: Yeah, I always find it beneficial to complete a task with other students also trying to complete the same task as you. It just makes the learning more interesting. And you’re more willing to go and spend the extra hour looking at the document to just make sure you understand what you’re writing, but also that it fits with whomever else you’re working with. I found it really beneficial or satisfying just the fact that we, like I said, could create something that any ordinary person could probably read and understand what was happening.

Maria: Yeah, I think the most rewarding part for me was just seeing that finished product and getting you know, positive feedback from Professor Kane and from my other group members. I think working in that group setting helps to motivate me to do the best of my ability. And I think it was just rewarding at the end to see everything come together well.

Victoria: I think it gave us all a deeper understanding of the material too because, instead of writing it yourself… because you can write something and not understand it. I’ve done it many times. [LAUGHTER] But when you’re sitting in a group, getting a presentation ready, you each need to understand the material. So you’re explaining what you learn to each other. And that’s something you don’t get by yourself. I found that really rewarding.

John: What about the public nature of the project? The fact that this will be out there, it will have your names on it, and it could be out there indefinitely.

Charlie: I found that portion of the project pretty intriguing and exciting. Just like I said, you can go out there, and obviously we’re college students, we’re looking for employment after this. So just showing an employer, “Hey, I’ve written something that’s been published. It’s out there, you can go read it for yourself and see what you think.” It gives something for the students to show.

Victoria: Yeah, it made it exciting because we knew what was at the end of the project. Rather than just a finished paper, we actually had something to like prove ourselves, like we did this.

Maria: I think for me, it was cool to know because at the end of the semester, I’m able to go to my family and say, “Oh, here you go. This is something that I worked on all semester long. Here’s something that you can read and you can better understand what I’ve studied for the past four years.” So I think it was helpful that I was able to show my family I’ve worked hard on this. This is something that is to show for that.

Charlie: I would definitely concur with that. Economics as a topic isn’t really discussed when you’re talking just with family members, so many of them don’t understand what you’re talking about. And you’ll try, but it’s hard sometimes. So to put something together that they’d be able to read and understand, I found that pretty satisfying.

John: And how did the class select the audience for this? What level was it written for?

Victoria: Students with a background in economics I think we decided on. But we came together as a class and decided on that. But you need economic background to understand some of the things we wrote.

John: But at an introductory level, so it wasn’t written at an advanced level. It was written for people who’ve had an economics course somewhere along the way.

Victoria: Or just no background. You don’t have to go to college to read the book.

Rebecca: How would you change this project in the future? We touched on a couple of things here and there, but do you have any other key things that, if the same exact project were presented to another group of students, how would you change the structure? Or the way it’s organized? Or the way that it’s presented the first day?

Victoria: Thinking about the class as a whole rather than just the book project, we did weekly presentations which was a lot of work in itself. So I would probably minimize those and focus on the book. Because we were sitting there reading 20+ page economic journals every week and making a presentation on it and doing the book project. So I think having more time dedicated to the book project and presenting on that material, rather than just economic journals that people have written, like it gives background, which is helpful, but maybe a little less, or maybe shorter ones, or ones that are just easier to understand. Because I know a lot of times you would say, like, “I know you guys don’t understand this, it’s challenging. But we still need to know it.” Like you would explain it in class, which would be helpful, but reading something you don’t understand is really difficult for students… in economics specifically. That’s challenging.

Maria: Yeah, I would agree. I think, at the beginning of the semester, it was a lot of work to have to juggle both the presentations and the book at the same time. So I kind of like the idea, I’m not sure if it was you Victoria, who mentioned it in class, of doing the presentation one week and then the next week working on the book and having class time devoted to the book in the week after. I think that would have been very helpful too because we did meet as groups, but if we were able to meet in a class setting than I think other classmates will be able to make comments on your chapter and offer advice. I think it would just help overall with the workload that we have.

Charlie: I also agree with that. I think the improvement can be made where we’d work on maybe a random topic every other week, and do a presentation on that, and then also incorporate the book project into that. I think it would help with the cohesiveness of the book along with just feedback and all the other problems that we had discussed.

John: One of the things I had suggested at the very beginning, you may recall, is I suggested one option is to spend the whole class focused on this. Another option is just to do it the way it was done in the past, or something else. And the class actually voted for the something else. Now having had the experience, the something else didn’t work quite as well, and that more class time should have been devoted, I think, to this and I saw that too.

Victoria: I think we’re just looking for something exciting. Like yeah, it’s a book project like we know what we’re going to do with that. But the presentations just added something else, but if I went back to a book project because then we could have taken the steps at a slower pace too, like the annotated bibliography, like the topics, we could have taken way more time with that than we did. Because once we did that very quickly, and then went into presentations, and then we just had due dates instead of meetings in class.

Maria: Yeah, I think for us, what appealed to us with this combination of the book and the presentations was that the presentations offered structure for us when we knew what we were getting with those presentations. We knew each week that we’d come in with the presentation. And I think with the book, we were excited because it was something new and different and I think we were a little too hesitant to go fully and choose the book, because we weren’t sure what we would be doing in class. We weren’t sure how we would be tested on that. So I think the combination of fields lost because we were able to have that structure, but we were also able to try something new.

Charlie: I know for some of the students in the class they had mentioned to me… they were hesitant to get rid of the presentations weekly because they were a fan of learning something new every week and learning a different topic, not just focusing on the book project. They really wanted to increase their knowledge base by just learning about multiple fields of economics. So I think that’s why we ended up going with what we went with in the end. But I think we all could all agree that if we had done that every other week, it would have been more efficient.

John: I agree. And I think some combination might be good for the reason you mentioned, but more class time devoted to it would be helpful.

Victoria: Maybe at first too, do a presentation. Like the first presentation, I don’t know what week that was, but maybe keep that one because when our group really met each other, we worked together, and then we planned a time every week where we would meet.

John: And if this is done again, and that will be if the class wants to do this in the future, perhaps that first topic for the readings could be related to whatever they choose to do so they’re actually doing some scaffolding with the presentations then.

Rebecca: I had something similar in my classes before where a team formed early on. We did something small, low stakes, to figure out how to work with each other and what doesn’t go well. So that when we did something a little more high stakes, you already knew what the wrinkles were going to be so that you could plan for that moving on. So it sounds like your presentations served that purpose, whether or not you intended that to happen or not.

John: But it became a lot of work when it was done every week, in addition to writing a book.

Victoria: That was difficult.

Maria: Yeah, I think it just helped to make us all more comfortable with each other and more comfortable speaking in front of the class.

Rebecca: So the big question is, of course, should other faculty do this?

Victoria: Yes, I’m working on my honors thesis right now, which is kind of what you would do in a traditional seminar. And it’s very difficult. So just having people there… write it with you… know what you’re talking about… You can ask them questions. In our group chat, we often ask, “What would you recommend for this part of the paper? Or what articles do you think are appropriate for this?” If you’re doing it by yourself, it’s very difficult. And the overarching topic… I feel like in a lot of seminars, they have that. It’s a topic for the seminar, but it doesn’t really filter through as well as the book project does, because we are all cohesive, all of us together working as a class of 27 people, which you never see. So, I found it really helpful and I liked it a lot. And it wasn’t like a crazy amount of work. You did the work, and you study, you did the presentations, and you wrote a paper, but it didn’t take you hours every day to work on. I feel like I learned more in this class than I have in other classes that I write individual papers for.

Maria: Well, I think I would partially agree and partially disagree with that. I think as a class, we all appreciated that Professor Kane was willing to change like the class structure and was willing to try something new. And I think that was definitely intriguing for us and provided something different as our last economic course. But I think if I had done my own topic paper, I think I probably would have learned a little bit more, I think just I would preferred that. But I think it was still important to get this experience and try something new.

Charlie: I think I would definitely suggest it to some other faculty members to maybe try this out. Like Victoria was saying, working with a group is pretty beneficial. And I feel like, from a personal standpoint, I learn more when I’m working with other people who I can ask questions to, get feedback from. Really, it helps your understanding of the class. In terms of incentive, I find that I wanted to work on the book project because you had that end goal of, “This is something that I can put out there and show to somebody.”

Victoria: Yeah, but at the same time, group work can sometimes be the worst thing that ever happens to you. Like we got really, really lucky because I know Charlie, we’re friends so we were like, “Okay, let’s work together. We’ll just get one random person.” Junwei was like such a blessing. We just work together so beautifully, but I feel like if we had someone that didn’t want to do the work… wasn’t willing to put in the work… didn’t show up to meetings… that would ruin the project for us. So I don’t know how you could fix that. But just if there’s a good group, it works. If there isn’t, I feel like it wouldn’t work as well.

Rebecca: So good to write one book during your time here, but maybe not many books. [LAUGHTER]

John: But there could be other things. For example, they could have been podcasts that were created. They could be collections of essays.They could be video projects that are put together by groups. So there’s a lot of different things that could be done.

VICTORIAL: Yeah, I would throw that out there. If you did this again with another book, like, yeah, you can write a book, but you can also do that… a different kind of form of the same kind of structure. That would be interesting.

Maria: I would be interested in doing some type of podcast because I know some of my friends in their classes have been required to do podcasts. And I feel like you have to prepare really well for that. So I think maybe that would have forced your teammates, if they weren’t doing the work, to do the work so that they wouldn’t get to the studio and not have anything to say. So I think that would have been another really cool option.

Charlie: I think it would be a good option for capstone classes, just because I know for a lot of majors, you hear what the capstone is about for the three years before you even get there. And I know personally for me, I’m also trying to get a political science degree, my capstone is next semester, like I’m already dreading the 25-page paper I’m gonna have to write. So to switch it up and have the students maybe not know exactly what they’re in for, I think it gives a little bit of an intrigue and like, “Okay, this isn’t just the I’m going to go and write a paper all year. It’s something else that I’m going to do.”

Victoria: Yeah, it’s more fun. I’m more willing to write a paper that my group members are in. Like we can all see each other too in the Google doc and talk to each other in the chat… be like, “What do you think about this part?” Or like Charlie can watch me while I’m writing my part of the paper and say, “This is good. Maybe change this. Or bring this sentence up.” You don’t do that in individual papers and even if you write an individual paper and have peer feedback, it’s not the same as having it right there, real time, or just people caring more because it’s theirs too.

John: We did have some issues with that early on though, in the first draft or two, because there were some people who really didn’t want to try using Google docs for writing. And were any of you involved in that?

Charlie: So, I’m not opposed to Google Docs. [LAUGHTER] I had just always used Word documents before. So it took a little bit of getting used to but once you commit to it, it’s a really nice thing to have in your repertoire. Google Docs, I feel like, is used by countless numbers of people, companies, places, businesses, the college. So honestly, as a student, you should just take the incentive to try to get to learn it. And once you learn it, it’s really beneficial to you.

John: One of the problems was that some people were writing in Word and then uploading it to the drive and that made it really hard for other people to edit. And eventually everyone switched over, but it did take a few iterations with some people.

Maria: So yeah, I think there were a couple of challenges with having different drafts because people made comments on separate drafts. So I think just sending out a reminder email would be helpful and letting people know because I know I think I made my changes on the wrong draft the first time and we had to send an email right away to have him fix that. So I think just having it set up all before the due dates like before you mention it in class would be really helpful too.

John: Yeah, there were some rough spots. This was new for me too.

Rebecca: So we always wrap up our podcast by asking what’s next for each of you?

Charlie: This December, I’m looking to graduate from Oswego, which is exciting for me. And after that, I’m not really sure what’s going to go on. We’ll see.

Maria: Well, I’m graduating this Saturday, and I’m going to be moving down to Florida for a little bit and doing an internship there.

Victoria: I’m also graduating Saturday and I’ll be working at HSBC this July in their graduate development program.

Rebecca: Sounds like exciting futures for each of you.

John: What are you doing in Florida? An internship where?

Maria: I’m doing the college program, the Disney College Program.

John: Oh, wonderful. Maybe I’ll see you there at the OLC conference. Well, thank you. It’s been a pleasure working with you all semester. And thank you for joining us.

Charlie: Thank you for having us.

Victoria: Thank you.

Maria: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.