48. The Culture of EdTech

As faculty, we engage with education technology as it relates to our classes but rarely consider the larger EdTech ecosystem. Dr. Rolin Moe,  the director of Academic Innovation and an Assistant Professor at Seattle Pacific University, joins us to discuss the politics, economics, and culture of EdTech.

Show Notes

Transcript

John: As faculty, we engage with education technology as it relates to our classes, but rarely consider the larger EdTech ecosystem. In this episode we examine the politics, economics and culture of EdTech.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Our guest today is Dr. Rolin Moe, the Director of Academic Innovation and an Assistant Professor at Seattle Pacific University. Welcome, Rolin.

Rolin: Thanks for having me.

John: We’re glad to have you here. Our teas today are…Rolin, are you drinking tea?

Rolin: I am John.

Rebecca: Yes!

Rolin: I am having the Maui Up Country blend that I picked up on a on a vacation that I had brought for the office, and we ran out. So I am drinking the wonderful Keurig inspired Celestial green tea today. But I am joining you guys over there. [LAUGHTER] What are you guys having?

Rebecca: I think it’s a green tea day. I’m having black raspberry green tea.

John: …and I have a ginger peach green tea.

Rolin: Excellent.

Rebecca: We’re all in sync without planning, so that’s nice.

John: We invited you here to talk a little bit about your April 2017 EDUCAUSE Review article (which has created a little bit of a stir) where you were talking about the growth of educational technology in higher ed. What types of EdTech in particular were you talking about?

Rolin: So, John that’s a good question… and a little bit of preface on the article itself. I wrote that with George Veletsianos, who is Canada Research Chair in Innovation and a Professor of Education and Innovation at Royal Roads University in British Columbia. We started this project in 2013 at a time when MOOCs had just come into conceptualization. Laura Pappano noted that the year before had been the “Year of the MOOC.” John Hennessy at Stanford said that the MOOCs were going to be a “tsunami that was going to wash away higher education as we knew it.” Clay Shirky compared higher education to a rotting tree that was in need of a lightning strike and this was going to change it… and so, this very optimistic (to the point of Pollyanna) thought on educational technology. And George and I both, as people who are scholars and practitioners in educational technology, were a little taken aback by this. The promises that were being related to educational technology didn’t match the literature. The history of educational technology didn’t match the present and the future track of these innovations, based on their previous experiences (kind of Silicon Valley startups) was not a positive one. As I mentioned, we started writing this in 2013 and the landscape kept changing. Ownership would change, or business models would pivot, and we had to rethink what we were doing. So we kind of, instead, came back to this more systematic review of what is educational technology, or EdTech, and we thought of it in socio-cultural terms as a phenomenon. So, thinking about that, it’s not necessarily a product that we are providing critique for but it’s more of the idea that by bringing products in, whether they be cloud based softwares, learning management systems, apps, learning technologies interoperability, or LTI, or outsourcing it to a third-party vendor, whatever that vendor may be. That approach cannot be thought of as altruistic in and of itself, but it is built in society that is usually, at best, tangential to education, but often completely separate… being brought in for profit bearing reasons, whereas our institutions, by and large, are education-bearing institutions that are looking to gain enough profit to continue operation. So, what EdTech are we looking at? We really want to be creating a more critical consumer of all EdTech. And you can definitely see that today in privacy issues that are coming out with Facebook and algorithmic issues that are happening with Twitter, and discussions of what constitutes free speech or hate speech on these platforms. When we wrote, we were much more thinking about the technology that’s getting into schools, but even there, some of the things that are happening in K-12: the data from these students is not necessarily protected, whether it’s getting hacked and sold to other places or if the companies themselves have connections to other products and other vendors. So, it’s a really meta piece to be thinking about. I don’t necessarily have an axe to grind with any particular software. That’s why we were very software agnostic when we were writing the piece. We just really want to be much more conscious of how we’re using technology in our teaching practice and what is happening because of the technology we’re bringing into our classrooms.

Rebecca: Thanks for laying down that groundwork. I think that foundation is gonna give us a good ground for discussion today and will help our listeners know exactly where we’re starting.

John: A lot of these things, where people were really optimistic about the introduction of MOOCs and so forth, we’ve seen all this before. Television was going to do the same thing. Before that radio was, if we go back further, printed books were going to have this big impact. So, these are issues that have been around for a long time. But, you focus on several issues that, perhaps, are more pressing now. One of the things you talked a little bit about is how colleges have been pressured by economic circumstances, by rising tuition costs and pressure to keep costs lower, to rely more on these external vendors. Could you talk a little bit more about that aspect?

Rolin: Absolutely. I need to preface here again, John, I appreciate you bringing up television. Because there’s a time that a lot of institutions invested in broadcast studios, with the idea being that we were going to be able to amplify education and we’re going to be able to have closed-circuit educational opportunities at senior centers and satellite campuses. And so you have in many land-grant colleges these forgotten studios, that in some cases are now being turned into teaching and learning centers where you have a green screen and you can show what you’re doing in Canvas, or Desire to Learn, or Blackboard, or whatever the system is that you may use… Moodle, I don’t want to leave anybody out. But, to think about my experience as an educator, I have a connection to this particular podcast. I cut my teeth as an educator in my first career, which was in film, at Duke University’s Talent Identification Program where I got to know John Kane, who has been kind of very foundational in how I think about teaching and learning. So John, thank you for that, and it’s wonderful to be on your show. We’ve seen all of this before and we failed to learn our lessons in education. So, we didn’t get out of television what we thought… what we thought we’d get out of radio we didn’t get. It’s important to look back and see “Well, what didn’t happen that we expected to happen? What did we plan for? What was the consequence? What were the unintended benefits? and what were the unintended pitfalls?” The problem, or the big difference today, is a lot of the technology is being looked at from an efficiency standpoint. So, television and radio and even if you go back, like you mentioned with the printed books, you go back to correspondence courses and using the Penny Post in order to be able to give keyboarding instruction for secretarial jobs. So, those technologies were based on much more inclusivity in education. You had a technology that made education available for more, and you had an opportunity to get away from geographic distance as what was keeping people from school. With digital technology what we’re seeing now is almost an inverse relationship that “Yes, we have this opportunity and we sell it.” So, the MOOCs were sold as an opportunity to democratize education for everybody. But, this is really framed in a cost-cutting perspective. That we’re going to bring in technology to keep costs down. That’s very important, costs in education, and higher education especially have skyrocketed, and to think about how we can be looking at this. But it’s disingenuous to say that our digital technologies are going to democratize education for all when we want to use them to save money more so than grant access. We have to look at both critically. We have to put the same research behind both. Moreover, what’s happening when our use of technology is in the gaining of data analytics that could be used, at best, in our spaces, but at worst by third-party vendors that we’ve signed contracts with that we don’t truly understand where they’re going or where they’re taking these things. So, I started with your question and went in a lot of different directions I’m realizing. But, I think it’s important to do that historical review and think about all those places because there’s a desire there, with what education’s supposed to be, if you want to think about Enlightenment-based thinking on education. But, we are at a different point now than we were with what someone like Soren Nipper would have called generations of technology. The first generation being radio, the second television, the third digital. This fourth kind, of web 2.0, has a much greater economic impact, both on the institutions as well as the whole purpose of education. That’s something that we don’t see a lot of in the literature and something that compelled George and I to write this article.

Rebecca: I’m hearing you talk about the the desire for more access but then also these rising costs. If we’re using EdTech, are students actually just getting more access? or are we just making things more expensive at the cost of actual learning?

Rolin: Yes. [LAUGHTER] It’s difficult because in some cases there is an upcharge on taking the course online. And there’s good reason for that because in order to teach a course online, if I’m an administrator, I now have to think about a faculty member who’s going to be working through that course. I have to think about any licensing that I need for contents. So, making sure that my reserves in the library can be easily flown into my LMS and that I have the rights for reproduction in that space. I have to think about instructional design, I have to think about information technology. I have this much larger infrastructure that’s involved, depending on what I’m doing: if I’m going to be using an anti-plagiarism software; if I’m going to be using an online proctoring software, a special grader, a video library of contents. There are four or five different buckets of LTI and those are the general ones, not anything discipline-specific. So, that brings this cost up. At the same time, if you think about Moore’s law, and as technology is increasing and the capacity to do things continues to increase, traditionally we have seen costs go down in this model. That hasn’t happened with education. So you have a space where students are presented in media and, I would say in a lot of cases by schools themselves, that this online efficiency opportunity to engage is going to bring your cost down, but then your cost is becoming more, because the cost on the institution is more. All of that is to say, at some point, if you’re gonna be selling both cost savings and access, that’s not a recipe for success. In many cases, we have the access, but it’s not to people, it’s not to a high impact educational experience that you have come to think with a stereotypical higher education space. I think of the Sally Struthers ITT Tech, you know, where you can do the courses in your pajamas. So we’re giving access in real time to curriculum and to materials, are we necessarily giving it to really engaging learning activities? In some cases, yes… but I don’t think the literature would say that those brightest cases of access are meeting that romanticized version of what it means to be a student in higher education. In many cases the most successful institutions in creating access and bringing costs down are the ones where faculty have been replaced by kind of quasi-administrators who work as admissions support specialists, tutors, retention specialists, program developers, and fundraisers. Kind of doing all of that from an office space, and that looks remarkably different from what we see in cinema, as somebody who works in film studies… what we see in cinema as that college experience. So, we’re gonna have to rectify what it is we think college is supposed to be with what it is we’re selling it as.

John: Might some of that be that, with new technologies… giving an example from economics… when steam engines were first introduced, we didn’t see any real improvements in productivity for decades after that. When the internet was first introduced and people shifted businesses to that, it’s taken decades before we’ve seen much of an increase in productivity. Is part of it that we try to use the new tools in the same way that we traditionally taught and we haven’t learned how to use it more efficiently, or is it something inherent in the shift to more digital media that limits the interaction between the instructor and student and may limit learning somehow.

Rolin: John, thank you for bringing that point up. If you think about professional development technology, the stereotypical overhead projector that is used to present material is then replaced by the PowerPoint…and what was interesting is, in some cases, the first uses of PowerPoint in classrooms (because of bandwidth issues) were printouts of PowerPoint slides that were then put onto overhead transparency. So what we see in many cases today what constitutes online learning is the lecture based approach the “sage-on-the-stage” model of teaching where we’re using our learning management system to do what we’ve traditionally done, and it’s what I would call a mediocre middle. It both misses the point of improving education and also misses the point of utilizing the technology, but it’s what we do. My fear is that there has been a financial success in doing things in this way, or at least creating a media culture that equates formal education to the lecture. So you think about a TED talk, or you think about a Coursera lecture… this idea that it is a faculty members responsibility to share their wisdom as the person who’s speaking through it. A podcast is another space, we’re people who are talking in a space. Now that doesn’t mean there’s not a space for podcasts and there’s not a space for lecture, but it’s easy to package that content and put it into a learning space that you’re hoping to monetize. For learning to be effective online and bring down costs, probably requires a pretty seismic shift in how we think about business as normal. Some of the early critiques of online education were that it would turn us into a fordist space, where it was gonna be the assembly line production. That was gonna get away from a faculty member as kind of an auteur, somebody who has the course from its implementation to its full assessment. With online that’s almost impossible to do for the sanity of anybody. So, in some cases, that model is going to need to change in order to be successful. We haven’t figured out what that looks like yet and the human capital costs of doing it right so far outweigh the benefits that you get from allowing students to be able to take classes from a distance and increasing your enrollment, hopefully through online. We haven’t figured out how to weigh the human labor that goes into that. And I think some of it is also we haven’t changed… I’m gonna get radical here, the expectation of what it means to be a professor is still the same as it was 50, 60 years ago, but what we consider is knowledge has changed pretty significantly with Ernest Boyer’s thoughts on scholarship. What it expects to be a faculty member… so the expectations of teaching at even teaching heavy institutions have gone up but the expectations on scholarship or service have not changed. So instead of it being a triangle of scholarship teaching and service it’s this odd triangle that is morphed into a parallelogram with no extra time given to these spaces. So, we’re gonna have to think about our governance structures and our infrastructure if we’re going to be successful. There’s an article in The New York Times this week we’re recording this in mid-September talking about what the next financial bubble may be, and it points to student loans that the cost of education has gone up fourfold over the last 30 years, outpacing everything, including healthcare… and the student loan debt has over the last five years, overtaking credit card debt. It’s the largest amount of debt that exists in any industry. That cannot keep up. Y et costs continue to rise. So another thing; in the next 7 years, that traditional college age, students 18 to 22, is going to decrease in 2025 because of demographic shifts. So, there’s a lot that’s going on at this point, and John, you mentioned the steam engine and how it took decades… Well, we keep saying we’re the Wild West and we only have years until we get to the cliff, and many people would say we’re already past that point; we’re at the point of no return. I like to be a little more optimistic than that.

Rebecca: I’m gonna go back to a little bit of discussion about access. Some of the things that I hear you describing is that the technology is allowing us to have access to information or the distribution of information. Which is why the lectures, the podcasts, et cetera are easy to package and deliver the access to that information. But, what I’m not hearing is access to learning or the access to becoming a scholar, or a way of thinking or being in the world. And I wonder if some of the movements in OER or the open education resources are trying to push the envelope or push the technology and access more in that direction, or if it’s really still emphasizing the ability to just deliver information.

Rolin: Rebecca, you bring up a really great point. And I’ll touch on OER because it’s a fascinating case study in this space, but if you look at the history of distance education with technology, the focus was on bringing people together… that the content operability was not the key point… but it was being able to bring people from disparate geographies or cultures or climates together to learn. And so it is based in constructivism and constructionism and social learning theory and activity theory and all of the wonderful progressive learning theory that is moving teaching and learning today. And the technology that is predominantly used stands much more didactic, maybe behaviorist, in approach because it’s easier to measure that than it is to measure the much more engaging work that happens when you bring people together. So I had an opportunity (I’ll try and not give away any disclosing information on this), but I had an opportunity to work with a group on a MOOC in after the first wave of MOOCs—this was 2013–2014. They were on a major platform and they had created a course, and it was not a traditional STEM course; this was an arts-based course that they had created. And the platform came to them at the end and said, here’s what happened in your class and had this ream of analytics and they said “Well, wait a second. We had a Facebook group, we had meetups, we had a lot of people create artifacts. Where does that fit into this?” And the platform just kind of shrugged their shoulders and said, I don’t know. We can tell you how long someone watched the video and they were saying, “That’s not what’s important to us. What’s important is what were the conversations that were happening and how is that gonna relate to where they’re going further.” We’re in a time of measurement today, yet our measurement structures are much more basic than our capabilities with technology. And so we’re engineering the technology to perfect those measurement techniques. We can’t do much more with bringing people together and engaging more progressive emergent learning theory with technology. I think what George and I were arguing is the technology, as it stands today, doesn’t feed that because that’s not what’s getting the clicks, that’s not what’s moving the needle, whatever metaphor that you want to use in that space. MOOCs are a fascinating space to look at this because the MOOC acronym actually comes from an experiment in social connectivist learning from 2008 with George Siemens, Stephen Downes, Dave Cormier and the great Canadian contingent. And then Sebastian Thrun didn’t even talk about it when he became the father of the MOOC in 2011. He was looking at a bold experiment in distributed learning at Stanford. It was a New York Times reporter Tamar Lewin who made the link between what George Siemens had done and what Sebastian Thrun had done and called it a MOOC. And it kind of stuck and that’s where we went with that. So it’s very interesting to look at the hype versus the research and why the hype is what’s pushing the cart when in academia we like to say it’s the research that does. Now you mentioned OER. I want to focus on that because this is a really fascinating space that in the last couple years you’ve seen this remarkable push on open educational resources, open textbooks, and I am a longtime advocate of open education… been attending the open ed conferences that David Wiley has been putting on since 2013. I ran the unconference there last year. So I’m advocate for what they’re doing. But it is interesting to think about their success and what their advertising is. Their paramount success is really focused in textbooks. So while you have the opportunity to edit a textbook and you have the opportunity for a faculty member to build artifacts of knowledge with students and cross collaboration, that’s not what’s moving the conversation today. What’s moving the conversation are these static textbooks that bring costs down for students. And I like to be the voice that’s saying, don’t forget about these places, because I worry that we’ll see something, and you can even see a little bit of it happening now with publishers who are wanting to open wash or green wash or astroturf what open is and say, “Oh, you know, here we are over here at Pearson or McGraw Hill and this is our contribution to the space.” When you look under the hood it looks remarkably different, but if the focus remains on this static text book in that adoption, it’s easy for that to co-opt. So, to answer the question in a more broad sense, I think in general we have research that’s telling us one thing and we have marketing and public relations and cultural ideology that’s saying something else. I don’t want to say we’ve done a poor job, but the two are very incongruent right now and usually it’s that media PR machine that’s pushing things and we’re playing catch-up and it’s easy to lose track of the research in that.

Rebecca: As a public institution like we are, obviously access as in all people should have access to the information is really important, but I always get concerned about the people who are generating the technology pushing it in the wrong direction and people who value everybody having education and learning not being able to push the envelope or push the technology in the direction that we want to push it in. They’re kind of butting heads in some ways.

Rolin: I would absolutely agree with that. And accessibility, it’s really wonderful to see accessibility being brought forward in terms not only of contents but also of learners, and so the stigmatism of having learning disability or an emotional or physical or some need to engage with content, that now is going to be supplemented by an institution. And that we are designing with that in mind. We’re designing a universal access and UDL that we’re engaging in this space, and that’s a really wonderful change that has happened in higher education. When people talk about the cost of higher education, it’s important to note that things like that are bringing the cost up, and I don’t think any of us would want to get rid of any of those pieces. The problem, of course, becomes “What is the historical understanding of this place?” and “What is our institutional objective and our institutional memory versus these changes that are happening in how we think about teaching and learning?” And I’ve done as much as I can locally at Seattle Pacific University to start conversations and meet people where they are and I think we’ve had some some pretty remarkable success in rethinking some of our structures, but we’re a private liberal arts institution not dealing with the state bureaucracies, not dealing with a state system, not dealing with tens of thousands of students, and it becomes difficult to navigate all of that. Bureaucracy is the least worst tool that we have in order to work with that. But it’s also a great straw man or easy fall guy for any problems that come up, and too often problems continue to exist rather than being tackled because it’s tough to think about what the benefit would be going forward.

John: In your article, you talked a bit about the increasing reliance on private vendors, outsourcing tasks from institutions to vendors on the grounds that that opens things up to the free market in some way, but when we look at the provision of most of these platforms, it’s a fairly unstable market. We’re seeing so much concentration in the market where many small publishers have disappeared, and many of the innovative educational technology providers have been bought up by other large firms. We’ve seen many providers disappear.

Rolin: I’m glad you bring that up, John, because if you think back… and John, you and I have a background in K-12 and it’s really fascinating to think about this from a K-12 perspective because in the 60s and 70s, the heyday of educational film, it was the job of the media resource specialists at a K-12 library to work with faculty to be able to understand how these pieces fit together, and so they were working with Encyclopedia Britannica and World Book and Disney and ABC and NBC and the different content providers of the time, who were making educational titles. It’s a fascinating, fascinating time. The computing craze in the 1980s came at the same time as a recession. And the idea being for people to think about how this was all going to fit together. When this was created the idea was that that role was going to be vital, and the change that happened was we got rid of the media resource specialist and believed it will be up to institutions and collaborations to grow this, to make this go further. Educational film died because it became less expensive to make it and the belief was more and more people would make it. What we instead make are lectures and YouTube videos, and there’s value to both but the great expectation that we had on what these contents could be is gone and we’ve lost that. And so there’s an opportunity… I think that if you think about learning as this contextualized and locally defined space… there’s an opportunity to be able to create these contents. But there’s a lot of risk that goes into that. There’s a lot of quality control that we we didn’t necessarily expect. And there are a lot of other costs that came in and so we output to these third-party vendors, hoping that we hit pay dirt with somebody. In many cases those companies are folding regularly or they’re being absorbed into others, the learning management system ANGEL, which was a rather popular system in the early 2000s and early part of this decade got bought out by Blackboard, and a lot of the people who liked ANGEL liked it for the reasons that it wasn’t Blackboard. But to think about, in that perspective, it’s almost impossible today for institutions to take this on their own. There’s just not a return on investment that works for that, so that means you either have to create these partnerships across institutions that historically have been at war with one another, or you invest in the promises of third-party vendor, either a small one that’s telling you what you want but may not be around in a month, or a large one that you have a lot of trust issues… best-case scenario, trust issues on the kind of service you’re going to receive; worst-case scenario, what does it look like what’s happening to your data, what’s happening to your analytics, what’s happening to the ownership of what you’re producing.

John: Going back to ANGEL a little bit, we used to use ANGEL here and in many ways I loved it; it had some really nice features that Blackboard is a ways away from getting. It had automated agents and so forth, but ANGEL was actually created at Indiana University. And one of the problems they had was that in the 2007 recession, state support for Indiana University was cut significantly and they owned this big cash cow that they could sell off… and so we lost a fairly viable provider in large part because we see in general a decline in federal and state support for higher ed and it puts institutions in a difficult bind where they often outsource more and more.

Rolin: Absolutely and ANGEL is a good example of that. You can go into the 60s with Plato. It was a Midwest State school that was doing Plato. I think about Quest Atlantis was another great thing that gets mentioned in all sorts of progressive educational research that was funded by grants and the funding dried up and there was no way to sustain it. The MEK Corporation, the people who created Oregon Trail and super munchers and that educational software, where is that today? And I work in educational film, I think about it from that perspective. How have we lost those film providers and now we just think that content will fit in for what was historically this really rich and vibrant place to engage, but we’ve lost it on the software side and the teaching and learning side, and we’re outsourcing so much of what we already do to the free market. Certainly there is benefit to that, but at what cost? And I don’t think there’s been enough analysis of what that cost has been.

Rebecca: So, you’re really bringing up the idea that EdTech is not neutral and that there’s competing goals. So, technology companies are obviously trying to make money and then we’re trying to have students learn, ideally. How do we help those things become more aligned? What needs to happen so that we’re not at odds but that we actually find alignment and essentially make the world better which, in theory or in PR, is what’s being said?

Rolin: I think for the first piece, Rebecca, is understanding that EdTech is not neutral, and once we have that foundation, that we understand what we’re using and what it relates to, we can be much more thoughtful about how we use it. So, I am a faculty member but I am primarily an administrator and I use our learning management system here on campus. I could go off the grid; I could try and do something completely different, but it’s important to show support of what we’re doing with an understanding of how that works, and so we have our LTI, whether it’s anti-plagiarism software or proctoring software and all these pieces, and as a scholar I can have criticism of that. So, as a practitioner, how do I help my students understand what they’re getting into with this and making informed decisions about that space. So, I think it really comes into this idea of understanding the learning environment and what my job is: to control… to create pathways for students to be able to learn and to scaffold that and to fill knowledge gaps and help people expand their zones of proximal development, to go Vygotsky on us. I need to cede some of the “management control” that goes into: “Well, we use this, and this is what’s going on.” But, let people make thoughtful decisions about what’s happening with the technology that they’re using. My son in K-12 can opt out of state standardized testing and that’s a decision that’s made as a family. Dealing with college students, we don’t give them the same rights to opt out of some of the technologies that are being used. So, I think about the proctoring technology that was out of Rutgers that was running in the background on computers using retinal scans to engage people and that’s just what you get when you sign up and there’s no informed decision or consent. There’s not even a Terms of Service that you have to read through and then click a button that you don’t actually end up reading. Can we have more of these conversations? Can we be more informed? Because, if we have that information, we’ll be much more thoughtful in the decisions we make on what vendors we choose. The vendors will then have to respond to that market in making software that is more open or more transparent in its use and the application of its data. People have to make a profit. Education has to make a profit. We can marry those pieces together and have a somewhat vibrant marketplace that is serving the learning of students. I think the issue is, right now for EdTech, the student is the customer, not the buyer, and so there’s a gap there that if we have students much more involved in all aspects of that and involved in those conversations, that becomes part of learning experience. I think that that could see some more direct improvements than just generally saying, “well, we’re thinking about this and we’ll continue to think about this going forward.”

Rebecca: I think one of the things I’m sorry I was gonna pick up on the threat of audience but okay

John: You mentioned keeping students in the zone of proximal development. One concern with standard lecture based teaching is that students are pretty much forced to move along at one pace. What’s your reaction to adaptive learning? Is that something that could help, or are there some limitations that we should be concerned with there?

Rolin: I mentioned Plato earlier, the first personalized learning network—basically adaptive learning. I think that there’s a wonderful opportunity for adaptive learning platforms and for being able to bring in competency-based education into spaces. Thomas Edison University has an amazing program that is built on the idea of competency-based education. Alternative pathways and moving away from “seat time…” there’s definitely viability for that. It just has to be thoughtfully executed… and what is the purpose of the learning that is happening in that space? So, if I think about a School of Health Sciences, I think about nursing… if I’m going to get a degree in nursing, there are really specific things that I need to do. I need to pass very specific exams that are proctored in very specific ways that expect me to maneuver in very specific fashions. The seat time is important for that, and that space there needs to model what I’m going to be getting into in an industry. So, I can’t be an intrepid change agent saying, “No, this needs to be social learning theory,” it needs to be what takes off in nursing. No, nursing students need to be able to be successful in the expectations of their field. There are the places that adaptive learning can fit into that. You see it in foreign language in many cases and the supplements that are happening there. Keyboard instruction is another one where that comes in. So, how could we use the best of that to be getting into other spaces. I think some things that we could explore there, as we rethink disciplines and what works for economics or film studies or education. I think there’s some places with that critical thinking… that soft skills, 21st century learner stuff… where the adaptive learning could come in…. so, misinformation, media literacy, fake news… big hot topic and I wrote an article in 2017 that got a lot of attention (not all positive) saying that fake news wasn’t the problem; it’s not what’s ended up resulting in Brexit or the results of the 2016 election. But it was a small part of a landscape that had been neglected and was suffering from blight for a long time because of how we teach this stuff. And I wonder if thinking about digital literacy, which we’re all expected to incorporate into our classrooms, if that could be served by an adaptive learning platform that engaged content, theory, criticism and evocative video to be able to move somebody on a pathway. That’s a place where all of us could come together because there’s no discipline that owns information literacy. It’s built out of information literacy in libraries. But librarians often are the most flexible in thinking about how their craft is going to change. Places like that, critical thinking, the stuff that we’re all told needs to be imparted to our students, but it’s just kind of this hooray concept of “Oh yeah, let’s have this.” Maybe those are the places to really focus on the successes of that and then the research can help define how economics could engage adaptive learning or film studies or education or cell biology.

Rebecca: One of the things that you said earlier is that students aren’t the audience of or aren’t the buyers of the technology. And I wanted to shift that a little bit to thinking about audience and who things are designed to and I think you’re right in that tech companies are selling to administrators who are the ones that are doing the buying and the purchasing who are trying to facilitate certain things, keep cost down, et cetera. How do we shift that conversation so that tech companies start to see the end users who are really students and faculty as the audience of their marketing, of their conversations, and actually shift things so that they focus on the research around learning and improve learning rather than just facilitating something?

Rolin: The key part of what you said, Rebecca, you kept going back to learning, and I think that’s what’s missing in these vendor conversations. We have this idea of what learning is and if I’m a vendor and have mounds of data I can point to achievement and I can point to the things that I measure in my platform that lead to that achievement, and for most instructors that’s not evidence of learning. That might be a small part of it but there’s a much larger picture. And we do a poor job of amplifying that research. That research doesn’t play well in mainstream media, so how do we do a better job of sharing that research. What constitutes learning? What makes learning happen? I love going to YouTube and looking up “do-it-yourself how to fold a fitted sheet,” ‘cause I don’t do a good job of folding a fitted sheet. And I’ve tried numerous times and I still struggle, so that video isn’t the piece that I need to be able to move me there. Now, there are other pieces, potentially making something for dinner that I would be able to replicate in that space, but replication again is not learning. So, even an understanding of: What is learning? What does that mean? How do we define what it is to have learned something? What it is to be a learner? “Lifelong learner” is a commodified term at this point when it really should be a state of being for, I would say, pretty much anybody. How do we engage those conversations? That’s a really complex question. In terms of an institution, how do we bring more student voices into these spaces? and not in a placating fashion of, “Well, we now have a student sitting on this committee.” But to really understand how that student can canvass and caucus with their peers to be able to provide us information. In the same way that if I’m serving on a faculty committee so that I’m meeting my service requirements, but if I’m getting something out of that and I’m giving back that’s a wonderful experience. A student serving on a committee… how can we provide them what they need for their CV or for their graduation in a way that what we’re asking from them they can provide us? …and not just sitting there and saying we’re listening to what they’re providing but often not doing that. So, more student voices in those decision-making processes… more research that’s going to be shown to the vendors… and I think we need to be more thoughtful about those vendor conversations. One thing we do here at Seattle Pacific University, we actually have… with our faculty… we provide entry points for vendor assessment when we do test demos. What are some of the things that faculty who are very interested in being part of these conversations but are coming in the middle of it… what’s happened so far and what are questions they can ask we’ll be able to draw out their expertise and what we need from the vendor? The more of that that we do, the better. I know the California State University systems doing something similar on automating a great deal of the pre-production that goes into assessing vendors so that the stakeholders who are asked these questions have that information in a repository and can access it very easily to make an informed decision, rather than it being brought down from higher administrators… lots of information that’s tough to digest in a small period of time.

John: What do you see as some of the most promising areas where EdTech has some potential?

Rolin: Excellent. My wife loves to say it’s very easy to show why you’re against something, but you get into this business to be for something. Get in education to really share the diffusion of knowledge and help people rise to heights they didn’t know were possible. Fall in love with things they don’t yet know exist as Dr. Gary Stager would say. So, what are some of the positive things that are happening? I really think there’s a chance for a revolution in multimedia. Here’s this podcast that is happening in an interdisciplinary fashion in SUNY Oswego bringing in a faculty member from a completely different perspective who serves an administrator having this conversation. More and more of this is happening. Before we went on the air, John, you were talking about editing your two channels and making sure the sound was right and all of these skills that were picked up that don’t come when you get your PhD in economics. So, as these pieces are coming in how do we value that and so you see more administrations and more governance bodies that are providing value to that. We were talking, Rebecca, about open education. The University of British Columbia now will recognize the editing of OER materials as part of promotion, tenure, and review for their School of Education. That’s a phenomenal change that has happened in how we think about the role of the faculty member as a distributor and conveyor of knowledge. I think people are being more thoughtful at this in this day and age. But, you did ask specifically about technology, so I need to pivot back there for a second. I love some of the stuff that’s happening in virtual and augmented reality. Some of the really interesting research that’s happening there. I like the drop-in classes that are happening around special interest topics that often, in many cases, are informal or non-formal learning spaces. Museums putting on areas where you can come in and learn in a certain time. Kind of a gap between a human experience and the MOOC but you’re kind of doing both at the same time. I think that the opportunities that we have with free and ubiquitous devices… and I don’t mean free as in cost but I mean free as in access to… and especially in the West with broadband capabilities, what’s going on with video and how we can better engage that and as more people learn about nonlinear editing and cinematography and camera and sound. What are some of the resources we’re gonna build there? Opportunities for students to share their knowledge is the main thing that comes forward for me. WordPress, which runs, what, a quarter of websites in the world is getting incorporated more and more into courses. You think about the WordPress camps. There’s a great thing happening in New York City coming up on managing the web and how you can work with students to be able to be creators and owners of the knowledge that they’ve created and what the implications are in that space. It is kind of a tough time to be bullish on technology if you think about Facebook and Google and Apple and Amazon and antitrust that’s going on in all of those spaces. And so a lot of the stuff that I’ve mentioned here is somewhat renegade, somewhat guerrilla even. So where are those opportunities to engage with environments through online? It comes back to community in that space. How do you find and foster that in your networked identity. There are opportunities and more and more that’s going to be happening. I think that we’re in this storm and after this there will be, not a calm, but there will be an opportunity to look at what’s been broken and how can we build and improve going forward, and I think that we’re getting to that point sooner rather than later.

Rebecca: We generally wrap up by asking what’s next. You talked a little bit about what’s next in EdTech, but what’s next for you?

Rolin: What we’re doing at Seattle Pacific University around academic innovation; we have been offering seed grants to faculty for the innovations that they see as necessary, whether that’s in a classroom, in a department, in a college across the entire campus working out in the community. We provided 45 of those over a two-year period, so almost a third of our faculty directly affected by those and it was very powerful, so we’re taking that a step further and engaging at a school or college level and finding innovations that we can then potentially put into day-to-day operations. So, one of the things we’re thinking about actually are adaptive courses. What would it look like for a course in nonlinear video editing to be almost entirely online. And you think about that with lynda.com. I can go to lynda.com and take a tutorial in using Final Cut or Adobe Premiere. What am I getting out of being in a higher education institution that I can’t get off of Lynda? That’s what we’re exploring: what does it look like to have that scaffolding and support that’s directed toward a greater understanding of knowledge? Other things are definitely around social justice. We are seeing at Seattle Pacific an increase in first-generation and historically underrepresented students who are coming in with the same scores as their peers but, once they get here, we’re seeing a discrepancy between where we would expect them to score and where they are scoring. And we have statistically significant research showing that that is the first-generation student demographic. So, what are some pieces we can put into play to be able to help them with their success? Because it’s not a matter of not being able to do it; it’s a matter of the structure and the culture is not befitting them. So, we have a program called the Bio Core Scholars where we are working with tutoring and mentorship on research, community, and knowledge gaps to be able to move these students. We’re in our fifth year of this program, we’re looking at expanding it. But we have brought the students up a full standard deviation in their scores, and we had an 86 percent success rate in graduating people to pre-professional health programs, which is just a remarkable number. Personally, I’m really big on what we can do with educational video. What are some of the things instead of it just being a lecture? I love Skunk Bear on NPR, taking a topic and in three minutes doing an entertaining, evocative dive into that topic, but again, that’s Oliver Gaycken would call “decontextualized curiosity.” How do we take that and actually put it towards learning? So, I’m looking at what does it look like to have lecture mixed with a very product based assessment mixed with more evocative filmmaking to move people into learning? How does that all go forward? It’s a very exciting time to be in higher education, even with all of the things that are looming on the horizon.

Rebecca: Certainly doesn’t sound like you’re gonna be bored any time soon.

Rolin: Not at all.

John: Thank you for joining us. We look forward to hearing more about this.

Rolin: John, Rebecca, thank you guys for having me.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

30. Adaptive Learning

Do your students arrive in your classes with diverse educational backgrounds? Does a one-size-fits-all instructional strategy leave some students struggling and others bored? Charles Dziuban joins us in this episode to discuss how adaptive learning systems can help provide all of our students with a personalized educational path that is based on their own individual needs.

Show Notes

In order of appearance:

Transcript

John: Do your students arrive in your classes with diverse educational needs? Does a one-size-fits-all instructional strategy leave some students struggling and others bored? In this episode, we examine how adaptive learning systems can help provide all of your students with a personalized educational path that is based on their own individual needs.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Charles Dziuban. Chuck is the Director of the Research Initiative for Teaching Effectiveness at the University of Central Florida, where he has been a faculty member since 1970 teaching research design and statistics. He’s also a founding Director of the University’s Faculty Center for Teaching and Learning. Welcome, Chuck.

Chuck: Oh, thank you both. I’m just so pleased to be here. The good feelings of Oswego continue… you know my growing up upstate New York and my attending Oswego. I’m really honored to do this… makes me feel good.

Rebecca: So glad to have you.

John: We’re very glad to have you.

Rebecca: Today our teas are:

Chuck: I have water from the water fountain.

John: My tea is Yorkshire Gold.

Rebecca: I have Lady Grey today.

John: We’ve invited you to join us to talk about adaptive learning. So, what is adaptive learning? Is it something new?

Chuck: Sure I’d be happy to talk about that. Let me answer that question in reverse order. It’s anything but new. I think adaptive learning has been around since we’re thinking about teaching and learning. I think if we went back to the Middle Ages we have an idea of this notion of adaptive learning. It is not new. In fact, where I came from, and what I’m in my career, I’m looking at a paper written by John Carroll called “A Model of School Learning…” where John laid out this set of equations that basically answers this kind of a proposition: if how much time a student spends learning is constant, what they will learn will be the variable; if what they learn is the constant, how much time they spend in the educational enterprise will be the variable. So that’s the basic sort of notion of that. When we define that, it kind of makes sense. If we give students a limited amount of time and anything… a course… a university… a semester… whatever you call it. It makes sense that what they acquire in that activity will be the variable. If what they learn is a constant, how long they’re going to spend in this enterprise will be the variable and we’ve all experienced that, but we’ve all butted our heads up against something for years and years and years until we finally understood it… and this is the basic notion of adaptive learning…. whatever time it takes, it takes for you to learn something… and he went further to break that down. When we say time, it’s just not how much time in the Augustan sense, it’s how much time do I spend actually in the learning enterprise and what is your perseverance… and what is your aptitude for this… so kind of modified for that. So, that makes eminent sense. That’s what adaptive learning is… basically it’s going to take whatever time you need as a student to learn this material to some sort of satisfaction level. That’s what it is. It’s a very simple concept.

John: …because students come in with very different backgrounds. Some have very rich backgrounds in some areas and weaker in others… and that’s going to vary quite a bit depending on what they’ve learned in prior courses or in their life experiences.

Chuck: Oh, absolutely. I mean I can remember my own experience walking in new courses at Oswego being virtually lost because I had no background and no experience in the course and having to really ramp up my energy and effort and use of time in order to: 1. reach the baseline in the course and go on in other courses. In adaptive learning we have students who enter courses who… don’t take this the wrong way… but fundamentally don’t need to be there. They’ve mastered the material… they can move on… and I’ll give you examples of that later. So, this gives a great deal of flexibility to the learning enterprise.

Rebecca: From the learning perspective, it sounds really wonderful. From a teaching perspective, it seems like it can be a little bit challenging – especially before some of these technologies were available.

Chuck: Nightmare. Yeah, it is. Of course, the way we’ve conceptualized this whole notion of how students learn, it really does… We’ve organized learning around this entity called the class… it’s a unit… yeah, and we call it a class… and if you think about it in terms of a course sequence, say in mathematics, the students who don’t pass their placement exam so they take intermediate algebra, and college algebra, and trig, then precalculus. You can conceive of that as these discrete units, but if you think about it, what we’re doing in our math department… we’re designing that as a series of skills across those things we’ve called classes and that makes the whole notion of teaching quite different. Really what you have to do at adaptive learning… if you begin to plan a course you really have to look at what you have taught, what the granularity of the course is, and your role as an instructor in all of that. It’s very very different and it really forces our instructors to think about what it is I’m teaching… it is much more than a syllabus. The granularity of what you teach becomes critically important.

Rebecca: How did UCF get involved in adaptive learning?

Chuck: It’s an interesting question. That was organic. You may or may not know that UCF is the fastest growing university in the country. It is one of the largest universities in the country. The good news is we’re growing; the bad news is we’re growing. [LAUGHTER] We cannot possibly build an infrastructure to house our students. So, besides adaptive learning, we have online courses, we have blended courses, we have lecture capture courses, and adaptive learning is a natural outgrowth of that sort of structure. 43% of our full-time credit hours are produced online so we’re almost a half online university… and one of the things is we became involved in this simply because one of the things that John said. Our students are very diverse. They come with very different backgrounds and even putting them in this thing called a course may not make sense for some of them in terms of their pre assessment of where they are… and the notion of adaptive learning pre-assessing them and then putting them in the proper learning sequence within a set of objectives is why we got involved in adaptive learning… to doing this… and what we did is we simply recruited our best faculty, say “Look, see if you can do this. See if you can, then kind of accommodate what students are, where they are, and how they’re learning, in a much more flexible environment. It just simply make sense for students… and given the fact that we have a diverse background…. students coming from underrepresented populations… it just makes sense at so many levels.

John: One case where we’ve done that institutionally here at Oswego is, beginning about three or four years ago, we adopted Aleks for the math requirement. It’s used for initial placement and then it’s also used for students to get up to the level needed to meet basic course requirements. So, students come in and then if they don’t meet the requirements for the courses they’d like to take, they can spend the summer working on that to move up to the level… and then test themselves again to basically get to the level they need to be to make good progress… and it’s been working very well from everything I’ve heard, but we haven’t done much more yet at an institutional level.

Chuck: Yeah, well there are, as you probably know, with technologies like this there’s a probably a new platform coming out every day. I mean this as a metaphor. Our math department uses Aleks and if you go into an Aleks class… and Aleks is a pre-designed course, there is some flexibility but it’s a course that’s pre-designed. These platforms come in basically two varieties: where courses are embedded in them like stuff with McGraw-Hill or Knewton or something like this, or content agnostic like Realizeit where you have to build the course. That’s two kinds of things. We have an Aleks course in college algebra, and if you look at it, it kind of looks like chaos. Students are coming in, there are people lecturing on the boards, there are student assistants helping them, they’re on the thing. When they master it, they master it and they leave. So, any concept of a lecture course is completely out the window, but what’s happening is: students are being assessed, they’re being reassigned to different areas within the Aleks platform to master the skills, and then moving on. So, some of them can complete a course in half a semester… and I assume you’re experienced at seeing this at Oswego, but one of the things we’re facing is where with adaptive learning, we’re trying to get over the teach then test model for a course. We’re trying to embed the learning assessment within the adaptive learning platform. So, we have a test-free course where essentially assessment becomes part of the learning platform assessment… actually becomes part of the teaching… and it’s really quite exciting, but it is very daunting to do. We’re kind of wedded to tests in our environment.

John: I believe that you’ve been working with Colorado Technical University.

Chuck: Yeah!

John: How did that partnership come about?

Chuck: Organically.

[LAUGHTER]

When we came to realize that adaptive learning may offer some advantages for us, we asked several vendors to come in and make presentations to faculty… and this was basically a faculty decision. They saw several platforms and they said “Realizeit is the one. This is the one… because we want the flexibility to build our own courses.” But, of course, there’s a Chinese proverb: “Be careful what you ask for….” and they got Realizeit… but now they had to build the courses. That’s daunting when you start building your courses. Realizeit’s been very helpful. They have a process called ingestion where they’ll try to take whatever you have in a course… and I’ll say this and I say this with love and kindness… all of these platforms have a bit of clunk associated with them. They all work, but they are all some problems associated with them and problems in quotes. I’ll say challenges, or in Provost speak, they have opportunities associated with them. [LAUGHTER] …and we work with them… but in doing this kind of thing, but we began presenting this with our partner (and I’ll talk about our research relationship with RealizeIt in a little bit)… but working with RealizeIt in some research that we began presenting and Colorado Technical University has been doing something that we’re not able to do yet, that is scale it. Scaling it is an issue. What they have done is, they’re a private for-profit institution, and they’ve scaled it at a remarkable kind of thing and thrown a lot of resources at it. So, we’re very good at research, they’re very good at scaling, and Realizeit is very good at research. So, when we saw them presenting and they heard us presenting, it was sort of a speed date. They said “hey, you’re doing pretty cool stuff” and we said to them “hey, you’re doing pretty cool stuff, you want to play?” …and that’s how it started… working with them and it’s led to several publications and several presentations, and then with RealizeIt looped into this, working in the background, we’ve been able to do this kind of thing where we think partnerships are really important in studying this… and of course I’m gonna recruit Oswego today. You should be joining us.

John: I’ve been looking at adaptive learning platforms for a few years.

Chuck: Yeah.

John: I’ve worked with some from the publishers but it would really be nice to build something from the ground up. So, we’ll talk about that perhaps a little bit more later.

Chuck: Alright. Well, I will say this. Our philosophy is we give away everything. You can have anything we have. We’ve published several papers. We have several projects underway now with adaptive learning. Any of your audience, and certainly my brothers and sisters at Oswego can have anything UCF has done. The idea is we need to do it for partnership. I did a presentation on, if you go to Google and search “the Grand Cafe.” The Grand Cafe was the first coffee house in England, and what it was was a result when the British discovered coffee houses ideas really blossomed. The Brits didn’t drink water for a long time because they were afraid of the water, and so for a couple of hundred years they basically started having a beer for breakfast.

John: Right.

Chuck: A couple of beers at mid break, and then a beer and a gin. They spent a couple hundred years drunk.

John: ….and so did the founding fathers.

Chuck: Yeah, absolutely. But when they discovered coffee houses… a coffee and tea (which you’re drinking)… they were in this space they were in this learning space and when you switch from a depressant to a stimulant there are many more ideas and that’s the grand Enlightenment… but the point being that it was a partnership. When we talk about these grand ideas they are not Eureka moments, they are not lightbulb moments, they are people working together creating ideas and, if you will, letting ideas have sex… in the idea that these ideas can grow and be developed and that’s what happened…. and that’s what happened with our partnership and now we’ve added another partner. We’ve added a Petroleum Geo-Services in Oslo, Norway. It’s an exploration company looking for oil all through the world, and suddenly they have to train these people aboard ship and they looked at this and said: “Whoa, it costs us a fortune to bring these geophysicists back to Oslo to train them. I wonder if we can do it aboard ship with adaptive learning.” So, we’re looking at that. So, there are a lot of possibilities and I’ll say we worried about the students adapting to adaptive learning. No problem. Of course, it’s embedded in our LMS so they don’t even know they’re not in Canvas. They still think they’re in Canvas.

Rebecca: So, you alluded a little bit to Realizeit’s potential, because you mentioned that it’s one of the few platforms where you can build courses from the ground up. Can you expand a little bit more about its strengths relative to some other platforms and maybe also some of the stumbling blocks associated with building something from the ground up?

Chuck: I’m not sure I know the characteristics of all the platforms. I know that Realizeit is content agnostic, and I’m sure there are other platforms that allow you to do that… and probably there are other platforms available too, as you begin to explore this where courses are assembled but they have some flexibility for faculty members to take components in and out. I would think that would be a real advantage. It’s sort of the continuum… building of course from ground up is daunting and one of the problems is the granularity of the course. Getting this level of granularity so the course flows evenly…. it’s always a rough start in terms of doing that and we’ve provided as much support we can for faculty.

John: How can faculty get started using adaptive learning? Is this something that’s best done at an individual level, or at a departmental or an institutional level?

Chuck: It’s a big task and whatever support you could provide… especially instructional design support… in terms of looking at a course, redesigning it, and putting it together, and that’s very important… and run-throughs are very important, test beds for courses before you roll them out. I, in my role having to evaluate our technology, I taught online, blended, lecture capture, adaptive, and each time I rolled it out… it’s been terrible. It has a lot of bumps along the road and our working two years with adaptive learning…. they’ve had some issues and by our work with them for two years, we have helped them clean up a lot of issues… and they would be the first to admit that. They help out on the research side of this. They’re very very strong. They can do research things we can’t. We can do research things they can’t do. It’s the perfect yin and yang… and this is a wonderful kind of relationship to have…. and Michael Feldstein and Phil Hill and the MindWires e-literate have built this project now they call the Empirical Educator where they’re trying to get vendors and academics together to work together… to begin to look at this.

John: We will include links to the articles that you’ve mentioned in the show notes.

Chuck: Absolutely. Okay.

John: The question I had about that (in terms of individual development or other forms of development) was basically whether people are developing their own variants of the course or is it done at the departmental level? Or the program level?

Chuck: Both.

John: Okay.

Chuck: How do we begin an adaptive learning? One of the things you have to do to demonstrate to faculty that it can be successful. So what we have done is we cherry-picked our best faculty, the kind of faculty that if you show them this platform, they’d say, “hey, we’d like to try this.” So our initial pilot study was done in Psychology, was done in College Algebra and Pathophysiology. And all of these faculty are the ones who would say, “I would try to do this anyway,” and we built it that way and then began demonstrating it with individual faculty. And now we have a pilot project in our College of Business to make as many courses as we can adaptive learning. So it’s now up to the college level and we are now making presentations to all of the individual departments saying, “look, this is the possibility— is this a strategic initiative for your department? Do you think this would be something that would be a value to a department? A value add to your department versus one-off courses?” Because, for instance, in Psychology, the Psychology professor developed it over three semesters and faculty really do have the option of how much they can lock down this course. You can lock down an adaptive learning course so it looks like a regular course or you could make it go completely adaptive. It’s pretty scary stuff and what Jeff did in Psychology is his third go-around, he said, “it’s adaptive, go!” That’s all there was to it. They could go through at any pace they wanted to. In a cohort of say 20% finished the course in three weeks and that sounds like heresy, okay. But they did it and they were finished. Their next thing they said, “okay, we want Psychology too, we’re ready for it” and wasn’t ready. Okay, it wasn’t ready. So what do you do with them— your students who have completed a course in three weeks. They’re done, they’re verified, they’re certified that they’ve completed the course and they have nowhere to go. So it has great consequences for building this out in terms of doing this and we work very hard. Intermediate Algebra (they didn’t make it into College Algebra) so Tammy Muse, who will be featured on 60 Minutes in a couple of weeks, has allowed them to start Intermediate Algebra when they complete [it], go directly to the Adaptive Learning College Algebra course and that cohort now is finishing the course on time and it’s really quite amazing. And what’s the objective is, well, make that cohort larger. Just the things that you were saying about Aleks, (that’s a bad word in Florida, “remediate”), but let them acquire the skills they don’t have and then go directly to the course they need. And what Realizeit will do is sequencing them back and Tammy has done an amazing thing. She has made all of her assessment items reflect the diversity of our campus with names and the diversity of the disciplines she made them have the disciplines reflect whatever their major is, whether it be Engineering or Physics or whatever. And though the problem sets “they work” all reflect those kinds of things. A lot of work, but it’s working beautifully.

Rebecca: Sounds really exciting.

Chuck: Well, on certain days, it is.

Rebecca: [Laughter] Exciting can be both scary and…

Chuck: Oh yeah, yeah yeah, yeah.

Rebecca: We talked a little bit about whether or not the platform allows for interleaved practice. How does it work, what does that look like? Can you describe one of these courses?

Chuck: As best I can.

Rebecca: Yeah.

Chuck: It looks like you’re in an LMS (Learning Management System) and you have exercises. RealizeIt has a decision engine built in and I know a good deal about the decision engine. I don’t know everything about it because these platforms are proprietary. These vendors do not like to give away their trade secrets but Realizeit is Bayesian-based. It gets prior knowledge assessment of a student, then based on that prior knowledge, it assigns a student to a location in a course and begins and then begins to assess them. It looks like an LMS, the learning can be anything from videos, to simulations, to reading, to practice exercises, to discussion boards— it can be any format that you would normally have in any particular course. Students can participate in any number of ways they are assessed and then based on the assessment, which can be (I’ll talk about that in a second), can be anything. It can be a simulation, it can be a practice exercise, it can be a performance— it can be anything. It evaluates them and based on the evaluation, it reassesses their learning path and then their sequence back into that and they can have any number of ways to go through the learning factor. So this platform is always thinking about them but it is re-assigning them and it is constantly re-assigning them to learning passes that goes all the way through. It’s kind of scary to look at in terms of what’s happening and in it then, it has for them, “you’ve mastered these, you have not mastered these kinds of things,” I wish I can tell you it works as smoothly as I just described it…

John: [LAUGHTER] But it’s getting there.

Chuck: It’s getting there and what we tend to do is overestimate the short-term impact of these things and underestimate the long-term impact— that’s something we do with all technologies. If you look at the latest MIT 10 technologies that are gonna revolutionize them, one of them is Babel Fish earbuds that instantly translate languages for you. Well, they can’t do that exactly yet, they will eventually maybe get very good at that, but that’s what it looks like. It looks like a regular LMS and if you experience it, you have this sort of seamless feeling that you’re moving through this with no real impediments to this so you kind of go through your own pace. And what it also does it learns how you best learn. If you do it best with reading, it’ll do it that way. If your best with simulations, it’ll steer you up to simulations, but it’ll steer you to them. But faculty have to prepare all of those things so “therein lies the rub” as a friend of ours in the round theatres in England would have said.

John: I looked at the Acrobatiq platform a couple years ago, I met with some of the representatives from the gardening. I haven’t looked at Realizeit very much yet, but these platforms are really good at giving students lots of retrieval practice and assessing where they have weaknesses and doing that type of [a]daptation. But one of the things we know is that interleague practice, as Rebecca mentioned, is really helpful in increasing recall when you ask students to go back and test them on things that they learned earlier in the course is really helpful and encouraging deeper long-term learning. And I asked about Acrobatiq’s ability to do that and there wasn’t any and that’s why we were a little bit curious about whether perhaps Realizeit had that ability to go back and bring in questions earlier. I know Aleks does that a little bit but most of the platform’s I’ve looked at so far haven’t.

Chuck: Well with Realizeit, students have that option. I guess if we could make interleave a verb, they had the option to interleave, I’ll give you an example of that. Pathophysiology— in the state of Florida there’s a requirement, now, most hospitals require their RN’s to become BSN’s and that’s causing some angst. Nursing is stressful enough but now nurses are given a certain timeframe to achieve their BSN’s so nurses are coming back, some of them unhappily. RNs to get their BSN’s and you’re doing it in adaptive learning platforms and online platforms in the Pathophysiology was adaptive so Julie Hinkle, who taught the course, said, “I’ve got 30-year RN’s who worked in Cardiology their whole career coming into my Pathophysiology course, taking the Cardio unit.” Well, they know more about cardio than I ever will and so within the adaptive learning platform, they simply go to the Cardio unit and test out. They’re done, ok, because they know everything but nurses are funny. If they don’t get quote “a hundred,” they’re not satisfied. So that’s their interleaving, they will go back and test themselves again and again and again and again until they get the satisfaction. High pressure, high pressure in that field. It’s not like beginning Psychology— if I pass, I pass in terms of this. And what Julia’s done is created incredible adaptive learning measurement devices. She’ll give them a series of bloodwork in blood gases that they all have to look at. I mean it’s hard and they all get different values and they all have to assess this patient and the protocol for this patient based on different values associated with and then she’s got them in discussion boards and if the values don’t make sense, the nurses, because of who they are, will go back until they get it. They had the option to do that. In this case that even you talked about her student-driven, you don’t have to force them— yeah, they have the option to do it, some will, some won’t.

Rebecca: Have you been using any open education resources as part of the content for the adaptive courses or are these all closed system faculty you’ve created? Materials…

Chuck: Our faculty, when they begin looking at it, you had this sort of adoption curve, we got the early adopters, who will do anything. When we look at our online courses— we’re very good at online teaching, we are very very good at it. However, we probably have 2,000 courses in our vault and faculty will ask me, “will I be any good teaching online?” Well, my response is, “are you any good face-to-face and if you’re not, you’re not gonna be very good online.” But we range from faculty who do things that are very text-based, faculty members who will not stop putting bells and whistles in their courses. And we say to them, “stop it, you’re being annoying. Stop with the gizmos in the course.” Somewhere, there’s a balance in this kind of stuff and obviously what you have for this preparation of course is a lot of work. I don’t know about you and Oswego, I assume everybody teaches one course and has a lot of free time [LAUGHTER]. We’re not that way at UCF. Our courses are very large, heavy teaching loads and right away, faculty say, “this is the too daunting.” So what I realize is doing in a lot of platforms are doing that are like us. We’re looking at OER (Open Educational Resources) and saying, “what of this can we ingest into this and make it available for a course.” In terms of, “can we take some of the right stuff, put it in there, and load it up so it’s ready to go” and then you can adjust it as we go along. Obviously those courses are good and if we can do that for faculty, it’s a great service because building a course ground up is daunting no matter what happened. And unless your institution provide faculty support, it’s probably too daunting for an individual faculty member to do this by themselves in my estimation.

John: Does Realizeit provide any package materials to help get people started, for example, including OER.

Chuck: Yeah they do, they provides a great deal of resources for it. They have said representatives down when things were not working as well as we’d hope. Basically all of these vendors and I’ll say they have the Veg-O-Matic, we have the platform for them to try it and it doesn’t chop vegetables equally well. That’s the partnership that’s so critical and there’s vendors are not just vendors anymore. They have to be active partners in this educational enterprise. They’re not just selling us stuff— they have to come and help us and Realizeit is very good about that. And others are, too. I’m not pitching Realizeit by any stretch of the imagination.

John: But it sounds like it’s worked pretty well for you guys.

Chuck: Yeah, it’s worked pretty well for us. Our issues are, “how do we scale it, how do we scale it?”

John: What sort of resistance has ever been from faculty? Is it mostly just a time issue or there are other issues that faculty are concerned about?

Chuck: I can probably give you a less of them. One: it’s a lot of work. For all faculty, I think one of the questions are, I’m not sure of the culture and Oswego, but at UCF, it still remains get-ahead equals teaching service and research and we know what of those three carries the great weight.

John: Yes, that’s an issue that’s discussed all the time.

Chuck: So if I’m gonna spend all of my time in preparation of this kind of enterprise, where’s the reward system within the culture of the University for this? And I don’t mean to be crass but what’s in it for me? Why should I do this? And we have to provide some sort of reward perfectly. We’ve done a lot, we have sowed awards that rewards faculty thing, but we tried by giving them course releases and all of the help we can have. We have a large staff that support faculty in the online environment and we have a bank of instructional designers that help faculty members load up their adaptive learning course. And basically our philosophy is if faculty can feel better about their teaching, they’ll follow you anywhere. I mean that’s yet— you know we all want to be the best teachers we can and it feels so good when it goes well and it feels so awful when it goes poorly and we’ve all had that experience.

John: Oh yeah.

Chuck: I am personally— I’m very fond of the quotes attributed to Augustine. I thought I understood it until I tried to teach it. Not one of us has not experienced that at one time in our position. [LAUGHTER] I’ve been talking about things that I’ve had no business talking about. I knew them, but I didn’t know how to teach them.

John: But you learn that as you go. [LAUGHTER]

Chuck: Yeah, oh yeah yeah, yeah.

John: In your recent EDUCAUSE paper on Adaptive Learning, you describe different types of student interaction with a platform. Could you tell us a little bit about that? I remember tortoise and hares and a few other animals in there. [LAUGHTER]

Chuck: Yes tortoise, hares…

Rebecca: Frogs and kangaroos.

Chuck: Frogs and kangaroos [LAUGHTER]— I’ve got to tell you, Realizeit is headquartered in Dublin, Ireland so one, it’s fun to listen to them talk. And two, they’re very bright so in doing this, what we decided, and then we’ve done this a lot, we had a research partnership and you’ve probably known this. If you begin presenting some of this stuff in tabular form and tables, your audience glazes over. In about two minutes, they stopped listening. So the idea there is, “how can we begin to portray this data in a way that’s more engaging to audiences?” And working with Realizeit and Colm Howlin, who was their Director of Research— is brilliant. We’re both big fans of Hans Rosling in Sweden, passed recently, but his approach to showing data in motion…

John: Oh, it’s wonderful, yes.

Chuck: It’s just absolutely wonderful. So we thought, “can we look at this” and what we weren’t able to do is portray the Hans Rosling Gapminder way, is looking at students traversing through an adaptive learning course of Psychology that is showing them in motion two things: what they’re achieving versus the number of activities. And that’s available from us in an animated form and then looking at them as they traverse through there, we have found that they produce certain behavioral styles and we found originally four. And the people in Ireland named this— they’re metaphors. Turned out to be animal[s]. The first one was the hare, and the hare— and remember what we’re plotting: number of learning activities against achievement level. The hare just goes zip. They start the course and they finish it in a very few days. That’s what they do— they’re finished. When I described in Psychology, that’s what happened in Psychology— they’re finished. And it sounds like heresy, but they’re finished. They’re done, ‘kay? And then the second one was the tortoise. The tortoise is the one who goes step by step by step by step by step through the course. They just progressed through the course in little increments and do this. And they get there, but they get there slowly. And then the Frog. The Frog did exactly what they would do in a regular course. There were eight modules in the Psychology course— they completed one a week. They did it week by week by week by week the way they would do it in a course. And interesting enough, when Jeff turned the cord, Jeff Cassisi, he turned the course open. There are a lot of students who did that— they went week by week by week because that’s what they’ve been taught. For them, that’s what of course was: you do it in these increments. And then the kangaroo— didn’t do a damn thing until the end of the course and then did it all in three weeks. Did it all— just zipped to the end of it and it’s fun to watch him in animation. He’s just dead in the water until the end and then he goes zip! And he’s at the end of the course. So there are four different ways that they approach it in terms of doing this and to see that in motion is very compelling. What it tells you is, at least in this course, there are various behavioral styles to the course and you have to be comfortable with them because they’ve all reached mastery and they’ve all reached mastery in some ways we wouldn’t approve of it. We wouldn’t approve of that kangaroo, would we? Doing nothing until the end of the course. That’s not right, but they finished so we have to learn how to deal with that in terms of doing this.

Rebecca: Despite the fact that many academics exhibit that exact behavior… [LAUGHTER]

John: That’s why we have deadlines. [LAUGHTER]

Chuck: Yeah, we have deadlines. Yeah, nothing is quite as motivating as fear, right? [LAUGHTER] yeah yeah yeah, but one of the things that I’ll say (is) there’s a lot of research that’s affecting us. And a really compelling book is called Scarcity, I don’t know if you’ve ever seen it, by Mullainathan, where they talk about students completely compelling to us at UCF for students coming from underrepresented populations in terms of what you think of students living close to the poverty line, what are they dealing with. They’re dealing with so many things in their life. Money, time, danger, single parenthood, finances, family, two part-time jobs, borrowing money from college. Fundamentally, when these students come to us, they’re exhibiting and expending so much cognitive energy just living life. Then when they come to us on campus, they burned up most of their cognitive bandwidth and then we put them in these lockstep courses where for some reason, they miss a class, they’re behind the power curve. We don’t design courses for these students. We firmly believe that they have as much intellectual capacity as any other students but we have not designed our University to accommodate. Adaptive learning is perfect for these students, it really is in terms of doing it so we’re trying to do that and accommodate these kinds of things. No wonder they drop out. In some cases, dropping out of a course becomes the optimal decision and what it is is their superstructure of life is so tenuous that if any one thing fails, this whole house of cards come tumbling down for these students. So they get behind a couple of classes and they’re done and we have to find a way to accommodate this. We’ve got to begin adjusting the way we organize ourselves.

John: What implication does adaptive learning have for the structure of the University?

Chuck: It has a lot. Our students are, now that they’re learning, asking really interesting questions like, “why do we need semesters” and the only answer we have is because we have semesters. [LAUGHTER] We don’t really have good answer for that and we’ve done this an adaptive learning we designed it. Now if you finish early, well you go on to the other course. The other end of adaptive learning is so what if you need an extra three weeks to complete the material and the semester is over? But that’s a nightmare for Financial Aid, it’s a nightmare for the Registrar, but we’re working on that in terms of doing this. But it hasn’t potentially turned on University structure on its head and I’m not sure we’re ready for that at the moment.

Rebecca: When a faculty member creates one of these courses, obviously they’re very involved in that content structure, but what’s their role in facilitating the course?

Chuck: Mm-hmm.

Rebecca: So if the class was to extend three weeks past for a particular student, does that mean then that faculty member is also engaged for that three weeks or what does that look like?

Chuck: Absolutely. In for a penny, in for a pound. If you’re involved with this kind of thing, if a student is still working within the platform and we’ve made agreements with the Registrar that they would get credit for the semester in which they did this, although they went past all of the due dates, a faculty member is still involved. So it changes the role of the faculty member immensely in terms of doing it. Some students (this is gonna come out wrong but), don’t need the faculty member. I mean, they need the faculty member, of course, I need the faculty member. This by no means abrogates the role of a faculty member, but it certainly does change it. Faculty members have to know when to intervene, when not to intervene— it changes us. It’s like teaching online: you have difficulty adapting to this. When I taught an online course, I did have difficulty adapting to my role, not being the center of attention all of the time. It was hard I have a big ego and it was difficult for me but but I got over it.

Rebecca: Are the assessments related to the adaptive learning stuff that’s like automated or something that the faculty member is manually grading using rubrics or things like that?

Chuck: Both, they are both. They get up to the faculty member we have a department who is now just getting over the notion of letting the assessments run within the adaptive learning platform and believing the assessments that it is as competency-based and students exhibited competency. That’s a hard sell for departments. Say a math department who say, fine, but they still have to take the test. You know what I’m saying? It’s a slow-moving thing and what we have done now is an a/b study in terms of demonstrating the fact that students who are assessed within the adaptive prep won’t do as well as students who were in other courses and they took the departmental exam and did equally well. We have to demonstrate that. It’s a hard sell. It’s very scary. It is a very scary phenomenon. Yes, you can design all kinds of platforms and you can build all kinds of intermediate testing devices, you can give tests within the platform and if students do well on the test, it can be, automated is the wrong word, it can be Bayesian decision to cycle them back to where they need to exhibit their skills and you can retest them. So it’s this continual cycling kind of thing.

Rebecca: I have a really easy time imagining how this would work for a knowledge gaining sort of class, maybe an a lower level of course, but a harder time envisioning when it might be like in an upper division class or one that might be more project-based or application based. What’s the experience been on your campus in terms of introductory level classes versus upper level classes giving adaptive learning a chance?

Chuck: That’s a good question. Question posted versus how would you teach Macbeth in adaptive learning? How would you do that? How would you teach clinical psychology in an adaptive learning course? The answer is that probably adaptive learning is not equally well suited for all disciplines or all levels. One is, adaptive learning is really really suited very well for hierarchical structured courses where achieving something at one level depends on achieving something at a slightly lower level. Like sequencing in math, or chemistry, or physics, or computer science. You can do it and we get it beginning psychology but is it really necessary? When you have eight modules in beginning psychology and there is a natural organic order but they don’t necessarily depend on each other. Now, Young came after Freud but, is it really dependent? That’s the kind of thing, I think there are some areas where it’s much better suited, and your question is well taken. I think we have to do a lot of exploration in terms of where adaptive learning is most suited and fits into our curriculum and it may not well fit equally well across all disciplines, all ecology. It’s a question we’ve got to do a lot of work on. Hopefully SUNY Oswego will answer most of those questions. [LAUGHTER]

John: Next semester. It’ll take a little while.

Chuck: Yeah, right. Right.

Rebecca: Do any of the classes involve, that you’ve been highlighting on your campus, have writing as a key component? I’m just curious.

Chuck: Yeah. I think probably in psychology there would be some writing involved in it. They would have to do some reaction papers and do that, yeah. So I think I can comfortably answer that question as, yes, there have been. It’s equally possible. The question is, can you teach creative writing in an adaptive learning course? In some ways it’s equally suited to it because I can imagine you could build a pretty good workshop in create a writing course, with a lot of work, but you can do it.

John: I looked at RealizeIt’s website and it said they create unique formative assessment items based on instructor provided question templates. How does that work?

Chuck: It works very well. In terms of doing it, I’ll tell you one of the things, yeah Ryan Baker wrote a really good paper. The technology is developed really well but the assessment in general is still kind of heuristic, if you know what I mean by that, we’re gonna assess your competency by whether you get four or five items right. That’s the heuristic part of that. You have to design better assessment devices. What we are doing now is we have to transform the assessment paradigm in terms of what they look like, in terms of are they authentic, are they reflective, and are they contextually relevant? Students respond much better to questions that are related to the disciplines that they’re going through, and we have to develop that. We’re nowhere near that, but we’re working on that. They’re very good about helping them. You give them a template, we want to do this, we want to do this in a simulation platform, they’ll help you work with it. They’re very good about doing that, but they can’t do it all. It is a partnership.

Rebecca: We usually hear about adaptive learning in online context. Have you had any experience in a hybrid environment or an in-person environment?

Chuck: Oh absolutely. We’ve taught some blended, adaptive learning courses, and it makes absolute sense. What’s a blended course? What do faculty members, the first time they think about blended, and they think about it incorrectly. What am I face-to-face, can I offload to the online environment? That’s about the worst way to go teaching a blended course, right? The thing is you look at what are the appropriate kinds of things for these two different formats? We have this all of the time where essentially we’ve had a blended nursing courses where they do material content offline and come in to the class and essentially do problem solving. It’s basically is very, very appropriate for this notion of a flipped blended course. We’re actually having a faculty member do it in statics, in engineering, which is really exciting. You know, it’s a whole notion of some of the courses are flipped some of the courses are flopped. [LAUGHTER]. But we sort of look at that and it makes kind of sense. But again, back to your question, is where is this modality most appropriate? Where does it fit? That’s the kind of question, and that’s really a departmental discipline sort of thing. Yeah, every discipline believes they are unique, right? Yeah, their pedagogical issues are unique to it. Yeah right [LAUGHTER].

John: Actually, along those lines we’ve had a number of reading groups here and we’ve had faculty from different departments get together and the faculty who are new to this type of thing expressed surprise at how common the concerns that they had were. To find out that people in other disciplines, very different disciplines, faced exactly the same problems and sometimes had some really good solutions.

Chuck: Yeah, we do this thing, I’m very fond of a concept by the sociologist, Susan Lee Star, she called it a boundary object. A boundary object is something like this, we do it all the time at UCF; critical thinking. We’re really into active learning now at the moment and active learning is another boundary object. We’ll do this say with the Faculty Senate, who’s in favor of critical thinking? Of course every hand goes up [LAUGHTER]. You wouldn’t dare say you’re not in favor of critical thinking. But when you get in a large community of practice nobody agrees what it is, right?

John: Exactly.

Rebecca: Yeah.

Chuck: Exactly. So, Susan Lee Star, this boundary object, is something like that. That it holds a community of practice together, but it’s very weak. It’s not strong enough to be really functional in a large community of practice but you go back to individual constituency, go back to physics, or rhetoric, or creative writing, or education, and they damn well know what critical thinking is in their discipline, right?

John: Right.

Chuck: They do. They absolutely can do it and they’re very powerful. When you bring it back to the community you’re back into the same dog fight and that’s a very powerful concept. I can name literally dozens of them: active learning, critical thinking, online learning, you go on. Very, very powerful.

Rebecca: I find writing is one that bubbles up. What do we mean by writing? What does that look like?

Chuck: Oh yeah. What is writing look like when you’re tweeting— are you writing?

Rebecca: Yeah, exactly.

Chuck: When you’re blogging, are you writing? How do you workshop writing now? Are workshops necessary? It’s all fascinating to me, but I guess that’s because we’re academics. If it weren’t for boundary objects we wouldn’t have anything to do. [LAUGHTER].

Rebecca: You start talking a little bit earlier about the implications of adaptive learning at a university. Can you expand upon that a little bit? [LAUGHTER]

Chuck: Sure. I mean the implications is; what does this say for the structure of the university? What does this say for the way when we’ve organized this enterprise called learning? There’s another great boundary object: student learning, student learning outcome, one of my favorites [LAUGHTER]. Go ahead, define that. My friend, Anders Norberg, from Sweden, we’ve written a paper called, a Time Based Model of Blended Learning. Where time becomes a fundamental design structure of a university and it’s virtually very very different from the way we organize learning at the moment and how we have organized learning is in the sense of discrete units called classes, called units, called semesters, called years and called matriculation period time. It has tremendous implications for that kind of structure. We are employed by all of these kinds of things and we’re organized by all of that so sooner or later we’re going to have to re-examine all of that, if we’re going to adopt these things and adapt these things and be environmental like that. There are three things I think associated with good ideas. One is was the adjacent possible, what’s the next reasonable step, it’s what’s next, what can we reasonably accomplish next, that’s the adjacent possible. Outside of that is the adjacent impossible, you can’t do it, you can’t do it. Secondly, you have to have the slow hunch, you have to stay the course. How many things have we done this but we’re gonna try it, it didn’t work right away and then we dumped it? I’m sure nobody at SUNY Oswego has done that, but we certainly have done that at UCF. You got to stay the course! If you read Darwin’s autobiography, he said “Yet a Eureka moment about natural selection when true”. If you carefully looked at his notes he founded months and years for it was in his notes he just didn’t know it was no Eureka moment, that’s stay the course. If you know this, you got to stay the course because it’ll be bad times, everything’s not gonna work, it’s just not going to be the way. Again, you over expect short term, you under expect long term. Then the third thing is you gotta have this liquid network, you’ve got to work with the vendors, you got to work with faculty, you got to work with administrators. SUNY has to work with UCF that has to work with CTU, that has to work with Carnegie Mellon, that has to work with Oleum Geo Services. You have to have this liquid network where we can share ideas. We can’t do it alone, frankly you can’t do it alone, we’ve got to work that way and if we work that way it’s going to change the whole way we do with the business. I guess the question for us is how much do we want to change? It’s way above my pay grade to change a University and I expect for you too, but sooner or later we’re gonna have to accomplish that there’s a lot of implications for the University.

Rebecca: There’s a lot that I’m hearing you talk about, that reminds me of agile design practice. That’s made a big boom in technology and design. The idea of small little sprints break a big problem into smaller problems that you can work towards. And then it’s iterative, you keep going back and it’s circular it’s not a straight line, that’s what I’m hearing you say. That’s the way that it needs to be tackled.

Chuck: I think that’s a good metaphor, we really be in an agile scrum and keep doing it until we get it right. That’s frankly not been our history right, part of our history is let’s declare a victory and move on to something else. I mean that in the kindest kind of way but we have some serious issues in my judgment. We have tremendous educational inequality in this country, unless we crack that, we’ve got some issues that we’re never going to solve.

John: And you guys are doing quite a bit on all those fronts and I hope we’ll have you back for some future podcasts to talk about some of those things.

Chuck: We’ve made some amazing breakthroughs in communities that you think wouldn’t achieve. The talent pool is as deep as anywhere else, but we’ve got to figure out how to let it up.

Rebecca: I think the other thing that I was hearing and what you’re talking about is the idea of these micro credentials is also surfacing that we had a prior podcast on. I really can see the adaptive learning move at the same pace as micro-credentialing because I think they’re directly related to one another and how the system might have to shift.

Chuck: Kahneman wrote that great book “Thinking Fast and Slow” and kind of admitted in terms of one of our habits is we attempt to solve a hard problem; we try to measure outcomes, we can’t figure out what to measure so we measure something else that we know how to measure. I’ll give you a great example “student learning outcomes”. We’re not too good at that, so we measure grades we measure course success, course success is not learning outcomes, we do an easier thing because we don’t know how to do the hard thing. I do that all the time! [LAUGHTER]

John: We all do that… I think.

Chuck: Yeah, I really love Oswego.

John: We enjoy it too, I’ve been here since 1983.

Rebecca: I came back.

John: That’s right! Rebecca was a student as well.

Chuck: Rebecca we knew you’d come crawling back…[LAUGHTER]… Where does the time go John? What happened?

John: I know, I just got here it feels like.

Chuck: Remember John Lennon? That song “My Boy My Beautiful Boy” that’s what happens when you’re busy making other plans.

John: Yes.

Rebecca: All the talk of time leads to the question of, Mike what does the future hold?

John: What are you doing next?

Chuck: What are we doing next? We’re reinventing the University, that’s what we’re doing. We have a new president, president designate Dale Whitaker who’s been our Provost. Dale is a very big thinker and he’s developing ideas like zero probation, all services, the hub for faculty, the faculty center and all services faculty be located in one kind of place that students become an active part of the instructional process. They’re no longer receptacles, if they become teachers as well we have to do that. They have a lot to teach us, it’s a different kind of world to do this. We have to understand better how our students acquire knowledge, we have to better understand the many generations that are existing on our campuses. I kind of like to get my news from “The Onion”.

Rebecca: hahaha !

John: That’s the best place these days and it’s not that far off.

Chuck: My favorite tagline in the onion is eccentric student reads entire book, we have to begin to accommodate the way students learn and use their learning devices. My graduate students at times will send me off to get a cup of coffee when they fix my technology problem. Technology for them is what Machan of the Reedy called “Living now in the InfoSphere” where information communication technologies talk to each other where we’re no longer in the loop, we’re on the loop. And I think you can understand that in terms of when you look at your Facebook page and what you were looking at on Amazon pops up on your Facebook page, you know that these technologies are talking to each other.

Rebecca: I thought they were reading my mind [Laughter]

Chuck: But the cover of last week’s Economist was “Epic Fail” and it was the Facebook F falling off the fail, lying on its back. The covered this week is AI spy, artificial intelligence spying you on the workplace, we have lots going on. I think the recent things that have happened over the weeks with Facebook give us the fact that we have some serious examination of our culture and our information to confront in the decades to come. I think the future is being defined for us by forces outside of our realm. This is pretty scary stuff for me. Thank you both, you guys are great. I really enjoyed it, thanks for having me

John: We’ve enjoyed this tremendously.

Rebecca: Yeah, it’s really interesting to hear what you’re doing.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

23. Teaching with comics

Looking for ways to increase student confidence in their ability to learn? Or their ability to see themselves as professionals in the field? In this episode, Carly Tribulli, a Biology Professor at SUNY-Farmingdale, joins us to discuss how comics may be created and used to meet students where they’re at, draw them in, and help them develop mental models of complicated processes and concepts.

We discuss Carly’s plans to create an OER biology textbook in which biological processes are represented using comic strips, her planned research on the effectiveness of instructional use of comics, as well the positive role model that she provides in Carly’s Adventures in Waspland, an instructional comic that Carly created for the American Museum of Natural History during her graduate study there.

Show Notes

Carly’s Work

Topics mentioned in the podcast (in order of their appearance):

Economics comic books:

STEM web comics recommended by Carly:

Transcript

Rebecca: Looking for ways to increase student confidence in their ability to learn? Or their ability to see themselves as professionals in the field? In this episode, we’ll explore how one faculty member uses comics to meet students where they’re at, draw them in, and help them develop mental models of complicated processes and concepts.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Carly Tribull, an assistant professor at Farmingdale State College, where she mostly teaches general biology for non major students in entomology. Her interests include bugs, biology, and of course, comics. Welcome, Carly.

Carly: Hi, nice to meet you guys.

John: Welcome. Today our teas are…

Carly: I’m actually drinking… a kind of cold coffee. But, but it’s good. I like it.

John: Yeah.

Rebecca: …and it used to be warm.

Carly: It used to be warm. I got it about an hour ago, so I knew this was going to happen, but I was like “You know, this is my only opportunity to get coffee, and I know you guys like to talk about what we’re drinking…”, and I was like “ooh, yeah a coffee, cool… I could have lied…”

Rebecca: That’s true. I have a Paris tea.

John: and I have blueberry green tea.

Rebecca: So, Carly, can you tell us a little bit about how you’ve been able to combine your interests in art and biology in your educational and career paths?

Carly: So, I’ve always been interested in both art and biology ever since I was a little kid. I grew up in a very science-forward family. There was a lot of interest in me becoming a biologist and my parents were both very encouraging, and my dad always sat and watched those sort of Wild Animal channel, Discovery Channel shows when I was a kid, with all like this farming animals and stuff like that. So, I was always interested in the animals and eventually that led to drawing animals. By the time that I was in high school, I was taking formal training in art and doing AP art and things like that, but also very much maintaining my biology education. By the time I was later in high school, I was drawing comics. I had discovered comics around early high school. I read a lot of manga, and then I started reading more graphic novels, never a lot of the superhero comics, but more of the weird offbeat stuff like the Sandman, and a bunch of manga series. So I started drawing comics, and I drew a bunch of weird comics and then I entered college at UC Berkeley, and I was a double major in art and biology, and I just continued that path all the way through. And I was really stubborn about not giving up art, despite the fact that I had chosen not to go to a traditional art school. I knew at that point I was going to go into biology, but I was very much stubbornly holding on to art, and so what happened when I was at Berkeley, is that I was actually able to do biological illustration as an undergraduate researcher. And that was the very first research experience I ever had, doing biological illustration for a paleontology lab. This has always made sense to me as a biologist, because there’s a really, really huge history of biology and art meeting together. Especially in entomology, when you consider the work of Maria Sybilla Marian, who is one the famous female entomologists of her time (probably the only major female entomologist of her time) and she was really the first person to study metamorphosis. And much of the way she shared that information, since this was obviously way before photography, was by these really elaborate illustrations that were shared with other entomologists at the time. So to me, it’s always made sense that there is some sort of crossover between biology and art, and I think while I was in college I was very stubbornly imagining myself as becoming that type of natural historian. And then when I was in graduate school there was a lot of encouragement for me to continue doing comics, weirdly enough.

John: Could you tell us a little bit about your graduate program?

Carly: I went to the Richard Gilder graduate school at the American Museum of Natural History, and that’s a pretty long name, but historically the museum has always funded graduate students from the City University of New York, and from the Bronx Botanical Gardens, and from NYU and Columbia, but only within the past seven or eight years or so, did they decide to start their own in-house PhD program. So, we still have all of those students that are coming from other institutions, but only recently where we like, we’re going to create our own graduate program. It was very, very, very different from your standard evolutionary biology PhD program. Usually the big state public schools, and a few of the private schools that are strong in the sciences, have an evolution in ecology, biology grad program that you spend five to six years and that you TA undergraduates to support your stipend. But at the AMNH, because it’s a museum, there are no undergraduates for you to TA. and you also have to finish in four years. So, because you had no formal TAships, and the funding was very good so you didn’t really need them, you were very much encouraged to do these informal teaching assistantships, and to find your way into the outreach education side of the museum, or working on exhibits and making yourself part of the contributing community to the museum. That is basically how the grad school ran, and I did my PhD in the evolutionary systematics of these parasitoid wasps that I study.

John: It sounded like a really natural blend of your interest and a superb educational path for you, in terms of giving you a way of continuing your earlier interest.

Rebecca: Before we jump forward I’m really curious, Carly, as an art faculty member, if you could talk a little bit about that first project, that first opportunity you had as a student and how you got that opportunity to combine your interests. Was it something that you pursued or was it something that your faculty helped to nurture?

Carly: Kind of a combination of both. My freshman year at Berkeley, I took an undergraduate symposium with Kevin Padian, who is a vertebrate paleontologist, and it was very much your standard freshman seminar. It was actually very small, it was only about 10 students. We did some readings, we did some talking, and around that time I think I was looking for research opportunities, and so I started talking with him and I started trying to get myself into the lab as an undergraduate researcher for future semesters, and it came up that I’m a biological illustrator, or that I was interested in biological illustration, and I think at some point he was like “okay, show me what you got, go draw the T-Rex,” because there’s a big T-Rex in the center floor of the Valley Life Sciences Building at Berkeley. And I went down and I drew it as best as I could and apparently he was pretty satisfied with my work. So, I joined the lab, and I was assigned to a current PhD student at the time named Katie Brakora, and I actually drew some of the images that were used in her dissertation. And that was excellent. I didn’t become Kevin Padian’s biological illustrator, but I was working with grad students that were going through grad student life, finishing their work… and at the same time I was taking the core art classes, because I was a double major and I knew I was going to be a double major for my freshman year. So, I was doing all of your standard intro to drawing, intro to painting, techniques classes, and things like that and it actually worked out really well for me to be a biological illustrator, as sort of a side biology undergraduate researcher, because Berkeley’s art program isn’t really focused on illustration or comics. It’s actually much more of a fine arts program. So, sometimes I was actually butting heads with the other art faculty, because I was very illustration focused and they’re very studio fine arts, and I was like not all of us are going to become studio painters. So, illustration seems like a skill that I should be investing in.

Rebecca: What a great story. Thanks, Carly.

John: While you’re in grad school, one of your projects was developing Carly’s Adventures in Wasp Land, and we’ve looked through that and it’s superbly drawn and fascinating. Could you tell us a little bit more about some of your work with illustrations and developing comics while you were at the American Museum?

Carly: I guess this goes back to how Carly’s Adventures in Wasp Land started, which by the way is not the title I came up with it, that was the title that the museum folks came up with it, I was just like, “okay.”

John: Did you have a title?

Carly: No, I did not have a title, that was probably an error on my part. I was opening myself up there, I think it’s a fine title. It’s a little bit goofy that it has my own name in it, but, whatever. In my interview to get into grad school, I had actually brought my portfolio in biological illustration, which was very unusual. Of course, evolutionary biology does attract people who can draw, but I think I was the first person who had come to that relatively new program with a portfolio. [LAUGHTER] I was kind of a scrappy undergraduate. I didn’t do that great in my courses. I’m a terrible memorizer, which allows me to sympathize with other students that aren’t doing that great in intro biology, especially my own students, because I actually didn’t do all that well for the first two years. And part of making myself an attractive student to graduate schools, was actually building up my research curriculum. I did a lot of research with Marvelee Wake at Berkeley after the Padian lab, and then also building up this biological illustration thing early on. I interviewed with Jim Carpenter, he accepted me to his lab, and I think he was very impressed with the fact that I did illustration and apparently it stuck with him enough that when he got a grant from the NSF, he came to me about helping him out with the broader impact section of that grant, and broader impacts is where you actually have to make your grant meaningful outside of academia. So, it’s where you would have outreach education. He remembered from my interview that I like to draw, he came to me and he was like “do you want to work with the digital outreach education side of the museum, and create a project with them? “And I was like “yeah, sure,” and as long as it was about teaching kids about wasps, and the different types of wasps, I pretty much had free rein. I started working with Ology, which is the digital outreach section of the museum, and a lot of what would happen is collaboration between me, Jim, and the Ology folks, especially when it came to writing the script for that comic, because the Ology folks have way more experience in writing for middle school readers than I did. So there was a lot of modification of my script but mostly I had free reign when it came to the illustration side of things, and I also mostly had free reign when it came to the creative decisions, like the decisions to make the wasps anthropomorphic and have them talking with you, that was something I decided on, even though it isn’t truly a hundred percent scientifically accurate. It was something that both the Ology folks and Jim signed off on.

John: I liked it.

Rebecca: I thought it worked well for adults too, I don’t think it’s just for middle schoolers. I’m just saying… I know way more about wasps now than I did before I read it.

John: Me too, and it was much more engaging than reading a textbook description of those things.

Carly: Thank you so much!

Rebecca: I also just really love that you’re like a superhero in the story. What a great way for little girls and boys to see a strong female scientist… taking on the wasp. I just thought it was a really great way to frame the story.

Carly: Yeah, and I think the first chapter in Carly’s Adventures in Wasp Land doesn’t actually talk about wasps but it sort of talks about me and how I became an entomologist. That wasn’t part of the original plan, but me and the folks at Ology, and eventually Jim was totally on board with this, felt that it was important that part of the broader impacts, should be showing young girls that they too could be an entomologist, this field that is commonly associated (at least by other people who are outside of entomology) as being male-dominated and being a career for boys… showing them that, that’s not necessarily the case. So, that’s when the strengths of comics especially when it comes to showing girls and underrepresented minority students that they can envision themselves also as scientists. That’s one of the things you can do with comics that I find really engaging… is that, in your choice of narrator, you can make those decisions.

John: I believe you’re releasing some of your materials under an OER license. Is that correct?

Carly: Yes, not Carly’s Adventures in Wasp Land. That is an OER in that it’s freely available, but it’s going to stick with the museum’s website for the time being (as far as I know). What I’m putting on an OER license is actually the comic textbook that I’m going to be eventually making for the Farmingdale State general biology students, but it’s certainly going to be available to any SUNY professor or any professor anywhere.

John: Have you requested an grant for that or are you doing this on your own?

Carly: So, I’ve requested a summer stipend and I should be finding out about that soon. As you might guess drawing and writing comics takes a lot of time, much longer than say a written textbook would take, and there are certainly many professors that are working on written OERs for their class. So, I’ve requested a summer stipend and SUNY Farmingdale has recently announced that there’s going to be an OER incentive grant, so I’ll be applying for that too.

John: Excellent.

Rebecca: You’ve also done some writing about using comics for science specifically, can you talk a little bit about the research that you’ve done in this area?

Carly: Yeah, so I think when I accepted the job at Farmingdale, I knew that I was going to be very, very, interested in making comics and researching the impact of comics… part of the research that I do for my tenure decision… and luckily the faculty here have been very supportive of that. Farmingdale is a primarily undergraduate institution, so there’s actually lots of professors that are also not only researching their scientific field or their artistic field, but are also researching educational techniques in their field. Part of preparing for that was actually some work that I did last year. I was actually invited to an open access issue from the Entomological Society of America, on educational communication in the sciences …and they had known over the years, because I kept presenting on comics, that my interest really lied in the use of comics as outreach education. So, I began actually searching through the literature because this was something I wanted to continue doing as a professor once I moved to Farmingdale, and it was also something that I just wanted to continue just as someone who was going to keep making educational comics regardless. And so what I found in doing this big review paper called “Sequential Science” is that there is much research in how comics impact the interest and attitudes towards the material, at students at a variety of levels, but there isn’t so much research in actually measuring their gains in content knowledge. So there’s lots of research to show that comics makes students at all levels more interested in the material, but not a lot showing and quantifying how much more they’re learning and retaining. So, I think that’s an area that I actually want to put more research into myself… but yeah I spent a lot of time for that paper reading a bunch of other papers about studies that had been conducted.

John: Have you started this research or is this a plan for future research?

Carly:This is definitely a plan for future. So the development of the OER textbook for gen bio is just happening right now, and anecdotally I’ve certainly seen students are more interested, so I do incorporate comics into my slides right now. They’re not my comics necessarily, they’re comics from a lot of different sources like Beatrice the Biologist or Your Wildlife, those are popular webcomics that are biology focused. I also make some drawings for them for the slides as well. In reality, any comic is just a set of sequential images. So, I can draw a set of sequential images that are explaining mitosis and meiosis. My students might not necessarily read those as comics or recognize them as comics, but they’re still comics because they’re telling an ordered set of events. So when I do that, anecdotally, I can tell you that the students are more interested… especially if there’s just been a slide with the textbook image and some complicated information, if I can show them that slide and then be like “oh let me break it down into these steps that I’ve drawn out” it seems to help them. But have I actually started measuring the impacts? No, not yet.

John: So do you have a research plan on that?

Carly: Yeah, so as the OER textbook is going to take some time to make. It’s probably going to take a couple of years to finish in its entirety, but there’s no reason that I can’t start exposing the students to the chapters as I complete them. So, until the OER is finished in its entirety, and given that I usually teach multiple sections of gen bio, I’m going to start setting up testing control groups just looking at small chapters, as I complete them. So, one class will receive the comics, the other class won’t receive the comics, and since both classes have the same test, I can actually see if there’s any improvement. Now, once the comic is finished in its entirety, that’s when I’ll actually begin the full-scale research… and what’s going to happen there is… again I teach multiple sections of gen bio… I can set up a test group and a control group. The test group will get the comic textbook and then the control group would get a traditional OER (probably the OpenStax gen bio textbook) and I can give them the whole textbook at that point and measure what their differences are in terms of performance using their midterms and their quizzes and their homework assignments. But I also plan on surveying them on interest, because although the interest and the attitudes might not seem as strong a topic as actual performance, I think when you’re teaching non major biology students (many of them who feel like they’re just there to check off a box), many of them who have prevailing biases against science… many of them who don’t feel like they can connect to science… I think it’s so important to measure those attitude changes.

Rebecca: Why does a sequential format work so well for a topic like biology? What do you see the benefit of being sequential in that way? This sequential art form.

Carly: So even in general biology, intro biology for non-majors, there’s still lots of processes that are multiple steps. So, I don’t know if either of you remember learning the Krebs cycle or photosynthesis. These are very complicated multi-step processes where something has to happen and then there’s a result… and then another thing happens and then there’s some sort of result. So, there’s plenty of stuff, even in the entry level biology classes, that lend themselves really well to a narrative. Comics really are any progression of images that build a narrative… now, that narrative doesn’t have to be fiction. The point is that there’s an order of events and together that order of events makes sense. You actually don’t have to add words for it to be considered a comic, but obviously the words help in the context of a biology class. I think given that there are so many multi-step processes whether you’re studying the Krebs cycle… or photosynthesis… or mitosis… or meiosis… or even natural selection or ecology… sequential comics… so these images, where you have processes that are laid out in order and broken down into steps, really help intro students.

John: Do you have an anticipated timespan on your textbook project?

Carly: I suspect that it’s going to be this summer. I probably have two and a half months that I’m not actually teaching, but I’ll also be doing research on my scientific stuff (on my wasp studies) at the same time. I suspect that I’ll be able to draft out the first half of the textbook and probably be able to complete about three to four chapters of it. So, I’ll have those chapters ready for the fall semester and then I’ll try to get some work done during the fall semester and keep building that project. I suspect in total it’s going to take me at least two summers and also the semesters between, where I’m actually doing much more work on sort of my regular school requirements to actually finish it.

John: Do you have any people who’ll be working with you on reviewing this and giving feedback?

Carly: Not yet, but I recognize the need for that. I want to have this textbook be one of the contributions that I have for getting tenure. Making a textbook is a common contribution for the tenure package, but to make a textbook you actually have to have some form of peer review if you’re going to go through a publisher. So, when you’re making your own OER and you’re publishing it on your own website, you might lose some of that aspect of peer review. The plan right now is to actually enlist a set of beta readers who are also science educators in their own field and have criticism from them. This isn’t quite the same as having peer review, but I think for now it’s the very best that I can do, but I’m certainly open to suggestion and open to constructive criticism and changing things up. One of the challenges of creating your own OER is that at some point you might lose the more rigorous aspects of submitting a textbook to a standard publishing company.

Rebecca: Will you have an editor working with you for this project?

Carly: Currently no one is lined up, but that’s a valid suggestion, to actually pay an editor… probably someone who works in science textbooks. But, I think before I can even get to that point, I actually have to have a fairly large body of material to show them in the first place.

John: I would think that one thing that would be useful is, once you have this material, adoptions and response from adopters could be used in place of the peer review.

Carly: Oh yes, certainly. And when I put it up on the website there’s definitely going to be a forum for educators to be like “You know what, this didn’t make a lot of sense. Can you change the wording on this?” So, treating this as a living body of work instead of: “oh, I published that, it’s done…” because there’s no cost associated with changing and the material outside of my own time cost because there’s no physical version. So, it actually wouldn’t be all that difficult for me to have those changes be something that’s constantly happening, especially as we find out better ways to teach say homologous chromosomes, or mitosis, or things like that. But even before I launch it, I still want to have beta readers that can give it a read-through even before that, but having the ability for educators to constantly give me comments would be something that’s on the main website.

Rebecca: What software are you using to manage the process?

Carly: The website build itself is through SquareSpace and that is because I have absolutely no training in making a website, whatsoever. So that’s the actual platform that I’m building the website through. In terms of drawing, I start a lot of stuff out by hand and then I usually draw it in Photoshop on a tablet. Certainly, there are times when my tablet is down and I have to draw it by hand, and then scan it… that’s also a possibility. There is something else I’m interested in and this is more of a conversation about OER versus publishers. On the major publishers textbooks right now… so, McGraw-Hill, Pearson, Cengage, stuff like that… I think they’ve recognized that students can get OERs for free, professors can get OERs for free. So, what these publishers are doing now is that they’re offering adaptive learning systems, where you have assignments that get harder or easier as the student does better or worse, where the grades go automatically into the professor’s LMS which (if you’re at a school that doesn’t have grad students) is great because you don’t have a TA to do your grading. The publishers are offering these adaptive learning systems that go seamlessly into your Blackboard or your Angel (or whatever you’re using), but if you’re developing an OER you don’t have that capability. You can make standard multiple choice quizzes on Blackboard and give them to your students, but that’s not the same thing as an adaptive learning system that tracks your students progress. So, I would also be interested in working with someone (or maybe even SUNY at large) to develop platforms that actually make these adaptive learning systems… because then I think they’ll actually be able to convince more professors to adopt OERs.

John: Some of the publishers do have that. I know that Cengage, Pearson, and McGraw-Hill have been putting together packages of OER materials, where they add other resources to them (including some adaptive learning tools) that they release under a fairly inexpensive license. Another option might be to investigate Lumen Learning. Lumen Learning works with OpenStax and they package OER materials with some other materials they’ve created through a variety of grant-funded activities. But that might be worth doing and SUNY does have a contract with Lumen Learning on these things.

Carly: Yeah, I would like to work with someone that is not just SUNY…

John: Right..

Carly: I’m a SUNY professor but I would like people at the University of California system to be able to use my comics.

John: Lumen Learning is not restricted to SUNY.

Carly: OK.

John: SUNY happens to have a contract where they get a discounted price on the bundles when colleges adopt the Lumen Learning platform, but it’s basically a bundling platform that works with OpenStax and other OER materials.

Carly: Yeah, so that’s worth considering, because not only do I want to make the comic, I also want to make assessment tools… so that whenever professors are using my comic they also have a test bank… a way to create these adaptive learning assignments and things like that. So, this is something I’ve talked about before in my presentations at the Entomological Society of America… that you can’t just make a comic and put it out there for educators, you actually have to provide study tools, study guides, teaching plans, teaching lessons, to actually make it useful for educators.
I really like the idea of there being a platform where a professor could create their own test bank and then assign levels and topics to those questions and then just be able to import those into something that is automatically going to make adaptive learning assignments.

John: I don’t think we’ve got that yet, but there are a couple platforms out there: CogBooks and Acrobatiq. Both are do-it-yourself platforms for creating adaptive learning solutions and based on the Carnegie Mellon system,… which they’ve been doing for quite a while there. But it’s a lot of work, and it automates some of the process so you don’t actually have to do the programming, but you still have to work through most of the structure yourself. I noticed that you give students the option of making their own comics for extra credit. Could you tell us about that? how have students responded? and how has that worked?

Carly: Sure, so this has really come out of a desire to actually start generating and using comics in my class while getting the OER ready… because I have people who are asking me “What results do you have already? How have students responded?” And I’m like, “I haven’t finished the comic yet.” So, I’m aware of that and so that’s where incorporating comics into the classroom right now, while I’m preparing, comes from. General biology is a very difficult course for most incoming freshmen (which is the vast majority of the students I have). What it feels like to me is that I give all of my students the benefit of the doubt… I assume that they’re all studying… and when they do poorly on their first test I don’t say to them “Oh, it’s cause you guys didn’t study enough.” I say to them “No one has taught you how to study.” So a lot of my students, when they do poorly on their exams and they come to me during office hours, I ask them how did you study? And inevitably the answer I get is “I reread the PowerPoint notes, I reread the slides,” and so I’m like “No, no that’s not how you study, that’s just reading”. I try to emphasize that studying is the active reorganization and recontextualization of all of the information sources I’m giving you, not just my PowerPoint slides, but the lecture notes your hopefully taking in class, the textbook itself, the homework assignments. There are all these different forms of information that I’m giving you, and what I’m hoping you’re doing is actively reorganizing it. So, we talk about rewriting your notes. We talk about how to actually make flashcards that are effective. We talk about making flowcharts… and really from that last one… making flowcharts… that’s kind of like making a comic already. With the making comics as an extra credit, I’m really just encouraging to do another form of studying, where they have to take all this material for a midterm and they have to draw their own comic. So, usually what I do is I start the first couple of pages for them. So, on my Twitter right now I can actually send you an image of this first page I’ve made to kickstart their own process. So, spring break is coming up and they have a midterm, not the day after spring break that would be cruel, but the Thursday after spring break. That midterm is going to cover mitosis, meiosis, inheritance, and DNA transcription and translation. And these all seem like different topics but in reality they’re all very interconnected topics. You really can’t talk about mitosis until you can talk about alleles, and genes, and Mendelian inheritance and things like that. So I’m trying to encourage the students to conceptualize that these are all interrelated things because I think that I’ll actually help them memorize things better than just treating them as separate slides that they’re just reading through. At the end of next week’s Thursday lecture, the one right before spring break I’m going to introduce this project and hopefully I get some results from it. Previously I had done this at my last teaching position, which was at Sam Houston State University. I was a visiting assistant professor there, and for extra credit, I offered students the opportunity to make a comic on the same set of materials and I get responses… but the problem is that I get responses usually from the students who don’t need the extra credit. I think this is something that’s a common problem with offering extra credit… that inevitably many, many, many times it’s the students that don’t actually need the extra credit that turn in the extra credit assignments. Now. I still enjoy reading them and they still say that it was helpful and it’s a new study technique that they’re going to do, but reaching out to the other students is one of the challenges I’m facing as a young professor.

John: We all do that, it’s not just related to age. I know in my class I give them lots of chances to retake tests… the people who do it the most are the students who are already doing best in class. So, it increases the variance in the outcomes quite a bit when that’s not entirely the goal, you’d like to have everyone rise up but not necessarily spread out further on that continuum.

Rebecca: So, I’m curious with a project like this, do you use the opportunity as being a scientist who also as an artist to sneak in some art teaching as well? Do you use things like Scott McCloud’s Understanding Comics or anything as a tool to help students understand how to put together a comic and the medium of a comic?

Carly: I love Understanding Comics, it was like one of my foundational books when I was an undergraduate taking my first comic drawing class. I actually tried to avoid situations like that because I don’t want to discourage the students that feel like they don’t know how to draw. Which is a silly thing because everyone can draw… drawing well is a different thing. I don’t want them to get hung up on how good their drawings are. I want them to get hung up on how much conceptual sense that it makes. So certainly Scott McCloud talks about this, about how you can still have a comic that’s just stick figures. And so for me, I don’t want them to freak out about the fact that I’m an artist, and that I’m pretty decent at drawing, and that I expect them also to be pretty decent at drawing. But the funny thing about teaching non-majors is that inevitably some of them are art majors. So, that’s that’s always fun, they’re always surprised to find out when they come to my office and they see that I have paintings that I made as an undergraduate up on my walls and things like that. I would love to refer classic comic making literature, but it’s just something that I don’t have enough time when I’m just spending five minutes to introduce something. But, certainly… the students that come to office hours… we do talk about you know what makes a comic because I also have students that read a lot of comics. I have lots of students that are going on the Manga reading websites and a lot of students that talk about superhero comics with me when they find out that I like comics. So it does come up, but it’s usually not something I have time to make part of my already jam-packed lecture.

John: Students often have this perspective that they’re either creative or they’re good at quantitative skills in STEM fields, and it’s really nice that you’re modeling the possibility that you can be both.. That they’re not mutually exclusive.

Rebecca: That’s also why I like McCloud as a reference book too, because it’s not really about fine art in the traditional sense but rather about how to tell a story. Which is interesting and helpful and doesn’t really necessarily emphasize being able to draw.

Carly: Yeah, I think he has that… what does an expression look like, and it’s just like two dots for eyes, and then eyebrows, and then a line for a mouth, and you can get the full range of human emotion. And then I show students comics like XKCD, that is just stick figures and it’s really effective so, yeah. I try to avoid things where they feel like they have to be a professional artist, not to say that’s what McCloud does, you just pointed out that it doesn’t do that. But I try to focus more on the conceptual – like how does this help you study, you’re not just making this to impress me. And you get that a lot with extra credits, sometimes you feel like students are just doing those projects to get extra credit. Instead I’m trying to be like “Mo, no this is a study tool. This benefits you.”

Rebecca: Have you had any students follow in your footsteps and develop a love of both art and science and pursued you as a mentor?

Carly: At Sam Houston State, I certainly had students that like to come and chat with me and sort of explore those topics. But unfortunately, I had to leave there to start the position I have at Farmingdale, and unfortunately I just haven’t been here long enough to build those connections. One of the things I want to do, as I’m at Farmingdale a bit longer, and I get settled in, is actually propose a biological illustration class. So we have the ability as biology faculty to offer these topics in biology courses, and one of the ones I really want to do is biological illustration… especially since we share our building with art, or rather… I think it’s design communications… whatever the technical college…

Rebecca: Communications design… probably.

Carly: Yeah… but they’re still students taking drawing and watercolor and painting so..

Rebecca: How cool. That would be so fun.

Carly: Yeah, and you know what I actually kind of taught that course at Berkeley. When I was an undergrad at Berkeley, there was this thing where students could actually teach one-credit non-graded courses. So, I actually offered a biological illustration course. Sort of one of those things to build my resume and make up for my not-so-great GPA, but I actually really loved doing it and it seemed like as long as you can get some specimens, and you can sit down, and you have a studio space, you can come up with some amazing work, and luckily I’m still a research associate at the Museum of Natural History, so hopefully they’ll let me borrow some animal mounts. But there’s also insects. Insects are great… they’re cheap and I’m also the entomology professor so it could just become entomological illustration and then of course Farmingdale also has a huge Horticulture Department and botanical illustration has always historically… much like art has been a big part of biology… art has been a big part of botany for a long time. So I think we have the ability to do this, and that there would be interest, and it’d be a cool collaboration with these two departments that are both in Hale Hall.

John: How have your colleagues responded?

Carly: I would say positively… extremely positively. I’ve been thinking more about transitioning into… not fully being a pedagogy researcher… but having it be a large part of what I do on the research side. So, I still plan on doing my usual wasp entomology taxonomy research, but I also want to do a lot of research that’s in comics and the use of comics. That was something that came up in my interview and I think it overall was a helpful thing, and even while I’ve been here I’ve talked about it a lot with my chair and she’s been extremely supportive, and my other colleagues have also been supportive. I haven’t received any negative pushback… which I think was something that I was expecting… because when you look at the literature about educators… whether they or not they want to use comics, there’s this fear… that comics have this bias against them. And so a lot of educators at the primary and secondary levels are kind of afraid of assigning them, and they’re afraid they’re going to be looked down upon by parents and by other educators. But I’ve been extremely fortunate, and I have faced none of that and largely the faculty have been very supportive.

Rebecca: I wonder if some people maybe perceive comics as just being not very rigorous. Which is crazy,because you can provide so much more information… because there’s a visual element as well as a text element. So they might actually be more rigorous.

Carly: Yeah. We talk about lack of rigor and lack of detail in textbooks anyways. If you look at a non major biology textbook it’s obviously not going to be as detailed as a major’s introductory biology textbook, and there’s a reason for that. You’re not teaching people who are going to continue in biology for the most part, so there’s less detail. But, still people harp on the lack of information and the lack of rigor. So, I feel like that’s going to be an argument that comes up no matter what assigned reading you’re going to use. Certainly with comics there’s another bias and that there’s a bunch of superhero comics… but comics are actually a lot more diverse these days.

Rebecca: Comics are probably a really great way to help students understand those basic concepts so that they can build their mental model because they probably come with all sorts of assumptions and things that are not correct, and I could see how demonstrating visually could help overcome some of that.

Carly: Yeah, certainly, and for me it really comes down to what is the point of general biology? What am I aiming to do? I still want my students to learn about photosynthesis, and the Krebs cycle, and mitosis, and meiosis. But I also want them to come away with an appreciation and a sense that they are able to understand it. I want them to walk away from the class with positive feelings towards science and not just- it’s a collection of facts I had to memorize.

John: I wish I had had a class like this when I was in college. it seems like a fascinating way of addressing this

Rebecca: It goes back to what we were talking about earlier. I didn’t know I wanted to know about wasps, but maybe I want to know more now after reading your comic.

Carly: Yeah. So these are all like my lofty aspirations as an educator, but I’m pretty sure I’m still making common mistakes and it’s still a bunch of facts that they have to memorize, sometimes. But I feel an awareness of these of these issues is helping and hopefully I only get better at that process.

John: …and there’s nothing wrong with it being fun for them to learn those facts. ..

Carly: Yeah.

John: … they do need to learn facts but there’s nothing saying it can’t be engaging.

Rebecca: Well, providing those sequences might make it easier to remember, because you have a clearer understanding of how the things connect. The visual representation can help provide those connections that words don’t always help because it’s too abstract.

Carly: I think with biology, especially at the introductory level, especially when you’re a professor that doesn’t have graduate student instructors or TAs, you don’t have a lot of time. So we always talk about wanting to have critical thinking questions and essays, but inevitably just because of time constrictions it does largely become scantron multiple-choice questions, and in that way it does become a lot of memorization. Now I still think that memorization is valid. I still think it’s important to know the steps and the processes and be able to call up that knowledge. But for me, the struggle is making that memorization easier. And if comics make that easier then I’m accomplishing my goal…

John: One of the things that really impressed me, though just following you on Twitter recently since I saw your work, is how engaged you are in the scholarship of learning and teaching in your discipline. It’s nice to see people starting their careers doing that. What got you interested in doing research on teaching and learning?

Carly: I think it actually comes down to who professors are. Professors tend to have PhDs, and in my case, I didn’t take any classes about how to teach. So I think most of us are just kind of thrown into this process and we learned slowly along the way. I was like “Well, there’s a whole body of research out there…” and I started reading some papers about how to be a more effective teacher. We have our own center here for teaching that has workshops and stuff like that, and I think recognizing my lack of formal training, I have no teachers certification or anything like that, made me more interested

Carly: I’ve got the list of questions.

Rebecca: One of the things that I think is kind of interesting is the way that we started this whole conversation… and it ties nicely back to the scholarship of teaching and learning… is that your first research position was doing illustrations. And I think that in academia, we don’t often see those sorts of actions as being research. So I really love that that role was called researcher and brings all this sort of together. It doesn’t have to be traditional to be effective or useful.

Carly: Now certainly that first position as an illustrator in the Padian lab… I still wanted to do traditional types of research, but that experience (as someone who is already sort of hanging around on the graduate student level and hanging around the research labs) made me a person that was visible in a crowd of something like 2,000 undergraduate biology students. So from the Padian lab, I was actually able to transition into a more traditional research role that actually led me to parasitism, to studying parasitism, and that was in the Wake lab with Marvalee Wake, who is one of my most important mentors as an undergrad. But yes, my first research position I was called an undergraduate researcher was actually just doing illustration. And I learned a lot about vertebrate anatomy because that was what Katie Brakora studied.

Rebecca: People don’t realize that when you’re doing that kind of illustration work, what kind of attention to detail you need to pay, and how much you can actually learn by just looking at something very carefully.

Carly: Oh yeah, being able to measure something… getting proportions down correctly. There’s a lot of math that goes into biological illustration and serve a lot of rigor. And then you just spend hours stippling, and that was my life.

Carly: Yeah, I would just say if this sounds like something that a faculty member is listening to this podcast and they’re like, “Ah I want to either start making comics or I want to incorporate comics even into a STEM class, I have lots of resources and I can sort of talk ad nauseam about that. You know like, “What are some good comics if you’re teaching biochemistry? What are some good comics if you’re teaching literature?” So certainly if there’s anyone who’s interested in either making comics or choosing comics for their classroom, I’d be happy to talk to folks.
I think unless you’re a comic book reader you probably don’t realize just how much comics have grown outside of what you might have imagined they were twenty years ago, and you’d be surprised by the amount of some relatable materials… especially in the social studies classes… especially in history, there’s a lot of memoirs… a lot of historical memoirs right now in comics.

John: Actually right now, I can think of at least a couple of examples in economics of comic book series that were created for instructional purposes. The Federal Reserve Bank of New York created a series of comic books to help provide middle- and high-school students with information about the monetary system and the role of the Federal Reserve Board; the other, a series of comic books created featuring Captain Euro… this was originally created to provide support for the introduction of the Euro and for the European Union in general.

Carly: Medicine has really moved with this, especially when they’re thinking about “How do we make information that transcends language barriers?” I follow a Twitter that is just medical graphics and there are conferences on medical comics as well. So I think that’s a field that’s really sort of latched on to making comics as a way to share information with patients, and there’s actually been some research showing that it’s more effective.

Rebecca: It’s used a lot in areas where there might be outreach for really low income or people in poverty who need important information about health or resources and things, and that’s where literacy might be an issue, and so sequential images are often used in those contexts as well. When I was doing a project in India, I discovered all of these really interesting graphics that were used… sequential graphics… to get people to do all sorts of things because there’s so many different languages… to kind of overcome that barrier. It was really interesting.
So we usually wrap up our interviews with the question of what are you gonna do next, you’ve already talked about a number of things that are on the horizon, but is there anything specific you want to share as your your next step, whatever it is that you want to research or do?

Carly: Yeah, so we’ve talked a lot about comics, but I can tell you a couple of other things that are on the horizon for me. My field season… the actual going out and studying wasps that I do that’s going to start up in the summer… and I’m hopefully going to bring an undergraduate or two with me, and then hopefully bring that undergraduate to present at the Entomological Society of America. So, that’s sort of the science side of my life, but sort of the swing back I’ve been talking a lot at the Entomological Society of America about using comics in entomology research… and sort of more in line with what you guys do generally, my next thing is actually proposing a symposium on education for undergraduates. Since most entomologists that are at a university don’t just teach entomology, we also generally teach any biology courses. So, kind of swinging more strictly into undergraduate education instead of the broader community outreach education that I’ve been doing with comics outside of academia. So, that’s exactly next on the horizon for me outside of just keeping working on comics.

Rebecca: So, where do your wasps take you this summer?

Carly: They’re going to take me hopefully to Puerto Rico for about a week, down to Florida for probably a week or two, and also local collecting. There hasn’t been a lot done around the Northeast, so going out to the Pine Barrens on Long Island and then probably making it up as far up as you guys and things like that and further up and down the East Coast.

John: Well if you do get up here let us know

Rebecca: Yeah.

John: … and we’ll take you out to lunch or dinner.

Carly: Oh, thank you. Yeah. This has been great guys, thanks for having me and inviting me to this.

Rebecca: Yeah, thanks for sharing all that you’re doing it sounds really exciting, I can’t wait to see it all happen.

John: It’s great to have you here, and you’re doing some wonderful work.

Rebecca: And you have two fans here and two advocates here.

Carly: Oh thank you, that’s important. I want to like tour all of the centers for teaching and learning excellence, however it’s called at every university, and you know be like “Comics, comics, comics, comics!”

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.