30. Adaptive Learning

Posted in active learning, adaptive learning, assessment, higher education, learning, OER, online learning, pedagogy, teaching
May 23, 2018

Do your students arrive in your classes with diverse educational backgrounds? Does a one-size-fits-all instructional strategy leave some students struggling and others bored? Charles Dziuban joins us in this episode to discuss how adaptive learning systems can help provide all of our students with a personalized educational path that is based on their own individual needs.

Show Notes

In order of appearance:

Transcript

John: Do your students arrive in your classes with diverse educational needs? Does a one-size-fits-all instructional strategy leave some students struggling and others bored? In this episode, we examine how adaptive learning systems can help provide all of your students with a personalized educational path that is based on their own individual needs.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Charles Dziuban. Chuck is the Director of the Research Initiative for Teaching Effectiveness at the University of Central Florida, where he has been a faculty member since 1970 teaching research design and statistics. He’s also a founding Director of the University’s Faculty Center for Teaching and Learning. Welcome, Chuck.

Chuck: Oh, thank you both. I’m just so pleased to be here. The good feelings of Oswego continue… you know my growing up upstate New York and my attending Oswego. I’m really honored to do this… makes me feel good.

Rebecca: So glad to have you.

John: We’re very glad to have you.

Rebecca: Today our teas are:

Chuck: I have water from the water fountain.

John: My tea is Yorkshire Gold.

Rebecca: I have Lady Grey today.

John: We’ve invited you to join us to talk about adaptive learning. So, what is adaptive learning? Is it something new?

Chuck: Sure I’d be happy to talk about that. Let me answer that question in reverse order. It’s anything but new. I think adaptive learning has been around since we’re thinking about teaching and learning. I think if we went back to the Middle Ages we have an idea of this notion of adaptive learning. It is not new. In fact, where I came from, and what I’m in my career, I’m looking at a paper written by John Carroll called “A Model of School Learning…” where John laid out this set of equations that basically answers this kind of a proposition: if how much time a student spends learning is constant, what they will learn will be the variable; if what they learn is the constant, how much time they spend in the educational enterprise will be the variable. So that’s the basic sort of notion of that. When we define that, it kind of makes sense. If we give students a limited amount of time and anything… a course… a university… a semester… whatever you call it. It makes sense that what they acquire in that activity will be the variable. If what they learn is a constant, how long they’re going to spend in this enterprise will be the variable and we’ve all experienced that, but we’ve all butted our heads up against something for years and years and years until we finally understood it… and this is the basic notion of adaptive learning…. whatever time it takes, it takes for you to learn something… and he went further to break that down. When we say time, it’s just not how much time in the Augustan sense, it’s how much time do I spend actually in the learning enterprise and what is your perseverance… and what is your aptitude for this… so kind of modified for that. So, that makes eminent sense. That’s what adaptive learning is… basically it’s going to take whatever time you need as a student to learn this material to some sort of satisfaction level. That’s what it is. It’s a very simple concept.

John: …because students come in with very different backgrounds. Some have very rich backgrounds in some areas and weaker in others… and that’s going to vary quite a bit depending on what they’ve learned in prior courses or in their life experiences.

Chuck: Oh, absolutely. I mean I can remember my own experience walking in new courses at Oswego being virtually lost because I had no background and no experience in the course and having to really ramp up my energy and effort and use of time in order to: 1. reach the baseline in the course and go on in other courses. In adaptive learning we have students who enter courses who… don’t take this the wrong way… but fundamentally don’t need to be there. They’ve mastered the material… they can move on… and I’ll give you examples of that later. So, this gives a great deal of flexibility to the learning enterprise.

Rebecca: From the learning perspective, it sounds really wonderful. From a teaching perspective, it seems like it can be a little bit challenging – especially before some of these technologies were available.

Chuck: Nightmare. Yeah, it is. Of course, the way we’ve conceptualized this whole notion of how students learn, it really does… We’ve organized learning around this entity called the class… it’s a unit… yeah, and we call it a class… and if you think about it in terms of a course sequence, say in mathematics, the students who don’t pass their placement exam so they take intermediate algebra, and college algebra, and trig, then precalculus. You can conceive of that as these discrete units, but if you think about it, what we’re doing in our math department… we’re designing that as a series of skills across those things we’ve called classes and that makes the whole notion of teaching quite different. Really what you have to do at adaptive learning… if you begin to plan a course you really have to look at what you have taught, what the granularity of the course is, and your role as an instructor in all of that. It’s very very different and it really forces our instructors to think about what it is I’m teaching… it is much more than a syllabus. The granularity of what you teach becomes critically important.

Rebecca: How did UCF get involved in adaptive learning?

Chuck: It’s an interesting question. That was organic. You may or may not know that UCF is the fastest growing university in the country. It is one of the largest universities in the country. The good news is we’re growing; the bad news is we’re growing. [LAUGHTER] We cannot possibly build an infrastructure to house our students. So, besides adaptive learning, we have online courses, we have blended courses, we have lecture capture courses, and adaptive learning is a natural outgrowth of that sort of structure. 43% of our full-time credit hours are produced online so we’re almost a half online university… and one of the things is we became involved in this simply because one of the things that John said. Our students are very diverse. They come with very different backgrounds and even putting them in this thing called a course may not make sense for some of them in terms of their pre assessment of where they are… and the notion of adaptive learning pre-assessing them and then putting them in the proper learning sequence within a set of objectives is why we got involved in adaptive learning… to doing this… and what we did is we simply recruited our best faculty, say “Look, see if you can do this. See if you can, then kind of accommodate what students are, where they are, and how they’re learning, in a much more flexible environment. It just simply make sense for students… and given the fact that we have a diverse background…. students coming from underrepresented populations… it just makes sense at so many levels.

John: One case where we’ve done that institutionally here at Oswego is, beginning about three or four years ago, we adopted Aleks for the math requirement. It’s used for initial placement and then it’s also used for students to get up to the level needed to meet basic course requirements. So, students come in and then if they don’t meet the requirements for the courses they’d like to take, they can spend the summer working on that to move up to the level… and then test themselves again to basically get to the level they need to be to make good progress… and it’s been working very well from everything I’ve heard, but we haven’t done much more yet at an institutional level.

Chuck: Yeah, well there are, as you probably know, with technologies like this there’s a probably a new platform coming out every day. I mean this as a metaphor. Our math department uses Aleks and if you go into an Aleks class… and Aleks is a pre-designed course, there is some flexibility but it’s a course that’s pre-designed. These platforms come in basically two varieties: where courses are embedded in them like stuff with McGraw-Hill or Knewton or something like this, or content agnostic like Realizeit where you have to build the course. That’s two kinds of things. We have an Aleks course in college algebra, and if you look at it, it kind of looks like chaos. Students are coming in, there are people lecturing on the boards, there are student assistants helping them, they’re on the thing. When they master it, they master it and they leave. So, any concept of a lecture course is completely out the window, but what’s happening is: students are being assessed, they’re being reassigned to different areas within the Aleks platform to master the skills, and then moving on. So, some of them can complete a course in half a semester… and I assume you’re experienced at seeing this at Oswego, but one of the things we’re facing is where with adaptive learning, we’re trying to get over the teach then test model for a course. We’re trying to embed the learning assessment within the adaptive learning platform. So, we have a test-free course where essentially assessment becomes part of the learning platform assessment… actually becomes part of the teaching… and it’s really quite exciting, but it is very daunting to do. We’re kind of wedded to tests in our environment.

John: I believe that you’ve been working with Colorado Technical University.

Chuck: Yeah!

John: How did that partnership come about?

Chuck: Organically.

[LAUGHTER]

When we came to realize that adaptive learning may offer some advantages for us, we asked several vendors to come in and make presentations to faculty… and this was basically a faculty decision. They saw several platforms and they said “Realizeit is the one. This is the one… because we want the flexibility to build our own courses.” But, of course, there’s a Chinese proverb: “Be careful what you ask for….” and they got Realizeit… but now they had to build the courses. That’s daunting when you start building your courses. Realizeit’s been very helpful. They have a process called ingestion where they’ll try to take whatever you have in a course… and I’ll say this and I say this with love and kindness… all of these platforms have a bit of clunk associated with them. They all work, but they are all some problems associated with them and problems in quotes. I’ll say challenges, or in Provost speak, they have opportunities associated with them. [LAUGHTER] …and we work with them… but in doing this kind of thing, but we began presenting this with our partner (and I’ll talk about our research relationship with RealizeIt in a little bit)… but working with RealizeIt in some research that we began presenting and Colorado Technical University has been doing something that we’re not able to do yet, that is scale it. Scaling it is an issue. What they have done is, they’re a private for-profit institution, and they’ve scaled it at a remarkable kind of thing and thrown a lot of resources at it. So, we’re very good at research, they’re very good at scaling, and Realizeit is very good at research. So, when we saw them presenting and they heard us presenting, it was sort of a speed date. They said “hey, you’re doing pretty cool stuff” and we said to them “hey, you’re doing pretty cool stuff, you want to play?” …and that’s how it started… working with them and it’s led to several publications and several presentations, and then with RealizeIt looped into this, working in the background, we’ve been able to do this kind of thing where we think partnerships are really important in studying this… and of course I’m gonna recruit Oswego today. You should be joining us.

John: I’ve been looking at adaptive learning platforms for a few years.

Chuck: Yeah.

John: I’ve worked with some from the publishers but it would really be nice to build something from the ground up. So, we’ll talk about that perhaps a little bit more later.

Chuck: Alright. Well, I will say this. Our philosophy is we give away everything. You can have anything we have. We’ve published several papers. We have several projects underway now with adaptive learning. Any of your audience, and certainly my brothers and sisters at Oswego can have anything UCF has done. The idea is we need to do it for partnership. I did a presentation on, if you go to Google and search “the Grand Cafe.” The Grand Cafe was the first coffee house in England, and what it was was a result when the British discovered coffee houses ideas really blossomed. The Brits didn’t drink water for a long time because they were afraid of the water, and so for a couple of hundred years they basically started having a beer for breakfast.

John: Right.

Chuck: A couple of beers at mid break, and then a beer and a gin. They spent a couple hundred years drunk.

John: ….and so did the founding fathers.

Chuck: Yeah, absolutely. But when they discovered coffee houses… a coffee and tea (which you’re drinking)… they were in this space they were in this learning space and when you switch from a depressant to a stimulant there are many more ideas and that’s the grand Enlightenment… but the point being that it was a partnership. When we talk about these grand ideas they are not Eureka moments, they are not lightbulb moments, they are people working together creating ideas and, if you will, letting ideas have sex… in the idea that these ideas can grow and be developed and that’s what happened…. and that’s what happened with our partnership and now we’ve added another partner. We’ve added a Petroleum Geo-Services in Oslo, Norway. It’s an exploration company looking for oil all through the world, and suddenly they have to train these people aboard ship and they looked at this and said: “Whoa, it costs us a fortune to bring these geophysicists back to Oslo to train them. I wonder if we can do it aboard ship with adaptive learning.” So, we’re looking at that. So, there are a lot of possibilities and I’ll say we worried about the students adapting to adaptive learning. No problem. Of course, it’s embedded in our LMS so they don’t even know they’re not in Canvas. They still think they’re in Canvas.

Rebecca: So, you alluded a little bit to Realizeit’s potential, because you mentioned that it’s one of the few platforms where you can build courses from the ground up. Can you expand a little bit more about its strengths relative to some other platforms and maybe also some of the stumbling blocks associated with building something from the ground up?

Chuck: I’m not sure I know the characteristics of all the platforms. I know that Realizeit is content agnostic, and I’m sure there are other platforms that allow you to do that… and probably there are other platforms available too, as you begin to explore this where courses are assembled but they have some flexibility for faculty members to take components in and out. I would think that would be a real advantage. It’s sort of the continuum… building of course from ground up is daunting and one of the problems is the granularity of the course. Getting this level of granularity so the course flows evenly…. it’s always a rough start in terms of doing that and we’ve provided as much support we can for faculty.

John: How can faculty get started using adaptive learning? Is this something that’s best done at an individual level, or at a departmental or an institutional level?

Chuck: It’s a big task and whatever support you could provide… especially instructional design support… in terms of looking at a course, redesigning it, and putting it together, and that’s very important… and run-throughs are very important, test beds for courses before you roll them out. I, in my role having to evaluate our technology, I taught online, blended, lecture capture, adaptive, and each time I rolled it out… it’s been terrible. It has a lot of bumps along the road and our working two years with adaptive learning…. they’ve had some issues and by our work with them for two years, we have helped them clean up a lot of issues… and they would be the first to admit that. They help out on the research side of this. They’re very very strong. They can do research things we can’t. We can do research things they can’t do. It’s the perfect yin and yang… and this is a wonderful kind of relationship to have…. and Michael Feldstein and Phil Hill and the MindWires e-literate have built this project now they call the Empirical Educator where they’re trying to get vendors and academics together to work together… to begin to look at this.

John: We will include links to the articles that you’ve mentioned in the show notes.

Chuck: Absolutely. Okay.

John: The question I had about that (in terms of individual development or other forms of development) was basically whether people are developing their own variants of the course or is it done at the departmental level? Or the program level?

Chuck: Both.

John: Okay.

Chuck: How do we begin an adaptive learning? One of the things you have to do to demonstrate to faculty that it can be successful. So what we have done is we cherry-picked our best faculty, the kind of faculty that if you show them this platform, they’d say, “hey, we’d like to try this.” So our initial pilot study was done in Psychology, was done in College Algebra and Pathophysiology. And all of these faculty are the ones who would say, “I would try to do this anyway,” and we built it that way and then began demonstrating it with individual faculty. And now we have a pilot project in our College of Business to make as many courses as we can adaptive learning. So it’s now up to the college level and we are now making presentations to all of the individual departments saying, “look, this is the possibility— is this a strategic initiative for your department? Do you think this would be something that would be a value to a department? A value add to your department versus one-off courses?” Because, for instance, in Psychology, the Psychology professor developed it over three semesters and faculty really do have the option of how much they can lock down this course. You can lock down an adaptive learning course so it looks like a regular course or you could make it go completely adaptive. It’s pretty scary stuff and what Jeff did in Psychology is his third go-around, he said, “it’s adaptive, go!” That’s all there was to it. They could go through at any pace they wanted to. In a cohort of say 20% finished the course in three weeks and that sounds like heresy, okay. But they did it and they were finished. Their next thing they said, “okay, we want Psychology too, we’re ready for it” and wasn’t ready. Okay, it wasn’t ready. So what do you do with them— your students who have completed a course in three weeks. They’re done, they’re verified, they’re certified that they’ve completed the course and they have nowhere to go. So it has great consequences for building this out in terms of doing this and we work very hard. Intermediate Algebra (they didn’t make it into College Algebra) so Tammy Muse, who will be featured on 60 Minutes in a couple of weeks, has allowed them to start Intermediate Algebra when they complete [it], go directly to the Adaptive Learning College Algebra course and that cohort now is finishing the course on time and it’s really quite amazing. And what’s the objective is, well, make that cohort larger. Just the things that you were saying about Aleks, (that’s a bad word in Florida, “remediate”), but let them acquire the skills they don’t have and then go directly to the course they need. And what Realizeit will do is sequencing them back and Tammy has done an amazing thing. She has made all of her assessment items reflect the diversity of our campus with names and the diversity of the disciplines she made them have the disciplines reflect whatever their major is, whether it be Engineering or Physics or whatever. And though the problem sets “they work” all reflect those kinds of things. A lot of work, but it’s working beautifully.

Rebecca: Sounds really exciting.

Chuck: Well, on certain days, it is.

Rebecca: [Laughter] Exciting can be both scary and…

Chuck: Oh yeah, yeah yeah, yeah.

Rebecca: We talked a little bit about whether or not the platform allows for interleaved practice. How does it work, what does that look like? Can you describe one of these courses?

Chuck: As best I can.

Rebecca: Yeah.

Chuck: It looks like you’re in an LMS (Learning Management System) and you have exercises. RealizeIt has a decision engine built in and I know a good deal about the decision engine. I don’t know everything about it because these platforms are proprietary. These vendors do not like to give away their trade secrets but Realizeit is Bayesian-based. It gets prior knowledge assessment of a student, then based on that prior knowledge, it assigns a student to a location in a course and begins and then begins to assess them. It looks like an LMS, the learning can be anything from videos, to simulations, to reading, to practice exercises, to discussion boards— it can be any format that you would normally have in any particular course. Students can participate in any number of ways they are assessed and then based on the assessment, which can be (I’ll talk about that in a second), can be anything. It can be a simulation, it can be a practice exercise, it can be a performance— it can be anything. It evaluates them and based on the evaluation, it reassesses their learning path and then their sequence back into that and they can have any number of ways to go through the learning factor. So this platform is always thinking about them but it is re-assigning them and it is constantly re-assigning them to learning passes that goes all the way through. It’s kind of scary to look at in terms of what’s happening and in it then, it has for them, “you’ve mastered these, you have not mastered these kinds of things,” I wish I can tell you it works as smoothly as I just described it…

John: [LAUGHTER] But it’s getting there.

Chuck: It’s getting there and what we tend to do is overestimate the short-term impact of these things and underestimate the long-term impact— that’s something we do with all technologies. If you look at the latest MIT 10 technologies that are gonna revolutionize them, one of them is Babel Fish earbuds that instantly translate languages for you. Well, they can’t do that exactly yet, they will eventually maybe get very good at that, but that’s what it looks like. It looks like a regular LMS and if you experience it, you have this sort of seamless feeling that you’re moving through this with no real impediments to this so you kind of go through your own pace. And what it also does it learns how you best learn. If you do it best with reading, it’ll do it that way. If your best with simulations, it’ll steer you up to simulations, but it’ll steer you to them. But faculty have to prepare all of those things so “therein lies the rub” as a friend of ours in the round theatres in England would have said.

John: I looked at the Acrobatiq platform a couple years ago, I met with some of the representatives from the gardening. I haven’t looked at Realizeit very much yet, but these platforms are really good at giving students lots of retrieval practice and assessing where they have weaknesses and doing that type of [a]daptation. But one of the things we know is that interleague practice, as Rebecca mentioned, is really helpful in increasing recall when you ask students to go back and test them on things that they learned earlier in the course is really helpful and encouraging deeper long-term learning. And I asked about Acrobatiq’s ability to do that and there wasn’t any and that’s why we were a little bit curious about whether perhaps Realizeit had that ability to go back and bring in questions earlier. I know Aleks does that a little bit but most of the platform’s I’ve looked at so far haven’t.

Chuck: Well with Realizeit, students have that option. I guess if we could make interleave a verb, they had the option to interleave, I’ll give you an example of that. Pathophysiology— in the state of Florida there’s a requirement, now, most hospitals require their RN’s to become BSN’s and that’s causing some angst. Nursing is stressful enough but now nurses are given a certain timeframe to achieve their BSN’s so nurses are coming back, some of them unhappily. RNs to get their BSN’s and you’re doing it in adaptive learning platforms and online platforms in the Pathophysiology was adaptive so Julie Hinkle, who taught the course, said, “I’ve got 30-year RN’s who worked in Cardiology their whole career coming into my Pathophysiology course, taking the Cardio unit.” Well, they know more about cardio than I ever will and so within the adaptive learning platform, they simply go to the Cardio unit and test out. They’re done, ok, because they know everything but nurses are funny. If they don’t get quote “a hundred,” they’re not satisfied. So that’s their interleaving, they will go back and test themselves again and again and again and again until they get the satisfaction. High pressure, high pressure in that field. It’s not like beginning Psychology— if I pass, I pass in terms of this. And what Julia’s done is created incredible adaptive learning measurement devices. She’ll give them a series of bloodwork in blood gases that they all have to look at. I mean it’s hard and they all get different values and they all have to assess this patient and the protocol for this patient based on different values associated with and then she’s got them in discussion boards and if the values don’t make sense, the nurses, because of who they are, will go back until they get it. They had the option to do that. In this case that even you talked about her student-driven, you don’t have to force them— yeah, they have the option to do it, some will, some won’t.

Rebecca: Have you been using any open education resources as part of the content for the adaptive courses or are these all closed system faculty you’ve created? Materials…

Chuck: Our faculty, when they begin looking at it, you had this sort of adoption curve, we got the early adopters, who will do anything. When we look at our online courses— we’re very good at online teaching, we are very very good at it. However, we probably have 2,000 courses in our vault and faculty will ask me, “will I be any good teaching online?” Well, my response is, “are you any good face-to-face and if you’re not, you’re not gonna be very good online.” But we range from faculty who do things that are very text-based, faculty members who will not stop putting bells and whistles in their courses. And we say to them, “stop it, you’re being annoying. Stop with the gizmos in the course.” Somewhere, there’s a balance in this kind of stuff and obviously what you have for this preparation of course is a lot of work. I don’t know about you and Oswego, I assume everybody teaches one course and has a lot of free time [LAUGHTER]. We’re not that way at UCF. Our courses are very large, heavy teaching loads and right away, faculty say, “this is the too daunting.” So what I realize is doing in a lot of platforms are doing that are like us. We’re looking at OER (Open Educational Resources) and saying, “what of this can we ingest into this and make it available for a course.” In terms of, “can we take some of the right stuff, put it in there, and load it up so it’s ready to go” and then you can adjust it as we go along. Obviously those courses are good and if we can do that for faculty, it’s a great service because building a course ground up is daunting no matter what happened. And unless your institution provide faculty support, it’s probably too daunting for an individual faculty member to do this by themselves in my estimation.

John: Does Realizeit provide any package materials to help get people started, for example, including OER.

Chuck: Yeah they do, they provides a great deal of resources for it. They have said representatives down when things were not working as well as we’d hope. Basically all of these vendors and I’ll say they have the Veg-O-Matic, we have the platform for them to try it and it doesn’t chop vegetables equally well. That’s the partnership that’s so critical and there’s vendors are not just vendors anymore. They have to be active partners in this educational enterprise. They’re not just selling us stuff— they have to come and help us and Realizeit is very good about that. And others are, too. I’m not pitching Realizeit by any stretch of the imagination.

John: But it sounds like it’s worked pretty well for you guys.

Chuck: Yeah, it’s worked pretty well for us. Our issues are, “how do we scale it, how do we scale it?”

John: What sort of resistance has ever been from faculty? Is it mostly just a time issue or there are other issues that faculty are concerned about?

Chuck: I can probably give you a less of them. One: it’s a lot of work. For all faculty, I think one of the questions are, I’m not sure of the culture and Oswego, but at UCF, it still remains get-ahead equals teaching service and research and we know what of those three carries the great weight.

John: Yes, that’s an issue that’s discussed all the time.

Chuck: So if I’m gonna spend all of my time in preparation of this kind of enterprise, where’s the reward system within the culture of the University for this? And I don’t mean to be crass but what’s in it for me? Why should I do this? And we have to provide some sort of reward perfectly. We’ve done a lot, we have sowed awards that rewards faculty thing, but we tried by giving them course releases and all of the help we can have. We have a large staff that support faculty in the online environment and we have a bank of instructional designers that help faculty members load up their adaptive learning course. And basically our philosophy is if faculty can feel better about their teaching, they’ll follow you anywhere. I mean that’s yet— you know we all want to be the best teachers we can and it feels so good when it goes well and it feels so awful when it goes poorly and we’ve all had that experience.

John: Oh yeah.

Chuck: I am personally— I’m very fond of the quotes attributed to Augustine. I thought I understood it until I tried to teach it. Not one of us has not experienced that at one time in our position. [LAUGHTER] I’ve been talking about things that I’ve had no business talking about. I knew them, but I didn’t know how to teach them.

John: But you learn that as you go. [LAUGHTER]

Chuck: Yeah, oh yeah yeah, yeah.

John: In your recent EDUCAUSE paper on Adaptive Learning, you describe different types of student interaction with a platform. Could you tell us a little bit about that? I remember tortoise and hares and a few other animals in there. [LAUGHTER]

Chuck: Yes tortoise, hares…

Rebecca: Frogs and kangaroos.

Chuck: Frogs and kangaroos [LAUGHTER]— I’ve got to tell you, Realizeit is headquartered in Dublin, Ireland so one, it’s fun to listen to them talk. And two, they’re very bright so in doing this, what we decided, and then we’ve done this a lot, we had a research partnership and you’ve probably known this. If you begin presenting some of this stuff in tabular form and tables, your audience glazes over. In about two minutes, they stopped listening. So the idea there is, “how can we begin to portray this data in a way that’s more engaging to audiences?” And working with Realizeit and Colm Howlin, who was their Director of Research— is brilliant. We’re both big fans of Hans Rosling in Sweden, passed recently, but his approach to showing data in motion…

John: Oh, it’s wonderful, yes.

Chuck: It’s just absolutely wonderful. So we thought, “can we look at this” and what we weren’t able to do is portray the Hans Rosling Gapminder way, is looking at students traversing through an adaptive learning course of Psychology that is showing them in motion two things: what they’re achieving versus the number of activities. And that’s available from us in an animated form and then looking at them as they traverse through there, we have found that they produce certain behavioral styles and we found originally four. And the people in Ireland named this— they’re metaphors. Turned out to be animal[s]. The first one was the hare, and the hare— and remember what we’re plotting: number of learning activities against achievement level. The hare just goes zip. They start the course and they finish it in a very few days. That’s what they do— they’re finished. When I described in Psychology, that’s what happened in Psychology— they’re finished. And it sounds like heresy, but they’re finished. They’re done, ‘kay? And then the second one was the tortoise. The tortoise is the one who goes step by step by step by step by step through the course. They just progressed through the course in little increments and do this. And they get there, but they get there slowly. And then the Frog. The Frog did exactly what they would do in a regular course. There were eight modules in the Psychology course— they completed one a week. They did it week by week by week by week the way they would do it in a course. And interesting enough, when Jeff turned the cord, Jeff Cassisi, he turned the course open. There are a lot of students who did that— they went week by week by week because that’s what they’ve been taught. For them, that’s what of course was: you do it in these increments. And then the kangaroo— didn’t do a damn thing until the end of the course and then did it all in three weeks. Did it all— just zipped to the end of it and it’s fun to watch him in animation. He’s just dead in the water until the end and then he goes zip! And he’s at the end of the course. So there are four different ways that they approach it in terms of doing this and to see that in motion is very compelling. What it tells you is, at least in this course, there are various behavioral styles to the course and you have to be comfortable with them because they’ve all reached mastery and they’ve all reached mastery in some ways we wouldn’t approve of it. We wouldn’t approve of that kangaroo, would we? Doing nothing until the end of the course. That’s not right, but they finished so we have to learn how to deal with that in terms of doing this.

Rebecca: Despite the fact that many academics exhibit that exact behavior… [LAUGHTER]

John: That’s why we have deadlines. [LAUGHTER]

Chuck: Yeah, we have deadlines. Yeah, nothing is quite as motivating as fear, right? [LAUGHTER] yeah yeah yeah, but one of the things that I’ll say (is) there’s a lot of research that’s affecting us. And a really compelling book is called Scarcity, I don’t know if you’ve ever seen it, by Mullainathan, where they talk about students completely compelling to us at UCF for students coming from underrepresented populations in terms of what you think of students living close to the poverty line, what are they dealing with. They’re dealing with so many things in their life. Money, time, danger, single parenthood, finances, family, two part-time jobs, borrowing money from college. Fundamentally, when these students come to us, they’re exhibiting and expending so much cognitive energy just living life. Then when they come to us on campus, they burned up most of their cognitive bandwidth and then we put them in these lockstep courses where for some reason, they miss a class, they’re behind the power curve. We don’t design courses for these students. We firmly believe that they have as much intellectual capacity as any other students but we have not designed our University to accommodate. Adaptive learning is perfect for these students, it really is in terms of doing it so we’re trying to do that and accommodate these kinds of things. No wonder they drop out. In some cases, dropping out of a course becomes the optimal decision and what it is is their superstructure of life is so tenuous that if any one thing fails, this whole house of cards come tumbling down for these students. So they get behind a couple of classes and they’re done and we have to find a way to accommodate this. We’ve got to begin adjusting the way we organize ourselves.

John: What implication does adaptive learning have for the structure of the University?

Chuck: It has a lot. Our students are, now that they’re learning, asking really interesting questions like, “why do we need semesters” and the only answer we have is because we have semesters. [LAUGHTER] We don’t really have good answer for that and we’ve done this an adaptive learning we designed it. Now if you finish early, well you go on to the other course. The other end of adaptive learning is so what if you need an extra three weeks to complete the material and the semester is over? But that’s a nightmare for Financial Aid, it’s a nightmare for the Registrar, but we’re working on that in terms of doing this. But it hasn’t potentially turned on University structure on its head and I’m not sure we’re ready for that at the moment.

Rebecca: When a faculty member creates one of these courses, obviously they’re very involved in that content structure, but what’s their role in facilitating the course?

Chuck: Mm-hmm.

Rebecca: So if the class was to extend three weeks past for a particular student, does that mean then that faculty member is also engaged for that three weeks or what does that look like?

Chuck: Absolutely. In for a penny, in for a pound. If you’re involved with this kind of thing, if a student is still working within the platform and we’ve made agreements with the Registrar that they would get credit for the semester in which they did this, although they went past all of the due dates, a faculty member is still involved. So it changes the role of the faculty member immensely in terms of doing it. Some students (this is gonna come out wrong but), don’t need the faculty member. I mean, they need the faculty member, of course, I need the faculty member. This by no means abrogates the role of a faculty member, but it certainly does change it. Faculty members have to know when to intervene, when not to intervene— it changes us. It’s like teaching online: you have difficulty adapting to this. When I taught an online course, I did have difficulty adapting to my role, not being the center of attention all of the time. It was hard I have a big ego and it was difficult for me but but I got over it.

Rebecca: Are the assessments related to the adaptive learning stuff that’s like automated or something that the faculty member is manually grading using rubrics or things like that?

Chuck: Both, they are both. They get up to the faculty member we have a department who is now just getting over the notion of letting the assessments run within the adaptive learning platform and believing the assessments that it is as competency-based and students exhibited competency. That’s a hard sell for departments. Say a math department who say, fine, but they still have to take the test. You know what I’m saying? It’s a slow-moving thing and what we have done now is an a/b study in terms of demonstrating the fact that students who are assessed within the adaptive prep won’t do as well as students who were in other courses and they took the departmental exam and did equally well. We have to demonstrate that. It’s a hard sell. It’s very scary. It is a very scary phenomenon. Yes, you can design all kinds of platforms and you can build all kinds of intermediate testing devices, you can give tests within the platform and if students do well on the test, it can be, automated is the wrong word, it can be Bayesian decision to cycle them back to where they need to exhibit their skills and you can retest them. So it’s this continual cycling kind of thing.

Rebecca: I have a really easy time imagining how this would work for a knowledge gaining sort of class, maybe an a lower level of course, but a harder time envisioning when it might be like in an upper division class or one that might be more project-based or application based. What’s the experience been on your campus in terms of introductory level classes versus upper level classes giving adaptive learning a chance?

Chuck: That’s a good question. Question posted versus how would you teach Macbeth in adaptive learning? How would you do that? How would you teach clinical psychology in an adaptive learning course? The answer is that probably adaptive learning is not equally well suited for all disciplines or all levels. One is, adaptive learning is really really suited very well for hierarchical structured courses where achieving something at one level depends on achieving something at a slightly lower level. Like sequencing in math, or chemistry, or physics, or computer science. You can do it and we get it beginning psychology but is it really necessary? When you have eight modules in beginning psychology and there is a natural organic order but they don’t necessarily depend on each other. Now, Young came after Freud but, is it really dependent? That’s the kind of thing, I think there are some areas where it’s much better suited, and your question is well taken. I think we have to do a lot of exploration in terms of where adaptive learning is most suited and fits into our curriculum and it may not well fit equally well across all disciplines, all ecology. It’s a question we’ve got to do a lot of work on. Hopefully SUNY Oswego will answer most of those questions. [LAUGHTER]

John: Next semester. It’ll take a little while.

Chuck: Yeah, right. Right.

Rebecca: Do any of the classes involve, that you’ve been highlighting on your campus, have writing as a key component? I’m just curious.

Chuck: Yeah. I think probably in psychology there would be some writing involved in it. They would have to do some reaction papers and do that, yeah. So I think I can comfortably answer that question as, yes, there have been. It’s equally possible. The question is, can you teach creative writing in an adaptive learning course? In some ways it’s equally suited to it because I can imagine you could build a pretty good workshop in create a writing course, with a lot of work, but you can do it.

John: I looked at RealizeIt’s website and it said they create unique formative assessment items based on instructor provided question templates. How does that work?

Chuck: It works very well. In terms of doing it, I’ll tell you one of the things, yeah Ryan Baker wrote a really good paper. The technology is developed really well but the assessment in general is still kind of heuristic, if you know what I mean by that, we’re gonna assess your competency by whether you get four or five items right. That’s the heuristic part of that. You have to design better assessment devices. What we are doing now is we have to transform the assessment paradigm in terms of what they look like, in terms of are they authentic, are they reflective, and are they contextually relevant? Students respond much better to questions that are related to the disciplines that they’re going through, and we have to develop that. We’re nowhere near that, but we’re working on that. They’re very good about helping them. You give them a template, we want to do this, we want to do this in a simulation platform, they’ll help you work with it. They’re very good about doing that, but they can’t do it all. It is a partnership.

Rebecca: We usually hear about adaptive learning in online context. Have you had any experience in a hybrid environment or an in-person environment?

Chuck: Oh absolutely. We’ve taught some blended, adaptive learning courses, and it makes absolute sense. What’s a blended course? What do faculty members, the first time they think about blended, and they think about it incorrectly. What am I face-to-face, can I offload to the online environment? That’s about the worst way to go teaching a blended course, right? The thing is you look at what are the appropriate kinds of things for these two different formats? We have this all of the time where essentially we’ve had a blended nursing courses where they do material content offline and come in to the class and essentially do problem solving. It’s basically is very, very appropriate for this notion of a flipped blended course. We’re actually having a faculty member do it in statics, in engineering, which is really exciting. You know, it’s a whole notion of some of the courses are flipped some of the courses are flopped. [LAUGHTER]. But we sort of look at that and it makes kind of sense. But again, back to your question, is where is this modality most appropriate? Where does it fit? That’s the kind of question, and that’s really a departmental discipline sort of thing. Yeah, every discipline believes they are unique, right? Yeah, their pedagogical issues are unique to it. Yeah right [LAUGHTER].

John: Actually, along those lines we’ve had a number of reading groups here and we’ve had faculty from different departments get together and the faculty who are new to this type of thing expressed surprise at how common the concerns that they had were. To find out that people in other disciplines, very different disciplines, faced exactly the same problems and sometimes had some really good solutions.

Chuck: Yeah, we do this thing, I’m very fond of a concept by the sociologist, Susan Lee Star, she called it a boundary object. A boundary object is something like this, we do it all the time at UCF; critical thinking. We’re really into active learning now at the moment and active learning is another boundary object. We’ll do this say with the Faculty Senate, who’s in favor of critical thinking? Of course every hand goes up [LAUGHTER]. You wouldn’t dare say you’re not in favor of critical thinking. But when you get in a large community of practice nobody agrees what it is, right?

John: Exactly.

Rebecca: Yeah.

Chuck: Exactly. So, Susan Lee Star, this boundary object, is something like that. That it holds a community of practice together, but it’s very weak. It’s not strong enough to be really functional in a large community of practice but you go back to individual constituency, go back to physics, or rhetoric, or creative writing, or education, and they damn well know what critical thinking is in their discipline, right?

John: Right.

Chuck: They do. They absolutely can do it and they’re very powerful. When you bring it back to the community you’re back into the same dog fight and that’s a very powerful concept. I can name literally dozens of them: active learning, critical thinking, online learning, you go on. Very, very powerful.

Rebecca: I find writing is one that bubbles up. What do we mean by writing? What does that look like?

Chuck: Oh yeah. What is writing look like when you’re tweeting— are you writing?

Rebecca: Yeah, exactly.

Chuck: When you’re blogging, are you writing? How do you workshop writing now? Are workshops necessary? It’s all fascinating to me, but I guess that’s because we’re academics. If it weren’t for boundary objects we wouldn’t have anything to do. [LAUGHTER].

Rebecca: You start talking a little bit earlier about the implications of adaptive learning at a university. Can you expand upon that a little bit? [LAUGHTER]

Chuck: Sure. I mean the implications is; what does this say for the structure of the university? What does this say for the way when we’ve organized this enterprise called learning? There’s another great boundary object: student learning, student learning outcome, one of my favorites [LAUGHTER]. Go ahead, define that. My friend, Anders Norberg, from Sweden, we’ve written a paper called, a Time Based Model of Blended Learning. Where time becomes a fundamental design structure of a university and it’s virtually very very different from the way we organize learning at the moment and how we have organized learning is in the sense of discrete units called classes, called units, called semesters, called years and called matriculation period time. It has tremendous implications for that kind of structure. We are employed by all of these kinds of things and we’re organized by all of that so sooner or later we’re going to have to re-examine all of that, if we’re going to adopt these things and adapt these things and be environmental like that. There are three things I think associated with good ideas. One is was the adjacent possible, what’s the next reasonable step, it’s what’s next, what can we reasonably accomplish next, that’s the adjacent possible. Outside of that is the adjacent impossible, you can’t do it, you can’t do it. Secondly, you have to have the slow hunch, you have to stay the course. How many things have we done this but we’re gonna try it, it didn’t work right away and then we dumped it? I’m sure nobody at SUNY Oswego has done that, but we certainly have done that at UCF. You got to stay the course! If you read Darwin’s autobiography, he said “Yet a Eureka moment about natural selection when true”. If you carefully looked at his notes he founded months and years for it was in his notes he just didn’t know it was no Eureka moment, that’s stay the course. If you know this, you got to stay the course because it’ll be bad times, everything’s not gonna work, it’s just not going to be the way. Again, you over expect short term, you under expect long term. Then the third thing is you gotta have this liquid network, you’ve got to work with the vendors, you got to work with faculty, you got to work with administrators. SUNY has to work with UCF that has to work with CTU, that has to work with Carnegie Mellon, that has to work with Oleum Geo Services. You have to have this liquid network where we can share ideas. We can’t do it alone, frankly you can’t do it alone, we’ve got to work that way and if we work that way it’s going to change the whole way we do with the business. I guess the question for us is how much do we want to change? It’s way above my pay grade to change a University and I expect for you too, but sooner or later we’re gonna have to accomplish that there’s a lot of implications for the University.

Rebecca: There’s a lot that I’m hearing you talk about, that reminds me of agile design practice. That’s made a big boom in technology and design. The idea of small little sprints break a big problem into smaller problems that you can work towards. And then it’s iterative, you keep going back and it’s circular it’s not a straight line, that’s what I’m hearing you say. That’s the way that it needs to be tackled.

Chuck: I think that’s a good metaphor, we really be in an agile scrum and keep doing it until we get it right. That’s frankly not been our history right, part of our history is let’s declare a victory and move on to something else. I mean that in the kindest kind of way but we have some serious issues in my judgment. We have tremendous educational inequality in this country, unless we crack that, we’ve got some issues that we’re never going to solve.

John: And you guys are doing quite a bit on all those fronts and I hope we’ll have you back for some future podcasts to talk about some of those things.

Chuck: We’ve made some amazing breakthroughs in communities that you think wouldn’t achieve. The talent pool is as deep as anywhere else, but we’ve got to figure out how to let it up.

Rebecca: I think the other thing that I was hearing and what you’re talking about is the idea of these micro credentials is also surfacing that we had a prior podcast on. I really can see the adaptive learning move at the same pace as micro-credentialing because I think they’re directly related to one another and how the system might have to shift.

Chuck: Kahneman wrote that great book “Thinking Fast and Slow” and kind of admitted in terms of one of our habits is we attempt to solve a hard problem; we try to measure outcomes, we can’t figure out what to measure so we measure something else that we know how to measure. I’ll give you a great example “student learning outcomes”. We’re not too good at that, so we measure grades we measure course success, course success is not learning outcomes, we do an easier thing because we don’t know how to do the hard thing. I do that all the time! [LAUGHTER]

John: We all do that… I think.

Chuck: Yeah, I really love Oswego.

John: We enjoy it too, I’ve been here since 1983.

Rebecca: I came back.

John: That’s right! Rebecca was a student as well.

Chuck: Rebecca we knew you’d come crawling back…[LAUGHTER]… Where does the time go John? What happened?

John: I know, I just got here it feels like.

Chuck: Remember John Lennon? That song “My Boy My Beautiful Boy” that’s what happens when you’re busy making other plans.

John: Yes.

Rebecca: All the talk of time leads to the question of, Mike what does the future hold?

John: What are you doing next?

Chuck: What are we doing next? We’re reinventing the University, that’s what we’re doing. We have a new president, president designate Dale Whitaker who’s been our Provost. Dale is a very big thinker and he’s developing ideas like zero probation, all services, the hub for faculty, the faculty center and all services faculty be located in one kind of place that students become an active part of the instructional process. They’re no longer receptacles, if they become teachers as well we have to do that. They have a lot to teach us, it’s a different kind of world to do this. We have to understand better how our students acquire knowledge, we have to better understand the many generations that are existing on our campuses. I kind of like to get my news from “The Onion”.

Rebecca: hahaha !

John: That’s the best place these days and it’s not that far off.

Chuck: My favorite tagline in the onion is eccentric student reads entire book, we have to begin to accommodate the way students learn and use their learning devices. My graduate students at times will send me off to get a cup of coffee when they fix my technology problem. Technology for them is what Machan of the Reedy called “Living now in the InfoSphere” where information communication technologies talk to each other where we’re no longer in the loop, we’re on the loop. And I think you can understand that in terms of when you look at your Facebook page and what you were looking at on Amazon pops up on your Facebook page, you know that these technologies are talking to each other.

Rebecca: I thought they were reading my mind [Laughter]

Chuck: But the cover of last week’s Economist was “Epic Fail” and it was the Facebook F falling off the fail, lying on its back. The covered this week is AI spy, artificial intelligence spying you on the workplace, we have lots going on. I think the recent things that have happened over the weeks with Facebook give us the fact that we have some serious examination of our culture and our information to confront in the decades to come. I think the future is being defined for us by forces outside of our realm. This is pretty scary stuff for me. Thank you both, you guys are great. I really enjoyed it, thanks for having me

John: We’ve enjoyed this tremendously.

Rebecca: Yeah, it’s really interesting to hear what you’re doing.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]