As colleges and universities plan for the uncertainties associated with the fall 2020 semester, it is fairly clear that faculty should receive more training in online instruction than was possible during the rapid transition to remote instruction that took place during the spring 2020 semester. Most professional development programs, though, are resource intensive and cannot be easily scaled given current college and university budget conditions. In this episode, Dr. Darina Slattery joins us again to discuss the less resource-intensive professional development program she developed in which groups of faculty complete two days of training to prepare them to efficiently transition their courses to online instruction.
Darina is the head of Technical Communication and Instructional Design at the University of Limerick. She is also the Vice President of the IEEE Professional Communication Society.
- Darina Slattery
- Gilly Salmon –
- Gagne, Robert. Gagne’s 9 Events of Instruction
- Darina Slattery, “E-Tivities.” Tea for Teaching podcast, May 13, 2020
John: As colleges and universities plan for the uncertainties associated with the fall 2020 semester, it is fairly clear that faculty should receive more training in online instruction than was possible during the rapid transition to remote instruction that took place during the spring 2020 semester. Most professional development programs, though, are resource intensive and cannot be easily scaled given current college and university budget conditions. In this episode, we discuss a less resource-intensive professional development program in which groups of faculty complete two days of training to prepare them to efficiently transition their courses to online instruction.
John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…
John: …and Rebecca Mushtare, a graphic designer.
Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.
Fiona: My name is Fiona Coll. I teach in the Department of English and Creative Writing here at SUNY Oswego, and this is my turn to sit in as a guest host.
John: Our guest today is Dr. Darina Slattery. Darina is the head of Technical Communication and Instructional Design at the University of Limerick. She is also the Vice President of the IEEE Professional Communication Society. Welcome back, Darina!
Darina: Thank you, John. And thank you, Fiona. It’s great to be here again.
Fiona: Today’s teas are… I’ll kick things off. I’m drinking Murchie’s Earl Grey Cream, which is my comfort tea.
Darina: Great. Well, I’m drinking traditional Irish tea, just black tea with loads of milk or cream as you say. So it’s just Barry’s tea. [LAUGHTER] I’m very traditional.
John: And I’m drinking an Irish breakfast tea, but from Twinings, so it’s a British Irish tea. [LAUGHTER]
Darina: That’s the one I would always get if I’m in the US having a cup of tea, yeah [LAUGHTER].
John: I do have Barry’s tea, but it’s up in the office, locked away along with most of our teas.
Darina: Okay. Okay.
John: So, we’ve invited you back again to talk about the DUO workshops you have created for developing blended and fully online programs. Could you tell us a bit about these?
Darina: Yeah, these are workshops that I first designed in 2014. One of our Dean’s needed a certificate program to go online in a very short period of time and the people who were going to be teaching on it were on-campus teaching staff. So, she wanted the program to go online. So she said, we need to have some kind of professional development for those staff. So I suppose I kind of took over a little bit in terms of I was very excited about this as it’s the kind of thing I do. This is what I teach, because I teach students how to design online courses. So, I was really excited at the prospect of developing some kind of professional development for my colleagues. So I kind of took over it in 2014, designed and developed it, rolled it out, and I’ve done about 13 of them since then, mostly to the faculty in my own university, but also one year, I rolled it out to an EU funded project where we had colleagues in five EU institutions who were going to try and teach their courses online together. So they all attended my DUO workshop. They actually came to Limerick to do the DUO workshop that particular year. So, essentially, it’s like a one and a half to two-day workshop. The length of it kind of depends on the group, it depends on how engaged they are, maybe what they’ve already done before, or if they have some experience or not. So I generally say one and a half to two days max. But usually by the afternoon of the second day, you know, things are wrapping up. And it’s mostly facilitation led, but there’s lots of activities I’ve built into it, then, as well over the years where I’ve tried to move it from being me telling everybody what to do, to them actually trying things out as they’re moving along, and that they can see some progress happening with their blended or fully online course. So, normally, it’s people who are thinking about moving a program online or developing a new online program, andthey don’t know where to start. So, they might have years of experience teaching on campus, but they’ve never taught online. So, that’s where then I’m asked, am I available? And when am I available? And we work from there.
Fiona: Can you tell us what DUO stands for?
Darina: Yeah, so I came up with the acronym. It stands for Developing UL, that’s my university, Online. So, I kind of liked the DUO idea as well because we’re on campus mostly, but now we’re going to be doing online courses as well, or online programs as well. I personally have been teaching online for a long time, but most of my colleagues only teach face to face. Well, they’re all teaching online now, but they weren’t teaching online a few weeks ago. [LAUGHTER] So, everyone’s an expert now. [LAUGHTER]
John: Have you been doing a lot of these workshops recently to help people move online?
Darina: Not so much, because when the pandemic struck, it kind of happened really quickly. So, there wasn’t time to kind of plan the normal DUO where normally it’s a face-to-face workshop, where you have everybody from the program team in the same room. They’re all working on their laptops, you know, planning things, and then we do storyboards, which we can talk about shortly. So, what has happened was, because it was such an emergency, like this week, you’re teaching on campus, but next Monday, you have to teach online, the University came up with other forms of professional development that would just kind of get the urgent things out there, like how you would do a meeting online instead, or, you know, how you’d add audio to your PowerPoint slides and stuff like that. So, I’m part of another group, a forum in the university where we all rolled out lots of different webinars and things like that. But now, we’re starting to plan for the fall in case things are online, and that’s where now we‘ll start planning. And we can schedule things a bit easier then. It’s just there was no scheduling time at all. It was just panic really, [LAUGHTER]for a couple of weeks there.
John: We experienced something very similar here.
Darina: Yeah. Yeah.
John: But I would think that a one and a half to two-day workshop could be a really nice model for campuses that are uncertain about what’s going to be happening in the fall, to help faculty transition, which is one of the reasons why we wanted to talk about this in light of this transition and about the uncertainty that we’re facing in the fall. In these workshops, you help faculty develop learning objectives, based on Mager’s three characteristics. Could you define these characteristics for those who aren’t familiar with this approach?
Darina: Yeah. So, I suppose, to take a step back… First, when I initially started running these workshops, I used to talk about Gagne’s learning outcomes and the five components of a really good performance objective. That’s because I teach that kind of stuff anyway, and I’m interested in it, but I actually found it was nearly a little bit too much for faculty who, you know, they were coming in wanting to know what tools to use. And suddenly they were talking about what learning capability verb to use with their objective and stuff like that. And they kind of want to get past that. So, I had to kind of meet them midway and say, “Right, you don’t want to do all that stuff that I think is really important. But, I’m going to give you an easier version of it that will still kind of partially address that concern.” I was not going to leave out the objective side of things and the learning outcome side. So, Mager’s model, then, was the other model that I normally talk about, even with my own students, and that’s, I suppose, simpler and easier to understand, so he recommends that a really good objective would have three components, it mightn’t have all three but there are three possible components. It will always have a performance, which is your action verb, like “What is it do you want the person to do?” Then, it may have conditions associated with it. So, like you might say, “without the aid of a calculator, compute x, y, z” or it may have other criteria. So instead of saying “at the end of this lesson, you’ll be able to speak French” you might say “you’ll be able to speak fluent French,” or that you will do a particular task within 10 seconds, if you determined that the 10 seconds are critical to the performance of that task. So I’m just trying to get them to think more. Because, in my experience, I mean, I’m the only person of all the colleagues I’ve spoken to who actually have objectives for all my lectures. I have objectives for all my assignments. Most of my colleagues would really only have objectives on their course outline, at the very start, like “this is what we’re going to cover and at the end of the course, you will be able to do these things.” Whereas I’m very much about telling students the purpose of everything we’re doing, and that’s kind of good practice, but a lot of people just don’t know that. So, you know, we can’t blame them for not knowing that yet. So, I’m trying to kind of get them thinking that way, that everything that you get your students to do needs to be aligned with the learning outcomes; that you need to be very clear when you’re articulating what you want them to do. Because I think back to when I was in school or in college, and you got your feedback after an assignment and you’re kind of aggrieved that I didn’t know I was supposed to compare and contrast. “You didn’t say compare and contrast, you said discuss.” So I discussed. So, I used to remember feeling upset about those kinds of things that I didn’t know that’s what you wanted me to do, and I would have done it if I had known. So it’s our responsibility really as teachers to tell them what we want. Now whether they do or not is up to them. But if we haven’t articulated, clearly, there’s going to be a problem. So that’s why I give them that aspect of the objectives. And they seem to kind of grasp that and think about it a bit more. And I often notice they start revising their objectives and their outcomes because I talk to them about Gagne’s five learning outcomes as well. And most of them are teaching cognitive outcomes, but you know, some of them are teaching psychomotor skills and attitudes and so on. I’ve kind of noticed over the years, a lot of them are writing learning outcomes because they’re required to write them for program accreditation, but they have no background in it, they don’t know why they’re doing this, they’re maybe just kind of copying what colleagues have written for their outcomes and so on. So again, I kind of managed to sideline that stuff into the DUO workshop as well, because you see a lot of them thinking “Oh, I didn’t know we were going to cover this but actually that’s great to know that. I didn’t know that’s why we do it this way.” So Magers is the kind of softer version of Gagne’s five more difficult components, even though they’re probably even more accurate. So, the performance, the conditions, and the criteria are the three components you could have in a good objective. You won’t always have conditions. You won’t always have criteria. So, it’s really important as well that people don’t just add in something like “within 10 seconds,” if 10 seconds isn’t critical to the performance. So again, it’s just about being very clear to your students, so they know what you expect of them.
Fiona: I like the way in which this opportunity for faculty or instructors themselves to learn something new becomes the moment when they can actually articulate what it is they want students to do. So, I like that there’s a dual, I like playing on the idea of the duo. [LAUGHTER]
Darina: I’m really chuffed at myself that I came up with that name. [LAUGHTER]
Fiona: Yeah. And so it seems eminently useful to have this moment where faculty are really digging in and thinking not just about modality or technology or tools, but actual learning outcomes, real learning outcomes, and it also seems like a good moment to investigate the difference between necessary challenges for learning and unnecessary barriers to learning that might be leftover from how an instructor was trained themselves or what they’re comfortable doing or disciplinary habits of assessment or evaluation. And so can you tell us where this fits in? Where this reflective piece fits into your two-day structure? How do you lead faculty from this reflective moment into learning about the online experience?
Darina: Yes. So in terms of the structure, I kind of start with this stuff. I start with learning outcomes. I always have this kind of feeling in the pit of my stomach at the start of the workshop that they’re going to go, “Oh, this isn’t what I thought this workshop was about. I thought we were going to look at cool technology. Why is she doing all this boring theory stuff?” But, I kind of have to get them on board and say, “Look, bear with me, this is important that you do this.” Most participants you can see them thinking, “Oh, okay, that’s not what I wanted today, but actually there’s value in it and now I need to revise my learning outcomes and objectives.” So, I get from that and then I move towards things like Gagne’s nine events of instruction, which is kind of practical steps that you do when you teach. And then I move them a little bit more towards like, “What kind of resources, what might we find online that you could use with your teaching?” And then I move towards then, maybe coming up with ideas for activities they might like to do online, but they don’t know how they would do it yet. They don’t know how they would have class presentations online, but they’re hoping I’m going to tell them. So we get to that as well. So it’s kind of bit by bit moving towards what they came in for, which was “Tell me how my course can be online.” [LAUGHTER] That’s the only objective they have coming into the room. Whereas, I actually get a whole lot of other little treats in there along the way. But it does start with the pedagogy. And I’ve had the odd workshop where you get the vibe from the room that we kind of just want to get to that other stuff. And it’s an uncomfortable place to be but I’m like, “This is the right way to do this.” You know what I mean? And you need to bear with me and they always see the value in this by the end, but there’s been a couple of times where I thought “Oh, they really don’t want to hear this stuff right now.” But, you know, if you want to be a good teacher, a good online teacher, it’s important.
John: After people work on the learning objectives, what would be the next step in the workshop process?
Darina: Yes. So I have an activity that and, over the years, I’ve incorporated more activities, because at the start, it was more me teaching them how to teach online. And I knew there was a need for them to do more than listen to me for one and a half to two days. So one of the early activities is “What are your learning outcomes for the course you’re trying to move online?” The activities that you’re currently doing, if it’s a current on-campus course, for example, are the objectives and the activities aligned? If they’re not, how might they be? So, they complete a collaborative Google doc at the same time. Now, I have a hidden agenda for using a Google doc. I know we all use Google docs all the time now. But, a huge amount of people have not collaborated in a Google doc. Maybe they have in the last few weeks, things have changed a bit. In the last few weeks, a lot more people have been exposed. But, normally a lot of faculty wouldn’t have any need to do that. If they’re writing a paper with someone else, they’ll write their Word document, they’ll email it to someone. They’ll add their content, then email it back, and stuff of that. So, I want them to see how they’re all contributing to that document live during the workshop, and it’s up on the big screen, and how this is something that they could get their students to do and it doesn’t seem that difficult. So, I’m trying to introduce them to the technology that way as well. So, that’s an early activity, you know: What are your outcomes? Are they aligned? Then I kind of move towards the events of instruction. So, the events of instruction are really a simple list of nine events that you should try and carry out in any teaching, whether it’s face to face or online. And we do a lot of these anyway. So even if you’ve never heard of Gagne’s nine events, you’re probably doing them, as a teacher does. But there are a couple of them that you might forget about, or that you might not put much emphasis in. So I find it really handy to just think of those nine steps in my head. So, the first thing you need to do is get the attention of your audience. Also related to that, by the way, you have to maintain their attention, which we know as well is another challenge. The second thing you need to do is inform them of the objectives. That’s one that a lot of people leave out. They have their objectives on their course outlines, but that’s the end of the objectives until the semester is over. So, you need to tell them why you’re getting them to do this activity, what they’re going to learn in today’s lecture and why it’s important. You need to stimulate recall of prior learning, that’s something that I see a lot of my colleagues leaving out, they forget to say how this material is connected to what we did before or what you did in some other course. Then you’re obviously presenting the stimulus material, which is the course content. And there’s a whole world of theory about how best to present your content. The next step is providing guidance. So telling them clearly what you want them to do, where, when, how and so on. Eliciting performance is not the same as the formal assessment. That’s like getting your students to engage regularly, which we all know we need to do more often. Giving them feedback, then the formal assessment. And then the one that is often forgotten about the final event is enhancing retention and transfer. So, I’m sure we’ve all taken courses where, years later, we realize why that thing was useful, because the teacher never told us… But we’ve just done something and suddenly, “oh, it all makes sense to us.” So, enhancing retention and transfer is another one that people really do forget. They feel once they get to the assessment that they’re done, and that there’s nothing else to be done. So, I really like that list because I can use it in my face-to-face teaching and in your online teaching. And if you’re trying to think like, “Am I missing anything?”, have a look at those nine events of instruction. I’m personally a huge fan of Gagne’s work. So I kind of talk about Gagne constantly. He was very particular, very organized, very structured, and so on. If you’re like me, you will love that. Other teachers are a bit more freelance, maybe they mightn’t like that structure as much. But it’s great to know that there is a template there that if you’re not sure what to do that you can consult it, you know?
Fiona: It seems really useful to have something that crystallized, I suppose, about what you’re trying to do in this new environment. Are there any of those learning events, those events of instruction, that are especially challenging online?
Darina: No, I think, in lots of ways, some of them actually, I won’t say easier to do, but there’s more options for them. So, you know, like for presenting your content, you can cater to multiple intelligences easier, maybe online than you can in a classroom where everybody’s just sitting there looking at one screen, for example. In online environments, you can get them podcasts and video and standard slides and whatever else. Certain things will be maybe more complicated because there’s a technology layer in between. But, then there’s also more options because of the technology layer in between. So you know, getting people to perform. I mean, we know there’s a wealth of activities you can get students to do online, but you still have to devise all of those, you still have to come up with activities, and then have the technical skills, know how, or whatever. Do you need to pick the right tools? and so on. So, there’s a huge amount of decisions to be made before you can do some of those events online. But if you make good decisions, it will be possibly even richer than it might be in your class. Again, it depends on what kind of a classroom teacher you are. I mean, some people are excellent classroom teachers, and there’s nothing that needs to be improved. But, for a lot of us, there’s nearly so much choice. That’s kind of daunting when it comes to teaching online. Just tell me which tool I should use for feedback. Just tell me which tool I should use for my lecture slides. You know, it’s those practical, urgent needs right now that most people are concerned with, not maybe the bigger picture sometimes.
Fiona: I think one of the things we’ve been talking about in terms of this emergency distance teaching situation is how to manage cognitive load for students, right? They’re dealing with so many other things, let alone this complete shift they didn’t choose, they didn’t ask for.
Fiona: But it’s really important to think about that from an instructor’s perspective as well, that we too can be totally overwhelmed by choice, as you say.
Darina: Of course, yeah. And that’s something that I’ve really seen in the last few weeks and as somebody who’s been teaching online for a long time, in some ways, this was an easy transition for me the last few weeks, but in other ways, there were moments where I was really overwhelmed by the amount of resources that were being sent to me on a daily basis about “here’s how to do your online lecture,” you know, “here’s how to have a meeting,” “use this tool” or “use this tool.” There were days when I just thought, “oh my god, I feel so overwhelmed.” And I wasn’t even looking for that information, but it’s being thrown at me, you know, [LAUGHTER] and if I was looking for that, I don’t know, if you’d presented to me at the right time anyway. So, I’ve thought several times over the last few weeks, there must be people who must have just gone: “That’s enough. I’m just gonna go with the first tool that’s recommended to me or if Mary who used to sit beside me says she uses Microsoft Teams, I’m going to go with Microsoft Teams, or I’m going to use Zoom, because that’s the word I’m hearing about all the time.” And I think there is a problem with all of that as well, you know, and one would probably see those problems emerging over the next few weeks and months, where people just are so overwhelmed with choice and everybody trying to help, I mean, myself included. I was creating resources and sharing them with people as well. So, I’m just as guilty. But for most people, you have a problem right now. You just want the resource to solve this problem right now. You don’t have the headspace when this is thrown at you to go back and evaluate multiple tools and pick which one appeals to you personally, you know. You’re just going to go with the most popular tool or the one that’s supported by your IT department or whatever. So, the overwhelming thing has really been on my mind the last few weeks and how did people make decisions about what to do. For our university, Zoom isn’t actually officially supported by our IT department, but because Zoom was mentioned so many times in that first week, everybody said “Right, I’ll do Zoom” [LAUGHTER], you know. And then suddenly it was all over the internet that there were problems with Zoom and people started to panic. But you know, they’ve committed to Zoom and they didn’t want to undo Zoom, you know, two weeks later. So, there’s a lot of that, I’d love to get into that to find out, you know, what were you thinking at those times when we were throwing all these resources at you? [LAUGHTER]
Fiona: Absolutely. This might be a good moment to come back to the wonderfully mindful structure of the two-day DUO experience. So, you’ve described bringing instructors in with this theoretical framework for thinking about their teaching, and you’ve described some of these exercises. How does it continue? Where do you take them next?
Darina: Okay, so say when we’ve come out of Gagne’s events of instruction, and I’ve spoken to them about different types of assessment options that you have, including some of the traditional ones that move online, like essays are just as good online as they were in the classroom environment, but I start talking to them then about, for example, possible social media assignments you could do, e-portfolios, podcasting, reflective blogging. And I show them examples of those with my own students, that’s when they start to get kind of excited then, because they can say, “okay, I’ve heard these words e-portfolios or I listen to podcasts all the time, but I’ve never created a podcast. I don’t know what you would do, what software you need. Do I have to have to buy equipment? Really, most people are at that basic level. And I want them to know that that’s okay that most people are at that basic level. And when they realize how easy it is to do it, or you point them towards a good resource, they’re really excited how “Oh, so I could do a podcast for my lectures and make it freely available” and stuff like that. So that’s where the kind of relief starts to set in a little bit, that there are loads of options, but she’s been doing this for a long time, and she just does these options, and they seem to be acceptable and, you know, very good or whatever. I think people need a lot of reassurance about methods because like, I’m not using all the latest technologies in my teaching, I’m using kind of good, reliable, consistent things like discussion forums, and well thought out activities and structured course materials and everything’s organized and they’re the kind of key practices that I’m just doing online. I’m not using, as I said, the latest tools for everything because the latest tools might not exist tomorrow, there has to be a rationale for the technology you’re using. So when I start showing them example assignments, then you can see people thinking about how they could do that with their students. Sometimes they’ll think Twitter, social media, I have no interest in that personally. I’m not going to do that with my students, that’s fine as well. Then I mention to them about learning object repositories and MOOCs. A lot of them still wouldn’t have done MOOCs before. You know, they might have heard the word MOOCs but never engaged in a MOOC. Most people haven’t really heard of learning object repositories, so I show them some examples, and I try and customize the examples to what their discipline is. So I showed them some economics resources that they could use if they’re economics professors, and then suddenly it’s like, “Wow, we can use those for free in our courses and we don’t have to develop them. That’s good.” So that’s another activity they do then I give them a little bit of time to have a browse. Because I know myself, I mean, there’s so much material available online, but I don’t really have time, most of the time, to actually look for stuff. So I just end up inventing it from scratch myself. So I just give them a little bit of time to dabble. At least they’ve looked at a MOOC. And there’s two reasons why I get them to look at MOOCs. Number one is to make them aware of what MOOCs are, and that they could engage in professional development themselves or study something they’ve always been personally interested in, but also to see how someone else teaches what they teach, because you get great ideas when you see how somebody overcame some kind of challenge that you personally have in your class. So that’s moving in then to the online assessments I’ve moved into, here’s some online resources you can use that will help you. That’s kind of roughly where the first stage of the workshop ends. That’s the kind of course planning and design, like what could you do? What are the options kind of a thing. The second part is the course delivery part. So now you can know what you want to do. How would you do it? So I talk to them about Gilly Salmon. She has a five-stage model of teaching and learning online. So I showed them the different phases that learners typically go through as they’re studying online, like from being afraid to access the technology and not knowing what password to use, etc, to kind of maybe reaching out to some other classmates to then starting to share resources about the assignment they’ve just been given to then trying to solve problems and do knowledge construction right through to kind of developing as independent active learners. So, I go through that model as well and kind of try to reassure them as well as that, you know, you will have learners at different phases at different times. Don’t feel this is you doing something wrong, that there are different phases, this is well known, it’s well researched, and so on. So I’m trying to constantly say to them, you know, you’re going to encounter these different issues along the way over the coming years, and just know that they have been documented before as being common problems, because people tend to blame themselves immediately when they have a problem teaching with technology. They always say, “Oh, I’m really bad at technology, it must be me, it must be my setup.” And it’s often not. Like, I’ve had so many bugs and problems with software the last few weeks, and I’m quite technically capable. And I’m pulling my hair out sometimes with technology. So, I’m thinking if you’re new to this, you know, you’re just going to be blaming yourself or think, “Oh, I just can’t do this” or you know what I mean? You’re just going to pull out too soon. So, I talk about Gilly Salmon’s five-stage model. So, it’s kind of the practical things you can do to help learners as they move through the phases. And I also talked to them about Gilly Salmon’s e-tivities. So, the e-tivities are structured forum based activities. I think we did our other podcast on that.
John: We’ll include a link to the earlier podcast in the show notes.
Darina: So I won’t go through all that again. But they’re just basically activities that are structured a certain way and they’re usually housed within the discussion forum. And you can get students to do potentially anything in an e-tivity. Whether it’s collect soil samples, and come back and report on it or have a debate or whatever it might be. So, I showed them examples of those and how I use those as the ongoing assessments in my own online courses. And then I kind of wrap up that course delivery section then with kind of some best practice guidelines, kind of tips that I’ve learned along the way: do this, don’t do that or you should consider this as well as, you know, other people’s guidelines as well. So that kind of brings you then to the end of this is how you would design and plan how you might deliver it. And then the third part then is where I kind of get them to do more work. And it’s me relaxing a little bit, and then kind of storyboarding their courses. So the storyboarding then is when I tell them to turn off their laptops for a while, because they’re all probably answering emails while they’re listening to me at the same time. So they turn off their laptops. And so I’m basically following actually, Gilly Salmon. I’m going to mention her again. She’s not paying me, by the way for mentioning her or something. [LAUGHTER] But she has a course design approach called Carpe Diem, and it’s for designing online courses. It’s been around a while. So one of the features of that kind of workshop was the A3 flip chart paper, different colored post-it notes and markers, and literally drawing columns for each week of your course, writing in the topics you’re going to cover each week, discussing them in a group. I find that a lot of my colleagues, there might be an initial program design team meeting, but once everybody has been assigned their course they tend to go off and do their own thing. And there doesn’t tend to be like regular discussion about well what assignments are you giving your students and are you doing reflective blogging, that’s great, I can continue from where you left off. It tends to be everybody working in silos, kind of once the program has started. So this is a great opportunity, even though it’s at the start, for people to actually talk out loud about, “well, I would love to do this, or I’d love to do that. But maybe you should do it because it would make more sense in your course.” And they have that conversation over the course of several hours. And it’s something that doesn’t happen a lot. It usually might happen by email, but not in a room where everybody’s literally kind of brainstorming together about what to do and when. And that’s when they also plot out then when their assignments are going to be issued. And the various activities or e-tivities. So I kind of convert all my participants to e-tivity fans by the end it. So it allows them to use that kind of model because they know it works for other courses in other programs in UL. So really the kind of agenda behind the storyboard is that when you walk out of the room, you have a big sheet of paper with what you’re going to do every week, what kind of activities you’ve committed to, are you’re going to do blogs or podcasting or portfolios or standard essays via the LMS, whatever it might be, but they have a template ready. They know what they need to do. They have an action plan. They might not have tried all the tools out yet, but they know that they don’t need to focus on Camtasia. Or they don’t need to use Zoom for their particular cohort or whatever it might be. They know what is a workable model by the time they leave the room. So that’s kind of why I like to wrap up with the storyboarding side of things.
John: And you have that collaborative aspect because most of your development has been within individual programs where they’re all teaching the same subject. So there’s a lot of opportunities for feedback.
Darina: There is.
John: Would that work as well, if it was a more mixed group of faculty,
Darina: I’ve done them with mixed groups as well. So I’ve done some for my own faculty where there’s been lecturers in languages, history, politics, public administration, obviously, each discipline will have their own concerns. So the language teachers are going to be really concerned about how to do oral examinations online. The politics people might be more interested in debates and discussions online. So individual questions will be a bit more tailored to their disciplines, but it has worked, but usually when people come to me to do this workshop, it’s “We want to move this program online, can you do one of your workshops?” And then I say, “Right, this is on economics,” or the last one I did was on artificial intelligence machine learning for finance. So it’s a new master’s program that’s starting in September, hopefully. And they had one of the courses on that program that is already online on another program. But all the rest of them have already been taught on campus or not at all. So they’re designing new courses from scratch, as well as thinking about how to do it online. So they went through the DUO workshop as well. That was the most recent one I did. And all the discussions at least were about kind of issues that would be relevant to the people teaching those courses in terms of the kind of tools they would use, the kind of assignments that they would do and so on.
John: We’ve done reading groups on campus for about six or seven years now, I believe. And one of the things that people have often been surprised by is how, when people from different disciplines are sharing ideas, it often sparks some creativity.
Darina: And even for myself, I’d rightly be writing down “This is a good idea.” I didn’t think of doing this myself, you know, and we often learn from our students. And in my case, in the DUO workshop, my students are colleagues, but they often have great ideas about “could you do this” or “Well, actually, I’ve done this.” And they might even be people who think they’re really poor at technology, but they can surprise you the things they’ve thought of doing with their students. And you think I could adapt that and do that with my own students. So I usually come up with lots of ideas for my own courses as well, after the workshop.
Fiona: I’m a literature specialist. And so I’m still thinking about your storyboard finale to the workshop and thinking about the story part more than the board a little bit and just thinking of the value of having a narrative of your course that’s been generated in this collaborative way that allows for connections to be visualized, but also thought through. You mentioned that part of what you talk to with faculty in your DUO workshop is the idea of assessment. And you’ve encouraged faculty to use both qualitative and quantitative assessments for students. Would you be able to talk a little bit about those and maybe provide some examples of qualitative and quantitative assessment measures.
Darina: It’s funny because like when I initially mentioned talking about this in this podcast, this was before all the COVID-19. And everybody’s obsessed with assessment right now, because they’ve all had to come up with new ways of doing things. But say, before that, if you just go with my standard DUO workshop, what I talk about with people, that’s one of their big concerns, you know, they have a face-to-face exam that lasts for two and a half hours. And now they’re wondering, “Well, how would we do that online?” Or, you know, “That’s fine that you have that kind of activity, but how would I do this type of activity online? …and so on. So, there’s a lot of talk about different types of activities, and usually people kind of latch on to different ones. And then the next question is, “Yeah, but how would I grade that? That’s fine for you, Darina, you have 30 people in your course I have a hundred and fifty…” and stuff like that, or they don’t know yet that there are rubrics out there for grading podcasts, for grading discussion forums, for grading reflective blog entries, for portfolios, for everything. So that’s one of the things I try and highlight for them. I say “Look, actually there are lots of universities that post really good rubrics up there. So you don’t even have to come up with your own or you can just adapt one of these. So they’re always thrilled to find that out, because they think that all has to come out of their own minds, and that they have to kind of devise what an excellent podcast is versus an average podcast. So I showed them those examples. And I have my own list of resources online, where I have grading rubrics, a page for that as well. So I highlight some of those, but also, then I tell them to kind of look at what, and most people don’t know this, their learning management system, Moodle, Blackboard, whatever it is that you’re using, they all have learning analytics data that’s available to instructors. And people have heard that, but they never browsed around their interface to find it. So, I would just show them examples of the kinds of things you can find out about your students. So, let’s say you decide you want to have 5% or 10% going for online participation. They’re thinking, but how would I grade that? You know, that’s gonna be really tough. So I say initially start off with quantitative data. Look at your LMS, you can click on an individual student’s name, you can see how often they logged in, how many words they wrote in each posting, for example, how many files they accessed, all of those kinds of basic quantitative things you can use. And if it’s a low number of marks that you’re giving for online participation, the quantitative might be enough. So that stuff that they might have manually done, if you don’t highlight it for them, they would have possibly copied and pasted all the postings and done a word count inside Microsoft Word. So it’s really important to show people those kinds of things. Qualitatively, then, you can still use the LMS analytics data to do things like, well in my own LMS the one we use is Sakai, but it’s very like, say, Moodle, and what do you use in your university?
Darina: Blackboard, okay. It’s very, like they’re all very similar anyway, so I can display students’ forum postings in context. So it’s one thing to know that John posted the required 100 words last week, but were John’s hundred words relevant to the topic of the forum? Was John actually answering the question that Mary asked him? So you can expand and view them in context so that you get a bit more qualitative analysis now, so you can tick the box that he’s done 100 words, but now you can see are they 100 useful words or relevant words or whatever. So that’s one kind of qualitative way of looking at it, to see do they post in the right forum, for example. You know, again, sometimes people can be very strategic about how many words they write and where they post them, but they might not know that you’re going to analyze it to that level and you’ll spot that they’re actually just repeating themselves or waffling or whatever they might be doing. You can have more advanced heuristics as well. So for example, qualitatively you might be interested in did students offer solutions to other students’ problems on the forum. So you know, I’ve said, “I’m really frustrated. I’m trying to do this assignment. I don’t know how to solve x, y, and Z,” even though there’s not officially any marks going for it, you might offer to give assistance to a classmate, for example. So that might be something that you build into your heuristics. Have they reached out to their classmates? And did they acknowledge other people’s contributions and so on, so all of these qualities of things could be incorporated into your rubrics and that’s just for kind of LMS participation. And as I said, then you have lots of rubrics for other tools they might use, like if they created a podcast, there’s rubrics for excellent, average, poor podcasts and so on. There are more advanced techniques then that some people who are interested in analytics research, myself included, like you can do things like cluster analysis or decision-tree classification. And so a lot of talk of, in the analytics field, is about trying to identify problem behaviors early on. So has a student logged in week one, or did they not log in until week six. If they don’t log in until week six, and they don’t access the week one lecture materials, are they more likely to get a bad mark versus those who do, and so on. So there are more advanced techniques you can use to kind of identify student behaviors, patterns that can inform when you intervene, or who you reach out to and so on. Obviously, the qualitatives are more time consuming. So it kind of depends on how many marks you’re giving. So I mean, I would usually have 10 to 15% going for participation, online engagement in e-tivities, and so on. So I’m looking at the quality of what they’ve written, but I’m also looking at the quality of their engagement and interaction with other students as well. Did they stay on topic? …all of the kinds of things that you would normally assess when students post something, but then quantitatively as well, I’m constantly keeping an eye to make sure that all my students are logging in when they should or, you know, if I haven’t received an assignment from John in a while, I’ll check to see has John been engaging with the LMS in the last few weeks? If not, maybe there’s something else going on with John that I need to follow up on. So there are some examples of the kind of quantitative qualitative techniques I would use with my students, it’s all very time consuming, but there is great data there, if you know where to look, you know.
Fiona: I feel as though time is the specter we keep coming back to in many ways. And it makes sense that the assessment opportunities you’re talking about can both save time in certain circumstances, but also involve time in other circumstances. And I know from the last few weeks, the one constant that seems to be coming up again and again, in conversations with faculty members is how time consuming this shift has been. And I know these are unusual circumstances, but do you talk to faculty about managing time and workload when it comes to online teaching?
Darina: I do. I think one of the most important things that I do in the workshop is I’m honest about what it involves. So like I’m teaching online for 13-14 years, and I still spend a huge amount of time every semester on like, I’m not just uploading the same podcast from last year I’m re-recording my podcasts. I’m spending hours in the forum at the start of the semester, setting up my discussion forums and the titles and the topics and changing privileges. Looking back over my notes for what I did wrong last year to make sure I don’t make that mistake this year. Coming up with activities and so on. It’s really time consuming for me and I have relatively small classes, even though they would be large online classes for our university, talking about 20 30, 35 students maybe, at most, taking a course it is really time consuming, but I suppose I’m lucky in that we fought over the years in our particular program to have it recognized that the work we do online is equivalent work to the work you’re doing on campus. So I might only have three hours of teaching on campus a week. But I’ve spent the whole rest of my week in my office, doing podcasts, answering questions on forums, doing live chat sessions, and so on. And that is recognized. So that’s something that I kind of emphasize when I’m talking to the program team, that this needs to be recognized. We’re luckier in recent years, I’m actually kind of envious of my colleagues now. Because the programs that I teach online, we’ve never had any ed tech support, educational technology support, everything that was done, all my courses are all done by me. You know, I don’t have somebody that I can say here on my slides, add my audio to it, [LAUGHTER] you know, that’s like a pipe dream for me. But in more recent times, like in the last two or three years in our university, there’s been more educational technologists hired for the individual faculties and in some cases, some programs have got their own educational technologist, which means then that person does a lot of that hard graft, you know, if you’re having technical problem and you’ve written your slides and you have your notes, but you don’t have time to fiddle around with software, there’s somebody who can do that fiddling around for you. So they have that advantage, I suppose. And in some cases in the bigger programs, they have tutors that they can hire if the number of students increases. Again, that’s something I’ve never had. That’s what the Open University has. That’s why they’re so good at what they’re doing as they have small groups dedicated to individual tutors, and they look after them all the way through. So the subject matter expert or the lecturer, the professor, doesn’t have to do all of those other things as well. And that is kind of what we should be doing. That’s the model we should have. But it really depends on the group I’m talking to. You know, some groups definitely have more resources than I do, for example, but I do emphasize to them that this isn’t easier. It’s equivalent, if not harder to what I do face to face. I mean, obviously, certain things have gotten easier than they were. But I’ve gone through lots of trials and tribulations and things working and not working. And I try a little tweak of an assignment and it all goes horribly wrong. And you say “Right, I won’t do that again.” And I think it’s really important that staff hear you say that because they assume because you’re teaching online for a long time that everything is easy for you, and it’s not, and it will never be easy, actually. It’ll never be easy. It will be very demanding, time consuming. But if you do it well, the rewards you get from your students and seeing what they produce, and so on, always makes up for in my eyes anyway, even though sometimes I think, “Why am I doing all these things, I don’t have to do it,” You know, there’s an easier way of doing this, I could do that. I could just throw out the podcast from last year and not customize it to developments this year. And they might not know any different. The students might not know any different but I would know, you know, and I would be worried that I made references in last year’s podcasts that are no longer relevant this year and things like that. So I probably make certain things more complicated than they need to be. But that’s just the way I am. [LAUGHTER] I’m a sucker for punishment.
John: I think I am too. I was thinking back when you were talking about how long you’ve been doing this. I started teaching online 24 years ago.
John: A lot of the tools have gotten better than they were back in those earlier years. But I’m still finding it takes at least as much time as it did when I first started back then.
Darina: In a way, you see you’ve so many more choices now. That’s a problem as well. I mean, it’s great to have a choice of technology in the sense that if you really dislike Microsoft Teams, for example, you can go and use something else. But on the other hand, you can invest a huge amount of time in that other tool and realize it still doesn’t do what you want it to do. So, like the other day, I was just trying to upload a video file that I recorded on my computer, my laptop at home, and it was on QuickTime because somebody told me the other day, you know what, if you have a Mac, you can use QuickTime, you can record yourself whatever and I said, “Great, I’m gonna use that now for this,” I had to install Adobe Premiere Rush to convert the iMovie to mp4 then when I played it, I discovered that my voice was not synchronized with my video. So then I had to download Adobe Media Encoder and convert it that way and then I had to do one other thing to be able to upload that video clip and that was a seven minute video clip. I’m thinking, I am pulling my hair out and I’m an expert and there’s “what is everybody else going through?” I think it is really important that people know that I have those moments as well. Because I do think when you’re encountering challenges if you know somebody else encounters those challenges, it makes it a bit easier.
Fiona: For sure.
John: Now that we’re nearly done with our spring semester, what should we think about in planning for the fall?
Darina: I have a huge concern about how we’re going to fix things that have gone wrong in the last few weeks. I’m usually concerned, like, I don’t know about your university, but I imagine it’s probably the case in most places that a lot of allowances were made for faculty to use whatever they could manage, and do whatever they can to come up with whatever assignment can reasonably assess the learning outcomes, and we’ll figure it out afterwards. You know, what, we’ll figure out how to grade them later, or whatever. And it was amazing, and also really exciting to think that universities could be that flexible when it really was needed. You know, it was really a big crisis, and it was needed. At the same time, I’m thinking right now, and I have seen other commentators mention this too, so it’s not just me, but a huge amount of people now think they’re online teachers. And this is where the language we’re using here is kind of important. This is where I’ve seen people refer to emergency remote teaching like you did, Fiona, at the start. I like that reference to it, rather than saying, I’m an online teacher because I’ve done a podcast or I’ve done whatever. We know, those of us that are teaching online, that to do well, it takes years of work. And every year it takes loads of work. It’s not just a thing you learn once and you’re sorted, it’s kind of another job on top of your teaching job. And I’m really concerned that the allowances that were made recently, that people will carry those through to the fall, and that when somebody says to them, Well, you know, I know we said you could use let’s say, I’m just picking Zoom as an example, just because it’s in the media all the time at the moment. I know, we said you could use Zoom in spring, but you know, it’s not safe. It’s not secure or whatever, you’re not allowed to use zoom anymore. And you will have a lot of people aggrieved that they’ve invested time and effort in technology that’s no longer supported. Or you have students complaining that technology didn’t work the way it should have, or they couldn’t do their assignments from home because you insisted on a particular tool that they didn’t have access to or they had to purchase or whatever. How are we going to get people to take a few steps back and say, “Okay, that was okay then but it’s not okay anymore.” And we’re now going to find time, somehow, between now and September, let’s say, to fix those things and to correct those problems that crept in along the way. And so that’s why I’m talking to my own faculty even, because apart from the whole university, even within my faculty, there’s lots of people to look after. So, within my own faculty, I’m thinking they need to be doing DUO workshops over the summer, you know, they need to kind of know, “That’s fine, you tried out these things, and you now know more than you would have six months ago if this hadn’t happened. But there are problems with some of what you’ve done.” There are brilliant things and they’ll have discovered things that I don’t know about myself and so on. I’m just worried about how would that conversation happen? How would we not sound like we’re being critical of people who did their best under exceptional stressful circumstances? So that’s kind of one of the research papers that’s floating around in my head at the moment, that the language we use will have to be very careful because they’ve been given permission to do it, whatever way they can manage. But does that mean we just let them do whatever way from now on? I don’t think so.
John: There wasn’t much choice at the time, but now there’s time to plan and I think on the positive side, faculty who have been teaching the same way for several decades, all of a sudden had to try some new things. And that might leave them open to think about how they might be able to use these tools more effectively in the future. And if it’s framed that way, I think it could be seen as a positive experience where if everyone gets together and talked about what worked, what didn’t work so well, and how those problems could be addressed, it’s an opportunity for people to bring in more effective tools, however they’re going to be teaching. And we don’t really know what that is going to be like for the next semester or two. But at least it will give us more time to plan and more people to reflect on their experiences and perhaps learn from those experiences so that it may not be seen as a constraint if there’s direction saying this tool was used, but perhaps there’s a better way of doing that, because I think everyone right now is questioning how this is working and what could work better.
Darina: Yeah, we don’t know yet how well it’s going to work. You know, our students still have to submit their assignments. We still have to grade them all. You know, we could be in for a big shock. We could be in for a pleasant shock. [LAUGHTER] We could realize that actually the students did so much better because of this other alternative way of teaching them and assessing them. So, there’s lots of exciting things we could find out yet. But it’s how we’ll have that conversation, how we’ll frame it, that will be interesting, because there will be people who did not want to use technology, who clearly never had plans on using technology who have been forced to use it, they’re going to look for any opportunity they can to dismiss technology as all useless and pointless and it doesn’t work when you want it to and so on. So, that’s one challenge. And then there’s all the other people in the middle who actually committed to doing a lot of really great things or they did their best, but maybe we might have to correct some of that. So it’s like giving constructive feedback to your students. This time, it’s to your colleagues. [LAUGHTER]
Fiona: It is. We’ve all become learners anew, whether we planned to or not in this semester.
John: By the time this podcast is released, some of those answers we’ll know about from how students have done because we’re recording this near the end of the semester in both of our institutions. We’ll have more answers,I think, by the time this is out
Darina: Great, interesting times ahead.
Fiona: Normally this podcast ends with the question: What next?
Darina: So if it’s what next for me, I have some papers planned when I get past some other deadlines about how institutions have supported staff and how would they could support them better moving forwards. Another thing I’m working on is, there’s a conference that I go to most years in North America, it was supposed to be on in Georgia in July, and like every other conference it has been moved, well, this one has been moved to virtual. So, I’m on the team. That’s the Professional Communication Society conference. So we’re now looking at what technology could be used to replicate a face-to-face conference, and how are we going to have breakout rooms and coffee breaks online and all those kinds of things. So that’s kind of exciting. It’s a lot of work for the team, but some work won’t be required, you know, that was required before when it was going to be a face-to-face conference. So it’d be great to see how that turns out in terms of moving that conference online, just this one year, hopefully. So they’re kind of my immediate things that I need to kind of get working on that research soon while it’s still fresh, and while people still have opinions about things and feel passionate about it, and so on. And in terms of DUO, a lot of resources were developed in the last few weeks to deal with the COVID crisis. So maybe some of DUO can be taken out, and we can point people to those other resources that were all developed under huge pressure, but really good resources were developed. So I might be able to repackage it a bit or maybe make it a bit shorter, or have the face-to-face component being just what needs to be done face to face and so on. So I’ll have to rethink that in the next few months as well, you know?
Fiona: That’s incredibly interesting and vital work. All the best.
Darina: Thank you very much. I appreciate that.
John: Thank you. It’s been great talking to you again, and I’m looking forward to hearing more of how things are going in the future.
Darina: Great and the best of luck to both of you as well in your wrapping up your semesters and I hope everything goes according to plan.
Fiona: Thanks a million.
John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.
Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Brittany Jones and Savannah Norton.