156. Social Annotation

Show Notes

• Hypothesis
• Darby, F., & Lang, J. M. (2019). Small teaching online: Applying learning science in online classes. John Wiley & Sons.
• Family Educational Rights and Privacy Act (FERPA)

Transcript

John: Do you struggle to get students to complete readings or to deeply discuss readings in an online environment? In this episode, we discuss how a social annotation tool can engage students in conversations with the text and with each other about the text.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted byJohn Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Margaret Schmuhl, an Assistant Professor in the Department of Criminal Justice at the State University of New York at Oswego. Maggie has also been working with us as the facilitator for our second cohort of faculty in the ACUE program here at Oswego. Welcome back, Maggie.

Maggie: Hi. Good to be back.

John: Good to see you, Maggie. Our teas today are:

Maggie: Well, I’m having an orange spice herbal tea.

Rebecca: That sounds nice and warming.

Maggie: It is. It’s very cozy for a cooler fall day.

Rebecca: I have Scottish breakfast, which apparently is my new default tea.

John: I’m having a ginger peach green tea, which I’ve been having a lot recently, too. We’ve invited you here to talk a little bit about how you’ve introduced the social annotation tool, Hypothesis, in your classes this summer and this fall. Could you talk a little bit about what prompted you to adopt Hypothesis for your classes?

Maggie: Yeah, so a couple of reasons. First, in my spring semester classes, I like to think that my students are very open and honest with me, especially when I asked them if they’ve done the readings. And for the most part, I get a very resounding “Nope,” like, “…haven’t done them.” And I just take a deep breath and carry on with the class, knowing that none of them have done any of these readings. And so, after feeling a bit frustrated for a long time with my classes not actually completing the readings that I’ve carefully curated for the class, I was looking for something that could keep them accountable to those readings, I had had colleagues who would assign reading summaries and such, and that seemed great, but I wanted to be able to see something, I wanted to see how they were understanding the reading. And I think John, you actually inspired me to consider Hypothesis because you had used it in some of your classes. So, when we had been meeting with the ACUE cohort, it was super interesting to me. So, I took a workshop, I think through CELT, with the Hypothesis representative, and it seems like a super easy functional tool. And I really liked that it was embedded right into Blackboard. So, it wasn’t necessarily throwing a lot of new technology out at students. And they didn’t have to create accounts, they didn’t have to go to a third-party website to use the annotation tool. It was something that I could throw right into the module, and all they had to do was click on it and start writing. So, it’s simplicity was super accessible, I think, for my classes.

Rebecca: One of the things that I think is really interesting about Hypothesis as a tool is that, if you’re using it for accountability purposes, it ends up being more of a dialogue with the readings rather than what can be perceived as busy work of summaries, or some of these other things that either just feel annoying to do, or annoying to read as a faculty member. And the same thing can happen with quizzing, too. It’s another thing to grade or another thing to look at. Sometimes that can be really effective. But, it’s a nice different way of doing it. And I think it’s really enjoyable as a faculty member to see how students are looking at materials.

Maggie: Yes, absolutely. Because with reading summaries, there’s an easy way out for students just to like look for a summary on the reading. But when you really want them to start asking questions about the reading, this tool helps them be able to locate certain segments of the reading that they may not have understood or something they found particularly interesting, or were able to connect it back to other classes or other information that we’ve talked about in just a really fluid way. So, yeah, I absolutely agree. That’s one of the big benefits I’ve found with this tool.

John: Did this replace some earlier activity? Or was this a new activity that you introduced in your class?

Maggie: So, actually, I haven’t had it replace anything. This has been more of a tool I’ve used, in addition to discussion posts, and so forth. But now that I’ve had a couple of courses under my belt with this, I do think I’m going to move towards replacing discussion posts. The discussion posts, from students’ feedback, they see discussion posts as just answering the questions that their professors wants, whereas the annotation allows them to pull out things that are interesting to them. And they’re able to engage, they think, in a more natural way than it is on the discussion posts. So, they’re reading through each document and, along the way, they see what their classmates are writing and where’s discussion posts, you have to go back into each of their classmate’s forums to see what they have written. And it seems a little more, I guess, artificial in discussion posts to just kind of comment like,”Oh, I agree. Here’s what I also wrote.” And it seems like a much more casual way of interacting that’s more akin to what we have in the classroom when we are face to face.

John: Do you think it encourages deeper and closer reading of the texts?

Maggie: Oh, definitely. I think a lot of my students have given me the feedback that they’re not just skimming the text anymore. They’re not just looking for the main findings or the points to summarize, but they’re actually considering each part of the text. And as they’re considering each part of the text, they’re using this tool to communicate to me their interpretations of the readings, but also the ways it connects back to their own experiences. So yeah, I found it to be quite invaluable for that kind of engagement.

Rebecca: Do you see it as a way to facilitate or to encourage community building around the content?

Maggie: Yeah, so I think that when I do replace discussion posts, there will probably be a little bit more of that. But, I already see where students are using the reply function. So when they create an annotation, they have an option to reply to another classmate’s annotation. And so I see dialogues begin to unfold between three or four students, whereas in discussion posts, if I tell students: “Okay, engage with someone else on their work,” they’ll pick one person, and they’ll respond to them. But again, it almost becomes like a text message to each other. And in a way, it seems, I think, more natural for them to just quickly write back and forth in response to each other’s questions, as opposed to having something a little more drawn out in a discussion post.

John: So how have students reacted to the use of Hypothesis.

Maggie: For the most part, my students have really enjoyed Hypothesis. Of course, there are some students who find it to be a little tedious. But, for the most part, when I asked them, whether they prefer discussion posts or their annotations, most of them prefer the annotations. They felt like they wouldn’t have completed the readings in a systematic way if it weren’t for Hypothesis. So, they’ve pointed to this level of accountability that the tool gives them to those readings, they actually feel like they’re retaining more information from those readings because of the way that they’re engaging with it. When I have a synchronous session, and we are diving into some of the issues that these readings bring up, engagement in those discussions are much greater than they used to be. I used to feel like I had to tailor questions so if they did the reading, or they didn’t do the reading, they could still participate. But, now I feel like we can actually dive into some of the nuances of that text in a way that we just couldn’t do before, when they didn’t do the reading.

John: it’s a whole lot easier, I think, for students to actually read the text when they have to actively be in the text to do their comments. So, it’s a little more difficult for them to evade doing the readings.

Rebecca: One of the things along those same lines that came up in a reading group discussion that we were holding yesterday was the idea of accountability and faculty talking through the concerns that they had about students being held accountable for things and that they seemed less accountable, or that employers have said that recent graduates seem a little less accountable than they had previously. So, it’s interesting to be able to use some of these tools to encourage accountability. But also, I think, it mimics a more professional experience about how you might engage with materials professionally. And so maybe it just feels more authentic, and therefore it’s easier to be more accountable.

Maggie: Oh, I love that, because I do think that, at least in the context of our careers as academics, we use annotation tools like this all the time, whether it’s in Google Docs, and we’re making comments and we’re working with co-authors and other faculty members on different projects and such, I definitely see where we use those tools and those skills that it’s a good skill set to encourage students to build.

John: Since we have this integrated into Blackboard with an LTI, it’s possible to do grading in the LMS. Have you been grading students on their participation?

Maggie: Yeah, so when I first started using Hypothesis in the summer, I was grading them, but at the time, the grading wasn’t embedded right into the Hypothesis platform. And so I was grading on a separate rubric and grading them sort of apart from each other. But now that I’ve been able to use the grading function right within, it makes grading much easier, because I can simply click on the student’s name, all of their annotations, and all of the replies that they’ve given to other students will show up right there so I can review them, give them a grade, move on to the next students, and it automatically loads right into the grading center. And when I’ve talked to students, they actually, not so surprisingly, said that if it wasn’t graded, they probably wouldn’t have done some of those readings. So, it certainly made me feel better by including this as a graded portion of their final grade, because I think without that incentive, they may not have engaged with it. But, I will say that I require students to do a minimum of three annotations, and I’ve several students who are doing 7, 8, 10, 12, just depending on the reading and their topic of that reading. They seem to be willing to move above and beyond that minimum standard, which I think is pretty cool.

John: I’ve seen exactly the same thing, that even though I did have some minimum specified, most of the students were doing2 to 10 times as much as a minimum when they were using Hypothesis.

Rebecca: Perhaps that attests to, in both of your cases, of actually helping students establish a habit of how to read or you get in the habit of using that tool to read and then you’re reading the whole document anyway, so you just annotate the whole thing.

Maggie: Yeah. And I was afraid that, as students were going through the readings, they would basically stop at the first page and put all of their annotations right on that first page, but I haven’t looked at all of their submissions. We do annotations every week on a reading, and so I’d have to pull it all out and compile that data to see what kind of patterns emerged. But, it seems to me that they are doing these annotations throughout the entire reading, they’re not just going a couple pages in and then being finished with it. Of course, there are students who are like that, but when I’m scrolling through that document, and I get to page 17, there’s still annotations there, which I find encouraging.

Rebecca: Probably, once a couple of students do it, and start modeling that, that becomes the standard of behavior, then people realize, like, well, even I’m not gonna read the whole thing, you got to read parts of the thing. [LAUGHTER]

Maggie: Exactly, yeah. And I’ve had a lot of student feedback that they like seeing what their classmates are writing about, because it’s given them insight into their perspectives on the reading and how it connects to their lives and their experiences. And I think it allows for an engagement in an online platform that I typically tend to enjoy in a physical face-to-face classroom.

Rebecca: And reading can seem like a really lonely activity generally. And if it’s difficult reading, it can feel extra lonely, especially if it’s asynchronous. So it seems like a good way to connect people through reading, which is not a way we generally think about being social.

Maggie: Absolutely. I’m teaching a class on the death penalty this semester. And so there are some Supreme Court cases that they are reading, and they are 200 pages long. Now, I required them to read one opinion and one dissent from the respective justices that are writing those cases. But, with that, they’re not so scared with the 200 pages of reading. They’re not just like totally shutting down and not doing it. They’re still engaging with the material, which is more than I can say wa’s happening in the classroom when we were face to face.

Rebecca: Can you talk a little bit about your future plans in using Hypothesis moving forward? Yeah, so another way I’ve used Hypothesis is for peer review, and I know that John has used Hypothesis for peer review too, so I think he probably has some comments as well on this. But, in my classes that are writing intensive, I like to incorporate some kind of a peer review feedback, because not only are we requiring them to write their papers with peer-reviewed research articles, and so forth, but I want them to understand what that process means by engaging in that. And I think when some of that feedback comes from their peers, they start to feel like the feedback isn’t so over their head, that that feedback is something that they can accomplish, and something that they’re perhaps a little less afraid of, than when it comes directly from me. So, what I really like about Hypothesis is I can create all of their submissions into a PDF, I can assign each student to review a certain number of papers, I typically tend to assign each student a particular paper so that not one paper gets all of the annotations over another. And I give them a requirement of making at least 10 or 15 comments on the paper. And before I know it, it’s actually eased up a lot of work on my own feedback, because they’re catching the things that I now don’t have to spend a whole lot of time, telling them to capitalize a particular word or explaining how to use commas in a particular sentence. And so it’s been really nice, because they can simply highlight, they can make little comments about when a sentence doesn’t sound right to them. And then I can come back, overlay my feedback over atop of it. And then the students have all of that in one place when they go to work on their final drafts and incorporate that feedback. So yeah, as far as planning in the future, I do plan to continue using that as a peer review feedback as well as in my readings. I’m teaching online courses next spring as well, and so I plan to go get some good scans of my readings so that I can allow for them to become annotatable. Is that a word? annotatable?

Rebecca: It is now. [LAUGHTER]

John: I had a similar experience when my students did peer reviews, and they really liked that ability. They like that they could see the comments, they could react to each other. They could reply to each other’s comments and sometimes they’d disagree about whether a change should be made and there were some really good discussions embedded right in the text, right at the point where it was occurring. And then I’d come in, and sometimes I’d say, “Well, you know, I think maybe the original actually worked pretty well here” or something similar. And it did make my work quite a bit easier, because the students were doing a lot of the basic editing. Initially, a lot of the comments were primarily grammatical. But after the first time we did that, we talked about how it worked. And the students were saying it would be nicer if we could get more substantive comments, actually suggesting ways in which we could improve the substance of the paper. And I was going to suggest the same thing, but they brought that up themselves. And they seem to have much more of a sense of ownership of the review process. And that worked really well. Did your students have any concerns or negative reactions about the use of Hypothesis?

Maggie: Yeah, so I have found that the tool seems to be much better suited for my upper-division seminar style classes. I think that, even though I find it to be really useful for my introductory survey courses, the students did not like it as much in those introductory courses. But, it’s hard to know exactly why. Some of them pointed to not wanting to actually engage with other people, which I kind of have to laugh and move on from those comments, because that’s part of the process of these courses, is engaging with other people. But, I do wonder if it’s between discussion posts, and low-stakes quizzes, if adding annotations in a lower-division course becomes a bit overwhelming for them. But, I do think that the benefits of being accountable to those readings and having better discussions because of those readings probably outweighs some of that concern. But, I have had some other student feedback. They didn’t like that there wasn’t specific feedback available for the grading function. So, when you grade in Hypothesis, you just give a number grade, it doesn’t allow you to submit a rubric to indicate different levels of content or grammaticals or whatever it is you want to grade in a rubric form. So I did have some students who wished that there was some more specific feedback available for that. But, it did make me wonder, and it kind of reminded me of some of the reading we were doing in our reading group, the Small Teaching Online, when they were talking about specs grading, I thought that these annotations might be a really good place for that… to incorporate some all-or-nothing kind of grading. But again, with low-stakes grading, it’s not a significant portion of their grades. So I guess that’s just one thing to keep in mind, is that sometimes students want some of that detailed feedback. And that tool doesn’t necessarily give you a place to comment on their annotations, except within the annotations, you certainly can comment by replying to their annotations, which I do.

John: But you don’t want to make it public because of FERPA, and so forth. But you always have the option of not using the grading feature within Hypothesis and just adding a column to the gradebook, attaching a rubric to it, and then just evaluating each student… looking at their comments using the same technique, and then just going to the rubric and adding that to the gradebook. So, there are workarounds.

Maggie: Yeah, and I’ve done it that way as well. That just brought to mind like, maybe I need to go back to using that method for some of these classes. The other thing is that sometimes scans aren’t the best. I do think it’s really better to use articles that are already searchable. Sometimes when you’re scanning material, making them searchable and accessible, is difficult. There’s really good scanners and technology that can help us with that. But, sometimes the students are highlighting certain segments of the text, and it’s jumping to other areas of the paragraph. And so I think with that it takes a little bit of time to complete. I’ve also had some students saying that they don’t like to highlight over other students highlights, but I think that’s more of a personal preference. So I just encourage them to reply, then, to those students’ annotations so that it’s about the same material. And that pushes them to engage with each other a bit. But while there’s certainly some areas that students want different features and improvement on, they overall very much like using this tool in Blackboard… at least that’s been my experience.

John: I suppose one nice side effect of this is the more people who use this, the more it will encourage the creation of accessible PDFs because basically the issue is that you need a text layer that contains all the text where it’s supposed to be basically.

Rebecca: Yeah, and if it’s a fully tagged PDF, it works better in Hypothesis than just an OCR’d PDF, for sure.

Maggie: Yeah, that’s fair. I think it is a great tool for faculty because it really does push them to make all of their readings accessible. So, in terms of accessibility, it’s a good way to push everyone to make their materials accessible.

Rebecca: We always wrap up by asking what’s next?

Maggie: Well, so in terms of using Hypothesis, I’m teaching some upper-division seminar courses next semester online, and so I plan to keep using this for both peer review and for reading comprehension. I’m hoping also that one day, we’ll be able to use inclusive access texts with Hypothesis so that we can move through some of the main readings, especially if we have a textbook, where students are able to annotate together.

Rebecca: I would like to be able to annotate images.

Maggie: Right. Yeah.

Rebecca: Well, thanks so much, Maggie for joining us. It’s always a pleasure to chat.

Maggie: Yes. Thank you for having me.

John: Thank you, Maggie. It’s great talking to you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]