39. Video feedback

Have you spent hours writing comments on student papers only to see them end up in the trash can as student file out of class? In this episode, Dr. Jessica Kruger, a clinical assistant professor in the Department of Community Health and Health Behavior at the University at Buffalo joins us to explore how providing video feedback may help motivate students to hear, see, use, and understand your feedback.

Transcript

John: Have you spent hours writing comments on student papers only to see them end up in the trash can as student file out of class? In this episode, we explore how providing video feedback may help motivate students to hear, see, use, and understand your feedback.
[Music]
John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today our guest is Jessica Kruger, a clinical assistant professor in the Department of Community Health and Health Behavior at the University at Buffalo. Welcome, Jessica.

Jessica: Thanks, glad to be here.

John: Glad to have you here. Today our teas are:

Rebecca: Jessica are you drinking tea?

Jessica: I am drinking tea.

Rebecca: Yes! Score one for us. [LAUGHTER]

Jessica: Today, I chose actually a chamomile tea from Yogi, and the Yogi tea proverb today is “not sharing is not caring.”

Rebecca: Sounds perfect. I’m drinking chai today.

John: …and I have blueberry green tea.

Rebecca: So, Jessica, can you tell us a little bit about the types of courses you teach?

Jessica: Sure. I teach in a brand new undergraduate public health program, and the courses that I currently teach: introduction to public health, which spans the gamut of information… anything can be public health… and we talked about it all in that course. I also teach a class called Social and Behavioral Aspects of Health and Methods and Mechanisms in Public Health.

Rebecca: What are your class sizes? or the kinds of things that you do in classes? What do they look like?

Jessica: My smallest class is 75 students and my largest is about 250. This upcoming semester it may be over 300. My class, when you walk into it, you’re going to hear music playing at the very beginning (probably pretty loud) to start the class. After the music stops, we get into the material, and if the students can guess it, the song usually actually relates to the topic that we’re covering, or something that’s going on in the world. I think my most apparent connection between music and academics and the world was, we had an ice storm here in Buffalo and, of course, the song was, you guessed it, Ice Ice Baby. [LAUGHTER]

John: What is the purpose of the music to start up the class.

Jessica: I like to get people comfortable and it usually fits with my personality… pretty fun… pretty open… pretty outgoing… and so when they come into class they should sit there, get acclimated to the environment, and then get ready to go for the day. Once the music stops, they know to stop their chatter, put away their phone, and get down to business.

John: Do you talk about how it relates to what you’re going to be doing or is that something that just flows naturally?

Jessica: Sometimes it flows naturally and other times the students will say “So, why did you play that song?” and it takes them a moment and I was like “didn’t you hear the reference to cigarette smoking in it? Obviously, we’re talking about tobacco today.” [LAUGHTER]

Rebecca: One of the reasons why we invited you to talk to us today is about how you give students feedback. Can you talk about the kinds of assignments you give students and the kinds of feedback that you want to be able to give them?

Jessica: Yeah. In my class of 75, the small one, we get to know each other very well. But, in that small class, we actually write a paper… and this paper isn’t just a 15 to 20 page paper. They do it in sections, so the students never know in the beginning… and I actually never give them a page number… of the paper that they’re going to write… and it’s all about the social and behavioral aspects of a specific health topic. For example, some students write about maternal mortality… infant mortality… tobacco use… and they have to describe the public health issue, but also what behavioral,or what social parts of the puzzle, fit into this health issue. They’re writing this paper in small chunks. They’re turning in about a page every other week, and with that they get written feedback on some of their papers. The typical written feedback where you say “Oh, you know, you need to cite this. You need to add an apostrophe here” or “Hey, could you explain this further?” But what I also do is, twice during the semester, as they’re putting their papers together to build it into a larger document, I provide them with video feedback… and this video feedback is something new. I tend to make videos to describe things for friends, families… and I said “why not just make videos for my students?” They can see my emotion… my face… how tired I am after reading all 75 papers, and they can also see the excitement when I say “Oh my goodness, that’s such a good point. I’m so glad you made that.” They can also hear, in my voice, how I’m feeling towards their paper or what’s going on, and I can even show them “This is actually how you do a citation” because they can see my screen.

Rebecca: That sounds really great. I can imagine that modelling behaviors or modelling how you think through writing could be really useful for students.

Jessica: It seems to be effective.

Rebecca: Can you talk a little bit about how it works, what your process is, in terms of how you’re recording it?

Jessica: Sure. Students turn in their paper via Blackboard and when I go in to grade the papers, I pull it up and I also pull up a program called Panopto. Panopto is a screen capture software similar to Screencast-o-Matic and it allows me to take a screenshot or a screen grab of my computer screen, and also you see a little picture of me in the corner. I enable that just so students know that “Hey, I’m there.” In a class of 75, you may not get that personalized attention in class, but you are getting a video made directly by the professor with your paper.

After I pull up the paper, I start going through it. I usually scroll through it once or twice to point out and start thinking and distilling my thoughts about it, and then I go as if I was just reading the paper. I start going through and saying “Oh, you know, you need a transition sentence here” and highlighting it or highlighting an area where I say “I think you need another citation here” or “That’s a really great point, I’m really glad you added that. Could you also add some more?” and so this allows students to: 1. visually see their paper and any sort of modifications they need to make, but also to see that I’m actually reading their paper and giving them some hopefully quality feedback that they can help improve upon.

John: There’s a lot of research out there where people are more likely to misperceive a negative tone or read a negative tone into written text, and they’re much less likely to do that when they actually see facial expressions and so forth. Have students taken the feedback a bit more appropriately?

Jessica: I actually think that’s a good point because I assess this and one of my students comments really struck me. They said: “I like that I can watch you read my paper and see where I went wrong. I feel videos helped increase the trust with the student and the professor. Sometimes written feedback makes you feel attacked or that you feel the professor’s just being unfair and biased towards your writing.” So, I feel that this does help break down some of those walls, and they can see the emotions, but also realize I’m also using a rubric to grade. It helps them better understand the rubric and how I’m figuring out their score in a fair way.

John: So, do you grade the whole paper with the video feedback? How long are these videos?

Jessica: So, because they’ve turned their paper in in multiple chunks in my class, I usually know what I’m looking for in each of their papers. I remember their topics. I know their writing styles, and so my videos for some of the students who are very good writers only last maybe about one to two minutes. For those who may need more correction and more additional support, I’ve had videos up to 10 minutes long. But the average is about 3 to 5 minutes.

Rebecca: Do you pre-read and then record or are you doing the whole process being recorded?

Jessica: I slightly pre-read, just so I’m not caught too off-guard. But, sometimes some of the most real and genuine comments come from when you see something and you’re like “Wow, I can’t believe they actually picked that up from lecture! That was awesome.” That point really struck a chord with them and they put that into their paper.

Rebecca: I really like that what you’re describing is annotating their papers but doing it live. Not only are you providing the feedback, but you’re also providing a model for having a conversation with a piece of writing, which is probably how we want students to read… which i think is really cool.

Jessica: I would agree. I never thought I would be doing this. It mostly came out of the fact that I have pretty terrible handwriting and my spelling is not always the best. So, instead of sitting down and typing out comments, which I found took me actually a lot longer than video grading, I feel like this adds that connection aspect, models some behavior, and also give some information that they can use versus my chicken scratch.

John: How do you share the information back to the students?

Jessica: I’m able, with Panopto, to get a live link and that link can actually be emailed to the student directly through the program. The student gets an email in their inbox saying: “Hello, there’s a video that’s being shared from Dr. Kruger. Please click on this link to watch it.” I also embed the link into our Blackboard shell, so that they can go back and look at it at a later time.

John: That’s a very convenient way of doing it. It’s much easier than having to render the videos and then send them out, and that is one of the nice features of Panopto. We use it at Oswego as well, but I don’t know of anyone who’s using it right now for this purpose, but it sounds like a great idea.

Rebecca: You’ve already talked about some of the advantages and disadvantages, but can you suss out a couple more details related to that?

Jessica: Most definitely. Because this was new to me and actually it’s quite new to the literature. Video grading compared to written feedback and audio feedback actually hasn’t been studied that much, and if it has been studied it’s been done in fairly small classes… not as small as mine, even smaller classes, about 14…16 students… and so I wanted to actually compare the students perception of the written feedback that I give them within this class and the video feedback that they’re receiving. So, they received written feedback about eight times in the class. They received video feedback about twice because their paper was assessed about ten times through the semester… and so at the end of the class I gave them a survey and I was looking at the quality that they felt (if it was better than written, about the same as written feedback, or worse than written feedback). their ability to understand information, the helpfulness of the information they were given, the accessibility… so, were they able actually get the information through the link that I provided… and also the ease for them to make changes within their paper. Overall, almost all of the students felt that this was better than written or about the same as written feedback. For example, 75% of the students that responded to the survey said that the quality was better than written feedback, and a quote that supports this from a student is that they said “Video seems to give a better chance to explain what you want fixed faster to say then write all the comments. So, I noticed you tend to include more information…” which was a neat observation of the student, even though they probably got their paper back covered in purple pen and stickers. But, they still felt there was more information from that video.

Rebecca: I wonder if some of the information that they’re perceiving is the facial expressions and stuff, so it’s not actually the commentary but how to interpret the commentary.

Jessica: Most definitely, and actually I didn’t tell the student that that’s a little face in the corner. I didn’t want to point it out, but actually most of the students noticed it, and the most common comments I’m seeing with all this information is “This information is really clear, but it also makes me feel really connected to my instructor.” One student even says “University of Buffalo makes me feel small, but the one-on-one interaction is really appreciated and makes me feel more personal and more connected to the instructor.”

John: So, it sounds like it benefits you by giving you a little bit less work, perhaps, and the students. Are there any problems in terms of students getting the feedback? Have they voiced any concerns? or are they as likely to read the feedback as when you give it back to them on paper?

Jessica: I wish I could sit down with all of them and say “So, tell me the truth, did you actually go through this?” What I’ve found is actually, within the students that I surveyed, about 67 undergraduates out of the 75 completed the survey, and 85 percent of them had actually watched the video feedback, which i think is pretty good. I’ll take 85 percent any day. With the papers, they continually got them back. So, I’m not sure if they actually read them. I hope… because they needed to incorporate those changes in their final paper in order to improve their grades. But that’s one thing with paper we actually don’t know. With Panopto, I can see if they’ve actually clicked on the video to watch it. Some students actually watched it multiple times, which I found to be interesting. They went back to review before they turn their paper in again to make sure that they made all those changes.

It’s actually neat. When I sent out links I would see the views spike up and then when their papers were due again I would also see the views spike. [LAUGHTER]

Rebecca: That seems like a good trend to have seen, right? Did you have any other interesting findings?

Jessica: So, with this very exploratory research in this, with a larger group of students, I was wondering about the users making change. When I’m making changes when I get comments from a reviewer. they’re usually on paper, right? Reviewer 2, you probably wouldn’t want video feedback from them. But, I thought it might be more challenging for them, because they’re watching a video and watching their paper, versus having a piece of paper next to them making the changes, but 81% of the students said it was better than the written feedback. One student said “It was much easier to understand what exactly you wanted fixed. With written feedback you have to first understand their handwriting and then understand exactly what went wrong, which is often hard.” So, I think that the voice and the emotions and the pictures helped them synthesize that information faster and easier than maybe that written feedback. I did have another student said something surprising. Although they knew I was going to do this in class, I had a student say “I got this email link about a video. I didn’t know that that was real, so I didn’t click on it. I didn’t watch it at all.” [LAUGHTER]

Rebecca: Sounds like a good excuse.

Jessica: Yes, exactly.

Rebecca: But that does bring up some issues related to accessibility and access. So, what do you do for students who maybe video isn’t the best method for them? Maybe they need a transcript or they need captions or something? Do you have a different method that you use for those students? or how would you approach that, if a student needed something like that?

Jessica: I always allow students to meet in person and discuss their paper so if the video option… if they need transcripts or other accessibility option, Panopto does have a built-in feature that may or may not be enabled for people’s campuses. Our captioning will be enabled soon, but with that does come a barrier of accessibility. So, I always give students the option of meeting with me. You can also upload the video to YouTube and create your own transcripts, which also have some challenges…. and there is actually, in the offices of accessibility, the ability to send them a video and they can send you back transcript as needed. But, in this case, if a student needs any sort of accommodations, I’m happy to meet them where they are and figure out a different way for that student besides the video feedback.

Rebecca: I think it’s always just important that as we’re thinking about these new methods that we’re not losing track of the fact that electronic documents may or may not be accessible naturally unless we take some extra steps. So, I think it’s always good when we let students know upfront that like “Hey, we’re trying this thing out, but if you need something else, let me know.” So, I’m glad that you’re doing that, but also that you’re making people aware of some of the tools that are available as well. This is such a great technique.

You said you were working on a paper, right?

Jessica: Yes, that’s where I was pulling the stats up. I’m like “Okay, I gotta pull that up.” [LAUGHTER]

Rebecca: I look forward to reading that when that’s out.

Jessica: Yeah, I have to figure out what journal I’m gonna send it to, which is always a fun thing to figure out, right?

John: One thing that I think might be worth mentioning is that for people who are using Blackboard who have really bad handwriting, as I do myself, we used to have that Crocodoc option to mark up text. It’s probably more work than video feedback, but at least you could embed comments and notes and other things. Now that Blackboard has lost that and they’ve replaced it with a New Box View, providing comments and providing feedback in Blackboard is so much more difficult, cumbersome, and so much less flexible than it was even with earlier releases. I think video feedback is a really good way of doing it, and Panopto is a really nice tool because it’s all stored on the server, you don’t have to store their videos, you don’t have to somehow transmit them to the students, or give them links to things in your own cloud services. So, it just seems like a really good approach.

Jessica: I think there’s a lot of utility for this for people who are teaching online. That’s how you can make that connection with the students. They can see who you are, if you’re comfortable with that, and also get to know them a little bit. I have colleagues who use tools such as Flipgrid, and one of their first assignments is: “Show me where you study” to kind of get to know each other. But, this is getting to know each other on an academic level, and knowing that I’m just not some person sitting behind a desk with a pen marking up your paper to bring you down… I’m here to help you learn and to help you grow as a writer and a future public health professional.

John: I used Voicethread in my online class last semester and it provided a similar experience. It got to the point where, and students commented on this too… some discussions were done in Voicethread and others were done in text… and when I was reading and some of the students noted that when they were reading text comments and text responses from students they could then hear the voice of the students. They could match the people to voices and it gave it a much deeper sense of connection then I had experienced before… because I had just used primarily text discussion forums.

Rebecca: I also want to note… 75 students in doing a writing assignment… like wow… that’s crazy. [LAUGHTER]

Jessica: Yeah. I think it’s important to help meet our learning objective for the course overall… and some would say I’m a glutton for punishment. But the students actually don’t complain about the paper as much as they complain to me about other things. Because they really don’t know that it’s going to be this long paper and grading it in these digestible chunks is actually much easier for me and the students. So, don’t tell my students next semester, but they’re still going to have to write the paper… and I still not to a page count, but the average page count for last semester’s class was just 15 to 20 pages at the end.

John: That’s a lot of grading.

Rebecca: It’s pretty incredible. How long does it take you to do each chunk? I mean you must have a down to a science because you’ve been doing it for a while.

Jessica: With the written feedback it takes me a few minutes per paper because they have a pretty solid outline that they follow and students have similar topics and I encourage them… I’m like “Work with a friend if you’re both writing about tobacco in Pacific population your background is going to be similar. If you both have the same source, be sure that you’re not plagiarizing.” But, you can work together and still have very separate papers… and so part of it becomes very repetitive at the beginning of the paper discussing a background on a public health topic… which is pretty straightforward, but where they get creative is actually where they get to health design and intervention that they think will work to help solve their public health problem.

Rebecca: At least it sounds like they get interesting.

John: It sounds like you scaffold it very nicely. How many times do you provide them with feedback over the course of the semester?

Jessica: About 10 times. Next semester, I’m going to do it a little bit less because it’s a lot of work on my part, but the smaller the chunks to grade, the faster it is to grade them… and the sooner I can realize that they might not understand something then we can cover that in more depth during the class. So, I use this project as a way to help them improve writing, but also it’s a nice check of “Oh, they actually don’t understand that concept so well” or “They’re applying it in a way that really isn’t the best, so let’s go back… let’s talk a little bit more about it and we can all be on the same page…” and I really, really like the way that it’s broken down and I don’t think I would ever wait til the end of the semester for one giant paper. Ever. [LAUGHTER] Because it causes a lot of anxiety for the students and it takes a long time to read such a long paper if you haven’t been working on it in chunks. I think my favorite thing that I hear from students is “You know, I never thought I would write like this” or “I didn’t like the idea that we had to turn in small pieces of paper but, you know what? I’m done with your paper in my class and I was procrastinating all the other papers. So, I’ve learned a new technique in writing and reducing procrastination.”

Rebecca: That must make it so that is a lot better to read at the end too.

Jessica: Oh, most definitely. If they make minor mistakes throughout it, they can fix it for when they turn it in for more credit, so that they’re not losing out on points.

Rebecca: So, we generally wrap up by asking what’s next? So, what’s next for you, Jessica?

Jessica: So, what’s next is I get to have a fun summer but coming back to the school year, I think I’m going to be writing a textbook with my students. I’m very much into the ope educational resource movement and so very passionate about it. Public health is a little bit behind other disciplines in adopting open educational resources… and I think one way to get students excited about this, but possibly other faculty, is to write a textbook… and like all of my classes… they’re small. [LAUGHTER] So, I’m going to attempt to do this with 75 students next semester. I’m either crazy, delusional, or ahead of the curve. I’ll let you know, hopefully on another podcast.

Rebecca: Sounds great. Sounds like it’s quite the adventure that you have planned for yourself for the fall.

Jessica: Life is always an adventure. [LAUGHTER]

I’ll let you guys in on a secret. I’m going to take a hundred students on a field trip this next semester to do some experiential learning… which will be really fun… and also terrifying to take that many students on a field trip. but I think experiential learning is so important in any academic field… and so, I’m this huge advocate and always trying to integrate it in my classes. One thing that I do is I’m highly involved in student-run free medical clinics and have been my whole career. As a student, I had to start my own medical clinics and now I want you just to suffer and do it like that. [LAUGHTER] I want them to have the same opportunities that I had and learn about the different disciplines and working together and helping out the community overall. So, each week I take undergraduate students with me to a free medical clinic and their job is to screen and advocate for patients and it’s so great to see how passionate the students are about helping these folks that they’ve never met… and also how invested they can get. The other day, when we went to the clinic, one patient needed eyeglasses and we actually didn’t know of any free places to get eye exams… and so the student took the initiative… got on their phone… called probably 10 eye exam places and then made it his mission to actually create us a document and call corporate offices to see if we could get eye exams for these patients that are in need. So, it really allows them to take charge… take off the training wheels… and become an advocate and use those public health muscles for good.

Rebecca: Sounds like a lot of rewarding days.

Jessica: Very much so.

John: That sounds like a lot of busy and long days as well. [LAUGHTER]

Jessica: I think the most common question I get is “Do you sleep?” and I do… eight hours a night. Sleep is very important for overall health. [LAUGHTER]

John: …and, in your field, that’s something especially to focus on.

Rebecca: Well, thanks so much for your time, Jessica. It’s been really interesting to hear what you’re up to… and I think you’ve offered a lot of fruit for thought for most of us.

John: Yes, thank you very much.

Jessica: Thanks so much. Happy to be here.

[MUSIC]
John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.