Category: metacognition

Show Notes

• Brown, Peter C.; Henry L. Roediger III, and Mark A. McDaniel (2014), Make it Stick: The Science of Successful Learning. Cambridge: Harvard University Press.
• Lang, James M. (2016).Small Teaching, Everyday Lessons from the Science of Learning. Hoboken, NJ: Jossey-Bass.
• Miller, Michelle (2015). Minds Online: Teaching Effectively with Technology.Cambridge: Harvard University Press.
• Raymond, Casey (2017). Casey Raymond’s January 12, 2017 workshop at SUNY-Oswego on Creating numerical questions in Blackboard Learn.

John: There is an overwhelming body of evidence that supports the use of retrieval practice to encourage long-term learning. While this has been known for decades, this was once very cumbersome and time-consuming to implement. In this episode, we’ll explore how learning management systems allow students to have unlimited practice.
Thanks for joining us for Tea for Teaching, an informal discussion of innovative practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist

John: and Rebecca Mushtare, aa graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today’s guest is Casey Raymond, the acting director of the Honors Program, an associate professor of chemistry and geochemistry, and a recent recipient of the President’s Award for teaching excellence at the State University of New York at Oswego.

John: Welcome, Casey.

Casey: Thank you John and Rebecca.

John: Our teas today are:

Rebecca: Jasmine green tea

Casey: elderflower Rooibus,

John: and Twinings English afternoon.

Rebecca: So, Casey, we know that you use low-stakes testing quite a bit in your introductory level courses. Can you talk a little bit about why low-stakes testing is so important for these particular courses?

Casey: So, particularly with general chemistry, the only way to get good at it is by practice and just telling the students to do questions at the end of the textbook doesn’t really work because they don’t do it. I could collect them all but I don’t want to grade all of those. I could grade just some, but again it’s still me grading them and so being able to do something electronically that encourages… forces… students to do something… is a really good avenue to get them to practice.

John: I don’t think that’s anything unique to just chemistry. There’s a tremendous amount of research out there involving the importance of retrieval practice in effective learning in pretty much all disciplines. Here at Oswego, we’ve had a series of three reading groups where that was a common theme to each of those. We had a reading group on Michelle Miller’s Minds Online: Teaching Effectively with Technology, and then we had a second reading group on Make it Stick, and then our current one is on Small Teaching: Everyday Lessons from the Science of Learning. Each of those has a significant portion on the learning gains associated with retrieval practice.

Casey: Yeah, to a degree, these are the modern version of flashcards kind of thing. It’s just a way to really get the students doing the work… and I announce in class if you’ve had to do a question 20 times and still don’t get it right or 20 times just to get it right, you probably need to be doing more questions like that and then go to the book, go to other textbooks, go to resources the publisher has available but it’ll help you determine what you don’t need to study or what you do need to study more.

Rebecca: Yeah, I think students often have no idea what\ they know or don’t know.

Casey: Yeah, and when it gets to the test and they realize that, it’s sometimes, it’s too late.

Rebecca: Yeah, that’s not not a good time to figure that out, right?

Casey: No

John: How do you weight these? And how many chances do students have to work on these problems?

Casey: So when I do this for homework, most of the time I give them unlimited attempts but there’s enough randomness in there that it’s unlikely they’ll see the same questions again… and my homework are usually two, three, or maybe four problems is all… but I’ll offer on average probably three homework assignments every two weeks, so I’m doing a few questions very frequently and in total that’s worth fifteen percent of their course grade. I do weekly quizzes. They’re usually over the weekend and again there are two or three questions is all, and that’s worth fifteen percent of their overall grade… and then at, by comparison, their midterm exam, their one in-class midterm is fifteen percent of their grade. So, in reality, these are all weighted equally. The homeworks, they usually can do as much as they want, I’ll take their highest score. The quizzes, I give them two attempts… and in some cases I do their best score. In some cases I take an average so I tell them if you get it right the first time, don’t do it again and so the quizzes are a little more information in terms of do they really know it because they only have the two attempts… and if they’re doing well on those they should do well in the midterm….
But, there’s always the students that have really high online scores because they’re using every possible resource to get the right answer… and then they get to the midterm and they don’t have all those resources and they score low because everything there is equally weighted. They still have plenty of time to improve before they get to the final exam.

John: …and one nice advantage of allowing unlimited attempts and keeping the highest grade is there’s always an incentive for them to do it again… so if they want to go back and space their practice and review (which is also important in terms of learning. If they want to do that they can go back and redo things from earlier in the course without worrying about the chance of blowing their grade.

Casey: Right. Yep, and what I typically do, though, as well as kind of impose some due dates is I just make a copy of the homework assignment and label it as practice…. and move all of the point values to zero…. and so once the homework two closes, I open homework two practice for zero credit… and it’s categorized differently in the gradebook in Blackboard.

John: OK.

Casey: So it has no bearing whatsoever on how blackboard calculates their grade and it gives them infinite practice after that. I also allow them, on the practice version, to see all of the answers, whereas when the homework is in play, it just lets them know which question they got right and which question they got wrong. They can’t actually see the answer…. how close they were…. and part of that is, in some cases, there’s limited randomness built into the problem. So, you know, I want them to really have to be working through it, but after the fact, when they’re just practicing…. getting ready for the midterm… getting ready for the final…. they should be able to see the answers.

Rebecca: Can you talk a little bit about how you implement the kind of low-stakes testing in our learning management system?

Casey: So, I really got started doing this when we were under ANGEL as our learning management system. I used other learning management systems before coming here, but when we, as a campus, settled on using ANGEL, I realized it was viable again to do some low-stakes testing and worked with Jeff Snyder to develop ways to write algorithmic mathematically based problems where there was really infinite variability in the question…. and we actually got pretty creative and wrote some pretty complicated questions within ANGEL…. and after Blackboard purchased ANGEL and we knew that it was likely our campus would shift to Blackboard, I had a sandbox given to me to start testing and we immediately recognized that most of what we did in ANGEL was going to fail horrendously in Blackboard…. and so it was, what do we do now? And part of my time during sabbatical was working out ways to do some of the things we did in ANGEL within Blackboard…. and I got a SUNY Innovative Instructional technology Grant to put that all together as tutorials… which most of the hard copy is complete now. It’s a matter of going through and implementing the figures and probably some short video tutorials on how I do that… whether it’s multiple choice questions or numeric questions…. but ways to get around the limitations that Blackboard presents us…. which I’ve been able to do in all but one situation.

John: How have students responded?

Casey: Overall, I think they’re okay with having the homework online. There’s certainly a learning curve for the students understanding that it’s a computer doing the grading and that there are limits to how much the computer can think. So if they don’t give the response the same way Blackboard is expecting it, it’s marked wrong and so part of it is educating the students in that context, and part of it is writing the questions such that there’s less ambiguity in entering the answers. I think initially the students don’t recognize the value of the low-stakes testing model, but once they, I think, start getting towards the midterm and see that…hey, those questions I had in homework and quizzes are just like questions on the midterm, I think they start to realize the value of it.

John: I’ve had a similar issue when dealing with algorithmic type questions where I have a question pool with a mix of variants and one of the problems, as you said, is if you have free response questions, students will come up with creative ways of giving you answers that you may not have anticipated. I’ll often have a list of ten or fifteen different possible correct answers and students will find new ways of writing it. Then I have to go back and vary that a little bit or search within a string. Have you been updating the questions or just giving them more guidance in the question itself.

Casey: For the most part I’ve been giving them more guidance in the question and especially the very first few homework assignments tend to be simpler… tend to be more straightforward… and to introduce that “how do you enter your answer?”… and in the sciences it’s really important to us that students include units in numeric responses and we really want to see that on their written work in their answers on it exam, but units are a disaster for online systems because there are so many different ways that you could write the units that I can’t even begin to fathom how many variations or iterations I would have to try and come up with to avoid problems …and so we’ve taken the stance of just not asking for units on the questions we asked the students to give your answer in nanometers or give your answer in meters or kilograms. We tell them what units we want the answer in, but we don’t ask them to put the units label on it.

Rebecca: Are there other workarounds you’ve had to also kind of adapt for this system? Some tips that you might have for others?

Casey: So to deal with the mathematical problems, the limitation that we encountered with Blackboard is that it only allows one dependent variable. I mean there were some limits with the way ANGEL l worked that we dealt with, but the real hurdle is you can only have the one dependent variable when you…

John: this is in terms of algorithmic…

Casey: in terms of algorithmic or even numeric questions… any kind of mathematical problem — and so if you wanted a multiple choice question that had a number as an answer and you wanted the numbers to be different for each student, Blackboard can’t do it or as an example if you had a string of three percentages blackboard could randomly generate two of the percentages but it can’t figure out what the third percentage is if you need to calculate something else in the question because that would be two dependent variables so what I’ve determined and figured out and really have mapped is to use spreadsheets whether it’s Excel, or Numbers, or Google sheets… it really doesn’t matter, but you can code your questions in a spreadsheet and then export that as a text file that can be uploaded as a pool of questions into Blackboard.
Now the nice thing about it is once you’ve done the question once and it’s all on one row of your spreadsheet all you have to do is copy and paste it to as many versions as you want. I’ve done as many as a few hundred versions of the same question this way and so there’s all different sets of numbers that the students could get and then I upload that set of let’s say 250 versions of the question as one pool of questions in Blackboard…. and so when it’s time to write a test, an assignment, a quiz – (Blackboard calls them all tests) — I can go to that pool and say pick one of any of these 250 it’s a good.

Rebecca: So, the time for you mostly is up front getting those questions set up?

Casey: And so most of the time is up front writing them in Excel and if you can write it out on paper first, once you learn a few key functions within Excel and how to handle a few things in Excel, the time really is just copy and paste and getting the text file created and uploaded. One of the other things that it allows you to do with the Excel is use HTML coding within your text so that, if you want a table of values presented to the students, you can do that, and the students will see a table of values within their Blackboard problem. Or you want to use any other kind of formatting or symbols, you just code it all with the HTML and it will upload with the text and Blackboard understands it.

John: When you have distractors on your multiple-choice do you generate them using common misunderstandings or mistakes that students make?

Casey: …and so I rarely use a multiple choice question, but I did figure out how to do it because I know many people will want to know how to do it, and so, yes when you create a mathematical multiple choice question you can have it calculate the distractors as typical mistakes that you know students are likely going to make: a negative sign, a flipped division, an improper conversion between units, all of those can be incorporated and there are ways within the Excel to basically set it up so it does a random representation of the correct and incorrect assignments. So it’s not always version B is the correct answer.

Rebecca: There’s a lot of nuance to kind of the workarounds. Do you have these in a in a format that you can share out?

Casey: Yeah, that’s the documents that I’m kind of creating now, and the steps and I’ve decided and made the choice in doing these to kind of present it at different levels. Kind of… first level is just a set of text of this — is what I’m doing — just kind of prose. If somebody really understands Excel or understands the concepts, they can read the prose and go apply it to a few more key steps that are required for any of this that you know walks through how to do it… and then my goal is with the videos is to actually go through step by step and show people this is how you do one. For those that might not be so comfortable with spreadsheets or using HTML and then the very last part of the document is going to be some of the really common code that you might want to use like how to make a table in HTML or what are some of these special characters and how do you code those, especially ones that you might want to frequently use, and so the goal would be people can just copy and paste that into whatever text they want to write and not have to go find it or really learn how to code HTML.

John: In the show notes we’ll include links to presentations that Casey has done on this as well as a link to these documents once they are available. So we’ll be updating those as things as things are posted.

Casey: Yep, and then you know I’m happy to share the kind of the initial ones that I have so far as a draft.

Rebecca: Great! So our last question always is: what are you gonna do next?

Casey: I don’t know that there’s going to be a next right now with Blackboard. You know I want to get these documents and tutorials and such in place and I don’t foresee what the next is and this one… let me put the bed huh.

John: Very good. Well, thank you, Casey!

Casey: You’re welcome.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Show Notes

• A free set of online mindset resources and lessons
• Brown, P. C., Roediger, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
• College Info Geek, created by Thomas Frank

Transcript

John:Today, our guests are Allison Peer and Alicia King, student academic success specialists at the State University of New York at Oswego.

Rebecca: Welcome, Allison and Alicia!

Alicia: Thank you for having us.

Allison: Thank you.

Rebecca: Today our teas are:

Alicia: I have Harney and Sons chocolate mint tea. It’s delicious.

Allison: I have Mandarin black pu’erh tea.

John: I have Twinings black currant black tea.

Rebecca: And I have Harney and Son’s Paris tea.

John: Many colleges have begun introducing student academic success specials. It’s a fairly new role. So could you tell us a little bit about what the role of academic success specialist is?

Allison: We see ourselves primarily as academic success coaches. There are a lot of differences between high school and college and we’re here to help students navigate that transition in the areas of time management, for example. So when students are in high school, they often have a lot of adults helping them manage their time for them- like teachers, parents, coaches. When they get here they have a lot more independence and free time and they have to make those decisions on their own, and sometimes they need support in doing that. Another thing we find is even though students may have been successful in high school, the strategies they relied on for success in high school may not be working for them as well in college. So we sometimes need to coach them on specific strategies for effective learning, and then we also have several students who report that they did not have to study in high school. So they, they get here and they first try to rely on, you know, the same methods that they used before, which might have included some cramming but otherwise they might not have had to study very much. And so then they find that they’re not being as successful and so we have to coach them on specific study and learn learning strategies as well.

Rebecca: How does your role complement or supplement the role of faculty members on campus?

Alicia: We have a few different things we use here. We offer to present at first choice courses and we’ll do that for any professor who’s, who would like to and-

Rebecca: What’s a first choice course?

Alicia: So it’s of course for first-year students. It’s usually a subject area of a requirement that they need for their degree program, but it’s also a class where we will teach them study strategies, teach them about the campus resources, they need to know about to be successful during their first year. So only first year students can get into those courses.

Allison: It’s actually SUNY Oswego’s version of a first-year seminar and there’s currently a group of people looking at modifying this a little bit. John you’re on that team, would you like to say a little bit about that?

John: So our Provost has talked a little bit about that. At his prior institution they had introduced a first-year program that were designed to improve student engagement and interest in the coursework. He referred to them as passion courses, where the instructor would find some topic that they were passionate about, the argument is it could help provide students with a much more engaging experience that they’d have a better tie to the community.

Allison: And we in the first choice courses, we also try to help build that tie to the college. One of the options we give to instructors that we can present for the first-choice class is “nailing your first semester,” and it focuses on campus resources to help support student success. There’s a couple other options of courses- or options of first-choice presentations that we offer, one is how to study in college, what to do and what not to do, and also balancing choices priorities and deadlines focuses more on the priority and time management aspect.

Rebecca: Are there other things that you do to support faculty other than just in the first year?

Allison: Yes, so for example we certainly encourage students to visit their faculty’s office hours. One of my students was reporting back to me on a conversation that she had with her professor in office hours he had suggested to her that, in addition to the traditional studying that she was doing for example, reviewing her notes from class he also suggested doing some additional practice problems and then if she experienced any difficulty with those problems or the ones that she struggled with she could bring those back to his office and discuss the solutions with him in office hours. And I could tell that the student really didn’t like the professor’s suggestion from my conversations with her, I know that she believes she needs to learn in certain ways and when someone was suggesting something different, she didn’t think it was going to work for her. So I took that opportunity to talk to her about why he was suggesting it. And I I showed her the book Make it Stick: The Science of Successful Learning, and explained to her that he was suggesting that she use one of the research-based strategies from that book. It’s a strategy that we know works. It’s been proven by research, and so after trying to you know convince her why she should give the the strategy a try we looked at her schedule to see you know where could she fit in some opportunities for this type of practice. So I’m looking forward to hearing back from her, to see how it went when she did some practice problems but she hadn’t gone back to the professor’s office hours yet to discuss those problems. So I’m looking forward to hearing back from her about how that part went.

Rebecca: So it sounds like the opportunities for the one-on-one interactions is really helpful. So that students can maybe start seeing the why. Certainly faculty members, I’m sure often try to indicate why they do things in classes, but maybe they’re not always good at communicating that, so having support outside can be really helpful. Because there’s often reasons why faculty might have certain requirement but you know they might not be so clear to students.

John: And students come in with some serious metacognitive issues in terms of what they find most effective in studying. As she said students generally believe that the most effective studying techniques are repeated rereading and cramming before an exam. And while that works really well and remembering things for a few minutes or a few hours, it really doesn’t do much in terms of long term learning and it’s one of the things but that you know both Make it Stick and the “Small teaching” reading group we’re doing this semester in which you’re both participating that point, out but it’s hard to convince students how successful are you, in general at convincing students a one-on-one approach I think would be much more effective than trying to have a faculty member, well in my case for example, try to convince three hundred and sixty students that they should try these techniques.

Allison: Yes, we definitely enjoy the one-on-one opportunities with students. In some ways students are already primed for new suggestions when we start working with them. We teach an academic success course in addition to seeing students on academic probation in one-on-one conversations. Students who take our academic success course, most of them are choosing to take it, because they know that the strategies that they have been relying on are not working for them. So they are ready to try something new. So we do have some buy-in already. In the past, the people in this position previously experimented with requiring students to take the course and it didn’t work as well, when the students felt like they were being made to take it, as I understand.

 

John:So extrinsic versus intrinsic motivation.

Allison: So another thing that we, do that kind of helps convince the students to try it, and this is suggested also in Make it Stick, take the time to explain to students how learning works. It’s about them, it’s about how their mind works and usually people are interested in things that relate to them and also that are going to help them improve and something that that they want to accomplish. So we have found a lot of engaging videos that you know help students see how learning works and then also we have videos that help introduce some strategies for effective learning such as spaced practice, retrieval practice, interleaved practice. One of the the video series that we really like is by “The College Info Geek,” Thomas Frank, I don’t know if you guys have seen him on YouTube one of our colleagues in the School of Education, I consider him to be an expert on like brain based learning strategies, he actually uses Thomas Frank with his students, so if he uses it then I feel comfortable relying on Thomas Frank as a resource also. But the videos that Thomas Frank does they are geared towards helping students improve and in all areas academically and otherwise like, for example, he has a video on how to get good sleep and how to take care of your health, so that you can then you know, in turn be more productive and successful with your academics. So, instead of students just having to listen to us the whole time we pull in some videos like that. And after we introduced the strategies to students we also ask them to share out examples of how they may already be using it, so that you know they might hear an idea from a classmate that they might think would work well for them and so then they’re willing to try it out, because they heard it from a peer who they can relate to.

Alicia: And Thomas Frank is young as well, so I think that helped quite a bit that- Allison is giggling because she knows how much I like Thomas Frank.

John: Actually a podcast we recorded just a few days ago and will be coming out probably a week or two before this we talked about a metacognitive cafe online discussion forum, where students look at similar things, but also share the thoughts with each other. So that not only are they thinking about how they’ve applied it- but applying it having hearing from peers about how it can be effective can help. And hearing from someone who’s young on a video, may also work better than hearing from an old professor, and so forth. One of the things that I’ve been really impressed with is all the materials you develop in the handouts that you’ve given out to students – we will include links to some of these in the show notes – could you tell us a little bit about the materials you’ve created and share with students?

Alicia: Sure, so using retrieval practice we understand that textbook reading isn’t always the students favored style of studying, even though to prepare for a test, you’re correct, John they they often will reread the text over and over and over again, so yes reading the text is important but also making connections to the class lectures are also important. So, we created these bookmarks using the retrieval practice strategy that helps them quiz themselves as they’re reading so that they’re pulling more information out of the text, their eyes just aren’t skimming, they’re not flipping pages and not really, they’re not really digesting the information that they’re trying to learn when they do that instead, they’re treating it kind of like a scavenger hunt. They have questions that they need to look for that they’re supposed to get out of that chapter, put it right on the back of the bookmark so they can quiz themselves with questions like what did I just learn or read? What is this mostly about so they can summarize it. Put it into their own words, which is another useful tool questions like how does this relate to what I already know which also helps them build connections so it’s just something handy that they can keep in their textbook to help them.

Allison: And we do have to coach them in how to use the bookmark, so the retrieval practice is only retrieval practice when you are forcing yourself to stop and and answer the question some students need a reminder that you know, you can’t just say “oh yeah I think I know that that’s that makes sense to me that’s easy I’m just going to move on to the next section,” they actually have to to stop and make themselves answer a question about it, put in that effort to do that.

Rebecca: I just had a student this morning that I had that same conversation with, this idea of fluency illusion where she was talking about yeah when you go over in class or we go over in class it makes perfect sense, when I go to try to do it myself like it makes- like I can’t figure it out at all and that’s you know that’s the idea of needing to practice and we were talking I was talking to her about the issue that she had been missing a lot of the review questions that we have at the beginning of class which is built-in practice, so she was missing that opportunity to kind of retrieve on her own she was only hearing like the solutions at the end, so we discussed that but it’s the same idea it’s like yeah sure does make sense to me when someone’s explaining it to me and like holding my hand through it.

Allison: Yeah, when I’m showing them the bookmark in my office, I actually find myself like turning in my chair turning away from the book and looking out of the book and you know like answering the question out of my head and then turning back to the book so they can see- yeah, I actually do need to to look out of the book and rely on my own thoughts to answer the question…

John: your own mental models

Allison: instead of you know, letting myself look back in the book to answer it. And we do tell them after you’ve answered the question for yourself, yes then you can check back in the book and see if there’s anything that you missed or it you know make sure you explain it accurately but do take the time to answer it on your own first. It’s definitely a habit that takes discipline to build.

Alicia: I like telling the students to use Cornell notes while they do it so they actually like in the left-hand column of the Cornell notes they can write a question like the ones on the back of the bookmark and then explain it and then use those notes as part with their lecture notes to study from so they get the most they can from the textbook and the lecture.

Allison: of course in our academic success course we make sure they know when you’re using those questions on the left hand side of the Cornell notes cover up the details on the right and there’s the questions to quiz yourself to do the retrieval practice.

John: Yeah, one of the things I’ve been using in my class is McGraw Hill’s Smart Book, but students have the option of either using a traditional eBook version of it which is just linear, or the smart book option which does exactly this- there’s a section they read and then they’re quizzed on it and then if they do well they can move on, if not it will tell them to go back and re-examine the material and then try it again with some different questions. What I’m finding is that the the stronger students of students who are doing better do that, but the students who are struggling find that very frustrating and they give up and it’s hard to develop a mindset to convince them that practice is really useful instead of focusing on the things that’s easiest, perhaps they need to focus more. What tactics could you use learning is hard work we have to convince some of that because when people come in with a fixed mindset, it’s hard for them to deal with from struggle.

Allison: So, you asked what tactics do we use to get them to understand that they have to practice the hard stuff, instead of going through the stuff that comes easier to them. I’m still working on figuring out the best way to do that with students but one thing that I think is helpful is a lot of students might have had the experience of playing a musical instrument or you know perhaps they took dance lessons and participated in dance recitals or sports yes, so and so if they had a good coach or teacher along the way, hopefully that person was explaining to them look when you’re playing this piece of music we don’t just start at the beginning and run through to the end every time we work on sections at a time and it’s not the section that we play really well that we need to stop and work on it’s the part that we’re struggling on. And so sometimes when we’re practicing it to try and get it better we don’t try to go home full-speed, we slow it down break it up into steps and then gradually build up the speed and work up to our performance level. And if we can relate it back to something like that that they’ve experienced, it may help them you know apply it to a new type of learning. We’re still looking for additional strategies.

Rebecca: After being sort of coached into some of these processes that we know science tells us work, you know how are students responding once they actually try? Are they seeing the effects and then although it’s a struggle they’re they’re doing it or what is your experience been?

Allison: We’ve gotten some good feedback actually we mentioned before that a lot of the students that we’re working with are primed for some change they’re ready to try some new strategies because the strategies they’ve been using have not led to success here in the college setting. So one thing that we do in our academic success course in addition to introducing the strategies with videos we give the students some opportunities to try those strategies out in the classroom on the day that we introduce it- but we also make it homework for a whole week, they have to choose a strategy they want to try out so we give them a handout actually, on the handout we list some strategies from Make it Stick: The Science of Successful Learning, we have self testing or retrieval practice, spaced practice, interleaved practice, we also list Cornell notes on there as a method of using retrieval practice so we have them choose a strategy to try out for the week, they have to write down specifically what they did to make that strategy a reality, write down the result of their actions, and then they also provide a little more detail on what their next steps are going to be. So perhaps they liked that strategy but they want to tweak it a little bit to make it even more effective for them, or perhaps they liked that strategy, found it effective for one course so they’re going to start using it in another course. So we find that when we give the students the time to do it by making an assignment, and we don’t have a lot of assignments in our class we try to make everything that we do beneficial for the students to promote their success, so we don’t feel like we’re over burdening them with work, but we do give them adequate time to try this out and as they’re doing the work for our class they’re also accomplishing work they have to do for another class anyway, so when they take the time to try it out while not being under stress about it we’ve gotten some good feedback, I have some some quotes for some students we have students write about this in their final reflection that they do for the course, they write about you know what worked well for them and kind of what their next steps are going to be as they’re working towards academic success. So for example, when students were trying out interleaving we had a couple students mention that when they were using interleaf practice or varied practice, they didn’t get tired out as fast during their study session because they weren’t studying one thing for a long stretch at a time they weren’t getting bored when they would switch between topics, students also mentioned that it helped them make connections between the things they were learning for example, I actually have an upper-division biology student who came to us because she’s considering medical school in the future and she’s doing well already but she knows medical school is going to be a challenge so she wants to start improving her learning strategies now. So she is taking several upper division biology classes and using interleaving helped her see the different connections between her different biology classes that she was taking. She cited higher grades on her midterm exam, but more importantly to her she said she didn’t feel like she had to rush to feel prepared for her exams. You know in the past she had relied on cramming and she was initially apprehensive about trying something new because she was used to studying in a certain way and she had been successful up to this point but when I shared with her that medical students use these strategies to be successful and even showed her some videos on YouTube that were geared towards medical students using the strategies I think that helped her give strategies a chance, and when she did she saw increased grades because of it, but she also saw a reduced stress because of it. Which is going to be important when she goes to medical school.

John: One thing I’ve been really impressed by talking to you since we started with the reading groups is the focus you placed on evidence-based practices giving students tools that allow them to be successful in any course rather than just focusing on specific short-term problems, and have students generally been buying in in general?

Alicia: We make sure that the students we invite into our class are students who are close to being on academic probation, or like see that they need the help so that the dynamic of the classroom is approachable for everyone.

John: So they’re more receptive.

Alicia: Yeah, and when I have a student add a class late I’m very careful to remind them this is a class full of students that are on academic probation, are close to it, and are very interested and being academically successful, and I think that helps a lot. Starting the class off with that mindset helps quite a bit, makes it a more open atmosphere. We also do some like ice breaking activities to help warm everyone up to let them know that it’s a safe zone for this kind of talk and that helps quite a bit. That seems to be my best strategy as a start.

Rebecca: It seems like you’re meeting them where they’re at, and that’s the whole structure of the course is like the assumption of being in this class is that you’re at a place where things aren’t working, we want to work better and that seems like it’s the key setup to being successful because, you know like that’s where you’re starting, you’re not starting with like well there’s some people were being really successful and some people who aren’t and you don’t have that wide range like there’s a little more focus and sometimes having that more focused group of students can make for maybe a better a better cell because, in John’s example earlier, the really large class he might talk about them maybe about the strategies but, perhaps it’s like group of students who were maybe already going to do fairly well already had that kind of growth mindset might adopt it because they’re willing to try something new, and that group of students who maybe weren’t willing who might not be successful might end up in your group of students but it’s nice that there’s kind of places that the information is getting to people in different places.

Alicia: I think you said it perfectly, we really do need to meet them where they’re at. We admitted them, they’re in the class they’re in there standing for a reason and that’s why Allison and I have jobs is to make sure we come to them.

John: It’s certainly more efficient than just discarding 20 to 40 percent of our students over first couple of years as was a common practice in colleges and universities for decades. You know people are spending a lot to be here and there’s a lot of investment in getting people here and encouraging them to be successful is a good thing.

Allison: Yeah we do find that a lot of students may not have had the opportunity to navigate through academic failure before, and certainly everyone has had struggles but some of the students that we’re seeing now may not have been having that many academic struggles while they were in high school, and you know some of the students we see they aren’t sure how to navigate a failure. So that’s another role that we and the faculty and staff as a team have to help students see those struggles and failures as a learning and growth opportunity rather than an experience that defines them negatively.

John: Going back to the mindset comment, that students with fixed mindsets are not always going to be students who are lower quality students- one of the problems is that when students have been successful but they have this fixed mindset they believe that it’s because they’re talented and the first time they experience failure can often be very disruptive and can lead them to giving up, so that growth mindset is important for all students even those who have been successful as well as those who’ve struggled. Those who’ve struggled regularly often have been forced to adopt a growth mindset because they can see it work but those students who are able to breeze through middle and high school without doing much more when suddenly they’re faced with a challenge often have those sort of troubles.

Allison: Yeah I remember having a conversation with a student like that last semester he had declared a certain major because he had been interested in it and based on his experience with the topic in high school he thought he was good at that subject that was his phrase you know “I thought I was good at it,” and then he got here and took an intro course in the same subject and he didn’t do as well as he had thought, and his comment to me was well I thought I was good at it but I guess I’m just not and you know the student definitely needs some help cultivating a growth mindset. I think it’s a long process for some students because you know their mindsets have been developing since they were young enough to start to understand language based on the types of praise they were hearing from their well-meaning parents-

John: and from some of their teachers along the way because they’ve been praised for being good at this or talented, rather than for their effort.

Allison: Right, so I think it’s something that we all need to be conscious of- the messages we’re sending to students, the way we’re praising them, what we’re praising them for, and also you know some of the feedback that we give that may not be praise but might be taken by the students as as more negative. Of course we have to give constructive criticism but some students-

John: some students just seem to have some problems in recognizing that, we’ve all seen people who say they’re just not good at math or they just can’t write or they just can give public speaking or they can’t draw, and the reason is mostly because they haven’t really tried to do those things and they haven’t put in the effort, and reminding students that they can get better at things by doing it there’s something I think we all need to work on.

Rebecca: I think it’s interesting that sometimes the the lack of a growth mindset is actually pretty prevalent and students who may be traditionally earned A’s and B’s and then those are the students who don’t want to take any risks, right? Which is really what college often is about is taking some risks, and like you know coming up with a hypothesis and finding out if you’re correct or not, you know there’s there’s a chance that you’re not, and so I certainly see this a lot in my classes too, that students you know it might be a student who maybe is a traditionally a C students and they really want to try something new you know and and try to get the most out of this situation so you know there’s there’s a benefit to maybe sharing some of the mindset strategies that maybe some of the students we might label as poor students like actually have that could that everyone else in the room could benefit from, and taking the time and energy to raise awareness about like “oh look at that risk,” you know that was a really interesting choice to make I’d like to see other people make take risks like that it could really be beneficial in helping to switch that mindset I think.

Allison: I think that’s a great suggestion Rebecca, we can’t just stop at telling students to put the time in and try and you’ll see yourself improve. I think we actually need to show them explicit ways that they can do that. So in your graphic design course you’re gonna have specific things that they can do to improve their work whereas in a writing course there’s going to be specific writing strategies students can use to improve their writing, so not only can the professor take the time to show the strategies that students can use, but also give some time to the other students in the class to share the strategies they’ve used I really do think that helps students buy in when the strategies are coming from their peers.

Rebecca: I did want to make sure that we took a little time while we were chatting today to find out like why you got into the roles that you’re in in the first place, because it’s an you know somewhat of an unusual position and I’m just like-

John: well, a new position

Rebecca: yeah, so I’m curious like how did you how did you end up where you’re at?

Alicia: So I went to school and studied finance actually, close to John over here and I started working for the school that I was going to and it helped pay tuition so that was a fantastic benefit and eventually I started academic advising, so I’ve always been an academic advisor of some sort and so advising was really important, to me not so much financial advising I found that that was a little more risky depending on the market and just tended to gravitate towards advising students so I really enjoy that aspect. My previous job here at SUNY Oswego was a Transfer Success Adviser where I helped students transfer from community college to a 4-year institution and there definitely is a gap there that’s very similar to what we experienced with our first year students, so I see that too and this just was like a very gradual happy next step for me.

Allison: I’ve had an interesting pathway to get to this role how much detail would you like?

Alicia: The whole story.

Allison: Okay well I have always been fascinated by how the brain learns and remembers, and when I was much younger I thought I wanted to be a brain surgeon, and everyone that I went to school with thought I was going to become a brain surgeon, and my parents thought I was going to become a brain surgeon, but through some different experiences that I had some before college and and some during college, I learned that physical surgery was not going to be the best way for me to help people personally, but I was still very interested in how the brain works. So I began to think about other ways that I could still learn all I could about how the brain functions and learns and remembers and help people improve their capacity for learning. So as an undergraduate, I took a lot of neuroscience courses in addition to my psychology major, got my Master’s in Education here at Oswego and taught for several years, but I was really missing the one-on-one interaction with students I found that I wasn’t really getting the time to have individual coaching sessions with students on how to improve, and how they could learn better. I had chunks of it here and there but I wanted to be able to focus more time on that, so when this opportunity became available it just seemed like a really good fit for me. So I’m glad I made the change.

Rebecca: So in your roles that you’re currently in, you’ve done so much already but what are you going to do next?

Allison: Well as we alluded to previously, we’re still you know looking for effective strategies to help students develop more of a growth mindset versus a fixed mindset and we know that the research is still taking shape in that area and I know there are some things out there that are already being discussed you know different conferences on student success also listservs it that we subscribe to first-year and transitional type listservs. Unfortunately, we don’t always have the time to digest all of that as its as it’s coming in because we do have so many things going on, but that’s one of our priorities you know I’m one of our slow times to investigate some more strategies to help develop that growth mindset with students.

Rebecca: Sounds like a fantastic plan. I think it you know taking the time to do that it’s gonna be really helpful I think we all probably wish we had a little more time to dig into the research on that topic well we really appreciate both of you joining us today and taking the time to chat with us and sharing what you’ve been up to.

Allison: Thank you.

Alicia: Thank you for having us.

John: Well thank you for being here.

Show Notes

References

• Foundation for Critical Thinking
• Scholarship of Teaching and Learning at Buffalo State
• Tanner, Kimberly D. “Promoting student metacognition.” CBE-Life Sciences Education 11, no. 2 (2012): 113-120 (http://www.lifescied.org/content/11/2/113.full)

Resources

• KB Tanner’s sample questions
• Paul-Elder model of Critical Thinking
• Dr John Draeger’s blog
• Learning How to Learn MOOC
• Metacognitive discussion prompts for John Kane’s Eco 101 class

Transcript

Rebecca: Our guest today is Judith Littlejohn, an instructional designer and historian from Genesee Community College in Batavia, New York. She is the 2014 recipient of the State University of New York Chancellor’s Award for Excellence in Professional Service and the 2015 recipient of the State University of New York’s FACT2 Award for Excellence in Innovative Instruction. Welcome, Judie.

Judie: Thank you.

John: Today’s teas are:

Judie: Oh, mine is a Twining’s Forest Blend that I got in Epcot.

Rebecca: Mine is English afternoon.

John: …and mine is a Tea Forte Black Currant tea.

John: Today we’d like to talk to you about the metacognitive cafe online discussion forum that you developed. What prompted you to develop this?

Judie: At Genesee Community College, we have a Provost who really promotes critical thinking and as part of her initiatives, in January 2014, we had a critical thinking workshop with Rush Cosgrove from the Foundation for Critical Thinking. He talked a lot about the “elements of thought” and different ways to critically analyze whatever we’re reading. So I took his elements of thought and incorporated them into my online discussions then and tried to focus the students on becoming aware of bias… and you know the different…. ‘cause I’m blanking out on what all the elements of thought are….Then shortly after that, in 2016, we had Dr. John Draeger come over and spend the day with us. He’s the director of the Scholarship of Teaching and Learning at Buffalo State and he has a blog called “Improve with Metacognition” and I found his ideas really interesting. He based a lot of what he does off of an article by Kimberly Tanner. She’s a biologist; she’s at San Francisco State University and she wrote an article about helping students in biology with their metacognition, and she had a table of discussion prompts that she uses. And so John Draeger shared Kimberly Tanner’s table of prompts, and it was all about what they were getting out of the reading, and being aware of how they learn, and what’s successful for them in their struggles to try to learn all this biology content. And so Draeger works with grad students and a lot of what he did was really interesting but, where we are in the Community College, we have the fundamental problem where it’s a struggle to get the students to read. So I thought it would be great to implement questions that help students reflect on how they’re learning what they’re reading, but initially we have to get him to read. So I took the table of prompts that Draeger had and kind of broke them down a little bit in a way that would force the students to have to read at least parts of the chapter in order to answer the questions, and so that led to a series of low-stakes discussion questions separate from the content-focused, you know, the higher-stakes discussions that they do throughout the term, and so the students have to engage with specific parts of the chapter of the book and discuss those. And then from that I kind of branched out into other ways that they think about how they’re learning, think about how they’re reading, and how they can transfer their knowledge, and things like that, and just become aware of the process of learning, thinking about how they learn and thinking about thinking, which, of course, is what metacognition is.

John: How do students respond to just the term “the Metacognitive Cafe?” Many of them probably haven’t heard that term before.

Judie: Well I do put the definition at the beginning of every discussion prompt. I just copy and paste it into each discussion.

John: So to remind them.

Judie: It reinforces what you’re doing for consistency. So they they see the definition all the time. Whether or not they read it, I don’t know, but I noticed that over the course of the semester – at first they’ll call their discussion post… you know… like “meta,” “meta one,” “meta two,” and now we’re into week… we just started week 11, I think… and so they’re starting to write out the words “metacognitive cafe” and so, to me, that indicates that they’re focused a little bit more on what they’re doing. They’re paying attention a little more, I think… I survey them a lot to try to get feedback on what they think of the discussions and it’s been overwhelmingly positive. I think one of the byproducts is it builds a lot of community in the class. The students are sharing what they’re struggling with content-wise and how they’re approaching content that’s unfamiliar to them… how they reread things… or if they take notes, and things like that, and they’re really giving each other advice and strategies.

Rebecca: This is taking place in an online environment, right?

Judie: Right. It’s completely online. I have two courses right now: History 101 (which is Ancient World) and History 104 (which is Early Western Tradition)… and so currently those two courses are engaging in these discussions.

John: Earlier, when I saw you present on this I was really impressed by it, so I’ve tried it myself and it’s worked really well. One of the things, as you mentioned, is that students start talking about their struggles in the class and they start to get to know each other a lot more, at least in my experience, and you had relayed the same sort of thing when you talked about it earlier. Could you talk a little bit about that aspect of it?

Judie: Yes, I find that. Maybe, I’m not sure if it’s because it’s low stakes and the pressure is off, but the students share a lot more in these discussions. They tell surprisingly personal stories of what their home life is like. I ask them to talk about their study space, or their work space. Do you read and write in the same location? Do you try to read in a quiet place? or whatever… I encourage them to share pictures of where they do their assignments and it’s surprising how much they share. I feel like you can kind of,,, you know… I think over the course of an online term the students kind of group together… by the early posters and the late posters. But it really is solidifying this community, and they’re supportive of each other and they’re talking about what they have in common: if they have the same major… if they have the same struggles… I think a lot of it is finding that other students are struggling with the same content… that they are is really affirming for them. They feel, you know, like: “I’m not the stupid one, I can learn this. If they figured it out, maybe I can too.” And they they really are supportive. So it’s been pretty interesting to watch.

John: …and they also share study strategies. When you talked about learning spaces, they probably… at least in my class I’ve used some of the same prompts…. they’ll often talk about how important it is to have a place where they can have focused concentration without interruptions… and for some other students that’s a bit of a surprise so…

Judie: It is. I have students who work two jobs… they go to school…. they have kids… they’re primary caregivers for their aging parents… you know… all kinds of things are going on… and they’ll write about how: “Well, I’ll read for fifteen minutes in my car before I go into my second job” or whatever, and a lot of them are studying in a very haphazard way like that… and I think they watch the posts of the students who say: “oh, I plan this into my day. I schedule my time…” and things like that. There’s a lot of discussion of time management, and it really wakes some of the students up to say “Okay, if my education is a priority, my studying has to be a priority too…” and then they think… they try to reorganize their time a little bit better.

Rebecca: What role do you play in the conversations because it seems like that’s moving away from the content area stuff to more about like… how you… how you are a learner… how you exist as a learner. One of the things that we talked a little bit about in a reading group that we had on our campus recently was about the faculty member becoming a little vulnerable in certain circumstances, so that students can relate to them a little bit more… and they’re not just like some sort of authority figure who has no emotions.

Judie: Mm-hmm. So with these discussions… I try to… I don’t insert myself a whole lot, but I try to get in there a little bit now and then. I use… we use… Blackboard as our learning management system, so I do turn on the post ratings. That means you can give the students stars on their discussion: 0 through 5 stars. So, whenever somebody posts what appears to be a genuine response in their initial post, I give them the five-star rating… so they know… and they can tell that the instructor gave them the rating…. so they know that I’m right there, that I read what they what they wrote. Even if I don’t comment, they see the stars… and then where I feel like it’s appropriate, then I will comment on somebody else’s posts… and I do so. In the one where we talk about our workspace, I do share a picture of my office set up at home… I teach as an adjunct, so most of this is going on when I’m at home after my workday… and so I show them that… and there’s a few different things that they come up with. I’m trying to…. just off the top of my head… at one student. So, I have one pretty basic question, which is “why is it important to read the chapter before you start to take the quizzes?” and one of the students made a great analogy… and she said “I read the chapter before I do the quiz or before I start the writing assignment because if I was trying to change a tire on my car and I had no idea how to – where to find the jack, how to how to take the tire off, where any of the tools were, and how to go about the process, I would not be able to change the tire in my car – and it was pretty good… and so I jumped in on that one and I said well I really liked your analogy… and then that kind of led to more discussion of analogies and things like that… which I like. Because I think sometimes, when the instructor posts, it kind of ends the conversation, and sometimes people won’t post after the instructor. So I like that one but, so yeah, I’m just kind of on the sidelines I’d say… and try to jump in now and then where I feel like it’s not going to hinder the conversation and….

John: but support it….

Judie: Right, yes…support it.

John: Going back to a point you made earlier about things bleeding through and them getting to know each other, the first time I use this was last spring in a labor economics class where I, in some of the discussions I’ll ask them to talk how the material relates to their future career or how it relates to other classes they’ll be taking… and they started opening up quite a bit in my classes too: about their work environment… about their family environment… about having to work a couple of jobs in some cases… or raising small children… or having relatives who had health issues that they had to assist with… and what was really interesting is that bled over into the content discussion. So when they were talking about various labor market topics, they make references to other people’s living conditions, and how they might be able to relate to this concept, or how the things they had said in the other discussion were relevant here… and it was nice to see that bleed through… because they they were making posts that were much more meaningful in both discussions than I’d normally noticed in the past.

Judie: I noticed too that they they post more, meaning: more sentences… more…. they seem to become a little bit more articulate in their posts, instead of just barely hitting the key words and and calling it good. They really elaborate on their ideas more… and they respond to each other a lot more so instead of – I mean there’s always a handful of students who respond to another student and say “I agree” but I find much more content like “I agree, especially when you said whatever… or when you said X it reminded me of Y” and so I find that the depth of their discussions is a lot better.

Rebecca: Have students articulated that the metacognitive cafe is something that they’re finding really beneficial? Other than your observations, have they actually articulated that?

Judie: Yes, they also… I survey them in addition to the institutional end-of-course survey. I do give them informal surveys in the class through Blackboard and I typically do a half-time survey and then an end-of-course survey, and I give them five extra credit points so that does encourage them to answer – I can see who responded, but not what they said, so it’s still anonymous in that respect – and we just finished up the half-time survey and I wrote down a quote that one of the students wrote. In the surveys that I write, I can ask very pointed questions about what we’re experimenting with and, as an instructional designer, I try all different things in my classes and I want to get feedback while it’s live and while we still have the other half of the semester to make changes. But one of them in History 104, the Western Tradition, said “the metacognitive cafe is a great idea… it seems like a small break from ordinary coursework where you can actually talk about how you do the coursework, which is interesting. It seems to me that most people enjoy the metacognitive cafe.” So I like that… and a quote I had from the past, I think from last semester, was: “the discussions sometimes did not mean much to the coursework, although it did help me learn how to learn the material better.”
I was having a conversation with my mom yesterday about teaching and she was asking me a lot of questions about teaching online and I said you know my whole thing is: of course I want them to grasp the learning outcomes, but I don’t care if they memorize dates and names as long as thematically they understand the major themes throughout history, but if if I can help them learn how to learn, then they’ll be unstoppable. That’s my whole goal… to make sure the students can figure out how to tackle some new ideas… and figure them out…. and look at them with a critical thinking perspective… and make the most of it ….and take it with them.

Rebecca: That’s something that I really value in my classes too. I teach mostly web design classes, so students really have to learn… to learn… because the stuff changes all the time.

Judie: Right.

Rebecca: …and we don’t realize how important that is, and so it seems to the metacognitive cafe is a good opportunity to help students kind of move away from the idea of fluency illusion… or the idea that like… oh this is really familiar…. so therefore I know it – kind of recognizing that there is a way that we retain information and we need to practice it and retrieve it and all of those sorts of things and by having kind of those guided questions they become more aware that’s even a thing right.

Judie: Yes, I agree. I think it’s been good. I think it’s not a whole lot of effort on my part and it’s not… you know, I estimate – every week I give the students a checklist of what they have to accomplish that week and approximately how long each activity will take and I usually put about 20 minutes for this. So I don’t think it it takes a lot of time for them to go through the process of typing up their response. I don’t know how much time they spend thinking about it, but I do think it’s really been helpful, and I think it’s worth carving that time out of the week for the students to do that.

Rebecca: Can you share some examples of some of your questions?

Judie: Let’s see…. I brought an index card with a couple of questions. So the first one that I asked them is… that first week… so say in History 101, the Ancient World, we talk about the Agricultural Revolution and I just fundamentally say “What did you already know about the Agricultural Revolution and what did you learn in this chapter that was new to you?….” and they can’t answer that if they don’t open the book.

John:…and that activates prior knowledge and it helps them to make…

Judie: Oh, absolutely. But remember, my first goal is to get them to read, so at least they have to look at the book. They’ve got to at least read the five or ten pages on the Agricultural Revolution and find some new idea that they can point to.

John: … and that’s a little nudge to get them to practice…

Judie: Exactly. …and I follow it up the next thing they do… well every week, every chapter, they do mastery quizzes, and if they don’t read the book first, the mastery quizzes take close to three hours and I tell them to budget two hours every week. You know, after they spend an hour reading then two hours for the quiz.

John: Are these the InQuizitive quizzes from Norton?

Judie: Yes, I use Norton InQuizitive. InQuizitive, if you’re not familiar with it, it’s sort of a gaming method of quizzing, and so the students have to wager. They wager points according to how confident they are that they know the answer, and they have to earn fifteen hundred points in the quiz to get ten points in the grade book.

John: So, if they’re wrong they lose points. so it forces them to think about how well they know this, which is also another way of developing metacognition.

Judie: Right, how well they know it. It’s set up really well to keep targeting the questions that they’re getting wrong and it tells them where to look in the book. It gives them a lot of helpful feedback and the students overwhelmingly like those quizzes. They dislike them at first because they’re not used to their chapter quiz taking so long, but once they get into it and get used to how the system works, then I think it’s… they really do like it. They’ve got different kinds too. So you can watch a short video clip and then answer a question, or they’ve got sorting and putting things in order… which I love …. I love to do timelines and cause-and-effect type things

John: It helps them make connections across events….

Judie: Right. So, it’s not just multiple choice quizzes, or just true and false. So, I like that a lot. But, yeah… so that was one question [for a metacognitive cafe discussion prompt]… So, yeah, I asked, what interesting fact did you learn? Oh, then another week, I’ll say “Did anything new or interesting in the chapter change the way you think about X, like change the way you think about the Middle Ages?” ….or what information changed any preconceived notions that people have about certain things, like: “What did you read that….and I think students have trouble… I think at this level the students have trouble understanding the difference between what they learned in high school and what history is in college because they don’t understand that high schools teach them citizenship and not history….not major global historical themes. They think they know global history, but they really don’t… and so a lot of what they learn in college is pretty eye-opening, if they will admit that their eyes need to be opened.

John: Because your students come in with pre-existing models of how the world works…

Judie: Exactly.

John: …and much of it is wrong…. and

Judie: yes

John: …we need to tear that down but they have to recognize it… and forcing them to confront it and think about that and reflect on that…

Judie: mm-hmm.

John: …could be really helpful.

Rebecca: Especially because students… they’re trying to just get the grade sometimes… especially if they have a lot of other things that they’re balancing in their lives and they don’t take the time to reflect… so by building it into your course, right… like when I finally get students to kind of make those connections….but we have to… it becomes sort of the responsibility of a faculty member to help students develop that practice, because it’s not intuitive and it’s not something that, you know, it seems like an extra, right?

Judie: Right.

Rebecca: After a little thing but… I don’t think they always see the value in it until they’ve done it.

Judie: Right. Yeah, and it’s tough…. I think it’s really tough with a gen ed that, you know. If I have a class of 32 students, typically there’s one who will say they want to be a historian; all the rest of them are only there because they need a world civilization requirement and they don’t want to take the course. They’ll say it right out: “I’m not interested in history,” “history is boring,” and so, you’ve got that uphill struggle right there… and then for them to realize they need to take serious time on this course that is not in their program that, as they see it it, can be tough. I have a couple students right now who just don’t seem to understand why you would have to take history, say, if you’re going to be an artist. Which, [laughing] how could you not want to take history as an artist…

Rebecca: …as an artist… what the heck?

Judie: It’s like influencing artistic movements like history and art, I think reflect each other and why would you only want half the story? But so, that’s tough.

Rebecca: I’m so glad that you brought up the stuff about general education because I think a lot of faculty struggle with some of the same issues, right, like “why am I here? and then trying to get those students engaged is such an upward battle or at least it feels like it is. Are you finding that the metacognitive cafe is starting to convert some of those folks?

Judie: A little bit… a little bit… that… and I also work really hard in all my classes to help the students find something in history that they can kind of latch on to thematically… So early on it within the first couple weeks of the class they need to find a theme to track so if you if you are an artist, you could choose art, or if you’re a sports person you could choose a sport history, and kind of track how what changes and you know what new trends and things started throughout whatever era we’re studying… and I think that helps a lot too… because…

John: …it gives them a more personal connection.

Judie. Exactly.

John: It’s also activating their prior knowledge and information.

Judie: Right…and so that’s helpful and do have them kind of share that too…. you know, what topics they’re researching and why… and then, and that’s fun. Those discussions are fun because when students are working on research papers and they spend all this time looking for sources and writing their outlines and all that and finally writing the paper, to finally be able to share what they’ve been researching I think is really good. It’s easy to do that in the classroom, but online it’s a little tougher, so I try to make sure they have a forum where they can point out what the most interesting things they discovered during their research process… where they’re finding sources… and the different facts that they will discover and I think it’s helpful.

John: One of the things that I’ve asked in my classes since I’ve been doing this (at the end of the term in the last metacognitive cafe discussion) is to reflect back on the class… and I asked them also if there’s anything… based on your question, one part of it is if what they’ve learned has caused them to alter their view of the world, or what was the most surprising thing they learned in the course…. and what’s been really interesting is the most common response is that what they enjoyed the most were the metacognitive cafes. One of the things I think we both do to some extent is we try to nudge them towards better learning strategies and as part of that, I have some on spaced practice, interleaved practice, and so forth, and also having them break that myth of learning styles that they’ve come in with… and what they’re always saying is that they amazed that so much of what they had learned and what they had been told in their earlier classes about rereading and highlighting and focusing on their learning styles is not based on evidence and many of them are saying they wish they had learned much of this stuff back in elementary school and it would have made their educational career much more productive.

Rebecca: Sounds like you know in in both of your cases an interesting way of you know if it’s a general education class that maybe students aren’t buying into the subject matter, at least they can buy into the idea of like learning how to learn and finding strategies and things that could apply elsewhere through the lens for a particular discipline which you know, if anything, at least that’s something that they can like connect to if they’re finding it hard to connect to the content.

Judie: Yeah that’s that’s a good insight.

John: Now one thing we we usually ask is: “what lessons have you learned while doing this? How has your practice changed over time?

Judie: Hmmmm

John: It sounds like it’s worked pretty well from the beginning.

Judie: I think a few of the questions could be a little bit more thoughtful. I don’t know. I asked some things about you know like motivation and how will they transfer the skills they’re learning in this class to their other courses but sometimes I try to mix up, so like throughout we have a 16 week semester so the first week we don’t do one of these. So we’ve got 15 of them and I try to kind of alternate between going back into the specific chapter, so they have to look at it and then, or else, just their own personal way of learning…. and try to line things up so that, to a certain extent, it goes along with what else they’re doing in the course… if it’s research or timelines or things like that. So every now and then I just had to look back and make sure that as I change the course activities that the metacognitive cafe discussions are still aligned correctly. And I think I like, John, some of what you said about having them read about the myth of learning styles and what not. Some of that material I send out in announcements throughout the week and just give them different resources that they can look through but maybe incorporating some of that more, because some of that is hard to dispel.

John: … and some of them resist it. When I’ve done that you, some of them said they just don’t believe it. This semester, I had an interesting experience where I posted a short video as a prompt on one of them from the Learning Scientists (they have some nice videos there on how we learn) and a few students were saying “Well, they only talked about a couple of studies and I’m not convinced, I’d like to see more evidence.” So I posted a list of five or six papers and they were really grateful, and they were discussing those, and that kind of surprised me because usually in an informal discussion like that it tends not to get quite as technical and it’s been interesting. So from then on I’ve been adding some research papers on at least the things that are related to learning science and so forth, and just to provide more support.

Judie: That’s a good idea, to add that upfront in the question because sometimes I’ll follow up on something that they’re discussing and if the student isn’t… if they’re not posting till the end of the week and the late posters, sometimes the students won’t go back and read it so I may have… I may post some information that would reinforce or sort of re-explain what it is they are trying to talk about, but they’ve already moved on and they’re not going to go back and look again. So that’s one thing. That’s a tough thing in Blackboard… if there were a way that you were alerted that somebody posted, you know, responded to your post or things like that like… one of the failings of Blackboard. I think maybe some of these questions could have supporting material sort of embedded within and they could choose to engage with that or not. But that might be a good idea to try in the future.

Rebecca: I like the idea of sort of having the opportunity to engage with those extra materials but not a requirement to do so. So that might help to engage like the wide range of students that we have, some who might want to take the the deep dive, and others who maybe don’t want to, but at least you can hit them somewhere.

Judie: Right, yeah.

Rebecca: I know I allow them to engage with the subject matter. Yeah they might watch a short video, but they don’t want to read the paper.

John: And the way I’ve been phrasing is since then is I have the short video, they’re usually five to ten minutes, and then I’ll put “(For those who would like to see more evidence, here are some resources.)”

Judie: That’s a great idea.

John: So, one of the things we normally ask people is what would you like to try next? It doesn’t have to be related to this.

Judie: Actually based on what we’ve just discussed I think the next thing I would want to do is in embed some of the articles and videos that you refer to and that type of thing and I think that would be good just you know kind of update the questions… update what they’re working on and just help the students learn how to learn.

Rebecca: Great.

John: Great. it’s working really well. I was so impressed when I saw the results and you shared with us. Some of the comments from students that I had to adopt it right away, and I’m gonna keep doing it as long as it keeps working.
Rebecca: Yeah, I adopted a couple of the questions. I didn’t do the full thing, but I did use some of the questions. Some of the questions, especially the workspace one, for some of my students and it was really interesting so I’d like to figure out how, how I can do it more – yeah I want in.

Judie: I love encouraging the students to share something like their workspace. I love when they post a picture. You could see it even shows a certain amount of trust, you know, that they’re gonna show a picture of their room. One I remember really well, she studied in her bedroom. She added like a little desk next to her bed and she had a sloped ceiling and a calendar… like she used her ceiling as a calendar and wrote all her due dates on the ceiling over her bed. It was crazy but I thought… wow…. like you’re just putting it all out there when you’re sharing things like that and it’s…. it’s really really good to see that sense of community. So I just think… I think these little discussions that started out just trying to get them to read sort of compounded into all these other benefits that I didn’t anticipate at first, but it’s really great to see. So I’m glad you guys are interested in it too.

John: I think it’s come up probably at least a dozen times in workshops just a semester. I’ve suggested to many of our faculty, particularly in online classes, because it really does help build a sense of community that I’ve never seen in my online classes before.

Judie: Good. One… oh I know… okay I know I knew I had one thing…. I want to do and we, John and I, were talking about this the other day when we were arranging all of this is… I think it would be really interesting if we could go back over… this is my fourth semester of this now, so if we could kind of go back and see how the students responded to these discussions and especially the ones who said: “well I already knew this” or you know the ones who are in denial and then see how their final grades were and… kind of chart that over time. Maybe see how they do in their program and, as opposed to the students who are, who you can see by the responses, are more open and saying “oh I’ll try this” and “I’ll try that” and see how their final grades were and if there’s any patterns over time. I think that would be really interesting.

Rebecca: It would be really interesting if other people are adopting stuff too to actually have it in some different disciplines and…

Judie: Yeah, that would be great.

Rebecca: …to see how that how that might turn out.

Judie: … we can all do that.

Rebecca: ….sounds like scholarship of learning article coming your way.

John: My informal observation is that the students who tend to be the last ones to post are the ones who say “I just don’t buy this” and they generally haven’t put a lot of thought into it… and they also generally are not the strongest students overall.

Judie: Yeah.

John: …and it’s sometimes difficult to get through to them. It’s perhaps an example of that old Dunning-Kruger effect, which is that the students with the worst metacognition tend to be those who have the the highest impression of their abilities and so forth. Not always, but it’s a pattern that I think if we did do some sort of analysis of that I wouldn’t be surprised.

Rebecca: Well sounds like we have a hypothesis, perhaps amended. [laughter]

Judie: I think I would like to follow up on that I think it would be interesting. So let’s get in touch with our institutional research folks and see what we can come up with.

Rebecca: Sounds like a plan.

John: Okay, and we will post show notes that will include links which will include the resources that Judie mentioned, will include the questions that we’ve included in the notes as well as any other materials that we find interesting.

Rebecca: Thanks for joining us today.

John: Thank you for joining us again, Judie. You’ve given a lot of workshops here and we really appreciate it. You do some really good things.
Judie: Well, thank you. It’s been fun