In this episode we discuss the metacognitive cafe online discussion forums developed by Judith Littlejohn, an instructional designer and historian from Genesee Community College in Batavia, New York. These discussion forums are designed to help students improve their metacognition and learning skills while also fostering an increased sense of community in the course.
Judith is a 2014 recipient of the State University of New York Chancellor’s Award for Excellence in Professional Service and the 2016 recipient of the State University of New York’s FACT2 Award for Excellence in Instruction.
- Foundation for Critical Thinking
- Scholarship of Teaching and Learning at Buffalo State
- Tanner, Kimberly D. “Promoting student metacognition.” CBE-Life Sciences Education 11, no. 2 (2012): 113-120 (http://www.lifescied.org/content/11/2/113.full)
Rebecca: Our guest today is Judith Littlejohn, an instructional designer and historian from Genesee Community College in Batavia, New York. She is the 2014 recipient of the State University of New York Chancellor’s Award for Excellence in Professional Service and the 2015 recipient of the State University of New York’s FACT2 Award for Excellence in Innovative Instruction. Welcome, Judie.
Judie: Thank you.
John: Today’s teas are:
Judie: Oh, mine is a Twining’s Forest Blend that I got in Epcot.
Rebecca: Mine is English afternoon.
John: …and mine is a Tea Forte Black Currant tea.
John: Today we’d like to talk to you about the metacognitive cafe online discussion forum that you developed. What prompted you to develop this?
Judie: At Genesee Community College, we have a Provost who really promotes critical thinking and as part of her initiatives, in January 2014, we had a critical thinking workshop with Rush Cosgrove from the Foundation for Critical Thinking. He talked a lot about the “elements of thought” and different ways to critically analyze whatever we’re reading. So I took his elements of thought and incorporated them into my online discussions then and tried to focus the students on becoming aware of bias… and you know the different…. ‘cause I’m blanking out on what all the elements of thought are….Then shortly after that, in 2016, we had Dr. John Draeger come over and spend the day with us. He’s the director of the Scholarship of Teaching and Learning at Buffalo State and he has a blog called “Improve with Metacognition” and I found his ideas really interesting. He based a lot of what he does off of an article by Kimberly Tanner. She’s a biologist; she’s at San Francisco State University and she wrote an article about helping students in biology with their metacognition, and she had a table of discussion prompts that she uses. And so John Draeger shared Kimberly Tanner’s table of prompts, and it was all about what they were getting out of the reading, and being aware of how they learn, and what’s successful for them in their struggles to try to learn all this biology content. And so Draeger works with grad students and a lot of what he did was really interesting but, where we are in the Community College, we have the fundamental problem where it’s a struggle to get the students to read. So I thought it would be great to implement questions that help students reflect on how they’re learning what they’re reading, but initially we have to get him to read. So I took the table of prompts that Draeger had and kind of broke them down a little bit in a way that would force the students to have to read at least parts of the chapter in order to answer the questions, and so that led to a series of low-stakes discussion questions separate from the content-focused, you know, the higher-stakes discussions that they do throughout the term, and so the students have to engage with specific parts of the chapter of the book and discuss those. And then from that I kind of branched out into other ways that they think about how they’re learning, think about how they’re reading, and how they can transfer their knowledge, and things like that, and just become aware of the process of learning, thinking about how they learn and thinking about thinking, which, of course, is what metacognition is.
John: How do students respond to just the term “the Metacognitive Cafe?” Many of them probably haven’t heard that term before.
Judie: Well I do put the definition at the beginning of every discussion prompt. I just copy and paste it into each discussion.
John: So to remind them.
Judie: It reinforces what you’re doing for consistency. So they they see the definition all the time. Whether or not they read it, I don’t know, but I noticed that over the course of the semester – at first they’ll call their discussion post… you know… like “meta,” “meta one,” “meta two,” and now we’re into week… we just started week 11, I think… and so they’re starting to write out the words “metacognitive cafe” and so, to me, that indicates that they’re focused a little bit more on what they’re doing. They’re paying attention a little more, I think… I survey them a lot to try to get feedback on what they think of the discussions and it’s been overwhelmingly positive. I think one of the byproducts is it builds a lot of community in the class. The students are sharing what they’re struggling with content-wise and how they’re approaching content that’s unfamiliar to them… how they reread things… or if they take notes, and things like that, and they’re really giving each other advice and strategies.
Rebecca: This is taking place in an online environment, right?
Judie: Right. It’s completely online. I have two courses right now: History 101 (which is Ancient World) and History 104 (which is Early Western Tradition)… and so currently those two courses are engaging in these discussions.
John: Earlier, when I saw you present on this I was really impressed by it, so I’ve tried it myself and it’s worked really well. One of the things, as you mentioned, is that students start talking about their struggles in the class and they start to get to know each other a lot more, at least in my experience, and you had relayed the same sort of thing when you talked about it earlier. Could you talk a little bit about that aspect of it?
Judie: Yes, I find that. Maybe, I’m not sure if it’s because it’s low stakes and the pressure is off, but the students share a lot more in these discussions. They tell surprisingly personal stories of what their home life is like. I ask them to talk about their study space, or their work space. Do you read and write in the same location? Do you try to read in a quiet place? or whatever… I encourage them to share pictures of where they do their assignments and it’s surprising how much they share. I feel like you can kind of,,, you know… I think over the course of an online term the students kind of group together… by the early posters and the late posters. But it really is solidifying this community, and they’re supportive of each other and they’re talking about what they have in common: if they have the same major… if they have the same struggles… I think a lot of it is finding that other students are struggling with the same content… that they are is really affirming for them. They feel, you know, like: “I’m not the stupid one, I can learn this. If they figured it out, maybe I can too.” And they they really are supportive. So it’s been pretty interesting to watch.
John: …and they also share study strategies. When you talked about learning spaces, they probably… at least in my class I’ve used some of the same prompts…. they’ll often talk about how important it is to have a place where they can have focused concentration without interruptions… and for some other students that’s a bit of a surprise so…
Judie: It is. I have students who work two jobs… they go to school…. they have kids… they’re primary caregivers for their aging parents… you know… all kinds of things are going on… and they’ll write about how: “Well, I’ll read for fifteen minutes in my car before I go into my second job” or whatever, and a lot of them are studying in a very haphazard way like that… and I think they watch the posts of the students who say: “oh, I plan this into my day. I schedule my time…” and things like that. There’s a lot of discussion of time management, and it really wakes some of the students up to say “Okay, if my education is a priority, my studying has to be a priority too…” and then they think… they try to reorganize their time a little bit better.
Rebecca: What role do you play in the conversations because it seems like that’s moving away from the content area stuff to more about like… how you… how you are a learner… how you exist as a learner. One of the things that we talked a little bit about in a reading group that we had on our campus recently was about the faculty member becoming a little vulnerable in certain circumstances, so that students can relate to them a little bit more… and they’re not just like some sort of authority figure who has no emotions.
Judie: Mm-hmm. So with these discussions… I try to… I don’t insert myself a whole lot, but I try to get in there a little bit now and then. I use… we use… Blackboard as our learning management system, so I do turn on the post ratings. That means you can give the students stars on their discussion: 0 through 5 stars. So, whenever somebody posts what appears to be a genuine response in their initial post, I give them the five-star rating… so they know… and they can tell that the instructor gave them the rating…. so they know that I’m right there, that I read what they what they wrote. Even if I don’t comment, they see the stars… and then where I feel like it’s appropriate, then I will comment on somebody else’s posts… and I do so. In the one where we talk about our workspace, I do share a picture of my office set up at home… I teach as an adjunct, so most of this is going on when I’m at home after my workday… and so I show them that… and there’s a few different things that they come up with. I’m trying to…. just off the top of my head… at one student. So, I have one pretty basic question, which is “why is it important to read the chapter before you start to take the quizzes?” and one of the students made a great analogy… and she said “I read the chapter before I do the quiz or before I start the writing assignment because if I was trying to change a tire on my car and I had no idea how to – where to find the jack, how to how to take the tire off, where any of the tools were, and how to go about the process, I would not be able to change the tire in my car – and it was pretty good… and so I jumped in on that one and I said well I really liked your analogy… and then that kind of led to more discussion of analogies and things like that… which I like. Because I think sometimes, when the instructor posts, it kind of ends the conversation, and sometimes people won’t post after the instructor. So I like that one but, so yeah, I’m just kind of on the sidelines I’d say… and try to jump in now and then where I feel like it’s not going to hinder the conversation and….
John: but support it….
Judie: Right, yes…support it.
John: Going back to a point you made earlier about things bleeding through and them getting to know each other, the first time I use this was last spring in a labor economics class where I, in some of the discussions I’ll ask them to talk how the material relates to their future career or how it relates to other classes they’ll be taking… and they started opening up quite a bit in my classes too: about their work environment… about their family environment… about having to work a couple of jobs in some cases… or raising small children… or having relatives who had health issues that they had to assist with… and what was really interesting is that bled over into the content discussion. So when they were talking about various labor market topics, they make references to other people’s living conditions, and how they might be able to relate to this concept, or how the things they had said in the other discussion were relevant here… and it was nice to see that bleed through… because they they were making posts that were much more meaningful in both discussions than I’d normally noticed in the past.
Judie: I noticed too that they they post more, meaning: more sentences… more…. they seem to become a little bit more articulate in their posts, instead of just barely hitting the key words and and calling it good. They really elaborate on their ideas more… and they respond to each other a lot more so instead of – I mean there’s always a handful of students who respond to another student and say “I agree” but I find much more content like “I agree, especially when you said whatever… or when you said X it reminded me of Y” and so I find that the depth of their discussions is a lot better.
Rebecca: Have students articulated that the metacognitive cafe is something that they’re finding really beneficial? Other than your observations, have they actually articulated that?
Judie: Yes, they also… I survey them in addition to the institutional end-of-course survey. I do give them informal surveys in the class through Blackboard and I typically do a half-time survey and then an end-of-course survey, and I give them five extra credit points so that does encourage them to answer – I can see who responded, but not what they said, so it’s still anonymous in that respect – and we just finished up the half-time survey and I wrote down a quote that one of the students wrote. In the surveys that I write, I can ask very pointed questions about what we’re experimenting with and, as an instructional designer, I try all different things in my classes and I want to get feedback while it’s live and while we still have the other half of the semester to make changes. But one of them in History 104, the Western Tradition, said “the metacognitive cafe is a great idea… it seems like a small break from ordinary coursework where you can actually talk about how you do the coursework, which is interesting. It seems to me that most people enjoy the metacognitive cafe.” So I like that… and a quote I had from the past, I think from last semester, was: “the discussions sometimes did not mean much to the coursework, although it did help me learn how to learn the material better.”
I was having a conversation with my mom yesterday about teaching and she was asking me a lot of questions about teaching online and I said you know my whole thing is: of course I want them to grasp the learning outcomes, but I don’t care if they memorize dates and names as long as thematically they understand the major themes throughout history, but if if I can help them learn how to learn, then they’ll be unstoppable. That’s my whole goal… to make sure the students can figure out how to tackle some new ideas… and figure them out…. and look at them with a critical thinking perspective… and make the most of it ….and take it with them.
Rebecca: That’s something that I really value in my classes too. I teach mostly web design classes, so students really have to learn… to learn… because the stuff changes all the time.
Rebecca: …and we don’t realize how important that is, and so it seems to the metacognitive cafe is a good opportunity to help students kind of move away from the idea of fluency illusion… or the idea that like… oh this is really familiar…. so therefore I know it – kind of recognizing that there is a way that we retain information and we need to practice it and retrieve it and all of those sorts of things and by having kind of those guided questions they become more aware that’s even a thing right.
Judie: Yes, I agree. I think it’s been good. I think it’s not a whole lot of effort on my part and it’s not… you know, I estimate – every week I give the students a checklist of what they have to accomplish that week and approximately how long each activity will take and I usually put about 20 minutes for this. So I don’t think it it takes a lot of time for them to go through the process of typing up their response. I don’t know how much time they spend thinking about it, but I do think it’s really been helpful, and I think it’s worth carving that time out of the week for the students to do that.
Rebecca: Can you share some examples of some of your questions?
Judie: Let’s see…. I brought an index card with a couple of questions. So the first one that I asked them is… that first week… so say in History 101, the Ancient World, we talk about the Agricultural Revolution and I just fundamentally say “What did you already know about the Agricultural Revolution and what did you learn in this chapter that was new to you?….” and they can’t answer that if they don’t open the book.
John:…and that activates prior knowledge and it helps them to make…
Judie: Oh, absolutely. But remember, my first goal is to get them to read, so at least they have to look at the book. They’ve got to at least read the five or ten pages on the Agricultural Revolution and find some new idea that they can point to.
John: … and that’s a little nudge to get them to practice…
Judie: Exactly. …and I follow it up the next thing they do… well every week, every chapter, they do mastery quizzes, and if they don’t read the book first, the mastery quizzes take close to three hours and I tell them to budget two hours every week. You know, after they spend an hour reading then two hours for the quiz.
John: Are these the InQuizitive quizzes from Norton?
Judie: Yes, I use Norton InQuizitive. InQuizitive, if you’re not familiar with it, it’s sort of a gaming method of quizzing, and so the students have to wager. They wager points according to how confident they are that they know the answer, and they have to earn fifteen hundred points in the quiz to get ten points in the grade book.
John: So, if they’re wrong they lose points. so it forces them to think about how well they know this, which is also another way of developing metacognition.
Judie: Right, how well they know it. It’s set up really well to keep targeting the questions that they’re getting wrong and it tells them where to look in the book. It gives them a lot of helpful feedback and the students overwhelmingly like those quizzes. They dislike them at first because they’re not used to their chapter quiz taking so long, but once they get into it and get used to how the system works, then I think it’s… they really do like it. They’ve got different kinds too. So you can watch a short video clip and then answer a question, or they’ve got sorting and putting things in order… which I love …. I love to do timelines and cause-and-effect type things
John: It helps them make connections across events….
Judie: Right. So, it’s not just multiple choice quizzes, or just true and false. So, I like that a lot. But, yeah… so that was one question [for a metacognitive cafe discussion prompt]… So, yeah, I asked, what interesting fact did you learn? Oh, then another week, I’ll say “Did anything new or interesting in the chapter change the way you think about X, like change the way you think about the Middle Ages?” ….or what information changed any preconceived notions that people have about certain things, like: “What did you read that….and I think students have trouble… I think at this level the students have trouble understanding the difference between what they learned in high school and what history is in college because they don’t understand that high schools teach them citizenship and not history….not major global historical themes. They think they know global history, but they really don’t… and so a lot of what they learn in college is pretty eye-opening, if they will admit that their eyes need to be opened.
John: Because your students come in with pre-existing models of how the world works…
John: …and much of it is wrong…. and
John: …we need to tear that down but they have to recognize it… and forcing them to confront it and think about that and reflect on that…
John: …could be really helpful.
Rebecca: Especially because students… they’re trying to just get the grade sometimes… especially if they have a lot of other things that they’re balancing in their lives and they don’t take the time to reflect… so by building it into your course, right… like when I finally get students to kind of make those connections….but we have to… it becomes sort of the responsibility of a faculty member to help students develop that practice, because it’s not intuitive and it’s not something that, you know, it seems like an extra, right?
Rebecca: After a little thing but… I don’t think they always see the value in it until they’ve done it.
Judie: Right. Yeah, and it’s tough…. I think it’s really tough with a gen ed that, you know. If I have a class of 32 students, typically there’s one who will say they want to be a historian; all the rest of them are only there because they need a world civilization requirement and they don’t want to take the course. They’ll say it right out: “I’m not interested in history,” “history is boring,” and so, you’ve got that uphill struggle right there… and then for them to realize they need to take serious time on this course that is not in their program that, as they see it it, can be tough. I have a couple students right now who just don’t seem to understand why you would have to take history, say, if you’re going to be an artist. Which, [laughing] how could you not want to take history as an artist…
Rebecca: …as an artist… what the heck?
Judie: It’s like influencing artistic movements like history and art, I think reflect each other and why would you only want half the story? But so, that’s tough.
Rebecca: I’m so glad that you brought up the stuff about general education because I think a lot of faculty struggle with some of the same issues, right, like “why am I here? and then trying to get those students engaged is such an upward battle or at least it feels like it is. Are you finding that the metacognitive cafe is starting to convert some of those folks?
Judie: A little bit… a little bit… that… and I also work really hard in all my classes to help the students find something in history that they can kind of latch on to thematically… So early on it within the first couple weeks of the class they need to find a theme to track so if you if you are an artist, you could choose art, or if you’re a sports person you could choose a sport history, and kind of track how what changes and you know what new trends and things started throughout whatever era we’re studying… and I think that helps a lot too… because…
John: …it gives them a more personal connection.
John: It’s also activating their prior knowledge and information.
Judie: Right…and so that’s helpful and do have them kind of share that too…. you know, what topics they’re researching and why… and then, and that’s fun. Those discussions are fun because when students are working on research papers and they spend all this time looking for sources and writing their outlines and all that and finally writing the paper, to finally be able to share what they’ve been researching I think is really good. It’s easy to do that in the classroom, but online it’s a little tougher, so I try to make sure they have a forum where they can point out what the most interesting things they discovered during their research process… where they’re finding sources… and the different facts that they will discover and I think it’s helpful.
John: One of the things that I’ve asked in my classes since I’ve been doing this (at the end of the term in the last metacognitive cafe discussion) is to reflect back on the class… and I asked them also if there’s anything… based on your question, one part of it is if what they’ve learned has caused them to alter their view of the world, or what was the most surprising thing they learned in the course…. and what’s been really interesting is the most common response is that what they enjoyed the most were the metacognitive cafes. One of the things I think we both do to some extent is we try to nudge them towards better learning strategies and as part of that, I have some on spaced practice, interleaved practice, and so forth, and also having them break that myth of learning styles that they’ve come in with… and what they’re always saying is that they amazed that so much of what they had learned and what they had been told in their earlier classes about rereading and highlighting and focusing on their learning styles is not based on evidence and many of them are saying they wish they had learned much of this stuff back in elementary school and it would have made their educational career much more productive.
Rebecca: Sounds like you know in in both of your cases an interesting way of you know if it’s a general education class that maybe students aren’t buying into the subject matter, at least they can buy into the idea of like learning how to learn and finding strategies and things that could apply elsewhere through the lens for a particular discipline which you know, if anything, at least that’s something that they can like connect to if they’re finding it hard to connect to the content.
Judie: Yeah that’s that’s a good insight.
John: Now one thing we we usually ask is: “what lessons have you learned while doing this? How has your practice changed over time?
John: It sounds like it’s worked pretty well from the beginning.
Judie: I think a few of the questions could be a little bit more thoughtful. I don’t know. I asked some things about you know like motivation and how will they transfer the skills they’re learning in this class to their other courses but sometimes I try to mix up, so like throughout we have a 16 week semester so the first week we don’t do one of these. So we’ve got 15 of them and I try to kind of alternate between going back into the specific chapter, so they have to look at it and then, or else, just their own personal way of learning…. and try to line things up so that, to a certain extent, it goes along with what else they’re doing in the course… if it’s research or timelines or things like that. So every now and then I just had to look back and make sure that as I change the course activities that the metacognitive cafe discussions are still aligned correctly. And I think I like, John, some of what you said about having them read about the myth of learning styles and what not. Some of that material I send out in announcements throughout the week and just give them different resources that they can look through but maybe incorporating some of that more, because some of that is hard to dispel.
John: … and some of them resist it. When I’ve done that you, some of them said they just don’t believe it. This semester, I had an interesting experience where I posted a short video as a prompt on one of them from the Learning Scientists (they have some nice videos there on how we learn) and a few students were saying “Well, they only talked about a couple of studies and I’m not convinced, I’d like to see more evidence.” So I posted a list of five or six papers and they were really grateful, and they were discussing those, and that kind of surprised me because usually in an informal discussion like that it tends not to get quite as technical and it’s been interesting. So from then on I’ve been adding some research papers on at least the things that are related to learning science and so forth, and just to provide more support.
Judie: That’s a good idea, to add that upfront in the question because sometimes I’ll follow up on something that they’re discussing and if the student isn’t… if they’re not posting till the end of the week and the late posters, sometimes the students won’t go back and read it so I may have… I may post some information that would reinforce or sort of re-explain what it is they are trying to talk about, but they’ve already moved on and they’re not going to go back and look again. So that’s one thing. That’s a tough thing in Blackboard… if there were a way that you were alerted that somebody posted, you know, responded to your post or things like that like… one of the failings of Blackboard. I think maybe some of these questions could have supporting material sort of embedded within and they could choose to engage with that or not. But that might be a good idea to try in the future.
Rebecca: I like the idea of sort of having the opportunity to engage with those extra materials but not a requirement to do so. So that might help to engage like the wide range of students that we have, some who might want to take the the deep dive, and others who maybe don’t want to, but at least you can hit them somewhere.
Judie: Right, yeah.
Rebecca: I know I allow them to engage with the subject matter. Yeah they might watch a short video, but they don’t want to read the paper.
John: And the way I’ve been phrasing is since then is I have the short video, they’re usually five to ten minutes, and then I’ll put “(For those who would like to see more evidence, here are some resources.)”
Judie: That’s a great idea.
John: So, one of the things we normally ask people is what would you like to try next? It doesn’t have to be related to this.
Judie: Actually based on what we’ve just discussed I think the next thing I would want to do is in embed some of the articles and videos that you refer to and that type of thing and I think that would be good just you know kind of update the questions… update what they’re working on and just help the students learn how to learn.
John: Great. it’s working really well. I was so impressed when I saw the results and you shared with us. Some of the comments from students that I had to adopt it right away, and I’m gonna keep doing it as long as it keeps working.
Rebecca: Yeah, I adopted a couple of the questions. I didn’t do the full thing, but I did use some of the questions. Some of the questions, especially the workspace one, for some of my students and it was really interesting so I’d like to figure out how, how I can do it more – yeah I want in.
Judie: I love encouraging the students to share something like their workspace. I love when they post a picture. You could see it even shows a certain amount of trust, you know, that they’re gonna show a picture of their room. One I remember really well, she studied in her bedroom. She added like a little desk next to her bed and she had a sloped ceiling and a calendar… like she used her ceiling as a calendar and wrote all her due dates on the ceiling over her bed. It was crazy but I thought… wow…. like you’re just putting it all out there when you’re sharing things like that and it’s…. it’s really really good to see that sense of community. So I just think… I think these little discussions that started out just trying to get them to read sort of compounded into all these other benefits that I didn’t anticipate at first, but it’s really great to see. So I’m glad you guys are interested in it too.
John: I think it’s come up probably at least a dozen times in workshops just a semester. I’ve suggested to many of our faculty, particularly in online classes, because it really does help build a sense of community that I’ve never seen in my online classes before.
Judie: Good. One… oh I know… okay I know I knew I had one thing…. I want to do and we, John and I, were talking about this the other day when we were arranging all of this is… I think it would be really interesting if we could go back over… this is my fourth semester of this now, so if we could kind of go back and see how the students responded to these discussions and especially the ones who said: “well I already knew this” or you know the ones who are in denial and then see how their final grades were and… kind of chart that over time. Maybe see how they do in their program and, as opposed to the students who are, who you can see by the responses, are more open and saying “oh I’ll try this” and “I’ll try that” and see how their final grades were and if there’s any patterns over time. I think that would be really interesting.
Rebecca: It would be really interesting if other people are adopting stuff too to actually have it in some different disciplines and…
Judie: Yeah, that would be great.
Rebecca: …to see how that how that might turn out.
Judie: … we can all do that.
Rebecca: ….sounds like scholarship of learning article coming your way.
John: My informal observation is that the students who tend to be the last ones to post are the ones who say “I just don’t buy this” and they generally haven’t put a lot of thought into it… and they also generally are not the strongest students overall.
John: …and it’s sometimes difficult to get through to them. It’s perhaps an example of that old Dunning-Kruger effect, which is that the students with the worst metacognition tend to be those who have the the highest impression of their abilities and so forth. Not always, but it’s a pattern that I think if we did do some sort of analysis of that I wouldn’t be surprised.
Rebecca: Well sounds like we have a hypothesis, perhaps amended. [laughter]
Judie: I think I would like to follow up on that I think it would be interesting. So let’s get in touch with our institutional research folks and see what we can come up with.
Rebecca: Sounds like a plan.
John: Okay, and we will post show notes that will include links which will include the resources that Judie mentioned, will include the questions that we’ve included in the notes as well as any other materials that we find interesting.
Rebecca: Thanks for joining us today.
John: Thank you for joining us again, Judie. You’ve given a lot of workshops here and we really appreciate it. You do some really good things.
Judie: Well, thank you. It’s been fun