Category: team based learning

302. Flipped Team-Based Learning

Posted in assessment, authentic learning, equity, evidence-based practices, flipped class, higher education, learning, mastery learning, pedagogy, retrieval practice, team based learning, two-stage exam
August 16, 2023

Flipped classrooms allow for class time to be used to put content into action. In this episode, Tina Abbate joins us to discuss the team-based approach that she uses in her classes to help develop the real-world skills important in her field.

Tina is a Clinical Assistant Professor at Stony Brook University’s School of Nursing. She holds a collection of credentials including a PhD, MPA, an MS, and is a registered nurse (RN). She teaches in-person and online undergraduate nursing classes at Stony Brook and conducts research on active learning strategies and the retention of information. She works as a nursing supervisor at two local hospitals.  She is the recipient of the 2023 SUNY FACT2 Award for Excellence in Instruction and was a recipient of the Stony Brook University Award for Excellence in Teaching an In-Person Course.

Show Notes

Transcript

John: Flipped classrooms allow for class time to be used to put content into action. In this episode, we look at one instructor’s team-based approach that emphasizes real-world skills important to the field.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist….

John: ….and Rebecca Mushtare, a graphic designer….

Rebecca: ….and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Tina Abbate. Tina is a Clinical Assistant Professor at Stony Brook University’s School of Nursing. She holds a collection of credentials including a PhD, MPA, an MS, and is a registered nurse (RN). She teaches in-person and online undergraduate nursing classes at Stony Brook and conducts research on active learning strategies and the retention of information. She works as a nursing supervisor at two local hospitals. She is the recipient of the 2023 SUNY FACT2 Award for Excellence in Instruction and was a recipient of the Stony Brook University Award for Excellence in Teaching an In-Person Course. Welcome, Tina.

Tina: Thank you. Thank you so much for having me here today.

John: We’re very happy to see you again. We saw you at the SUNY Conference on Instruction and Technology (or CIT) about a month or so ago. And our teas today are:…. Tina, are you drinking tea?

Tina: I am. I am drinking a chai tea. Very good.

Rebecca: That sounds nice and warming.

Tina: Yup.

Rebecca: It’s a little chilly here, although it’s summer and it was hot yesterday. It is not hot today. [LAUGHTER]

Tina: Yes, for sure the weather has been very odd.

Rebecca: So I have my tea for teaching mug today. And in it, I think actually a mix of a couple of different black teas because I switched when I had a half a cup left. [LAUGHTER] I’m not sure what we call this today, but it’s a mix of black teas.

Tina: That sounds delicious.

John: Well, it sounds like a great tea to have while discussing blended learning.

Rebecca: A high quality blend. [LAUGHTER]

John: And we’re having a real cold spell here in Durham, North Carolina. The temperature has dropped down to 87 today, and I am drinking a tea forte black currant tea.

Rebecca: That’s a nice summer tea.

John: It is.

Rebecca: So we’ve invited you here today to discuss your use of active learning tools. But before we jump into that, we were curious about your wide range of degrees, credentials, and certifications. We didn’t even list them all. Can you share a little bit about your pathway into your current position at Stony Brook?

Tina: Sure. Well, when I went back to grad school, I certainly didn’t intend to get three graduate degrees. I had gotten into Binghamton and gotten into their BS to PhD program because I wanted to do research and my ultimate goal was to do executive leadership position at a hospital because I really enjoyed the leadership role of nursing. So just to backtrack, I graduated Binghamton University in the year 2000 and started right in the NICU (neonatal ICU) at Stony Brook and I worked as a NICU nurse for six years. And in that time, I knew that I wanted to go back to school. And like I said, I got into the BS to Ph. D. program at Binghamton. They awarded me a fellowship program. So I moved from Long Island. My daughter was one at the time. And I started my education there at Binghamton, continued it for the graduate program. And about a year into my doctoral studies they had asked if I wanted to teach clinical and I’ve taught in other capacities. I used to teach violin and piano when I was younger and I never really thought of teaching as a career goal for me. However, I was a poor graduate student, and I said, “Sure, I’ll do it.” And I had about six students in the NICU. I was teaching clinical, and, I don’t know, something came over me. I found my professional soulmate, something clicked so hard for me in that clinical that I wanted more. So I continued asking for teaching assignments. And it’s hard to articulate the feeling that you have, but I felt like I found my niche. And so I did clinical instructoring for about six years and then I moved into the classroom setting. So at that time, I still worked as a nursing supervisor, so I enjoyed the leadership role. And Binghamton started a dual master’s degree program, where you get your master’s in nursing with a concentration in whatever you wanted, I chose education. And the other part of the dual degree was a Master of Public Administration. So I was the first cohort to move through that program. So I graduated first with my Masters of Science in Nursing and my functional role was educator. Then two years later, I completed the Master of Public Administration, and then eventually the PhD. And it all just aligned so perfectly in my current career, because obviously I’m an academic at heart through and through. So those degrees have assisted me in that role. I still work in administration. I teach research, I teach leadership and management. So all of the degrees I’ve utilized and I still utilize actively every day. So this pathway was kind of carved out for me, I think, and I just feel very fortunate that I’m able to apply all of the degrees that I’ve gone for.

John: At the SUNY Conference on Instruction and Technology, you gave a presentation on how you structure your courses. And you mentioned that you were using a flipped team-based learning class structure. Can you tell us a little bit about how your classes are structured, and what a typical class day would look like in one of your classes?

Tina: Sure. So any class that I’m involved in or coordinate, the structure that I utilize is a flipped team-based learning approach. And this essentially requires students to prepare prior to coming to class. It has some benefits there, there’s flexibility, students can learn at their own pace, it really amps up the student responsibility for learning, as we know, and then it also gives us the opportunity for higher level learning because they’re interacting with the concepts outside prior to class. And the team-based part of it I like is because that increases that collaboration amongst students. We know that nursing healthcare is a team sport, so I like to engage the students in teamwork so that they can collaborate and work on their team dynamics, and their own personal team skills. So how my classes operate is, prior to each class, students complete a set of videos, and they’re interactive videos, they’re accessible videos for all types of learners, and it carries weight in their grade. So basically, in these pre-class videos, students get a little voiceover content from me about a concept, and then they get tested on it using a variety of types of questions: matching, true-false, multiple choice, hotspots, you name it. As they move through the videos, they are taking notes on a note-taking guide. So all the concepts are there for them to just follow along, take notes. So they’re seeing, hearing, they’re doing something as they move through the videos. And that note-taking guide eventually acts as a study guide for them, because they have to take a quiz every single class. So they complete these videos before class. And then I start each class with a micro-lecture review using Kahoot!, which is just a game-based learning platform. And in this micro-lecture review, I’m really drilling down to the concepts and helping these students reconcile any last residual confusion that they may have about these concepts. And then after the Kahoot!, they take a quiz. Now, since they’ve interacted with the concepts so many times prior to taking this quiz, I push the level of the questions in these quizzes. There are 15 questions and I try to push the level as high as I can. And the students are able to rise to the occasion because they are not hearing the information for the first time when they walk into class. They have a vague sense of the concepts, we nail it down, and then they take the quiz. After the quiz, the rest of class is comprised of team-based activities. And that’s how every class looks like for me.

Rebecca: Can you talk a little bit more about the embedded questions that you have in the videos and how students have responded to that aspect of a flipped classroom?

Tina: Absolutely, I use a program called Articulate 360. Articulate 360 has many different types of functions in it. But I focus more on the storyline aspect of this product, where I’m able to set up these video clips. So if you already have voice overs, you can basically chop up that voice over into different bits, put it into a story, the type of file that they reference there. And then in between each clip, you can embed any type of quiz question that you could possibly imagine. And you can set up different parameters. So for example, I like to elevate the stakes a little bit, so the students, for these pre-class videos, the grade that counts is their first pass. So it’s not like they can retake the video for a higher grade. It’s whatever they get at the end of that first pass of the video is the grade that counts. And they have two opportunities to answer each quiz question correctly. And I also embed a lot of feedback, so if they get the answer wrong, they’ll see a pop up with some review, and then if they still got it wrong, or they got it right, then there’s an explanation that pops up for the right answer. So I do survey my students in the middle of the semester using a Google form. And then at the end using the university platform, and the feedback about the videos has been very positive, they really do appreciate even though it means extra work, I’m still not giving them 20 chapters to read. I’m giving them something that passes along a bit more quickly and has a better chance of sticking in their memories. And they also appreciate the note-taking guide because it also becomes a study guide, not just for the quiz, but for the final exam at the end.

Rebecca: Like I’ve counted four or five layers of countability on that same content. [LAUGHTER]

Tina: Exactly.

Rebecca: We’ve got the note taking guide. We’ve got the embedded questions, and we’ve got the Kahoot!, and then we’ve got the quiz, and then the exam at the end.

Tina: Yeah, so it’s all about building on these concepts, having the knowledge and then being able to apply it in the classroom

John: In your presentation, you mentioned that you were de-identifying the names of students taking the Kahoot!, but maintaining a leaderboard in the classroom. Could you tell us a little bit about how that works.

Tina: So Kahoot is based on answering the questions correctly or incorrectly. And part of the score is how quickly you answer the question. So ideally, you want to answer quickly and answer the questions correctly. So at the end of the Kahoot!, they get a score. And just again, to raise the stakes, students have to hit a certain benchmark of points to receive full credit. And I try to push that benchmark a little bit, not to make it impossible, but just to make it a little bit challenging for them to give them something to work towards. So for example, in one semester, they have to reach 70,000 points to get the full credit, and then it’s prorated from there. So every time I have a class, I load the data into this program that was built by our Center for Excellence in Learning and Teaching on campus. One of the computer scientist was able to put this leaderboard, showed me how to upload the files, which are basically just CSV files. And what it does is this leaderboard shows their rank in the class, their total score, and the score for that week, so that they can monitor their progress. And everybody else is de-identified and random words, but they can see their name, and they could see their rank in the class.

John: And one of the advantages, I think, of using Kahoot! is it does provide some practice in developing automaticity. So that students can practice retrieving information quickly, which I would think would be especially important in health-care situations.

Tina: Absolutely. And I’ll have some students that come to me and they just absolutely despise Kahoot! because of the stress. And if you’ve ever taken a Kahoot!, and I have, it is stressful, you have to really think on your feet very quickly, especially since your score is based on how fast you answer the question. So what I tell them from the beginning is if you really are struggling with Kahoot!, and you don’t like Kahoot!, Kahoot is really for you, it’s meant for you, because I want you to think of a situation in a hospital setting. If a patient is deteriorating, we call something called a rapid response. And a team of people flow to the room to address whatever issue it is, perhaps the patient’s having difficulty breathing, whatever, chest pain, this now has become a very emergent situation. And in that situation, you have to be, as the primary nurse or a nurse assisting someone else, you have to have laser focus, and someone may ask you to just go get a piece of gauze. And if you’re new in the role you may be so flustered, just by getting that piece of gauze. So, this is really like a precursor to that. So I tell the students to use Kahoot! as a mechanism to help with your laser focus in situations where the outcome is dependent on what you’re doing.

Rebecca: Another thing that seems really relevant to a healthcare setting is the team-based learning aspect of your course. Can you talk a little bit about how you arrange the team-based activities and also how you set your students up for success on teams.

Tina: So with team-based learning, as we know, it’s simply a collaborative learning strategy and how the team activities look depends on the course. So I can talk to you about my research course. That happens every fall semester, and I have 160 students, this is the graduating class. These are the seniors, they’re in the last two semesters of the program. And what we do in that course is the team-based activity portion of class is working on a project. So I’ll tell you a little bit about the project which is experiential in nature. Stony Brook University is attached to Stony Brook University Hospital. So every year, I pick a unit, I meet with the manager, and they give us a clinical problem to solve. So for example, this fall the students and I will be working with the surgical ICU and the clinical topic is nurse wellbeing. So, as we know, we’re in this post-pandemic world and wellbeing has really moved to the forefront. Things like burnout, compassion fatigue is very prevalent in the healthcare environment and just globally as humans. I think we’re just a little tired of living in this fight or flight for so long. And now we’re trying to come back from this. So this fall semester, the students will be working in teams to find a solution for the surgical ICU for nurse wellbeing. So what we do is we search for articles together, and that’s how they get to their solution. We use a framework we use Melnyk’s seven steps for evidence-based practice. So in undergrad nursing, even though it’s called the nursing research course, the students are expected to utilize the research that has been done on a topic to make changes to their practice. Our expectation is not for them to actually conduct research. That’s a PhD level thing, but according to our essentials in baccalaureate nursing, that our accrediting body tells us what curriculum to teach to the students, the expectation is that they know how to read the research, how to critique it, how to appraise it, how to synthesize it, and how to use the research to develop solutions. So from there, they work in teams of eight throughout the semester, they develop their solution, they put it into a video project, the six-minute video project, and I choose the top two projects. Those top two projects then move on to the implementation phase. So then the unit will implement and evaluate the solution. And in addition to that, we put in for posters at conferences. For example, last year, we had two posters at ENRS. I was assigned the course of research, I was like, “Oh, boy, how am I going to make this interesting?” …because we know that research content can be a bit dry. So I ran the course for a couple years, and I knew that I had to do something with it. And that’s where I started moving towards this more experiential learning opportunity for the students. And so far, it’s been going really well.

Rebecca: So I heard you say something about teams of eight, and I almost maybe had a heart attack, [LAUGHTER] just thinking about how big that team is, and how to manage that. Can you talk a little bit about some of the structures you have in place to help a group that size, which is relatively large, be successful?

Tina: Sure. So teams of eight… that means I have 20 teams in total. And we’re all reviewing the same articles. So then I know the answers to all the questions. And basically, Google Drive is my answer. Every team has their own folder, within that folder are subfolders, I have them buddy up and be assigned to a certain number of articles. As a team, they have like individual and buddy responsibilities, which is clearly articulated in a contract that they review and fill out at the beginning of the semester. So they have individual responsibilities, buddy responsibilities, and they have team responsibilities. And every single class looks the same. So by the second class, they’re already into the mode. I don’t throw them any curveballs, every class structure is exactly the same, so they know what to expect. And they have appraisal forms to fill out. They have tables to fill out as a team to keep all of their literature organized. And the structure that I have in place seems to be working because there’s very little confusion now that I’ve kind of worked out all of the kinks. And I also always keep instructions projected just to make sure that they are apprised of the flow of class.

John: You mentioned Melnyk’s, seven steps of evidence-based practice. Could you give us a brief overview of that framework?

Tina: Absolutely. So there’s many evidence-based practice models out there. Stony Brook goes with Melnyk, and there are seven steps and actually I begin with step zero, step zero is igniting that spirit of inquiry. And that’s one of my main end goals of the course is for them to stay curious about how they can improve practice as a nurse for their patients. So that’s step zero. And then basically, what we do is we take the clinical problem, and then we frame it in the form of a question, a PICO question. And that helps us to find our articles. So once we find our articles, we go through the articles, we decide what we’re going to keep, what we won’t want to keep, then we start to critically appraise these articles, review them, read them, understand them, the students put that information into a literature review table, which is just the main elements of each article. After we’ve appraised all of the articles, the next step is to synthesize all of the articles. So what is the bigger picture? For that synthesis class students do complete synthesis tables. And when they create these synthesis tables, now the beginnings of their proposed solution begin to emerge. So then students put their solution together based on the synthesis table. And then the next step in this process is to implement the solution and then evaluate the solution. And of course, dissemination is always the last step.

John: You also mentioned that you use collaborative testing on exams. I’ve done this with a two-stage exam process where people take the exam individually first and then submit that but then take it again as a group. It also appears to have been and that’s been tremendously successful. It’s also appeared to have been really beneficial in terms of student learning, and it’s just so much more fun to watch the students work in groups on exams, than it is to go over the exam the next day with the whole class. That collaborative exam format has been so much much better than I ever expected it to be. Could you tell us a little bit about how you do collaborative testing on your exams?

Tina: I absolutely adore collaborative testing. If you have to assess students using exams, this is really maximizing the use of exams. So in my courses, students take collaborative exams in teams of three. And as we know, the research says that collaborative testing may decrease test-taking anxiety, the students have to take a large licensing exam at the end of the program. So it may help some of these students with that, like you said, immediate feedback on test performance, it really scales back the number of questions, I don’t even do exam reviews anymore, because the immediate feedback that they get, they’ve reconciled any confusion on the exam, that an exam review is no longer required. It increases student engagement and collaboration. I love how they, like you said, they debate, they discuss, that peer instruction. There are some people out there who can read a book and retain 100%. But generally speaking, you’ll have a better chance of retaining more information if you’re teaching someone else versus reading a book. Of course, that just varies learner to learner. So that could be really something to hone in on when it comes to collaborative testing. So, yes, the traditional way is to take the test individually, and then they take it again, in a team. And in our program, the clinical courses like medical surgical nursing, pediatrics, all of those, I would always recommend to do individual than collaborative because you really want to assess that individual on their performance and understanding of the concepts. And so I teach research, and I teach leadership and management, these are non-clinical courses, I skip the individual part, and take them right to a collaborative exam. So for example, for my research course, the students don’t know who they are paired up with, or in a team with, until about an hour before the exam. They get two articles, a quantitative article and a qualitative one. And then they have a set of questions to answer. Essentially, we’ve been preparing for this type of exam throughout the semester. So they end up doing really well. In my transitions to professional practice, where I’m teaching leadership and management, that is a traditional final exam, multiple choice, select all that apply, type of questions. And again, I actually do it on Zoom, they go into breakout rooms, they share their screen, and they take the exam, there’s a scribe who enters the answers. And also when it comes to accommodations, kind of as a side note, I’ve been able to set up strategies for individuals that do have accommodation so that they can maximize their experience as well.

John: When I first tried this, I was so excited about how the students were reacting with the collaborative exam that I took a short video clip while they were doing it and sent it to Rebecca. She was working with me in the teaching center at the time. It was just a remarkably positive experience.

Tina: Do you notice a difference like I would say an estimate of 10 points between the individual and the collaborative mean.

John: Generally, yeah. And the group one is virtually always higher than each individual score, except in one case in my class, where one student had a higher score than his group, and that’s because during the group discussions the student gave in to peer pressure within the group. I encouraged him to be more assertive when he’s confident about his answer. But that only happened with one student on one exam.

Tina: that’s pretty rare. I just love just watching them engage like that. So I’ll pop into like the breakout sessions, and they’re collaborating and negotiating and it’s just fantastic.

Rebecca: You mentioned earlier about your research class having a project coming up about well being. And I think that’s a topic that we’ve been talking a lot about in higher ed in a lot of situations. Can you talk a little bit more about that project and some of the research that’s going into it and some of the outcomes of it?

Tina: Absolutely. I mean, wellbeing is such a hot topic right now in probably every type of job you could think of. And it’s interesting wellbeing is kind of always been in the background. And I think the pandemic really shoved it into the forefront where it really should have been. That really needs to be, in my opinion, the top priority of any workplace because if your employees are well, it has a positive trickle down effect. So it has gotten to the point now where our accrediting body who tells us the essentials that we need to teach to our students, they have added a wellness component, and we’re adopting these new essentials in the next year. These are new essentials for us to follow. So it made it into the essentials, which is very telling. And now faculty are charged with teaching students, monitoring students, about their wellness and wellbeing. So this was pretty timely, because of the pandemic, the clinical topic that we’ve been doing for this EBP project has been things like compassion fatigue, burnout. And now this year, we’re doing wellbeing. Last fall, we worked with the cardiothoracic ICU. And the EBP project topic was compassion fatigue. And we wove in a lot of wellbeing into the solution, which is actually kicking off on July 1. So this year, instead of doing compassion fatigue, which has a bit of a negative connotation, let’s flip it to the positive. And like I said, we’re working with the surgical ICU, and we want to customize a wellness solution for that unit. So in the meantime, by proxy, I can teach the students about their own wellbeing and their own wellness. So I have a lot of content in there, so that they’re learning about this clinical topic to help develop a solution, but they’re also learning about it for themselves. And I do a few things with them, and definitely evolving this as we move along. And I’m lucky enough that I have the graduating class in the fall and the spring. So I move it through from the fall to the spring semester. So in addition and educating them on the different ways to promote your own wellness, we start each class with a mindfulness activity. I have a sound bowl that actually a student gifted to me, we do meditation, mindful breathing, every class is something different. This year, I’m inviting students to lead some of these sessions. So I want it to grow so that other students can participate and lead us and it’s literally three to five minutes at the beginning of every class, all lights down, devices off, phones flipped down, and we just take the time to be as present as possible. And I also help them keep an eye on their level of burnout. And I give them the professional quality of life survey at the start of every class. And halfway through, I’ll do a comparison of statistics between the different cohorts. Because I have the traditional cohort and I have the accelerated one, we look to see how our scores are doing over time, just to have that educational component to it. And then also the Insight Timer app, that’s an app that you don’t have, I would highly recommend that you download it. It has so many mindfulness type of activities that you can do. There’s a journal, you can track your progress. They have classes, and even the paid version, which is I think, maybe $60 for the year, they offer so many different bells and whistles, it’s really just a phenomenal app to use if you’re looking to promote your own wellness. So the other thing I wanted, I attended that CIT conferences, is I would love to use ChatGPT to develop a wellness assignment. So I’m still thinking about the inner machinations of how that would work. But hey, you know, if AI is here, might as well see if we can use it to promote wellbeing.

John: And it’s nice to have that focus of using ChatGPT positively because this is something that’s going to be part of students’ lives going forward, maybe not this specific tool, but AI tools are not going to disappear and using them for good would be a nice alternative for the concerns that many faculty have about the use of these tools. During your presentation at the CIT conference, you also mentioned using a variety of edtech tools. What are some of the tools that you use in your classes?

Tina: Sure. So I’ve trialed some apps here and there, I’ve used Plotagon. I used Go Animate for Schools, which is now VYOND, just for them to create case scenarios in their leadership and management class. And based on feedback, the one that they really liked is now a bit pricey. So I tried a free version of an app, and it really didn’t go well based on feedback. And that’s how it works in education. You try something out and you survey the students and if the experience over time is really not positive, you need to move on to something else. But things that really have stuck is I told you about Kahoot! and Articulate 360. How I communicate with the students. I use GroupMe. I prefer to communicate with them using that application over Brightspace or traditional email. They join via QR code and I have them all in one group chat and I can post quickly. They could send me direct messages, they could post questions in our group chat. And it just seems to really streamline communication because we’re all competing for their cerebral real estate, they have a lot going on, a lot of deadlines, so I find that this GroupMe app is really helpful. And I also try not to spam them with too many messages, thoug. It really seems to work. And then again, Google Drive, I can’t even begin Google Drive for everything, whether I want to survey them or whatever it is, Google Drive has it for us.

Rebecca: So speaking of Google, [LAUGHTER] you mentioned earlier using a Google form for a mid-semester evaluation. Can you talk a little bit more about that, and how you’ve used that to make adjustments in your class for the latter half of the semester?

Tina: Sure. So a Google Form is a pretty nice way to just give a quick survey to your students, I do that in the middle of the semester. And I have to tell you, that’s where I get my best data, because they are in the throes of it. And my response rate is typically over 90%, as compared to at the end, where they’re kind of just fizzling out, tired, maybe a bit over it, generally speaking. So I don’t get the response rate in the final that I do in the mid semester, when I analyze it, very short, a couple Likert questions: What do you like? What don’t you like? …and if there’s enough of a theme in the qualitative questions, or in the Likert scales, I’m able to make changes prior to them departing from me, instead of waiting for the next cohort to come in. For example, some things that came up was: “It can be a bit loud in the classroom.” So I’ve done something to control the volume in there, because it’s a very active classroom, or we feel like we’re sitting around too long during the TBL activities. So now I have a mechanism for them to let me know when they’re done with their activities, so that they’re not sitting around waiting. So those types of things. If they say, “let’s skip the final exam,” then that’s not anything that I can honor. But I’ve gotten some really good raw feedback that’s helped me evolve my classes. I’m just always so grateful for the student experience, because they inform me where this needs to go. Another way that I use a Google Form is with team-based learning. Michelson says that you should have the team members evaluate each other on their team performance. And typically, this is done at the end. But I like to do it in the middle of the semester, where they’re evaluating each other so that they have an opportunity to remediate, and then by the end, hopefully, their team’s performance scores have gone up. The challenge, though, with a Google Form is it’s very hard for me to share the feedback back to the students, it requires a lot of copying and pasting. And there’s a lot of room there for error, human error. So currently, I do bring in the students that are rated poorly just to give them some one-on-one guidance on how to improve their team performance. But in the meantime, to work around that I did trial a product called Kritik that offers that ability where the students will get their feedback back. But I reached out to our Center for Excellence in Learning and Teaching. And right now what we’re doing, we have a sandbox, and we’re working on trying to do a Kritik-like type of peer evaluation in Brightspace, using PeerMark. And we’re getting very close to ironing out some of the finer details. So I’m going to finally have an evaluation where every student can see their feedback from their team members based on their performance, so they know what they’re doing well and where they need to improve.

John: You teach both face to face and also online. Do you use many of the same techniques in your online classes that you use in your face-to-face classes? How do you modify your class for online delivery?

Tina: So I do everything the same, except that it’s in an asynchronous format. So students really have to be self disciplined in an asynchronous online type of environment. The online classes that I teach are post-licensure undergrads, so they have their two-year Registered Nurse license, and they’re looking to get their four-year degree. So some of the assignments, we tailor a little bit differently just because they have nursing experience, whereas my pre-licensure students do not. So maybe the assignments vary a little bit, but the structure is the same, using Articulate. I don’t use Kahoot! with them, only because I don’t have them in front of me, but they do have the quiz. And they have the TBL activities and things of that nature. So it’s the same, but it’s just in an asynchronous format.

Rebecca: I know that we mentioned in the intro that you do some research on some of your teaching practices. Can you tell us a little bit about some of that work?

Tina: Sure. So a colleague and myself got IRB approval, and we’re just starting to do some research on this evidence-based practice project that the students do in my class. And we’re just starting off with a cross-sectional study. We have a valid tool that’s been out in the literature that measures their perceived knowledge, skills and attitudes regarding evidence-based practice. So, I’m not building logistic regression models or anything yet, but starting off with a cross-sectional study to understand pre and post, the beginning of class and at the end of their research class, if there’s any impact or change in their knowledge, skills, and attitudes regarding evidence-based practice. So that’s where I’m starting. And I’d like to move on from there eventually.

John: And speaking of moving on, our last question is: what’s next?

Tina: So, I just would like to continue publishing and presenting. And continuing my research. Like I mentioned earlier, I’d like to introduce an AI tool for wellbeing, and Stony Brook just purchased several VR headsets. And because my courses include a lot of content about compassion, wellness, well being, I would love to develop a simulation about empathy. I think that would be a fantastic use of VR, apart from like, typical clinical scenarios. And that’s really my plan for now.

John: Well, thank you for joining us. And when you do have some results from your research, we’d love to have you come back and talk about it.

Tina: Thank you. Definitely. I really appreciate you inviting me. This is a wonderful opportunity for me. Thank you.

Rebecca: Yeah, thank you for letting us use your class as a little case study for folks to think about ways that they could change, improve, and reconsider their own classes. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

193. Making Team Projects Work

Posted in collaboration, collaborative learning, higher education, learning, pedagogy, teaching, team based learning
June 23, 2021

The ability to work effectively in teams is a skill that is highly valued by the employers of college graduates. Group projects in college classes, though, are not always designed to develop teamwork skills. In this episode, Lauren Vicker and Tim Franz join us to discuss strategies that we can use to create group activities that help students develop their teamwork skills while addressing complex problems. Lauren is a Professor Emeritus in the Department of Media and Communication at St. John Fisher College. Tim is a Professor and Interim Chair in the Psychology Department, also at St. John Fisher College. They are the authors of Making Team Projects Work: A Resource for High School and College Educators, which was released earlier this year.

Shownotes

Transcript

John: The ability to work effectively in teams is a skill that is highly valued by the employers of college graduates. Group projects in college classes, though, are not always designed to develop teamwork skills. In this episode, we explore strategies that we can use to create group activities that help students develop their teamwork skills while addressing complex problems.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guests today are Lauren Vicker and Tim Franz. Lauren is a Professor Emeritus in the Department of Media and Communication at St. John Fisher College. Tim is a Professor and Interim Chair in the Psychology Department, also at St. John Fisher College. They are the authors of Making Team Projects Work: A Resource for High School and College Educators, which was just released earlier this year. Welcome Lauren and Tim.

Lauren: Thank you.

Tim: Thanks for having us.

Rebecca: Today’s teas are…. Lauren, Are you drinking tea?

Lauren: I am drinking tea. I am drinking Trader Joe’s Moroccan mint green tea, one of my favorites.

Rebecca: That sounds like something John would rock. [LAUGHTER]

John: I actually have a backup tea here which is Moroccan mint, but it’s a different brand.

Tim: …and Wegmans decaf green for me.

Rebecca: It’s a good one. Wegotta have the Wegmans on…

John: Wegmans has a wonderful collection of teas, especially in the larger stores. And I have ginger peach green tea.

Rebecca: …and I am back to my good old English afternoon, John.

John: It’s been a year. I think you only had that once on the podcast in the last year or so.

Rebecca: Yeah, yeah, I know. I was thinking like, I haven’t been drinking it very often. I need to get back to it.

John: Well, we have six boxes of it still in the office for when we return.

Rebecca: I have six boxes in this office too. [LAUGHTER]

John: So, you should be set for the week.

Rebecca: I’m good. I’m good.

Tim: I took all my tea home last August, because I knew it would be a while. I did finish all the office tea.

Rebecca: It’d be hard for us to do that.

John: We have hundreds of teas in the office, so yes. We’ve invited you here to discuss Making Team Projects Work. Could you tell us a little bit about how this project got started?

Tim: Yeah, this was an interesting project, because it’s really been something that Lauren and I have been talking about for years. When I first started here back in 2000, at St. John Fisher, we realized pretty quickly that we offer two similar courses, one in small group communication that Lauren was offering and one in small group dynamics that we were offering in psychology. And so what we did was merge the courses. And then we started team teaching it and have been team teaching for about 20 years together. Over the time of our team teaching, we realized that we were going way beyond with our team projects than most of our peers, most of our colleagues at St. John Fisher. And we realized that a lot of people don’t know all the details about running a team project. So, we wrote the book.

Rebecca: Sometimes students complain about group projects. There’s a lot of strengths and benefits, but also some reasons to maybe not do group projects. So can you talk a little bit about both some of the benefits and some of the weaknesses of doing group projects.

Lauren: So one of the best reasons came out last week in Inside Higher Ed a story about a survey that was done by a AAC&U (American Association of Colleges and Universities) where they surveyed 500 employers, CEOs, and hiring managers, and asked them about the top skills that they were looking for in their new hires. And number one, a top skill turned out to be ability to work in teams. So we really need to be preparing our students for the workplace. And that’s one of the best reasons to use a group project. It’s also a much better way to engage students while they are in the classroom. Get them involved, have them work with other people, give them some of those professional skills that they need, and also keeps them more engaged than say, listening to a passive lecture. So there are a lot of reasons why people don’t use group projects. And it is a lot of work to set up a group project, and to do it well. A lot of faculty think about group projects as a way that they can minimize their workload. So what they’ll do is they’ll take an individual project that they might give to students, and just turn it over to a group and say, “Here, do this.” Give it out. And then you don’t hear anything until the end of the semester or close to the time when the project is due. And that is definitely not the way to do it. And so what we’re proposing is that people follow a very systematic process. And we actually have a model that shows how you can walk through each of the steps and be able to turn what might be an individual project into a really good team project.

Tim: Rebecca, there are other reasons why faculty think you shouldn’t run group projects… for example, student complaints, sometimes you get some pretty serious student complaints about a project, or the problem with so many projects and so many team projects, especially, is social loafing, where one person just sits back and lets the other do it, or conflict, or all these other problems that teams can run into. But the reality is that the well designed team project can help to minimize a lot of those problems, especially if the faculty member uses a structured process, such as we suggest in our book, where there’s lots of steps involved, and the faculty members are checking in regularly with the team. Now, the other problem is that does take a little more work. But with good planning and practice, team projects can be really effective.

Rebecca: I’m certainly an advocate for collaborative work. I do a lot of team projects in my own classes and know there are a lot of planning things to do at the beginning. Can you outline some of the key things to think about before introducing a collaborative project to your students,

Lauren: We talk about an input-output model. So let’s start maybe with the inputs to the group. A lot of people think that they can, as we said before, just take an individual project and make it a team project. So attention to the task is really important. And we can talk about that more if you want to talk about what makes a good team task. But also, the people are a huge input. And one of the biggest mistakes that faculty make is allowing students to choose their own teams and just to say, “Okay, everybody break up in groups of four, or five, or six,” and go ahead and do it. And that is the worst way to do it. Because you are not going to get any heterogeneity, you’re not going to get people with diverse viewpoints and experiences, people are just going to be working with their friends. And the final input is actually the context. A lot of people overlook that, but it depends on what kinds of experiences do students have with working in teams. And there are some colleges and universities that have a lot of teamwork going on, and others that are still using a lot of lecture-based sage-on the-stage type of teaching. And so if the culture of the school isn’t used to doing teams, or if you don’t even have a physical setup for teams, where people are in kind of an amphitheater classroom, and it’s hard to move around into groups, all of those things can actually thwart it. And you have to also consider what else is going on at the time. So those of us who were using teams when the pandemic hit know exactly the challenges that that entails. So what we find is that you have to start at that very beginning in the planning… on planning your task and planning the people who are going to be there and then looking at the context that you’re going to be considering.

Tim: If I can follow up on one thing, Lauren, this idea of picking your own teams that so many people, and in our presentations, we’ve gotten some pushback on this, that “Oh, the students love picking their own teams.” But number one, and we’ve seen this in our class, and we both seen it separately. Sometimes friendships break apart in those teams, because the friends realize they have very different working styles and get very frustrated with one another. And then the other problem with letting people pick their own teams is the elementary school kickball on the playground problem where somebody is the last to get picked. And honestly, I had somebody in tears in my class a couple of years ago, when I did a very short project and decided I don’t have time to do all that. And the person didn’t get picked until very last. And that was awful. And this is not the way we want people in college to be picking their teams. When they get out in the workplace, they’re not picking their own teams there either.

John: I’ve had a similar experience when I’ve used group projects in my classes, and students will always say, “Can we make our own groups?” And what I’ve done the last several times I’ve done it, and it’s worked really effectively, is to ask them how they knew each other. These are upper-level classes, primarily, where I’m doing this. And they’ll say, “Well, we’ve taken a lot of classes together.” I said, “So we’d like to have teams where everyone has a good mix of experience on all the teams. But if you know these people because you’ve taken a lot of classes with them, that means you’ve probably taken more courses in the discipline than other people have. So if we put everyone together who has the most background in the material, we won’t get as much diversity in the group, but we’ll also end up with some teams having some really rich backgrounds in the discipline, and others having a somewhat weaker background. And that may not be the most equitable way of creating teams.” And once you say that to students, you get much more buy in and they’ll generally accept it. And then I’ll often ask them, “What might we use as a criteria to balance the teams?” …and they’ve come up with some good suggestions. And that’s worked pretty well.

Lauren: You’re absolutely right, John. We’re a really big fan of the team-based learning approach to forming teams, which is: make the criteria very transparent. So if you say “I don’t want you with people that you’ve had two or three classes with,” or “I don’t want you with people who are in the same major or at the same level,” that’s great. We’ve done a number of different things. Sometimes faculty can pick the criteria as you have done, John, other times I know Tim has used a questionnaire that he’s had students fill out like a self assessment of their skills, and then he’ll put the teams together that way and tell the students this is how you were put together. I actually had one class where I did let them pick the criteria, or suggest the criteria, and I said “That sounds like a good idea. Let’s give it a try.” And when we put the teams together, we realized that they were perfectly balanced. And it was one of the best TBL classes that I had. And when we went online during the pandemic, it was almost seamless, because the teams had already really been formed, and they had an identity, and they’d had successes together. And so it was a really great way to do it. So we’re a huge advocate of not letting students pick their own teams,.

Tim: …and letting it be that transparent process. That transparent process is so important.

John: I’ve used a Google form in the classroom, where, when it’s in the classroom, where it’s displayed on the screen, they submit their responses, I’ll sort them from highest to lowest according to that criteria, and just go down the list assigning the teams 1-2-3-4, etc., and it’s worked really well. And another nice thing about it is when the teams are formed with this sort of criteria, instead of by social network, the team has been created for a specific purpose and they tend, when they’re working together, to focus on their purpose, rather than talking about what they’re going to do that weekend and other things. I found that the students tend to be much more on task when the teams were created to be balanced, separate from any friendship relationships. They tend to separate it from the social networks that otherwise might tend to dominate some of the discussions when they’re in a physical classroom. The groups have been really productive that way.

Tim: That’s a fabulous point.

Rebecca: Yeah, I’ve had very similar experiences in my classes as well. I tend to have a lot of different majors that come together and so I often use that as one way of dividing up the differences of experience for these collaborative projects, and it tends to work out well, and they tend not to know each other, as a result. [LAUGHTER]

Tim: Well, and then on certain teams, Lauren, I don’t think you even know this, but two of the students in our last class, our last group dynamics class, are now the closest of friends. And they didn’t know each other before a class, so that division can actually open up the doors to new friendships as well.

Rebecca: One of the things that you talk about in your book, and this seems like a good moment to bring it up, is that teams need to form and get to know each other, understand the project, understand what each member’s expertise might be, what the tasks are at hand, and also a need for someone to kind of step into a facilitator role. We might call it a leader, we might call it a facilitator, whatever that might be, can you talk about how to make that process go smoothly? …because if that process doesn’t happen, as you indicate in your book, the team doesn’t work, because nothing ever gets done. [LAUGHTER]

Tim: Right. And the team can take so much longer to get to a level of performance. When we talk about team performance, the key theory of development is Tuckman, where most people have heard this… that forming, storming, norming, and performing… and we want to get our class projects right into that performing stage as quickly as possible, and a team charter can do that. And in that team charter, it’s allowing the team to create some guidelines. In those guidelines should be things like attendance and deadlines, how to deal with conflict, the levels of participation that they expect from one another, their communication standards and rotating responsibilities, as you said, Rebecca, who’s going to be taking notes at the meeting, who’s going to be facilitating the meeting, when is each person going to be taking the lead on each thing, meeting times and places and then even things like decision rules and ways to solve problems when they occur. Those are all things that we encourage our teams to develop upfront ahead of time so those discussions are productive, rather than during a time of conflict.

Lauren: And if some people think that a team charter is too formal, we’ve actually had the class as a whole agree on: What are the norms for the class? How are we going to run this class? …kind of giving them ownership. And we have all the teams get together and generate different rules for the class and then we post them up on our course management system. And so when there’s an issue, we say, “Hey, look, we said that people were going to answer any texts or email within 24 hours, or within 12 hours, or everybody was going to show up prepared to meetings.” So we can actually point to those. But I want to back up before we actually start the team charter, we are huge fans of icebreaker exercises. And I know one of the things that we had the luxury of doing in group dynamics was we were teaching about groups, so we could spend a lot of time talking about these issues. However, most faculty have content that they need to cover. I also teach a course in the Wegmans School of Pharmacy and I know how much content people in the sciences and the humanities… they have so much to cover. They don’t really have a lot of time to talk about the group dynamics. And so they might assume that just because you’re working on a team project, you’re learning to work in a team, but we really want the teams to be able to do some icebreaker exercises in the beginning, even if it’s just fun stuff, you know, like what do you binge watching? What kind of pets do you have? Where are you from? …just getting to know each other. We think that that is hugely important. Just to get to know each other on a personal level, and then they get a little bit more ownership of the team. So while John’s right that not knowing people may be good and make you a little more task oriented, you still have to be concerned about all the people. And we know that, like this past year, a lot of students have really suffered from some anxiety and mental health issues. So we want them to feel comfortable talking to the group when they need to, we think that that is an important piece. So starting with that, and then they can constructively work together on that team charter.

Rebecca: To take one of those icebreaker activities that I’ve done in my classes that’s been really fun is to design an emoji. Of course, I teach a design class, so it’s related, but the students have had a lot of fun doing that activity, but it immediately gets them figuring out a way to work together and just talk a little more socially. So it’s kind of a task to do, it doesn’t really matter what the outcome is.

Tim: Some of the best icebreakers are actually relevant to the course or relevant to the team project. If you can make them relevant, they’re even better. And so I think that’s a perfect one.

Lauren: Yeah, emojis or I have them design a logo for their team… come up with a team name, and then a logo, and in the olden days, I would print up the logo and put it on their team folder. Now we have to do that virtually. [LAUGHTER]

John: One of the issues that, as you mentioned before, often shows up for students in terms of past experience with groups, are those people who may be sharking. In your book several times you mentioned the student named Fred. Could you tell us a little bit about Fred and how to deal with students like Fred.

Tim: Fred is actually a real student. This was not a hypothetical story. I think we embellish a little, but you saw us both laugh when you mentioned, Fred.

Rebecca: I think I’ve met Fred. [LAUGHTER]

John: I think we’ve all met Fred. [LAUGHTER]

Lauren: So again, we go back to what’s happening with this project? What is Fred doing or not doing? What has the team decided on their norms, or their roles for the team? And what are the sanctions for someone who is social loafing and not pulling their weight in the team? And one of the things that we talk about extensively, and this is also a big part of team-based learning, is peer evaluations. People who are not doing what they’re supposed to be doing on the team should be getting feedback from each other, and not just at the end of the project. Very often, we wait to the end, and we say, “team evaluations.” And all of a sudden it says, “Wait, Rebecca didn’t show up for any of the meetings. John wasn’t prepared for the presentation.” We need to know that along the way, and we recommend check-ins frequently with the groups. So we spend time informally checking in with the teams, meaning just wandering around the class or, if they’re in breakout rooms, popping into the breakout rooms to see what they’re doing, or actually having formal check-in times. We’ve sometimes given surveys to the class to find out how things are going, sometimes we make them anonymous. So we find out where are you on the project? How is everybody doing? And other times we ask them to specifically evaluate the contributions of each member. Another thing that we’ve done in the past is actually set up Google folders for all of the teams with instructor access. And that way we tell them, “Okay, everything that you do is going to go into that Google folder.” So the instructor has a way of looking in and saying, “Fred, it’s been three weeks, and you have put nothing into the Google folder, what’s going on?” So we can talk to the teams individually, but also talk to Fred individually, as well.

Tim: Yeah, and just as instructors, it’s our job to give students those skills for teamwork, because that’s what their employers and grad schools want. It’s also our job as instructors to develop our students. And that process of multiple check-ins, though, that’s one of the areas where it does take more work, we need those multiple check-ins to see how things are going. And Lauren, I think you emphasized and I can’t stress enough, the importance of these formal and informal peer and instructor evaluations that are going on throughout the process of this team project to keep them on track and develop their skills so that they can improve and be better team members when they leave our campus.

Rebecca: You have a couple other scenarios of student or learning situations around leadership that I think are maybe important to address as well, the idea that the team seems it’s going great, but come to find out it’s the one person doing all the work and no one else is allowed to do anything. And then there’s also the opposite where just nobody’s doing anything because nobody knows who’s in charge. Can you talk a little bit about how to make sure that there’s maybe a leader who’s not a dictator? …someone who’s really acting more as a facilitator within a team.

Tim: Well, I think this is another area where it does take a little extra time. And if we want to develop these teamwork skills in our students, we need to teach to the teamwork skills and teach to the leadership skills, at least a little time …and I’m not talking taking whole weeks of class but 15 to 20 minutes to put our expectations down and have them help get those expectations out there so that they know what it means to rotate through leadership. They know what the five or 10 top leadership characteristics are that their team expects of them when they’re leading. And we do emphasize the importance of rotating in the leadership role. That is important because we want everybody to lead team meetings and everybody to take notes at team meetings, not leave all that to one person.

Lauren: And students are really reluctant to take on the leadership role. They don’t want to seem like it’s a power grab sort of thing. And so it’s important for them to understand the nature of leadership, that it isn’t one autocratic person telling everybody what to do, that they understand the different perspectives on leadership. And again, we have the luxury of being able to talk about that, about different types of leadership, and we have our students do leadership assessments. And it’s helpful for them to be able to talk about when they go to a job interview. They’ve got something where they can discuss how they see themselves as leading the team.

Tim: Right, those demonstrable activities that they can actually show on paper in a portfolio, I say on paper, but it certainly could be electronic in these days.

Rebecca: I think one thing that comes up frequently when we talk about groups and group dynamics is setting on and establishing roles. And I’m hearing you both emphasize the idea of rotating some of those roles. And I think this is a place where faculty struggle to set up good structure. Can you talk a little bit about some of the rotating roles that should be there and how do we encourage students to rotate over the course of a semester?

Tim: Some of those roles I think we’ve already mentioned, certainly leadership should rotate at meetings. A lot of those roles occur in meetings. So, at a meeting, you want somebody who’s leading or facilitating, we often prefer the term facilitating, because that is somebody who’s just leading that meeting, then you need a note taker, then you need on top of that a timekeeper, somebody who’s keeping that team on task. But even beyond just the simple meeting strategies, where everybody should be getting some practice and learning opportunities throughout all those, when you’re thinking about roles, there’s also those informal roles, like somebody who could be the cheerleader for the team and trying to get people to feel better. And somebody who could be the one who’s asking the questions or giving the task-based answers, keeping them on track during their problem solving or decision making. These are all roles that each person should be practicing. We all tend to fall into our own roles that we are used to. And by forcing them out of their comfort zone and into some of these other roles, they can get better at being a team member, and bring more inputs, as Lauren introduced at the beginning, into the team project.

Lauren: But having said that, it is a lot more challenging to do when you’ve got an online class or online team projects, because, especially if it’s an asynchronous class, so it really depends on your circumstances. And another thing we found is that as the project progresses and students are getting closer to the deadlines, they definitely have to start solidifying their roles, they need one person who’s going to collect all the data and one person who’s going to do the data analysis and one person who’s going to organize, whether it’s a portfolio or slides or whatever that they’re going to do. So that part becomes important as well. And so one of the things we’re teaching them is to be flexible in their roles and realize that maybe you were a leader on the last project, but you’re not the expert to be the leader of this particular project. Or maybe you’re a really good graphic designer. And so you should do our slides for our presentation or design the portfolio. So trying to make sure that everybody gets a chance to show their strengths, as well.

John: I know when Rebecca and I have presented together jointly, she was always very quick to volunteer to do it, [LAUGHTER] because she wanted to make sure I wasn’t doing it. [LAUGHTER]

Tim: Well, Lauren and I have the same exact relationship, because I’m going to give Lauren some credit here, [LAUGHTER] she is way better at doing slides. And now when I’m doing the slides, I do my outline and hand it over to Lauren electronically to clean it up and make it look nice. [LAUGHTER] And it looks a lot better when we get it to the point where we’re going to use it.

John: So that does suggest though, that using the expertise of some of the group members can be helpful. But you do also want to provide some rotation in tasks. And that could be a bit of a challenge. Would you recommend instructors giving students are a rotating list of at least some of those positions? Or would you encourage the teams to do that on their own, such as the leadership role and other roles?

Tim: Well, I think the areas where you have expertise, you should certainly…. that’s the advantage of teams… you can spread the work out but you can also at least get closer to that idea of team synergy because you’re pulling together all those diverse views, all those diverse backgrounds, all that diversity in expertise. And oftentimes when we use the term diversity, we use it to mean race, but diversity is much, much broader than that. And so you can pull together that diversity in expertise and come up with a much better outcome, a much better product that the students are proud to share, and show off in the future. So yes, you certainly want to leverage the expertise of your students. But there are certain areas where you want the students to get some practice with those roles, as you pointed out, John.

Lauren: And I think if we don’t set them up, they probably won’t happen naturally. And so we need to be talking to them about it, giving them some experience doing it. A lot of it is putting the responsibility on the team and the students to say, “Okay, here’s something that we want to see you doing,” and have them explain to us how that happened. So one of the things that we’ve done when we have students give team projects is not just talk about what they found out when they did the project, but what was their process like? …and describing that, because you can learn a lot from hearing how other teams managed it. And you can actually see, during presentation time or reading portfolios, how they approached it, and which processes were most successful.

Rebecca: One of the things that we do in our design classes is something called a process video for just that. So if they’re working collaboratively, they describe and show their process for the project in a short video, like a three- to five-minute video. And it’s really interesting sometimes to see the way that different team members describe the same process.

Lauren: I love it. I love it.

Tim: That’s fabulous. The importance of reflection on their group work, it can’t be understated, because a lot of times is, Professors, we focus on the task outcome, but what we want for our students is also all the other stuff that comes along with teamwork, where they learned what it means to be a team member. And it’s those reflective activities at the end: “How did you get to this? Where did you help? Where could the team do better?” Those are the things that can really help our students develop those teamwork skills in the future.

Rebecca: Sometimes those things are so invisible too, unless we directly ask them to explain or narrate. I’ve been surprised often, in watching the process videos like”Oh, is that how you did that? I didn’t realize.” [LAUGHTER] It’s really interesting sometimes to see how they did something technically or how they arrived at a particular idea which hadn’t been explained to me in a one-on-one meeting or something or with a group meeting.

Lauren: And one of the things that we have done at the end of a big project is we have asked the teams to self assess, and actually tell us verbally, in front of the whole class, what they think they did well, what they’d like to do if they were going to do that project again, or going forward, how they will use that. And then we have other teams give peer feedback too, so it’s a good discussion. And it’s after they have finished the project, so there’s a sense of relief. But also, it’s important to say “Just because you turn the project in, that’s not the end of the process, you’ve got to look back and take those lessons with you to the next group experience.” And we should point out that there are some programs, especially in graduate programs, where people are in the same teams for a year or even two years. And so if you’re going to be doing more projects with the same team, it’s just invaluable to be able to learn from each of those experiences and take it forward.

John: One of the issues along those lines that came up with a podcast we did earlier with Olga Stoddard was an examination of long-term group projects and leadership roles in terms of gender. This was in an MBA program, which was disproportionately male dominated. And one of the things that happened in groups where women were in the minority, their leadership tended to be undervalued, or their rating of their leadership skills tended to be rated relatively low, while in groups where they were the majority or represented the whole group, their leadership evaluation was quite a bit higher. So one of the things, in terms of roles, that could be an issue is gender bias, and so forth in constructing the group. There’s also lots of research that shows that women are more likely to be asked to take minutes in meetings or to be the recorder in groups. Might it be worthwhile to address some of these issues with the groups before the groups create their charter or before they start their processing?

Lauren: Absolutely. And we do talk about implicit bias, not just gender, but on other factors as well, that it is important for students to have that call to their attention. Fisher has more females than males. So we haven’t had that much of an issue in classes. But I have seen that happen. I taught at the Simon School of Business at the University of Rochester for a while and you see that sort of thing happening, but students may not even realize what they’re doing. And so calling it to their attention is really important. I recently hosted a DEI panel at the TBLC conference. And one of the things that came out was, if you only have a few people of color in the class, don’t spread them out. So each group has one person of color who has to represent their entire race. And it’s that sort of thing students don’t really understand. And I think sometimes faculty don’t even understand what the implications are. But how valuable that lesson can be for the students going forward.

Rebecca: We spent a lot of time talking about interpersonal relationships in groups in our discussion today, but maybe we can also talk a little bit about the kinds of activities that might be appropriate to do as a group, as opposed to what might be more appropriate for individuals.

Tim: Well, there are definitely tasks that are not appropriate for teamwork. For example, writing a paper, if you want people to write a paper together, that’s a task that really isn’t a typical team task. A team task, one that’s designed for a team, should be complex, it should be challenging, it should require lots of ways to solve it, and it should force some level of cooperation. And I wish we could give some examples. But the problem is that the task is very discipline dependent. And what task works for one discipline is different for another. But if you’re trying out a team task, then you can use it, you can see what works and see what you need to change or have some of your colleagues and peers review it and see what they think is relevant and what they think works and what they think might not work about it. Because we’re all trying to improve. We’re all lifelong learners. And so we can improve ourselves, too.

Lauren: And that’s another reason why you don’t want to take an individual project and turn it over to a group. Because those cooperation requirements are so important. They’re valuable for the students learning the content for the course, but also learning how teams work and how they can develop in those.

Tim: And back to those multiple check-ins. Once you start to force people to cooperate in a complex and challenging task, then it’s really difficult for certain students to let go their control and let others do it. But if you’re doing the multiple check-ins, you’re getting the information about which members are doing their work, and which members are not in time so that they can change and improve and develop.

Rebecca: Although cooperative tasks are different in different disciplines. Can you give an example from the classes that you’ve taught together, of where cooperation becomes an important key or important element to a project that you’ve assigned?

Tim: Oh, yeah, we’ve got quite a few of them, because we use team projects, not just together in our group dynamics class, but in many other classes. And so I ‘m going to pick my industrial and organizational psychology class. And in that class, they do a client project where they go out and they do a survey within an organization. And when they do that survey in an organization, I have them divided up to one person who’s the main client contact, and one person who does the data analysis, and one person who’s the lead for the first half of the project. And the project takes a lot of steps, I give them a stepwise document for what they should do, but there’s a lot of steps involved. And it’s something they’ve never done, writing a client type report instead of…. I’m in psychology… that dreaded APA-style paper that is [LAUGHTER] so frustrating for so many students. And in this case, they haven’t done this, so they’re trying to figure this out, dividing it up, and then coming back together and building on each other’s work to do this client survey.

Lauren: Now another project that we did in our group dynamics class was we had students actually do an observational report on a real-life group on campus. And so they had to choose a group, they had to get permission to observe the group, they had to observe the group at least three times, they had to give a certain number of instruments, whether they were things that measured interpersonal skills, leadership skills, roles, conflict resolution, that sort of thing. And then they had to go and observe the group and do a post-meeting analysis and ask people to say, “How did you think the meeting went?” So it required a lot of coordination, because they had to find a group that would give them permission to observe. And then they had to come back and figure out how were you going to collect this data, who was going to be responsible for doing that. When a meeting’s over, most of the time the students all get up and they want to leave the meeting, get to class or whatever. So they had to figure out how they were going to get that information from people. And when it was all done, they had to put it together and display their data. They had to show the results of all of the surveys they had done, the observations, they had to look at the task, the people, the contexts, and analyze those things. And then they were asked to analyze all those things that happened when the group got together. How was their communication style? What kind of norms did you notice? And it became really interesting to watch the students recognize things that they had done, and maybe not even realized, that they were either supportive of the group goals or not so supportive of the group goals, depending on what was happening. And then they had to come and give recommendations. And the groups that they observed were told that, if they wanted the recommendations, they would give them the executive summary from their report. So that was a task that required everybody being involved because they couldn’t all do all of the things. And it was pretty complex, and a long semester project.

Rebecca: So it sounds like scope is an important piece of the puzzle with collaborative assignments.

Tim:: Absolutely.

John: Do you have any other advice for our listeners?

Lauren: One of the things that we haven’t talked about was making sure that everybody is involved in the group. And it’s easy for people in groups to get lost if they don’t know the people well, if they may be shy and not comfortable speaking up… maybe it’s their first class in a particular subject matter. And so we’ve talked about this idea of checking in and making sure that everybody’s involved and that’s one thing that you can do is to encourage that sort of feedback, and making sure that everybody understands that everyone has to be involved, that that’s part of the requirements for the group tasks. Starting off with those icebreakers is just great, because it gets you to know everyone, and so people start to feel more comfortable. And especially again, I’ll say with online classes, it’s an issue because people don’t put their cameras on, they mute their microphones, they don’t show up for team meetings because of a scheduling conflict, and that sort of thing. So it makes it difficult, but you’ve got to make sure that with formal and informal check ins that everybody is involved and know what their role is and what their deliverable is going to be for the project.

Tim: And if I can follow up on a different thing, but it’s related to what Lauren finished on, in the online environment. We are all teaching in this different environment now, getting used to teaching online. And that’s difficult. And it’s difficult for us, it’s difficult for our students. What we found is our students often don’t know all the tools available to them, nor do they know how to use all those tools that are available to them. And so helping, especially on a team project, helping them realize all the different ways they can communicate, because if left to their own devices, it’ll be primarily by text, [LAUGHTER] and if other students are like ours. And so helping them to see that they have all these other things available that are both things they can do at the same time (synchronous tools), and asynchronous tools (things that they can use that drop information where other team members can get to it later).

Rebecca: I think that’s a really important point, Tim, not just in the online environment, just generally when students are collaborating, because they may not always be able to be in person together at the same time in any context. So having strategies to deal with communication or deal with sharing materials can be really helpful. And sometimes that means, in my experience at least, doing some little activities to introduce them to those tools so they can kind of level up in the technical skill sets that might be necessary before expecting them to be using it in their teams.

Tim: Absolutely. What’s interesting is, I’m going to date myself here, and sorry, Lauren, you’re coming along on the ride with me on this one, [LAUGHTER] but when Lauren and I first started teaching group dynamics together, we used to talk about teams that were either virtual or not virtual, because that’s the way it was at the time, your team was one or the other. And now in today’s environment, no team is all face to face, and no team is all virtual. I have a colleague next to me that I’ve texted, emailed and called and her office is right next to mine. And so we have a lot of computer-mediated communication, even in face-to-face teams now. So it’s the level of virtuality, everybody is using these tools. How can we use them to match the task and to match the skill set that the people on the team have?

Lauren: So at the beginning of this conversation, we talked about our book being based on a model that was an input-output model. And so we talked a little bit about the inputs, the things that happen in the middle are the communication and the conflict and the norms and roles, but to really be aware of what the outputs are. And faculty need to be aware of those before they assign the project. As much as we hate having to put together rubrics, it’s something that students need to have, they need to know how are they going to be evaluated. So we tend to focus, though, a lot on the task rubric. And that’s important. Obviously, we want students to get the content that relates to the course material, but we also need to have a rubric for those self and peer evaluations. And so the output is not only the task output, how did they do on the task? …but what did they take as a result from working on that task together? What kind of feedback did they get? How are they seeing themselves? …and actually having structured rubrics not just saying, “Well, what do you think? How did you do?” actually giving them a form to fill out and we’ve got some examples in the book and on our website too, that show what people can do. We’ve even got some from high school that were not as detailed as the ones that we gave our college students, but it’s important that they see the output of it is not just what the grade was on the project. But there’s more detail for that.

Rebecca: Students have a tendency to think that the output is the thing that the most weight or value was placed on. And I know, in my classes, it’s really the process. [LAUGHTER] So when I show them those rubrics and show them the weighting between the task versus the actual process of making the thing that you’re outputting, they’re often surprised. And I have to remind them constantly throughout the process, or through the project, that this process piece is important, you need to stop shortcutting this, this is the thing that actually matters the most, this is where the learning is happening.

Lauren: This is so true, I teach public speaking, and students think that the person with the best delivery is going to get the best grade. And I said, but look at the rubric, delivery is only 15% of the grade, you’ve got to do research, you’ve got to do organizing, you’ve got to have your citations in there, and you’ve got to have visuals, and how you handle Q&A, and all of those sorts of things. So it is a good example, Rebecca, of how there’s a lot more to it. And we need to lay it out so students know what it is.

John: And sharing those rubrics in advance with students in the learning management system or in person if it’s face to face, but preferably in the learning management system, so they have access to them anytime, and referring back to them regularly, will help remind them and help you be more transparent and how you’re assessing student work.

Tim: We remind our students: we want to start with the end in sight, what is it that you need at the end and build the project based on that end goal. So if we can provide those rubrics and those processes through which we’re going to be evaluating them, we get better work from our students. And that’s what we all want is better work for our students.

Lauren: And from time to time, we actually pull the rubric up during class and say, “Does this look familiar? Did anybody notice that this is the way you’re going to be evaluated?” …because they get involved in their project and then they lose sight of some of those details that are going to be important. For example, we had a project where students had to use two synchronous tools and two asynchronous tools when they were working on their project. And we went around and started asking them well, like, “What’s one of your synchronous tools?” They were saying: “Well, we’re using Google Docs.” We’re like, “Well, wait a minute, just because you’re both on the Google Doc at the same time, doesn’t make it a synchronous tool,” [LAUGHTER] and so it gave us a chance to really give them some clarification about what they needed to do. And so that’s why the rubric at the beginning is so important.

Rebecca: I think this is a good moment to wrap up in some ways, because it’s like we’ve got our end in sight, we’ve got a process in place. And so we always wrap up by asking: what’s next?” (that nice reflection kind of question) [LAUGHTER]

Lauren: Great.

Tim: That’s a great question, because we do have the “what’s next.” Number one is we post blog posts on our LinkedIn sites every Tuesday. So we’re constantly developing content. For example, on May 18, we talked about giving feedback and peer evaluations, we have something about how to teach peer evaluation, so your students do a better job at it. That’s on our blog posts. And we also have a student version of our handbook coming out in September. It’s going to be matched to the professor version, the handbook for the instructor, except it’s going to be more student friendly, a lot less writing a lot less “how to” in the text and a lot more based on checklists and exercises and guidelines, rather than simply explaining the things.

John: That sounds like a great project.

Tim: Thank you.

Lauren: Thank you. And we’re looking forward to being able to get more feedback from educators. We have a lot of professionals who follow us on LinkedIn as well, and so respond to some of those topics. And we’ve done a number of professional development seminars for different colleges and universities that have been pretty well received. So we’re looking forward to it. But we’ve got a lot of resources on our website, which we can put in the show notes and people can find a lot of information there and just get an idea of some tools they can use as well as contact us if they want more information.

Rebecca: Excellent. Thanks so much.

John: Thank you.

Lauren: Thank you so much for having us. It’s been a lot of fun.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

102. Team-Based Learning

Posted in active learning, classroom community, classroom management, collaboration, collaborative learning, evidence-based practices, higher education, learning, lecturing, pedagogy, peer instruction, project based learning, team based learning
October 9, 2019

A large body of research finds that active learning approaches result in larger learning gains than traditional lecture approaches. In this episode, Dr. Kristin Croyle joins us to discuss how she transitioned from  explore using interactive lecture to collaborative learning, and then to team-based learning. Kristin is a Psychologist and our new Dean of the College of Arts and Sciences at SUNY Oswego.

Show Notes

  • Johnson, R. T., & Johnson, D. W. (2008). Active learning: Cooperation in the classroom. The annual report of educational psychology in Japan, 47, 29-30.
  • A discussion by Dan Ariely explaining why asking for shorter lists of positive features in a relationship can engender positive feelings appears in this March 24, 2014 video clip.
  • Michaelsen, L. K., Knight, A. B., & Fink, L. D. (2004). Team-based learning: A transformative use of small groups in college teaching.
  • Team-Based Learning Cooperative
  • Epstein Educational Enterprises, What is the IF-AT?
  • Sweet, M., & Michaelsen, L. K. (2012). Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement. Stylus Publishing, LLC.
  • Croyle, K. L., & Alfaro, E. (2012). Applying team-based learning with Mexican American students in the social science classroom. Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement, 203-220.
  • Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251-19257.
  • 74. Uncoverage – David Voelker – Tea for Teaching podcast episode discussing the uncoverage movement in history, March 27, 2019

Transcript

John: A large body of research finds that active learning approaches result in larger learning gains than traditional lecture approaches. In this episode, we explore one faculty member’s transition from using interactive lecture to collaborative learning, and then to team-based learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Today our guest is Dr. Kristin Croyle. Kristin is a Psychologist and our new Dean of the College of Arts and Sciences at SUNY Oswego. Welcome, Kristin.

Kristin: Thank you.

John: Our teas today are:

Kristin: Earl Grey

Rebecca: I am having Mama’s work tea, because Ada made it this morning and she calls it work tea, which means she pulls the tea bag tag out and puts it in the big cup. Also, it’s just my normal English Afternoon. But, that was a better story.

John: And I’m drinking Spring Cherry green tea.

Rebecca: We invited you here today to talk about collaborative and team-based learning in your classes. But before you do that, you’ve noted that you had a strong preparation in teaching before you got started. Can you talk a little bit about that. We’ve talked a little bit about that on the show before and how a lot of faculty aren’t prepared…

Kristin: Um hmm.

Rebecca: So, could you talk about how your preparation may be informed what you’ve done.

Kristin: My graduate program, I went to the doctoral program in clinical psychology at the University of Montana in beautiful Missoula. And that program takes the preparation of their grad students very seriously, but across several areas not just in clinical work and research, but also knowing that some of them are going to end up in positions in which there will be teaching. So, while I was there, that very first semester I was brought in, they had a structure for teaching their introductory psychology classes where graduate students were assigned our own classes where we were the instructor in the classroom, but we had a supportive network around us. So, the syllabus was already there, the textbook was already there. We collaborated in writing tests. We had a structure of TAs that supported us and they would have recitation sections in which the TAs also received development. And we joined in that so we could see how more hands-on kind of things could be done with students in smaller groups. We even assigned our final grades together. And some of those pieces are pieces that are areas of skill that people don’t often think about developing. So, that first semester, all I had to do was think about working within the structure: How am I going to handle the day-to-day teaching and learning in the classroom? I didn’t have to worry about course design because the course was already designed in front of me. And I also didn’t have to, at the end, think: “When you assign grades, is that rigid? Do you really have to follow the exact, you know, 90/80 that it is in the syllabus? Or what if there are natural breaks around 88 or 89? Is it okay to flex that? What kind of power does an instructor have that is fair to students and evaluation?” I got to do all of that in a collaborative setting with a very experienced faculty member as a guide. There was also a credit-bearing course for teaching psychology that we were encouraged to take… which I really enjoyed. And then I was given opportunities to function more independently. When they needed a stats teacher over the summer, and they knew I was living there over the summer, I got to teach on an adjunct basis, but still with the support of faculty around me. So kind of putting students in the deep end, but with a high level of support around them, I felt very prepared when I was done with the graduate program to enter into an assistant professor position. And I still appreciate the preparation that they gave me.

Rebecca: I think with the preparation like that you’re probably far more willing to experiment and do new things as a faculty member too and to maybe even break away from what faculty around you are doing. Do you find that to be the case? Or were there other faculty doing some of this collaborative work in the department that you were in?

Kristin: Yes, and no. One of the experiences I had at my previous institution, which was the University of Texas – Pan American that then transitioned through a merger to be the University of Texas Rio Grande Valley. I was talking with a colleague in another department about the kinds of things we were doing in the classroom. And I still remember him saying “Oh, I didn’t know you could do that in the classroom and that was like teaching.” He had a very restricted idea of what teaching was, and what would be acceptable to colleagues, which he had never had the opportunity to test with other people around him. And that was something that I arrived from day one… that you talked about your teaching… that you can do many different things in the classroom. And it’s all teaching, as long as you are trying to work with students to create a learning environment and they are learning, then it counts as teaching. So I did come in with a much more flexible idea, then certainly some of my colleagues who hadn’t had an opportunity to ever have those discussions. And of course, some people are hired into departments in which those discussions don’t ever happen, so they may persist with those misconceptions for many years.

John: Or throughout their entire careers at times. [LAUGHTER] The scaffolding that was provided is really nice, because we’ve talked to a few people who’ve been in teaching training program or had some training in graduate programs. But usually, it’s not quite as structured as that and that’s a nice feature.

Rebecca: Yeah, I came from a program like that, but it was like very front loaded. It wasn’t that ongoing…. So I felt a lot more prepared, because I did have a lot of those experiences, but I didn’t have that same kind of supportive network throughout. Which is incredibly valuable.

Rebecca: So, you want to take us through what some of your collaborative experiences have been in the classroom and the ways that you set up some of the team-based learning exercises, maybe starting with what are those?

Kristin: Sure. So kind of the way that I journey through my teaching, particularly when I was an assistant professor, I felt comfortable in the classroom, but I didn’t feel expert. I felt like I was still trying to figure out what was going on, which is a perfectly fine way to be and a good state for learning to occur. So I felt like I was a talented lecturer, like I can engage students. I teach in psychology, I also think psychology is naturally very engaging, but part of that is because I really love the field. So, I felt like I could engage students and that they would listen and that they would be interested. But I started to become dissatisfied that there was always a core of engaged students and I had no idea what was happening with the other students in the class. And then sometimes I would be disappointed when we have tests or homework. Everyone said they had no questions. Clearly that was wrong. I was wondering how do you engage the majority or all of the students in their learning so that they aren’t coasting through class believing that they understand until they really don’t. And then I also felt like I was kind of fooling myself into thinking that students were with me when they were not with me. So I had an opportunity at that time to do some intensive cooperative learning training along the model of Johnson and Johnson collaborative learning. And that model from the University of Minnesota, it focuses on the importance of cooperation in the classroom, and that in cooperative settings, students learn more, develop a stronger sense of self efficacy around their learning; that they together are able to achieve more than they would individually. And it also has impacts on retention… that if students are feeling like they are individually known and valued in the classroom by their peers, they’re more likely to continue showing up to class and to develop relationships outside of the classroom that supports them along the way. So through that training, it was intensive, it was like eight hours a week, one day set for several weeks. The very first day, I could see what a difference I was going to make in my classroom. So, for example, I was using group assignments in class and they had all the same disadvantages that group assignments and most classes have, because I had no idea how to structure the group work so that it would be successful. I was doing group work to save me grading time, honestly.

Rebecca: That’s why a lot of people go to group work.

Kristin: Yes! Without understanding that all I had to do was some structural changes, and then it would actually be effective for learning as well, instead of just saving me grading time. In that cooperative learning training, I learned how to structure intensive group work that could be the length of an entire semester, or it could be the length of a single class day. I learned how to structure less intensive moments of team time. So how do you do a think-pair-share that works versus how do you do a think-pair-share that doesn’t work very well. So, that within the course of that training… actually just within a few days… I suddenly had, instead of 10% of the students in the class engaged on a daily basis, I had 100% of the students engaged on a daily basis. So, that was a huge breakthrough and I continued that way for several years.

John: What were some of the structural changes that you made that did lead to increased engagement?

Kristin: So, the cooperative learning approach of Johnson and Johnson, is kind of theoretically heavy, in the sense that they outline the pieces that are necessary for strong collaboration to occur. And then they turn it over to you as the instructor to say, “How do I build those pieces in?” So, for example, they emphasize positive interdependence as one of the essential components of cooperative learning… that when you create a group and a group activity for them to do, the activity has to be structured in such a way that each person is necessary to contribute. You can’t structure it in such a way that you can have three people talking when one person is only needed, and there are specific recommendations on how do you structure it so that everyone is needed. At the same time you have to build an individual accountability as another required component, so that, even if each person is needed, people can still slack off, say, “Yes, you all can’t do as well without me because you need me, but I don’t really care about what is happening here.” There has to be a level of individual accountability that’s also built in. Along with that, some of the skills that I thought were most important, they build an emphasis on group processing and social skills, so that if you have people consistently working together in class, they may not have developed the social skills to do that effectively, especially over time. You can work with someone for two minutes on a think-pair-share and really be bad at social skills. But, if you have to work with them over an extended period of time on a project and things are going south in terms of group conflict, it’s the instructor’s responsibility to help them to develop the social skills to work together. For example, on the first day of class, when I first start having students talk to each other so that they know that’s going to be a pattern in the class, I give them something quick to talk about. And I say introduce yourself to the person next to you… spend two minutes talking about this. And then I’m going to ask you about what you talked about. And then I run around the class real quick… pair up people who aren’t participating, introduce them to each other so that they understand this is a part of the class. So, then I follow back. So, what pieces are important there? …that I explicitly instruct them, you turn with your body… you actually make eye contact. And I will point out as people first start doing this, look at these two people, they are looking at each other, because many times students won’t do that, and it’s very hard to have a cooperative interaction if you don’t make eye contact… and I will say, “Who was the person you talked to? Tell me their name.” So they understand that I was serious when I said, introduce yourself, tell me something about them and that there’s individual accountability through just random calling on… that they need to participate in the cooperative portion. And then there’s also the self-reinforcing aspect of it that five minutes later, when I say to talk about something else, they realize they already know somebody in class, they have a connection. The next day, when I come in, they’re not quiet, they’re already talking to each other, they’ve created those connections.

John: A nice thing about that, too, is for people who are uncomfortable talking about themselves in class, having one person tell you something about the other person, it’s a little bit less pressure, it’s a little less revelation to the whole group. There’s some evidence that that type of thing is more effective in providing a more comfortable environment.

Rebecca: Kristin, can you also talk a little bit about a specific example of a cooperative activity where all of the members are held accountable, and all have a role? …just to provide an example for people who have less experience.

Kristin: So cooperative learning can be divided into informal and formal cooperative learning. Informal cooperative learning tends to be much shorter activities that can be done kind of on the fly if you already have an idea in your mind of how you might want to do that. Formal cooperative learning tends to be more intensively structured… longer-term activities. So that could be a single class session. If you’re going to do an activity that takes an hour, that would be more formal… or if you’re going to do something that takes an entire semester. The pair-and-share that I just talked about is an example of informal cooperative learning. Something like a jigsaw classroom activity can be structured as a formal cooperative learning activity. And it already shares almost all of the components: there’s individual accountability, because each student is given a specific role. There’s also positive interdependence, because the success of everyone depends on each person doing their role. So there are ones that are already structured with a built in component. The pieces that aren’t built into something like a jigsaw classroom activity, would be the group skills and group processing, and the ways that you can build that in. You can, for example, ask groups to reflect on what went well. I typically emphasize that more than asking them to reflect on things that went poorly, because asking to reflect on what went well tends to maintain a positive atmosphere, but also helps them to cover both bases at the same time anyway.

Rebecca: …or realize that my list for what went well is not very long… [LAUGHTER]

Kristin: Right. So, a common group processing thing I would have students do after their first more lengthy or more formal cooperative learning activity would be: list three things that your team did well together and one thing that you could improve on. And another thing I might ask them to do is to provide positive feedback to each member of the group at the end of the activity. And the kind of feedback that they provide is usually pretty specific, and helps to shape their behavior throughout the rest of the semester. So when they say things like, “I like it when you disagreed, and you said that this other thing would be a better way to go” that provides important feedback, and it helps to encourage better processing going forward. But I will go around and give individual social skill feedback too. But it’s usually things like, “Oh, I see you’re sitting so far away from your group, I’m not sure they can hear you, let’s scoot your chair in so that they can hear you.” Or I might ask, “Oh, do you know this person’s name next to you? What’s her name?” …and we’ll make sure that people maintain the social and cooperative connections that enable to do that kind of good group.

John: Just as an aside, it’s useful if you’re asking about things that went well, to keep the list fairly short. I’m reminded of a study that Dan Ariely talked about where they did a controlled experiment where in one case, they asked people to reflect on three things they liked about their partner and another case to list 10 things I liked about the partner, and then they surveyed them on the quality of the relationship. And those who were asked about three things generally rated their relationship with a partner fairly high. But when they were asked to come up with 10 things, they struggled with that and they rated that relationship lower. So keeping the list short…

Kristin: Right.

John: …is really good so you don’t…

Kristin: There’s kind of analogous thought about keeping things like gratitude lists. If you list too much stuff, it can have a negative effect, because you start to identify things that you really don’t think are that important, and it makes you think the whole thing is less important.

John: And if you want to get the opposite effect, ask people to list 10 things that were bad, and then they’ll struggle beyond the first few. You talked about having continuous relationships or persistent relationships with collaborative learning. Did you try to keep the group relationships consistent for the same groups throughout the term? Or did you vary that?

Kristin: I varied it. There are some good data to suggest that in collaborative learning… they refer to them as base teams… that base teams have a persistent positive effect, particularly on things like student engagement and retention throughout the semester and throughout the year…. that you have a team that is expecting you every day. But when I was doing cooperative learning, I didn’t restructure my courses. I restructured the day. Does that make sense?

Rebecca: Um hmm.

Kristin: So I didn’t have a reason for base groups. And I felt strange imposing them on the students without a reason. Besides, they would maybe be socially a good idea. I had to completely rebuild my courses from the ground up before I started using base teams. And that’s when I transitioned to team-based learning.

John: …and in team-based learning, persistent teams are recommended as part of the process.

Kristin: Absolutely.

John: Could you tell us a little bit about this transition to team-based learning. What prompted you to introduce that? …and how it worked?

Kristin: So I was happy with how courses were going. People were interested and engaged. I had students telling me, “I know every single person in this classroom.” and when you’re teaching a class of 30, or 40, or 50, that’s unusual. “I know everyone in here, I feel really supported.” I feel like things were going well. But I was unsatisfied with what I was teaching. I wasn’t clear, in my own mind, about what persistent learning outcomes I wanted for my students. I had not sat down and really thought through if I were to follow up with a student in a year or five years, what would I want them to recall from this class? What would I want them to be putting into use in their lives or in their careers? I had never thought that through. And I was fortunate enough to run into team-based learning at that time, right as I was primed to start thinking about this questions. Team-based learning originated by Larry Michaelsen. He was coming from the perspective of enrollment increases. He had been assigning some pretty challenging work. He was a faculty member in business. And as his course enrollments increased, he started to wonder how can you maintain the same kind of interesting, really challenging in class… by case work, for example… with a large enrollment. So he developed team-based learning to address that piece, but it also requires you to completely rethink the design of the course. And to start from the course outcomes: “What do you want the persistent outcomes to be?” …and then structure the course forward in that way. So in team-based learning, after you make a decision about your course outcomes, and what you really want students to be able to do, then you structure the course in a modular fashion. And each module has certain steps. So the beginning is student preparation, then when they come into class, you test. You say, “okay” …and it’s called the readiness assurance process. So you want to know what students are ready to do after they’ve individually prepared, and what they’re not ready to do. So they prepare, they test. And then, since it’s a team focus, they also test as a team. After that you have a good idea as an instructor, what are they ready to do? What are still the fuzzy areas? What do they really not get at all? What are their competencies as a team already, even if every individual student doesn’t have it, and then you can do some corrective lecturing, basically, so many lectures that fill in some of the gaps. And that’s all part of the readiness process, because you’re getting them ready to do some interesting application work in class. And the rationale for that is… and actually what I had been doing prior to that, was giving interesting application material to work on at home individually, while doing lecture and cooperative learning in class. But the interesting application material was actually the heart of the course, and the much more challenging piece. So it was better to bring the hard piece where they needed support into the classroom. And the piece they were ready to do, which was to do their own self study back into their own lives. So you do this readiness assurance process to make sure they are ready for interesting application, and then the majority at the time for the module you spent on application. Doing that after I had already worked with cooperative learning was really helpful, because all of that application work is done in a team setting. So when you already have some experience with how to build teams, how to maintain and develop their social skills, that’s really, really, really helpful. That’s a short version.

John: One of my colleagues, Bill Goffe, who was on one of our very early podcasts, noted that when he gave the group test, the performance always went up significantly, so that they could see the benefits of the peer discussion that was part of that. And he was really impressed with it. And he noted that, oftentimes, if a student didn’t show up for class one day, they get a hard time from their classmates from the group because they let the group down. And he said his attendance had never been better than when he was using a team-based learning approach.

Kristin: Absolutely. And a lot of people who do team-based learning, use the same methodology for doing the team testing, which is honestly really cute. It’s a scratch-off form. And the scratch-off form is used so that the team gets immediate feedback on each option. So on any particular item in a multiple choice test, if they want to select “B” they scratch off “B.” If it’s not there, then they continue to scratch until they get the right answer. For one thing, they love it. But also they are getting immediate team feedback. If this person is not speaking up, if they say I think it’s “B” and then they stop advocating and then it turns out to be “B” later than the team immediately knows, by the time they get to the next question. “Okay, we need to incorporate more feedback from all of our team members, wait a minute, this person who’s not speaking up actually has a lot to say.” In the course of just a few multiple choice questions, it brings their team development forward leaps and bounds. And they kind of have fun with a scratch off, which is also a bonus.

John: And it also gives them incentives to come prepared and to listen to other people in ways that they might not otherwise.

Kristin: Yeah, and their team will give them grief, if they say “Oh, I don’t know, because I didn’t read,” their team members will be like, “But we are depending on you, you need to read, we all read.”

John: And it also gives them a little bit, perhaps, of improvement in metacognition because they’re getting that immediate feedback, and it’s being coupled with the reactions of the peers. So if someone was insistent on a wrong answer, and they dominated that discussion, they might be a little more careful in the future and more willing to listen to the other people and reflect.

Kristin: Exactly, and it doesn’t have to wait till next week, it can happen right away. Right on the next question. The team application activities are also structured in a particular way. In team-based learning, they talk about the four S’s for the application activity, the first one is that you have to select a significant problem. So what they’re working on is something that will be important to them, something that they will identify with, or that they recognize is worth their time in thinking about and trying to think through. The second one is that they need to be working on this same problem. You can’t say teams one and two are working on this, three and four are working on this, five and six are working on this. Third one is that they structure in so that they make a specific choice as the outcome. Because it’s easier to solicit team feedback if everyone is making a specific choice rather than having kind of an open-ended narrative response. And it helps to stimulate whole group discussion as you’re moving. Now it can sound like it’s limiting to say that you have to make a specific choice, but you can do in a very broad way. And the fourth one is simultaneous reporting. So all the teams are asked to report at the same time on what the choice was that they made, so that they can’t piggyback off another team who’s putting in effort. So, as an example, one of the courses that I taught in the psychology major in Texas was the tests and measurements course in psychology, and test and measurements starts with a stats review. They’ve all had statistics, it usually comes prior to tests and measurement. But it’s the first time that they have an opportunity to work with statistics in kind of a decision-making way. So you start with a stats review. So one of the activities that I would do, I gave them two hypothetical first-grade teachers with how many questions 10 of there students got right on a spelling test. And the two distributions had the same mean, but one was fairly normal, and one was highly skewed. So they had to do their quick statistics review… Do the mean, median, mode and standard deviation describe the shape of the distribution. But the question I was asking them was, “If you were the principal, which teacher would you offer an after-school tutoring program to for extra pay? And which teacher would you potentially nominate for a teaching award?” They found that question to be a really interesting question. For one thing, students think a lot about what good teaching is, and what constitutes a good teacher. So they already come in with very strong opinions. And they also understand the complexity of, you know, if everybody’s passing but people aren’t excelling, is that good teaching? Whereas if most people are failing, but a few people are getting an “A” is that good teaching? …and how the data contributes to good decision making, but can also be kind of manipulated to contribute to decision making in not such a good way. So instead of just saying, “Let’s review the stats, here they are,” it was a question with a specific choice that they simultaneously reported on. And then we could discuss together. And of course, their answers are different. There’s different rationales in both ways. So then we could discuss together what their rationale was, if they want to debate they can debate a little. It generates a lot of student enthusiasm, and everybody’s doing it instead of just 10% of the class.

John: And once they’ve committed to an answer, they have a stake in and they really want to know, that’s something we’ve seen a lot of things we’ve talked about in the past, too.

Kristin: Absolutely.

Rebecca: When you were doing the team-based learning, were you sticking specifically to problems that were on a class-by-class basis still, like you were discussing in the co-operative setting, or were you doing some longer term activities that went across multiple class periods?

Kristin: I had the… what I consider gift… to often be teaching in a three-hour time slot, which is my very favorite time slot. So I would have activities that would extend two or three hours, but typically not between classes, I found that to be more of a sweet spot, at least for me. At my previous institution we had a very high commuter population. And I promised, in both models, that I would never ask them to do something out of class with their teams, that was one of my rules… that it was just simply too burdensome for students who have multiple outside of school commitments… family and work, or living potentially 150 miles apart, which was not unheard of. I promised them no out of class stuff. I structured that intentionally so that the individual preparation that they were doing, they could do anywhere on their own time schedule, but they were expected to be there. And their team expected them to be there to be able to engage in class. And it was also one of the ways that you talk people into it, when they say “I worked with other groups who were all slackers and we would always set times and they wouldn’t show up.” And I said “That’s not going to happen in here. We already have a time we’re all going to show up together.”

John: And the philosophy that’s very similar to the flipped classroom approach where you let students do the easy stuff outside and then give them assistance with or have them work in a framework where they’re getting more assistance with the more challenging issues.

Kristin: Absolutely. I think TBL [team-based learning] is definitely a flipped classroom approach.

Rebecca: I think the other thing that helps too with that model… of making sure you’re not working outside of class… really helps students with really different backgrounds start working together, because you might have students who are more traditional who are on campus. And so for them to meet outside of class is often not such a big deal. But then if you have students who are working or have families, and there’s a disconnect in the class, even, between those two populations, that helps make that more obvious and work a little bit better,

Kristin: Right. Absolutely. Yeah. And I didn’t want to set up anything where people were made to feel like unvalued team members, because they couldn’t do what was asked of them because of other commitments. Since that was in my control, I wanted to make sure that people felt welcome.

Rebecca: I’ve tried to even do that with long-term projects. In the field that I’m in, we tend to do things that go across class periods, but there’s always the “Are we going to do this outside of class or are we going to do this inside of class, and I try to have them do anything that needs to be collaborative, and decision making, in class, and then things that can be done on their own, even if that means doing some creative work, or whatever, outside of class. But those are independent things that can be done for the same reasons. And I find that students will try to manipulate that system, so that they’re gonna: “Oh, we’ll just do it outside of class, because we don’t want her to know whether or not we’re on top of something,” or whatever. But I call them out on it, because it’s really devaluing some of that exact thing. People have other commitments and things.

John: You mentioned, you started to use a backwards design approach where you started with the things you want them to remember five years later. Did you have to cut back on the breadth of the coverage in the class, to some extent, by doing that?

Kristin: Yes, I did. When I was going with the straight up cooperative learning approach, I did not have to cut back on the content at all. Without the full redesign, I found I could cover the same amount of material in straight lecture versus in a cooperative setting. But it was all coverage. It was just a different kind of coverage. When I approached it from a backward design perspective, and I really was able to focus on the objectives that I thought were important, I did have to reduce the amount of things that we were covering. I have no regrets about that, of course, because I completely recognize that covering material isn’t just covering it. What are students going to do with something I covered in class? They didn’t cover it, I was the one who was learning it and talking about it. So I’m much happier with an approach in which I am consistently hitting on the objectives that I really want them to recall, and that they are working hard to apply those throughout the semester.

John: If they’re not going to remember it passed the final exam, covering more material isn’t terribly useful.

Kristin: No.

John: We talked about that in a previous podcast with David Voelker, who talking about the coverage approach in History…

Kristin: Right.

John: …which is the same logic.

Kristin: Exactly. And I actually now consider that to be a complete waste of time. So why am I spending class time on something that I actually don’t really care if they remember, it’s not the most important thing to me, and they really don’t care if they remember.

Rebecca: You have some compelling arguments for why team-based learning and collaborative learning are good options. If one wanted to start moving in that direction, what would you suggest their first steps be?

Kristin: For team-based learning, there are a couple of great books that are very easy to approach. There are several great resources for team-based learning. Larry Michaelsen published a book in 2008, for example, that covers that from front to back. It gives examples of applications in different disciplines. There’s also a book published a few years later on team-based learning in the social sciences and humanities. That also covers the basics, but has applications that are more specific to social sciences and humanities. Team-based learning has really caught on in medical education and in business education. So in the original book, there are more application examples that are in MD preparation or in business schools. So if you’re looking for other examples, the second book might be a good choice as well. And that one is edited by Michael Sweet and Larry Michaelsen.

John: And in fact, I read your article, or

Kristin: Oh, did you?

John: …your chapter in there as background.

Kristin: I’m glad someone read it.

John: Now I have to read all the others. But, I, at least, did read that. It was very good. So for faculty who are moving to this, what are some pitfalls that they might run into? Or what sort of problems might they encounter?

Kristin: Team-based learning as being a much more structured approach… Michaelsen does a really nice job of laying out the pieces that he thinks are critical. And I agree they are critical. So, for example, he talks about explaining, and testing the model with students on the first class day, and you cannot skip it. So the very first class day, I give students an example individual application test, like they would get for their readiness assurance. It includes basic psychology knowledge that may or may not be present in the culture. So they have some chance of getting some of them right and some not. And then I have them do it as a team. And the team scores, of course, are always dramatically higher than the individual scores. And the team testing process is so much better. [LAIGHTER] It’s more pleasant and interesting and collaborative than they expect it to be. That simply going through that, it allays many of their fears about what a team is going to be like to work with. Plus, when they see that the team has tripled their individual score, they’re like, “Hey, maybe I could depend on other people to help me learn, and maybe this will pay off for me.” So going through an explanation of what the rationale is, having them experience it a little is really, really critical in helping them stay open minded while they experience it. And then regularly throughout the semester, I will keep reinforcing them with those messages. I’ll say, look at this amazing thing you guys did. You used all the intellectual resources around you, and you analyzed this difficult problem and came up with some great solutions. I’ll remind them how much they’re learning and what kinds of challenging tasks they’re able to do as a team when they have the preparation to do it, which helps as they’re starting to think “Well, wouldn’t it just be easier if I could do this by myself?” It helps them to kind of remember, ”Well, yes, but you wouldn’t be doing this, you would be doing something not as challenging, not as integrative.”

John: and probably not learning quite as much either&hellp;

Kristin: Yes. He also emphasizes an aspect that is also emphasized in cooperative learning… of helping the teams develop and giving them feedback, helping them give each other feedback. That’s also really critical, especially very early in the semester, as they’re starting to develop group norms and bond together to make sure that you don’t short the time in class for them to have some group processing time and to build their team skills. So, for example, when I taught last spring, I had a student who came to me after I think it was the second week. So it’s very early in the semester, and she said, “I really need to reassigned teams. My team hates me, they won’t make eye contact with me.” She was really upset. And I’m reluctant to reassign people teams, because often what they’re experiencing, they take with them. It’s not always a function of that team process. So we talked some, and I tried to get a handle on what she was experiencing. I knew where she sat, I had an idea of the team composition. And I asked her to try one more day, just one more day. And then we would talk about reassigning her teams. And that day, I was sure to build in plenty of time for group processing, where they talked about what they were doing well as a team and something to improve. Their team turned around immediately. She was a relatively assertive person, which I already knew. I knew that she could handle this. So she went back to the team. She was able to talk with her team about not feeling heard. They immediately turned around in the way that they were with her. And by the very next class day, they were a relatively high functioning team. They did well all semester. They brought doughnuts for each other. I mean, it was a really nice supportive group. What they needed was the time in class to do some processing. And if I, as the instructor, had been moving too fast, and not giving them time to do that, and not giving them a prompt to do that, it would have been a really negative experience for her. So, also building in time for the team to develop and prompts for them to do that.

Rebecca: So you mentioned liking to have a three-hour teaching slot.

Kristin: That’s my favorite. It’s not required.

Rebecca: So, in that amount of time, how much time would you designate towards this group processing, for example, to give people an idea of what that proportion or the amount of time to dedicate so that you don’t shortcut it and you don’t rush through it?

Kristin: If I were to do an activity that might take an hour, I might spend 10 minutes for group process, it doesn’t have to be very long, or even five. And you don’t have to do it every time, you could do 10 minutes after the first one or two more intensive activities, and then not do it for another few times… and another five minutes just every so often to help them resolve their underlying dissatisfactions and to recognize that what they’re doing is not just application activity, it’s also group interaction. So please take time to do both. Another really important required component that I didn’t mention is peer evaluation, I always incorporate peer evaluation as part of the grade.

John: How did you form the teams in these classes?

Kristin: They’re heterogeneous, first, with a very open process so students can see it happening and know there are no shenanigans… that this is all very open… talking about the rationale that people of different backgrounds bring different strengths. So you want a group that has people of different backgrounds, so you can have a larger kind of learning base between you. So usually, I’ll pick a few characteristics that might be important in that kind of background. And I will line them up around the room based on those characteristics. And if it’s 200 people, it’s a really long line. And then we count off. So when I teach introductory psychology, students who have had a high school psychology class usually are starting a big leg up on the other students. So I’ll include that as a characteristic. Sometimes I’ll include the distance that people are coming from, because then they have different experiences, depending on what class I might also include if their student athletes, just because if you put too many together in a team, then they’re all gone on the same day. They have interesting backgrounds, but they also have patterns of attendance and of absence that need to be adjusted around. And we’ll count off all the way around so people can see how the teams are made. But heterogeneous teams are really, really critical. Having students with pre-existing relationships will throw off the team process in a way that automatically excludes people that don’t have pre-existing relationships… plus they tend to be lower performing teams. And I don’t want to set that up on purpose.

John: One of my colleagues once did this in a class of, I think it was about 350 students, but he just sorted them alphabetically. So he had them organize himself that way, and it was a fairly long process. But, it was kind of amusing for those of us wandering by and just seeing…

Kristin: …this huge line… Yeah.

John: He didn’t do it that way In the future, he used other criteria.

Kristin: I’ve had colleagues that I’ve talked with that think that this is a long process. It’s not. You can sort 200 people in 10 minutes, and then you’re done for the whole semester,

John: Doing it alphabetically…

Kristin: takes a lot longer.

John: …can be more challenging, because they were self forming that… it didn’t convert rapidly.

Kristin: The other thing I never do is I don’t put the students who didn’t come the first day into a team, because there are characteristics about why they didn’t come the first day. If you put them all together in one team, they share some of those characteristics… It tends not to be a very high performing team. So I make sure they’re sorted out among the other teams. But that was one of the things that I learned in cooperative learning. That, before I did cooperative learning training, and I was assigning group work, I would assign people based on if you didn’t come the day we did the assignments, you were in another group. And that group typically did not do very well. And as an instructor, it’s my responsibility to create a learning environment in which students can excel, it’s on them whether they do their part. But if I’m setting up a team in ignorance, with predictable characteristics, so that they’re going to have a failure experience, that’s on me to correct. And it’s not on them. So afterwards, I felt guilty when I had come to a new realization. But, yeah, it’s my responsibility to set up an environment in which those students can be successful in their teams.

John: In your chapter in that book, you mentioned that when you switched over, it did affect your course evaluations a little bit. Can you tell us a little bit about that?

Kristin: Just a little bit. But yes, it did. So when I was doing straight lecture, I was shooting for engaged lecture. And in psychology, you can build in little experiences, especially in introductory psychology, where the topics are changing frequently, you can always build things in that are kind of interesting. You can do a little optical illusion here and a little bit of memory trick there. And there’s these ways to build it in, but it is still basically straight lecture. And I got high evaluations for that. I was careful about trying to build those in every day, you know, every few minutes. And when I went to cooperative learning, where it was essentially the same approach, but in in a much more engaged and cooperative fashion, those evaluations stayed very high. Students knew each other, they were happy in class. When I went to team-based learning and I was actually asking every student to participate all the time, and be prepared in class in a way that their contributions were much more obvious than mine. My evaluations did drop just a little bit, not a lot, but a little. And I am grateful that I was teaching in a context where I knew that my department wouldn’t care. They were more interested that I was doing good teaching. And they understood the many factors that influence student evaluations. But I also recognize that it’s incumbent on me to help students understand how they are learning, what kinds of things encourage learning and retention, and then you kind of let the student evaluations fall where they may.

John: When I read that, it reminded me of that study that came out a few weeks ago from Harvard in their physics program, where they found that students in active learning classes did demonstrably better on tests, but they perceived their learning as being lower. So there was a pretty strong inverse relationship between their perception of learning and actual learning. That seems to be fairly common, there have been a number of other studies where what students think to be most effective, is often not what most enhances their learning.

Kristin: Right.

John: Do you have any other advice for our listeners, who might think about using either collaborative or team-based learning in their classes,

Kristin: The one thing I would say is that teaching a cooperative learning or a team-based learning structure class is a lot more fun. You have to be willing to give up control, because when you’re lecturing, you have absolute control… meaning even that students can’t ask you weird things, because you haven’t opened the door for that to happen. But when you structure the learning experience, and then you give up the control to the students, it is an exciting environment to be in. I wasn’t as tired when I was coming out of class. I was energized, you could feel the difference in the room just walking into class… they were excited and talking with each other. When I would circulate around before class started, they’re talking about the class instead of talking about other stuff. It completely changes the environment in the classroom in a way that I think really matches what I expect out of a university education for students, it creates a environment of intellectual enthusiasm around the topic that you’re teaching.

Rebecca: So we always wrap up by asking what’s next?

Kristin: That’s a great question. So right now I’m 100% administrative. And since I’m in a new position, in a new institution, I’m gonna spend some time figuring out all the newness pieces. But I’d like to go back to the classroom, at least for a course here and there when I can. There’s nothing different about students than there is about people. So I also think often about how what we do in the classroom, what we understand works and what we understand doesn’t work, how that applies in administrative settings as well. We know for example, that people tend to try and find the shortest path. So if they’re trying to learn something, they want to put in the least effort to learn it. If you ask a faculty member to do a task for the department, they are obviously going to choose the easiest path to do that… not necessarily the best path. So how do I take the experiences of learning and teaching, that in some ways are better understood to an environment of administration that in some ways is not as well understood? What kinds of lessons can I apply there as well?

Rebecca: Well, thank you so much. It’s been a really interesting conversation. I’m sure it gives a lot of people things to think about as they move forward in this semester and future semesters.

Kristin: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.