66. Just-in-Time Textbook

What would you do if you are scheduled to teach a class of 75 students and discover that several very expensive textbooks would be required to address the full range of course topics?  In this episode, Dr. Jessica Kruger rejoins us to discuss how she responded to this challenge by working with her students to  create their own textbook. 

Jessica is a Clinical Assistant Professor in the Department of Community Health and Health Behavior at the University at Buffalo.

Show Notes

Transcript

John: What would you do if you are scheduled to teach a class of 75 students and discover that several very expensive textbooks would be required to address the full range of course topics? In this episode, we talk with someone who responded to this challenge by having her students write their own textbook as they progressed through the course.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Jessica Kruger, a Clinical Assistant Professor in the Department of Community health and health behavior at the University at Buffalo. Welcome back, Jessica.

Jessica: Thank you. Happy to be back.

Rebecca: Today our teas are:

Jessica: Not today, but maybe a little bit later to relax, thinking about all of this stuff I need to do before the start of the semester. [LAUGHTER]

John: …and I am drinking Rose Garden Black Tea that you brought back from…

Rebecca: …Epcot. So I’m drinking the same thing. We’re having a tea trial this afternoon.

John: It’s one of the blends that they only sell at the Twining store in Epcot I believe…

Rebecca: …Yeah, and it’s a nice counterbalance to the lovely weather we have outside.

John: We have invited you here to discuss the open pedagogy project that you ran last semester in a class with 75 students. Once again, scaling things to a higher level than people normally do it. Could you tell us a little bit about that class?

Jessica: This class is a 300 level public health course. It’s titled Methods and Mechanisms in Public Health. It covers three main topics, so we start out with environmental health and then we move into information about health theories and health behavior theories and then we end with disparities. And so with this class, there was not one single book that would encapsulate all the topics. Instead of having the students buy three or more different books, I started to think, well, what else could I do?

John: The natural thing is to have them write their own book…

Jessica: Of course, why not?! I think the cost of textbooks are continuing to rise, and especially with this sort of course that specialized with these three different areas. I don’t think I would ever find a publisher that would make something quite like that. So why not write your own?

Rebecca: How did you pick the topics that were included and how did you get going on this project?

Jessica: So I actually heard about open pedagogy and writing a textbook with your students at the CIT conference that happened last year, and I was really inspired by Robin de Rosa and what she had done. And so immediately after hearing from her, I thought, “I bet I could do this.” At that time, I don’t know if I was deranged because it was the end of the semester, or maybe it was a stroke of brilliance. Probably a mix of the two. But I thought, let’s figure this out let’s see how to do this. And so, as I was looking over the syllabus and the topics, I started flipping through their textbooks, I started looking at other resources that were available and began to put the topics together and I broke it down by the weeks of the syllabus. So my students were actually writing the textbook before they learned the content. Which for them, was very scary and for me, a little bit scary too. But the great part is they actually had to go out they had to find resources, they had to put it together. And as I was building this, I created Google documents and made skeleton outlines for the chapters and that’s how they kind of got started.

John: So you created the skeletal outlines and then they fleshed it out?

Jessica: Yes, literally. The outlines were a title of the chapter, some objectives, and headings, different headings of sections. I attach some information in each Google Doc, some resources that were out by the CDC or other peer reviewed sources that I thought could be helpful. And of course, I invited them to come meet with me, especially if they have no clue what I was talking about with this topic.

John: …And within each week, did you have a subset of students work on it? Or did you divide it up among all the students? How did you arrange that?

Jessica: So in this class, my very small class of 75, which is actually my smallest class this past semester, I broke them up into groups of four to five students. They didn’t get to choose their groups, but they did get to choose their topics. As they went through and looked at it, we broke it down and said, “Okay, this group decide what is your top picks,” and they could choose. And they knew the order of the chapters, so if they wanted to get it done out of the way, they could do it at the beginning of the semester or wait till the end. And so each group worked together to make a contract, divide up the work and choose how they’re going to execute this.

Rebecca: So what worked well about that method and what didn’t work well about that method?

Jessica: Oh, students love group work. We all know that they love group work, right? If I could figure out the secret to making it go smoothly all the time, I guess I would probably be a millionaire. But nevertheless, I think there was some strategies that did work well. And the fact was, I had the maker group contract. Barbara Oakley actually has a article called “How to work with a couch potato,” and in that article it talks about how to deal with someone who’s not pulling their weight and how to create a group contract that’s actually useful. And so the students worked together with their group and talked about how they’re going to evaluate each other. I didn’t set a peer evaluation, they did, and they also broke up what they’re going to do. So as they’re creating this chapter they would all right in a different colored text. So one student was green, one student was orange, and so they can see visually what each person did. And my caveat was, if someone drops the ball, they drop the ball, you don’t have to make up their work, as long as it’s stated in the group contract, that is fine. If they don’t do the summary they don’t do this summary, your chapter doesn’t have a summary. That’s okay.

John: How did that work? Did everything get completed?

Jessica: Actually, we only had one chapter I believe that doesn’t have a summary and most everyone did their part. There’s always some squabbles back and forth. There were some groups who did really good strategizing and had someone go back through and create one voice for it, other groups didn’t. But overall it worked out quite well breaking it up that way, but also giving them that out, realizing that they didn’t have to make up something that someone else didn’t do.

Rebecca: So how did you handle chapters that weren’t fully complete and you needed the other students to read those chapters?

Jessica: I had a wonderful TA last semester, and the chapters would be due about a week before they needed to be put on to UBLearns. She would alert me to anything that was going on or anything that should be changed and I would look over the chapters. And then I would bring in some other content or modify it. But overall they actually did a really great job of putting this together and finding sources…

John: …you mentioned UBLearns…

Jessica: …UBLearns is actually our Blackboard learning management system. So they would offer it in Google Docs, and then I would take that and create it so that they could comment on it, but not actually edit it. And this allowed them to get feedback from their peers, which we plan to preserve the feedback for the next time is classes taught, so this book is a living continuing iteration.

Rebecca: So as the students were reading the chapters as assigned reading, is that when they were providing the comments and the feedback?

Jessica:Yes, they would go through and we had some very astute grammar students that would go and pick commas, and also asked for more explanation, which was excellent. I could actually use that in my teaching to talk further about areas where I could tell that students needed more help.

John: Did the original writers go back and fill in some of the gaps at that point?

Jessica: Sometimes they did, but for the most part, we left it as is. We used version one and the plan is for the students who take this class next to take those comments, continue adding, continue changing, and revamp the book. So there’ll be multiple versions for each semester that it’s taught.

Rebecca: Can you talk a little bit about how you handled copyright and giving credit where credit is due for each of these chapters and how you might handle that in future iterations?

Jessica: Overall the class work together to figure out their creative commons license, and what they chose was a CC BY-NC-SA. Which means, people must attribute and give credit, it’s non-commercial and it’s share alike. We talked about this as a group, we learned about copyright and they all actually signed a contract as if they were publishers, as they are. We discussed how to attribute content, some chapters did it one way some chapters did it another way. So you’ll see in tech citations in some areas and and others you’ll see, “this was modified from the CDC website at this location.” And so some of it was actually openly sourced information that was reused. Now, this is their first time writing a textbook, so can I stand behind that everything is completely cited perfectly? Absolutely not. But they did a good faith effort to make sure that their information was cited properly.

John: …and there is a little note at the bottom of each page listing sources and saying that, to the best of our knowledge, this is not subject to copyright. If you find anything that appears to violate it, please notify us. So there is a procedure for addressing that stated on each page, I believe?

Jessica: Correct.

Rebecca: Can you talk a little bit about how the groups decided to evaluate each other?

Jessica: Yeah, I think group evaluation and teamwork is such a challenge. I myself, despised group work as a student, and as a faculty member sometimes I’m like, “Ah, I know they need to work with others, but…” But really having them create their own contracts and having them evaluate each other on their own terms, so some groups decided that “we will do everything perfect and we will come to every meeting.” No, that’s not a group contract. That’s not something that you can actually achieve. But what you can achieve is open communication, and so most of the groups actually used GroupMe or other tech tools to stay in contact, they would set up meetings sometimes before after class. They would also work on the Google Doc in tandem, you can actually see when someone else is working on a Google doc and point out different sources to each other and discuss how they’re going to put it together. And so that allowed them to evaluate each other and most students gave each other pretty high marks in peer review. And I don’t know if that’s because they all like each other and they’ve all been in multiple classes together, or if it’s actually how they feel that this came together. But overall, it happened, it got done and so I do think they work together pretty well. Some groups obviously better than others depending on their strategy, but having them create the contract I think is the important aspect that I found

John: And you were able to monitor that by seeing the colors of the contributions?

Jessica: Yes. So I had them color code their contributions and if they didn’t color code because someone was an editor or whatnot, they would note that so they knew what each person was doing. And then before I would send it out to the class, I would take out all those colors and do some type editing. But overall, each of them were contributing and most of the students in the class are juniors and seniors, so they’re upper level students.

Rebecca: Did you have students evaluate each other at the end of their contribution, or at the end of the semester?

Jessica: It was at the end of their contribution, and it’s fascinating because I asked them during that process, not only to evaluate themselves, but to evaluate this project. And it’s fascinating because as I’m going through it, I’m looking at it and asking them, “Do you think other classes should do something like this?” And about 50% said no, at that time. I was like, “Oh my goodness. Is this going to fail? What’s happening? Why do they despise this so much.?” But, at the end when we did an overall evaluation, it was actually overwhelmingly positive, it was all positive about the whole project. So it just goes to show you that as you’re in the thick of something, you may see it as challenging, overwhelming, but in the end, when you see that final product, when you see that 19th chapter for 200 page book that you’ve created with your fellow students, that’s powerful.

Rebecca: You ended up getting your book printed and copies distributed to each of your contributing students. How did you pull that off during the semester you were writing it?

Jessica: Very carefully. I worked very closely with my TA and I spent countless hours editing and making sure pagination was correct. I think in the end I probably put in a whole work week just getting that together and working with the university printing service. I sent it in, it was excellent. I’ve never printed a book before not knowing how to do anything like that, and thankfully our university had some funding to allow me to print books for all of the students. And to see their faces when they got that book, was just outstanding. I actually created a celebration and so I invited the Director of the program, the Dean, the Chairs, and even people from OER Services at SUNY. And the students walked into a cake and people clapping for them, and then I reveal the printed book.

Rebecca: So you had to have had the book finished multiple weeks before the end of the semester to make that happen?

Jessica: Well, I was hoping to have it finished sooner. But you know, life happens, and so they actually got it on the day of their final and their first question is after it’s revealed is, “Do we still have to take a final?” In fact, they did, but it turned out alright. So it was the day of their final which was a week after the end of the semester. But it all turned out fine, everyone was happy, and they got to eat cake.

John: What proportion of their grade was based on this collaborative work?

Jessica: It was actually about 10% of their grade… [LAUGHTER]

John: [LAUGHTER]… low stakes assignment, relatively, for writing a book.

Jessica: It was very low stakes, which, as you choke on your tea, I couldn’t believe that they would do it for 10% of their grade. But in fact, when you take all of these small pieces of writing and put them together, it actually wasn’t a huge whole. When you have 75 people writing a little bit at a time, they got into it when they had to do it but when they were done, they were done. So it was actually just a small percentage of what they had to do. I like to use non-traditional teaching techniques and experiential learning, and so this class also took a field trip and did a lot of other exciting teaching techniques. So this book was something that was a small percentage of their grade.

John: [LAUGHTER] That’s impressive.

Rebecca: How much time do you think each student actually spent on the writing that they contributed?

Jessica: I would speculate a few hours. They were writing in chapters and I didn’t give them page limits, which was interesting because most groups wrote about six pages on average. But then you had some groups write more and they were allowed to put pictures and videos and diagrams because some of these are models. So some of the chapters were far longer and some students are more lengthy writers, some are more succinct. And so it just depended on the group and what topic it was.

Rebecca: Did they also have to present their chapter?

Jessica: They don’t have to present their chapter but they did have to read the chapters. The whole test was based on the book and the presentations that I gave during class on the content.

Rebecca: What advice do you have for other people who might want to take on such an adventure?

Jessica: [LAUGHTER] Ah man… [LAUGHTER] I think you should do what sparks your interest and you should really follow your passion and if this is what you want to do, proceed with caution, but dive in. That seems to be my approach to a lot of these quote unquote weird pedagogical techniques that I like to use. But this was really a joy. It was a lot of work so much work, not only to set it up and get it ready, and then convince students to do it for only 10% of their grade. If you would have seen their faces but, I said, “Hey, guys, guess what? We’re going to write a textbook.” They all looked at me like I was insane. And so did my colleagues. [LAUGHTER]

John: [LAUGHTER] That was gonna be another question. Has anyone there considered following up with this and doing something similar on their own?

Jessica: I have not had anyone take me up on that. I’ve had a few people asked me, “Do you think others should do this?” And my answer is usually, “You should think about it.” It’s a lot of work, but it’s very rewarding the students get so much out of it. For my evaluations, the students not only said that this helps them learn the content better than a traditional textbook, because it was written by other students, but it also helped build their confidence. They were writing a book, and now they can put that on their vitae, and all the students were so excited because they’re going to take it home to their parents and show them what they’ve done in class. They also talked about just that it was new and novel and how that makes them excited about it. But it depends how much time you have, how patient you are, and what the topic area really is.

Rebecca: When you go to do a revision next time you teach the class, do you expect your time costs to be the same, or do you expect it to be a bit different now that you have a structure?

Jessica: I think now that I have a process and understand how much time it’s going to take versus saying, “This will be fine. It’ll work out well,” I think it will take a little bit less time. But I think with that you also have to get students buy-in, because it’s not completely new, it’s something else is someone has created. And sometimes that’s more challenging to add to it, to modify it to make it better. But I think it’s a good exercise in teaching students the importance of revision and adding to something and building it.

John: Did any students object to having the work being posted publicly, or were they all happy with that?

Jessica: They were actually really excited. When we first were doing that hey said, “So where is this going to go?” I said, “Well, it’s going to be public. Everyone’s going to be able to read this.” And they looked at me, and at that time I don’t think they really understood what open source was, completely being on a website in the Lumen platform, being able to see this content. But once it was printed in a book, they said, “Well, where does it go next? Are we going to print more of these is in the library?” I said, “Well you just wait.” And then I was able to send them the URL, thanks to the SUNY OER Services who put it together, so that the students can now show it to their friends and post it digitally and share it.

Rebecca: [LAUGHTER] What kind of kool-aid did you hand out the day you’re talking about this assignment? [LAUGHTER]

Jessica: [LAUGHTER] Well, I should mention that all of these students were actually students that I’ve had previously and I think having that level of trust and understanding that, “you never know what’s going to happen and Dr cougars class, just be ready to roll with it.” I don’t think I could have done that with students that I haven’t had for multiple semesters. So that trust and rapport was really important and saying, “You can do this, I believe in you. Let’s do this together.” I think without that, this would have been more challenging and students would have said, “Who cares about that 10%, I’m out.” But in this case, they understood and they followed along and they were happy to do it. It was something of a challenge. I think some of the comments in which students told me about this, were just amazing. One of the students said that they generally felt this was a great experience and a wonderful opportunity and that experiences like going on field trips and writing a textbook was exciting, and made me feel like a kid again in elementary school. It makes me more motivated and looking forward to learning experiences. So I think having novel experiences and having something that’s new but also exciting and exhilarating and gosh, a little challenging, is good for the students.

John: …and they’re actually creating something themselves, which by itself should be a little bit more motivating than passively consuming a textbook that someone else created.

Jessica: Exactly.

Rebecca: I think the report that you mentioned earlier is important too. I think that’s an interesting component to this particular project that some people might not realize how important that that can be. But having that little bit of trust to go on a bigger adventure, then maybe they’d be willing to otherwise, I think is key, but something that we all can be thinking about.

Jessica: Oh yeah, I think it’s so important to make connections with your students and people do it in many different ways. I’m usually known as being (taryn?? 22:00), that shameful word in academia. But in fact, I think it’s so important and that’s how I can get so much buy-in from students and get them to join me in these learning adventures that we tend to go on.

John: We do have a note to ask you again, to come back at some point and talk about the field trip, things that you do with these large groups of students as well. But I think maybe we should leave that for a future podcast, if you’re willing?

Jessica: Excellent. Sounds great. I love being on the show.

Rebecca: I don’t know if I dare ask our final question. But we always wrap up by asking what’s next? [LAUGHTER]

Jessica: [LAUGHTER] What’s next? Well, this semester I vowed to focus and more self care and I’m actually teaching a new course called Stress and Population Health. And so with that course, I’m trying to take my own advice, which is sometimes the most difficult, and only doing a few crazy activities during the semester. So my students will go on more field trips, they will do some experiential learning, but they’re also going to be focusing on stress within the college campus, and performing some stress reduction tabling around public health and also learning a little bit more about meditation and how overall in the US we’re a little bit too stressed. So with that, I think “what’s next” is we should all take a little bit more care for ourselves, to be around for students and to give a little bit more. So, that’s where I’m putting “what’s next.”

John: That sounds like a good strategy and perhaps chance to relax a little bit and I believe that when I hear about it later, at the end of the semester. [LAUGHTER]

Rebecca: [LAUGHTER] You have to remember that it has to be in comparison to, you know, someone’s lack of stress is really dependent upon how much stress they generally pile upon themselves.

Jessica: [LAUGHTER] Exactly. So, I’m only doing half of what I typically do in my crazy teaching. But still, I think it’ll be fun, exciting, and I’m looking forward to another great adventure and semester.

John: …and you’re also doing a COIL course at some point, aren’t you?

Jessica: I have currently already done one COIL course and I have actually just created a another COIL connection in Jamaica and have plans to create additional COIL connections so that we can actually compare components of health cross culturally and cross nationally.

Rebecca: Sounds really cool.

John: …and we should note for listeners outside of New York that COIL courses are Cooperative Online International Learning courses where classes pair up with classes from other countries.

Rebecca:Thank you so much for joining us again, Jessica and sharing with us how you rolled this project out and giving us all a little bit of inspiration and a little motivation to do some of this work ourselves.

John: Yes, thank you. I’m still amazed by the 10% but I looked through much of the work that your students have done, and it’s a really impressive work.

Jessica: Thank you. They’re very impressive students. I’m honored to have work with such amazing people. It couldn’t have been done without them believing in themselves and believing in what we were doing was important.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer and Jacob Alverson.

[MUSIC]

54. SOTL

As faculty, we face a tradeoff between spending time on  teaching and on research activities. In this episode, Dr. Regan Gurung joins us to explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.  Regan is the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research.

Show Notes

Show Notes

John: As faculty, we face a tradeoff between spending time on teaching and on research activities. In this episode, we explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.

[MUSIC]

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Regan Gurung, the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research. Welcome.

John: Welcome.

Regan: Thanks a lot, Rebecca and John.

John: Our teas today are…

Rebecca: I’m drinking Prince of Wales today.

Regan: Alright.

John: I’m drinking ginger tea.

Regan: Ooh, now you’re making me want to. [LAUGHTER]

John: We’ve invited you here today to talk about research in the scholarship of teaching and learning, or SOTL. You’ve conducted a lot of research on teaching and learning as well as research within your discipline. In most disciplines there has been an increase in the journals devoted to teaching and learning and an increase in research in teaching and learning, but it hasn’t reached everywhere yet. SOTL research is often not discussed in graduate programs and is sometimes devalued by campus colleagues. Why does that occur?

Regan: So. I think there are multiple reasons why the—and I’m going to start with the devaluing. I think there’s a lot of uncertainty about what it exactly it is, so on one hand, when people say a scholarship of teaching and learning… very often if it’s somebody who hasn’t really read up on it recently the sense is, oh, you know, that’s research on teaching; that’s not as good as your regular research. Now, I think that’s a misperception and once upon a time, and here I mean maybe even 15 years ago, there was some scholarship of teaching and learning that wasn’t done very well and I think people have heard about that in the past and that’s why there’s that knee-jerk reaction. Far too often it’s seen as something where it’s not as rigorous, perhaps, or it’s not done in the same way and most of that is wrong. What I like to tell folks who see that is, if you think the scholarship of teaching and learning is not rigorous, well, you haven’t tried to submit something to a journal recently. I co-edit a journal on the scholarship of teaching and learning in psychology and I can actually see some people submit poor work and I send it right back; I do the classic desk rejection and I say, look, this is just not good enough. So my favorite tip for “How do you write for a scholarship of teaching journal?” is very simple: just like you write anything else. There’s a lot of baggage, but I think that as you alluded to, John, it has changed more recently and I think part of what you notice now or what I’ve been seeing is that this kind of work, this kind of examination is being called different things. For example, a term that I’m hearing more and more often is DBER: disciplinary based educational research. And I’m hearing this come out of medical schools and engineering schools and social work schools and many professional programs where they’re doing DBER, which is essentially what the scholarship of teaching and learning is. So, I think because of that baggage with the term, people are calling it different things but in general the work is getting much more rigorous.

John: Excellent, and if changing the name is sufficient to do that, it’s a valuable step.

Regan: I think that’s why, when I talk about it I like to talk about it as: “Do you want to know if your students are learning? Do you want to know if your teaching is effective?” Well, then you should do some research on it. You can call it what you want. I started really calling it pedagogical research because that’s what it was, but it’s truly a rose by any name.

John: And that’s something that Carl Wieman has emphasized.

Regan: Absolutely, yup.

John: In the sciences, you test hypotheses and there’s no reason we couldn’t do the same thing in our teaching.

Regan: Exactly.

John: And that’s starting to happen, or it’s happening more and more.

Rebecca: In some disciplines, the scholarship of teaching and learning is not accepted as being part of their tenure and promotion file, for example. What would you recommend faculty do in a department like that if they really want to get started in SOTL?

Regan: Well, so, Rebecca, let me take you a half step back.

Rebecca: Yeah.

Regan: When you say “in some disciplines it isn’t as accepted.” What has surprised me is that most disciplines have actually been doing the scholarship of teaching and learning and publishing it for the longest time. I mean, if you take a look at chemistry, it goes back, gosh, seventy years or so. Almost every discipline out there has a journal that publishes the scholarship of teaching and learning, but, and here’s the big but: most of us in our normal training never run into it. So, I’ll take my own case. In psychology, the Teaching of Psychology Journal has been around for 46 years, yet all through grad school, all through my post-doc I never even knew the journal existed. Why? Because the programs that I went through weren’t focused on teaching the individuals—wonderful as they may be—who I worked with didn’t do that kind of work, so they didn’t know about it. So I think that’s a really important fine-tune there: there is a journal in almost every discipline—almost every discipline—for the scholarship of teaching and learning. So, it’s just a question of discovering it… it’s a question of finding it. Now, that said, where can they start? I think I can answer your question from a conceptual level and from a practical level, so I’ll start with the practical. The easiest place to start, there are lots of compilations of how to do it. For example, I think both of you have my website. On my website I have a simple tab called SOTL. On that tab is a list of places to get going, and I’ve organized it so that there’s a brief introduction to SOTL, there are journals, there are resources, there are little handouts. So, if a faculty member has even ten minutes, go to my website, hit SOTL, scroll through. That’s the more practical, that’s the easiest way to get started. From a conceptual standpoint it really starts with the question, what aspect of your teaching or your student learning are you curious about? John, I know you do some work in large-class instruction in economics. Why is this assignment not working? Can I get my students to remember certain concepts better if I change how I present information? It starts with a question. And you don’t have to read anything, you don’t have to look at any manual. If you look at your class and you go, “Hmmm, why isn’t this working, or why isn’t that working?” That’s where it begins, and from there you follow the same route that we always do: go look at what’s been published in it, fine-tune your question, design, think about what do you want to change and so on and so forth. I think it’ll help if I give you my working definition of the scholarship of teaching and learning, and when I think about it I think of SOTL as encompassing those theoretical underpinnings of how we learn. And more specifically, I see it as the intentional and systematic modifications of pedagogy and here’s the important part: the assessment of the resulting changes in learning. So that’s the key: you intentionally, you systematically, modify what you’re doing and then you measure whether it worked or not. That’s it. I could say that nonchalantly. There’s a technique , there’s a robustness to it, but at the heart, where do you start? You start by asking the question.

Rebecca: I think one of the things that I hear you saying is not much different than someone has a really reflective teaching practice—they’re doing it but not in that systematic way?

Regan:Yeah, absolutely right. There’s a term called scholarly teaching and in this kind of literature there’s a distinction made between scholarly teaching and the scholarship of teaching and learning, and all the distinction is is that scholarly teacher is reflecting on their work and then you’re right, you’re absolutely right; making those intentional systemic changes. That’s scholarly teaching. When it becomes the scholarship of teaching and learning is when you present it or you publish it, preferably through peer-reviewed ways, but you’re absolutely right; at the heart of it it’s scholarly teaching. It’s reflective intentional systematic changes.

John: One of the barriers, that people who are considering doing research in the scholarship of teaching and learning, is going through IRB approval, and in many disciplines that’s something they haven’t experienced before. It’s common in psychology. It’s less common in economics and perhaps in art.

Rebecca: It doesn’t exist in design. [LAUGHTER]

John: Could you tell us a little bit about that process?

Regan: Sure. Every university has an institutional review board and essentially what that board does is it’s in place to make sure that any research that’s being done isn’t harmful. Now, normally when we think about harmful we think about a drug or a food substance being tested, but here it just means any research that’s being done, and so when you do the scholarship of teaching and learning or when you’re examining your classes, yes, you could just look at your exams and see if exam scores are changing, but, if you do want to publish that, if you do want to share that, you really should go through institutional review board review. Now, the key thing here: it does sound like this whole new world, and it is, but at the heart of it is a very simple process. Now, there are three levels of review and I think knowing about the levels helps. For example, the first level is called an exempt review. The next level is called an expedited review, and the third level is called a full board review. I don’t think I’ve run into scholarship of teaching and learning that has gone through a full board review, because we’re not doing things that are more than minimum level of stress. Now when you say, hey, hang on, I didn’t know they were stress involved. Well, anytime you ask anybody to fill out a survey, there’s a minimal level of stress. And when you’re asking your students to reflect on their learning, well that’s a minimum level of stress. Every university has its own procedure. SUNY Oswego probably has a forum online. It’s a short forum; you’re basically telling this board what you plan on doing, what you plan on doing with the information, and most importantly, in these kind of cases, you are letting the board know whether or not students will be put under duress. What the IRB is going to look for is are you the instructor in some way forcing your students to do things that normally wouldn’t be done in the normal course of the educational process. But at the heart of it, all you’re doing is you’re sharing with this board whether or not you can do it and most scholarship of teaching and learning is at that exempt level. That exempt level essentially translates to exempt from further review. It doesn’t mean exempt from being reviewed; it just means this is mundane and low stress enough that it’s exempt from further review. Now that second level, expedited. If you do want to measure or keep track of names, if you want to look at how certain names relate to scores down the line—and that’s actually some really key research—well that’s expedited review. Now, even there it’s reviewed by one person. Both the expedited and the exempt review are reviewed by one person, often the chair. It often takes no longer than a week, and by doing that you just know that all your t’s are crossed and your i’s are dotted and it’s the ethical thing to do. So, whenever people say: “Oh, this is really mundane and I’m not really doing much more than just measuring student learning,” I still sa y if there’s any chance you want to present it or publish it make sure you go through the IRB.

John: And many journals will require evidence of completion of the IRB process.

Regan: Oh, absolutely. The moment you want to publish it you have to sign off saying that you got IRB review..

John: We do use an expedited review process on our campus. I was going to say, though, that we’re recording this a bit early because we’ve recorded a few things in advance, so we’re recording this in late October, but just yesterday I read that Rice University has introduced a streamlined expedited review process or IRB and apparently that’s something that’s been happening at more and more campuses. Are you familiar with that?

Regan: You know, not as much, because right now there’s so much up in the air with the IRB because national guidelines are changing. They were supposed to have changed in January, then it was moved to July. The latest I heard is it’s moved to next January. So, for the most part actual regulations are changing. Even on our own campus we switch from one form of human subjects training to another form, but this so called short-form expedited process will definitely help. That said, even the regular expedited, it’s a very easy process and I think the neat thing about this—and I tell students this when I’m teaching research methods, too—as the instructor or the researcher, just going through that IRB form really reminds you of some key things that you may have otherwise forgotten about, so, yes.

Rebecca: Do you talk a little bit about your own research to give people an overview of what project might look like from the beginning to the end?

Regan: Sure. What really got me interested in this is I teach large introductory psychology classes, the class is 250 individuals and I was struck by how when publisher reps come into my office and try to convince me to adopt one book over the other they would talk about the pedagogical aids in the textbook; “oh, look, our book has this and our book has that.” And that really got me started studying textbooks and how students use textbooks. So the umbrella under which I do research is student studying: What’s the optimal way for students to study? …and I use both a social psychology and a cognitive psychology lens or approach to it and it really started with looking at how they use textbook pedagogical aids. So, for example, in one of my really first studies I measured which of the different aids in a textbook the student uses and then I used their usage to predict their exam scores. Now, what I found, and this is what really surprised me and got me doing this even more, is that even those students were using and focusing on key terms a lot. Now, mind you, I’ll take a half step back—you may not be surprised to know that students use bold terms, they use italics, that’s what they focus a lot on. But students in my study also said that they use key terms a lot. Now if you’re studying key terms that should be good. If you’re making flashcards and studying those key terms that should be good, but what I found is that the more students use key terms the worse their exam scores. There was this negative correlation and that’s completely counterintuitive. Why would they go the opposite direction? So, I dug into it some more and I realized that students spend so much time on key terms or so much time on flashcards that they’re not studying in any other way. So even though they’re using flashcards, they’re so intent on memorizing and surface-level processing that they’re not doing deeper level processing. So, that was some years ago and I’ve been building on that, trying to unpack how students study. My most recent study… that’s actually under review right now… a colleague, Kate Burns, and I took two of the most recommended cognitive psychology study techniques, which is repeated practice or testing yourself frequently and spacing out your practice or spacing out your studying, and we took both of these and across nine different campuses divided up classes such that the students in those classes were either using high or low levels of each of these. So, in one study across multiple campuses we tested is there a main effect of one of these types of studying or is there an interaction? And what we found is that there is an interaction and the critical component seems to be spacing out your studying. Not so much even repeating your studying, but really spacing out your studying, and I think what’s interesting here is the reason this is happening is the students who said that they were testing themselves repeatedly, that sounds great, and if you’re a cognitive psychologist you say, hey, the lab says repeat testing is great; the problem is in the classroom a lot of students who were repeatedly testing themselves were repeatedly testing themselves during a really short period of time.

John: Right, I’ve seen that myself.

Regan: And I think that’s the issue, but because we had both these factors in the study, we could actually tease that out. So that’s the kind of work that I do… is take a look at what the cognitive lab says is important; let’s see how it works in the actual classroom.

John: Now was this a controlled experiment? Or was this based on the students’ behavior?

Regan: So, yes and no, okay. [LAUGHTER] I love this study because of a number of reasons. Number one, we tested two different techniques in the same thing. Number two, we did it at multiple institutions, so it’s not just my classroom. A lot of SOTL is one class. So, here we went beyond to try and generalize. But, to get to your question, we actually used a true experimental design. So we recruited these different campuses and we assigned a classroom. So, for example, I’d say, “Hey John, thanks for taking part. If you can have your students do high repetition and high spacing?” “Hey Rebecca, thanks for taking part. Could you have your students do high repetition and low spacing?” And that’s how we spread it out. We had about two campuses in each of these cells. That’s the true experiment on paper. To get to the other part of what you said… in reality, that’s not exactly what students always did. And you know students; we can tell them to do something but a whole bunch of things gets in the way. Fortunately, of course, we measured self reports of what students said they actually did and it was relatively close to the study cells, but even though it varied a little bit we could still control for it. So, yes, it was close to a controlled study as much as you could control something in the real world across nine campuses.

John: That brings us to the general question of how you construct controls. Suppose that you make a change in your class; how do you get the counterfactual?

Regan: Right.

John: What would be some examples for people designing an experiment?

Regan: The word control, especially in research, has the true connotation of the word control group and that’s controlling for factors as different from having a control group. Optimally we’d love a control group. The problem with the control group is that it means no treatment. So, very often a true control group means this group of students is not getting something. From a philosophical and an ethical standpoint, I don’t like the notion of one group not getting something. So, the word I like to use is comparison group. So, your question still holds, but what’s the comparison group? I think here’s where if you’re fortunate enough to teach multiple sections, well one of the sections can be the comparison group. If you’re not fortunate enough to have multiple sections, you compare the students this semester with the students the last semester when you weren’t doing that new, funky innovation. So, there are a bunch of different ways to gather the comparison group, but you’re absolutely right: having a comparison group is important. Most commonly in scholarship of teaching and learning, the comparison is the students before that intervention, so it’s a classic pre- and post- measure. I’ll give you this quiz before I’ve introduced the material, I give you an equivalent quiz after, let’s see if there are changes in learning. And that’s the most common comparison; you’re comparing them with them before but optimally again you want a different section, you want a group of students, a different semester, or so on, and so on.

John: And it’s best if you have some other controls…

Regan: Absolutely.

John: for student ability and characteristics.

Regan: You nailed one of the key—my two favorite are effort and ability. As much as possible, measure their GPA. If they’re first-year students, measure their high school ACT scores or their high school GPA and then you have to measure ability, and I think those two are probably the usual suspects for control. And again, a lot of SOTL doesn’t do that and it should.

Rebecca: I think one thing that comes up a lot for me (and maybe some others who are in disciplines maybe more similar to my own) is that the kind of research that we do is not this kind of research generally, but we’re really interested in what’s happening in our classrooms. So, for faculty who might be in the arts or some other area where we’re doing really different kinds of research, how would you recommend being able to partner or do this kind of work without that background?

Regan: And I think implicit in your question is the “Do I need to have a certain methodological tool bag?” and I remember I was at a conference once and somebody accosted me and said “Hey, is it true that you have to be a social scientist to do this work?” And the answer is no, and I wrote a pretty funky essay called “Get Foxy,” which is how social scientists can benefit from the methodologies of the humanists and vice versa. But, you’re right; you can collaborate if you need to do that kind of work, but there are a lot of questions even within your discipline… and when I think about SOTL I think about answering questions about teaching and learning with the tools of your discipline. Now, I’ll give you an example: a good friend of mine was an art and her project, or something that she wanted to dig into, was to improve student critiques in an art class. Here we have students learning how to do art (and I think it was drawing or jewelry making) and across the course of the semester everybody had to present their work and then critique each other’s work… and those critiques, they just didn’t have the teeth that she wanted them to, so she was giving them skills and how to do it. So here’s a case of how did she know whether or not the critiquing tools were increasing? Well, she came up with a simple rubric and to score them against and look at if the scores changed. Now, you may say, well, we very often in the arts and theater you don’t get skills to do that, which is true, but that’s where I think collaboration comes in and that’s why what’s really neat about scholarship of teaching and learning is very often there are class collaborations. I have a historian on my campus who wanted to change the quality of his essays and he and David Voelker changed how he was teaching and wanted to see it roll out and had students on their essays use teams in a different way. Well, he compared, and John this goes back to your point, he compared essays from before the change with essays from after the change, counted up the number of teams students had and then, Rebecca, to your point went over to my colleague in psychology and said, hey, can you tell me if this is statistically different. So, he didn’t even bother with doing the stats; he just said, “Hey look, I don’t need to do the stats.” But you can, in a click, and literally within minutes my colleague in psychology had done the stats for him. I think that’s the kind of stuff that can happen to truly get at those answers if you go, “You know, I don’t know how to do that.” But, you’d be surprised… the basic skills for SOTL can give you enough to test questions pretty well.

Rebecca: I think John and I have also found in the teaching center that it’s really exciting when faculty from different disciplines start talking about their research when they’re looking at learning because there’s things that we can learn from each other and the more that we’re talking across disciplines can be really valuable as well.

Regan: Right, and I think this is where reading the rich literature that exists in your discipline or even across disciplines on scholarship on teaching and learning really gives you the leg up, because I find now when I do workshops and somebody says, “You know, I’ve got this question; I don’t know how to start.” More often than not it’ll remind me of a study that I can say, hey, here’s what you can do. And it’s just because I read a lot and I’ve got all that in my head and I just matched to that question and it’s pretty easy. I mean, very rarely do we have to invent something from scratch. We go, “Hey, yeah, you know what? Here’s the study that’s pretty close to the question you have, let’s use that methodology.”

Rebecca: So, how do we build a culture of the scholarship of teaching and learning—the departments who might have faculty who are resistant to the idea of their colleagues spending their time doing that? How do we start changing minds and really building a culture that embraces the idea of the scholarship of teaching and learning?

Regan: Well, I think you’ve got to attack it from two different levels. You definitely want a champion in the administration who is educated enough about the scholarship of teaching and learning and how it can be done robustly. If you can convince somebody of it’s worth and then if you go “How do you do that?” …well that’s where you need to make sure you have at your fingertips, as a teaching and learning center, the exemplars of really robust work… and I think if you have that really robust work at your fingertips, that’s definitely a key place to start. One of my favorite examples along those lines of trying to convince (especially administrators) about the worth of scholarship of teaching and learning, I recommend a 2011 publication by Hutchings, Huber, and Ciccone, it’s called A Scholarship of Teaching and Learning Reconsidered and this 2011 publication is a great collection. It does your homework for you. That one book pulls together evidence for why scholarship of teaching and learning helps students, helps faculty, helps institutions. So that’s where the top down—get your administrators to check that book out and go, “Oh yeah, look, there is actually some good research.” Coming at it from the other angle—I know this for a fact—there are people on your campus doing some of that work, but often they may be isolated, they may be a small group. You want to strengthen them so that they can spread that to their circles, and that’s really how it starts. On my campus, when Scott was the Dean at Green Bay, we did a lot to develop scholarship of teaching and learning through the teaching center. There was one year where we had 14 faculty who got together every month and talked about their projects. Now you may say, well, that’s 14 and you had 160 faculty. You know what, you do 10 of working every year and colleagues see the value of the work those 10 or 14 are doing, pretty soon you’re gonna have a culture where people recognize it more and appreciate it more. So I think that’s how it goes… you put your efforts on those people who are already doing it to make them stronger and that’s gonna spill over and pretty soon you’re gonna win over folks.

John: We generally had support from the upper administration and there’s often been a lot of faculty who are new, interested in doing it; it’s usually the promotions and tenure committees that have served as a barrier in some departments, but we’ll work on that and we need to keep working on that.

Regan: Well, just along those lines on our campus we felt so strongly about the scholarship of teaching and learning that the Faculty Senate actually passed a resolution recognizing the importance of scholarship of teaching and learning. Now again, it still gave department chairs some leeway, but at least the faculty voted on it as something that the university values and that goes a really long way to having especially junior faculty say, you know, I can do this.

Rebecca: Certainly makes faculty, especially junior faculty, feel supported when the Senate is saying, “Yes, we believe in this” and it’s not just one person saying we don’t.

Regan: Absolutely. And they’ll be naysayers. We started off this conversation with “There are people out there who think it’s not good enough” and there are people out there but I’ve had conversations with such people on my campus where sharing some information, sharing things about how it’s done goes a long way towards changing minds.

John: In my department, it’s helped that I’ve been the chair of our search committee for a few decades now. We’ve generally hired people who are interested in this, but that’s not the case in all of our departments yet, but we’re hoping that’ll change. For those who have small classes or may not be interested in doing research in their own classes, one other option is meta-analysis. Could you talk a little bit about that?

Regan: So meta-analysis, where one study is taking a look at a lot of different studies, there is the mother of all meta analyses… is one that we should talk about because I think the interested person can run to it. John Hattie, now at the University of Melbourne, did a meta-analysis where actually he did a meta-meta-analysis; took 900 meta analyses and then synthesized the data from those 900 studies that had already synthesized data, and the reason I like talking about that is the sample size when you take all those 900 meta analyses is a quarter of a billion with a “b”; that’s a lot of data points, it’s a lot of students. And what’s neat about meta analyses is that instead of just being one study at one place it’s now multiple studies over multiple contexts, and if you can find an effect over multiple contexts, that’s really saying something because a lot of single studies are so geared into the local context of where that place is that if you run into a meta analysis, so even if anybody listening pulls up an educational journal or an SOTL journal and sees meta analysis in the title, I would spend more time reading that one because it’s gonna be more likely to generalize from that. So, I think it’s statistical and methodological advances now mean that there are more meta analyses around and more meta, meta analyses around as well.

Rebecca: As an advocate for the scholarship of teaching and learning, where do you hope the scholarship of teaching and learning goes in the next five years?

Regan: Honestly, I think it should be a part of every teacher’s repertoire. When I think about a model teacher, and it’s not just when I think about it—I’ve published on evidence-based college and university teaching and when my co-authors and I looked at all the evidence out there and what makes a successful university teacher… one of those components, and we found six… I mean, it wasn’t just student evaluations, no, it was your syllabi, it was your course design, but one big element was doing the scholarship of teaching and learning… and to answer your question, I think if in five years from now we can see it be part of teacher training to look at your class with that intentional systematic lens, I think that’s where the field needs to get to.

John: At the very least it would get people to start considering evidence-based teaching practices instead of just replicating whatever was done to them in graduate school.

Regan: Absolutely. People would be surprised at how much good SOTL there is out there, and I always like sending folks to the Kennesaw State Center for Teaching and Learning where they have a list of journals in SOTL in essentially every field. You will scroll through that list for ages and it is just mind-boggling to realize that, “Wow, SOTL has been going on for a very long time.” And Rebecca, you mentioned art and performance arts and theater and music… not as much, but even there there is a fair amount and I think it’s just a question of getting folks making those resources more available to individuals and that’s why whenever I interact with teaching and learning centers I have a short list of key resources to look at. And again, that’s on my SOTL link. But, even that small list is an eye-opener to most people who never knew this existed, and I think once they realize it’s there they will start seeing it everywhere and once you start doing it it really energizes you. For those of us who’ve been teaching for 20-plus years to look at our classes with that new eye of how can I change something, how can I make it better and then seeing the positive effects of those changes, that’s invigorating.

Rebecca: I’m energized after having this conversation.

Regan: It is good stuff.

Rebecca: Yeah.

Regan: I just got back from a three-day conference and all we did was sit around and talk about cool SOTL. And you’re right …came back and sitting on the plane I was texting people with study ideas to collaborate on. It was that exciting.

Rebecca: The more you talk… collaborate… the more it happens.

Regan: There you go.

Rebecca: So, we always wrap up by asking, what’s next?

Regan: You know, I think I like getting the bang for my buck and you mentioned this in the intro: right now I’m working on the American Psych Association’s Introductory Psychology Initiative and what’s next is basically two years of really focusing on the introductory psychology course. It’s taken by close to a 1.5 million students a year and I’d like to make sure we can make that course the best learning experience for our students as possible, so that’s where my energy is gonna be for the next little bit.

John: That’s a big task and a very useful one.

Rebecca: And definitely worthwhile. Well, thank you so much for spending some time with us this afternoon. it’s been eye-opening and exciting… energizing. I can’t wait to look through some of the resources.

Regan: You know, is there anything else that you’d like, get in touch and I welcome anybody listening to get in touch as well.

John: Thank you, and we’ll share links to the resources you mentioned in the show notes.

Regan: Sounds good.

[MUSIC]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen, and Dante Perez.

[MUSIC]