134. Convergent Teaching

New faculty often enter college classrooms with little training on how to best support student learning. While peer evaluations of teaching are commonly used, these evaluations are often conducted by other faculty who also have little training in the science of learning. In this episode, Aaron Pallas and Anna Neumann join us to discuss how we might build a culture in which we all continue to develop our ability to support our students’ learning. Aaron and Anna are Professors of Education at Teachers College, Columbia University. They are also the co-authors of Convergent Teaching: Tools to Spark Deeper Learning in College.

Show Notes

Transcript

John: New faculty often enter college classrooms with little training on how to best support student learning. While peer evaluations of teaching are commonly used, these evaluations are often conducted by other faculty who also have little training in the science of learning. In this episode, we discuss how we might build a culture in which we all continue to develop our ability to support our students’ learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York
at Oswego.

[MUSIC]

Fiona: My name is Fiona Coll. I teach in the Department of English and Creative Writing here at SUNY-Oswego and
this is my turn to sit in as a guest host.

John: Our guests today are Aaron Pallas and Anna Neumann. Aaron and Anna are Professors of Education at Teachers College, Columbia University. They are also the co-authors of Convergent Teaching: Tools to Spark Deeper Learning in College. Welcome.

Aaron: Thank you.

Anna: Hello.

Fiona: Today’s teas are:

Aaron: For me, it’s a ginger and turmeric herbal tea.

Anna: Mine is a peppermint tea.

Fiona: I am drinking something called peppermint bark.

Aaron: Ok.

John: And detecting a pattern, I have a peppermint and spearmint blended tea today. We’ve invited you here today to talk about Convergent Teaching. You begin this book with a story about one faculty member’s introduction to teaching in higher ed, and that experience seemed to characterize the experience of many people when they first started teaching… certainly it characterizes mine. Could you tell us a little bit about that story?

Aaron: Sure. We open the book with a fictitious vignette about a tenure-track professor, who we call Chris Felton, who obtains a tenure-track job in the political science department at Roseville University, a small private institution in the Midwest. And he’d been a TA in graduate school, but he had no formal preparation for college-level teaching, because what he learned as a doctoral student was how to study political science, not how to teach it. He wasn’t given much guidance about what to do in the classroom once he started his job. The
department didn’t discuss teaching at its monthly faculty meetings, and what his colleagues hoped that students would learn in the introductory courses, or even cumulatively as a political science major, was never addressed publicly. He had heard that there was some kind of teaching center on campus, but it seems to be directed at
faculty who have been flagged as weak teachers, and that’s not how he saw. Chris understood the key contradiction of faculty life at many institutions, and that is he was hired to teach, but his prospects for
promotion and tenure would be judged primarily on the basis of his research productivity. There were no incentives to be a great teacher, there were only incentives not to be an awful one, and although he took his teaching seriously, thinking carefully about class readings and assignments, he was mainly flying blind. He worried especially about how to translate complex political ideas, so that they’d be comprehensible to students who had never been exposed to political science as a discipline. Which topic should he start with? How could he
get the concepts and theories without turning off his students? How could he assess, in the middle of the class in real time, that students were picking things up without disrupting the flow of discussion in the class? These were all mysteries to him. And although he knew that there were journals in the field that dealt with the teaching of political science, he looked at them, didn’t really see things that seemed to be relevant to the issues that he was struggling with. Perhaps, he thought, Roseville’s mandatory teaching observation system could help. Once a year, a senior colleague came into his class to observe him teach, but he was never really sure what the colleague was looking for. There were no written guidelines for the observer that he knew of, no training that the observer had had, but Roseville’s policies required that junior faculty be observed once annually. And the senior colleague would smile and nod as she walked out of the classroom, but Chris got no formal feedback from her on the class, and when he had his annual meeting with the department chair, the observation wasn’t even mentioned. Roseville encouraged undergraduates to fill out course evaluations, as so many institutions do, and Chris’s ratings fell in the satisfactory range. So in some classes, he seemed to be doing quite well. But for the most part, the comments on these evaluations provided little insight into whether and what his students were actually learning. They talked about how much reading assigned, number of papers, things that they liked, but not much guidance about how he might improve his practice and he wondered if he should be doing something fundamentally different in the classroom, but he really didn’t know how to go forward.
So that’s the vignette that we opened the book with and we do think that it does characterize the experience of great many college faculty upon starting careers as college teachers.

Fiona: It is a very illuminating way to address this paradox, that our day-to-day work as faculty members involves so much teaching, and yet, it’s a mystery, as you say, many of us come to it without much guidance. Why do you think institutions of higher education dedicate so few resources to teaching?

Aaron: I think it’s partly a function of institutions focusing on their faculty’s mastery of subject matter and viewing that as the most important thing about teaching, teachers know their subjects without realizing that there’s a lot more to teaching than knowing your subject, that’s necessary, but not sufficient, and no one has really taken on the responsibility in investing in cultivating faculty teaching. You see this disjuncture where the graduate schools that prepare faculty aren’t doing it, and the institutions that hire them aren’t doing it either. And I think that there is a cost issue. Doing it well, as you all know, investing in helping faculty develop as teachers is not a cost-free enterprise, and most institutions, I think, don’t believe that they have the resources to devote to large-scale teaching improvement.

Anna: There’s one additional issue that’s larger than institutions, and that is that teaching is an extremely complex endeavor, so that we have books and seminars and other podcasts that speak to aspects of how to do some particular thing about teaching well, but there are very few, call it theories, of overall approaches to teaching that can be taught to junior faculty or individuals wishing to somehow get the larger lay of the land. And it’s not just about knowing, say, how to work with groups, or how to call on people equally in a classroom, or how to manage activity in the classroom. It’s a matter of being able to integrate many relevant things. And the fact that that kind of a theory has been missing has made it difficult for institutions and other organizations or people to really think about, “How do we go about improving teaching?”

Aaron: Yeah, I would say that we don’t have a common vocabulary for teaching and learning in colleges and universities. It’s difficult to measure to begin with, for lots of reasons. And why do institutions focus on other things? Well, other things are easier to count. It’s easier to to figure out the millions of dollars that an institution garners in external grants or the number of books and articles its faculty published, but figuring out what’s a good metric for good teaching and what students are learning is something that has eluded us so far, in part because we don’t have this common vocabulary.

Anna: I will say that there is a very good body of work out there on students’ learning in schools. So I don’t want to say that that doesn’t exist, it definitely does exist. The work that’s been done on teaching that’s come out of that can infer how we go about supporting learning, but we are still missing some broader strategies for putting the many different pieces of teaching together in such a way that it can be taught to faculty who want to improve their teaching.

Fiona: And one of the remarkable interventions that your book Convergent Teaching does is to start to put together that larger picture to perhaps develop something like a common vocabulary, and I was struck by how early on in that conversation, you introduced this concept you call American ambivalence about higher education,
generally speaking. Could you describe what you mean by American ambivalence?

Aaron: Sure. American ambivalence is the term that we use to represent the fact that there’s a continued high demand for post- secondary education in the U.S., but declining confidence in the American higher education system, and its ability to deliver on the American dream. Americans have long viewed education, and higher
education in particular, as the best route to getting ahead in American society and moving up, particularly for those who have faced structural barriers to their upward mobility, and we’ve also viewed education as the engine of the economy. If you look at presidential speeches about education over the past 6 years, that’s what they talk about: education is how you get ahead, it’s a route to social mobility, and it’s what we collectively depend on to keep our economy going. That kind of rhetoric puts an awful lot of weight on the education system to solve social problems, to affect things that are probably far beyond its capacity… and we have seen growing economic inequality in the U.S. over time, more and more young adults struggling to secure stable and secure
jobs, much as their parents did, and that can allow them to make a decent living and coupled with the rising costs of post secondary education, which for public institutions has coincided with reduced state and public investments in higher education, college seems riskier and riskier. It’s still clearly, on average, a good investment for almost everybody, but changes in technology, making our economy seem riskier with the gig economy and less stable pathways to work, make it harder to see a predictable path to getting ahead just via education.
So that’s the American ambivalence story and in response to that we see a variety of reform proposals coming forward, and part of the argument for our book is that the major reform proposals ignore the centrality of college teaching.

John: Could you talk a little bit about those, you focus on three approaches that are often suggested: powering it up, staying the course, and blowing it up? Could you tell us a little bit about each of those and why they don’t work so well?

Aaron: Sure. Powering it up is basically the term we use to describe proposals that place their faith in technology as the means to transform higher education with a modern internet, artificial intelligence and big
data, and these proposals emphasize vocational outcomes and preparation for work, and they sometimes do attend to learning, they think about “How can we customize a learning experience for students using adaptive machine learning techniques and a variety of other fancy high-powered tools?” You can sense perhaps from the way I
described that I’m a little skeptical, but they don’t talk about teaching. And so you can envision and some of these views are quite forward looking, the potential for big data to customize learning experiences to create the real-time records of competence that could be used as a kind of online competency set of badges or transcripts that would help match students to jobs in the labor market, but the reality is we’re not there yet.

It’s quite some distance away in the future and the proponents of this approach don’t always acknowledge that technological advances, even things like online classrooms seem to work best for those who are already advantaged. They don’t seem to work as well for those who are starting out from lower starting points, and the other thing is that it’s often argued that this is cheaper, but we also know that it’s not always cheaper. Technology has not always proven to be a more efficient way of educating students.

John: Certainly in terms of online education, that’s true, that it does seem to advantage those who come from continuing- generation households and those who have stronger preparations, but I would just say that there is at least some evidence that adaptive learning techniques can help narrow the achievement gaps between first- and continuing-generation students. There are some shortcomings with it: it doesn’t provide the same sense of community, the same sense of engagement, and those are issues that perhaps need to be addressed by not relying exclusively on those. But, I think some techniques do seem to be successful in at least reducing that gap, but they’re certainly far from being cure alls, and they’re certainly far from being perfect in these goals.

Aaron: So the other sort of major reform initiatives that we mentioned in the book, the one that was kind of the response to what we call the techno optimists we label “staying the course,” and this is basically the defenders of the liberal arts, the people who defend the idea that higher education is a site for critical thinking and the search for meaning in life, and that we should be focusing on higher education as a means for developing the person and not narrowly on preparing them for jobs. And again, it’s not a perspective that says jobs aren’t important, we all want to have good, stable, secure jobs, but it’s a move away from a narrowly instrumental view of the purpose of higher education and emphasizing the importance of liberal arts and, frankly,, learning
disciplinary knowledge and the benefits of exposure to the structure of discipline, not any particular discipline, because one can learn a lot from studying any discipline and empowered organizer’s knowledge. And then the final theme is the radical version of, what we called blowing it up. And, John, you’re an economist, you may be familiar with Bryan Caplan, who’s a controversial economist who argued that students don’t learn much of anything in college, let alone things that contribute to their workplace productivity, and so his argument is
we should reduce public investments in higher education overall, that college doesn’t really work for anybody very well, and we hear his argument, but we don’t buy it.

John: He’s a bit of an outlier in economics too, most economists argue that we’ve seen some really growing and significant returns to investment in education, and it explains a great deal of the growth in income inequality. So there’s not a lot of economists who really fully buy into his arguments.

Fiona: So if this landscape you describe is missing this vital piece of the higher education puzzle, which is teaching, tell us how you imagine putting teaching back into the picture.

Aaron: Putting teaching back into the picture begins with the recognition that good teaching does matter, and there’s actually a large body of evidence that shows that good teaching does affect students’ learning outcomes, that we’re still kind of collectively hamstrung by the fact that we don’t always agree on what should be learned in college. And we don’t always have good assessments of what students have learned in particular subject areas, or even in the realm of broader critical thinking skills, but in spite of that, we know that teaching practice is related to student learning, and we know this extensively from the literature of K-12 education, which we often use to try to transpose what we think we should be able to demonstrate and know in the world of higher education as well.

Anna: And, to stepping back from that a moment and attending also to the three strategies that Aaron laid out a few minutes ago as sort of prominent right now, especially the one on techno-optimists. There’s attention there to students’ learning to some degree, And I mentioned to you that there’s been a fair amount of research done on students’ learning, but we are actively bringing the teacher and teaching back into the picture. One of the underlying assumptions of this view is that while there is a good amount of learning that people can do on their
own, and I certainly don’t want to argue with that view, that truly making learning available to the broad public and to people who are underrepresented in American higher education, support for learning is absolutely essential. That, in our view, typically comes from human beings and those human beings are those who support learning and supporting learning is what we mean by teaching. And so, as simple as this may sound, we’re basically saying that teaching matters and that finding ways to bring together some of the key things that we know about teaching into an overarching theory of good teaching is where we’re trying to take this.

John: And the way you take that, I think, is by focusing on three aspects of teaching, targeting, surfacing and navigating. Could you talk a little bit about each of those? It seems like a pretty reasonable way of classifying things.

Anna: This is not something we’ve pulled out of the sky, nor is it something that we necessarily drew purely from our own practice. I’ve been working with instructors through the larger New York metro area and in a number of other places as well, trying to help them in improving their teaching… sitting in on classrooms. We’ve also
reviewed the literature on human learning and ways to facilitate that, and one of the things that we realize is that there, and this goes back to some basic research on learning, and that some critical elements of how people learn includes there’s got to be a learner, there’s got to be something that you learn, there’s got to be subject matter, a teacher or someone who supports learning is in the picture, and we need to take into consideration the larger context or milieu in which learning happens, because that can shape any number of things. Now, given that, the thing that often falls out of conversations about learning and support for learning, namely teaching, is, frankly, the question of what’s learned, what is taught and with little systematic attention to which ideas we teach, and why, with regard to what it is that students need to learn. One of the first things that I noticed is that the expert instructors who really seem to be getting through to their students are giving a whole lot of attention to identifying what we call core concepts in the discipline. In other words, are there particular ideas or ways of thinking that, in essence, you could say either they’re a building block of the field or they model the field, they model thinking in the field. So for example, we could argue that thinking in economics is quite distinctive and that it differs in systematic and interesting ways, say from thinking in English literature, or in sociology for that matter or chemistry. When a student is in a chemistry class or a sociology class or an English Lit class or economics class, the question of deciding where
to begin the teaching can begin with an instructor figuring out what some core ideas are that that instructor can target, and then teaching those ideas very deeply and carefully in many different ways, sometimes more than once, using different texts, using different exercises, using different assignments and then building out from
those core ideas, either to other topics that resonate with what we saw on the core idea or that argue with them. And so targeting refers to the effort by an instructor to literally target, to identify those core ideas.
There’s a second activity here and that we refer to as surfacing, this moves attention from the subject matter to the students. We all know that students do not walk into a class empty headed, we cannot assume that we are laying subject matter on them. They come in knowing something, they come in with knowledge from their homes, from their cultures, from their lives. And some of those things that they bring from their lives can serve as a starting point for learning the ideas that are in the text that we want to teach, the core ideas that we want to
teach, and if we manage to identify those aspects of their lives that can be used as entry points or doorways into teaching of those core ideas, then we have a foothold, we have someplace to start with them. That’s what we mean by surfacing, surfacing in students that prior knowledge that can be used as a stepping stone into the learning of new ideas. There’s more I could say about that because prior knowledge can also get in the way, but that’s for another podcast.

John: But even then, just to briefly address that, when you’re trying to connect to prior knowledge, sometimes those are barriers that have to be
onfronted and perhaps students need to address those things in order to make progress. So, I think it’s good to point out that you’re teaching students who come in, as you said, with pre-existing views of how the world works. Some of those are good building blocks, some of them need to be knocked down a little bit, sometimes gently, but you need to get a solid foundation to build up.

Anna: That’s a very challenging thing to do, because you don’t want to be disrespectful, and often a good part of learning is figuring out how to hold on to your personal life, your personal values, personal meanings, while somehow getting a grip on a different way to think about things.

Aaron: And it can be threatening, scary, anxiety provoking, our term convergent teaching has two different meanings. One is the convergence of subject matter, learner and context (the milieu in which the learning takes place), but the other is the joint attention to cognition, emotion, and identity that teachers have to think
about as attributes of individual learners. Because sometimes you’re asking students to confront ideas that are scary, that are threatening, and that is a challenge, but it is part of moving them from what they already think they know to where you want them to get to.

John: In our increasingly polarized world, that’s particularly an issue, although it always has been to some extent.

Anna: And that, by the way, begins to speak to navigating… the third activity, which in essence is sort of steering between those core subject matter ideas and students’ prior knowledge and on the one hand, bringing in appropriate images, representations, text, videos, lecture, whatever you do that you do well in class… activities, assignments toward helping students come face-to-face with those particular ideas… steering before that and getting a handle on what students are bringing and figuring out how to put these pieces together.

Fiona: I’d like to ask you to maybe give an example or two of something that might be involved in targeting, surfacing or navigating, but I wonder if I might begin by asking how explicit you might be with students about these approaches? In the example that Aaron gives from his own teaching, there’s a moment where you describe breaking the fourth wall and talking to students about the very nature of what you’re doing as a teacher. How important is that metacognitive piece for teaching?

Aaron: That’s a great question. I think that the example that I gave about targeting did originate in an actual class where I was teaching introductory statistics and figured out that the middle of a distribution was a fundamental concept that was going to be a building block for all sorts of things in basic descriptive and inferential statistics and learned the hard way that the kind of rote way of teaching it, students could
memorize formulas and produce correct answers but didn’t really understand the middle or why a particular statistic was a good representation of the middle. I think it’s helpful for students to know what’s coming. You can think of it, a little bit, as scaffolding of students as they move from what they already know to where you want to get them to.

Anna: It’s a complicated question that I have thought about several times in my own teaching. So speaking from that, to be honest, I think it’s a judgment call, and this is something I want to keep an eye on as I keep doing this work. I’m nowhere near finishing it, and that is that I don’t often give them an abstract lay of the land, largely because I fear that they will put it in a bottle aside from real life. I want them to deal with real life. I want them to know that they know something that’s of value in the class. And so we work at a very basic level and then I start moving them up to ideas, up to concepts. Later on in the semester when I’m done that kind of thing a couple of times and one or two students catches on and says it in class, which they catch me at
times, then at that point, I may stop and explain, but as a teacher it is what I am doing. So that’s an example, but I do purposefully articulate what it is that I’m up to. Now, I like to think of myself as teaching largely inductively, but there are times that I just sort of have to stop and explain what the heck it is I’m doing.

Fiona: Could you tell us a little bit about some effective strategies for surfacing?

Anna: That’s a good question, the most effective strategy that I have found, there are a couple of things that I’ve seen done. One is that at the beginning of the course, I simply write out a list of questions that I view as some of the core ideas in my class and have people write initial responses, and then we have an hour-long conversation about them without my interrupting. And sometimes at the end of the semester, I’ll return those sheets to them, and they’ll remember what they said in that first class. But some of the best way, that becomes a place for me to go about, you know, fishing for prior knowledge is frankly, class discussions. And I never, you know, just sort of open it up to anything, I typically have thought through the questions that I’m going to
use to open discussion with, and to move it along with, but then I spend a lot of time listening, and to the extent that I can, when I hear a student say something that might be usable, I might ask them to say more, I’ll put them in groups and then I’ll go listen to that little group where the student is sitting and that begins to give me a little bit of a handle on how they’re thinking about this thing. There are times when they will offer, say prior knowledge from pop culture that I’ve never heard of. I don’t live in that world, I acknowledge that, I recall one particular instance where I can’t even recall what we were talking about, but the student said, “Oh, that’s like a meme.” My response was, “Oh, what the heck is a meme? What do you mean by that?” And they had to teach that to me, but that then became a stepping off point for the act of teaching whatever I was doing at the time, which I’m not recalling.

Aaron: I do something similar, I think, often starting a class asking students what do you think you already know about x, where x is a concept that I know I’m going to be invested in developing through the course of the semester just to get an idea about initial beliefs and assumptions. We also are cognizant of the fact that in many institutional settings, there’s a lot of social distance between faculty and their students, students are often from different backgrounds than the faculty and have had very different life experiences. And of course,
it’s always a risk to generalize, you can learn a lot by taking a student out to lunch or coffee and just hearing them talk about their experiences in their lives in ways that may surprisingly be able to connect with you to their beliefs about disciplinary knowledge.

John: One other thing that you talk about in your book is the issue of addressing the needs of the rising proportion of adjunct faculty in college. With a focus on teaching, it’s important to develop their skills, but often these are people who come in at night who teach one or two courses at each institution and they may be teaching at three or four, sometimes even five institutions. What can we do to help improve their work and to help them be more productive in some very challenging circumstances?

Aaron: They are challenging circumstances, we know that contingent faculty adjuncts are second- class citizens on most institutional campuses. They don’t have access to some of the same basic resources that regular full-time faculty do, even things like office space to be able to meet with students for office hours or access to computers or phones or things like that. And they are often on the road, which makes it hard to even envision how do you get them to a department meeting of the regular faculty because they may be teaching at another campus at that time of day. I think the big issue is we have to recognize that increasingly, institutions are relying on contingent faculty to teach more and more courses and more and more students and that’s true across all institutional types, and institutions need to get serious about investing in developing adjunct faculty. There are campuses where campus centers for teaching and learning, which we are big fans of, recognize are often under budgeted are simply unable to offer the kinds of services that they make available to regular faculty to adjuncts, and we have a rough back of the envelope calculation that suggests that in order to give contingent faculty the same level of access or participation as full-time tenure-track faculty, you might have to double the campus’s budget for teaching improvement. And most campuses don’t have slack piles of money lying around, but the reality is if you’re concerned about teaching and you know that adjuncts increasingly are bearing the burden, the idea that tenure-track faculty can benefit from supervision, support, observation, coaching, but contingent faculty can’t, that’s ridiculous.

John: What types of support can campuses provide to help improve the quality of teaching? You mentioned teaching centers, we’re a fan of those too, or at least I am [LAUGHTER], but there’s a lot more that campuses perhaps could do and teaching centers often have very limited staffing and budget. So what are some approaches that could be used to provide more support for faculty?

Aaron: I think the biggest is a realignment of faculty reward systems that recognize the importance of good teaching, and limit reliance on student evaluations of teaching as evidence of good teaching. Virtually every institution has student ratings of instruction, that’s not going away. But far too many institutions rely on that as the primary metric for judging whether someone is a good teacher… used in high stakes situations for contingent faculty and tenure-track faculty alike, and the thing about student ratings is they provide very little guidance as to how to get better, even well designed forms, and there’s clearly a variation in the quality of what actually gets asked on a student form. They’re used for summative purposes, rather than for helping faculty actually do better in the class. So, I think we need to think about other ways of doing that. We’re big fans of peer observation, of ways of having faculty benefit from having peers observe them in the classroom and engage in structured observations followed by conversations about “What happened here? Why did you do this? Do you think it worked? What might have happened if you had done it differently?” as one part of a strategy coupled with other things that most of which boil down to making teaching a more public and accountable activity, but accountable to peers, accountable to other faculty who have expert knowledge about subject matter and about pedagogy.

Anna: You know, the only thing that I would add to that is that while I am a great fan of evaluations and self evaluations or peer evaluations, sometimes whether it’s evaluation or assessment it’s confused with
instructional development. They’re not the same thing. Just because you’re assessing something doesn’t mean that that’s a straight route into improving teaching. Just because you assess something doesn’t mean things will get better. There is real activity that’s involved in making teaching better and backing up from that, you need to
understand what it is, what it would look like.

Aaron: Teaching improvement requires sustained activity. In the K-12 world there are kind of derogatory references to one-off professional development sessions where you’ve got some speaker in front of an auditorium of 300 teachers for two hours, and the belief that that’s going to somehow transform practice, it never happens. You have to think about teaching improvement as a longitudinal process that involves sustained effort over a good period of time, sustained engagement with others who can help you think about practice and obviously, some teaching centers have strategies for doing that. I know Oswego, you’ve got this teaching squares initiative, which in many respects, parallels our ideas about peer observation, bringing faculty into conversation about concrete examples of classroom practice that they have shared together from observation. There are a few, I think, initiatives that are becoming more common. I think Oswego has a cohort for ACUE, the Association of College and University Educators, which is a structured program that allows a cohort of faculty to engage with one another around a range of teaching practices. And again, it’s not something that happens within two hours. It’s sustained. I think, John, you’re running a bunch of these through the course of the semester or a year. And that’s another approach that recognizes the benefit of engaging with peers and sustained engagement with
teaching practice concern.

John: The ACUE program has been really successful on our campus, we’ve had a cohort of, I believe, close to 30 people go through it last year, and we’ve got our second cohort now of, I believe, 28 people moving through it, and it’s a 25-week program, and it does provide that type of sustained involvement or engagement. And also Anna mentioned how she often will try to surface things at the beginning of an activity. One of the nice things about the ACUE modules is they all start by having faculty address their preconceptions about the topic before actually addressing them and providing research and examples. There’s a lot to be said for that approach and people participating in this program automatically picked it up just by observation. You
also mentioned the Carl Wieman Science Education Initiative as providing a nice set of tools. Could you talk just a little bit about that?

Aaron: Carl Wieman is a world renowned physicist who has taught at University of Colorado at Boulder and the University of British Columbia. And he developed an initiative at those two campuses that involved trying to create sustained improvement in science education by offering a competitive grants program to the science departments in both schools that would allow them to hire a science education specialist, basically postdoctoral fellows with disciplinary knowledge whose job was to help develop courses, to improve often introductory courses in the sciences on each campus, and they were sizable awards, department might get a million dollars over five years to help do this. What he discovered is that he was underestimating the importance of incentives for individual faculty, that the unit of change he thought, eventually, was really the faculty member more so than the course… in part because courses, they’re owned by departments more so than by faculty, they’re a little bit harder to move. And he came to the view that you could start with those faculty who seemingly were interested in changing their practice, figure out ways to provide them with incentives to work on their courses in the form of compensation, of course releases, supplemental research assistance and the like, as a mechanism for getting them to work at their teaching. The initiative was about $15 million across the two campuses. It’s hard to sustain, he found that it wasn’t well aligned with the campus’ culturally inscribed reward structures for faculty, it was still the case that research productivity dominated how faculty were evaluated, so it wasn’t
well coordinated with that. And it benefited when there were senior administrators at the Provost level, or similar levels, who were vocal and public about their commitments to trying to improve teaching. So it’s a kind of cautionary tale and I think that he can point to the fact that thousands of students were exposed to courses that had benefited from this kind of effort, but it wasn’t clear that it was going to be sustainable with a large concentration of faculty once the money ran out.

John: I should note that we also had a discussion with Doug McKee in one of our earlier podcasts, we talked about an implementation of that program at Cornell. The title of that was “The Cornell Active Learning Initiative,” which basically built upon that, in fact, I believe that Doug and some other people from Cornell went to a workshop that Carl Wieman offered, and that’s still underway, it’s still under development. I haven’t heard as much about the results, but that’s something we should probably check back with Doug about at some point.

Aaron: I think that Wieman, by virtue of his stature, was able to find the money, to get institutions to support these big initiatives. You don’t see institutions investing in teaching improvement to the tune of five to ten million in just the sciences, and there’s no reason why we couldn’t imagine that in other fields of study as well, finding the money and finding the institutional support is a big issue.

Fiona: One of the downsides of the incredible richness of this book is that it’s hard to package it all in a very short conversation like this. Is there anything we haven’t mentioned that you’d really like to talk about?

Aaron: I think perhaps it’s recognizing that, for lack of a better term, it takes a village… that teaching improvement is a shared responsibility of the institutions that prepare future faculty, of the institutions that
hire future faculty, of disciplinary associations, of the federal government and its ability to generate the sources for the studying of teaching and learning in colleges and universities, philanthropic organizations, and individual faculty as well. No one group can do this. We do offer at the end of the books some guidance for what individual faculty themselves can do, starting with reading the book, of course… [LAUGHTER] but also in the context of their campuses, the kinds of likely local supports that they can seek out in their immediate
surrounds, which can be the disciplinary associations with which they’re affiliated, or a campus teaching center, or colleagues who they just discover are passionate about teaching. All those are things that we think individual faculty can leverage, but no one can do it alone.

Anna: And I think I’ll add one thing that’s really more an idea than anything else. And that is that I spent a number of years studying learning. And one of the things that my students and I sometimes struggle with is the idea that one never learns fully alone. Even if you’re in a quiet room with the book in front of you. You’re there with the author. You’re there with the author’s thinking, the author’s thinking comes to you in whatever way. But, the point here being that teaching is a way of bringing another person into an individual’s learning
and that that individual, if that individual has thought a lot about how people learn, and how to support people’s learning, that that learning can be extended and deepened in a number of rich ways. So, I think of
teaching as part of a larger learning experience. And I guess that’s where I would want to end that.

Fiona: Thank you for that reminder that unfortunately, necessary reminder that we’re talking about people in all of this.

Anna: Yes.

John: We always end our podcast with the question: What’s next?

Anna: I want to continue to study teaching and learning. Teaching is the advancement of students’ learning. I’ve done this for a number of years in colleges, largely those that serve underrepresented learners, undergraduate institutions, and I will always be interested in those institutions. More recently, I’ve thought about improving teaching in law schools that also serve underrepresented populations. There are a number of those and I have been in contact with a number of law school faculty who are very eager to improve their teaching. And that strikes me as an important route into the future. The fact of the matter is that we want to bring new populations into undergraduate education, but we want them to go on as well. And that will involve improving
teaching and diverse sites.

Aaron: For me, most of my work is on K-12 schooling. And throughout the book, we draw on the literature on K-12 teaching and learning and organization to inform what we think might happen in higher ed. And so I’m often anticipating what’s going to move from the K-12 world to the higher ed world. Sometimes that’s a little dystopian. And the current sort of dystopia that I’m working on is how K-12 classroom teachers are evaluated, of the kinds of accountability mechanisms that exist in New York State and New York City that they are subject to and how they experience these accountability systems, and with what consequences for their orientations towards improving their practice. My suspicion is that, as is often the case, if it’s happening in the K-12 world, the higher ed world may not be that far behind.

Fiona: They sound like incredibly important directions to go.

John: …and as one form of convergent teaching: convergence between K-12 and college teaching.

Aaron: Yeah.

John: Well, thank you very much for joining us. I very much enjoyed reading your book, and I’m happy we’re able to share this with our listeners.

Aaron: Well, thank you so much.

Anna: Thank you very much. We’ve enjoyed talking to you.

Fiona: It’s been an absolute privilege to speak with you. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Savannah Norton.

[MUSIC]

133. Signature Pedagogies

Many disciplines have well-developed signature pedagogies that are designed to help students develop the skills needed to view the world from their disciplinary lens. In this episode, Regan Gurung, Nancy Chick, and Aeron Haynie join us to discuss signature pedagogies and to examine how the COVID-19 pandemic has challenged us to adapt our teaching approaches and encouraged faculty to seek out and share pedagogical advice as we attempt to provide enriching learning experiences for our students.

Regan is a Professor of Psychological Sciences at Oregon State University, Nancy is the Director of the Endeavour Foundation Center for Faculty Development at Rollins College, and Aeron is the Executive Director of the Center for Teaching and Learning at the University of New Mexico.

Show Notes

  • Gurung, R. A., Chick, N. L., & Haynie, A. (2009). Exploring Signature Pedagogies: Approaches to Teaching Disciplinary Habits of Mind. Stylus Publishing, LLC.
  • Regan Gurung (2018). 54. SOTL. Tea for Teaching Podcast, November 7th.
  • Schulman, L. S. (2005). Signature Pedagogies in the Professions. Daedalus, 134 (3), 52-59.
  • Angela Bauer, Professor and Chair of Biology at High Point University
  • Catherine Denial, Bright Professor and Chair of History at Knox College
  • Punch Through Pandemic With Psychological Science – Course description at Oregon State
  • Huston, T. A., & DiPietro, M. (2007). 13: In the Eye of the Storm: Students’ Perceptions of Helpful Faculty Actions Following a Collective Tragedy. To improve the academy, 25(1), 207-224.
  • Wiggins, G., & McTighe, J. (2006). Understanding by Design (Expanded 2nd edition). US: Pearson, 2005, 16.

Transcript

John: Many disciplines have well-developed signature pedagogies that are designed to help students develop the skills needed to view the world from their disciplinary lens. In this episode, we examine how the COVID-19 pandemic has challenged us to adapt our teaching approaches and encouraged faculty to seek out and share pedagogical advice as we attempt to provide enriching learning experiences for our students.

We should note that this podcast was recorded shortly after our campuses shut down in mid-March, but the discussion today remains as relevant as it was at that time.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guests today are:

Regan: Regan Gurung,

Nancy: Nancy Chick,

Aeron: and Aeron Haynie.

John: Regan is a Professor of Psychological Sciences at Oregon State University and had been a guest on an earlier podcast. Nancy is the Director of the Endeavour Foundation Center for Faculty Development at Rollins College. And Aeron is the Executive Director of the Center for Teaching and Learning at the University of New Mexico. Welcome, everyone.

Regan: Thank you, John.

Aeron: Welcome.

Nancy: Thanks.

Rebecca: Today’s teas are:

Regan: I’m drinking some Darjeeling tea grown on the family estates on the foothills of Darjeeling.

Aeron: And I’ve just been enjoying some nice loose Earl Grey tea from the St. James Tea Room in Albuquerque.

Nancy: And since I’m in Florida where it’s 93 degrees outside, I’m drinking some strawberry fizzy water.

John: My tea today is Irish breakfast tea.

Rebecca: With your lack of selection because it’s all locked up. [LAUGHTER] Mine is blackcurrant tea today.

John: Regan, Aeron, and Nancy are the co-authors of Exploring Signature Pedagogies: Approaches to Teaching Disciplinary Habits of Mind and a follow-up volume Exploring More Signature Pedagogies. We’ve invited you all here today to talk a little bit about signature pedagogies and how that might relate to the situation we’re experiencing today, where faculty have suddenly, with very little notice, moved to remote teaching in the U.S. and for much of the rest of the world. Could one of you first define what is meant by a signature pedagogy?

Regan: We’ll let Nancy take this as this was her idea that got us all started.

Nancy: Okay, signature pedagogies were originally defined by Lee Shulman in 2004 when he had culminated some of his research on the professions and learned about how professors in those professions taught in ways that captured the ways of knowing, doing, thinking, and valuing of those professions. So the examples that he often gives… in law, law is typically taught with the very Socratic questioning, the spitfire Q&A, where the students need to recall details from cases on the spot, which very much resembles the courtroom; and in medicine, you have the rounds where the group of students and the doctor move around to a patient and diagnose collaboratively based on what they find in a very quick report from the patient, and that is how medicine works. And so Shulman ended his 2004 keynote at The International Society for the Scholarship of Teaching and Learning with a challenge to the academic disciplines. What are your signature pedagogies? What are the ways that you do or can teach in ways that embody the ways of knowing, doing, thinking, and valuing of your disciplines?

Aeron: In putting together this volume that we co-edited, one of the questions that came up from many of our authors was, “Are we describing the way that our discipline generally teaches, which we can think of that as a default or a traditional way? And how is that different than a signature pedagogy?” And I remember many of those conversations, and the real distinction is that the default pedagogy isn’t something that’s necessarily been examined as really helping promote ways of thinking as a practitioner, and so I think that’s an important distinction to make, too.

Regan: I think just building on that point, I remember a conversation with Angie Bauer, where she talked about how biology does it a certain way, but there are better ways to do biology signature pedagogy, and I think that was the very neat thing about their chapter, where they said, even though biology does it this way, there’s a better way to do biology.

Nancy: And I know that one thing that really triggered her and the other authors, and I don’t remember where this came from, maybe it came from Shulman, is the question of “What does it mean to think like a biologist?” And that question seems to open up a world of teaching when it comes to thinking about signature pedagogies and I think that’s really what we’re talking about.

John: So signature pedagogy, basically, is an ideal way in which people are training to become participants in the discipline, but not all disciplines have developed a very good alignment between what they’re trying to achieve in terms of student outcomes and the way in which they actually practice it, and that shows up in a number of the chapters. In fact, the chapter on economics I paid a little more attention to because it was pretty clear there that economics, at that time at least, did not have… and it still doesn’t, to a large extent… have a very well defined signature pedagogy, that there’s not always a very close alignment between how people teach and what types of skills they’d like to develop. If the purpose of a signature pedagogy is to help people understand the world through the lens of the discipline, i s this something that faculty generally make transparent to their students?

Aeron: I think no, in many cases, no. And I think that partly, that’s because as an expert, it’s so obvious and natural to us to look at the world as a historian or as a sociologist, or a biologist; that we don’t realize we’re doing it, and that’s one of the things that I think can be really lovely drawing on our experience and editing all of the different chapters is to realize that we actually do have a signature pedagogy, that we do have a disciplinary way of looking at the world, and as a faculty developer, I try very much to get instructors to think about articulating that in a way that makes sense and that’s coherent to undergraduate students, and I think this is particularly important with the general education core courses. The students in a gen ed course aren’t really going to be interested in learning a bunch of content just for the sake of providing a foundation that they can do cool stuff with, meaningful stuff with, later because that might be the only course they’re ever going to take in sociology or history or biology, so it’s so important to give them a more authentic and meaningful experience of seeing the world through that disciplinary lens. So I think this is important work to think about why your discipline matters. So right now, in this moment, if a student is struggling with being able to keep their attention span and prioritize your class over all of the other worries that they have, and child care and all of those things, why should they care about your class? And I think that we always need to articulate that. Why does history matter? Why does biology matter? We don’t always do a good job of explaining that, but it should always be something that we address. “Why should you care about my class? Why do I think it matters?” And say that in a way that makes sense to students.

John: How does this relate to the situation we’re facing now in terms of different disciplines’ approach to how they try to train their students, when suddenly they move from the modalities they’re used to into one that in some cases, they very rarely have experienced?

Aeron: I think we’re probably all seeing, as faculty developers, which is what all three of us are doing in many ways. At this point, what we’re seeing is that different departments have different anxieties, different specific anxieties about how to transfer, sometimes their default pedagogy, and sometimes we can say a signature pedagogy, but a specific way that they believe learning needs to be enacted or has often been enacted in the classroom. For example, we have a lot of science faculty saying, “Okay, so how do we do our science labs online?” or I’ve also had conversations with folks in art studio, “So how do we do metals? How do we do printmaking remotely?” and foreign languages as well. So on one hand, I think that really shows that different disciplines are impacted in different ways in terms of thinking about “how to,” and this is what’s so extraordinary about this moment, really kind of immediately, without much planning or forethought, just pick their courses up in the middle of the semester and pivot them to online. So that’s quite different than I think what Nancy is going to lead us to talk about in terms of stages two or three of this experiment, which is what would be a more reflective, thoughtful way, or evidence-based way to create a signature pedagogy online.

Nancy: And I build on that… You mentioned studio artists, and I’ve been having some really fascinating conversations with some of the artists here and they’re talking, like you said, about “How do we do printmaking or metallurgy,” or whatever but they’re also talking about “How do I do critique with a static desktop and my students are working on their art in their living rooms.” And so people are not talking about glossy and fancy technology, we’re talking about “Take your phone camera,” and the students take their phone camera and walk around and show their sculpture or their watercolor. So it’s this real foregrounding of the pedagogy even more than the technology, because I think when we talk about developing online courses, traditionally, we talk about okay, everyone is assumed to have a really nice computer with an LMS, and we focus on the LMS. But now, like Aeron said, it’s foregrounding the key pedagogies in these different departments.

Regan: I think that’s where the problem comes in, in some ways, when we talk about how well are faculty taught to train to teach in the first place. Because, interestingly enough, even before the pandemic, if we think pre-pandemic, there were many faculty in many disciplines who were not teaching their students the habits of mind of their profession. So in two volumes of multiple chapters, every author in those chapters are people who’ve taught about teaching, who’ve been reflective about that teaching, who’ve trained themselves to teach, and I think now when each of us look out at our respective campuses as directors of centers for teaching and learning, you see the vast number of individuals who aren’t really even teaching according to the signature pedagogies of their discipline, and that was pre-pandemic. Then you add the pandemic, and you build in all those factors about technology and remote teaching and things like that. So in many ways, this is a great wake up call for so many to say, “Do I even have the fundamentals of teaching down? Let me build on those fundamentals.” Because when it comes down to it, it’s engagement, right? One of the big questions that I see coming up is “How do I engage my students online?” And I think for all of us who’ve taught online before, we have a great advantage, there are a number of faculty who have never taught online and it’s a whole new way of thinking. So I think thinking about signature pedagogy is almost a luxury. I hope we can get there. Let’s get everybody going. When Nancy talked about different stages, today is day one of spring term at Oregon State, and so the last week was crazy. We have 1,300 faculty and 3,000 plus classes that had to move from face-to-face to online but all of last week, I can tell you, we weren’t fielding pedagogical questions, we were getting “How do I use Zoom? How do I use the LMS?” I think those pedagogical questions I’m looking forward to starting next week, not even this week.

Aeron: Yeah. And I want to add to that, I don’t know what day we’re on. This is the world’s longest month. [LAUGHTER] But we’ve been teaching, supposedly, pivoted to online for maybe I guess a week, officially. And I will say that last week, some of the most interesting conversations, and again, we did it primarily department by department. Some of our most interesting conversations were with faculty who were either able or forced to take that big view and just say, “What’s the most important thing? What do I really need students to experience or engage in through this semester, when the semester is over?” And actually some of the art studio faculty… I want to give a shout out to here at University of New Mexico, they’re extraordinary… they really had a very human and humane response, which goes to Regan’s point about engagement and connection and all of the evidence about belonging and they were really concerned with their students on the most human level. “How can I stay connected to my students? How are my students doing both medically and emotionally?” And they kept asking questions, “I’m worried about our graduate TAs, I’m worried about our graduate students.” So I think there have already been, here and there, some productive conversations about “Okay, we can’t continue the plan that we began when we originally planned this Spring 2020 semester. So if we’re going to scrap it, what’s most important?” And I want to give a shout out to Professor, and I don’t know how to pronounce her name, it’s Cate Denial in Knox College in history. This is on the Twitter, she shared that she had just changed her semester, and instead of the planned lessons in history, she gave them all notebooks and nice pens and said, “Record what’s happening to your individual lives right now and then we’re going to store these in the Knox College, I believe, library because your reflections are going to be part of an historical artifact.” And that is a way for us maybe to think about how signature pedagogies could eventually really revitalize these conversations. What does it mean to think like a historian? It means to think about that this will someday be history, and how do we decide what this was like? And how can students if all they remember from this semester is, “Oh, I’m actually part of history and my thoughts and my everyday experience might be interesting for folks, 20,50,100 years from now, that’s a really important thing, and it kind of a little bit segues into this conversation about the signature pedagogies in courses for majors versus gen ed students who aren’t going to be majors.

Rebecca: I think what’s really interesting is the idea of thinking about what it looks like to be an expert in a different field and how they’re going to perceive this experience in helping students process their experience through that lens, whatever that lens might be, and you’ve highlighted a couple of those examples could be really powerful. It also is one of those opportunities that we can do a multidisciplinary approach to studying something specific, which I think is really exciting.

Regan: I think what’s interesting here and the way you mentioned the historian taking history, I didn’t think about what we’re doing in this way, but we at Oregon State created a brand new class for coping with the pandemic and it’s called Punch through the Pandemic using Psychological Science. And in the lens of signature pedagogy is… talk about meta-metacognition, right? We’re psychologists offering a course on coping with the pandemic using psychological science. So there are all these different levels there going on and I bet you’ll see more of that going on as different disciplines take their lenses towards dealing with what’s going on. You know, John, you mentioned Econ, I bet all the economic stuff going on here and public health, and what a great opportunity to make learning real for our students, even more real than it has been.

Nancy: We’ve also seen this happening with literature and the art. I think of all of the examples on social media of people writing poetry, or sharing poetry, or sharing powerful photographs or works of art. Just how people are using the arts and humanities right now. As Regan said, to cope with what’s happening, we’ve been having these conversations for so long about the death of the humanities, and we are certainly seeing that the arts and humanities are far from dead. So I think they’re right about how this moment is really revitalizing a conversation about the role and the importance of all the disciplines and how they are all contributing to understanding and surviving and thriving soon, hopefully, in this moment.

Aeron: Absolutely, Nancy, and I wanted to give a shout out. A friend of mine has a daughter who’s just been accepted to Oberlin College, and as an admitted student, she got an email inviting her to be part of a two-credit interdisciplinary course that looks at economics and writing and sociology and biology and math, I think, and maybe others examining the virus and if the students who are admitted elect to take this course it would count for credit. First of all, I’m so in awe of them being able to get this faculty to develop something so rich, so quickly. Being at a large state university myself, I can’t quite picture how we would do that. But what this would do, I think, is very much as Nancy was saying, this would allow a freshman student to see, “Okay, here’s this big event that’s happened that’s impacting my life in all these ways. How does looking at the world with the lens of a sociologist, how does that help me start to answer this question of what’s happening? How does art and literature help me understand this question? How does history help me understand this current moment?” My daughter’s only in high school, but boy, I wish that she was able to take a course like that right now because what’s happening instead, and her school is lovely and her teachers are wonderful, but what at least started happening for her online schooling as a sophomore in high school, they were continuing the lessons as they had planned them and there’s such a disconnect between her lived experience and now being online and just having to do work in these separate, disparate disciplines that really aren’t connected to each other and aren’t connected to this important historical moment. And even though the virus has made this more intense, isn’t that what happens anyway? …that students go in and they take a bunch of courses that are not connected to each other, and they’re not connected to the lived realities of our students’ lives in the historical moment. So it’s making it more pointed, but I think that this is a critique we can make of higher ed and K-12 education in general.

Nancy: Just to build on that, I feel like we need to throw into the mix… some years ago, Therese Huston and Michele DiPietro did some research on how students reacted and what they needed, basically, from their professors after 9/11, after Hurricane Katrina, after some of the early school shootings, and among a range of ways that professors reacted, ultimately what these students wanted was for their professor to do something, to do something that, like Aeron says, connects whatever it is that’s happening to their lived reality. It can be small, it can be large, but I think now we’re not just talking about a moment of silence. I think what we’re seeing is an opportunity, like Aeron says, to use this moment to more fully integrate everything we know about learning across the disciplines. So I think this is a real moment to reach the lived experiences of students’ lives in the way that our disciplines are being enacted at this very moment. So it’s this fascinating kind of alignment of the stars for some really rich learning once we’re all ready to get to that stage, I think.

John: We threw out the plan for my class tonight. We’re going to be focusing on the economics of recovering from a pandemic. But one of the things I’m hearing is this notion that this is a great opportunity to think more deeply about our disciplines and about how we train our students. Instructors tend to teach in very much the same way that they have always seen and they tend not to change. There’s a lot of inertia in how we approach life, more generally. But there was a suggestion that everyone’s been getting instructions basically, to focus on “What are the most important learning outcomes that you want your students to have by the end of the class?” and “What’s the most efficient way of getting them there?” And this is forcing people to rethink everything about their teaching, and might this be a good opportunity to develop the signature pedagogy of their disciplines?

Nancy: You know, that Jay McTighe and Grant Wiggins are alive and well right now and very excited because this is truly a moment of uncoverage. Aeron was the one earlier who talked about how people are really thinking, “What’s most important, what do I want my students to remember?” So we’re talking right now about everyone is going through this process of uncoverage, getting rid of that coverage model and really focusing on what’s important.

Aeron: Yeah, and as much as I agree with Nancy, and I love how you’ve been, John, sort of pivoting your course, I also want to say that I’m nervous that it may not go as optimistically this semester and, as we can all imagine, that there’s opportunities, but I’m also worried that what we’re really going to find out is that a lot of faculty find this process so frustrating. And we Regan said at the beginning of our conversation, that a lot of initial comments are about the technologies and not the pedagogy. I myself had a problem going from Zoom to a Zoom Pro account, and I got frustrated. I’m the Executive Director of the Center for Teaching and Learning and I thought, “My goodness, if I’m frustrated for a few minutes, what are the rest of the faculty experiencing?” So, this is not the ideal way for this to happen, aside from loss of life and all of that, but just pedagogically and institutionally, it’s just not the best way for this to happen, so it is an opportunity. I don’t know what we’re going to see. I’m thinking myself, and my excellent staff are spending a lot of time thinking, “How can we best support faculty in leading them into these larger, richer conversations, and away from just conversations where they’re focusing on the mechanisms of teaching?”

Regan: I think it’s also, when you think about the conversations, one of the neat things that I keep trying to remind people of when I talk to them about the remote switch is when you go online and when you are relying on Zoom, but more importantly, you’re relying more on your LMS. Now the opportunities to essentially have one-on-one conversations increases dramatically, and I think what’s going to happen, that we haven’t started talking enough about yet, is what if, in two weeks from now or three weeks from now, faculty are sick and students are sick. I think there are many disciplines where we focus so much on the dynamics of the course that we don’t think about “How is the student actually taking this?” and what’s going on in their lives that could influence how they experience the course, and I think this is the time that that realization and openness is more important than ever. And I’m sure we’ve all had conversations with individuals who will say, “Look, that’s University 101,” or “That’s Academic Student Services’ job, not mine,” and I think right now the realization is no, it’s all of our jobs.

Nancy: And Regan, I really appreciate you saying that because part of me is cringing a little bit at the idea of an opportunity because all of us right now, we’re seeing not only the people getting sick and people dying, but as Aeron said earlier, faculty are first and foremost right now worried about their students. Yes, we have to make all this transition to an LMS, to Zoom, to whatever, but first and foremost, “Are my students okay?” Those are the conversations that I’m hearing and, “Are my colleagues okay?” So right now again, we’re in that early stage where I don’t know if it’s an opportunity for anything right now. R ight now we have a moment of care for each other and our students just to make it work, and just to survive and thrive together, then we’ll get to some, I think, pedagogical opportunities.

Rebecca: I think the reminder of care is really important, care for ourselves, care for each other, and I think students are demonstrating care for their faculty as well. There’s a lot of stories of students reaching out to faculty to make sure they’re okay too and I think that just demonstrates how we’re all human and that humanness is coming out right now. And the care that goes both ways actually is coming out in these communities. So I think that’s really important. And being forgiving of yourself as you’re teaching in these crisis moments. It’s not gonna be perfect, and I think reminding everyone that it’s not going to be perfect is a good thing to be doing. But then looking forward to, not in a joyful way necessarily, the idea that we may need to be planning for this again in the summer, and in the fall, depending on how the virus experience unfolds, that’s when some of these signature pedagogy ideas could maybe start to be implemented.

Aeron: I think that the way that I’m seeing signature pedagogies is the way that disciplines are reaching out and I know there’s a lot of resources being shared by historians. I know there’s folks in the sciences that are sharing resources and in math, so that is a movement toward a sort of disciplinary signature pedagogy approach, which is “How can we share methods and ways of engaging in this new modality that will be effective?” What, of course, we hope eventually can be afforded is some sort of evidence-based way of evaluating the effectiveness of these new modalities. For the record, I’m not saying that we should study this semester, I just mean, in general, that we do want to go toward evidenced based. But, thinking about compassion and flexibility, which has been our mantra in every department consultation, compassion and flexibility for our students and for ourselves. Again, shouldn’t that be our mantra all the time, because even though we don’t always have this many people facing a health crisis and employment crisis, and mental health crisis, we have students facing those things and faculty and staff facing significant health challenges, and mental health challenges, and economic challenges all the time. It’s just not all happening in the same way. And so probably you’ve all seen and read studies and disability rights folks saying “Well now you know what it’s like to really have to think about these health concerns and to feel isolated,” and I think that’s a really important part of this conversation, that some form of this virus has been going around all the time. People have been affected in many ways, people have been losing jobs, people have been overcome by stress that makes them unable to perform cognitively at the level that we keep expecting, so I wanted to throw that out there too.

Rebecca: I think it’s really interesting to see how all these things that tend to be invisible have become visible, and that maybe is a really useful outcome of this experience.

Nancy: This really is a moment of forced empathy, if you will, and it’s hard not to think about how desperately we needed to empathize with each other in the historical moment we were in a few months ago. And now we have this moment where we’re having to really think about people across the globe and people who are very different from us in ways that I think a lot of people haven’t, so it is this moment of care and empathy and compassion.

Regan: I just sort of, especially at this time where many faculty may be struggling with “How do I teach this in this format? How do I do what I normally did in this remote teaching environment?” And it actually reminds me of something when we edited the first book in particular, where I know for me, as a social scientist, reading all the other chapters was really neat to go, “Oh, that’s how you do it there. That’s how you do it there.” And I know something that the three of us shared with all our authors, and even the readers, is don’t just read the chapter from your discipline, read the other chapters. And at this time, I think of that because I go, you know what? There may be another discipline’s signature pedagogy that may help you in your discipline at this time, and I think that’s just another neat thing about nicely describing a signature pedagogy for your discipline, because the reality is some of the elements and how you do it may really help somebody from a different discipline… and the example about the art critique and the phone… yes, that makes perfect sense for a sculpture, but that also makes sense If I want to do something in a different format in what I’m doing.

Nancy: Actually, Regan, that’s a great example because the conversation with the artist and using the phone for critique came as some scientists were talking about doing a biology lab with students with their phones so they could see what the students were doing. so that’s exactly what you’re describing, an example of one discipline working out its signature pedagogy in this environment, and another saying, “Aha, that’s how we can do ours.”

Rebecca: We’ve had a lot of those interesting intersections, not just at this time, which has certainly happened. We’ve had a really nice social media group that’s been helping each other out and sharing some of those ideas and examples, but also, I’ve run an accessibility fellows program that is cross disciplinary too and those kinds of things happen all the time, where it’s like I’m trying to overcome this accessibility barrier, and then someone from another discipline has encountered something, it’s not exactly the same but has some of the same kinds of issues, like in sciences, and the arts, for example, certainly helped each other out a lot in that area. So I think it’s always fun and maybe a nice opportunity to get to know colleagues and ways of knowing that are different from what you always have experienced before. One other question that I had thinking about signature pedagogies is maybe a lot of disciplines haven’t really thought about where remote plays into their discipline, or what it means to be a professional, and if this is an opportunity to think about what kinds of remote experiences actually happen in our disciplines, as professionals or the kinds of things that we engage in that maybe we might start incorporating into our classes anyways. And this might be an opportunity to experiment, maybe not right in this moment, but maybe as we plan in moving forward.

Nancy: I’m just thinking about all of the authors I’ve seen who’ve come out and said “If you’d like for me to visit your class, now I can do that,” or virtual book launches. So I just think even in my discipline of English, how it’s making the authors, and publishers, so much more accessible.

Aeron: Yeah, it’s interesting. We’ve had a little bit of a controversy here at the University of New Mexico. Arts and sciences, I believe, last semester issued a statement saying that faculty have to be present a certain number of days on campus, and I think that this comes from an understandable desire to make sure that faculty are accessible to their graduate students and on committees and that they’re doing service to their department. But we’re starting to see already, even before this current moment, that there are faculty who are just as engaged, if not more so, remotely than folks who are next door in their office with the door shut. So that notion of what does it mean to be present? What does it mean to be engaged? What does it mean to do good work and be a good colleague, I think is being further troubled in this semester.

John: Following up on that a little bit, one of the things that a lot of faculty had said is that they’re going to continue using Zoom or other tools to connect, to hold office hours, because we have a lot of students who commute who just can’t make it very easily to office hours because of schedules, and they found it really helpful as a way of students showing what they’re working on, sharing the screens, and so forth. And my department is continuing a workshop series, but it’s now going to be offered over Zoom and that makes it a whole lot easier for people who are more distant, who don’t have to commute into campus. So, I think we’ll see a lot of those things being rethought when we return to something that’s a semblance of normal.

Nancy: And it’s really helping us push back against that narrative that you cannot have community in virtual environments. That’s been a narrative for a long time, and we’ve known, in pockets, that that’s not necessarily the case, that it can be done, if done intentionally and deliberately, and I think we’re seeing that right now on a global scale. So, I think you’re right. Redefining presence, redefining community, redefining collaboration with great implications for the classroom.

Regan: And I think something else that’s going on here is, to build on that a little bit, we’re discovering some exemplary ways to use this technology that are being shared more, but that probably would not have been shared as much if this was not going on. I think within every discipline, there’s a lot of variance, and there are some faculty who have better developed signature pedagogies who are maybe practicing them more, and some who are not, and I think with the amount of sharing that’s been going on now, I think there’s a little bit of an equalization or where more people are getting access to a “best practice” of doing something that they wouldn’t have been paying attention to before. I’m liking that notion of edits. “Here’s how we can do labs better, here’s how we can do our critiques better.” That’s been shared more than I think it was before, so the way that is getting more scholarship on teaching and learning out there than I think it would have.

Nancy: Another really important thing that’s happening right now is exactly what Regan’s talking about, this sense of sharing. The social media communities built up around teachers, educators, people in specific disciplines sharing resources, sharing advice, sharing experiences on a global scale. During the first week when this happened, I was helping to moderate a Facebook group for educators started by a woman in Thailand who I’ve never met, and in five days, there were over 90,000 members of this group. So we started to divide them down by grade, but just the level of sharing is unprecedented, to say the least. So I really appreciate Regan’s point about the role of scholarship in that sharing, and earlier Aeron talked about the role of evidence-based practices as part of that sharing.

Aeron: That sense of generosity that goes across disciplines and across institutions and across countries as well, I think that is the most powerful message from this crisis as globally, we are all connected, and we’re going to sink or swim together. And we’ve seen even on our campus, a lot of generosity, and folks who are more experienced with online tools volunteering to be consultants, participating graduate students offering, volunteering as well, who are more savvy with tools, and it’s really been lovely to see that.

Rebecca: If we think a little bit about next steps or moving beyond the next few weeks, which are really urgent, and we finish up the semester and we start thinking about reflection. What are some of the things that you want to encourage faculty to reflect on as they move forward?

Regan: I know from a center perspective, something that I’ve been actively trying to do, even right now, is trying to anticipate what the next needs of the faculty would be, and I think, like we’ve all talked about, right now it’s still stage one, “Let’s get remote and let’s get comfortable doing that.” And I think we might anticipate those next level of questions. The next level of needs is key, but I think, again, building on what we just said in terms of the sharing, I think what’s happening is these really neat signature pedagogies are emerging from different schools and different colleges, and I think being able to capture that and then connect with some of what’s been emerging at other institutions is pretty key. I mean, I know locally when I speak to, let’s say, engineering, I hear certain ways that tackling the lab situation and they talk to forestry, and then try to get to share across there, and I think the immediate next step seems to be alright, let’s come up with a better way of sharing these signature pedagogies even amongst other universities in the same disciplines, I think would be pretty neat way to go. So, it’s informal right now, and I think we’re tiptoeing towards a better way of doing it.

Aeron: We’re in the process also of thinking about our phase two after the triage and I think one thing seems apparent, and that is that we’re going to always need to have a remote component or an online component. I hope that in moving ahead that faculty who hadn’t interacted with our teaching center will realize, “Well, okay, this is a resource.” And also will be a little bit less nervous about having Zoom meetings and putting things online. But I think the most powerful thing will come when people, after this semester is over and all of us sit down and think, “Okay, what was lost by pivoting to remote teaching and learning and what wasn’t lost?” And I think a lot of that, going back to how is it changing us as professionals to work remotely? I’ve spent probably as much time as the rest of you thinking, “Okay, what do I miss? And what, strangely, do I not really miss that much? How productive can we be in non-traditional ways, and how engaged can we be in non-traditional ways?” That will be interesting, I think, when the dust settles and when this semester is over and we really have some time to reflect, for us to ask “What was lost? What is it that we want to build into our courses for the fall, and what do we realize that we can live without?”

Nancy: That idea, that part of the reflection is prioritizing, based on “What did I learn would work well, and what can I live without?” as Aeron said… What I actually would like to see people reflecting on afterwards has nothing to do with signature pedagogies. It’s more “What did they learn about being human?” And what did they learn about, I hesitate to say, work-life balance, but that’s the phrase that we all recognize. So much of what’s happened over the last few weeks has forced people to really not only think about “What’s important in my course, what can I get rid of and what do I really need to focus on in my course?” but with our entire lives, and I think we’re going to, in a few weeks or months, start looking back and really re- evaluate how we spend our time, how we spend our time in our courses, how we spend our time preparing for our courses, how we spend our time as faculty, how we spend our time as friends and partners and family members and humans. And I think all of that coming together, that kind of integrated way of thinking about our lives is parallel, or maybe the other side of the coin of, the integrated way the disciplines right now are helping to make sense of what’s happening to us. This is really just all about integrative thinking.

Regan: This is the scary reality for me, that at the end of this we’re gonna ask the same question of both our lives and our classes, which is what’s really important, especially when we think about learning outcomes. At the end of all this are those learning outcomes that so many people sweat so much to cover, was that really important? How our learning outcomes gonna change, I bet that’s gonna be different coming this fall.

Aeron: Backwards design your life.

Regan: There you go, there you go.

Rebecca: I think one thing that’s interesting that you’re highlighting is the idea that to be able to articulate your own disciplinary way of looking at things, you almost need to know what other ways of looking at things are. So, by looking at other chapters of your book, for example, or exploring as we’re figuring out ways to handle our current situation from other disciplines, it’s a good way to then be able to articulate the ways that we actually learn and see the world in our own discipline. By knowing what we don’t do, [LAUGHTER] can be really helpful. Our worlds have collided, there is no silo between my personal life and my work life at this moment, as we noticed when my two-year-old walked in earlier when we were chatting, and I think that that’s important, that integrated way of thinking has been forced because there is no possibility of silo at the moment. Before it was really easy to exist in silos or really separate our personal lives from our work lives.

Nancy: Remember, it wasn’t that long ago when a man was being interviewed on the news and his child walked into the room, and that hit the news all over the place because it was, “This doesn’t happen, and isn’t that cute,” and now it’s just reality.

John: But it does open up some possibilities of better connections with students during this event, because they are in their home, they’re really scared, and I’ve noticed, at least, that they’re much more likely to open up about their concerns than they would be in a typical class session, because in class they see it as very narrow, very focused… when they’re sitting at home and they’re worried and they come in a little bit early or they stay a little later, they’re much more likely to open up about all of their issues and talk about how the class is going as well, but also their concerns and what sort of barriers they have in ways that many faculty don’t normally discuss with students, or at least not in a large-class session. Going back to a point that was made just a few minutes ago, there’s the suggestion that for gen ed classes, it’s really important to convey to students why it’s important and so forth, but it’s also important within disciplines. This came up a little bit in the chapter on economics where economists often say that they’re trying to prepare students for grad school, yet those students make up probably less than 1% of most of the students in our classes, and that’s something that perhaps a lot of faculty don’t always think about. And if we do focus a little bit more on the things that motivate students and why students are in our class and trying to help explain to students why this is important and why it’s interesting, maybe the focus that people are getting now might help people work to address that more generally to improve their disciplinary approaches as well.

Aeron: As someone with a PhD in the humanities, I don’t think we should be thinking about educating future graduate students at all. I think we should be thinking about, in gen ed courses, educating future citizens and human beings.

John: We always end our podcast with a question. What’s next?

Regan: I think something that has a lot of pedagogical implications I know, and life implications, is how long are we going to look at this as, and I think I’m really glad that we’ve moved from the “Let’s reassess every two weeks,” to a school is closed through the fall, or at least to the summer. And I think decisions like that really help people cope and get control, and I think that’s something… I know it’s a mid-range plan… is really getting people used to the fact that we’re looking at minimum this for three months, and don’t do something just for today, change that house around, change that routine around now, because who knows, and I’m one of the most optimistic people normally and I continue to be so, but I just worry about when our students actually start getting sick and when our faculty start getting sick, because they are going to, and I think a lot of what we’re talking about, I saw a meme just last night, “The Titanic’s going down and the musicians are still playing.” This is happening and we’re worried about remote teaching, and it’s important, but I don’t know if we’re having enough discussions about the big picture.

Rebecca: Our contingencies need contingencies.

John: One of my colleagues mentioned that she received a note from one of her students that her mother has been diagnosed with this, and we’re going to be seeing a lot of that, that is a serious issue.

Nancy: What’s next for me, perfectly in line with what Regan was saying, what’s next for me is, it’s a beautiful day outside. We’re on lockdown, but we’re allowed to go outside if we stay away from people, so I’m going to go for a walk.

Rebecca: It’s raining here, but I’m going to do the same thing.

Regan: I teach online in an hour, but I think I’m going to take the dog outside in the meantime.

Aeron: And I’ve never been so happy to be in under-populated New Mexico, where really you never are going to be within six feet of someone, and so I’m going to go take a nice long hike. Shout out to SUNY, I’m a SUNY grad, SUNY Binghamton, SUNY Buffalo.

John: Thank you for joining us. This has been a wonderful conversation.

Rebecca: Yeah, thank you so much. Really good conversation.

Aeron: Thanks for inviting us, it was such a good excuse. Well, nice to meet you too, but so nice to see you, Regan and….

Regan: Good to see you guys.

Nancy: Yay.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Savannah Norton.

[MUSIC]

126. Pandemic-Related Remote Learning

Over the last two weeks colleges across the U.S. have made the decision to shift all classes from face-to-face to remote instruction in an attempt to reduce the spread of the COVID-19 virus. In this episode, Flower Darby joins us explore the challenges and the opportunities associated with this transition.  Flower Darby is the Director of Teaching for Student Success, an adjunct instructor in several disciplines, and the author, with James Lang, of Small Teaching Online. She is also one of the developers of the Online Teaching Toolkit created by the Association of College and University educators (or ACUE).

Show Notes

Transcript

John: Colleges across the U.S. have recently made the decision to shift all classes from face-to-face to remote instruction in an attempt to reduce the spread of the COVID-19 virus. In this episode, we explore the challenges and the opportunities associated with this transition.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Flower Darby, Director of Teaching for Student Success, an adjunct instructor in several disciplines, and the author, with James Lang, of Small Teaching Online. She is also one of the developers of the Online Teaching Toolkit created by the Association of College and University educators (or ACUE). Welcome, Flower.

Flower: Hi, John. Hi, Rebecca.

Rebecca: Hi.

Flower: Thanks for having me.

Rebecca: Thanks for joining us quickly.

Flower: Absolutely.

Rebecca: Today’s teas are

Flower: I’m drinking a tall iced tea.

Rebecca: It can never be tall enough these days, right?

[LAUGHTER]

Flower: That’s right.

John: I’m drinking Ginger Peach Black tea.

Rebecca: I’m drinking English afternoon. Sometimes you just got to go with comfort.

John: This would be one of those times.

Rebecca: Yeah, I think so.

John: We’ve invited you here to talk about the transition that faculty are having to go through throughout the country, and probably throughout the world, on fairly short notice to migrating classes from face-to-face to remote instruction. And a lot of faculty are really anxious about that. What advice do you give faculty in terms of where
they should focus their attention, especially if they haven’t done much work with online instruction?

Flower: I think the most important thing to start with is frequent communication with your students. Students are also very anxious, and so even if all you’re saying to your students is that you don’t know yet how it’s going to go, I think that that really puts students at ease, and the transparency will really serve everyone well. In addition to that, I know that centers like the one that you have there are offering all kinds of support workshops, tutorials, self-help, articles, all kinds of resources that faculty can avail themselves of. I’m not sure that the word is consistently getting out to faculty members. So, I would encourage you to turn to your teaching and learning center or your learning management system support area in order to find out what they’re offering. Schools across the country have broken down: Here’s the basics that you need to know to get up to speed with your LMS, especially for people who aren’t familiar with or don’t typically use it.

John: One of the nice things is how widely those resources are being shared. Look online and there’s lots of places, and your teaching and learning center is likely to be sharing them with you fairly regularly as long as you open their emails. [LAUGHTER]

Flower: I was working one-on-one with several faculty members yesterday in our drop-in support sessions, and my experience was that faculty didn’t know about all the websites that we launched last week and all the resources. So, as you said, John, one of the heartwarming aspects of this current situation is to see how institutions are sharing resources with each other very openly. So, as you said, do a little exploring, see what’s available generally online and what your own institution is offering as well.

Rebecca: I think along those lines, though, there’s also information overload because there is actually so much being shared right now. So if we’re looking for specific topics or subject matter related to this, what are maybe a couple of things that faculty should focus on to just get going?

Flower: That’s a great point, Rebecca, because I myself have felt that I’ve been in a blizzard of emails and resources, and distilling the awesome information into usable and organized material has been a real challenge this past week. So, again, I think the main thing at this point is to communicate. So, learning how to use the announcement tool in your LMS, deciding what is going to be your primary communication strategy, it might be that
you’re going to use email. Then the next thing that we recommend is getting your syllabus and a course schedule into your LMS if you haven’t already done that, and then third, understanding how to use the grade center to allow students to track for themselves how they’re doing in your course. Those are communications, uploading basic files or putting basic, like I said, course syllabus type of information and then beginning to understand how to use the grade center. That would be my recommendation.

John: And in terms of the syllabus, perhaps an updated or some type of addendum for the syllabus might be useful to let students know how things might be modified given the shift in instruction, right?

Flower: Great point, John. Faculty, while we don’t want to let go of or change our learning outcomes for the course, it is absolutely the case that many of us might be modifying what the original plan was, we might be changing the structure of tests and quizzes or creating new and different assignments. So yes, I love that idea of post an updated or an addendum to syllabus. And of course, maybe you also include a prominent statement that says “Subject to change with appropriate notice” to the students. One of the things that I’ve always communicated to my students is that if I make any changes in the syllabus or the course schedule, it will always be to your advantage. And I think students appreciate that sort of sense of security, that knowing things could change, but it will be done to help them if needed.

Rebecca: I think one of the other things that faculty are feeling a little overwhelmed by are all the possible tools and technologies that they can use, right, and sometimes this is opening wide doors of possibilities that they didn’t know existed, but then also, there are so many possibilities… “And my colleague here is using this tool and
my colleague over here is doing this tool, should I be doing that, too?” ends up being this common question, and I know my response has been “Don’t introduce too many new tools because information overload or that now there’s a whole learning curve there.” What are your thoughts on this? There’s so many tool possibilities.

Flower: Sure, Rebecca, I think we’re exactly on the same page, and I’ve been doing this kind of support work for several years now. And my philosophy really does align, as you might imagine, with the book, Small Teaching Online, and James Lang’s original , which is to start small. In terms of technology, I always recommend faculty to choose
something that they themselves are comfortable with, and to not make the mistake of trying, as you said, either several new tools, or possibly trying a tool, a technology, that is so sophisticated and complex that it’s outside of faculty’s comfort zone. So, I’m a big fan of deciding something that you feel comfortable with… starting small. One of the things that I’ve been thinking about is we’re kind of in triage mode, I mean, pretty clearly right now,
but maybe four weeks from now, we may have settled into a better rhythm and you may be able to add or layer on additional approaches or technologies or different ways of engaging with your students. Again, as long as you’re communicating with your students, this is what we’re going to start with. And then later on, if you have the bandwidth personally to learn something new, or maybe after surveying what your colleagues are doing, you identify and isolate the one thing that you really want to bring in. Definitely keep it simple and understand that if you as
faculty are not comfortable using a tool, it’s going to create additional challenge for yourself and for your students as well. So stick with what you’re comfortable with.

Rebecca: I think that’s really good advice, and sometimes faculty just need a little reassurance that what they know is okay.

Flower: Absolutely. For sure.

John: Might you also recommend that they check with the students to see how it’s working and ask them if they have any barriers to whatever they’re doing? We have some people in the region who live in areas without broadband, or there’s some people who are living in households where they can’t afford wireless.

Flower: Yeah, there’s a really robust national conversation going on, which is helpful to really think about the student experience. Once again, I think it’s really important to remember that we’re all people and worldwide this is a weird situation, and everybody is under unusual stress and anxiety. And so another one of my basic rules of advice or guidelines here is to practice empathy and flexibility as much as you can. And so that said, I’m seeing contradicting opinions or different approaches, I should say, “Should we be asking students what technology they have available? Or should we not? Is that too intrusive?” I think, again, being transparent with your students and saying, “Hey, we’re going to try this, and if it’s not going well we will figure it out, we’ll change the plan,” can be a really helpful way to go. And then the other guideline that I’ve been sharing with faculty is to go low tech as
much as possible. So I know many faculty and many institutions are suggesting synchronous, live lecturing, audio and video options. Those are actually the hardest and most complicated, and the most prone to fail or challenge. That’s the peak of what we could be doing. In order to ensure the greatest accessibility, including both student access to technology and also any students who might be using things like screen readers or other tools, going lower tech and
using the tools and functions that are within the learning management systems, such as PDF readings and online discussion forums, and quizzes and assignments, those are actually the most possible to create success. Again, for the moment, it may be that later, after you’ve taken the pulse of how your students are doing, you might add more. Or you might host optional synchronous sessions, maybe a virtual office hour or a review session. But for the most success, I recommend going low tech, aiming for the lowest common denominator.

John: And that would certainly satisfy lower bandwidth requirements for people who might be on slower connections.

Flower: Absolutely.

Rebecca: What are some things faculty might want to think about if they know that a lot of their students are using mobile technology rather than desktops and laptops? So in addition to this lower tech approach, are there other things that you would recommend when you know screens might be small that our students are relying on?

Flower: Great question, and I’ve long thought that higher ed is way behind the curve on mobile learning. If you look at industry or corporate training and professional development, there’s some really great mobile apps now. I’m not
saying that now is the time to go out and find a new mobile learning app, let me be clear about that. But I think higher ed has some work to do here, just a couple of simple strategies to consider. First of all, keep in mind the powerful computer that the smartphone is, and again, you have to be careful not to assume that everybody has a smartphone, but it can be a really interesting tool. Maybe students will record video reflections on their smartphone camera, the tablet camera, and upload those or maybe instead of a long, robust written assignment, maybe
you’re going to be okay with little blurbs of text that students can type with their thumbs on their device. It’s a time for flexibility, for creativity, for rethinking the way that you normally do things, and just embracing the adventure, really.

Rebecca: One of the things that I’ve thought a lot about too is making sure that you’re not putting too much emphasis on things that have nothing to do with your learning objectives. So if grammar and spelling really isn’t part of your learning objective, then some forgiveness over mistyping,[LAUGHTER] and maybe using voice commands and
things like that and using voice to text maybe is appropriate in this case.

Flower: Yeah, Rebecca, I think this is a time to rethink everything, honestly, in higher ed. And I’ve been thinking hard about “What are we going to reflect on looking back when we’re through this immediate crisis situation?” Absolutely, I’m a fan of: if the scholarly citation isn’t really needed for this particular demonstration of student knowledge, then maybe you don’t need to require that. So, I would invite faculty to really critically examine all of their usual practices because it’s not the use right now.

Rebecca: What are some of the questions that you’re getting hammered with?

Flower: Faculty have different levels of experience and comfort, and so some faculty are like, “Okay, I already use the Learning Management System, help me think about additional ways to engage my students online.” The necessity of building and creating and maintaining community cannot be overstated. Again, especially in precarious times such as
these, so many faculty want to know how to engage and interact meaningfully with students. Then we also have, of course, the very predictable question about “How do we do what we do in person such as a lab, or a studio or performance class or field work experience? How do we do that in an online setting?” That’s complicated and challenging, but faculty are resourceful and creative people, and I know that they’ll figure it out. The main takeaway for that question is really identify what the learning goal is for that activity, and then think creatively about how students can achieve that learning in an online or remote setting. Now, keeping in mind that it may be the case that you start to have students do some kitchen sink science labs, or some living room dance moves, or whatever it might be. Students don’t have to stay in the learning management systems, again, with their devices, they can capture video, they can take pictures and upload the evidence of what they did. Just a matter of really focusing on that learning goal and then thinking about the activities that will help students… and again, we have to be careful
not to assume that all students have all the things, that providing options for students to achieve that learning no matter where they are.

Rebecca: Can we circle back to this community piece? You’ve mentioned facilitating community is really important. You’ve talked a lot about communication. What are some ways to get students to come together and feel like they’re still a cohesive whole, rather than disparate people who have been dispersed across the world or across the nation?

Flower: So, before all of this happened, if you’re familiar with my work, you may know that I’ve really focused a lot on increasing the social connections in online classes because there is an inherent distance. It is most often the case that students doing online classwork are by themselves sitting at their home desk or at a coffee shop. It
is unusual for students who are doing online classwork to be sitting with another student or with others. And so just really thinking about that physical isolation, and then thinking about how we can’t use the non-verbal cues that we use when we’re in the classroom. So if we’re explaining something as we’re presenting a mini lecture, and we see a whole bunch of furrowed brows or we see that students are clearly off daydreaming about something else, we can
adjust our approach, we can stop, slow down, re-explain, ask the students what their questions are, and we don’t have that real-time feedback in an online environment. So it’s just very important to be really intentional to cultivate that. It can absolutely be done, you think about how we interact in social media settings. We can engage with other people in online situations, but it takes a little bit more intentionality. So, be visibly present for your students, post those announcements, return assignments, timely answer emails… students still say all the time
that they just wish their online instructors would answer their emails. But those are just ways that you want to be visibly present, posting in an online discussion forum, those kinds of things. And then encourage students. You know, I wouldn’t be surprised if it might be helpful to just create a discussion forum just to say “What’s on your mind right now? How can we help? What are you dealing with? What are the challenges?” and just encourage people in
the class to interact with each other as people.

Rebecca: Wait, we’re all people? [LAUGHTER]

John: Students don’t always have that perception of their faculty face-to-face. This is a nice opportunity to open up in ways that perhaps you haven’t done in the past.

Flower: For sure.

Rebecca: I think a lot of faculty will be teaching remotely from their homes with their own levels of distraction and pets and kids and relatives.

Flower: Yes.

Rebecca: Other habitats, right, to their households, just like students. And I think the more that we can share that and that we’re also trying to manage, or even strategies that we’re using for managing that, could actually be useful as a model for students as well.

Flower: I read a really funny, the beginning of what I think will be a series in The Chronicle of Higher Ed this morning, and it was basically, I think The Chronicle has now dispersed to all working from home. And this one reporter was just describing the challenges of sharing a house with three other working adults and two pets that
don’t get along with each other, and trying to be professional, and be on video conference calls or whatever when the cat’s rear end is brushing against the monitor. [LAUGHTER] So I do think, Rebecca, your point about being really authentic about the challenges that we’re all facing, and again, just practicing flexibility, both for your own approach, and then encouraging your students to do the best they can and you’re there to support them. I don’t think
we can message that frequently enough.

Rebecca: I certainly had cars getting drawn across my keyboard yesterday. So… [LAUGHTER] And up my arm and on my head. [LAUGHTER]

Flower: It’s a challenging situation, and this is just one aspect. You think about the potential financial impact that students and their families might be facing as so many businesses are shutting down right now. You think about if a student becomes sick themselves or somebody in their family becomes sick, there’s so many potential challenges
and barriers. I really think this is a moment for humanity to shine without overstating that, and just supporting each other and being willing to be as flexible as we possibly can, helping students achieve the learning, holding them accountable, but being really willing to flex and empathize as needed. I guess I would just reiterate that we should be kind to ourselves, we should not expect to be online teaching rockstars, we should remember that this is not online teaching as we traditionally think of it, this is a triage mode remote delivery of instruction. And we can’t become really well developed online teachers on the spin of a dime. So be kind to yourself, be patient, take it slow, do what you know how to do. It may be the case that in coming weeks, you can add more, you can become more educated as you avail yourself of the resources that your center and others are providing. But ,just kindness is all that I can really recommend to yourself and to your students.

Rebecca: I think that’s a perfect note to end on, and a good reminder that that flexibility goes both to yourself as well as to your students.

Flower: Absolutely. These are unusual times. We’re all freaking out about lots of different things. And so we have a job to do, and students have a job to do, and we can band together and support each other. I’m just thinking about what movie will be made by Hollywood. [LAUGHTER] I mean, there’s got to be tons of movies that will come out after
this, but specifically about higher ed, that would be interesting. [LAUGHTER]

Rebecca: I think the term you used earlier was weird times, and I would agree, we can all be together in these weird times. [LAUGHTER]

Flower: Just practicing basic humanity and consideration for your fellow human beings, I think, is gonna go a long way.

John: It was just so much nicer reading Stephen King novels than it is to live in one of them.

Flower: That’s true. This is bizarre. Let’s just admit that and determine how we can best move on.

Rebecca: Well, thank you so much for the tips, and also just allowing us all to realize that we’re not alone.

Flower: That’s a great reminder, Rebecca, we are not alone. Let’s help our students feel like they are not alone, and we’ll get through this. We’re resourceful people.

John: We always end with a question. What are you doing next?

Flower: So, John, just this morning I offered and was encouraged to write a new column for The Chronicle. Now, I haven’t written it yet. But the tentative title of my new column, which I hope to develop in the next couple of days, is “Okay, So You’ve Pivoted Online. Now What?” and I really want to talk to people about, there was all this adrenaline, and all this frenzy and excitement, and then after we get through this rush, I can imagine that people might begin to deflate, they might become discouraged, the prospect of finishing this semester, it might not be really energizing to people. So I would like to write out some suggestions for regaining your energy and your enthusiasm and discovering the benefits of teaching remotely in this particular situation. We’ll see, I haven’t written that yet. We’ll see.

Rebecca: Yeah, I know that some of the conversation about graduations, and in my department it’s senior exhibitions and things, those kind of capstone moments that are really special and how we can make them special remotely, maybe they’ll be extra special.

Flower: So much opportunity for creative thinking right now. I’ve been wondering if this is going to be the demise of the higher ed conference or other industries as well, if we’ll ever get back together in person or if we’ll find so many other ways to interact virtually that things might be really different from here on out.

John: I know in our workshops, we’re seeing a lot of people coming in over Zoom that we’ve never seen before in workshops, and we’re hoping to see a lot more of them in the future. So, it’s opening up this type of remote access to people who have never tried it before, and that’s a really positive aspect.

Flower: Yeah, I love the focus on the opportunities that this situation is affording us. And then let’s think carefully, when we get to a point that we can kind of look back on the situation, I think higher ed leaders really need to be thinking critically about what needs to change to support effective teaching with technology, because if
there’s one thing we’re learning here, it’s a staple, it’s a support that we can’t do without. And yet many institutions don’t really support the effective use of technology in our teaching in a really sort of simple and sustainable way. So I’m, I’m encouraging, again, specifically leadership in higher ed to think critically about centers such as the ones that we live in, and about the role of instructional designers and “How do we make this much more of a core function in support of our institutions?”

Rebecca: I hope those conversations start.

Flower: I’m gonna do everything I can, talk to everybody I can about it. [LAUGHTER]

Rebecca: Agreed.

John: And I think those types of support are more valued than they ever have been before. I think a lot of people have discovered the instructional designers and the teaching centers across the country.

Flower: 100%. Again, that’s an opportunity that we’re being afforded right now is to help people see what we can do and access those people who haven’t come to our workshops before and demonstrate our value, a real opportunity to do that right now.

Rebecca: Well, thank you so much.

John: Thank you. It’s great talking to you again.

Rebecca: Stay well.

Flower: Thank you, you too. Thanks for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Brittany Jones and Savannah Norton.

[MUSIC]

121. Persistence Scholars

A college degree generally results in higher incomes, more pleasant and more stable jobs, greater life satisfaction, and lower unemployment probabilities. Many students that enter college, though, leave without a degree, but with high levels of student debt. In this episode, Dr. Michelle Miller joins us to discuss an innovative program she helped develop at Northern Arizona University in which faculty members work together to discover ways of helping their students successfully complete their educational goals.

Michelle is the Director of the First Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of  Minds Online: Teaching Effectively with Technology, and has written about evidence based pedagogy and scholarly as well as general interest publications. She has been working with a Persistence Scholars program at NAU for the past two years.

Show Notes

Transcript

John: A college degree generally results in higher incomes, more pleasant and more stable jobs, greater life satisfaction, and lower unemployment probabilities. Many students that enter college, though, leave without a degree, but with high levels of student debt. In this episode, we discuss an innovative program in which faculty work together to discover ways of helping their students successfully complete their educational goals.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Michelle Miller. Michelle is the Director of the First Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology, and has written about evidence based pedagogy and scholarly as well as general interest publications. She has been working with a Persistence Scholars program at NAU for the past two years. Welcome back, Michelle.

Michelle: Hi. Thank you. It’s great to be here.

Rebecca: Today’s teas are… Michelle, are you drinking tea?

Michelle: I sure am. I’ve got a… I think it’s some type of green tea. It’s actually imported from China, so I can’t read the label, but it tastes great. [LAUGHTER]

John: I have Forest Fruits green tea, which I picked up at the OLC conference in Orlando.

Rebecca: And I have vanilla coconut tea.

John: We invited you here to talk about the Persistence Scholars program at Northern Arizona University. Could you tell us about this program?

Michelle: Yeah. So this is a faculty professional development experience that works very much as a blended course, and it’s run for the past few years. And it was something that we developed and designed right in our institution to address, as you can probably guess from the name, student persistence, and broadly speaking, student success. And to do so in ways that would complement programs that we already had in the works, as well as some other more traditional kinds of faculty professional development programs and courses that focused more exclusively on teaching or course design. So, that’s how we brought this into this space. And it’s been a really exciting experience to get to build this from the ground up and to run it with a number of cohorts of our faculty at Northern Arizona.

John: Could you tell us a bit about how the program was structured?

Michelle: Maybe I should back up a little bit and tell you a little bit more about some of the roots of this program and why there was such great support for it from the beginning. This came out of some real brainstorming. I was in a group a few years ago that was charged with just really open ended brainstorming about this topic of student persistence. And as you can imagine that, from an institutional standpoint, that fits into some very important questions, such as retention, especially retention from the first to second year which, those who are in this arena know is a really critical area for ensuring that we keep the students who we recruit to our institution and ensure that students can accomplish the goals they set out to when they sign up with us. So, I was in this group and as I said, we had this very open ended charge of saying “What else could we do to support student persistence?” And because of my background with the First Year Learning Initiative, which is another kind of student success initiative at Northern Arizona, my perspective is always “What about the faculty? What about the academic side of student persistence and engaging faculty in advancing that, and getting excited about that question?” So, together with some of the other folks I was working with, notably John Doherty, who I’ve collaborated with a number of times on student success initiatives, we got to thinking, “Well, how could we reach out?” I had seen quite a number of programs or appeals to faculty, which really came at it from a very emotional, or sort of heart perspective, saying, “You know, really think about your students, have compassion for the backgrounds that many of them come from and the challenges they’re facing.” And I think that’s wonderful. That’s great and conventional wisdom about how to recruit people and get them excited about something. They say “Speak to the emotions, get to why.” Well, I think that’s true. But, faculty are a bit of a special case. I think that we’re wired a little bit differently [LAUGHTER] in some ways. And I think that we have to come at this intellectually as well. So I said, “What if we had some kind of a program that would bring people in and really engage them in this very rich scholarship that’s around, not just teaching and learning, but also everything we’ve come to know about the factors… institutional factors… psychological… social factors… all these things that play into students persisting until they do attain that degree?” So, that was the idea. Now it sort of went down on paper and sort of stayed on ice, stayed in a file drawer for a few years. But then my leadership came back to me and said, in the context of some other things we were doing, they said “Wait a minute, what about this program that we had thought up?” And at that point, we were able to really put it together and make it happen.

Rebecca: What are some of the things that you covered as part of this program that would be different than the first- year program that you’ve ran, or other things that are focused on pedagogy?

Michelle: We tell faculty when we recruit them in that this is not the place to start if you do want that traditional, like, “How can I improve my classroom?” teaching, or “How can I brush up on these skills?” We have lots in place for that. So, what is different about this is that it does focus on the scholarship of persistence. And you know, from my background, I’m a psychologist, I’m a research psychologist by training. So I actually didn’t know, and I started to get into the course design and student success game. I really wasn’t aware of just how much really good quality scholarship has gone into this and how people have thought about and really committed to many books and articles, all this knowledge that they’ve come up with, or what impacts student persistence and what institutions can do, what faculty can do. So, it does have that flavor of a slightly different content area that, again, many of us are just not aware of, even if we care a lot about teaching in our own discipline. And I think what’s also different about it is that it doesn’t take a traditional kind of workshop or book group kind of approach. I think those are really, really great. We all see great examples of those in faculty professional development, but this was structured as a blended course, specifically. So it’s designed with a kickoff workshop that lasts about a day. And then we go online and do just some very structured weekly modules, largely focusing on some readings and discussions and one culminating project. So, I think that, as well, is something that faculty rarely have the opportunity to engage in. I think there’s some national programs out there, for example, ACUE’s program… that’s online, but that’s also a full year. And this is a little bit more compact, and I think it’s designed in a way that’s a little bit more manageable with a typical teaching and research load that faculty have.

John: And you also had people do some visits to various places on campus too, as part of that, I believe.

Michelle: Right. This is the culminating project which we tried in the first few iterations to kind of refine this and I think we ended up with something that’s really a standout and here I have to credit my leadership K. Laurie Dickson. Dr. Dickson is a colleague of mine and part of the upper leadership at Northern Arizona University. This was her idea and she really encouraged me to develop this. We didn’t want to have, as a culminating project, kind of a very typical five paragraph essay or research project or something like that. We wanted to push faculty out into some areas that were particularly new. And we wanted to have them engage in some perspective taking on angles and aspects of the students experience and the university experience that they just normally would never do. So we called this the field project… so, a very generic name, but here’s how this played out. It was up to them to design an experience. It didn’t have to be lengthy… didn’t have to be some gigantic multi-day thing, but just something that they could go and do and experience, and then write about it from a very first person, very subjective perspective. And also, we did ask them to kind of tie it back to some of the readings that we had done and some of the concepts that we had seen over the course of the experience. The examples of what faculty came up with were just… it’s mind boggling, the creativity that people brought to this. Now one of the popular ones was to simply go on a campus tour. Now, how many times do we as faculty ever do that? And I mean, I work in a building where the campus tours originate. So, I see them every single day going and coming, the parents, the students and everybody, the student tour guides. And it’s just never occurred to me to ask, “What are they saying? What’s the little back conversation? What’s the mood like among people who are on these tours? What do we tell students and their parents, as they’re coming into our campuses?” So people could opt to go on one or more of these tours, you could also go on a department specific tour, which is also a fairly popular twist, and then reflect back again on “What does this tell us about what it’s like to be a student here?” and to start taking that perspective as a student and thinking about “What would affect my likelihood of persistence?” So that was one, but we’ve also seen many other options on this as well. One very creative faculty member decided to go out physically to these different student support spaces and organizations. And we all read about those, I know I do, I get the email that says, “Oh, here’s the center that we have for veterans. Here’s the center that we have for Native American students. Here’s where you go, if you need help with writing.” Well, we see those, but what do they look like? What do they feel like? Are students there when you visit, and what sorts of activities are taking place there? And she actually put her reflections together as a photo essay. So, she took pictures of the spaces, she thought about the look and feel of the spaces, and through that she demonstrated that she was taking this new perspective. And this was not an art or design professor, by the way, her specialization is in foundational math, so you can see they’re crossing out into other disciplines. So, even something like observing a class that’s not yours outside of your discipline, you can make that work as well. If you come at it from this perspective, not as like “I’m here to critique the teaching and get ideas for my own teaching,” but “What’s going on in the back row? What’s more clear, what’s less clear, how might the mood or the feel of the classroom change if I come over a couple of different weeks of the semester, and how does that seem to me?” So those are some of the things that faculty actually did to experience some of these things from the other side.

John: How many faculty were part of this program?

Michelle: We usually have run cohorts between about 12 and 20 faculty per semester. And I think we’re about four semesters in, so it’s not an enormous program. But you could see over time with a concerted effort and continued dedication to the program, continued support for it, that we’ve now directly engaged quite a few faculty from around the university. And I should say as well, here’s another little twist that I was not anticipating when we sat down to design this program, is that it’s not entirely all faculty either. We’ve also reached out to staff members, for example, people who work within our advising center or our academic support centers, which function as our tutoring centers on campus. In the first cohort or so I just received a request of somebody’s saying, “Hey, my staff would really benefit from this, do you mind if we have a person or two participate in it?” First I said “Well, okay, I wasn’t planning on that. But I can’t see why not?” Well, I soon learned that having that mix of individuals in the cohort is part of the power of it. Because you think academic disciplines are siloed, we are tremendously siloed in terms of units of student support across campus. To see the interplay in discussions and in meetings between people who work in these more direct student support roles, and people in more traditional faculty roles is really amazing. It really cuts across several of those silos as well just in the participation.

Rebecca: Can you talk a little bit about some of the content that participants were surprised by that was counter to what their assumptions were?

Michelle: One of the challenges in pulling the content for this was that I did have to be really, really selective. Being excited about this, of course, I’ve got 100 articles and things that I want to share, and all these concepts to lay on them, and I went with just a very, very few that I felt were the most powerful and the most backed by research. I would say that one of the ones that surprised them, maybe pleasantly so, is some of the academic work around lay theories and belongingness, which is related to mindset. And probably a lot of your listeners are familiar in some way with those. But, in particular, the work of David Yeager, he’s one of the major researchers in this area. His work focuses on how you can communicate to students that things like intelligence and the potential for academic success are not fixed, they’re something that can be built up through effort. A piece of this is normalizing struggle in a way that, just because you get to campus and you feel lonely, and you feel overwhelmed, that a lot of successful people start out that way, so don’t quit. And what’s I think surprising to faculty and definitely was to me as well when I started reading the work is just how powerful some relatively small interventions can be. Just going through, say an online module that exposes students to some of these mindset concepts can result in statistically significant changes to the likelihood of persistence, retention, and things like that later on down the line. So, I think I was surprised, and I think many faculty are surprised by that as well. And that that work is really high quality in terms of the scholarship behind it, the statistical analysis, how the studies are set up. That’s another kind of pleasant surprise too.

John: A while back, we had Angela Bauer on the podcast, who’s now at High Point University. And she had an intervention in the chemistry department there, where just growth mindset messaging that was delivered by slides that were used by all the people in the department eliminated the achievement gap there. So it was a remarkably powerful effect, which is very consistent with what you’re describing there.

Rebecca: Can you talk about a couple of other small interventions that faculty can implement that are really powerful?

Michelle: Another theme that’s come out of the work on this has looked at the effect of structure… increasing course structure so that, for example, instead of the two midterms and a final, we have those distributed smaller assignments over the course of the semester. And that’s one of those things that there’s got to be a dozen good reasons, from the memory research all the way down to mindset, why this is a really good and powerful thing to do. Now, whether that’s a small intervention or not, that could be a matter of perspective, because for some people, if their course is designed in a completely different direction, that could be some major overhaul there. However, I should say that many of the faculty, in fact, most of the faculty who participate in this, are part of our First-Year Learning Initiative already. In fact, that’s kind of why we decided to develop the program as strongly as we did, is we felt it was a really good complement to those courses that were already part of this initiative we have to ensure really best practices in design for key first-year courses. So, many of those courses are already supposed to have that type of design. But this is a way to continue to engage faculty, particularly those who maybe weren’t on the scene when that course was first designed, they show up and they’re saying, “Why do we have all these grading quizzes?” or “How come it’s set up this way?” Well, this gives them some of the backing behind doing that. I think as well, some of the things that we can look at are simply the communications we have with students. So, that’s another area where I think it may be a little bit under the radar, just how important this stuff is for student persistence, that it’s not even the course design or how the course is taught, just the words that get exchanged in, say, office hours, or the tone of the email that you send to a student to respond to them when they write to you with a question. I think that an experience like this gets us to stop and think and say, “How can I tweak my phrasing or bring in some of that good perspective taking to make those communications either more compassionate or gentle?” or to communicate something like a growth mindset that, “Hey, it’s not a matter of whether you got it if you don’t, we’re just going to jump in where you’re at. And with effort, you can succeed at this.” So, I think those are some of the key things that we can bring in as faculty to affect this very big issue of persistence.

Rebecca: Can you talk a little bit about things that you do in the kickoff workshop? Can you talk a little bit about some of the things that faculty come in knowing or not knowing or mis-knowing?

John: Mis-knowing, is that a word?

Rebecca: I don’t know, I just invented it. [LAUGHTER]

John: It is now.

Michelle: Yeah. I love that term. So, there’s that kickoff workshop where we most directly start to query people’s assumptions, knowledge, and misconceptions about persistence, and to introduce them to this idea that yeah, this is a serious area of academic inquiry that’s interdisciplinary, and we can all access it for the benefit of our students. And in a blended course, it’s generally a good practice to offer face-to-face bonding and group cohesion experience first, before we all go off to our separate online corners. At this kickoff workshop, there are elements of it that are recognizable to anybody who comes to faculty development workshops, but I think there were some novel components too. I mean, one of the things that we do is, it’s simple, but it’s a really effective kickoff exercise. So, we would have either a sticky wall where people can put ideas, or we’ve sometimes bedded rooms that have whiteboard walls, and we have pens, and I say, “Okay, what are some things you’ve heard about why students don’t persist? Just give me reasons. You don’t have to endorse them. They could also be the things that we do here in the faculty meetings.” So, that’s a nice kind of permission giving, kind of opening, I think, to let people say things that they know that are maybe not backed up or they’re not sure, or they don’t agree with them, but they think they’re important to put and they’re also encouraged to put those ideas in groupings. So, there’ll always be some around economic factors, or psychological and social emotional factors, family support. So, we all write on the walls and have these things in front of us for the rest of the day, statements about what barriers there are… to keep that in the front of our minds of what really is affecting our students. As we engage with this work, there is a presentation component, but I really center that around five key claims. So, I think too, it’s important to come with this not just like, “Well, here’s some tips that you can have and some things that some people believe.” I say, “I am not neutral on this. And here’s my five beliefs and these underlie everything that we’re going to do. And you can agree or disagree with these, but I can back them all up, that this is what drives us” and you know, as faculty I think that’s appealing. We want to know what are those assumptions and just to list them off real quick, there are academic persistence matters, so this is important. There are disparities that both reflect and perpetuate inequalities, ethnic class, economic, that we do know a lot about how persistence works. So, that knowledge base does exist. That there are effective strategies for addressing those disparities, although they’re not easy or cheap, I’m not there to sell faculty on magic bullets or “Hey, if you just tweak with this one thing, everything will be fine,” because we all know that’s not the case. And then lastly that faculty do have the ability to positively affect persistence through their teaching, but also through those interactions that they have informally advocating for certain kinds of policies with the institution. So, I really present that. And then lastly, we have a hands-on data exercise. Now one of the things that I think can be a barrier for faculty as they want to get involved with this is we think, or we really don’t have, access to the information that is specific to our campus. So we also have an exercise where I bring in librarians, this is really great. They’ve supported me a great deal in this and we get people on laptops and say, “Alright, here are some sites to explore, national sites about student persistence, databases, article databases you can look at. Use these to uncover solutions, facts about student persistence right here, right now, just do this right now.” And we also get them access doing some basic working knowledge of our institutional dashboard for looking at things like pass rates or grade breakdowns which you can do, you can do it by course, you can do it by semester, you can do even more fine grained by student characteristics. This is all out there, but the vast majority of faculty just do not either know that or they don’t have that working knowledge. So, what I envision is okay, a faculty member can, if it comes up in their department, “Oh, hey, what can we do about this course that’s maybe a bottleneck or we think we’re ready to redesign this one over here?” They can pull the data for themselves and say, “Well, here’s how things changed when we brought in, say, a courseware system, or here are the students who are having the most difficulty, or if a student passes this course, here’s their likelihood of succeeding in this one down the line.” Faculty love that. And once that power is in their hands, I think that they really can carry that out. That’s all the stuff we do, and the kickoff that we have right there and how we establish that grounding for them.

John: It’s great that you have that data. Many institutions are very protective of data, even though it could be really useful in helping us learn about what works.

Michelle: Then to turn around and say, “Well, faculty are kind of in the way here, faculty are this or that?” Well, yeah, we do have to look at what have we empowered faculty to be able to do reasonably and in ways that are appropriate to their own discipline?

John: What are some of the myths that people come into this with in terms of what leads to students dropping out, or failing, or withdrawing?

Michelle: I don’t know if I’m ready to quite call it a myth, but there is perhaps a sort of counterproductive concept, which is the old “If we would just admit better students” who are, and I’m going to use a terrible phrase, “college material.” I mean, that phrase is awful on many different levels as we look at our students, who are these complex human beings, who’ve come to us willing to step up and try to do these incredibly challenging things to accomplish goals that benefit them and benefit our whole society. There is that. And I think an associated belief is, all of this should just be addressed in K-12. And aside from the practical issues there, especially if you teach at a public institution, which we are, I don’t think that’s right to just say “This has to be sort of repaired as a problem by the time it gets to me, or I can’t… or shouldn’t… do anything.” So that whole complex of beliefs about something didn’t happen before this student graduated from high school therefore kind of what’s the point and if the school wants to retain more students, we need to admit the more academically skilled students from the beginning. I say I’m not ready to call that a myth, that is because, yes, absolutely, things like the accomplishments and achievements, academic experiences you’ve had before you come to college. Yeah, those are all great predictors of retention. It’s not that that doesn’t matter at all. But a great deal of other things do matter. And I think that those are maybe where we want to redirect students. And I think as well among faculty who still have themselves a form of fixed mindset, that is really problematic too. And, you know, this really hit home for me. There’s a recent article by Elizabeth Canning and her colleagues at Indiana University Bloomington, titled “STEM Faculty who Believe Ability is Fixed Have Larger Racial Achievement Gaps.” Wow, just think about that for a second. They were looking at the beliefs that are in the heads of the faculty, not even their teaching or what they selected, or what they said to students overtly, but the beliefs they have about who achieves and why, and whether that capacity, that potential is fixed. That plays out in accentuating the exact types of gaps and disparities that we are here to shrink and get rid of. That is surprising and disturbing. And they also find there’s less student motivation overall in those courses. So, I think that that’s maybe another constellation of very counterproductive problematic beliefs that, “Oh, the students aren’t motivated. They don’t want to do the work, but maybe they can’t do the work, maybe they aren’t cut out for this.” If that’s in my head, that is going to leak out and infuse the teaching that I do. And then we have more of these gaps at the end of the day. Those are some of the beliefs that I think are more of an issue. I think less frequently, we’ll see some version of “Well, we’re maybe trying to come from a place of compassion and look at things like oh, family issues, caregiving responsibilities, jobs that students have to hold down in order to be able to support themselves and their families as they go through their education.” It’s great to acknowledge that, but then I think that sometimes faculty can then have this very kind of dead end view of it and say, “Wow, I don’t know if there’s any way this could work.” And yeah, there are only so many hours in the day and we can’t just say, “Oh, education can happen on the margin, no big deal.” But I think too, what we need to step back and look at those beliefs and say, “Well, what are some institutional policies? or “What are even some things written into my syllabus that accentuate that barriers, or put barriers up for students who have those responsibilities? Do they all have to be there? What can I take away that doesn’t get in the way of what students are accomplishing or what’s expected of them, but simply make some of these much more possible?” So that’s kind of a set of those ideas too.

John: One other point there is that students who are most at risk often end up leaving with a large amount of debt and have the most struggle trying to pay for it, putting them at further disadvantage. So, the more we can help these students to be successful, the better off they’ll be.

Michelle: Right. And so many faculty, I mean the faculty who I’ve talked to, and I’ve talked to many at this point, I really believe that they care. They do care about that issue… that deeply disturbs them and deeply bothers them, the idea of somebody leaving with tremendous amount of debt that’s going to limit their lives, and what if they leave with that debt and without the degree that they came for? That’s a tragedy. And I think that we can take that intention and that reaction and channel that into positive action.

John: What types of incentives were there for faculty to participate in this program?

Michelle: For those of us who do work in this space of faculty professional development, we know that that’s an issue. There’s so many demands on faculty time, and so it’s important to have that. So, quite simply, we have a small honorarium. And in fact, it’s small enough to where at first I thought, “Well, do we really need this?” but the feedback I got from my staff and also from faculty was that yes, this is important, if only as a gesture, that we realize this takes your time. So that was $150, and they came in the form of professional development funds. So it’s just enough to plug in, maybe get some books or help make up a gap in some funding for a conference. Those are some of the typical things that faculty use that for. So we have that, and as a kind of a less tangible, but still very important incentive was, as I said, this is part of the First-Year Learning Initiative. And so courses that want to maintain their presence in that program and kind of stay in good standing have to demonstrate this ongoing engagement. So, especially after the first semester or two, we started to say, “Yeah, this is a powerful program, and we really want to make this First-Year Learning Initiative participation contingent on doing this.” So many of the faculty who are there, they do come in because it’s really required for their participation in this bigger program. But, then we have some who come because they’re simply interested and they’ve heard good things about the program as well. So there’s a spectrum of those incentives, both tangible and intangible.

Rebecca: What are some of the students that are the most at risk for persistence? What students are we really helping by engaging in this literature and these methodologies?

Michelle: I think that for people who have some familiarity with the area of student persistence, no surprises here. It’s students who are first generation, as a large proportion of our students at Northern Arizona are. So there is that, there is kind of a constellation of socio-economic factors which can play out in everything from just simply the financial resources one has to attend college, all the way down to the quality of the schools, and the preparation, the pre-college preparation that you were able to get as part of the education you were provided in K-12. So, there is that. Students of color, students of color definitely are going to have a number of barriers and challenges that are going to play out in terms of persistence. And then there are, within particular disciplines, as many of us are familiar with… in particular disciplines where the gender representation or representation of women is relatively low, there can be some persistence issues there as well. So, in the more traditionally male STEM fields, engineering, mathematics and so on, but really, largely these issues of class, of race, and economic opportunity are what all are coming to a crux when students are in these crucial early semesters of college participation. That’s what we’re seeing.

John: It’s fairly early. You’ve only been doing this program for two years, but do you have any evidence of its success in terms of impacts on students?

Michelle: This is a very faculty- and staff-oriented initiative. And there are so many different factors that impact retention and that all go on at once. And by the way, that’s something that I’ve definitely learned as… when I got into this as well… is that there are just this enormous number of options, and even outside of the classroom. Then you have things like learning communities, residential communities, bridge programs, mentorship opportunities, all of these things are kind of getting into the mix sat once, which is probably not a bad thing to have all of these, but it does make it difficult to tease that apart when you look at something like overall retention rates or persistence rates for an institution. However, we have gathered some really systematic assessment data through our participants specifically. So what we did over the past few semesters is we brought in a kind of a pre-assessment so we could capture some very key things about participants’ knowledge and commitment to and ability to advocate for student persistence at the beginning… at the outset of this… before we did anything, and then at the end, after they’d done this about six to eight week program, and so there we do see some pretty dramatic changes and some really dramatic improvements. So, one in particular that stands out is that we asked participants how capable they feel to discuss and apply concepts from the research literature on persistence. And that is very, very low at the beginning. It’s about two and a half on a scale of one to five. And that went up to a little bit over an average of four on that same scale of five after the program. So, that’s something where faculty said, “Yeah, I feel like I can come into this as an informed advocate.” Knowledge about student persistence, that’s another area where the self-rated capability goes way, way up. And also, another thing we asked them is how capable they feel to identify and dispel some of the major misconceptions about attrition and persistence. So there too, the numbers are very, very similar. So we get positive comments, but I also feel like those quantitative ratings have really targeted what I wanted to change as a function of this program.

Rebecca: We talked a little bit about institutional concerns about retention and persistence. Why should faculty be engaged in this piece? We often think, “Well, that’s not our responsibility.” But, why should it be a faculty responsibility, in part?

Michelle: So here’s the thing. I think that this really fits with my experience over about 10 years of working on this at the institution. I think that so many of the initiatives that institutions spend all this money and their political and social capital on setting up, those live or die in faculty meetings. And I think that there’s very limited realization of that on the part of leadership. And it’s understandable because that’s one place where they don’t get to go. But I’ve sat in many, many, many such a meeting over my career. And here’s the thing, in my experience, it can just take one person who thinks that this initiative is misguided, or they think we ought to just admit better students that that should be fine, or they only care about retention for financial reasons. It only takes one highly vocal person to shut that down in that department and there may be other people who are sitting there who are interested in this… they’re saying, “You know what, I care. I think that social inequality is perpetuated when students don’t persist. I see real disparities, and I’m not comfortable with that. And I think this is a social justice issue.” Well, especially if that person is more junior or is not tenured, and the person who’s highly vocal is senior and is tenured, that initiative is not going anywhere. And I don’t care how much money you put into it, or what kind of big stipend is attached to it, it’s not happening. So that’s where I really had this vision as a designer of this program that I wanted people to be able to kind of raise their hand and say, “Well, actually, there’s some research that shows this”, or “I learned about this one concept,” or “Have you thought about how inequality is perpetuated, and maybe we should care for those reasons.” So, to equip and emboldened people to do that… Now that’s always up to them. They can take persistent scholars and come away with whatever conclusions that they want. I honestly come at it that way, that it is up to them to draw their own conclusions, but I do feel, especially given those things they tell us on our assessments, that we’ve done the best we can to equip them to go in to be those advocates. And it isn’t just teaching too. Don’t forget faculty, even though we can’t always affect things like financial aid or how drop/add policies are handled or any of that, we do have faculty senates, and sometimes we can weigh in on those issues. So, if we can bring pressure to bear in a positive way on our administrations, we usually think about it as “Oh, the administration is kind of leaning on us to support student success,” well that runs the other direction, too. And it can. And how does that happen? When we have the information because, again, faculty, we run on evidence… that’s baked into our culture, and that is who we are. So if you are the person at the meeting, you can say, “Well, I read this entire book by Vincent Tinto, who’s the most respected researcher in this area, I’ve actually read that book. And here’s what I took away from it. And so here’s why we should maybe give this initiative a second thought.” That’s what I think can be very, very powerful for creating change.

John: Faculty are well intentioned, but they don’t always know what they can do to be effective, and it’s really easy to blame the students when students aren’t successful. And we see that in lots of departments and lots of people. Providing them with information, I think, could start to make a big difference.

Rebecca: I think a lot of faculty are overwhelmed. They might be interested in these topics, but don’t have time to dig around and find the research and sort through it. So, having a curated opportunity like this is a good way to engage deeply with some key materials and come out of it with that perspective, which I think is really valuable. And we see that in other areas too, where we want to learn more about memory, or we want to learn more about learning strategies or whatever. If we can curate those things, then it’s often easier for faculty to engage and think about how they can individually commit to those ideas because they don’t have to sort through all of the information. It’s collated for them.

Michelle: And that’s just such a perfectly articulated way of describing what our design philosophy really was. And yeah, to say you can make a website or a giant compendium of “here’s a lot of suggested resources,” but it’s a different challenge to say, “Okay, you can assign three things. You can select three things for us to read over this three-week period. That’s it, what are those three things going to be?” And I did, it really did force me to really focus on quality and what was powerful. Yeah, that belongingness mindset lay theory piece was one, transparency was another that I selected. And really the last iteration to it, I also selected an excerpt from Lisa Nunn’s book, 33 Simple Strategies for Faculty. It is oriented towards first generation, but it really crossed over into so many practical applications of the research we were reading about. So that was a huge hit with the last cohort of participants as well. So being selective, having one targeted experience that you can simply share in a very informal way, rather than sitting down to write the giant literature review, I think that’s the sort of thing that we do need. And we did design it with that blended approach with that idea of maximum flexibility. Every week was its own modular piece where we did the same thing, so there wasn’t a whole lot of thrashing around about “what are the expectations” and so on. Even things like designing it so that it starts up about three weeks or four weeks into the semester and wraps up, like in the fall, we wrap up before Thanksgiving. That’s a big, big deal to faculty. If you coordinate it with the students’ semester, that’s just going to be too much. And you’re going to hit people with way too many demands right at their busiest time. So, that was also really appreciated as a factor that promoted faculty participation.

John: In an email exchange prior to this conversation, you mentioned something about the AR program at NAU that you’ve been working with and some results that were relevant to this discussion. Could you tell us a little bit about that?

Michelle: It’s so funny, this faculty professional development program ended up intersecting with a completely separate piece of my research agenda right now. I’ve been working for the last about two years with our amazing cross disciplinary group here, the Immersive Virtual Reality Laboratory at NAU with Professor Norman Medoff and Professor Giovanni Castillo. They had designed, already, this program for organic chemistry so students get to actually manipulate with molecules and they worked with a chemistry professor to make actual meaningful assignments that would use that program in this really cool way. We even set up kind of a almost experimental study where we did the classic flip a coin and one section has access to the VR and the other section does not… it has a substitute assignment instead. Of course I’m interested in looking at the impact on student success. Well, I got into the data, and I looked at the overall impact and there is, there’s a reasonably consistent trend towards better grades in O-Chem. And also better final exam scores if you have access to this particular technology and way of interacting with the material. But then I started doing the subsidiary analyses and I was really surprised. We broke it out by first-generation status and first-generation college students, which was about half of our participants, in this case, experience improvements, positive impacts of this intervention that were larger and more consistently they were significant. All the measurements that we looked at were consistent in terms of the advantage that they got. And we’re working on writing this up for publication right now, and we did present them at a conference over last summer. And it’s really stretching my mind as well to try to say, “Well, why is that? What does that maybe communicate to students when we offer them this? How might it actually maybe shore up the experiences of students who have not had access to as good of a chemistry education, most likely, before they got to our university, compared to students who come from continuing generation families?” I was so surprised. And now there’s something that once again is telling me persistence has a lot to do with these other factors. Can we control them? Can we address them? Of course we can’t, as faculty, but we can look to discover ways that extend what we’re doing in the classroom or take particular approaches, and like so many of the interventions that we do in course design, this is one that doesn’t bring anybody down. I mean, if I’m from an advantaged background, I’m from a majority group, I’ve had this great background when I come in, I can benefit too, that’s fine, but somebody else is going to experience disproportionate benefits. And it’s maybe in a way, replicating a pattern that we’ve seen time and again with other ways of approaching these challenging foundational level courses.

Rebecca: That sounds really exciting.

Michelle: Thank you.

John: I’m looking forward to reading that.

We always end by asking, “What are you doing next?”

Michelle: Well, I have handed off the Persistence Scholars program. So while I’m still very proud of the work and feel very engaged with it, I have stepped away from the First-Year Learning Initiative, and as part of that the Persistence Scholars program is going to be led by a colleague of mine, Cody Canning at NAU, and I’ve handed off that program before as part of sabbatical and so on. So it is neat to build a program from the beginning that can be taken on and have it structured in depth enough to where you could take it on and then bring your own expertise and particular perspective to it. I’m still very engaged nationally though with spreading out these ideas about student persistence, learning and success in the first year, and looking at how we can take those and develop those in other places and really spread those efforts out, since I know so many of us nationally are just really fired up about this. So that’s where that stands right now. I’m working on a book right now with West Virginia University Press, with a very dynamic editor and a group of writers who are all working right now on writing about different issues in pedagogy in higher education. So that’s an honor, and I’m having a lot of fun with that book. So, memory and technology is what I’m writing about, and that’s something that springboards off a lot of the teaching that I do and some other writing as well. And that is something that I think is an issue that we see recurring now as being a very timely issue for people who are teaching. So that is taking a lot of my intellectual effort right now, and I’m looking at ways to keep engaging people in Minds Online, which, although it does have that specific technology angle, I think does pick up on many of these issues of promoting student success, and reducing disparities, and finding sometimes very surprising things that happen when we start to teach in new ways. So, that book came out around five years ago, it’s hard to believe, but I’m also looking at all the ideas and research that’s come out since then, and new applications that faculty have come up with. So, I’m looking at some new ways to keep that percolating along and kind of harness some of that energy we all have around that topic. So, I would say with that, just stay tuned or contact me to learn more, and we’ll see how that develops over the next year or so.

John: And when is this new book coming out?

Michelle: Oh…

John: Tentatively?

Michelle: It’s coming out after I write it. Let’s just say 2021. So it is well, well underway. We’re in striking distance of having that out in 2021.

John: And that’ll be part of the West Virginia University Press series edited by James Lang.

Rebecca: Well, thank you so much for joining us. It’s always a pleasure.

Michelle: Likewise, it’s always great to talk about these issues with both of you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

110. Fostering a Growth Mindset

Some students with fixed mindsets enter our classes expecting to be unsuccessful while others believe that they have a natural talent in the discipline. In either case, these students often get discouraged when they experience challenging tasks. In this episode, Sarah Hanusch and John Myers join us to discuss how they have revised their classes and used metacognitive exercises to help students develop a growth mindset and to recognize the benefit of learning from mistakes. Sarah and John are both Assistant Professors in the Department of Mathematics at SUNY Oswego.

Show Notes

Transcript

John K.: Some students with fixed mindsets enter our classes expecting to be unsuccessful while others believe that they have a natural talent in the discipline. In either case, these students often get discouraged when they experience challenging tasks. In this episode, we examine how two faculty members have revised their classes and used metacognitive exercises to help students develop a growth mindset and to recognize the benefit of learning from mistakes.

[MUSIC]

John K.: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John K.: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Sarah Hanusch and John Myers. Sarah and John are both Assistant Professors in the Department of Mathematics at SUNY-Oswego. Welcome, John and welcome back, Sarah.

Sarah: Thank you.

John M.: Thank you.

John K.: Our teas today are?

Sarah: None today

John M.: Yeah, imaginary tea. No tea for me.

Rebecca: The imaginary tea…that’s what my daughter likes to drink. That kind.

John M.: Yeah, I’m in good company there&hellp;

Rebecca: I have English afternoon.

John K.: And I have a ginger tea.

Rebecca: We invited you here today to talk a little bit about how you’ve introduced a project on metacognition in some of your mathematics courses. Can you tell us a little bit about the project?

John M.: Sure, this began, I believe, in the spring of 2018 in a Calculus I course. And the idea was that, Calculus I is known across, basically the entire country…every school in the country…as being a very difficult course. So, you have a lot of students who are coming in, especially in the spring semester, who had bad experiences with calculus in the past. And in particular, I’ve been told by some colleagues that there’s going to be some students in there that more support than I suppose you would imagine. The situation was that on the very first day of class, I had students coming in who have had bad experiences with it in the past. And then at the same time, I have the students that are typically high performing. And they have difficult times also with perfection, you know, being obsessed with 4.0s and grades and that type of stuff. So the idea was that I wanted to simultaneously address failure with the students and perfection at the same time. And I was sort of led to think about this metacognition project, actually, funnily enough, on a flight back from San Diego. I was at what are called the joint meetings for mathematicians, and a lot of progressive newer teaching techniques are talked about at this conference. And I’m flying back from the conference on the airplane and I’m getting really introspective and I’m thinking like, I really need to do something to talk to my kids about failure and perfection. And then it occurred to me that there was this blog post that I had just read a couple weeks before by a mathematician by the name of Matt Boelkins at Grand Valley State University. And he had this idea for a metacognitive project that addressed all sorts of things like growth mindset, fixed mindset, productive failure, and all these different things. And I decided about a week before classes started that this is what I was going to do.

Rebecca: That’s when all the best ideas happen.

John M.: I know…right before class and on an airplane. I get really introspective when I’m on airplanes and staring out the window and thinking of all the big things in life and stuff.

Sarah: And essentially, John came to me and said, “I’m thinking about doing this project.” And I said “Well, that sounds cool. And let’s see if we can measure if it has any positive effect or not.” So, I sort of came in on the research side of it…of “let’s see if this is effective for changing attitudes towards mathematics.” And since then, I’ve stolen the project to use in my own classes. But, it really started as I came in sort of more on the research side of things

John M.: I think stolen might have been a strong word, but…

Sarah: I didn’t ask…I just took it. [LAUGHTER]

John K.: For the research project did you do pre- and post-tests on attitudes?

Sarah: We did a pre- and post-test, we use an assessment called MAPS which is the Mathematics Attitudes and Perceptions Survey. It’s a 31-item survey. It assesses, I think, it’s seven different dimensions. Some of them are growth mindset. Do they view mathematics as being answer focused or process focused? The categories were growth mindset, the applicability of mathematics to the real world, their confidence in mathematics, their interest in mathematics, their persistence in mathematics, their ability to make sense of mathematics, and do they view mathematics as being answer focused or process focused?

John K.: Sounds like a good instrument. Before we talk about the results, let’s talk a little bit more about how you implemented it. How was the project structured in terms of what activities did the students do during the class?

John M.: So the idea was that over the entire semester, they would have a selection of articles online to read, they would have a selection of YouTube videos to watch and it was essentially experts that are addressing these various topics. So, like for example, there is a clip by Carol Dweck, one of the originators of the theory of growth and fixed mindsets, and they were to watch these clips and read these articles across the semester. And then I think it was probably with two weeks or three weeks left in the semester, they’d have to write a reflective essay. It was an attempt to sort of shift the culture in the classroom towards viewing mistakes and failure as productive and as opportunities for learning. Because I think in wider culture, everybody believes that math is just about the right answer. And that if you can’t get the right answer, then there’s no worth in whatever effort it was that you put in to get to that point. And I wanted to provide sort of a counterpoint to that, so a counter narrative. Being honest about how many times per day mathematicians actually do fail, you know, that type of thing. So yeah, the main component was this essay that was reflecting on the stuff that they read and watched over the semester, and then there was sort of like daily conversations.

John K.: Were the conversations online or were they in class conversations?

John M.: In class…in office hours, just kind of whenever they popped up. I remember a couple conversations that happened after I gave back exams, for example, or rather right before I gave back exams. So for example, I would say, you know, I’m about to hand back exams. And I want you when you see the score, when you put the paper over and see your score, I want you to immediately think how are you going to frame this result in your mind. Are you going to look at that score and be happy with it and chalk it up to just your natural talents? Or are you going to say, “Oh, this is a result of hard work?” And then if you’re not happy with your score, are you going to put it away and never look at again, or are you going to engage with your mistakes and make them productive mistakes? It was sort of intervention through conversation that happened on an almost daily basis.

Rebecca: Did you notice a difference in the kinds of conversations you were having in class because they were doing these readings and watching these videos, maybe conversations you hadn’t experienced before in the classroom?

John M.: Yes. In particular, I had students come into office hours and they were relentless with trying to understand the material because they knew that they were going to have another shot to get it right. And I had never experienced that before. In fact, in one of my student’s essays, I had a student tell me that when she’s not done well on exams in the past, she would just take the exam and stuff it into her book bag and never look at it again. And she told me that just because of because of how I was structuring the course that she doesn’t do that anymore. She actually pulls it out and engages with the mistakes and the comments that I put on the exam and comes and talks to me about the exam and everything. So I did see a change in the students.

John K.: Was some of it based on the reflections or was it also partly based on a restructuring of a course to give students more opportunities to redo things or to try things again?

John M.: I believe the latter had something to do with it. Because the idea was that I could say these things out loud to them. But I wanted to actually build components into the course in addition to the essay that sort of reflect the themes that I’m trying to communicate to them.

John K.: Telling them that they can learn from mistakes, if you don’t give them the opportunity…

John M.: Right.

John K.: …to learn from mistakes might not be as productive. I think both components are really valuable. I just want to make sure we were clear on that, too.

John M.: I think that you risk sounding like a cliche motivational poster, if you don’t actually put some meat on the bones with it.

Rebecca: Can you talk about some ways that you actually built that into the course?

John M.: I did test corrections. I don’t remember exactly, I think it was get back half the credit they missed or something like that. So, the idea was that they had to engage with the mistakes on their exams and correct them. And it had to be perfect. So they had a week to turn in their test corrections, and then I would re-grade them. This was very time consuming, as you might imagine, but the students I believe, really responded to it. It really sort of hooked in with the theme that I was trying to send.

Sarah: And since then, we’ve both moved to more mastery based grading. John before I did, but a system where students keep trying things until they get it right. And that really helps sort of drive that “learn from your mistakes” message home.

John K.: Are you able to do some of that in an automated way? Or is this all involving more grading on your part?

Sarah: The way I’m doing it, unfortunately, it’s more grading on my part. Although I will say this semester I’m doing these mastery based quizzes, but I’m not collecting homework. So, it’s kind of a toss up in terms of how much…it isn’t really extra grading. I’m just grading more things in another category.

John M.: Right, I would not do test corrections again. Not only was it a lot of time to grade, but then I had issues with academic honesty. The mastery based thing I have found is, I believe, much more effective.

John K.: Another thing you may want to consider that we’ve talked about in a couple of past podcasts is having a two-stage exam, where in the first stage, they do it themselves. And then you have them break up into groups and do either all the questions or a subset of those as a group. So, you’ve got some peer instruction going on as well…and that way it’s done right in class and it can be done, if the exam is short enough or the class period is long enough you can do both of it. A common practice is to do two-thirds say individual and then one-third for the group activity, which has many of the same things. They don’t know what they’ve gotten wrong, but when they’re sharing with their peers, they’re talking it over and it means you only have to grade the group exams on the second stage, which makes it a whole lot easier than individual ones.

John M.: Right. Yeah, I have a friend I believe he has done that stuff like that. So yeah,

John K.: The Carl Wieman Science Education Institute, I believe, has a lot of information on that. I’ve been doing it the last couple of years, and it’s been working really well. Doug Mckee was a guest on an earlier podcast, we talked about that as well. Are there other things we want to talk about in terms of what you’ve done in the courses?

Sarah: One thing that we’ve both done since this initial project is we’ve taken some of the ideas of this project, but interspersed it more throughout the course. One thing I know at the time that John observed was that he felt like a lot of the students started the projects in the last week, right? And so what I’ve done instead of doing a big project of these topics is I’ve taken these articles and done the second week of class, you have to read one of them and respond on it. And then the fourth week, you have to do another one, and so on. So it’s a little bit of it throughout the whole course instead of all loaded at the end. I think it helps having some of those conversations with the students as well because they’re not just seeing the ideas in the conversations. They’re not just seeing the ideas in the paper. They’re kind of seeing both and it just helps intersperse it a little bit throughout the semester. I know I’ve done that a couple times now. I think you’ve done that since as well.

John M.: I did a pre-semester sort of essay and then I did a post-semester essay. But it was in response to the first time we did that, which is referred into the paper, and one of my students actually told me in their essay, he was like, ‘Hey, I wish I had this at the beginning of the semester.” So yeah, it’s definitely like a “duh” moment. Like, I probably should have done something earlier in the semester, instead of waiting all until the end. But, you learn as you do these things, so. But the essays that the students wrote… I provided them with prompts just to alleviate any sort of writer’s block that they may have. But, the students who basically ignored my prompts and told me their personal stories were the essays essentially that I still remember. I had students that were straight A students that were telling me exactly what I thought was going to happen: that they’ve been the smart person their entire life, and they kind of feel trapped by being a smart person. They don’t want to take any risks because if they risk something and fail, then that’s their identity as a smart person, right? They’re not smart anymore. I’ve had students from the other end of the grading spectrum who basically told me that the first day they walked into the class before I even said anything, they were already convinced that they were going to fail the class. I had students tell me about mental health problems. I had adult learners talking about balancing life and school issues. I mean, it’s just absolutely amazing what they told me, they opened up basically. That made a big impression on me.

John K.: Tying into an earlier podcast, Judie Littlejohn and I had introduced something really similar where we have weekly discussion forums. And I also noticed the same sort of thing, that I got to know the students much better because when they were talking about some of the barriers or the issues they face, they were sharing a lot of details about their life. And you get to know them better and they also seem to form a little bit more of a tighter classroom community because they also got to know each other a little bit more.

Rebecca: It is kind of interesting how when students are talking about their process or who they are as learners, is very different than talking about the subject matter. And it does get them to open up and may be engaged with faculty in a way that they wouldn’t otherwise.

John M.: And I have found being honest about my own failures in the past has been a catalyst for conversation, right? Because they view us as professors, they view us as the authority figures, the experts in that we never fail. And basically telling them how many times I fail on a daily basis in my own mathematical research. It goes a long way, I think… finding common ground with them. And acknowledging how difficult the subject material is. I mean, there’s a reason that calculus has a high failure rate because it’s a hard course, among other reasons. Yeah, just having the humility with the students and kind of stepping down off of the pedestal in front of them, I think that it helps.

Rebecca: So do you want to share some of the results that you got from your study?

Sarah: We saw some very significant quantitative results. I mentioned the MAPS instrument is what we use. It’s a 31-point scale. Its reliability and validity has been established pretty well, especially in calculus classes. One of the things that they did was they looked to see if the items were consistent with expert consensus…. So, with how mathematicians view it and all of the items were valid with the attitudes of mathematicians except some of the growth mindset scales. Research says that that’s an important scale as well. And on this 31-point scale, we saw an almost 4-point improvement from pre-test to post-test…of the students becoming more aligned with the expert opinions, which is a really significant amount…I mean, almost 10% improvement, which is even more remarkable, because when this assessment was first validated, they found that there was usually a negative result from taking a Calculus I class. So, the attitudes get worse pre-post in a calculus class and ours had statistically significant improvement. In addition, we saw statistically significant improvement among all of the sub scales. Now some of them were better than others. Some were just barely below .05 in terms of significance and others were much more significant. I mean, we really saw that over the course of this semester, they really did change their attitudes. We also had some evidence, as John’s already talked about, from their essays…where they said how they started to view mistakes as productive, and they started to feel like there was value in making mistakes and learning from them.

John K.: You mentioned alignment with an expert scale, can you explain that for our listeners?

Sarah: Essentially, what the original authors and it was Code et. al. that did this paper and develop this instrument. They gave this survey to students and they gave it to mathematicians and looked for alignment. Particularly they were looking for whether or not the mathematicians agreed on the items. And the idea was our goal is to get math students to have attitudes more like mathematicians, because that’s our goal, right? …is to develop future mathematicians. And so we would like those attitudes to get closer to how mathematicians view mathematics. They had high agreement among the mathematicians on every item, like I said, except one or two of the growth mindset questions. So, in other words, this survey reflects how mathematicians view mathematics. And that was how they determined the right answers on the survey, whether a particular item is something you should agree with or something you should disagree with. They went with the expert consensus.

John K.: So now, I may be misconstruing this, but are you suggesting that perhaps a lot of mathematicians had adopted a fixed mindset? So, there was a bit more variance there on that?

Sarah: I will say that was what the results of their validation showed.

John K.: Okay.

Sarah: And leave it at that. [LAUGHTER]

John K.: It does remind me of that study a few months ago, that found that when instructors had a growth mindset, the achievement gap narrowed and the drop-fail-withdrawal rate was much lower in courses, then for those instructors who had a fixed mindset. I think that maybe even more of an issue in the STEM fields than it is in humanities and social sciences, but I think it’s not uncommon everywhere.

Rebecca: I say it’s a common problem everywhere.

John M.: I’ll say it…mathematicians suffer from fixed mindsets. I’ll just say it, right? [LAUGHTER]

John K.: Many academics do.

Sarah: Yeah.

John M.: Yes, of course.

Sarah: I mean, the people who choose to become academics are often the people that were successful in school and they decide to continue with it. I mean, it is less likely that people who felt unsuccessful decide to keep going and to go into academia.

John K.: Selectivity bias there and that reinforces a belief in a fixed mindset, perhaps.

Sarah: Precisely.

Rebecca: What kind of response have you seen from students from…I mean, it sounds to me like this one study lead to good results, and then that changed many classes in that you’ve taught or the way that you’re teaching, how have students responded?

Sarah: Generally positively. I think doing the projects at the end of the semester wasn’t the best idea because they just feel so overwhelmed at the end of the semester with exams and projects and everything coming due. So, I did get some responses of “W hy do I have to do this now.” But generally, I think they appreciated learning about learning.

John M.: I think that given the opportunity to talk about their past experiences, I think they appreciated that. For the most part, I’ll agree with Sarah. I think that the message landed with an awful lot of students like I wanted it to. Some of my favorite essays were students who told me that they thought I was crazy on the first day. I mean, you go into a math class to learn math, you don’t go into a math class to study metacognition, or whatever it may be. I had one student the first time around, who basically told me it was all a load of crap, like why this is not working at all. And I had a student the last time that I did this, she was very skeptical towards the end even. Basically, aliken it to just some cheesy self-help stuff. I think that most students responded positively.

Rebecca: Have you seen the response impact other faculty in your area? For example, if they really liked having those techniques and things introduced in your class, have they asked other math faculty to do that in future classes or are you finding that its not many math students who were actually in that particular class?

Sarah: We haven’t done any tracking, so I don’t know where his students have gone. I mean, I’m sure some of them went on to Calc II…I’m sure some of them did not. Right. I mean, I guess most of them would have had Jess the following semester, right? Did she say anything?

John M.: No, she didn’t say anything. I’m teaching Calc III right now, and I have some of my former calculus students that were in this and they’re doing well.[LAUGHTER] Small sample size, but yeah, they’re doing well.

John K.: That could be an interesting follow up though to see how successful they were in the subsequent classes.

Sarah: Yeah.

Rebecca: Sometimes we’ve heard anecdotes, of departments and things when there’s been change that if students really respond well to whatever the techniques are, that they will demand it of other faculty members, and John’s talked about this before in economics.

John K.: Yeah, when you can show results…

Rebecca: Yeah.

John K.: …that there’s been some gain, and especially if it comes from students at the same time, it often puts pressure on other people in the department because if you’re able to show people that your technique has been successful and students are coming in and saying, “G ee, I wish you would consider doing this. I did this in my intro classes, and it was really helpful.” That sometimes helps make change much easier.

Sarah: Yeah, so one of the things that we did look at was we compared the final exam scores of John’s sections to the other sections of calculus that semester. Now, there was some other issues that clouded that data a little bit. His scores were a little bit lower than the other instructors. But what was really surprising, essentially, if you look at, I don’t remember if it were just the final exams or the semester grades. The DF rates were the same among the sections, but the withdrawal rates were significantly different. And that almost no one withdrew from John’s sections. I think there were two if I remember the data correctly, whereas there was like five or six on average from the other sections. And so the DFW rates were different, but the DF rates weren’t. So I just thought that was an unusual circumstance. So, it seems like the students were sticking with his class… and pushing through.

John K.: And if there is a larger portion of students staying with the class, then perhaps a slightly lower average grade is not necessarily a bad sign…

Sarah: Exactly.

John K.: …because student success is partly measured for persistence to completing the course.

Sarah: Exactly. I think because there were more students who stuck it through to the final exam, then his final exam scores ended up being a little bit lower. But again, if you looked at like overall course grades, they ended up being pretty consistent, other than the W rates. I wanted to make sure that there weren’t significant differences in the rates and I think it was just shy of being statistically significant. Like, if you had one more student that would’ve been significant. But just to make sure that, especially like adding the test corrections in wasn’t substantially making the class too easy, right? Because that’s often a critique that, you know, “Well you make these changes, but is that just making the class too easy and people who aren’t really prepared, are they passing?” And so I just did this analysis of the, like I said, it was really just a t-test analysis, but just to see whether or not it was significantly lower and it wasn’t significant. It was lower, right, just not significantly. And then like I said, I looked at retention rates just more as an explanation for why the average was lower.

John K.: In a lot of studies of interventions, the dependent variable is the drop-fail-withdrawal rates, because that’s a measure of success in completing the course. That by itself could be an interesting focus of a study. I’ve been running this metacognitive cafe in my online classes for a while and I did have a student in the class who wrote a few times about the metacognitive development that was introduced in one of your classes. They didn’t specify who but they said, we’re also doing some work on metacognition in the math class, and they said it was really useful and it was nice to see it in two classes.

Sarah: Yay!!

John M.: Good.

John K.: So there’s at least one positive data point there or one additional data point there. So are you going to continue this in the future? And if so, what might you do differently?

Sarah: Well, I think we’ve mentioned already that we’ve worked on including some of the ideas at the beginning of the semester and throughout the semester, rather than one project at the end. For the reason that it really benefits them most at the beginning of the semester when things are getting started. I think we’ve also both changed different things about our grading systems to incorporate more opportunities for growth.

John M.: The last time I did this, I introduced some articles that were a little bit more rigorous with the data and the science, because I sort of wanted to counter that kind of criticism that all this “Oh this is just a bunch of TED Talks…” that kind of thing. So, I really wanted the students to see some of the science behind it, the science of learning, because I really wanted to send that message that “No, this is not me just standing up here saying, ‘Oh, this is going to help you or anything, right?’ This is actually stuff that researchers have thought about before.”

John K.: I had a very similar response the first time I did this. I had a video I posted which was a TED talk by a cognitive scientist who talked about research that showed that learning styles were a myth. And some students had come to believe in the existence of learning styles because they’ve heard of them and often been tested, multiple times in multiple years, on their learning styles. Sometimes even through college and that’s rather troubling. The students said, “Well, this is just one researcher, I’m sure there’s lots of other studies. I don’t believe it because it’s not consistent with what I’ve always been told or what I’ve heard.” So I decided to modify it then and I added to that discussion, five or six research studies. In case you don’t believe this TED talk by someone who’s done a lot of research on this, here’s a number of studies, including some meta analyses of several hundred studies of this issue, and that has cut much of that discussion. They’re less likely to argue against it when it’s not just a talking head or not just a video when they can actually see a study even if they don’t understand all the aspects of it.

Sarah: Yeah. So I think that’s one thing we’ve tweaked what articles and what videos are we showing. I know the semester I gave my students a article that had just come out this September, that students perceive active learning as being less efficient, even when they’re learning more. In some physics classes at Harvard, they gave two weeks at each thing… two weeks of active and two weeks of lecture, and then they had them switch. And the students learned more with the active learning, but felt they learned less. And my students have been feeling frustrated because they feel like they’re not learning enough and that I’m not telling them what to do.

Rebecca: You’re not “teaching” them.

Sarah: I’m not teaching them. And we spend the class period, letting them vent. So all their feelings were out in the open. But, then I sort of countered with this article saying, “Look, I promise you really are learning things. You just don’t feel like you are. But you really, really are. And you’re actually learning it better than if I were using a different style.” So, that’s one way that we’re tweaking the articles because sometimes the research comes out that’s pertinent.

John K.: We refer to that Harvard study in a few past podcasts. We touched on it in a podcast that will release on October 9th. I haven’t shared it with my class yet, but I’ve been tempted to.

Rebecca: What was the discussion like talking about that particular article? Given that they were frustrated?

Sarah: I mostly was just trying to acknowledge that I understand their frustrations…and that, yes, the way I’m teaching this class can be frustrating. I agree. Sometimes I get frustrated about it. But I know that ultimately, they are learning things and that they are going to be stronger writers and stronger students of mathematics by using this structure. And so I kind of use it as evidence of I’m not changing.

Rebecca: So I hear you…

Sarah: Yeah.

Rebecca: …nut…

Sarah: I hear you, but…

John K.: I had this very conversation with my class today. They’re coming up for an exam very shortly. And I asked them, how did they review before an exam and the most common answer was they like to reread the material over and over again. And I mentioned some of the research on that. And I said, the best way to review is to work on problems with this. And I gave them several ways in which they could do that, that are built into the course structure. And I said, “But that doesn’t feel as effective. Why?” And one of the students said, “Well, I get things wrong.” And I said, “And when would you rather get things wrong, when you’re reviewing for an exam, or when you’re taking exams?” And I think some of them got that message. So I’m hoping we’ll see when they take the test next week.

John M.: Right? It seems like anytime you do anything that’s just not a standard straight lecture, there’s a certain amount of buy in that you need to get from the students. And sometimes that can be very difficult. There’s almost a salesmanship that you have to do throughout the semester to make sure that everybody’s on the same page and to kind of fight those feelings where the students give you a lot of pushback. Yeah, that’s the great fear is that when you innovate or you experiment that’s going to go horribly wrong. And sometimes it does, but, you know, we still keep going.

John K.: Because students are creatures of habit. They’ve learned certain things and they want to keep doing things the same way. And anything new can seem troubling, especially if they’re getting feedback along the way that says they need to work more on things…that’s not as pleasant as rereading things and having everything look familiar.

John M.: Right

Rebecca: Passively sitting in a lecture when things all seem like it makes perfect sense to you, because an expert is describing it who knows what they’re talking about, right? Always feels easier than trying to apply it yourself. And I think that students, even though the lecture might feel better, and learning is hard…over time…at the end, when they’ve seen how much they’ve accomplished, and you do have them reflect…many of them appreciate or come around. Sometimes, it’s not in that same semester, sometimes it’s emails, months or years later.

John K.: Yes.

John M.: Right. Right, right.

Sarah: If only if we could do course evals, you know, a whole year later,

John K.: Or five years later. That may not work too well in my tenure process, though.

Rebecca: We always wrap up asking what’s next?

Sarah: Well, the first thing is we’re hoping our article gets published. It’s been submitted. We’re waiting for reviewers. I’m going on maternity leave next semester…that’s really what’s next.

Rebecca: Sounds like a new adventure.

Sarah: It is a brand new adventure.

John M.: Wow, I don’t think that far ahead, I guess. Yeah, I guess I’m that unoriginal, huh. But, yeah, no I’m just trying to…

Sarah: We’re moving to a new building.

John M.: Yeah, moving to a new building, and getting a new department chair. Yeah, that’s right.

John K.: A new desk to go with the chair?

John M.: No. Ah… Yeah, funny, funny, funny.

Sarah: if only…

Rebecca: Well, thanks so much for joining us, this has been really interesting.

[MUSIC]

John K.: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John K.: Editing assistance provided by Brittany Jones and Kiara Montero.

109. Active Learning

Moving from a familiar instructional format such as lectures to a more active learning environment can be daunting. In this episode, Dr. Patricia Gregg joins us to discuss how she flipped her classes and embraced active learning. Trish is an Assistant Professor of Geophysics at the University of Illinois at Urbana-Champaign.

Show Notes

Transcript

Rebecca: Moving from a familiar instructional format such as lectures to a more active learning environment can be daunting. In this episode, we share the story of one faculty member who fully flipped her classes and embraced active learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Patricia Gregg. Trish is an Assistant Professor of Geophysics at the University of Illinois at Urbana-Champaign. Welcome.

Trish: Thanks for having me.

Rebecca: Today’s teas are:

Trish: I am drinking a peppermint decaf tea.

Rebecca: …in what looks like a very nice handmade mug.

Trish: Yes! This was made last summer at the YMCA of the Rocky Camp in Colorado.

John: My tea today is a Harney and Sons chocolate mint.

Trish: Mhmm.

Rebecca: And I have a Prince of Wales tea today.

John: We’ve invited you here to talk about some of the active learning techniques that you’ve used in your class and also a little bit about how you use a flipped classroom approach. But, before that, could we talk a little bit about your own experience in science classes, and whether active learning was common while you were a student?

Trish: I was thinking about this and it’s interesting because it’s sort of a yes or no type of situation. Geosciences in general is the fun major in that we pull together a lot of different disciplines. So, you have chemistry and physics and math and computer science and you’re using those all in applied ways to understand the structure and evolution of the earth. And so our classes typically have a lecture-based meeting time and then a laboratory that’s associated with it. So, when I was matriculating, most of my classes, there would be three one-hour meetings throughout the week where we’d be lectured at, and then we’d have a three-hour laboratory class at some point during the week or a field experience that would help to apply some of the knowledge that we gained in the passive-learning setting. But, then as you get at higher levels, and things become more theoretical, it really did switch to more of this passive-learning mode. And I don’t want to age myself, but I matriculated a while ago, so I didn’t ever really experience these new active-learning techniques that have become so much more widely adopted nowadays. So, even through graduate school, most of the classes were me sitting passively scribbling furiously to try to take notes as quickly as I could, while a professor lectured and basically tried to stuff as much knowledge into my brain as possible. So, I didn’t really know a lot about the types of things that you can do to engage learners until after I was out of that student mode. But, yeah, geology is cool, though, because you still do have active portions where you get to go on field trips with your professors, and they show you things in the field and you apply that knowledge directly. But, in the classroom, it really was sort of divided, like, “This is your passive lecture that you’re going to sit and listen to, and you may never get called on through the entire semester.” And then “Here’s your lab where you will look at a microscope and look at hand samples or do other types of things that are a little more active.”

Rebecca: What motivated you to do something different in your own classroom?

Trish: As a graduate student, I really didn’t have a chance to do a teaching assistantship. I was on fellowships through most of my PhD time. So, I knew that I was woefully underprepared for entering academia and teaching my own classes. So, as a postdoc, I applied to this call that I saw out by the Center for Astronomy Education. And it was in 2011, they had this course called Improving College General Education and Earth Astronomy and Space Science through Active Engagement. And I saw the ad for this course and I thought, “Oh, this sounds great.” And then I saw that it was three days in Hawaii, and I said, “Oh, man, I must apply to this.”[LAUGHTER] And so I applied and it was mostly astronomy graduate students and postdocs, and the workshop was run by Ed Prather and Gina Brissenden out of the University of Arizona through the Center for Astronomy Education. And they had been doing all this amazing research about how to engage students in 100-level classes, mainly for the idea that they would sort of entrain new majors and new science students. But, it was just a mind blowing experience. I for the first time learned what think-pair-share was, I’d never heard of that before. We did lecture tutorials, I didn’t even know that was a thing. They did all of these, like voting and role playing and these different pedagogical things that I didn’t even know it existed. And they use them on us throughout the workshop. So, we were learning about these techniques through them actively using us as guinea pigs. And then we each had the opportunity to sort of develop a little module. They gave us specific astronomy, like 101-type things, that we would be teaching and we got to teach the other workshop participants and get feedback immediately on things that we didn’t do so well and things that we could improve on. But, it really just blew my mind. I think that was one of the most transformative experiences for me because, up to that point, all the experiences that I had, had been very research focused and how to improve as a scientist and how to improve my research approach, but I’d never had an opportunity to actually learn how to teach and how to teach effectively. So, yeah, I credit that three-day workshop in Hawaii, which was awesome… to be in Hawaii. It’s just sort of changing my entire worldview on how education can be and how I could be a better educator. Had it not been for the Center for Astronomy Education, I don’t know what I would be doing now. So, I think what I took away from it more than anything is that not every student is going to learn simply through lecture… passive engagement… type of situation. And I was fortunate that I seemed to do well in that mode, but it was amazing. I loved that workshop……. It was great.

Rebecca: It sounds really transformative. But, the one takeaway that I hear is next time we want a faculty member to change what they’re doing, we just need to woo them to Hawaii. [LAUGHTER]

Trish: Well, I have to admit that being a postdoc gave me some flexibility in that regard. So, yeah, when I saw that call, I was like, “Oh, I want to do that… three days in Hawaii.” I took my mom with me and she hung out and snorkeled during the day while I was in workshops. It was wonderful.

John: Could you tell us a little bit about some of the techniques that you’ve used in your classes?

Trish: The first semester that I taught, I was given a class that had already been developed and it was sort of easing me into that mode of becoming the head lecturer of a course. So, I didn’t really have a lot of wiggle room to change the curriculum yet, because I was still sort of learning how one gets in front of a class and does things. And so in that first term, I started to use some of the approaches that I had experienced through the Center for Astronomy Education, and sort of trickled them into my class. I use lecture tutorials and think-pair-share a lot during that term. And then I even used some small group activities and jigsawing to try to figure out ways that I could engage the students. And it was sort of a perfect situation to get my feet wet because I had the scaffolding of a well- developed course where I could put in some of my own ideas and try them out and if things weren’t working, I could get immediate feedback from the students and change my trajectory. I was also really fortunate that the students were super kind to me, it was my very first time teaching, I told them straight out. I was very communicative throughout the course. Every time I tried something new, I’d say, “okay, we’re going to try this. I don’t know if it will work, but this is why I’m doing it.” And the students were sort of brought in as collaborators in that process, so they didn’t see me as sort of this professor that was telling them “Oh, you’re going to do this, this and this and just follow along and trust me blindly.” They realized that I was trying to learn how best to teach them and so they were very helpful and when things didn’t work, they’re like, “Yeah that didn’t work.” And then when things went well, they say, “Oh, I really liked that.” And even after that semester, I’d get emails from students. They do a lot of journal reading and science reviews. And one of the students had emailed me over the summer and said “Oh, that really helped. At my first job they asked me to review some literature and I was able to use the template that you provided in class and what we did as groups to do that for my job.” So, I gained a lot of confidence through that process. And then after that first term, I started looking around campus to see if there were faculty development potential to help me to do a better job of developing my next courses. Because while that one had already been developed, I was then sort of slated to develop three new courses, which would be mine and I’d have to start from scratch and really think about how I wanted to develop my teaching as a portfolio. So, one of the things that I really wanted to try was this idea of flipping and mainly it came from a place that I didn’t enjoy lecturing. I would get bored hearing myself talk… like there would be times where I’m up there at the dry erase board writing out things and then suddenly I forget what I was saying, because mentally I’d fallen asleep at that point… like, alright, I’ve been talking so long, I don’t even know what I’m talking about anymore. And I enjoyed the parts where we were actively learning as a group so much more. That was so exciting to me, where the students were doing things hands on, and I could walk around and help them to gain more insight on what they’re working on. So, I contacted the Center for Innovation in Teaching and Learning on campus and looked at the different things that they had available. And one of the facilities that they had advertised on their page was the Illinois iFLEX classroom. So, this is the Illinois flexible learning experience classroom. And I was able to get some training on how to use IFLEX classrooms from Dr. Eva Wolf here on campus. And that also then immediately changed my perspective of how teaching could be because these were classrooms where all of the tables were on wheels. So, you could move them around. They had monitors that students could plug in their computers or laptops, iPads or whatever too so that they could do collaborative learning. And she showed me some of the things that other faculty members were doing in these flexible spaces and it helped me to be inspired to think about what sorts of things I could do. So, as I started to develop my next class, I was like “Alright, I want to be able to use the computing facilities, I want to be able to use these flexible classroom configurations and I really want it to be flipped.” So, the first time I taught a flipped class, I recorded lectures and put them online and naively thought my students are going to watch them and they’re going to do the readings and they’re going to come to class prepared, and I did not have the assessment structured as such that the students had points awarded for doing those things. And boy did I learn quickly that students are not going to be these wonderfully motivated pupils that do all of the things on the list ahead of coming to class. So, I quickly spoke to colleagues around the department and found out about this edtech tool called PlayPosit. I don’t know have you guys had an experience with PlayPosit?

Rebecca: No.

John: That one I haven’t heard of.

Trish: So, PlayPosit is an edtech tool that you can integrate with a learning management system. We use Moodle on our campus. And you take your video lectures, and it embeds questions and prompts within your video lecture. So, students can’t fast forward and they don’t know when these questions are going to pop up. But, it’s a way to assess how they’re doing with the video… with the lecture as they’re watching it. So, sometimes I’ll use multiple-choice questions. In the upper-level classes I mostly use essay questions because I really want them to delve into the topic a little bit more. I also sprinkle in questions from the readings that they’re supposed to do because it’s another way to assess that they’re actually looking at the text or reading the papers that I’ve suggested. And then at the end, it’s great because you can put in some questions about what concepts did you not understand? What do you want to learn more about? Are there sticking points that are kind of confusing you? And this fed directly into learning about the just-in-time teaching method. So, I could have these PlayPosits that the students had to watch before class and I set them for midnight the day before. And I could come in the morning before class, assess how they did on the lecture and immediately I have a lot of information going into the classroom that day for where they’re stuck and I could modify my approach to the learning goals for that day based on how they did on their PlayPosit. And that just changed everything. That made it so much better, and I think from the students perspective, they felt more accountable because there were points that were associated to watching these lectures. And then I would come into class and the first thing I would do is sort of go through the questions and the things that they missed and talk to them about it. And it gave this really nice back and forth. And it sort of broke the ice a bit, because there’s always that little awkward start when you get into classroom, or at least there is for me, and this was an easy way for, say, “Okay, so on the lecture that you guys completed for yesterday, here are some topics that you didn’t really understand. So, let’s go through them together and maybe we can make sure that everyone’s on the same page.” And that sort of changed the game for me for the flipped classroom model.

John: Going back to the PlayPosit, you can also do the same thing with Camtasia and upload the videos as a SCORM package into Blackboard, Canvas, or other things as well.

Trish: Oh, I have to check that out.

John: Once the students arrived in class, you mentioned that you used a just-in-time teaching approach. How did you structure the class? What would your class generally consist of during the class time?

Trish: I originally taught on Monday, Wednesday, Friday for 50 minutes and realized that that was not a long enough time period for us to do what we wanted to do. So, I switched to a Tuesday-Thursday class so I could have a full hour and 20 minutes with my students twice a week. So, typically when students come to class, we have this sort of icebreaker where we go through the lecture material. And sometimes that might take 10 or 15 minutes if there’s a concept that the students really need to get for us to do our activity for the day. And then we usually go straight into a prompt for what the activity is going to be. So, for example, one of the classes that I teach… my favorite classes… junior-level class in volcanology… so, it’s just called volcanoes. And it’s a sneaky class because it’s actually a geophysics class. It’s very math and physics heavy, but I don’t tell the students that when they come, and they do not have an upper-level math requirement to take the course. And this was sort of my sneaky way to entrain students that might not realize that they can absolutely do this. So, I get a lot of diversity in that class: we’ll have communications majors, advertising, education majors, as well as the geology majors. The first time I taught the class, of the 20 students in the class only 4 were geology majors and the other 16 were just spread from throughout the campus. So, it was a really cool opportunity to empower students that “Wes, you can use math and physics and it’s not that intimidating”. So, we go straight into these activities and every exercise is quantitative. They get real geophysical data from deforming volcanoes and active volcanoes around the world, and they analyze it. And there’s a large social sciences component because I want them to think about the societal impacts of those volcanoes and potentials for eruption and how it might impact the communities that are around the volcanoes. And then also that communication thruway of how as we as scientists communicate hazards to local populations. So, they have a lot of different levels of work that they’re doing. Almost every class period is done in a jigsaw manner, where they’re broken into small groups and each small group is going to be working on some component that at the end of the class we’re going to come together and discuss. I typically start the prep of the activity, for example, “Okay, today we’re going to be looking at this type of volcano.” And maybe it’s a stratovolcano. And we’re going to look at Mount St. Helens in the US, we’re going to look at Mount Fuji in Japan, we’re going to look at Ruapehu in New Zealand. But, each group will have a different volcano. And they’re going to look at data directly from that volcano and the surrounding areas and do a small activity that helps them to understand that data set, the type of physical processing, and then at the end of class, each of the groups will come back together and present what they found to the group and then there’s some larger full-classroom group discussion questions that will go over as a class. So, it’s usually the activity I hope will take about 45 minutes, but it really depends. I try to keep them short in my mind, but then oftentimes they go a little bit long if the students get really excited about it. One of the things I think is really good about the flipped class is that I’m able to do so much more than I was able to do in sort of a classical passive learning model. And with that came a lot of grading. And so the first term I did flipped classes, I had not learned about light grading, and was buried in the amount of feedback I was trying to provide to students. So, if we had PlayPosits a couple times a week, we had these activities twice a week, they had additional outside of class things, they had midterms… so, it was unreal, the amount of grading. So, I was very fortunate to find out about light grading, and how to maybe back up the amount of feedback and time I’m spending on student papers and that really helped a lot. So, I think that one of the things that has to be said in conjunction with this particular model is: you need to do some sort of light grading, because there’s no way to stay on top of everything without losing sleep. [LAUGHTER]

Rebecca: Can you talk a little bit about how you adjusted your grading in specific ways?

Trish: Originally, when I did the PlayPosits, again, there are a lot of essay questions. So, I would take a lot of time to really think through the answers to the essay questions and making sure that I’d have a rubric for what I wanted, what points I wanted them to hit on each of these essay questions, and I was very detailed about when they miss things and providing feedback. And so as I shifted into a light grading model, I would do that for the first couple of weeks. And then from then on out, it would just be a quick glance, like, did they hit this? And I wouldn’t spend as much time with the subtleties of “Yes, they wrote out a great answer, and they hit all these points.” And then for just the in-class exercises, the thing I started doing too was originally I had each student turn in their own exercise, even in the groups because I wanted to see their individual contribution. But, I recognize that there was enough individual assessment through the playp osits, and through their midterms, that having that additional individual assessment through the group activity really wasn’t necessary. And it wasn’t really contributing to their success in the class. By having individual assessment on the group assignments, it wasn’t helping students who were falling behind do better. And so after that point, I started allowing the groups to just do their presentations as a Google slideshow, and then I would have their Google slideshow. So, basically, in the jigsaw puzzle, they go into their group, they work on this presentation and when we come back together, each group shows a Google slide or Google doc of what they’ve been working on, and they present it to the entire class. At that point, I just say that’s good enough and I don’t require that each student then hands in the answers to the discussion questions in their own words. So, little things like that made it a lot more tangible for me. Whereas before when I was having each student providing responses for the discussion questions, and then on top of that having the discussion in the class, it was just too much. [LAUGHTER] But, I admit that it came from a perspective that I was concerned that some students would not fully contribute to the group activity, and I wanted to try to hold people accountable, but it really was a little bit too much micromanaging. And I think that the groups ended up holding each other accountable in their own ways without needing me to sit and say, “Okay, everyone needs to answer this question.” The other thing I really like about the small groups is that I’ve noticed that it brings out a lot of discussion from students that otherwise do not participate in the larger group discussions. And one of my favorite things in those small group activities is going around the room… and I typically spend a couple minutes with each group, and I just sort of keep roving around the classroom and it helps me to get to know individual students a lot better. And it also gives them so much more confidence to talk to me. And I feel like it’s made me more approachable as an instructor because they’ve had these smaller group interactions with me where I’ve sat at their table and said, “Oh, that’s a great idea…” or “Oh, have you thought about this” and those just little micro interactions really build up and it creates a student population where they feel more comfortable in the class. And then by the end of the term, I feel like, as a class, it’s much more energetic and engaged. And even in those larger groups discussion, some of the quieter students that you would never have heard from previously are starting to speak out and oftentimes with the encouragement of their group members. That’s another thing I really like about the small group setup.

John: Could you give us some feel for the size of your classes? How large are they typically?

Trish: My typical class size is 20. I usually keep the classes up to 40 students because that’s what the flexible classroom configurations will hold. One of the interesting things about the flipped class is that the first day of class, I do tell the students, these are my expectations. This is a flipped class you’ve signed up for and we go through what that model looks like and I always have students drop after that first day. That’s kind of fascinating for me, maybe it would be nice to follow up and find out…. Did you drop because of the model that was being used in the classroom? Or did you drop because of a schedule conflict? Or were there other things going on? But, I typically end up with 20 students that end up through the entire term. I teach mostly upper-level junior-, senior-level courses. So, I’ve not had the opportunity to try these techniques in the large introductory level classes.

John: I think most of them should scale pretty nicely, except for the grading aspect of it.

Trish: Yeah, I think that were to be done in an introductory class, you probably would want to have some TAs involved as well, just to help. I recognize that other instructors do these amazing small group activities in these large format lecture classes. But, I think having the logistical setup so that you can walk around and interact with groups, maybe not every group every time, but enough so that you can hit most of the groups once in a while, would be imperative because I really think that the students greatly benefit from that almost one-on-one interactivity with the professor.

John: I teach a class typically between three and 420 students in the fall, and I do wander around and I’ve found something similar. I don’t get to sit with each group. But, the students that I do interact with become dramatically more likely to stop by and ask questions, or if they see me in the hallway, to come up and just say hello. So, those individual interactions can make a big difference in practice.

Rebecca: I think it’s just a far more efficient way to give feedback as well. You can disrupt misconceptions and reframe things for small groups. And then if you stop by a couple of groups and hear the same kinds of misconceptions, you can address those more holistically to the whole group. I found that works really well for me, too.

Trish: Yeah, absolutely. And I always get tickled when I see that. I mean, this isn’t necessarily a good thing. But, when there are lots of groups that have the same misconception, because it means that there’s something that… or a piece of information that I have not given the students or something that’s missing in how we’ve set up the activity. And that’s always kind of nice to see and it helps me to redevelop how I’m going to teach it the next time. So, I really do like that. Because otherwise, if I were just lecturing, I would never realize that there was this piece of information that nobody got until the exam comes back and at that point it’s sometimes too late.

John: That’s one of the advantages of a just-in-time teaching approach. It allows you to focus your class time on the things that students are struggling with, and to skip over the things that they already understand. So, it lets you use your class time much more efficiently.

Rebecca: At the end of one of those class periods or even during that class period, I jot down what those things are, so that if it’s a while between each semester when you teach it again you don’t forget what those are because sometimes you can lose track. So, coming up with a system to routinely to check in on those things can be really helpful.

Trish: Yeah, a journaling effort or something. Yeah.

John: And I saw you also do something called Trashcano?

Trish: [LAUGHTER] Yes, Trashcano, Trashcano is an activity that we do late in the term once the weather gets nice. In the class we talked about different styles of eruptions. And one of the styles that we get to later in the class is explosive eruptions andTrashcano is a demonstration that was developed by my colleagues at Colgate: Karen Harpp, Danny Geist, and Alison Koleszar. And they basically developed this experiment where you take a trash can and you fill it up about two-thirds to three-quarters of the way filled with water. And if you submerge a two-liter bottle with liquid nitrogen in it, that bottle represents a pressurized magma chamber and it ends up rupturing because liquid nitrogen is boiling at ambient temperature. And so the two-liter bottle ruptures in that water and creates a column that sprays into the air. So, for this activity, the students do some calculations of plume heights so they can use their iPhones to measure the angle of the trajectory of the water and they can say, “Okay, the plume went this high” and they can do some back calculations to discuss what sort of pressurization caused that amount of uplift to the water. And then we also put styrofoam balls of different sizes and shapes into the trash can. And they can make isopach maps… basically how we actually map explosive eruptions where we take the different grain sizes, and we create a map of how far the different grain sizes spread from the center of the eruption. It’s a fun day outside. This past year, we did it in the rain, which was rather interesting to see how rainfall dampens the amount of distance the styrofoam can spread. I’m not sure that we’d want to do that in the rain again, but it was an interesting experiment. Yeah, we do a lot of little things like that so that the students can take their concepts, the actual equations that we’re working on in class and apply them in a tactile, physical way.

Rebecca: Trish, do you use consistent teams throughout the semester or do you rotate how your groups are formed?

Trish: I’ve done it a couple different ways. I’ve now had the opportunity to teach my flipped classes two to three times each at this point. And some terms I do let them switch around and some terms I keep it consistent. And I’ve found that overall, it works a little better when they’re consistent teams the whole way. I feel like the students build a lot of teamwork and camaraderie with their groups. But, I don’t know… I try to take it by a term-by-term basis because I have had situations where the students are eager to switch around and meet other members of the classroom. We do this a little bit with our jigsaw discussions. So, for example, we do a role playing exercise where each group is a volcano monitoring agency. So, in your monitoring agency, you have a volcanologist, you have a seismologist, you have a geodesist you have a communication specialist, but then all the communication specialists from each group will have to get together and work as a team for one of the activities and all the geodesists will have to get together and work as a team for the activities and then bring them back to their initial group. So, they do get some chances to interact with one another through these, I don’t know, is it a jigsaw puzzle within a jigsaw puzzle? [LAUGHTER] I’m not sure how you describe it, but they do get these opportunities to move around to other groups. But, that’s something that I still am thinking a lot about. I think you had a guest on recently. Dakin Burdick. In his he talked about how sometimes he likes to let the students all which groups all the time because then they get to know everybody in the class, and then sometimes it keeps them all together. It seems like a lot of people do different things with this. I don’t have a great method yet. But, I do tend to go on a sort of term-by-term basis and get a feel for the culture of the class and how people are melding. I do find sometimes when you do the consistent groups, it can happen that the group tends to congeal really well. And then it lifts up all of the students in the group. And so people attend class more regularly, and they’re much more engaged. But, I have seen it happen where groups have sort of fallen apart because one or two members just aren’t attending regularly, and they’re really not committed or engaged. And that becomes difficult. And then you really kind of need to reshuffle a little bit.

John: We talked about that in episode in early October with Kristin Croyle when she was talking about team-based learning where there are persistent teams. And one of the things she suggested is it’s really important to form teams that are constructed to be balanced so that you don’t run into that. But, there are some advantages of having persistent teams. But, if it’s a persistent, dysfunctional team with people missing, then that could be problematic. I think a lot depends on the nature of the activities. If you’re going to have persistent activities like in team-based learning, having well defined teams may be useful, but for other types of activities that vary class to class it may not matter as much.

Rebecca: Yeah, I think there are certainly advantages to both. I’ve had experiences where we’re doing long-term projects. So, doing some preliminary shorter activities with those groups that they’re going to have for their long-term projects can be really helpful. And getting those teams gelled before it really matters. And I’ve also had experience doing persistent teams when I’ve gamified a classroom. And that actually works really well in getting people to hold each other accountable and be competitive. So, I’ve had really good luck when I’ve done that as well. You’ve also received a lot of grants for your research.

Trish: Yeah, I’m in a fortunate position that my primary position is research. So, I don’t actually teach that much. I only teach two classes a year. So, I do try to find ways to integrate all of the exciting research that my group is doing into what we do in the classroom setting, but not just my classes, the other classes in the department too. We teach a 100-level course in oceanography and a lot of my research centers around seagoing expeditions and collecting geophysical data at sea to understand submarine volcanoes. So, we try to bring that experience back into the classroom for our introductory level students. Especially in a landlocked state of Illinois, many of our students have never seen the ocean. They’ve never been to the beach, and they don’t really have a concept for why would scientists be at sea collecting data? And what are scientists doing in our marine setting? So, bringing that into our introductory classes, I think, is really critical. So, the big push there was… I was chief scientist of an expedition we’ve just wrapped up. We had two seagoing missions to the eastern Pacific, it was called the Oasis expedition. And we’re investigating a line of seamounts on the sea floor. So, these are volcanoes that have been active over the past million years, and we were using this submarine to collect data, to collect rock samples from the sea floor, and it started because I have a young daughter and I was going to be gone for about 45 days, and I wanted her to feel connected to me while I was away. We don’t really have great internet at sea, as you can imagine. So, it’s hard to continue to feel connected with loved ones at home. So, I decided with the help of my husband, to create a YouTube channel that would chronicle our life at sea and link back to my daughter’s classroom and some local schools that they could watch what scientists are doing. And then we also ended up using those videos in the introductory courses on campus at a higher level so that students could see a sort of a hands on of what we do when we’re at sea. But, yeah, it started out predominantly as me wanting to stay connected to my then six year old while I was sailing, and became a really great way to provide outreach to a broader learning ecosystem. So, lots of people throughout the community,

Rebecca: I think it might seem more obvious that students in a landlocked state don’t have experience with marine life. But, at the same time, I think that our students don’t have much experience with many professional experiences in what it’s like to be in any kind of industry or research setting. So, I think that that same methodology works in a lot of circumstances to give students exposure to what it might actually be like to be a professional in the field.

Trish: Yeah, and I really like that sort of informal blogging aspect. So, these videos were [LAUGHTER] very informal. I had a blogging camera and I basically just filmed myself doing things. And I remember at one point, my husband sent me this email that basically said, “Wow, you look really tired. Are you doing okay?” Because I just was like, “Alright, it’s 4:30 in the morning, we’re getting ready to do some scientific stuff. I’ll put my camera on me and film what we’re doing.” But, yeah, it’s an exhausting process because it’s a 24/7 operation when you’re out at sea collecting data. You have this facility. For 30, 40 days, whatever it is, and you want to use every second of it to get as much information as possible. But, I think that’s important because a lot of people don’t know like, “Oh, that’s what an ocean scientist….” well, what my particular volcanology centered ocean scientist “…does for research.” And then the other arm of my research program is very much in volcano hazard. So, that feeds directly into the volcano geophysics courses I teach because my group works on developing forecasting mechanisms and algorithms for taking volcano monitoring data and providing monitoring agencies with information about how volcanoes are evolving and we have a lot of monitoring agency partners that we’re working with to try to provide some new quantitative methods for assessing volcanic unrest. So, these are things that we’re thinking about every day, but we certainly can infuse them into classes on volcanology and volcano geophysics,

John: Having that video channel would also let you do some time shifting… where much of the work that you’re doing takes place during breaks when classes wouldn’t be in session. And it still allows you to bring this into your own classes as well. I’ve watched several of your videos, and they’re really good. We’ll share a link to those in the show notes.

Trish: Oh, great… Thanks…. [LAUGHTER] They haven’t been updated for years. But, yeah, I think the asynchronous aspect is really cool. One of the things that we struggled with when we were first doing these expeditions was we were trying to schedule, within reason, because it’s really hard to schedule your ship time because you’re working with all the other scientists that are utilizing the facility, but we’re trying to schedule them such that students could participate synchronously with what we were doing. So, while we’re out at sea we’re sending back Q&As and doing videos. But, what we found was that you could still use all of this information after the fact so students have been benefiting from these videos for the last three, four years, which is really fantastic. And I think that it’s something that a lot of fields scientists could take advantage of. For example, the Antarctic field season is when everyone’s off for holiday. But, perhaps if they’re doing these videos, that they could bring them back and create learning modules for students to see more of what is it like to be a scientist working in Antarctica during the Antarctic summer… and not in a documentary way, I felt like one of the things I really wanted to do was provide that informal feeling for students so that they could look at that and say, “Wow, I actually feel like I could do that. And I could see myself in that role.” Whereas when you have that documentary, shininess, it’s harder to imagine that it’s not this esoteric thing that you could never aspire to be as I wanted to show like, “Yes, we’re up at 4:30 in the morning and we’re tired.” Yeah. I like that.

Rebecca: I think you’re right, that that polish sometimes makes it seem really not approachable to students, or that they don’t belong in the field, or they don’t belong in the discipline. But, if you’re showing that realness in that authentic moment through your own lens, it’s really beneficial to students… and I can imagine this working in just about any context, actually, to help students understand the day in the life that they might be pursuing.

Trish: Yeah, absolutely. It would be so helpful too for K through 12 students, because a lot of times they have no idea. Geology is an interesting discipline in that we’re kind of a found discipline. Students usually come to college thinking, “Oh, I’m going to go do chemistry or I’m going to do physics or engineering…” and geology is not really on students radar, but then they start to see how they can apply chemistry, physics, engineering, and math, all in one discipline, and they sort of gravitate towards us. But, we don’t get a lot of freshmen into our major but maybe if K through 12 students saw what geologists do on a daily basis and what a career looks like, they might say, “Oh, yeah, that’s a major I could be interested in.”

Rebecca: There’s a lot of disciplines and careers that students have no idea exists. The lens in which they see the world is largely through whatever classes they’ve been taking. So, it’s like, world is math, English, science, these really broad categories of things.

John: And they see it from the textbook perspective, as a well defined body of knowledge that they just have to learn or memorize, and not as an active, ongoing endeavor. And those videos you created, and these types of connections that you’re making for students help open up that possibility to them. As part of the OASIS project, you used a variety of social media including Twitter, Reddit, and I believe you did a Reddit Ask Me Anything. Could you tell us a little bit about your use of social media for this project?

Trish: One thing that we learned very quickly is that the internet on the ship was not great. So, the day we did the Reddit Ask Me Anything, it was a day that the Alvin submarine was on the sea floor collecting samples so we knew the ship could stay in one spot. So, it’s sort of like having an aerial antenna on your old TV and you’re trying to like bend it in just the right way so you have a good connection. So, we were able to set the ship in one location and then rotate it [LAUGHTER] so that the satellite was in the right spot so we could get on Reddit, and then we had to like shut down everything using the internet and we all crowded around one computer [LAUGHTER] and did the Ask Me Anything. And I think it was a really good experience. One of the things that cracks me up is that there was a scientist on a sister ship in the northern Pacific that responded to one of our questions and said, “Hey, we’re up here on the RV Armstrong. Hello.” [LAUGHTER] Yeah, I think that they’re Reddit Ask Me Anything was a great experience. It was mostly done by the graduate students on the ship. I was sort of running around doing other things. But, it was a great way, again, to provide outreach. It gives you a demographic that otherwise you may not have interacted with, which I think is important.

Rebecca: Sounds like another sneaky method, like not telling students that your volcano class has math in it. [LAUGHTER]

Trish: Yes. Exactly…[LAUGHTER] Exactly.

John: We always end the podcast by asking, what are you doing next?

Trish: One of the things that we’re working on right now, as I mentioned, we’ve collected a lot of video at sea and on the last expedition, we collected a lot of virtual reality 3D video with GoPro fusion. So, we use GoPro fusions to collect really nice 3D videos on the ship. So, things like how the scientists were cleaning tube worms that were collected from hydrothermal vents and how they’re processing rocks and just the day-to-day life on the ship. So, we collected all this virtual reality video and now we’re working with colleagues and the CITL, the Center for Innovative Teaching and Learning on campus to develop VR modules for introductory classes. And this is been a really crazy learning experience for me because one of our goals is to give these experiences to students that would not necessarily ever have a chance to go to sea or to do this type of work. But, building and structuring the learning goals for these VR experiences is really difficult and I didn’t realize how big of a leap it would be from just the video content and lectures to creating a VR structured activity for students. And there’s some really cool things they’re doing here on campus. The medical school’s been doing a lot with VR techniques for med students and different procedures in VR. And there’s an Archaeology Professor on campus who’s been using it to like simulate an archaeological dig using VR. So, we’re working with some really amazing educators. And hopefully, that will come out with some fascinating modules for our students and upcoming offerings of our oceanography class. But, that’s sort of the big thing that we’re doing right now. I’m kind of excited to see how that will turn out.

John: We talked a little bit by email about you and your husband coming back on later to talk about some of that work. So, for our listeners, we will be revisiting this sometime in the next couple months, I think.

Trish: Yeah, hopefully we have a paper in review right now, where we look at asynchronous linkages to field expeditions and ways that you can collect videos and content while you’re in the field, sort of non-disciplinary-specific, of course we’re looking at marine sciences. But, again, you could use it in other fields and how you can bring that back to produce learning modules for your classrooms.

Rebecca: …sounds really exciting.

John: We’re looking forward to hearing more about that as well.

Trish: Thanks.

Rebecca: Well, thank you so much for joining us. It’s been really interesting.

Trish: Thank you so much for having me. It’s been a pleasure to talk to you guys. And I really appreciated your podcast, it’s been so inspirational to me and my work. I really appreciate it. ‘

John: It’s been a lot of fun for us too. We get to talk to people in depth. Normally, when we gave workshops, we’d hear little bits and snippets of what people on campus were doing, but being able to explore things like this is so much more valuable for us.

Rebecca: And we get to learn about all kinds of different disciplines too which is really exciting. to

Trish: I think what’s so cool is exactly what you said in your hundredth podcast that a lot of times faculty can’t go to those workshops. So, giving them a way to listen to the podcast while they’re commuting or traveling is just awesome. It’s very, very cool. Before going to sea, I always load up my phone and computer with all the podcasts I can get my hands on, because once you’re there, you’re there. That’s pretty much it. [LAUGHTER] Just download the entire catalog.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

108. Neuromyths

Faculty design their classes based on their perceptions of how students learn. These perceptions, though, are not always consistent with the science of learning. In this episode, Dr. Kristen Betts and Dr. Michelle Miller join us to discuss the prevalence of neuromyths and awareness of evidence-based practices in higher ed.

Kristen is a clinical professor in the online Ed.D. program in Educational Leadership and Management in the School of Education at Drexel University. Michelle is the Director of the First-Year Learning Initiative, Professor of Psychological Sciences and the President’s Distinguished Teaching Fellow at Northern Arizona University. She’s also the author of Minds Online: Teaching Effectively with Technology and a frequent guest on this podcast.

Show Notes

  • Miller, M. D. (2014). Minds online. Harvard University Press.
  • Online Learning Consortium
  • Betts, K., Miller, M., Tokuhama-Espinosa, T., Shewokis, P., Anderson, A., Borja, C., Galoyan, T., Delaney, B., Eigenauer, J., & Dekker, S. (2019). International report: Neuromyths and evidence-based practices in higher education. Online Learning Consortium: Newburyport, MA.
  • Mariale Hardiman
  • Tracey Noel Tokuhama-Espinosa
  • Dekker, S., Lee, N. C., Howard-Jones, P., & Jolles, J. (2012). Neuromyths in education: Prevalence and predictors of misconceptions among teachers. Frontiers in psychology, 3, 429.
  • Alida Anderson
  • Macdonald, K., Germine, L., Anderson, A., Christodoulou, J., & McGrath, L. M. (2017). Dispelling the myth: Training in education or neuroscience decreases but does not eliminate beliefs in neuromyths. Frontiers in psychology, 8, 1314.
  • Universal Design for Learning,” CAST website
  • Mchelle Miller, “65. Retrieval Practice” – Tea for Teaching podcast, January 23, 2019.
  • Vygotsky, L. (1987). Zone of proximal development. Mind in society: The development of higher psychological processes, 5291, 157.
  • Michelle Miller, “86. Attention Matters” – Tea for Teaching podcast, June 19, 2019.

Transcript

John: Faculty design their classes based on their perceptions of how students learn. These perceptions, though, are not always consistent with the science of learning. In this episode, we examine the prevalence of neuromyths and awareness of evidence-based practices in higher ed.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today are Dr. Kristen Betts and Dr. Michelle Miller. Kristen is a clinical professor in the online EDD program in Ed.D. Educational Leadership and Management in the School of Education at Drexel University. Michelle is the Director of the First-Year Learning Initiative, Professor of Psychological Sciences and the President’s Distinguished Teaching Fellow at Northern Arizona University. She’s also the author of Minds Online: Teaching Effectively with Technology and a frequent guest on this podcast. Welcome, Kristen and welcome back, Michelle.

Kristen: Thank you so much for having us.

Michelle: Hi, it’s great to be here again.

John: Were really pleased to talk to you. Our teas today are…

Kristen: I’m drinking Apricot Oolong, a green Tea. Nice for the afternoon.

Michelle: And, I have a wonderful hibiscus tea.

Rebecca: And, I have… big surprise… English Afternoon tea.

John: And, I have ginger peach black tea.

We invited you here to talk about the study that you both worked on together on neuromyths and evidence-based practices in higher education. Could you tell us what prompted this study?

Kristen: Sure. As a lifelong learner, I decided I would enroll in a wonderful program being offered at Johns Hopkins University several years ago in mind, brain, and teaching led by Dr. Mariale Hardiman. In one of the courses, I read several articles that looked at the high prevalence of neuromyths in K through 12 education. And, one of the things that caught me by surprise was: One, I was a K through 12 teacher early in my career. I was, at the time, a professor in the School of Education, and in looking at some of the neuromyths, they actually looked like things that I had studied as part of professional development. And, I had not assumed they would be neuromyths. And, so it really intrigued me in terms of: Why is there this high prevalence and why are we not more aware of some of the evidence-based practices that are out there? Not just in the United States, but clearly these were studies that were taking place internationally. So, I decided to start looking at this through the lens of higher education, because that’s where I work and it’s my area of expertise, and I reached out to Dr. Michelle Miller. I was at the Online Learning Consortium conference. Her focus is on cognitive psychology. So, I approached her after the session and told her about this interest in looking at neuromyths within the field of education… really, across disciplines, in trying to see was it similar to what the findings were in K through 12 education, and what was really being done to integrate evidence-based practices into pedagogy or even andragogy. So, we decided to connect and start looking at this. I had a wonderful PHD student who I was working with at the time as well, who is from Armenia, very interested in this topic, and we quickly grew our small group to include a total of ten researchers from the total of seven different institutions nationally and internationally across three countries. And, everybody brought different expertise, everyone from two-year colleges, four-year colleges, public, private. And, we also were very fortunate because we were able to find, really some of the seminal researchers in the area of mind-brain education science, such as Tracey Tokuhama-Espinosa. And, we reached out to the researchers who actually conducted the studies looking at neuromyths like Sanne Dekker, and we reached out to a Alida Anderson who worked with McDonald et. al. in their 2017 publication. So, it quickly grew from a point of interest in trying to identify what was happening in higher education, to really a much broader international study.

Michelle: Oh, and just echoing what Kristen has said here, we first did meet through the Online Learning Consortium, first at a conference and then they set up calls where we got to talk to each other and realize that even though we came from somewhat different academic backgrounds and published in some different areas, we really had this common ground of interest in how do we bring more evidence-based teaching to faculty in higher education and really throughout the world. And, to me, as a cognitive psychologist, it’s just an inherently fascinating question of, even though we live in our own minds, why do we not sometimes understand some basic principles of how the mind and how the brain works? So, that’s just an intellectually interesting question to me. But then it does take on this tremendous practical importance when we start to look at teaching practices throughout the world and bringing that really quality evidence-based design of teaching and learning experiences for our students.

Rebecca: Can you talk a little bit about how, once all of these researchers are now together, how did you put the study together and how was it conducted?

Kristen: I have to say it was not easy. Thank goodness, we reached out to some of the original authors. The survey instruments that looked at neuromyths and general knowledge about the brain. And, what was so interesting is almost all the studies were truly K through 12 focus, so the questions were very different. Even looking at lexicon, “girl and boy,” where we would want to look at male/female. So, we had a look at absolutely every question and make sure that we were able to revise that question within the framework for the lens of higher education. So, it was not an easy process, just in terms of time, because we had to go through so many iterations. And, I think that really helps with the integrity of the research. We had two pilot studies, even down to looking at the Likert scales that we used. One of the things that really stood out was the primary study that we looked at, which was a 2012 study by Sanne Dekker and several other researchers. They had a Likert scale that looked at correct, incorrect and I don’t know. There was a study by McDonald and colleagues in 2017 and they changed it to true and false. So, we decided early on, we would go with true and false. And, when we did that pilot, we ended up with half the participants stopping midway and simply putting, “I’m not sure if it’s true or false…” and they just didn’t complete the survey. And, I think, just looking at how we phrase the questions, it really affected the participation of our respondents. So, we went back, we modified some of the questions based on that, and we change the Likert scale. And, I think being able to have the ability to say whether it was correct, incorrect, or you didn’t know took away from saying it was true or false, because you can base it on knowledge or what you perhaps had been exposed to. And, we ended up having a wonderful pilot making some additional changes. And the feedback that we got, even after sending out the survey, we had a flood of emails saying “Can you please send us a copy of the study, we’re really interested?” So, we really looked at everything. And, I would say one thing that stood out most; and again I go back to the time we spent over two years on this study from point of inception to where we actually send out the survey, collected this study and then published it, was when we looked at the neuromyths, what we quickly realized was we needed to examine evidence-based practices as well. And, we looked at all of this from a metacognitive perspective. The prior studies that were done, looked at what they called “endorsing neuromyths,” and we weren’t so much looking at endorsing, we wanted to look at awareness, because all of us were involved in teaching… professional development. And, so it was a matter of trying to identify what the gaps were, what were instructors, instructional designers, and professional development administrators aware of and, if there is that gap, how could we develop a study where people would say “Wow, I also thought that was correct, but it’s incorrect… but, I would love to find out what the response is and how I can change my knowledge or understanding.” And, so we looked at absolutely everything and wanted to create a study that people would pick up and say, “This is where I am now. Gosh, after going through this in reading the report, this is where I am and my circle of knowledge needs to continue to expand, as things continue to expand through mind-brain education science.”

Michelle: As a collaborative effort, I haven’t been involved really in a study of this scale and scope. And, it’s simply the level of collaboration. You just heard about one of the iterations of the survey instrument that we put together and just how that piece of the study came about. But all the way through the analyses, the writing, it was such an opportunity, even apart from what we were able to share with the rest of the world, just from my own niche piece of the study as well. The opportunity, as a cognitive psychologist, to start infusing what I feel is more attention that needs to be paid to cognitive psychology and learning sciences. The opportunity to infuse that into this field in this area of thinking was also really exciting as well.

Kristen: So, in terms of how it was conducted, we sent the survey out for the Online Learning Consortium. When we originally started, we were just going to look at instructors, we were looking at neuromyth prevalence in instructors because all of the other studies that had been done were primarily K through 12 teachers and pre-service teachers. (although the McDonald study looked at a wider range). Once we started to bring together our team, then we started thinking, “Gosh, well, it’s not simply the instructors. It’s going to be the instructional designers, it’ll be anybody conducting some type of professional development as well because no course is truly an island.” There are so many people today involved in course design, course development and so the Online Learning Consortium was such an amazing partner for us and they touch on absolutely every part of that population. So, we reached out to them early on and said “We’d love to collaborate with you. You’ve got an extensive membership and listserv. Would we be able to develop this survey instrument, send it out through your membership, and ask them through snowball sampling to share it with others who may actually be involved in higher education, in one of these roles.” And, they could not have been a better partner. They’re just incredible to work with. So, that’s how it was conducted.

John: And, we were actually part of that snowball. I sent it out to a list of about 1200 faculty, staff, and professional development people on my campus alone. How large was your ultimate sample?

Kristen: We ended up with approximately 1300 respondents. And, then we actually looked at the full study, we ended up with 929, who met the criteria for inclusion. So, one of the things we wanted to make sure when we looked at the criteria for inclusion that they worked in higher education. You’d be surprised. So many people complete surveys, but they don’t necessarily meet the criteria. Even when you explicitly state you have to be within higher education: teaching or one of these areas. So, we had a total of 929 who met the criteria, and of those they also had a complete 95% of the questions for the neuromyths, and also for the evidence-based practices because we didn’t want to have any gaps. I would say it was an incredible response rate, especially for those completing the survey. They filled out I would say the majority of everything within the survey itself. The respondents were just incredible as well, because you talked about the cross section of participants, but we ended up with really an incredible number of instructors and that was broken down into full-time, part-time, instructional designers, the professional development administrators and it allowed us to run a lot of different tests that we’ll talk about when we look at the findings.

Rebecca: I think one of the things that’s really interesting about how you discuss the setup of the study is thinking about how many different individuals play a role in perpetuating myths, or even perpetuating good evidence-based practices too. That administrators is where funding comes from, so you have to have everybody in the institution on board with what you actually want to essentially Institute.

Kristen: Well, what’s interesting, and you bring up such a great point. One of the top neuromyths out there is learning styles. And, so when you’re looking at learning styles, this is something that almost seems to permeate. It doesn’t matter when you started teaching, whether it’s K through 12, or higher education at some point if you’ve been involved in education, you’ve come across learning styles. Now there are learning preferences and there’s lots of wonderful research on that. But this concept of teaching to learning styles, I think, unfortunately… we talk about this in section seven of our report kind of got mixed in with multiple intelligences. And, that is not at all what multiple intelligence was about, but it was almost the timing of it and so, having been a K through 12 teacher, I remember going through a professional development where we learned about learning styles and how it was something to look at in terms of teaching to learning preferences. And, even to this day when I do presentations, and I know Michelle has run into this as well, especially when we co-teach some of the OLC workshops, somebody will inevitably raise their hand or type in the chat area “Are you kidding? Learning styles is a neuromyth? We just had somebody on our campus six months ago, who taught us how to do an assessment to teach to learning styles.” So, it’s still out there, even though there’s so much in the literature saying it’s a neuromyth. It’s still prevalent within education across all areas.

John: So, you mentioned the issue of learning styles. And, that’s something we see a lot on our campus as well. We’ve even had a couple of podcast guests who we edited out there mention of learning styles and then had a chat with them later about it. I won’t mention any names because they had some really good things to say, but it is a really prevalent myth and it’s difficult to deal with. So, you mentioned learning styles. What are the most prevalent myths that you found in terms of neuromyths?

Kristen: When you look at the report, the first part of our survey had 23 statements. We had eight statements that were neuromyths. If you look at the K through 12 studies, they had many more neuromyths, but we had eight. And, I will tell you, the top five neuromyths in higher education, very closely parallel what you find in K through 12. Now our prevalence is not as high, but it still shows that instructors, instructional designers, and administrators are susceptible to them and that goes back to awareness. So, the top one: listening to classical music increases reasoning ability and that’s really that Mozart Effect. Another one: individuals learn better when they receive information in their preferred learning styles. Some of us are left brained and some of us are right brain due to hemispheric dominance and this helps explain differences in how we learn. So, that’s really that concept of “Oh, I’m right brained. I’m left brained.” And, this again, is something that goes across higher ed and K through 12. Two other really big ones: We only use 10% of our brain. And, if you look at section seven of the report, you will find all of the responses, literally evidence-based practices or research-supported responses to make sure that people aren’t simply saying, “Oh, it’s incorrect. Well, we want people to know why it’s incorrect. So, they can reflect on that and change their understanding, really the rationale and the research behind it. And, then lastly, it is best for children to learn their native language before a second language is learned. This, again, is a big neuromyth. And I think one of the things I’m hoping that will come out of this study, because we talked about this really when we go into evidence-based practices, is this concept of neuro-plasticity, the fact that the brain changes every time you learn something new. When you’re engaged in an experience, the brain is changing. And, sometimes the brain is changing at a cellular level before you might even see that change in behavior, and so we’re able to see now through technology through f-MRI through fNIR so much more than we were able to see before. So, really keeping abreast of what’s happening in the research should be informing our practice because we have more information available than ever before. But, somehow we need to get that into our professional development training, seminars, and workshops or into the classes that we’re teaching in our schools of education or into our onboarding. But yeah, these are the top five neuromyths in terms of susceptibility, and they cut across higher ed and K through 12.

John: In your paper, you also provide some crosstabs on the prevalence by the type of role of individuals, whether they’re instructors, instructional designers, or administrators. Could you tell us a bit about how the different groups due in terms of the prevalence of these neuromyths?

Kristen: Well, the one thing I will say is, everybody is susceptible to neuromyths, so it wasn’t as if there was one group, and I know that’s always in the back of someone’s mind, “Gosh, who’s the most susceptible?” Well, we didn’t find any significant differences, and one of the things that we wanted to do as well was to really be break the participants down and look at other factors. So, when we look at full-time versus part-time faculty, is one group more susceptible to neuromyths. And we found no significant difference in terms of gender, in terms of age, in terms of working at a two-year institution, a four-year institution. And I really think that talks to the amazing reality of the opportunity to integrate professional development in looking at the learning sciences and mind-brain education science in the opportunity to decrease that gap. So, it wasn’t one group over another. But it’s everybody who has this opportunity to increase this awareness across all of these areas.

John: Didn’t you also find that some of these myths were less common among instructional designers relative to faculty,

Kristen: We found with evidence-based practices, when we looked at significant difference with evidence-based practices, instructional designers actually had in terms of percent correct, higher awareness of evidence-based practices. It wasn’t a large difference, but there was a significant difference and Michelle can certainly talk to this point as well. But, this is really the importance of having an incredible team when you’re looking at course design, course development, and part of that may have to do with, when you look at instructional design, there is so much new literature and research that’s getting infused in to that area, and so that may have something to do with it. But, I think there’s lots of additional studies that we could do to follow up.

Michelle: Kind of circling back to the point of the design and delivery of instruction in a contemporary university or college is fundamentally more collaborative than it was in prior eras. And, so I think we definitely need to have everybody involved start to really break out of that old school mold of class is identified with the teacher who teaches it and that’s what a course is. And no, courses reflect, today, everything from the philosophy and the support that comes down from the top to the people that the students may never meet, but who put their stamp on instruction such as instructional designers. And, this is something that I get pretty fired up about in my just practical work as a program director and just being involved in these things in the university, that there are still faculty who you say, “Hey, do we have any instructional designers who are working with us on this project to redesign? Is anybody assigned to help us as we develop this new online degree program or something?” …and you sometimes still get blank look.? Or you get “Oh, aren’t those the people who you call when the learning management system breaks down and that’s their specialty?” I mean, this report, I think, just really hammers home that idea that instructional designers are a key part of this collaborative team that goes into really good quality higher education instruction today. And it isn’t just about the technology. I think that they’re getting exposure to and staying abreast of what’s going on in research that relates to teaching and learning. And, what a great opportunity for faculty to not just rely on them for technology, but to learn from them and to learn with them as we build better courses together.

Rebecca: Can you talk a little bit about the awareness that you found in general about evidence-based practices? So, we focused a lot on the neuromyths, but what shook out when you started looking at the evidence-based practices?

Kristen: Well, one thing that stood out was awareness was much higher. And, that’s really exciting. I think that’s a huge testament to the professional development that we are offering. But, there were still gaps in areas where there certainly could be a lot of improvement. So, a couple of examples that I’ll give because we literally spent months looking evidence-based practices, and we wanted to make sure that we could support them. So, for example, when we look at percent correct, where most individuals across all three groups were not as aware, like “differentiated instruction is individualized instruction.” So, we know that this is incorrect. But most of the respondents did not put that that was an incorrect statement. So, they either stated it was correct, or they didn’t know. So, again, this is an area that we certainly want to explore. Because differentiated instruction is something that really, I think, adds to the classroom. And, there are other ones. For example, we’ll look at Universal Design for Learning. So, one of the statements we had in there actually comes directly from the CAST website, and it says “Universal Design for Learning is a framework to improve and optimize teaching and learning for all people based on scientific insights into how humans learn.” Well, the instructional designers, they were the most aware. So, 87% of them got that correct. Of the professional development administrators 74% got that answer correct. For the instructors, 58% got that correct. So, you can see the difference in the responses and when we share this nationally or internationally…. when we talk about the study, you’ll have a lot of individuals who’ll say “No, universal design for learning, that’s about accessibility.” Well, it certainly is about accessibility. But, most importantly, it’s about learning and how humans learn. It is probably the most dynamic and the most powerful aspect that we can add into pedagogy or into andragogy. But just by looking at the data here, it may not be something that everybody’s aware of, and that’s again a great opportunity to integrate that into professional development. So, there are a number of things. I mean, it’s exciting because when you look at it, there are 28 statements. And, as I mentioned, overall, the awareness was much higher across all three groups, compared to neuromyths or general knowledge about the brain.

Michelle: Just to jump in here, again, from my kind of cognitive psychology perspective, those evidence-based practices that we’re talking about also include, specifically, some items that are related to memory, a topic that’s really close to my heart. So, I think those are just fascinating as well. So, for example, we asked a variation on a classic question that many cognitive psychologists have looked at: “whether human memory works a lot like a digital recording device or a video camera.” So, is your memory basically taking in information that’s in front of you? And, here again, we’ve got 69% of our instructors saying, “Oh, yeah, that’s right. That’s how it works.” And, that is not how it works. 79% of our instructional designers identify this as an incorrect statement and 74% of our administrators, and we have a few other related things such as we asked people whether testing detracts from learning. And, as Tea for Teaching listeners know, that goes to retrieval practice. Testing doesn’t detract from learning, testing builds up learning. So, these are some as well that I think it’s very interesting to tap into what people know and really think about while these maybe seem like inside baseball, or very metaphorical or philosophical questions, if I’m an instructor, and I believe these things, that students are basically just running video cameras in their heads… well, that is going to lead to some different practices. I might be very puzzled as to why I got up and gave this lecture and the students eyes were pointed at me and yet it didn’t end up in memory. So, those are some of the items that I was particularly interested to see when we got all the numbers in.

Kristen: You know, I would say one thing: when anybody reads the report, what we want them to do is look at how it’s presented in terms of the tables, because everything is looking at the percent of correct or accurate responses. So, as Michelle said, when we look at “human memory works like a digital recording device,” 69% of the instructors got that correct. 79% of the instructional designers got that correct. And, 74% of the administrators got that correct. So, that means we still have a fairly large percentage, basically 20 to 30% that either got the answer incorrect, or they didn’t know. And, even looking at these responses, do they actually know why they knew it? Or did they guess or did they make that assumption like, “Oh, that’s got to be right.” And so, really, the intentionality of this study was awareness, really bringing out statements from the literature to help anybody who’s involved in teaching, course design, professional development to look at these questions, and really think “Do I know this?” And, “If I know it, how do I know this? Is it based on some type of research or literature? Could I defend that? If I don’t know with certainty, where do I find that answer? And how can I learn that? And, how can I integrate those practices?”

John: On the day when your report came out, we shared that on our campus to everyone on our mailing list. One of the nice things about the report is that it has all the questions and also provides references for the answers explaining why the specific answer is true or false. And, it’s a really great resource and we’ll share a link to that in the show notes. It is long. When I shared it two people sent back email saying “Maybe we should use this as a reading group for next semester.” And it’s not a bad idea, actually. But, much of that is appendices and so forth. And, it’s a really informative document. I believe in your survey, you were asking people about their participation in professional development, and you looked at the relationship between participation in professional development and the prevalence of these myths. Is that correct?

Kristen: We did. So, one of the things that we wanted to look at was trying to find out if educators were involved in professional development, whether it be neuroscience, psychology or in mind-brain education science, did that actually increase their awareness of neuromyths, general information about the brain, and evidence-based practices? And it did. We found that that it was definitely a predictor and it was found to be a significant predictor and so, for us, again, it looked at what a wonderful opportunity to be able to say that training does have a positive impact. And, that was really the crux of the study… and it’s interesting, you talked about the length of this study, because originally we had thought about doing two different or three different studies. So, we do one on neuromyths, one on evidence-based practices, one on professional development. Then when we brought the data in, the question was: “Do we separate them out into three different long articles or three different reports?” And, we collectively, across all disciplines said, “No, we need to bring them together.” Because first and foremost, it’s about awareness. You can’t really talk about evidence-based practices, until you’re aware of what the neuromyths might be. What are some of the fallacies that you might actually believe? What are things about the brain that you may or may not know? And, once you’re there, and you have that understanding, you can then move into the evidence-based practices, because it’s all really connected. So, when Michelle talks about memory, you can’t really talk about memory without having some understanding of the mind or the brain. And, so we decided collectively, we would bring it together as hopefully a seminal piece that would really present anyone with a continuum as to: “Where am I? What am I possibly doing in my classroom?” …being able to really do that self assessment and then find the answers, as you said, in that section seven, and realize that they’re not an outlier. I mean, chances are anybody that goes through this is going to fall within that span in terms of their understanding and knowledge.

Michelle: And, what I hope is coming out here is that this study is unusual, not just in its scale, its scope and that we focused on higher education, but that it is so explicitly geared to not just identifying gaps in knowledge or awareness, but addressing those. It’s not like we came along six months later and said, “Oh, by the way, here’s a really nice resource we put together.” It is one stop, it’s right there. And, what an exercise that was, as well. Kristen, I think you’ll remember back just saying, “Okay, in a paragraph… this item, all of us look at this and go ‘oh my gosh, that’s wrong’ or ‘that’s right.’ Why is that? and what are the very best empirical sources that we will trace back to, to demonstrate that?” So, we are trying to provide that and also to really be a model to say: next time that you get that handout or that workshop that says, “Oh, here’s some great stuff about the brain.” What are they backing that up with? Can you trace it back to the solid research sources that makes some of these really powerful principles for learning, and make other things just misconceptions.

Kristen: One of the things that I would say was probably the most exciting and the most challenging. We had 10 researchers, we had 10 researchers from different fields: people from nursing, biomedical engineering, psychology; we had people who work in the area of neuroscience, education (as I mentioned), and we needed to come out with a collective voice, writing a report that would be understood across disciplines. And, so when we wrote section seven, all of us had to be reviewers and we vetted it multiple times. Not just within our group, but outside, to make sure when you read about neural pathways, it actually made sense. Because to write something where somebody would not understand or not be able to connect would be a challenge. And, we wanted people to walk away. I know one of the things that we were looking at: Why neuromyths? Well, a lot of the research out there looks at the fact that when you teach, your teaching and your pedagogy is based on your knowledge, and in your understanding of how people learn, and so we wanted to really look at this area in terms of awareness, because it may impact pedagogy. Our study did not do that. And, I want to make sure it’s really clear. Our study was not designed to say, “Oh gosh, the awareness of neuromyths wasn’t very high in this area, therefore, you must be integrating neuromyths into your teaching. That was not the intentionality of our study and that’s not something that we’ve ever said. There are certainly recommendations we put in the study to look at. If there is a high prevalence of neuromyths,how does that affect pedagogy? But ours was simply looking at awareness and could professional development address gaps? So, we could do this across all different groups that would be involved in course design and delivery.

John: That’s one of the things I really like about it, that you do address all these things well, you provide the evidence, and it’s going to be a great go to reference for those of us when faced with neuromyths, with issues about evidence-based practices. We can just go and grab some of the citations and share them back out or refer them to the whole document as I’ve done several times already. These things are really common even in professional development. I was at a session not too long ago, where there were two neuromyths presented during the session. One was the learning styles thing. But the nice thing is, unlike other times when I’ve seen that done, there were two of us who went up and waited until everyone else talked to the presenter. And, we were both ready to do it after other people had gone so we didn’t embarrass her, but it’s starting to get out there. And, I know on our campus, we’ve got a growing number of people who are aware of this partly because of the reading groups we’ve had, where we’ve had a growing number of participants… and that all started actually with Michelle’s book about five years ago now when we first did the group. You came out, you visited, people wanted to do more, so we started a reading group. We’ve done four additional reading groups since then. We’ve had many of the same participants, but it’s spreading out wider. I’m hoping we’re making a difference through these reading groups.

Michelle: And, that’s so gratifying as an author and as a researcher, and I remember well working with your group in Oswego and the great ideas I took away as well. So, I’m a big believer in virtuous cycle. So, maybe we’ve started one.

Kristen: I think what really came out of this study is the passion that everybody has for student success. Everybody from those that are offering the professional development, the instructional designers that want to make sure that the students are successful, even though they might not be teaching the course. And, then the instructors themselves… and so to be able to work with that many individuals who are not only subject matter experts across their disciplines, but so passionate about making a difference. But I think being able to integrate all of this new research relating to neuroscience, psychology and education, it’s going to transform not only how we teach, but it’s going to transform pedagogy, andragogy, and this whole concept of learning.

Rebecca: I really appreciate the bringing it together and that you decided to keep it all together and not to make three separate reports. I think it’s actually really important to understand how these are all connected and related. And, I think that’s one of the most unique things about the report. I think the community is probably very grateful that we have this resource available now.

Kristen: Oh, thank you.

John: One of the things I’ve often been concerned about is how some of these neuromyths, particularly the left brain – right brain thing, and the learning styles belief, often serves as a message to students that they can only learn in certain ways or they only have certain types of skills, and they’re not able to make progress in other ways. And, it can serve as a barrier and can lead, perhaps, to the development of a fixed mindset in students which may serve as a barrier.

Rebecca: …or not even allow those students to feel like they can enter particular disciplines.

John: If people become more aware of this, perhaps it could lead to more opportunities for our students or fewer barriers placed in the way of students.

Rebecca: …or maybe even just more inclusive pedagogy in general.

Kristen: You bring up such a great point. So, if you believe in learning styles, and you believe that you are truly a visual learner, Michelle and I’ve talked about this a lot, it almost becomes a self-fulfilling prophecy. But you probably are an incredible visual learner because you’ve been told you learn better in this learning style, so you’re going to seek materials in that learning styles. So, the challenge with that, especially when you’re looking at younger students or anybody during their education, you’re precluding really other ways to enhance your learning. So, when you look at Universal Design for Learning, it’s so important because you’re looking at multiple means of engagement, representation, action, and expression. And, when you’re looking at learning styles, if a student believes they’re a visual learner and suddenly asked to go in and take a Spanish oral exam, it could trigger, all of a sudden, stress. Well, what do we know about stress? And, Michelle can talk more about that. But, when you’re stressed, it affects working memory. And, so just that thought of, “Oh my gosh, it’s an oral exam. I’m a visual learner. How can I perform well on that?” And it’s really creating, as you talked about, a barrier or it may decrease, possibly, performance. I know that Dr. Tracey Tokuhama-Espinoza is very passionate about this. And, you’ll see in her presentations, she’ll come out and say “Neuromyths do harm.” And so, I think it’s certainly something that needs to be explored. And, Michelle, from a psychological point, I’d be curious to find out what you have to say as well.

Michelle: When you say “self-fulfilling prophecy” and things like that, it also kind of reminds me of a placebo effect, in a way… and learning styles, and continuing that as an example, yeah, I might go: “Oh, visual learning. It is absolutely me,” like “Now I feel like I can tailor all this to myself. I’ll just find teachers, opportunities, and disciplines that are right there in visual learning.” And, I might have some subjective impression that that’s helping me, or from the teacher’s perspective, I might feel like “Well, I brought in some different materials and engaged different modalities and, what do you know, because of learning styles, we’re doing better.” Well, there’s lots of different reasons why that might be happening. An individual may walk away, and maybe they weren’t individually harmed. I just feel like… just like in modern medicine, there’s sort of a promise that we can do better than mere placebos. I think that ought to be the promise of modern pedagogy as well, that we can do better than simply trying to build up expectations or giving people a false sense that they have something based on science that’s going to help them individually do better. And, I hear so many kind of missed opportunities that really kind of get me activated as well. I think about, for example, the energy that goes into faculty professional development. These things come from good impulses. I really believe that. I believe that people who really pursue something like learning styles or things like that, they want to do better and they want to be more inclusive, but that effort is directed down the wrong path simply because of this gap in knowledge and gap in information in getting the right information to the right people at the right time. And, I can’t stand the thought of faculty, especially as limited as faculty time is and as spread as thin as faculty are, to think that they might try to pick up on some better information about teaching and learning and go down the wrong path. I never want that to happen again. And, maybe our report will be a step in the right direction.

Kristen: I’ll say one thing that we’re trying to do with the report, is really to align the report with best practices and evidence-based practices. So, when you look at the concept of neuromyths the wonderful study that was written by McDonald (and this was in 2017) and her colleagues, the title is “Dispelling the Myth: training and education in neuroscience decreases but does not eliminate beliefs in neuromyths” and so professional development is not a silver bullet. Simply offering one workshop that’s going to address neuromyths is not going to necessarily get rid of neuromyths. So, we have to do what? We have to look at spacing. We have to look at interleaving. So, with professional development, how do you take information related to evidence-based practices and integrate spaced practice into our own professional development? How do we integrate interleaving? How do we integrate low-stakes assessment? So, maybe when faculty or instructional designers come in, you do a quick self assessment and find out what that baseline knowledge is, and then at the end to say, “Okay, at the end of professional development, we need to get to 95% or higher.” But, they’re able to actually test their own knowledge. So, we need to kind of turn professional development upside down and make it active learning and really engage everybody in what we’re looking at within pedagogy and andragogy.

Rebecca: Yeah, I always find it really ironic that a lot of training and things on evidence-based practices is not using evidence-based practices… or using really traditional formats: lecture or getting lectured at and not really engaging with the material. And, it’s no different when we’re working with our students. And, if they’re practicing in a way that’s not going to be effective for them, and they’re not successful. They could spend tons of time on something and just not really make progress. The same thing can happen with our faculty and staff who are designing curricula and what have you as well. They can be really invested.

Michelle: Absolutely.

John: We do have an excellent podcast on retrieval practice. In fact, it’s one of our most popular episodes. We’ll share a link to that in our show notes. We don’t yet have any podcasts on interleaved and spaced practice, but I’m sure we’ll be asking Michelle to come back and talk about these things at some point in the future, if she’s willing. So far, we’ve been focusing on the types of neuromyths that are common. What can we do to reduce the prevalence of these neuromyths?

Kristen: Professional development is certainly key. But, I would look at things such as onboarding, making sure that when people are getting hired on, that they’re really introduced to evidence-based practices from the very beginning. And, even individuals that would say, “Gosh, I’ve been in instructional design for 20 years, I’m familiar” …there may still be those gaps. And, it’s almost like adaptive learning. Everybody that comes in very much like the Vygotsky’s work of zone of proximal development, they may have all been teaching for 20 years, but it doesn’t mean that we don’t have neurodiversity in terms of experience, knowledge about different practices. So, it’s important that it’s from the very onset of when people get hired and making sure it’s understood that we’re committed to best practices, evidence-based practices and what we do builds upon the literature and the research. Not only do we introduce it here, but we move it forward and integrate it into our pedagogy and what we’re doing in our classrooms.

Rebecca: So, we always wrap up by asking: What’s next?

Michelle: Conference season is upon us. We’re recording this fall of 2019. I’m gearing up to go to the Online Learning Consortium’s Accelerate conference in November. And so, I will just personally say come find me if you’re there and you want to talk more about this. I will be presenting on a related but different topic having to do with our ongoing Attention Matters project, which is also the subject of another Tea for Teaching episode. So, I’m really working on getting ready for that, and also the upcoming POD network conference. So, for those educational developers who will be attending that, I’ll be speaking there and hopefully having lots and lots of sidebar conversations with plenty of other people who are interested and fired up about these very topics. So, I/m working on those. I’m working on what I will now call a forthcoming book. It’s under contract with West Virginia University Press, tentatively titled Remembering and Forgetting in the Age of Technology. So, maybe someday in the not too far off future, we’ll be talking about that project as well.

John: We should note that this podcast will be released during the OLC conference. In particular, it’s coming out on Wednesday of the conference.

Kristen: Oh, that’s exciting.

John: And, I should also note that we’ll be presenting there as well. I’m hoping we’ll get some people to listen to this podcast because we’re presenting the next day. So, we might get some new listeners. [LAUGHTER]

Kristen: Oh, that’s exciting. In terms of projects that I’m engaged in and working on. We’ve just launched a new lab in our School of Education at Drexel University. So, we’re bringing everything together and trying to align projects coming up for 2020. But it’s a lab called ELABS, Education, Learning, and Brain Sciences Research Collaborative. So, we’ll be looking at different studies related to the learning sciences and mind-brain education science. I am wrapping up an article with several researchers at Drexel University, some of our PhD students, that looks at immersive virtual reality and practice as well as transfer of learning. We also have a report that I’m working on. It’s an update to research that I conducted earlier on online human touch. So, I’m wrapping up that study and putting together an article there. And, then also looking at two publications for books looking at neuro plasticity and optimal learning. One would be for students to really understand neurodiversity, neuroplasticity, how you can optimize the stress response, and then looking at neuroplasticity and optimal learning from the instructor or instructional design perspective. How do you integrate this into your practice? So, those are the initiatives that I’m working on.

Rebecca: Sounds like lots of things for all of us to look forward to.

John: Thank you very much for joining us. This was a fascinating conversation. And, we’ve been looking forward to this report since I first heard a bit about it when you initially did the survey, and when I saw a preliminary presentation at all see last year.

Kristen: Well, thank you so much for having us. It’s such a pleasure to discuss this topic with you. And, I’m looking forward to listening to many of your upcoming podcasts that clearly is connected to this report.

Michelle: Thank you so much. It makes all the hard work worthwhile and we love the opportunity to get the work out to exactly the people with the power to spread it to faculty and instructional designers and leaders in universities today.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

107. Project NExT

Faculty beginning their teaching careers often rely on the teaching methods that were inflicted on them when they were students. These practices are not always consistent with evidence on how we learn. In this episode, for Assistant Professors from the Math Department at SUNY-Oswego join us to discuss how our math department is transforming its instructional practices through the use of professional development opportunities provided by the Mathematical Association of America.

Show Notes

Transcript

John: Faculty beginning their teaching careers often rely on the teaching methods that were inflicted on them when they were students. These practices are not always consistent with evidence on how we learn. In this episode, we examine how one department is transforming its instructional practices through the use of professional development opportunities provided by its national professional organization.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today we are joined by four assistant professors from the Department of Mathematics at SUNY Oswego. Our guests are:

Sarah: Sarah Hanusch.

Rasika: Rasika Churchill.

Jessalyn: Jessalyn Bolkema.

Zoe: And I’m Zoe Misiewicz.

Rebecca: Welcome everyone!

John: Our teas today are:

Rasika: I’m having Earl Grey.

Jessalyn: I just poured myself a cup of lemon ginger.

Sarah: I’m not having any tea today. I’m not much of a tea drinker.

Zoe: I’m not having any tea today either. I just haven’t unpacked to that point yet.

Rebecca: And I have… English afternoon.

John: And I have Bing Cherry Black Tea. So, we invited you here to talk about Project NExT, which is something that people in our math department have been involved with. Could you tell us what Project NExT is?

Sarah: So, Project NExT stands for New Experiences in Teaching. It’s a program that is sponsored by the Math Association of America that brings new mathematics faculty…so you have to be in your first or second year of a full-time job…but they bring these new mathematicians in from all over the country to teach them about active learning.

Rebecca: How did your involvement, or the department’s involvement, with Project NExT get started?

Sarah: I learned about it as a graduate student, and was highly encouraged by a lot of people to apply. And so I kind of brought it into the department by saying, “Dear Department Chair, will you pay for this?” And since then, in part because of my starting it, we’ve encouraged everyone we’ve hired to apply. And as a result, there’s now five members of the department that have either completed or are still in Project NExT.

Jessalyn: Yeah, I will echo that experience. It was something that I was aware of as a graduate student, in part because some of my mentors had gone through Project NExT…it’s now 25 years old…just celebrated 25 years. And so for me, it was something that I knew I was interested in. And in fact, when I visited Oswego for a campus interview, and the department said “Oh, yeah, we have Project NExT fellows on the faculty, and we would be happy to support you in that,” that was a really exciting and encouraging thing about the department.

Rasika: For me, actually, I didn’t heard about that before. But, when I got the job offer, it came with that. I said “Yeah, sure.”

Zoe: I was just hired this past year and so I’m doing Project NExT, but I think I can already see the effects that it has had. It was a program I already knew about, I really wanted to participate in. So, as I was going through the hiring process, one of the first things I would ask the chair at a place was “Would you support an application for Project NExT?” …because it does require a bit of funding. And so seeing that there were already multiple Project NExT fellows in this department was also a good sign for the department as a whole when I was thinking of what sort of department I’d want to be at. And so I think it’s just showing that it’s already been recruiting people who are interested in it already, at this point.

Sarah: I was just going to clarify a little bit about how the funding for it works. There’s actually no fee to participate in Project NExT. The way it’s organized is that you attend special sessions at three of the national conferences in mathematics. So, you attend two math fests in the summer, and then the joint math meetings, which is in January. And so these are big nationwide meetings in mathematics. And so the idea is that you’re going for some special sessions during the meetings. And then your first year, you go for a couple days pre-conference for the really heavy duty workshop. So, the financial commitment from a department is just the funding to go to those three conferences.

Rebecca: You mentioned active learning. Can you talk a little bit more about how those workshops and things are structured?

Zoe: There were a lot of workshops about active learning and just using evidence-based pedagogy, so saying not only active learning is good, but we have evidence to support it and here are some of the things that you could do in terms of active learning. And all the sessions obviously are structured with that in mind. So, we’re not just sitting there listening passively to someone tell us about active learning, but they really make sure you’re doing something, whether it’s a fun little game like building a marshmallow tower, or some other interactive activity in each session. The sessions aren’t only about active learning, there’s a lot about inclusivity and diversifying the profession. So, a lot of sessions on that, or maybe I just chose sessions on that. But, there’s also a whole professional development stream. So, there’s stuff about how to get started in your career in terms of grants and so on. It’s really a lot of everything in there.

Rasika: It’s categorized like if you interest on the tactile learning, so are you interest on the group work, are you interest on some other…you know, inquiry based and mastery grading and so forth. So, depending on your interest, actually, they give more opportunity to listen, go talk with people and have a conversation: what they had, what they tried and what failed and what succeed. Which is like a really nice thing for us, as a beginner, to see what people have gone through and what I should expect, and so forth. Actually, I was interested about the whole program.

Sarah: So, they do some three-hour breakout workshops where you get to go based on what your interests are. So, I did one that was focused on teaching future educators because that’s my background, but I doubt any of these other ladies chose that same session because that’s not their expertise and not what their job is going to be about fundamentally.

Jessalyn: I will add, I attended two workshops that stand out to me in retrospect. One on making active learning intentionally inclusive. That was all about inclusive pedagogies and ways to incorporate group work in the classroom in a way that benefits all students and allows all students to participate fully. I also did a longer breakout workshop that was building a toolkit for student-centered assessment, that was all about learning objectives and exam structures from a more experienced instructor. And then there are also facets of Project NExT that extend well beyond the physically meeting in person. So, as Rasika mentioned, there are lots of ways that you can navigate the workshop according to themes that are of particular interest to you. So, if tactile learning or kinetic activities are of interest, or you’re really focused on educating future teachers or whatever that might be, you might be encouraged to declare a goal for yourself in your first year related to one of those areas of interest. And then we’ve got little email exchanges that go on for people who’ve declared interest in one of those goals like “this email list is all about mastery based grading, check in when you’ve tried something. check in with your questions.” So, there’s a little bit of accountability built into that structure that these people know what you’re trying to do, and they’re going to check in with you on it. But, then just the larger structure of email lists is that you have this cohort of other new instructors who will fire off questions like, “Oh, I’m teaching this class next semester I’ve never taught before, what textbook might I use?” or “I had this really strange interaction in my classroom, and I’m not sure how to handle it” or “I think this part of my syllabus is just crashing and burning. Help! Has anyone been here before?” And so you have this sort of communal resource and the community experience of brainstorming and problem solving together.

Sarah: …and included in that they assign each of us a mentor. So, a more experienced instructor that’s a mentor is assigned to each person in the program currently, and it’s always someone that is outside of your department. In fact, they will not allow anyone to be a mentor who has a fellow in their department. So, as long as we keep having fellows, we won’t have any mentors here. But, what’s nice is when you do send emails out on that list of “I’m trying this and it’s not going well, help!” you do get responses from your peers. You also see responses from all of the mentors for that cohort, which I think is also valuable because sometimes they have a little more experience than your actual cohort.

Rasika: We have a group that people who are interested on the inquiry based or tactile work, they have their own little Zoom conversation whenever they have time together. You get to know all different schools, what they’re doing and, you know, share your experience.

Rebecca: Would you all like to talk a little bit about how Project NExT has influenced your own teaching?

Rasika: For me actually, I was really interest on the tactile experience from this Project NExT. So, I decided to do some activities this semester starting as a beginner and also some group work. And also something that… not exactly what I’m getting from the Project NExT, but it’s like I will say, part of the SUNY Oswego Reading Group, that I was so interested on the book that we are reading. And I decided to give a couple of pages for the students every week to read, and I assigned them 5% for the final grade that they have to read and write half a page to one-page report for me and tell me what they think. Do they think like it’s feasible for them to change and try and do the things in that nature? So far, it’s really going well, and I have good comments from students saying that “you are opening up different ways of thinking…that we were stuck and never complaining about everything. But, we are now having, you know, in a broader way of looking at the things about growth mindset and so forth.” So, I was speaking here and there like chapters from some interesting books. So, that’s what my experience so far this semester, as a beginner.

Sarah: I think for me, it just gave me a lot more lesson plans and ideas to draw from. I already had a pretty active approach to my teaching, but it just opened a broader view of what kinds of things could work well. Especially in some of the more tactile things available that can be helpful for helping students to learn.

Jessalyn: Within my own teaching, I think it’s been really easy or natural to draw on resources from Project NExT in setting up my class or setting up lessons. When I taught Calc I, on day one, we made zip lines out of ribbon and key chains and measured average velocities and it was fun and it was memorable and it got students working in groups and they reported at the end of the semester. “Hey, remember when we did zipline? That was fun!” and I 100% would not have pushed myself to do something that involved or non standard, I’ll say, without thecontext of Project NExT saying “Oh, just try one new thing each semester.” I completely overhauled the Calc II class to be entirely mastery based grading in response to some of my own frustrations with how I had been setting up my class. And Project NExT supplied a whole lot of resources, a whole lot of people, a whole lot of information and motivation to try something like that, which I think was helpful. As far as department culture goes, I think the fact that we’ve had this many Project NExT fellows and continue to have Project NExT fellows gives us a shared language to talk about teaching. Some shared frame of reference on “Oh, yeah, you know, this person who tried this technique,” or “Have you heard anything about…. “”…Oh, hey, this came through on my Project NExT list.” That I think has encouraged just our conversations about teaching and being intentional in how we’re structuring our classes, or how we’re handling things.

Sarah: I’m experimenting with mastery based grading this semester because of the information you and John got, from your experience in Project NExT. And so your experiments with it last year has led me to experiment with it this year. So, it definitely has changed just how we even hear about new things to try.

Jessalyn: That’s delightful. I appreciate that it’s trickling around.

Sarah: It is trickling for sure.

Zoe: So, I’d say it’s still obviously fairly early. We’re only one month into my first semester after going through the first part of Project NExT. But, I’d say a lot of it has been both an affirmation of things that I have been doing and also it’s sort of given me the confidence to do the things that I was doing even more fully and to advocate for these approaches, even though I am brand new in this department. So, I’m not afraid to send to the whole department email list like “we need to be more positive toward our students and not say that it’s all their fault if they’re struggling. we need to take responsibility for that.” Or just to try things that may or may not work well. For example, I’m doing mastery-based grading just of the homework in my general education math course. And I’m using an online system that,it turns out, is not that great for mastery-based grading of that course, even though I’ve used it for other courses. Students, I think, still benefit from it, but it’s not quite as effective as I might have hoped. But, I’m just willing to try these things and willing to speak up about things, so those are the main impacts in my courses.

John: Could you tell us a little bit about how you’ve implemented mastery learning technique?

Sarah: I think we’ve all done it a little different. Why don’t you start, Zoe, since you were just talking about it.

Zoe: I’ve done it only in the homework, so not in their exams. So, the homework is done online, it’s 15% of their grade. And so for each little subtopic, they have to do a little quiz. It’s five questions: three medium, one easy, one hard, and they need to get at least 90% on it. And they can try as many times as they want, but they do have to keep trying. And so, in courses like college algebra…is the one that’s most similar to where I’ve done it before…the material all builds on itself and it divides nicely into little component and there, I’d say it’s going well. The students complain about it at the beginning, but already after I asked them to reflect on their first test performance, a lot of people said, “Oh, it’s actually really helpful that I had to go back and keep learning these things until I fully understood them.” Whereas in the first couple of weeks, there’s always a bit of pushback about “Why do we need to get 90% on this. It’s too hard to get 90%…couldn’t it be lower?” And then once the results come in, they see it’s worthwhile. The other course I’m doing is similar, the gen ed math course…it’s also their online homework…15% of their grade, but that textbook just doesn’t break down the material into as nice sections and the questions are longer and the grading of the online system is pickier. So, that one has some issues, but the same basic idea.

John: Are you using publisher provided questions then, and tools?

Zoe: Yeah, publisher provided questions and tools.

John: Are you allowing unlimited attempts or is a limit on the number of attempts?

Zoe: Yeah, unlimited attempts, and flexible deadlines too. So, I do say they need to achieve a certain amount before each of the test. But, the idea is that if you haven’t yet mastered something, you can still go back and do it several weeks later. As you keep practicing the material, we keep building on it. So, it’s not that you have just one chance and you’re done. The goal is to get them all to understand it fully by the end of the semester.

Jessalyn: My approach to mastery based grading in my first implementation was to go totally off the deep end, and just structure the whole class with a mastery-based grading scheme. So, what this meant was that I did away with midterm exams, everything was broken down into learning objectives roughly correlated to the sections that we were intending to cover in the textbook. And the primary mode of assessment was quizzes. So, my students had quizzes that they could retake as many times as they needed to. And each quiz had three questions and I wrote problem banks of many many questions for each quiz. And in order to earn an A at the end of the semester, the expectation was something like, “Oh, you need 18 of your quizzes to be three out of three and the rest of you two out of three.” So, it was not a points accumulation scheme, it was just quizzes and repeated quizzes. They also had online homework through web work and that was unlimited attempts. There were deadlines, and they just needed to… there was sort of a threshold percentage associated to an A or a B, or a C. And then I had a few more other activities and elements going on. But, primarily, the structure involves these mastery quizzes. And I owe a great deal in the structure of this class to Laura Taalman from James Madison University, who shared a lot about how she structured her class that way and so I sort of borrowed and adapted from her setup for my experiment.

Sarah: So, my class is pretty similar to Jess’s. The main difference is I’m doing it in a proof-based course, so it’s fewer questions. She had three questions per objective. I have one, because they’re a little bit longer questions. The only exam in my class this semester is the final and that’s only because I’m required to have some common questions on a final exam. So, I had to have a final exam…instead I’m doing weekly quizzes. Each week, we add one to two new objectives. There’s about 20 for the entire semester. So, our first week we had two questions on the quiz. The second week, we had four questions on the quiz, but questions one and two were the same objective as one and two from the first quiz. So, the questions are just going to grow cumulatively…so our last quiz will have about 20 questions on it. Although I did tell them once everyone has mastered a question, it’s just going to say mastered, it’s going to be no new question writing and at some point, I’m going to recycle some of the early ones.

John: Your building in some interleaved practice and spaced practice as well.

Sarah: But, the idea is that once they have mastered a question, they no longer have to do it again. They’ll have the questions for practicing and for getting ready for the final. In addition to these mastery quizzes, I’m having them write a portfolio, which is going to have a little bit more of that interleaving practice and making sure that at the end of the semester, they still remember how to write some of these early proofs and it’s also to focus on the writing aspect. So, to help make sure they’re really using the language precisely. Sometimes with a quiz when it’s timed, you’re a little more flexible, but I want to make sure that they have that precision of language down by the end of the semester. So, I’m sort of balancing those two aspects of it that way. They have “unlimited attempts” in air quotes…restricted by what? …there’s 12 times I could quiz during the semester…13 for something…So, restricted to…they need to do number one all semester long. They can have all semester to do it, but we are eventually going to run out of time.

Rasika: So, for me, I haven’t tried to mastery based grading yet. Maybe in the future.

John: Are there any other new techniques any of you have used in your classes?

Sarah: I’ve done a lot of experiments with this idea of embodied cognition, where you actually have students sort of using their bodies to experience things mathematically. One way that we did this with my pre-service elementary school teachers, I give them a bunch of clothesline, and I have them make a circle. So, you may think, “Okay, no big deal.” But, what happens is, it’s not good enough until it’s a perfect circle. Part of this is to elicit the definition of a circle, because to non-mathematicians, I’m going to pick on you for just a moment, Rebecca, how would you define a circle?

Rebecca: One continuous line that’s in a loop.

Sarah: So, a lot of times they come up with something like that. Well, how does that distinguish, though, a circle from an oval. So, it’s not really a precise definition of a circle, right? With the precise definition is being it’s all of the points that are a fixed distance from the center. But, what happens is, by forcing them to make their circle better and better and better and better, they actually all know that’s the definition of the circle. Maybe they don’t remember it, but they know that there’s this radius thing involved. And so by not allowing them to sort of quit until they actually are in a perfect circle, the only way to do that is you have someone stand in the center, and you take another piece of clothesline to measure your radius, and you move everyone in and out as appropriate. So, that activity of physically making the circle and by having to have that person in the center, and that radius gets them to say the definition of the circle properly, first of all, but they get to experience it in a way that they don’t get to otherwise. And that’s an activity that I never would have thought of without going to Tensia Soto’s session at my first Project NExT meetings.

John: It is certainly safer than giving them all compasses with sharp points where they can stab each other, which was how people used to do it.

Sarah: We still do compass and straightedge constructions in geometry, but again, that doesn’t actually help you really understand what the definition is. I think doing this physically actually helps them understand why a compass works. I know that sounds silly, but it really helps make those kinds of connections. I have another activity where we take clothesline and I make a triangle on the ground, and I make them walk the interior angles of the triangle and you spin 180 degrees and it, again, helps them experience that the sum of the internal angles of a triangle is 180 degrees. And again, that’s something that, the first time I did it, it was baffling because first of all, it’s hard to turn the interior angles. Your instinct is to turn the exterior ones, but you end up backwards. From a geometry standpoint, it makes sense, but somehow that physical aspect just really changes things.

John: It makes for a much more memorable experience, where they’re seeing things from a different perspective. And I think that’s really useful.

Sarah: I agree. That’s why I do it.

Rebecca: Does anyone else want to chime in about how having so many fellows from Project NExT has influenced the larger department? Because you’re not just five people in your department, you’re how many?

Sarah: There’s 14 of us tenure/tenure-track now. I do think it’s changing the way some things are done. It’s slow going. I think everyone would concur with that. Jessalyns’s smirk is definitely confirming that. It’s slow going, some of us would like change to happen faster. But, I do think change is happening. I think there’s a lot of respect from our colleagues that we are trying new things. I think a lot of them have a “You can do what you want, but don’t make me change yet.” But, I think we’re starting to get them a little bit, too.

John: If your students are more successful, that often convinces people and sometimes when students say, “I did this in this other class and was really helpful,” that’s often really persuasive to other faculty. But, it’s convenient that you had so many people all come in at once, because that’s not typical in most departments that have such a large cohort, in a short period of time.

Sarah: We have had a lot of retirements, one right back on top of each other. So, we have had an influx of young faculty in our department, which…that alone…to have so many in this program as well. Definitely.

Rebecca: I think it really helps to have models of ways that you can do things because if you didn’t learn using these methods or you didn’t have exposure to that as a student, you have no way of knowing how those really play out unless you have examples. So, it sounds like Project NExT played that role for you, but then you are playing that role for other faculty in your department.

Jessalyn: Thinking about department culture more broadly, not just among discussions and relationship among faculty, but in terms of the student experience, and this engagement that we’re having from our majors and the sort of activities that we’re involving them in. I think there has been a Project NExT influence there as well. Sarah, you and John started the Putnam Competition before I came even and a lot of other conversations and gatherings have come out of that, like we’re getting together with our majors and talking about preparing them for graduate school if that’s something they want to do. The math club or other organizations have taken on a different role in the department and I think a lot of that comes out of some of the ideas in Project NExT, like hearing about how another department celebrates their students participating in something like the Putnam Competition. But, it also comes out of the relationships you build in an active learning classroom and the way that we connect with students when we are trying new things. And we’re being honest with them and saying, “Hey, I’m trying something new. And I’m going to want your feedback.” The community that you build in a classroom flows into the community that we support and foster as a department.

Zoe: So, it’s a bit hard for me to talk about departmental culture change in the one month that I’ve been there not having seen it before I did Project NExT. But, I can certainly talk about how the department seems different from other departments, just in the willingness to embrace new ideas. And there’s also a sense that these ideas are just supported. Even if we haven’t had an explicit conversation, I know that there will be support for trying something new that was suggested in Project NExT. And it seems, when it comes time to make policies, that we have almost a majority just of Project NExT people. Obviously, we need a couple more people, but there are other people who haven’t participated in the program who would still support these sorts of initiatives. Knowing that that base of similar views is there, makes a big difference in what sorts of ideas we would even suggest or consider.

Sarah: I think a lot of our Project NExT fellows have also been very active with doing undergraduate research with students.

Rasika: I think even like talking to colleagues. For me, like I have a personal experience, because my husband is also a mathematician and teach at SUNY Oswego. If I learn something new, I share with him of course, he’s not a Project NExT fellow, but…

Rebecca: So, it sounds like the program’s working really well. You’re all really excited about it. It sounds like it’s engaging all of you. So, glad that you’re able to share it with us.

Sarah: The MAA has definitely done a lot to support improving teaching in mathematics and I do think it is a program that other disciplines could look at and possibly model. I will say they have put a lot of money and a lot of investment into making this a success. It is well run and has been well funded, which is a testament to how important professional organization views it.

John: We always wrap up by asking, what’s next?

Sarah: Well one thing that’s next is we’re trying to get one of our other new faculty, his application was rejected last year. We’re also hiring two people, hopefully this year…So, possibly trying to send them next year as well.

Jessalyn: Another immediate thing that’s next is that our two current NExT fellows will be attending the joint math meetings in January and maybe organizing some Project NExT sessions or at least attending some sessions.

Zoe: I’ll be helping to organize a session on getting started in math education research, which I was made part of because I said it was something I wanted to do, but it’s not something I have any background in. So, I’m finding it a bit of a challenge to assist in this organizational process. But, I also, possibly for Math Fest next summer, helping organize a session on reducing math anxiety, which is something that a previous NExT fellow who I follow on Twitter help organize this session. So, having attended NExT, I think, gave me the confidence to respond on Twitter to this senior mathematician and say, “Oh, yes, I’m interested in this topic.” And so that will come later. And that’s something I actually feel like you could contribute to in a meaningful way, unlike math education.

Rebecca: Well, thanks so much for joining us. This has been really interesting.

Sarah: It’s our pleasure.

Jessalyn: Yeah, thanks for having us.

Zoe: Thank you very much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Brittany Jones and Kiara Montero.

102. Team-Based Learning

A large body of research finds that active learning approaches result in larger learning gains than traditional lecture approaches. In this episode, Dr. Kristin Croyle joins us to discuss how she transitioned from  explore using interactive lecture to collaborative learning, and then to team-based learning. Kristin is a Psychologist and our new Dean of the College of Arts and Sciences at SUNY Oswego.

Show Notes

  • Johnson, R. T., & Johnson, D. W. (2008). Active learning: Cooperation in the classroom. The annual report of educational psychology in Japan, 47, 29-30.
  • A discussion by Dan Ariely explaining why asking for shorter lists of positive features in a relationship can engender positive feelings appears in this March 24, 2014 video clip.
  • Michaelsen, L. K., Knight, A. B., & Fink, L. D. (2004). Team-based learning: A transformative use of small groups in college teaching.
  • Team-Based Learning Cooperative
  • Epstein Educational Enterprises, What is the IF-AT?
  • Sweet, M., & Michaelsen, L. K. (2012). Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement. Stylus Publishing, LLC.
  • Croyle, K. L., & Alfaro, E. (2012). Applying team-based learning with Mexican American students in the social science classroom. Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement, 203-220.
  • Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251-19257.
  • 74. Uncoverage – David Voelker – Tea for Teaching podcast episode discussing the uncoverage movement in history, March 27, 2019

Transcript

John: A large body of research finds that active learning approaches result in larger learning gains than traditional lecture approaches. In this episode, we explore one faculty member’s transition from using interactive lecture to collaborative learning, and then to team-based learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Today our guest is Dr. Kristin Croyle. Kristin is a Psychologist and our new Dean of the College of Arts and Sciences at SUNY Oswego. Welcome, Kristin.

Kristin: Thank you.

John: Our teas today are:

Kristin: Earl Grey

Rebecca: I am having Mama’s work tea, because Ada made it this morning and she calls it work tea, which means she pulls the tea bag tag out and puts it in the big cup. Also, it’s just my normal English Afternoon. But, that was a better story.

John: And I’m drinking Spring Cherry green tea.

Rebecca: We invited you here today to talk about collaborative and team-based learning in your classes. But before you do that, you’ve noted that you had a strong preparation in teaching before you got started. Can you talk a little bit about that. We’ve talked a little bit about that on the show before and how a lot of faculty aren’t prepared…

Kristin: Um hmm.

Rebecca: So, could you talk about how your preparation may be informed what you’ve done.

Kristin: My graduate program, I went to the doctoral program in clinical psychology at the University of Montana in beautiful Missoula. And that program takes the preparation of their grad students very seriously, but across several areas not just in clinical work and research, but also knowing that some of them are going to end up in positions in which there will be teaching. So, while I was there, that very first semester I was brought in, they had a structure for teaching their introductory psychology classes where graduate students were assigned our own classes where we were the instructor in the classroom, but we had a supportive network around us. So, the syllabus was already there, the textbook was already there. We collaborated in writing tests. We had a structure of TAs that supported us and they would have recitation sections in which the TAs also received development. And we joined in that so we could see how more hands-on kind of things could be done with students in smaller groups. We even assigned our final grades together. And some of those pieces are pieces that are areas of skill that people don’t often think about developing. So, that first semester, all I had to do was think about working within the structure: How am I going to handle the day-to-day teaching and learning in the classroom? I didn’t have to worry about course design because the course was already designed in front of me. And I also didn’t have to, at the end, think: “When you assign grades, is that rigid? Do you really have to follow the exact, you know, 90/80 that it is in the syllabus? Or what if there are natural breaks around 88 or 89? Is it okay to flex that? What kind of power does an instructor have that is fair to students and evaluation?” I got to do all of that in a collaborative setting with a very experienced faculty member as a guide. There was also a credit-bearing course for teaching psychology that we were encouraged to take… which I really enjoyed. And then I was given opportunities to function more independently. When they needed a stats teacher over the summer, and they knew I was living there over the summer, I got to teach on an adjunct basis, but still with the support of faculty around me. So kind of putting students in the deep end, but with a high level of support around them, I felt very prepared when I was done with the graduate program to enter into an assistant professor position. And I still appreciate the preparation that they gave me.

Rebecca: I think with the preparation like that you’re probably far more willing to experiment and do new things as a faculty member too and to maybe even break away from what faculty around you are doing. Do you find that to be the case? Or were there other faculty doing some of this collaborative work in the department that you were in?

Kristin: Yes, and no. One of the experiences I had at my previous institution, which was the University of Texas – Pan American that then transitioned through a merger to be the University of Texas Rio Grande Valley. I was talking with a colleague in another department about the kinds of things we were doing in the classroom. And I still remember him saying “Oh, I didn’t know you could do that in the classroom and that was like teaching.” He had a very restricted idea of what teaching was, and what would be acceptable to colleagues, which he had never had the opportunity to test with other people around him. And that was something that I arrived from day one… that you talked about your teaching… that you can do many different things in the classroom. And it’s all teaching, as long as you are trying to work with students to create a learning environment and they are learning, then it counts as teaching. So I did come in with a much more flexible idea, then certainly some of my colleagues who hadn’t had an opportunity to ever have those discussions. And of course, some people are hired into departments in which those discussions don’t ever happen, so they may persist with those misconceptions for many years.

John: Or throughout their entire careers at times. [LAUGHTER] The scaffolding that was provided is really nice, because we’ve talked to a few people who’ve been in teaching training program or had some training in graduate programs. But usually, it’s not quite as structured as that and that’s a nice feature.

Rebecca: Yeah, I came from a program like that, but it was like very front loaded. It wasn’t that ongoing…. So I felt a lot more prepared, because I did have a lot of those experiences, but I didn’t have that same kind of supportive network throughout. Which is incredibly valuable.

Rebecca: So, you want to take us through what some of your collaborative experiences have been in the classroom and the ways that you set up some of the team-based learning exercises, maybe starting with what are those?

Kristin: Sure. So kind of the way that I journey through my teaching, particularly when I was an assistant professor, I felt comfortable in the classroom, but I didn’t feel expert. I felt like I was still trying to figure out what was going on, which is a perfectly fine way to be and a good state for learning to occur. So I felt like I was a talented lecturer, like I can engage students. I teach in psychology, I also think psychology is naturally very engaging, but part of that is because I really love the field. So, I felt like I could engage students and that they would listen and that they would be interested. But I started to become dissatisfied that there was always a core of engaged students and I had no idea what was happening with the other students in the class. And then sometimes I would be disappointed when we have tests or homework. Everyone said they had no questions. Clearly that was wrong. I was wondering how do you engage the majority or all of the students in their learning so that they aren’t coasting through class believing that they understand until they really don’t. And then I also felt like I was kind of fooling myself into thinking that students were with me when they were not with me. So I had an opportunity at that time to do some intensive cooperative learning training along the model of Johnson and Johnson collaborative learning. And that model from the University of Minnesota, it focuses on the importance of cooperation in the classroom, and that in cooperative settings, students learn more, develop a stronger sense of self efficacy around their learning; that they together are able to achieve more than they would individually. And it also has impacts on retention… that if students are feeling like they are individually known and valued in the classroom by their peers, they’re more likely to continue showing up to class and to develop relationships outside of the classroom that supports them along the way. So through that training, it was intensive, it was like eight hours a week, one day set for several weeks. The very first day, I could see what a difference I was going to make in my classroom. So, for example, I was using group assignments in class and they had all the same disadvantages that group assignments and most classes have, because I had no idea how to structure the group work so that it would be successful. I was doing group work to save me grading time, honestly.

Rebecca: That’s why a lot of people go to group work.

Kristin: Yes! Without understanding that all I had to do was some structural changes, and then it would actually be effective for learning as well, instead of just saving me grading time. In that cooperative learning training, I learned how to structure intensive group work that could be the length of an entire semester, or it could be the length of a single class day. I learned how to structure less intensive moments of team time. So how do you do a think-pair-share that works versus how do you do a think-pair-share that doesn’t work very well. So, that within the course of that training… actually just within a few days… I suddenly had, instead of 10% of the students in the class engaged on a daily basis, I had 100% of the students engaged on a daily basis. So, that was a huge breakthrough and I continued that way for several years.

John: What were some of the structural changes that you made that did lead to increased engagement?

Kristin: So, the cooperative learning approach of Johnson and Johnson, is kind of theoretically heavy, in the sense that they outline the pieces that are necessary for strong collaboration to occur. And then they turn it over to you as the instructor to say, “How do I build those pieces in?” So, for example, they emphasize positive interdependence as one of the essential components of cooperative learning… that when you create a group and a group activity for them to do, the activity has to be structured in such a way that each person is necessary to contribute. You can’t structure it in such a way that you can have three people talking when one person is only needed, and there are specific recommendations on how do you structure it so that everyone is needed. At the same time you have to build an individual accountability as another required component, so that, even if each person is needed, people can still slack off, say, “Yes, you all can’t do as well without me because you need me, but I don’t really care about what is happening here.” There has to be a level of individual accountability that’s also built in. Along with that, some of the skills that I thought were most important, they build an emphasis on group processing and social skills, so that if you have people consistently working together in class, they may not have developed the social skills to do that effectively, especially over time. You can work with someone for two minutes on a think-pair-share and really be bad at social skills. But, if you have to work with them over an extended period of time on a project and things are going south in terms of group conflict, it’s the instructor’s responsibility to help them to develop the social skills to work together. For example, on the first day of class, when I first start having students talk to each other so that they know that’s going to be a pattern in the class, I give them something quick to talk about. And I say introduce yourself to the person next to you… spend two minutes talking about this. And then I’m going to ask you about what you talked about. And then I run around the class real quick… pair up people who aren’t participating, introduce them to each other so that they understand this is a part of the class. So, then I follow back. So, what pieces are important there? …that I explicitly instruct them, you turn with your body… you actually make eye contact. And I will point out as people first start doing this, look at these two people, they are looking at each other, because many times students won’t do that, and it’s very hard to have a cooperative interaction if you don’t make eye contact… and I will say, “Who was the person you talked to? Tell me their name.” So they understand that I was serious when I said, introduce yourself, tell me something about them and that there’s individual accountability through just random calling on… that they need to participate in the cooperative portion. And then there’s also the self-reinforcing aspect of it that five minutes later, when I say to talk about something else, they realize they already know somebody in class, they have a connection. The next day, when I come in, they’re not quiet, they’re already talking to each other, they’ve created those connections.

John: A nice thing about that, too, is for people who are uncomfortable talking about themselves in class, having one person tell you something about the other person, it’s a little bit less pressure, it’s a little less revelation to the whole group. There’s some evidence that that type of thing is more effective in providing a more comfortable environment.

Rebecca: Kristin, can you also talk a little bit about a specific example of a cooperative activity where all of the members are held accountable, and all have a role? …just to provide an example for people who have less experience.

Kristin: So cooperative learning can be divided into informal and formal cooperative learning. Informal cooperative learning tends to be much shorter activities that can be done kind of on the fly if you already have an idea in your mind of how you might want to do that. Formal cooperative learning tends to be more intensively structured… longer-term activities. So that could be a single class session. If you’re going to do an activity that takes an hour, that would be more formal… or if you’re going to do something that takes an entire semester. The pair-and-share that I just talked about is an example of informal cooperative learning. Something like a jigsaw classroom activity can be structured as a formal cooperative learning activity. And it already shares almost all of the components: there’s individual accountability, because each student is given a specific role. There’s also positive interdependence, because the success of everyone depends on each person doing their role. So there are ones that are already structured with a built in component. The pieces that aren’t built into something like a jigsaw classroom activity, would be the group skills and group processing, and the ways that you can build that in. You can, for example, ask groups to reflect on what went well. I typically emphasize that more than asking them to reflect on things that went poorly, because asking to reflect on what went well tends to maintain a positive atmosphere, but also helps them to cover both bases at the same time anyway.

Rebecca: …or realize that my list for what went well is not very long… [LAUGHTER]

Kristin: Right. So, a common group processing thing I would have students do after their first more lengthy or more formal cooperative learning activity would be: list three things that your team did well together and one thing that you could improve on. And another thing I might ask them to do is to provide positive feedback to each member of the group at the end of the activity. And the kind of feedback that they provide is usually pretty specific, and helps to shape their behavior throughout the rest of the semester. So when they say things like, “I like it when you disagreed, and you said that this other thing would be a better way to go” that provides important feedback, and it helps to encourage better processing going forward. But I will go around and give individual social skill feedback too. But it’s usually things like, “Oh, I see you’re sitting so far away from your group, I’m not sure they can hear you, let’s scoot your chair in so that they can hear you.” Or I might ask, “Oh, do you know this person’s name next to you? What’s her name?” …and we’ll make sure that people maintain the social and cooperative connections that enable to do that kind of good group.

John: Just as an aside, it’s useful if you’re asking about things that went well, to keep the list fairly short. I’m reminded of a study that Dan Ariely talked about where they did a controlled experiment where in one case, they asked people to reflect on three things they liked about their partner and another case to list 10 things I liked about the partner, and then they surveyed them on the quality of the relationship. And those who were asked about three things generally rated their relationship with a partner fairly high. But when they were asked to come up with 10 things, they struggled with that and they rated that relationship lower. So keeping the list short…

Kristin: Right.

John: …is really good so you don’t…

Kristin: There’s kind of analogous thought about keeping things like gratitude lists. If you list too much stuff, it can have a negative effect, because you start to identify things that you really don’t think are that important, and it makes you think the whole thing is less important.

John: And if you want to get the opposite effect, ask people to list 10 things that were bad, and then they’ll struggle beyond the first few. You talked about having continuous relationships or persistent relationships with collaborative learning. Did you try to keep the group relationships consistent for the same groups throughout the term? Or did you vary that?

Kristin: I varied it. There are some good data to suggest that in collaborative learning… they refer to them as base teams… that base teams have a persistent positive effect, particularly on things like student engagement and retention throughout the semester and throughout the year…. that you have a team that is expecting you every day. But when I was doing cooperative learning, I didn’t restructure my courses. I restructured the day. Does that make sense?

Rebecca: Um hmm.

Kristin: So I didn’t have a reason for base groups. And I felt strange imposing them on the students without a reason. Besides, they would maybe be socially a good idea. I had to completely rebuild my courses from the ground up before I started using base teams. And that’s when I transitioned to team-based learning.

John: …and in team-based learning, persistent teams are recommended as part of the process.

Kristin: Absolutely.

John: Could you tell us a little bit about this transition to team-based learning. What prompted you to introduce that? …and how it worked?

Kristin: So I was happy with how courses were going. People were interested and engaged. I had students telling me, “I know every single person in this classroom.” and when you’re teaching a class of 30, or 40, or 50, that’s unusual. “I know everyone in here, I feel really supported.” I feel like things were going well. But I was unsatisfied with what I was teaching. I wasn’t clear, in my own mind, about what persistent learning outcomes I wanted for my students. I had not sat down and really thought through if I were to follow up with a student in a year or five years, what would I want them to recall from this class? What would I want them to be putting into use in their lives or in their careers? I had never thought that through. And I was fortunate enough to run into team-based learning at that time, right as I was primed to start thinking about this questions. Team-based learning originated by Larry Michaelsen. He was coming from the perspective of enrollment increases. He had been assigning some pretty challenging work. He was a faculty member in business. And as his course enrollments increased, he started to wonder how can you maintain the same kind of interesting, really challenging in class… by case work, for example… with a large enrollment. So he developed team-based learning to address that piece, but it also requires you to completely rethink the design of the course. And to start from the course outcomes: “What do you want the persistent outcomes to be?” …and then structure the course forward in that way. So in team-based learning, after you make a decision about your course outcomes, and what you really want students to be able to do, then you structure the course in a modular fashion. And each module has certain steps. So the beginning is student preparation, then when they come into class, you test. You say, “okay” …and it’s called the readiness assurance process. So you want to know what students are ready to do after they’ve individually prepared, and what they’re not ready to do. So they prepare, they test. And then, since it’s a team focus, they also test as a team. After that you have a good idea as an instructor, what are they ready to do? What are still the fuzzy areas? What do they really not get at all? What are their competencies as a team already, even if every individual student doesn’t have it, and then you can do some corrective lecturing, basically, so many lectures that fill in some of the gaps. And that’s all part of the readiness process, because you’re getting them ready to do some interesting application work in class. And the rationale for that is… and actually what I had been doing prior to that, was giving interesting application material to work on at home individually, while doing lecture and cooperative learning in class. But the interesting application material was actually the heart of the course, and the much more challenging piece. So it was better to bring the hard piece where they needed support into the classroom. And the piece they were ready to do, which was to do their own self study back into their own lives. So you do this readiness assurance process to make sure they are ready for interesting application, and then the majority at the time for the module you spent on application. Doing that after I had already worked with cooperative learning was really helpful, because all of that application work is done in a team setting. So when you already have some experience with how to build teams, how to maintain and develop their social skills, that’s really, really, really helpful. That’s a short version.

John: One of my colleagues, Bill Goffe, who was on one of our very early podcasts, noted that when he gave the group test, the performance always went up significantly, so that they could see the benefits of the peer discussion that was part of that. And he was really impressed with it. And he noted that, oftentimes, if a student didn’t show up for class one day, they get a hard time from their classmates from the group because they let the group down. And he said his attendance had never been better than when he was using a team-based learning approach.

Kristin: Absolutely. And a lot of people who do team-based learning, use the same methodology for doing the team testing, which is honestly really cute. It’s a scratch-off form. And the scratch-off form is used so that the team gets immediate feedback on each option. So on any particular item in a multiple choice test, if they want to select “B” they scratch off “B.” If it’s not there, then they continue to scratch until they get the right answer. For one thing, they love it. But also they are getting immediate team feedback. If this person is not speaking up, if they say I think it’s “B” and then they stop advocating and then it turns out to be “B” later than the team immediately knows, by the time they get to the next question. “Okay, we need to incorporate more feedback from all of our team members, wait a minute, this person who’s not speaking up actually has a lot to say.” In the course of just a few multiple choice questions, it brings their team development forward leaps and bounds. And they kind of have fun with a scratch off, which is also a bonus.

John: And it also gives them incentives to come prepared and to listen to other people in ways that they might not otherwise.

Kristin: Yeah, and their team will give them grief, if they say “Oh, I don’t know, because I didn’t read,” their team members will be like, “But we are depending on you, you need to read, we all read.”

John: And it also gives them a little bit, perhaps, of improvement in metacognition because they’re getting that immediate feedback, and it’s being coupled with the reactions of the peers. So if someone was insistent on a wrong answer, and they dominated that discussion, they might be a little more careful in the future and more willing to listen to the other people and reflect.

Kristin: Exactly, and it doesn’t have to wait till next week, it can happen right away. Right on the next question. The team application activities are also structured in a particular way. In team-based learning, they talk about the four S’s for the application activity, the first one is that you have to select a significant problem. So what they’re working on is something that will be important to them, something that they will identify with, or that they recognize is worth their time in thinking about and trying to think through. The second one is that they need to be working on this same problem. You can’t say teams one and two are working on this, three and four are working on this, five and six are working on this. Third one is that they structure in so that they make a specific choice as the outcome. Because it’s easier to solicit team feedback if everyone is making a specific choice rather than having kind of an open-ended narrative response. And it helps to stimulate whole group discussion as you’re moving. Now it can sound like it’s limiting to say that you have to make a specific choice, but you can do in a very broad way. And the fourth one is simultaneous reporting. So all the teams are asked to report at the same time on what the choice was that they made, so that they can’t piggyback off another team who’s putting in effort. So, as an example, one of the courses that I taught in the psychology major in Texas was the tests and measurements course in psychology, and test and measurements starts with a stats review. They’ve all had statistics, it usually comes prior to tests and measurement. But it’s the first time that they have an opportunity to work with statistics in kind of a decision-making way. So you start with a stats review. So one of the activities that I would do, I gave them two hypothetical first-grade teachers with how many questions 10 of there students got right on a spelling test. And the two distributions had the same mean, but one was fairly normal, and one was highly skewed. So they had to do their quick statistics review… Do the mean, median, mode and standard deviation describe the shape of the distribution. But the question I was asking them was, “If you were the principal, which teacher would you offer an after-school tutoring program to for extra pay? And which teacher would you potentially nominate for a teaching award?” They found that question to be a really interesting question. For one thing, students think a lot about what good teaching is, and what constitutes a good teacher. So they already come in with very strong opinions. And they also understand the complexity of, you know, if everybody’s passing but people aren’t excelling, is that good teaching? Whereas if most people are failing, but a few people are getting an “A” is that good teaching? …and how the data contributes to good decision making, but can also be kind of manipulated to contribute to decision making in not such a good way. So instead of just saying, “Let’s review the stats, here they are,” it was a question with a specific choice that they simultaneously reported on. And then we could discuss together. And of course, their answers are different. There’s different rationales in both ways. So then we could discuss together what their rationale was, if they want to debate they can debate a little. It generates a lot of student enthusiasm, and everybody’s doing it instead of just 10% of the class.

John: And once they’ve committed to an answer, they have a stake in and they really want to know, that’s something we’ve seen a lot of things we’ve talked about in the past, too.

Kristin: Absolutely.

Rebecca: When you were doing the team-based learning, were you sticking specifically to problems that were on a class-by-class basis still, like you were discussing in the co-operative setting, or were you doing some longer term activities that went across multiple class periods?

Kristin: I had the… what I consider gift… to often be teaching in a three-hour time slot, which is my very favorite time slot. So I would have activities that would extend two or three hours, but typically not between classes, I found that to be more of a sweet spot, at least for me. At my previous institution we had a very high commuter population. And I promised, in both models, that I would never ask them to do something out of class with their teams, that was one of my rules… that it was just simply too burdensome for students who have multiple outside of school commitments… family and work, or living potentially 150 miles apart, which was not unheard of. I promised them no out of class stuff. I structured that intentionally so that the individual preparation that they were doing, they could do anywhere on their own time schedule, but they were expected to be there. And their team expected them to be there to be able to engage in class. And it was also one of the ways that you talk people into it, when they say “I worked with other groups who were all slackers and we would always set times and they wouldn’t show up.” And I said “That’s not going to happen in here. We already have a time we’re all going to show up together.”

John: And the philosophy that’s very similar to the flipped classroom approach where you let students do the easy stuff outside and then give them assistance with or have them work in a framework where they’re getting more assistance with the more challenging issues.

Kristin: Absolutely. I think TBL [team-based learning] is definitely a flipped classroom approach.

Rebecca: I think the other thing that helps too with that model… of making sure you’re not working outside of class… really helps students with really different backgrounds start working together, because you might have students who are more traditional who are on campus. And so for them to meet outside of class is often not such a big deal. But then if you have students who are working or have families, and there’s a disconnect in the class, even, between those two populations, that helps make that more obvious and work a little bit better,

Kristin: Right. Absolutely. Yeah. And I didn’t want to set up anything where people were made to feel like unvalued team members, because they couldn’t do what was asked of them because of other commitments. Since that was in my control, I wanted to make sure that people felt welcome.

Rebecca: I’ve tried to even do that with long-term projects. In the field that I’m in, we tend to do things that go across class periods, but there’s always the “Are we going to do this outside of class or are we going to do this inside of class, and I try to have them do anything that needs to be collaborative, and decision making, in class, and then things that can be done on their own, even if that means doing some creative work, or whatever, outside of class. But those are independent things that can be done for the same reasons. And I find that students will try to manipulate that system, so that they’re gonna: “Oh, we’ll just do it outside of class, because we don’t want her to know whether or not we’re on top of something,” or whatever. But I call them out on it, because it’s really devaluing some of that exact thing. People have other commitments and things.

John: You mentioned, you started to use a backwards design approach where you started with the things you want them to remember five years later. Did you have to cut back on the breadth of the coverage in the class, to some extent, by doing that?

Kristin: Yes, I did. When I was going with the straight up cooperative learning approach, I did not have to cut back on the content at all. Without the full redesign, I found I could cover the same amount of material in straight lecture versus in a cooperative setting. But it was all coverage. It was just a different kind of coverage. When I approached it from a backward design perspective, and I really was able to focus on the objectives that I thought were important, I did have to reduce the amount of things that we were covering. I have no regrets about that, of course, because I completely recognize that covering material isn’t just covering it. What are students going to do with something I covered in class? They didn’t cover it, I was the one who was learning it and talking about it. So I’m much happier with an approach in which I am consistently hitting on the objectives that I really want them to recall, and that they are working hard to apply those throughout the semester.

John: If they’re not going to remember it passed the final exam, covering more material isn’t terribly useful.

Kristin: No.

John: We talked about that in a previous podcast with David Voelker, who talking about the coverage approach in History…

Kristin: Right.

John: …which is the same logic.

Kristin: Exactly. And I actually now consider that to be a complete waste of time. So why am I spending class time on something that I actually don’t really care if they remember, it’s not the most important thing to me, and they really don’t care if they remember.

Rebecca: You have some compelling arguments for why team-based learning and collaborative learning are good options. If one wanted to start moving in that direction, what would you suggest their first steps be?

Kristin: For team-based learning, there are a couple of great books that are very easy to approach. There are several great resources for team-based learning. Larry Michaelsen published a book in 2008, for example, that covers that from front to back. It gives examples of applications in different disciplines. There’s also a book published a few years later on team-based learning in the social sciences and humanities. That also covers the basics, but has applications that are more specific to social sciences and humanities. Team-based learning has really caught on in medical education and in business education. So in the original book, there are more application examples that are in MD preparation or in business schools. So if you’re looking for other examples, the second book might be a good choice as well. And that one is edited by Michael Sweet and Larry Michaelsen.

John: And in fact, I read your article, or

Kristin: Oh, did you?

John: …your chapter in there as background.

Kristin: I’m glad someone read it.

John: Now I have to read all the others. But, I, at least, did read that. It was very good. So for faculty who are moving to this, what are some pitfalls that they might run into? Or what sort of problems might they encounter?

Kristin: Team-based learning as being a much more structured approach… Michaelsen does a really nice job of laying out the pieces that he thinks are critical. And I agree they are critical. So, for example, he talks about explaining, and testing the model with students on the first class day, and you cannot skip it. So the very first class day, I give students an example individual application test, like they would get for their readiness assurance. It includes basic psychology knowledge that may or may not be present in the culture. So they have some chance of getting some of them right and some not. And then I have them do it as a team. And the team scores, of course, are always dramatically higher than the individual scores. And the team testing process is so much better. [LAIGHTER] It’s more pleasant and interesting and collaborative than they expect it to be. That simply going through that, it allays many of their fears about what a team is going to be like to work with. Plus, when they see that the team has tripled their individual score, they’re like, “Hey, maybe I could depend on other people to help me learn, and maybe this will pay off for me.” So going through an explanation of what the rationale is, having them experience it a little is really, really critical in helping them stay open minded while they experience it. And then regularly throughout the semester, I will keep reinforcing them with those messages. I’ll say, look at this amazing thing you guys did. You used all the intellectual resources around you, and you analyzed this difficult problem and came up with some great solutions. I’ll remind them how much they’re learning and what kinds of challenging tasks they’re able to do as a team when they have the preparation to do it, which helps as they’re starting to think “Well, wouldn’t it just be easier if I could do this by myself?” It helps them to kind of remember, ”Well, yes, but you wouldn’t be doing this, you would be doing something not as challenging, not as integrative.”

John: and probably not learning quite as much either&hellp;

Kristin: Yes. He also emphasizes an aspect that is also emphasized in cooperative learning… of helping the teams develop and giving them feedback, helping them give each other feedback. That’s also really critical, especially very early in the semester, as they’re starting to develop group norms and bond together to make sure that you don’t short the time in class for them to have some group processing time and to build their team skills. So, for example, when I taught last spring, I had a student who came to me after I think it was the second week. So it’s very early in the semester, and she said, “I really need to reassigned teams. My team hates me, they won’t make eye contact with me.” She was really upset. And I’m reluctant to reassign people teams, because often what they’re experiencing, they take with them. It’s not always a function of that team process. So we talked some, and I tried to get a handle on what she was experiencing. I knew where she sat, I had an idea of the team composition. And I asked her to try one more day, just one more day. And then we would talk about reassigning her teams. And that day, I was sure to build in plenty of time for group processing, where they talked about what they were doing well as a team and something to improve. Their team turned around immediately. She was a relatively assertive person, which I already knew. I knew that she could handle this. So she went back to the team. She was able to talk with her team about not feeling heard. They immediately turned around in the way that they were with her. And by the very next class day, they were a relatively high functioning team. They did well all semester. They brought doughnuts for each other. I mean, it was a really nice supportive group. What they needed was the time in class to do some processing. And if I, as the instructor, had been moving too fast, and not giving them time to do that, and not giving them a prompt to do that, it would have been a really negative experience for her. So, also building in time for the team to develop and prompts for them to do that.

Rebecca: So you mentioned liking to have a three-hour teaching slot.

Kristin: That’s my favorite. It’s not required.

Rebecca: So, in that amount of time, how much time would you designate towards this group processing, for example, to give people an idea of what that proportion or the amount of time to dedicate so that you don’t shortcut it and you don’t rush through it?

Kristin: If I were to do an activity that might take an hour, I might spend 10 minutes for group process, it doesn’t have to be very long, or even five. And you don’t have to do it every time, you could do 10 minutes after the first one or two more intensive activities, and then not do it for another few times… and another five minutes just every so often to help them resolve their underlying dissatisfactions and to recognize that what they’re doing is not just application activity, it’s also group interaction. So please take time to do both. Another really important required component that I didn’t mention is peer evaluation, I always incorporate peer evaluation as part of the grade.

John: How did you form the teams in these classes?

Kristin: They’re heterogeneous, first, with a very open process so students can see it happening and know there are no shenanigans… that this is all very open… talking about the rationale that people of different backgrounds bring different strengths. So you want a group that has people of different backgrounds, so you can have a larger kind of learning base between you. So usually, I’ll pick a few characteristics that might be important in that kind of background. And I will line them up around the room based on those characteristics. And if it’s 200 people, it’s a really long line. And then we count off. So when I teach introductory psychology, students who have had a high school psychology class usually are starting a big leg up on the other students. So I’ll include that as a characteristic. Sometimes I’ll include the distance that people are coming from, because then they have different experiences, depending on what class I might also include if their student athletes, just because if you put too many together in a team, then they’re all gone on the same day. They have interesting backgrounds, but they also have patterns of attendance and of absence that need to be adjusted around. And we’ll count off all the way around so people can see how the teams are made. But heterogeneous teams are really, really critical. Having students with pre-existing relationships will throw off the team process in a way that automatically excludes people that don’t have pre-existing relationships… plus they tend to be lower performing teams. And I don’t want to set that up on purpose.

John: One of my colleagues once did this in a class of, I think it was about 350 students, but he just sorted them alphabetically. So he had them organize himself that way, and it was a fairly long process. But, it was kind of amusing for those of us wandering by and just seeing…

Kristin: …this huge line… Yeah.

John: He didn’t do it that way In the future, he used other criteria.

Kristin: I’ve had colleagues that I’ve talked with that think that this is a long process. It’s not. You can sort 200 people in 10 minutes, and then you’re done for the whole semester,

John: Doing it alphabetically…

Kristin: takes a lot longer.

John: …can be more challenging, because they were self forming that… it didn’t convert rapidly.

Kristin: The other thing I never do is I don’t put the students who didn’t come the first day into a team, because there are characteristics about why they didn’t come the first day. If you put them all together in one team, they share some of those characteristics… It tends not to be a very high performing team. So I make sure they’re sorted out among the other teams. But that was one of the things that I learned in cooperative learning. That, before I did cooperative learning training, and I was assigning group work, I would assign people based on if you didn’t come the day we did the assignments, you were in another group. And that group typically did not do very well. And as an instructor, it’s my responsibility to create a learning environment in which students can excel, it’s on them whether they do their part. But if I’m setting up a team in ignorance, with predictable characteristics, so that they’re going to have a failure experience, that’s on me to correct. And it’s not on them. So afterwards, I felt guilty when I had come to a new realization. But, yeah, it’s my responsibility to set up an environment in which those students can be successful in their teams.

John: In your chapter in that book, you mentioned that when you switched over, it did affect your course evaluations a little bit. Can you tell us a little bit about that?

Kristin: Just a little bit. But yes, it did. So when I was doing straight lecture, I was shooting for engaged lecture. And in psychology, you can build in little experiences, especially in introductory psychology, where the topics are changing frequently, you can always build things in that are kind of interesting. You can do a little optical illusion here and a little bit of memory trick there. And there’s these ways to build it in, but it is still basically straight lecture. And I got high evaluations for that. I was careful about trying to build those in every day, you know, every few minutes. And when I went to cooperative learning, where it was essentially the same approach, but in in a much more engaged and cooperative fashion, those evaluations stayed very high. Students knew each other, they were happy in class. When I went to team-based learning and I was actually asking every student to participate all the time, and be prepared in class in a way that their contributions were much more obvious than mine. My evaluations did drop just a little bit, not a lot, but a little. And I am grateful that I was teaching in a context where I knew that my department wouldn’t care. They were more interested that I was doing good teaching. And they understood the many factors that influence student evaluations. But I also recognize that it’s incumbent on me to help students understand how they are learning, what kinds of things encourage learning and retention, and then you kind of let the student evaluations fall where they may.

John: When I read that, it reminded me of that study that came out a few weeks ago from Harvard in their physics program, where they found that students in active learning classes did demonstrably better on tests, but they perceived their learning as being lower. So there was a pretty strong inverse relationship between their perception of learning and actual learning. That seems to be fairly common, there have been a number of other studies where what students think to be most effective, is often not what most enhances their learning.

Kristin: Right.

John: Do you have any other advice for our listeners, who might think about using either collaborative or team-based learning in their classes,

Kristin: The one thing I would say is that teaching a cooperative learning or a team-based learning structure class is a lot more fun. You have to be willing to give up control, because when you’re lecturing, you have absolute control… meaning even that students can’t ask you weird things, because you haven’t opened the door for that to happen. But when you structure the learning experience, and then you give up the control to the students, it is an exciting environment to be in. I wasn’t as tired when I was coming out of class. I was energized, you could feel the difference in the room just walking into class… they were excited and talking with each other. When I would circulate around before class started, they’re talking about the class instead of talking about other stuff. It completely changes the environment in the classroom in a way that I think really matches what I expect out of a university education for students, it creates a environment of intellectual enthusiasm around the topic that you’re teaching.

Rebecca: So we always wrap up by asking what’s next?

Kristin: That’s a great question. So right now I’m 100% administrative. And since I’m in a new position, in a new institution, I’m gonna spend some time figuring out all the newness pieces. But I’d like to go back to the classroom, at least for a course here and there when I can. There’s nothing different about students than there is about people. So I also think often about how what we do in the classroom, what we understand works and what we understand doesn’t work, how that applies in administrative settings as well. We know for example, that people tend to try and find the shortest path. So if they’re trying to learn something, they want to put in the least effort to learn it. If you ask a faculty member to do a task for the department, they are obviously going to choose the easiest path to do that… not necessarily the best path. So how do I take the experiences of learning and teaching, that in some ways are better understood to an environment of administration that in some ways is not as well understood? What kinds of lessons can I apply there as well?

Rebecca: Well, thank you so much. It’s been a really interesting conversation. I’m sure it gives a lot of people things to think about as they move forward in this semester and future semesters.

Kristin: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

98. Developing Metacognition

Many students arrive in our classes with relatively little understanding of how they learn. In this episode, Dr. Judith Boettcher joins us to discuss how well structured project-based or problem-based learning activities can help students develop their metacognitive skills so that they become more successful as learners.

Dr. Boettcher is the author of many books and articles on higher education and has long been a leader in the field of online education. The Online Teaching Survival Guide: Simple and Practical Pedagogical Tips, co-authored by Judith has been an important resource for faculty transitioning to online teaching. At Oswego (and many other institutions), many faculty have been using materials that Judith has developed for ACUE (the Association of College and University Educators).

Show Notes

Transcript

John: Many students arrive in our classes with relatively little understanding of how they learn. In this episode, we examine how well structured project-based or problem-based learning activities can help students develop their metacognitive skills so that they become more successful as learners.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Today our guest is Dr. Judith Boettcher. She is the author of many books and articles on higher education and has long been a leader in the field of online education. The Online Teaching Survival Guide: Simple and Practical Pedagogical Tips, co-authored by Judith has been an important resource for faculty transitioning to online teaching. At Oswego many of our faculty have been using materials that Judith has developed for ACUE (the Association of College and University Educators). Welcome Judith.

Judith: Thank you very much, Rebecca and John, it’s great to be here.

John: We’re really pleased to have you here. Our teas today are… are you drinking tea?

Judith: I actually… yes, made a special point. Here’s my cup, which you can’t see. But I chose one of my favorite teas, which is a lemon and ginger tea from England, of course,

John: the Twinings version.

Judith: No, this is a Hamptons tea from London.

John: We have the Twinings version of that in our tea collection.

Rebecca: I’m drinking Jasmine green tea today.

John: And I’m drinking oolong tea today.

Judith: Sounds good.

Rebecca: Judith, your mug looks really interesting. Is it abstract art. Is that what was on it?

Judith: It’s actually pears.

Rebecca: Okay, I only saw the bottom part of it. I can see it now.

Judith: Yeah, right. Well, I will confess that at some point, I finally decided to clear every vendor cup out of my cupboard.

Rebecca: That sounds refreshing.

Judith: Yes.

John: I have vendor cups all over… in my vehicles… in my offices, everywhere.

Judith: Well, that was part of my retirement process that I went through. I said, “Okay, that’s it.”

John: One of the things we had trouble with is picking a topic because you’ve worked on so many topics, but we settled on having you talk a little bit about how students can work to improve their metacognition using project-based or problem-based learning. But before we do that, could you talk a little bit about what metacognition means?

Judith: I would really like to John, partially because I had this book with the title of metacognition that I was reading when I was with a family event. And one of my relatives said, “What in the world is this? Meta what?” [LAUGHTER] So, an easy way I like to think about metacognition is the definition of just it’s thinking about thinking. It’s a definition, I think. that we can all just really grab on to and we can really use. But then I kind of like to expand that definition into one that’s in the How People Learn report that I go back to pretty regularly and that is that “metacognition is the process of reflecting on and directing our own learning.” And I really like that one, because it’s got the two steps, I think, what we want to really kind of focus on with metacognition, and that is reflection… really stopping thinking, pausing… and then actually directing our own thinking, because that leads to action. So then we have reflection and action, which I think is the core of metacognition skills.

Rebecca: One of the things I think we often talk about in education context is this reflection piece. And we always tell students to reflect, but we don’t always give them the time and space to do that.

Judith: Yeah. [LAUGHTER] Very much.

Rebecca: So can you talk a little bit about project-based and problem-based learning and how metacognition connects to those rather than standard ways of operating in the classroom?

Judith: Yes, I would love to do that. In fact, that was one of the first things when we’ve got into online learning, was that there was a real struggle as to how do we maintain the security and everything of people taking tests. And so it turned out that we decided that one of the best ways of gathering evidence of student learning was not by doing these tests… that we would actually have the students do projects. And that kind of evolved into the following process. And that is that in some of the work I’ve done for ACUE, and also in the book, I’ve mentioned that I really like to design a course that starts with the students selecting and doing a project, actually in week one or two of the course. And then that students actually focus on that project throughout the entire course. And that’s the mechanism by which we gather evidence of student learning. It kind of also avoids this whole process of buying papers and buying other kinds of things. Because you really have milestones along the way.

I’m going to stop and talk about the first step for just a moment. And that is choosing a project. One of the wonderful things about choosing a project is that then students actually have to stop and then think about the kind of project that they want to do. It gives them an opportunity to actually customize a course to their interests, which starts getting past that big motivation problem that sometimes teachers might say, “Well, how do I motivate them?” Well, we give them the opportunity to choose something that is of interest to them. So they choose a project, and then they actually write up what that project is that they want to do. But that’s not the end of selecting the project. Before the project is really kind of finalized that they’re going to be working on, they actually then sit down and talk with some of their other students and the other peers. They switch and swap their proposed project descriptions, so that they actually talk out loud about the project. And then hopefully, by talking to their peers about it, they get some additional ideas, and they refine it a bit. And then it goes to the faculty member. So the faculty member doesn’t get it right away, but it goes through this first the individual students thinking about it, and then the other students thinking about it, and then the faculty member can take a look at it. And that’s only milestone one that can take up to about the third week of a course. But by then, hopefully, that gives them a real focus of the course rather than having just the topics thrown at them from week one through week 16… that they’ve got a real focus of: “Oh, how is this going to affect my project?” We can come back to that I just want to mention briefly then there’s like four other milestones, five other milestones for every project throughout the course. And the first is that project description, then the second one would be planning how do I plan to do it, which is a really important metacognitive skill. And then another milestone would be some type of other “Just checking in, how are you doing?” …kind of a thing. And then there’s two final things. One is where they actually share their project with the other students, like in a mini conference, whatever. And then the final thing is the actual final thing, which might not be a paper, it might be a video or an interview, it could be one of a number of things that would go into their student portfolio then. But, that gives the student a focus throughout the entire course that way. That’s a lot, isn’t it?

Rebecca: It is a lot. [LAUGHTER]

John: When you have them give feedback to each other, do you recommend that that’s done synchronously or asynchronously If it’s an online course?

Judith: You know it’s actually best, I think, that the students do it somewhat like a brainstorming event. What you and I are doing right now here with Zoom, students can do with FaceTime, they can do it with just an online gathering chat, whatever. If they’re really, really busy online students, and they have to do it asynchronously, that can be done too with email. I mean, that works too. But it’s often really great for the students themselves to talk out loud. We don’t have them talking enough, I think. They read passively, and they kind of think and everything else, but we don’t have ways for them to use their voice to talk about what they’re thinking.

John: It sounds like the project’s really nicely scaffoldied. But how do you bring in the metacognitive development? How do you get students to improve their metacognitive skills? Is that something they’re explicitly thinking about? Or is it something that’s done as part of the structure of the project?

Judith: Well, of course, it depends on how the faculty member wants to do it. As I had the chance to go back and look more intensely at the metacognitive skills, it occurs to me that thinking metacognitively is such a basic intellectual skill in many respects, particularly now in our 21st century. It’s as fundamental as reading and writing. And, you know, I think we need to look at our entire curriculum from pre-K through whatever as to really how do we explicitly teach and model and coach metacognition skills, which includes with that initial project proposal, we can talk about the fact that what they need to do is think and be sure to build on their interest to give them the criteria and coaching as to how that’s going to really work for them. It’s really a problem when they have to select a project. So what are the constraints on that problem? What are the features or benefits that they’re going to get from this? …and to build that into the assignment and actually writing out the initial proposal and then meeting with their colleagues and peer students, that’s all part of the coaching and the modeling of the metacognitive process.

Rebecca: I think your emphasis on talking about things is really interesting. Because I find that students often will passively write something, and it makes no sense to anybody. But as soon as they try to explain it out loud, they realize it makes no sense. Because when it’s in writing, or they’re just reading in their head, they don’t often have that realization. But as soon as you try to say it out loud as a sentence, it’s not structured in a way that makes sense. They catch themselves or they realize, “Wait, there’s a big hole in what I’m talking about here… nobody knows what I’m talking about.” You can tell it by other people’s faces looking blankly back at you.

Judith: And then they realize that they don’t know what they’re talking about.

Rebecca: Yeah. [LAUGHTER]

Judith: So yes, in fact, that’s apart of the power of talking with their peers about the project, because the peers would most likely say, “Well, wait a minute, why did you choose this? Why are you interested in this?” And so that’s when they have to dig more deeply. It almost goes back to that Socratic questioning, “Why do you think that? Why is that important to you? Do you think this is going to make any difference in your life?” One of the things I really recommend in online learning is that the students, when they read and look at the objectives for a course, to stop… and in the first week, actually, to have the students process those learning objectives, and then personalize one or two of them and say, “This is what I really want to make certain I know when I finish this course. This is how I’m going to be a different person.” Thinking “What skill am I going to be able to do when I finish this course?” They kind of set that goal and set those expectations early on. Hopefully, students setting those goals and objectives become again, more personal to them, and something that they can check themselves. And then as they go through the course, “What progress am I making on this particular goal? How is this coming?” …kind of fun.

Rebecca: Are there ways that you would suggest a faculty member model metacognition early on in an online course?

Judith: Well, I think a really easy way for faculty to model meta cognitive skills is… say, for example, in biology or one of sciences, one of the things I love to look at is the biography of a scientist and looking at, “Well, how did they come to this point? What made them think X, Y, or Z as opposed to A, B, or C?” and we find it’s really rooted in their personal lives and their thinking that they’ve been doing. So faculty member can somewhat do the same thing, in fact, very early on in their introduction to the course. Now, what do you teach Rebecca?

Rebecca: I teach web design.

Judith: You teach web design, okay. So in your introduction, then, to your students, you can say the reason I love web design is as follows. And you can go back into your life and your experiences and say, This is what happened to me… and this is what I was thinking… this is how I got to this point. So you really speak out loud, and you share your processes by which you arrived at that point. And we can do that for any kind of thinking… as we’re talking about an experiment. We do want faculty to share their expertise and their dissertation, for example, they say, “Wow, you know, this is why I’m interested in this. And you know, after 25-30 years, I’m still interested in this because of this.” So they share their thinking processes. Another really easy time is when students ask them a question. And obviously, this can happen online in the synchronous activities. And the students might ask them a question about, “Well, gee, Professor, so and so what does this really mean in this instance, or in this context?” And the faculty member can say, “You know, I really am not sure about that. And I would like to, before I answer that, I would like to do a little more research and thinking, and I will get back to you on that.” So it sends the message that the process of thinking is something that we keep doing all the time.

Rebecca: I think that moment of admitting that you’re not always the expert in everything in the moment, is always a good thing for students. And they respond really positively to that.

Judith: That’s right. It also gives them the opportunity to say, “You know, Professor, I would like a little more time to think about that. Let me get back to you on that.”

John: How can we tell that students have improved their metacognitive skills? I like the idea of having the peer instruction do that. Or to work with students to help them recognize what they know and what they don’t know. But how can we measure that? Or how can students know that they’re more metacognitively aware? How can they observe improvement?

Judith: Well, one of the things I did as I was thinking about this was, you know, I was kind of preparing for you asking me the question, “What are the actual metacognitive skills?” and I came up with five thinking skills that we really want to encourage and model for students. The first one is an easy one. And that is, number one, we really want to encourage students to think. I know that sounds somewhat simplistic, but oftentimes think about in the online environment, we give them an assignment and students, they don’t even read it, really, they kind of scan it. And they kind of assume without thinking that that’s what that assignment is going to give back. And so one metacognitive skill is to pause and stop to really take some time to think and process. In fact, I kind of like to think about the assignment as a briefing. You know, we use briefings in business and politics and detective crime solving and all the rest of it we use briefings. Maybe as a faculty member, you want to encourage the students to think of an assignment as a briefing and a briefing is, “Well, this is what we know now, what do we want to know about next? How are we going to find out what that next is? And when you finish this assignment, what do you expect to know or think about or become clear about?” So step number one is really pausing and thinking. There’s a book out recently called, was it Kahneman? It’s that Thinking Fast and Slow or Slow and Fast?

John: Yes, Thinking Fast and Slow. It’s a great book.

Judith: Thinking Fast and Slow, right. Well, you know, he makes the point that we live in such a fast-paced world that our first response to anything is that, “Oh, I know the answer to that.” It’s like Jeopardy, and you really don’t stop and pause and think. And so, because it’s really easy, that’s the short-term memory, we don’t have to think about it. We know that in order for learning to occur, that we really have to stop and give time for information going into our short-term memory to get into the long-term memory. The only way anything goes from the short-term to the long-term memory is for us to connect what we know already with the new information coming in. And you know what? …that takes time. So that’s the one thing for all of us. Time is so seriously, we don’t want to take the time. But if we don’t take the time, the only way learning occurs, you know, if we grow dendrites in our brain, and so if we don’t take the time the dendrites don’t grow, and nothing lasts. It’s kind of a fast bullet shot, so to speak. And then we forget it. As soon as we use it in a sentence, it’s gone. So the idea that we have to really discipline ourselves to stop and think and process, what is it that I’m needing to do? What is it that I want to learn? What do I want to get out of this? And when I finish this assignment, how will I know that I’ve really finished the assignment. Which brings us to the next thinking skills. One of the things I really liked, I use myself a great deal. And this is kind of a hard won practice of my own. And that is that, whenever I was given an assignment whether it was at work or wherever, whenever I had a hard time getting started… you know, getting started is sometimes one of the most difficult parts of anything, right? So anyhow, I finally figured out that one of the reasons it was hard getting started was that I didn’t quite know what it was I was going to do. I finally realized that my best practice is that I visualize what it is going to be when I finish. I get requests to review articles for journals, etc, etc. Well, I know now that my visualization works is that I’ll be done when I finish answering all those questions, you know, read the article, finish answering all the questions, I compose my response back to the editor. And that’s when I’m done, okay? So with any kind of a project, we want to encourage the students to say, “Okay, what is my assignment going to look like when I’m really done? When I have finished reading this article, or reading this core seminal research project? When will I really be done? Will I be done when I can explain the research to someone else? Will I be done when I can actually implement these ideas in a web design project? Is that when I’m going to be done? Just what is that project going to look like when I’m really done?” So we just celebrated the Apollo landing on the moon, in July. I just had the recent opportunity to see the Apollo 11 program that they created out of the original footage from everything that was happening at Cape Kennedy and Houston… the pictures, it showed like, literally, a room full… it almost look like 100 guys, and they were all guys at that time… 100 guys sitting at computer terminals, and you think when they worked on that project, how did they envision success? They had to envision success as actually a man being on the moon. How did he get to the moon? What did he land on the moon in? And then how are we going to get him back from the moon safely? I mean, think of all the things that had to be planned and worked on and everything had to be coordinated to make that thing happen.

So thinking skills, we have to think, we have to visualize, and then we have to plan. Once we know what that final vision is, we need to then plan each of those steps along the way. And again, we can model some of that and coach that by building the planning into the assignments for the project. We definitely want the students to give me a plan with the date, the milestones, the resources they’re going to be needing,]… if they need to make appointments or interviews when that’s going to happen. So we help them realize that projects just don’t happen. They happen after all of these various steps. And then, of course, we build in step number four, evaluating and pausing to then debrief each step along the way: How am I doing? Do I need more time? What else might I need? What else do I really want to know, if I do have enough time? I mean, this really does happen with web design, right? We get to a certain point, we think of it’d be great if I did this. Great if I did that. Do I have enough time to do that? Do I have the knowledge to do that? Or do I need to learn a new skill to insert that into that. So those are all questions that we want to ask encourage the students to plan for and to ask along the way. And then of course, the final one is the final debriefing. When you hand something in, you get feedback both from your peers and from the faculty member. And in many online courses, you do have these little mini conferences where you invite alumni or experts or just friends in to say take a look at this and see what you think.

Rebecca: I like your framework of the briefing. And it’s actually one that I use in my classes pretty regularly.

Judith: Oh, great.

Rebecca: When I’m doing long term projects, I have students on a weekly basis do basically a little briefing of what did they do? What do they need to do? What are the planning steps? So not just like that big scope of the whole project, but on a very routine basis, checking in with their schedule and checking in with their plan and what they’re struggling with to demonstrate what they’re doing, and what their thought processes is as they’re working on it. So I asked them, “What were the big design decisions you made this week? And what did you base those on?”

Judith: Wonderful. In fact, I sometimes like to use the example of Mark Harmon, the actor in NCIS. I love those programs, actually. Like every morning, he kind of just drives into the office and says, “Okay, what do we have.” And each one of his team members have to then report “Okay, since we saw you yesterday, this is what we’ve done. And this is what we know.” And then from that briefing, decide on their next steps, “Okay, we need to do X, Y, or Z.” It’s a really kind of a nice example of planning.

John: Building it into the project just seems to make an awful lot of sense. If you want students to improve their skills, have them apply those skills and structuring it so that they’re doing it is a very reasonable way of doing this.

Judith: And I love the fact that, Rebecca, as you were saying is to ask the students to say what have you been thinking for? And what made you arrive at the decision that you’d like to do something a little different there? So yeah, again, verbalizing the thought processes, is part of the metacognitive abilities.

Rebecca: I think one of the things that surprises students with that assignment is that they don’t realize that they’ve made decisions.

Judith: Oh, interesting.

Rebecca: That happens in projects. But also, if they’re writing a paper, whatever, they’ve made decisions, but they don’t necessarily think of it like they’ve made a decision. They’ve just kind of moved forward. So it causes them to stop and pause and do that first thing that you were talking about, and think for a second before they move forward.

Judith: Yeah.

Rebecca: I do have to say, that since I instituted that, my students are far more articulate when they’re talking about their projects. [LAUGHTER]

Judith: When they’re doing their own projects, they get their passions involved. There was another thing I wrote recently on curiosity, how important curiosity is and how we want to really build that in. In fact, one of the things when I was doing a workshop, I suggested to faculty, I said, “You know, let’s get the students to stop answering questions. Let’s get them posing questions. And let’s give them problems that, in our discipline, we don’t know the answers to, because what fun is that? Ok, so we set up the situation that we the faculty know the answers, and then they have to figure it out. So switch it around a little bit and say, here are some problems and we don’t know the answers, how am I to approach this? And then we’ll give them the challenge.” And that really, particularly John, I’m sure you’ve seen this with the TIP program, the students come up with things that you never would have expected.

John: It’s one of the reasons I enjoy it so much.

Judith: Yeah.

Rebecca: So one of the things that I think can be a challenge is if you’re doing a big project throughout the semester, we don’t want it to end up being just one big high-stakes assignment. Are there methods or ways or strategies that we can manage bigger assignments to have some lower-stakes moments or ways so that there’s not so much pressure, and that they’re allowed to make mistakes and improve and learn?

Judith: Yes, and I’m glad you asked that. Actually, somewhere in a couple of tips online that I wanted to refer to. It’s, I think, tip… is it 38 and 60, that talk about project-based learning online… that’s something to think about. But I also have a chart that recommends the grading process for an online course. And so you assign points to each of the milestones in the project. So you assign points with the project proposal and selection, but you also then have smaller team-based events where it’s worth a little bit. Also, the discussion board is a really important aspect of online learning. But it’s hard. How do you evaluate and grade that? So I recommend that about 15%, even 20% on the discussion board. And it’s pretty much a given like the classroom discussion is… so long as students are reasonably consistent about following the rubric for the discussion boards. So you’ve got a little bit there, you have also shorter essays and shorter leadership opportunities in an online course where they might summarize a week’s discussion. So there’s other kinds of activities within an online course. At no point is everything dependent on that one big project, but it does require the students to invest time and energy in that.

John: Many people often think of a need to develop metacognitive skills most for our less able students, thinking of the Dunning-Kruger effect, and so forth… that those who know the least often overestimate their learning by the most. But I think those issues may apply for students at all levels.

Judith: Yeah, we assume that great kids, actually, automatically have these skills and think that way, and they don’t.

Rebecca: Which I think is also a really great thing to talk about, too, because we often talk about metacognition, and helping students who are struggling, do better. But it’s also a great way to challenge students who are already doing well, to do better.

Judith: Absolutely, a great point.

John: For the last 32 summers, I’ve been teaching in the Talent Identification Program (or TIP) at Duke University. And one of the things I’ve observed is that many of the students there have generally been able to breeze through their regular classes without ever having to really learn how to learn. So they sometimes face a little bit of a challenge when they arrive at TIP, and suddenly they’re faced with a challenge. But a really nice part about this is that it’s an ungraded program, so that they can develop their learning skills without worrying about what sort of grades they’re going to get in their classes. And it’s much better to do it there than it would be at some future point, perhaps in a physical chemistry class, or a differential equations class, or some other class later in the career, when they haven’t really had to develop those metacognitive skills that are going to be useful in their future. Developing metacognitive skills really are important for students all along the spectrum.

Judith: That’s a really good example too, John, it’s an interesting phenomenon to talk about, really.

John: How can we tell whether students have higher levels of metacognition? Or how can we tell when students have not developed their metacognitive skills?

Judith: Well, you know, this is obviously a great question. And so let me share just a little bit about my thoughts on the students. How do we know when the students are totally clueless about their thinking processes? And I think one really red flag, particularly in online courses, that these are the students who rather consistently post comments on the discussion board: “What are we doing now? What was the assignment about? When is this? Oh, there’s a rubric, I didn’t know that.” These comments often are the ones from students that they really haven’t taken the time to really read the assignment and to make plans on that. These are also the students that we know for online students that they’re trying to do too many things. They’ve got 1000 things going on with their families, and work, very likely. And they just simply don’t take the time. So I think… just simple reminders all the way along the way in an online course from the faculty member. In the assignment, to be really clear about the assignment and to say, “Be sure to think about the following.” For example, if an article has been assigned, that you as a faculty member say, “Okay, this is why I’ve selected this article. Here are some of the core concepts that are really important in this” and guide the reading to say, “Why do you think the person did this? Be sure to be able to answer these kinds of questions?” And then also to be clear about maybe what you want to do when you finish this reading assignment, to talk about it. If necessary, explain it to your 12-year old. See if you can explain it to that person. So being really clear in the assignment would be, I think, super helpful for all students in an online course, because again, it’s the kind of thing a faculty member would do in the classroom, you’d say, “Okay, I want you to read this article. And you know, this is one of my favorite articles. This is why I want you to do it” …and all the rest of it.” I think we need to do more of that in the assignments in the online course. That’s one thing.

The other students that are clueless are the ones that keep thinking that all they have to do is passively read and reread and reread. We hear that all the time from students that “Oh, well, I read it 15 times. I don’t know why I don’t understand it.” Without saying, while giving help and modeling, as you’re reading this, think about the following.

Just as an aside, I signed up to take a course on brain, dendrites, and synapses from the Hebrew University of Jerusalem, and oh, my gosh, lesson three, they said, “Well, you know, if you’re not familiar with the electrical circuits, you probably need to go into the Wikipedia and get all this information.” I’m thinking, “Oh, I’m lost right here.” But the idea was that there was some foundational content that I didn’t have. And so again, if in an online course, if some of their content is dependent on some of those other core concepts that they might not have, to remind students that, “hey, this is a difficult reading. And you may want to get some further help in X, Y, or Z.” So again, going back to that whole briefing, and using the assignment as a briefing and giving them clues how to do well in the reading. Some of those comments really have to do with whether students are struggling or not. Because we want to emphasize more clearly the reasons for why we make choices. When we design a course, we really make lots and lots of design decisions and selections as to why I want my students to read X, Y, or Z. And I think sharing that rationale and sharing those reasons, I think, really helps give insight into the metacognitive thinking of the faculty member as well.

Then John, you ask the question, where we talked about little bit about how even when students are doing well, they may not be thinking metacognitively. And I think that it’s important to recognize that these explicit thinking skills about thinking, about visualizing, and about planning, etc, that it would be a good idea to build those visualization and planning into the assignments.

Now, one of the challenges with online courses, and I’ve seen this for years, is the fact that some students don’t have good places to study, they can be living in very busy environments, they don’t have an office, they don’t have a really quiet place to go. And so one of the things that we can do is actually ask the students to post a picture of where they’re going to be doing their studying. It kind of gets them thinking, “Okay, where am I going to be doing my studying? Is it going to be a place that I can really concentrate?” There was one study hint… that I actually just pulled it out, because I gave it my granddaughters…. and it was called “How to study” …and the little hint in there was that when you are sitting down to study, what you do is you find, select or design, some kind of a flamboyant little hat… think Cat in the Hat kind of thing, you know… just flamboyant things, design some kind of a flamboyant hat so that when you put that hat on, you tell your brain and you tell the other people around you that you’re going internal now… you’re really going to think… you’re going to work, you’re going to study. We almost do that nowadays… when I go into my Starbucks, or my local coffee shop… that’s where I do my writing, by the way… I put on my Bose headphones, because again, that signals to the people around me that, “Hey, I’m working, I’m really concentrating.” So if we asked the students to really stop and think, “where am I going to study?” If I’m a family person, and I’m working, “When am I going to study?” That I have to schedule my week, and the times and places that I’m going to do my work and do my studying. And oh, by the way, I kind of have to get my family and my friends on board to say “Yes, John or Rebecca, they’re studying now I can tell they’re studying, you got to leave them alone, give them the time and the space to leave them alone.” For online students I think this is super important for them to build a schedule ahead of time. And again, it recognizes the fact that metacognitively takes time and it takes space in order to do it well.

John: That reminds me of a couple of our previous podcasts, a few episodes back Mathew Ouelett from Cornell, when we were recording, he mentioned that he has a drawing or a painting, I think he did of a tomato, I don’t remember if it was a painting or if he colored it, but he has a big poster with a tomato on his door that he puts up as a signal for faculty to leave him alone because he’s engaged in a pomodoro technique, and he wants to be just focused on this. In a much earlier podcasts, our very second one, Judie LittleJohn was here and we talked about a metacognitive cafe online discussion forum she and I had both used. And one of the things she uses in it, and Rebecca has introduced that in her classes too, is exactly what you suggested, having students describe their study space and perhaps post a picture of it. And then to address all those issues about how well it works, how they deal with distractions, and so on. This ties in nicely to some of our earlier discussions,

Judith: Well great, actually as I was in my Lucky Goat local coffee shop, actually getting ready for this podcast a couple weeks ago, it turns out this young business person came in and he had a briefcase and all the rest of it and I could tell he was kind of settling in… a little too close to me… but that’s okay, there weren’t many seats available. But anyhow, he was talking pretty loudly. And I thought, “Oh, he’s on his phone.” You know, there’s everybody who talks to themselves these days is on the phone, right? And anyhow, we ended up talking to each other partially because he was talking. He said, you know, what he was doing was he was talking to himself about what he was going to do during his time there. And obviously setting his own personal goal. And then he had learned that from his mother. [LAUGHTER] But that does remind me of something else I do want to share. And that is when I do sit down… my own metacognitive practice… when I sit down to do a task… say, I go to the coffee shop, and I got a couple of things I need to do, I write out a mini plan. And list the time I’m starting… list my first subtask, the second sub tasks with approximate time to completion, and everything else. And of course, it doesn’t happen exactly like that. But I do get done, and I do manage to check the pieces off. But then another really important metacognitive practice that I’d like to share is that when I finish that task, I say “What is my next important step?” In fact, David Allen and his book Getting Things Done on Stress Free productivity, thee whole environment, even in business. His primary question is, “What is your next step?” So before you stop where you are, particularly in a writing project, you write down what your next step is. Because it’s totally short circuits, that transition. Because when you sit down, “Yeah, what was I doing now? What do I have to do? What is my next step?” Hey, I can’t tell you the number of hours that that particular little hint has saved me in terms of really making progress and stuff. So that’s another little hint to build into all that project planning. “What is my next step? What do I have to have next?” And sometimes for online learners, particularly, it’s not sitting down and studying, they have to get a resource, or they have to get a book or they have to make an appointment or they have to do something. But then that’s something that doesn’t have to necessarily be done exactly in order. To kind of almost wanting to build a little calligraphy thing, saying “What is my next step?” is a really, really great little hint.

Rebecca: I think experience certainly teaches us that, but it’s something that students who have less experience don’t explicitly know to do. Because we all know that we’ve tried to cut corners and hurry through something. And then if we don’t do that… I know I’ve spent hours figuring out now what was I doing? Like it’s been a while since I’ve worked on this project. Now, where was that? What was my filing system?

Judith: Yes. And what was I thinking? And oh, dear, I wish I bought that along with this other thing with me. Yeah, exactly.

Rebecca: I also wanted to circle back a little bit to the workspaces too. One thing that I found with asking students to talk about their workspace is also to show them professional workspaces. And to talk about the different kinds of environments and how space can help facilitate certain kinds of activities, but also can short circuit certain kinds of activities as well.

Judith: I think that’s really, really helpful. I know, I can’t tell you the number of conversations I had with people when I was in a work environment, and management was making all kinds of decisions that were not conducive to good collaborative work or good independent work. I mean, it’s a real discipline to think about.

John: And you need both spaces, but you will often only have something that’s better for one or the other.

Judith: Exactly, exactly. For any kind of a course, as we’re talking about metacognition, we get overly focused on “what” the students are learning as opposed to the “how.” So just in a capsule comment, that as we are designing online our classroom based courses that we really include in our assignments.. in the design and the various activities… that we include both the “what” of the content and the “how…” How do I get there? And I think we started getting there when they start setting those personal goals. Because some of the courses now are starting to include a goal or a learning outcome that they want their students to think like a scientist. You’re not just learning biology, we want you to think like a scientist. Or we want you to think like an engineer, or an entrepreneur, or whatever. But getting this… it’s a different mindset. You just don’t want to learn what biology is and learn about the content of biology, but I love finding out about the biographies of like Einstein and a few other folks. It’s just fascinating to get into their heads as to the process that they use to do that. Eric Kandle, by the way, is another Nobel Prize winner. He’s got some fabulous book out about his thinking and his processes by which he actually investigated and learned about memory. So… fun.

Rebecca: I think that using biographies is a great way to introduce students to that a little bit.

Judith: Yes.

John: A number of software packages, such as Lumen Learning’s Waymaker package, Norton’s Inquiszitive, and some of McGraw-Hill’s and CENGAGE’s products include attempts at building student metacognition in their products. For example, they’ll ask students questions and they’ll also ask them about their confidence in their responses. The Norton Inquizitive package, in particular, sets it up in a somewhat game-like situation, where they get to bet points on how confident they are. And then it gives them feedback on how they did versus how they perceived they were doing. Do you think this type of approach might be useful?

Judith: I was just reading a research study about that, John, and the students… Who was that by? I would have to go back and find it, I’ll have to email it to you. But the results showed that the students who were less confident…

John: …did better.

Judith: Yeah. Who was that?

John: I don’t remember. I think it was on the POD list, but maybe not, it might have been on an economics list. Consistent with the Dunning-Kruger effect, the students who did relatively poorly, had relatively high self perceptions of how well they did that were not reflected in their test scores.

Judith: …So important to be asking the question: “What do I know?” and “What don’t I know?” Because that’s a core, isn’t it? If we can answer that question. In fact, one hint that’s in that book called Make It Stick… which is a real good one… one of the key things I took away from that was the technique or practice of having in a textbook, taking the heading and turning it into a question and then seeing if you can answer that when you finish reading something. So again, “What do I know when I start? What do I know when I finish? And am I able to answer that question? Or am I able to pose another good question based on that?” So explaining to ourselves what I know and what I don’t know, I wanted to go back to that study that we were just both talking about, because my question that occurred to me as I was reading this is that, you know, the problem, I think, is that the study really didn’t really give the students an opportunity to verbalize why they felt confident or less confident, which is, I think, a whole missing piece of that. And I don’t know how they obviously would make designing a study… actually, my dissertation, in fact, I will bore you with the title of it is “Fluent Readers’ Strategies for Assigning Meaning to Unknown Words in Context.” And the thing is that, half the time, they didn’t know that was an unknown word, they just assumed they knew the word. So again, if we don’t know we don’t know something, we can be very confident. [LAUGHTER] But anyway, in order to get the answers to that question of the study, I actually had the students talk out loud to me, they verbalized and I would go back and ask them about the word and say, “What were you thinking when you came to that?” …and all the rest of it. So they had to verbalize their thinking. For me, it was a good study, it really worked. So again, going back to the value of verbalizing,”What do I know?” and “What don’t I know?” and “What do I think I know?” And Rebecca, we got that when you said, “Well, you started talking about what to do, and then you realized you didn’t know anything.”

John: As you suggested before, a really good thing with any project is to think about where you’re going next. And we always end our podcast with the question, “What are you doing next?”

Judith: What a great question. Okay, what am I working on next? I did mention that I just finished a little video for the Distance Learning Conference in Madison in early August. On my back burner, and I’ve been saying this for a couple of years, I would really like to write a book on concepts. What I’d like to do is… I mentioned I love working with faculty, and when we design courses it so often on a topic-to-topic basis. And yet, as I started working with faculty towards the last 10 years or so, I started asking faculty to tell me what their core concepts of a course is. And, you know, faculty, that’ve been teaching for 10,15, 20, even 30 years, they would pause and they’d say, “Uhhhh, [LAUGHTER] you know, we just teach topics,…” as opposed to core concepts. And we think about core concepts are what stay with us when we finish a course, hopefully. So the question really is, what do you expect your students to know, five years, 10 years down the road from what you have been spending all this time and energy on? So I’d like to write a book about thinking about concepts and how to design a course, and focus on problems and concepts rather than topics.

Rebecca: I like that, I think that would be really helpful. Can’t wait to read it.

Judith: [LAUGHTER] I can’t either.

Rebecca: You gotta visualize, you gotta visualize. [LAUGHTER]

Judith: Thank you. You can give it right back to me.

John: We talk a lot about backwards design. But a lot of the classes that many of us teach were not designed in that way. They did not start with those major course learning objectives, and then work backwards to get to that point. And they’re just series of topics taught in the same way that they were taught to them when they were students. And they were taught in the same way as their previous generation taught them. And there’s not always a lot of thought going into that. And that sounds like a really good project.

Judith: Well, thank you, I may call you and see. I’ll need faculty to work with on that project. So I may contact you for that.

Rebecca: I’ll sign up.

John: Be happy to.

Judith: I’d be willing, God be willing that I get going on that. [LAUGHTER]

John: I think we all have a few projects like that. But eventually they often happen.

Rebecca: Eventually.

Judith: Yes, well, actually talking about it is a good thing. Because the more things we write down, and the more things we actually talk about are more likely to happen.

Rebecca: And I have just told the world so it’s gonna have to happen, right?

Judith: Oh, dear. [LAUGHTER]

John: Although if you change your mind, we can edit it out.

Judith: Scratch that…

John: I know I come up with some things after a podcast where I say I want to do this in my class next semester. And once it’s in a recording now, I pretty much have to do it. [LAUGHTER]

Judith: Well, John, before we break up here, when you do go to Duke, what do you teach?

John: I teach economics, introductory micro and macro economics.

Judith: Okay. Sounds great.

John: It’s been a lot of fun. I love doing it. The kids are just so amazing.

Judith: Well, kids are amazing at that age. They really are. It’s wonderful to see them evolving to young men and women. You know, I’ve got eight grandchildren. My oldest is now 19 and a half. In fact, she did microeconomics online, both one and two this summer.

John: Thank you for joining us.

Judith: Well, thank you very much. I really enjoyed being here.

Rebecca: This is a lot of fun. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.