Faculty are often excited after attending professional development workshops and plan to implement new techniques, but often don’t follow through. In this episode Dr. Penny MacCormack joins us to talk about one program that provides scaffolding and structure to help faculty improve their teaching using evidence-based practices.
Penny is the Chief Academic Officer of the Association of College and University Educators (ACUE). Before joining ACUE, Penny had served as the Chief Academic Officer for the New Jersey State Department of Education and as an adjunct professor at Southern Connecticut State University, and Montclair State University. She began her career in education as a science teacher.
- ACUE — Association of College and University Educators
- ACE — American Council on Education
- Linda Nilson — Director Emeritus of the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University
- 80. Self-Regulated Learning. Tea for Teaching podcast (with Linda Nilson)
- Tom Angelo — Professor of Educational Innovation & Research at the University of North Carolina at Chapel Hill
- Elizabeth Barkley — Professor of Music History at Foothill College
- Saundra McGuire — Director Emerita of the Center for Academic Success at Louisiana State University
- Efficacy studies
Rebecca: Faculty are often excited after attending professional development workshops and plan to implement new techniques, but often don’t follow through. In this episode we talk about one program that provides scaffolding and structure to help faculty improve their teaching using evidence-based practices.
John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…
John: …and Rebecca Mushtare, a graphic designer.
Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.
John: Our guest today is Dr. Penny MacCormack, the Chief Academic Officer of the Association of College and University Educators, or ACUE. Before joining ACUE, Penny had served as the Chief Academic Officer for the New Jersey State Department of Education and as an adjunct professor at Southern Connecticut State University, and Montclair State University. She began her career in education as a science teacher. Welcome, Penny.
Penny: Thank you. I’m happy to be here.
Rebecca: Today’s teas are…
Penny: Green tea.
John: I have Bing Cherry Black tea.
Rebecca: And I have Lady Grey.
John: We’ve invited you here to join us to discuss ACUE’s effective practice framework and the associated professional development program. How did this program come about?
Penny: So I think, like many ideas, initially with a conversation among leaders in higher education, some very respected leaders, talking about some of the challenges and changes happening in higher ed. An increasingly diverse student body, certainly more attention being paid to retention and graduation rates, and increasing contingent faculty, as well as the public starting to question the quality and the value of a degree in higher ed. And as we looked at the student success agenda, with many strategies that made good sense, really paying attention to maybe more nuanced financial supports, guided pathways with better advisement, data analytics, instructional supports, et cetera. We felt that there was a missing element and we felt like that element was more foundational than just one of the strategies that folks should be thinking of. For example, guided pathways or advisement make really good sense to us…that a student would have a clear path to a meaningful degree. But what we thought attention needed to be paid to was the quality of instruction in those courses along the pathway, and then across an entire institution, the quality of teaching. And we were very aware of the fact that faculty—including contingent faculty—are experts in their discipline, in their subject area, and they’re experts in the research processes. But most have little—sometimes no—training in evidence-based teaching practices in teaching. So we felt like that missing foundation needed to be addressed and set about to develop a comprehensive…we wanted something that would give folks a foundational base of the evidence-based teaching practices we know to be effective in the college classroom. So we wanted to be comprehensive, we wanted it to be research based, we wanted it to be high quality, and we wanted to be scalable. Recognizing that while it’s important for small groups of instructors to become better teachers, the reality is, all of our students, and all of our faculty deserve to be interacting with the evidence-based teaching practices we know actually improve engagement and deepen learning. So we set about to do that.
Rebecca: It’s a pretty big undertaking. It sounds like you probably had a lot of people involved in that process. Can you talk a little bit about how did the design of the program happen and who was involved?
Penny: So you’ll notice here one of the things I said was comprehensive, that we wanted faculty to gain a foundation in evidence-based practices. And so we needed to identify, what are the core set of knowledge and skills you need to be effective in the college classroom? And to be very honest, we had hoped perhaps that already existed somewhere. [LAUGHTER] But lo and behold, that was not the case. And so we reached out to scholars in teaching and learning across the country and worked with them, did a deep dive into the literature, and worked through an iterative process to identify that core set of knowledge and skills. And once we had that, we also worked with the American Council on Education, to endorse our courses and our framework. And they brought to bear their own set of experts across the country in the Scholarship of Teaching and Learning to review the framework. And then eventually, ACE endorsed the framework and so we feel pretty confident at this point through the processes we used and ACE used to say that our framework and effective practice does outline the core set of knowledge and skills you need to be effective in the college classroom. So in that case, the folks who really informed that work are the experts in the scholarship of teaching and learning across the country, folks like Linda Nilson, Tom Angelo, Elizabeth Barkley, Saundra McGuire, really making sure again, to involve those folks that teaching centers across the country know really have done the majority of scholarship in that area.
Rebecca: Of course, once you came up with the framework and that comprehensive knowledge, you had to figure out how to deliver it. Can you talk a little bit about how that decision was made?
Penny: Absolutely. You point out something that is quite important. It’s one thing to develop a list, right? “Here’s the core set of knowledge and skills.” It’s yet another thing to do that those other three describers, right? Research based—that was kind of easy, because the list was research based—but high quality. And for me, when I’m talking with folks, high quality really means that faculty will love it. Because if faculty are not going to be engaged in this course and engaged enough to actually change the practices that they’re using in the classroom, then we’re not going to realize that student level impact that is our mission. So in order to design the course now—to your point, got to do that part—we did a couple of things. So one, we paid a lot of attention to the research on how people learn, how does the brain work, and specifically, how do adults learn. The course needed to be scalable. It needed to be offered online, so a lot of attention to online practices. But then we did something really important. And that was to talk to faculty focus groups across the country and do a couple of things. One, put some materials in front of them. Some questions, some video, some text, and ask them to critique, which they did happily, because faculty are quite good at critiquing. [LAUGHTER] The second thing we did was we asked them, “What would you need to consider changing the practices you use in the classroom?” And so they were crystal clear. One, they wanted to see those evidence-based practices in action, in authentic classrooms, by their peers…peers teaching…people that they could see would be instructors in the classroom. Two, they wanted to hear from those instructors why they were using those practices. Icing on the cake would be to hear from students as well, how those practices were working for them. Three, they wanted to hear from researchers. They wanted to hear from the folks who demonstrated that these practices are effective in the classroom. Makes sense, they’re higher ed folks, they want to hear from the folks that did the research. And four, they wanted opportunities to learn, discuss with their colleagues as they were learning, to learn with and from their colleagues. And so just as we paid attention to the research on how people learn, how adults learn, online practices, we paid really careful attention to what faculty asked for, and we delivered it. We made sure that those four things that I heard over and over and over again—from faculty across the country—we delivered on. We listened to them.
John: Maybe it would help if you sketch out the process of a typical module, because it incorporates all those things. And we’re new to ACUE, but our faculty so far have really been enjoying it and they really appreciate the design of the program. But it might help for our listeners who aren’t as familiar to know how a typical module is structured.
Penny: I’m happy to discuss the learning design because we spend a lot of time and a lot of attention to it. Each module includes 12 components. I can divide those 12 components into four groups of three. So the first three components are really designed to pique somebody’s interest and to activate prior knowledge. So we show an introduction video, where that includes clips from our classroom demonstration, kind of like how 60 Minutes gets you interested in the rest of the show, we’re showing little clips to get folks interested in the topic. We outline very clearly the learning objectives and the rationale for the module, so we connect the practices that they’re going to learn to the research that demonstrates it does impact students, and then we offer a group of questions to activate that prior knowledge because what we know about that is if you activate prior knowledge, you’re more ready for new knowledge. So that’s the first three components. The second three are designed to build that foundational knowledge. We decided to show before tell first. And so we have a classroom demonstration video, where you see faculty utilizing the evidence-based practices being recommended in that module. You hear from those faculty why they’re using those practices and you hear from students about how those practices are impacting their learning. Next component, you hear from the researchers about the research behind that component. We actually utilize speed drawing there, so that it’s not just a talking head, but there’s a little bit more interaction going on and then finally, we offer resources to faculty so that when they implement any one of the practices that they’ve just seen in that classroom demo, they have all the resources they would need to implement. The next three components are about deepening learning, and allowing for that collaboration to happen with their colleagues. And so the first component is some text. We wanted faculty to read a little bit deeper about the practices and the way we do that is to address some of the common misconceptions, common challenges that faculty might think of, and we address those with the research. And so a common challenge or a common misconception will include a couple of paragraphs from the research about why that’s a challenge and how to overcome it or why that misconception exists in the information that kind of helps you see it differently. We follow that by two sections of what we call observe and analyze. Up to this point in any module, faculty would be able to do all of those components on their own online when it’s most convenient for them. With the observe and analyze, oftentimes faculty will schedule a particular day that they’re all going to engage in watching these videos, and the videos are of what I call developing practice. So you’ll remember that faculty would have seen effective practice, they would have heard from the researchers, but now we show them developing practice—somebody doing some things well and some things that could be adjusted some—and that is the conversation that faculty have. So they watch this video, and then they engage in an online conversation—some of our partners will sometimes bring folks together face to face—but they engage in a rich conversation about what that person is doing well, and what they might adjust or tweak.
John: We should note that no actual students were harmed during these demonstration component videos.
Penny: Absolutely. As a matter of fact, during the demonstration videos where we were doing developing practice, students knew what we were doing, and it’s completely scripted. So I think what was interesting about students is they understood when a practice was really effective, because remember, it’s developing. So it’s not like a train wreck, it’s some things being done well, and some things that could be tweaked. And when you think about it, the faculty watching the video are in the same shoes as the person trying it for the first time. So they’re watching somebody try something for the first time making some mistakes but doing some things that are quite good. And they’re able, they have that opportunity, before they’re asked to implement one of those practices in their classroom so it’s a really rich learning opportunity that they get to do with their cohort to collaborate with their colleagues. The last set of components, faculty are asked to practice and reflect and then we do a closing video. So we indicate to faculty, “Here are the learning objectives for the module and here are the practices.” And there’s always between five to 10 practices offered in every single module. And we say to faculty, “Choose one,” and that’s important. In adult learning you don’t want to say, “This is the one thing you have to do and you have to do it now,” because faculty are teaching different classes, have different students that they’re working with, we want to give them a choice. So they choose one of those practices and they implement it in their classroom. And then what we require is they reflect on that experience in writing. And that written reflection is submitted to us to be scored. We do present to faculty a rubric for how we’re going to score that reflection. So those requirements are up front, we try to practice what we preach, as far as teaching and learning goes. Faculty submit the reflection, we have national readers that score it using the rubric, and if a faculty’s reflection isn’t quite up to our meets category, we get it back to them with specific feedback and they can resubmit. Now we finish every module with a closing summary—again, practicing what we preach, good teaching and learning—close with a summary of the learning objectives and some more commentary from the researchers.
John: A lot of our faculty have commented how they appreciate the fact that the course itself uses all the practices that are implemented—as you mentioned—and they really enjoy the skeletal outlines, they like the ability to go in and critique these demonstrations. And one of the things that we as working with our teaching center appreciate is that we’ve done workshops on many of these topics and some people have attended them two or three years in a row without actually implementing them. And what we really appreciate is the fact that now people have to get past that barrier of actually trying it in the classroom. And a lot of people who have been coming to our gatherings have said they did this for the course and now they’re doing it in every class. So it’s already making some big changes in people’s teaching practice. So it’s been working really well.
Rebecca: I think another real strength is the external reviewers is really important so that as teaching and learning center staff, we can support our colleagues and not feel like there’s some sort of punitive relationship where we’re judging.
Penny: Yeah, we are a learning organization and so actually when we first piloted a smaller number of the modules, we had the facilitators—our course facilitators, often folks from an institution’s teaching and learning center—scoring their reflections, and they were crystal clear with us that that didn’t feel right. And so we took that on, so that they could really be the coaches that we want them to be with the cohorts.
Rebecca: I think that works really well and I think that really encourages faculty to follow through and to do them and to actually take the actions in the classroom. So I think we really benefited from that particular feature.
Penny: Yeah. I know our mission has been to realize student outcomes— better retention, graduation rates, better learning— through quality instruction. And so in order to impact students, we knew faculty had to go beyond learning these evidence-based practices, but actually using them and so the requirement to complete a module became the implementing of one of the practices. And then what we know to be true in professional development is reflection is such a strong way to not only implement but actually to continue thinking about what went well, what didn’t go well, what might I refine, et cetera. That’s really putting you on the trajectory to becoming a better and better instructor.
Rebecca: I think one of the other interesting advantages of this particular online course is that a lot of our faculty may never have taken an online course but may be asked to teach online courses, so having the experience of a well designed online course is an important experience, especially as faculty move more and more into teaching online and having an idea of how to implement some of these practices, not just in face-to-face situations, but also in online or hybrid situations.
John: And we should also note that in each module, the options that people have could be either for a face-to-face class, or there’s a set of options for people who are teaching online, so it facilitates both types of instruction directly for people with different teaching schedules.
Penny: And we have actually even brought that to a more sophisticated level. So we will be offering our course in online essentials coming up in the next few months, where if we had a cohort of online instructors, they would be doing an observe and analyze about online instruction versus face-to-face so that they would really have that full experience of, “How do I do this core set of skills needed to be an effective instructor online?” So we’ve gone beyond just offering the online resources, to making sure we offer some real high quality learning experiences for them.
Rebecca: That’s great.
John: You mentioned the goal of improving instruction and improving all these outcomes. I know that there’s been some research that has been done at some campuses in terms of what sort of impact this has had. Could you tell us a little bit about what’s been found in terms of the effectiveness of this program in improving student outcomes?
Penny: Absolutely. We’re really, really proud of the work that we’ve done with regards to efficacy. And I think it’s important to recognize that when we partner with any institution, we partner to assist and support implementation. So when you partner with ACUE, we don’t say, “You can click on here and get to our courses, and good luck!” [LAUGHTER] When we partner, every institution has an academic director who will work with the campus lead—oftentimes the teaching and learning center folks as well—to design the course sequence and cadence and make sure that it makes sense for that particular group of faculty. And then in addition to assisting with implementation, we actually study efficacy. And we are very proud of multiple studies now demonstrating student impact. But I always like to indicate that the first set of data that we collected was around faculty, because as I was mentioning before, if faculty aren’t engaged with the course, faculty aren’t learning, and faculty aren’t changing their practices, then you have no hopes of seeing student impact. And we’re particularly proud of what we have with regards to faculty data across over 2,000 faculty members. Ninety-seven percent on average report that the course is relevant. On average, faculty report learning 55 new practices and learning more about 71. And then on average, faculty report implementing 28 new practices as they engage with the 25 modules and a plan to implement 28 more. So we’ve got that faculty data that says to us, “Hey, you know what, you’ll likely have student impact data,” because again, all of the practices in the course are evidence based, they’re already research based. And we’re, again, really proud to share some of the findings we have at Delta State, we have a study where we were able to show an increase in A’s, B’s and C’s, and a decrease in DFW’s. At Miami Dade College, we were able to show an all of these results are statistically significant. In fact, I invite anyone to go on our website, look at the impact page, if they’re particularly interested in the statistical analyses. At Miami Dade, we saw increased student engagement, comparing faculty to themselves before and after they engaged in the course as well as to a matched cohort. We saw an increase in grades. At Texas Women’s University we saw an elimination of course completion gap, a rracial course completion gap. And at Broward, we actually gave students surveys where they indicated that they had engaged regularly in evidence-based teaching practices. And we’ve got a number of studies currently going on so we have been able to show and realize the student level impact that you might expect as faculty start to regularly use evidence-based teaching practices. It’s really, pretty quite amazing.
John: How many schools have participated in this program?
Penny: So currently, we are partnering with over 100 colleges and universities across 37 states. And again, as we partner with any university, we work with them to design the course offering for that particular set of faculty at that particular institution.
John: We appreciated the fact that since we started in late January that the structure was able to accommodate teaching schedules of our faculty, so that people were doing things that were relevant at that portion of the year.
Penny: Yeah, I am particularly proud of the fact that this is not just some lockstep set of courses we ask you to follow, but rather thoughtfully sequenced, dependent on when faculty are starting to engage in the course, and we sequence in a way so that faculty pretty early on—as they implement in their classrooms—start to have some positive feedback from students because that itself is pretty motivating.
Rebecca: Yeah, I think one thing to point out is that we often think about when you teach someone how to teach, you start with the syllabus or you start at the beginning, and we started in the middle, because we were in the middle of the semester, and it made perfect sense for our faculty. I think that it was really effective and I think that the faculty really appreciated that they were able to do stuff right away and not plan things for a semester out.
Penny: Yeah, what we found essentially is as much as I love to think about learning outcomes, and aligning my assessments and aligning my activities, that’s not what everybody enjoys doing. And it’s best to put that towards the end of a sequence. So that faculty really can utilize practices that connect with their students, motivate their students, really embrace the diversity in their classroom, and have those kinds of interactions and then get to, “Okay, so how do I structure this? How do I write a learning outcome that really helps students learn more? How do I make sure my assessments are aligned,” et cetera. That’s work that’s best after they’ve had some of those other experiences.
John: And after the toolkits have been developed, so they have activities they can plug into those learning objectives.
Penny: I do think that when an institution feels like, “Gosh, we need to do something about courses,” they’ll often go to course design as their strategy and leave out the how the course is taught all together and just think the redesign is going to do it, but it really is the combination.
Rebecca: So we always wrap up by asking what’s next?
John: For either you or ACUE?
Penny: For both me and ACUE—I’m happy to say—as I described before, we’re a learning organization. So we are constantly listening to our partners, seeing what’s happening in higher ed where we think we might be able to have some positive impact. But one of the key areas—no surprise—is continuing education. So, we’re helping faculty have this strong foundation, but we know it takes a lifetime to become an effective instructor. And so we want to support faculty in continuing to build on that strong foundation. As well as looking at what are some other areas in higher education where we might be able to offer some courses and some learning that would assist with, again, realizing student success.
John: We’ve really enjoyed talking to you and we’re really enjoying the program here.
Rebecca: Yeah, thank you so much for joining us today.
Penny: I’m so happy that folks are enjoying the program. When we hear from faculty and we hear the kinds of appreciation and even as they talk about how their students are more engaged or learning at deeper levels, there’s simply nothing better than that, and so we’re excited to be working with you folks and with folks across the country.
John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.
Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.
John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.