Editor’s Note: This blog originally appeared on the U.S. Department of Education’s Office of Educational Technology (OET) Medium page and is being re-posted with their permission. You can follow the full series at https://medium.com/building-evaluation-capacity.
by Brad Flickinger under Attribution 2.0 Generic (CC BY 2.0)
When it comes to the process of selecting and implementing ed tech in K-12 schools, change is needed. While districts are estimated to spend 14 billion dollars on digital learning tools this year, many of those tools will not actually be used by teachers and students. One reason for this is that those making purchasing decisions don’t always know what will meet educators’ needs and improve student learning before trying it out in the classroom.
Faced with this challenge, some districts have begun conducting rapid cycle evaluations to determine if a particular educational technology is moving the needle for students. When done well, these short studies generate sufficient evidence to help district leaders make better, more informed decisions, and ultimately help improve student outcomes.
But like adopting any new process, finding time and resources to effectively conduct rapid cycle evaluations that generate meaningful evidence can be hard. A critical step for districts to seamlessly integrate an evidence gathering system is to identify a rapid cycle evaluation champion. Below we highlight five characteristics to look for when identifying a rapid cycle evaluation champion.
Champions can articulate why evidence matters and create buy in.
At a recent conference, former California Superintendent of the Year Dr. Devin Vodicka advised education leaders looking to create positive change: invite everyone who cares to work on what’s possible.
This simple phrase encompasses two key qualities of any champion of change: they care enough to take the lead, and they are able to articulate a clear vision to encourage others to participate. Rapid cycle evaluation champions are able to help all stakeholders understand why it’s important to gather data on whether or not a tool is helping them reach their goal of supporting student learning and engagement.
Champions can leverage their connections to embed evidence gathering and rapid cycle evaluation into the district ed tech decision making process.
An effective rapid cycle evaluation champion is able to help districts modify their ed tech decision-making process to include deliberate evidence gathering. Rebekah Kim, the Director of Instructional Supports at Highline Public Schools, collaborates with district leaders on how to implement their evidence-based ed tech decision-making vision. With support from team members from the Data & Assessment Team, Technology Department, and Personalized Learning Specialists, she has successfully refined district processes for ed tech implementation to focus on learning conditions and strategies that make a positive impact on student outcomes.
Champions have the access and skills needed to manipulate student data.
The usefulness of rapid cycle evaluation findings depend on the quality and relevance of data gathered. Rapid cycle evaluation champions can identify the best data sources to evaluate the success of an innovative program and they have access to it. While champions don’t have to be data gold medalists, they should either feel confident collecting and analyzing data themselves, or know when and who to ask for help.
Champions are humble and actively seek input from all stakeholders.
Rapid cycle evaluation champions know they don’t have all the answers. They recognize the importance of educator and student voices, and they value input from researchers and product developers when evaluating a digital learning tool. For example, Aileen Owens, Director of Technology and Innovation at South Fayette School District, has been a champion for gathering evidence on computational thinking programs to encourage underrepresented groups to engage in AP computer science. Recognizing the value of stakeholder input, she has created space for educators to engage in and co-lead the process, developed instruments to gather student feedback, and partnered with researchers at Carnegie Mellon University.
Champions see value in sharing what they’ve learned.
Dr. Todd Keruskin, Assistant Superintendent at Elizabeth Forward School District (which you can learn more about here), has been a rapid cycle evaluation champion in his district for five years. He understands how difficult it can be to make decisions about ed tech without evidence. After each trial, he shares what he has learned through formal and informal channels — at conferences, with peers, and through Pilot Study Briefs. He said, “We’ve learned a lot about innovative programs from other district leaders. It’s our responsibility to continue to work together to gather and share evidence so we ensure education technology meets the promise of supporting all learners.”
To find free tools and resources to support a rapid cycle evaluation champion in your district, visit the the Ed Tech RCE Coach and Edtech Pilot Framework.
