Feasibility and Conduct of an Implementation and Impact Evaluation to Inform High Quality Data-Driven Instruction (DDI)

2012-2018
Prepared for
U.S. Department of Education, Institute of Education Sciences

Data-driven instruction (DDI), whereby schools and teachers use student data in a formative way to improve instructional practices, is receiving increasing attention and support as a promising approach to school reform, but little is known about the strategy’s effectiveness in changing teaching practices and raising student achievement.

In this study, Mathematica is conducting an experimental impact evaluation of the effects of DDI on student achievement. This involves the implementation of high quality DDI professional development and estimating its effects on student achievement. Schools implementing the intervention receive a half-time data coach who focuses on implementing DDI in grades 4 and 5. With the school principal, the coach organizes and facilitates a school leadership team that ensures that teachers have access to student data from formative assessments and oversees intervention activities across the school. The data coach also organizes and facilitates teacher collaboration teams in grades 4 and 5, professional learning communities of the grade teachers that collectively examine data to identify key concepts that students have not mastered and work to determine and implement improved instructional practices to address these weaknesses. To help data coaches, principals, and other instructional leaders with implementation, the intervention includes a series of professional development sessions focused on various aspects of DDI as well as the study's DDI provider, Focus on Results.

The study will address the following research questions:

  • What is the impact of comprehensive DDI on student achievement after a year and a half of full implementation
  • How do teachers’ instructional strategies, collaboration, and satisfaction levels differ between treatment and control schools?    
  • What challenges do schools face implementing a comprehensive DDI program?
To answer the study's impact questions, the evaluation is using an experimental design. Within 12 districts recruited to participate in the study, 102 schools have been randomly assigned to either a treatment group that is implementing a comprehensive DDI program or a control group that is not implementing the study's key DDI features during the 2014-2015 and 2015-2016 study years. While both treatment and control schools are in districts in which interim assessments are regularly administered and that have data systems that produce data for teachers, only treatment schools are receiving support from Focus on Results to implement the study's DDI program.