Education Policy Research


	Scientifically based methods are the hallmark of our work evaluating education programs and studying education policy issues. Our studies cover the earliest learning experiences of infants as well as education in the K-12 grades and college years. Our education studies have provided important counsel to policymakers as they seek ideas for improving American education. We have also played an important role in advancing the state of the science in education research. Read more about our work on specific education topics.

Highlights

Teacher Advancement Program Improves Retention But Not Test Scores

The Teacher Advancement Program (TAP) aims to improve schools by raising teacher quality. A new report on TAP schools in Chicago notes that TAP teachers were five percentage points more likely to return to their schools than were non-TAP teachers. However, the program did not produce measurable impacts on student test scores through March of the start-up year. Read more.

Reading Comprehension Curricula Show No Positive Impacts on Achievement

To become successful learners, students need to "read to learn." A report on first-year findings from Mathematica’s study of four reading comprehension programs sheds light on the effectiveness of these curricula in helping disadvantaged students improve their reading comprehension. Overall, the curricula had no positive impact on student test scores, and in some cases, had a negative impact. Read more.

Latest Work

	Reports:
	"Rigorous Evaluation of Roads to Success: Design Report." Duncan Chaplin, Martha Bleeker, and Claire Smither, May 2009. The Roads to Success program was launched in early 2005 to forge connections between students’ school experiences and their aspirations for adulthood, as an ongoing part of middle and high school. This report describes the intervention, research design, evaluation data, analysis methods, and a comparison of baseline characteristics of the treatment and control groups. Funding for the program and study was cut in 2008, so both are scheduled to conclude in 2009. The report also reviews the evaluation plan and possibilities for further research if funding is restored. Back to Top
	"Effectiveness of Selected Supplemental Reading Comprehension Interventions: Impacts on a First Cohort of Fifth-Grade Students." Susanne James-Burdumy, Wendy Mansfield, John Deke, Nancy Carey, Julieta Lugo-Gil, Alan Hershey, Aaron Douglas, Russell Gersten, Rebecca Newman-Gonchar, Joseph Dimino, and Bonnie Faddis, May 2009. To become successful learners, students need to comprehend what they read. Children from disadvantaged backgrounds, in particular, may have difficulty comprehending text because they often lack general vocabulary and strategies for organizing information and gleaning knowledge from text. This report, from Mathematica’s federal study of four reading comprehension programs, sheds light on the effectiveness of these curricula in helping disadvantaged students improve their reading comprehension. Overall, the curricula had no positive impact on student test scores, and in some cases, had a negative impact. The study, a large-scale randomized control trial involving 268 teachers and 6,350 students in 89 schools in 10 mostly large disadvantaged urban districts in 8 states, examined the effects of these curricula on fifth-grade students. Back to Top
	"Evaluation of Experience Corps: Student Reading Outcomes." Nancy Morrow-Howell, Melissa Jonson-Reid, Stacey McCrary, YungSoo Lee, Ed Spitznagel, January 2009. A Mathematica team led by Emily Dwoyer, Kathy Sonnenfeld, and Susan Sprachman provided the data collection services for this study of the Experience Corps (EC) initiative, a program that brings older adults ages 55 and up into public elementary schools to tutor and mentor children who are at risk of academic failure. EC members also help teachers in the classroom and lead after-school enrichment activities. The findings presented in this report demonstrate that students who received EC program tutoring and mentorship showed significant improvements in their reading and comprehension skills and, at the same time, the program was also found to be a low burden to teachers. Back to Top
	“An Evaluation of the Teacher Advancement Program (TAP) in Chicago: Year One Impact Report.” Steven Glazerman, Allison McKie, and Nancy Carey, April 2009. The Teacher Advancement Program (TAP), a whole-school intervention that aims to improve schools by raising teacher quality, provides teachers with opportunities for professional growth, promotion to school leadership roles without leaving the classroom, structured feedback, and performance-based compensation. This report focuses on the Chicago Public Schools, which began implementing TAP in 2007. Early findings from Mathematica’s study, which focused on the district’s K-8 schools, note that teachers in TAP schools reported significantly more mentoring and support than their peers in similar schools. Although TAP led to changes inside schools, these changes did not produce measurable impacts on student test scores through March of the start-up year. In addition, the program had a significant impact on teacher retention. TAP teachers were five percentage points more likely to return to their schools than were non-TAP teachers. Back to Top
	“2008 Principal/Vice Principal Survey Results for Evaluation of the Effective Practice Incentive Community (EPIC).” Duncan Chaplin, Hanley Chiang, Margaret Sullivan, Virginia Knechtel, Dominic Harris, Shinu Verghese, Kathy Sonnenfeld, Barbara Kennen, and John Hall, March 2009. This report presents findings from a baseline survey of principals and vice principals in three sites—the Memphis and DC school districts, and a consortium of charter schools—partnered with New Leaders for New Schools in the EPIC. Within each site, EPIC offers incentives to educators in eligible schools with significant student achievement gains to help document and disseminate their practices. The survey collected information on factors important for informing the implementation and evaluation of EPIC, including school administrators’ knowledge of EPIC, attitudes toward performance pay, and appraisal of educational practices in their schools. Most principals have positive attitudes toward EPIC and merit pay in general. Principals generally prefer the use of school-level performance and achievement growth, rather than teacher-level performance alone or test-score levels, to determine incentive awards, but they exhibit an incomplete understanding of how growth is captured in these performance measures. More Reports Back to Top

	Journal Articles:
	“Statistical Power for Random Assignment Evaluations of Education Programs.” Peter Z. Schochet, Journal of Educational and Behavioral Statistics, March 2008. This article examines theoretical and empirical issues related to the statistical power of impact estimates for experimental evaluations of education programs. The author considers designs where random assignment is conducted at the school, classroom, or student level, and employs a unified analytic framework using statistical methods from the literature. Focusing on standardized test scores of elementary school students, this article discusses appropriate precision standards and, for each design, the required number of schools to achieve those standards using empirical values of intraclass correlations, regression R2 values, and other parameters. Clustering effects vary by design but are typically large. As a result, large school samples are required for education trials, and many evaluations will have sufficient power to detect precise impacts only for relatively large subgroups of sites. Back to Top
	“After-School Program Effects on Behavior: Results from the 21st Century Community Learning Centers Program National Evaluation.” Susanne James-Burdumy, Mark Dynarski, and John Deke, Economic Inquiry, January 2008. This paper presents evidence on after-school programs’ effects on behavior from the national evaluation of the U.S. Department of Education’s 21st Century Community Learning Centers after-school program. Findings come from both of the study’s components: (1) an elementary-school component based on random assignment of 2,308 students in 12 school districts, and (2) a middle-school component based on a matched comparison design including 4,264 students in 32 districts. Key findings include higher levels of negative behavior for elementary students and some evidence of higher levels of negative behaviors for middle school students. Back to Top
	"Academic Achievement and School Functioning Among Non-Incarcerated Youth Involved with the Juvenile Justice System." Jonathan D. Brown, Anne W. Riley, Christine M. Walrath, Philip J. Leaf, and Carmen Valdez, Journal of Education for Students Placed at Risk, January-March 2008. This article reports on the education problems of youth involved with the juvenile justice system but not incarcerated. More than half demonstrated deficits in academic functioning, with standard achievement scores as low as five standard deviations below the normative mean. Non-Caucasian youth and those who received special education services or lived in an urban area had lower achievement. These findings suggest that youth involved with the justice system but not incarcerated demonstrate problems in academic achievement similar to incarcerated youth and may benefit from targeted education interventions. Back to Top

	Issue Briefs:
	“Infusing Academics into Career and Technical Education.” Joshua Haimson, James R. Stone, III, and Donna Pearson, Trends in Education Research, Issue Brief #3, December 2008. Integrating academic learning into career and technical education (CTE) classes can be challenging for educators and curriculum developers but can be aided by securing detailed feedback from CTE teachers. Drawing on a recent study, this issue brief identifies challenges developers faced in infusing more math into CTE curricula and notes that incorporating academic learning into CTE requires substantial time, effort, and other resources. Back to Top

	Other:
	The What Works Clearinghouse (WWC) has released the following new intervention reports and quick reviews: An adolescent literacy intervention report looks at SuccessMaker^®, a set of computer-based courses used to supplement regular classroom reading instruction in grades K–8. Using adaptive lessons tailored to a student’s reading level, SuccessMaker^® aims to improve understanding in areas such as phonological awareness, phonics, fluency, vocabulary, comprehension, and concepts of print. New early childhood education intervention reports look at the following programs: Doors to Discovery™ focuses on the development of children’s vocabulary and expressive and receptive language through a learning process called “shared literacy,” where adults and children work together to develop literacy-related skills. Literacy activities, organized into thematic units, encourage children’s development in a number of areas identified as the foundation for early literacy success. Bright Beginnings is an early childhood curriculum, based in part on High/Scope^® and Creative Curriculum^®, with an additional emphasis on literacy skills. The curriculum consists of nine thematic units designed to enhance children’s cognitive, social, emotional, and physical development. Special emphasis is placed on the development of early language and literacy skills, and parent involvement is a key component. Another new intervention report looks at the research on Sheltered Instruction Observation Protocol (SIOP), a framework for planning and delivering instruction in content areas such as science, history, and mathematics for English language learners. Using the SIOP framework, teachers modify the way they teach so that the language they use to explain concepts and information is comprehensible to these students. Quick reviews include the following: Impacts of Comprehensive Teacher Induction: Results from the First Year of a Randomized Controlled Study—This study examined the effects of comprehensive teacher induction programs on the job retention of new teachers and the academic achievement of their students. Effects of Social Development Intervention in Childhood 15 Years Later—A study that examined the long-term effects of the Seattle Social Development Project, an elementary-school-based intervention designed to improve students’ social skills and engagement. The Enhanced Reading Opportunities Study: Findings from the Second Year of Implementation—A study that examined whether supplemental literacy classes improve the reading skills of struggling ninth-grade readers. Back to Top
	Both Sides Now: The Story of School Desegregation’s Graduates. Amy Stuart Wells, Jennifer Jellison Holme, Anita Tijerina Revilla, and Awo Korantemaa Atanda, January 2009. This book, based on a five-year study of six high schools that had undergone some form of desegregation in the late 1970s, provides a firsthand account of how desegregation affected students during high school and later in life. Their stories, set in a social and historical context, underscore the benefits of school desegregation while providing perspective on the backlash against it. The team conducted open-ended interviews with more than 500 graduates, educators, advocates, and local policymakers. Back to Top
	Mission and Money: Understanding the University. Burton A. Weisbrod, Jeffrey P. Ballou, and Evelyn D. Asch, Cambridge University Press, 2008. Many academic institutions pursue mission-related activities that are unprofitable and engage in profitable revenue raising activities to finance them. This book presents research on schools’ revenue sources from tuition, donations, research, patents, endowments, and other activities. It considers lobbying, distance education, and the world market, as well as advertising, branding, and reputation. The pursuit of revenue, while essential to achieve the mission of higher learning, is sometimes in conflict with that mission. The tension between mission and money is also highlighted in a chapter on the profitability of intercollegiate athletics. More Back to Top