In a world where data are proliferating as never before, more policymakers are relying on research evidence to serve the public good. What are the research methods that offer the most useful data to policymakers in this rapidly changing landscape? Find out in #EvidenceInsight, a new video series from Mathematica Policy Research.
Randomized controlled trials (RCTs) are considered the gold standard of rigorous research design. For decades, Mathematica researchers have designed, executed, and replicated large-scale RCTs in many different policy and program areas. Today, greater availability of high quality administrative data—along with an abundance of emerging technologies—have increased demand for faster program evaluation with equally robust results. This demand, coupled with shrinking resources, has motivated researchers to consider new methods that are more efficient and less expensive than RCTs, but just as reliable.
Our new video series is designed to help policymakers who need access to strong evidence. Watch the videos below to learn more—and share your #EvidenceInsight with us on social media.
Ann Person, director of Mathematica’s Center for Improving Research Evidence, kicks off the new #EvidenceInsight video series by describing each of the four cutting-edge research methods featured in the series: Bayesian methods, adaptive randomization, predictive analytics, and rapid-cycle evaluation. She also talks about how valuable it is for policymakers to rely on evidence in making decisions that affect public policy. Watch the video.
Predictive Analytics: Transforming Decision Making in Three Steps
In the fourth and final episode of Mathematica’s #EvidenceInsight video series, we dive into predictive analytics, an emerging research field in which data and evidence are used to predict outcomes and inform decision making. In this video, Mathematica experts break the process into three essential steps: (1) define the problem, (2) design the predictive model, and (3) make the predictions. When applied to policy-relevant research questions, predictive analytics can be a valuable tool for organizations seeking to improve their services and their decision making processes by identifying what works for whom under what conditions. Watch the video.
Rapid-Cycle Evaluation: Determining What Works in Less Time
In the third installment of Mathematica’s #EvidenceInsight video series, Alex Resch, associate director of Mathematica’s human services research, describes how rapid-cycle evaluation can quickly determine the impact of small program changes on efficiency and effectiveness. Rapid-cycle evaluations measure impacts, involve the use of rigorous methods to identify causal links, rely predominantly on existing program data, and focus on results that can be observed in a short time. Because rapid-cycle evaluations proceed relatively quickly, the results give decision makers the timely and reliable evidence needed for continual program improvement. For instance, the Office of Education Technology has partnered with Mathematica to help local school districts use rapid-cycle techniques to identify effective educational technologies and software applications. Watch the video.
Adaptive Randomization: A Fresh Perspective on Traditional Research Design
In the second installment of Mathematica’s #EvidenceInsight video series, data scientist Mariel Finucane and researcher Ignacio Martinez discuss adaptive randomization, an innovative extension of traditional randomized controlled trials (RCTs). Faster and less expensive than traditional research methods, adaptive randomization lets us adjust our methods in response to accumulating evidence without losing the methodological principles of randomization. This new research method, when combined with access to rich data sets such as electronic health records, contributes to our understanding of how specific program elements can be refined to better meet the needs of particular subgroups. Watch the video.
Bayesian Methods: A Faster, Probabilistic Approach to Research Design
In our first #EvidenceInsight video, statistical analyst Lauren Vollmer talks about how Bayesian methods can help researchers overcome the limitations of traditional evaluation techniques. She reviews the strengths, limitations, and context of the Bayesian methods, which allow researchers to use information from outside resources to supplement traditional data collection techniques, enabling insights that would not have been possible with the traditional methods alone. Bayesian methods, which produce precise probabilities, enable policymakers to target programs and policies to meet the needs of specific audiences. Watch the video.