Comparing Job Training Impact Estimates Using Survey and Administrative Data
Download
Associated Project
Workforce Investment Act Adult and Dislocated Worker Programs Gold Standard Evaluation
Prepared for:
U.S. Department of Labor, Employment and Training Administration
The data source used for analysis can affect the accuracy and precision of a study’s estimates. In this paper, we examine three sources used to produce impact estimates for the WIA Adult and Dislocated Worker Programs Gold Standard Evaluation: (1) survey data, (2) administrative data from the National Directory of New Hires (NDNH), and (3) Administrative Tax Data. We further explore the strengths and drawbacks of administrative and survey data by directly comparing estimates produced using the survey data and NDNH. We found that both sources were consistent in their main conclusions about program effectiveness but that impacts calculated using the NDNH were smaller than those calculated using survey data, consistent with previous studies of similar sources. Three factors likely explain most of the observed differences: (1) many survey respondents reported jobs not captured by the NDNH; (2) survey respondents typically overreported earnings in any given job, especially early in the follow-up period; and (3) survey respondents typically underreported the number of jobs they held early in the follow-up period.
More like this from Mathematica
How do you apply evidence?
Take our quick four-question survey to help us curate evidence and insights that serve you.
Take our survey