Case Study: Common Data and Measurement Standards Enhance a Program to Help Low-Income, Low-Skilled Seniors Find Jobs

Benchmarks for the Senior Community Service Employment Program Help Track Milestones

. . . the SCSEP Performance and Results Quarterly (SPARQ) performance report system software allows reporting of individuals and integration of grantee reporting systems.

- expectmore.gov
Project Facts

SPARQ, an automated system for managing participant data and evaluating program performance, launched in 2004 as a CD for desktop computers. SPARQ then moved to a web-based application, merging with a business intelligence module.

By 2012, all grantees moved to SPARQ as their official case management system. Today, SPARQ serves more than 70 grantees and has more than 54,000 active participants.

Since its inception, SPARQ has served more than 365,000 participants.

Notes on Technical Assistance. Mathematica's technical assistance involved a three-pronged approach:  

  • System support to grantees and subgrantees in their use of the SCSEP software
  • Program technical assistance that involved developing and distributing documentation such as data definitions, guidance regarding SCSEP policy, and recommendations on how best to capture and validate SCSEP data
  • Reporting and performance measurement processes for grantees and subgrantees

 

The Issue

Grantees receiving funds for the SCSEP, a program serving low-income, low skilled seniors, faced the challenge of inconsistent program performance reporting and monitoring. Without universal standards, benchmarks, and best practices, it was difficult to know whether these programs met goals, milestones, and overall outcomes—providing low-income, hard-to-employ seniors with service-based experience and job training.

The Approach
From 2004 to 2007, SCSEP enacted new program requirements calling for better performance monitoring and measuring as well as more stringent data validation. Mathematica’s Data Management and Analytics department was one of three stakeholders that designed, developed, and supported a case management, reporting, and performance measurement system—the SCSEP Performance and Results Quarterly Progress Report (SPARQ) system. 

Mathematica employed its deep knowledge base across a range of disciplines including data analytics, programming, systems implementation, statistics, employment and training research, and policy. Our business development team and employment and training researchers’ experience navigating SCSEP policy helped inform the system’s design and implementation, and our expert statisticians created algorithms to ensure high quality data validation. Programming contributions included identifying and documenting all end user requirements and reviewing those requirements with program stakeholders, as well as verifying and validating all software enhancements.

The Impacts

Mathematica tapped its multidisciplinary team of experts to create a solution that brought order, processing standards, and best practices to a complicated data collection and validation system previously conducted independently by individual grantees. Today, SPARQ uses the most granular data available to create performance measures and a case management system that ensures grantees meet the Senior Community Service Employment Program’s specific goals and overall high standards—helping low-income, low-skilled seniors find gainful, fulfilling employment. In addition:

  • SPARQ helped ensure pre- and post 9-11 veterans and their spouses received employment and training assistance.
  • Enhancements to the case management system helped track participants who had reached time limits on their benefits.
  • SPARQ’s program performance measures inform and help determine grantee funding.
To learn more about how Mathematica Policy Research can provide solutions to your communications challenges, contact: Communications@mathematica-mpr.com

This case study is for informational purposes only. Mathematica Policy Research, a nonpartisan research firm, provides a full range of research and data collection services, including program evaluation and policy research, survey design  and data collection, research assessment and interpretation, and program performance/data management, to improve public well-being. Its clients include federal and state governments, foundations, and private-sector and international organizations. The employee-owned company, with offices in Princeton, N.J.; Ann Arbor, Mich.; Cambridge, Mass.; Chicago, Ill.; Oakland, Calif.; and Washington, D.C., has conducted some of the most important studies of education, disability, health care, family support, employment, nutrition, and early childhood policies and programs.

About the Project

The Senior Community Service Employment Program (SCSEP) in the Office of Workforce Investment (OWI) at the U.S. Department of Labor (DOL), Employment and Training Administration (ETA) provides subsidized, service-based training for low-income adults 55 or older who are unemployed and have poor employment prospects. The program helps bridge these workers into long-term, unsubsidized unemployment. 

OWI wanted to implement a program-wide performance measurement system and benchmarks for grantees to monitor SCSEP’s effectiveness. Performance measures included hours of community service, participants’ employment and retention rates and average earnings, service level to participants, and service to seniors most in need. OWI also wanted to standardize the way grantees collected and validated data.Prior to this initiative, grantees used their own systems and approaches to calculate performance measures and collect and validate data. Without common data and measurement standards it was impossible for OWI to aggregate data across grantees or objectively compare grantee performance.

To support the new requirements, Mathematica’s Data Management and Analytics Department helped design, develop, and support a case management, reporting, and performance measurement system—the SCSEP Performance and Results Quarterly Progress Report (SPARQ) system.