Employment service providers looking to replicate evidence-based interventions will likely face challenges in determining how to implement the most important elements, or core components, of those interventions. There are several reasons that this is challenging, including that employment and training interventions are often a bundle of many services, making it hard to disentangle which of those services may be driving results. Another challenge is that the employment and training literature has very few studies that use rigorous, quantitative methods to test the impacts of different components of an intervention. In the absence of quantitative data on core components for an intervention, employment service providers can use qualitative strategies to carefully determine how to implement and intervention. Currently, federal agency staff, researchers, and service providers are actively applying qualitative strategies to try to identify the core components of an intervention. This brief explores those strategies, providing guidance that may be useful to employment service providers and other human services agency staff who want to replicate evidence-based interventions.
Primary Research Questions
- What are recommended strategies for qualitatively identifying and documenting the core components of an intervention?
- How can organizations successfully implement core components of evidence-based interventions?
- How do organizations overseeing the implementation of evidence-based programs monitor fidelity?
Purpose
This special topics report is designed to provide guidance for employment service providers and other human services agency staff who wish to implement evidence-based programs but find little information about the core components of those programs. The report discusses recommendations from the literature and informant interviews about strategies to qualitatively identify core components. The report then summarizes the processes and methods that other human services agency staff have used to identify, document, and implement the core components of interventions.
Key Findings and Highlights
As a results of a scan of the literature and informant interviews, the Pathways to Work Evidence Clearinghouse shares recommendations that can help organizations identify and document core components that have not been well specified, and provides suggestions for beginning the process of implementing core components and monitoring fidelity. The recommendations are to:
- Scan the available research to look for common features of evidence-based interventions focused on achieving similar results.
- Engage other agency staff and partner organizations to draw on their experience implementing program elements and begin to test implementation of those elements.
- Begin to develop descriptions of each core component of the intervention to establish a shared understanding that will guide future implementation.
- Take a deliberate, step-by-step approach to plan for a new intervention, considering model fit and consciously adapting as needed before implementation begins.
- Build and maintain an environment that supports successful implementation of core components by creating detailed intervention descriptions, securing funding and staffing to support this work, and expanding data management capacity to create a database to track partner implementation of core components.
- Use data from program monitoring and evaluation to understand how participants engage with individual program components as a way of monitoring fidelity.
- Use or build a fidelity monitoring tool
- Partner with researchers, key model developers or original staff, and experts to provide guidance on implementation.
- Make intentional adaptations during implementation as needed.
Methods
The Pathways Clearinghouse team conducted qualitative, 60-minute interviews with a targeted set of respondents from six organizations that have experience related to researching or implementing the core components of evidence-based programs. Respondents included federal staff who have experience using qualitative approaches to identify core components, researchers with expertise identifying core components and supporting their implementation, and human services agencies that have been through the process of implementing employment programs and reporting on their core components.
Citation
McCallum, Diana, Mastri, Annalisa, Christensen, Benjamin, Boland, Bethany, and Cadena, Candace (2022). Key Elements of Employment Programs: Strategies from the Field for Identifying, Implementing, and Sustaining Core Components. OPRE Brief # 2022-212, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.