7. Investing in What Works through Direct Services

The Labor and Workforce Development Agency administers a range of programs that it evaluates to ensure services are evidence based. Through investments in the CAAL-Skills workforce data governance system, a study led by the California Policy Lab (CPL) published in August 2022 provides causal impact analysis for six out of ten job training programs included in the CAAL-Skills data and evaluation system. Analysis suggests these programs:

1. Increased employment and earnings for participants in the WIOA Title I – Adults, WIOA Title I – Dislocated Workers, and the State Certified Apprenticeship programs.
2. Increased employment but not earnings for participants in the Trade Adjustment Assistance and Career and Technical Education programs.

This evaluation analysis complements individual workforce program evaluations, such as the ETP Program Evaluation, America Job Centers of California (AJCC), Regional Plan Implementation (RPI) evaluations, and Breaking Barriers evaluation reports.

The California Tobacco Prevention Program recently evaluated their Behavioral Health Initiative. This project funded residential behavioral health treatment programs to increase tobacco use treatment, smoke-free campus policies, and other wellness initiatives. The evaluation found a significant decrease in smoking rates and a significant increase in receiving nicotine replacement therapy among clients after the intervention.

The Colorado Reading to Ensure Academic Development (READ) Act, amended in 2019, emphasizes the use of evidence-based instructional practices to support early literacy. It required the use of program funds for external program evaluation; created a science-based early literacy block grant program of $2.5 million annually; and required districts receiving funds in the school year 2021-2022 to have K-3 teachers complete evidence-based training in teaching reading.    

Colorado’s Behavioral Health Administration manages and operates the I Matter program, which connects youth and parents to counseling services. The program offers up to 6 virtual therapy sessions, promotes services through an advocacy campaign, and will reimburse participating therapists for services. This program meets our standards of evidence-based as the program collects and analyzes performance and evaluation data and is a program rooted in evidence of positive outcomes for those with mental health issues.

The Colorado Governor’s Office of State Planning and Budgeting (OSPB) invests in evidence-based policy through its own grant program, where departments submit, outlined in our yearly budget guidance, grant applications in a format similar to OSPB’s yearly budget requests which use a decision item template. All requests must include information on where the proposed program falls on the Colorado evidence continuum. Below you will find a full repository of OSPB’s currently funded programs, however specific funding amounts cannot be shared at this time. Specific grant outcomes and details can be found on OPSB’s website under “Grantee End-of-Year Reports.”

A 2015 Connecticut law defines three tiers of evidence for programs operated by the Connecticut Departments of Correction, Children and Families, and Mental Health and Addiction Services, and the Court Support Services Division of the Judicial Branch: evidence-based, research-based, and promising. The law requires these agencies to categorize their programs by the evidence tiers in even-numbered fiscal years. Additionally, the law charges the Institute for Municipal and Regional Policy at University of Connecticut with submitting an annual report containing a cost-benefit analysis of the programs. In 2020, the report entitled Benefit-Cost Analyses of Evidence-Based Programs showed that 108 programs and services administered by the Judicial Branch’s Court Support Services Division and the Department of Correction were identified as evidence-based.

The Maryland Department of Public Safety and Correctional Services provides Victims Service Coordinators to respond to telephone inquiries from crime victims and parole and probation agents regarding court ordered restitution and domestic violence services. Results for America’s Economic Mobility Catalog describes the correlation between community-based victim services for restorative justice and economic mobility. Maryland utilized these data to provide American Rescue Plan resources to continue the work of Victims Service Coordinators without disruption during the pandemic.

At the Maryland Department of Housing and Community Development, the Emergency Housing Program (EHP) is intended to reduce the number of individuals and households experiencing homelessness. Through the American Rescue Plan, an additional $15 million in funding was provided to Maryland’s 16 Continua of Care (CoCs) and Local Homeless Coalitions to mitigate the impact of COVID-19 on vulnerable households. Speed was required to help people regain stability in permanent housing and to prevent households from becoming homeless. Data, including the Urban Institute’s Eviction Risk Average Percentile Score, was used to establish the funding formula. And use of evidence was required, specifically the use of a Housing First approach from the Results First Clearinghouse, noting its positive impacts based on rigorous evaluations.

In 2012, Michigan passed Act 291, requiring the state departments of Community Health, Human Services, and Education to allocate funding to those home visiting programs that have strong evidence of effectiveness. The law specifies that two categories of programs are eligible for funding: evidence-based programs—those that are supported by rigorous research and have clear designs and implementation standards—and promising programs. To be eligible for continued funding, promising programs are required to undergo an evaluation of their effectiveness. Because effective implementation plays a critical role in achieving expected outcomes, all programs are required to maintain fidelity with key elements of their designs. Similarly, in 2017, Michigan passed Act 5, requiring the Michigan Department of Corrections to allocate 100% of funding to evidence-based practices within four years of implementation.

The State of Missouri provides grant funding to the University of Missouri to implement the Age-Friendly Sustainable Smart and Equitable Technologies (ASSETS) to combine components of aging in place (AIP) and care coordination with low-cost smart home activity sensors. ASSETS for AIP allows rural Missouri older adults and those with disabilities remain in their homes by passive sensor-enhanced care coordinated by healthcare and social service professionals.  ASSETs for AIP combines components of two other evidence-based programs, Aging in Place and CAPABLE, both designed to improve health outcomes for older or disabled adults living in the community. Both interventions have been extensively tested in experimental and quasi-experimental studies supported by Centers for Medicare and Medicaid, National Science Foundation, National Institutes of Nursing Research, & National Institutes on Aging.

In New Mexico in 2014, a Pew Charitable Trusts report cited an example where a cost-benefit analysis revealed a program was not generating cost-savings. The report stated that the New Mexico Corrections Department program inventory, conducted in 2012, “found that fewer than 10 of the more than 40 programs studied were evidence-based.” Further, “the department’s drug abuse prevention programs, Therapeutic Communities, was not operating according to its evidence-based design and, as a result, was not achieving desired outcomes. The department subsequently replaced the program with an alternative, Residential Drug Abuse Treatment, that the Results First analysis showed would generate a return of $4 in benefits for every $1 invested.”

For over a decade, New Mexico’s Children, Youth, and Families Department has used the Missouri model for its juvenile justice program. This program focuses on providing nonpunitive approach to rehabilitation for youth including keeping youth near their family and providing evidence-based behavioral health services as needed. This model has been recognized as evidence-based by Annie E Casey among others. Additionally, since 2012, New Mexico has been investing in home visiting. In FY22, more than 6 thousand families were served with the program and 4,200 of these families were served through an evidence-based or research- based model. The state currently uses two evidence-based models, Nurse Family Partnership and Parents as Teachers and one research-based model, First Born. The results of the home visiting program are annually published in an outcomes report.

Beginning in 2013, the New York State Division of Criminal Justice Services developed a cost-benefit analysis, based on an initial technical report, to outline the impact, costs, and benefits of specific criminal justice interventions. As a result of these efforts, New York has continued to operate Alternatives to Incarceration programs, a $17 million initiative in 2021-2022, to support evidence-based interventions.

Governor Kathy Hochul allocated $75 million to the State University of New York (SUNY) Transformation Fund in the FY 2024 budget. With this investment, 25 SUNY campuses will replicate the City University of New York (CUNY) Accelerated Study in Associate Programs (ASAP), which dramatically increases associate degree completion, and Accelerate, Complete, and Engage (ACE), which is the baccalaureate version of ASAP. ASAP|ACE have been shown in randomized controlled trials to significantly improve associate and bachelor’s degree completion rates and help close opportunity gaps, offering resources and supports including tuition waivers, funding to offset transportation and textbook expenses, academic assistance, comprehensive personalized advisement, and career development activities. The expansion at SUNY will support at least 3,750 students, or inaugural cohorts of 150 students, at each of the 25 participating campuses.

Ohio prioritized a need to reduce infant mortality rates across the state. The IOP Platform was used to expand and enhance predictive profiling models that leveraged cross-agency data to determine those most at risk for infant mortality, and design targeted interventions based on the data. The project involved securely linking more than 30 data sets from multiple state programs to form a 360-view of the clients that allowed the state to: (1) determine indicators that are significantly tied to infant mortality – leading indicators of positive and negative outcomes; (2) create models to predict characteristics of mothers most likely to benefit from interventions; and (3) explore which State programs for mothers are most effective. Over $300,000 was spent in fiscal year (FY) 2023 to develop additional data-informed models and dashboards to implement the evidence-based interventions. In FY 2024 the state general fund will be investing the following: Healthy Beginnings at Home $3,000,000; Help Me Grow $46,500,000; Infant Vitality $15,361,000.

Ohio partnered with the Dolly Parton Imagination Library of Ohio by investing $16 million during FY 2022 and FY 2023, and proposed an additional $16 million in FY 2024 and FY 2025. This is an evidence-based program to increase literacy by providing a new book each month to registered children ages five and under. Several local studies in Ohio show the positive impacts of children receiving these books, including increasing the percentage of children considered “on track” during Kindergarten Readiness Assessments.

A 2003 Oregon law provides a definition of evidence-based program as a program that “incorporates significant and relevant practices based on scientifically based research; and is cost effective” and states that the Oregon Department of Corrections, the Oregon Youth Authority, the Oregon Youth Development Division, and “the part of the Oregon Health Authority that deals with mental health and addiction issues” shall (1) spend at least 75% of state moneys that the agency receives for programs on evidence-based programs; (2) perform cost-benefit analyses; and (3) compile a biennial program inventory with results from funded programs. As of 2022, the Oregon Youth Authority spent 88% of General Fund ($59.6 million) and 91% of total funds ($88.5 million) subject to Oregon’s Senate Bill 267 (SB267) on evidence-based programming. In a 2022 report to the legislature, the Oregon Department of Education Youth Development Division indicated that 94% ($2,420,087) of reported expenditures support local programs that have met evidence-based practice criteria.

As part of the implementation of the 2003 Oregon law on evidence-based health, youth, and corrections programs, Oregon Health Authority (OHA) maintains an ongoing inventory of approved evidence-based practices in behavioral health, including culturally relevant services. OHA has also established the Health Evidence Review Commission to continually review and update an ongoing inventory of evidence-based practices at work within OHA. The commission vets medical evidence to identify new and valuable practices and maintains a publicly available list of practices in a wide range of clinical and treatment areas.

In 2020, the Pennsylvania Department of Education (PDE) launched the PDE Evidence Resource Center to help Pennsylvania schools thoughtfully apply high-quality, relevant research to their local settings. By listing evidence-based interventions in the areas of instruction and curriculum, student support and professional development, and student wellness, this tool helps school districts identify and select strategies to address every aspect of school improvement. This tool also is being used for the PDE’s research agenda that allows the department to identify and measure its progress in meeting its research and learning priorities.

For over 10 years, the Department of Corrections (DOC) has published a biannual State Intermediate Punishment (SIP) performance report that shows significantly lower recidivism rates for the program participants of SIP back to 2005. To increase participation in these effective programs, Act 115/Justice Reinvestment was passed in 2019. DOC expanded the SIP program (now called State Drug Treatment Program) into more institutions, and the savings generated from the inmates going through the program (as a result of their reduced stay in prison and not receiving parole supervision) will be reinvested into county probation programs, sentencing, and victim services.    

Pennsylvania has been recognized as a national leader for its efforts to increase availability of the lifesaving opioid overdose reversal medication naloxone. The Pennsylvania Commission on Crime and Delinquency (PCCD) administers the Naloxone for First Responders Program in collaboration with Department of Drug and Alcohol Programs (DDAP) and other sister agencies. Since the program’s launch in 2017, PCCD has collected, tracked, and analyzed naloxone distribution data via a network of 48 Centralized Coordinating Entities and other organizations receiving naloxone directly via a statewide portal (launched in 2021). This distribution data, coupled with drug overdose surveillance data from the DOH ODSMP, has informed the development of a statewide “Naloxone Saturation Plan” by DDAP as part of Pennsylvania’s State Opioid Response (SOR) Plan last year. Drug overdose surveillance data also is shared regularly with the eight agencies that are part of the Interagency Substance Use Response Team (ISURT) for its use in response planning. Several agencies monitor if medication is reaching people and places with the greatest need.

PCCD conducted original recidivism research with assistance from the Indiana University of Pennsylvania to confirm if the County Intermediate Punishment (CIP) program (i.e., diversionary sentence of probation with restrictive conditions with intensive supervision for drug and alcohol offenders), which is supported annually by an $18 million state appropriation, yields better recidivism results for individuals that participate in the program versus a cohort of similar individuals sentenced to state prison. The study found that an individual diverted to a CIP sentence had a lower three-year recidivism rate (25.7%) than a comparable individual sentenced to state prison (34.9%). This study has been used to solidify support for maintaining CIP funding. A presentation of the study was reported during a commission meeting.

Tennessee’s Office of Evidence and Impact (OEI) defined five evidence steps. The Office also completed inventories of state- and federally-funded program areas including corrections, mental health, substance abuse services, early literacy, labor and workforce, economic development, and higher education as a part of the state’s evidence-based budgeting initiative.  Several evidence-based programs are directly administered by state staff and agencies submit updated program outcome data to OEI annually. Of those programs currently inventoried, 30 have evidence or strong evidence and over $1.6 billion state and federal has been invested in these programs annually. As Tennessee continues to inventory programs across every department, those numbers will grow to show the full picture of evidence-based spending on direct services.