Colorado
Leading Example
8. Investing in What Works through Budgeting
The government has a budget policy that prioritizes evidence of effectiveness in order to achieve equitable outcomes.
8. Investing in What Works through Budgeting
The Colorado Governor’s Office of State Planning and Budgeting (OSPB) continues to make evidence an integral part of the budget process. Since 2016, OSPB has mandated evidence information in budget requests for the Governor’s annual proposal. OSPB provides annual guidance to state departments, stressing the use of data and evidence in the budget process, promoting transparency and informed decision-making. OSPB’s guidance includes the use of a five-step “evidence continuum”, which measures budget requests on the level of evidence gathered and assessed. Pew researchers have also conducted an analysis and summarization of Colorado’s evidence continuum.
By applying the evidence continuum, Colorado is investing in programs and projects that promote the use of evidence-based and informed practices like wraparound services for 500 Medicaid members, drawing from learnings from a Denver housing social impact bond project and a pilot Community Aging in Place – Advancing Better Living for Elders (CAPABLE) program, an evidence-based fall prevention program that will serve 400 aging residents; and funding the University of Colorado Health Sciences Center and for training and education for health-care, behavioral health-care, and public health-care professionals, to further promote the use of evidence-based models of care for treatment of pain and substance use disorders. These specific programs have a focus on promoting equitable outcomes, which state agencies are instructed to analyze and speak upon in their request as detailed in the FY 24-25 budget guidance on EDI and evidence-based policy created by OSPB.
In the 2022 legislative session, OSPB and the Colorado Legislature’s Joint Budget Committee began implementing HB21-284, Evidence-based Evaluations for Budget, which codified requirements to use a consistent evidence framework to assist the legislature’s budget decisions. The Governor’s budget includes assessments of the evidence for each proposal in accordance with HB21-284, and Joint Budget Committee staff review the Governor’s Office characterization of the evidence (e.g., reviewing and critiquing studies cited in the proposal) and also may perform their own independent assessment of the evidence.
Promising Examples
1. Evaluation Leadership
The government has a designated leader and/or team responsible for coordinating evaluation policy and use of findings (e.g., Chief Evaluation Officer).
1. Evaluation Leadership
The Colorado Governor’s Office of State Planning and Budgeting (OSPB) and its director lead the use of data and evidence in the state’s budget process through the review of state agency budget requests based upon their level of evidence and evaluation to inform priority policy and budget decisions. This office engages with the University of Denver’s Colorado Evaluation and Action Lab as well as the Colorado Evidence-Based Policy Collaborative composed of state agency representatives and community organizations. The office also administers a $500,000 annual fund for program evaluation and implementation grants, which provides competitive funding for agencies to undertake evaluation and implementation science projects. OSPB continuously engages with the legislative branch to determine how evidence can be better utilized in the budget process.
With the large influx of the America Rescue Plan Act (ARP), OSPB has applied the evidence continuum as a cornerstone of its decision-making process to ensure the strongest outcomes for residents. The Governor’s Office is evaluating all ARP-funded programs against the evidence continuum and includes information on evidence-based practices in all federal reports.
2. Data Leadership
The government has a designated leader and/or dedicated team responsible for coordinating the governance, collection, sharing, and analysis of data (e.g., Chief Data Officer).
2. Data Leadership
The State of Colorado’s Office of Information Technology (OIT) includes a Chief Data Officer. The Chief Data Officer is responsible for developing and directing the state’s data strategy, data governance, and data management. The position establishes enterprise data policies, standards, and procedures to maximize the state’s investment in data management systems. The OIT’s statewide data management program leverages data and information as enterprise assets and establishes standards and processes to enable more agile solutions and government services. Additionally, OIT’s Government Data Advisory Board (GDAB), created by a 2009 law, advises the Chief Data Officer on activities and policies necessary to develop the interdepartmental data protocol, made up of members from state and local government. GDAB’s mission is to improve the efficiency and effectiveness of state government, citizen service delivery, and policymaking by providing guidance and recommendations on how the state should govern and manage data and data management systems.
In 2022, GDAB created a data governance subcommittee, which will define a statewide protocol for collecting, managing, storing, and linking personally identifying information (PII) and develop best practices for data management and data governance, including data lifecycle, reconciliation, and retention policies and procedures.
3. Investment in Capacity to Learn What Works
The government invests at least 1% of discretionary program funds in evaluations [and evidence building] and uses evaluation findings to advance the agency’s mission.
3. Investment in Capacity to Learn What Works
The Colorado Department of Public Safety’s Juvenile Diversion Program invests 3% ($120,000) of program funds to conduct evaluations. Crime Victim Services’ competitive funding opportunities require all applicants to evaluate the effectiveness of their project. It also requires those who currently receive funding to answer questions regarding their current effectiveness at meeting their funded projects’ goals and objectives, as well as their adherence to their budget.
The Colorado Department of Early Childhood (CDEC) has utilized ARPA funding to embark on a $500,000, 4-year Early Childhood Mental Health Consultation (ECMH) Evaluation, and ongoing funding of nearly $375,000 to maintain a robust data collection system. This program receives nearly $6.5M each year including temporary stimulus funding to build the capacity of adults and early childhood professionals to prevent, identify, treat, and reduce the impact of mental health problems among children from birth to age six and their families. CDEC is putting evidence at the core of its activities, building a causal link between activities and the improved physical and mental health of parents and young children.
Since FY 2017-18, Colorado’s Office of State Planning and Budgeting (OSPB) administers a $500,000 annual fund for program evaluation and implementation grants, which provides competitive funding to support program implementation or evaluation of outcomes. Through this grant, the Governor’s Office has invested approximately $3,735,800 in grants to support evidence-based policy. This includes a $150,000 grant to the Department of Higher Ed for contract evaluation and assessment of scholarship interventions on degree completion, Employee evaluations.
4. Evaluation Policy and Learning
The government has an evaluation policy and a learning agenda to support the building and use of evidence.
4. Evaluation Policy and Learning
The Colorado Governor’s Office of State Planning and Budgeting (OSPB) is home to many research and evidence-based policy initiatives and publishes the results of evaluations funded by the grant program on its website each year. OSPB also publishes annual guidance to state departments requiring the use of data and evidence in the state’s budget process. It also provides definitions for key research and evaluation terms and “encourages agencies to use the evidence continuum outside the budget process as a means to evaluate programming internally, and to guide discussions with OSPB analysts and internal and external stakeholders.” With the large influx of the America Rescue Plan Act (ARP), OSPB has applied the evidence continuum as a cornerstone of its decision-making process to ensure the strongest outcomes for residents.
The Colorado Department of Higher Education (CDHE) developed a research agenda to inform education and training policies and programs. Data sharing is required by Colorado law among the Department of Education, Department of Labor, and Department of Economic Development, which collectively generate research covering the education-to-workforce continuum.
Several projects funded through the American Rescue Plan Act (ARP) plan include program evaluations. The Colorado Evaluation and Action Lab (Colorado Lab) partnered with the Governor’s Office of State Planning and Budgeting to determine appropriate evaluation strategies for ARPA-funded projects with a number of state agencies, including the Departments of Higher Education, Health Care Policy and Financing, Human Services, Labor and Employment, and the Department of Early Childhood.
5. Data Policy and Practices
The government has documented, user-friendly policies and practices to expedite internal and external data sharing, linkage and access.
5. Data Policy and Practices
A 2021 Colorado law modified many of the statewide data practices by specifying requirements regarding the creation of a data-sharing and privacy master plan and sharing of personal identifying information between state agencies. Moreover, it codified the “share first” expectation (24-37.5-704 C.R.S) for data sharing statewide, specifically that, except where prohibited by state or federal law, agencies are preemptively authorized to share data with other state agencies, legislative and judicial branches, political subdivisions, and non-governmental organizations. HB21-1236 requires each agency to conduct a data inventory with assistance from the Office of Information Technology and the Government Data Advisory Board (GDAB). GDAB is also tasked with creating data-sharing agreements, which is being accomplished through subcommittees established in 2022.
The Colorado Information Marketplace is the state’s open data repository with more than 2,000 datasets intended to increase government transparency. Beyond state open data, the Colorado Governor’s Office and the Colorado Evaluation and Action Lab co-designed the Linked Information Network of Colorado (LINC) to facilitate data sharing for research and analytics in 2019. The network is designed to share data across state agencies and provide de-identified data to perform robust, academically rigorous research to inform policy. LINC has a three-tier legal structure, which includes: (1) an enterprise memorandum of understanding (eMOU) signed by all data providers; (2) data-sharing agreements to secure, handle, and anonymize data for all LINC projects; and (3) data licenses with roles and responsibilities for users of LINC project data.
The Governor’s Office of Information Technology (OIT) has also established three new subcommittees (Data Governance, Data Sharing, and Data Inventory) which build on interagency collaboration to inform the GDAB on data policy best practices. In 2022, OIT published the Digital Government Strategic Plan which outlined the State’s vision for harnessing the data that we collect to build an ecosystem that puts customer service first.
OIT does not currently have a public-facing policy on equity in data sharing and management. However, OIT is dedicated to internal conversations on Equity, Diversity, and Inclusion in employment and personnel practices. This includes the implementation of an annual Equity, Diversity, and Inclusion (EDI) plan, and the expansion of training programs, coaching opportunities, and employee resource groups to promote inclusivity and support individuals from marginalized backgrounds. OIT acknowledges the significance of accessibility and is currently developing guidelines that will shape the organization’s technology solutions. On a cross-agency level, the Enviroscreen Tool, developed by CDHPE, generates “scores” using disaggregated census and environmental data to measure environmental impacts on disproportionately impacted communities. The department works with communities and other agencies to be transparent and accessible in this data for better decision-making rooted in environmental justice principles.
6. Investing in What Works through Grants and Contracts
The government defines and prioritizes evidence of effectiveness in grant programs or contracts in order to achieve equitable outcomes.
6. Investing in What Works through Grants and Contracts
In 2021, the Colorado Department of Education used an evidence-based framework to allocate funds to the Colorado Comprehensive Literacy State Development Grant (CLSD) program. Grantees were required to utilize evidence-based interventions that help children develop reading, writing, speaking, and listening skills, as defined by the Colorado Reading to Ensure Academic Development (READ) Act. The State’s grant program placed an emphasis on funding directly supporting historically underserved students, including children in poverty, English learners, and children with disabilities.
The Colorado Department of Education grant program for school improvement, Empowering Action for School Improvement, has required schools to use evidence-based practices as defined by the federal Every Student Succeeds Act (ESSA) since 2018. The application gives points for the use of evidence-based strategies, as well as points based on past performance for applicants who are seeking to expand existing initiatives.
In July of 2021, the Colorado Workforce Development Council launched the Upskilling, Reskilling, and Next-Skilling Workers program to support evidence-based projects that improve workforce development and employment capacity. The application awards points for seven criteria, one of which includes the level of evidence of the proposed approach as defined by the Colorado Evidence Continuum.
7. Investing in What Works through Direct Services
The government directly administers evidence-based interventions (i.e. government employees deliver interventions) in order to achieve equitable outcomes.
7. Investing in What Works through Direct Services
The Colorado Reading to Ensure Academic Development (READ) Act, amended in 2019, emphasizes the use of evidence-based instructional practices to support early literacy. It required the use of program funds for external program evaluation; created a science-based early literacy block grant program of $2.5 million annually; and required districts receiving funds in the school year 2021-2022 to have K-3 teachers complete evidence-based training in teaching reading.
Colorado’s Behavioral Health Administration manages and operates the I Matter program, which connects youth and parents to counseling services. The program offers up to 6 virtual therapy sessions, promotes services through an advocacy campaign, and will reimburse participating therapists for services. This program meets our standards of evidence-based as the program collects and analyzes performance and evaluation data and is a program rooted in evidence of positive outcomes for those with mental health issues.
| Program | Summary | Funding (July 2020 through present) | Evidence Base |
| Colorado Reading to Ensure Academic Development (READ) Act | Colorado Reading to Ensure Academic Development Act (READ Act) in 2012 to ensure that all children in Colorado reach grade level proficiency in reading by the end of third grade. | ~$75M | The statute explicitly requires that practices supported by this program must be evidence-based practices. The Colorado Department of Education ensures compliance with this requirement as it reviews and approves training programs that are added to its list of approved programs. |
| I Matter program | Colorado House Bill 21-1258 established I Matter in 2021 to provide access to mental health and substance use disorder services for youth, and to address needs that may have resulted from the COVID-19 pandemic. | $21,000,000 | Outpatient treatment of mental health concerns has been shown to significantly successful according to research from North Carolina State University, the Research Triangle Institute, and the University of South Florida. |
The Colorado Governor’s Office of State Planning and Budgeting (OSPB) invests in evidence-based policy through its own grant program, where departments submit, outlined in our yearly budget guidance, grant applications in a format similar to OSPB’s yearly budget requests which use a decision item template. All requests must include information on where the proposed program falls on the Colorado evidence continuum. Below you will find a full repository of OSPB’s currently funded programs, however specific funding amounts cannot be shared at this time. Specific grant outcomes and details can be found on OPSB’s website under “Grantee End-of-Year Reports.”
9. Performance Management
The government has performance management practices for incorporating evidence and data results to improve decision-making processes and create a culture of accountability.
9. Performance Management
A 2013 Colorado law required all state agencies to submit annual performance reports to the Colorado state legislature. Reports must include: (1) performance measures for the major functions of the department; (2) performance goals for at least the subsequent three years; (3) a description of the strategies necessary to achieve those goals; and (4) a summary of the department’s most recent performance evaluation. The Governor’s Director of Operations and Cabinet Affairs oversees the development and execution of statewide performance goals. The Governor’s Office of Operations maintains the statewide performance management system required, by law, to be published on an annual basis. The office provides guidance on developing annual strategic goals and annual performance plans and identifies opportunities for process improvement (CRS 2-7-204). Consistent with this management system, executive branch agencies continue to identify annual Wildly Important Goals (WIGs) that align with their agency’s mission, the statewide strategic goals, working group goals, and the Reimagine State Government Initiative.
To support the Governor’s strategic priorities, the Governor’s Office of Operations maintains working groups (Economic Development, Environment and Renewables, Health, and Education and Workforce) with cabinet members and relevant leadership, including: deputy directors, division directors, and/or key program staff, as needed. Those working groups set annual performance goals aligned to support the Governor’s priorities and progress toward the goals displayed on the Governor’s Dashboard with data updated monthly. The Office of Operations, Office of State Planning and Budgeting, and Policy Office within the Governor’s Office regularly coordinate to provide feedback on performance goals, budget requests, and legislative items, respectively. Additionally, the Governor’s Office will be launching a website where the public can review the spending of ARP SLFRF, and the outputs and outcomes that the state is receiving as a result.
10. Results-Driven Contracting
The government uses strategies to structure, evaluate, and actively manage contracts strategically, using data to help leverage procurement as a tool to achieve equitable outcomes.
10. Results-Driven Contracting
In 2015, the Colorado General Assembly passed HB 15-1317, which authorized the Governor’s Office of State Planning and Budgeting (OSPB) to enter into Pay for Success (PFS) agreements. Colorado continues its Pay for Success programs to improve outcomes for youth at risk of out-of-home placements. These contracts seek to achieve equitable outcomes by targeting youth “involved in the child welfare system or juvenile justice system”, which are systems that disproportionately impact low-income populations and youth of color. Evaluation metrics include attendance rates, reductions in suspensions, and improvement in credit accumulation; and research by The Brookings Institute and other entities show strong correlation between high school education attainment and upward economic mobility.
Three programs were implemented with payments dependent on the achievement of a set of preselected outcomes. Per the annual report: “As of May 2021, both the Fostering Opportunities and Multi-Systematic Therapy (MST) projects have hit their desired implementation benchmarks. In March 2021, the independent evaluator certified that the MST project had once again exceeded its desired implementation metrics, resulting in an additional success payment to investors of $305,190, the maximum success payment possible under the contract. In April 2021, the evaluator certified that the Fostering Opportunity project had reached a success level three (out of four levels), yielding a payment to investors of $427,105. The Denver Runaway Project was terminated in September 2020 under the contract guidelines due to low take-up of services.”
11. Community Participation
The government has practices for proactively engaging and empowering residents and community partners to inform policy development, program design and delivery processes with an emphasis on historically underserved communities.
11. Community Participation
In 2021, HB21-1266 authorized the creation of the Environmental Justice Program within the Colorado Department of Public Health and Environment (CDPHE). The department’s goal was to address environmental justice inequities through increased community engagement and cooperation between state, tribal, and local governments. CDPHE provides inclusive access to Air Quality Control Commission meetings and Greenhouse Gases (GHG) rulemaking. Services provided by CDPHE include translation services, community connector programs, 30-day notices of meetings, virtual/online meetings, and increased accessibility services. HB21-1266 ensures greater community involvement and due process in all phases of environmental law and policy. The Environmental Justice Task Force must submit the recommendations by November 14, 2022.
The Enviroscreen Tool, developed by CDHPE, generates “scores” using census and environmental data to visualize the level of unjust environmental impact across the state. The tool provides greater transparency for communities that face greater environmental burdens. CDPHE also provides The Environmental Justice Report Tool for Regulation 3 for air pollution permit applicants to use when implementing the Disproportionately Impacted (DI) Community Permitting Rule adopted by the AQCC in May 2023.
After the passage of the American Rescue Plan Act (ARPA) in March 2021, Colorado initiated the “Build Back Better Tour” to gather input from residents and stakeholders on the challenges they were facing. Housing, mental and behavioral health, workforce development, and targeted economic relief were identified as key areas for transformative investment. A strategic recovery framework was developed based on bipartisan engagement and robust public input, aiming to build back a stronger Colorado. The tour highlighted the opportunity to use ARPA funds for once-in-a-generation transformational change in the state.
12. Identifying Key Outcomes and Tracking Progress
The government has a strategic plan or priorities with measurable goals to achieve equitable outcomes that were informed by underserved communities and the government tracks progress on these key outcomes.
12. Identifying Key Outcomes and Tracking Progress
The Colorado’s Governor’s Dashboard, established in 2019, outlines high-priority strategic goals: energy and renewables, health, crime prevention, reducing homelessness, land use, and the workforce. Progress towards these goals, set by cabinet working groups, is displayed on the publicly available Governor’s Dashboard, with monthly updates. The goals align with agency performance plans. The Governor’s annual budget request links these strategic goals to specific agency activities and outcomes. Agencies develop performance goals considering their impacts on broader equity, diversity, and inclusion efforts, via performance measures in line with broader state goals and analysis of disaggregated demographic data.
Following the enactment of the American Rescue Plan Act (ARPA), the Colorado governor and the legislature identified three strategic areas for investing State Fiscal Recovery Funds (SLFRF), General Fund, and other ARPA awards: Fiscal Integrity; Transformational One-Time Investments (affordable housing, behavioral and mental health, workers and the workforce); and Economic Recovery and Relief. Over the next four years, Colorado will monitor key performance indicators in these areas, such as: housing affordability; homelessness rates; access to broadband; unemployment rate by county; access to quality healthcare, behavioral and mental health, and victim assistance. The Governor’s Office launched a publicly available website detailing ARPA-related spending trends and the outputs and outcomes associated with the major programs.
To enhance state policies and programs, the Colorado Governor’s Office and the Colorado Evaluation and Action Lab jointly developed the Linked Information Network of Colorado (LINC) in 2019. LINC enables data sharing for research and analytics, generating informative evidence about the impacts of state programs. Evidence is shared with agency leadership and staff during cabinet working group meetings. Collaborative projects between the lab and state agencies have improved outcomes in areas such as education, renewable energy, reentry, and racial disparities in criminal justice, prenatal health, and the early childhood education workforce.
- Leading Example
- Promising Example
- No Examples Featured