Principles for Continuous Improvement: Santa Cruz Human Services Department’s Journey

and

This is the third and final post in our data-driven continuous improvement blog series. Read Part 1 on Building a Data Dashboard and Part 2 about having Collaborative, Data-Driven Conversations.

So what does all of this look like in reality? How do you take these abstract concepts and implement them in the “real world?" We want to share the story of Santa Cruz’s Human Services Department (HSD) outcomes contracting transformation, in which they moved through the steps we outlined in the first and second blog series. By the end of their two-year engagement with Third Sector that focused on improving their TANF Welfare to Work (WTW) subsidized employment programming, Santa Cruz HSD had built a more collaborative relationship with their two providers and had developed new outcomes and metrics to regularly review and work towards. 

Data Considerations

At the start of the Santa Cruz HSD’s journey, the WTW contracts identified a number of outcomes, but providers did not receive data regarding their performance on some of these outcomes. These outcomes focused on milestones such as skill development and program completion, but did not measure long-term success such as job attainment or wage increases, and providers were unsure of which measures were most important to participants and HSD. The data was self-reported via provider follow-up calls with participants six months after program exit, leading to data quality issues.

What should we measure?: Third Sector collaborated with HSD, Stanford’s Center on Poverty and Inequality (CPI), and the two providers via a human-centered design approach that included quantitative and qualitative data analysis, service provider, program participants, and employer interviews, and co-creation of service improvement ideas. One result of the collaborative process included clear outcomes that reflected the goals of the WTW program, funding, and participants themselves. The identified outcomes were:

  • A program participant earned unsubsidized wages or enrolled in an institution of higher education
  • A program participant’s quarterly earnings grew 10% or more one year after program exit
  • A program participant successfully discontinued CalWORKS (California’s TANF program) due to an increase in family income

How will we measure it?: Third Sector and CPI helped HSD leverage and link data from California Employment Development Department (the state administrative database that contains employment and wage data) and CalWIN (a provider-inputted database including employer invoices, program entry and exit, etc.) to measure progress. Third Sector and HSD compared outcomes performance to historical data and included a 10% improvement year-over-year as an expectation. CPI additionally helped further disaggregate data on the priority measures to help HSD and providers understand the barriers that were most likely to prevent people from achieving the outcome goals, including lack of transportation, housing, or child care. 

Moving From Data to Continuous Improvement

As we emphasized in our last post, simply deciding on and sharing data is not enough. To really create a data- and outcomes-driven program, it’s important to move that data into action. Santa Cruz HSD worked collaboratively with their provider partners, Goodwill and Community Action Board (CAB), to ensure a sustainable forum to analyze outcomes and make program adjustments:

Invest in building collaborative relationships with providers and partners: Third Sector modeled human-centered design principles by researching the community context in partnership with HSD and providers and guiding them through problem definition, ideation and prototyping. Third Sector developed protocols for providers to test elements of improvements with users and refine ideas before implementing, further building trust and collaboration. As a result of Third Sector’s consultation and support, HSD’s contracted providers felt more empowered to be creative and outcomes-driven. Internal HSD staff, as well as both contracted service providers, have indicated that they now have an incentive to work collaboratively, align everyone around common goals, and have increased service providers’ ability to be more client responsive in their service design and delivery. One provider stated that the new focus on long-term employment outcomes “gave permission to be creative and do whatever it takes in a really innovative way to meet the contracted outcomes.”

Plan and hold Continuous Improvement meetings that foster data discussion: Santa Cruz HSD continues to leverage these strong relationships to hold ongoing continuous improvement meetings that examine trends in participant journeys, examine new data insights, and develop new program improvement ideas. HSD and providers now review outcomes data semiannually, consistent with the availability of state data. HSD also continued to meet monthly with providers and discuss qualitative results from testing service improvements until the pandemic hit. The three parties have adapted the cadence of their meetings (flexibility in these structures is key!) and look forward to getting back to monthly meetings when that’s possible.   

Follow-through on the promise to act on the data: The workgroup also leverages a clear decision-making process to ensure follow up and action from these rich, data-driven discussions. Providers continue to use the new flexibility and human-centered design approach to prototype, test, and get agency approval on new service improvement ideas. 

We hope that this “case study” helps illustrate how our simple frameworks and suggestions can be applied across a variety of social services and programming. Santa Cruz HSD and their provider partners’ journey may seem “simple” upon reflection, but we also want to recognize the intention and effort they all put into creating a new, collaborative, outcomes-focused continuous improvement mindset. We are pleased that this new mindset and process helped the providers to feel more empowered in their work with jobseekers and in their relationship with HSD.

Upon completion of our project with them in 2019, HSD leadership said "Third Sector really helped the agency focus on the north star of whether they are actually achieving outcomes." We are excited for both the continuous improvement process and mindset to continue to drive improved outcomes for WTW participants and families for years to come.