Fresh Powder and Data Myths

In late January, in the mountains of the Wasatch Valley, over 600 city, state, and local government leaders, social impact investors, service providers, academics, and thought leaders came together at the University of Utah to discuss solutions to some of the most intractable challenges facing the social service sector in the United States and internationally. This unique event is the annual Winter Innovation Summit, hosted by the Sorenson Impact Center in Salt Lake City, Utah, which offers a diverse array of panels and workshops featuring leaders who are moving the needle in their respective fields.

The Winter Innovation Summit began in 2015 and has continued to grow in scope over the past three years. Third Sector’s work has been highlighted at each Summit since its inception and, this year, Third Sector’s Empowering Families initiative was featured as a partner track. The Empowering Families initiative is a “Learning Community” of seven state and county governments that are receiving technical assistance on outcomes-oriented contracting from Third Sector and on developing an integrated data system (IDS) from Actionable Intelligence for Social Policy (AISP) in order to achieve better two-generation (2Gen) outcomes for children and families. Learning Community members led sessions for their peers on accessing and analyzing inter-agency data to inform program improvements. They also participated in discussions with First Step House, the service provider from one of Third Sector’s Salt Lake County PFS projects, and co-hosted a joint session with the Results for America Local Government Fellows.

The Empowering Families Learning Community was also represented in the Sorenson-led panels. Dave Wilkinson, the Commissioner of the Connecticut Office of Early Childhood, and a Learning Community member, led a panel about how 2Gen and outcomes-oriented contracts can work together to improve results for families, featuring Third Sector’s CEO Caroline Whistler.

Following the panels, the Empowering Families Learning Community spent an afternoon digging deeper into outcomes-oriented contracting and data integration. One of the highlights of this exercise was a presentation led by Melissa Wavelet, the Director of the Office of Performance and Strategic Outcomes at the Colorado Department of Human Services (DHS) and a Learning Community member. Melissa walked us through her agency’s performance management process and focused on how she worked to bust commonly held “data myths” across the state. Some of these myths included concerns around:

• Data quality: “If data is entered by many, it will not be reliable or accurate to discuss”
• Ownership: “If state staff aren’t doing the work directly, they can’t be held directly responsible for improving it”
• Sample size: “Small and medium sized counties cannot be held to the same goals as large counties because they serve so few people”
• Accuracy: “If services are delivered more quickly, accuracy will suffer”

These myths resonated with many members of our Learning Community and Melissa’s guidance on how to gain buy-in and break down barriers related to data feedback loops were both practical and transferable. Melissa shared an example from her team that illustrated how they busted the sample size myth. Her team at DHS ensured that all monthly county performance reports include the average performance of similarly-sized counties as a data point of comparison. Melissa explained that showing counties how their same-sized peers were performing has been a key tactic in making the data actionable and applicable.

Practical solutions like this sparked a fruitful discussion, as members of the Learning Community discussed their own challenges and wins with using data to inform continuous program improvement. Namely, one site discussed how they have managed the unintended consequence of cherry picking the highest potential participants for a program, known as “creaming,” that resulted from setting misplaced data targets. Another site shared its successful technique of circulating anonymized comparison data among service providers in order to encourage service improvement through healthy competition and mutual learning.

This presentation on busting data myths, and the discussion that followed, is one of many examples of how the Empowering Families Learning Community has enabled governments to learn from each other in order to improve service delivery and, ultimately, improve outcomes for families. The Winter Innovation Summit gave our Learning Community the opportunity to connect with other social impact professionals, and with each other, in order to make tangible progress that will be brought back to benefit their communities. This learning is an ongoing process that will continue to move forward with every convening. The Empowering Families initiative will meet again as a Learning Community in mid-March, at a seminar on developing an IDS and using it to drive outcomes-oriented contracts hosted by our partners at AISP.

The Empowering Families initiative is funded through a 2016 grant from the Social Innovation Fund.