Creating Positive Change with Data and Evaluation – Part I

Brian Beachkofski leads Third Sector’s efforts in data, evaluation, and modeling to include developing and executing strategy, capturing best practices, and providing training.

This post is based off a recent speech I made at the Harvard University Center on Education Policy Research for a meeting of the Proving Ground partners. Proving Ground is an initiative committed to helping education agencies meet their practical needs, by making evidence cheaper, faster, and easier to use. Their goal is to make evidence-gathering and evidence-use an intuitive part of how education agencies conduct their daily work. 

This is part one of a three part series.

I was talking to someone over the weekend about my my career and they pointed out that my career going from a research engineer, to a budget analyst, to emerging technology scout, to a non-profit addressing social sector issues appeared to be a series of very different jobs. To me, it’s been very consistent. The unifying thread through my career has been using data to fix problems. I think that using data better can address problems that have had stagnant, poor outcomes for years, potentially decades.

As a contrary example, when we look at how technology and medicine have improved over the last 25 years it’s literally awesome. As an example, Paste Magazine created a list of the top ten technologies to come out in 2016. It’s amazing:

1. Google’s Tensor Processing Unit Beats Go Champion
2. SpaceX landed a rocket vertically in the ocean
3. Eternal nano-structured data recording
4. SolarCity opens Buffalo facility, Tesla opens Gigafactory 1
5. Dust-sized sensors that can be implanted within the body
6. Carbon nanotube transistors outperform silicon for the first time
7. Hyperloop one begins actual high speed test
8. A breakthrough in lithium-metal batteries could double efficiency
9. First Amazon package delivered by drone
10. The first 1TB SD card

The reason we have seen this rapid advancement is that the scientific method is used to test concepts and commit research funds to the most promising approaches. The ideas that work well are researched further and get larger investment to grow. Conversely the idea of “fail fast” is a cultural identity in tech start-ups. Biotech firms trim their research pipeline as quickly as possible to shift resources to more promising projects.

Things are very different in the social sector. Typically a good model is loved and revered until it’s accepted as the best possible approach. Further experimentation is slowed and resources flow into project strictly adhering to the evidence-based practice. A great model becomes a mandatory approach and becomes practically frozen in time when compared to how innovation occurs in the technology or medical sectors.

As an example let’s look at the “War on Poverty” that LBJ started in 1965. Today’s poverty rate is 13.5%.  Based on forty years of effort, multiple anti-poverty programs, and billions of dollars spent it would be natural to assume that the poverty rate in 1965 was much higher.

The national poverty rate in 1965 was 17.3%, a rate so unacceptable that within the next decade the Great Society initiatives and other poverty reduction efforts, such as Head Start, WIC, the Earned Income Tax Credit, and the Food Stamp Act (now SNAP) fundamentally changed the available support to those in poverty. While these program have undoubtedly lessened the personal cost of poverty and created a floor that can prevent complete destitution, their impact on reducing overall poverty in the US is less clear.

Forty years later the poverty rate in the US is 13.5%, which varies considerably by state. Fifteen states have poverty rates equivalent or higher than the US average in 1965. During the entire period between 1966 and 2015 the poverty rate hovered between 11 and 15.5%. This is the stagnation in outcomes that we need to address. Locking a model and only making changes at the margins is not going to continually improve a national challenge like poverty. Legislating programs further slows innovation because changes literally require an act of Congress.

A real question is how do you provide the needed support, use effective practices, and still allow the flexibility to find what works better than what we’re doing today. In fact, we know that social mobility happens and multi-generational poverty ends. It ends at different rates in different locations, so perhaps we can learn more about why certain areas have greater social mobility? Can we use the data to figure out policies and practices that work, expand those, and continually test and experiment?