Achieving Population-Level Impact by Maximizing Learning

Posted by Alison Gold on

America’s cities are the engine of national prosperity and economic opportunity. For generations, people have moved to cities because they contain the systems to educate, train, and create opportunity, while providing the transportation, technology, financial and social infrastructure to connect it all together. However, the systems designed to support national prosperity and individual economic opportunity have often – implicitly and explicitly – excluded many people since their inception. As our country has grown and become more diverse, this exclusion has led to more pronounced economic disparity.

The conventional wisdom has been that the challenges the urban poor face are side effects of geographic isolation in blighted neighborhoods. If cities improved these neighborhoods through programmatic interventions — like upgrading buildings and creating training programs — residents’ opportunities and incomes would improve as well. For generations, philanthropy, government and nonprofits have advanced the belief that if we do more of this work, it will lead to the end of poverty. Yet poverty in America and its cities has continued to grow.

Living Cities believes that the delivery of high quality programs is important work, but we also believe that it is not sufficient to achieve population-level impact. Addressing poverty requires the transformation of systems—the interconnected set of actors and institutions which created our current, inequitable outcomes in the first place.

Achieving Improved, Population-Level Outcomes for Low-Income People

Living Cities believes that change begins with the recognition that broken urban systems must be changed. Through our efforts, particularly The Integration Initiative, we are testing our hypotheses on what it takes to make transformational change happen, and what the process is to get from the status quo to a population-level impact for low-income people. The following are the hypotheses we believe are embedded in this process:
Building a new type of civic infrastructure that brings together decision-makers from across sectors to address tough problems is critical to begin changing the way things work;
• Changing stakeholder boundaries, perspectives and relationships produces new results;
• Using a mix of grants, flexible and senior debt can help drive private markets to work on behalf of low-income people;
• Producing project and place-based outputs and outcomes leads to systems change;
• Reaching enduring systems outcomes will take 10 years or more; and
• Achieving these milestones will result in improved outcomes for low-income people.

Twenty-two months into this effort we are seeing promising progress on-the-ground (see article “Halfway There” in this issue of At the Table), and our hypotheses about how to create systems change that improves the lives of low-income people are becoming more refined.

Transforming a system is not easy. It requires hard work by a lot of people, openness, patience, and fearlessness. Many forces—the way that philanthropy and government provide funding, the public’s apathy or even hostility toward the urban poor—reinforce the status quo of project and program-level results when we need population-level change. While projects and programs are not enough to systematically change our communities for the better, these efforts can be powerful tools for learning about how systems work (or don’t).

Using Programs to Learn How to Build the Best System

After reading Eric Ries’ book The Lean Start Up, I was struck by the framework he offers for building a successful start-up and how applicable it is to doing systems change work. Ries is an entrepreneur and a writer who has written about addressing complex problems where there were no existing solutions. While his efforts focused on the intersection of technology and behavior, the process that he shared in his book has equal significance for other types of complex problems without existing solutions—like poverty.

Ries puts forth two ideas for how we should be thinking about addressing complex problems. First, he states that “The goal…is to figure out the right thing to build.” (If we’re adapting this idea to The Integration Initiative, the “thing” is the new system that will lead to better outcomes for low-income people.) Ries’s second organizing idea is that, “Learning is the essential unit of progress.” In the process of figuring out the right system to build, it is not how many outputs or outcomes (individuals receiving services, housing units built) that you produce, but what you learn that will help you build a better system.

Ries also shares a couple of tools for trying to make these ideas more concrete. He posits that any organization trying to change how people or institutions behave functions like the triangle outlined here.

At the base of everything that you do is the vision you have—where you want to get to. In The Integration Initiative, a site’s vision is what the community looks like if it has achieved population-level impacts for low-income people. The middle layer of the triangle is strategy, which Ries describes as the hypotheses and assumptions and interpretations about what needs to change-- and how-- in order to achieve the vision.

At the apex of this triangle are programs and projects, which test the strategy. It is great if these experiments succeed, but the most important thing is to learn from the work. Even if the programs and projects fail, the efforts can still contribute enormously to refining the strategy for making systems change.

So, how do you use your programs and projects to test hypotheses and refine strategies? Ries came up with a very simple model for it that he calls: Build-Measure-Learn:

First, build a program or project that tests your hypothesis, or with existing programs, take a step back and make explicit which hypotheses are being tested. Then you measure, or capture what has happened as a result of the program. In The Integration Initiative, site teams work with their evaluators and data partners to figure out what data is needed to capture the outputs and outcomes.

The third step is to learn from what has been built and measured. In most work focused on addressing poverty, this process is relegated to an evaluation that takes place after the program has been completed, and the data is interpreted in only one way. So a program could have been going on for five years and produced poor outcomes throughout, and provided no learning toward building a better system. Instead of being a tool for after-the-fact reflection, the Build-Measure-Learn process can be used throughout a program’s lifespan. It can provide real-time feedback on your programs’ effectiveness and on the validity of your hypotheses. It also can help guide action from tweaking your program, to generating new hypotheses within your strategy.

It matters less whether the program or project succeeded or failed, as long as you maximized your learning toward building a better system. Recently, we introduced this model to The Integration Initiative site teams and asked them to practice “maximizing their systems learning” from real programs. As a team, they had to answer the following questions:
a. What are the population-level result(s) that you’re trying to achieve?
b. What do you think needs to change in order to achieve these results? (Aim to identify 5-10)
c. Build: What hypotheses are you testing through this program/project? (Aim to identify at least 3)
d. Measure: What results did the program yield?
e. Learn: How can you interpret these results? (Try to identify at least different 3 ways to interpret them.)
f. How did the results validate or call into question your hypotheses about what needs to change?
g. What other factors contributed or detracted from your results? How will you account for them in the future? What new or additional assumptions/hypotheses have you developed?
h. What are the implications of what we learned from this program for your overall strategy in terms of practice? Policy? Financing? Partners?

As The Integration Initiative moves forward, we plan to keep using the Build-Measure-Learn framework in site-based work, and with the Initiative as a whole. We think it’s a powerful tool for disciplined reflection and continuous improvement.