Measuring What Matters from Knight Foundation on Slideshare

In his 2012 annual letter to grantees, Bill Gates stressed the value of measurement as a critical tool for delivering social impact – in classrooms, clinics and cities. “Setting clear goals and finding measures that will mark progress toward them can improve the human condition,” he said.

It’s a familiar and important refrain and it turns out most of us agree. More than 80% of nonprofit leaders recently surveyed believe that demonstrating impact through performance measurement is a top priority.  Yet still, when we get down to evaluating our work, it can feel like a time-sensitive and daunting task that delivers little value.

How, then, do we improve our practice of it? How can we use it to strengthen our programs without overtaxing our organizations?

These are questions we often grapple with at Knight Foundation. At the annual Philanthropy Miami Conference in March, we shared a few simple exercises on how to use evaluation to deliver better programs and promote greater effectiveness within organizations.

A lot of what we shared was drawn from three resources that we’ve found valuable in our work. Each comes at the topic of measurement from a different angle. Together they offer a great starting point on various approaches, techniques and tools for using data to make progress towards your goals.

• Measuring the Networked Nonprofit (Beth Kanter and Katie Delahaye Paine) – Provides strategies and step-by-step guides for measuring relationships, social connectivity and engagement in nonprofits.
• Lean Analytics (Alistair Croll and Benjamin Yoskovitz) – Offers a guide on how to use data to build a better startup, by tracking indicators that help you iterate and understand market needs and user engagement.
• Leap of Reason: Managing to Outcomes in an Era of Scarcity (Mario Morino) – Provides a call to action and a set of case studies that highlight the importance of outcomes-based management in the nonprofit sector.

A few highlights from these resources include:  

1.  Developing Good Metrics. How do we know if we’re succeeding? If we want to answer this question, we need to track information that tells us something about our work, not just what is easy to collect. One potentially misleading indicator that organizations often rely on is: “number of people touched/served.” Although this number might give you a sense of your organization’s reach, it tells you little about the quality of the programs delivered, what happened as a result of your efforts or what you might do to promote improvement. As Croll and Yoskowitz note, if a metric won’t change how you behave, it’s a bad metric. In sum, good metrics tend to be understandable, comparable, typically a ratio or a rate and help you change how you do your work.

2. Prioritizing What You Collect.  As you gather more information, it’s easy to feel overwhelmed by data and uncertain how to act on it.  When struggling with more than a handful of data points, prioritize. Remember, as Amanda Cox, head of the New York Times Graphic Department noted, “Data isn’t like your kids.  You don’t have to pretend to love them equally.” Force yourself to identify the “one metric that matters” in your organization.  In other words, if you could only collect one indicator of progress, what would it be? In reality, you end up tracking a few different measures of progress, but it’s a useful exercise to do with your team.  Here are a few tips from Croll and Yoskovitz:

3. Building a Shared Culture: We tend to think of evaluation as a practical issue, that our troubles would be solved if we could identify the magic formula that would reveal our impact. In reality, measurement and analytics is more about building a culture where data is used to inform decisions and shape programs.  Beth Kanter summarizes it well in this post: “Data-informed cultures have the conscious use of assessment, revision and learning built into the way they plan, manage and operate. From leadership, to strategy, to decision-making, to meetings, to job descriptions—a data-informed culture has continuous improvement embedded in the way it functions.”

We hope you find these thoughts useful as you work to improve measurement at your organization. 

By Mayur Patel, vice president/strategy and assessment at Knight Foundation