Jon Sotsky: Use assessment to add value, not prove it
Measuring social impact can be an expensive and time-intensive proposition says Jon Sotsky, Knight Foundation’s director of strategy and assessment.
Foundations and nonprofits should carefully consider what Sotsky calls the “return on assessment” rather than treating evaluation as a blunt tool to achieve what he considers the ultimate goal of philanthropy: delivering outsized impact with limited dollars.
We recently sat down with Sotsky to find out how assessment can develop a better understanding of what’s working and what’s not, and why. He also previews a few upcoming Knight evaluations and discusses how he recently joined Miami’s growing entrepreneurial movement.
Why should we as a foundation assess our work at all? What’s the value?
J.S.: Because Knight – and philanthropy as a whole – doesn’t have nearly enough money to squander it funding ineffective programs and making mistakes that could be avoided by learning from experience. Our goal is to gather information that enables the foundation and its grant partners to make better decisions.
Related links
What are some of the biggest lessons you’ve learned about doing assessment effectively?
J.S.: The importance of buy-in from leadership and program staff cannot be overstated. You can gather the best information under the sun, but it ultimately comes down to people possessing a genuine interest in applying data to their work. Thankfully at Knight, we have program staff eager to incorporate assessment into their strategy and a board of directors that has placed a premium on measurement and evaluation.
I’ve also learned that effective assessment is predicated upon finding grant partners with a genuine desire to learn. If you need to convince grantees that measuring impact is important, you’ve already lost. That said, metrics and analyses need to be integral, not extraneous, to the management of grantee projects. When developing performance metrics with grantees, I’ll often say, “If this metric isn’t meaningful to you or won’t influence the way you perform your work, let’s not measure it.”
Are you seeing any trends in how philanthropy is assessing impact?
J.S. For one thing, it’s happening more frequently. Several foundations have recently hired a full-time evaluation staff member for the first time or are bringing aboard additional evaluation staff; it’s good to see this level of rigor becoming more of the norm at foundations.
The biggest shift I’ve noticed relates to the underlying purpose of evaluation. More and more funders are using assessment to influence grantee success, not just judge success. This mindset has fueled the growing popularity of ‘developmental evaluation,’ an approach that encourages continuous, real-time learning by grantees as they navigate complex environments. I also believe the drive towards more consistent metrics and online grants management systems, like the Fluxx system Knight adopted, stems from a desire to encourage more frequent and timely exchanges with grantees about how projects are progressing, rather than waiting a year for a report that summarizes how the work unfolded.
Though I’m not quite ready to call it a ‘trend,’ there’s definitely growing interest in ‘big data’ to philanthropy. The Foundation Center has been an important advocate for more timely and precise data sharing through its Reporting Commitment. I’ve also admired Packard Foundation’s “Open File Cabinet” project where they’ve sought to be more transparent about their grant-making process and share lessons and resources from their work with external audiences.
What current projects at Knight are you most excited about?
J.S.: We’re analyzing the emerging landscape of ‘civic tech’ organizations including open government platforms, neighborhood forums and other technologies enabling residents to address social causes and improve their communities. We’ve partnered with the analytics software company Quid on the project, and it’s been a great experiment with applying big data to our work at Knight. The project has yielded some really interesting findings about funding trends in civic tech as well as the role of private vs. philanthropic capital. In the spirit of openness, we’ll be publishing the analysis on our website this fall along with all the underlying data in a machine-readable format so others can build on the conclusions and contribute additional data.
We’re also currently conducting a review of Knight’s Tech for Engagement portfolio. There have been some really interesting findings about effectively deploying technology to address underlying community needs, recruiting and engaging residents, and developing business models to sustain projects. We’ll highlight pieces of the report via a blog series this fall.
What websites do you read to stay up-to-date on trends related to philanthropy or assessment?
J.S.: I’ve been tuned in lately to sites that sit at the nexus of healthy cities, tech and social entrepreneurship. I’m a big fan of TechPresident and Next City, which have both been great sources of information for the Tech for Engagement assessment. Other staples for me include TechCrunch, Co.Exist, Good and Stanford Social Innovation Review.
When you look back at past Knight’s evaluations, what are some highlights?
J.S.: The recent report on Open Contests stands out because it paired each key insight in the report with a set of discrete, actionable takeaways for readers interested in operating effective grant challenges. The Social Impact Games evaluation was a great demonstration of the creativity that can be applied to the communication of evaluation findings; it’s a credit to Mayur Patel’s vision and work with our communications team to experiment with approaches beyond developing cookie-cutter PDF reports. Finally, the Connected Citizens report exploring the role networks play to influence community change in an increasingly digital age was one of the best reports we’ve put out because the topic was top of mind for people at the time and the insights strongly resonated across audiences.
You’ve lived in big cities like New York and Chicago. How does Miami compare?
J.S. The city feels like an adolescent compared to grown-up cities like New York and Chicago. I miss several aspects of living in those cities, but it’s been incredible to live in Miami during its formative years and have a chance to shape its future. My fiancée and I have felt since moving here that the city needs more entertainment options and social offerings for young professionals and residents. So, we actually just launched a new business in Miami called Bottle & Bottega that offers art classes and parties where people can enjoy a night out drinking wine and painting with friends with the opportunity to meet new people. We have our first four events scheduled in September which will be hosted at local bars and restaurants, and we’re preparing to open our own BYOB studio space in 2014 – stay tuned!
By Elizabeth R. Miller, communications associate at Knight Foundation