Measuring Collective Impact - from Standardized Tools to Common Frameworks
When agencies work together, the impact is often far greater than the sum of its parts. But how much greater? Measuring a common impact among diverse partners is often a challenge. Here’s how we’re handling it in one of our projects.
In working with the Creditvale Community Wellness Hub, we’re working with a diverse group of community agencies, health-service providers, social clubs, housing providers, and municipal interests. The common goal between all these organizations is to improve the wellness of the tenants of Creditville Mills, a two-tower apartment complex in Mississauga, as well as people living in the surrounding area who use Hub services. Many of the tenants are seniors, and there are also a number of young people in the buildings.
Our approach to this challenge is inspired by the Common Approach to Impact Measurement and SenseMaker’s methodology. Some of the steps and stages of our project so far:
Flexible Framework Development
We worked with Hub members to review their data collection practices, focusing on how they look at impact and outcomes in their organizations. Some agencies are already very robust in this area, and are using their own impact measurement tools; other organizations have less capacity, and may not be measuring outcomes or impact at all.
The challenge in this stage was balance: we wanted to ensure that the Hub could gather the right kind of data to help measure its collective impact. At the same time, we couldn’t overwhelm smaller organizations with new burdensome data collection processes, or ask agencies to radically change their established tools. Where we could, we made suggestions and tweaks for existing tools, and helped draft new tools when it was requested. Rather than require standardized measurement, we will summarize the various Hub partners’ impacts using a common framework.
Designing Common Measurements
Our framework was organized around the principle of “wellness,” and recognizes that wellness is a broad concept that can be improved a number of ways, including through culture, social and civic activity, improving health, and meeting basic needs. We know that while every organization in the Hub is working to improve wellness, they all work in different ways.
We built our common framework off the Canadian Index of Wellbeing. To take an equitable and culturally relevant evaluation approach; we refined the framework with significant input from funders, Hub organizational members, and most importantly, the people who access the Hub in their everyday lives. Their input helped shape a framework that would capture changes in users’ wellbeing, despite the wildly diverse activities.
The framework asks tenants and community members to tell their stories of how they have benefited from the Hub, and then asks them to answer closed-ended questions about their stories. Essentially, we are having the storytellers code their own responses using common wellbeing domains. We did not want a complicated set of rules for what counts as a particular outcome. In our framework, we recognize a particular type of outcome occurred when a community member codes their open-ended response as that type of outcome. Again, this ensures that our measurement is aligned to diverse worldviews of various community members, rather than forcing a single worldview of what “counts.” It also provides us with both stories and statistics.
Collective Impact Goals
It is our hope that measuring the collective impact of these organizations will result in a strong, cohesive Hub identity with plenty of cross-pollination between organizations, as well as improved wellbeing outcomes among Hub users. Our evaluation activities will help tell the story of the Hub’s successes and identify places for it to change and grow.