What is the purpose of a metrics system for my community?

How is impact-assessment useful for communities to work and track progress towards their shared goals? 

First and foremost, it is important to understand the vocabulary. Social impact measurement was originally defined by Hehenberger, Harling, and Scholten (2013) in the context of entrepreneurship as an approach to understand, manage, and improve the process of creating social impact with the goal of maximising or optimising it for social enterprises and their stakeholders. When translating this to communities and its projects, social impact measurement serves as a system for benchmarking a community’s commitment curve and its progress towards meeting shared objectivesthrough goal-specific KPIs.

Put simply. Building and sustaining a vibrant community is very time and resource consuming. Through the strategy, you know the purpose of this community, its objectives, and how it aims to create value for its members and beyond. Thinking about how you assess if the effort put into the community and the impact it creates for members and society or the planet needs to feel balanced. CMX assumes that a new community formes about every five seconds. There a lot of 'homes' for our members and thus its valid to regularly reflect on the results of your community. 

In general, people often jump to tracking and measuring different indicators and vast data sets. But the important first step for setting up a sustainable, goal-related metrics system is to precisely determine what success for your community will look like. 

Get back to your objective. What do you want to achieve? Ask your team, what would be meaningful data and information for your community to collect and analyse? 

“Before setting any indicators or KPIs you need to know what outcomes you are trying to achieve. A theory of change or logic model can serve as a framework to articulate your impact. On the basis of outputs and outcomes, you can then set indicators and define how you plan to measure your progress towards these goals”

Sarah Stamatiou Nichols, Impact Hub Global

Defining the purpose of your impact measurement effort is essential for gaining perspective on what to measure in your online community. You can easily get lost in different KPIs(Key Performance Indicators),metrics, sub-metrics and fail to see the big picture. To give you context, here is an example of how to think about indicators of success: 

Let’s say you have an online community that gathers alumni of certain programs associated with your organisation. One of the key activities is to help organise an alumni field trip to foster networking.

There would be three key indicators:

1. How many trips did they book? (i.e. can be measured annually and is directly describing the success of your activity)
2. How many people went on the trip? (This is a conversion indicator showing the number of people who went on a trip compared to the people who applied. It is important as it will show you overall process success for next field trip planning)
3. Do participants feel more connected after returning from the trip? (A simple survey at the end of the trip done by the participants will help you understand the level of satisfaction.)

The third objective is already more subjective compared to the first two indicators. Nevertheless, indicators one and two alone will not actually give you an answer. On the contrary, just the satisfaction level alone might not capture the depths of some of the connections made or the learning, or even a personal transformation. The key is to get as close to what you want to know as possible. 

Certain variables can influence  how your impact management system is defined. If a community is connected to an organisation or larger entity that is providing input on impact management, this will influence what success means and what indicators will be. 

WWF has about 50 communities but they all aim to serve a concrete purpose of a larger strategy. So they are not stand alone and can decide on their own what success looks like. The plastic community for example is closely connected to the Ocean strategy which predetermined the purpose and the indicators of success. 

If you have multiple indicators of success (we recommend no more than five) decide if they are all equally important or if you will weight them in some way. This can lead to the development of a weighting system that prioritises outcomes. 

When your community is still young, tracking data can help you stay on track. But keep your tracking simple. Eventually, there might be a point when people ask about the successes or the impact of your community. 

What indicators should I measure?

Once the success indicators are defined, the next step is to identify Key Performance Indicators (KPIs) to track progress around Objectives and Key Results (OKRs). A KPI is a measurable value that demonstrates how effectively a community is achieving key objectives set collectively or a decision-making committee. Organizations and communities use KPIs at multiple levels to evaluate their success at reaching targets. There are many different KPIs that can be measured but here are some of the most relevant for your virtual community-building activities:

• For open communities: Unique visits/Traffic on platform
• For closed communities: Registered members
• Content pieces published 
• Engagement with the content
• Time spent on the platform
• # Power Users
• # of referred members
• The lifetime value of members 

Next to those overarching KPIs, you can also have KPIs focused more on your concrete activities for the community or by the community aiming at impact: 

• # participants for in-person or online event/program/mentorship/grant
• # Members joining in-person meeting
• # Members participating in learning offers
• # Workgroups initiating projects
• # Members supporting other members projects/work 
• # Active sub-communities 

Those KPIs would not cover one of the main purposes of a community: belonging.

• Members feeling like being in the community provides them with value 
• Members engaging in peer support 
• Members making connections for other members 
• Members meeting outside of the community 
• Members becoming mentors 
• Members that feel the community empowers them to create impact 

These final KPIs are very hard to track but can, depending on your purpose, be the only ones that show you if your community endeavours are successful. You can check those in an annual member survey for example. Being interested in those more personal results also provides an opportunity to regularly listen to your community and hear what they have to say. Not every suggestion, comment, or criticism should be acted upon, but all should be listened to.

It is easy to develop 20 KPIs but it’s a lot of work to then actually collect all this data. So make sure whatever KPI you pick that tracking this data point allows you to better understand if the community serves its purpose. It is always important to continuously review that what you are tracking still makes sense. Whatever your goals, communities will evolve over time and take on new aspects that may require you to change your approach. Measuring and analyzing your impact over time will help you to adapt your community-building as your data points will indicate trends over time and allow you to react to those in a timely manner. 

An example. The popularity of Learning Exchange Grants created by the Bosch Alumni Network is a clear example of a community determining that their offer for knowledge sharing has value to its members. They know about the popularity because they capture key data and acted accordingly.

“The number of applications for this grant has been increasing continuously and so we decided to dedicate a larger share of our financial resources to this offer. This is a clear indicator that the grant scheme has been successful so far and, with this increase in resources, we hope to further substantiate it, strengthen members’ ties within and around the network and partially contribute to their further accomplishments.”

Vinzenz Himmighofen, Bosch Alumni Network 

A final step is to think about how you capture and give meaning to your data. Infographics or dashboards are great ways to create handy overviews that can be easily understood. Moreover, analytics give meaning to KPIs. For example, if a community exists to generate sales, you will be looking at sales conversion rates. Google Analytics, as well as other programs, offer a wide variety of metrics to look through.

Can we measure the community’s impact? 

Let’s not forget that there are limitations to data collection. The main impact communities have on their members is based on relationships and knowledge sharing. Belonging can not be captured in data. Quantitative impact measurement rarely tells the full story as they will not show community members being there for each other in challenging times, community members becoming life-long friends, community members having crucial advice or a solution to someone’s challenge. These in-between moments that make a community magical are hard to capture and this is when storytelling can play a big role. Members quotes expressing the value they see in the community, a story of two members starting a business together, a video capturing a testimonial. 

Here an example of the league of intrapreneurs:

It is always important to combine qualitative and quantitative measurements to provide a full picture. Let’s say a TEDx community is organizing a local TEDx conference and they are looking to gain insights about how to improve the community experiences they offer, the first step would be to identify success indicators and second to layout useful KPIs that would combine both quantitative and outcome-driven approaches. For example, a potential quantitative KPI the community would look at would be the number of tickets sold. Then for a sub-KPI from that could be the participation rate. Once you have that data, the TEDx organizers could send out a community survey after the conference to capture outcome-driven data. For example by asking people to share their most inspiring moments or key takeaways of the experience. 

Measuring environmental or social impact

This is the point where it gets very complicated. The most substantial impact communities have often is on their members that then potentially create impact for people and planet through their behaviour, their work or their leisure time activism. It is already challenging to track the impact on members, to then go even a level deeper and trying to track the positive impact members have can be impossible. You will need to move to a project level. 

The league of intrapreneurs operates across all the SDGs as they empower intrapreneurs across all fields. To show the impact the community has they show member stories across all areas of change but it would be impossible to find joint themes and capture data for those. 

Whereas, if the plastic community of WWF co-creates an initiative to minimise plastic waste the impact measurement can be done on a project level and then be linked back to the community. 

Similarly to other organizations in the sector, Impact Hub wants to move from SDG engagement to SDG measurement. Members are being mapped to the SDGs through an annual member survey and insights complemented with additional impact data. Their focus now lies on deepening understanding and measurement of the most prominent impact outcomes and SDGs among our membership. Getting to those levels can be very challenging. 

It is definitely worth having a conversation about the impact you are having on people and the planet in alignment with the community’s purpose. 

Show your success and your learnings

Finally, you need to think about how to show your data to the outside world. Many communities produce blog posts or report to show what they have achieved but also share their learnings. 

Impact Hub, for example, produces an annual report. The report showcases and explores the paths that we can all follow to scale meaningful impact collaborations and create a better future. In addition to inspiring stories of entrepreneurs who are doing well while doing good, it includes valuable insights from Impact Hub’s more than 10 years of building communities for impact around the world and five years of gathering data about it.

A stakeholder-based approach to impact management

The most common use of monitoring and evaluation (M&E) frameworks and tools is for impact reporting to donors and investors. For nonprofits, social enterprises, and mission-driven organizations, performance evaluation is extremely tied to M&E as they are not only accountable to their donors but also their beneficiaries. M&E was built out in organizations in a top-down way with an overvaluation of quantitative metrics. However, recently, more comprehensive and integral approaches are being designed for organizations to conduct impact analysis and measurement collaboratively and cross-functionally.

An M&E plan or framework focuses on the processes to implement and improve a certain program. For evaluating the impact of a given community project or event etc, it’s essential to consider the larger picture of a community’s organizational health and how an M&E framework would ensure the longevity of its efforts. 

Once you have a main success measurement, break it down to support your measurement process across different milestones in a community’s growth. Moreover, taking a stakeholder-based approach to M&E offers a more holistic and flexible framework to impact assessment. Checking in with our own biases to M&E ensures inclusive and representative impact assessment. It’s important to consider how we measure impact and how data collection can be rooted in socio-economic and cultural biases around what is “data” and meaningful information. Here, systems mapping can be extremely helpful to better understand the breadth and depth of impact, and how your community might fit into a large ecosystem of actors. 

The metrics used to measure the social impact of social enterprises should be linked to the scope of the measurement, depending on stakeholders’ needs (Zappalà and Lyons, 2009). 

Here are some suggested activities to try out. They are based on a higher-level comprehensive process to M&E that would be ongoing and stakeholder-centric. These are activities that work best in the case of a community connected to a larger organization. 

Stage 1- The Why: Gaining organizational & internal systems context

• Review existing theory of change (the inputs, outputs, activities that generate change) through systems mapping with staff, advisors and key stakeholders. Long-term vision and mission statements should guide the impact assessment framework. Output: Clearer community picture around structure, current operations, and programming
• Define M&E goals in collaboration with stakeholders i.e. better meet the needs of the beneficiaries and key stakeholders. Output: Defined OKRs (Outcomes and Key Results) to align community priorities with organizational priorities and guide the M&E framework direction. 

Stage 2 – The How: Determining the usefulness and efficacy of impact management processes to ensure continuous performance management.

• Stakeholder mapping: Identify key informants relevant to community actors. Output: Solidify the scope of the framework to understand resource and impact distribution flows.
• Key Informants and user surveying /interviewing by community persona type (i.e. active members, passive members, observing members, external supporters etc.) Output: Insights, storytelling, and personal testimonials.

Stage 3 – The What: Building a comprehensive impact management framework based on gaps assessment.

• Identify “best practices” methodologies for community members and stakeholders based on user/graduate feedback and surveys. Output: case studies on best practices and progress reports. 
• Create a documentation portal and database for continuous M&E. Output: Living library of impact assessment tools and resources, as well as user/community profiles and success stories. 
• Benchmark the effectiveness of projects across time and resource allocation. Output: continuous feedback “social accounting” system. Submit data and apply it to certifications.

We are witnessing an exciting time where data and tech are more at our disposal than ever before. The implications of this for impact management are that we can make more informed and collective decisions if we create systems to proactively and thoughtfully co-design metric systems that reflect our communities’ voices and priorities. 

After determining how to measure an organization’s or community’s success and impact, the question remains regarding how leadership will make impact management an integral priority of its community and/or organization and how it will build capacity across functions and teams to ensure that sufficient resources are allocated/budgeted in advance strategically.