Social Metrics

Outcomes measure for articulating the impact of programming
Francis Nolan-Poupart
March 17, 2017

Social metrics are the measurement and articulation of social outcomes.

They tend to describe ways to measure the change that individuals experience as a result of a particular intervention and are often used by organizations to articulate the impact of their programming. Social metrics represent the next big challenge for delivering outcomes-based results for all involved stakeholders.

Social metrics are foundational to the implementation of pay-for-performance, impact investing and social enterprise blended value accounting and are incredibly valuable for advancing social innovation, social enterprise, social finance and pay for performance arrangements. While social metrics can commonly refer to as either outcomes measurement, social impact measurement, social value, social accounting, and/or blended value, there are generally three types of social outcome measures:

  • Qualitative measures: including storytelling, case study and anecdotal experiences.
  • Quantitative: quantified outcomes are a count (of outputs) plus a valuation (an indicator of quality) and sometimes monetized (assigned a dollar value). Quantitative for example counts the numbers of individual that get a job plus the quality of the job (part time, full time, sustainable or not), to which a monetized value could be applied. Tools such as Social Return on Investment include all those elements, other include the count plus valuation without monetizing.
  • Blended: qualitative with scales or indicators along a pathway such as Sustainable Livelihoods, and Life Course Assets analysis.

Social metrics matter to people and organizations that deliver services, interventions, and/or programs because they can be used to demonstrate the effectiveness, efficiency, and impact of their work. These metrics also matter to funders, who want to know their contributions are making a genuine difference regarding outcomes.

Elements of an outcomes measurement process can include:

  • Inputs: e.g. dollars invested, human capital (including volunteer effort), capital
  • Outputs: e.g. # of participants, retention, test scores, exit surveys
  • Outcomes: e.g. # of participants who have gained employment, increased employability (along a pathways of indicators), improves self-sufficiency, improved self-confidence.
  • Impact: e.g. social inclusion, increased well-being, increased productivity, increase government revenue (tax sources, reduction in support payments) decreased social costs. Social impact measurement is distinguished from evaluation in that SIM is ongoing and repeated, generating ‘dashboard’ information to help daily activity be more effective; evaluation is periodic and examines activity and results over years.

Advantages

The primary benefit of measuring outcomes in the social sector is the value of the information gained from the process. As outcomes are better understood, through both qualitative and quantitative measures, they can be better articulated and better understood.

  • For organizations: Measuring and communicating the social outcomes help to demonstrate the importance of the effort, skills and funding
  • For organizations: Social metrics gives organizations valuable outcomes information that can be used to look forward and do strategic planning to achieve objectives more effectively and efficiently, allowing organizations to focus on what really makes a difference.
  • For funders: social service organizations can more easily communicate the value and importance of their work.
  • For funders and organizations: Help to identify interventions that could be scaled up, based on the outcomes they achieve. Social metrics help to align the missions of funder and organization by identifying relative effectiveness of one intervention over another or an innovative versus an existing approach. Metrics provide evidence for targeting resources to innovative interventions.
  • For funders: To make funding decisions based on the articulated value (qualitative, blended or quantitative) of the outcomes, not just the quality and quality of the activities and outputs.

Limitations

  • Measuring social value is a difficult process.
  • Measuring social outcomes is more challenging than measuring easily valued products or services like the value of a car, consulting expertise, or the value of a business where there is a market place for exchange to determine a ‘market value’. (Social Value International)
  • Funders typically do not make resources available for this effort or the development of the capacity; there have been rare exceptions (e.g. City of Calgary has invested in the development of Social Return on Investment capacity building in its funded social services organizations).
  • The vast range of tools, and approaches make it difficult to navigate the space and select a best fit or reasonable option. In the social metrics field there is an overarching lack of standards which has encouraged the proliferation of single use methods and proprietary approaches used by a single funder (e.g. Tools and Resources for Assessing Social Impact).
  • There is no single ‘silver bullet’. The key is to measure what best reflects the interests of the organization and the interests of stakeholders, making the temptation to create another single purpose tool for a single organization or a single funder even greater.
  • There is the risk that reports on social impact measurement will be construed to be more robust than they actually are; this is an inexact and emerging practice and numbers will not be able to tell the complete story of social outcomes.

Policy Opportunity

  • Government programs provide billions of dollars to organizations across Canada and internationally to make a difference in the lives of vulnerable people. Government receives rigorous accountability information on how the funds are spent, accounting for the activities undertaken by organizations to deliver services. Government also collects some amount of output data (number of people who received which services) and may be able to use it effectively for outcomes measurement or analysis.
  • The evidence of outcomes that social metrics can provide gives important information in the policy development and program design stages that may otherwise be missing or challenging to identify as an evidence base.
  • As funders, particularly government, are better able to understand outcomes, funding could be targeted to outcomes are achieved in the most effective and efficient ways; with carefully designed indicators ‘creaming ’ can be avoided.
  • Government may be called upon to help develop standards for social impact measurement. While most practitioners do not recommend a prescriptive approach, a ‘principals-based’ approach is being adopted in the UK and internationally. This involves agreed upon principals for measurement with standardized indicators of the quantitative values for outcomes plus transparency about what outcomes are counted and how they are valued.
  • Measuring social outcomes, although challenging and costly, offers the opportunity to better understand more directly the outcomes that result from funding. However funders rarely get outcomes information for funded projects, and rarely provided the funding required to for social metrics results.

Considerations

  • Social metrics are not easy. It is challenging, expensive and resource intensive. Organizations report that time, cost and lack of resources and expertise are significant barriers. Knowing what to measure and how to measure it are not obvious or easy. There is, however, a growing body of resources, tools and approaches to social metrics options.
  • As a funder, the government can play a role in providing funds for organizations to develop capacity and implement social metrics reporting systems, or to hire the expertise for specific government funded projects.
  • Government action in this area will also have to be explicitly inclusive of the organizations that deliver services to help tease out what can and cannot be measured within a funding period, what can and cannot be claimed and to ensure that outcomes measurement does not undermine organizational autonomy or attribution for their program interventions.

Government of Canada

  • ESDC has done preliminary work with Sametrica in applying social metric measurement to some of its programs.

Best in Class

Sources

‍‍‍© 2021 Policy Community