Measuring What Matters Pt. I: How We Evaluate Impact

Nov 28

Making the decision about where to invest is one of the biggest questions any foundation faces. At the GitLab Foundation, we take impact estimation seriously and have implemented a robust process to estimate the social impact of our potential investments. We also take transparency seriously, which is why we’re sharing how we measure our impact and how we developed our processes.

The four types of impact modeling

Most philanthropic and impact investing organizations have an investment selection process to guide their investment decisions. Our experience finds that there are four primary ways to vet potential organizations:

Relationship-based process: This process leans on program officer networks, referrals, and meetings to enable flexibility, speed, and adaptability to opportunities as they arise.

Quiet research: These funders conduct due diligence and research on which organizations may be most effective. They then fund those organizations they believe are most effective. An organization that famously employs this style of giving is Mackenzie Scott’s Yield Giving.
Impact scorecard rubric: This process requires all potential grantees to go through a rubric-scoring process on a variety of dimensions that a funder believes are important. These scorecards are often based on the five dimensions of impact. For an example of a scorecard, see Calvert Impact’s scoring process described here.
Quantifying social impact effects: These organizations use a cost-benefit or Social Return on Investment (SROI) model to quantify the social benefits of a project and compare it to the total costs of the program to ensure they are cost-effectively using their resources.

Our impact estimation model

We use a cost-benefit model, as described in number four above. This impact modeling process estimates the social impact (benefits) compared to the requested grant amount (cost). While we also rely on relationship-based processes and impact rubrics, our unique focus on impact modeling centers our work on finding solutions that cost-effectively improve people’s lifetime earnings. A key factor is estimating how much additional income our grants generate for program participants compared to what they would earn without the grant. We have an ambitious initial impact target of driving over $100 in lifetime earnings gained per dollar spent.

We call this benefit-to-cost ratio our North Star metric. It helps us determine which interventions are likely most effective at raising incomes for our focus communities: those earning below a living wage in their local context. Before we make any grant, our program officers and impact measurement team work with potential grantees to understand and quantify how their programs improve incomes for their participants. We learn how many people would be impacted and the estimated income increase their participants could expect.

To better understand the process, calculate your own simplified North Star return on investment metric through this interactive visualization.

Building on a strong foundation

The concept of quantifying the potential social impact of an investment is not a new idea. Cost-benefit analysis emerged in the United States during the mid-1950s to help make public infrastructure investment decisions. Quantifying the social benefits from philanthropic investments emerged through the “Social Return on Investment” concept developed by the Roberts Enterprise Development Fund (REDF) in 1999. Since then, many organizations have built impact models, such as the Robin Hood Foundation’s Benefit-Cost Ratio in the early 2000s, GiveWell’s cost-effectiveness methodology, and Global Innovation Fund’s ‘Practical Impact Units ’ model in 2019.

At the GitLab Foundation, we seek to build on the work of these innovative leaders to create accurate and meaningful estimates on the potential impacts of our investments. We believe taking this approach will enable us to support organizations that have an outsized impact on the world. We are excited to share our learnings and updates as we continue to refine our impact estimation model.

Learn about our insights to date in part two.