Measuring What Matters Pt. II: Impact Insights

At the GitLab Foundation, we are committed to continuously improving how we estimate impact. In early 2023, we set up a cost-benefit model that centers our work on finding solutions that cost-effectively improve people’s lifetime earnings while taking into account relationship-based processes and impact rubrics. As we have implemented our model and made our first grants, we have already learned a lot and are committed to transparently sharing the insights we’ve gained.

Impact estimates should not be the only investment criteria

We look at many factors when assessing potential impact, including what outcome will be achieved, who the stakeholders are, how much change occurs, our unique contribution to the change that is created, and the risk that the project doesn’t meet its goals. The North Star metric helps us clearly estimate the how much and contribution dimensions, but the metric is unable to provide clear answers on the other dimensions. 

We use these five dimensions in our own impact due diligence process because they originate from a leading framework for evaluating the social impact of investments, the Impact Management Project’s Five Dimensions of Impact. Our impact estimation model is a valuable tool in our due diligence process and we recognize it is most powerful when alongside the other dimensions of impact.

Nonprofits often struggle to understand their own impact

Impact measurement can be challenging for nonprofits due to competing priorities, limited resources, and limited staff expertise. For example, they may know the number of people who were involved with their program, but struggle to understand how their participants' incomes or employment rates have changed after completing their programs. Even for those that are able to understand how participants’ incomes have changed after their program, they often struggle to hear back from participants after it has been more than a few months.

We believe that the philanthropic community has a responsibility to support nonprofits in their capacity to understand their work's short-term and long-term impact. Later this year we will launch our first impact measurement and client feedback fund. This fund will invite proposals from our grantees to apply for up to $50,000 in additional funds that can be used for advancing their impact measurement and client feedback activities, , in which insights could lead to significant programmatic improvements or future funding opportunities. 

Consider relative income increases when comparing projects internationally

Originally, we evaluated all of our work using an absolute income change model. When we focused on the United States, working with similar populations, this model worked well for coming the effectiveness of different programs.. But, as we expanded our work to Kenya and Colombia, we found that using an absolute income model that quantifies income gains in dollars created an unintentional bias against lower-income countries.

A relative income change methodology focuses the Foundation’s grantmaking on organizations that are the most cost-effective at increasing participants’ relative income. This impact model increases our focus on grants that are able to create transformational income change for low-income communities around the world. We are excited to share more details on this new model in the coming months.

What’s next

We are excited to continue to improve and iterate how we measure expected impact and welcome feedback. We will further develop our relative impact model that values percentage changes in income and model interventions that indirectly affect lifetime income, such as improved secondary education. Additionally, we look forward to learning about the realized impact of our investments as our grantees report on their progress in the coming months.