Evaluation Report 2022/23
We recently completed the first round of our Maximum Impact research program, which launched at the end of 2022 and continued until mid-2023. At the conclusion, we conducted an in-depth evaluation to assess the program’s effectiveness in different areas and to identify how we could improve it going forward.
As part of our commitment to transparency, we are pleased to share the full report and key findings from our review publicly. We invite you to examine either the high-level conclusions we drew from our research, or the attached full report. For privacy reasons, we will not post participants’ private information online.
We are truly grateful to all the program participants who generously donated their time to assist with our data collection efforts.
The Maximum Impact program was established to increase awareness and facilitate cost-effectiveness research in the field of data-driven philanthropy in Israel. During the pilot year, we collaborated with 21 local NPOs to perform and publish cost-benefit analyses of select programs run by each.
In the current research, we will evaluate the outcomes and impact of our pilot program. The goal was to identify opportunities for improvement going forward, share key learnings, and establish a benchmark for similar efforts in Israel and abroad. Going through this reflective process would allow us to strengthen the program for the benefit of future participants, partners, and the philanthropic sector overall.
In this evaluation, we utilized a mixed methods research approach, incorporating both quantitative and qualitative elements to gain nuanced insights. Specifically, we combined structured questionnaires using rating scales with in-depth interviews and self-assessments to analyze our work across four key indicators for program effectiveness:
1. Participant satisfaction levels
2. Best Practices Identification
3. Effect on Participants
4. Ecosystem Ripple Effects
To strengthen the credibility of this study, we partnered with an advisory team from the international research institute Rethink Priorities. They assisted in constructing survey questionnaires for program participants, which serve as the primary data source for this evaluation. Their guidance helped ensure our data collection processes were rigorous.
Participant satisfaction – We found a very high overall satisfaction level with the program, averaging 8.2 out of 10. The Net Promoter Score was also high at 48.
Best practices – key features for participant success were individual guidance from an experienced researcher, initial research grants, and firm deadlines. Surprisingly, financial incentives like awards for the most effective organizations or donation matching campaign were less influential.
Effect on participants – Clear positive short-term effects included strengthened research capacities, greater investment in evaluation, and increase in data-informed decision-making. Participants also expressed higher willingness to re-evaluations in the future. However, the readiness to make major changes based on findings varied among participants.
Ecosystem ripple effects – While participants substantially increased incorporation of research findings within their grant and funding submissions, this alone does not necessarily reflect a transformative shift in mindsets across the entire sector. In order to facilitate more substantive practice changes, additional cohorts of the program will likely be required, specifically to influence how resources are allocated based on evidence from our evaluations. Expanding the initiative’s reach through increased participation has potential to catalyze broader ecosystem impacts over the long run.
Overall, the pilot demonstrated that the research program exceeded expectations and yielded promising outcomes that can guide future progress. Contingent upon adequate funding, we aim to continue executing the program with adjustments based on insights from this evaluation.
For the next periods, we see an opportunity to strengthen two key elements in the program:
First, communications around our work must clarify that influencing resource allocation practices is long-term, as impacts may not be immediately visible. Second, we will continue building long-term partnerships with other partners in the philanthropic ecosystem in Israel to increase our program’s holistic nature and value – especially by integrating donors into research efforts.
Strengthening these two facets has the potential to magnify the program’s influence on evidence-based decision-making and optimize our contributions to shaping how social needs are addressed in Israel.