Former NCRP Director of Evaluation Lisa Ranghelli on what philanthropy can learn from NCRP’s largest self-evaluation project to date.
In the fall of 2016, we at NCRP began implementing a 10-year strategic framework that re-focused the organization’s efforts in helping philanthropy actively co-create a just and equitable world where all communities get the resources they need to thrive. We wanted to more intentionally connect with and funnel support to movements that are important drivers of national progress and social change. Additionally, we expanded our scope to increase the effectiveness and impact of high-net-worth donors who do not give through foundations all while continuing to play our historical role as philanthropy’s watchdog and critical friend.
For 45 years, NCRP has worked with foundations, nonprofits, social justice movements and other leaders to ensure that the sector is transparent with – and accountable to – those with the least wealth, power and opportunity in American society. We do this, in part, by producing the kind of quality, action-oriented research that the sector can trust and use to analyze important trends, critique practices that don’t measure up and praise those modeling efforts that equitably build, wield and share power.
We also do this by being transparent about turning that lens inward and trying to publicly learn from the same practices and challenges that face our colleagues.
So, midway through our 10-year framework, we sit down with former NCRP Senior Director of Evaluation Lisa Ranghelli, who spent much of 2021 looking all that we have tried to accomplish in the last 5 years. We are still processing the lessons, but present these initial thoughts as a way of modeling the kind of public transparency we hope becomes standard in our sector.
Elbert Garcia: Why did NCRP undertake a large-scale evaluation at this point in time?
Lisa Ranghelli: We knew we were going to want to reflect on our progress and make course corrections along the way. In turn, we augmented our internal evaluation and learning capacity by creating tools and processes to collect data and reflect on progress on an annual basis. We also set our sights on doing a more significant evaluation halfway through our decade-long strategic framework in case any major adjustments were needed.
This year was also our 45th anniversary, so it was a great opportunity to reflect on our impact at this milestone.
EG: What were you hoping to learn?
LR: The midpoint review goals were:
• Understand internally how we got here: We wanted to get our staff and board all on the same page about what we’ve done over the last 5 years and the pivots we’ve made.
• Understand externally how we’ve shown up: We sought to gain confidential feedback on our impact, strategies and partnership from our primary constituents (i.e., nonprofit members and movement allies, funders and donors, and philanthropic allies).
• Understand what we’ve accomplished so far: We wanted to aggregate multiyear quantitative and qualitative data to gauge progress on our primary objective – influencing grantmakers to shift funding and practice in ways that support social movements and advance intersectional racial equity.
The timing was auspicious because one of NCRP’s key supporters, the Hewlett Foundation, decided to repeat a 2017 field scan in 2020 that included a survey of foundations about their uptake of new ideas and the quality and utility of content provided by philanthropy-serving organizations (PSOs). As a member of the field scan advisory committee, I was also able to give input on the assessment tools and process, and the Hewlett Foundation team were very responsive to our ideas.
More than 1,500 grantmakers responded – a number we could never have achieved had we done our own survey.
Without the heavy lift of doing our own funder survey, we were able to focus on getting feedback from other stakeholders. We anonymously surveyed 49 of NCRP’s nonprofit members and other movement allies and conducted 37 confidential interviews with NCRP board members, PSOs, donor networks, past Impact Awards winners and philanthropic consultants. We assembled tracking information from the last 4 years on specific funders, including any evidence that we had influenced their practice. Our approach throughout was to discern signs of contribution, not attribution, knowing that many forces and actors sway any one person or institution’s behavior.
Q: What were some of the key findings?
A: The good news is that we know we are making a difference. We have contributed to more than 60 grantmakers changing their practice in ways that align with our objectives – which averages to more than 1 per month since 2017. Moreover, the Hewlett Foundation survey affirmed that NCRP content has influenced the thinking of hundreds of funders and that more organizations were likely to recommend NCRP to a peer compared to 4 years ago. This was due to the perceived quality of NCRP’s research and content, the appreciation for our willingness to be critical of philanthropy and hold the sector accountable, our progressive values, our focus on racial equity, and our alternative or thought-provoking perspective.
We learned through our nonprofit survey that our Movement Investment Project and Nonprofit Membership Program are working in tandem to help movement leaders better understand the philanthropic landscape (67%), make connections with funders, and become more empowered in those relationships (41%). A smaller subset received the opportunity to organize with other nonprofits to jointly influence a funder or group of funders and found this peer support extremely valuable. Among survey responses from NCRP members, 62% view NCRP as a very or extremely effective ally and partner. As one respondent wrote:
“It’s been so great to be part of a community of immigrant and refugee rights movement organizations convened by NCRP. We feel less alone, inspired by and learning from our peers in the network and totally supported by NCRP staff with information and strategic guidance.”
Finally, we were heartened to hear positive feedback from our PSO allies and other sector leaders about how we’ve partnered with them and about NCRP’s niche in the sector. Folks most appreciated that NCRP has been both proactive and responsive in programming and content, uses both the carrot (such as the Impact Awards) and the stick (such as “Black Funding Denied”) to motivate funders, and centers social movements and their needs in philanthropic engagement. It seems that all these roles come together to make NCRP a valued leader in philanthropy, as one PSO interviewee described:
“For me it’s a combination of NCRP’s thought leadership and research that are pushing the envelope and agenda, then complementing that with tools and resources. Not just igniting a conversation, but by having Power Moves to back that up, having the immigrant interactive website with numbers and work you can do, is really key. You’re not just calling out, but calling into action.”
If anything, our allies want us to partner with them more and bring them into our projects earlier, rather than when we are ready to launch a new report or tool. They want us to convene funders more often and provide more one-on-one and peer coaching opportunities. Given NCRP’s commitment to holding an accountability stance with grantmakers, it behooves us to be explicit and transparent about what we do see as our role and to manage expectations of our sector partners.
EG: What challenges remain for the organization – you know, areas where we might have fallen short of expectations or are in need of improvement?
LR: We learned that our strongest influence is with funders who are already well on the road to social justice and racial equity. We’ve been much less effective at reaching funders who invest in under-resourced communities, but have not really explored equitable systems change yet. And though we’ve set an intention to engage foundation decision-makers more, we haven’t made inroads in reaching trustees.
NCRP has also fallen short in building effective programming to sway the giving from high net worth individuals, nor have we communicated clearly what that strategy entails and which donors we are trying to influence. Our comprehensive review of the last 5 years showed that the level of ambition we brought to fully implementing all the priorities in the strategic framework was beyond our capacity at that time. This effort to reach major donors was a casualty of that. We also realized we couldn’t focus on a dozen social movements at once and pivoted to one or two at a time. The last several years have been a process of reprioritizing, narrowing focus areas, and expanding budget and staffing to better position us to meet all our goals.
Another area of critical feedback was around how we’ve used publicly available foundation grants data to show the dramatic shortfall in sector giving designated for specific marginalized communities (immigrants and Black people). We learned that the confusion and blowback around the sources and quality of that data undermined our credibility and made it hard for funder and PSO allies to amplify those messages. This creates a greater urgency for us to seek better data through existing or alternative methods and better educate our audiences about the grants categories we use and why.
Finally, we discovered that many of our allies don’t know about the totality of NCRP’s work, resulting in missed opportunities to engage our networks across project silos. Although we have made strides in making our content a lot more accessible and “digestible,” stakeholders had great suggestions for how we can continue to improve. We also gained insights on the accessibility, practicality and rigor of our content from the Hewlett Foundation scan.
EG: What was most surprising or stuck with you?
LR: Because I wore two hats at NCRP – internal learning and evaluation expert and “Power Moves” author and team member – I was in the position of both collecting data and feeling the impact of it at a personal level. Of course, I hope that makes me more compassionate in sharing evaluation results with my peers at NCRP. However, it was also a source of competing emotions. It was both gratifying and hard to see evaluative information about “Power Moves,” a program dear to my heart.
On one hand, it was nice to see that the guide has been helpful for many funders and still has visibility and resonance in the sector after 3 years – if anything, it’s in greater demand now. On the other hand, I gained humility in hearing critical feedback about “Power Moves.” While I’ve known for a while that it can be a challenging tool to implement for funders with limited time and capacity, we got critical feedback from several consultants that was eye-opening.
It made me realize that despite the equity goals and values of the guide and the diversity of those who gave input and helped write it, it is very much a product of our white supremacy culture and perhaps speaks best to a white funder audience. While consultants want to use it with funders, several who were interviewed found the guide and/or the project challenging to work with, and we have not done a good enough job of trying to partner with them to use it, even though we held a year-long consultant cohort for this purpose after launch.
That’s a missed opportunity, given that philanthropic equity consultants are uniquely positioned to influence practice, and they are often invited into the funder board rooms that are hard for PSOs to reach. And personally, it’s an opportunity for me to reflect on how I still center whiteness in my work, despite my values and intentions.
EG: What’s changed in the last 5 years and how is that reflected in this evaluation?
LR: Shortly after NCRP began implementing its 10-year strategy, Trump was elected president, causing many philanthropic leaders to step up to defend communities of color, our democracy and so much more. NCRP’s goal to drive more foundation and donor resources to social movements became all the more urgent, and our initial focus on immigrant and refugee rights organizing was in part a response to the government’s harmful new xenophobic policies.
Then, the pandemic and racial justice uprisings made it hard for foundation leaders to ignore the legacy of white supremacy ideologies that manifested in these twin crises, and they saw the further unleashing of grassroots power at the local level. Funders were being forced to grapple with questions of power as they watched the corruption of it at the federal level.
I think this midpoint review showed that NCRP’s programs and tools were flexible, adaptable, and able to help funders and donors meet these moments. Simultaneously, we have been deepening our commitment and sense of accountability to our nonprofit members and other movement groups. This has affected how we approach our work. For example, “Black Funding Denied” grew directly out of conversations with our Black-led nonprofit members.
The evaluation – and continuing efforts to actively reflect on its practices – provides us with useful information and feedback about how we continue to navigate this stronger accountability stance toward funders with integrity.
EG: Indeed, we are still processing that evaluation and internally discussing how to build on the lessons we have learned in the next 5 years. Thank you so much, not only for carrying out this evaluation of our past work, but for all that you have done at NCRP since 2008. You will be missed, even though we know you are only a phone call – or email – away.
Lisa Ranghelli was NCRP’s senior director of evaluation and learning until August 2021. After 13 wonderful years with the organization, she is leaving to pursue a local opportunity with the Public Health Institute of Western Massachusetts. There, she will be providing inclusive research, evaluation services and capacity building for community-based organizations and institutions working on health equity initiatives.