Pew Charitable Trusts staff reviewed this page prior to publication.
In July 2014, we made a general support grant of $3 million over two years to the Pew Charitable Trusts (Pew) for the Public Safety Performance Project (PSPP). This page lays out the reasoning behind our decision not to renew this funding following expiration of the grant in July 2016.
Although we decided not to renew the grant, one major goal of ours in making the original grant was to learn about philanthropic efforts to change public policy. Prior to the Pew grant, we had not focused on criminal justice reform or other policy philanthropy, and indeed had made very few grants at all. Through a partnership with Pew, we sought to understand the strategic approach and decision-making processes of a major national nonprofit organization with a strong reputation for advancing evidence-based policy. We believe we have learned multiple useful lessons from our partnership, though we think the bulk of that learning came early in the process, before we actually disbursed the funds.
Our decision not to renew our support for PSPP is based largely on our changing high-level strategic picture of criminal justice reform, which in turn is driven by a change in personnel: at the time of this grant, we were generalists learning about the field, but our work in this area is now led by Program Officer Chloe Cockburn.
Our altered high-level picture has not fundamentally changed how we see Pew’s work on technical assistance to states to inform their development and passage of reform legislation, which we think produced impressive results, but it has changed the way we view its other work, which accounts for a significant part of what our funds have supported on the margin. This other work includes research, publications, convenings, and engagement with other national organizations. We previously had very little idea of how to view these activities, and we implicitly modeled them as being equivalent in cost-effectiveness to the state-level technical assistance (i.e., we previously based our overall cost-effectiveness estimate on the per-dollar impact of the latter alone). With our more developed picture of the landscape, we no longer see this other work as fitting into our priorities. Going forward, we think there are other grants we can make in this space where our money will go further. Nonetheless, we note that we continue to think that Pew is an effective organization doing high-impact work.
We believe the changes in our views can be illustrated by revisiting our 2014 estimate of the cost-effectiveness of a grant to PSPP. We wrote:
PSPP’s budget is not public but Pew and the Bureau of Justice Assistance together have spent no more than $25 million on state assistance across all states that have participated in the policy development phase of the justice reinvestment process going back to Texas and Kansas in 2006. To be conservative, we use this $25 million figure as though it were our estimate of combined Pew/federal spending on state assistance.
PSPP assisted with 29 reforms that passed by the end of 2013 in 27 states. At its peak in 2010, the aggregate prison population among these states was 780,000. Assuming that the impact forecasts accurately predicted each bill’s impact on prison populations, we would expect the bills to lead to about an 11% decrease (the average forecasted reduction among states for which we received forecasts from PSPP was 11%). An 11% reduction in the prison population of these 27 states would lead to about 85,800 fewer prisoners at a given point in time.1 If we assume that the gains from reforms last about ten years and we assume that these gains would not have occurred in PSPP’s absence, then PSPP’s state assistance would have led to about 858,000 fewer person-years in prison. 2
If (as noted above) we assumed for the sake of the calculation that total spending to achieve these gains was $25 million, we would then calculate that Pew and BJA spent about $29 per “prison year averted,” again assuming none of the reforms would have happened without PSPP’s involvement. In addition, these averted prison years would have led to billions of dollars in savings to state budgets.
A different way of summarizing the assumptions that this cost-effectiveness estimate yields:
- Average expenditures are approximately $925,000 per state.3
- Average prison population in the states PSPP works in is approximately 29,000 per state.4
- PSPP’s work would reduce prison populations by approximately 11%, on average, in the states that it works in, relative to their modeled counterfactuals.
We also implicitly made an assumption that the state work was representative of the work that our funds would support, and that to the extent that our funds would support PSPP’s national work rather than the state work, the national work was equally as cost-effective as the state work.
In comparison, based on our current understanding:
- About half of our funds supported the expansion of PSPP’s national work (e.g. research, publications, convenings, and engagement with national organizations), rather than state technical assistance. Because the national work is less direct and quantifiable than the state technical assistance work, it is harder to come to a clear view on its cost-effectiveness. But we now believe, in large part based on the view of Program Officer Chloe Cockburn, that PSPP’s state technical assistance work is more cost-effective than its national work and that there are other funding opportunities we could support (compared to the latter) where our money would go further on the margin. This is the biggest qualitative change in our view, though we do not know how to quantify it.
- Our grant also supported additional work in five states PSPP would have worked in with or without the grant (Alabama, Alaska, Maryland, Nebraska, Utah). After discussion with Pew, our (and Pew’s) best estimate is that our funding provided something like 10% increased impact in those states, where increased impact means the legislation that passed is expected to result in a larger decrease in incarceration than would have been possible without the extra capacity provided by our grant. Assuming that roughly half of our grant was used to augment the state work, this implies an effective cost of reform in one state that is roughly 3.2x higher (meaning lower cost-effectiveness) than our 2014 estimate.5
- The average baseline prison population in the five states was approximately 13,000, which is approximately 2.2x lower than our 2014 estimate for the average prison population, meaning work in each state was less cost-effective than in our model.6
- PSPP projects that the prison reforms achieved in these states will reduce prison populations by an average of approximately 16% relative to their modeled counterfactual levels.7 This is approximately 45% higher (meaning higher cost-effectiveness) than our 2014 estimate.8
Taken together, these changes to the key inputs would reduce our estimate of the cost-effectiveness of marginal funding to PSPP’s state technical assistance work by about 5x,9 and leave us uncertain about the overall cost-effectiveness of our contribution because of the challenge of assessing the national work.
We note that PSPP disagrees with portions of our analysis here. In particular, PSPP points out that the project and its partners conducted work in additional states during the grant period that were excluded from our calculations.
As in our original writeup of the grant, we emphasize that quantifying the effect of policy work is very difficult, that we consider these estimates extremely rough, and that we treat them much more as a check for whether a grant could plausibly compete with grants to, e.g., GiveWell’s top charities, rather than as a precise estimate of impact. In addition, while we remain focused on maximizing the expected return on our investments, we are approaching criminal justice from an increasingly long-term perspective. We believe that PSPP achieves meaningful reform in the current landscape, but our view is that the landscape itself needs to shift, and that our resources can help achieve such shifts, so we are focusing our investments accordingly. On this point, PSPP states: “We are supportive of this long-term approach and note that our funding of and partnership with coalitions of business, faith, crime-victim, and conservative leaders has contributed to a landscape, which is more receptive to reform than any in the past several decades. Our view is reinforced by scholars who have studied the issue closely.”
Our understanding is that in the absence of our support, PSPP will continue to receive considerable funding from other sources, and we continue to see PSPP as a worthy recipient of investment from donors. We believe PSPP does effective work, successfully assisting states with the development and passage of legislation each year aiming to safely reduce the number of people incarcerated, and providing research and analysis to foster an environment friendly to reform. However, given our change in staffing, we now think there are other grants we can make in this space where our money will go further on the margin. In light of this, we are not planning to renew our support.