This appendix details statistical checks that the authors applied to test that our principal findings — that there is no clear relationship between crime and the election of a pro-reform prosecutor — held even when subtle changes were made to our methodology.
Specifically, in the accompanying piece, researchers compared crime trends for cities with pro-reform prosecutors to those in cities without pro-reform prosecutors. But defining these categories is an inherently subjective determination based on prosecutors’ policies, practices, rhetoric, and professional affiliations. Our principal analysis categorized prosecutors based on recognition from recent academic literature. Sources used and categorization details may be found there.
We note, however, that due to heterogeneity in cities’ crime rates and trends, the inclusion or exclusion of outlier cities could significantly shift our findings. To check that our analyses were robust to such cases, researchers tested the following alternative coding rules.
First, researchers tested the groups in figures 1–3 by recategorizing pro-reform prosecutor cities, one at a time, and rerunning the analyses. Appendix figures 1–3, below, plot each of those cases for homicide, aggravated assault, and larceny, with the original analysis plotted in bold. Excluding individual pro-reform prosecutor cities does not fundamentally change the findings of the analysis in any of the charge types. We have annotated points where the exclusion of an individual city shifts the group trend. Note that reading this graph may be counterintuitive at first glance. A city’s exclusion increases the average when the city’s own rate is below the group average. Additionally, note that groups of cities may differ slightly between analyses of offenses, due to data availability.
Second, we also tested the effects of recategorizing cities from the comparison group. We note that a small number of cities with large net changes have significant effects on the trends for the smaller pro-reform group, but their exclusion from the comparison group does not meaningfully change its trends.
Lastly, we tested alternative rules for coding pro-reform prosecutors. Recall that our principal analysis included a prosecutor in the “pro-reform” group only if they were listed as reform-oriented in three out of the three previous papers on this subject that we consulted. However, coding prosecutors as pro-reform based on recognition from at least two out of three papers adds only one new pro-reform prosecutor, and coding based on recognition from at least one paper adds seven new pro-reform prosecutors.
Broadening our criteria for inclusion in the pro-reform group does not result in significant changes to our comparisons for homicide, aggravated assault, or larceny.
In sum, our findings are robust to several methods for recategorization. Neither including nor excluding individual cities nor testing non-unanimous-recognition rules fundamentally changes the findings of this analysis.
