A litany of politicians, academics, communities, and organizations are firmly on board with the idea that the criminal legal system should be reformed. And they increasingly agree on how we should approach reform: through an evidence-based, data-driven approach. This approach, which I have labeled the evidence-based paradigm, has become so ubiquitous as to seem inevitable. Over time, it has shaped prominent approaches to policing, bail, adjudication, sentencing, and corrections.
It isn’t difficult to understand why the evidence-based paradigm has gained such widespread support and influence. On its face, it seems designed to redress the pathologies that led to mass incarceration — by flipping the conventional system on its head. It promises to be smart on crime instead of tough. It purports to let data drive instead of allowing racist anecdotes to inspire policy. Instead of basing consequential decisions on potentially biased and presumptively inaccurate intuitive guesses, it anchors them in evidence. Instead of lamenting that “nothing works,” it seeks out “what works” to combat mass incarceration. And perhaps the most appealing aspect of this approach is that it is seemingly apolitical and objective, with no agenda other than reducing the size and strain of the incarceration system.
In a forthcoming article in the BYU Law Review, I argue that this turn to the evidence-based paradigm is neither neutral nor inevitable. It is the result of a choice, and a deeply political one at that. And that choice — along with the subsequent decisions about what we measure, why we measure, and how we measure — favors the status quo. The evidence-based paradigm is characterized by three core commitments: First is the collection and analysis of data about how the system operates and how reforms impact its operation. Second, this collection should occur through quantitative methods. And third, the collected evidence should be used to increase the efficiency of the criminal legal system. The interplay of these commitments within the paradigm ultimately strengthens the influence of neoliberalism, punitive impulses, and white supremacy over criminal system policy and procedure.
The evidence-based paradigm ultimately strengthens the influence of neoliberalism, punitive impulses, and white supremacy over criminal system policy and procedure.
Proponents of the evidence-based paradigm commonly start from the premise that we do not know enough about the operation or impact of certain criminal legal processes in order to know what reforms are warranted — despite decades of insights, experience, and advocacy that make plain the system’s harms. According to paradigm proponents, we must engage in robust data collection and analysis before we take broad action, because “the road to reform is paved with data.” A recent proposal by the Brennan Center for Justice illustrates this data-first commitment: It proposes that, in order for President Biden to fulfill his campaign promise to stop incarcerating people who cannot pay court debt, the administration should start not with immediate and direct action to encourage jurisdictions to cease these predatory practices, but rather with robust research. Suggested research would involve pilot-testing provisions that consider an accused person’s ability to pay court fees and collecting data on a number of metrics, including the “costs associated with conducting ability-to-pay assessments” and “changes in defendants’ lives due to reduced fines and fees.”
But data, along with the research questions and methodologies that create data, do not simply exist; they are created in a power-infused system. As revealed in the in-depth scrutiny of algorithmic risk assessment instruments — a pillar of many evidence-based reforms — much of the data that drive this paradigm are biased and therefore reproduce racially biased outcomes.
But the problems with this approach run deeper than the data; the evidence-based methodology itself participates in the reproduction of epistemic biases and white supremacy. It sends a clear message about what (and who) counts. Borrowing its name and methodological orientation from evidence-based medicine, the paradigm valorizes quantitative data, ideally gathered from randomized controlled trials, to guide and assess reforms. It thus centers reform initiatives and assessments in institutes, academies, and organizations with the credentials and resources to conduct rigorous, large-scale studies. Individual or collective accounts of personal experience with a certain reform or procedure explicitly and intentionally do not qualify as evidence under the evidence-based paradigm. As a result, the paradigm reinforces a hierarchy of knowledge and promotes epistemic injustice: only the findings of the researchers matter, while those whose experiences are often the subject of such studies will not be heard.
Taken to its logical conclusion, this evidence-based orientation suggests we should not take bold action to redress problems until, or unless, we can show such actions are necessary and effective through quantitative empirical testing. It embodies what Ruha Benjamin has called the “datafication of injustice . . . in which the hunt for more and more data is a barrier to acting on what we already know.” In this way, it imposes a burden of proof on those who want to implement change to prove the impact of the desired changes before they will be embraced.
Taken to its logical conclusion, this evidence-based orientation suggests we should not take bold action to redress problems until, or unless, we can show such actions are necessary and effective through quantitative empirical testing.
The paradigm promotes this quantitative data collection and analysis to prove whether a reform is effective. But effectiveness is itself a value judgment. Effective for what purpose? What does the paradigm value? Many evidence-based reforms presume the answer to this question is obvious: We should do what works to reduce recidivism.
There are many problems with how the criminal system, in general, and evidence-based reforms, in particular, define recidivism. Technically, recidivism means repeated criminal behavior — committing a crime after having already committed one. But recidivism is used to measure and guide evidence-based reforms in contexts like bail reform, for example, in which individuals have not yet been convicted. To speak of repeated future criminal activity in that context is simply inaccurate — it presumes prior criminal activity. What’s more, many reforms define recidivism to include a future arrest. But an arrest is merely an allegation of criminal behavior, not dispositive proof it has occurred. And given that policing practices are often concentrated in low-income communities of color — thanks, in no small part, to evidence-based predictive policing programs — the risk of arrest is heavily influenced by race and socioeconomic status. Thus, recidivism is a malleable, misleading, biased, and vague metric. Moreover, emphasizing recidivism reduction maintains a focus on the person who may engage in behavior the system deems criminal as the source of the problem, thereby deflecting from systemic conditions that encourage and sustain mass arrest, prosecution, and punishment policies.
But recidivism reduction is not an end in and of itself. Rather, it is an instrumental step in achieving two broader values: saving money and increasing (or at least not decreasing) public safety. Therein lies the “bipartisan” appeal of this reformist paradigm — why deeply conservative reform organizations like Right on Crime have joined with left-leaning think tanks in support of evidence-based criminal system policies. Reforms that reduce recidivism are presumed to save money by avoiding the cost of “unnecessary” incarcerations, while reducing the commission of crime is equated with an increase in public safety. By focusing on “what works” to reduce recidivism, the paradigm produces reforms that continue in the neoliberal project of centering monetary concerns and quantifying the “cost” of a particular policy in very literal financial terms. Thus, the paradigm prioritizes reforms that provide a sufficient “return on investment” through recidivism reductions, as proven through quantitative study. It therefore disregards the potential intrinsic value of spending money on services such as quality education, engaging summer programs for teenagers, or meaningful employment opportunities.
At the same time, the paradigm holds fast to the reductive definition of public safety as linked to behavior defined as criminal, while ignoring a whole range of deeply harmful conduct that threatens safety through official, “non-criminal” means, such as coercion and violence by law enforcement and lack of secure housing, quality education, and stable employment opportunities. In so doing, the paradigm also furthers the troubling assumption that the public whose safety matters are distinct from those who are suspected of committing a crime.
More from our decarceral brainstorm
Every week, Inquest aims to bring you insights from people thinking through and working for a world without mass incarceration.
Sign up for our newsletter for the latest.
Newsletter
Whether one engages in this paradigm to “right-size” or reduce the scope of the carceral system, the goal of evidence-based reforms is to make the system run more efficiently — to optimize its operation so that only those who supposedly need or deserve to be incarcerated as a means of crime reduction are caged. This orientation, therefore, continues to center incarceration in reform conversations; incarceration is the default response, until or unless a reform has proven it is not necessary.
As noted above, I do not claim that those who promote the evidence-based paradigm intend to sustain the existing system and all of its dysfunctions, even though appeals to data and the scientific method can certainly provide cover for people who do. For the most part, it’s an approach embraced by people who want to develop reforms that can marshal bipartisan support. But this pragmatic rationalization is part of the problem. The paradigm is appealing to many because it does not challenge existing structures of power or privilege. It seems pragmatic because it lacks the imagination required to create a different relationship between criminal law, safety, and the public. At most, any reforms this paradigm produces will tinker at the edges of a deeply rotten system.
There have been proposals to address a few of the problems that plague the paradigm. For example, many have focused intensively on how to “debias” recidivism risk predictions, while others have proposed that we expand or shift our metrics. At best, these alternations offer improvements on a fundamentally flawed model. For example, even if we define public safety in a way that accounts for the safety risks caused by law enforcement and gather data that reflect that definition, we are still viewing reform through a lens that privileges quantitative knowledge claims and advances an epistemological hierarchy in an effort to perfect the operation of an unjust system.
If we want to transform instead of tinker, we must reimagine why we collect data, for what purpose, what counts as “evidence” — and, most importantly, whose voices are privileged in answering those questions. In other words, we must abolish the evidence-based paradigm and replace it with something else. What does that something else look like? I do not purport to have a conclusive answer and do not believe there is one singular alternative. But drawing on the insights of abolitionists and critical scholars who have been engaging these questions and envisioning different futures for decades, I offer a few guiding thoughts.
If we want to transform instead of tinker, we must reimagine why we collect data, for what purpose, what counts as evidence — and, most importantly, whose voices are privileged in answering those questions.
First, we should acknowledge that we already know a lot — without further study — about the impact of certain criminal system policies and procedures. For example, if we listen to those who have been harmed by police, we know that many common policing practices are inherently violent, coercive, and unsafe.
Second, we need to fundamentally shift who is centered in setting the reform agenda. This will require a shift in power from the government organizations, research institutes, and academies to the communities that have been most harmed by traditional governance structures.
Third, to the extent our new reform agenda requires more information about the areas and impacts of reforms, we must expand our data collection and analysis practices to include and prioritize those that center the insight and goals of those who have experienced the violence and injustices of the system. This notion is not radical. In fact, the original evidence-based approach to medicine embraced a “bottom-up approach” that considered insight from the needs and choices of the patient who was receiving treatment. Nor is it unrealistic; community-based participatory research methods were created to achieve these very aims. Meanwhile, we should heed the advice of QuantCrit scholars and embrace an attitude of “principled ambivalence” towards the quantitative data we do have. We need not categorically reject statistical data — but to the extent we engage with it, we should do so with an awareness of how it represents the product of a series of choices by those empowered to set the research agenda and conduct the studies — and on how those choices impact the outcome.
Activist groups are already doing this work and showing how these insights can come together. Data for Black Lives, for example, works to highlight the racist origins of popular data-driven practices and calls for the abolition of “big data” while also embracing the potential for data systems and technologies to serve as “powerful instruments for fighting bias, building progressive movements, and promoting civic engagement” — but only when such tools are “in the right hands.” Ultimately, we must remember that data, on its own, and regardless of how it is defined, does not answer the question of how to change the criminal legal system. Data can help describe the world. It can help us visualize the scope and depth of inequity and injustice. But data cannot and will not prescribe the path forward.
Image: Claudio Schwarz/Unsplash
