Open PDF in Browser: Seema Tahir Saifee,* Aggregation at the Bottom
Scholars and reformers who aim to reduce reliance on incarceration offer different visions for how to achieve that change. Some call for technocrats to play a greater role in criminal policymaking. Another camp supports shifting power downward to populations most harmed by mass incarceration. Many scholars describe these approaches as advancing competing visions of expertise.
The framework of expertise is incomplete; the label does not fully capture the ways in which traditionally powerless populations create change. This Article introduces a less visible way that directly impacted people uncover systemic problems in criminal law and policy and shape new ideas to intervene in those problems—what I call aggregation at the bottom.
American criminal punishment operates collectively. As a consequence, mass clusters of people experience the same legal or social disadvantage, often alongside each other. For judges, prosecutors, and academics, these facts are highly dispersed. But in prisons and neighborhoods that disproportionately replenish prisons, empirical facts emerge in systemic form—the same unusual verdict, the same forgotten plea deal, the same social disadvantage. In these low‑status spaces, people observe, collect, and conceptualize recurring legal and social facts in the aggregate, opening up new ways to intervene in group‑based problems in criminal law, procedure, and policy. Outside these spaces, many community‑based allies take a similar approach.
This Article engages with and broadens criminal law scholarship on aggregation. I begin by identifying a common theme in the existing literature: Legal scholars value aggregation as a path to seeing recurring problems in criminal law, procedure, and policy and intervening in those problems more systemically. Put simply, legal scholars champion aggregation as a way to think about and create change in the criminal system—but only when the aggregators are in positions of power. I then introduce and conceptualize aggregation as a path to change from below. This Article argues that an innovation that legal scholars have deployed to improve the criminal system from the top down—aggregation—opens a window into an approach to reduce incarceration from the bottom up. Shifting this perspective reveals that the “data‑driven” approach to criminal law reform excludes data inspired from below that can open up new paths to reduce incarceration and reduce crime. Aggregation is not just one source of knowledge, but it is an understudied source of power at the bottom.
Introduction
American criminal punishment operates collectively.[1] As a consequence, mass clusters of people experience the same legal or social disadvantage, often alongside each other. For courts, prosecutors, and academics, these shared facts are highly dispersed due to the transactional nature of criminal adjudication.[2] But in low‑status spaces, empirical facts emerge in systemic form. A few examples illustrate:
– A group of men held in a New York state prison detected a pattern around them: In every prison they were in, they kept seeing the same people—someone they or a friend knew from the neighborhood. Based on this anecdotal observation, the group formed a concept that the state prison population appeared to be drawn from a small pool of neighborhoods. To investigate how small that pool was, the group collected data to identify the zip codes that supplied the growth in the state prison population. They found that 75 percent of the state prison population came from only seven neighborhoods, all in New York City. The neighborhoods were marked by extreme poverty, unemployment, failing schools, and the lowest life expectancy in the city. With data to support their concept, the group proposed reallocating funds from the state prison budget toward economic and social development in the seven neighborhoods. A group of people confined in a maximum security prison pioneered the concept “invest/divest.” Their research upended the dominant narrative of crime and punishment, shifting attention away from where crimes are committed toward where people lived before entering prison. The group fundamentally redefined public safety and the metrics used to measure it. Their innovation—divesting monies from prisons and investing in high‑incarceration neighborhoods—has now won scholarly, movement, and empirical support as a promising path to reduce crime and punishment.[3]
– Assisting people in prison with their cases, a man held in a Louisiana penitentiary observed a pattern around him: Numerous people imprisoned alongside him were convicted of felonies by a 10-2 or 11-1 verdict, where one or two jurors voted to acquit. Observing a recurrence of split verdicts in prison, he often thought the dissenting jurors got it right. Framing the state’s majority‑jury rule as a civil rights issue, he urged the indigent defense bar to move systematically for unanimous juries in every criminal case that went to trial. After he built up the issue in the courts, Louisiana state courts rejected defense claims that the Jim Crow‑era rule violated the Equal Protection Clause of the Fourteenth Amendment, repeatedly noting the vacuum of data showing the jury rule’s present‑day disparate impact. Journalists who covered the courts took notice and began a project to collect jury polling data from trial records statewide. The data showed that between 2011 and 2016, in parishes across the state, Black people were 30 percent more likely than white people to be convicted by split juries, and Black jurors in Louisiana’s most populous parish were almost three times as likely as white jurors to cast a dissenting vote. The data collection surfaced the profound and enduring racial impact of the state’s jury scheme. The data played an instrumental role in putting a constitutional amendment on the state ballot that would require unanimous juries in future felony trials.[4] Louisiana voters overwhelmingly approved the amendment, ending a law that “carve[d] a large[] footprint in Louisiana’s towering incarceration rate.”[5]
Individually, each intervention seems exceptional: a one‑off idea that gained traction or an indigent person in prison who got lucky. Examining the moves together, a different picture emerges. People most harmed by mass criminalization detected a group problem in the legal or social data[6] that surrounded them and reflected on those group harms to develop—and inspire in others—ideas that shaped new ways to intervene in mass incarceration. This phenomenon is not restricted to prisons. People embedded in communities most affected by policing and prosecution have also opened up new ways to intervene in mass incarceration by extracting patterns from combined datasets to reveal new insights. Consider the following examples:
– A research lab sought to empirically test whether providing jobs and cognitive behavioral therapy could reduce gun violence among Chicagoans at the highest risk. Researchers recruited participants for the intervention through multiple pathways, including: (1) a machine learning algorithm that used police data to predict men at the highest risk of gun violence in five Chicago neighborhoods with the highest rates of gun violence; and (2) local outreach workers from the neighborhoods served who were instructed to use community contacts and sources to refer men whom they thought were at highest ex ante risk of gun violence involvement. Over 2,400 men were randomly assigned to the intervention or a control group. Their progress was tracked over twenty months. A randomized controlled trial was conducted. Participants referred by outreach workers showed enormous and statistically significant declines in arrests (79 percent) and victimizations (43 percent) for shootings and homicides. Both the algorithm and outreach workers selected common variables to observe within a population dataset, but outreach workers with extensive experience in the neighborhoods identified unobservables that the algorithm did not have access to that produced a systematic variation in treatment effects across the pathways.[7]
– After the election of a “progressive” prosecutor, a community bail fund sought to track whether the prosecutor’s actions met his public promises to end cash bail in the city. Court watchers observed over one hundred bail hearings, documenting bail requests made by the prosecutor and bail amounts set by the magistrate. The bail fund combined the data collected by its court watchers to extract patterns. The bail fund reported that in 70 percent of cases observed where cash bail was set, the prosecutor requested higher bail than what the magistrate ultimately set. In no observed case did the magistrate set cash bail at a higher amount than requested by the prosecutor. Observing bail policy on the ground, the bail fund and its court watchers created a data‑based counternarrative to mainstream media accounts that the prosecutor had overhauled the city’s cash bail system.[8]
In these four illustrations, people most harmed by policing and incarceration (1) observed a recurring pattern in the legal or social data surrounding them, or (2) compiled large amounts of data, combined it, and chose variables to observe in the combined dataset to generate new insights from the whole. In each move, people most harmed by mass criminalization and mass incarceration uncovered systemic problems in criminal law and policy and shaped new ideas and strategies to intervene in those problems. Each example represents what this Article calls aggregation at the bottom.[9]
One definition of aggregation is a cluster of things that have been brought together.[10] A second definition of aggregation is compiling large amounts of data from different sources and combining it to extract patterns or insights from the whole that would not be visible in individual data points.[11] Tracking these definitions, I describe two strains of aggregation at the bottom. First, recurring legal and social facts are concentrated, sometimes side by side, at the bottom. People in low‑status spaces observe, collect, and reflect on those facts in the aggregate to understand group‑based problems and use the knowledge that flows from aggregation to create new paths forward.
Second, people in communities most harmed by policing and prosecution collect data from different sources, combine it, choose variables to observe in the combined dataset, and extract patterns that others might miss. As Dan Solove has described, “[a]s data gets aggregated, information that is not identifiable can become identified.”[12] Take a modern example—algorithms. In machine learning, developers compile a dataset, select variables to observe in that dataset, and feed that data to an algorithm.[13] Tool developers train the algorithm to aggregate the data to detect patterns that humans might miss.[14] Put simply, an algorithm can be used to uncover hidden patterns in large datasets.[15] Algorithms do not hold a monopoly over this skill set. When law and policy are examined from the places where individual determinations of guilt, probable cause, and bail are aggregated[16]—such as, respectively, prisons and jails, high‑incarceration neighborhoods, and courtroom galleries—that bottom‑up approach can generate legal and social observations common to the group.
People most harmed by the criminal system are aggregated, physically, in carceral institutions and poor, segregated neighborhoods, and digitally, to be exploited for patterns, in order to control and predict.[17] Out of this physical aggregation, people subjected to the harms of the criminal system have turned the system into the focus of study. People most harmed by policing and prisons become the subjects, not the objects, of aggregation.[18]
To be clear, I do not use aggregation as a jargony synonym for community knowledge. A number of scholars, notably, Ngozi Okidegbe, Lisa Washington, Monica Bell, Bennett Capers, and Russell Robinson, have written about knowledge that is shared by members of a community.[19] Although both concepts overlap, I aim to articulate and tease out something distinct. Community knowledge is typically known to those who experience, share, and pass it down to others.[20] One of the values of aggregation is its potential to extract new insights that are hidden—even from the aggregators themselves.[21] But as an approach that people most harmed by policing and prisons can—and already do—use to uncover and intervene in systemic problems, aggregation is undertheorized.
I present a theoretical and normative account for why the frame of aggregation is an important addition to dominant ways of thinking about how knowledge is acquired and created by people most harmed by policing and prisons. First, criminal law scholars think and write extensively about aggregation, conceptually and methodologically.[22] I argue that a shared theme emerges from the existing literature: Legal scholars value aggregation as a path to uncovering recurring problems in criminal law, procedure, and policy and intervening in those problems differently and more systemically.[23] Scholars encourage academics, courts, prosecutors, and algorithm developers to use aggregation as a lens, a method, and a tool to identify and confront latent systemic problems in criminal law, procedure, and policy.[24] Put differently, criminal law scholars value aggregation as a path to change—but only when the aggregators are in positions of power. This Article introduces aggregation at the bottom as a perspective to understand one way in which traditionally powerless populations uncover and intervene in systemic problems in criminal law, procedure, and policy.
Second, proponents of “evidence‑based” criminal law reform have expressed skepticism that bottom‑up contestation can reduce the use of incarceration.[25] Evidence‑based reformists have argued that “proposals [to reduce the use of imprisonment] are the stuff of experts and bureaucrats. And they are best justified using social science evidence.”[26] Aggregation at the bottom offers a rebuttal to this claim. It adds a distinctive way to think about data that is unobservable, undiscovered in systemic form, or just conceptualized and problematized differently at the “top,” but acquired or produced by people who do not exercise the levers of decision‑making. It is one less visible way in which historically disempowered populations access and generate data that can advance the long‑term project of decarceration. And it reveals that the “data‑driven” model for criminal law reform excludes data created or inspired from the bottom that can open up—and has already ushered in—new paths to reduce incarceration and reduce crime. Related to this point, aggregation at the bottom adds critical layers to dominant ways of thinking about lived experience.[27] Put differently, aggregation at the bottom complicates critiques that lived experience is insufficiently evidence‑based.
When ideas from the bottom enter the fray to deepen scholarly discussions on reducing incarceration, one common response is that bottom‑up participation does not guarantee decarceral outcomes or, worse, can entrench existing inequities.[28] This criticism is valid; but it can also be leveled against elites.[29] Aiming this critique at people most harmed by prisons and policing performs a legitimating function: In that, to value aggregation as a path to systemic change when used by courts, prosecutors, and algorithms—but to overlook the ways in which aggregation operates as a path to change from the bottom up—legitimates existing structures of power. Taking aggregation seriously as a way of knowing at the bottom can shift that power downward.
Finally, unveiling the workings of aggregation at the bottom raises questions about whether “expertise” is the appropriate framework for change in criminal law. Mass incarceration is “one of the most pressing human‑rights challenges of our time.”[30] There is strong momentum among scholars and reformers to reduce our reliance on incarceration. One of the most vibrant debates in criminal law scholarship focuses on who should drive that change—credentialed elites using empirical research to guide decisions about public safety, or people who speak from experience about the harms of criminalization and incarceration.[31] The fault line is around “expertise,” and who has it.[32] To be clear, reclaiming the expertise of people most affected by policing and incarceration is destabilizing and inspiring. But expertise often serves as a catch‑all[33] that can relieve us of the responsibility to think about how new ideas are developed and can continue to be developed. Resort to the expert label can often be a reflex. In fact, an impulse may emerge to characterize the opening examples to this Article as “expertise.” My concern is that the expert label does not do enough work. The turn—or, more aptly, the default—to expertise as the relevant framework for change skips an important step: When scholars lay competing claims to expertise, evaluating those claims requires unpacking the ways in which knowledge is developed. While no single theory captures the complex ways in which knowledge is produced, I argue that something else is at stake in the opening examples and fixating on expertise as the promised land risks obscuring it. Legal culture displays an “infatuation with expertise”[34] that obviates the need to consider alternate theories that might aid in understanding how knowledge is created.[35]
This Article proceeds in three parts. Part I engages with various treatments of aggregation in criminal law scholarship and identifies a through line uniting that scholarship. It then examines factors that constrain the capacity of aggregation to offer a path to change when used only by elites. Part II presents a descriptive account of what I call aggregation at the bottom, using examples to illustrate, and contrasts that phenomenon with three related but distinct forms of communal knowledge. Part III presents an account of why aggregation at the bottom should inform debates on evidence‑based reform and the larger project of decarceration, examines how aggregation unsettles dominant understandings of the knowledge of people most harmed by policing and incarceration, and explores some implications for taking—and failing to take—aggregation seriously as a way of knowing at the bottom.
Aggregation at the Top
Scholars have characterized mass incarceration as a “group and systemic problem,”[36] where “punishment and marginalization operate collectively”[37] in neighborhoods with concentrated disadvantage. A hallmark of this phenomenon is the patterns that emerge at every stage of the criminal process. Scholars, judges, and political leaders have written about recurring problems in the criminal system uncovered by practitioners,[38] academics,[39] social scientists,[40] and journalists.[41] Confronting the mismatch between a model of case‑by‑case adjudication and the systemic problems endemic to mass incarceration, scholars and advocates have developed creative ideas to treat these recurring issues systemically.[42]
How were these once‑latent, recurring issues laid bare in the first place? Often, advocates, scholars, investigators, reporters, and others compiled a dataset and observed a series of common factors across that dataset. The Innocence Project and academics assembled hundreds of DNA exoneration cases and observed recurring issues across the combined dataset.[43] Social psychologists combined known false confession cases to observe common factors that lead people to confess to crimes they did not commit.[44] And defense attorneys, investigators, and journalists have studied a disgraced detective’s prior cases in the aggregate to observe the patterns that emerge, such as the detective using the same “eyewitness” in multiple prosecutions or reciting the same implausible narrative to support a probable cause determination in multiple unrelated cases.[45]
Aggregation is not solely a hallmark of post‑conviction review. Scholars have written about aggregation as a lens, a method, and a tool in criminal law, procedure, and policy. In this Part, I engage with scholarly treatments of aggregation in criminal law and procedure, which are collected together here for the first time, and identify a common theme: Legal scholars value aggregation as a path to uncover recurring problems in criminal law and policy and to open up new ways to understand and intervene in those problems systemically. My aim is not to provide an exhaustive account of how scholars discuss aggregation in the criminal law context—certainly the harms of aggregation in policing, prediction, and criminal trials abound[46]—but rather to open a window into the connections that legal scholars have drawn between aggregation, the criminal system, and legal and social change.
Scholars
Alexandra Natapoff offers aggregation as a lens to think about the ways in which the American misdemeanor process “treat[s] people and cases by group.”[47] Order maintenance police stop large numbers of people based on age, race, and neighborhood;[48] prosecutors and defense counsel resolve “entire classes” of minor cases based on the “going rate” for the offense in the particular jurisdiction;[49] bail is often set on a schedule based on the offense charged;[50] judges process petty cases in bulk with little to no consideration for individual facts;[51] and punishment is decided based on offense category and criminal history.[52]
These and other “group‑based processing [habits],” Natapoff shows, stand in “deep tension” with criminal law’s “core commitment[] to individuation” in evidence, procedure, and the ultimate question of culpability.[53] Misdemeanor processing “can thus be understood as a series of iterative aggregations” where people are “largely [identified,] evaluated, convicted, and punished by category and based on institutional habit.”[54] Natapoff demonstrates that the misdemeanor system “does not function as a traditional ‘criminal’ system” that seeks to punish individual crime, but rather “amount[s] to a crime control system” that is “concerned with [the] management of groups.”[55]
Aggregation is a way to think about “what the criminal system does and how it . . . do[es] it.”[56] Natapoff concludes that “[a]ggregation thus provides a powerful conceptual lens through which to understand and critique one of the largest and most dysfunctional segments of the American criminal process.”[57] Put another way, aggregation is not only a trend to lament; it offers scholars a valuable way to think about systemic issues in criminal law.
Courts and System Actors
Brandon Garrett introduces aggregation as a method that courts can use to detect and remedy criminal procedure problems that recur across cases.[58] Garrett’s focus is “procedural aggregation,” or the “formal disposition of common issues or claims in more than one case using techniques such as joinder, consolidation, or a class action.”[59] Garrett describes how criminal courts have used aggregation as a method to uncover deep, systemic problems in criminal justice institutions and to design group solutions.[60] Garrett argues that “courts have experimented with . . . [novel] aggregative approaches” to detect, redress, and prevent systemic violations of criminal procedure rights—from inadequate indigent representation to fabrication of forensic evidence to racial disparities in death sentences—that would otherwise “be without any effective redress” in our individualized model of criminal adjudication.[61] As a method, aggregation enables courts to unveil the systemic nature of criminal procedure violations that “only arise piecemeal in our current [criminal] system”[62] and to design interventions to “enjoin [the] institutions responsible for [those] harms.”[63] Garrett shows how treating persistent patterns as group harms can create opportunities to craft structural reform in the criminal system.[64] Garrett therefore introduces the remedial promise of “court‑centered aggregation” and its potential to reshape criminal adjudication to serve the deterrence goals of criminal procedure.[65]
Andrew Crespo also unveils the capacity of courts to use aggregation to (over)see law enforcement behavior at an institutional level and “to reform the failed criminal justice state.”[66] As Crespo explains, criminal courts collect a considerable stockpile of facts every day, often in digital form.[67] This data, however, is dispersed across individual cases.[68] Crespo argues that courts can use technology to “catalogue[], organize[], and systematically stud[y]”[69] this data to unlock insight into common police practices and the institutional behavior of local prosecutors.[70] This capacity to access, search, and understand their internal data in the aggregate enables courts to uncover the systemic nature of law enforcement activities.[71] Judges can then “integrate[] [this aggregate analysis] into the regular process of constitutional criminal adjudication” to improve their systemic oversight of law enforcement.[72] Though his focus is on criminal courts’ latent capacity to serve as “systemic actors,”[73] Crespo displays the power of aggregation to “catalyze” institutional insight into “the systemic dimensions of criminal justice administration.”[74]
Scholars urge other repeat players to use aggregation as a method to track serial actors in the criminal system.[75] Jonathan Abel notes that when a police officer engages in misconduct, the current presumption is that the misconduct is a one‑off occurrence.[76] Abel calls for “cop tracing”—urging prosecutors to investigate every prior case handled by an officer whenever that officer’s credibility is discredited.[77] “The aim of [cop] tracing is to aggregate data across numerous cases so that larger patterns are revealed.”[78] Many groups create misconduct databases, compiling lists of officers with credibility problems.[79] Creating these databases for pending and future cases is important, but as Abel notes, they do little for people who are already convicted.[80] Abel urges “backward‑looking aggregation” to “identify the systemic nature of seemingly episodic problems.”[81] Abel argues that the “failure to engage in cop tracing is symptomatic of the failure to see the misconduct of even a single bad officer in systemic terms.”[82]
Algorithms
Algorithms are perhaps the most prominent use of aggregation in criminal law. Police, prosecutors, judges, parole boards, and other actors in the criminal system increasingly use algorithms to guide decision‑making.[83] Perhaps best known—and widely critiqued—in criminal law for their role in risk assessment and prediction,[84] algorithms essentially automate the process of extracting patterns from large amounts of data.[85] Tool developers specify an outcome to study, feed a known dataset into the algorithm, select factors to observe in that dataset, and train the algorithm to aggregate that data to detect patterns.[86] The algorithm reveals patterns in the historical data based on variables that have correlated with the outcome of interest in the past.[87]
Algorithms are a key part of technocratic expertise in criminal law. System actors and reformers rely on algorithms as tools for data aggregation in criminal law and policy. Their sophisticated power to mine data to uncover previously unseen patterns sustains the central role that algorithms play in criminal law administration. Algorithms are also the subject of widespread criticism for exacerbating and entrenching existing inequalities.[88] A number of scholars have called for algorithms to be deployed in different ways—to uncover and redress systemic problems in criminal law, procedure, and policy. Scholars have argued that algorithms can be used to detect factors correlating with police use of force in order to institute early interventions to reduce police violence,[89] uncover contradictions in probable cause affidavits from the same police department to improve judicial oversight of law enforcement,[90] and identify geographic areas to target for social and economic development.[91] In these new ways, scholars have shown how algorithms can be deployed to aggregate data for diagnostic ends.[92]
A Unifying Theme
This Part demonstrated that criminal law scholars think about aggregation—of people, cases, issues, and data—as a conceptual lens, a remedial mechanism, and a technological tool. Putting this scholarship in conversation, I identify a unifying theme: Legal scholars value aggregation as a route to seeing recurring problems in criminal law and policy and intervening in those problems differently and more systemically.
Scholars urge academics, judges, system actors, and data scientists to use aggregation as a lens, a method, and a tool to generate new insights into institutional issues in criminal law, procedure, and policy. As a lens, aggregation offers scholars a way to conceptualize the ways in which the criminal system manages to “masquerade” as an individualized process.[93] As a method, aggregation offers courts a way to pool information about recurring criminal procedure problems to reach policy issues[94] and to shape state action at an institutional, not individual, level.[95] As a tool, aggregation offers reformers a process for technologies to distill patterns from large amounts of data. A through line unites this seemingly disparate body of scholarship: Aggregation is a perspective that enables scholars, courts, other actors in the criminal system, and technologies to understand systemic problems in criminal law and policy, and these new understandings open a window for those same groups to design structural reforms to interrupt those problems.
This scholarship is pioneering in concept and innovation. Perhaps not surprisingly, the aggregators are elites who exercise authority by virtue of their position, educational credentials, or technocratic command. To use a management term, the aggregators are “at the top.” I now turn to four factors that constrain the ability of aggregators at the top to unveil insights into recurring problems in criminal law and policy and develop interventions to reduce them: position, unobservables, accessible data, and normative choices.
Limits
Position
The position of the aggregators limits the power of aggregation to achieve meaningful change. In legal culture, courts are the go‑to institution for resolving problems.[96] To be sure, courts enjoy distinct advantages in detecting and remedying systemic issues that arise in the criminal system. They are “uniquely positioned to see a large stream of cases,”[97] “serially engage with systemic criminal justice issues,”[98] and implement deterrent remedies.[99] Yet criminal law’s “fundamental commitment to individuation”[100] opens only a narrow window for court‑based aggregation.[101] Indeed, Brandon Garrett carefully limits court‑centered aggregation in criminal law to cases involving a pattern of constitutional criminal procedure violations “in which common factual and legal issues affect a group of criminal defendants.”[102]
Andrew Crespo expands this window by showing that criminal courts can now automate the process of data aggregation to reach systemic problems beyond criminal procedure rights. Crespo details the ways in which new technologies can aggregate information across cases, enabling courts “to ‘see’ beyond the truncated transactional horizon of a given case.”[103] Algorithms, for example, can automatically search and catalog digitized filings to detect patterns in the same police department and prosecutor’s office court‑wide, enabling courts to better see, understand, and monitor institutional practices of police and prosecutors.[104] “[C]rucially,” Crespo argues, “it is this capacity to aggregate dispersed information that is the key to unlocking criminal courts’ capacity for greater institutional awareness.”[105]
By hand or machine, criminal courts can use aggregation to detect, remedy, and deter recurring criminal procedure violations, surface common law enforcement practices, design systemic reforms, and create economies of scale.[106] The aim of criminal court‑based aggregation is “to create a more efficient, accurate, and fair criminal justice system”[107] and to enable courts better to fulfill their constitutional responsibility to regulate state behavior at a systemic level,[108] purposes that are inherently tied to the (limited) role and power of courts.
Aggregation’s usefulness in criminal law and policy, however, is not limited to improving the operation of the criminal system, making it more efficient, or reducing “errors.” As an approach to change, aggregation has the potential to reach broader problems in criminal law and policy, including structural inequities. Litigators, scholars, and social scientists have revealed the capacity of data aggregation—manual or algorithmic—to uncover and target a range of systemic problems in criminal law and policy beyond a court’s purview.[109] Still, a number of factors related to the position of these actors constrain their efforts. I begin with two distinct access‑to‑data issues.
Unobservables
Jonathan Abel has demonstrated that efforts to “identify the systemic nature of seemingly episodic problems” via aggregation—such as by tracing a dishonest officer’s prior cases, tracking a prosecutor’s use of peremptory strikes across that prosecutor’s cases, or identifying how often an officer’s requests to file charges are declined—“collide with the reality of a [criminal] . . . system” that does not collect data systematically.[110] “[I]mportant connections among cases are obscured by the atomization of the criminal justice system.”[111] As Abel has argued, “[c]ourts, prosecutors, and police departments curate case information in ways that [conceal] . . . . patterns of misconduct that emerge from their own data.”[112] To address these information gaps, scholars turn to the same entities—courts, prosecutors, police, and academics—to find ways to reveal “case‑to‑case connections.”[113]
Automation can facilitate some of these connections, but it cannot resolve the problem that much of the information needed to uncover systemic problems in criminal law is not documented, systematically or at all;[114] is difficult to access;[115] or is buried or forgotten.[116] And, despite automation’s power, many meaningful variables are simply unobservable by algorithms.[117] Put simply, what aggregation can do is limited by what the aggregators can see. Some systemic problems in criminal law are difficult to observe from any vantage. Other times, the data is out there, but whether it is observed episodically or in aggregate form can depend on who is doing the observing.
Accessible Data
Aggregation’s utility is limited not only by the data the aggregators can’t access but also by the data they can access. To open a window into systemic problems in criminal law and policy, aggregators at the top rely heavily on records maintained by police, courts, and prosecutors.[118] Although many actors have manually unearthed latent patterns from this data,[119] increased reliance on automation to extract patterns from this data poses distinct concerns. Data mining, which “automates the process of discovering useful patterns,” can have “a disproportionately adverse impact on protected classes, whether by specifying the problem to be solved in ways that [systematically disadvantage protected groups,] . . . reproducing past prejudice, or considering an insufficiently rich set of factors.”[120] Put differently, the patterns that algorithms purport to unveil are shaped, limited, and potentially tainted by the data to which the algorithm has access.[121] Ngozi Okidegbe calls this a “data source selection problem.”[122]
Normative Choices
Finally, in data collection and analysis, choices must be made at every inflection point. Before data is collected, someone must specify a question or an outcome to study.[123] How to define the problem, what data to collect, from where to collect the data, what variables to observe in the dataset, and how to interpret the results are choices.[124] Each choice is political and is shaped by the normative values of those conducting the research.[125] Each choice can lead to different insights, interventions, and paths forward. Asking a different question or framing it differently shapes a decision about what data to collect. Even if different people assemble the same dataset, the variables they choose to observe in that dataset can “change[] what we [a]re able to see.”[126] Data “echoes its collectors.”[127] Even if the same question is posed, the same data compiled, and the same variables observed, someone must interpret and frame the results[128]—a choice that can fundamentally change the path forward. Automation can detect variables in datasets that humans might miss, but automation does not choose what meaning to assign to the data. Every choice is influenced by the position, agenda, and commitments of the aggregators.[129]
Criminal law scholars have introduced aggregation as an innovation that offers new ways of seeing, understanding, and intervening in systemic problems in criminal law, procedure, and policy. Still, the perch of the aggregators limits what aggregation can achieve. I turn now to a strain of aggregation that receives far less attention in legal scholarship but holds possibilities to open up new horizons for change.
Aggregation at the Bottom
Aggregation at the bottom is a frame to understand one way in which people in communities most harmed by mass incarceration uncover and produce new ideas to intervene in systemic problems. I describe two types of aggregation at the bottom. First, incident to the collective operation of mass incarceration, recurring legal and social facts are clustered—or aggregated—at the bottom. People in low‑status spaces observe a recurring pattern in the legal or social data that surrounds them and reflect on those recurrences to develop group‑based theories. They then use the knowledge that flows from aggregation to develop new paths forward.[130] Second, people embedded in places that experience the greatest impacts of policing and incarceration compile large amounts of data, combine it, and select variables to observe in the combined dataset to reveal new insights from the whole that would not be discernible in individual data points alone. Section II.A gives two examples to illustrate the first type, and Section II.B gives two examples to illustrate the second.
Clustered Data
After the 1971 Attica prison rebellion, a group of men incarcerated in a New York state maximum security prison formed a study group called the “Think Tank.”[131] Between 1971 and 1981, New York’s prison population more than doubled.[132] Members of the study group observed a pattern: “In every prison they were in, they kept seeing the same people,” someone they or a friend knew from the neighborhood.[133] Based on this anecdotal evidence, the group formed a concept that “a very small pool” of neighborhoods supplied the growth in the state prison population.[134] The group “investigat[ed] just how small that pool was.”[135] The study group cross‑referenced state census data with department of corrections’ population data to identify the zip codes that produced the growth in the state prison population.[136] The group found that 75 percent of the state prison population came from only seven neighborhoods, all in New York City.[137] The neighborhoods were set apart by “social conditions that by every possible measure—health care, housing, family structure, substance abuse, employment, education—rank at the very bottom in the state.”[138]
With geographic data to support its hypothesis, the Think Tank proposed redirecting funds from the state prison budget toward economic and social development in the seven neighborhoods.[139] Their radical proposal was met with little support.[140] After their study made the front page news,[141] researchers replicated the group’s findings, collected more granular data, including the home address of everyone held in the New York state prison system, and framed that data into large scale visuals using geo‑mapping software.[142] The maps depicted that the vast majority of people in New York state prisons came from a tiny number of poor, segregated neighborhoods and were mainly concentrated on specific blocks in those neighborhoods.[143] The maps “made the social and economic dimensions of incarceration more understandable to a wide range of stakeholders.”[144] And “[t]he data visuals showed the same stark pattern in cities across the nation.”[145]
Traditional crime mapping tools detect crime “hot spots,” measuring public safety using crime and arrest rates—metrics that are critical to policing success.[146] The Think Tank’s concept and research “shifted attention from where crimes are committed to where people lived before entering prison, fundamentally redefining public safety and the metrics used to measure it.”[147] The dominant worldview of law enforcement, “whose muse is high‑crime areas, now competed with the stark view from below: high‑incarceration neighborhoods.”[148] Most data on imprisonment at the time was state and county level.[149] “[T]here ha[d] been few studies of the spatial concentration of incarceration in neighborhoods in the nation’s largest cities.”[150]
A group of men confined in a maximum security prison pioneered the concept “invest/divest”—divesting monies from prisons and investing in high‑incarceration neighborhoods—a “once‑fantasy” idea that has now won scholarly, movement, and empirical support as a promising path to reduce crime and punishment.[151] The Movement for Black Lives “r[a]n with it” as a way to dramatically reduce reliance on incarceration.[152] The Think Tank’s pathbreaking work sparked the famed concepts of “justice reinvestment” and “million‑dollar blocks”—prison spending maps that visually depict how much money a state spends to imprison residents of a single city block.[153] The Think Tank’s work ushered in new ways of thinking about criminal law and new ways of advocating for change.[154]
How states apportion their penal spending is a normative choice. People from the very neighborhoods where incarceration was concentrated urged a targeted “community specific” investment agenda rooted in social justice.[155] Others might have recommended increasing police presence in those neighborhoods. In fact, just a few years ago, two Harvard professors proposed “shift[ing] resources from incarceration to policing” to reduce crime.[156] Despite acknowledging the “good evidence” that long‑term investments in social programs “can be efficient at reducing crime,” the professors concluded that redistribution is “not feasible.”[157] The professors proposed “us[ing] the money saved by cutting prison populations to hire [500,000 more] police officers.”[158] In his searing critique, Alec Karakatsanis exposed flaws in the authors’ data and in the premise that “the answer to structural inequality is . . . more police.”[159] Karakatsanis stated, “these two professors are proposing the greatest expansion of militarized police and surveillance in modern Western history.”[160]
Once data is collected, what to do with that data is a normative question.[161] Someone must decide “where[] the data takes us.”[162] My aim in telling this account is to explore a more preliminary question. People in prison uncovered a systemic problem: A handful of neighborhoods were disproportionately replenishing the state’s prisons. What prompted them to collect the data that identified the seven neighborhoods in the first place? Indeed, anyone—police, researchers, cartographers—could have collected government data to identify the seven neighborhoods. The events that occasioned the data collection are commonly described as experience‑based knowledge or lived experience.[163] I aim to draw out one undertheorized source of that experience.
A group of people observed a recurring pattern surrounding them: “[T]hey often found themselves imprisoned alongside many of the people they knew in the street.”[164] This recurrence was clustered around them in every prison in which they were held.[165] The observation of a recurring fact concentrated at the bottom and conceptualized by people at the bottom sparked different questions, theories, and subjects of data collection, ushering in compelling new ideas to intervene in mass incarceration. As I have argued elsewhere, “[d]ue to the systemic operation of mass imprisonment in this nation, people in prison have insight on what is happening on the outside by the aggregated phenomena they observe on the inside.”[166]
Aggregation at the bottom is a way of knowing. It adds critical layers that are missing from dominant ways of thinking about knowledge production by people most harmed by policing and incarceration. People most affected by mass criminalization have frequent encounters with clusters of people subject to the same legal or social disadvantage. In spaces where these recurrences accrue, empirical facts emerge in systemic form, often at a granular level. Observing a fact in the aggregate can unveil the systemic nature of a problem. Indeed, legal scholars urge courts, prosecutors, and algorithms to aggregate to see the systemic dimensions of criminal justice problems. In spaces where recurring facts are clustered by the criminal system, observing, studying, and reflecting on those facts from the bottom up can generate new ways of thinking about and intervening in group‑based problems and—as the next example shows—can motivate new topics of litigation and data collection, including by people with greater resources.
Assisting people in prison with their cases, Calvin Duncan observed a recurring fact in the Louisiana penitentiary where he was held: Many people imprisoned alongside him were convicted by a 10-2 or 11-1 verdict, where one or two jurors voted to acquit.[167] Louisiana and Oregon were the only states that allowed a person to be convicted of a serious felony by a nonunanimous jury.[168] The split jury rule was not hidden from on high. Practitioners, Supreme Court experts, and journalists were very familiar with each state’s unusual jury rule. But law professors and defense counsel brought one‑off petitions to the U.S. Supreme Court[169] because “lower courts were bound to reject future [Sixth Amendment] challenges to split verdicts” based on Supreme Court precedent.[170] By contrast, Duncan framed and intervened in the problem differently. He “relentless[ly]” pursued the jury issue up to the U.S. Supreme Court,[171] but he also developed new strategies in the state courts.
The nonunanimous jury rule sent thousands of people to prison and kept them there for a long time or a lifetime.[172] Observing split verdict convictions in aggregate form, where he often thought the dissenters were correct,[173] Duncan problematized split verdicts as “a civil rights issue.”[174] He urged the indigent defense bar to systematically move for unanimous juries prior to trial.[175] From inside prison, Duncan “drilled into” the Orleans Public Defenders to preserve the jury unanimity issue in every criminal case that went to trial.[176] The defenders did so, and private defense attorneys followed suit.[177] In its template pleading, the defense bar raised an equal protection claim: The nonunanimous jury rule was first enshrined in Louisiana’s constitution in 1898.[178] The stated purpose of the 1898 constitutional convention was to “establish the supremacy of the white race.”[179]
Louisiana state courts rejected the equal protection claims due, in part, to the absence of data showing the jury rule’s present‑day disparate impact, which one court warned “would be impossible . . . to show.”[180] Journalists covering the courts took notice.[181] A team of reporters began a project to collect jury polling data in the state.[182] The newspaper found that between 2011 and 2016, in parishes across Louisiana, 40 percent of jury convictions ended with one or two holdouts, and Black people were 30 percent more likely than white people to be convicted by split juries.[183] More limited data from Louisiana’s most populous parish showed that Black jurors were almost three times as likely as white jurors to cast a dissenting vote.[184] The data collection surfaced a systemic problem: the profound and lasting racial impact of the state’s jury scheme.[185]
The newspaper’s investigative reporting and data analysis played an instrumental role in the Louisiana legislature passing a bill to place a constitutional amendment on the state ballot that would require jury unanimity in future felony trials.[186] Louisiana voters approved the amendment by a nearly 2-1 margin, ending a Jim Crow‑era rule that “play[ed] a significant role in keeping Louisiana at the top of the nation’s incarceration pyramid.”[187]
Again, what prompted the jury voting data to be collected in the first place? Duncan observed, reflected on, and intervened differently and more systemically in a group‑based problem in criminal law. The Louisiana courts took notice. The state courts’ constant refrain that the defense’s claims could not succeed without contemporary data paved the way for a project to gather juror voting data buried behind courthouse doors.[188] The journalists now had a “reason to do th[e] project.”[189] As one of the journalists told me, anyone could have studied jury polling data in Louisiana at any time,[190] but it was the momentum that Duncan built in the state courts that made gathering the data relevant.[191] “Duncan’s work inspired new subjects of data collection, dramatically changing how the courts, prosecutors, legislators, and the public thought about—to borrow a phrase from Professor Kimberlé Crenshaw—the ‘endurance of the structures of white dominance.’”[192]
Months after Louisiana voters amended their state constitution, the U.S. Supreme Court granted certiorari in Ramos v. Louisiana, the twenty‑third nonunanimous jury conviction that Duncan brought to the high court.[193] The Court in Ramos held that the Sixth Amendment required jury unanimity in state criminal trials.[194] Although the new rule announced in Ramos did not apply retroactively on federal collateral review,[195] the top prosecutor in New Orleans vowed to reexamine the hundreds of split jury convictions in his jurisdiction that became final before Ramos,[196] and the Oregon Supreme Court ruled that everyone convicted by a nonunanimous verdict in the state—including those whose convictions were final—was entitled to a new trial.[197] The decisions to reexamine hundreds of old cases to be retried, pled out, or dismissed have resulted in reducing the time people stay in prison, releasing people from prison, or both.[198]
Aggregation as Praxis
In the last two examples, people who bore the costs of mass criminalization and carceral punishment observed systemic problems in the facts concentrated around them and created new ideas to intervene in those problems. In this Part, I examine a second type of aggregation at the bottom: People in communities most harmed by policing and prosecution collect data from multiple sources, combine it, and choose variables to observe in the combined dataset to extract patterns. They reveal new insights from the whole that would not be visible in individual data points. This sophisticated process is increasingly entrusted to algorithms. I provide two examples.
Following Chicago’s unprecedented spike in gun violence in 2016, researchers at the University of Chicago, University of Michigan, and Cornell University sought to empirically test whether the combination of jobs and cognitive behavioral therapy (CBT) could reduce shootings among men in Chicago at the highest risk of gun violence.[199] The researchers partnered with community‑based organizations to launch “READI Chicago.”[200] Eligibility was limited to men eighteen and over at the highest risk of gun violence who lived in five Chicago neighborhoods with the highest rates of gun violence.[201] To identify and recruit participants for the violence intervention program, researchers designed three referral pathways:
(1) a machine learning algorithm trained to predict Chicagoans at the highest risk of gun violence;[202]
(2) local outreach workers with extensive on‑the‑ground experience in the neighborhoods served who were instructed to refer men whom they thought were at highest risk of gun violence;[203] and
(3) people released from jails and prisons who might be at a particularly sensitive transition point.[204]
Over 2,400 men were randomly assigned to a READI offer or a control group free to pursue other services.[205] Men assigned to READI were offered eighteen months of paid work combined with group CBT sessions.[206] Researchers tracked their progress over a twenty‑month period.[207] A randomized controlled trial was completed, and results were published in 2024.[208]
READI produced no statistically significant change in the study’s primary pre‑specified outcome of interest: an index that combines multiple measures of serious violence.[209] But participants referred by outreach workers experienced large and statistically significant declines in arrests (79 percent) and victimizations (43 percent) for shootings and homicides.[210] “No results in the other referral pathways approach statistical significance.”[211]
For an individual‑level intervention that targets people for social support, identifying and engaging prospective participants is critical.[212] READI’s effects varied systematically based on the pathway by which a participant was referred.[213] All pathways identified men at immensely high risk of gun violence.[214] But “men with equally high risk of gun violence seem to respond quite differently depending on how they were selected.”[215]
The first referral method was an algorithm designed to identify men at the highest risk of gun violence involvement.[216] To estimate the likelihood of some future event, algorithmic risk assessment makes predictions about individuals that are grounded in statistical generalizations about populations in the underlying dataset.[217] READI’s algorithm was trained on arrest and victimization records from the Chicago Police Department.[218]
The second referral pathway was local front‑line workers from community‑based organizations.[219] The organizations have “longstanding roots in their neighborhood[s],” and “[t]heir front‑line workers are recruited from the communities they serve and often have backgrounds similar to those of program participants.”[220] Outreach workers were instructed to identify men whom they thought were at highest ex ante risk of gun violence involvement in the coming months.[221]
In exploratory analysis, researchers tested several hypotheses to try and explain the differences in treatment effects between algorithm and outreach referrals.[222] One possibility was that outreach workers may have referred men who, in their view, would benefit from READI or were “ready for READI.”[223] Although interviews suggested that outreach workers filtered out people they believed would be unlikely to engage in programming or faced other barriers to participation, outreach workers were not systematically referring people who experienced the highest gains from participation.[224] In fact, only the outreach referrals with higher algorithmic risk predictions experienced enormous declines in serious violence.[225] Most outreach referrals had somewhat lower predicted risk—which was not the subgroup most responsive to READI.[226] This suggests that outreach workers were not simply selecting on expected responsiveness to the program.[227]
Neither the take‑up rate of outreach referrals nor their dosage of programming appeared to explain the different treatment effects by pathway.[228] Based on the findings, researchers were also skeptical that personal relationships between outreach workers and their referrals drove the treatment heterogeneity.[229] Outreach referrals were subsequently involved in gun violence at even higher rates than algorithm referrals with the same level of predicted risk,[230] suggesting that outreach workers may have selected men based partly on information unobservable to the algorithm that is predictive of future gun violence involvement.[231] However, the data did not support the hypothesis that unobservable “risk” factors that predict gun violence involvement drove treatment heterogeneity.[232]
In short, the researchers do not know why treatment effects were larger for outreach referrals.[233] There was something different about the individuals whom the outreach workers referred that was unobservable to the algorithm and the researchers.[234] Both the algorithm and the outreach workers chose variables to observe within a given population using different rules of thumb.[235] The algorithm aggregated large amounts of historical police data to look for factors that correlated with gun violence in the past,[236] drawing conclusions about individuals based on group averages.[237] People from high‑incarceration neighborhoods aggregated a large amount of information from the communities in which they were embedded, selected common characteristics for observation, and generated different heuristics to screen and nominate candidates.[238] The researchers concluded that “human and algorithmic referral mechanisms worked better together than alone,”[239] but cautioned that if programs are limited to one referral pathway, “using only outreach workers to identify [candidates] . . . may be the best way to maximize treatment effects.”[240]
Human‑based aggregation offers one way to understand the treatment heterogeneity. It offers one way to understand the critical role of people in communities most harmed by the criminal system in diagnosing, envisioning, and generating new ways to intervene in systemic problems, including gun violence. Something the outreach workers were doing or knew produced a variation in treatment effects in a systematic way.[241] Outreach workers captured a different set of variables,[242] picking up on something the algorithm did not have access to that turned out to be crucial to reducing violence.[243]
The next example shows that aggregation by communities at the bottom is not spatially restricted; in fact, it can happen in a variety of spaces. After Philadelphia District Attorney Larry Krasner entered office in 2018 on a “progressive” platform, the Philadelphia Bail Fund combined data collected by its court watchers across cases to track whether Krasner’s actions met his public promises to end cash bail in Philadelphia.[244] Put another way, after its court watchers observed and documented hundreds of court proceedings, the community bail fund aggregated that data across cases to extract patterns.
Court watching groups organize and train members of the public to watch criminal court proceedings and report their observations to the public.[245] Volunteers with Philadelphia Bail Watch, a court watching initiative, regularly sit through hundreds of bail hearings in shifts, observe, and take notes.[246] They “go to the places where decisions are made” to observe decision‑making from a position of sousveillance.[247]
In 125 arraignment hearings observed over a three week period, court watchers documented bail requests made by the prosecutor and bail amounts set by the magistrate.[248] Aggregating that data, the Philadelphia Bail Fund found that in 70 percent of cases observed where cash bail was set, the prosecutor requested a higher bail amount than what the magistrate ultimately set.[249] As the bail fund explained, the “magistrates consistently assign[ed] less punitive bails than Krasner’s office request[ed].”[250] In no case observed did the magistrate issue cash bail at a higher amount than that requested by the prosecutor.[251] In every case observed where the prosecutor requested release without the requirement to post bail, the request was granted.[252]
Andrew Crespo has argued that the capacity “to aggregate dispersed information” is “key to unlocking . . . greater institutional awareness” into the “systemic dimensions of criminal justice administration.”[253] Exhibiting its own capacity to aggregate dispersed data, the Philadelphia Bail Fund unlocked institutional awareness into the practices of the city’s new prosecutor, educated the public about the power of the prosecutor to determine bail outcomes, and opened up conversations in the district attorney’s office.[254] “[O]bserving the implementation of Krasner’s bail policy on the ground, . . . [the bail fund] arm[ed] [them]selves with facts,” producing a data‑based counternarrative to “the myth that Krasner ha[d] overhauled the cash bail system in Philadelphia.”[255]
In 2022, the bail fund highlighted another pattern in the city’s cash bail system.[256] Bail fund organizers partnered with volunteer data scientists from the group Code for Philly to download publicly available court records for every person charged with a crime in Philadelphia in 2020 and 2021 and to scrape bail‑related case information from the dockets.[257] The bail fund combined that data across cases to produce new insights from the whole—revealing that five of the city’s zip codes accounted for nearly one third of the over $21 million that Philadelphians spent on cash bail in 2021.[258] Bail fund organizers then went door-to‑door in those five zip codes to survey over 800 residents about their understandings of cash bail and community safety.[259] The bail fund collected this information in a report that “center[ed] the community’s insight[s]” and “ma[d]e connections between the ways that cash bail extracts money from already poor communities of color.”[260] These are just a few instances of the community bail fund using data aggregation to uncover, publicize, and mobilize against systemic harms[261] in “everyday criminal adjudication.”[262]
Relationship to Communal Knowledge
A rich body of scholarship in criminal law and adjacent fields describes the value of bottom‑up knowledge, from the narratives that flow when centering the voices and experiences of people most harmed by policing and incarceration;[263] to the information gaps that narrow when directly impacted populations “document and present their lived experience[s]”;[264] to the bold visions for social change that expand our critiques of criminal law.[265] These sources and expressions of knowledge expand our possibilities for change. Aggregation is not a fancy stand‑in for these forms of communal knowledge; it operates in solidarity with them. Aggregation at the bottom is a perspective that also centers the knowledge and ideas of directly impacted people as essential to achieving legal and social change. In this Section, I connect and contrast aggregation at the bottom with three related but distinct concepts: collective action, crowdsourcing, and emergent themes.
Collective Action
Jocelyn Simonson is noted for her pioneering scholarship on bottom‑up contestation and collective action in criminal procedure.[266] Aggregation at the bottom is not a substitute or synonym for collective action. I do not use the term aggregation to describe people coming together to think, act, or share experiences as a collective—just as criminal law scholars do not use the term aggregation to describe judges coming together to figure out solutions, prosecutors joining forces to document or observe, or public defenders coming together to engage in direct action. Certainly, group‑based problems can and do surface when people unite in collective reflection, action, or observation; in fact, they often do emerge in concert.[267] But a single person—a solo aggregator—can also observe, amass, or conceptualize common cases or facts in the aggregate and, alone or with others, use the knowledge that emerges to mobilize for systemic change.[268] In drawing this distinction, I do not mean to distance aggregation and collective action from each other. Both aggregation and collective action intersect in critical ways; indeed, one often catalyzes the other.[269] These intersections are a strength, a reflection of the rich and diverse ways in which people most affected by the criminal system produce knowledge. My aim is simply to clarify that both ideas are distinct in theory and practice.
Crowdsourcing
Aggregation at the bottom is distinct from crowdsourcing.[270] Soliciting input into a project by enlisting ideas from a large number of people[271] is invaluable to driving innovation.[272] But unless a person or a group combines that input and examines it in the aggregate to detect trends, the practice of crowdsourcing on its own is a different concept.
Emergent Themes
Lived experience produces what Bennett Capers has called “pools of knowledge” in Black communities.[273] Ngozi Okidegbe has described the “patterns that emerge” when people share their individual and collective experiences and the “common threads” within individual anecdotes.[274] Okidegbe’s insights might parallel what sociologists describe as “emergent themes” in qualitative research.[275] The concept of emergent themes in social science is rooted in grounded theory, a qualitative research method where the researcher begins with open‑ended data collection and allows theory to emerge naturally from the qualitative data, without imposing a pre‑existing theoretical framework.[276] Aggregation at the bottom adds critical layers to these invaluable forms of communal knowledge. It shares important parallels and connections with these expressions of communal knowledge, but it also expands how scholars and reformers think about the ways in which data and new ideas are created from the bottom up.
A common element emerges in considering aggregation by elites and non‑elites in criminal law and policy: uncovering common issues affecting groups of people; treating a chronic pattern as a group harm; grouping common claims; designing group interventions; extracting patterns from combined datasets that affect the group. Because mass incarceration has been described as the “systematic imprisonment of whole groups of the population”[277] and “incarceration [that is] so extensive and concentrated that it imprisons not just the individual but the group,”[278] adopting a group‑based approach that holds the potential to occasion group‑based effects is critical.
Expertise, Legitimation, and Power
An innovation that legal scholars have deployed to improve the criminal system from the top down—aggregation—opens a window into an approach to reduce incarceration from the bottom up. Attending to aggregation as an approach from the bottom expands the terrain of critique and reform in criminal law. This Part provides a sketch of some implications, focusing on three: Aggregation at the bottom offers new ways to think about the connection between bottom‑up struggle and the project of decarceration; it adds critical layers to dominant ways of thinking about the knowledge of people most harmed by policing and prisons; and failing to attend to aggregation as an approach to change from the bottom up—while valuing aggregation when used by elites—legitimates existing power structures. I situate these arguments alongside the evidence‑based model for criminal law reform to show that the reigning approach to reform excludes data created or inspired from below that can open up new paths to reduce incarceration and reduce crime.
The Expertise Debate
“We are living in a moment of possibility . . . .”[279] There is strong momentum among scholars and reformers to find ways to reduce our reliance on incarceration. One major disagreement turns on who has the relevant “expertise” to achieve that change.[280] One camp has called for agencies led by social scientists and policy experts to guide decisions on public safety and crime reduction through data‑driven methods (“evidence‑based model”).[281] A second camp opposes technocratic control, embracing instead what Jocelyn Simonson calls a “different kind of expert”—directly impacted people who speak from experience about the harms of policing and incarceration (“bottom‑up change”).[282]
The principle behind the evidence‑based model is that irrational political pressures got us into mass incarceration, and credentialed elites relying on data will get us out.[283] The model’s clarion call to let the data drive the reforms[284] is sustained by at least two sets of beliefs. The first is that installing experts to “follow data and science toward reforms that will reduce our reliance on incarceration” is a “rational” approach to identifying effective reforms.[285] A vigorous body of scholarship has debunked the myth of “value‑neutral expertise.”[286] Erin Collins has argued that evidence‑based criminal law reform embraces an ideological commitment to efficiency:[287] the most public safety (typically measured by a single contested metric—reducing recidivism)[288] at the lowest cost (defined mainly in fiscal terms).[289]
A second tenet of the evidence‑based model is to collect more data to improve public safety.[290] But the model only endorses reforms that are “proven” effective at enhancing public safety through empirical research.[291] As a consequence, evidence‑based reform excludes most “community‑level investments,” many of which have long‑term social and economic benefits, but few, if any, that promise speedy reductions in crime.[292] Also, the model’s concept of rigorous evidence inevitably excludes different knowers. Put simply, social scientists and experts with elite credentials produce the “right” kind of data to reduce incarceration, and people outside those realms (generally) do not.[293]
Erin Collins has argued that the evidence‑based model “disqualifies wide swaths of knowledge as a basis for reform or intervention, including observational, community, and experience‑based knowledge.”[294] Collins and Ngozi Okidegbe have promoted a different concept of data, namely, knowledge from the lived experience of the harms of the criminal system that individuals and communities document using qualitative methods.[295] Collins and Okidegbe call for “re‑envision[ing] what information ‘counts’ as data” to guide new paths forward and “whose voices matter in setting the research agenda.”[296]
The push to elevate data from below recalls Mari Matsuda’s famous call to legal scholars to “look[] to the bottom” as “a new epistemological source.”[297] Amna Akbar, Jocelyn Simonson, Allegra McLeod, and other scholars have demonstrated how concepts and interventions from below have shaped new ways of thinking about criminal law.[298] Bottom‑up experiences and visions for change can create alternative conceptualizations of the problems to be addressed and “offer alternative frameworks for the way forward.”[299] A growing number of scholars have argued that large‑scale decarceration must be driven from the bottom up.[300]
A number of evidence‑based proponents have expressed skepticism that—and questioned how exactly—bottom‑up contestation can reduce the use of incarceration.[301] John Rappaport has argued that “proposals [to reduce the use of imprisonment] are the stuff of experts and bureaucrats. And they are best justified using social science evidence.”[302] Aggregation at the bottom offers a rebuttal to this claim. It reveals that the “data‑driven” model for criminal law reform excludes data‑driven work created or inspired by those at the bottom that can advance the long‑term project of decarceration.
People most harmed by policing and incarceration have access to systemic data. They have also created—and inspired others with greater resources to create—new ideas to unearth systemic data. Bottom‑up aggregation has ushered in new ideas to intervene in recurring problems in criminal law, new ways to reach structural inequities, and new interventions to reduce incarceration on a group basis—unsettling dominant ideas about criminal law and opening up new routes to reduce crime and punishment.
Aggregation at the bottom has generated insights that are missing from top‑down models. People most affected by the criminal system have aggregated legal and social facts that are systemically unobservable or inaccessible to elites, or are buried, forgotten, undocumented, too difficult to acquire, or otherwise unaggregated by elites. Even when data is observable in the aggregate by elites, for reasons of time, interest, or institutional and normative commitments, its systemic nature may remain undiscovered or disregarded from up above. Aggregation is one understudied way in which systemic knowledge is created by those who have the least resources in society and who are denied access to social, economic, and political power.
Aggregation at the bottom disrupts the dichotomy between numbers and narrative. It offers a distinctive way of thinking about lived experience that is missing from criminal law scholarship. It unsettles dominant ways of understanding the insights of people most affected by the criminal system. It puts communal knowledge in a different light. The knowledge of people directly impacted by the criminal system is commonly described by both allies and critics as “lived experience,”[303] “anecdote,”[304] “narrative,”[305] and “firsthand knowledge”[306] in the harms of criminalization and incarceration. These terms describe invaluable sources of knowledge that are too often dismissed as subjective or individual.[307] I do not privilege aggregation over other forms of knowledge at the bottom or suggest that all bottom‑up work must expose systemic problems or generate group‑based interventions. Nor do I limit aggregation to quantitative data. Aggregative moves at the bottom open a window to value other forms of bottom‑up work and contestation that are too often quickly dismissed because they lack an evidence base.[308] They are a less visible and understudied slice of knowledge creation at the bottom, and a piece of the puzzle in the decarceral effort. And they reveal how the evidence‑based model stands in the way of reducing reliance on incarceration.
Obstacles
Taking aggregation at the bottom seriously demands a reorientation of the role and structures of funding. Elite aggregators have significant capital through their institutional or philanthropic affiliations. Indeed, legal scholars urge courts and system actors to spend the funds necessary to unearth systemic data through aggregation.[309] Meanwhile, people in communities most affected by incarceration have limited resources. On top of that, governments and philanthropies often invest only in “proven” strategies that promise speedy crime reduction outcomes, which are impracticable for long‑term interventions.[310] Almost every intervention in this paper is not “evidence‑based” as conceived by the evidence‑based model.[311] Most interventions from the bottom therefore do not and will not receive funding and, as a result, will not come to light without elite partnership or scholarly or media inquiry. And when elite partners drive the research agenda, ideas by bottom‑up partners risk being suppressed or even erased. These power imbalances and structures of funding dramatically limit the possibilities of community‑based initiatives, interventions, methods, and partnerships that prioritize “community‑generated research questions.”[312]
Of course, many scholars have discussed the limits of lived experience.[313] The access‑to‑data obstacles that limit what elite aggregators can see and achieve will also constrain those at the bottom. Each step of data collection and aggregation involves normative choices all the way down. Aggregation is not a cure‑all, and people at the bottom are not a monolith. Their aggregative observations and interventions may lead to ideas that are at odds with decarceral futures. None of these concerns, however, are specific to the bottom. These criticisms can be—and have been—leveled against courts and algorithms.[314] Yet there remains a sustained demand by scholars and reformers for aggregation by courts, system actors, and machines.
The simultaneous use and disregard of aggregation as a path for change in criminal law reveals how concepts used in law can serve a legitimating function. Attending to aggregation as a framework for systemic change demands studying the less visible ways in which it operates in the spaces where individual determinations of guilt, probable cause, and bail are aggregated. Failing to attend to aggregation as an approach at the bottom—while valuing it up above—legitimates the narrative that expertise in criminal law resides at the top. Evidence‑based proponents claim that people most harmed by the criminal system do not have the right kind of data to achieve the change envisioned by a decarceral agenda.[315] Shrugging off aggregative moves at the bottom helps to smooth their path.
When insights are developed by people with the least advantage, the traditional response is to see the phenomenon as an instrumental problem to be resolved by more top‑down solutions—improve the defense bar, if someone in prison intervenes in a systemic issue; build court capacity to produce more statistics, so court watchers don’t have to collect systemic data; improve the algorithm so it can predict whom the community members refer.[316] Although strengthening indigent defense and court data collection practices are laudable goals, the instinctual response to fix the ills of elites and call it a day is neither neutral nor inevitable. Shifting the locus of change back to the top legitimates existing power structures. Taking aggregation seriously as a way of knowing at the bottom shifts that power downward.[317]
Conclusion
One of the most pressing debates among criminal law scholars and reformers is whether people most harmed by policing and imprisonment have the expertise to reduce incarceration. This Article argues that the language of expertise is incomplete; it obscures alternate theories to understand how knowledge is produced. This Article introduces aggregation as a perspective to understand one way in which people in communities most harmed by the criminal system uncover and intervene in systemic problems in criminal law.
This Article also surfaces a trend in criminal law scholarship: Scholars value aggregation as a path to understand and uncover recurring problems in criminal law and policy and to intervene in those problems differently and more systemically. Scholars use it. Social scientists use it. Prosecutors can use it. Courts can use it. Algorithms use it all the time. Legal scholars champion aggregation as a framework, and rightly so—but when the aggregators are in positions of power. Yet in low‑status spaces we do not attend to aggregation qua aggregation. Endorsing a path for change when used by elites while glossing over its use by the poor reproduces the politics of knowledge. This Article theorizes and studies aggregation from down below, in those spaces where people are most harmed by policing and incarceration. It reveals that aggregation is one source of knowledge, systemic data, and power—and a hidden path to change from the bottom up.
* Assistant Professor, Rutgers Law School. For invaluable comments and conversations, I thank Amna Akbar, Anna Arons, Erin Collins, Adam Crews, Seth Endo, Jay Feinman, Ellen Goodman, Bruce Green, Eve Hanan, Stacy Hawkins, Paul Heaton, Thea Johnson, Max Kapustin, Amy Kimpel, Heidi Liu, Sandy Mayson, Dan Richman, Ryan Sakoda, Barry Scheck, Matthew Shapiro, Jocelyn Simonson, Elenore Wade, and Adnan Zulfiqar, as well as participants at the University of Michigan Law School Junior Scholars Conference, Legal Ethics Schmooze at the University of Denver Sturm College of Law, ABA-AALS Criminal Justice Roundtable, Markelloquium at Brooklyn Law School, CrimFest 2022 and 2023, Decarceration Law Professors Work in Progress Workshop, and faculty workshops at the University of Pennsylvania Law School and Rutgers Law School. Rutgers Law librarian Nancy Talley and the Biddle Law Library provided excellent research support. Many thanks to Noah Brown for early research assistance. I am grateful to Rutgers University’s Institute for the Study of Global Racial Justice, whose early career faculty fellowship made this work possible. The dedicated editorial team at the Colorado Law Review provided meticulous editing and superb insights. Finally, a special thank you to Paul Heaton for an especially formative conversation in the early stages of conceptualizing this piece. All errors are mine alone.
- Professor David Garland has stated that imprisonment becomes “mass” imprisonment when it ceases to incarcerate the individual and becomes the “systematic imprisonment of whole groups of the population.” David Garland, Introduction: The Meaning of Mass Imprisonment, in Mass Imprisonment: Social Causes and Consequences 1, 1–2 (David Garland ed., 2001); see also Benjamin Levin, The Consensus Myth in Criminal Justice Reform, 117 Mich. L. Rev. 259, 277 (2018) (arguing that critiques of “mass” incarceration “stress[] that punishment and marginalization operate collectively, rather than simply on an individual basis”); Anne R. Traum, Mass Incarceration at Sentencing, 64 Hastings L.J. 423, 427 (2013) (describing mass incarceration as “a group and systemic problem, not merely an individual problem”); Bruce Western & Christopher Muller, Mass Incarceration, Macrosociology, and the Poor, 647 Annals Am. Acad. Pol. & Soc. Sci. 166, 168 (2013) (“[I]ncarceration must be so extensive and concentrated that it imprisons not just the individual but the group.”). ↑
- See Traum, supra note 1, at 436 (“[T]here is a ‘mismatch’ between mass incarceration, which is a systemic problem . . . and our case-by-case system of criminal adjudication.”); see also Andrew Manuel Crespo, Systemic Facts: Toward Institutional Awareness in Criminal Courts, 129 Harv. L. Rev. 2049, 2053, 2057 (2016) (stating that this “transactional mode of adjudication” means that criminal courts “[l]ack[] a holistic picture of how the criminal justice system operates”); Jonathan Abel, Cop Tracing, 107 Cornell L. Rev. 927, 994 (2022) (“[I]mportant connections among cases are obscured by the atomization of the criminal justice system.”); Eve Brensike Primus, A Structural Vision of Habeas Corpus, 98 Calif. L. Rev. 1, 9 (2010) (“Systemic violations affect large groups of criminal defendants, but they are currently unaddressed by a system oriented toward individual errors.”); Brandon L. Garrett, Aggregation in Criminal Law, 95 Calif. L. Rev. 383, 439 (2007) (“[I]n our balkanized system, many courts lack a systemic picture of issues . . . .”); Spencer S. Hsu, FBI Admits Flaws in Hair Analysis Over Decades, Wash. Post (Apr. 18, 2015), https://www.washingtonpost.com/local/crime/fbi-overstated-forensic-hair-matches-in-nearly-all-criminal-trials-for-decades/2015/04/18/39c8d8c6-e515-11e4-b510-962fcfabc310_story.html [https://perma.cc/S7AF-WCSS] (“The tools don’t exist to handle systematic errors in our criminal justice system . . . .” (quoting Professor Brandon Garrett)). ↑
- See discussion infra pp. 245–50. ↑
- See discussion infra pp. 251–55. ↑
- Gordon Russell et al., Louisiana Leads Nation in Locking Up People for Life; Often, Jurors Couldn’t Even Agree on Guilt, Advocate (Apr. 21, 2018), https://www.nola.com/news/article_175540ba-e44d-5ea0-a734-970600159c77.html [https://perma.cc/MPU2-US5H] (noting arguments that the majority-jury rule “play[ed] a significant role in keeping Louisiana at the top of the nation’s incarceration pyramid”). See generally Todd R. Clear & James Austin, Reducing Mass Incarceration: Implications of the Iron Law of Prison Populations, 3 Harv. L. & Pol’y Rev. 307, 316 (2009) (arguing that ending mass incarceration requires “chang[ing] the laws that send people to prison and sometimes keep them there for lengthy terms”). ↑
- “Most dictionaries define data as facts, numbers, or just information that is used as the foundation for reasoning and decision-making.” Katharina Pistor, Rule by Data: The End of Markets?, 83 Law & Contemp. Probs. 101, 104 (2020) (citing Data, Merriam-Webster Online Dictionary, https://www.merriam-webster.com/dictionary/data [https://perma.cc/G257-A4DV]; Data, Cambridge Dictionary Online, https://dictionary.cambridge.org/us/dictionary/english/data [https://perma.cc/74R8-Q3MC]). ↑
- See discussion infra pp. 255–61. ↑
- See discussion infra pp. 261–64. ↑
- I use the phrase “the bottom” to describe people and places most affected by policing and incarceration. See Mari J. Matsuda, Looking to the Bottom: Critical Legal Studies and Reparations, 22 Harv. C.R.-C.L. L. Rev. 323, 324–26 (1987); Jocelyn Simonson, Police Reform Through a Power Lens, 130 Yale L.J. 778, 811 (2021). ↑
- Google, “Aggregation”, 56,000,000 results (Nov. 30, 2025), https://www.google.com/search?q=aggregation [https://perma.cc/E54R-AS3X]. ↑
- See Masha Efy, Data Aggregation Techniques for Effective Data Analysis, Owox (July 22, 2023), https://www.owox.com/blog/articles/data-aggregation-techniques-for-effective-data-analysis [https://perma.cc/TM79-YLGQ] (“The data aggregation definition involves compiling information from different sources to extract essential insights.”); see also Usman Hasan Khan, All You Need to Know About Data Aggregation, Astera: Blogs (July 23, 2024), https://www.astera.com/type/blog/data-aggregation [https://perma.cc/AR26-7R5W] (“Data aggregation . . . prepares data for analysis, making it easier to obtain insights into patterns and insights that aren’t observable in isolated data points.”); Pauline T. Kim, Data-Driven Discrimination at Work, 58 Wm. & Mary L. Rev. 857, 900 (2017) (stating that “once aggregated, data may reveal far more . . . [information] than the individual data points alone would suggest”). ↑
- Daniel J. Solove, Introduction: Privacy Self-Management and the Consent Dilemma, 126 Harv. L. Rev. 1879, 1891 (2013). ↑
- See, e.g., Jessica M. Eaglin, Constructing Recidivism Risk, 67 Emory L.J. 59, 72–80 (2017). ↑
- Introduction to Machine Learning, GeeksforGeeks, https://www.geeksforgeeks.org/introduction-machine-learning [https://perma.cc/VF63-3QNJ] (last updated July 29, 2025); Katrina Wakefield, A Guide to the Types of Machine Learning Algorithms and Their Applications, SAS, https://www.sas.com/en_ie/insights/articles/analytics/machine-learning-algorithms.html [https://perma.cc/5BS4-V46E]; Andrew Guthrie Ferguson, The Rise of Big Data Policing: Surveillance, Race, and the Future of Law Enforcement 8 (2017). ↑
- What Is Machine Learning?, GeeksforGeeks, https://www.geeksforgeeks.org/ml-machine-learning [https://perma.cc/46TL-C7JL] (last updated Sep. 13, 2025). ↑
- Cf. Traum, supra note 1, at 437 (“Mass incarceration . . . is a widespread social problem that results from, and is recognized as, the aggregation and concentration of many convictions and sentences.”). ↑
- I thank Dan Richman for highlighting this point. ↑
- I thank Jocelyn Simonson for highlighting this point. ↑
- See Ngozi Okidegbe, Discredited Data, 107 Cornell L. Rev. 2007, 2014, 2052–54, 2062–64 (2022) (calling for the creation of pretrial algorithms that draw on community knowledge about the harms of incarceration); S. Lisa Washington, Pathology Logics, 117 Nw. U. L. Rev. 1523, 1583–87 (2023) (centering community knowledge as essential to dismantling the family regulation system); Monica C. Bell, The Community in Criminal Justice: Subordination, Consumption, Resistance, and Transformation, 16 Du Bois Rev. 197, 208 (2019) (arguing that people most affected by the criminal system “are especially knowledgeable about systemic injustice”); I. Bennett Capers, Race, Policing, and Technology, 95 N.C. L. Rev. 1241, 1246 (2017) (stating that acts of racialized police violence are “talked about in barbershops and hair salons, on church pews and on street corners, and yes, in prisons”); Russell K. Robinson, Perceptual Segregation, 108 Colum. L. Rev. 1093, 1120 (2008) (“In general, black and white people obtain information through different informational networks, which results in racialized pools of knowledge.”); David R. Maines, Information Pools and Racialized Narrative Structures, 40 Socio. Q. 317, 323–24 (1999) (explaining that information pools are often race-based, where Black people are aware of certain information that is largely unknown to white people). ↑
- See sources cited supra note 19. ↑
- See, e.g., Bruce Green & Ellen Yaroshefsky, Prosecutorial Accountability 2.0, 92 Notre Dame L. Rev. 51, 104 (2016) (contrasting the “visible misconduct” of prosecutors, such as misconduct noted in court opinions or heard in summations, with previously less visible misconduct, “such as overcharging in order to extract guilty pleas,” explaining that the latter “comes more readily to light” through the aggregation of information); Solove, supra note 12, at 1891 (“As data gets aggregated, information that is not identifiable can become identified.”); see also infra Part I (examining the power of aggregation to uncover latent patterns and the systemic dimensions of seemingly isolated events). ↑
- See discussion infra pp. 230–38. ↑
- See infra Sections I.D and I.E. ↑
- See discussion infra pp. 231–38. ↑
- See infra p. 273. ↑
- John Rappaport, Some Doubts About “Democratizing” Criminal Justice, 87 U. Chi. L. Rev. 711, 811 (2020) (questioning whether community-based approaches can achieve reductions in prison populations). ↑
- See infra pp. 274. ↑
- See infra p. 276. ↑
- See infra p. 276 and notes 313–314. ↑
- Katherine Beckett, Mass Incarceration and Its Discontents, 47 Contemp. Socio. 11, 21 (2018). ↑
- See infra Section III.A. ↑
- Benjamin Levin, Criminal Justice Expertise, 90 Fordham L. Rev. 2777, 2782 (2022) (noting that both camps adopt a “shared appeal to the language of experts and expertise”). ↑
- See Pierre Schlag, Expertopia—The Rule of Expertise, at 65 (June 2, 2022) (unpublished manuscript) (on file with author) (“[E]xpertise has but one move, or one tendency: to reduce everything to the order of expert knowledge.”). ↑
- Anna Lvovsky, Rethinking Police Expertise, 131 Yale L.J. 475, 475, 492 (2021) (challenging the “assumption that more expert policing is, invariably, more lawful policing”). ↑
- I explore this question further in future work. See generally Seema Saifee, Expertise or Struggle? (July 2025) (draft in progress) (on file with author). ↑
- Traum, supra note 1, at 427 (discussing scholarly debate on the term “mass incarceration”). Sociologist David Garland coined the term “mass imprisonment” to describe the “systematic imprisonment of whole groups of the population.” Garland, supra note 1, at 2. ↑
- Levin, supra note 1, at 277. ↑
- See, e.g., Katherine R. Kruse, Instituting Innocence Reform: Wisconsin’s New Governance Experiment, 2006 Wis. L. Rev. 645, 645–46 & n.2 (crediting the Innocence Project for creating a reform agenda based on “patterns of dysfunction in the investigation and prosecution of crimes” revealed by DNA exonerations); Barack Obama, The President’s Role in Advancing Criminal Justice Reform, 130 Harv. L. Rev. 811, 861 & n.295 (2017) (noting that exonerations of wrongly convicted individuals revealed widespread errors in FBI testimony on microscopic hair evidence); Jed S. Rakoff, Jailed by Bad Science, N.Y. Rev. of Books (Dec. 19, 2019), https://www.nybooks.com/articles/2019/12/19/jailed-bad-forensic-science [https://perma.cc/99Y2-R673 ] (describing how DNA exonerations “exposed how bad much other forensic evidence was”); Abel, supra note 2, at 1003–04 (describing public defenders who uncovered patterns of police and prosecutorial misconduct). ↑
- See, e.g., Brandon L. Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong (2011); Kara MacKillop & Neil Vidmar, Decision-Making in the Dark: How Pre-Trial Errors Change the Narrative in Criminal Jury Trials, 90 Chi.-Kent L. Rev. 957, 957 (2015) (stating that Garrett’s work “quantitatively evaluated the first 250 DNA exonerations and exposed clear patterns of error within those cases”). ↑
- See, e.g., United States v. Hall, 974 F. Supp. 1198, 1203–05 (C.D. Ill. 1997) (stating that social scientists engage in systematic observation of false confessions in exoneration cases); Erica Beecher-Monas, Blinded by Science: How Judges Avoid the Science in Scientific Evidence, 71 Temp. L. Rev. 55, 87–88 (1998) (stating that social scientists “rely on data generated through systematic observation”). ↑
- See, e.g., Abel, supra note 2, at 941–42 (stating that an investigation by The New York Times revealed a pattern of misconduct in cases of former detective Louis Scarcella); Frances Robles & N.R. Kleinfield, Review of 50 Brooklyn Murder Cases Ordered, N.Y. Times (May 11, 2013), https://www.nytimes.com/2013/05/12/nyregion/doubts-about-detective-haunt-50-murder-cases.html [https://perma.cc/N397-XSXY] (uncovering “disturbing patterns” in a dozen of Scarcella’s cases, including using the same “eyewitness” for multiple murder prosecutions). ↑
- See, e.g., Barry C. Scheck, Conviction Integrity Units Revisited, 14 Ohio St. J. Crim. L. 705, 743–44 (2017) (recommending “root cause analysis” of wrongful convictions); Primus, supra note 2, at 12, 41 (proposing restructuring federal habeas review to focus on systemic errors); Abel, supra note 2, at 934–35, 939–41 (calling for “cop tracing”—a systematic effort to identify and investigate the universe of prior cases handled by a discredited officer to recognize the systemic dimensions of a single officer’s misconduct); Robles & Kleinfield, supra note 41 (announcing an audit by a district attorney’s office into a disgraced detective’s past homicide cases); Charlie Nelson Keever, Mass Exonerations: Protocols for Reviewing Convictions for Serious Crimes in Cases of Systemic Misconduct, 28 Berkeley J. Crim. L. 229, 241–42 (2023) (discussing “mass” exonerations where dozens, hundreds, or thousands of convictions were overturned due to a pattern of misconduct or a common bad actor). ↑
- See Jim Dwyer et al., Actual Innocence: Five Days to Execution and Other Dispatches from the Wrongly Convicted 246, 257, 258 (2000) (describing common factors in DNA exonerations, including inadequate and underfunded defense, prosecutorial misconduct, eyewitness misidentification, false confessions, forensic fraud, and use of jailhouse informants); see also sources cited supra note 39 (noting academic research that identified recurring patterns in wrongful conviction cases). ↑
- See Hall, 974 F. Supp. at 1203–05; Beecher-Monas, supra note 40, at 88– 89. ↑
- See Robles & Kleinfield, supra note 41; Abel, supra note 2, at 941–42, 964– 65, 1002–04. ↑
- See, e.g., Andrew Guthrie Ferguson, Persistent Surveillance, 74 Ala. L. Rev. 1, 19–20, 27–29 (2022) (considering the Fourth Amendment implications of new surveillance technologies that enable police to aggregate many data points “to see patterns of movement, associations, and activities that were not observable before”); Ariel Porat & Eric A. Posner, Aggregation and Law, 122 Yale L.J. 2, 34–45 (2012) (noting special concerns that aggregation of claims and elements in criminal trials would pose to the rights of the accused); infra notes 88, 120 (citing scholarship critiquing predictive algorithms). ↑
- Alexandra Natapoff, Aggregation and Urban Misdemeanors, 40 Fordham Urb. L.J. 1043, 1043, 1046 (2013) (describing the “aggregating tendencies” of the misdemeanor system as “conceptual game-changers”). ↑
- Id. at 1043, 1053, 1063, 1066 (contrasting the particularized suspicion required for searches and seizures under the Fourth Amendment with the “generalized nature of the selection criteria” that shape police decisions to stop and arrest); see also Bernard E. Harcourt & Tracey L. Meares, Randomization and the Fourth Amendment, 78 U. Chi. L. Rev. 809, 813 (2011) (arguing that police suspicion “attaches to group-based traits”). ↑
- Natapoff, supra note 47, at 1043, 1070 (explaining that plea deals are determined largely by reference to the local “price” for the offense, rendering the process a “categorical exercise” where “bargains are struck based on the institutional habits of the local jurisdiction”); see also Malcolm M. Feeley, The Process Is the Punishment: Handling Cases in a Lower Criminal Court 187 (1979) (describing the reality in American criminal courts as “more akin to modern supermarkets in which prices for various commodities have been clearly established and labeled”). ↑
- Natapoff, supra note 47, at 1045, 1066–67 (noting that the concept of bail requires an individual determination of flight and risk but that in practice jurisdictions often maintain bail schedules that “automatically set bail amounts based on the nature of the offense”). ↑
- Id. at 1072; Feeley, supra note 49, at 10 (describing criminal cases where “[a]rrestees were arraigned in groups and informed of their rights en masse”). ↑
- Natapoff, supra note 47, at 1055–56 (“[I]n theory, the imposition of punishment is a highly individuated process.”); see also Albert W. Alschuler, The Failure of Sentencing Guidelines: A Plea for Less Aggregation, 58 U. Chi. L. Rev. 901, 902 (1991) (lamenting “the movement from individualized to aggregated sentences,” where punishment is determined based on offense category and criminal history, as “mark[ing] a backward step in the search for just criminal punishments”); Dorothy E. Roberts, The Social and Moral Cost of Mass Incarceration in African American Communities, 56 Stan. L. Rev. 1271, 1301–02 (2004) (“[T]he current sentencing regime that generated the enormous prison population is far from individualized. Indeed, the prison explosion is largely attributable to sentencing changes that made punishment less individualized.”). ↑
- Natapoff, supra note 47, at 1043, 1045, 1047, 1049, 1055 (“identif[ying] aggregation as a key feature of what [ails]” the American misdemeanor system). ↑
- Id. at 1043, 1074. ↑
- Id. at 1043, 1084 (“[T]he aggregating tendencies of the petty offense process in fact amount to a crime control system.”); see also Malcolm Feeley & Jonathan Simon, Actuarial Justice: The Emerging New Criminal Law, in The Futures of Criminology 173, 175 (David Nelken ed., 1994) (arguing that punishment and treatment goals of the criminal system have been replaced with risk management); Issa Kohler-Hausmann, Managerial Justice and Mass Misdemeanors, 66 Stan. L. Rev. 611, 611 (2014) (describing misdemeanor justice as operating under a managerial model that does not punish individual criminal conduct but “us[es] the criminal process to sort and regulate the populations targeted”). ↑
- Natapoff, supra note 47, at 1085. ↑
- Id. at 1049 (concluding that “aggregation is a prime contributor to the urban criminal system’s loss of legitimacy”). ↑
- Garrett, supra note 2, at 411–12, 420, 435, 448 (noting that forensic fraud by a crime lab can affect hundreds of cases, inadequate indigent defense does not concern just one person’s rights, but “implicates the indigent defense funding system of [an entire city],” and racial disparity in death sentencing recurs across capital cases). “When viewed in the aggregate each set of problems may resemble mass torts or mass injuries to criminal defendants. A mass remedy is only appropriate.” Id. at 449. ↑
- Id. at 386; e.g., id. at 418, 420–21, 441 (distinguishing between a court consolidating cases raising the same claim to examine a systemic problem—aggregation—with a court assigning all habeas petitions to a single judge who considers each petition individually—non-aggregation). ↑
- Id. at 416–19, 424 (describing a Louisiana court that consolidated ineffective assistance of counsel claims against the same public defender raised by people in multiple cases accused of unrelated crimes, pooled information across cases to uncover evidence of systemic inadequacies, and fashioned systemic remedies to secure indigent defense funding); see also Stephanos Bibas, The Psychology of Hindsight and After-the-Fact Review of Ineffective Assistance of Counsel, 2004 Utah L. Rev. 1, 8 (“Unfortunately, this victory was only temporary. Over time, the money failed to keep up with inflation and caseloads, and today New Orleans defense counsel still have heavy caseloads.”). ↑
- Garrett, supra note 2, at 387, 410–11, 424, 428–29, 450 (discussing the limits of court-based aggregation, including the “narrow[] . . . range of issues that aggregation can reach in criminal law,” namely criminal procedure rights common to a group, not questions of guilt or elements of a crime, and court reluctance due in part to conceptions of the right to an individual day in court). ↑
- Id. at 427, 446; see also id. at 412–14, 417, 420–21 (detailing the ways in which court-based aggregation can detect gross inadequacies and systematic improprieties in criminal justice institutions by pooling information, collecting and reviewing data, and investigating the institutions responsible). ↑
- Id. at 446; see also id. at 412–16, 418, 428, 447 (describing how West Virginia’s highest court used aggregation—grouping cases together to “create[] [in effect] an issue class action”—to uncover a pattern of corruption in the state crime lab, ultimately ordering group reversals of convictions and structural reform of the crime lab). ↑
- Id. at 388, 431, 446, 448 (showing that “aggregating . . . [a] group problem” empowers courts to examine a systemic issue that “no one criminal defendant would have a great chance of prevailing on” because appellate courts typically find the constitutional violation in any individual case to be “harmless, defaulted, or unexhausted”). ↑
- Id. at 387, 447–50. ↑
- Crespo, supra note 2, at 2053. ↑
- Id. at 2069–70. ↑
- Id. at 2057, 2068–69 (arguing that data aggregation enables courts to “‘see’ beyond the truncated transactional horizon of a given case” to gain a “holistic picture of how the criminal justice system operates”); id. at 2088–92 (stating, for example, that judges are often unaware of problems in the same prosecutor’s office that arise in neighboring courtrooms, but technology can aggregate digital records of disclosures by the same prosecutor’s office court-wide to make visible recurring Brady violations in a specific office). ↑
- Id. at 2070, 2073. ↑
- Id. at 2082–92; id. at 2113 (arguing that “empiricism that exposes the systemic nature of [criminal justice] problems is likely to further efforts at reform”). ↑
- Id. at 2069–70. Crespo describes this process as “systemic factfinding.” Id. at 2052, 2101; see also id. at 2072 (noting that Fourth Amendment events “fall into readily identifiable patterns”); id. at 2070, 2073–75, 2083–85, 2109 (explaining that courts can see these patterns in the same police department, for example, by using software that can automatically search, code, and organize probable cause affidavits and warrant returns to assess the likelihood that specific police observations will lead to the discovery of evidence); id. at 2073–85 (demonstrating in detail how this process can enhance systemic oversight of police); accord Max Minzner, Putting Probability Back into Probable Cause, 87 Tex. L. Rev. 913, 939 (2009) (arguing that “[a]dding a success-rate requirement” to the probable cause analysis “will begin to shift law enforcement incentives in positive ways”). ↑
- Crespo, supra note 2, at 2070. Crespo’s proposal stems from an understanding that criminal courts’ core constitutional mandate is to regulate state power at a systemic level. Id. at 2055–56; see also id. at 2101 (arguing that systemic factfinding can “reshape constitutional criminal adjudication as a tool” to deter law enforcement misconduct at an institutional level); Barry Friedman & Maria Ponomarenko, Democratic Policing, 90 N.Y.U. L. Rev. 1827, 1865 (2015) (criticizing the case-by-case mode of adjudication through which courts currently undertake this duty as a “completely inadequate” means of regulating police); Daphna Renan, The Fourth Amendment as Administrative Governance, 68 Stan. L. Rev. 1039, 1056, 1114 (2016) (arguing that a transactional, one-off mode of adjudication denies courts the “tools to put th[e] pieces together, to see a whole greater than the sum of its parts” when “overlapping and interconnected” police practices operate “in combination”); Tracey L. Meares, Programming Errors: Understanding the Constitutionality of Stop-and-Frisk as a Program, Not an Incident, 82 U. Chi. L. Rev. 159, 162, 164 (2015) (arguing that police practices are “carried out systematically” as a “program” and that “individual-level analysis is unsuitable for assessing the nature of violations” presented by proactive policing). ↑
- Crespo, supra note 2, at 2056; see also id. at 2107, 2117 (explaining that “systemic factfinding’s highest and ultimate purpose is to assist criminal courts in fulfilling [their core constitutional responsibility]”) (emphasis removed). ↑
- Id. at 2066. “And crucially, it is this capacity to aggregate dispersed information that is the key to unlocking criminal courts’ capacity for greater institutional awareness.” Id. at 2069. ↑
- See, e.g., Abel, supra note 2, at 994; Adam M. Gershowitz, Prosecutorial Dismissals as Teachable Moments (and Databases) for the Police, 86 Geo. Wash. L. Rev. 1525, 1529 (2018) (“[P]rosecutors could use their dismissal notifications to create a database that allows the prosecutor’s office to see which officers are bringing in weak cases that are ultimately dismissed.”). ↑
- Abel, supra note 2, at 939. “An officer whose corruption comes out in one case will likely have corrupted other cases.” Id. at 934. ↑
- Id. at 934. “When an officer’s credibility is destroyed in one case . . . . no mechanism exists to force criminal justice actors . . . to examine or even identify the discredited officer’s other cases.” Id. at 935. “This Article argues for making systemic review of a [discredited] cop’s cases automatic, rather than ad hoc.” Id. at 939. ↑
- Id. at 995. ↑
- Id. at 998–1000. These are often referred to as “Brady lists.” Id. at 998. ↑
- Id. at 1000. ↑
- Id. at 995, 998. ↑
- Id. at 927. “With paid informants, lab technicians, and forensic experts, the criminal justice system has acknowledged that serial witnesses pose systemic problems—that an individual’s misconduct in one case may well be repeated in numerous cases. That same acknowledgment is absent when the serial witness is a cop.” Id. at 992. ↑
- Eaglin, supra note 13, at 61 (stating that algorithmic tools are used “at every stage of the criminal justice process”); Sandra G. Mayson, Bias In, Bias Out, 128 Yale L.J. 2218, 2218 (2019) (noting the same); Ngozi Okidegbe, To Democratize Algorithms, 69 UCLA L. Rev. 1688, 1692 (2023) (same). ↑
- See e.g., Eaglin, supra note 13; Mayson, supra note 83; Okidegbe, supra note 19; Sonja B. Starr, Evidence-Based Sentencing and the Scientific Rationalization of Discrimination, 66 Stan. L. Rev. 803 (2014); Erin Collins, Punishing Risk, 107 Geo. L.J. 57 (2018); Bernard E. Harcourt, Against Prediction: Profiling, Policing, and Punishing in an Actuarial Age (2007); Megan Stevenson, Assessing Risk Assessment in Action, 103 Minn. L. Rev. 303 (2018). ↑
- See Solon Barocas & Andrew D. Selbst, Big Data’s Disparate Impact, 104 Calif. L. Rev. 671, 677 (2016) (“In particular . . . [data mining] automates the process of discovering useful patterns, revealing regularities upon which subsequent decision making can rely.”). ↑
- Eaglin, supra note 13, at 72–75; id. at 68 n.41 (noting that some algorithmic tools allow the computer to derive the factors to observe); Ferguson, supra note 14, at 147; see also sources cited supra notes 13–14. ↑
- Eaglin, supra note 13, at 68; Mayson, supra note 83, at 2224. ↑
- See, e.g., Okidegbe, supra note 19; Barocas & Selbst, supra note 85; Virginia Eubanks, Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (2018); Ruha Benjamin, Race After Technology: Abolitionist Tools for the New Jim Code (Polity 2019); Cathy O’Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (2016); Vincent M. Southerland, The Intersection of Race and Algorithmic Tools in the Criminal Legal System, 80 Md. L. Rev. 487 (2021); Andrew D. Selbst, Disparate Impact in Big Data Policing, 52 Ga. L. Rev. 109 (2017); Julia Angwin et al., Machine Bias, ProPublica (May 23, 2016), https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing [https://perma.cc/X8GM-45XG]; Michele Gilman, How Algorithms Intended to Root Out Welfare Fraud Often Punish the Poor, PBS (Feb. 17, 2020, at 7:15 AM), https://www.pbs.org/newshour/economy/column-how-algorithms-to-root-out-welfare-fraud-often-punish-the-poor [https://perma.cc/J8SY-NDP6]. ↑
- See Ferguson, supra note 14, at 147–50 (describing efforts to create algorithms to identify officers at risk for negative police-civilian interactions); Jaeah Lee, How Science Could Help Prevent Police Shootings, Mother Jones (May/June 2016), https://www.motherjones.com/politics/2016/07/data-prediction-police-misconduct-shootings [https://perma.cc/RV5H-2YQT]. ↑
- Crespo, supra note 2, at 2073–78, 2083–85 (arguing that algorithms can detect text patterns in probable cause scripts to assist judges in assessing how often police searches based on the same set of observations yielded relevant evidence). ↑
- Tim Goddard & Randolph R. Myers, Against Evidence-Based Oppression: Marginalized Youth and the Politics of Risk-Based Assessment and Intervention, 21 Theoretical Criminology 151, 162 (2017); Mayson, supra note 83, at 2218, 2284, 2287–90, 2293–94 (proposing that predictive algorithms be used as “diagnostic tools” to respond to those deemed “high risk” with support not restraint). ↑
- Mayson, supra note 83, at 2284–85, 2295 (considering the use of predictive algorithms for “diagnostic or supportive purposes”). ↑
- Natapoff, supra note 47, at 1063. ↑
- Garrett, supra note 2, at 388, 448; cf. William J. Stuntz, The Uneasy Relationship Between Criminal Procedure and Criminal Justice, 107 Yale L.J. 1, 70 (1997) (criticizing courts for failing to “enforce minimum levels of funding for public defenders’ offices”). ↑
- Crespo, supra note 2, at 2112. ↑
- Cf. Beth Stephens, Translating Filártiga: A Comparative and International Law Analysis of Domestic Remedies for International Human Rights Violations, 27 Yale J. Int’l L. 1, 24 (2002) (“[I]t seems noncontroversial to conclude that the United States is among the countries with the highest rate of recourse to the courts to resolve disputes.”). ↑
- Crespo, supra note 2, at 2068 (internal quotation marks omitted) (quoting Daniel J. Meltzer, Deterring Constitutional Violations by Law Enforcement Officials: Plaintiffs and Defendants as Private Attorneys General, 88 Colum. L. Rev. 247, 286 (1988)). ↑
- Id. at 2064. ↑
- Id. at 2051, 2063–64; Garrett, supra note 2, at 450. ↑
- Natapoff, supra note 47, at 1045. ↑
- Id. at 1057–58; Garrett, supra note 2, at 401, 428–29. ↑
- Garrett, supra note 2, at 429; accord id. at 401 (“I mean to only indicate that there are some recurring errors that could benefit from some mechanism to pursue the aggregate remedies.”); id. at 448–49 (listing, as examples, “chronic under-funding of indigent defense, repeat prosecutorial misconduct, malfeasance in forensic laboratories, or patterns of concealing exculpatory evidence”); id. at 428 (“Only [procedural] . . . rights [affirmatively asserted by people charged with crimes], not questions of guilt or elements of the crime, are appropriate for aggregation.”). ↑
- Crespo, supra note 2, at 2069. ↑
- Id. at 2074–75, 2088–92. ↑
- Id. at 2069. ↑
- Garrett, supra note 2, at 386–91, 450; Crespo, supra note 2, at 2107. ↑
- Garrett, supra note 2, at 385; see also id. at 391 (“Aggregation not only permits efficient adjudication, but it can also improve the quality of adjudication, including its accuracy and fairness.”); id. at 450 (“[A]ggregation promises significant improvement in the efficiency, accuracy, fairness and integrity of our criminal justice system.”); Crespo, supra note 2, at 2075, 2107. ↑
- Crespo, supra note 2, at 2101, 2117. ↑
- See supra notes 43–45, 89, 91 and accompanying text. ↑
- Abel, supra note 2, at 994–95 (citing scholars who have called for aggregating such data across cases to reveal larger patterns); see, e.g., Ronald F. Wright et al., The Jury Sunshine Project: Jury Selection Data as a Political Issue, 2018 U. Ill. L. Rev. 1407, 1442; Gershowitz, supra note 75, at 1525. ↑
- Abel, supra note 2, at 994; see also Crespo, supra note 2, at 2051 (arguing that the process of case-by-case adjudication “inculcates in criminal courts a transactional myopia”). ↑
- Abel, supra note 2, at 1002; accord Green & Yaroshefsky, supra note 21, at 53, 56, 59–61, 103–07 (arguing that aggregating information across cases is crucial to seeing prosecutorial misconduct “as a widespread, systemic problem” and describing the challenges to such aggregation prior to the information age). ↑
- Abel, supra note 2, at 939–41, 968, 995 (calling on prosecutors’ offices and police departments to look backward when an officer is discredited to identify the officer’s case history); Crespo, supra note 2, at 2088–90 (calling on courts to collect a database of “systemic facts” to reveal trends across cases); Gershowitz, supra note 75, at 1550 (calling on prosecutors to maintain a database to track officers who routinely bring in weak cases that result in dismissals). ↑
- See Abel, supra note 2, at 995 (stating that “something as basic and banal as a list of an officer’s [past] cases” is “logistically impossible to compile” in a criminal “justice system that is disorganized, decentralized, and disinclined to provide access to [systemic] data”); Wright et al., supra note 110, at 1442 (“It is startling that public courts, in an age when electronic information surrounds us on all sides, make it so difficult to track jury selection practices across different cases.”). ↑
- See Abel, supra note 2, at 956–58, 996–98 (describing the legal and logistical limits on the ability of defense counsel, prosecutors, and police to identify an officer’s case history); Rebecca Wexler, Privacy Asymmetries: Access to Data in Criminal Defense Investigations, 68 UCLA L. Rev. 212, 215 (2021) (explaining that privacy statutes prevent defense counsel from accessing information that police and prosecutors can access); Wright et al., supra note 110, at 1407, 1442 (describing the limits on public access to jury selection data). ↑
- E.g., Roby Chavez, Aging Louisiana Prisoners Were Promised a Chance at Parole After 10 Years. Some Are Finally Free, PBS (Nov. 26, 2021, at 4:13 PM), https://www.pbs.org/newshour/nation/aging-louisiana-prisoners-were-promised-a-chance-at-parole-after-10-years-some-are-finally-free [https://perma.cc/AT6T-QTVY] (discussing Louisiana’s “10/6 lifers,” dozens of people sentenced to life imprisonment with the promise of parole eligibility after ten years and six months but who, for decades, were forgotten). ↑
- See generally Monica P. Bhatt et al., Predicting and Preventing Gun Violence: An Experimental Evaluation of READI Chicago, 139 Q.J. Econ. 1 (2024) (concluding that factors contributing to success of a gun violence intervention were captured by community members with on-the-ground knowledge but unobservable to the algorithm). ↑
- See, e.g., Crespo, supra note 2, at 2069 (discussing aggregation of court data); Abel, supra note 2, at 939–40, 958, 965 (calling for a complete list of a discredited officer’s past cases to be compiled from police and prosecutorial records); Okidegbe, supra note 19, at 2012 (stating that pretrial algorithms are built on law enforcement and court data); Eaglin, supra note 13, at 101, 103 (explaining that arrest data is low cost and easy to access); id. at 66 (arguing that the desire for cheap and accessible data “incentivizes [algorithm] developers to make construction choices” that diverge from the interests of society). ↑
- See supra notes 43–45 and accompanying text. ↑
- Barocas & Selbst, supra note 85, at 675, 677; id. at 671 (“[A]n algorithm is only as good as the data it works with.”); see also Khan, supra note 11 (“Incomplete data can disrupt aggregation and generate inaccurate or skewed results.”); Mayson, supra note 83, 2251–52 (explaining that most risk assessment tools in the criminal system predict arrest because “the data do not allow for direct crime prediction”); id. at 2253 (discussing “[t]he choice to predict arrest”); David A. Harris, Profiles in Injustice: Why Racial Profiling Cannot Work 77–78 (2002) (explaining that arrest rates measure law enforcement activity and do not fully or accurately portray all crime or who commits it); Okidegbe, supra note 19, at 2007, 2039 (explaining that police data reflects the over-policing of Black communities and that algorithms built on this data reflect and reproduce the inequities in the data); Selbst, supra note 88, at 123, 133–34 (explaining that algorithms “will learn that people of color commit a higher percentage of ‘crimes’ than they do in reality” given the history of racialized policing). ↑
- Spates of law review articles have raised these concerns with predictive algorithms, but similar concerns are present with non-predictive technologies. Consider an example outside the criminal law context: The City of Boston sought to improve its ability to find and fix potholes. To allocate its resources efficiently, the city released the smartphone app, Street Bump, which drew on drivers’ GPS and accelerometer data to detect potholes in Boston. A large stream of data “reveal[ed] the status of street surfaces at low cost to the city.” Data on “three or more bumps at the same location trigger[ed] an inspection.” Lena V. Groeger, When the Designer Shows Up in the Design, ProPublica (Apr. 4, 2017, at 8:00 AM), https://www.propublica.org/article/when-the-designer-shows-up-in-the-design [https://perma.cc/ZUU9-666M]. But poor people, particularly older people, are less likely to have smartphones. As a result, the dataset “miss[ed] inputs” from “significant parts of the population” who have the fewest resources, further entrenching existing inequities. Kate Crawford, The Hidden Biases in Big Data, Harv. Bus. Rev. (Apr. 1, 2013), https://hbr.org/2013/04/the-hidden-biases-in-big-data [https://perma.cc/M59B-NXWE]. ↑
- Okidegbe, supra note 19, at 2012; id. at 2025, 2051, 2060, 2063 (arguing that the choice to train pretrial risk assessment algorithms on police data sustains a conception of public safety that associates “risk” with developer-chosen indicators of crime commission and nonappearance in court). Okidegbe argues that pretrial algorithms “omit considerations of the harms of [pretrial] incarceration.” Id. at 2064 (emphasis added). Okidegbe calls for pretrial algorithms to be built with data from “community knowledge sources.” Id. at 2014, 2032, 2052–54 (arguing that pretrial algorithms should draw upon qualitative data on the injustices of the bail system produced by people most harmed by pretrial incarceration); id. at 2053 (stating that this knowledge is “inaccessible to outsiders”). ↑
- Groeger, supra note 121 (noting that before data is analyzed, a decision must be made about what data to seek to help answer a question). ↑
- This is true for humans and algorithms. Eaglin, supra note 13, at 75 (stating that algorithm developers must specify the outcomes they wish to study by translating a problem into a formal question about variables); id. at 72–89 (discussing the normative judgments embedded in algorithmic construction); Barocas & Selbst, supra note 85, at 678 (stating that this is a “necessarily subjective process”); see also Selbst, supra note 88, at 132 (“Using data mining . . . tends to bias organizations toward questions that are easier for computers to understand.”); Mayson, supra note 83, at 2252 (explaining that accurate crime data does not exist and that this “fundamental data limitation” is why most risk-assessment tools predict future arrest rather than future crime). ↑
- See The Politics of Numbers 3 (William Alonso & Paul Starr eds., Russell Sage Found. 1987) (arguing that “political judgments are implicit in the choice of what to measure, how to measure it, how often to measure it, and how to present and interpret the results”); Erin Collins, Abolishing the Evidence-Based Paradigm, 48 B.Y.U. L. Rev. 403, 426 (2022) (“Research questions do not simply exist; they are chosen by those empowered to set the research agenda.”); Eaglin, supra note 13, at 87–88 (demonstrating that algorithmic construction reflects the normative choices of developers). ↑
- Groeger, supra note 121. ↑
- Id. (“The way in which data is collected often reflects something about the people who collect it.”). ↑
- Id. (“Data doesn’t speak for itself . . . .”). ↑
- See Eaglin, supra note 13, at 87–88; Collins, supra note 125, at 409–10, 437 (arguing that the contemporary data-driven approach to criminal law reform prioritizes efficiency and fiscal savings); Benjamin, supra note 88, at 11 (“[E]ven just deciding what problem needs solving requires a host of judgments; and yet we are expected to pay no attention to the man behind the screen.”); see also Sarah Wall, A Critique of Evidence-Based Practice in Nursing: Challenging the Assumptions, 6 Soc. Theory & Health 37, 49 (2008) (“[T]he choice of research questions, the funding of research, and the consequences (uses) of research are deeply structured by the interests and values of powerful groups, including the professions. What passes for objective research is a search for what elites want knowledge about.”). ↑
- I thank Sandy Mayson for clarifying this point. ↑
- Seema Tahir Saifee, Decarceration’s Inside Partners, 91 Fordham L. Rev. 53, 84 (2022); Jessica T. Simes, Punishing Places: The Geography of Mass Imprisonment 43 (2021). ↑
- The Corr. Ass’n of N.Y., The Prison Population Explosion in New York State: A Study of Its Causes and Consequences with Recommendations for Change 1 (1982). ↑
- Seema Saifee, One of the Best Ideas for Ending Mass Incarceration Was Thought Up in a Prison, Slate (Apr. 11, 2023, at 10:30 AM), https://slate.com/news-and-politics/2023/04/green-haven-think-tank-history-invest-divest-prison-study.html [https://perma.cc/R7X6-NTCH]; Orisanmi Burton, Attica Is: Revolutionary Consciousness, Counterinsurgency and the Deferred Abolition of New York State Prisons 152 (2016) (Ph.D. dissertation, University of North Carolina at Chapel Hill) (on file with author); Pam Widener, Man of the Year: Eddie Ellis at Large, Prison Life, Oct. 1996, at 44, 49–50, https://www.prisonlegalnews.org/media/publications/Prison_Life_October_1996.pdf [https://perma.cc/GNM9-Q9F8] (“Every prison I was in,” Think Tank member Eddie Ellis said, “I seemed to know everyone . . . . [p]eople who came from the neighborhood. And if I didn’t know them personally, some friend of mine knew them.”). ↑
- Widener, supra note 133, at 50; Burton, supra note 133, at 152 (“Captives knew this from experience, as they often found themselves imprisoned alongside many of the people they knew in the street.”); Saifee, supra note 131, at 85. ↑
- Widener, supra note 133, at 50. ↑
- Burton, supra note 133, at 153. ↑
- Ctr. for NuLeadership, The Seven Neighborhood Study Revisited 3 (2013), https://static1.squarespace.com/static/624def0f36d3686eb5ecc0f6/t/664971897f5e377e1f7f1f8a/1716089227842/THE+SEVEN+NEIGHBORHOOD+STUDY+REVISITED_Center+NuLeadership.pdf [https://perma.cc/4Y2U-YCB9]. ↑
- Widener, supra note 133, at 50. ↑
- Saifee, supra note 131, at 86–87. ↑
- Id. at 87. ↑
- Francis X. Clines, Ex-Inmates Urge Return to Areas of Crime to Help, N.Y. Times (Dec. 23, 1992), https://www.nytimes.com/1992/12/23/nyregion/ex-inmates-urge-return-to-areas-of-crime-to-help.html [https://perma.cc/2HML-3GJL]. ↑
- Saifee, supra note 131, at 88–89; Saifee, supra note 133. ↑
- Saifee, supra note 131, at 88–89; see also Simes, supra note 131, at 44 (“[T]he Think Tank inspired [this] further groundbreaking work” that “aim[ed] to visualize the Think Tank’s original insights”). ↑
- Saifee, supra note 131, at 93. ↑
- Id. at 90. ↑
- Id. at 90–91. ↑
- Saifee, supra note 133 (arguing that the Think Tank “upended the dominant narrative of crime and punishment”). ↑
- Saifee, supra note 131, at 91. ↑
- Id. at 91–92. ↑
- Jeffrey Fagan, Valerie West & Jan Holland, Reciprocal Effects of Crime and Incarceration in New York City Neighborhoods, 30 Fordham Urb. L.J. 1551, 1552, 1568 (2003) (citing to the New York Times article on the Think Tank’s seven neighborhood study); accord Simes, supra note 131, at 44 (“Long before any such publications in academic journals of the social sciences, the Think Tank’s report drew a direct connection between community and prison.”). ↑
- Saifee, supra note 133; Saifee, supra note 131, at 93; see also, e.g., Invest-Divest, Movement For Black Lives, https://m4bl.org/policy-platforms/invest-divest [https://perma.cc/XV2N-SSCV] (demanding divestment from prisons and policing and investment in education, healthcare, employment, and safety of Black people); Amna A. Akbar, An Abolitionist Horizon for (Police) Reform, 108 Calif. L. Rev. 1781, 1820 n.172 (2020) (citing a “good deal of data” showing that access to a living wage, jobs, education, housing, and healthcare leads to a decline in crime); Steven Hawkins, Education vs. Incarceration, Am. Prospect (Dec. 6, 2010), https://prospect.org/special-report/education-vs.-incarceration [https://perma.cc/KW64-V7TL] (calling for greater investments in education in high-incarceration neighborhoods as a way to reduce crime); Eugenia C. South, To Combat Gun Violence, Clean Up the Neighborhood, N.Y. Times (Oct. 8, 2021, at 2:12 PM), https://www.nytimes.com/2021/10/08/opinion/gun-violence-biden-philadelphia.html [https://perma.cc/CF2G-PKSZ] (describing empirical studies demonstrating that greening and cleaning vacant land in segregated, disadvantaged neighborhoods resulted in up to a 29 percent decline in gun violence); Katherine Beckett, Ending Mass Incarceration: Why It Persists and How to Achieve Meaningful Reform 4 (2022) (noting evidence that investments in social welfare reduce crime and violence). ↑
- Mariame Kaba, We Do This ‘Til We Free Us: Abolitionist Organizing and Transforming Justice 174 (2021); Saifee, supra note 131, at 97. ↑
- Saifee, supra note 131, at 88–97; see also Simes, supra note 131, at 43–44 (stating that the Think Tank’s insights, methods, and data collection “deeply inform the work and research presented throughout this book”). ↑
- Saifee, supra note 131, at 89–93. The Think Tank’s research also motivated public health scholar Robert Fullilove to shift his own research to the role of mass incarceration in driving the HIV epidemic. Id. at 92 n.243. Fullilove said he “owe[d] it all to [the Think Tank’s] pioneer[ing] contributions.” Id. (noting that when he looked at the Think Tank’s data, “it became clear that those were the [seven] neighborhoods with the highest rates of HIV/AIDS in [New York] [C]ity”) (all alterations in original); see also Robert E. Fullilove, Mass Incarceration in the United States and HIV/AIDS: Cause and Effect?, 9 Ohio St. J. Crim. L. 353, 357 (2011) (arguing that “the greatest engine driving the [HIV] epidemic was the cycling of inmates in and out of prison and in and out of their communities”). ↑
- Ctr. for NuLeadership, supra note 137, at 5; Saifee, supra note 131, at 93; Burton, supra note 133, at 152; Widener, supra note 133, at 53 (stating that the Think Tank’s perspective was premised on the idea that the failure of social institutions serving Black and Latinx communities was directly responsible for crime and punishment). ↑
- Christopher Lewis & Adaner Usmani, The Injustice of Under-Policing in America, 2 Am. J.L. & Equal. 85, 95 (2022). This is despite their acknowledgement that “[c]oncentrated disadvantage is the root cause of most serious crime.” Id. at 88. ↑
- Id. at 97. But see David Greaves, Eddie Ellis: Prison Reform Visionary, Our Time Press (Aug. 5, 2019), https://ourtimepress.com/eddie-ellis-prison-reform-visionary [https://perma.cc/2XS8-JTE7] (interviewing Think Tank member Eddie Ellis, who argued that the criminal system should aim to address social and economic problems which will lead to less people going to prison and less need for prisons). ↑
- Lewis & Usmani, supra note 156, at 95. The authors speculate, “we guess that a world of more policing would be one of less police violence.” Id. at 101 (first emphasis added). ↑
- Alec Karakatsanis, A Warning to Journalists About Elite Academia, Alec’s Copaganda Newsl. (Nov. 3, 2022), https://equalityalec.substack.com/p/a-warning-to-journalists-about-elite [https://perma.cc/MAL4-AX9G] (arguing that the professors “knew available data [undermining their claim that the United States is under-policed] may refute the headline-grabbing point . . . and they chose to hide it”). ↑
- Id. ↑
- See Levin, supra note 32, at 2814 (“[E]ven if we all agreed on data about crime rates, the frequency of police stops for different racial groups, or the likelihood of recidivism, that wouldn’t tell us how to respond or what to do with that data.”); Collins, supra note 125, at 462 (noting that data cannot speak). ↑
- Sociologist Jessica Simes has argued that “[t]aking the Think Tank’s insights seriously means going wherever the data takes us.” Simes, supra note 131, at 46. Simes examined prison admissions data in Massachusetts from 2009 to 2017 to see “whether the concentrations the[] [Think Tank] observed in the 1980s and 1990s persist today.” Id. at 46–47. Simes found that the correlation between poverty, disinvestment, segregation, and incarceration rates remained significant, but the “spatial distribution of high incarceration rates” has stretched beyond major urban cities to small cities and suburbs, which are experiencing concentrated disadvantage, resource deprivation, and severe economic decline. Id. at 49–50, 54. ↑
- Burton, supra note 133, at 152–53; see also Levin, supra note 32, at 2782– 83, 2786 (contrasting expertise rooted in lived experience with more traditional notions of expertise). ↑
- Burton, supra note 133, at 152. ↑
- Saifee, supra note 133. ↑
- Id. ↑
- Saifee, supra note 131, at 66–68. ↑
- Id. at 68. ↑
- See, e.g., Eugene Volokh, Supreme Court Declines to Reconsider Constitutionality of Convictions by Non-Unanimous Juries, Volokh Conspiracy (Jan. 10, 2011, at 11:38 AM), https://volokh.com/2011/01/10/supreme-court-declines-to-reconsider-constitutionality-of-convictions-by-non-unanimous-juries [https://perma.cc/M882-KPSK] (describing a law professor’s unsuccessful petition for certiorari challenging Oregon’s nonunanimous jury rule). ↑
- Emily Bazelon, Shadow of a Doubt, N.Y. Times Mag., https://www.nytimes.com/interactive/2020/01/15/magazine/split-jurors.html [https://perma.cc/7RA9-66VA] (last updated Jan. 17, 2020); see also Saifee, supra note 131, at 68 and accompanying notes (describing the “deeply fractured set of opinions” in Apodaca v. Oregon, 406 U.S. 404 (1972), in which five justices found that the Sixth Amendment did not require unanimous verdicts in state criminal trials). ↑
- Adam Liptak, A Relentless Jailhouse Lawyer Propels a Case to the Supreme Court, N.Y. Times (Aug. 5, 2019), https://www.nytimes.com/2019/08/05/us/politics/supreme-court-nonunanimous-juries.html [https://perma.cc/4Y8D-KG66]. ↑
- See infra note 187 and accompanying text; cf. Beckett, supra note 151, at 176 (noting that “the proliferation of long and life sentences, which are mainly imposed in cases involving violence, is a fundamental driver of mass incarceration”). ↑
- Saifee, supra note 131, at 68. ↑
- Bazelon, supra note 170 (quoting then-capital defense lawyer Ben Cohen); Saifee, supra note 131, at 67–69. ↑
- Saifee, supra note 131, at 68–69 and accompanying notes. ↑
- Id. at 69 n.91; Telephone Interview with Colin Reingold, Dir. of Strategic Crim. Lit., Promise of Just. Initiative, former Lit. Dir. & Senior Couns., Orleans Pub. Defs. (Jan. 6, 2022) (noting that before this point, not many public defenders were raising the jury issue “as a matter of course”). ↑
- Saifee, supra note 131, at 69 n.92. ↑
- Id. at 69–70 and accompanying notes. ↑
- Official Journal of the Proceedings of the Constitutional Convention of the State of Louisiana 374 (H.J. Hearsey 1898). ↑
- Saifee, supra note 131, at 70 (quoting State v. Webb, 2013-0146, p. 45 (La. App. 4 Cir. 1/30/14), 133 So. 3d 258, 286). ↑
- Telephone Interview with John Simerman, Rep., New Orleans Advoc. (Dec. 17, 2021). ↑
- Jeff Adelson, Gordon Russell & John Simerman, How an Abnormal Louisiana Law Deprives, Discriminates and Drives Incarceration: Tilting the Scales, Advocate (Apr. 1, 2018), https://www.theadvocate.com/baton_rouge/news/courts/how-an-abnormal-louisiana-law-deprives-discriminates-and-drives-incarceration-tilting-the-scales/article_16fd0ece-32b1-11e8-8770-33eca2a325de.html [https://perma.cc/SL8X-WFAK] (“The Advocate reviewed about 3,000 felony trials over six years, turning up 993 convictions rendered by 12-member Louisiana juries in which the newspaper was able to document the jury votes.”). ↑
- Id.; John Simerman, U.S. Supreme Court Refuses to Make Louisiana Ban on Non-Unanimous Juries Retroactive, NOLA.com (May 17, 2021), https://www.nola.com/news/courts/article_40f11aa4-a8dd-11eb-ae3e-dfa9c5d97cc6.html [https://perma.cc/U7EL-YCTS]. ↑
- Adelson, Russell & Simerman, supra note 182; Simerman, supra note 183. ↑
- Adelson, Russell & Simerman, supra note 182. ↑
- “[W]ithout [the reporting] . . . it would have been impossible to be successful, not just with the legislators but in getting the public to vote for it.” Chris Granger, The Advocate Wins First Pulitzer Prize for Series that Helped Change Louisiana’s Law, Advocate (Apr. 15, 2019), https://www.theadvocate.com/baton_rouge/news/the-advocate-wins-first-pulitzer-prize-for-series-that-helped-change-louisianas-split-jury-law/article_dba87282-5f28-11e9-92b3-bfba0cf08ab2.html [https://perma.cc/SAY3-XA57] (quoting Louisiana state senator JP Morrell, sponsor of the bill). ↑
- Russell et al., supra note 5 (noting that the jury rule gave prosecutors an advantage in plea negotiations, leading accused people to “weigh[] a guilty plea and a hefty prison term against the tall odds of convincing at least three jurors that [the state] got it wrong”); see also Granger, supra note 186; Saifee, supra note 131, at 75–76 (noting that the jury rule “incentivized prosecutors to charge more serious crimes than the evidence warranted—crimes that carried more severe penalties—resulting in ‘more people serving more time in prison’”) (quoting Russell et al., supra note 5); John Simerman, For Prosecutors, Louisiana’s Split-Verdict Law Produces Results, NOLA.com (Apr. 21, 2018), https://www.nola.com/news/courts/article_e737f0e7-7d8a-5fc7-84bf-22f33277ea89.html [https://perma.cc/M29R-QCZZ] (quoting a former prosecutor in New Orleans who admitted to filing more severe felony charges than the evidence could support to ensure that jury unanimity would not be required); Lea Skene, Louisiana’s Life Without Parole Sentencing the Nation’s Highest – and Some Say That Should Change, Advocate (Dec. 7, 2019), https://www.theadvocate.com/baton_rouge/news/article_f6309822-17ac-11ea-8750-f7d212aa28f8.html [https://perma.cc/JP3H-M2FA] (stating that Louisiana has the highest percentage of people serving life without parole in the nation); New Report: 80% of People Still Imprisoned Due to Jim Crow Jury Verdicts Are Black, Most Are Serving Life Sentences, Promise of Just. Initiative (Nov. 18, 2020), https://promiseofjustice.org/news/2020/11/18/new-promise-of-justice-initiative-report-80-of-people-still-imprisoned-due-to-jim-crow-jury-verdicts-are-black-most-are-serving-life-sentences [https://perma.cc/Z77P-CUXJ] (stating that of the people who remain in Louisiana prisons from split verdict convictions, 80 percent are Black and most are serving life sentences without the possibility of parole, and that the jury rule “helped make Louisiana the state with the highest incarceration rate[] and the most wrongful convictions per capita in the Deep South”); Adelson, Russell & Simerman, supra note 182 (“On average . . . [Louisiana] sends one person to prison every five days on the word of a divided jury. About half of them face sentences so stiff they likely will die in prison.”). ↑
- Telephone Interview with John Simerman, supra note 181 (“We got involved because judges kept saying we don’t have enough data to show [the jury rule’s] lasting [racial] effect today.”); see also Adelson, Russell & Simerman, supra note 182 (referencing a rare hearing in 2017 where a New Orleans trial judge denied an equal protection challenge to the split-jury rule in the absence of “a full-scale study” that “shows disproportionate impact”). ↑
- Telephone Interview with John Simerman, supra note 181. ↑
- Id. To be sure, the data collection was a tall order. “Parishes and judges var[ied] widely in how and whether they record[ed] juror votes.” Saifee, supra note 131, at 71 n.106 (citing articles stating that no Louisiana court consistently and comprehensively collected jury voting data; Louisiana juries were often not polled and, when they were, judges usually sealed or tore up the results; and even the total vote count was absent from many trial records). ↑
- Telephone Interview with John Simerman, supra note 181; Telephone Interview with Colin Reingold, supra note 176 (stating that the court opinions and the rare court hearing were the result of Duncan building up the issue in the state courts and educating the defense bar). ↑
- Saifee, supra note 131, at 72–73 and accompanying notes (citing Kimberlé Williams Crenshaw, Race to the Bottom: How the Post-Racial Revolution Became a Whitewash, Baffler (June 2017), https://thebaffler.com/salvos/race-to-bottom-crenshaw [https://perma.cc/7K2Y-HK8Y]). ↑
- Ramos v. Louisiana, 586 U.S. 1221 (2019), cert. granted; Telephone Interview with G. Benjamin Cohen, Chief of Appeals, Orleans Parish Dist. Att’y’s Office (Oct. 11, 2021) (describing the twenty-three certiorari petitions as the “tip of the iceberg”). ↑
- Ramos v. Louisiana, 590 U.S. 83, 93, 107 (2020). ↑
- Edwards v. Vannoy, 593 U.S. 255, 262 (2021). ↑
- Saifee, supra note 131, at 73 & n.125. ↑
- Watkins v. Ackley, 523 P.3d 86, 103 (Or. 2022); see also Jason Breslow, The Supreme Court Outlawed Split Juries, but Hundreds Remain in Prison Anyway, NPR (May 14, 2023, at 6:00 AM), https://www.npr.org/2023/05/14/1175226037/supreme-court-ramos-louisiana-split-juries-oregon [https://perma.cc/X88L-NT9P]; Non-Unanimous Jury Verdicts: The Ramos and Watkins Decisions, Or. Pub. Def. Comm’n: Gen. Info. & Res., https://www.oregon.gov/opdc/general/pages/non-unanimous.aspx [https://perma.cc/5V6T-N6JW]. ↑
- Saifee, supra note 131, at 73–74 and accompanying notes (describing Duncan’s work and the long-term consequences Ramos carries to reduce the time people charged with serious felonies in Louisiana and Oregon spend in prison); see also Breslow, supra note 197 (“For hundreds of Oregon inmates, the [Oregon Supreme Court] decision means prosecutors must now decide whether to pursue a new trial, cut a plea deal or dismiss charges altogether.”). ↑
- Bhatt et al., supra note 117, at 6–10. ↑
- Id. at 3–4, 10. ↑
- Id. at 10, A-14; see also id. at 8 (noting that shootings in Chicago, as in most cities, are “extremely concentrated” in a handful of neighborhoods and among a small number of people in those neighborhoods). ↑
- Id. at 3, A-15–A-16. ↑
- Id. at 3, 5, 12, A-18. ↑
- Id. at 3-4, 12 (explaining that due to limitations in the third pathway, the study focused on differences between the first two pathways); see also id. at A‑19– A‑20. ↑
- Id. at 4. “Among all men randomized to READI offers, 55% started the program (defined as attending orientation),” a higher take-up rate than interventions involving much less disconnected populations. Id. at 29; see also Patrick Smith, Anti-Violence Programs Are Working. But Can They Make a Dent in Chicago’s Gun Violence?, ChicagoPublicMedia: WBEZ Chi. (Nov. 1, 2021, at 5:00 AM), https://www.wbez.org/criminal-justice/2021/11/01/chicago-anti-violence-efforts-succeed-but-shootings-rise [https://perma.cc/E3QX-SPNV] (describing 55 percent of those who were offered READI showing up as “incredible” considering the men “ha[d] been disappointed so many times in their lives by different social systems”) (quoting Cornell University Professor Max Kapustin). ↑
- Bhatt et al., supra note 117, at 4. ↑
- Id. at 5, 19 n.19. ↑
- Id. at 1. ↑
- Id. at 5 (tracking three measures of serious violence involvement over a twenty-month period: shooting and homicide victimizations; shooting and homicide arrests; and other serious violent-crime arrests, such as robbery and aggravated battery). ↑
- Id. at 6. ↑
- Id. at 36. ↑
- Id. at 3, 9, 13–16 (discussing the significant challenges to identifying people at high risk of future gun violence, engaging them to participate, and locating them as many are disconnected from social institutions, like schools or formal employment, or may be unhoused); Smith, supra note 205 (stating that the only public institution many in the target population had any sustained connection with was the criminal legal system (quoting Cornell University Professor Max Kapustin)). ↑
- Video Interview with Max Kapustin, Professor of Econ., Cornell Univ., Jeb E. Brooks Sch. of Pub. Pol’y (June 15, 2023). ↑
- Bhatt et al., supra note 117, at 28, 49. “Prior to program referral, 35% of men in the study had been shot and 98% had been arrested, with an average of more than 17 prior arrests.” Id. at 4. ↑
- Id. at 50. ↑
- Id. at 3, 11–12. ↑
- Starr, supra note 84, at 828, 848; see infra notes 236–237. ↑
- Bhatt et al., supra note 117, at 18. READI’s algorithm was trained to predict “whether someone would be either arrested for, or the victim of, a violent crime involving a gun during the next 18 months.” Id. at A-16. The phrase “violent crime involving a gun” included homicide, assault, battery, or robbery with a firearm. Id. “[N]ew [prediction] models were trained 10 times over the referral period using the most up-to-date records available from [the Chicago Police Department].” Id. at A‑15. ↑
- Id. at A-18. ↑
- Id. ↑
- Id. at 6; accord id. at A-18, A-49. ↑
- Id. at 6–7. The researchers did not anticipate the effects would be greater for people in the outreach pathway, so this analysis is exploratory. Video Interview with Max Kapustin, supra note 213; Bhatt et al., supra note 117, at A-43 (noting that differences in impacts by pathway were unexpected). ↑
- Bhatt et al., supra note 117, at 43 n.34; Video Interview with Max Kapustin, Professor of Econ., Cornell Univ., Jeb E. Brooks Sch. of Pub. Pol’y (June 21, 2023). ↑
- Bhatt et al., supra note 117, at 6, 42–43, A-50; Video Interview with Max Kapustin, supra note 223. ↑
- Bhatt et al., supra note 117, at 50. The large declines in serious violence were concentrated among the smaller group of outreach referrals—25 percent—with above-median predicted risk. Email from Max Kapustin to author (July 3, 2023, at 9:40 PM) (on file with author); see Bhatt et al., supra note 117, at 1. “Yet there do not appear to be parallel declines among the above-median predicted risk algorithm referrals.” Bhatt et al., supra note 117, at 41. ↑
- Bhatt et al., supra note 117, at 43; see also id. at 42 fig. III. ↑
- Id. at 43; Video Interview with Max Kapustin, supra note 223. ↑
- Bhatt et al., supra note 117, at 43–45. Outreach referrals had much higher take-up rates than algorithm referrals, but they engaged in programming about as much as algorithm referrals after starting. Id. at 29, 30 tbl. 3, 44; see id. at 42 fig. III (depicting this outcome graphically); Video Interview with Max Kapustin, supra note 223 (stating that if outreach referrals were more “ready,” one would expect them to be more ready to start and to stick with it, but researchers were only seeing the former). “Taking up”—or the start of formal participation in READI—is defined as attending the first day of READI orientation. Bhatt et al., supra note 117, at 14, 30; see also id. at 41–44 (noting that outreach referrals who worked the most hours did not experience the biggest declines). Also, “the incapacitation effect of READI activities,” namely “keeping people busy during the workday,” did not appear to explain the treatment effects. Id. at 35, A-28, A-52 (noting that “the decline in arrests for shootings and homicides are driven by declines in weekend incidents”). ↑
- Bhatt et al., supra note 117, at A-49. ↑
- Id. at 43 (“[O]utreach referrals typically had higher realized risk than algorithm referrals at similar predicted risk levels.”); Video Interview with Max Kapustin, supra note 223; Email from Max Kapustin, supra note 225. ↑
- Bhatt et al., supra note 117, at 43. ↑
- Id. at 43–44. ↑
- Id. at 5–6, 45. ↑
- Video Interview with Max Kapustin, supra note 223; Bhatt, et al., supra note 117, at 28 (noting that the findings suggest that the pathways identified “different kinds of people”); id. at 28 n.25 (“[R]eferrals from one pathway rarely included people who had previously been referred via another pathway . . . .”); id. at 44 (noting some evidence that “the unobservables correlated with pathway . . . matter for treatment heterogeneity”). ↑
- I thank Paul Heaton for this point about rules of thumb. ↑
- Bhatt et al., supra note 117, at A-16 (noting that READI’s statistical model used over 1,400 features from police records to predict future gun violence involvement); see also Mayson, supra note 83, at 2251 (“[W]hat prediction does is identify patterns in past data and offer them as projections about future events.”); id. at 2270 (“All prediction presumes that we can read the past with enough reliability to make useful projections about the future.”); Jessica M. Eaglin, Racializing Algorithms, 111 Calif. L. Rev. 753, 761 (2023) (“Algorithms rely upon statistical analyses of large historical datasets consisting of observations about the past behaviors of people involved in the criminal legal process.”). ↑
- See Starr, supra note 84, at 827–28, 848 (arguing that economists often argue on efficiency grounds that “if a decisionmaker lacks detailed information about an individual, relying on group-based averages . . . will produce better decisions in the aggregate”); Kelly Hannah-Moffat, Actuarial Sentencing: An “Unsettled” Proposition, 30 Just. Q. 270, 278 (2013) (original emphasis omitted) (explaining that a high risk score means that an individual “shares characteristics with an aggregate group of high-risk offenders,” not that the individual is “a high risk offender”); Eaglin, supra note 13, at 85 (stating that risk-assessment algorithms produce a quantitative outcome that suggests the likelihood that the target event will occur with individuals sharing those same characteristics). ↑
- Qualitative research from interviews and field observations provided anecdotal insight into outreach workers’ selection, screening, and recruitment process. Bhatt et al., supra note 117, at A-47–A-52 (discussing outreach workers’ use of inclusion and exclusion criteria); id. at A-18–A-19, A-49 (noting that outreach teams regularly met to discuss interested candidates before deciding whether to nominate them for randomization); Video Interview with Max Kapustin, supra note 223 (stating that the outreach organizations would send a list of names to the research team for random assignment). ↑
- Bhatt et al., supra note 117, at 6. ↑
- Id. at 50. ↑
- Video Interview with Max Kapustin, supra note 223. ↑
- I thank Heidi Liu for highlighting this point. ↑
- Bhatt et al., supra note 117, at 34; Video Interview with Max Kapustin, supra note 223. ↑
- Malik Neal & Cal Barnett-Mayotte, Court Watch Observations Show Shortcomings in Krasner’s Promise to End Cash Bail, Phila. Bail Fund (May 8, 2019), https://www.phillybailfund.org/da-report [https://perma.cc/LW5X-9UWS] (describing the gap between Krasner’s rhetoric and “what happens daily in the basement of the Criminal Justice Center” (quoting Malik Neal, cofounder of the Philadelphia Bail Fund)); Bryce Covert, Progressive Philly D.A. Larry Krasner’s Bail Reform Plans Seem Stalled, Advocates Say, Appeal (June 25, 2019), https://theappeal.org/progressive-philly-d-a-larry-krasners-bail-reform-plans-seem-stalled-advocates-say [https://perma.cc/QXA3-F3SX]. Krasner is often described as a “progressive prosecutor.” Jennifer Gonnerman, Larry Krasner’s Campaign To End Mass Incarceration, New Yorker (Oct. 22, 2018), https://www.newyorker.com/magazine/2018/10/29/larry-krasners-campaign-to-end-mass-incarceration [https://perma.cc/4TFR-6C39]; Carissa Byrne Hessick, Pitfalls of Progressive Prosecution, 50 Fordham Urb. L.J. 973, 973, 978 (2023) (noting the lack of consensus on “what, precisely, it means to be a progressive prosecutor” but observing that the term encompasses those who “r[a]n on . . . decarceral platforms”). ↑
- Jocelyn Simonson, Radical Acts of Justice: How Ordinary People Are Dismantling Mass Incarceration 57–60 (2023). Court watch volunteers often sit in solidarity with people impacted by the criminal system and may include people who themselves have been directly harmed by the system. Id. at 69–74, 84. ↑
- Id. at 62–65, 80; Philadelphia Bail Watch, Phila. Bail Fund (2018), https://www.phillybailfund.org/bailwatch [https://perma.cc/R6JK-JF3B]. ↑
- Simonson, supra note 245, at 54, 65–66. I use this example because courtrooms are places where people are most affected by policing and prosecution, and the court watchers observe decision-making from a position of sousveillance. See Steve Mann & Joseph Ferenbok, New Media and the Power Politics of Sousveillance in a Surveillance-Dominated World, 11 Surveillance & Soc’y 18, 26 (2013) (“The practice of viewing from below when coupled with political action becomes a balancing force that helps—in democratic societies—move the overall ‘state’ toward a kind of veillance (monitoring) equilibrium . . . .”); Robert M. Cover, Violence and the Word, 95 Yale L.J. 1601, 1607 (1986) (discussing the relationship between legal interpretation and the pain and violence it produces “even in the most routine of legal acts”). ↑
- Malik Neal & Cal Barnett-Mayotte, Phila. Bail Fund, Observations of 125 Recent Bail Requests 3 (May 8, 2019), https://static1.squarespace.com/static/591a4fd51b10e32fb50fbc73/t/5cd369f5b3f45700013a01f3/1557359094458/DAO+Bail+Request+Report+5.19.pdf [https://perma.cc/SJ45-E22W]; Simonson, supra note 245, at 80. ↑
- Neal & Barnett-Mayotte, supra note 248, at 4 (charting this occurrence in thirty out of forty-four cases during the period of observation). In several cases the magistrate described the prosecutor’s bail request as “punitive” or “ridiculous.” Id. at *1 tbl. 1. The bail fund does not track the underlying criminal charges. Covert, supra note 244. ↑
- Neal & Barnett-Mayotte, supra note 244 (quoting Malik Neal, co-founder of the Philadelphia Bail Fund). ↑
- Neal & Barnett-Mayotte, supra note 248, at 4. ↑
- Id. ↑
- Crespo, supra note 2, at 2066, 2069; see also Simonson, supra note 245, at 84 (“Courtwatchers sit in solidarity with the accused, but in a collective, systemic way as they amass information about cases in the aggregate.”). ↑
- Covert, supra note 244 (noting that the bail fund’s report “caused a good internal discussion” in the district attorney’s office (quoting a representative of the district attorney’s office)). ↑
- Malik Neal & Christina Matthias, Broken Promises: Larry Krasner and the Continuation of Pretrial Punishment in Philadelphia, 16 Stan. J. C.R. & C.L. 543, 551, 557 (2021); see also Simonson, supra note 245, at 78 (describing court watchers as producing a community “counternarrative” to traditional accounts of the criminal system). ↑
- Jordan Levy, How Is Your Neighborhood Affected by Cash Bail? The Philly Bail Fund Plans To Hand You the Hard Data, Billy Penn (Aug. 11, 2022), https://billypenn.com/2022/08/11/philadelphia-bail-fund-online-tool-data-salih-israil [https://perma.cc/T8VH-D4TC]. ↑
- Phila. Bail Fund, Ransom and Freedom: Ending Cash Bail in Philly 3 (Aug. 27, 2022), https://www.phillybailfund.org/ransom-and-freedom-ending-cash-bail-in-philadelphia [https://perma.cc/R7SV-VQJS]. ↑
- Id. at 2. The neighborhoods that make up those zip codes are among the poorest in the city, with a federal poverty rate of between 30 to 41 percent. Id. at 5. ↑
- Id. at 2–3, 8, 10–11; Simonson, supra note 245, at 127–29. ↑
- Simonson, supra note 245, at 129. ↑
- See, e.g., Phila. Bail Fund, Rhetoric vs. Reality: The Unacceptable Use of Cash Bail by the Philadelphia District Attorney’s Office During the COVID-19 Pandemic 4 (July 2020), https://www.phillybailfund.org/s/PBF_RhetoricvsReality_072920-8c9t.pdf [https://perma.cc/T9TL-PX5X] (analyzing 451 bail hearings over two months in 2020 after Krasner pledged to limit pretrial incarceration during the coronavirus pandemic); Chris Palmer, Tensions Are Boiling Over Between Philly DA Larry Krasner and Bail Reform Advocates, Phila. Inquirer (July 29, 2020), https://www.inquirer.com/news/philadelphia/philadelphia-da-larry-krasner-cash-bail-reform-advocates-20200729.html [https://perma.cc/U4QF-DGMN]. ↑
- I borrow the phrase “everyday criminal adjudication” from Jocelyn Simonson. See Jocelyn Simonson, The Place of “the People” in Criminal Procedure, 119 Colum. L. Rev. 249, 252, 270 (2019) (discussing bottom-up interventions into “everyday criminal adjudication” such as court watching and community bail funds). ↑
- See, e.g., Okidegbe, supra note 19, at 2032, 2057–58 (calling for algorithm developers to draw on community knowledge sources to consider the risks that pretrial incarceration poses to the accused, their families, and their communities); M. Eve Hanan, Invisible Prisons, 54 U.C. Davis L. Rev. 1185, 1191, 1223 (2020) (arguing that sentencing policy must be informed by the voices of incarcerated people to “understand[] the qualitative experience of imprisonment”). ↑
- See Seema Kakade, Environmental Evidence, 94 U. Colo. L. Rev. 757, 783, 800 (2023) (discussing “community evidence” as a source of knowledge in environmental law); see also Okidegbe, supra note 19, at 2032, 2054 (explaining that pretrial algorithms are built exclusively on carceral data and exclude mitigating data that is accessible to communities). ↑
- See, e.g., Amna A. Akbar, Toward a Radical Imagination of Law, 93 N.Y.U. L. Rev. 405, 412 (2018); Amna A. Akbar, Sameer M. Ashar & Jocelyn Simonson, Movement Law, 73 Stan. L. Rev. 821, 864, 849 (2021); Allegra M. McLeod, Envisioning Abolition Democracy, 132 Harv. L. Rev. 1613, 1623–37 (2019); Simonson, supra note 262, at 298–99; Simonson, supra note 9, at 783. ↑
- See, e.g., Jocelyn Simonson, Copwatching, 104 Calif. L. Rev. 391, 409 (2016) (describing local residents coming together to record police-citizen interactions); Jocelyn Simonson, The Criminal Court Audience in a Post-Trial World, 127 Harv. L. Rev. 2173, 2193–94 (2014) (conceptualizing the power shifts when audience members collectively observe routine criminal proceedings); Jocelyn Simonson, Bail Nullification, 115 Mich. L. Rev. 585, 585 (2017) (arguing that community bail funds, where members of the public post bail for strangers, shift popular and constitutional understandings about money bail); see also Andrew Manuel Crespo, No Justice, No Pleas: Subverting Mass Incarceration Through Defendant Collective Action, 90 Fordham L. Rev. 1999, 1999 (2022) (considering plea unions, where people facing prosecution join together in a strike to “bring the penal system to a halt”). ↑
- See discussion supra pp. 245–47. ↑
- See discussion supra pp. 251–55. ↑
- See discussion supra pp. 245–48, 250–51. ↑
- I thank Jane Manners for drawing this distinction. ↑
- Google, “Crowdsourcing”, 6,370,000 results (Nov. 30, 2025), https://www.google.com/search?q=crowdsourcing [https://perma.cc/W865-7H3D]. ↑
- An example of crowdsourcing occurred during the campaign to enact a reparations ordinance in Chicago in the aftermath of the decades of torture that Chicago Police Department Commander Jon Burge and his ring of white detectives perpetrated on Black people. A multiracial and intergenerational grassroots coalition embarked on a “different conceptualization of the injustice of police conduct,” McLeod, supra note 265, at 1625, challenging themselves to imagine a “holistic package of relief” that criminal and civil remedies were “incapable of providing.” Joey L. Mogul, The Struggle for Reparations in the Burge Torture Cases: The Grassroots Struggle That Could, 21 Loy. Pub. Int. L. Rep. 209, 209, 224 (2015). The Chicago Torture Justice Memorials project, a group of activists, survivors, artists, educators, and attorneys, put out an open call to the community to submit proposals to memorialize the police torture, leading to broader ideas for possible redress. Id. at 219–20, 223–24 (describing the landmark reparations ordinance); McLeod, supra note 265, at 1625; 2012 Speculative Proposals, Chi. Torture Just. Mem’ls, https://chicagotorture.org/memorial-proposals [https://perma.cc/XFF4-B9GJ]. ↑
- Capers, supra note 19, at 1246 (stating that acts of racialized police violence are “talked about in barbershops and hair salons, on church pews and on street corners, and yes, in prisons”). Capers adopts the term “pools of knowledge” from Russell Robinson. Robinson, supra note 19, at 1120 (“In general, black and white people obtain information through different informational networks, which results in racialized pools of knowledge.”); see also Maines, supra note 19, at 323–24 (1999) (explaining that information pools are often race-based, where Black people are aware of certain information that is largely unknown to white people). ↑
- Okidegbe, supra note 19, at 2054–55, 2062 (2022) (describing the “patterns that emerge” from listening sessions, panels, surveys, and interviews conducted by community groups and the “common threads” in individual accounts, such as the harms of parental incarceration). ↑
- Cf. J. Patrick Williams, Emergent Themes, in 2 The SAGE Encyclopedia of Qualitative Research Methods 248, 248 (Lisa M. Given ed., SAGE Publ’ns Inc. 2008) (“Themes emerge from the close analysis of any data source, including fieldnotes, ethnographic and reflective memos, interview transcripts, and various print, visual, and digital media.”). ↑
- Id. at 249 (noting critiques by constructivists “that theoretical bias is inevitable”); Barney G. Glaser & Anselm L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research 2–3 (1967) (“The basic theme in our book is the discovery of theory from data systematically obtained from social research.”); Kathy Charmaz, Grounded Theory: Objectivist and Constructivist Methods, in 2 The Handbook of Qualitative Research 509, 510–12 (N.K. Denzin & Y.S. Lincoln eds., SAGE Publ’ns Inc. 2000) (explaining that grounded theory was developed by Glaser and Strauss, who challenged “assumptions that qualitative research could produce only descriptive case studies rather than theory development”); Bhatt et al., supra note 117, at A-43 (noting that the use of emergent themes can “reveal unexpected insights and perspectives that may have been missed with a more structured or predefined approach to analysis”). ↑
- Garland, supra note 1, at 2. ↑
- Western & Muller, supra note 1, at 168. ↑
- Akbar, Ashar & Simonson, supra note 265, at 830. ↑
- Levin, supra note 32, at 2782 (noting that both camps adopt a “shared appeal to the language of experts and expertise”). ↑
- Rachel Elise Barkow, Prisoners of Politics: Breaking the Cycle of Mass Incarceration 165 (2019) (“We need to establish expert agencies charged with instituting and evaluating criminal justice policies so that we get better outcomes . . . .”); Rappaport, supra note 26, at 810–11 (endorsing “an evidence-based approach to criminal justice problem-solving” that relies on social science evidence); Brandon L. Garrett, Evidence-Informed Criminal Justice, 86 Geo. Wash. L. Rev. 1490, 1493 (“Evidence-informed practices refer to a family of approaches that have brought greater use of data and science into the criminal justice system.”). ↑
- Simonson, supra note 9, at 850–52; Akbar, supra note 265, at 425; Bell, supra note 19, at 208–10. ↑
- Barkow, supra note 281, at 5, 10–11, 15; Levin, supra note 32, at 2781, 2804; Collins, supra note 125, at 413–15. A number of scholars have argued that politicians in the United States tend to set criminal policy by catering to ill-informed voters who are driven by emotions and “gut reactions” after hearing stories, anecdotes, and “panics” about crime. Barkow, supra note 281, at 1, 5; e.g., Levin, supra note 32, at 2780 (noting that “criminal legal literature generally has adopted a story [that] mass incarceration [w]as caused by popular punitive impulses” but that “scholars disagree widely” about this); Naomi Murakawa, The First Civil Right: How Liberals Built Prison America 15 (2014) (tracing how liberals played a central role in building the carceral state by focusing on formal procedural reforms to avoid addressing structural inequalities); Ruth Wilson Gilmore, Golden Gulag: Prisons, Surplus, Crisis, and Opposition in Globalizing California 26–29 (2007) (analyzing the geographic, economic, and political incentives that led to mass incarceration); Alice Ristroph, An Intellectual History of Mass Incarceration, 60 B.C. L. Rev. 1949, 1992 (2019) (“Scholars have offered several different explanations for mass incarceration . . . .”). The notion that criminal policy is set based on emotion and intuition makes the turn to data and empirical research appear reasonable at first glance. Collins, supra note 125, at 411, 414 (arguing that “in this rush to change course,” proponents of the evidence-based model have overlooked its harms). ↑
- See Collins, supra note 125, at 403, 406–07, 424 (describing the evidence-based model as “criminal law’s leading paradigm for reform”). ↑
- Id. at 405; see also Barkow, supra note 281, at 167 (“[One] premise behind this recommendation for expert oversight is that empirically valuable information on criminal law can lead to better decisions.”); Levin, supra note 32, at 2811 (deconstructing the technocratic logic that “social science and apolitical decision-making could lead more efficiently to objectively good policy”). ↑
- Evan Selinger & Robert P. Crease, Introduction, in The Philosophy of Expertise 3 (Evan Selinger & Robert P. Crease eds., Columbia Univ. Press 2006) (“[T]he authority so conferred on experts . . . risks elitism, ideology, and partisanship sneaking in under the guise of value-neutral expertise.”); see also, e.g., K. Sabeel Rahman, Democracy Against Domination 100–01 (2017) (describing “the political and moral dimensions of expert judgment”); The Politics of Numbers, supra note 125, at 3 (arguing that “what to measure, how to measure it . . . and how to present and interpret the results” are political choices); Kimani Paul-Emile, Foreword: Critical Race Theory and Empirical Methods Conference, 83 Fordham L. Rev. 2953, 2956 (2015) (arguing that social science claims to “objectivity” and “neutrality” in knowledge production “mask hierarchies of power that often cleave along racial lines”); Lvovsky, supra note 34, at 493–94 (noting that critics “deride the notion of the ‘objective’ expert as an anti-democratic myth, an attempt to sell the people a dictatorship under the guise of technocratic neutrality”). ↑
- Collins, supra note 125, at 403, 450 (arguing that the evidence-based model has a political agenda informed by neoliberalism where economic efficiency is the primary metric of success); see, e.g., Barkow, supra note 281, at 6 (discussing the importance of engaging experts “to make sure we are making the right calls to maximize public safety and are spending our limited resources most effectively”). ↑
- Collins, supra note 125, at 409, 431–35 (noting that the evidence-based model measures the public safety impact of a particular reform through recidivism); see also Marie Gottschalk, Caught: The Prison State and the Lockdown of American Politics 101 (2015) (describing recidivism reduction as “a leading penal policy goal and indeed the preeminent yardstick by which to judge the success or failure of justice reinvestment and other penal reforms”); Robert Weisberg, Meanings and Measures of Recidivism, 87 S. Cal. L. Rev. 785, 785–88 (2014) (discussing the contested definition and potential measures of recidivism); Marie Gottschalk, Did You Really Think Trump Was Going To Help End the Carceral State?, Jacobin (Mar. 9, 2019), https://jacobin.com/2019/03/first-step-act-criminal-justice-reform [https://perma.cc/F7MC-7FJ6] (arguing that recidivism is a “misleading gauge of public safety”); Dana Goldstein, The Misleading Math of ‘Recidivism,’ Marshall Project (Dec. 4, 2014, at 11:15 AM), https://www.themarshallproject.org/2014/12/04/the-misleading-math-of-recidivism [https://perma.cc/7L5R-HFAV] (showing the ways in which the definition of recidivism can mislead the public). ↑
- Collins, supra note 125, at 429–31 (arguing that the evidence-based model reflects a “scarcity mindset” that prioritizes fiscal savings and efficient decisions over larger structural questions). ↑
- See Barkow, supra note 281, at 185; Collins, supra note 125, at 430 (“This endless search for data to guide us towards optimal performance deflects structural questions and criticisms.”). ↑
- Collins, supra note 125, at 414, 442 (“The implication of this empirical mindset is clear: we cannot know whether something is true unless it has been proven scientifically . . . .”); Barkow, supra note 281, at 175 (“Any agency responsible for criminal justice policy [should be] require[d] . . . to base its decisions on empirical evidence about what will best promote public safety at the lowest cost.”); Meghan Guevara & Enver Solomon, Crime & Just. Inst. & Nat’l Inst. Of Corr., Implementing Evidence-Based Policy and Practice in Community Corrections, at ix (2d ed. 2009) (“[E]vidence-based practice focuses on approaches demonstrated to be effective through empirical research rather than through anecdote or professional experience alone.”). ↑
- James Austin et al., Reinvigorating Justice Reinvestment, 29 Fed. Sent. Rep. 6, 12 (2016) (“The only investments that can qualify under this portfolio would be programs of other criminal justice system agencies.”); id. (noting a strong literature on the long-term social and economic benefits of community investments but stating that evidence-based reform is restricted to “proven” strategies with immediate crime reduction). ↑
- See Rappaport, supra note 26, at 811 (arguing that proposals to reduce imprisonment “are the stuff of experts and bureaucrats” and “are best justified using social science evidence,” and “there’s no reason to think laypeople are necessary—or even helpful—in achieving these outcomes”); Barkow, supra note 281, at 166 (arguing that “criminologists or social scientists who study these issues on a regular basis” have a comparative advantage over prosecutors to guide decisions about public safety using data and empirical research); Collins, supra note 125, at 410 (“The decision to valorize the evidence-based methodology is itself a choice to privilege quantitative scientific inquiry over other ways of knowing.”). Rappaport has noted that evidence-based reform “is not necessarily antagonistic toward lay participation or community-based solutions. Its posture is contingent and skeptical, in a scientific sense. If reliable evidence shows these solutions to work, great—run with them.” Rappaport, supra note 26, at 812. This argument takes funding imbalances for granted. See infra Section III.B.; see also Collins, supra note 125, at 441 n.175 (“Rappaport’s reform vision allows space for solutions to criminal justice problems that are generated by communities most impacted by the system—subject to empirical validation.”). ↑
- Collins, supra note 125, at 442. Collins has explained that the evidence‑based model devalues alternate ways of knowing that are not readily susceptible to quantification or measurement. Id. at 410, 428, 438–40, 459 (stating that the evidence-based model excludes lived experience from its definition of evidence). “We must value the insights of people who are most impacted by criminal legal policies as evidence of the policies’ impact—regardless of whether their observations and experiences have been ‘validated’ by a controlled trial or quasi-experimental study.” Id. at 459; see also Gottschalk, supra note 288, at 261 (arguing that the evidence-based model “contributes to a denigration of other kinds of knowing and evidence that are not the result of controlled experiments, including policy studies and qualitative work”). ↑
- Collins, supra note 125, at 410–11, 438–40, 459–60; Okidegbe, supra note 19, at 2054–55. ↑
- Collins, supra note 125, at 411, 459; accord Okidegbe, supra note 19, at 2055, 2058–59. ↑
- Matsuda, supra note 9, at 324–25. Matsuda encouraged critical legal scholars to study and support the struggles of people of color who have experienced subordination. Id. at 324–25, 349. Matsuda has argued that the perspective of “grass roots philosophers who are uniquely able to relate theory to the concrete experience of oppression . . . can lead to concepts of law radically different from those generated at the top.” Id. at 325–26. ↑
- Akbar, supra note 265, at 405–06, 457–59, 473 (arguing that radical social movements offer a deeper set of critiques than liberal law reform projects); Simonson, supra note 262, at 256–57, 270, 286–87, 294–95 (arguing that facilitating communal interventions on behalf of the accused can open up an “alternative vision of criminal procedure”); McLeod, supra note 265, at 1615–19 (describing abolitionist conceptions of justice); cf. Matsuda, supra note 9, at 362, 373 (discussing reparations as “a legal concept generated from the bottom”). ↑
- Akbar, supra note 265, at 476; see also Monica C. Bell, Safety, Friendship, and Dreams, 54 Harv. C.R.-C.L. L. Rev. 703, 710 (2019) (“The legal scholar’s impulse is to say: Enough description. We know the problem. How are we going to fix it? But ‘we’ do not have a rich understanding of ‘the problem.’”); Simonson, supra note 9, at 853, 860 (arguing that directly impacted people “might also seek data and information from less traditional sources”); McLeod, supra note 265, at 1623–28 (describing an abolitionist initiative to respond to state violence). ↑
- See, e.g., Akbar, supra note 265, at 425–26; Simonson, supra note 262, at 298–99; Gottschalk, supra note 288, at 282; Allegra M. McLeod, Beyond the Carceral State, 95 Tex. L. Rev. 651, 705 (2017) (reviewing Gottschalk, supra note 288); Dorothy Roberts, Democratizing Criminal Law as an Abolitionist Project, 111 Nw. U. L. Rev. 1597, 1605, 1607 (2017); Jonathan Simon, Racing Abnormality, Normalizing Race: The Origins of America’s Peculiar Carceral State and Its Prospects for Democratic Transformation Today, 111 Nw. U. L. Rev. 1625, 1650 (2017). ↑
- See, e.g., Rappaport, supra note 26, 810–11 (arguing that “there’s no reason to think laypeople are necessary—or even helpful” to reducing incarceration); Trevor George Gardner, By Any Means, A Philosophical Frame for Rulemaking Reform in Criminal Law, 130 Yale L.J. F. 798, 805 (2021) (responding to Simonson, supra note 9) (“It would be a categorical mistake to equate the pursuit of an equitable process of crime policymaking—even as it relates to race-class subordinated communities—with the pursuit of equitable crime policy.”); see also Levin, supra note 32, at 2826–28 (questioning whether shifting power to the bottom will serve decarceral ends). ↑
- Rappaport, supra note 26, at 811. ↑
- E.g., Simonson, supra note 9, at 853; Monica C. Bell, Katherine Beckett & Forrest Stuart, Investing in Alternatives: Three Logics of Criminal System Replacement, 11 U.C. Irvine L. Rev. 1291, 1326 (2021); Collins, supra note 125, at 410; Okidegbe, supra note 19, at 2052–53; Levin, supra note 32, at 2782, 2786. ↑
- E.g., Wright et al., supra note 110, at 1438. ↑
- E.g., Rachel López, Participatory Law Scholarship, 123 Colum. L. Rev. 1795, 1818, 1824 (2023). ↑
- Simonson, supra note 9, at 856. ↑
- Okidegbe, supra note 19, at 2062. The discounting of lived experience as a form of expertise is not limited to the criminal law context. Cf. S. Lisa Washington, Pathology Logics, 117 Nw. U. L. Rev. 1523, 1578–81 (2023) (discussing the family regulation system’s disregard of parents’ expertise in their children’s needs); Leah M. Litman, Melissa Murray & Katherine Shaw, A Podcast of One’s Own, 28 Mich. J. Gender & L. 51, 66 (2021) (“Too often, when minorities and women advert to their own lived experiences . . . their commentary is viewed as relying unduly on anecdote and narrative, as opposed to real expertise.”). ↑
- Cf. Austin et al., supra note 292, at 12 (noting that restricting funding to “proven” strategies with speedy crime reduction outcomes excludes virtually all community-level investments); McLeod, supra note 300, at 657–58 (arguing that even limited initiatives can serve as an opening toward more transformative ends). ↑
- E.g., Crespo, supra note 2, at 2105–08 (arguing that courts can partner with data experts in universities or government agencies to aggregate facts across cases or create positions for empiricists in the court system); Abel, supra note 2, at 939– 40, 993 (noting that merely compiling a list of prior cases handled by a discredited officer would impose no additional costs on the prosecution, and although investigating each case on the merits is expensive, “that is not sufficient justification for ignoring the systemic effects of an officer’s misconduct”); id. at 1008 (“There are costs, to be sure, in tracking an officer’s cases. But there are also costs in letting . . . [police misconduct] go undetected.”); Garrett, supra note 2, at 423–24 (discussing the costs to the accused and the public from the failure to aggregate). ↑
- Austin et al., supra note 292, at 12; see also Bhatt et al., supra note 117, at 3 n.3, 8. Particularly, Arnold Ventures, one of the largest philanthropies in the United States, aims to identify “what works” based on “what we already know from research.” See Evidence and Evaluation, Arnold Ventures: Our Work, https://www.arnoldventures.org/work/evidence-evaluation [https://perma.cc/4SSH-9M82]. READI Chicago, for example, was funded primarily through private philanthropy, including Arnold Ventures. Bhatt et al., supra note 117, at 47 n.39. ↑
- See Collins, supra note 125, at 417–18 (internal citations omitted) (explaining that the National Institute of Justice embraces a “strict empiricist approach, defining a program as evidence-based only when randomized controlled trials conducted at three different sites have demonstrated its efficacy” while “flexible empiricists” rely on “quasi-experimental” methods or “‘findings from empirically sound social science research’ . . . to identify the most effective reforms”). ↑
- Lauren Johnson et al., Reclaiming Safety: Participatory Research, Community Perspectives, and Possibilities for Transformation, 18 Stan. J. C.R. & C.L. 191, 194, 198–99 (2022) (describing community-based participatory research, a methodology that “prioritizes the needs, questions, strategies, and expertise of communities”); see also Collins, supra note 125, at 454–55 (noting that this method “center[s] the needs and desires of those most impacted . . . . [and] positions members of these [historically marginalized] communities as agents, not subjects, of research”); Monica C. Bell, Next Generation Policing Research: Three Propositions, J. Econ. Persps., Fall 2021, at 29, 37, 41 (suggesting that social scientists focus their research on evaluating community-based strategies for violence reduction while cautioning that “some norms of ‘evidence-based policymaking,’ which base normative decisions about good policy on clear, countable results, [may be] in some ways out of step with creative efforts to ‘reimagine’ public safety”). ↑
- Two core criticisms are that lived experience is opinion-based and that lay people can hold punitive opinions. Rappaport, supra note 26, at 759–66, 791 (citing studies that lay people can be punitive and arguing that “while public opinion is certainly less punitive today than it was three decades ago . . . it remains quite harsh”); Gardner, supra note 301, at 807 (“[W]hat value is equitable process to the project of criminal-justice reform if the punitive crime politics seemingly endemic to American culture permeate marginal communities?”); see also Stan Orchowsky, Just. Rsch. & Stat. Ass’n, An Introduction to Evidence-Based Practices 8–9 (2014) (“In particular, opinions, testimonials, and anecdotes are not evidence of effectiveness in and of themselves.”). But see Rachel E. Barkow & Mark Osler, Designed to Fail: The President’s Deference to the Department of Justice in Advancing Criminal Justice Reform, 59 Wm. & Mary L. Rev. 387, 459 (2017) (calling for a presidential criminal justice advisory commission that includes formerly incarcerated people who can speak to their experiences). ↑
- See Crespo, supra note 2, at 2112–14 (noting concerns that ideological commitments may drive some judges to aggregate facts across cases in a manner that “exacerbates existing criminal justice pathologies” but explaining that such concerns should not be overstated); see also Levin, supra note 32, at 2811 (arguing that elite experts “have been key players in constructing the carceral state”). See generally supra notes 88, 120–121 (citing scholarship on how algorithms perpetuate racial inequities). ↑
- Whether decarceral ideas need to be grounded in data is an important normative question that this Article does not address. Some scholars have argued that our current moment does not demand more data. See Benjamin, supra note 88, at 78 (arguing that “the hunt for more and more data is a barrier to acting on what we already know”). Erin Collins has argued that the “endless search for data to guide us towards optimal performance deflects structural questions and criticisms.” Collins, supra note 125, at 430. Collins does not reject data; she calls for “let[ting] go of the mandate to test and measure” and “redefin[ing] what evidence means—what data ‘count.’” Id. at 411, 449, 459. Evidence-based proponents also recognize “limits to what the expertise model can accomplish.” Barkow, supra note 281, at 177 (“Some policies might be the humane and just thing to do, even if they cannot be supported by data and studies . . . . That would require cultural shifts beyond the scope of an institutional fix.”). ↑
- See Video Interview with Max Kapustin, supra note 223 (noting this question coming up repeatedly after the READI randomized controlled trial was completed). ↑
- See generally Simonson, supra note 9 (examining the social movement focus on power shifting in police reform); K. Sabeel Rahman & Jocelyn Simonson, The Institutional Design of Community Control, 108 Calif. L. Rev. 679, 730 (2020) (discussing the importance of power-shifting in institutional design); Matthew Clair & Amanda Woog, Courts and the Abolition Movement, 110 Calif. L. Rev. 1, 35 (2022) (discussing bottom-up work that has created “material power shifts”); Marbre Stahly-Butts & Amna A. Akbar, Reforms for Radicals? An Abolitionist Framework, 68 UCLA L. Rev. 1544, 1560 (2022) (“In a system plagued by profound power differentials between dominant and dominated classes, radical reforms cannot be top down: they must be bottom up.”). ↑