A Comparative Simulation Study of the Fairness and Accuracy of Predictive Policing Systems in Baltimore City

Abstract

There are ongoing discussions about predictive policing systems being unfair, for example, by exhibiting racial bias. Law enforcement in some cities, such as Los Angeles, California, and Baltimore, Maryland, have initiated the integration of these systems into their decision-making processes, and some of these systems were advertised as being unbiased. However, later studies discovered that these methods could also be unfair due to feedback loops and being trained on historically biased recorded data. Comparative studies on predictive policing systems are few and insufficiently comprehensive. Crucially, the relative fairness of predictive policing methods with regard to traditional hot spot-based policing has not been established. Moreover, the relationship between fairness and accuracy is complex and requires further study. Furthermore, the case of Baltimore City, Maryland, USA, has not yet been systematically analyzed despite its relevance as an early adopter of predictive policing technologies with a fraught history of social justice concerns around policing. An improved understanding of these questions could better inform policy decisions around predictive policing technologies both in Baltimore and beyond. Therefore, in this work we perform a comprehensive comparative simulation study on the fairness and accuracy of predictive policing technologies in Baltimore. Our results suggest that the situation around bias in predictive policing is more complex than previously assumed. While we find that predictive policing exhibits bias due to feedback loops, as previously reported, we also find traditional hot spot-based policing to have similar issues. Although predictive policing is found to be more fair and accurate than hot spot policing in the short term, it also amplifies bias more quickly, suggesting the potential for worse long-run behavior. In Baltimore, the bias in these systems tended toward over-policing White neighborhoods in some cases, unlike in previous studies. However, when the analysis was restricted to some specific crime types, this tendency differed. Overall, this work demonstrates a methodology for city-specific evaluation and compares behavioral tendencies of predictive policing systems, showing how such simulations can reveal inequities and long-term tendencies. We recommend that authorities and community stakeholders use simulation methodologies to assist in collaboratively navigating the complexities around fairness in predictive policing.

1. Introduction

Advancements in AI have led to the development of predictive policing systems with the potential for higher accuracy in predicting crime [1]. Predictive policing has been shown to provide improvements over simpler prior approaches [2], referred to as hot spots or hotspot policing [3,4], a strategy that uses data and simple statistical models to find areas with high crime rates and allocates officers accordingly. This strategy is based on the assumption that crime distribution is affected by environmental factors such as a neighborhood not being patrolled enough or presenting profitable burglary targets, as well as by successful crime attempts on the part of the criminals, that is, there is a higher chance of crime happening where it was been previously observed [5]. These simple models have evolved into more sophisticated predictive policing models that can take into account more factors, including changes in crime distribution over time, allowing them to predict future crime distributions [6]. These predictive models promise higher accuracy in identifying future crime locations [4,7,8].

If police resources are allocated based on these more accurate predictions, one might also expect that the resulting crime distribution would better align with the underlying crime patterns, and consequently produce more proportionate allocations across neighborhoods. However, theoretical and simulation studies have suggested that feedback loops between police allocations and recorded crimes may amplify racial and regional disparities over time [9,10,11]. These competing perspectives suggest that predictive policing may promise higher accuracy while simultaneously posing risks for the development of long-term bias. Understanding how these dynamics unfold remains an important question. Most existing studies have examined these dynamics in relatively simplified settings, often focusing on a single model, limited fairness metrics (either racial or regional), or synthetic data [10,11,12]. The relative fairness and accuracy of predictive policing methods compared to traditional hot spot policing approaches remain insufficiently understood in realistic urban contexts. In particular, prior work has not examined how different policing algorithms compare when evaluated simultaneously across multiple fairness metrics and over extended operational periods. Existing studies typically evaluate a single algorithm or focus on a small number of performance indicators, and none have had the goal of demonstrating a comparative, localized, and multi-metric simulation example. To address these gaps, we propose a comparative simulation framework for evaluating policing systems that jointly analyzes fairness and accuracy across multiple metrics over time using real-world data. Our comparative study provides bases upon which these models’ long-term behavioral tendencies can be evaluated against one another using real crime data. Our results interpretation provides insight about the complications of using real crime data and how the observations can be explained.

Along with comparing the performance of predictive policing and hot spots policing, we investigate the long-term potential of using these methods in Baltimore City, Maryland, USA. Baltimore provides an interesting and important context for several reasons:

• Its history of reports of violence [13,14,15].
• The evident geographical racial divide around the city [16,17].
• The animosity that arose among police and citizens around 2015 after the case of Freddie Carlos Gray Jr. [18,19].
• Its use of predictive policing models law in enforcement, starting around 2018 [20,21].

Neighborhoods in Baltimore remain largely segregated. This is not merely an after-effect of historical segregation policies and resulting “White flight”, which kept the Black majority in inner Baltimore while driving White residents toward outer suburbs [22]. Rather, according to the “New Jim Crow” theory [23], mass incarceration in Black neighborhoods functions as a modern way of maintaining segregation by ensuring that people of color reside in the low-income part of the city [23]. This segregated status of neighborhoods reinforces racial and economic disparities among the neighborhoods, which in turn makes it increasingly vital to address the fairness of predictive policing algorithms.

Further complicating fairness evaluations is the difference in the effect of police presence in Black neighborhoods vs. White neighborhoods. In her book “The New Jim Crow,” Alexander, points out that policing in Black neighborhoods is militarized to the extent that Black youth started referring to police presence in their community as “The Occupation”. Alexander mentions that the aggressive tactics used in Black neighborhoods are not even newsworthy, while the same tactics would be “political suicide” in White urban neighborhoods. According to Alexander, the reason for this is the lower political influence of the Black community [23].

These policing tactics affect the psychological and social atmosphere in these neighborhoods. In White communities, police presence is often perceived to bring safety and order. In Black neighborhoods, however, it causes fear, anxiety, and instability—conditions that can escalate tension and end up in aggression by both police forces and residents. Therefore, the same presence can have varied effects in different communities [23]. If policing decisions made using the results of predictive models do not consider these dynamics, they might reinforce harm rather than reduce crime. Pre-evaluation of these systems has uncovered tendencies towards concentration of policing, which could help to promote better-informed police decision-making.

The death of Freddie Carlos Gray Jr., a 25 year-old Black man, was caused by injuries sustained while in police custody and led to riots and city-wide protests, heightening the tension between the police and the Black community [15]. The protests were rooted in the over-policing of Black neighborhoods by heavy surveillance and wrongful arrests, as well as under-protection with delayed and insufficient police response to calls for service [24,25]. In the wake of these protests, there was a spike in crime rates, especially in homicides and non-fatal shootings, in Baltimore [26,27]. Two reasons have been reported as probable causes of this spike:

1.

Police pullback: The police department underwent a month-long retreat in response to the protests [26]. There was a drop in the number of arrests [27].

2.

Opportunists and emboldened criminals: A public survey by the PEW Research Center showed that about 60% believed the main cause of the crime rise was that some people took advantage of the situation [28]. During the unrest and subsequent police pullback, opportunists looted approximately 30 pharmacies and drug clinics, stealing substantial amounts of narcotics, which some believed caused a rise in crimes as well [29,30,31].

The grassroots movements following the death of Freddie Gray and the mass arrest of protesters [32] resulted in the establishment of several community-led organizations advocating for police reform and racial justice, namely, the Baltimore Action Legal Team (BALT) [33], the No Boundaries Coalition Community [34], and Leaders of a Beautiful Struggle (LBS) [35]. These local organizations, with the help of activists, lawyers, and community members, are trying for reform by providing legal assistance, public education, and policy advocacy for Black residents affected by biased policing practices. Following the goals of this grassroots movements, our study extends their vision by addressing the technological aspect of policing reform.

The segregated nature of Baltimore’s neighborhoods along with its history of aggressive law enforcement tactics may have taken root in Baltimore’s crime records, on which predictive policing algorithms are trained. Baltimore’s law enforcement has been using predictive policing tools without prior independent evaluations and analysis or estimating potential long-term performance tendencies in Baltimore’s specific context, which is a problem that needs to be addressed.

With this study, our contributions are:

1.

A simulation study of the fairness of predictive policing in the context of Baltimore/

2.

A comparative analysis of hot spot policing and predictive policing.

3.

Introducing a logical basis on which predictive policing models can be compared.

4.

Visualization of results with suitable graphics, promoting the comprehensibility of our comparisons and the interpretability of related findings.

5.

Last and most importantly, demonstrating the necessity of localized evaluation of policing systems prior to real-world implementation and providing a method with which to achieve this.

In our 300 day-simulation period, predictive policing developed less bias on average in most scenarios compared to hot spot policing while maintaining higher accuracy; however, this could change if the simulation were run for longer because of predictive policing’s higher rate of bias development. Contrary to the prevailing wisdom, the feedback loop in Baltimore trends toward assigning a higher average share of officers to White neighborhoods, as determined for Kernel Density Estimation (KDE) and PredPol when trained on all crime records from 2018-2019.

2. Related Work

Since the emergence of predictive policing systems, research in this field has focused on four main areas: accuracy improvement [2,8,36], fairness enhancements [9,10,12], comparative studies [37,38], and contextualized implementation studies [9,11,12,39,40], with some works contributing to more than one aspect.

2.1. Accuracy Improvement

Advancements in predictive policing have grown with improved data collection methods and more sophisticated techniques. Earlier predictive systems in this field relied on hot spot mapping methods such as Kernel Density Estimation (KDE) [41,42] to predict crime densities [43]. Temporal-aware crime mapping claimed accuracy improvement over non-aware static mapping by taking into account the time distance between prior crimes and the prediction target time [4]. By accounting for the time and space dependency of crime distribution, the proposal of spatiotemporal KDE for improved hot spots policing set the foundation for more enhanced predictive models [8]. The Epidemic-Type Aftershock Sequence (ETAS) crime forecasting model, which makes use of the Expectation Maximization (EM) algorithm, showed improved accuracy in assigning crime risk scores to different geographical areas [2,36]. This ETAS-based framework was adopted and operationalized by the PredPol predictive policing system to generate hot spot forecasts [44].

More recent years have seen the use of textual data for crime analysis and prediction using data from social media [45,46]. Vivek and Prathap [45] used crime-related tweet counts and compared the predictions of crime counts by LSTM, ARIMA, and SARIMA. They showed that ARIMA performed best at predicting crime counts while LSTM performed worst, probably due to limited training data. They also showed that crime tweet counts fairly matched real-world crime counts and could be used as a proxy for real-world crime data in certain crime categories. Similarly, S. Tam et al. [46] combined twitter sentiment analysis with historical crime data and proposed the ConvBiLSTM model, which integrates CNN and BiLSTM networks. They fused processed textual and non-textual data, which they fed into a Bidirectional LSTM. The resulting model predicted whether a tweet could be indicative of a crime with about 97% accuracy.

Reinforcement learning has contributed to the enhancement of police allocation logics [39,40,47,48,49,50]. Barbosa et al. proposed a model-free RL approach for optimizing patrol agent positioning. The model learns from digital “pheromones” left over from previous events. After repositioning an agent, the model is rewarded if an agent is present where a crime is happening and penalized if a crime happens where there is no patrol [40]. Joe et al. [47] used deep reinforcement learning and temporal difference learning for patrol and incident response. Their model learns dispatch and rescheduling jointly with two main objectives: maximizing successful incident response and minimizing patrol schedule disruption. Another work on patrolling improvement used IDQN and deep reinforcement learning to learn dynamic patrol routes by maximizing patrol coverage and prioritizing high-risk areas. The city is represented by a graph, with locations as nodes and roads as edges. Each road has two attributes: crime risk and distance. Patrolling a high-risk road is rewarded, while revisiting a road diminishes the reward [48,49]. Multi-Agent Reinforcement Learning (MARL) has also been used to optimize patrolling and dispatching by a joint policy. The model is rewarded if the decision results in faster response and better coverage, and is penalized if a decision results in delays or gaps in patrols [39].

The aforementioned studies did not address fairness; other studies that have evaluated fairness are discussed in the following subsection.

2.2. Fairness Studies

In 2016, the unfairness tendencies of predictive policing models were brought to light when a simulation study by Lum and Isaac [9] on drug-related crime in Oakland, California demonstrated how police would frequent minority and low-income neighborhoods when these models were applied. This motivated more studies focusing on fairness analysis of these systems.

Brantingham ran a study explaining how bias enters into data when an officer downgrades or upgrades a crime based on personal bias [51]. In this study, simulations were run after downgrading crime counts by removing a percentage of crimes from the actual dataset and after upgrading counts by adding an additional percentage of crimes, using setting of 2, 5, 10, 15, and 20 percent. They compared model parameters and demonstrated that for downgrading, the bias needs to be substantial in order to have a noticeable impact on the model parameters. By “noticeable impact” they meant that the estimated parameter values were more than one standard deviation from what they would have been if there was no downgrading. When upgrading crime counts, the impact became noticeable at a lower percentage (10 percent) than when downgrading [51]. In 2018, Ensign et al. mathematically analyzed how this bias could happen using a Polya-urn model in four different scenarios involving allocating a single officer to one of two neighborhoods:

1.

High-crime-rate neighborhood vs. low-crime-rate, training only on discovered crimes.

2.

The same, except this time training only on reported crimes.

3.

Two neighborhoods with high crime rates, training only on discovered crimes.

4.

The same neighborhoods, now trained only on reported crimes [10].

The authors proposed several remedies that could improve that specific model [10]. The same year, a study by Brantingham and Mohler et al. investigated evidence of bias in a randomized controlled trial of the ETAS model, which their results suggested was nonexistent [52]. However, their bias measurement was different from what is measured in the fairness of policing distribution when using predictive policing algorithms in simulations [5,11,52]. They did not look at police concentration after long-term use of the predictive policing model or at the effect of the feedback loop. Their bias detection focused on detecting significant racial differences in arrests between a control (day 20 hot spots chosen by human analysts) and treatment days (day 20 hot spots chosen by the predictive policing algorithm) [52]. In another paper, Mohler introduced a new version of the ETAS model with a penalized likelihood method that incorporated demographic parity, demonstrating give-and-take between accuracy and fairness [12].

Akpinar et al. contributed to the discussion of fairness by demonstrating that even when using victim-reported data instead of arrest data (i.e., discovered crime), the models would show bias because crimes are under-reported in some areas compared to others [11]. They pointed out that the unfair rankings were data-driven and not model-specific. They used a synthetic crime dataset generated by the Self-Exciting Point Process model (SEPP) over Bogota, Colombia. They thinned the generated dataset by the victimization rate of each district in Bogota to form a synthetic true crime dataset, then thinned it further using the victim report rate of each district to create the reported crime dataset. A simulation was run once using the Moving Average (MAVG) model and once using the SEPP model. In both cases, it was shown that fewer hot spots were predicted in low-reporting districts regardless of the actual crime rate. While this study demonstrates the inherent risk of data-driven bias even across different model types, it did not simulate police deployment or detection feedback, nor did it compare models in terms of predictive performance. Our work builds on this insight by using real-world crime data, modeling officer deployment and detection, and examining how retraining based on observed and reported crime influences the fairness and accuracy of hot spot-based policing and predictive policing over time [11].

In 2022, Chapman et al. argued that the cause of bias lies in the theoretical assumptions behind model development [5]. They asserted that criminological theories are categorized into two beliefs:

1.

Neo-classical: Crime is the product of rational choice, and police presence is preventive. Because a successful burglary can motivate another one nearby, there is a relation among crimes within space and time.

2.

Positivist: Crime is based in genetics, social setting, and biology, rather than by choice; thus, its distribution is mostly random.

To show that bias arises from the model, Chapman et al. created three sets of synthetic crime data: one with a uniform and random distribution, the second mostly uniform with two hot spot regions added, and the third consisting of real-world data. They attempted to show that simulating PredPol over each of these datasets caused clearer hotspots to form after 48 days of simulation. Police were distributed among the ten top-ranked regions.

We note that even if the model had ranked them the same, only ten regions would be allocated police. From then on, hot spots would appear. Whether this demonstrates bias was caused by the model or bias due to limited policing resources is unclear. The authors also tried to remove the effects of feedback in the simulation by using a random crime dataset. They observed that no hot spots were formed, countering their assumption that the model was the cause of bias. In this experiment, the authors changed the data fed into the model, not the model itself. They concluded that bias is model-driven, unlike Akpinar [5,11]. In 2023, Mashiat et al. presented an extended abstract proposing the use of a counterfactual causal model to assure fairness. They compared their PropFair model with Ensign’s PolyaUrn model based on police allocation between two neighborhoods with constant crime rates of 0.3 and 0.7, claiming that their model better matched the fairness line by allocating about 0.45 of police to beat 1 and the rest to beat 2 [37]. There have also been fairness studies on some investigative tools [38] and criminal recidivism prediction systems [53,54,55]. Here, we focus on fairness studies concerning crime prediction and patrolling-related tools.

2.3. Comparative Analysis

There have been numerous works comparing the accuracy of two models with noticeable differences in their underlying logics [4,8,40,46,47,48,50]. Among these, several proposed new models for incident response or route-patrolling scheduling [40,47,48,50]. Others have proposed crime prediction models for helping with policing decision-making [2,4,8,46]. To show the enhancement in accuracy provided by their proposed framework, these studies compare a proposed model to other models or consider specific features by comparison to the same base model without optimization.

Other studies have focused on comparing an unbiased version of an algorithm to its original version in terms of fairness [10,12,39]. Ensign theoretically compared PredPol with a modified version of it, with the comparison focusing on allocating an officer between two regions [10]. Mohler [12] compared a Hawkes process-based model to a version of it with a penalized likelihood, while Repaskey studied fairness in the officer-response time of a Multi-Agent Reinforcement Learning (MARL) model and an optimized model by rewarding even patrol coverage [39].

There are also some existing works comparing both the fairness and accuracy of two different algorithms with different levels of complexity. In an extended abstract, Mashiat et al. presented a theoretical fairness and accuracy comparison between PropFair, their proposed counterfactual causal model, and Ensign’s PolyaUrn model, using the same two-region police allocation scenario as in Ensign’s 2018 study [37]. Griffard et al. also studied fairness and accuracy, finding a pattern between newly police-reported crime and previous cases by looking at the rate at which patterns were found in minority neighborhoods versus others. They compared Patternizr, an investigative tool deployed by the NYPD, with baseline models such as gradient-boosting decision trees [38].

2.4. Contextualized Studies

There have been a number of studies analyzing the performance and impact of predictive policing systems in certain regions, some through simulation and others through real-world experiments. The simulation studies have investigated model behavior in cities such as Oakland, Indianapolis, Atlanta, Denver, and Bogota [9,11,12,39,40]. Real-world performance analyses have been conducted in Los Angeles, Kent (UK), and Memphis [7,52,56]. Baltimore has not been systematically studied before, although the city is currently making more use of AI tools in their law enforcement than previously [57,58]. We believe it is imperative to be aware of the unfairness tendencies of such models specific to the context prior to real-world application, since any city policies can further impact the demographic distribution and be a cause of economic, social, and racial divide, as seen in the past [22]. Therefore, one of the aims of this study is to analyze model performance in Baltimore, an important deployed real-world context which is also representative of areas with a history of racial segregation.

As these studies show, fairness distortions in predictive policing systems can emerge from both modeling assumptions and biased data inputs. Yet, few have examined how these biases evolve over time in real-world deployment settings, another gap we aim to address through simulation in the Baltimore context. For more information on the body of work in this field, see previous literature reviews by Raji et al. [59] and Mandalapu et al. [60].

3. Materials and Methods

In this section, we describe the datasets used and some basic terminology that facilitates our explanation of the simulated scenarios, followed by the details of those simulations.

3.1. Data Description

We use the crime and neighborhood dataset available on the Open Baltimore website [61,62,63] (data source: https://data.baltimorecity.gov/datasets/, accessed on: 11 March 2023, 23 August 2022, 15 March 2023, respectively). As the Open Baltimore website periodically updates these datasets and its directory structure may change, we also maintain archived copies to ensure reproducibility (downloaded version: https://github.com/saminsemsar/Data_Analysis_Portfolio/tree/main/PredictivePolicing/Data, accessed on 1 April 2026). The crime dataset is a spatiotemporal dataset which includes information such as the date and time of the crime, latitude, longitude, neighborhood, and crime description. For the study, crime locations outside residential areas were excluded. After performing data preprocessing, a total of 207,447 crime records that occurred in Baltimore City in the years 2019–2020 were utilized. Crime records were sorted chronologically by their timestamp. Note that these crime records are only a subset of the incidents that occurred in that period, as they are the only ones documented and known to law enforcement, and this is likely to impact the behavior and potential bias in predictive policing algorithms [9].

3.2. Definitions

In this paper, several key terms will be used, which we have defined below.

• Black neighborhoods: We define a neighborhood as Black if the number of Black residents in that neighborhood exceeds the number of residents of any other racial group. For example, if a neighborhood has 21 Black residents, 20 White residents, and 10 Latino residents, it is flagged as a Black neighborhood. This majority-based definition allows us to consistently categorize neighborhoods by their predominant racial composition for fairness analysis.
• White neighborhoods: Neighborhoods with majority White residents.
• Neither Black nor White neighborhoods: Neighborhoods where the majority of the residents are neither Black nor White. These areas are populated primarily by groups including the Latinx community, Alaskans, and other racial or ethnic minorities.
• Noisy-OR: We use a Noisy-OR function to model the probability of an effect Y given binary causes $X_1,\dots ,X_n$. Here, $X_i$ indicates whether cause i is present or not. Each cause $X_i$ is associated with a failure probability $q_i$. The Noisy-OR formulation assumes conditional independence and is defined as follows:

  $$P(Y=1\mid X_1,X_2,\dots ,X_n)=1-{\displaystyle \prod _{i=1}^n}{q}_i^{X_i}.$$

  (1)
  We incorporate the Noisy-OR probabilistic model to introduce uncertainty into crime detection. Noisy-OR models the independent contribution of multiple officers in detecting a crime by increasing the likelihood of detection as the number of officers within the detection radius increase. Then, an event (e.g., crime detection) is determined as having occurred if any of several coin flips come up as heads, i.e., if a random probability is higher than the Noisy-OR calculated probability, that crime is marked as “detected”.
• Detected crimes: The algorithm determines whether a crime is detected by applying the Noisy-OR model to the probability of detection (p), taking into account the number of police officers within the neighborhood k and the probability of an officer detecting a crime, which is a hyperparameter $p=0.5$. This detection probability parameter (p) is selected as a neutral baseline in the absence of reliable empirical estimates of an officer’s detection rate. Choosing $p=0.5$ allows us to model detection uncertainty while avoiding strong assumptions that could bias results toward specific outcomes.

  $$P\left(\mathrm{Crime}\mathrm{Detected}\right)=1-{(1-p)}^k$$

  (2)
  A crime is labeled as “detected” based on a coin flip with the probability in Equation (2).
• Reported crimes: We create a report dataset by flipping a coin with 0.4 probability to decide whether or not each crime would be reported Equation (4). The value of 0.4 was calculated using a weighted average based on the number of each type of crime in our dataset and the probability of reporting that crime based on the 2019 report of the Bureau of Justice Statistics (BJS) [64]. We weight each crime category’s count by its national reporting probability from the BJS. The estimate is computed as follows:

  $$\mathrm{Average}\mathrm{Report}\mathrm{Probability}\mathrm{of}\mathrm{Crime}={\displaystyle \frac{{\displaystyle \sum _{i\in \mathcal{CT}}}{N}_i·r_i}{{\displaystyle \sum _{i\in \mathcal{CT}}}{N}_i}},\mathrm{in}\mathrm{our}\mathrm{case}\simeq 0.4.$$

  (3)

  $$\mathrm{Crime}\mathrm{is}\mathrm{Reported}\mathrm{if}(0.4>\mathrm{rand})$$

  (4)

  where $N_i$ is the number of crimes of type i that occurred after 2019 in Baltimore and $r_i$ is the national reporting rate of crime type i (see Figure 1).
  To clarify the concepts of reported and detected crimes, consider a simple example. Suppose that ten crimes actually happen in a neighborhood, but only four are reported to the police due to under-reporting. A predictive system trained on historical data will only observe these four reported incidents, not the full set of ten true crimes. In our simulation, “detected” crimes refers to the subset of true crimes that become visible to the system through policing activity, which may partially overlap with reported crimes.
• KDE: Kernel Density Estimation (KDE) is a statistical technique used to estimate the underlying Probability Density Function (PDF) of a set of data points. It involves creating a smooth and continuous function by placing a kernel (a predefined shape, such as a Gaussian) on each data point and then summing them. The resulting estimated density function provides insights into the distribution and intensity of the data across the entire range. We implemented Kernel Density Estimation using the KernelDensity class from the scikit-learn Python library (version: 1.1.3) [65], which provides a computationally efficient implementation of the density estimation method originally introduced by Rosenblatt [42] and Parzen [41].
• Short-term KDE: KDE which receives short-term crime history. In our simulations, we set this short term to equal a month’s worth of crime history.
• Long-term KDE: KDE which receives a longer history of crime compared to short-term KDE. In our simulations, we set it equal to one year’s worth of crime history to make it comparable to PredPol in terms of the data they receive.
• PredPol: PredPol is a self-exciting point process model for crime prediction introduced by Mohler et al. [2,7]. The model is based on the Epidemic-Type Aftershock Sequence (ETAS) framework originally developed in seismology, in which each past event increases the short-term likelihood of nearby future events. In the context of crime forecasting, this formulation captures near-repeat victimization by modeling crime intensity as a combination of a background rate and a self-exciting component. We implemented the simplified formulation explained in Mohler’s 2015 work [7].

3.3. Methods

A simulation-based framework was developed to investigate how policing algorithms perform over time with respect to fairness and accuracy. Real crime data and neighborhood data from Baltimore were applied in the simulation, including neighborhood identifiers, timestamps, and racial demographic indicators. The goal was to observe and quantify the temporal dynamics of bias and accuracy in policing models. For this purpose, three variables were calculated in each scenario setting during the course of the simulation for each day and for each neighborhood:

1.

Number of real crimes in the Baltimore city dataset.

2.

Number of police officers assigned.

3.

Number of crimes detected.

The simulation was run 20 times in each scenario setting; therefore, the number of police assigned and number of crimes detected are both averaged over 20 runs.

The simulations involved running PredPol, short-term KDE, and long-term KDE over the eight distinct settings summarized in 

Table 1. The eight simulation scenario settings described in the study.

Scenario	Crime Input	Number of Officers	Report Setting
S1	AGG. ASSAULT	40	Detected only ($r=0$)
S2	AGG. ASSAULT	40	Detected + reported ($r=0.4$)
S3	AGG. ASSAULT	400	Detected only ($r=0$)
S4	AGG. ASSAULT	400	Detected + reported ($r=0.4$)
S5	TOTAL	40	Detected only ($r=0$)
S6	TOTAL	40	Detected + reported ($r=0.4$)
S7	TOTAL	400	Detected only ($r=0$)
S8	TOTAL	400	Detected + reported ($r=0.4$)

. Our implementation approximates hot spot policing using a spatial kernel density estimation model (short-term and long-term KDE) applied to crimes observed within a rolling historical window. While KDE is a spatial smoothing technique, the simulation introduces temporal dynamics by updating the input crime data sequentially over time. This approach reflects how hot spot policing is commonly implemented operationally through analysts who generate hotspot maps based on recent crime observations. Let t denote the current simulation day and ${\tau }_i$ the occurrence time of crime i. We define the set of crimes available for prediction at day t as

$${\mathcal{H}}_t^{\left(W\right)}=\{x_i=(la{t}_i,lo{n}_i):t-W\le {\tau }_i<t\},$$

(5)

where $(la{t}_i,lo{n}_i)$ represents the spatial coordinates of crime i in terms of latitude and longitude and W is the historical window length. In our simulations, $W=$ one month for short-term KDE and $W=$ one year for long-term KDE.

Using the historical events, the KDE model estimates a spatial crime intensity surface

$${\widehat{f}}_t^{\mathrm{KDE}}\left(x\right)={\displaystyle \frac{1}{|{\mathcal{H}}_t^{\left(W\right)}|}}\sum _{x_i\in {\mathcal{H}}_t^{\left(W\right)}}{K}_h(x-x_i),$$

(6)

where $x=(\mathrm{lat},\mathrm{lon})$ and $K_h\left(\right)$ is a Gaussian kernel with bandwidth h.

For a Gaussian kernel in two dimensions,

$$K_h(x-x_i)={\displaystyle \frac{1}{2\pi h^2}}exp\left(-{\displaystyle \frac{\parallel x-x_i{\parallel }^2}{2h^2}}\right).$$

(7)

Unlike hot spot-based policing systems, the PredPol model that represents predictive policing in our study models how crimes influence the likelihood of future crimes nearby in space and time. PredPol is modeled as a self-exciting point process in which past crimes increase the short-term likelihood of future crimes. Let n index neighborhoods and let ${\mathcal{H}}_t$ denote the set of crimes observed prior to day t. The predicted crime intensity for neighborhood n at day t is written as

$${\lambda }_n^{\mathrm{PredPol}}\left(t\right)={\mu }_n+\sum _{i\in {\mathcal{H}}_t}\mathbf{1}\{n_i=n\}\theta \omega e^{-\omega (t-{\tau }_i)},$$

where ${\mu }_n$ is the background crime rate for neighborhood n, $\theta$ controls the strength of self-excitation, $\omega$ controls temporal decay, $n_i$ is the neighborhood in which crime i occurred, and ${\tau }_i$ is the occurrence time of crime i. The neighborhood-level intensities are then normalized into a probability distribution over neighborhoods

$$p_n^{\mathrm{PredPol}}\left(t\right)={\displaystyle \frac{{\lambda }_n^{\mathrm{PredPol}}\left(t\right)}{{\sum }_k{\lambda }_k^{\mathrm{PredPol}}\left(t\right)}},$$

and police officers are allocated by sampling from this distribution.

PredPol is applied to a dataset of all known crimes to predict future crime distributions. This model gives higher weights to more recent crimes; on the other hand, hot spot-based policing typically receives only recent crimes and only calculates the crime density of different areas. This method does not perform any time-wise calculation. Thus, short-term KDE more accurately represents the practice of hot spot policing than long-term KDE. However, we include long-term KDE for a fair comparison with PredPol, as it uses the same amount of data while short-term KDE only uses the most recent data. The algorithms were simulated over various settings defined by the following three variables:

1.

Number of police officers: Distribution of 40 or 400 police officers.

2.

Probability of reporting a crime: Using only detected crimes (probability of report = 0) or using both detected and reported crimes (probability of report = 0.4). The value $r=0.4$ was calculated as the weighted average of reporting probabilities across crime categories, where the weights correspond to the observed frequency of each crime type in the dataset (Section 3.2). It is used as a simplified uniform reporting rate across neighborhoods and crime types to represent under-reporting of crimes. This assumption allows us to examine how under-reporting affects policing algorithms. We note that in reality, reporting probabilities may vary across neighborhoods and crime types; we discuss this impact in Section 6.

3.

Crime type: Using all crime types in the dataset (’TOTAL’) or only aggravated assault records (’AGG. ASSAULT’).

The simulation horizon was set to 300 days. This duration was chosen to be long enough for feedback effects between crime detections and police deployments to emerge and influence algorithm behavior. At the same time, extremely long simulations can become difficult to interpret because the underlying environment is unlikely to remain stationary over extended periods. For instance, the demographic map, crime patterns, and reporting behavior may change over time. In such settings, very long simulated horizons may show recurring reversals in the bias direction, or even show no bias at all if examining average police concentration over time. Therefore, the 300 day horizon represents a compromise that allows for the study of long-term feedback tendencies while maintaining a reasonably stable environment for interpreting the results.

PredPol and long-term KDE used all available detected and reported crime data, while short-term KDE used detected and reported crimes within one month before the prediction date. The crime events were taken from the Baltimore dataset, while the simulation modeled officer allocation using alternative policing algorithms, crime detection, and prediction updates based on the observed events.

During the simulation, police locations were determined for each day over a period of 300 days, starting from 1 January 2019. Note that the police assignment for the first date was determined by considering all real crimes that occurred before that date, assuming that all the crimes were detected. However, for the subsequent days we appended that crime history to a subset of the real crimes from the previous days that were specifically labeled as detected by the simulation, plus the reported crimes of the previous days. Noisy-OR was applied to the number of police officers within the neighborhood in which each crime happened in order to generate the detected crimes dataset for the current day. This dataset was then appended along with the reported crimes dataset and used to determine police locations for the following day. Predictions were generated at each simulation day, after which police officers were accordingly allocated to the model predictions, crimes were detected according to the detection technique (Section 3.2), and detected crimes were added to the crime dataset before generating the next day’s predictions. The pseudo-codes in Algorithms 1 and 2 clarify our strategy.

Algorithm 1: Creating a Reported Crimes Dataset

Input: Crime dataset crimes, Report probability report_probability

Output: None

reported_crimes = an empty crime dataset;

save CSV file of reported_crimes;

Algorithm 2: Police Allocation Simulation

Input:  Crime dataset crimes; prediction algorithm Alg; start date start_date; end date end_date; detection probability detectProb; reported-crimes dataset reported_crimes  Output:  Result dataset res containing daily neighborhood-level police allocations and detected crimes

Initialize crimes_h ← all crimes before start_date;

Initialize detected_crimes ← crimes_h;

Initialize res with columns: date, neighborhood, crime_num, police_num, detected_crime_num;

Fill res with simulation dates, neighborhoods, and number of crimes from crimes;

return res;

3.4. Fairness and Accuracy Metrics

The results gathered from running KDE and PredPol in different scenarios were analyzed using different visualizations and statistical assessments.

To improve legibility and reduce redundancy, we use the notation $\overline{x}$ and $Avg\left(x\right)$ to denote the arithmetic mean (i.e., the sum of values divided by the number of elements). This notation is used interchangeably throughout the formulas depending on which better improves clarity and visual flow.

To quantify fairness and accuracy in our data analysis, we focus on five metrics to analyze three major concepts: the racial fairness gap, the neighborhood-level fairness gap, and coverage accuracy. For both racial group fairness and individual neighborhood fairness, equality of treatment is a combination of having equal resources in general and equal resources assigned to similar individuals. For this purpose, both racial and neighborhood-level fairness consider the equality of police share or police officer number as general resource equity and the equality of police share to crime share ratio as a measure for equity in treatment of individuals having similar crime rates, i.e., treatment proportional to crime rate.

The most fair model or simulation setting in terms of any fairness gap metric is defined as the one with the minimum average value for that metric during simulation, while the most accurate is defined as the one with the maximum average accuracy value during the simulation.

1.

Racial Fairness Gap

Racial fairness in this study is a group fairness metric. In this work, the racial disparity or fairness gap is defined as an absolute difference in averages. The use of a difference or absolute difference is a standard approach in quantifying group disparity (cf. Equation (5) in [66] and Definition 6.3 in [67]).

• Inequality of Average Police Share between White and Black Neighborhoods
  This measure determines the disparity of average police share between the two races, meaning the groups’ general equality of resource allocation:

  $${\mathrm{RacialFairnessGap}}_{\mathrm{PoliceShare}}=\left|\overline{P_{\mathrm{Black}}}-\overline{P_{\mathrm{White}}}\right|$$

  (8)

  where $\overline{P_{\mathrm{Race}}}$ is the average police share in neighborhoods of the given race. Based on the explanation above, the most fair model or simulation setting would be determined by

  $${\mathrm{MostFair}}_{\mathrm{PoliceShare}}=argmin\left(Avg\left({\mathrm{RacialFairnessGap}}_{\mathrm{PoliceShare}}\right)\right).$$

  (9)
• Inequality of Average Police-to-Crime Ratio (PCR) between White and Black Neighborhoods
  The racial fairness gap is defined as the absolute difference of the average police-to-crime ratio between groups, analyzing proportional treatment or similar treatment of individuals with similar crime rates (see Equations (10)–(14), where $N=\left|\mathrm{Neighborhoods}\right|$).

  $$PoliceShareSmoothe{d}_i={\displaystyle \frac{PoliceNu{m}_i+\epsilon }{TotalPoliceNum+N\epsilon }},i\in Neighborhoods$$

  (10)

  $$CrimeShareSmoothe{d}_i={\displaystyle \frac{CrimesNu{m}_i+\epsilon }{TotalCrimesNum+N\epsilon }},i\in Neighborhoods$$

  (11)

  $${\mathrm{PCR}}_{\mathrm{Race}}=Avg\left({\displaystyle \frac{PoliceShareSmoothe{d}_i}{CrimeShareSmoothe{d}_i}}\right),i\in Neighborhood{s}_{Race}$$

  (12)

  $${\mathrm{RacialFairnessGap}}_{\mathrm{PCR}}=\left|{\mathrm{PCR}}_{\mathrm{Black}}-{\mathrm{PCR}}_{\mathrm{White}}\right|$$

  (13)

  $${\mathrm{MostFair}}_{\mathrm{PCR}}=argmin\left(Avg\left(\left|{\mathrm{RacialFairnessGap}}_{\mathrm{PCR}}\right|\right)\right)$$

  (14)

2.

Neighborhood-Level Fairness Gap

To study inequality of treatment among individual neighborhoods, we use the Gini coefficient. This measure of inequality has been used in health [68], education [69], and especially the economic-related literature [70]. The Gini coefficient measures the distance from the equality line; the higher the coefficient, the more unequal the values. The Gini coefficient is calculated using the trapezoidal method:

$$Gini\left(X\right)={\displaystyle \frac{1}{n}}\left(n+1-2·{\displaystyle \frac{{\sum }_{i=1}^n{\sum }_{j=1}^i{x}_{\left(j\right)}}{{\sum }_{i=1}^n{x}_{\left(i\right)}}}\right),\mathrm{where}{x}_{\left(i\right)}\mathrm{are}\mathrm{sorted}\mathrm{values}.$$

(15)

• Inequality of Police Distribution
  This metric calculates the Gini coefficient of police numbers in each neighborhood during the simulation in order to determine the overall inequality of the resource distribution.

  $${\mathrm{Gini}}_{\mathrm{Police}}=\mathrm{Gini}\left({\left\{P_i\right\}}_{i=1}^N\right)$$

  (16)

  $${\mathrm{MostFair}}_{\mathrm{Gini},\mathrm{Police}}=argmin\left(Avg\left({\mathrm{Gini}}_{\mathrm{Police}}\right)\right)$$

  (17)
• Inequality of Police-to-Crime Ratio (PCR) Across Neighborhoods
  We measure the Gini coefficient of each neighborhood’s police-to-crime ratio, which is the inequality of the police distribution of similar individuals with similar crime ratios and the inequality of officer shares proportional to crime shares.

  $${\mathrm{PCR}}_i={\displaystyle \frac{P_i+\epsilon }{C_i+\epsilon }},{\mathrm{Gini}}_{\mathrm{PCR}}=\mathrm{Gini}\left({\left\{{\mathrm{PCR}}_i\right\}}_{i=1}^N\right)$$

  (18)

  $${\mathrm{MostFair}}_{\mathrm{Gini},\mathrm{PCR}}=argmin\left(Avg\left({\mathrm{Gini}}_{\mathrm{PCR}}\right)\right)$$

  (19)

3.

Coverage Accuracy — Proportion of Crimes Detected

Coverage accuracy measures the effectiveness of the police distribution relative to the actual crime distribution. It is defined as the proportion of detected crimes.

$$\mathrm{CoverageAccuracy}={\displaystyle \frac{\mathrm{Total}\mathrm{Detected}\mathrm{Crimes}}{\mathrm{Total}\mathrm{Crimes}}}$$

(20)

$$\mathrm{MostAccurate}=argmax\left(Avg\left(\mathrm{CoverageAccuracy}\right)\right)$$

(21)

3.5. Experimental Environment

All experiments were implemented in Python (version 3.9.13) within an Anaconda environment, using libraries including scikit-learn (version 1.1.3), NumPy (version 1.22.3), pandas (version 1.5.1), GeoPandas (version 0.12.1), Shapely (version 1.8.5. post 1), inequalipy (version 1.0.5), Matplotlib (version 3.6.2), Plotly (version 6.5.2), Folium (version 0.13.0), and seaborn (version 0.12.2). The primary simulation results presented in the main text were obtained using standard computing resources. To support robustness analysis over extended parameter ranges, additional large-scale experiments reported in the appendix were conducted on the UMBC High Performance Computing Facility (HPCF) cluster (chip). Code and processed data are publicly available to facilitate reproducibility.

4. Results

This section describes the results of our experiments. The results dataset and code for the analysis are available online (accessed on 1 April 2026) (https://github.com/saminsemsar/Data_Analysis_Portfolio/tree/main/PredictivePolicing). To summarize, across the scenarios we observe three main and mostly consistent patterns. First, PredPol achieves higher average coverage accuracy and fairness than KDE-based methods in most of the scenarios. Second, PredPol often shows faster deterioration in neighborhood-level fairness. Third, crime type affects police concentration patterns. In what follows, we provide the details of these findings. The results are organized to address a series of guiding questions aimed at examining key aspects of our research objectives:

• Which algorithm is fairer or more accurate in our specific location and time period?
• Do different systems react differently to feedback loops?
• Is the bias model-driven or data-driven?
• How do the police concentration and distribution change over the course of the simulation in different scenarios?

Before presenting the findings, we begin with a map-based visualization to provide an intuitive overview of how police were distributed on the first and last days of simulations in an example scenario (40 police officers, 300 days) (Figure 2). The red circles are crime hot spots in Black neighborhoods, the black circles are those in White neighborhoods, and the blue markers are police locations. It can be observed that police are allocated more evenly among neighborhoods on the first day of allocation compared to the last day (about a year later), as shown in Figure 2, illustrating the feedback loop phenomenon [9].

We present the detailed findings organized by themes corresponding to the guiding questions listed above. Some of these findings have been further analyzed for robustness to different report probabilities; this analysis can be found in Appendix A.

1. 

Comparative Fairness and Accuracy Across Models

This subsection compares short-term KDE, long-term KDE, and PredPol in terms of overall fairness and accuracy within the Baltimore study region and time period. The findings summarize average and aggregate performance across scenarios, including whether the improvements in fairness affected accuracy.

Finding 1.1 

PredPol was the Most Accurate at the Beginning

In 50% of scenarios, PredPol had the highest coverage accuracy compared to the other two on the simulation’s first day. However, on the last day of the simulation, long-term KDE became the most accurate in 75% of the scenarios, cf. Figure 3.

Finding 1.2 

PredPol was generally the most accurate algorithm.

PredPol was the most accurate algorithm in 87.5% of scenarios and the second-most accurate in the rest. Conversely, short-term KDE was the least accurate in 75% of scenarios (Figure 4). Further analysis of this result’s robustness against different report probabilities (Appendix A) proved that although the extent of difference in accuracy of these models was influenced by $pReport$, their relative ranking remained mostly the same (Figure A1).

Finding 1.3 

PredPol was the most racially fair, while long-term KDE was the least.

PredPol was the most racially fair, in terms of equality of police-share to crime-share ratio, in 75% of scenarios, and long-term KDE was the least racially fair in 87.5% of scenarios (Figure 5). In terms of average police share, PredPol was the most racially fair model in 62.5% of scenarios. Long-term KDE, on the other hand, was the least racially fair model in 75% of scenarios (Figure 6 and Figure 7). Further analysis of this result’s robustness against different report probabilities (Appendix A) proved that although the extent of the difference in racial fairness gap between these models was influenced by $pReport$ (Figure A1, right column), their relative ranking remained mostly the same (Figure A1, left column), with PredPol having a higher probability of achieving the best performance across $pReports$.

Finding 1.4 

PredPol was the most fair in regard to neighborhood-level fairness, followed by long-term KDE, while short-term KDE was the least fair at the neighborhood level.

Based on both equality of average police share and equality of average police-to-crime ratio, PredPol (in 75% of scenarios) and long-term KDE (in the other 25%) were the most fair models at the neighborhood level. According to both metrics, short-term KDE was the least fair at the neighborhood level in 100% of scenarios (Figure 8 and Figure 9). Further analysis of this result’s robustness against different report probabilities (Appendix A) proved that although the extent of difference between these models in terms of the neighborhood-level fairness gap was influenced by $pReport$ (Figure A1, right column), their relative ranking remained mostly the same, with PredPol maintaining the highest probability of achieving the best performance across $pReports$ (with the exception of $pReport=0$, where long-term KDE and PredPol swapped places); see Figure A1, left column. In general, long-term KDE and PredPol performed more alike in terms of the extent of neighborhood-level fairness gap, but with a higher decrease in gap for PredPol compared to long-term KDE as the report probability increased (Figure A1, right column).

To reinforce the scenario percentage pattern reporting, we also computed paired differences between PredPol and each KDE-based method across the eight main scenarios. The average differences are provided in

Table 2. Paired scenario-level differences between PredPol and KDE-based methods across the eight main scenarios. Positive values indicate higher accuracy for PredPol; negative values indicate lower fairness gaps (better fairness) for PredPol.

Metric	Avg. Dif. Long-Term KDE	Avg. Dif. Short-Term KDE
Average accuracy	$+0.0127$	$+0.1306$
Racial fairness gap (PCR)	$-3.3727$	$-1.8443$
Racial fairness gap (avg. police share)	$-0.0003$	$-0.0002$
Neighborhood fairness gap (PCR Gini)	$-0.039$	$-0.1908$
Neighborhood fairness gap (police Gini)	$-0.0329$	$-0.2171$

.

Finding 1.5 

Higher accuracy is accompanied with lower neighborhood-level fairness-gap.

In Figure 10, it can be observed that higher neighborhood-level fairness gaps (lower fairness) were accompanied by lower accuracy. The accuracy–fairness correlation is clearer when we group the data points by the number of officers allocated, which was the variable creating the most distinct clusters (Appendix B). Further correlation analysis can be found in Appendix B.

Finding 1.6 

Racial fairness is not correlated with accuracy.

Unlike neighborhood-level fairness, we did not observe any correlations between racial fairness and accuracy (Figure 11, Appendix B). All levels of racial fairness were observed over all levels of accuracy. Further correlation analysis can be found in Appendix B.

2. 

Differential Responses to Feedback Loops

This subsection examines how the different algorithms responded to feedback loops over time. These findings focus on temporal trends, including the direction and speed of change in fairness and accuracy, rather than on average outcomes. Overall, we found that long-term KDE had the slowest pace of change (slope of trend-line) for accuracy and fairness metrics over the days of simulation, with PredPol and short-term KDE competing for the least stable. In some cases where this trend moved towards focusing officers on a race different from the one at the start of simulation, the trend became positive, showing a temporary improvement in bias. This happened more for KDE-based models. A more detailed report is presented below.

Finding 2.1 

Long-Term KDE had the slowest pace of change in most scenarios, especially for neighborhood-level fairness.

Long-term KDE had the smallest slope of the trend line in:

• 50% of scenarios for racial fairness in terms of average police share (Figure 12).
• 37.5% of scenarios for racial fairness in terms of police-to-crime proportionality (Figure 13).
• 75% of scenarios for neighborhood-level fairness in terms of police-share equality (Figure 14 and Figure 15).
• 75% of scenarios for neighborhood-level fairness in terms of police–crime proportionality (Figure 16).

Finding 2.2 

Short-term KDE had the fastest trend of racial bias based on average police share in most of the scenarios.

Short-term KDE had the fastest trend of racial bias in 62.5% of scenarios for equality of average police share (Figure 12). Taking a closer look at individual scenarios, although short-term KDE ranked second in pace of neighborhood-level bias in most of the scenarios, its change in neighborhood-level fairness over the days showed a more volatile pattern compared to the other two models (Figure 15; to see the other scenarios, refer to the code [71]).

Finding 2.3 

PredPol had the fastest trend of neighborhood-level bias, and showed a trend in racial bias metrics for most of the scenarios.

We have already established that PredPol was generally the fairest model in terms of both racial and neighborhood-level fairness; however, looking at the pace of change for each metric over different scenarios, PredPol shows a fast neighborhood-level fairness gap trend (Figure 14 and Figure 16) and racial fairness trend in terms of police-to-crime ratio in most of the scenarios (Figure 13). This suggests that it is indeed vulnerable to bias from feedback loops, as previously reported by [9].

Finding 2.4 

Bias amplification was observed in a higher percentage of scenarios for PredPol compared to the other two models.

Although PredPol was fairer in most of the scenarios, amplification of bias was seen in its trends in a higher percentage of scenarios than for the two KDE approaches. PredPol’s neighborhood-level fairness dropped in 100% of scenarios, while the drop for short-term KDE and long-term KDE occurred in 75% and 37.5%, respectively (Figure 17 and Figure 18).

The drop in racial fairness for PredPol happened in 75% of scenarios for police–crime proportionality and 50% of scenarios for average police share, while these numbers were 50% and 25% for short-term KDE and 75% and 25% for long-term KDE, respectively (Figure 19 and Figure 20). In Figure 19 and Figure 20, the hatched or striped bars are those in which the race receiving focus at the start of the simulation contrasted with the race the trend was toward; this demonstrates how all the fairness gaps with negative slopes were those where the race focus at the beginning of the simulation contradicted the trend. For instance, refer to the models’ behaviors in the example scenario in Figure 7.

Finding 2.5 

Short-term KDE experienced an accuracy drop in a higher percentage of scenarios compared to the other two models.

Although we observed previously that neighborhood-level fairness and accuracy seemed to be correlated to some extent, here we saw that short-term KDE, not the model with constant and highest speed of neighborhood-level fairness drop, experienced accuracy drops in 75% of the scenarios. The corresponding drop percentages were 62.5% for PredPol and 37.5% for short-term KDE (Figure 21).

Finding 2.6 

Distribution uniformity did not guarantee the defined racial fairness.

While the percentage of scenarios with a drop in neighborhood-level fairness ranged from 75 to 100 for PredPol and short-term KDE, the percentage of scenarios with a drop in racial fairness ranged from 25 to 75 (Figure 17, Figure 18, Figure 19 and Figure 20). On the other hand, for long-term KDE racial fairness based on police–crime proportionality fell in 75% of scenarios, while both neighborhood-level fairness metrics worsened in 37.5% of scenarios. Long-term KDE’s racial fairness based on average police share only fell in 25% of scenarios. This indicates that the models with the least police distribution uniformity might stay racially fair in areas with certain demographic maps and crime records.

Finding 2.7 

Bias typically worsened over time except when the trend and current state contradicted.

We observed a consistent pattern across scenarios: in cases where racial fairness appeared to improve unexpectedly, the demographic group receiving the greatest police attention at the start of the simulation was different from the group receiving the most attention at the end. In the case of Baltimore and using all crime records, this focus at the end was surprisingly on White neighborhoods. This shift largely accounted for the observed improvement (see Figure 19 and Figure 20). For these contradictory cases where the initial state and the direction of change did not align with bias amplification, the slope of the bias trend must have been sufficiently steep relative to the initial disparity for the trajectory to have crossed the parity point and begun moving toward amplification. In such situations, longer simulation horizons are required for this transition to appear.

These apparent improvements hint at the possibility of seasonal cycles of bias reduction and subsequent amplification when simulations are run for extended durations. Changes in the data distribution for real crimes may arise from multiple underlying factors. Two such factors are illustrated below:

i.

Seasonality: Any variable with cyclic effect, such as weather, could cause seasonal changes in the distribution.

ii.

Police policy change: Changes in police distribution or enforcement practices could reduce crime in a given location or suppress specific crime types affected by that policy, which would then induce a shift in crime distribution.

Therefore, when data on real crimes are used and when both reported and detected crimes contribute to the updates, the distribution of reported crimes may begin to favor a neighborhood that was not previously predicted as high-risk. As officers gradually shift their attention from earlier high crime rate areas towards this newly emergent hot spot, a temporary reduction in measured bias can occur. This is typically followed by renewed amplification once police resources become concentrated in the new focal area, until the distribution of reported crimes eventually shifts again.

3. 

The Effects of Data Variation on Bias

This subsection examines how different crime data affect the fairness and accuracy outcomes across the algorithms. We compared scenarios using aggravated assault records with those using all crime records in order to analyze the change in fairness and accuracy.

Finding 3.1 

Expanding from aggravated assault to all crimes increased inequality of the police-to-crime ratio and reduced accuracy for long-term KDE and PredPol.

When changing the crime type from aggravated assault to all crimes, the neighborhood-level gap based on the police-to-crime ratio and accuracy dropped in most scenarios (see Figure 22 and Figure 23).

Finding 3.2 

Different models could have different bias outcomes when applied to different data, and vice versa.

When looking at the slope of the trendline to determine which race is receiving a higher average police share (Figure 24) or higher average police-to-crime ratio (Figure 25) as the simulation days pass, we observe that when feeding aggravated assault crime records to PredPol, the focus is more on Black neighborhoods, whereas the focus flips to White neighborhoods when using all crime records. This is observed in most scenarios and for both metrics. This result is noteworthy because the debate around bias in predictive policing typically makes the implicit assumption that algorithms are solely biased against Black neighborhoods, following the work of Lum and Isaac [9].

4. 

Police Concentration Areas in Baltimore

This subsection focuses on how police resources are distributed across neighborhoods over the course of the simulation. To be more precise, we looked at the number of scenarios in which each of the top-10 most frequently over-policed neighborhoods were ranked as top-3.

Finding 4.1 

Assigning officers based on aggravated assault led to over-policing more Black neighborhoods than White neighborhoods

Over-policing is defined as having a higher police share compared to crime share. Looking at the top-10 most frequently over-policed neighborhoods by each algorithm when using aggravated assault records, we observed that other than Downtown, which is the most frequently over-policed neighborhood in the scenarios, all other top-10 frequently over-policed neighborhoods were Black, such as Blair-Edison and Sandtown-Winchester. (Figure 26).

Finding 4.2 

Assigning officers based on all crime records by PredPol over-policed mostly White neighborhoods such as Mount Vernon and Canton, unlike the other two models.

Looking at the top-10 most frequently over-policed neighborhoods by each algorithm when all crime records were used, we saw that other than Sandtown-Winchester and Cherry Hill, all other top-10 frequently over-policed neighborhoods by PredPol were White, such as Mount Vernon, Canton, and Brooklyn. (Figure 27). Among short-term KDE’s over-policed neighborhoods, only two of six (Paterson Park and Inner Harbor) were White; among long-term KDE’s over-policed neighborhoods, one out of four (Brooklyn) was White.

5. Discussion

We performed simulations in a range of scenarios comparing different policing systems using a Baltimore crime dataset. Based on our simulations and analyses, we draw the following overall conclusions:

• Fairness and Accuracy Comparison
  By comparing simulation results of representative predictive policing and hot spot policing algorithms, we discovered that predictive policing via PredPol was generally more accurate and had higher racial and neighborhood-level fairness than hot spot policing via short-term and long-term KDE across most scenarios.
  Long-term KDE, where we widened KDE’s data input to use the same data received by PredPol, improved accuracy and neighborhood-level fairness over that of short-term KDE but worsened racial fairness. The improvement in accuracy did not overtake PredPol’s level of accuracy but almost matched its neighborhood-level fairness.
  The theory of hot spot policing claims that distribution of recent crimes can approximate their near-future distribution. Recent crimes are often defined as those within either a month or a year of the prediction date [72]. In our experiment, both long-term KDE and PredPol received the crime history from a year before the starting date of the simulation. The simulation’s starting prediction date was 1 January 2019, and the crime history included all crimes after 1 January 2018. If the crime history was more than a year ago, we expect that the accuracy at the starting date would drop, but that overall accuracy might improve because of our finding that a longer crime history slows down the effect of neighborhood-level bias for KDE, which in turn slows the drop in accuracy.
  Our results show that higher accuracy does not guarantee a racially fairer model. Both fair and unfair outcomes were observed across the fairness spectrum for high-accuracy scenarios (Figure 11). However, neighborhood-level fairness seemed to correlate with accuracy (Figure 10). These results extend the work of Mohler et al. [12] on accuracy–fairness tradeoffs by providing evidence from extended simulations on real crime data. Unlike Mohler’s observation that fairness comes at a cost in accuracy, some of our scenarios were both fairer and more accurate than others. This allows us to infer that the give-or-take between accuracy and fairness depends on both the fairness metric and the demographic geography of the location the system is customized for (Figure 10 and Figure 11).
• Temporal Feedback Loop Effects
  We saw a trend of bias amplification in most of the scenarios for all three systems, except when there was a change in the pattern of data during the simulation causing the police focus to change from certain neighborhoods to others. In scenarios where bias did not become worse, the race that received the highest focus at the start of the simulation contradicted the race at the end, as shown in Figure 19 and Figure 20. Note that the negative slopes of the fairness gaps indicate those scenarios with improvements in regard to that metric. For these scenarios, bias amplification might have happened later on if the simulation had been continued for a longer period, unless reported crime distribution changed drastically again. Therefore, when using real crime data we saw that bias amplification was not constant, instead depending on changes in the distribution of newly discovered and reported crimes. For neighborhood-level fairness, we saw a continual increase in bias for most of the scenarios, especially for PredPol and short-term KDE (Figure 17 and Figure 18). For racial fairness, a lower number of scenarios experienced bias amplification compared to neighborhood-level fairness, especially for the metric related to average police share (Figure 19). However, it should be noted that all scenarios with bias improvement also showed a contradiction between which race the initial police focus was on and which race the focus was trending towards.
  It is important to consider that we defined bias amplification as a falling trend in a system’s equality of treatment. In all of scenarios where we saw improvement in equality, we also observed that the race receiving a higher policing share at the beginning of the simulation was different from the race that the trend was towards; therefore, a longer simulation duration is needed in order to make sure there is not a point of racial equality in the future after which a bias amplification towards the trending race occurs. We did not extend the simulation further, as longer durations introduce a shift in the underlying crime distribution that interacts with the feedback effect, making it difficult to attribute the changes in fairness to a single mechanism. Unlike Ensign’s assumptive theoretical study that showed constant bias amplification when assigning an officer between two neighborhoods [10], in a more realistic situation such amplification is not constant, and could temporarily subside before intensifying again.
  Although PredPol was generally fairer and more accurate on average in most of the scenarios, its speed of bias amplification was higher for most of the metrics in most of the scenarios, while the speed of bias amplification for long-term KDE was substantially slower compared to the other two in most scenarios. This provides further evidence that different models respond differently to feedback loops.
• Crime Type Effect
  In the context of Baltimore, using different data from all crime records flipped the police concentration from one demographic to another for PredPol, and in some scenarios for short-term and long-term KDE as well. This result shows that the direction of racial bias can be affected not only by the predictive policing system but also by the data it was applied to.
  We also observed that the average accuracy and average neighborhood-level equality based on the police-to-crime ratio dropped for long-term KDE and PredPol. This suggests that systems with more focus on the long-term effects of present crime on predictions of future crime might become less accurate when daily records are sparse. Alternatively, it could be that data records had a higher change in distribution over a short period of time. The sparseness of crime data and its effect on different systems could be studied further in future work.
• Data-Driven vs. Model-Driven Bias
  Our analyses found that both data and model matter: the same data fed into different models (KDE vs. PredPol) produced different fairness outcomes as previously shown by Chapman et al. [5]. The same model fed different data (aggravated assault vs. total crime) can affect the speed of bias development, or even flip the direction of bias.
• Baltimore-Specific Insights
  Black neighborhoods received more policing on average in most simulations, although this varied with crime type and algorithm. In most scenarios, the trend was toward assigning a higher average share of officers in general, and also toward a higher average share of officers per share of crime to White neighborhoods when all crime records were used. This finding counters the widely held assumption that predictive policing is biased specifically against Black neighborhoods, as found by Lum and Isaac in data from Oakland, California [9]. These patterns are not solely explainable by neighborhood count (Baltimore has more Black than White neighborhoods), suggesting that model behavior plays a larger role than geography alone. Furthermore, when examining the top-5 highest crime neighborhoods in different period lengths before the start of the simulation (cf. Figure 28), we observed the percentage of crimes occurring in White neighborhoods to be substantially higher for total crimes as compared to aggravated assault. This indicates that the crime distribution differs by crime type, and could affect the direction of bias.
  However, when comparing the police allocation of different models with crime percentage concentrations (cf. Figure 29), we observe that the models produce different police distributions even on the first day of the simulation. These differences reflect inherent differences in model behavior. Since longer term include the influence of feedback loops, these initial discrepancies are likely to compound over time, leading to the observed long-run differences in speed or extent of bias.
  Thus, both data and model behavior appear to affect not only the magnitude of bias in Baltimore, but also its direction and how the direction evolves over time.
  Downtown, Sandtown-Winchester, and Blair-Edison were repeatedly among the top over-policed neighborhoods when aggravated assault records were used. This consistency across models suggests that some neighborhoods are structurally favored or targeted regardless of which predictive algorithm is used.
  However, when looking at over-policed neighborhoods when using all crime records, we see that the models are not as consistent. This is another demonstration of how data and algorithms can both affect the results. The only neighborhood the algorithms commonly over-policed when using total crime records was Sandtown-Winchester, a Black neighborhood.
  These Baltimore predictive policing simulation results are consistent with previous studies on other cities, including both real experimental studies [7] and simulation studies [9], in that police concentration tendencies occurred in the models due to feedback loops.
  Based on the duration of the simulation, these systems might change rank in terms of each bias metric or even average accuracy. A short simulation duration might indicate PredPol as causing the most uniform police distribution (based on police Gini coefficient), followed by long-term KDE and then short-term KDE, while a longer simulation duration might cause a rank swap between long-term KDE and PredPol. These variables could be studied in future system-comparative works.
  For Baltimore City, our recommendations are as follows:

  –

  Although predictive policing remained fairer and more accurate than hot spots-based policing, it had a higher speed of bias amplification than hot spots policing in most scenarios. Hence, we advise the city to be aware of the long-term tendencies of any predictive policing system they might use.

  –

  Evaluation studies like the one performed here should be performed prior to real-world implementation. These studies can highlight bias issues arising from the combination of a particular algorithm and data, leading to decision-making that is better informed.

  –

  When distributing resources using predictive policing models based on aggravated assault records, authorities should be mindful of assigning too many officers to the Downtown, Blair-Edison, and Sandtown-Winchester neighborhoods. We advise careful interpretation of results when applying all crime records to the predictive policing model, since Sandtown-Winchester, Brooklyn, Mount Vernon, and Canton could incorrectly appear to have higher crime rates due to feedback loop phenomena. Similarly, the city could prepare a list of neighborhoods that might be under-rated, under-policed, and in need of more attention.

  –

  Our results highlight the importance of actively monitoring long-term fairness trends when deploying predictive policing systems. In particular, it is important to distinguish between estimated or predicted trends in model behavior obtained through pre-deployment evaluation and observed trends that emerge from real-world implementation. In practice, this could involve periodic reassessment of both predicted and observed system behavior over time. At the end of each evaluation period, stakeholders could conduct audits of police allocation distributions and fairness indicators (e.g., racial or neighborhood-level fairness) to detect emerging bias. These audits should compare observed trends with previously estimated tendencies and current outcomes with those from previous periods. Such comparisons could help to identify deviations between expected and actual system behavior as well as shifts in bias over time. Decisions made based on estimated trends may also influence future outcomes, which should be documented and studied.

  –

  Engaging domain experts and community stakeholders in defining additional fairness metrics and reviewing these trends could bring to light new aspects involving fairness and community welfare and provide critical context for interpreting system behaviors.

  Together, these steps can help to ensure that predictive policing systems remain aligned with fairness objectives over extended deployment periods.
• Policy Implications
  Based on our examination of various aspects of social fairness, policymakers should look beyond the promised ability to reduce and prevent crime when authorizing deployment or continuation of any smart policing system. More broadly, achieving a society where every individual has equal opportunity to succeed, and in turn to help their community prosper, requires coordinated efforts among stakeholders along with a continuous evaluation process to ensure that these systems operate as intended.
  Our results demonstrate that predictive policing systems exhibit dynamic behavior over time, affecting how these system deployments need to be regulated. Policies regarding their evaluation should be ongoing, meaning that continuous pre- and post-deployment evaluations through periodic audits need to be performed rather than a one-time pre-deployment evaluation. By enforcing appropriate laws and policies, it is necessary to establish a cyclic evaluation framework that includes the following steps:

  1.

  Defining/updating fairness and accuracy metrics.

  2.

  Defining/updating the system dynamics and behavioral responses, e.g., how police presence changes crime distribution, how different environmental factors change the probability of crime being reported, etc.

  3.

  Defining/updating policies and regulations.

  4.

  Pre-deployment evaluation of the policing system’s behavior.

  5.

  Monitoring real-world outcomes.

  6.

  Conducting causal and experimental analysis of outcome data to better understand interactions among the metrics and the system’s responses.

  7.

  Interpretation of the results by community stakeholders and domain experts.

  This approach shifts the focus from selecting the best policing system to establishing a robust evaluation and governance process that directs the development of the policing system, simulation framework, and evaluation framework itself. Such a process can adapt to long-term system dynamics while ensuring that the system meets fairness objectives over time.
  Furthermore, a dynamic governance process and continuous re-evaluation that engages community stakeholders to promote transparency and accountability will result in higher levels of community cooperation, thereby building trust between citizens and law enforcement agencies. When stakeholders are aware that system performance is regularly assessed and corrective actions are being taken, confidence in the system will improve and tolerance for inadequacies and enforcement errors will increase.

6. Limitations

Limitations of this work include:

• Every individual scenario parameter (e.g., crime-type, number of officers, probability of report, etc.) and hyperparameter (such as base probability of crime detection by an officer within detection radius (the crime’s neighborhood), could be explored over a wider range of values to asses robustness and their influence on the results.
• We assumed that the crimes in the database represented all crimes that actually happened, which is untrue. These records could be the result of an already biased system, such as by having less crime reported or discovered in certain areas.
• We used real crime data, which we filtered by using a fixed report probability for all neighborhoods. The approach would be more realistic if different crimes were filtered based on their report probability instead of using an average report probability for all crimes. Using varying report probabilities across neighborhoods and/or crime types could change the distribution of observed crimes (detected + reported) and potentially influence the fairness and accuracy outcomes of the algorithms. Exploring neighborhood-specific and/or crime type-specific reporting rates would be a valuable expansion for future work.
• We did not account for how the presence of police in a neighborhood could affect the crime distribution. The only change in crime distribution over the course of our simulations consisted of what was hidden in the original crime data, without any calculated manual changes. Currently, we expect that by running the simulation for longer periods, these systems would potentially experience a flip in their bias direction tendency when the distribution of the filtered reported crimes eventually changes by suddenly having several days of high crime rates in neighborhoods with different demographic majorities.
• In this work, we considered a detection formula that implicitly assumes crime detection to be determined exclusively by the number of deployed officers. Nevertheless, in real-world deployments detection rates may also be affected by the model-estimated level of crime risk itself. Officers entering a predicted high-crime area typically behave with elevated vigilance, which can inform how they evaluate behavior and distribute attention [73]. As a result, detection probability may be determined not only by officer count but also by the broader sociotechnical context induced by the predictive mechanism.

7. Conclusions

In this study, we develop an agent-based modeling simulation to investigate whether the use of hot spot policing (short-term and long-term KDE) and predictive policing (PredPol) lead to feedback loops and hence to racial bias in Baltimore. We define accuracy and fairness metrics to evaluate and compare the systems’ behavior in this locality. In doing so, we extend the body of existing literature by providing a framework consisting of fairness and accuracy metrics to compare and evaluate different systems’ long-term localized tendencies. This approach provides insights into the resource distribution tendencies of each model, recounting neighborhoods that might be over- or under-rated as crime hotspots by looking at which neighborhoods are over-policed or under-policed during a 300-day simulation period. Our results using 2019–2020 data from Baltimore City show that predictive policing and hot spot-based policing are indeed subject to both feedback loops and to racial and neighborhood-level bias. Although the speed of bias amplification was higher for predictive policing, it was still the fairer and more accurate system on average compared to hot spot-based policing systems in our 300-day simulations. However, our results suggest that the relatively high vulnerability of predictive policing to feedback loops could result in poorer fairness and accuracy than hot spot policing over the long term.

Our findings in these 300-day simulation scenarios of Baltimore show that, contrary to the generally assumed bias trends, the police concentration trends when using all crime records were mostly towards White neighborhoods. Downtown Baltimore is a mostly White-dominant area, and appeared as the neighborhood receiving the highest policing shares (with a rising trend) in most of the simulation scenarios. This shows that fairness issues in predictive policing algorithms have impacts beyond the Black community, which could motivate a broader set of stakeholders to work together to find solutions that benefit the entire community. It is also crucial to continuously monitor the fairness dynamics in real-world deployments. In particular, comparing predicted model behavior with observed outcomes over time, along with periodic auditing of fairness metrics, would help practitioners to identify emerging biases and adjust their decision-making strategies accordingly. Overall, our results emphasize the need for continuous localized fairness assessments and cautious use of predictive policing tools in sensitive environments.