Detecting Abnormal Behavior Events and Gatherings in Public Spaces Using Deep Learning: A Review

Abstract

Public security is a crucial aspect of maintaining social order. Although crime rates in Western cultures may be considered socially acceptable, it is important to continually improve security measures to prevent potential risks. With the advancements in artificial intelligence methods, particularly in deep learning and computer vision, it has become possible to detect abnormal event patterns in groups of people. This paper presents a review of the deep learning techniques employed for identifying gatherings of people and detecting anomalous events to enhance public security. Some of the open research areas are identified, including the lack of works addressing multiple cases of anomalies in large concentrations of people, which leaves open an important avenue for future scientific work.

1. Introduction

Public security is a paramount concern in modern societies. While crime rates may be considered socially acceptable, ensuring the safety and well-being of individuals and communities remains a pressing need. In recent years, there has been a growing recognition that traditional approaches to public security may not be sufficient in addressing emerging threats and challenges [1].

One such challenge is the rise of new forms of violent and anti-social behavior, often exhibited by multicultural youth gangs. These gangs, comprising individuals from diverse backgrounds, engage in coordinated acts of physical brutality using weapons such as knives and blunt objects. Their activities pose a significant risk not only to rival gangs but also to innocent bystanders, increasing the urgency to find proactive solutions to mitigate potential harm.

Additionally, the changing dynamics of public demonstrations have further underscored the need for advanced security measures. Some Western countries are considering allowing spontaneous demonstrations without the obligation of prior notification to authorities. While this allows for greater freedom of expression, it also introduces potential risks. Spontaneous gatherings with illegal intentions can result in injuries to the general public and property damage, necessitating innovative approaches to identify and prevent such incidents.

Given these pressing challenges, there is a critical need to explore advanced technological solutions that can enhance public security measures. Artificial intelligence (AI) methods, particularly in the realms of deep learning and computer vision, offer promising avenues for detecting abnormal behavior patterns and gatherings in public spaces [2]. By harnessing the power of AI, it becomes possible to detect and predict potential security risks, allowing for timely interventions and improved public safety.

To understand the current capabilities and limitations of AI systems in this domain, it is essential to trace the historical evolution of video-based anomaly detection technologies. The automatic detection of anomalous events in video has evolved significantly over the past two decades. In the early 2000s, most approaches relied on traditional computer vision techniques with hand-crafted features and statistical models to describe normal behavior [3]. Techniques such as Gaussian mixture models for background subtraction or trajectory-based clustering using SVMs were common [4]. These classical systems were based on explicit assumptions and expert-defined rules, often combining motion vectors or texture patterns with machine learning methods such as k-NN [5], HMMs [6], random forests [7], or shallow neural networks [8]. Around the mid-2010s, a paradigm shift occurred with the advent of deep learning. Enabled by powerful hardware and larger video datasets, researchers began using deep neural networks to automatically learn features from data. Early architectures such as autoencoders and 2D CNNs modeled normal video patterns through reconstruction or prediction, triggering anomaly alerts when errors spiked [9,10,11].

Shortly thereafter, more specialized video architectures emerged. Temporal modeling was introduced through RNNs and LSTMs, as well as ConvLSTM and 3D CNNs like C3D or I3D, which learned spatiotemporal features directly from video [12]. Generative models such as GANs and VAEs allowed for the synthesis of future frames or latent scene representations, detecting anomalies through generation discrepancies [13]. More recently, the 2020s have seen a push toward even more generalizable and semantically rich approaches. Hybrid models now combine classical insights with deep learning, while self-supervised learning uses pretext tasks like frame ordering to learn useful representations [14]. Vision–language models (VLMs) have also entered the field, integrating textual context to better distinguish between benign unusual behavior and true security threats [15,16].

Existing reviews [17,18,19,20] in the field of computer vision have extensively explored monitoring the behavior of individuals or groups. However, the focus of much of the literature has been on the post-event detection of anomalous events, rather than addressing the challenge of prior detection. Alternatively, the available literature primarily concentrates on detecting events after they have occurred or, at best, during their ongoing unfolding.

Considering the identified gaps in prior research on public safety risk detection, our research aims to investigate the potential of AI to provide highly accurate predictive tools for anticipating imminent anomalous events. Our interest lies in methods that go beyond existing detection techniques for anomalous events in large gatherings of people, enabling the anticipation of future events and ensuring a higher degree of general satisfaction among the population.

This paper aims to address these pressing concerns by conducting a literature review of AI methods, specifically in deep learning and computer vision, that are capable of detecting gatherings of people and abnormal events. The review provides readers with 35 papers pertaining to mass congregations of people and the analysis of event patterns, thereby fulfilling the criteria for quality assessment. Additionally, a comprehensive examination of the data obtained from the selected studies is conducted, leading to the formulation of recommendations and guidelines to support future work in this research domain.

The remainder of the paper is organized as follows: Section 2 describes the steps used to select those primary studies of interest for the analysis. Section 3 presents the results of the searches and their quality assessment. Section 4 attempts to answer the questions presented in Section 2.2. Section 5 and Section 6 highlight the different techniques of computer vision and the conclusions.

2. Research Methodology

2.1. Prior Research

Regarding the identification of potential anomalous events with significant consequences in densely populated areas, our review of the literature revealed inconclusive systematic reviews. While the consulted sources primarily concentrate on aspects related to detecting and monitoring groups or congregations of individuals, we did encounter noteworthy research that can serve as an initial exemplification, such as learning to predict crowd flows [21], monitoring of subjects and relationships with lots of people [22], and identification of abnormal behavioral patterns in crowds [22].

2.2. Research Goals

In this phase, the questions that gave rise to the research were examined, along with the methods applied to find and focus the information. For instance, the databases that were consulted, the keywords that were used, and so on, were considered. The following six questions were considered, as presented in

Table 1. Research questions.

Questions	Route
Q1. What is considered to be a gathering of people?	The foundational concept to consider at the outset of this research is the notion of a gathering. Valuable outcomes can be achieved by accurately determining the head count and confirming the presence of a group surpassing a specified threshold. Furthermore, it is crucial to establish the capability of tracking individuals and their interactions.
Q2. Can AI recognize what a gathering of people is?	The integration of deep learning techniques with well-trained models plays a crucial role in facilitating real-time crowd scene monitoring. By employing AI techniques and leveraging meticulously trained datasets alongside mathematical models, it becomes possible to effectively discern between organized congregations of individuals and mere streams of people occupying the same vicinity.
Q3. How can we detect with videos the difference between a peaceful and a nonpeaceful gathering?	For those occasions when a gathering may lead to violent actions, the system should be able to predict and even anticipate them. Once the need for the number of participating individuals is covered, whether the behavior of that crowd of people corresponds to something that could be called normal, or whether, on the contrary, actions outside the norm are evident.
Q4. What are the different types of deep learning networks and architectures used for abnormal event and gathering detection?	This section will be addressed by examining several prevalent learning mechanisms frequently employed in anomaly detection, including autoencoders, GANs, and diverse variations of CNNs.
Q5. What are the performance and accuracy of these methods on public security datasets?	The performance and accuracy of the CNNs and autoencoders examined in the preceding section can be influenced by several factors. These factors include the size and quality of the datasets, the complexity of the problem under consideration, the architectural design of the models employed, and the choice of evaluation metrics for performance assessment. Consequently, our research will focus on identifying the situations where the utilization of one AI technique over another is more advantageous or suitable.
Q6. What key elements of the datasets can provide evidence of the outcome of the gathering, and which datasets are most commonly used to monitor gatherings using deep learning?	We need studies to be able to analyze actions in a gathering of people, but also consider the combination of other determining factors, and whether a bladed weapon or blunt object can be found in the crowd. In these cases, it is necessary to use deep learning to detect other important factors.

, in which the question’s path can be determined by the column called “route”:

We mainly used the databases of Scopus, Web of Science, IEEE Xplore Digital Library, and Google Scholar for the collection of related papers, and depending on the platform, the search focused on the title, abstract, or keywords.

The criteria used in the searches referred to syntaxes such as citizen security, vision camera, computer machine, etc., using Boolean operators AND and OR. The strings used in the search were as follows:

• TI=(crowd gathering) OR TI=(spontaneous demonstration) AND TI=(crowd aggregation) AND TI=(abnormal aggregation) AND PY=(2017–2025);
• TITLE-ABS-KEY ((crowd OR gathering) AND ( (movement AND patterns) OR (weapon AND detection) ) AND analysis AND (human OR person OR people)) AND PUBYEAR > 2016;
• TITLE-ABS-KEY (“public safety”) AND TITLE-ABS-KEY (“computer vision”) AND TITLE-ABS-KEY (“deep learning”) AND PUBYEAR > 2016.

Due to the increase in the number of smart cities, where those have sensors that capture almost every possible kind of image such as traffic control or crowd counting, we focused on the results that were more directly related to public security, searching for spontaneous gatherings of people in unusual areas (crowd aggregation evolution, or abnormal crowd aggregation), as well as, for example, the detection of items capable of carrying bladed weapons or abnormal behaviors such as running at unexpected times or in unexpected ways, falls, etc. Hence, once the results of the metasearch engines were obtained, screening was carried out to only keep the publications that are relevant to our research through exclusion/inclusion criteria, which will be presented in Section 2.3.

2.3. Inclusion and Exclusion Criteria

The outcomes derived from this literature review relied on the identification of crowd formations, wherein machine learning and object tracking techniques were employed to extract pertinent and substantial information regarding behavior or patterns that deviate from commonly established norms.

Table 2. Inclusive and exclusive criteria.

Decisions	Related to
Inclusive Decisions	Related to people detection
	Related to the detection of objects
	Related to public safety
	Related to behavioral patterns
Exclusive Decisions	Not related to deep learning techniques
	Not related to camera analysis (computer vision)
	Not related to engineering (computer science)

shows the inclusive and exclusive criteria applied to the filter that was obtained with the metasearch engines.

2.4. Selection Results and Data Extraction

The search strings in Section 2.2 show 698 publications between the years 2016 and 2025. Once the 698 results related to the core of our research had been identified, using the criteria shown in Table 2, the ones that most closely matched the desired final objective and that could provide a response to the questions raised in Section 2.1 were reviewed.

The first exclusion criterion that was taken into account (after removing duplicated) was to eliminate all the results that had nothing to do with the area of computer science, which then left 481 publications. Evidently and almost automatically, the first thing the review focused on was the title of the remaining papers, which resulted in a further 261 being discarded, leaving 210. Then, we conducted a detailed review of the conclusions, summaries, and results. Following the latest filtering process, 35 documents were considered to be the closest to our search. Figure 1 shows the discarded papers through the filtering process.

The selected papers highlight the recognition of events such as recognizing interpersonal distance and people density, predicting behavior based on human actions [23], detection of small objects at a low altitude, learning to predict crowd flows [21], early prediction of anomalies in surveillance cameras [24], vision-based systems for security and protection through machine learning techniques [5], learning from anomalous events in crowds [25], learning to prevent crime [26], monitoring of subjects and relationships with many people [22], identification of abnormal behavioral patterns in crowds [22], crowd flow predictions [27], detection of anomalies in both critical and noncritical crowded environments [28,29], and detection of knives or bladed weapons [30,31].

The 35 selected results contain some characteristics that could be useful for a definitive solution to the analysis of the state of the art. These publications cover many different aspects of the study of citizen security, including detecting a massive concentration of people, counting the number of participating entities, detecting patterns of behavior, locating objects that can be used to detect a mass concentration of people, the ability to determine whether the object poses a social danger (knives, firearms, etc.) or whether it is not relevant to public security.

2.5. Temporary Chart of Publications

About the dates of the publications found, Figure 2 shows the number of publications since 2017 concerning crowd analysis, movement patterns, and weapon detection, with the 2023 index indicating an increase at an exponential rate. The decision to use this range of dates is because the interest in this type of technology began to increase from 2017, as well as to limit the number of publications in this paper. Upon examining the graph, it becomes evident that there is a growing interest in the matter.

2.6. Most Representative Keywords

To verify the alignment between the search results and the intended objective of our research,

Table 3. Inclusive and exclusive criteria (keyword filters are highlighted in bold).

Keywords	Count	Keywords	Count	Keywords	Count	Keywords	Count
detection	903	video	569	datasets	534	crowd	512
training	443	prediction	442	network	430	object	349
flow	264	anomaly	259	camera	204	crime	201
ratio	197	abnormal	186	performance	176	count	167
people	160	convolutional	152	framework	144	public	136
distance	128	tracking	114	density	108	recognition	108
deep learning	90	architecture	89	target	86	behavior	80
pattern	76	anomalous	75

presents a compilation of keywords that appeared across all revised papers, with the 35 selected studies after applying filters highlighted in bold. It is evident, as anticipated, that six keywords—detection, video, datasets, crowd, training, and prediction—emerge as the most frequently recurring terms. This finding substantiates the research interest in leveraging computer vision to enable intelligent analysis of crowd gatherings.

3. Findings and Quality Assessment

The obtained papers were thoroughly examined to facilitate their categorization into distinct groups, utilizing both quantitative and qualitative data. Quantitative data pertain to the information derived from experimental procedures, while qualitative data encompass insights drawn from the authors’ conclusions. Based on the conducted analysis, it was determined that the papers could be classified into three interrelated categories. Despite their interconnectedness, these categories exhibit notable distinctions that form a crucial criterion for the accurate assessment of quality.

The initial category can be referred to as “crowd aggregation” or “crowd gathering,” encompassing studies centered around the identification and analysis of typical crowd behaviors. Another category, termed “abnormal aggregation”, specifically focuses on detecting and studying unusual behaviors within crowds. Within the latter category, a further subdivision exists, known as “critical abnormal aggregation”, which pertains to the identification and examination of highly significant abnormal crowd behaviors. These categorizations correspond to the research questions outlined in Table 1.

By conducting this quality assessment, we can classify the primary studies and ascertain the quantity of papers positioned at the forefront of knowledge relevant to our research.

The first category encompasses studies dedicated to examining various aspects of crowd formations, such as people counting, peer relationships, crowd analysis, and distance estimations. In contrast, the second category explores deeper into the research of abnormal behavioral patterns that deviate from what is conventionally considered “normal”. Lastly, the third category introduces an additional risk factor, focusing on studies involving the detection of concealed weapons within crowds or the identification of potentially violent or extreme anomalies.

Figure 3 displays a graph presenting the distribution of studies across the different categories based on their extracted and classified data. It illustrates the percentage composition of each category. Additionally,

Table 4. Categorized primary studies.

Category	Primary Studies
Crowd aggregation	[21,22,25,27,28,32,33,34,35,36,37]
Abnormal crowd aggregation	[20,23,24,25,28,30,31,38,39,40,41,42,43,44,45,46,47]
Critical abnormal aggregation	[26,39,48,49,50]

provides a categorized reference list of the papers.

The identification and categorization of primary studies highlight the balance between research focused on the analysis of incident-free crowd gatherings (42%) and studies investigating anomalous situations (46%). Notably, only a minority of studies (12%) concentrate on critical anomalies, suggesting that the field of hazardous event detection is still in its nascent stage of development. Therefore, there is considerable value in expanding knowledge and understanding of such critical situations.

4. Analysis of the Results

In this section, a discussion about the questions raised in Section 2 of this paper is presented. These questions serve as important guiding points for our research and explore the key aspects related to the detection of abnormal events in large gatherings of people using artificial intelligence methods. By addressing these questions, we aim to gain a deeper understanding of the existing knowledge and explore potential solutions to the identified research gaps. Through a comprehensive analysis and synthesis of the available literature, this section aims to provide valuable insights and contribute to the advancement of the field of public security through AI-based anomaly detection techniques.

4.1. Q1. What Is Considered a Gathering of People?

It is important to consider the concept of gathering as the initial foundation for the start of the research. Because there are no clear definitions provided in the existing literature, we can initiate anomaly detection by considering a minimum group size of 20 individuals as our starting point. Results that provide a head count and indicate that it is a gathering of more than that number (or similar) of individuals will be useful. In addition to that, it is also important to be able to keep track of individuals and the interactions between them. Elias et al. [22] present real-time videos capturing individuals walking in densely populated areas such as squares, shopping malls, and train stations. The study aims to comprehend both crowd behavior and the distinction between individual elements and group entities. This is achieved by enclosing each subject within a frame, characterizing them, and analyzing their behavior based on proximity to others in consecutive video frames. Additionally, they employ the deep high-resolution representation learning neural network [40] to extract information from body poses and joints, enabling the identification of relationships and human interactions. In a similar line, Fitwi et al. [35] estimate interpersonal distance between entities in the dataset and measures the density of the nearest crowd using E-SEC. This approach considers the interpersonal distance between dynamic objects, the occupied area of the crowd, and density. The dataset was captured using CCTV cameras, and the YOLOv3 algorithm was applied for detection and tracking purposes.

It is important to determine which of a multitude of people may correspond to a family nucleus or a couple, as well as the interactions between them and the rest of the group. Understanding the concept of group gatherings, tracking individual elements in the dataset, and analyzing interactions between entities (including differentiation between family and non-family units) are crucial for accurately detecting spontaneous crowd flows that occur within a short timeframe. Moreover, the significance of people congregating can vary based on external factors beyond control, such as holidays that encourage participation in demonstrations or rainy days when individuals prefer to stay indoors. Therefore, predicting crowd behavior using multisource data (including vacation schedules, public holidays, and traffic data) becomes imperative. In [21], the ST-B-ResNet method leverages multisource data and employs a BRBM data reconstruction mechanism for crowd flow analysis.

4.2. Q2. Can AI Recognize What a Gathering of People Is?

Deep learning can be employed to detect gatherings of people by utilizing trained neural networks that can recognize specific patterns in images or videos associated with such groups or situations. This process entails utilizing a dataset comprising images or videos showcasing instances of people congregating, followed by training a neural network that learns from this dataset and can subsequently identify common characteristics of these gatherings.

One possible strategy involves utilizing object detection algorithms like YOLO, which have the ability to detect and track objects in video sequences. By training a model using YOLO on a collection of image and video datasets containing people congregations, the model can learn to identify specific patterns within these groups. For instance, the algorithm can be trained to detect clusters of people in close proximity or individuals looking toward a shared point of interest.

An alternative method involves employing a convolutional neural network (CNN) to analyze images or videos depicting gatherings of people, enabling it to learn and identify behavioral patterns based on the visual features extracted from these visual inputs. For instance, the CNN can be trained to recognize distinct body shapes, sizes, and the spatial relationships among individuals within the scenes.

Upon training the network, it serves as a foundation for detecting people gatherings in new videos or images that were not included in the training datasets. This detection capability holds potential for numerous applications, including crowd behavior management, CCTV surveillance, and tracking, as well as social distancing monitoring.

4.3. Q3. How Can We Detect with Videos the Difference Between a Peaceful and a Non-Peaceful Gathering?

For those occasions when a gathering may lead to violent actions, the system should be able to predict and even anticipate them. Once the need for the number of participating individuals is covered, whether the behavior of that crowd of people corresponds to something that could be called normal, or whether, on the contrary, actions outside the norm are evident. The study of foreground people’s actions allows for the establishment of a learning model [29] that helps to determine whether a behavior is anomalous or not, highlighting those behaviors that represent a prediction error. The work of Nauman and Shoaib [38] mainly focuses on detecting anomalies within a crowd of people, fusing a Mask R-CNN convolutional network with Res-Net-101 as the main architecture. These studies are useful when analyzing actions in a gathering of people, but they do not consider the combination of other determining factors, nor whether a bladed weapon or a blunt object can be found in the crowd. In these cases, it is necessary to use deep learning to detect other important factors that are not taken into account in the previously mentioned papers. Recent methods such as MSMC-Net utilize multiscale spatiotemporal attention to detect crowd-level anomalies like turbulence and counterflow, enabling real-time risk scoring without tracking individuals [49].

Table 5. Significant datasets for crowd analysis.

Dataset	Type	Nº Videos/Images	Method	Model
UCF-Crime [51]	Surveillance	2047	Abnormal event detection	Autoencoders, CNN, 3D-VAE, transfer learning.
UCF-Crime2Local [52]	Surveillance	13,662	Abnormal event detection	CNN, Spatiotemporal Autoencoder, transfer learning.
CrowdHuman [53]	Crowd	15,000 images, 5000 annotated	Pedestrian detection, multi-person tracking	Faster R-CNN, RetinaNet, YOLOv3, Cascade R-CNN
PETS2009 [54]	Surveillance	24,259	Object Tracking	Multiple Object Tracking (MOT), R-CNN, DeepSORT
UMN Dataset [55]	Surveillance	1200	Object detection	Faster R-CNN, YOLOv3, SSD, Mask R-CNN
ShanghaiTech [56]	Crowd	1119	Crowd counting	CNNs, DCNN, MCNN, FCN
UCSD [57]	Pedestrian	Over 2000	Anomaly detection	CNN, VAEs, GANs, Transfer learning
RWF-2000 [58]	Surveillance	2000	Abnormal event detection	LSTM, CNN, Reinforcement Learning
PublicVision [43]	Surveillance crowd	1413 videos	Behavior and violence-level recognition	Swin Transformer

presents several significant datasets commonly used in the papers from Section 2.2. To clarify the types of anomalous situations commonly addressed in the literature, two representative examples are described below:

• Example 1: Movement in the Opposite Direction to the Crowd Flow: A typical anomaly in video surveillance scenarios involves detecting an individual walking in the opposite direction to the prevailing movement of the crowd. While this behavior does not necessarily imply violence, it does constitute a significant deviation from the usual collective dynamics. Such anomalies may indicate potential risk situations, such as attempted escapes, suspicious behavior, or the possible onset of an incident.
• Example 2: Sudden Transition from Normal to Violent Behavior: Another paradigmatic situation involves a group of people who, initially, are behaving in a completely peaceful or routine manner. However, at a given moment, two individuals begin to fight or engage in physical aggression. These events highlight the importance of automated systems being able not only to identify atypical patterns but also to detect, at an early stage, the transition between normal behaviors and scenarios that may become dangerous.

Both examples underscore the need for artificial intelligence-based solutions capable of efficiently anticipating or detecting both subtle deviations in group behavior and the emergence of imminent risk situations for public safety.

4.4. Q4. What Are the Different Types of Deep Learning Networks and Architectures Used for Abnormal Event and Gathering Detection?

Deep learning can be employed to detect anomalous events in crowd gatherings by training neural networks to recognize patterns that deviate from normal behavior. To achieve this differentiation, two distinct datasets are utilized: one representing “normal behavior” and the other representing “anomalous behavior.” The neural network is trained using these datasets to accurately identify and distinguish such deviations. For example, the ACSAM model uses CNNs with abnormality-aware training layers designed for dense crowds, improving accuracy by over 12% compared to previous deep learning methods [46].

One potential approach is to employ anomaly detection algorithms, such as autoencoders or GANs, to recognize patterns that diverge from the anticipated behavior of a group captured in video datasets. For instance, an autoencoder can be trained using a dataset of regular occurrences (e.g., individuals walking in a public area) and subsequently leverage this training to detect events that deviate from the norm (e.g., an individual running or a sudden convergence of a group towards a specific location). Similarly, GANs can also be utilized to generate images depicting regular occurrences and subsequently identify images that deviate from the anticipated pattern.

Another approach involves employing a neural network, specifically a CNN, to classify events as either normal or abnormal. This method entails training the CNN using a labeled dataset of images or videos that indicate the normality of each event. Subsequently, the trained CNN can be applied to analyze new images or videos and classify them accordingly based on these two models.

In both instances, continuous training with new data enables the neural network to undergo refinements, enabling it to adapt to evolving behavior patterns and enhance its predictive capabilities in detecting anomalies. The insights gained from this detection process hold applicability in diverse domains, including public safety and the management of crowds within public or private spaces.

A notable innovation is the PublicVision system, which integrates a Swin Transformer to classify crowd behavior by size and violence level, while securing surveillance data using encrypted VPNs [43].

4.5. Q5. What Are the Performance and Accuracy of These Methods on Public Security Datasets?

The performance and accuracy of the previously discussed CNNs and autoencoders can vary due to several factors, including the dataset’s size and quality, the complexity of the addressed problem, the chosen model architecture, and the evaluation metrics employed to gauge performance. Although CNNs and GANs are frequently employed in the domain of public safety, each exhibits distinct strengths. CNNs excel in image segmentation, classification, and object detection, making them highly effective in such tasks. On the other hand, GANs are predominantly utilized for synthetic image generation and enhancing low-resolution images. CNNs and GANs have demonstrated significant advancements in performance and accuracy when applied to public safety datasets. GANs have been successfully employed to generate highly realistic facial images, contributing to the advancement of facial recognition systems.

Autoencoders have found extensive utility in a range of public safety applications, including anomaly detection, surveillance, and image and video analysis. Nonetheless, the performance and accuracy of autoencoders when applied to public safety datasets can fluctuate based on factors such as the specific application, dataset quality and size, and the design of the autoencoder model.

Autoencoders have demonstrated encouraging outcomes across diverse public safety applications. Particularly in anomaly detection, they have been employed to detect and characterize aberrant behavior or events in surveillance videos or sensor data. The accuracy of the anomaly detection system hinges on the quality of the input data as well as the design of the autoencoder.

Autoencoders have found utility in various image and video analysis tasks, including compression, reconstruction, and super-resolution. They possess the capability to acquire a compressed representation of the input data through training and subsequently employ this representation to reconstruct the original image or video.

Nevertheless, it is crucial to acknowledge that the performance and accuracy of autoencoders in public safety datasets can be impacted by ethical and legal considerations associated with the utilization of these technologies.

Specifically in anomaly detection, deep models have been used to identify and characterize aberrant behaviors or events. One such model, based on an Optimized Deep Maxout architecture, achieved 97.28% accuracy in anomaly detection using evolutionary optimization techniques [45]. Another enhanced SlowFast model with attention and dropout mechanisms detects five public-space behaviors with up to 40.5 FPS, meeting real-time needs [44].

A practical implementation using YOLOv5 combined with Twilio API enabled the real-time detection of overcrowding and alert dispatch in educational and public facilities, demonstrating strong applicability in live environments [47].

4.6. Q6. What Key Elements of the Datasets Can Provide Evidence of the Outcome of the Gathering, and Which Datasets Are Most Commonly Used to Monitor Gatherings Using Deep Learning?

These studies are useful when analyzing actions in a gathering of people, but they do not consider the combination of other determining factors, nor whether a bladed weapon or blunt object can be found in the crowd. In these cases, it is necessary to use deep learning to detect other important factors that are not taken into account in the previously mentioned papers.

To efficiently conduct computer vision-based monitoring of a crowd, it is essential to utilize datasets that offer ample information and learning capabilities, enabling us to achieve conclusive outcomes in our monitoring objectives.

Deep CNNs have been trained to estimate kinetic energy and entropy from simulated evacuations, capturing crowd macro-behavioral shifts useful for anomaly detection [42]. Group-level feature learning has also enabled crowd behavior reconstruction without tracking individuals, enhancing flexibility and realism in simulated datasets [59]. Hybrid models combining deep learning with enhanced social force simulations enable realistic crowd behavior modeling with fewer collisions and better handling of high-density scenarios [50].

Table 6. Categorized primary studies.

Method	Application	Metric	Value	Reference
Optimized deep maxout	Anomaly detection	Accuracy	97.28%	[45]
Enhanced SlowFast	Detection of 5 behaviors in public spaces	Processing speed (FPS)	40.5 FPS	[44]
YOLOv5 + Twilio API	Overcrowding detection	Real-time response (latency)	Not defined	[47]
ACSAM	Anomaly detection in dense crowds	Accuracy improvement over previous methods	>12%	[46]
PublicVision (Swin Transformer + encrypted VPN)	Crowd behavior classification	Global accuracy Mean Average Precision (mAP) Inference latency	89.76% 93.3% ∼20 frames/inference	[43]

presents some of the significant datasets commonly employed in the papers retrieved from Section 2.2.

5. Discussion

Following the analysis of the potential answers in Section 2, a concise table was derived in Section 4, featuring selected papers that highlight distinct technologies, datasets, and objectives.

Table 7. Summary of different techniques in video analysis.

Title	Objective	Model and Method	Tool	Dataset	Site
[33]	Determine the stillness state in every different place and situation.	RNN. Stillness level model and the leaky bucket model (LBM)	Not defined	PETS2009	Outdoor and indoor
[34]	Generate an alarm for the security personnel to take appropriate actions	DADCG algorithm. MHI and Optical Flow techniques.	CVST—Matlab R2012a	YouTube	Outdoor and indoor
[25]	Reduce over-fitting because of scarcity of data.	CNN: C3D, GAN, N3D, ResNet	SHADE	UCSD, UCF-Crime, UMN	Outdoor
[26]	Develop a unified model to predict crime, leveraging latent relationships from social media	Smote-TomekLinks iterative, 1D-CNN	Not defined	Twitter, 4-month city-wide dataset	Outdoor
[28]	Detection of anomalies in real-world surveillance sites	CNN MobileNet	Not defined	RGB+D, UCF-Crime2Local	Outdoor and indoor
[60]	Detect violent actions on streets in urban spaces	CNN	A conventional laptop, VLC tool	Created by the authors	Outdoor and indoor
[31]	Develop two novel weapon detectors applying deep learning	Faster R-CNN, RPN	Not defined	COCO 2017, Gupta dataset, Open Images dataset	Outdoor and indoor

provides a summarized overview of the various computer vision techniques employed.

Upon acquiring these data, the identification of a solution becomes feasible, leading to the proposal of three potential solutions to address our questions on gatherings.

The problem definition would focus on “what is happening” in an agglomeration of people in front of a given event, and whether there is a relationship between the actions and that event. The system should behave in the same way as a human being, after observing a certain behavior on a monitor. It has to be able to realize that something anomalous is happening.

Table 8. Quantitative results of crowd analysis models across various categories.

Category	Sub-Category	Accuracy
Crowd aggregation	Liquor law violation	59.8% [26]
	Narcotics	76.8% [26]
	Prostitution	86.9% [26]
	Deceptive practice	71.0% [26]
	Grappling	95.0% [60]
	Arrest	63.5% [25]
	Chase	59.1% [25]
	Run	63.2% [25]
Abnormal crowd aggregation	Robbery	77.3% [26]
	Theft	74.7% [26]
	Assault	75.1% [26]
	Punching	92.0% [60]
	Kicking	93.0% [60]
	Fight	55.1% [25]
	Knife Detection	85.44% [31]
Critical abnormal aggregation	Homicide	70.0% [26]
	Gun Detection	46.68% [31]
	Shoot	83.2% [25]
	Strangulation	90.0% [60]
	Kidnapping	52.1% [26]
	Arson	54.6% [26]
	Weapons violation	77.3% [26]

presents the accuracy results of the selected papers in different scenarios.

The current literature contains other methods capable of detecting anomalies in a video sequence with high precision. These methods can, for example, detect if a vehicle is in an area designated for pedestrians only or if people are running in typically calm areas. However, these methods lack the ability to interpret whether there is violence or the potential for an altercation in the area.

Table 9. Performance of anomaly detection methods across various datasets.

UCF-Crime		UCSD		ShanghaiTech		RWF-2000
Method	AUC (%)	Method	AUC (%)	Method	AUC (%)	Method	Accuracy
[61]	87.24	[62]	99.7	[63]	87.72	[64]	93.4
[65]	86.98	[66]	98.9	[67]	85.94	[68]	90.4
[69]	85.99	[70]	98.7	[71]	85.9	[72]	90.25
[73]	85.38	–	–	[74]	84.3	[75]	89.75

presents a summary of the leading state-of-the-art techniques for detecting anomalous events in images and videos.

In addition to algorithmic performance, the reliability of abnormal behavior detection systems also depends on the physical configuration of the surveillance infrastructure. Camera planning is thus a critical factor in ensuring the effectiveness of AI-based monitoring systems. A strategic deployment and configuration of a camera network maximizes the visual coverage of urban areas, intersections, and public squares, minimizing blind spots and optimizing the capture of unusual behaviors. Indeed, camera planning is considered the first step when designing surveillance networks for public safety applications [76]. Networked cameras, when interconnected and often centrally managed, can collaborate to detect anomalous events more robustly than isolated units. A well-planned camera network facilitates continuous tracking of individuals or crowds across different zones, enhancing the detection of suspicious activities or unauthorized gatherings from multiple viewpoints [77,78].

A careful network design must consider environmental characteristics (urban versus rural, open versus narrow spaces) and the specific surveillance objectives. Recent research highlights that coverage requirements and optimization algorithms for camera placement must be tailored to the type of scene being monitored—for example, busy urban areas require different strategies than rural settings [79]. In deep learning-based anomaly detection systems, it becomes essential to plan the location, angle, and range of cameras to ensure that relevant events are not missed. Proper multi-camera coverage not only increases the likelihood of capturing unusual activity but also reduces false positives by corroborating incidents from different perspectives [80,81].

Modern surveillance systems are further enhanced by smart cameras equipped with embedded analytics that can operate in coordination [82]. These edge-computing cameras are capable of detecting objects and behavioral patterns in real time and communicating with other nodes in the network. The recent literature underscores the value of such intelligent multi-camera systems, from multi-camera surveillance reviews to scene-aware planning strategies adapted to different geographic contexts [83,84]. Optimal camera planning—supported by networked cameras and computer vision algorithms—forms a critical technological foundation for reliable anomaly detection in public environments. This integration enhances not only accuracy but also the responsiveness of the entire surveillance system, ensuring a more effective early warning and mitigation framework.

Another important aspect to discuss is that, despite recent progress, deep learning-based systems for abnormal event detection still face significant limitations due to the lack of representative datasets. Most current models are trained on traditional datasets such as UCSD, CUHK Avenue, or UCF-Crime, which do not adequately reflect the diversity of real-world surveillance scenarios. There is a growing need for datasets that encompass multiple environments (e.g., streets, transit stations, and campuses), varied camera perspectives (e.g., fixed, mobile, and PTZ), and different environmental conditions. Recent studies emphasize that the quality and diversity of datasets directly influence model performance and that the absence of comprehensive training data remains a critical barrier to effective anomaly detection [85,86]. To address this, new benchmarks such as the NWPU Campus dataset have been proposed, offering extensive multi-scenario coverage, diverse viewpoints, and a rich taxonomy of anomalous and normal behaviors, significantly advancing model generalization capabilities [87]. Additionally, enhancing dataset annotations has become essential [88]. While many existing datasets adopt unsupervised or weakly supervised formulations, future datasets should provide richer labels, such as temporal markers, spatial localization, and anomaly type classification (e.g., violence, accidents, unauthorized gatherings, fires, or structural failures). Including both human-related and environmental anomalies would help reflect the full spectrum of real-world public safety scenarios.

In recent years, the field of abnormal behavior detection in video using deep learning has seen significant growth across multiple countries. According to a literature review considering peer-reviewed scientific publications restricted to articles published in indexed journals and top-tier international conferences, China has emerged as a global leader in this domain, with a high volume of publications focused on anomaly detection in surveillance footage, supported by institutions like the Chinese Academy of Sciences, Shanghai Jiao Tong University, Northwestern Polytechnical University, and Tsinghua University. The United States follows closely, with prominent contributions from universities such as the University of Central Florida, Carnegie Mellon University, and the University of California system, particularly in integrating spatiotemporal modeling and generative learning approaches. Other countries with increasing research output include India, with active participation from the Indian Institutes of Technology, which has ranked second in terms of the number of publications in recent years. South Korea also stands out, especially through the Universities of Sejong and Yonsei.

European institutions also play a central role in advancing this field. Universities like the University of Amsterdam, the Technical University of Munich, and ETH Zurich have published influential work in unsupervised learning, vision–language integration, and multi-view anomaly detection. In the United Kingdom, the University of Oxford and Imperial College London have contributed methods focused on real-time detection and privacy-aware video analytics. Overall, research efforts are geographically diverse but consistently emphasize deep learning-based methods, including hybrid and self-supervised approaches, tailored to real-world surveillance scenarios.

Finally, scientific advancements in video anomaly detection have rapidly translated into practical applications within the electronic security industry. Several companies now offer intelligent surveillance cameras equipped with embedded deep learning analytics capable of autonomously detecting abnormal behaviors. A notable example is Avigilon (https://docs.avigilon.com, accessed on 29 June 2025) (Motorola Solutions), which has introduced Unusual Motion Detection (UMD)—a system that continuously learns the typical activity in a monitored scene without relying on predefined rules. By highlighting unusual movements, the system allows security operators to quickly review large volumes of footage and focus on potentially critical incidents. Axis Communications (https://www.axis.com, accessed on 29 June 2025) has also integrated deep learning capabilities into its latest camera models. With dedicated deep learning processing units (DLPUs), Axis cameras can detect anomalies directly at the edge without the need for centralized servers, enabling scalable, real-time surveillance with reduced infrastructure demands.

In the global market, leading Chinese companies such as Hikvision (https://www.hikvision.com/, accessed on 29 June 2025) and Dahua (https://www.dahuasecurity.com, accessed on 29 June 2025) have incorporated AI-driven behavioral analytics into their product lines. Hikvision, for instance, offers real-time abnormal pattern detection through its HikCentral platform, particularly suited for high-traffic environments like airports or stadiums. These systems can automatically raise alerts upon detecting unusual crowd dynamics, such as sudden gatherings or erratic movement. In parallel, software-focused companies like Scylla AI (https://www.scylla.ai, accessed on 29 June 2025) have emerged, offering plug-and-play solutions that enhance existing camera systems with real-time behavior recognition. Their platforms can identify a range of abnormal events—including violence, theft, or vandalism—across multiple video streams simultaneously. These developments demonstrate how deep learning is empowering surveillance systems to move from passive recording to proactive, intelligent monitoring, aligning commercial tools with the cutting-edge techniques explored in academic research.

6. Conclusions

This paper provides a review of methods and techniques that use computer vision and machine learning to determine a gathering of people and analyze abnormal events. The findings reveal that while there are various approaches in the literature, deep learning-based systems are gaining prominence due to their ability to learn complex patterns and achieve higher accuracy in detecting gatherings and analyzing behavior. While some works address the problem individually using classical computer vision methods, the adoption of deep learning techniques offers significant advancements in detecting and understanding gatherings of people. Deep learning models, with their capacity to automatically extract relevant features from large-scale datasets, have shown superior performance in detecting abnormal events and behaviors. However, despite the progress made, there is a notable gap in the literature when it comes to a comprehensive approach that addresses the entire pipeline of gathering detection. While some studies focus on detecting normal behavior in gatherings using one-class learning systems, there is still a need for further research on the comprehensive detection and combination of abnormalities. This entails not only identifying anomalous behaviors but also classifying them and understanding their implications for public security. Furthermore, the utilization of different types of devices, including both mobile and static ones, for gathering detection presents an important avenue for future scientific work. By leveraging the capabilities of mobile devices such as smartphones and static devices like surveillance cameras, it becomes possible to enhance the coverage and effectiveness of gathering detection systems.