Enhancing Weakly Supervised Video Anomaly Detection with Object-Centric Features

Abstract

Surveillance cameras are extensively deployed across public and private environments, driving the need for intelligent video monitoring systems. However, a major challenge arises in Weakly Supervised Video Anomaly Detection (WSVAD), where supervision is limited to video-level labels, making snippet-level anomaly localisation particularly difficult. This challenge is often formulated as a Multiple Instance Learning (MIL) problem. Although recent approaches have achieved encouraging results by modelling spatio-temporal dynamics, they often overlook the semantic information within videos that could further enhance anomaly detection. To bridge this gap, we propose enriching feature representations by applying object detection techniques to extract object-centric features. These features provide supplementary high-level semantic information that supports the discrimination of anomalous events. Experiments conducted on two benchmark datasets, UCF-Crime and ShanghaiTech, demonstrate that our approach achieves performance comparable to state-of-the-art (SOTA) methods. The results highlight that incorporating object-level semantics offers a promising direction for improving WSVAD, underscoring the potential of semantic-aware approaches for more effective anomaly detection.

1. Introduction

Anomaly detection (AD) is a fundamental research topic that focuses on identifying data samples that deviate from normal behavioural patterns or expected distributions [1]. Due to growing demands across diverse domains, anomaly detection plays an important role in various application areas such as cybersecurity [2], financial surveillance [3], healthcare and medical realm [4], and UAV-based surveillance and monitoring [5]. Among them, one of the most critical applications lies in video surveillance, where it can be used to identify irregular or hazardous events from surveillance videos, e.g., robbery, shooting, and shoplifting, to prevent social harm and ensure public safety.

Nevertheless, detecting anomalies in long and complex surveillance videos remains extremely challenging [6] due to high labour costs, low annotation precision and poor data quality. To address these issues, weakly supervised video anomaly detection (WSVAD) has been proposed as a promising alternative.

A representative example of WSVAD is the Robust Temporal Feature Magnitude (RTFM) framework proposed by Tian et al. [7]. It employs pre-trained Inflated 3D ConvNets features [8] and learns feature magnitude based on top-k instances. Chen et al. [9] further advanced this concept by designing a Magnitude-Contrastive Glance-and-Focus Network (MGFN) to better capture spatial–temporal dependencies. However, a notable limitation is that the feature magnitude is not only affected by abnormality, but also by other factors such as object motion and object count. Consequently, simply pushing normal features away in the feature space may not accurately reflect anomaly boundaries. This observation underscores the importance of incorporating object-level information into the network, as object dynamics and interactions convey rich video semantics that are crucial for improving anomaly detection performance.

In view of the above challenge, some previous studies leveraged optical flow features to capture motion information, which can indirectly reflect object-level dynamics. However, such approaches typically yield suboptimal results [1] and double the feature dimensionality, thereby complicating the training process.

To overcome these limitations, modern object detectors such as YOLO [10,11] are adopted as a practical and efficient solution for generating supplementary object-level features. Unlike pre-computed video features, detection outputs inherently encode additional rich semantic cues, including object categories, motion patterns, and object count, which are highly informative for anomaly understanding. Instead of designing and training additional networks to capture object-related contextual information, mature object detectors are applied directly to the original videos to generate the supplementary object-centric representations. Compared with optical flow, object detection offers lower feature dimensionality and achieves superior detection performance. To overcome the issue that YOLO’s native outputs, which consist of discrete class IDs and bounding box coordinates, are not ideal for direct network training, we instead extract the class probability distributions from the penultimate layer. These distributions are then fused with the corresponding positional information to form a compact but semantically informative feature representation. These object-level features are subsequently fused with pre-trained video representations obtained from the Inflated 3D ConvNet (I3D) model [8], yielding richer contextual cues while effectively reducing detection errors.

The contributions in this work are summarised as follows:

• The training network is effectively and efficiently enabled to incorporate object-level information, such as object motion and count, by combining the base model with supplementary object detection features. Furthermore, it can be easily adapted to any existing object detector and holds significant potential for future applications;
• A new feature format has been designed to adapt the direct YOLO outputs to the proposed framework, to ensure they can be better concatenated with the existing pre-trained features (I3D);
• Attention-based mechanisms have been incorporated to capture the contextual information and enhance the quality of the extracted object features, resulting in improved performance compared with previous methods;
• Experiments have been conducted on two benchmark datasets: ShanghaiTech [12] and UCF-Crime [13]. The proposed method achieves better results on both datasets than the baseline model, offering a promising alternative for tackling weakly supervised video anomaly detection tasks that previous methods have not fully explored.

This paper is structured as follows. Section 2 reviews recent research in both video anomaly detection and object detection. Section 3 presents the structure and implementation details of the proposed method. Section 4 introduces the dataset preparation and experimental setup, followed by an analysis of the results. Finally, Section 5 concludes the paper by summarising the key ideas and contributions.

2. Related Works

2.1. Video Anomaly Detection

Supervised anomaly detection [14] generally achieves the best performance across different categories, as it benefits from precise frame-level annotations that provide strong supervision during training. However, it is less commonly studied for complex or large-scale datasets due to the high cost of detailed annotation and the susceptibility to human error, which makes it difficult to ensure high-quality labels. In contrast, unsupervised Anomaly detection methods [15,16,17,18,19,20,21] eliminate the need for expensive manual annotation but often perform suboptimally because they lack clearly defined anomalies and ground-truth supervision for training.

Between these two extremes, weakly supervised learning offers a balanced trade-off between accuracy and annotation cost. By relying solely on coarse video-level labels rather than detailed frame-level annotations, weakly supervised methods significantly reduce the burden of manual labelling while still retaining enough supervisory signal for effective model training. The mainstream WSVAD approaches are primarily based on the MIL framework [13], in which a video (or a set of snippets) is treated as a “bag”, containing multiple “instances” (snippets or frames), and instance-level anomaly patterns are learned using only bag-level video annotations. The objective is to assign higher anomaly scores to abnormal snippets within abnormal videos, while reducing the scores of normal snippets [22]. Sultani et al. [13] initiated this direction by introducing the large-scale UCF-Crime dataset and a MIL-based framework using video-level labels to localise abnormal segments, sparking significant research interest. However, this MIL-based formulation introduces label noise that normal snippets may be incorrectly treated as highly abnormal in an abnormal video, which can degrade the learning process. To eliminate the noise, Zhong et al. [23] adopted a graph convolutional neural network (GCN) and formulated the problem as a supervised learning task under noisy labels, aiming to reduce the influence of incorrectly labelled snippets. While GCN can explicitly model relationships among snippets, joint training with the MIL framework is computationally costly. It can lead to an unconstrained latent space that can cause unstable performance. Building upon these foundations, Wan et al. [24] introduced the Anomaly Regression Network (AR-Net) to learn more discriminative features and improve anomaly detection accuracy to a practical level. Tian et al. [7] further advanced the field with the robust Temporal Feature Magnitude (RTFM) network, which leverages Inflated 3D ConvNets [8] for feature extraction and employs a temporal network to model spatio-temporal dependencies, along with a tailored loss function that encourages separation between normal and abnormal patterns. Following the RTFM structure, Chen et al. [9] observed that a loss function which simply encourages abnormal features to have larger magnitudes than normal ones is fundamentally unreasonable, as this assumption contradicts the inherent magnitude distribution across different videos. To address this, a Magnitude-Contrastive Glance-and-Focus Network (MGFN) was proposed, which introduces a magnitude-contrastive loss and performs a global scan over the video while selectively focusing on specific portions for finer analysis, thereby achieving performance improvements and demonstrating the effectiveness of integrating global and local feature reasoning.

Although these approaches have achieved effective progress, most rely on pre-computed features extracted from models such as I3D [8] or C3D. These features, while effective for general video representation, often lack semantic richness and fail to capture object-level context that is essential for understanding anomalies. To address this limitation, recent progress in object detection offers a promising solution. Modern object detectors achieve high accuracy and real-time performance, making them practical for extracting supplementary features that provide richer semantic information. Building on these strengths, we adopt a weakly supervised learning approach as a balanced compromise between annotation cost and detection performance, enhanced with object-level features to better capture contextual information and improve network learning.

2.2. Object Detection

Object detection aims to develop computational models capable of identifying what objects are present and where they are located within an image or video frame, forming a foundation for many computer vision applications [25,26], including surveillance systems [27]. Among existing approaches, the YOLO family represents the mainstream in real-time object detection [28].

The original YOLO was introduced by Redmon et al. in 2015 [29] as the first one-stage detector in the deep learning era, achieving a significant improvement in detection speed. Since then, the YOLO series has evolved rapidly, each iteration addressing prior limitations and enhancing detection performance. YOLOv7 [10], developed by the same author of YOLOv4 and YOLOR, introduced a trainable bag-of-freebies method that allows the model to outperform all known detectors at that time in both speed and accuracy across various frame rates (5–160 FPS). Impressed by its superior detection accuracy and fast inference speed, YOLOv7 is adopted as one of the object detectors in this study. YOLOv9 [11] further improved upon earlier versions by addressing the issue of information loss during feature extraction and spatial transformation. It introduced Programmable Gradient Information (PGI) for more reliable gradient propagation and a lightweight Generalised Efficient Layer Aggregation Network (GELAN) architecture that improves parameter utilisation using standard convolution operations. As YOLOv9 was built based on YOLOv7 and developed by the same author, jointly evaluating them provides a clearer assessment of their relative performance.

Since object detectors inherently capture motion cues, they can provide valuable supplementary information for anomaly detection, as sudden or unusual movements often signal anomalous events. However, traditional object detectors primarily output bounding boxes, which alone are insufficient as high-level features for WSVAD. Hence, a key research gap remains in transforming these detection outputs into rich, semantically meaningful representations suitable for anomaly analysis. This work aims to bridge that gap by developing strategies to extract and integrate object-centric features that enhance WSVAD.

3. Proposed Methods

Figure 1 shows the proposed framework’s architectural design, which equips the network with the capacity to incorporate object information adequately and effectively exploit these features. This integration is crucial as it significantly impacts the anomaly scores.

Given a weakly labelled video set V, where each video is associated with a binary label $y\in \{0,1\}$ indicating normal or abnormal, the video is first divided into T snippets, which are then independently processed by a feature extractor and an object detector. The feature extractor produces representations denoted as $F_{i3d}$, while the object detector produces YOLO label data, which is then used for feature generation to produce $F_{yolo}$. Both $F_{i3d}$ and $F_{yolo}$ are structured as ${\mathbb{R}}^{T\times C\times D}$, where T is the number of temporal snippets per video, C is the number of spatial crops per snippet, and D denotes the feature dimension.

Following feature adaptation and concatenation, the resulting representations are fed into a classifier that produces anomaly scores for each snippet. These scores are then passed through a Top-K magnitude selection mechanism, which identifies the top-k snippets based on their highest feature magnitudes. This mechanism ensures that the most informative snippets with the highest anomaly potential are prioritised for further analysis. Network optimisation is driven by the feature magnitude learning of the selected Top-K snippets, along with a binary cross-entropy loss, enabling the model to effectively maximise the separability between the normal and abnormal videos.

3.1. Generation of Additional Features

Object detectors are employed to generate the feature representation $F_{yolo}$. The raw detection output produced by YOLO includes object information such as class IDs, bounding box coordinates, and bounding box dimensions (width & height). To ensure the features match the desired dimension $F_{yolo}\in {\mathbb{R}}^{T\times C\times D}$ for subsequent concatenation, a conversion strategy is necessary. Specifically, to match the number of snippets T and ensure consistency with the shape of pre-trained I3D features, object detection is performed on the first frame of every K-frame snippet, which is treated as the key frame. Given the high frames per second (fps) of surveillance videos, the temporal differences between adjacent frames are minimal. Therefore, using only the first frame of each snippet is sufficient to capture anomalies, reducing computational overhead and overall processing time. To match the required number of crops C, the extracted feature is duplicated C times to ensure dimensional consistency.

During the object detection process, a confidence threshold is set to filter out the less reliable results, ensuring that only objects detected with sufficient reliability are included in the textual output. However, focusing solely on the class category with the highest class confidence from the reliable object may result in limited and potentially inaccurate information, especially when the threshold value is low. Hence, to further incorporate the confidence levels of all classes, the final classification step of YOLO that produces discrete class IDs is bypassed, and the class probability distributions from the preceding stage are used as the desired features. As the object detector is trained to recognise N classes, these distributions provide N probabilities for each frame.

To incorporate the bounding box coordinates and dimensions, which are also known as object position information, each input frame is uniformly divided into P patches, with each patch capable of representing N class probabilities. Consequently, the final feature dimension of the $F_{yolo}$ becomes $D=P\times N$, as illustrated in Figure 2, and the feature is denoted as $F_{yolo}\in {\mathbb{R}}^{T\times D}$. Since each detected object is associated with a bounding box, its spatial overlap with each patch is computed. The ratio of the overlapping area to the total bounding box area is then calculated. If this ratio exceeds a predefined threshold, the corresponding patch is assigned the N-dimensional class probability vector of the detected object, as shown in Equation (1). This approach enables each patch to encode class information for one or more objects, allowing for multiple objects to be represented within the same patch if overlaps occur.

$$\mathrm{IoU}={\displaystyle \frac{\mathrm{Area}(\mathrm{Patch}\cap \mathrm{BoundingBox})}{\mathrm{Area}\left(\mathrm{Boundingbox}\right)}}>R_{threshold}$$

(1)

3.2. Feature Pre-Processing

After obtaining $F_{i3d}$ and $F_{yolo}$, a multi-scale temporal network [7] is applied to $F_{i3d}$, while feature pre-processing is performed on $F_{yolo}$. Feature pre-processing is required to be performed before concatenation to transform distinct feature sets into a compatible representation. This will facilitate more effective feature fusion and enhance overall training. Although no single pre-processing method consistently outperforms others, all tested approaches yield better results than omitting pre-processing altogether, which will be listed in Section 4. To minimise the computational overhead introduced by additional features, lightweight pre-processing techniques are preferentially adopted.

Inspired by the squeeze-and-excitation network (SE-Net) [30], we employ a lightweight channel recalibration mechanism to enhance the network’s representational capacity by capturing channel-wise dependencies at minimal computational cost, shown in Figure 3.

The generated additional feature $F_{yolo}$ undergoes a global average pooling operation, serving as the squeeze step to embed global contextual information. Unlike the original SE-Net, which operates on spatial feature maps, $F_{yolo}\in {\mathbb{R}}^{T\times D}$ contains only temporal and channel dimensions, without spatial height and width. Nevertheless, the temporal dimension T can be analogously treated in the squeeze-and-excitation framework to achieve a similar recalibration effect. The squeeze operation can be formulated using Equation (2):

$$F_{yolo}^{sq}={\displaystyle \frac{1}{T}}\sum _{i=1}^T{F}_{yolo}(i,:),\mathrm{with}{F}_{yolo}\in {\mathbb{R}}^{T\times D},F_{yolo}^{sq}\in {\mathbb{R}}^D$$

(2)

Once the squeeze is done, two simple fully connected (FC) layers are applied to $F_{yolo}^{sq}$, with a ReLU activation after the first FC layer and a sigmoid activation after the second. The resulting excitation vector is then used to re-weigh the original feature through channel-wise multiplication across the temporal dimension, yielding the final recalibrated representation.

Another pre-processing layer explored in this work is the Hydra attention module [31]. It is an efficient general attention module with multi-heads, which increases the number of attention heads and aligns them with the number of features, while reordering operations within the linear attention framework to maintain constant computational cost.

3.3. Fusion of Features

After pre-processing, the pre-trained feature $F_{i3d}$ is transformed into $X_{i3d}$ via the mapping $s_{\theta }:{\mathcal{F}}_{i3d}⟶{\mathcal{X}}_{i3d}$, which is implemented by the multi-scale temporal network. Similarly, the object detection feature $F_{yolo}$ is transformed into $X_{yolo}$ through $s_{\varphi }:{\mathcal{F}}_{yolo}⟶{\mathcal{X}}_{yolo}$, which is realised by the feature pre-processing blocks. With both representations prepared, feature fusion is performed through direct concatenation, the simplest method of merging since the two features share the same shape along the temporal and crop dimensions. The resulting feature is denoted as $X=\mathrm{Concat}(X_{i3d},X_{yolo})$, where $X\in {\mathbb{R}}^{T\times C\times D}$ and $D=D_{i3d}+D_{yolo}$.

3.4. Training Phase

After processing $X_{i3d}$ and $X_{yolo}$ and concatenating the resulting features into X, model training begins.

The training process involves the joint optimisation of feature magnitude learning loss and a binary cross-entropy (BCE)-based classification loss under the Multiple Instance Learning (MIL) framework adopted from the baseline work [7]. The corresponding loss functions are presented in Equations (3) and (4).

$$L_{\mathrm{mag}}={\mathbb{E}}_i\left[{\left(m-\left({∥{\displaystyle \frac{1}{k}}\sum _{t\in {\Omega }_k\left(X_{\mathrm{abn}}^i\right)}{X}_{\mathrm{abn}}^i∥}_2-{∥{\displaystyle \frac{1}{k}}\sum _{t\in {\Omega }_k\left(X_{\mathrm{nor}}^i\right)}{X}_{\mathrm{nor}}^i∥}_2\right)\right)}^2\right]$$

(3)

$$L_{\mathrm{bce}}=-{\displaystyle \frac{1}{k}}\sum _{t\in {\Omega }_k\left(X\right)}\left[ylog\left(p\right)+(1-y)log(1-p)\right]$$

(4)

Equation (3) defines the feature magnitude learning loss $L_{\mathrm{mag}}$, which encourages a larger difference between the magnitudes of top-k abnormal and normal snippet features. Here, $X_{\mathrm{abn}}^i$ and $X_{\mathrm{nor}}^i$ denote the abnormal and normal snippet features of the i-th video in the top-k set ${\Omega }_k\left(X^i\right)$, containing the k snippets with the largest l2-norm. The margin hyperparameter m defines the desired separation between the normal and abnormal features. The squared term penalises deviations from the margin symmetrically, and the loss is averaged across all videos in the batch.

Equation (4) defines the binary cross-entropy (BCE) loss $L_{\mathrm{bce}}$, which is computed over the top-k snippet scores selected from each video. Here, p and y denote the predicted probability and the ground-truth label of a snippet in the top-k set ${\Omega }_k\left(X\right)$, respectively.

In addition, two auxiliary loss components, namely, temporal smoothness and sparsity regularisation, are incorporated [13]. The loss components are summarised in Equations (5) and (6):

$$L_{\mathrm{smooth}}=\sum _{t=1}^{T-1}{(s_{t+1}^{\mathrm{abn}}-s_t^{\mathrm{abn}})}^2$$

(5)

$$L_{\mathrm{sparsity}}=\sum _{t=1}^T{\left(s_t^{\mathrm{abn}}\right)}^2$$

(6)

Equations (5) and (6) define the auxiliary losses applied to the abnormal snippet scores $s_t^{\mathrm{abn}}$, computed over all the abnormal snippets rather than the top-k. The temporal smoothness loss $L_{\mathrm{smooth}}$ encourages gradual changes between consecutive abnormal scores, penalising abrupt changes between adjacent snippets. The sparsity loss $L_{\mathrm{sparsity}}$ promotes sparse activation, keeping most abnormal scores low while allowing a few to be high.

The overall loss is defined in Equation (7), where the coefficients $\alpha$, ${\lambda }_1$ and ${\lambda }_2$ control the relative contributions of the losses.

$$L_{\mathrm{total}}=\alpha L_{\mathrm{mag}}+L_{\mathrm{bce}}+{\lambda }_1{L}_{\mathrm{smooth}}+{\lambda }_2{L}_{\mathrm{sparsity}}$$

(7)

4. Experiments and Discussions

4.1. Datasets

The experiments are performed on two benchmark datasets, the ShanghaiTech [12] and UCF-Crime [13] datasets. These two benchmarks only provide video-level labels and include both the normal and abnormal videos. For the normal videos, all snippets are considered normal, whereas in abnormal videos, only some snippets exhibit abnormal behaviours.

ShanghaiTech campus dataset is a medium-scale level dataset and covers 13 different scenes. It has a total of 437 videos, with 130 belonging to the abnormal videos category. These videos are taken in complex lighting conditions and different camera angles, but are less complex than the UCF-Crime dataset [12]. The original dataset was reorganised by Zhong et al. to move a subset of the abnormal testing videos to the training set to construct a weakly supervised training set [23]. This is widely accepted in the research community, and this work will adopt the same configuration.

UCF-Crime is a large-scale dataset consisting of 1900 long untrimmed videos and has an average frame number of 7247. These anomalies are selected because they have a significant impact on public safety, with 13 different real-world anomalies that impact public safety, such as robbery and shooting. Videos in this dataset encompass a complex and diverse environment, including both indoor and outdoor scenes, and day-time and night-time scenes. For the experiments, a total of 1610 videos will be used as the training set with video-level labels, while the remaining 290 videos will be used as the testing set with frame-level labels.

4.2. Metrics

In line with the previous work and to ensure a fair comparison with the baseline, the frame-level Area Under the Receiver Operating Characteristic Curve (AUC) is used as the evaluation metric for both datasets. AUC measures the model’s ability to distinguish between normal and anomalous frames across all threshold settings, with a higher AUC indicating better discriminative performance. This makes it especially suitable for evaluating models on imbalanced datasets, where accuracy can be misleading.

4.3. Implementation Details

Following prior works, 2048-dimensional video features are extracted using the I3D network [8]. For a fair comparison, the same setup as the baseline RTFM model [7] is adopted. The basic hyperparameters follow default settings: $T=32,m=100,C=10,k=3$, where each video is divided into 32 snippets, the margin for training purposes is set to 100, and the top 3 snippets are selected for calculating separability. The normal and abnormal weights are set to ${\lambda }_1={\lambda }_2=1$. Adam optimiser is used with a learning rate of $\alpha =0.005$ for ShanghaiTech and $\alpha =0.001$ for UCF-Crime. The batch size B is the same for both the normal and abnormal videos, meaning that each mini-batch contains B normal and B abnormal videos, and is selected randomly. The batch size B is 16 for ShanghaiTech and 64 for UCF-Crime.

For YOLO detector settings, the default configuration for generating text results is used, employing the model weights ‘yolov7.pt’ and ‘yolov9-c-converted.pt’, with a confidence threshold of 0.25 and an IoU threshold of 0.45. The text result includes object class IDs, bounding box coordinates, bounding box dimensions (width & height) and class confidence scores. To generate the object detection feature $F_{yolo}$, the feature dimension is set to $D_{yolo}=P\times N=1280$, where $P=16,N=80$, and $R_{threshold}=0.3$ are selected. We empirically found that $P=16$ patches offer a good trade-off between detection accuracy and computational efficiency.

4.4. Benchmark Performance

The experiment was first conducted on the ShanghaiTech dataset. The frame-level AUC performance on the ShanghaiTech dataset is shown in

Table 1. Frame-level AUC performance on the ShanghaiTech dataset. * indicates the result from retraining by the author. The best result is in Bold.

Supervision	Method	Features	AUC (%)
Unsupervised	Conv-AE [6]	-	60.85
	Stacked-RNN [32]	-	68.00
	Frame-Pred [12]	-	73.40
	Mem-AE [15]	-	71.20
	MNAD [33]	-	70.50
	VEC [34]	-	74.80
Weakly Supervised	GCN-Anomaly	C3D RGB	76.44
	Zhang et al. [35]	I3D RGB	82.50
	GCN-Anomaly [23]	TSN Flow	84.13
	GCN-Anomaly [23]	TSN RGB	84.44
	AR-Net [24]	I3D Flow	82.32
	AR-Net [24]	I3D RGB	85.38
	AR-Net [24]	I3D RGB & I3D Flow	91.24
	RTFM [7]	I3D RGB	97.21
	RTFM * [7]	I3D RGB	95.82
	Ours	I3D RGB & YOLOv7	96.51
	Ours	I3D RGB & YOLOv9	$\mathbf{97}.\mathbf{45}$

. Compared with the unsupervised approach, the proposed method demonstrates superior performance. The baseline model is retrained using the I3D RGB feature; likewise, for the proposed model, which also uses the same parameters. The result shows that the proposed method performs 1.63% better than the reproduced baseline model, and further improvement is observed when using the YOLOv9 object detector instead of YOLOv7.

Table 2. Frame-level AUC performance on the UCF-Crime dataset. * indicates the result from retraining by the author. The best result is in Bold.

Supervision	Method	Features	AUC (%)
Unsupervised	SVM Baseline	-	50.00
	Conv-AE [6]	-	50.60
	BODS [36]	I3D RGB	68.26
	GODS [36]	I3D RGB	70.46
	Zaheer et al. [37]	ResNext	71.04
Weakly Supervised	Sultani et al. [13]	C3D RGB	75.41
	Sultani et al. [13]	I3D RGB	77.92
	Zhang et al. [35]	C3D RGB	78.66
	GCN-Anomaly [23]	C3D RGB	81.08
	GCN-Anomaly [23]	TSN Flow	78.08
	GCN-Anomaly [23]	TSN RGB	82.12
	Wu et al. [38]	I3D RGB	82.44
	RTFM [7]	I3D RGB	84.30
	RTFM * [7]	I3D RGB	83.03
	Ours	I3D RGB & YOLOv7	83.81
	Ours	I3D RGB & YOLOv9	$\mathbf{84}.\mathbf{42}$

presents the frame-level AUC results on the UCF-Crime dataset. The proposed method achieves a notable improvement over unsupervised approaches and consistently outperforms the retrained baseline under identical experimental settings, with a performance gain of 1.39%. Additionally, incorporating the more advanced YOLOv9 object detector further enhances performance compared to its YOLOv7 counterpart, highlighting the benefit of stronger object-level representation in video anomaly detection tasks.

4.5. Ablation Study

Ablation studies have been carried out to evaluate the effectiveness of the proposed additional feature format (Section 3.1), the pre-processing methods (Section 3.2), and to analyse the impact of different object detectors.

4.5.1. Comparison with Feature Formats

The I3D features are first passed through an MTN block for the baseline model RTFM to unify the feature representations. However, this block is not applied to YOLO-generated features, as they encode class probability distributions across spatial patches rather than temporal dynamics.

The FeatVec method proposed by Doshi et al. [39] was evaluated to better align the feature to the training network. It combines the $X\&Y$ coordinates, object area, and class probabilities into a single feature vector (shown in Equation (8)) for further training. As an alternative, the proposed YOLO feature format, represented as $F_{yolo}$, has been tested through various experiments based on the YOLOv9 features to evaluate the effectiveness of the new feature format. The results, presented in

Table 3. Performance of different feature formats, including FeatVec and the newly proposed format. Various pre-processing methods are evaluated, utilising the YOLOv9 detector. The best result in each dataset is highlighted in bold.

Dataset	Pre-Processing			AUC (%)
	MTN	SE Layer	Hydra Attn	FeatVec [39]	Proposed Format
ShanghaiTech	✓			94.73	95.16
		✓		96.07	97.45
			✓	96.15	95.93
UCF-Crime	✓			82.89	83.18
		✓		83.49	83.5
			✓	82.73	84.42

, include comparisons across different pre-processing methods to provide a more comprehensive evaluation.

$$F_t^i=\left[\begin{array}{c}{C}_x\\ C_y\\ \mathrm{Area}\\ p\left(C_1\right)\\ p\left(C_2\right)\\ ⋮\\ p\left(C_n\right)\end{array}\right]$$

(8)

The results show that (1) the proposed feature format consistently outperforms the traditional FeatVec approach, validating its effectiveness; and (2) applying traditional processing blocks MTN to the additional YOLO features offers limited improvement over using I3D features alone, highlighting the importance of dedicated pre-processing strategies.

4.5.2. Comparison with Object Detectors

Since the proposed method can be seamlessly integrated with all existing object detectors, it is essential to assess the impact of different detector choices. As YOLO serves as the primary feature generator, a comparative analysis between YOLOv7 and YOLOv9, developed by the same team and based on similar principles, is conducted. The results of this comparison are presented in

Table 4. Comparison between different object detectors—YOLOv7 and YOLOv9. The best result in each dataset is highlighted in bold.

Dataset	Pre-Processing		AUC (%)
	SE Layer	Hydra Attn	YOLOv7	YOLOv9
ShanghaiTech	✓		96.51	97.45
		✓	96.28	95.93
UCF-Crime	✓		83.81	83.5
		✓	83.06	84.42

.

Analysis of the table reveals that while YOLOv9 does not always achieve better results than YOLOv7 on every instance, it generally achieves better overall results. In cases where YOLOv7 performs slightly better, the margin is minimal, whereas YOLOv9 often yields significantly stronger performance when it excels. This trend suggests that the more advanced architecture of YOLOv9, including specifically its improved gradient flow via PGI, enhanced layer aggregation with GELAN, and more efficient multi-scale representations, enables more accurate object detection, which in turn enhances the effectiveness of the additional object-level features in our model. Since this proposed technique can be seamlessly applied to any object detector, it holds strong potential for future improvements.

4.6. Qualitative Analysis

Qualitative analysis is presented in Figure 4, which shows the predicted anomaly scores generated by our network. Four example videos from the UCF-Crime dataset, including three anomalous and one normal video, are analysed. The results demonstrate that our model can effectively detect abnormal snippets across different types of anomalous events and can identify multiple events within a single video, which is a more challenging task in weakly supervised video anomaly detection. By incorporating object-level information through the additional feature $F_{yolo}$, our model achieves better performance in capturing abnormal events. Specifically, it produces higher and more stable anomaly scores for abnormal snippets (Figure 4a–c) and lower scores for normal videos (Figure 4d), compared to using only the I3D feature.

5. Conclusions

In conclusion, this paper proposes a novel weakly supervised approach for video anomaly detection. The proposed approach enables the network to leverage additional object-level information by effectively integrating object detection capabilities and an attention mechanism into the baseline framework. A custom-designed feature format is developed to encapsulate all the relevant object information, allowing seamless integration with other features, such as I3D, to support effective network training. For this work, the YOLO object detector is employed to obtain accurate object class categories and positional data, thereby enhancing the reliability and effectiveness of anomaly detection in surveillance videos. The proposed method demonstrates improved performance over the baseline and achieves competitive results to state-of-the-art (SOTA) methods on benchmark datasets, offering a promising alternative for tackling WSVAD tasks that previous methods have not fully explored.

While the proposed method leverages object-level information, the object detector is limited to a fixed set of predefined object categories, which may not fully capture all relevant objects in specific surveillance scenarios, such as crime-related events. Some object classes may also be redundant or less informative for anomaly detection. In future work, we plan to investigate detectors capable of learning a larger variety of object categories and models that can focus on scenario-relevant objects, potentially further enhancing the effectiveness of WSVAD.