TransLowNet: An Online Framework for Video Anomaly Detection, Classification, and Localization ^†

Abstract

This work presents TransLowNet, an online framework for video anomaly detection, classification, and spatial localization. The system segments incoming video streams into clips processed by an X3D-S feature extractor to obtain spatio-temporal representations, which are analyzed by dedicated modules for anomaly detection and recognition, while a MoG2-based stage estimates the spatial regions of anomalous activity. Evaluated on the UCF-Crime dataset, TransLowNet achieved 80.0% AUC, 54.5% accuracy, and 20.3% mAP@0.5, offering an efficient and interpretable approach for continuous video surveillance.

1. Introduction

Video Anomaly Detection (VAD) is a key field in computer vision with wide applications in intelligent video surveillance. Its main goal is to identify atypical events within video sequences. However, most existing approaches are limited to binary detection (determining only whether an anomaly is present) without recognizing the anomaly type or its spatial location, which substantially reduces their practical utility. Moreover, many methods operate under an offline scheme, where analysis is performed only after the entire video has been processed. To enable more efficient and practically deployable systems, anomaly analysis should instead be conducted in an online manner, that is, while the video stream is being received, jointly integrating detection, classification, and localization within a unified framework. In this work, we present TransLowNet, an online modular framework that unifies these three tasks. The framework begins by segmenting video streams into clips, each composed of N frames sampled at temporal intervals of G frames. Each clip undergoes a preprocessing step consisting of a central crop followed by a resize to $H\times W$ pixels, ensuring spatial consistency across clips. Once preprocessed, the clips are processed by an Expanded 3D ConvNet (X3D-S) [1], which extracts compact spatio-temporal representations of each clip. These representations are then evaluated by an anomaly detection module [2] that determines whether the observed patterns correspond to normal or abnormal events. When an anomaly is detected, the extracted features trigger two independent processes: (i) a multiclass classifier [3], which assigns the detected action to one of the predefined anomaly categories, and (ii) a spatial localization module based on Mixture of Gaussians (MoG2) [4], which operates at the pixel level to highlight the regions associated with the anomalous behavior. TransLowNet was evaluated on the UCF-Crime dataset [5], achieving competitive results in detection, classification, and localization compared with state-of-the-art methods, while maintaining the advantage of being fully online and modular.

2. Methodology

Figure 1 provides an overview of the proposed framework, TransLowNet, built upon the previous works of Flores et al. [3,6], where the Anomaly Detection Module and the Anomaly Classification Module were independently trained. In this work, the same training configuration is preserved, while the inference stage is extended by incorporating an additional Anomaly Localization Module to estimate the active regions where the anomalous event occurs.

During inference, each processed clip $x\in {\mathbb{R}}^{H\times W\times Ch}$, where $Ch$ denotes the number of channels, is fed into the Feature Extraction Module (X3D-S) [1], following the scheme described in [3,6]. This module produces a compact spatio-temporal feature representation $f\in {\mathbb{R}}^{1\times D}$, where D is the dimensionality of the feature vector. The resulting representations are then evaluated by the Anomaly Detection Module [2], adapted in [6] to operate in an online mode. The detector assigns to each clip an anomaly score that quantifies its deviation from normal scene behavior. This score is compared against a threshold ($thr$) set to the 90th percentile of the ROC-derived AUC distribution, following the criterion established in [6]. If the score exceeds $thr$, the clip is considered anomalous and is simultaneously forwarded to the Anomaly Classification Module [3] and to the proposed Anomaly Localization Module.

The Anomaly Localization Module relies on the observation that anomalous events often produce abrupt variations in visual dynamics or motion magnitude relative to the background. Following this principle, the Mixture of Gaussians (MoG2) method [4] is employed to adaptively model the background using Gaussian distributions updated over time. This process isolates the regions exhibiting unusual motion, generating a binary mask that is subsequently refined to remove noise and merge nearby regions. Finally, a coherent region representing the area of highest anomalous activity is obtained and expressed as a bounding box that constitutes the final spatial localization estimate of the event.

3. Results

The proposed TransLowNet framework was evaluated on the UCF-Crime dataset [5], which consists of 1900 untrimmed videos divided into 1610 for training (810 anomalous and 800 normal) and 290 for testing (140 anomalous and 150 normal), covering 13 anomaly categories under the standard evaluation protocol. Following the current state-of-the-art practice [2,3,5,6,7,8,9], four metrics were used to assess the performance of the system, each corresponding to a specific task. The Area Under the ROC Curve (AUC) was employed to measure frame-level anomaly detection performance, while Accuracy (ACC) was used to evaluate video-level multi-class classification. For spatial localization, the mean Average Precision at IoU 0.5 (mAP@0.5) was computed using the ground-truth bounding boxes provided in [9]. Finally, GFLOPs were reported to quantify the computational complexity of the model, providing an additional measure of efficiency and scalability for online environments. The detection and classification modules preserved the same training configuration, hyperparameters, and loss functions reported in Flores et al. [3,6], ensuring consistency and reproducibility across experiments. Following those works, each video sequence was divided into clips composed of $N=13$ frames sampled at a temporal interval of $G=6$. Each frame was center-cropped and resized to a spatial resolution of $H\times W=182\times 182$ pixels with $Ch=3$ RGB channels. The resulting feature dimensionality was $D=192$.

Table 1. Comparison of performance between state-of-the-art methods and the proposed framework on UCF-Crime.

Method	GFLOPs	AUC (%)	ACC (%)	mAP@0.5 (%)
Sultani et al. [5]	∼61.638	75.41	23.00	–
Tan et al. [7]	∼38.950	82.69	28.40	–
Gao et al. [8]	∼26.503	91.34	–	–
Flores et al. [3]	28.873	82.27	58.96	–
Al-Lahham et al. [2]	∼17.824	83.40	–	–
Ours (TransLowNet)	2.844	80.00	54.48	20.32

Note: Bold values indicate the results obtained by the proposed method (TransLowNet).

summarizes the performance comparison between TransLowNet and several state-of-the-art methods. The proposed framework achieved an AUC of 80.00%, a classification accuracy (ACC) of 54.48%, and a spatial localization performance of 20.32% mAP@0.5, with a computational cost of only 2.844 GFLOPs. These results demonstrate a competitive balance between accuracy and efficiency, highlighting that TransLowNet operates in an online mode, processing video streams sequentially as they are received. This property contrasts with offline methods such as Sultani et al. [5] and Tan et al. [7], which require complete video sequences before identifying anomalies, limiting their applicability in continuous surveillance scenarios that demand immediate analysis. Compared with other approaches, Gao et al. [8] and Al-Lahham et al. [2] achieve strong detection accuracy but lack classification or localization capabilities, restricting them to binary detection tasks. Conversely, Flores et al. [3] report the highest classification accuracy (58.96%) but employ a heavier architecture (28.873 GFLOPs) and do not include a localization component, preventing spatial identification of anomalous regions. In contrast, TransLowNet integrates detection, classification, and localization into a unified, modular, and efficient framework, capable of operating online with a computational cost nearly ten times lower than most prior models. Although the mAP@0.5 value (20.32%) indicates that spatial anomaly localization remains an open challenge (mainly due to high visual variability and low contrast between normal and abnormal patterns) the inclusion of this module represents a meaningful step toward a more comprehensive understanding of anomalous behavior. Overall, TransLowNet maintains competitive detection and classification performance while extending the functional scope of surveillance systems by providing additional spatial interpretability with minimal computational overhead.

4. Conclusions

This work presented TransLowNet, an online modular framework that jointly integrates video anomaly detection, classification, and spatial localization. The results obtained on the UCF-Crime dataset demonstrate a competitive balance between accuracy, interpretability, and computational efficiency. The modular design of the system enables flexible and scalable deployment in continuous surveillance environments. Although the experimental validation was limited to the UCF-Crime dataset, future work will include evaluations on additional benchmarks, as well as per-class and temporal stability analyses to further strengthen the robustness and generalization of the proposed model.