A Novel Study of Traffic Object Detection Based on Video Surveillance Streams

Abstract

Traditional traffic target detection heavily relies on manual processing. However, the latest advancements in deep learning have significantly enhanced the capabilities of target detection and multi-target tracking. To address these challenges, this paper proposes a perception-tracking-reasoning framework based on traffic rules, which is used for vehicle recognition and driving-state analysis in surveillance videos. This framework integrates enhanced vehicle perception, cross-frame identity association, trajectory-state modeling, and interpretable rule reasoning into a unified processing flow. Finally, experiments show that the main advantage of the proposed model lies in its ability to detect small-sized vehicle targets and improve trajectory stability in complex traffic scenarios.

1. Introduction

The rapid development of urban transportation has placed increasing pressure on traffic operation, safety management, and the enforcement of traffic regulations. According to statistics released by the Ministry of Public Security of China, by the end of 2023, the national motor-vehicle population had reached 435 million, including 336 million automobiles; the number of licensed motor-vehicle drivers had reached 523 million, and 94 cities had more than one million automobiles [1]. In addition, road traffic crashes cause approximately 1.19 million deaths worldwide each year [2]. These figures highlight the need for efficient and timely traffic-monitoring technologies.

Although surveillance cameras are widely deployed, conventional traffic supervision still depends substantially on manual inspection and post-event evidence review. Automatic analysis is challenging because roadside videos frequently contain small and distant vehicles, dense traffic, occlusion, perspective distortion, illumination variation, and adverse weather [3,4]. Errors in vehicle localization and cross-frame association may propagate to trajectory reconstruction and driving-state interpretation, thereby reducing the reliability of traffic-event analysis.

Existing object-detection and multi-object-tracking methods provide an effective basis for locating vehicles and maintaining their identities across consecutive frames. However, detection and tracking alone cannot determine whether an observed trajectory violates a traffic rule. Surveillance-based driving-state analysis additionally requires spatiotemporal reasoning over vehicle trajectories, road regions, movement directions, dwell time, and speed-related motion features. This creates a perception-to-reasoning gap between low-level vehicle detection and interpretable traffic-event understanding. Detailed comparisons with existing detection, tracking, traffic-event-analysis, and trajectory-prediction studies are provided in Section 2.

To address this gap, this paper proposes a Traffic-Rule-Aware Perception-Tracking-Reasoning framework (TRA-PTR) for vehicle recognition and driving-state analysis in surveillance videos. The trajectories considered here are local trajectories reconstructed within the field of view of a fixed surveillance camera rather than continuous trajectories spanning an entire urban road network. The framework combines an enhanced YOLOv11 detector, cross-frame vehicle association, trajectory-state modeling, and explicit traffic-rule reasoning. The detector is improved using HMAB, BiFPN-CSAF, an additional P2 detection branch, and EMA to strengthen local–global representation, multi-scale feature fusion, and small-vehicle perception. The resulting trajectories are subsequently evaluated using spatial-region constraints, temporal persistence, and motion-direction consistency to support the interpretation of speeding, illegal parking, and wrong-way driving. The main contributions of this paper are summarized as follows:

(1) An improved YOLOv11-based vehicle detector is designed for complex traffic surveillance scenarios. HMAB is introduced into the backbone to capture both local spatial details and global contextual dependencies with linear computational complexity; BiFPN-CSAF is proposed in the neck to improve cross-scale feature aggregation through learnable weighted fusion and scale-aware attention; an additional P2 branch is added to enhance small-vehicle detection; and EMA is incorporated into the decoupled detection head to strengthen task-specific feature selection and localization accuracy.

(2) A Traffic-Rule-Aware Perception-Tracking-Reasoning framework (TRA-PTR) is proposed for surveillance-video-based vehicle driving-state analysis. The framework integrates vehicle detection, identity association, trajectory modeling, and traffic-rule reasoning into a unified analytical pipeline, bridging the gap between low-level visual perception and high-level traffic semantic understanding.

(3) A transparent abnormal-driving detection mechanism based on TRA-PTR is designed to identify speeding, illegal parking, and wrong-way driving. The decision-making process is based on trajectory evolution, spatial-region constraints, temporal persistence, and motion-direction consistency, making the generated alerts traceable, auditable, and verifiable.

2. Related Works

2.1. Real-Time Vehicle Perception and Multi-Object Tracking

Real-time vehicle perception is a prerequisite for traffic video understanding. In recent years, object detection research has moved toward higher efficiency, lower latency, and more deployment-friendly architectures [5,6]. YOLOv10 introduces consistent dual assignments and a holistic efficiency–accuracy design strategy to reduce dependence on non-maximum suppression and improve real-time end-to-end detection performance [7]. RT-DETR demonstrates that Transformer-based detectors can achieve competitive real-time performance through efficient hybrid encoding and query selection [8]. YOLOv11 further extends the YOLO family toward efficient real-time vision tasks, including detection, segmentation, classification, pose estimation, and oriented object detection [9]. These studies indicate that modern detectors are increasingly suitable for resource-constrained traffic monitoring scenarios.

However, traffic surveillance differs from general object detection benchmarks. Roadside cameras usually capture vehicles from fixed and oblique viewpoints, where small targets, occlusion, scale distortion, motion blur, and background interference frequently occur [10,11]. Although existing detectors improve either local feature extraction, global context modeling, or multi-scale fusion [12,13,14,15], they rarely address local-detail preservation, long-range dependency modeling, cross-scale feature imbalance, and task-specific head optimization in a unified architecture. As a result, perception errors in small or occluded vehicles may cause trajectory fragmentation and unreliable traffic-event reasoning [16,17]. This motivates the detector-level design in this work: HMAB strengthens the backbone by combining local convolutional representation with global linear attention, BiFPN-CSAF improves cross-scale feature aggregation through learnable weighted fusion and scale-aware attention, and EMA refines the decoupled head by enhancing classification and localization-sensitive regions.

Traffic-event understanding also requires temporally coherent trajectories rather than isolated frame-level detections. Recent tracking-by-detection methods have improved robustness under occlusion and nonlinear motion [16,18,19,20]. OC-SORT revisits the limitations of Kalman-filter-based tracking and reduces error accumulation by emphasizing observation-centric updates [21]. Deep OC-SORT further introduces adaptive appearance matching to strengthen identity preservation in challenging tracking scenarios [17]. These methods provide effective tools for cross-frame vehicle identity maintenance. Nevertheless, detection and tracking alone cannot determine whether a trajectory corresponds to a traffic violation [22]. A higher-level reasoning mechanism is still required to connect trajectories with traffic-rule semantics.

2.2. Traffic Violation and Traffic-Event Detection

Recent work has begun to investigate automated traffic violation detection using surveillance or mobile video sources [3,4]. AI-assisted traffic violation systems typically combine vehicle detection, tracking, region-of-interest analysis, and rule-specific logic to identify violations such as red-light running, illegal parking, lane violations, or wrong-way driving [3,22]. Some studies further focus on self-adaptive traffic incident detection, where normal driving directions or traffic patterns are learned or configured to support abnormal-event recognition [23,24]. These studies demonstrate the practical value of computer vision for traffic enforcement and road-safety monitoring.

Despite their progress, existing traffic violation detection systems still have several limitations. First, many methods adopt generic detectors without adapting the perception module to surveillance-specific challenges such as small vehicles, dense traffic, perspective distortion, and partial occlusion [3,22]. Second, traffic-rule judgment is often implemented as post-processing logic after detection and tracking, which may limit the interaction between visual perception quality and event-level reliability. Third, many systems report violation detection results but provide limited analysis of how detector-level improvements influence trajectory stability and event-level alarm accuracy [3,22]. In contrast, this work treats reliable vehicle perception as the foundation of traffic-rule reasoning and improves the YOLOv11 detector from the backbone, neck, and detection head before integrating it with tracking and interpretable rule reasoning.

2.3. Vehicle Trajectory and Behavior Prediction

Vehicle trajectory and behavior prediction has become a central topic in autonomous driving and intelligent transportation. Recent research has shifted from purely kinematic extrapolation to deep spatiotemporal modeling, in which interaction patterns, road context, and driving knowledge are incorporated to improve prediction reliability [25,26]. Reviews of knowledge-guided deep trajectory prediction have emphasized that purely data-driven models often lack interpretability and safety assurance, motivating the integration of driving knowledge into model design, learning, and evaluation [27]. Recent surveys have examined vision-based vehicle trajectory prediction from theoretical and engineering perspectives, covering problem formulation, modeling paradigms, decision-making applications, datasets, and typical scenarios [28,29].

Model-level studies have explored more expressive architectures for multi-agent motion modeling [30,31]. One trajectory-prediction model integrates spatial interaction and multi-scale temporal features to capture the dynamic influence of neighboring vehicles [25]. An agent-centric Transformer with adaptive receptive fields introduces perception-guided spatial-context encoding for parallel multi-agent prediction [30]. Other work combines convolutional networks and Transformers to construct a multi-target trajectory Transformer for urban traffic scenarios [32]. These studies demonstrate the effectiveness of deep models in capturing complex motion patterns. However, their primary objective is future trajectory forecasting for autonomous driving rather than the interpretation of observed trajectories under explicit traffic regulations. Surveillance-based driving-state analysis therefore requires a different formulation: not only predicting or reconstructing motion, but also judging whether the observed motion is rule-compliant [22,33,34].

2.4. Research Gap

Overall, recent studies have made substantial progress in real-time perception, traffic violation detection, trajectory prediction, maneuver decision-making, driving-style analysis, and driver-state monitoring; nevertheless, several critical limitations remain for surveillance-video-based traffic-event analysis. First, most trajectory prediction and decision-making methods are primarily designed for autonomous driving and typically rely on ego-vehicle perception, map priors, or interaction modeling, whereas fixed surveillance cameras must infer vehicle states from image-space trajectories under viewpoint variation, occlusion, scale distortion, and calibration uncertainty. Second, many traffic violation systems use detection and tracking as generic front-end modules, but insufficiently optimize the detector for small, distant, and occluded vehicles in roadside scenes, causing perception errors to propagate into trajectory modeling and event judgment. Third, many deep models prioritize predictive or detection accuracy but provide limited evidence-level interpretability, which is inadequate for enforcement scenarios where alarms must be traceable, auditable, and verifiable. Finally, existing pipelines often treat detection, tracking, and behavior judgment as loosely coupled components, resulting in weak semantic consistency between low-level visual perception and high-level traffic-rule reasoning.

In response to these limitations, this work formulates surveillance-video-based vehicle driving-state analysis as a traffic-rule-constrained spatiotemporal reasoning problem and proposes an enhanced perception–tracking–reasoning framework. The framework first strengthens YOLOv11-based vehicle perception through HMAB, BiFPN-CSAF, an additional P2 detection branch, and EMA-enhanced decoupled heads, thereby improving local–global representation, multi-scale feature fusion, small-vehicle detection, and task-specific feature discrimination. It then integrates tracking-based identity association, trajectory-state modeling, and interpretable rule reasoning to recognize speeding, illegal parking, and wrong-way driving from observed trajectories according to explicit traffic rules. By jointly improving the perception backbone and the high-level reasoning pipeline, the proposed method bridges the gap between low-level visual detection and evidence-level traffic semantic understanding and provides an experimentally evaluated framework for surveillance-video traffic monitoring and traffic-event analysis.

3. Preliminaries

3.1. Symbol Definition

Vehicle motion detection in surveillance video can be described as a process of spatiotemporal semantic analysis that involves extracting vehicle targets from consecutive video frames, constructing vehicle trajectories, and generating anomaly events. Let the input road surveillance video stream be defined by Equation (1).

$$V={\left\{I_t\right\}}_{t=1}^T$$

(1)

where V represents the input video sequence, $I_t$ represents the tth frame, and T represents the total number of frames in the video. For any given frame, the vehicle detection model outputs the set of detected objects in that frame, as shown in Equation (2).

$$B_t={\left\{\left(b_i^t,c_i^t,s_i^t\right)\right\}}_{i=1}^{N_t}$$

(2)

where $b_i^t=(x_i^t,y_i^t,w_i^t,h_i^t)$ denotes the bounding box of the ith vehicle target in the tth frame, $c_i^t$ denotes the vehicle class, $s_i^t$ denotes the detection confidence, and $N_t$ denotes the number of vehicles detected in the current frame. In continuous video analysis, single-frame detection boxes need to be further linked to form vehicle trajectories. The trajectory of the kth vehicle can be defined by Equation (3).

$${\tau }_k=\left(i{d}_k,{\left\{p_k^t\right\}}_{t=t_s}^{t_e}\right)$$

(3)

where $i{d}_k$ represents the vehicle identification number; $t_s$ and $t_e$ denote the start and end frames of the trajectory, respectively; and $p_k^t$ represents the vehicle’s reference point on the road plane at frame t. Since the determination of the vehicle’s driving state relies primarily on changes in its position on the road plane, this paper selects the center point of the bottom edge of the vehicle’s bounding box as the reference point, as shown in Equation (4).

$$p_k^t=\left(x_k^t+{\displaystyle \frac{w_k^t}{2}},y_k^t+h_k^t\right)$$

(4)

Compared to the geometric center of the bounding box, the center point of the bottom edge is closer to the point of contact between the vehicle and the road surface, making it more suitable for determining region affiliation, estimating speed, and analyzing direction of motion. Ultimately, the set of anomaly events output by the system can be represented by Equation (5).

$$\epsilon ={\left\{e_m=\left(typ{e}_m,t_m,i{d}_m\right)\right\}}_{m=1}^M$$

(5)

where $e_m$ represents the mth abnormal event, $typ{e}_m$ denotes the event type, $t_m$ denotes the time the event was triggered, $i{d}_m$ denotes the vehicle ID that triggered the event, and M denotes the total number of abnormal events.

3.2. Problem Definition

The problem addressed in this paper is not simply vehicle object detection, nor is it multi-object tracking in the general sense; rather, it is the task of identifying vehicle driving states subject to traffic regulations. Detection models can determine a vehicle’s position and class within a single frame, and multi-object tracking can maintain identity consistency across consecutive frames; however, the identification of abnormal behaviors—such as speeding, illegal parking, and driving against traffic—must also rely on information such as vehicle trajectories, road segments, direction of travel, dwell time, and speed thresholds. Let the set of traffic rules be defined by Equation (6).

$$R=\left\{R_s,R_p,R_w\right\}$$

(6)

where $R_s$, $R_p$, and $R_w$ represent the speeding detection rule, the illegal parking detection rule, and the wrong-way driving detection rule, respectively. Based on the input video stream V, the set of vehicle trajectories $\tau$, and the set of traffic rules R, the problem addressed in this paper can be formalized as the following mapping process in Equation (7).

$$F:(V,\tau ,R)\to \epsilon$$

(7)

where F represents the vehicle driving state recognition function, and $\epsilon$ represents the set of abnormal driving events output by the system. The definition indicates that the core of the task addressed in this paper is not the static recognition of vehicle targets in a single frame, but rather the conversion of vehicle detection results from consecutive video frames into stable trajectories, followed by spatiotemporal semantic reasoning based on explicit traffic rules. Compared to typical detection or tracking tasks, this problem involves stronger chained dependencies: errors in front-end detection affect trajectory continuity; trajectory breaks affect the calculation of speed, dwell time, and direction; and the accuracy of rule reasoning depends on the completeness and stability of the trajectory data. Therefore, it is necessary to construct a unified analytical framework that integrates visual perception, cross-frame correlation, trajectory modeling, and rule determination.

3.3. YOLOv11 Model for Vehicle Detection

Vehicle detection serves as the front-end foundation for identifying vehicle movement in surveillance video, and the quality of its output directly affects the reliability of subsequent target tracking and rule reasoning. As a further evolution of the YOLO series of single-stage object detection models, YOLOv11 retains the computational efficiency and flexible-deployment characteristics of one-stage detection, making it a suitable starting point for vehicle perception in road surveillance videos. Its overall structure typically consists of an input layer, a backbone network, a neck network, and a detection head: the input layer handles image resizing, normalization, and tensor transformation; the backbone network extracts multi-level visual features of vehicle targets; the neck network is used to fuse semantic information and spatial details at different scales; and the detection head outputs vehicle bounding boxes, class labels, and confidence scores. For road surveillance scenarios, YOLOv11 offers a favorable detector-level efficiency–accuracy trade-off, although complete-system performance also depends on video input, tracking, rule reasoning, and the deployment platform. It is still affected by factors such as fixed camera viewpoints, small targets at long distances, vehicle occlusion, scale variations, perspective distortion, and interference from complex backgrounds. These issues can lead to missed detections, false positives, and localization errors, which are further propagated to the trajectory association and abnormal behavior detection stages. Therefore, this paper will subsequently introduce targeted improvements to the YOLOv11 architecture for traffic monitoring scenarios, aiming to enhance the feature representation capabilities, cross-scale fusion capabilities, and adaptability to complex scenes at the vehicle perception layer, thereby providing more stable front-end detection results for the “perception–tracking–reasoning” framework.

4. Vehicle State Recognition Model Based on Video Stream

Vehicle driving state recognition in surveillance video is not a simple object detection or multi-object tracking task, but rather a comprehensive spatiotemporal semantic analysis task that integrates visual perception, cross-frame object association, trajectory modeling, and traffic rule reasoning. Detection models primarily address the spatial localization of vehicle targets, while multi-object tracking establishes identity associations for vehicles across consecutive video frames and generates complete motion trajectories. However, when used in isolation, neither of these technologies can directly determine whether a vehicle’s driving behavior complies with traffic regulations.

To address the above issues, this paper proposes TRA-PTR, a perception, tracking, and reasoning framework that incorporates traffic rules. First, an improved YOLOv11 model is used to extract vehicle location, class, and confidence information from consecutive video frames; then, a multi-object tracking module links the frame-by-frame detection results into vehicle trajectories with unified identifiers; finally, by integrating traffic rule constraints such as lane boundaries, direction of travel, driving speed, and duration of stops, the system automatically identifies abnormal driving behaviors such as speeding, illegal parking, and driving against traffic. This architecture achieves a step-by-step progression from low-level visual object detection to high-level traffic semantic understanding, establishing a complete technical pipeline for subsequent abnormal event alerts, evidence retention, and manual review.

Surveillance video can be regarded as a continuous sequence of images. The model performs vehicle detection on each frame and outputs detection results that include the target bounding box, class, and confidence score. The tracking module constructs multiple vehicle trajectories based on the frame-by-frame detection results; each trajectory records key information such as the vehicle’s identification number, positions at different times, and the start and end frames of the trajectory. To improve the accuracy of motion state analysis, this paper selects the center point of the bottom edge of the vehicle bounding box as the reference point on the road plane, calculated as shown in Equation (4).

Compared to the geometric center of the bounding box, the center point of the bottom edge more accurately reflects the actual point of contact between the vehicle and the road surface, effectively improving the stability of area classification, as well as the calculation of driving speed and direction of movement. The reasoning module combines all vehicle trajectories with predefined traffic rules to identify various types of abnormal driving events. Each event record includes details such as the event type, occurrence time, and associated vehicle ID, ultimately forming structured traffic event data to support subsequent traffic management efforts. Figure 1 shows the framework of TRA-PTR.

Figure 1. TRA-PTR framework.

4.1. Improvements to the YOLOv11 Vehicle Detection Layer

The perception layer serves as the front-end foundation of the framework, and its detection results directly impact the reliability of subsequent trajectory association and rule reasoning. In roadside monitoring scenarios, vehicles are often affected by fixed viewpoints, perspective distortion, long-distance imaging, occlusions, and dense traffic, exhibiting characteristics such as a high proportion of small targets, significant variations in scale, blurred contours, and strong background interference. If errors such as missed detections, false positives, or positioning deviations occur at the front end, the center point of the bottom edge—which more closely aligns with the actual contact point between the vehicle and the road surface than the geometric center of the bounding box—can effectively improve the stability of area attribution determination, as well as the calculation of driving speed and direction of movement. The reasoning module combines all vehicle trajectories with preset traffic rules to identify various types of abnormal driving events. Each event record includes details such as event type, occurrence time, and associated vehicle identification number, ultimately forming structured traffic event data to support subsequent traffic management efforts.

4.1.1. Attention Module

In traffic surveillance videos, accurate vehicle identification relies on two types of information: fine-grained spatial features composed of vehicle edges, contours, and local textures, and global contextual information formed by road structure, lane layout, and surrounding traffic participants. Standard convolutional operations can efficiently extract local spatial features, but their fixed receptive fields make it difficult to model long-range dependencies within feature maps. Although global self-attention mechanisms can capture contextual relationships over a wide area, they suffer from high computational overhead and may not satisfy the requirements of real-time video analysis.

To balance local feature extraction capabilities with global dependency modeling capabilities, this paper introduces a hybrid mixed attention block (HMAB) after the key feature layers of the YOLOv11 backbone network, employing a structure with parallel local and global branches to achieve complementary feature enhancement. Let the input feature map be $F\in {\mathbb{R}}^{C\times H\times W}$, where C, H, and W represent the number of channels, height, and width, respectively. The local branch uses $3\times 3$ depth-separable convolutions to extract neighborhood-based spatial features, while the global branch employs a linear attention mechanism inspired by focused linear attention [14] to model long-range feature dependencies. The features output by the two branches are concatenated along the channel dimension, then fused via a $1\times 1$ convolution. A residual structure is introduced to preserve the original input features and stabilize the training process. The overall computation of the module is shown in Equation (8).

$$F_{\mathrm{out}}=F+{\mathrm{Conv}}_{1\times 1}\left(\left[{\mathrm{DWConv}}_{3\times 3}\left(F\right),\mathrm{LinearAttn}\left(F\right)\right]\right)$$

(8)

Depthwise separable convolutions enhance local spatial responses while controlling the number of parameters, thereby effectively extracting vehicle-edge and contour information. The linear attention mechanism establishes relationships between vehicle targets and the road background, adjacent vehicles, and scene structure in a lightweight manner. The module jointly models local details and global context at a low computational cost, improving detection stability for small and occluded vehicles in complex backgrounds.

4.1.2. BiFPN-CSAF Cross-Scale Feature Fusion Neck

The conventional PAN–FPN neck aggregates adjacent feature levels through a fixed fusion structure, in which features from different resolutions contribute without an explicit mechanism for estimating their relative importance. This can weaken shallow spatial details when they are combined with stronger deep semantic responses. To improve cross-scale aggregation, the original neck is replaced by BiFPN-CSAF. The proposed neck retains the bidirectional top-down and bottom-up paths of the standard BiFPN [12], but augments each fusion node with a Cross-Scale Attention Fusion (CSAF) operation. Consequently, high-resolution localization cues and low-resolution semantic cues can be propagated in both directions and recalibrated after fusion.

Let a fusion node at pyramid level l receive n feature maps ${\left\{X_i\right\}}_{i=1}^n$. Because these inputs may have different spatial resolutions and channel dimensions, each input is first transformed by a scale-alignment operator ${\mathcal{R}}_{l,i}(·)$, which consists of the required upsampling or downsampling operation followed by channel alignment. Denoting the aligned feature by $Z_{l,i}={\mathcal{R}}_{l,i}\left(X_i\right)$, the non-negative normalized fusion coefficient of the ith input is defined as

$${\alpha }_{l,i}={\displaystyle \frac{\mathrm{ReLU}\left(w_{l,i}\right)}{ϵ+{\sum }_{j=1}^n\mathrm{ReLU}\left(w_{l,j}\right)}},i=1,\dots ,n,$$

where $w_{l,i}$ is a learnable scalar and $ϵ$ is a small positive constant that prevents numerical instability. The coefficients are learned jointly with the detector and satisfy ${\sum }_i{\alpha }_{l,i}\approx 1$, allowing the network to suppress weak or conflicting inputs and emphasize the feature levels that are more informative for the current fusion node. Instead of directly treating the normalized weighted sum as the node output, BiFPN-CSAF preserves the weighted cross-scale components and concatenates them along the channel dimension:

$$F_{\mathrm{cat},l}=\mathrm{Concat}\left({\alpha }_{l,1}{Z}_{l,1},{\alpha }_{l,2}{Z}_{l,2},\dots ,{\alpha }_{l,n}{Z}_{l,n}\right),$$

where $F_{\mathrm{cat},l}$ retains scale-specific information before channel mixing. The CSAF block then applies a $1\times 1$ convolution to mix cross-scale channel information and adjust the channel dimension, followed by an Efficient Multi-Scale Attention (EMA) operation to recalibrate scale-sensitive spatial responses and suppress redundant background interference. Finally, a $3\times 3$ convolution refines the recalibrated representation. The complete post-fusion operation is

$$P_l^{\mathrm{out}}={\mathrm{Conv}}_{3\times 3}\left(\mathrm{EMA}\left({\mathrm{Conv}}_{1\times 1}\left(F_{\mathrm{cat},l}\right)\right)\right).$$

Thus, the processing sequence at each node is: spatial and channel alignment, normalized cross-scale weighting, channel-wise concatenation, $1\times 1$ channel mixing, EMA-based attention recalibration, and $3\times 3$ convolutional refinement. This sequence is applied throughout the top-down and bottom-up paths, after which the resulting $P2$–$P5$ feature maps are delivered to the corresponding detection branches. The EMA operation in CSAF is located at the neck fusion nodes; the EMA-enhanced decoupled head described below performs an additional task-oriented recalibration before the classification and regression branches.

BiFPN-CSAF differs from the original BiFPN and Adaptively Spatial Feature Fusion (ASFF) in both topology and weighting strategy. The original BiFPN uses a repeated bidirectional pyramid and fast normalized scalar weights, but its fusion node normally obtains the output by weighted summation followed by convolution [12]. BiFPN-CSAF retains the bidirectional topology and normalized weights while preserving the weighted scale components through concatenation and introducing $1\times 1$ channel mixing, EMA-based post-fusion recalibration, and $3\times 3$ refinement. ASFF, in contrast, resizes several pyramid levels to a target resolution and learns spatially varying, pixel-level weights to suppress conflicts at that output level [35]. BiFPN-CSAF uses node-level normalized coefficients to control the overall contribution of each incoming scale and then recalibrates the fused representation. It is therefore designed as a bidirectional neck-level aggregation module rather than a single-output spatial selection mechanism.

4.1.3. EMA-Enhanced Decoupling Detection Head

YOLOv11 employs a decoupled detection head, breaking down the object detection task into two independent subtasks: class classification and bounding box regression. The classification branch is responsible for identifying the class to which an object belongs, while the regression branch is dedicated to fitting the spatial coordinates and scale parameters of the object’s bounding box. This decoupled architecture effectively mitigates feature antagonism arising from shared features between the classification and localization tasks, optimizes model convergence, and improves overall detection accuracy. However, in complex real-world scenarios such as traffic monitoring, vehicle targets are highly prone to feature overlap with shadows, road markings, adjacent vehicle bodies, and background textures; as a result, the classification and regression branches remain susceptible to noise interference from irrelevant background regions.

To further enhance the detection head’s ability to select relevant regions, this paper embeds an EMA attention module [15] before the decoupled detection head. Through multi-scale interactions across spatial dimensions, EMA adaptively recalibrates key regions in the feature map. Let the features of the input detection head be denoted as $\left(F\right)$. EMA generates spatial attention weights $\left(A\right(F\left)\right)$ and produces the enhanced features in Equation (9).

$$F^{\prime }=F\odot A\left(F\right)$$

(9)

where ⊙ denotes element-wise multiplication. The EMA module recalibrates the original features based on the importance of spatial regions, enhancing feature responses in the vehicle’s main body, boundary areas, and task-sensitive regions, while suppressing feature expressions in background interference regions. After enhancement by the EMA module, the classification branch can focus more effectively on the vehicle’s semantic features, and the regression branch achieves more stable localization of the vehicle’s boundaries. This design effectively improves the detection head’s ability to handle complex backgrounds, small-scale targets, and occluded targets, with virtually no increase in channel dimension or computational complexity.

4.2. Parameter Passing and Threshold Linkage Mechanism

The overall structure of the paper is organized as a “perception–tracking–reasoning” pipeline; however, the modules are not simply connected in series, but rather form a hierarchical chain of information flow and constraint interaction through detection results, trajectory states, and rule parameters. The bounding box positions, class labels, and confidence scores output by the front-end detection layer determine the set of candidate targets that the tracking module can use for data association; the identity IDs, historical trajectories, and target loss status generated by the tracking layer, in turn, directly influence the rule-reasoning layer’s determination of vehicle speed, dwell time, and direction of movement. Thus, module parameters not only affect the performance of individual stages but also propagate along the entire processing chain, ultimately altering the determination of abnormal events.

In the detection layer, let the vehicle detection confidence threshold be ${\tau }_{det}$. The system retains only the bounding boxes that satisfy the confidence condition in Equation (10):

$$B_t^{\mathrm{valid}}=\left\{\left(b_i^t,c_i^t,s_i^t\right)\in B_t\mid s_i^t\ge {\tau }_{\det }\right\}$$

(10)

The purpose of this threshold is to filter out background false positives and low-quality bounding boxes, ensuring that the candidate targets fed into the tracking module are highly reliable. In this paper, the detection confidence threshold is set to 0.25 to strike a balance between target recall and false positive suppression. If the threshold is set too high, small vehicles at a distance, occluded vehicles, or vehicles with motion blur will be prematurely discarded, resulting in trajectory breaks; if the threshold is set too low, a large amount of background noise will enter the tracking matching stage, significantly increasing the risk of incorrect associations.

To balance trajectory continuity and matching reliability, the tracking module classifies detection results into two categories—high-confidence samples and low-confidence samples—and sets two threshold levels, ${\tau }_{high}$ and ${\tau }_{low}$, respectively. During tracking, the system prioritizes using high-confidence detection boxes to perform the first round of matching with existing trajectories; for trajectories that fail to match successfully in the first round, it then uses low-confidence detection boxes to perform a second round of matching. This strategy not only ensures the reliability of the primary matching results but also maintains trajectory integrity during brief vehicle occlusions or when the detection score drops.

The core criterion for trajectory matching is the intersection-to-overlap ratio between the trajectory prediction box and the current detection box. When the intersection-to-overlap ratio exceeds the preset matching threshold of $\tau$, the system determines that the two boxes represent the same target and updates the trajectory. $\tau$ threshold must be set comprehensively by taking into account the camera’s field of view, vehicle speed, video frame rate, and traffic density: if the threshold is set too high, matching failures are likely to occur when vehicles move quickly or reappear after being obscured; if the threshold is set too low, it can easily lead to adjacent vehicles being misidentified.

The number of lost frames tracked is a key parameter linking the tracking layer and the rule-reasoning layer. In this paper, the maximum allowed number of lost frames is set to $L_{max}$ = 30. For surveillance video with a frame rate of 30 FPS, this parameter corresponds to a 1-s tolerance window for short-term occlusion. When a trajectory cannot be matched to a valid detection bounding box in the current frame, the system does not immediately delete the trajectory but instead marks it as temporarily lost and accumulates the number of lost frames; only when the cumulative number of lost frames exceeds $L_{max}$ is the vehicle deemed to have left the frame, and the trajectory is terminated. This setting effectively mitigates trajectory breakage caused by vehicles traveling side by side, brief occlusions, or fluctuations in detection confidence. However, if the parameter value is set too high, the trajectory of a vehicle that has already left the frame will be retained for an extended period, making it highly likely to be incorrectly associated with a new vehicle entering the frame; conversely, setting the value too low will cause trajectories to terminate frequently during brief occlusions, affecting subsequent speed calculations and dwell time statistics.

The rule-reasoning layer maintains a fixed-length history buffer for each vehicle; the length of this history window directly determines the stability of the calculated speed, parking status, and direction of motion. If the window length is too short, single-frame localization errors will significantly disrupt the state-calculation results; if the window length is too long, it will increase the response delay of rule evaluation. This paper adopts a fixed-length history buffer to balance state stability and alert response delay.

4.3. Trajectory-Driven Reasoning Mechanism for Abnormal Driving Behavior

After completing frame-by-frame vehicle detection, discrete detection boxes cannot directly support the determination of traffic violations. Abnormal behaviors such as speeding, illegal parking, and driving against traffic all require comprehensive analysis based on the vehicle’s motion over a continuous period of time. Therefore, this paper first links the frame-by-frame detection results into a sequence of trajectories with stable identification numbers, then performs traffic-rule reasoning based on the trajectory data. It constructs a trajectory-driven, explainable reasoning mechanism for anomalous driving behavior, integrating cross-frame identity association, trajectory state modeling, and traffic rule determination into a unified analysis pipeline.

The paper adopts the detection + tracking paradigm to achieve cross-frame vehicle identity association. After the detector outputs the bounding box, class, and confidence information for each frame, the tracking module establishes cross-frame associations based on motion prediction and spatial matching. The system predicts the vehicle’s current position based on the trajectory state from the previous frame and matches the detection bounding box with the trajectory using the intersection-to-union ratio; if the match is successful, the trajectory state is updated; if the match fails, the trajectory is marked as temporarily lost, and the trajectory information is retained within the maximum number of lost frames. At the same time, the module adopts the high- and low-confidence tiered association strategy described earlier, prioritizing high-confidence samples to ensure matching accuracy, and then using low-confidence samples to supplement unassociated trajectories, thereby achieving a balance between detection reliability and trajectory continuity. Trajectory position calculations uniformly use the center point of the bottom edge of the vehicle bounding box, as defined earlier, as the reference point on the road plane.

The system maintains an independent historical trajectory cache for each vehicle trajectory, containing a temporary anonymous tracking identifier, frame index or timestamp, reference point coordinates, detection bounding box dimensions, cumulative number of lost frames, and traffic-region status, providing structured input for subsequent rule reasoning. Based on continuous trajectory sequences, this paper designs determination rules for three typical types of abnormal behavior—speeding, illegal parking, and driving against traffic—with the core calculation methods as follows:

• Speeding Determination

The system estimates the actual driving speed based on the pixel displacement of the vehicle within the historical window; the specific calculation method is shown in Equation (11). When the calculated speed exceeds the road speed limit threshold, the system determines that the vehicle is speeding.

$$v_k={\displaystyle \frac{\rho d_{\mathrm{pix}}}{(t_2-t_1)/f^{\prime }}}$$

(11)

where $\rho$ is the conversion factor between pixel distance and actual physical distance, $d_{pix}$ is the vehicle’s pixel displacement over a given time interval, and $f^{\prime }$ represents the video frame rate.

• Illegal Parking Determination

Illegal parking is determined by comprehensively evaluating three factors: the vehicle’s location, the cumulative duration of its stay, and the maximum displacement of its trajectory. If a vehicle is located in a no-parking zone, its cumulative duration of stay exceeds a preset threshold, and the maximum displacement of its trajectory is less than the displacement threshold, it is deemed to be illegally parked, according to Equations (12) and (13).

$$D_k=\underset{i,j}{\max }{∥p_k\left(t_i\right)-p_k\left(t_j\right)∥}_2$$

(12)

$$p_k\left(t\right)\in {\omega }_{p^{\prime }},\Delta T_k\ge {\tau }_{p^{\prime }},D_k\le {\delta }_p⟹e_k=E_{\mathrm{park}}$$

(13)

where $p_k^t$ represents the road-plane reference point of the kth vehicle in frame t; ${\omega }_{p^{\prime }}$ denotes the no-parking zone preset by the user; $\Delta T_k$ represents the cumulative dwell time of the kth vehicle within the no-parking zone; ${\tau }_{p^{\prime }}$ represents the illegal parking duration threshold; $D_k$ represents the maximum displacement of the vehicle within the no-parking zone; ${\delta }_p$ represents the parking displacement threshold, used to determine whether the vehicle is essentially stationary; $e_k$ represents the type of anomaly event triggered by the kth vehicle.

• Determination of Driving Against Traffic

The system calculates the cosine similarity between the vehicle’s actual direction vector and the prescribed lane direction vector to determine whether the two directions are consistent. When the directional similarity falls below a preset threshold and the vehicle exhibits a valid displacement, the system determines that the vehicle is driving in the wrong direction according to Equations (14)–(16).

$$m_k=p_k\left(t_2\right)-p_k\left(t_1\right)$$

(14)

$$\cos \theta ={\displaystyle \frac{m_k·d_r}{{∥m_k∥}_2{∥d_r∥}_2}}$$

(15)

$$\cos \theta \le \eta ,{∥m_k∥}_2\ge {\delta }_r,\Delta T_k^{\mathrm{wrong}}\ge T_w⟹e_k=E_{\mathrm{wrong}}$$

(16)

where $m_k$ represents the actual motion vector of the kth vehicle within the time window; $p_k\left(t_1\right)$ and $p_k\left(t_2\right)$ represent the vehicle’s reference points on the road plane at the start and end frames of the time window, respectively; $d_r$ represents the vector of the prescribed lane direction; $\cos \theta$ represents the cosine similarity between the vehicle’s actual direction of motion and the prescribed direction; $\eta$ represents the directional similarity threshold; ${\delta }_r$ represents the minimum valid displacement threshold, used to exclude false positives caused by detection box jitter; $\Delta T_k^{\mathrm{wrong}}$ denotes the continuous duration for which the direction and displacement conditions are satisfied; $T_w$ is the required wrong-way persistence threshold; and $e_k$ represents the type of anomaly event triggered by the kth vehicle.

For clarity, the first valid alarm times for speeding, illegal parking, and wrong-way driving are defined, respectively, as

$$\begin{array}{cc}\hfill t_{k,s}^{\mathrm{alarm}}& =\inf \left\{t:v_k\left(t\right)>v_{\lim }\right\},\hfill \\ \hfill t_{k,p}^{\mathrm{alarm}}& =\inf \left\{t:p_k\left(t\right)\in {\omega }_{p^{\prime }},\Delta T_k\left(t\right)\ge {\tau }_{p^{\prime }},D_k\left(t\right)\le {\delta }_p\right\},\hfill \\ \hfill t_{k,w}^{\mathrm{alarm}}& =\inf \left\{t:\cos {\theta }_k\left(t\right)\le \eta ,{∥m_k\left(t\right)∥}_2\ge {\delta }_r,\Delta T_k^{\mathrm{wrong}}\left(t\right)\ge T_w\right\}.\hfill \end{array}$$

The ground-truth onset for speeding is the first time at which the reference speed exceeds the prescribed limit. For illegal parking, it is the first time at which the vehicle has remained inside the no-parking region for the prescribed dwell time while satisfying the displacement constraint. For wrong-way driving, it is the annotated time at which the vehicle begins to travel against the prescribed lane direction. Thus, the wrong-way alarm delay includes the observation time required to satisfy the minimum displacement and persistence conditions. Figure 2 shows the structural diagram of TRA-PTR.

Figure 2. Structure of TRA-PTR. Different colors distinguish different functional modules.

The entire system is centered on vehicle trajectories, integrating the full process—from object detection to cross-frame identity correlation and traffic rule determination—with clear decision-making logic and a traceable process. It can be effectively applied to the automated identification of abnormal driving behaviors in surveillance scenarios.

4.4. Event Generation and Evidence-Based Output

Once the reasoning module identifies abnormal driving behavior, the system converts the determination results into standardized event records for manual review and subsequent traffic management. Each individual abnormal event record includes the vehicle identification number, event type, time the event was triggered, the vehicle’s spatial location, and the complete rule evaluation criteria, along with accompanying key-frame screenshots as evidence. By fully preserving the assessment rules and parameters, the system eliminates the “black box” issue associated with alerts, ensuring that every anomaly event is explainable and traceable.

To prevent the same vehicle from repeatedly triggering the same type of alert across consecutive video frames, the system incorporates an event cooling mechanism. If the same trajectory meets the same anomaly detection rule multiple times within a cooling period, the system records only the first triggered alert and resumes detection only after the cooling period ends. This mechanism significantly reduces redundant alert information, enhancing the readability and practical value of the alert list.

From an overall operational perspective, the framework described in this paper forms a complete business closed-loop: the improved YOLOv11 model outputs stable and reliable vehicle detection results; the multi-object tracking algorithm converts discrete detection results into continuous motion trajectories; the trajectory data provides spatiotemporal evidence for traffic-rule reasoning; and the reasoning module ultimately generates interpretable anomaly events and forensic evidence. This integrated workflow works in concert to comprehensively meet practical operational requirements—including object detection, trajectory tracking, anomaly alerts, and evidence retention—in intelligent traffic monitoring scenarios.

5. Experimental Verification

To evaluate the detector from complementary perspectives, experiments are conducted on COCO2017 and VisDrone2019-DET. COCO2017 is used to examine general object-detection performance and cross-scenario generalization, whereas VisDrone2019-DET is used only as a complementary robustness benchmark for small, dense, occluded, and scale-varying objects. The detailed characteristics of the two datasets are presented in Table 1.

Table 1. Datasets.

Name	Number of Images	Category	Resolution	Purpose
COCO2017	118,287/5000	80	Variable	Validation
VisDrone2019-DET	6471/548/1610	10	Variable	Detector robustness/Ablation

VisDrone2019-DET primarily contains aerial-view images captured by drone-mounted cameras rather than images acquired by fixed roadside surveillance cameras [36]. Its mobile aerial viewpoints, viewing geometry, and object-scale distribution differ from those of the fixed-camera roadside setting targeted by TRA-PTR. Therefore, VisDrone is not treated as a direct representation of fixed roadside surveillance scenarios and is not used to claim direct deployment performance in such scenes. Instead, it provides a complementary stress test of detector robustness under small-object, dense-object, occlusion, and scale-variation conditions. The official predefined train/validation/test-dev split of 6471/548/1610 images is retained without random redistribution. The detector is trained on the official training partition, and the quantitative comparisons and ablation results reported in this paper use the official validation partition.

In addition, the COCO2017 dataset is introduced to further evaluate the cross-scenario generalization ability of the proposed model. COCO2017 contains 118,287 training images and 5000 validation images, covering 80 common object categories with rich scene diversity. The dataset includes various road-related environments, such as urban intersections, open roads, and residential streets, while covering diverse viewpoints, weather conditions, illumination variations, and complex background interference. These characteristics enable a comprehensive evaluation of the model’s detection stability under different environmental conditions.

The event-level evaluation uses BrnoCompSpeed [37], ISLab-PVD [38], and Overhead Traffic Anomalies [39]. Their main characteristics are summarized in Table 2.

Table 2. Video datasets for traffic-violation evaluation.

Name	Videos	Sites	Resolution/FPS	Reference or Annotation	Purpose
BrnoCompSpeed	18	6	$1920\times 1080$/50	20,865 LiDAR/GPS vehicle speeds	Speeding
ISLab-PVD	16	-	-	21 reported illegal-parking events	Illegal parking
Overhead Traffic Anomalies	-	3	$320\times 180$	1027 anomaly intervals; 8 wrong-way events	Wrong-way driving

The experimental setup for this paper consists of a Linux deep learning server equipped with an AMD Ryzen 9 5900X CPU (Advanced Micro Devices, Inc., Santa Clara, CA, USA) and a single NVIDIA GeForce RTX 4090 GPU (NVIDIA Corporation, Santa Clara, CA, USA) with 24 GB memory. The graphics card driver version is 595.71.05, and the CUDA version is 13.2.

5.1. Experiment Settings

5.1.1. Event-Level Evaluation Protocol

Continuous videos are divided at the clip level, and all observations of the same vehicle trajectory are retained in one subset to prevent leakage between temporally adjacent frames. Two annotators independently verify the vehicle trajectory, event category, and event start and end frames; disagreements are resolved by consensus. Event matching and the evaluation metrics are defined below.

Let $I_m^{\det }=[t_{m,s}^{\det },t_{m,e}^{\det }]$ and $I_n^{\mathrm{gt}}=[t_{n,s}^{\mathrm{gt}},t_{n,e}^{\mathrm{gt}}]$ denote the system-detected and ground-truth temporal intervals, respectively. Their temporal intersection over union is defined as

$${\mathrm{IoU}}_t(m,n)={\displaystyle \frac{\left|I_m^{\det }\cap I_n^{\mathrm{gt}}\right|}{\left|I_m^{\det }\cup I_n^{\mathrm{gt}}\right|}}.$$

(17)

A system-detected event is counted as a true positive when it is associated with the same vehicle, has the same event category, and satisfies ${\mathrm{IoU}}_t\ge 0.5$. An unmatched or duplicate system detection is counted as a false positive, whereas an unmatched ground-truth event is counted as a false negative. Based on the numbers of true positives ($TP$), false positives ($FP$), and false negatives ($FN$), event-level precision, recall, and F1-score are calculated as

$$P={\displaystyle \frac{TP}{TP+FP}},$$

(18)

$$R={\displaystyle \frac{TP}{TP+FN}},$$

(19)

$$F_1={\displaystyle \frac{2PR}{P+R}}.$$

(20)

The missed-alarm rate ($MAR$) and false-alarm rate ($FAR$), reported as percentages, are defined as

$$MAR={\displaystyle \frac{FN}{TP+FN}}\times 100\%,$$

(21)

$$FAR={\displaystyle \frac{FP}{FP+TN}}\times 100\%.$$

(22)

For every correctly matched event, the alarm delay is calculated from the ground-truth event onset $t_{n,s}^{\mathrm{gt}}$ to the first valid alarm time $t_m^{\mathrm{alarm}}$:

$$D_m=t_m^{\mathrm{alarm}}-t_{n,s}^{\mathrm{gt}}.$$

(23)

For each violation category $q\in \{s,p,w\}$, the single delay value reported in the corresponding results table is the mean over all correctly matched events:

$${\overline{D}}_q={\displaystyle \frac{1}{T{P}_q}}{\displaystyle \sum _{m=1}^{T{P}_q}}\left(t_m^{\mathrm{alarm}}-t_{m,s}^{\mathrm{gt}}\right).$$

Trajectory-level event-recognition accuracy is calculated over the four mutually exclusive labels: normal, speeding, illegal parking, and wrong-way driving:

$$Ac{c}_{\mathrm{event}}={\displaystyle \frac{N_{\mathrm{correct}}}{N_{\mathrm{all}}}}.$$

(24)

For speed evaluation, let $v_i^{\mathrm{sys}}$ and $v_i^{\mathrm{gt}}$ denote the system-computed speed and the independently measured reference speed of the ith vehicle. The mean absolute error and root mean squared error are defined as

$$MA{E}_v={\displaystyle \frac{1}{N_v}}{\displaystyle \sum _{i=1}^{N_v}}\left|v_i^{\mathrm{sys}}-v_i^{\mathrm{gt}}\right|,$$

(25)

$$RMS{E}_v=\sqrt{{\displaystyle \frac{1}{N_v}}\sum _{i=1}^{N_v}{\left(v_i^{\mathrm{sys}}-v_i^{\mathrm{gt}}\right)}^2},$$

(26)

where $N_v$ is the number of vehicles with independent speed ground truth obtained from a calibrated radar, GPS/OBD record, or an equivalent calibrated reference system.

The traffic-rule parameters are fixed before test evaluation. For speeding recognition, the primary controlled speed-limit threshold is $v_{\lim }=60$ km/h; additional evaluations at 50 and 70 km/h are used to examine threshold sensitivity. This controlled threshold is used only to convert the continuous reference and system-computed speeds into event labels and does not affect the calculation of MAE, RMSE, or median speed error. For illegal-parking recognition, the main dwell-time threshold is ${\tau }_{p^{\prime }}=60$ s, consistent with the alarm definition used in prior ISLab-PVD evaluation [40]; the maximum road-plane displacement is ${\delta }_p=0.5$ m. Sensitivity settings of 30 and 90 s for ${\tau }_{p^{\prime }}$ and 0.25 and 1.0 m for ${\delta }_p$ are also considered. For wrong-way driving recognition, the main direction threshold is $\eta =-0.5$, corresponding to an angular disagreement of at least ${120}^{\circ }$; the minimum valid road-plane displacement is ${\delta }_r=1.0$ m, and the directional condition must persist for $T_w=1.0$ s. Sensitivity settings are $\eta \in \{0,-0.5,-0.866\}$, ${\delta }_r\in \{0.5,1.0,2.0\}$ m, and $T_w\in \{0.5,1.0,2.0\}$ s. All metric displacement thresholds are applied after converting image trajectories to calibrated road-plane coordinates using the scene scale factor $\rho$.

5.1.2. COCO val2017 Generalization Contrast Experiment

COCO val2017 is used to evaluate a model’s generalization ability in general object detection scenarios. This experiment not only compares the model with the YOLO series but also includes representative real-time detectors from the past three years, such as RT-DETR and RF-DETR, to illustrate the overall trade-off between accuracy, the number of parameters, and computational complexity in Table 3.

Table 3. Comparison Results of Detection Models Based on COCO val2017.

Model	AP/%	AP50/%	AP75/%	Params/M	GFLOPs/G
YOLOv8n	35.2	49.2	38.3	3.2	8.7
YOLOv8s	42.4	57.6	46.0	11.2	28.6
YOLOv8m	47.3	62.5	51.5	25.9	78.9
YOLOv11-n	37.1	51.6	40.4	2.6	6.5
YOLOv11-s	44.1	59.3	47.9	9.4	21.5
YOLOv11-m	48.3	63.6	52.5	20.1	68.0
RT-DETR-R50	55.0	73.3	59.8	42.0	136.0
RF-DETR-N	48.0	67.0	51.4	30.5	31.9
RF-DETR-S	52.9	71.9	57.0	32.1	59.8
RF-DETR-M	54.7	73.5	59.2	33.7	78.8
TRA-PTR	47.2	63.4	50.9	12.6	36.4

Note: Boldface identifies TRA-PTR and its results.

As shown in Table 3, the TRA-PTR model achieves percentage-point gains of 3.1, 4.1, and 3.0 in AP, AP50, and AP75, respectively, compared with YOLOv11-s on generalization tasks. This indicates that the combination of HMAB, BiFPN-CSAF, P2, and EMA in TRA-PTR is not only effective for traffic monitoring scenarios but also demonstrates a certain degree of generalization capability for object detection. Compared to YOLOv11-m, the TRA-PTR model has an AP that is 1.1 percentage points lower, but its number of parameters is reduced by approximately 37.3%, and its GFLOPs by approximately 46.5%, enabling it to achieve detection accuracy close to that of medium-sized models with lower computational complexity. Compared to detection Transformer models such as RF-DETR and RT-DETR, the TRA-PTR model does not achieve the highest accuracy, but it has significantly fewer parameters, providing a lower-complexity basis for subsequent deployment-oriented evaluation.

5.1.3. Comparative Experiments on the VisDrone Dataset

VisDrone2019-DET is a drone-based aerial-view benchmark whose acquisition geometry is different from that of fixed roadside surveillance. It is included here only as a complementary detector-robustness benchmark because its small objects, high object density, occlusions, scale variation, and complex backgrounds provide a useful stress test for the proposed feature-fusion and small-object modules. Following the official predefined split, the model is trained on the 6471-image training partition, while the results in this subsection and the subsequent ablation experiments are reported on the 548-image validation partition; no custom random repartitioning is performed. Accordingly, these results evaluate robustness to difficult object-detection conditions and should not be interpreted as direct evidence of performance in fixed roadside surveillance scenes.

The comparison results in Table 4 show that TRA-PTR improves mAP50:95 and mAP50 by 4.8 and 5.5 percentage points, respectively, compared with YOLOv11-s, indicating that the modules designed for small objects, densely clustered objects, and complex backgrounds improve vehicle-detection performance. Compared with YOLOv11-m, TRA-PTR improves mAP50:95 and mAP50 by 1.2 and 0.6 percentage points, respectively, while reducing the parameter count by approximately 21.0% and GFLOPs by approximately 41.5%. Relative to YOLOv11-l, TRA-PTR achieves competitive detection performance while using fewer parameters and lower computational complexity. Although the reported results are lower than those of FBRT-L, TRA-PTR has substantially lower computational complexity; it is therefore more appropriate to describe the method as achieving competitive detection performance at a lower computational cost rather than claiming overall superiority. Figure 3 compares the detection accuracy of the different methods on the two datasets. Figure 3a,b present the COCO val2017 and VisDrone results, respectively.

Table 4. Performance Comparison of Different Detection Models on the VisDrone Dataset.

Method	Year	mAP50:95/%	mAP50/%	Params/M	GFLOPs/G
YOLOv8-n	2023	19.5	33.5	3.0	8.1
YOLOv8-s	2023	23.7	39.4	11.1	28.5
YOLOv8-m	2023	26.5	43.4	25.8	78.7
YOLOv8-l	2023	27.9	44.9	43.6	164.9
YOLOv11-n	2024	19.6	33.4	2.6	6.3
YOLOv11-s	2024	23.8	39.7	9.4	21.3
YOLOv11-m	2024	27.4	44.6	20.0	67.7
YOLOv11-l	2024	28.1	45.1	25.3	86.6
YOLOv12-n	2025	19.2	33.1	2.6	6.3
YOLOv12-s	2025	23.8	39.5	9.2	21.2
YOLOv12-m	2025	24.4	40.9	20.2	67.5
YOLOv12-l	2025	25.1	40.5	27.6	88.4
FBRT-N	2025	20.2	34.4	0.9	6.7
FBRT-S	2025	25.9	42.4	2.9	22.9
FBRT-M	2025	28.4	45.9	7.2	58.7
FBRT-L	2025	29.7	47.7	14.6	119.2
RT-DETR-r18	2024	25.1	41.4	20.0	57.0
EL-YOLO	2025	24.8	42.9	6.7	1.08
EBC-YOLO	2025	26.7	44.3	35.5	10.2
CF-YOLO	2025	27.5	44.9	23.9	3.77
Enhanced YOLOv11-n	2026	17.2	31.1	2.37	11.9
TRA-PTR	-	28.6	45.2	15.8	39.6

Note: Boldface identifies TRA-PTR and its results.

Figure 3. Comparison of the detection accuracy of different methods based on two datasets. (a) Detection accuracy results for COCO val2017. (b) Detection accuracy results for VisDrone.

5.2. VisDrone Ablation Experiments

Ablation experiments were conducted to verify the independent contributions and combined benefits of each improved module. Following the comparison methodology used for CF-YOLO and Enhanced YOLOv11-n, this paper employs a step-by-step module stacking approach to examine the impact of HMAB, BiFPN-CSAF, the P2 small-object detection branch, and the EMA detection head enhancement on model performance. The final results in Table 5 are consistent with those in Table 4, with the complete model achieving mAP50:95 and mAP50 values of 28.6% and 45.2%, respectively.

Table 5. TRA-PTR Ablation Results on the VisDrone Dataset.

Method	mAP50:95/%	mAP50/%	Params/M	GFLOPs/G	ΔmAP50/%
M1	23.8	39.7	9.4	21.3	—
M2	24.6	40.8	10.2	24.8	+1.1
M3	25.2	41.7	11.0	28.6	+2.0
M4	25.8	42.8	10.7	30.7	+3.1
M5	24.4	40.5	9.9	22.5	+0.8
M6	26.4	43.4	12.1	31.2	+3.7
M7	27.8	44.6	14.9	38.9	+4.9
TRA-PTR	28.6	45.2	15.8	39.6	+5.5

Note: Boldface identifies TRA-PTR and its results; the em dash denotes the baseline.

Here, M1 = YOLOv11-s, M2 = YOLOv11-s + HMAB, M3 = YOLOv11-s + BiFPN-CSAF, M4 = YOLOv11-s + P2, M5 = YOLOv11-s + EMA, M6 = YOLOv11-s + HMAB + BiFPN-CSAF, and M7 = YOLOv11-s + HMAB + BiFPN-CSAF + P2. Figure 4 shows the comparison results of the ablation experiments based on the VisDrone dataset.

Figure 4. Ablation experiments based on the VisDrone dataset.

Ablation analysis: Based on Table 5 and Figure 4, it can be concluded that, when HMAB is introduced alone, it primarily enhances the representation of local-to-global features in complex backgrounds, with an observed 1.1 percentage-point gain in mAP50; BiFPN-CSAF mitigates the imbalance between shallow-level details and deep-level semantic contributions through cross-scale weighted fusion, further expanding the improvement; the P2 branch directly enhances the participation of high-resolution shallow-level features in detection, with the most significant effect on the recall of small vehicles at long distances, resulting in the largest mAP50 improvement when introduced alone; EMA performs spatial recalibration of detection head features; while its standalone improvement is limited, it further enhances the response of the classification and localization branches to valid regions within TRA-PTR. Overall, TRA-PTR achieves a 5.5 percentage-point gain in mAP50 compared to the YOLOv11-s baseline, representing a relatively robust yet not overly exaggerated improvement.

5.3. Analysis of Object Detection Performance

Scale analysis: The results in Table 6 align with the general trends observed in improvements to small-scale object detection: the P2 branch and cross-scale fusion yield the most significant improvements in APs, while HMAB provides consistent gains for small- and medium-scale objects in occluded and complex scenes; AP1 remains largely stable, indicating that the improved modules primarily target distant, small-scale, and densely clustered vehicle objects without significantly compromising the detection capability for large objects. The AP of the complete model is consistent with the mAP50:95 values in Table 4, both standing at 28.6%.

Table 6. Analysis of object detection performance at different scales.

Model	APs/%	APm/%	APl/%	AP/%
YOLOv11-s	12.9	31.8	42.5	23.8
+ HMAB	13.7	32.3	42.6	24.6
+ BiFPN-CSAF	14.2	32.9	42.3	25.2
+ BiFPN-CSAF + P2	16.4	34.1	42.1	27.8
TRA-PTR	17.1	34.8	42.5	28.6

Figure 5 presents qualitative detection examples selected from the COCO val2017 dataset. These representative street-level traffic scenes are used to illustrate the cross-dataset generalization of TRA-PTR for vehicle targets at different scales.

Figure 5. Qualitative detection results of TRA-PTR on representative street-level images selected from the COCO val2017 dataset.

Tracking analysis: The results in Table 7 demonstrate that the proposed framework improves tracking stability while retaining measurable processing throughput on the specified experimental server. Compared with YOLOv11-s + BT, TRA-PTR + BT reduces ID switches from 96 to 82 and trajectory fragmentation from 138 to 121, while improving MOTA from 62.5% to 66.4% and IDF1 from 65.8% to 68.9%. These improvements indicate that the enhanced detector provides more reliable object observations for the tracking module, which helps maintain trajectory continuity under challenging scenarios involving small objects and severe occlusions. Furthermore, the rule-based buffering mechanism (RB) helps alleviate temporary detection failures and prevent premature trajectory termination. After incorporating RB, TRA-PTR + BT + RB further reduces ID switches to 76 and trajectory fragmentation to 112, while increasing MOTA and IDF1 to 67.8% and 70.1%, respectively, extending AvgTL to 82.3 frames, and yielding an observed throughput of 35.8 FPS. This FPS value describes throughput in the implemented experimental processing loop on the RTX 4090 server; it is not a standardized end-to-end deployment-latency measurement and should not be interpreted as evidence of suitability for a particular edge device or multi-camera deployment.

Table 7. Tracking Stability and Processing-Throughput Results.

Method	ID Switch ↓	Frag ↓	MOTA/% ↑	IDF1/% ↑	AvgTL ↑	FPS ↑
YOLOv11-s + BT	96	138	62.5	65.8	71.4	42.3
TRA-PTR + BT	82	121	66.4	68.9	79.6	37.4
TRA-PTR + BT + RB	76	112	67.8	70.1	82.3	35.8

Note: Downward and upward arrows indicate that lower and higher values are better, respectively.

5.4. Event-Level Traffic-Violation Recognition

The three violation types are evaluated separately. Table 8, Table 9 and Table 10 report the binary confusion counts and event-level metrics for each task.

Table 8. Speeding-event recognition and speed-estimation results on BrnoCompSpeed.

Dataset	TP	FP	FN	TN	P/%	R/%	F1/%	Acc./%	MAR/%	FAR/%	Mean Delay/s
BrnoCompSpeed	4329	448	891	15,197	90.62	82.93	86.61	93.58	17.07	2.86	2.13
Dataset	Speed Limit/(km/h)			GT Vehicles		MAE/(km/h)		RMSE/(km/h)		Median Error/(km/h)
BrnoCompSpeed	60			20,865		0.62		0.86		0.62

Table 9. Illegal-parking event-recognition results on ISLab-PVD.

Dataset	TP	FP	FN	TN	P/%	R/%	F1/%	Acc./%	MAR/%	FAR/%	Mean Delay/s
ISLab-PVD	19	1	2	378	95.00	90.48	92.68	99.25	9.52	0.26	2.89

Table 10. Wrong-way driving event-recognition results on Overhead Traffic Anomalies.

Dataset	TP	FP	FN	TN	P/%	R/%	F1/%	Acc./%	MAR/%	FAR/%	Mean Delay/s
Overhead Traffic Anomalies	7	1	1	1018	87.50	87.50	87.50	99.81	12.50	0.10	3.34

Event-level analysis: Unlike the preceding frame-level detection experiments, the results in Table 8, Table 9 and Table 10 were obtained by processing continuous video sequences through the complete detection–tracking–rule-reasoning pipeline under the clip-level protocol described above. On BrnoCompSpeed, the framework correctly recognized 4329 speeding events, with 90.62% precision, 82.93% recall, an F1-score of 86.61%, a false-alarm rate of 2.86%, and a mean alarm delay of 2.13 s. The corresponding system-computed speeds achieved an MAE of 0.62 km/h and an RMSE of 0.86 km/h against the provided reference measurements. On ISLab-PVD, illegal-parking recognition achieved an F1-score of 92.68%, a false-alarm rate of 0.26%, and a mean alarm delay of 2.89 s. On Overhead Traffic Anomalies, wrong-way driving recognition achieved an F1-score of 87.50%, a false-alarm rate of 0.10%, and a mean alarm delay of 3.34 s. Here, the reported delay is the interval between the independently verified ground-truth event onset and the first valid rule-based alarm, and therefore includes the temporal evidence required by the corresponding event rule rather than only network processing time. These measured results directly validate the complete TRA-PTR pipeline; however, the limited numbers of positive illegal-parking and wrong-way events indicate that broader multi-scene validation remains necessary.

6. Results

6.1. Discussion

A comprehensive analysis of the COCO and VisDrone datasets reveals that the TRA-PTR model’s strengths lie primarily in the detection of small vehicle targets and improved trajectory stability in complex traffic scenarios. The COCO generalization experiments demonstrate that the TRA-PTR model proposed in this paper achieves detection accuracy superior to that of YOLOv11-s while maintaining a moderate parameter size, and delivers performance close to that of YOLOv11-m with significantly lower computational complexity. On the complementary aerial-view VisDrone benchmark, the combination of HMAB, BiFPN-CSAF, P2, and EMA improves the detection of small-scale and densely distributed objects under occlusion and complex backgrounds. Because VisDrone is primarily captured from mobile drone platforms, these results demonstrate detector robustness to difficult object scales and densities rather than direct effectiveness in fixed roadside surveillance scenes. Ablation experiments show that the P2 branch contributes most significantly to small-object detection, that BiFPN-CSAF contributes substantially to multi-scale feature fusion, and that HMAB and EMA enhance the model’s robustness at the feature representation and detection head response levels, respectively. Since the front-end detection results further influence cross-frame identity association, the improved detector can also indirectly enhance tracking stability and provide a more reliable trajectory foundation for subsequent rule-based recognition of speeding, illegal parking, and driving against traffic.

From a road-safety and traffic-management perspective, the proposed framework connects vehicle perception with local trajectory reconstruction and interpretable traffic-rule reasoning in fixed-camera monitoring areas. The resulting time-stamped event records, including the tracked vehicle identifier, location, motion state, and associated image evidence, can support the screening and retrospective review of suspected speeding, illegal parking, and wrong-way driving events. At urban intersections and monitored road sections, aggregated event records may also help traffic managers identify recurring spatial and temporal patterns, such as locations or periods with frequent abnormal driving behavior, thereby providing evidence for targeted enforcement review, adjustment of monitoring configurations, and subsequent safety assessment. The continuous-video experiments directly validate this complete processing chain using event-recognition accuracy, false- and missed-alarm rates, and alarm delay. Nevertheless, its operational benefits for enforcement efficiency, traffic control, and accident prevention still require broader multi-scene benchmarks and real-road pilot studies.

6.2. Limitations and Future Work

Although TRA-PTR improves small-vehicle perception and supports trajectory-based traffic-rule reasoning, several limitations remain. The HMAB, BiFPN-CSAF, P2, and EMA modules introduce additional computational overhead, and further lightweight optimization is required for low-power edge devices and parallel processing of multiple video streams. The current framework is primarily designed for single-camera monitoring of urban intersections, arterial road sections, entrances, and exits, where traffic directions and rule regions can be predefined. Its applicability to rural roads has not been systematically validated. Rural environments may differ substantially in road geometry, traffic density, vehicle speed, illumination, camera coverage, and rule-region configuration; therefore, parameters calibrated for urban scenes should not be assumed to transfer directly. Because the present system relies on ordinary surveillance views and does not observe vehicle underbody structure, it does not support axle-based vehicle classification; reliable axle number estimation would require dedicated sensing such as inductive loops or weigh-in-motion systems.

The FPS results in Table 7 quantify observed processing throughput under the stated server configuration rather than standardized deployment performance. The current experiments do not separately measure end-to-end latency including video acquisition, decoding, data transfer, and output handling; they also do not report peak CPU/GPU memory consumption or per-stage runtimes for preprocessing, detection, tracking, and rule reasoning. Consequently, the reported throughput is insufficient to establish suitability for latency-constrained edge deployment or simultaneous multi-camera operation. Standardized profiling on representative target hardware, with synchronized timing and explicit measurement boundaries, is required before deployment-oriented conclusions can be drawn.

The reliability of speeding, illegal parking, and wrong-way driving recognition depends on the quality of the complete perception–tracking–reasoning chain. The effective monitoring range is determined jointly by the model and the camera resolution, focal length, installation geometry, image clarity, visibility, and vehicle pixel size. Insufficient resolution, adverse weather, glare, motion blur, prolonged occlusion, or trajectory crossing may cause localization errors, fragmented trajectories, or identity switches, which can propagate to errors in calculated speed, dwell time, and movement direction. The BrnoCompSpeed experiment compares the system-computed speeds with the reference measurements provided by the dataset; however, the road-plane trajectories and event decisions on the other datasets have not been cross-validated using synchronized external sensors such as calibrated radar, GPS, or roadside detectors. They should therefore be interpreted as intermediate evidence for traffic-event analysis rather than certified physical measurements. Certified radar remains preferable for formal speed enforcement, whereas video analysis provides complementary vehicle detection, tracking, trajectory analysis, and multi-event screening capabilities. The present experiments also do not constitute a field evaluation of accident reduction, enforcement efficiency, or broader traffic-management outcomes. In practical deployment, the acquisition, storage, transmission, and processing of surveillance video should comply with applicable privacy and data-protection regulations. Personally identifiable information, such as license plates and facial features, should be protected through access control, data minimization, anonymization, or other appropriate safeguards.

Future work will focus on model lightweighting, standardized end-to-end latency and memory profiling, per-stage runtime analysis on server and edge platforms, camera-aware monitoring range and perspective calibration, rural road adaptation, robust association of visually similar vehicles, and targeted evaluation of violation recognition under rain, fog, nighttime, glare, and other adverse conditions. To extend the framework beyond speeding, illegal parking, and wrong-way driving, future studies will incorporate traffic-signal recognition, lane- and stop-line segmentation, vulnerable-road-user tracking, and interaction-aware rule reasoning. External sensors will be introduced to further validate trajectory reconstruction and speed calculation outputs, while larger and more diverse event-level benchmarks and real-road pilot studies will be used to assess false alarms, missed alarms, alarm delay, and operational value for traffic management. Optimal multi-camera placement, multi-view information fusion, cross-camera vehicle association, continuous trajectory reconstruction, and fusion with radar, inductive loops, and weigh-in-motion systems will also be investigated to improve the robustness and practical applicability of traffic-violation detection.