Optimization of Indoor Pedestrian Counting Based on Target Detection and Tracking

Abstract

Real-time, precise monitoring of the number and distribution of indoor personnel is crucial for building safety management, operational optimization, and personnel scheduling. However, narrow entrances and high-density passageways often lead to missed detections, false positives, and tracking failures in pedestrian detection, thereby reducing cross-line counting accuracy. Additionally, edge devices deployed in practical scenarios frequently process multiple video streams simultaneously, resulting in computational resource constraints. To address these challenges, this paper proposes a lightweight, enhanced multi-object pedestrian tracking and counting method tailored for indoor scenarios by optimizing deep learning models. Firstly, modular optimizations are applied to the YOLOv8n model to construct a more lightweight detector, RL_YOLOv8, reducing computational overhead while maintaining accuracy. Secondly, correlated pedestrian auxiliary prediction and pedestrian position change constraints are employed to mitigate ID switching, tracking interruptions, and trajectory jumps in dense scenes. Finally, a buffer zone auxiliary counting strategy is designed to further reduce missed detections of pedestrians crossing lines. Experimental results demonstrate that compared to the original detection-and-tracking-based line-crossing counting method, the improved approach effectively enhances counting accuracy and real-time performance, better meeting the requirements of practical intelligent security and crowd monitoring systems.

1. Introduction

Pedestrian cross-line counting aims to estimate the number of pedestrians passing a designated counting line within a given time interval [1]. It has been widely used in intelligent surveillance, shopping mall footfall analysis, and traffic management, playing an important role in public-safety assurance and operational optimization [2]. Existing cross-line counting methods can generally be categorized into two groups. The first group combines density-map estimation [3] with optical flow [4] or velocity-field information for flux-based flow estimation, whereas the second group performs individual counting based on detection and tracking [5]. The former computes crossing counts by integrating the crowd density over the flow region intersecting the counting line and is well suited for open areas or macro-level flow statistics. However, in typical indoor scenarios such as teaching building corridors and classroom entrances, camera viewpoints are often constrained, pedestrian trajectories are complex, and short-term back-and-forth motion occurs frequently. These characteristics make density-map-and-flow-based methods difficult to reliably associate with fine-grained line-crossing events, thereby limiting their applicability to precise indoor counting tasks [6].

Although existing detection-and-tracking frameworks can provide individual trajectories, they still face severe challenges in indoor environments. First, double-line (gate-based) crossing rules [7] can suppress duplicate counts caused by short-term oscillations, but in narrow indoor spaces, pedestrians may not fully traverse both lines, which can lead to missed detections and counting bias [8]. Second, heavy occlusions under high crowd density significantly degrade bounding-box stability; jitter in the lower-body region easily causes trajectory interruptions and crossing misjudgments. Some studies use the head region as a more stable reference point [9,10], yet in teaching building scenes, head motion is not always consistent with full-body crossing behavior, and near-wall regions may cause the head to miss the counting line, resulting in unreliable judgments.

To address these issues, this paper proposes a lightweight detection-and-tracking-based counting method for indoor scenarios. We introduce modular improvements to YOLOv8n [11] to build an efficient detector referred to as RL_YOLOv8, and develop an occlusion-robust trajectory refinement scheme that combines bottom-boundary suppression and correction with related-pedestrian-assisted motion prediction. This design mitigates occlusion-induced bounding-box jitter and tracking drift, thereby improving trajectory stability and the reliability of cross-line judgment. The experimental results demonstrate that the proposed method achieves notable gains in both counting accuracy and processing efficiency in dense indoor environments.

2. System Framework and Optimization Methods

2.1. System Framework and Process

The fundamental framework of the indoor pedestrian cross-line counting system based on video surveillance comprises three primary components: pedestrian detection, pedestrian tracking, and cross-line counting [12]. Initially, video frames are extracted from the surveillance footage, and the YOLO [13] detector is employed to identify pedestrian targets. Next, the DeepSort [14] tracker processes the pedestrian position and boundary information, performing ID association across consecutive frames to establish stable trajectories. Subsequently, the two nearest adjacent points within the pedestrian trajectory are combined to create motion vectors, which are then used in vector operations with the endpoints of the counting line to ascertain whether the pedestrian has crossed it. The direction of pedestrian movement is determined using the vector cross product, facilitating the calculation of bidirectional pedestrian flow. To improve the accuracy and efficiency of the counting system, this paper proposes the following optimizations for the system model and process:

(1)

Make lightweight improvements to the YOLOv8n model, reduce its size, and lower the consumption of computing resources while ensuring its detection performance.

(2)

Pedestrian trajectories are smoothed by correcting pedestrian position, which reduces the occurrence of trajectory jumps and enhances the stability of the trajectories.

(3)

The motion trends of related pedestrians are utilized to correct the predicted trajectory of the occluded pedestrians, significantly enhancing the accuracy of predicting the positions of occluded pedestrians.

(4)

An auxiliary counting zone is designed to reduce counting mistakes that are prone to occur in narrow areas, which will enhance the accuracy of counting pedestrians.

The overall optimization process is illustrated in Figure 1, where the yellow sections are improved. In the detecting phase, a more lightweight detector RL_YOLOv8 is constructed to reduce computational overhead while maintaining accuracy. In the tracking phase, the correction and prediction of the pedestrian position is proposed to make the predicted trajectory more closely align with the actual trajectory, thereby alleviating interruptions caused by occlusion.

Figure 1. The overall flowchart of the optimized pedestrian counting system.

2.2. Improvement of Pedestrian Target Detection

Pedestrian target detection serves as the foundation for cross-line counting. Traditional detection methods, which depend on manual features and sliding windows, face challenges such as high computational cost and low efficiency, making it difficult to satisfy real-time requirements. With advancements in deep learning, object detection techniques utilizing convolutional neural networks (CNNs) have gradually gained prominence, and are typically categorized into two types: two-stage and single-stage methods [15]. Two-stage methods, such as Faster R-CNN, initially generate candidate regions for subsequent classification and regression. While these methods achieve high detection accuracy, their inference speed is comparatively low, which complicates deployment in scenarios that demand high real-time performance [16]. In contrast, single-stage methods, including SSD, RetinaNet, and the YOLO series, accomplish target localization and classification in a single forward pass. These methods are characterized by their compact structure and high speed, making them the preferred choice for real-time target detection. Among these, the YOLO series stands out due to its end-to-end architecture and exceptional inference efficiency, making it a widely adopted choice for real-time object detection.

Despite the continuous advancements in detection speed and accuracy of the YOLO series models, optimization for specific application scenarios remains essential. The teaching building environment presents two primary challenges: First, the large number of cameras produces a substantial volume of data, necessitating a lightweight model to improve throughput and deployment efficiency. Second, as illustrated in Figure 2, the scale variations of pedestrians in indoor settings are considerable due to varying camera–target distances and strong perspective effects [17], requiring the model to possess robust multiscale detection capabilities [18]. This paper proposes an optimization model in response to the requirements above.

Figure 2. The corridor scene of a teaching building. In the scene, the detection bounding boxes of four pedestrians at different positions are 126 × 309, 119 × 328, 46 × 124, and 32 × 82, respectively.

Firstly, the RGCSPELAN (Residual Ghost Channel Spatial Pyramid Efficient Layer Attention Network) [19] modules are used to replace some C2f modules in the YOLOv8 structure to reduce the computational load while preserving detection accuracy. By integrating the lightweight design philosophy of GhostNet and the efficient feature aggregation strategy of the ELAN architecture, RGCSPELAN effectively lowers computational complexity and improves inference efficiency, which is particularly beneficial for deployment on resource-constrained devices. Secondly, the detection head is replaced with the LSCD (Light-weight Spatial-channel Dual-attention) module to minimize computational redundancy and enhance the detection capability of multiscale targets. Consequently, the proposed RL_YOLOv8 model (Figure 3) achieves substantial reductions in parameters and computational complexity while maintaining comparable detection accuracy, making it well suited for practical indoor applications with constrained computing resources.

Figure 3. The structure of the RL_YOLOv8 model.

2.2.1. Lightweight Module Design: RGCSPELAN

On resource-constrained edge devices, especially in scenarios where multiple video streams need to be processed simultaneously, enhancing the model’s lightweight characteristics becomes crucial [20]. While YOLOv8n achieves high accuracy in handling complex environments and occluded pedestrians, its computational load is relatively heavy. To reduce the computational load of YOLOv8n, we introduced the RGCSPELAN module. RGCSPELAN, by integrating the advantages of CSPNet and ELAN (Efficient Layer Aggregation Networks), reduces computational complexity and optimizes gradient flow and feature aggregation, significantly improving inference speed and ensuring efficient operation on resource-constrained devices.

RepConv (Re- parameterization Convolution) [21] in the RGCSPELAN module enhances feature expression in occluded and complex target scenarios while improving inference efficiency. The RepConv structure, illustrated in Figure 4, employs convolution re-parameterization for optimizing computational efficiency. During the training phase, three branches, 3 × 3 Conv, 1 × 1 Conv, and BatchNorm (BN), are utilized for parallel computing to optimize feature extraction. The output feature maps from these branches are then summed element-wise and passed through an activation function layer. In the inference stage, the multiple branches are fused into a single 3 × 3 convolution using the convolution re-parameterization method. This approach not only preserves the original feature capabilities but also reduces the model’s computational load and memory consumption.

Figure 4. The structure of RepConv.

2.2.2. LSCD Detection Head

The YOLOv8n architecture incorporates three detection heads, each designed to manage targets of varying scales. However, this configuration imposes a substantial computational burden during deployment. Each detection head features an independent convolutional layer and normalization module, leading to significant parameter overhead and redundant computations, which restricts the model’s practicality on edge devices [22]. To solve the problem, our approach adopts a lightweight shared convolutional detection LSCD head, illustrated in Figure 5.

Figure 5. The structure of the LSCD module.

The LSCD head is designed to balance lightweight architecture and inference efficiency. A shared convolutional mechanism, where GroupNorm (GN) is incorporated to replace Batch Normalization (BN), is employed in it. Existing studies have demonstrated that Group Normalization (GN) can enhance the stability of gradient flow in the regression and classification branches within object detection frameworks, thereby benefiting overall performance [23]. The structure utilizes 1 × 1 convolution to process input features from the P3, P4, and P5 levels, followed by two 3 × 3 shared convolutions to fuse these features. Finally, a learnable scale-adaptive module, Scale Layer, is used to dynamically adjust the scale distribution of the predicted boxes, alleviating the problem of positioning deviation caused by excessive multiscale differences among pedestrians in monitoring and ensuring accurate positioning of pedestrians [24].

2.3. Improvement of Pedestrian Target Tracking and Counting Model

2.3.1. Problems Existing in DeepSort

Multi-object tracking is a crucial component of the cross-line counting system. The DeepSort algorithm combines spatio-temporal motion predictions from Kalman filtering with appearance features extracted by a deep convolutional network to ensure stable tracking of pedestrians. The process involves several steps: First, the Kalman filter utilizes historical motion state information to predict pedestrian positions in the current frame. Simultaneously, an appearance-embedding network extracts appearance features from the detection bounding box. Finally, target matching and trajectory updates are accomplished by assessing the similarity between the weighted fusion of motion trajectory predictions and the appearance features.

In the motion prediction stage, the DeepSort tracker employs an 8-dimensional state space $(x,y,r,h,v_x,v_y,v_r,v_h)$ to characterize the motion state of the target. Specifically,$(x,y)$ denote the center position of the bounding box,$r$ represents the aspect ratio of the target detection bounding box, $h$ indicates the height of the target bounding box, and $(v_x,v_y,v_r,v_h)$ reflect the variation values of the aforementioned four parameters. Utilizing the Kalman filtering algorithm, the DeepSort predicts the target’s position in the subsequent frame based on this state vector. When the target is unobstructed, the DeepSort tracker can continuously detect, update, and correct errors by incorporating the latest positional information of pedestrians, thereby maintaining stable tracking of the target. However, when the target becomes occluded, the system must rely solely on Kalman filtering to estimate the position of pedestrians. In this scenario, the linear prediction capabilities of the Kalman filter reveal significant limitations. Pedestrians exhibit nonlinear motion, characterized by deceleration, acceleration, turning, and other dynamic behaviors. Despite this, the Kalman filter continues to perform linear extrapolation based on the speed and direction prior to occlusion, failing to account for changes in the actual motion state. Consequently, this oversight results in an increasing deviation between the predicted and actual positions. Furthermore, discrepancies in the predicted positions of pedestrians contribute to variations in their apparent scale, which exacerbates the challenges associated with target re-identification and may lead to issues such as trajectory mismatch and identity loss [25].

2.3.2. Correction of Pedestrian Position

In this paper, we adopt the center point of the bottom boundary of the pedestrian target bounding box as the designated position of the pedestrian. However, in vision-based pedestrian detection and tracking tasks, occlusion frequently leads to jitter in the boundaries of the detection bounding boxes, with the bottom boundary exhibiting the most significant variability. This phenomenon occurs because the bottom portion of the pedestrian is typically the first to be obscured, resulting in the absence of bottom features, which causes the bottom boundary of the target bounding box to ascend. Once the occlusion ceases, the bottom boundary rapidly retracts, leading to sharp fluctuations in pedestrian positioning. The erratic movement of pedestrian trajectories directly impacts cross-line detection; specifically, the abrupt shifts in the bottom boundary can cause the trajectory to instantaneously jump over the counting line. Additionally, the recurrent fluctuations at the bottom boundary may result in pedestrians being counted multiple times when oscillations occur near the counting line.

In contrast, the head area is typically the last to become occluded, resulting in more stable temporal variation of the top boundary [26]. Consequently, the smooth motion of the top boundary can serve as a reference to constrain and correct the sudden changes of the bottom boundary. The correction of the bottom boundary will reduce misjudgments in trajectory cross-line detection. Firstly, the vertical movements of the top and bottom boundaries of the pedestrian target bounding box are calculated in two consecutive frames. Secondly, the jitter deviation of the bottom boundary can be corrected by utilizing the stable motion changes of the top boundary. Assuming that ${∆y}_{top}$ and ${∆y}_{bottom}$ denote the vertical displacements of the top and bottom boundaries between frames $k-1$ and $k$ respectively, they are computed as follows:

$$\mathrm{\Delta }{y}_{top}=y_{top}^k-y_{top}^{k-1}$$

(1)

$$\mathrm{\Delta }{y}_{buttom}=y_{buttom}^k-y_{buttom}^{k-1}$$

(2)

where $y_{top}^k$, $y_{top}^{k-1}$, $y_{bottom}^k$, and $y_{bottom}^{k-1}$ denote the vertical positions of the pedestrian of the top and bottom boundaries of the target bounding box in frames $k$ and $k-1$, respectively. We set a tolerance threshold $\tau .$If the difference between ${∆\mathrm{y}}_{top}$ and ${∆\mathrm{y}}_{bottom}$ exceeds $\tau$, the change will be regarded as abnormal. Then, we use the top-boundary displacement ${∆\mathrm{y}}_{top}$ as a reference to correct it. The corrected bottom-boundary displacement $y_{bottom}^{k^{\prime }}$ is defined as follows:

$$\mathrm{\Delta }{y}_{bottom}^{k^{\prime }}=\left\{\begin{array}{c}{\mathrm{\Delta }y}_{\mathit{bottom}},\\ \mathrm{\Delta }{y}_{top},\end{array}\right.\begin{array}{c}\mathrm{\Delta }{y}_{bottom}\in [\mathrm{\Delta }{y}_{top}-\tau ,\mathrm{\Delta }{y}_{top}+\tau ]\\ \mathrm{\Delta }{y}_{bottom}\notin [\mathrm{\Delta }{y}_{top}-\tau ,\mathrm{\Delta }{y}_{top}+\tau ]\end{array}$$

(3)

When detecting pedestrians, the lack of pedestrian bottom details will cause the lower boundary of the detection frame to rise, and its height will not be lower than the real bottom boundary of the pedestrian. Consequently, it is essential to ensure that the predicted bottom-boundary value is not higher than the detected bottom boundary. However, if this substitution is carried out for a long time, the predicted lower boundary will gradually accumulate deviation and deviate from the true bottom position. To suppress this drift, we introduce a stepwise alignment mechanism where the predicted bottom boundary of the target detection box is gradually pulled to the bottom boundary of the target detection box detected by YOLO in a certain step $s$. The parameter represents the number of pixels that move from the predicted bottom boundary to the lower boundary of the detection box in each frame. Finally, the vertical position of the predicted bottom boundary is

$$y_{bottom}^{k^{\prime }}=y_{bottom}^{k-1}+\mathrm{\Delta }{y}_{bottom}^{k^{\prime }}+s$$

(4)

By smoothing the bottom boundary and incorporating a stepwise alignment mechanism to correct drift, the proposed method yields trajectories that are smoother and more closely aligned with the real motion path. Figure 6 illustrates a typical example. As shown in the figure, the smoothed trajectory is much more stable, indicating that the bottom boundary smoothing effectively reduces the errors caused by occlusion, ensuring a stable trajectory. The improvement of trajectory smoothness and stability can increase the accuracy and robustness of cross-line counting.

Figure 6. Comparison of original and smoothed pedestrian trajectories. Subplots (a–f) present representative frames for pedestrian ID 10, comparing the original bounding boxes (green) with the smoothed bounding boxes (yellow) across the occlusion period, including pre-occlusion, during-occlusion, and post-occlusion frames. Subplot (g) shows the corresponding trajectories, where the original trajectory is plotted in green and the smoothed trajectory in yellow.

2.3.3. Improvement of the Target Prediction Position

In educational environments such as classrooms and corridors, peak periods during class transitions result in increased pedestrian density and significant occlusions, which present considerable challenges for target detection and cross-line counting. Furthermore, specific social relationships, including friendships and class affiliations, create an attraction among individuals, leading them to maintain proximity throughout their movement. In high-pedestrian-density environments, pedestrian flow is further constrained by physical space, resulting in a phenomenon known as “flow alignment,” where movement direction remains consistent and speeds synchronize [27]. The combined effects of social relationships and flow alignment can lead to prolonged blockage of some pedestrians.

When the target is occluded and cannot be tracked temporarily, the traditional DeepSort tracker relies on the Kalman filter to estimate the target’s position. This reliance can lead to position drift, which is detrimental to counting accuracy [28]. To address this issue, this paper introduces using the surrounding crowd to assist in predicting the positional changes of the occluded target.

Assuming the frame where the target disappears is the current frame, we use the first frame and the $N$th frame before the current frame to calculate the related pedestrians whose movements are consistent with that of the occluded pedestrian. The motion trends of related pedestrians are then utilized to correct the predicted trajectory of the occluded targets. The approach significantly enhances the accuracy of predicting the positions of occluded targets, thereby facilitating more-reliable cross-line counting.

Firstly, select the pedestrian whose Manhattan distance from the occluded target in the previous frame of the current frame is less than half the height of the bounding box of the target. Let $∆x_i$ and $∆y_i$ denote the distances of the $i$th pedestrian from the lost target in the $x$ and $y$ directions, respectively. The proximity constraint condition is then expressed as

$$\mathrm{\Delta }{x}_i+\mathrm{\Delta }{y}_i<{\displaystyle \frac{h_{t-1}}{2}}$$

(5)

Secondly, we perform direction consistency filtering based on the pedestrian motion directions from the $N$th frame to the first frame. Specifically, the displacement vectors of the target and the surrounding pedestrians are calculated by the two frames, and the direction angles are recorded as $\mathrm{\angle }\mathrm{\alpha }$ and $\mathrm{\angle }{\beta }_j$, respectively. Individuals whose angle between the motion direction and the target direction is within 30 degrees are selected.

$$\left|\angle \alpha -\angle {\beta }_j\right|<{30}^{\circ }$$

(6)

Thirdly, for the candidate pedestrians that satisfy Formula (6), the relative difference in moving distance between them and the target pedestrian between the two frames is calculated. When the relative difference meets the following conditions, these candidate pedestrians are identified as the final relevant pedestrians.

$${\displaystyle \frac{\left|d-d_j\right|}{d}}<0.5$$

(7)

where $d$ denotes the total movement distance of the target pedestrian over the course of $N$ frames, while $d_j$ denotes the total movement distance of the $j$th candidate pedestrian during the same interval. If the absolute value of the difference between these two distances is less than 0.5 times the total movement of the pedestrian across the $N$ frames, the candidate pedestrian is deemed related to the target pedestrian and is classified as a related pedestrian.

Finally, the average position change of these related pedestrians is utilized to estimate the predicted position change $(∆x_p,∆y_p)$ of the occluded target as follows.

$$\left\{\begin{array}{c}∆x_p={\displaystyle \frac{1}{n}}\sum _{i=1}^n∆x_i\\ ∆y_p={\displaystyle \frac{1}{n}}\sum _{i=1}^n∆y_i\end{array}\right.$$

(8)

where n denotes the number of related pedestrians,$\mathrm{a}\mathrm{n}\mathrm{d} (∆x_i,∆y_i)$ denote the position change of the ith related pedestrian.

In Figure 7a, the pedestrian with ID 2 is obstructed at the classroom door, resulting in detection failure. By considering proximity, directional consistency, and motion similarity, the model identifies pedestrians with IDs 3 and 4 as related. It then uses their average displacement instead of the predicted displacement of the pedestrian with ID 2 during the occluded frames, effectively preserving trajectory continuity and improving cross-line counting accuracy.

Figure 7. Detection of occluded pedestrians based on pedestrian correlation: (a) shows the scene of occluded pedestrians in surveillance, with five pedestrians present, two of whom are occluded; (b) provides a schematic diagram of relevant pedestrian detection, illustrating how the model determines related pedestrians by utilizing the movement characteristics of surrounding pedestrians.

2.3.4. Counting with Auxiliary Zone

Cross-line counting is typically determined using the cross product to ascertain the movement direction of pedestrians across the counting line. Before deployment, it is essential to establish counting lines across various scenarios and to define the entry and exit directions. As illustrated in Figure 8, let the endpoints of the counting line be designated as C and D, forming a vector $\overrightarrow{CD}$. When a pedestrian crosses the counting line from left to right, the action is recorded as exiting; conversely, crossing from right to left is recorded as entering. Let the pedestrian’s position in the previous frame be denoted as point A and the current frame’s position as point B. The coordinates of these two points create a vector $\overrightarrow{AB}$. A crossing event is first detected by checking whether A and B lie on different sides of the counting line. The sign of the cross product of vectors $\overrightarrow{AB}$ and $\overrightarrow{CD}$ is then employed to ascertain whether the pedestrian is entering or exiting.

Figure 8. The diagram of pedestrian crossing line counting. Here, the red line CD is a counting line and the black line AB represents the line connecting the positions of a pedestrian in two consecutive frames.

In actual monitoring scenarios, corridor entrances and lobbies serve as critical nodes for pedestrian flow and are significant areas where individuals tend to congregate densely, leading to frequent occlusions [29]. For pedestrians exiting a room and entering the monitored area, their initial positions are typically near the counting line. However, they are often impeded by individuals ahead, resulting in cases where the initial tracking position has already crossed the counting line, thereby causing counting omissions due to delayed detection or late track initialization. The small image on the left in Figure 9 depicts the scene at the classroom door as captured in the video. It illustrates that the two pedestrians in front partially obstruct the view of the three individuals exiting the classroom from behind. The small image on the right displays the positions of pedestrians within the two-dimensional building floor plan. This phenomenon indicates that in dense-crowd or occlusion scenarios, cross-line counting that relies solely on the positional data of trackers may encounter issues with counting omissions. In contrast, the buffer counting strategy effectively addresses this limitation by compensating for late-initialization cases near the counting line.

Figure 9. Illustration of missed counts at a classroom doorway. (a) Surveillance scene where pedestrians exiting the classroom are partially occluded; due to delayed detection, a pedestrian may already be beyond the counting line when first detected. (b) Corresponding 2D floor-plan schematic.

To solve the problem, an auxiliary counting zone is established at one end of the counting line, as illustrated in Figure 10. The auxiliary counting zone is positioned at various exit locations, including corridor entrances, lobbies, and room doors, and extends outward along the normal direction of the counting line. The buffer width is adaptively set to one-third of the pedestrian’s bounding box height, accounting for perspective-induced scale variations. When a new ID first appears in the buffer, the system records the positional changes of the target over the subsequent 10 frames. If the movement direction aligns with the corresponding predefined crossing direction, it is classified as a valid cross-line event. The approach effectively identifies the appearance position and local movement direction of pedestrians across different scenarios, thereby reducing missed counts caused by delayed detection and improving counting robustness.

Figure 10. Auxiliary counting zone for cross-line counting: (a) Surveillance scene with the auxiliary counting zone placed near the counting line; (b) Schematic illustration of the auxiliary-zone-based counting principle, where short-term motion cues are used to validate crossing events for newly appearing targets.

3. Experiment and Analysis

3.1. Experimental Dataset

To assess the effectiveness and generalization of the proposed RL_YOLOv8, we produced a campus indoor pedestrian dataset by extracting frames from surveillance video captured in a teaching building. The dataset covers typical conditions, including daytime and nighttime scenes, peak periods during class transitions and breaks, and varying weather conditions such as sunny and rainy days, thereby reflecting real-world challenges such as large pedestrian density fluctuations and complex illumination. The dataset comprises a total of 7604 images, which is partitioned into training, validation and test sets in a ratio of 7:2:1.

To further test the generalization ability of the model outside of the campus scenario, we also conducted experiments on two public pedestrian benchmarks, the WiderPerson and CityPersons datasets. On WiderPerson, we use the labels of full-body, partial-body, and riders as pedestrian instances.

To assess the applicability and robustness of the proposed cross-line pedestrian counting model in real-world scenarios, an experiment was conducted in a typical teaching building environment. The experimental area included a classroom and its adjacent corridor, with dense pedestrian flow during peak commuting hours. We set up three detection lines Line 1, Line 2 and Line 3 at the entrances of the two classrooms and along the corridor, and designated buffer zones at the classroom entrances. The experiment was conducted ten minutes prior to the first afternoon class to evaluate the model’s missed detection rate and ability to avoid duplicate counting under complex motion conditions such as continuous passing, parallel walking, and multidirectional interweaving. The specific settings are illustrated in Figure 11.

Figure 11. Experimental scene with three pedestrian counting lines.

3.2. Environmental Configuration and Evaluation Indicators

All training and validation was performed on the training machine (test environment, TE1) listed in Table 1. To further evaluate deployability, we additionally conducted model testing on a test environment (TE2) with more limited computing resources, as also listed in Table 1.

Table 1. Environment configurations.

Configuration	Training & Test Environment (TE1)	Test Environment (TE2)
Purpose	Model Training & Test	Model Test
Operating System	Windows 11	Windows 10
Memory	128 GB	16 GB
CPU	Intel Core i9-13900K	Intel Core i5-12400F
GPU	NVIDIA GeForce RTX 3090	NVIDIA GeForce RTX 4060 Ti
GPU Memory	24 GB	8 GB
CUDA Toolkit	12.1	11.3
Python version	3.11	3.8
Deep Learning Framework	PyTorch 2.4.1 + cu121	PyTorch 1.13 + cu117

For a fair comparison, all models were trained using identical hyperparameters, including 300 epochs, a batch size of 32 and the same data augmentation. For object detection, we use Precision, Recall, FLOPs, number of parameters, FPS, mAP@50, and mAP@50:95 to evaluate the quality of the models. For cross-line counting, we use Precision, Recall, and the F1-score to evaluate counting accuracy.

3.3. Comparative Experiment on the Improvement of Object Detection Models

To assess the performance of the enhanced model, the RL_YOLO model was compared with three other YOLO models: YOLOv5n, YOLOv8n, and YOLO11n. The experimental results are presented in Table 2.

Table 2. Performance comparison between the improved model and existing models.

Models	Precision (%)	Recall (%)	mAP@50 (%)	mAP@50:95 (%)	FLOPs (G)	Param (M)	FPS
YOLOv5n	94.68	90.06	95.58	75.97	7.1	2.50	157.37
YOLOv8n	94.25	91.18	95.89	76.41	8.1	3.01	225.81
YOLO11n	94.75	90.32	95.58	76.35	6.3	2.58	196.61
RL_YOLOv8	94.99	91.25	96.40	76.50	5.5	1.67	233.80

The data in bold indicates that the data is the optimal value of this column.

According to Table 2, RL_YOLOv8 achieves the best detection accuracy, with mAP@50 reaching 96.40% and mAP@50:95 reaching 76.50%. At the same time, it significantly reduces computational cost. The FLOPs drops to 5.5 G and the number of parameters decreases to 1.67 M, both clearly lower than the baseline models. These results indicate that RL_YOLOv8 provides a better balance between accuracy and efficiency, making it suitable for real-time pedestrian detection and cross-line counting on resource-constrained indoor devices.

In addition, we test the models under different runtime environments. The detection accuracy remains largely stable across platforms, while inference speed varies due to hardware and software differences. The FPS comparison is reported in Table 3.

Table 3. Inference speed comparison under an alternative runtime environment.

Model	YOLOv5n	YOLOv8n	YOLO11n	RL_YOLOv8
FPS	60.27	67.15	54.69	72.15

To further compare the performance of RL_YOLOv8 with other models, we additionally conduct experiments on two public benchmark datasets, WiderPerson [30] and CityPersons [31], in the environment of TE1. Since neither of the two public datasets has an official verification set label, 20% of the training set is randomly selected as the verification set, the test set remains unchanged, and all model training parameters are consistent with those of the campus dataset. The test results are shown in Table 4 and Table 5.

Table 4. Test results on WiderPerson.

Models	Precision (%)	Recall (%)	mAP@50 (%)	mAP@50:95 (%)	FLOPs (G)	Param (M)
YOLOv5n	0.7698	0.6084	0.7120	0.4295	7.1	2.50
YOLOv8n	0.7723	0.6125	0.7153	0.4341	8.1	3.01
YOLO11n	0.7704	0.6040	0.7091	0.4315	6.3	2.58
RL_YOLOv8	0.7631	0.6156	0.7151	0.4328	5.5	1.67

The data in bold indicates that the data is the optimal value of this column.

Table 5. Test results on CityPersons.

Models	Precision (%)	Recall (%)	mAP@50 (%)	mAP@50:95 (%)	FLOPs (G)	Param (M)
YOLOv5n	0.7826	0.4970	0.5753	0.3494	7.1	2.50
YOLOv8n	0.7825	0.5023	0.5880	0.3591	8.1	3.01
YOLO11n	0.7871	0.5025	0.5823	0.3561	6.3	2.58
RL_YOLOv8	0.7588	0.5245	0.6021	0.3670	5.5	1.67

The data in bold indicates that the data is the optimal value of this column.

According to Table 2, Table 4 and Table 5, the detection accuracy of all models in the public datasets was significantly lower than that in the self-made data set from the campus, mainly because the public dataset is more crowded and sheltered by people, and there are some label errors in the public datasets. In addition, on the two public datasets, L_YOLOv8 has better detection accuracy than YOLOv5n and YOLO11n, and is close to the YOLOv8n model, but its FLOPs and parameters are significantly smaller than those of other models, which makes it easy to deploy and gives it better operation efficiency.

3.4. Ablation Experiment of Target Detection Model

To further assess the contribution of each module, a series of ablation experiments was performed utilizing the campus indoor pedestrian dataset. The symbol “√” denotes the application of the improvement module based on the baseline model YOLOv8n. The results of the experiment are presented in Table 6.

Table 6. Results of the ablation experiment.

Models		Precision (%)	Recall (%)	mAP@50 (%)	mAP@50:95 (%)	FLOPs (G)	Param (M)	FPS
RGCSPELAN	LSCD
-	--	94.25	91.18	95.66	75.97	8.1	3.01	225.81
√	-	94.91	90.68	95.66	76.35	7.1	2.31	234.84
-	√	94.46	90.55	95.83	76.76	6.5	2.36	227.36
√	√	94.99	91.25	96.40	76.50	5.5	1.67	233.80

The data in bold indicates that the data is the optimal value of this column.

The ablation study in Table 6 evaluates the contributions of RGCSPELAN and LSCD on the YOLOv8n baseline using the campus indoor pedestrian dataset. Replacing the C2f blocks with RGCSPELAN reduces model complexity, decreasing FLOPs from 8.1 G to 7.1 G, and parameters from 3.01 M to 2.31 M, while keeping detection accuracy at a comparable level. Introducing the LSCD head further improves detection performance, increasing mAP@50 from 95.66% to 95.83%, and reduces computational cost and model size to 6.5 G FLOPs and 2.36 M parameters.

When RGCSPELAN and LSCD are combined, RL_YOLOv8 achieves the best accuracy on this dataset, with mAP@50 reaching 96.40% and mAP@50:95 reaching 76.50%. At the same time, the model becomes substantially more lightweight: FLOPs decreases from 8.1 G to 5.5 G, and the number of parameters drops from 3.01 M to 1.67 M. Overall, these results show a better accuracy–efficiency trade-off, making RL_YOLOv8 more suitable for real-time indoor pedestrian detection and cross-line counting on resource-constrained devices.

3.5. Influence of Parameters on Experiments

All parameters in the algorithm will affect the final counting result. To find the optimal parameters, we conducted multiple sets of experiments for different parameter values. Compared to Line 2 and Line 3 scenarios, Line 1 exhibits more severe occlusion and is more challenging, so we chose Line 1 for the experiment.

3.5.1. Influences of Tolerance Threshold and Pulling Step

As described in Section 2.3.2, the tolerance threshold $\tau$ and the pulling step $s$ are the two key parameters for suppressing the lower boundary. Both affect the effectiveness of pedestrian position point correction and thus have a certain impact on cross-line counting. To test the effect of these two variables on cross-line counting, experiments were conducted with different parameter settings while fixing other variables. Compared to Lines 2 and 3, the occlusion near Line 1 is more severe and representative, so we chose Line 1 for experimental analysis. The experiment sets $s$ as 1, and sets different values for $\tau$. The results are shown in Table 7.

Table 7. Counting results with different tolerance thresholds.

Tolerance Value	True Count (In + Out)	Model Counting (In + Out)	TP (In + Out)	FP (In + Out)	FN (In + Out)	Precision (%)	Recall (%)	F1 (%)
3	55 + 8	50 + 11	48 + 7	2 + 4	7 + 1	90.16	87.30	88.71
5	55 + 8	55 + 10	52 + 7	3 + 3	3 + 1	90.77	93.65	92.19
8	55 + 8	56 + 9	52 + 7	4 + 2	3 + 1	90.77	93.65	92.19
10	55 + 8	56 + 9	52 + 7	4 + 2	3 + 1	90.77	93.65	92.19
15	55 + 8	55 + 9	51 + 7	4 + 2	4 + 1	90.63	92.06	91.34
20	55 + 8	55 + 9	51 + 7	4 + 2	4 + 1	90.63	92.06	91.34
30	55 + 8	54 + 9	50 + 7	4 + 2	5 + 1	90.48	90.48	90.48
40	55 + 8	54 + 9	50 + 7	4 + 2	5 + 1	90.48	90.48	90.48

Figure 12 shows the impact of different tolerance thresholds on counting accuracy. From Table 7 and Figure 12, when $\tau$ is too small, the suppression effect becomes too sensitive, resulting in excessive constraints on the update of the bottom edge and an increase in missed counts. When $\tau$ is set to 5, 8, or 10, the best balance can be achieved between suppressing occlusion jitter and maintaining motion tracking, resulting in the best counting accuracy. When $\tau$ exceeds 15, some jumps in the lower boundary of the target box will be recognized as normal and no jump correction will be made, resulting in a decrease in counting accuracy. Therefore, we set the default value of $\tau$ in the system to 8.

Figure 12. The impact of different tolerance thresholds on counting accuracy.

To test the influence of the pulling step on the experimental results, we fixed $\tau$ to 8, and set $s$ to 0.5, 1.0, 1.5, 2.0. The test results are shown in Table 8.

Table 8. The impact of s setting on counting accuracy.

Frame Window	True Count (In + Out)	Model Counting (In + Out)	TP (In + Out)	FP (In + Out)	FN (In + Out)	Precision (%)	Recall (%)	F1 (%)
0.5	55 + 8	55 + 9	51 + 7	4 + 2	4 + 1	90.63	92.06	91.34
1	55 + 8	56 + 9	52 + 7	4 + 2	3 + 1	90.77	93.65	92.19
1.5	55 + 8	54 + 9	50 + 7	4 + 2	5 + 1	90.48	90.48	90.48
2	55 + 8	55 + 10	51 + 7	4 + 3	4 + 1	89.23	92.06	90.63

The data in bold indicates that the data is the optimal value of this column.

According to Table 8, when $s$ is 1, the counting accuracy is the highest; when the value of $s$ is low (0.5), it will cause a significant deviation between the corrected lower boundary and the true boundary; When the value of $s$ is high (1.5, 2), it will cause the lower boundary to quickly rise to the detected lower boundary, and the improvement in correction effect is not significant, resulting in a decrease in counting accuracy.

3.5.2. Analysis of Buffer

When a pedestrian walks out of a classroom, he will not be detected at first for he is obstructed by doors, walls, or other pedestrians, shown as Figure 13a. When the pedestrian is detected and assigned an ID number of 6, his detection position (the red point) has already appeared outside the counting line (the red line), shown as Figure 13b. At this point, relying on a single counting line cannot detect the pedestrian’s crossing behavior, resulting in missed detections. To address this issue, a buffer zone (the yellow rectangle) is introduced. The width of the buffer zone is affected by occlusion and pedestrians in other directions. When the width is too narrow, the problem of missed detections will still exist; When the width is too wide, there may be situations where pedestrians outside the classroom suddenly appear in the buffer zone due to obstruction, resulting in false positives. The buffer zone mainly solves whether pedestrians are obstructed when crossing lines. When the obstruction is light, there is no need to set a buffer zone, such as in scene Line 3.

Figure 13. The buffer zone reduces the situation of omissions in recording. (a) At the 15th frame before the current frame, a pedestrian walking out of a classroom is not detected; (b) At the current frame, the pedestrian is detected and assigned an ID number of 6, and his position is located in the buffer; (c) At the 15th frame after the current frame, the pedestrian left the buffer zone.

3.5.3. Influences of the Length of Frame Window

To analyze the impact of frame window length $N$ on counting results, we conducted experiments using Line 1 as the test line. During the experiment, set τ to 8 and s to 1. The counting results under different values of $N$ are shown in Table 9.

Table 9. Experimental results with different $N$ values.

Frame Window	True Count (In + Out)	Model Counting (In + Out)	TP (In + Out)	FP (In + Out)	FN (In + Out)	Precision (%)	Recall (%)	F1(%)
5	55 + 8	50 + 9	48 + 7	2 + 2	7 + 1	93.22	87.30	90.16
10	55 + 8	54 + 9	51 + 7	3 + 2	4 + 1	92.06	92.06	92.06
15	55 + 8	56 + 9	52 + 7	4 + 2	3 + 1	90.77	93.65	92.19
20	55 + 8	55 + 9	50 + 7	5 + 2	5 + 1	89.06	90.48	89.76
25	55 + 8	57 + 9	52 + 7	5 + 2	3 + 1	89.39	93.65	91.47
30	55 + 8	58 + 9	50 + 7	8 + 2	5 + 1	85.07	90.48	87.69

The data in bold indicates that the data is the optimal value of this column.

According to Table 9, the highest accuracy is achieved when $N$ is set to 15; when the value of $N$ is too small or too large, it can lead to a decrease in pedestrian counting accuracy. The reason is that when the value of $N$ is too small, it is difficult to accurately determine the proximity relationship between the occluded person and other pedestrians around them before occlusion. When it is too large, pedestrians may exhibit sudden behaviors such as turning, slowing down, and pausing during walking, which can cause changes in the proximity relationship between pedestrians and those around them, resulting in inaccurate predictions.

3.6. Pedestrian Counting Experiment

The pedestrian detection and cross-line counting experiment was conducted in a corridor scene during the 10 min before class, when pedestrian density is high. In both the original method and the improved cross-line system, to avoid the counting interference caused by a pedestrian wandering near the counting line, we stipulate that the movement of frequent back and forth crossing lines in a short period of time is not included in the crossing count. That is, if a pedestrian crosses the counting line in one direction and then turns back to cross the line again in 15 frames, the two cross-line movements of the pedestrian back and forth are not counted. The detailed experimental results are shown in Table 10 and Figure 14.

Table 10. Experimental results of pedestrian cross-line counting using original and improved methods.

	Scene	True Count (In + Out)	Model Counting (In + Out)	TP (In + Out)	FP (In + Out)	FN (In + Out)	P (%)	R (%)	F1 Score
Original counting result	Line 1	55 + 8	59 + 8	52 + 6	7 + 2	3 + 2	86.57	92.06	89.23
	Line 2	12 + 4	12 + 1	11 + 1	1 + 0	1 + 3	92.31	75.00	82.76
	Line 3	104 + 28	111 + 32	103 + 28	8 + 4	1 + 0	91.61	99.24	95.27
Improved counting result	Line 1	55 + 8	56 + 9	52 + 7	4 + 2	3 + 1	90.77	93.65	92.19
	Line 2	12 + 4	11 + 4	10 + 4	1 + 0	2 + 0	93.33	87.50	90.32
	Line 3	104 + 28	107 + 28	104 + 28	3 + 0	0 + 0	97.78	100.00	98.88

Figure 14. Comparison of cross-line counting results between original and improved methods.

For Line 1, located at the classroom entrance, pedestrians are frequently occluded by others, causing noticeable jitter in the bottom boundary of the bounding box. Such instability may lead to trajectory “line jumping” and duplicate counting near the counting line. With the proposed trajectory refinement and bottom-boundary stabilization, false positives decrease from nine to six and false negatives decrease from five to four. Consequently, the F1 score improves from 89.23% to 92.19%, indicating more stable counting under frequent occlusions.

For Line 2, it is farther from the camera, where pedestrians appear smaller and are more prone to missed detections under occlusion. The auxiliary counting zone helps compensate for missed crossings caused by delayed detection or late track initialization near the doorway. As a result, the number of missed counts is reduced (FNs decrease from four to two), which increases recall from 75.00% to 87.50%. The F1 score increases from 82.76% to 90.32%, showing that the buffer-zone strategy improves robustness mainly by recovering missed events.

For Line 3, placed in the corridor close to the camera with a long counting line, pedestrians occupy larger pixel heights and are easier to detect. The improved system notably reduces false counts (FPs drop from 12 to 3) while maintaining a similar Recall level. This yields a clear improvement in precision from 91.61% to 97.78% and in F1 score from 95.27% to 98.88%, demonstrating that trajectory stabilization and the anti-repetition rule effectively suppress duplicate counts caused by trajectory fluctuations.

Overall, by integrating trajectory refinement, bottom-boundary stabilization and the auxiliary counting zone, the improved system reduces missed counts and suppresses false counts across different lines, leading to more stable and reliable cross-line counting in crowded and occluded indoor scenarios.

4. Conclusions

To solve the low cross-line counting accuracy in critical areas, such as corridors and classroom entrances, this paper proposes an optimized approach for pedestrian cross-line counting. The approach utilizes an enhanced model of YOLOv8n (that is RL_YOLOv8) combined with DeepSort tracking. The RL_YOLOv8 model serves as the foundation for the enhanced cross-line counting system, which reduces computational complexity while maintaining high detection accuracy. To improve the accuracy of cross-line counting, we introduce three key strategies: First, bottom boundary suppression reduces cross-line counting errors caused by trajectory jumps by suppressing drastic fluctuations in pedestrian position points. Second, a related-pedestrian prediction method corrects the predicted trajectory during periods when pedestrians are occluded, ensuring that occluded pedestrians can still successfully trigger the counting line. Finally, the buffer counting mechanism addresses missed detections caused by pedestrians already crossing the counting line when first detected, especially in scenarios such as classroom entrances.

Experimental results show that the improved system exhibits significant improvement in counting accuracy, effectively avoiding the original cross-line counting error caused by factors such as occlusion, and overall performance is better than the original system. Future work could further expand the adaptability of this method to other application scenarios, such as large shopping malls, subway stations, and airports, and explore more advanced pedestrian detection and tracking models to further improve the accuracy and performance of pedestrian counting systems. In addition, to explore the practicality of the system, we will carry out further work, such as adaptation to varying camera viewpoints and integration with multi-camera systems.