Search Results (68)

In this paper we propose a novel set of techniques for real-time Multi-Target Multi-Camera (MTMC) tracking of vehicles in congested, low speed environments, such as those of drive-thru scenarios, where metrics such as the number of vehicles, time of stay, and interactions between vehicles and staff are needed and must be highly accurate. Traditional methods of tracking based on Intersection over Union (IoU) and basic appearance features produce fragmented trajectories of misidentifications under these conditions. Furthermore, detectors, such as YOLO (You Only Look Once) architectures, exhibit different types of errors due to vehicle proximity, lane changes, and occlusions. Our methodology introduces a new tracker algorithm, Multi-Object Tracker based on Corner Displacement (MTCD), that improves the robustness against bounding box deformations by analysing corner displacement patterns and several other factors involved. The proposed solution was validated on real-world drive-thru footage, outperforming standard IoU-based trackers like Nvidia Discriminative Correlation Filter (NvDCF) tracker. By maintaining accurate cross-camera trajectories, our framework enables the extraction of critical operational metrics, including vehicle dwell times and person–vehicle interaction patterns, which are essential for optimizing service efficiency. This study tackles persistent tracking challenges in constrained environments, showcasing practical applications for real-world surveillance and logistics systems where precision is critical. The findings underscore the benefits of incorporating geometric resilience and delayed decision-making into MTMC architectures. Furthermore, our approach offers the advantage of seamless integration with existing camera infrastructure, eliminating the need for new deployments. Full article

In multi-vehicle traffic scenarios, achieving accurate environmental perception and motion trajectory tracking through LiDAR–camera fusion is critical for downstream vehicle planning and control tasks. To address the challenges of cross-modal feature interaction in LiDAR–image fusion and the low recognition efficiency/positioning accuracy of traffic participants in dense traffic flows, this study proposes an attention-based 3D object detection network integrating point cloud and image features. The algorithm adaptively fuses LiDAR geometric features and camera semantic features through channel-wise attention weighting, enhancing multi-modal feature representation by dynamically prioritizing informative channels. A center point detection architecture is further employed to regress 3D bounding boxes in bird’s-eye-view space, effectively resolving orientation ambiguities caused by sparse point distributions. Experimental validation on the nuScenes dataset demonstrates the model’s robustness in complex scenarios, achieving a mean Average Precision (mAP) of 64.5% and a 12.2% improvement over baseline methods. Real-vehicle deployment further confirms the fusion module’s effectiveness in enhancing detection stability under dynamic traffic conditions. Full article

In highway scenarios, factors such as high-speed vehicle movement, lighting conditions, and positional changes significantly affect the quality of trajectories in multi-object tracking. This, in turn, impacts the trajectory clustering process within the multi-target multi-camera tracking (MTMCT) system. To address this challenge, we present the weak-cue mixed similarity matrix and boundary expansion clustering (WCBE) MTMCT system. First, the weak-cue mixed similarity matrix (WCMSM) enhances the original trajectory features by incorporating weak cues. Then, considering the practical scene and incorporating richer information, the boundary expansion clustering (BEC) algorithm improves trajectory clustering performance by taking the distribution of trajectory observation points into account. Finally, to validate the effectiveness of our proposed method, we conduct experiments on both the Highway Surveillance Traffic (HST) dataset developed by our team and the public CityFlow dataset. The results demonstrate promising outcomes, validating the efficacy of our approach. Full article

Tunnels are commonly found in small and enclosed environments on highways, roads, or city streets. They are constructed to pass through mountains or beneath crowded urban areas. To prevent accidents in these confined environments, lane changes, slow driving, or speeding are prohibited on single- or multi-lane one-way roads. We developed a foreground detection algorithm based on the K-nearest neighbor (KNN) and Gaussian mixture model and 400 collected images. The KNN was used to gather the first 200 image data, which were processed to remove differences and estimate a high-quality background. Once the background was obtained, new images were extracted without the background image to extract the vehicle’s foreground. The background image was processed using Canny edge detection and the Hough transform to calculate road lines. At the same time, the oriented FAST and rotated BRIEF (ORB) algorithm was employed to track vehicles in the foreground image and determine positions and lane deviations. This method enables the calculation of traffic flow and abnormal movements. We accelerated image processing using xfOpenCV on the PYNQ-Z2 and FPGA Xilinx platforms. The developed algorithm does not require pre-labeled training models and can be used during the daytime to automatically collect the required footage. For real-time monitoring, the proposed algorithm increases the computation speed ten times compared with YOLO-v2-tiny. Additionally, it uses less than 1% of YOLO’s storage space. The proposed algorithm operates stably on the PYNQ-Z2 platform with existing surveillance cameras, without additional hardware setup. These advantages make the system more appropriate for smart traffic management than the existing framework. Full article

This review paper provides a comprehensive examination of millimeter-wave radar technologies in automotive systems, reviewing their advancements through signal processing innovations. The evolution of radar systems, from conventional platforms to mmWave technologies, has significantly enhanced capabilities such as high-resolution imaging, real-time tracking, and multi-object detection. Signal processing advancements, including constant false alarm rate detection, multiple-input–multiple-output systems, and machine learning-based techniques, are explored for their roles in improving radar performance under dynamic and challenging environments. The integration of mmWave radar with complementary sensing technologies such as LiDAR and cameras facilitates robust environmental perception essential for advanced driver-assistance systems and autonomous vehicles. This review also calls attention to key challenges, including environmental interference, material penetration, and sensor fusion, while addressing innovative solutions such as adaptive signal processing and sensor integration. Emerging applications of joint communication–radar systems further presents the potential of mmWave radar in autonomous driving and vehicle-to-everything communications. By synthesizing recent developments and identifying future directions, this review stresses the critical role of mmWave radar in advancing vehicular safety, efficiency, and autonomy. Full article

Multi-Object Multi-Camera Tracking (MOMCT) is essential for real-world applications but remains challenging in low-FPS environments where motion-based tracking is ineffective. This study proposes a novel vehicle tracking approach that integrates YOLOv9 for object detection, SwinReID for feature extraction, and a KNN-based matching algorithm to enhance tracking across multiple cameras. Our research leverages the AICUP 2024 Spring Competition dataset, which includes surveillance footage from Chiayi City, along with ImageNet for model training. To improve the accuracy, we incorporate scene segmentation to reduce false detections and optimize the buffer size per camera through an exhaustive search, adapting to varying perspectives and frame rates. Our method achieves a total score of 1.263626 based on IDF1 and MOTA metrics, demonstrating its effectiveness in low-FPS settings. These results contribute to the development of more robust MOMCT systems for traffic monitoring and surveillance. Full article

Multi-camera target tracking is a critical technology for continuous monitoring in large-scale environments, with applications in smart cities, security surveillance, and emergency response. However, existing tracking systems often suffer from high computational costs and energy inefficiencies, particularly in resource-constrained edge computing environments. Traditional methods typically rely on static or heuristic-based camera selection, leading to redundant computations and suboptimal resource allocation. This paper introduces a novel framework for efficient single-target tracking using edge-based distributed smart cameras with dynamic camera selection. The proposed framework employs context-aware dynamic camera selection, activating only the cameras most likely to detect the target based on its predicted trajectory. This approach is designed for resource-constrained environments and significantly reduces computational load and energy consumption while maintaining high tracking accuracy. The framework was evaluated through two experiments. In the first, single-person tracking was conducted across multiple routes with various target behaviors, demonstrating the framework’s effectiveness in optimizing resource utilization. In the second, the framework was applied to a simulated urban traffic light adjustment system for emergency vehicles, achieving significant reductions in computational load while maintaining equivalent tracking accuracy compared to an always-on camera system. These findings highlight the robustness, scalability, and energy efficiency of the framework in edge-based camera networks. Furthermore, the framework enables future advancements in dynamic resource management and scalable tracking technologies. Full article

Cooperative visual tracking involving unmanned aerial vehicles (UAVs) in low-altitude environments is a dynamic and rapidly evolving domain. Existing models encounter challenges with targets, such as scale variation, appearance similarity, and frequent occlusions, which hinder the effective use of target information for cross-view identity association. To address these challenges, this study introduces a model for multi-view, multi-target tracking in low-altitude scenes involving UAVs (MVTL-UAV), an effective multi-target tracking model specifically designed for low-altitude scenarios involving UAVs. The proposed method is built upon existing end-to-end detection and tracking frameworks, introducing three innovative modules: loss reinforcement, coupled constraints, and coefficient improvement. Collectively, these advancements enhance the accuracy of cross-view target identity matching. Our method is trained using the DIVOTrack dataset, which comprises data collected from a single UAV and two handheld cameras. Empirical results indicate that our approach achieves a 2.19% improvement in cross-view matching accuracy (CVMA) and a 1.95% improvement in the cross-view ID F1 metric (CVIDF1) when compared to current state-of-the-art methodologies. Importantly, the model’s performance is improved without compromising computational efficiency, thereby enhancing its practical value in resource-constrained environments. As a result, our model demonstrates satisfactory performance in various low-altitude target tracking scenarios involving UAVs, establishing a new benchmark in this research area. Full article

Recently, aerial manipulations are becoming more and more important for the practical applications of unmanned aerial vehicles (UAV) to choose, transport, and place objects in global space. In this paper, an aerial manipulation system consisting of a UAV, two onboard cameras, and a multi-fingered robotic hand with proximity sensors is developed. To achieve self-contained autonomous navigation to a targeted object, onboard tracking and depth cameras are used to detect the targeted object and to control the UAV to reach the target object, even in a Global Positioning System-denied environment. The robotic hand can perform proximity sensor-based grasping stably for an object that is within a position error tolerance (a circle with a radius of 50 mm) from the center of the hand. Therefore, to successfully grasp the object, a requirement for the position error of the hand (=UAV) during hovering after reaching the targeted object should be less than the tolerance. To meet this requirement, an object detection algorithm to support accurate target localization by combining information from both cameras was developed. In addition, camera mount orientation and UAV attitude sampling rate were determined by experiments, and it is confirmed that these implementations improved the UAV position error to within the grasping tolerance of the robot hand. Finally, the experiments on aerial manipulations using the developed system demonstrated the successful grasping of the targeted object. Full article

In this study, an improved stable multi-object simple online and real-time tracking (StableSORT) algorithm that was specifically designed for maritime environments was proposed to address challenges such as camera instability and irregular object motion. Specifically, StableSORT integrates a buffered IoU (B-IoU) and an observation-adaptive Kalman filter (OAKF) into the StrongSORT framework to improve tracking accuracy and robustness. A dataset was collected along the southern coast of Korea using a small autonomous surface vehicle to capture real-world maritime conditions. On this dataset, StableSORT achieved a 2.7% improvement in HOTA, 4.9% in AssA, and 2.6% in IDF1 compared to StrongSORT, and it significantly outperformed ByteTrack and OC-SORT by 84% and 69% in HOTA, respectively. These results underscore StableSORT’s ability to maintain identity consistency and enhance tracking performance under challenging maritime conditions. The ablation studies further validated the contributions of the B-IoU and OAKF modules in maintaining identity consistency and tracking accuracy under challenging maritime conditions. Full article

Accurate multi-object tracking (MOT) is essential for autonomous vehicles, enabling them to perceive and interact with dynamic environments effectively. Single-modality 3D MOT algorithms often face limitations due to sensor constraints, resulting in unreliable tracking. Recent multi-modal approaches have improved performance but rely heavily on complex, deep-learning-based fusion techniques. In this work, we present CLF-BEVSORT, a camera-LiDAR fusion model operating in the bird’s eye view (BEV) space using the SORT tracking framework. The proposed method introduces a novel association strategy that incorporates structural similarity into the cost function, enabling effective data fusion between 2D camera detections and 3D LiDAR detections for robust track recovery during short occlusions by leveraging LiDAR depth. Evaluated on the KITTI dataset, CLF-BEVSORT achieves state-of-the-art performance with a HOTA score of 77.26% for the Car class, surpassing StrongFusionMOT and DeepFusionMOT by 2.13%, with high precision (85.13%) and recall (80.45%). For the Pedestrian class, it achieves a HOTA score of 46.03%, outperforming Be-Track and StrongFusionMOT by (6.16%). Additionally, CLF-BEVSORT reduces identity switches (IDSW) by over 45% for cars compared to baselines AB3DMOT and BEVSORT, demonstrating robust, consistent tracking and setting a new benchmark for 3DMOT in autonomous driving. Full article

To enhance the accuracy and efficiency of the deflection response measurement of concrete bridges with a non-contact scheme and address the ill-conditioned nature of the inverse problem in influence line (IL) identification, this study introduces a computer-vision-aided deflection IL identification method that integrates edge detection and time-domain forward inference (TDFI). The methodology proposed in this research leverages computer vision technology with edge detection to surpass traditional contact-based measurement methods, greatly enhancing the operational efficiency and applicability of IL identification and, in particular, addressing the challenge of accurately measuring small deflections in concrete bridges. To mitigate the limitations of the Lucas–Kanade (LK) optical flow method, such as unclear feature points within the camera’s field of view and occasional point loss in certain video frames, an edge detection technique is employed to identify maximum values in the first-order derivatives of the image, creating virtual tracking points at the bridge edges through image processing. By precisely defining the bridge boundaries, only the essential structural attributes are preserved to enhance the reliability of minimal deflection deformations under vehicular loads. To tackle the ill-posed nature of the inverse problem, a TDFI model is introduced to identify IL, recursively capturing the static bridge response generated by the bridge under the influence of successive axles of a multi-axle vehicle. The IL is then computed by dividing the response by the weight of the preceding axle. Furthermore, an axle weight ratio reduction coefficient is proposed to mitigate noise amplification issues, ensuring that the weight of the preceding axle surpasses that of any other axle. To validate the accuracy and robustness of the proposed method, it is applied to numerical examples of a simply supported concrete beam, indoor experiments on a similar beam, and field tests on a three-span continuous concrete beam bridge. Full article

Mixed non-motorized traffic is largely unaffected by motor vehicle congestion, offering high accessibility and convenience, and thus serving as a primary mode of “last-mile” transportation in urban areas. To advance stochastic capacity estimation methods and provide reliable assessments of non-motorized roadway capacity, this study proposes a stochastic capacity estimation model based on power spectral analysis. The model treats discrete traffic flow data as a time-series signal and employs a stochastic signal parameter model to fit stochastic traffic flow patterns. Initially, UAVs and video cameras are used to capture videos of mixed non-motorized traffic flow. The video data were processed with an image detection algorithm based on the YOLO convolutional neural network and a video tracking algorithm using the DeepSORT multi-target tracking model, extracting data on traffic flow, density, speed, and rider characteristics. Then, the autocorrelation and partial autocorrelation functions of the signal are employed to distinguish among four classical stochastic signal parameter models. The model parameters are optimized by minimizing the AIC information criterion to identify the model with optimal fit. The fitted parametric models are analyzed by transforming them from the time domain to the frequency domain, and the power spectrum estimation model is then calculated. The experimental results show that the stochastic capacity model yields a pure EV capacity of 2060–3297 bikes/(h·m) and a pure bicycle capacity of 1538–2460 bikes/(h·m). The density–flow model calculates a pure EV capacity of 2349–2897 bikes/(h·m) and a pure bicycle capacity of 1753–2173 bikes/(h·m). The minimal difference between these estimates validates the effectiveness of the proposed model. These findings hold practical significance in addressing urban road congestion. Full article

Autonomous driving systems are a rapidly evolving technology. Trajectory prediction is a critical component of autonomous driving systems that enables safe navigation by anticipating the movement of surrounding objects. Lidar point-cloud data provide a 3D view of solid objects surrounding the ego-vehicle. Hence, trajectory prediction using Lidar point-cloud data performs better than 2D RGB cameras due to providing the distance between the target object and the ego-vehicle. However, processing point-cloud data is a costly and complicated process, and state-of-the-art 3D trajectory predictions using point-cloud data suffer from slow and erroneous predictions. State-of-the-art trajectory prediction approaches suffer from handcrafted and inefficient architectures, which can lead to low accuracy and suboptimal inference times. Neural architecture search (NAS) is a method proposed to optimize neural network models by using search algorithms to redesign architectures based on their performance and runtime. This paper introduces TrajectoryNAS, a novel neural architecture search (NAS) method designed to develop an efficient and more accurate LiDAR-based trajectory prediction model for predicting the trajectories of objects surrounding the ego vehicle. TrajectoryNAS systematically optimizes the architecture of an end-to-end trajectory prediction algorithm, incorporating all stacked components that are prerequisites for trajectory prediction, including object detection and object tracking, using metaheuristic algorithms. This approach addresses the neural architecture designs in each component of trajectory prediction, considering accuracy loss and the associated overhead latency. Our method introduces a novel multi-objective energy function that integrates accuracy and efficiency metrics, enabling the creation of a model that significantly outperforms existing approaches. Through empirical studies, TrajectoryNAS demonstrates its effectiveness in enhancing the performance of autonomous driving systems, marking a significant advancement in the field. Experimental results reveal that TrajcetoryNAS yields a minimum of 4.8 higger accuracy and 1.1* lower latency over competing methods on the NuScenes dataset. Full article

In intelligent traffic monitoring systems, the significant distance between cameras and their non-overlapping fields of view leads to several issues. These include incomplete tracking results from individual cameras, difficulty in matching targets across multiple cameras, and the complexity of inferring the global trajectory of a target. In response to the challenges above, a deep learning-based vehicle tracking algorithm called FairMOT-MCVT is proposed. This algorithm con-siders the vehicles’ characteristics as rigid targets from a roadside perspective. Firstly, a Block-Efficient module is designed to enhance the network’s ability to capture and characterize image features across different layers by integrating a multi-branch structure and depth-separable convolutions. Secondly, the Multi-scale Dilated Attention (MSDA) module is introduced to improve the feature extraction capability and computational efficiency by combining multi-scale feature fusion and attention mechanisms. Finally, a joint loss function is crafted to better distinguish between vehicles with similar appearances by combining the trajectory smoothing loss and velocity consistency loss, thereby considering both position and velocity continuity during the optimization process. The proposed method was evaluated on the public UA-DETRAC dataset, which comprises 1210 video sequences and over 140,000 frames captured under various weather and lighting conditions. The experimental results demonstrate that the FairMOT-MCVT algorithm significantly enhances multi-target tracking accuracy (MOTA) to 79.0, IDF1 to 84.5, and FPS to 29.03, surpassing the performance of previous algorithms. Additionally, this algorithm expands the detection range and reduces the deployment cost of roadside equipment, effectively meeting the practical application requirements. Full article