Search Results (137)

Search-and-rescue (SAR) missions following earthquakes and other disasters require aerial video perception systems that do more than detect objects in isolated frames. Operators must maintain the identities of access points, vehicles, responders, hazards, and other mission-relevant targets despite UAV ego-motion, dust, debris, occlusion, scale variation, and abrupt scene transitions. This paper presents E’CHIT (Edge-Oriented Colour Histogram Instance-Guided Tracking), a deployment-oriented, operator-centric UAV tracking framework for real-world disaster-response applications. Its primary scientific contribution is an identity-stabilised, detector-assisted tracking methodology. YOLOv8-seg proposals trained on D’RespNeT initialise and refresh tracks; a Custom-RE3 recurrent module propagates target states through short detector dropouts; and a lightweight EOMC verifier, based on edge orientation, mean colour, and shape consistency, determines whether tracks should be accepted, refreshed, or reacquired. A scene-cut watchdog that combines luminance mean absolute difference (MAD) with HSV histogram divergence prevents stale identities from carrying over after hard edits or sudden feed changes. Custom-RE3 is the continuation module implemented and evaluated in this study. The surrounding E’CHIT wrapper follows an initialise–reseed–verify–reset cycle and is tracker-adaptable at the software-interface level: another compatible SOT or MOT continuation module can be integrated through adapter modifications, state and bounding-box conversion, and method-specific retuning, followed by independent validation. All reported quantitative results therefore apply to the Custom-RE3 implementation. D’RespNeT, the optional reinforcement learning (RL) warm start, the HUD, and the deployment stack support this central tracking contribution. D’RespNeT provides 28 polygon-annotated SAR classes. An author-developed PPO/SAC script is used only during offline detector training. In the reported runs, it produces different early optimisation trajectories for selected difficult or under-represented classes, while the default supervised schedule remains the strongest final global mAP reference. No RL policy runs during deployment; the detector architecture, parameter count, and inference graph remain unchanged. Evaluation on D’RespNeT and authentic disaster-response UAV footage shows that E’CHIT increases Success@IoU ≥ 0.5 from 0.62 to 0.79, reduces identity switches by approximately 71%, and maintains real-time 1080p performance, achieving 164–330 FPS for single-target tracking and 24–100+ FPS for end-to-end multi-target operation on an RTX-class GPU using FP16. The VOT2014, NT-VOT211, and VOTS2024 figures reproduce historical result spaces reported in the literature and include a clearly labelled, non-official E’CHIT operating-point marker solely for context. This marker was not produced using the corresponding official datasets, toolkits, reset rules, or submission routes; it is excluded from the primary quantitative claims and must not be interpreted as a leaderboard rank or a protocol-identical comparison. Overall, the system demonstrates how identity-stable UAV tracks can provide actionable operator cues for target monitoring, entry-point assessment, and UAV–UGV/ground-team coordination in cluttered disaster scenes. Full article

Predicting vessel motion and environmental dynamics is essential for safe operation of autonomous maritime navigation systems. Transformer-based models have achieved strong results in AIS-trajectory forecasting and in anticipating future sonar observations, however, their use in maritime radar frame prediction has received little attention, despite radar being a key sensing modality in challenging weather and visibility conditions. In an effort to address this gap, this paper introduces a transformer architecture for predicting future maritime radar frames from sequences of past X-band observations and vessel ego-motion derived from GNSS, adapting the EchoPT paradigm originally developed for simulated in-air sonar imagery to the real-world MOANA dataset. We detail the model architecture and evaluate its prediction performance under both single-frame and autoregressive settings on held-out test data, and benchmark the model against persistence and rigid geometric warp references. A complementary failure mode analysis links the observed prediction errors to specific architectural and dataset choices, providing concrete directions for further research. Full article

End-to-end autonomous driving systems have demonstrated remarkable potential in navigating complex environments by directly mapping sensory inputs to control actions. However, many existing approaches rely on a single encoder to compress driving big data into a latent vector, which does not explicitly convert driving conditions into internal safety constraints. In addition, some approaches often depend on static, inflexible safety boundaries driven by mathematical equations, and they lack a clear mechanism to adapt control actions under varying conditions. This disconnection between conditions and control actions limits their ability to maintain safe operational controls in diverse kinds of scenarios. To address these limitations, we propose a Big Data-Driven Multi-Constraint Learning framework to generate safe driving action proxies derived from ego motion represented by steering and acceleration. Our architecture uses two encoders in cascade: a Temporal State Encoder that extracts a latent state representation from input driving states using temporal self-attention, and a Latent Constraint Generation Encoder that transforms this latent state into a compact internal constraint vector representing flexible and dynamic safety boundaries instead of static mathematical equations. This constraint vector is fused with the latent state representation to guide a Control Network for action generation, while a Condition Classifier and a Safety Discriminator enforce condition adaptability and safety. We evaluated the framework on the large-scale nuScenes big data repository, and experimental results demonstrated that our deep learning analysis successfully uncovers hidden safety patterns across massive driving logs, yielding stable convergence with low errors, notably a normalized Steering Mean Absolute Error (MAE) of 0.0189, a normalized Acceleration Mean Absolute Error of 0.0387, and strong safety discrimination with a Safety Accuracy of 0.9996, indicating that the learned internal safety constraints effectively modulate control actions under diverse driving states and uncertainties. Full article

Egocentric multi-view image analysis refers to the processing of utilizing synchronized video streams captured from multiple wearable cameras worn on the head or body, providing complementary first-person perspectives of dynamic, real-world interactions. Unlike single-view egocentric vision, which may suffer from severe occlusions, motion blur, and limited field-of-view or traditional fixed-camera multi-view setups (assuming static geometry and controlled environments), egocentric multi-view systems leverage body-worn rigs to enable a more robust and flexible 3D understanding in open-world, mobile scenarios. In this work, we present a systematic survey of advancements in cross-view feature fusion, geometric consistency enforcement, open-world detection, human–object interaction (HOI) modeling, action segmentation, 3D reconstruction, and novel-view synthesis specifically tailored to wearable multi-camera platforms. Key datasets released between 2024 and 2026—including HOT3D (833 min of synchronized multi-view hand/object interactions from Project Aria and Quest 3), MultiEgo (first multi-egocentric dataset for 4D social scene reconstruction), and Ego-1K (large-scale 12-camera rig for dynamic 3D video synthesis) are thoroughly examined alongside an analysis of integrations with large language models (LLMs) and vision–language models that drive performance gains, typically in the 15–30% range over single-view baselines in hand tracking, HOI recognition, and reconstruction fidelity, although we show through a consolidated meta-analysis that this gain is task-dependent: larger for geometry-bottlenecked tasks such as in-hand object lifting, and smaller, method-dependent, or occasionally negative for semantic-recognition tasks such as keystep recognition under naive view fusion. These methods cover work in multi-view stereo, cross-view learning, and novel-view synthesis while addressing several real-time wearable constraints. Practical applications such as immersive Augmented Reality/Virtual Reality (AR/VR), assistive robotics, and healthcare monitoring are also discussed together with the challenges in motion calibration, benchmark diversity, and edge deployment ability. Thus, in this review, we attempt to fill a critical gap by focusing exclusively on wearable multi-view systems in an open-world setting, synthesizing the latest literature to chart future directions toward more embodied and continual learning agents. Full article

Autonomous driving technology enables precise motion control and creates substantial opportunities for improving the operation of signalized intersections. However, the performance trade-offs among autonomous vehicles at signalized intersections, and their impacts on the operation process, will directly affect the further application of autonomous driving in the intelligent transportation system. Addressing this research focus, this paper proposes a multi-objective driving strategy for autonomous vehicles based on the scene characteristics of signalized intersections. Firstly, the intersection and its adjacent control area are treated as an integrated decision region, and an autonomous vehicle driving performance model at signalized intersections is established to evaluate economy, comfort, and efficiency performance. Secondly, a multi-stage trajectory generation method combining phase division, candidate trajectory generation, and real-time trajectory adjustment is further developed to adapt the ego vehicle to traffic conditions while maintaining safe and smooth motion. Thirdly, a multi-objective optimization problem is formulated to generate optimal trajectories for each autonomous vehicle within the region. Finally, weight sensitivity analysis, application adaptability analysis, and an analysis of system-level key factors and system-level impacts are conducted to explore the strategy optimization potential. The case studies reveal that the proposed strategy achieves improvements of 50.0%, 33.3%, and 21.6% in comfort, economy, and efficiency, respectively, compared with the common strategy. In future research, more specific and complex practical factors will be incorporated into the proposed strategy. The strategy helps to reveal the performance-oriented behavior of autonomous vehicles at signalized intersections and provides methodological support for the wider application of autonomous driving in intelligent transportation systems. Full article

Autonomous vehicle (AV) technologies have advanced rapidly in recent years, driving an increasing demand for large-scale, high-quality annotated data. However, collecting and annotating real-world pedestrian video datasets is time-consuming, costly, and often insufficient to cover rare and safety-critical scenarios. Recent world foundation models have demonstrated impressive capabilities in generating realistic videos, yet their suitability for safety-critical autonomous driving applications remains largely unexplored. In this work, we investigate whether current world foundation models can generate driving scenarios that are sufficiently realistic and behaviourally consistent for autonomous driving research. We conduct a case study centred on pedestrian–vehicle interactions captured from ego-vehicle dashcam viewpoints, where subtle behavioural and geometric errors can have significant safety implications. To support this investigation, we develop SynPeDAS, an open research framework comprising a collection of synthetic pedestrian-interaction videos, a reusable generation pipeline for transforming real-world driving footage into synthetic scenarios, an automated evaluation suite, and downstream demonstration code. Through quantitative evaluation and structured human assessment, we identify several recurring failure modes, including dynamic misalignment, depth drift, and object persistence inconsistencies. More importantly, we find that commonly used evaluation metrics frequently exhibit ceiling effects and weak alignment with human judgement, limiting their ability to detect safety-critical behavioural errors. These findings indicate that, despite high perceptual realism at the frame level, current generative world models and existing evaluation methodologies remain insufficient for capturing physically grounded motion and task-critical semantics. Consequently, significant challenges remain before world model-generated videos can be considered reliable for safety-critical autonomous driving applications. SynPeDAS provides an open platform for systematically studying these challenges and developing improved generation and evaluation methods. Full article

This paper presents a metrology-oriented review of vision-based displacement and deformation measurement for civil structural health monitoring (SHM), with an emphasis on field robustness and uncertainty quantification (UQ). The review focuses on image- and video-based methods that convert visual information into quantitative physical measurements, such as displacement, strain, or derived dynamic indicators. The literature is organized according to the main stages of the measurement chain: image formation, image-plane motion estimation, and geometric conversion to metric motion. Within this framework, measurement pipelines are interpreted through three levels of geometric mapping, namely, a scalar scale-factor model, a planar homography-based model, and a full Jacobian-based model. The review synthesizes major method families, including marker-based and markerless tracking, feature-based tracking, optical flow, digital image correlation (DIC), phase-based motion magnification, edge-based estimators, fixed- and moving-camera configurations, UAV-based acquisition with ego-motion compensation, hybrid vision–sensor fusion, and deep-learning-enhanced pipelines. A structured taxonomy of uncertainty sources is then presented along the processing chain, covering camera geometry and calibration, imaging noise and blur, quantization, timing and synchronization, environmental disturbances, optical turbulence and heat haze, platform motion, algorithmic failure modes, and reference-sensor uncertainty. The paper also compares UQ practices, including GUM-aligned analytical propagation, Monte Carlo methods, DIC-specific error budgets, bootstrap and resampling strategies, and probabilistic deep learning. The main contribution of this review is to connect computer-vision-based displacement pipelines with metrological requirements by explicitly linking measurement models, uncertainty sources, UQ methods, and field-validation evidence within a unified framework. A practical uncertainty-budget template is compiled to support traceable reporting across different pipelines and deployment scenarios. The paper concludes with prioritized research gaps and future directions, including standardized benchmarks and datasets, traceable UQ for moving-camera systems, multi-sensor fusion with end-to-end uncertainty propagation, long-term drift characterization, optical-turbulence and adverse-weather modeling, validated subpixel limits at extreme range, probabilistic deep learning–metrology integration, and standardized reporting practices. Full article

This study presents a denominator-conditioned functional-coverage framework for rear-end automated emergency braking (AEB). The framework estimates what share of a declared modeled rear-end hazard denominator is controllable at a given intervention lead time. Ego–lead episodes come from NGSIM US-101 and I-80; the default D0 denominator is a modeled no-action counterfactual from episode-entry states, not a replay of observed future motion or a crash-rate estimator. A longitudinal brake-only replay model estimates policy-specific required lead times for transparent reference policies: a brake-only upper envelope, TTC-family references, a stopping-distance reference, and an RSS-style baseline. Under the default pipeline, the upper envelope saturates near 99%, the weak TTC anchor covers roughly half of the denominator, and stronger references reach about 95–96% frequency-weighted saturation. Duration filters change the saturation by at most 4.00 percentage points, and increasing physical-uncertainty realizations from R = 3 to R = 30 changes the saturation by at most 0.33 percentage points. An extended TTC-family audit shows that a coverage-only candidate reaches 99.00% saturation but 30.70% nuisance intervention, whereas a burden-constrained candidate reaches 97.88% saturation with 9.23% nuisance. The reported values are conditional controllability measures, not certification evidence or deployment-level crash-rate estimates. Full article

Deep learning perception in autonomous vehicles (AVs) has created a critical attack surface in which adversarial patches and sensor-spoofing perturbations cascade from perception errors into unsafe driving decisions. Existing defenses face three limitations: most require retraining the perception network, making them impractical for already-deployed fleets; they operate almost exclusively at the perception layer, without verifying whether a compromised detection actually altered the driving action; and they leave temporal consistency across frames largely unexploited. This paper presents SENTINEL, a zero-modification, plug-and-play defense that wraps any deployed AV perception-and-planning stack without updating its weights, calibrating only the detection thresholds, score combination weights, and reference exemplars once on a small held-out calibration set. SENTINEL integrates a frozen foundation model verification ensemble (CLIP, DINOv2, SAM-2), a temporal consistency scorer that flags patches through anomalous frame-to-frame stability under ego-motion, a counterfactual policy verifier that replans under reconstructed perception and measures action-space divergence, and a risk-adaptive safety shield that modulates driving aggressiveness by verification confidence. Across CARLA, nuScenes, KITTI, and BDD100K, against five adversarial attacks and an adaptive adversary, SENTINEL reduces the attack success rate by up to 92%, keeps the clean accuracy loss to approximately 1.8 percentage points, reduces the collision rate under attack by approximately 87%, and adds under 45 ms latency on an RTX 4090 GPU. SENTINEL reframes adversarial robustness as a runtime property of the complete autonomous decision pipeline. Full article

Control-barrier-function-based safety filters are promising for autonomous driving, but most existing formulations treat obstacle perception as deterministic or account only for bounded ego state-estimation errors. This becomes limiting when obstacle existence, position, motion, and sensing quality vary online. We present a sensor-health- and belief-aware risk-adaptive high-order control barrier function (HOCBF) safety filter for dynamic obstacle avoidance. The method uses obstacle belief from a perception/tracking module, inflates residual obstacle uncertainty according to an object-wise sensor-health score, and converts upper-tail risk into adaptive HOCBF tightening through conditional value-at-risk (CVaR). Sensor health enters the controller through both covariance inflation and online CVaR confidence scheduling. The resulting quadratic program combines deterministic ego-error robustness with probabilistic perception uncertainty while minimally modifying the nominal control input. The zero-slack solution guarantees forward invariance of the risk-tightened safe set under the stated assumptions, whereas the slack-activated mode provides a quantified least-violation fallback rather than a strict safety guarantee. Simulations on a nonlinear 3-DOF bicycle model evaluate critical cut-in, sudden perception degradation, merge-bottleneck, fixed-CVaR, sensitivity, runtime-scaling, heterogeneous multi-obstacle, and heavy-tailed uncertainty cases. Full article

Rear-view human tracking and re-identification remain critical challenges for robotic visual sensing in unmanned vehicles, particularly under adverse weather conditions and severe occlusion. Conventional deep learning models often suffer from feature contamination and trajectory drift under dynamic illumination. To overcome these bottlenecks, we propose a lightweight tracking framework driven by spatiotemporal prediction and multimodal feature fusion. Specifically, an ego-motion-aware Kalman prediction mechanism maintains temporal continuity during complete occlusions. Upon target reappearance, a multi-factor descriptor—fusing color histograms with geometric constraints—is employed within a dynamic Mahalanobis search region. This is coupled with a specular-reflection-penalized adaptive learning rate (η_k) that actively freezes template updates during severe environmental degradation conditions. Evaluated on a custom Mecanum-wheeled robot, the proposed method achieves a peak precision of 94.2% and a tracking success rate of 93.4%. Extensive experiments in extreme rainy night scenarios demonstrate a 35% reduction in average tracking error, maintaining a Center Location Error (CLE) below 11 pixels. Furthermore, the system achieves a rapid target re-identification response of 72.83 ms during occlusion phases. Ultimately, this framework delivers a highly robust and real-time solution for autonomous navigation in complex dynamic environments. Full article

Safety-oriented UAV motion planning relies on distance-to-obstacle fields and their gradients, yet onboard mapping is typically limited to bounded local distance updates. Consequently, optimization may stall outside the updated band due to missing gradients, while enlarging the update range substantially increases computational cost. Our key insight is that motion-planning locality implies only a small subset of obstacles governs local trajectory refinement. We term this subset points of interest (POIs). Motivated by this observation, we develop a locality-aware sequential motion planning framework with a POI-driven feedback mechanism that continuously identifies and augments these trajectory-relevant obstacles during search and optimization. The mechanism tightly couples mapping, search, and optimization and enables safe trajectory refinement without requiring global distance updates. The framework adopts a heuristic mapping strategy that combines a long-term occupancy grid with bounded incremental distance updates and a POI-based short-term k-d tree, enabling efficient nearest-neighbor queries and gradient proxies beyond the update band. The search process generates a dynamically feasible initial trajectory in the long-term map while collecting POIs, which are then used to construct the short-term component. The trajectory is subsequently refined through iterative optimization loops, where newly exposed closest obstacles are incorporated into the POI set and the short-term map is updated until convergence. Safety is enforced through conservative collision checking against the inflated long-term occupancy map. Simulations in building and forest environments show that 99.7% of trials converge within two refinements in sparse scenes and none exceed four overall. Compared with FastPlanner and EgoPlanner, the proposed method achieves consistently larger obstacle clearances. Onboard experiments further validate its practicality under real sensing and computational constraints. Full article

To reduce the conservatism of local trajectory planning in dynamic road scenarios caused by redundant projection of predicted trajectories, this paper proposes a hierarchical hybrid trajectory-planning framework with a time-to-arrival (TTA)-driven dynamic risk-filtering mechanism. In the Frenet coordinate system, road boundaries, ego states, and static and dynamic obstacles are represented uniformly to construct an S–L fused risk field and an S–T spatiotemporal interaction graph, enabling the filtering of temporally irrelevant conflict regions based on TTA relationships. At the path-planning layer, risk-guided adaptive sampling is integrated with dynamic programming and quadratic programming to improve search efficiency and trajectory quality. At the speed-planning layer, spatiotemporal coordination is achieved through non-uniform discretization, safe-corridor extraction, and speed-profile optimization. Simulation results show that the proposed method generates safe, smooth, continuous, and executable local trajectories in scenarios involving static-obstacle avoidance, adjacent-vehicle cut-ins, non-motorized road-user crossings, and mixed multi-obstacle interactions, while reducing unnecessary deceleration and detours. Ablation results further indicate that adaptive sampling reduces the number of DP search nodes by approximately 50% and the average planning time by about 30%, while maintaining a nearly unchanged minimum safety distance. These findings demonstrate that the proposed framework effectively suppresses redundant conflict regions and improves planning efficiency, solution feasibility, and motion continuity without compromising safety. Full article

The use of Unmanned Aerial Vehicles (UAVs) has become quite popular in several applications during the last few years. Their spread is motivated by the flexibility of usage of UAVs and by their ability to automatically execute several tasks, mostly thanks to the availability of Global Navigation Satellite Systems (GNSSs), which usually allow reliable outdoor localization of aerial vehicles. However, the extension of task automatic execution indoors, and in other challenging working conditions for the GNSS, requires an alternative positioning system able to compensate for the unreliability or unavailability of GNSS in those cases. To this end, additional sensors are usually considered. Among them, cameras are probably the most popular ones. The most common case of a vision-based positioning system is a camera mounted on a moving platform used to determine its ego-motion in a dead-reckoning approach, i.e., visual odometry. Although this solution is affordable and does not require the installation of any infrastructure, it enables absolute positioning of the camera, i.e., of the UAV, only if certain landmarks, with known position, are visible in the flying area. In contrast, this work considers the use of external cameras installed in the flying area to track the UAV movements. This approach is similar to the one implemented in motion capture systems as well, where a set of static cameras is used to triangulate some target positions using calibrated cameras. Instead, this work investigates the use of vision and machine learning tools to (i) extract the UAV position from each video frame and (ii) estimate its 3D position. Estimation of the 3D UAV position is performed with a single camera, exploiting machine learning tools in order to avoid the need for camera calibration. Performance analysis is provided for a dataset collected at the Agripolis campus of the University of Padua. Full article

Autonomous vehicles (AVs) must make predictable and socially compliant behavioral decisions to ensure safe and efficient interactions with other road users. To address this challenge, this paper proposes a game-theoretic behavioral decision-making model integrated with spatial motion planning to capture the interactive intentions between the ego vehicle (EV) and target vehicle (TV) in pairwise scenarios. First, the study defines an intention representation method that characterizes intentions using spatial area boundaries, feasible speed ranges, and a set of goal points (speed goal points, position-orientation goal points). Second, a spatial motion planning approach is adopted to evaluate the intention, which optimizes the driving scheme using a multi-objective cost function (incorporating pursuit precision, comfort, energy efficiency, and travel efficiency). Finally, the game-theoretic decision-making model is constructed. The Social Value Orientation (SVO) is introduced to quantify drivers’ social preferences, and the payoff function, which integrates safety rewards (based on inter-vehicle distance) and performance rewards (based on motion planning indices), is established. Simulation results verify that the proposed model can effectively address the interactive intention decision-making problem between the AV and other road users and handle different scenarios. Full article