Search Results (73)

Control-barrier-function-based safety filters are promising for autonomous driving, but most existing formulations treat obstacle perception as deterministic or account only for bounded ego state-estimation errors. This becomes limiting when obstacle existence, position, motion, and sensing quality vary online. We present a sensor-health- and belief-aware risk-adaptive high-order control barrier function (HOCBF) safety filter for dynamic obstacle avoidance. The method uses obstacle belief from a perception/tracking module, inflates residual obstacle uncertainty according to an object-wise sensor-health score, and converts upper-tail risk into adaptive HOCBF tightening through conditional value-at-risk (CVaR). Sensor health enters the controller through both covariance inflation and online CVaR confidence scheduling. The resulting quadratic program combines deterministic ego-error robustness with probabilistic perception uncertainty while minimally modifying the nominal control input. The zero-slack solution guarantees forward invariance of the risk-tightened safe set under the stated assumptions, whereas the slack-activated mode provides a quantified least-violation fallback rather than a strict safety guarantee. Simulations on a nonlinear 3-DOF bicycle model evaluate critical cut-in, sudden perception degradation, merge-bottleneck, fixed-CVaR, sensitivity, runtime-scaling, heterogeneous multi-obstacle, and heavy-tailed uncertainty cases. Full article

The use of Unmanned Aerial Vehicles (UAVs) has become quite popular in several applications during the last few years. Their spread is motivated by the flexibility of usage of UAVs and by their ability to automatically execute several tasks, mostly thanks to the availability of Global Navigation Satellite Systems (GNSSs), which usually allow reliable outdoor localization of aerial vehicles. However, the extension of task automatic execution indoors, and in other challenging working conditions for the GNSS, requires an alternative positioning system able to compensate for the unreliability or unavailability of GNSS in those cases. To this end, additional sensors are usually considered. Among them, cameras are probably the most popular ones. The most common case of a vision-based positioning system is a camera mounted on a moving platform used to determine its ego-motion in a dead-reckoning approach, i.e., visual odometry. Although this solution is affordable and does not require the installation of any infrastructure, it enables absolute positioning of the camera, i.e., of the UAV, only if certain landmarks, with known position, are visible in the flying area. In contrast, this work considers the use of external cameras installed in the flying area to track the UAV movements. This approach is similar to the one implemented in motion capture systems as well, where a set of static cameras is used to triangulate some target positions using calibrated cameras. Instead, this work investigates the use of vision and machine learning tools to (i) extract the UAV position from each video frame and (ii) estimate its 3D position. Estimation of the 3D UAV position is performed with a single camera, exploiting machine learning tools in order to avoid the need for camera calibration. Performance analysis is provided for a dataset collected at the Agripolis campus of the University of Padua. Full article

Unmanned vehicles are becoming increasingly essential in executing high-risk missions in unknown environments such as search and rescue. As the complexity of operational environments escalates, carrying out unmanned tasks becomes cumbersome or even infeasible for a single vehicle, hampered by limited perception and operational constraints. Aiming at enhancing the flexibility of unmanned operations under complicated scenarios, this study introduces AGC-NeRF, an innovative air–ground collaborative exploration framework that harnesses the functional complementarity of UAVs and UGVs—enabling a UGV to navigate through a complex scenario with the assistance of a UAV via referencing a neural radiance map. First, a UAV is employed to collect aerial images for reconstructing the environment to be explored by a UGV, leveraging its aerial perspective to achieve wide-area coverage and global environmental perception that is unattainable for a single UGV. Concurrently, an innovative image saliency evaluation approach is introduced to meticulously select landmarks that are contributive to the UGV’s navigation system, yielding a pre-trained NeRF model of the operation scene. Then, a landmark-aware 6-DOF ego-motion estimator and collision-free trajectory optimizer are designed for the UGV based on the NeRF map. Finally, an online replanning architecture is established which relies on a ground station for NeRF training and state optimization by synergizing the trajectory planner and the state estimator, which forms a dual-agent vision-only navigation pipeline. Simulations and experiments validate that AGC-NeRF enables reliable UGV trajectory planning and state estimation in unknown environments, demonstrating superior efficacy and robustness of the air–ground collaborative paradigm. Full article

Event-based vision sensors offer microsecond temporal resolution and low power consumption, making them attractive for edge robotics and simultaneous localization and mapping (SLAM). Contrast maximization (CMAX) is a widely used direct geometric framework for rotational ego-motion estimation that aligns events by warping them and maximizing the spatial contrast of the resulting image of warped events (IWE). However, conventional CMAX is computationally inefficient because it repeatedly processes the full event set and a full-resolution IWE at every optimization iteration, including late-stage refinement, incurring both event-domain and image-domain costs. We propose coarse-to-fine contrast maximization (CCMAX), a computation-aware CMAX variant that aligns computational fidelity with the optimizer’s coarse-to-fine convergence behavior. CCMAX progressively increases IWE resolution across stages and applies coarse-grid event subsampling to remove spatially redundant events in early stages, while retaining a final full-resolution refinement. On standard event-camera benchmarks with IMU ground truth, CCMAX achieves accuracy comparable to a full-resolution baseline while reducing floating-point operations (FLOPs) by up to 42%. Energy measurements on a custom RISC-V–based edge SoC further show up to 87% lower energy consumption for the iterative CMAX pipeline. These results demonstrate an energy-efficient motion-estimation front-end suitable for real-time edge SLAM on resource- and power-constrained platforms. Full article

Coupled simultaneous localization and mapping (SLAM) and multi-object tracking have been studied in recent years. Although these tasks achieve promising results, they mainly associate keypoints and objects across frames separately, which limits their robustness in complex dynamic scenes. To overcome this limitation, we propose KOM-SLAM, a tightly coupled SLAM and multi-object tracking framework based on a Graph Neural Network (GNN), which jointly learns keypoint and object associations across frames while estimating ego-poses in a differentiable manner. The framework constructs a spatiotemporal graph over keypoints and object detections for association, and employs a multilayer perceptron (MLP) followed by a sigmoid activation that adaptively adjusts association thresholds based on ego-motion and spatial context. We apply a soft assignment on keypoints to ensure differentiable pose estimation, enabling the pose loss to supervise the association learning directly. Experiments on the KITTI Tracking demonstrate that our method achieves improved performance in both localization and object tracking. Full article

Thermal cameras are known for their ability to overcome lighting constraints and provide reliable thermal radiation images. This capability facilitates methods for depth and ego-motion estimation, enabling efficient learning of poses and scene structures under all-day conditions. However, the existing studies on depth prediction for thermal images are limited. In practical applications, thermal cameras capture sequential frames. Unfortunately, the potential of this multi-frame aspect is underutilized by the previous methods, resulting in limitations on the depth prediction accuracy of thermal videos. To leverage the multi-frame advantages of thermal videos and to improve the accuracy of monocular depth estimation from thermal images, we propose a framework for self-supervised depth and ego-motion learning from multi-frame thermal images. We construct a multi-view stereo (MVS) cost volume from temporally adjacent thermal frames. The construction process is adjusted based on the estimated pose, which serves as a motion hint. To stabilize the motion hint and improve pose estimation accuracy, we design a motion enhancement module that utilizes self-generated poses for additional supervisory signals. Additionally, we introduce RGB images in the training phase to form a multi-spectral loss, thereby augmenting the performance of the thermal model. The experimental results, conducted on a public dataset, demonstrate the proposed method’s accurate estimation of depth and ego-motion across varying light conditions, surpassing the performance of the self-supervised baseline. Full article

Visual odometry in dynamic environments is particularly challenging, as moving objects often cause incorrect data associations and large pose estimation errors. Traditional EKF-based VO methods rely on 1-point RANSAC to reject outliers under the assumption of a static world, thereby discarding dynamic landmarks as noise. However, in practice, outliers may arise not only from measurement errors but also from the motion of objects. To address this issue, we propose a modified 1-point RANSAC framework that detects dynamic objects and leverages both static and dynamic landmarks for ego-motion estimation. Inspired by adaptive strategies observed in biological vision systems, our approach integrates EKF-based state estimation with dynamic object tracking to achieve simultaneous ego-motion and object-motion estimation, improving robustness in complex and dynamic scenes. Full article

In recent years, SLAM, visual odometry, and structure-from-motion approaches have widely addressed the problems of 3D reconstruction and ego-motion estimation. Of the many input modalities that can be used to solve these ill-posed problems, the pure visual alternative using a single monocular RGB camera has attracted the attention of multiple researchers due to its low cost and widespread availability in handheld devices. One of the best proposals currently available is the Direct Sparse Odometry (DSO) system, which has demonstrated the ability to accurately recover trajectories and depth maps using monocular sequences as the only source of information. Given the impressive advances in single-image depth estimation using neural networks, this work proposes an extension of the DSO system, named DeepDSO. DeepDSO effectively integrates the state-of-the-art NeW CRF neural network as a depth estimation module, providing depth prior information for each candidate point. This reduces the point search interval over the epipolar line. This integration improves the DSO algorithm’s depth point initialization and allows each proposed point to converge faster to its true depth. Experimentation carried out in the TUM-Mono dataset demonstrated that adding the neural network depth estimation module to the DSO pipeline significantly reduced rotation, translation, scale, start-segment alignment, end-segment alignment, and RMSE errors. Full article

Monocular fisheye camera distance estimation is a crucial visual perception task for autonomous driving. Due to the practical challenges of acquiring precise depth annotations, existing self-supervised methods usually consist of a monocular distance model and an ego-motion predictor with the goal of minimizing a reconstruction matching loss. However, they suffer from inaccurate distance estimation in low-texture regions, especially road surfaces. In this paper, we introduce a weakly supervised learning strategy that incorporates semantic segmentation, instance segmentation, and optical flow as additional sources of supervision. In addition to the self-supervised reconstruction loss, we introduce a road surface flatness loss, an instance smoothness loss, and an optical flow loss to enhance the accuracy of distance estimation. We evaluate the proposed method on the WoodScape and SynWoodScape datasets, and it outperforms the self-supervised monocular baseline, FisheyeDistanceNet. Full article

Global navigation satellite systems (GNSSs) exhibit significant performance limitations in signal-deprived environments such as indoor spaces and underground spaces. Although visual SLAM has emerged as a viable solution for ego-motion estimation in GNSS-denied areas, conventional approaches remain constrained by static environment assumptions, resulting in a substantial degradation in accuracy when handling dynamic scenarios. The EMS-SLAM framework combines the geometric constraints and semantics of SLAM to provide a real-time solution for addressing the challenges of robustness and accuracy in dynamic environments. To improve the accuracy of the initial pose, EMS-SLAM employs a feature-matching algorithm based on a graph-cut RANSAC. In addition, a degeneracy-resistant geometric constraint method is proposed, which effectively addresses the degeneracy issues of purely epipolar approaches. Finally, EMS-SLAM combines semantic information with geometric constraints to maintain high accuracy while quickly eliminating dynamic feature points. Experiments were conducted on the public datasets and our collected datasets. The results demonstrate that our method outperformed the current algorithms of SLAM in highly dynamic environments. Full article

Estimating ego-motion in autonomous vehicles is critical for tasks such as localization, navigation, obstacle avoidance, and so on. While traditional methods often rely on direct pose estimation or AI-based approaches, these can be computationally intensive, especially for small, incremental movements typically observed between consecutive frames. In this work, we propose a brute-force-based ego-motion estimation algorithm that takes advantage of the constraints of autonomous vehicles, which are assumed to have only three degrees of freedom (x, y, and yaw). Our approach is based on a genetic algorithm to efficiently explore potential vehicle movements. By generating an initial seed of random motion candidates and iteratively mutating and selecting the best-performing individuals, we minimize the cost function that measures image similarity between frames. Furthermore, we implement the algorithm using CUDA to exploit parallel processing, significantly improving computational speed. Experimental results demonstrate that our approach achieves accurate ego-motion estimation with high efficiency, making it suitable for real-time autonomous vehicle applications. Full article

As the demand for autonomous driving (AD) systems has increased, the enhancement of their safety has become critically important. A fundamental capability of AD systems is object detection and trajectory forecasting of vehicles and pedestrians around the ego-vehicle, which is essential for preventing potential collisions. This study introduces the Deep learning-based Acceleration-aware Trajectory forecasting (DAT) model, a deep learning-based approach for object detection and trajectory forecasting, utilizing raw sensor measurements. DAT is an end-to-end model that processes sequential sensor data to detect objects and forecasts their future trajectories at each time step. The core innovation of DAT lies in its novel forecasting module, which leverages acceleration data to enhance trajectory forecasting, leading to the consideration of a variety of agent motion models. We propose a robust and innovative method for estimating ground-truth acceleration for objects, along with an object detector that predicts acceleration attributes for each detected object and a novel method for trajectory forecasting. DAT is trained and evaluated on the NuScenes dataset, demonstrating its empirical effectiveness through extensive experiments. The results indicate that DAT significantly surpasses state-of-the-art methods, particularly in enhancing forecasting accuracy for objects exhibiting both linear and nonlinear motion patterns, achieving up to a $2\times$ improvement. This advancement highlights the critical role of incorporating acceleration data into predictive models, representing a substantial step forward in the development of safer autonomous driving systems. Full article

Moving objects in the environment have a higher priority and more challenges in growing domains like unmanned vehicles and intelligent robotics. Estimating the motion state of objects based on point clouds in outdoor scenarios is currently a challenging area of research. This is due to factors such as limited temporal information, large volumes of data, extended network processing times, and the ego-motion. The number of points in a point cloud frame is typically 60,000–120,000 points, but most current motion state estimation methods for point clouds only downsample to a few thousand points for fast processing. The downsampling step will lead to the loss of scene information, which means these methods are far from being used in practical applications. Thus, this paper proposes a motion state estimation method that combines spatio-temporal constraints and deep learning. It starts by estimating and compensating the ego-motion of multi-frame point cloud data and mapping multi-frame data to a unified coordinate system; then the point cloud motion segmentation model on the multi-frame point cloud is proposed for motion object segmentation. Finally, spatio-temporal constraints are utilized to correlate the moving object at different moments and estimate the motion vectors. Experiments on KITTI, nuScenes, and real captured data show that the proposed method has good results, with an average vector deviation of only 0.036 m and 0.043 m in KITTI and nuScenes under a processing time of about 80 ms. The EPE3D error under the KITTI data is only 0.076 m, which proves the effectiveness of the method. Full article

Autonomous driving has long grappled with the need for precise absolute localization, making full autonomy elusive and raising the capital entry barriers for startups. This study delves into the feasibility of local trajectory planning for Level-2+ (L2+) semi-autonomous vehicles without the dependence on accurate absolute localization. Instead, emphasis is placed on estimating the pose change between consecutive planning timesteps from motion sensors and on integrating the relative locations of traffic objects into the local planning problem within the ego vehicle’s local coordinate system, thereby eliminating the need for absolute localization. Without the availability of absolute localization for correction, the measurement errors of speed and yaw rate greatly affect the estimation accuracy of the relative pose change between timesteps. This paper proved that the stability of the continuous planning problem under such motion sensor errors can be guaranteed at certain defined conditions. This was achieved by formulating it as a Lyapunov-stability analysis problem. Moreover, a simulation pipeline was developed to further validate the proposed local planning method, which features adjustable driving environment with multiple lanes and dynamic traffic objects to replicate real-world conditions. Simulations were conducted at two traffic scenes with different sensor error settings for speed and yaw rate measurements. The results substantiate the proposed framework’s functionality even under relatively inferior sensor errors distributions, i.e., speed error $v_{\mathrm{err}}$∼$\mathcal{N}(-0.1,0.1)$ m/s and yaw rate error ${\dot{\theta }}_{\mathrm{err}}$∼$\mathcal{N}(0.57,1.72)$ deg/s. Experiments were also conducted to evaluate the stability limits of the planned results under abnormally larger motion sensor errors. The results provide a good match to the previous theoretical analysis. Our findings suggested that precise absolute localization may not be the sole path to achieving reliable trajectory planning, eliminating the necessity for high-accuracy dual-antenna Global Positioning System (GPS) as well as the pre-built high-fidelity (HD) maps for map-based localization. Full article

Ultra-wideband (UWB) has gained increasing interest for providing real-time positioning to robots in GPS-denied environments. For a robot to act on this information, it also requires its heading. This is, however, not provided by UWB. To overcome this, either multiple tags are used to create a local reference frame connected to the robot or a single tag is combined with ego-motion estimation from odometry or Inertial Measurement Unit (IMU) measurements. Both odometry and the IMU suffer from drift, and it is common to use a magnetometer to correct the drift on the heading; however, magnetometers tend to become unreliable in typical GPS-denied environments. To overcome this, a lightweight particle filter was designed to run in real time. The particle filter corrects the ego-motion heading and location drift using the UWB measurements over a moving horizon time frame. The algorithm was evaluated offline using data sets collected from a ground robot that contains line-of-sight (LOS) and non-line-of-sight conditions. An RMSE of 13 cm and 0.12 (rad) was achieved with four anchors in the LOS condition. It is also shown that it can be used to provide the robot with real-time position and heading information for the robot to act on it in LOS conditions, and it is shown to be robust in both experimental conditions. Full article