Search Results (51)

Nonlinear optimization is a central component of visual odometry and simultaneous localization and mapping (SLAM), but its repeated small- and medium-scale linear algebra operations are difficult to deploy efficiently on embedded hardware. This paper presents a synthesizable C++ library for AMD/Xilinx Vitis high-level synthesis (HLS) that provides field-programmable gate array (FPGA)-oriented dense linear algebra kernels and Lie-group primitives on SO(3) and SE(3). The library supports configurable scalar types, including IEEE floating point, posit arithmetic, and reduced-precision floating-point formats, enabling design-space exploration between numerical accuracy and hardware cost. The proposed kernels are integrated into the back-end of a monocular direct mesh-based visual SLAM system and evaluated on an AMD/Xilinx Kria KV260 platform. Compared with the software reference running on the embedded processor, the integrated FPGA implementation reduces the end-to-end optimization iteration time from 32.0 ms to 8.9 ms, corresponding to a speed-up of 3.6×, and reduces the dominant kernel latency from 25.0 ms to 4.9 ms. The most resource-efficient reduced-precision configuration reduces lookup table (LUT) usage by 29.6%, flip-flop (FF) usage by 25.7%, block random-access memory (BRAM) usage by 25.9%, and digital signal processor (DSP) usage by 38.6% relative to the floating-point hardware baseline, while keeping the relative trajectory error within 1.42%. The results show that Lie-group-aware optimization back-ends can be mapped to embedded FPGAs efficiently when fixed-size algebraic kernels, synthesis-aware memory structures, and configurable arithmetic are considered together. Full article

Visual–inertial odometry (VIO) often struggles with illumination variations, sparse visual features, and inertial drift in complex indoor settings, leading to scale uncertainties and accumulated errors. To address these issues, this paper proposes a new UWB–VIO initialization method combined with an enhanced Robust error-state Kalman filter (Robust ESKF) fusion technique for mobile robot localization. During initialization, common problems include scale drift and heading inconsistency. To solve these, a direction-consistent constrained initialization model is developed. By jointly optimizing the scale factor and yaw angle, this model ensures consistent alignment between the visual–inertial and ultra-wideband (UWB) coordinate frames. This approach removes the need for external calibration and independent coordinate transformation, which are typically required by traditional methods. In the fusion process, an improved residual-weighted robust filtering mechanism is employed to minimize the impact of abnormal UWB ranging data and noise interference. This mechanism adaptively suppresses outliers caused by UWB multipath reflections and non-line-of-sight (NLOS) propagation, thereby reducing VIO drift and improving the overall robustness and stability of the localization system. Experiments conducted in narrow-corridor environments, where both UWB and visual sensors are affected by interference, demonstrate that the proposed method significantly reduces trajectory drift and attitude jumps, resulting in better positioning accuracy and trajectory continuity. Compared to conventional UWB–VIO fusion algorithms, the proposed method enhances average localization accuracy by over 50% and maintains stable estimation even in severe multipath interference conditions, demonstrating high precision and strong robustness. Full article

High-precision autonomous localization remains a critical challenge for intelligent mining vehicles in GNSS-denied and unstructured coal mine roadways, where traditional odometry-based methods suffer from severe cumulative drift and perceptual aliasing. Inspired by the synergy between mammalian visual cues and cognitive neural mechanisms, this paper proposes a robust biomimetic localization framework that integrates multi-source perception with a prior cognitive map. The core contributions are three-fold: First, a semantic-enhanced biomimetic localization method is developed, leveraging roadway sign data as absolute spatial anchors to suppress long-distance cumulative errors. Second, an optimized head direction (HD) cell model is formulated by incorporating a speed balance factor, kinematic constraints, and a drift correction influence factor, significantly improving the precision of angular perception. Third, boundary-adaptive and sign-based semantic constraint terms are integrated into a continuous attractor network (CAN)-based path integration model, effectively preventing trajectory deviation into non-navigable regions. Comprehensive evaluations conducted in large-scale underground scenarios demonstrate that the proposed framework consistently outperforms conventional IMU-odometry fusion, representative 3D SLAM solutions, and baseline biomimetic algorithms. By effectively integrating semantic landmarks as spatial anchors, the system exhibits superior resilience against cumulative drift, maintaining high localization precision where standard methods typically diverge. The results confirm that our approach significantly enhances both trajectory consistency and heading stability across extensive distances, validating its robustness and scalability in handling the inherent complexities of unstructured coal mine environments for enhanced intrinsic safety. Full article

Indoor localization remains a challenging task due to the unavailability of reliable global navigation satellite system (GNSS) signals in most indoor environments. One way to overcome this challenge is through visual–inertial odometry (VIO), which enables real-time pose estimation by fusing camera and inertial measurements. However, VIO suffers from performance degradation under high-speed motion and in poorly lit environments. In such scenarios, motion blur, sensor noise, and low temporal resolution reduce the accuracy and robustness of the estimated trajectory. To address these limitations, we propose a motion-aware event-based VIO (MA-EVIO) system that adaptively fuses asynchronous event data, frame-based imagery, and inertial measurements for robust and accurate pose estimation. MA-EVIO employs a hybrid tracking strategy combining sparse feature matching and direct photometric alignment. A key innovation is its motion-aware keyframe selection, which dynamically adjusts tracking parameters based on real-time motion classification and feature quality. This motion awareness also enables adaptive sensor fusion: during fast motion, the system prioritizes event data, while under slow or stable motion, it relies more on RGB frames and feature-based tracking. Experimental results on the DAVIS240c and VECtor benchmarks demonstrate that MA-EVIO outperforms state-of-the-art methods, achieving a lower mean position error (MPE) of 0.19 on DAVIS240c compared to 0.21 (EVI-SAM) and 0.24 (PL-EVIO), and superior performance on VECtor with MPE/mean rotation error (MRE) of 1.19%/1.28 deg/m versus 1.27%/1.42 deg/m (EVI-SAM) and 1.93%/1.56 deg/m (PL-EVIO). These results validate the effectiveness of MA-EVIO in challenging dynamic indoor environments. Full article

In recent years, SLAM, visual odometry, and structure-from-motion approaches have widely addressed the problems of 3D reconstruction and ego-motion estimation. Of the many input modalities that can be used to solve these ill-posed problems, the pure visual alternative using a single monocular RGB camera has attracted the attention of multiple researchers due to its low cost and widespread availability in handheld devices. One of the best proposals currently available is the Direct Sparse Odometry (DSO) system, which has demonstrated the ability to accurately recover trajectories and depth maps using monocular sequences as the only source of information. Given the impressive advances in single-image depth estimation using neural networks, this work proposes an extension of the DSO system, named DeepDSO. DeepDSO effectively integrates the state-of-the-art NeW CRF neural network as a depth estimation module, providing depth prior information for each candidate point. This reduces the point search interval over the epipolar line. This integration improves the DSO algorithm’s depth point initialization and allows each proposed point to converge faster to its true depth. Experimentation carried out in the TUM-Mono dataset demonstrated that adding the neural network depth estimation module to the DSO pipeline significantly reduced rotation, translation, scale, start-segment alignment, end-segment alignment, and RMSE errors. Full article

The loss of localization in robots navigating GNSS-denied environments poses a critical challenge that can compromise mission success and safe operation. This article presents a method that fuses visual odometry outputs from both direct and feature-based (indirect) methods using Iterated Conditional Modes (ICMs), an efficient iterative optimization algorithm that maximizes the posterior probability in Markov random fields, combined with uncertainty-aware gain adjustment to perform pose estimation and mapping. The proposed method enhances the performance of visual localization and mapping algorithms in low-texture or visually degraded scenarios. The method was validated using the TUM RGB-D benchmark dataset and through real-world tests in both indoor and outdoor environments. Outdoor experiments were conducted on an electric vehicle, where the method maintained stable tracking. These initial results suggest that the technique could be transferable to electric vehicle platforms and applicable in a variety of real-world conditions. Full article

Visual–inertial odometry (VIO) is the primary supporting technology for autonomous systems, but it faces three major challenges: initialization sensitivity, dynamic illumination, and multi-sensor fusion. In order to overcome these challenges, this paper proposes stereo direct sparse visual–inertial odometry with efficient second-order minimization. It is entirely implemented using the direct method, which includes a depth initialization module based on visual–inertial alignment, a stereo image tracking module, and a marginalization module. Inertial measurement unit (IMU) data is first aligned with a stereo image to initialize the system effectively. Then, based on the efficient second-order minimization (ESM) algorithm, the photometric error and the inertial error are minimized to jointly optimize camera poses and sparse scene geometry. IMU information is accumulated between several frames using measurement preintegration and is inserted into the optimization as an additional constraint between keyframes. A marginalization module is added to reduce the computation complexity of the optimization and maintain the information about the previous states. The proposed system is evaluated on the KITTI visual odometry benchmark and the EuRoC dataset. The experimental results demonstrate that the proposed system achieves state-of-the-art performance in terms of accuracy and robustness. Full article

Visual simultaneous localization and mapping (Visual SLAM) based on RGB-D image data includes two main tasks: One is to build an environment map, and the other is to simultaneously track the position and movement of visual odometry estimation (VOE). Visual SLAM and VOE are used in many applications, such as robot systems, autonomous mobile robots, assistance systems for the blind, human–machine interaction, industry, etc. To solve the computer vision problems in Visual SLAM and VOE from RGB-D images, deep learning (DL) is an approach that gives very convincing results. This manuscript examines the results, advantages, difficulties, and challenges of the problem of Visual SLAM and VOE based on DL. In this paper, the taxonomy is proposed to conduct a complete survey based on three methods to construct Visual SLAM and VOE from RGB-D images (1) using DL for the modules of the Visual SLAM and VOE systems; (2) using DL to supplement the modules of Visual SLAM and VOE systems; and (3) using end-to-end DL to build Visual SLAM and VOE systems. The 220 scientific publications on Visual SLAM, VOE, and related issues were surveyed. The studies were surveyed based on the order of methods, datasets, evaluation measures, and detailed results. In particular, studies on using DL to build Visual SLAM and VOE systems have analyzed the challenges, advantages, and disadvantages. We also proposed and published the TQU-SLAM benchmark dataset, and a comparative study on fine-tuning the VOE model using a Multi-Layer Fusion network (MLF-VO) framework was performed. The comparison results of VOE on the TQU-SLAM benchmark dataset range from 16.97 m to 57.61 m. This is a huge error compared to the VOE methods on the KITTI, TUM RGB-D SLAM, and ICL-NUIM datasets. Therefore, the dataset we publish is very challenging, especially in the opposite direction (OP-D) when collecting and annotation data. The results of the comparative study are also presented in detail and available. Full article

Aiming to address the problems of fewer related studies on autonomous navigation algorithms based on multi-sensor fusion in complex scenarios in pastures, lower degrees of fusion, and insufficient cruising accuracy of the operation path in complex outdoor environments, a multimodal autonomous navigation system is proposed based on a loosely coupled architecture of Cartographer–RTAB-Map (real-time appearance-based mapping). Through laser-vision inertial guidance multi-sensor data fusion, the system achieves high-precision mapping and robust path planning in complex scenes. First, comparing the mainstream laser SLAM algorithms (Hector/Gmapping/Cartographer) through simulation experiments, Cartographer is found to have a significant memory efficiency advantage in large-scale scenarios and is thus chosen as the front-end odometer. Secondly, a two-way position optimization mechanism is innovatively designed: (1) When building the map, Cartographer processes the laser with IMU and odometer data to generate mileage estimations, which provide positioning compensation for RTAB-Map. (2) RTAB-Map fuses the depth camera point cloud and laser data, corrects the global position through visual closed-loop detection, and then uses 2D localization to construct a bimodal environment representation containing a 2D raster map and a 3D point cloud, achieving a complete description of the simulated ranch environment and material morphology and constructing a framework for the navigation algorithm of the pushing robot based on the two types of fused data. During navigation, the combination of RTAB-Map’s global localization and AMCL’s local localization is used to generate a smoother and robust positional attitude by fusing IMU and odometer data through the EKF algorithm. Global path planning is performed using Dijkstra’s algorithm and combined with the TEB (Timed Elastic Band) algorithm for local path planning. Finally, experimental validation is performed in a laboratory-simulated pasture environment. The results indicate that when the RTAB-Map algorithm fuses with the multi-source odometry, its performance is significantly improved in the laboratory-simulated ranch scenario, the maximum absolute value of the error of the map measurement size is narrowed from 24.908 cm to 4.456 cm, the maximum absolute value of the relative error is reduced from 6.227% to 2.025%, and the absolute value of the error at each location is significantly reduced. At the same time, the introduction of multi-source mileage fusion can effectively avoid the phenomenon of large-scale offset or drift in the process of map construction. On this basis, the robot constructs a fusion map containing a simulated pasture environment and material patterns. In the navigation accuracy test experiments, our proposed method reduces the root mean square error (RMSE) coefficient by 1.7% and Std by 2.7% compared with that of RTAB-MAP. The RMSE is reduced by 26.7% and Std by 22.8% compared to that of the AMCL algorithm. On this basis, the robot successfully traverses the six preset points, and the measured X and Y directions and the overall position errors of the six points meet the requirements of the pasture-pushing task. The robot successfully returns to the starting point after completing the task of multi-point navigation, achieving autonomous navigation of the robot. Full article

Visual navigation is a key capability for robots and animals. Inspired by the navigational prowess of social insects, a family of insect-inspired route navigation algorithms—familiarity-based algorithms—have been developed that use stored panoramic images collected during a training route to subsequently derive directional information during route recapitulation. However, unlike the ants that inspire them, these algorithms ignore the sequence in which the training images are acquired so that all temporal information/correlation is lost. In this paper, the benefits of incorporating sequence information in familiarity-based algorithms are tested. To do this, instead of comparing a test view to all the training route images, a window of memories is used to restrict the number of comparisons that need to be made. As ants are able to visually navigate when odometric information is removed, the window position is updated via visual matching information only and not odometry. The performance of an algorithm without sequence information is compared to the performance of window methods with different fixed lengths as well as a method that adapts the window size dynamically. All algorithms were benchmarked on a simulation of an environment used for ant navigation experiments and showed that sequence information can boost performance and reduce computation. A detailed analysis of successes and failures highlights the interaction between the length of the route memory sequence and environment type and shows the benefits of an adaptive method. Full article

High-precision indoor positioning is essential for various applications, such as the Internet of Things, robotics, and smart manufacturing, requiring accuracy better than 1 m. Conventional indoor positioning methods, like Wi-Fi or Bluetooth fingerprinting, typically provide low accuracy within a range of several meters, while techniques such as laser or visual odometry often require fusion with absolute positioning methods. Ultra-wideband (UWB) and Wi-Fi Round-Trip Time (RTT) are emerging radio positioning technologies supported by industry leaders like Apple and Google, respectively, both capable of achieving high-precision indoor positioning. This paper offers a comprehensive survey of UWB and Wi-Fi positioning, beginning with an overview of UWB and Wi-Fi RTT ranging, followed by an explanation of the fundamental principles of UWB and Wi-Fi RTT-based geometric positioning. Additionally, it compares the strengths and limitations of UWB and Wi-Fi RTT technologies and reviews advanced studies that address practical challenges in UWB and Wi-Fi RTT positioning, such as accuracy, reliability, continuity, and base station coordinate calibration issues. These challenges are primarily addressed through a multi-sensor fusion approach that integrates relative and absolute positioning. Finally, this paper highlights future directions for the development of UWB- and Wi-Fi RTT-based indoor positioning technologies. Full article

The increasing interest in developing robots capable of navigating autonomously has led to the necessity of developing robust methods that enable these robots to operate in challenging and dynamic environments. Visual odometry (VO) has emerged in this context as a key technique, offering the possibility of estimating the position of a robot using sequences of onboard cameras. In this paper, a VO algorithm is proposed that achieves sub-pixel precision by combining optical flow and direct methods. This approach uses only a downward-facing, monocular camera, eliminating the need for additional sensors. The experimental results demonstrate the robustness of the developed method across various surfaces, achieving minimal drift errors in calculation. Full article

Simultaneous Location and Mapping (SLAM) is a common algorithm for position estimation in GNSS-denied environments. However, the high structural consistency and low lighting conditions in tunnel environments pose challenges for traditional visual SLAM and LiDAR SLAM. To this end, this paper presents LiDAR and optical flow fusion odometry (LOFF), which uses a direction-separated data fusion method to fuse optical flow odometry into the degenerate direction of the LiDAR SLAM without sacrificing the accuracy. Moreover, LOFF incorporates detectors and a compensator, allowing for a smooth transition between general environments and degeneracy environments. This capability facilitates the stable flight of unmanned aerial vehicles (UAVs) in GNSS-denied tunnel environments, including corners and long-distance consistency. Through real-world experiments conducted in a GNSS-denied pedestrian tunnel, we demonstrate the superior position accuracy and trajectory smoothness of LOFF compared to state-of-the-art visual SLAM and LiDAR SLAM. Full article

Visual simultaneous localization and mapping (VSLAM) enhances the navigation of autonomous agents in unfamiliar environments by progressively constructing maps and estimating poses. However, conventional VSLAM pipelines often exhibited degraded performance in dynamic environments featuring mobile objects. Recent research in deep learning led to notable progress in semantic segmentation, which involves assigning semantic labels to image pixels. The integration of semantic segmentation into VSLAM can effectively differentiate between static and dynamic elements in intricate scenes. This paper provided a comprehensive comparative review on leveraging semantic segmentation to improve major components of VSLAM, including visual odometry, loop closure detection, and environmental mapping. Key principles and methods for both traditional VSLAM and deep semantic segmentation were introduced. This paper presented an overview and comparative analysis of the technical implementations of semantic integration across various modules of the VSLAM pipeline. Furthermore, it examined the features and potential use cases associated with the fusion of VSLAM and semantics. It was found that the existing VSLAM model continued to face challenges related to computational complexity. Promising future research directions were identified, including efficient model design, multimodal fusion, online adaptation, dynamic scene reconstruction, and end-to-end joint optimization. This review shed light on the emerging paradigm of semantic VSLAM and how deep learning-enabled semantic reasoning could unlock new capabilities for autonomous intelligent systems to operate reliably in the real world. Full article

In conventional point-line visual–inertial odometry systems in indoor environments, consideration of spatial position recovery and line feature classification can improve localization accuracy. In this paper, a monocular visual–inertial odometry based on structured and unstructured line features of vanishing points is proposed. First, the degeneracy phenomenon caused by a special geometric relationship between epipoles and line features is analyzed in the process of triangulation, and a degeneracy detection strategy is designed to determine the location of the epipoles. Then, considering that the vanishing point and the epipole coincide at infinity, the vanishing point feature is introduced to solve the degeneracy and direction vector optimization problem of line features. Finally, threshold constraints are used to categorize straight lines into structural and non-structural features under the Manhattan world assumption, and the vanishing point measurement model is added to the sliding window for joint optimization. Comparative tests on the EuRoC and TUM-VI public datasets validated the effectiveness of the proposed method. Full article