Next Article in Journal
Parallel Structure from Motion for Sparse Point Cloud Generation in Large-Scale Scenes
Previous Article in Journal
Foot Gesture Recognition Using High-Compression Radar Signature Image and Deep Learning
Article

ReS2tAC—UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices

1
Fraunhofer Center for Machine Learning, Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB), 76131 Karlsruhe, Germany
2
Institute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany
3
Vision and Fusion Laboratory, Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany
*
Author to whom correspondence should be addressed.
Academic Editor: Jonathan Black
Sensors 2021, 21(11), 3938; https://doi.org/10.3390/s21113938
Received: 13 April 2021 / Revised: 20 May 2021 / Accepted: 28 May 2021 / Published: 7 June 2021
With the emergence of low-cost robotic systems, such as unmanned aerial vehicle, the importance of embedded high-performance image processing has increased. For a long time, FPGAs were the only processing hardware that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded systems. However, the recently increasing availability of embedded GPU-based systems, such as the NVIDIA Jetson series, comprised of an ARM CPU and a NVIDIA Tegra GPU, allows for massively parallel embedded computing on graphics hardware. With this in mind, we propose an approach for real-time embedded stereo processing on ARM and CUDA-enabled devices, which is based on the popular and widely used Semi-Global Matching algorithm. In this, we propose an optimization of the algorithm for embedded CUDA GPUs, by using massively parallel computing, as well as using the NEON intrinsics to optimize the algorithm for vectorized SIMD processing on embedded ARM CPUs. We have evaluated our approach with different configurations on two public stereo benchmark datasets to demonstrate that they can reach an error rate as low as 3.3%. Furthermore, our experiments show that the fastest configuration of our approach reaches up to 46 FPS on VGA image resolution. Finally, in a use-case specific qualitative evaluation, we have evaluated the power consumption of our approach and deployed it on the DJI Manifold 2-G attached to a DJI Matrix 210v2 RTK unmanned aerial vehicle (UAV), demonstrating its suitability for real-time stereo processing onboard a UAV. View Full-Text
Keywords: embedded stereo vision; real-time stereo processing; disparity estimation; semi-global matching; GPGPU; SIMD; UAV embedded stereo vision; real-time stereo processing; disparity estimation; semi-global matching; GPGPU; SIMD; UAV
Show Figures

Figure 1

MDPI and ACS Style

Ruf, B.; Mohrs, J.; Weinmann, M.; Hinz, S.; Beyerer, J. ReS2tAC—UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices. Sensors 2021, 21, 3938. https://doi.org/10.3390/s21113938

AMA Style

Ruf B, Mohrs J, Weinmann M, Hinz S, Beyerer J. ReS2tAC—UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices. Sensors. 2021; 21(11):3938. https://doi.org/10.3390/s21113938

Chicago/Turabian Style

Ruf, Boitumelo, Jonas Mohrs, Martin Weinmann, Stefan Hinz, and Jürgen Beyerer. 2021. "ReS2tAC—UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices" Sensors 21, no. 11: 3938. https://doi.org/10.3390/s21113938

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop