Next Article in Journal
Pulse Radar with Field-Programmable Gate Array Range Compression for Real Time Displacement and Vibration Monitoring
Previous Article in Journal
Design of Enhanced Rotation Locked Loop for Roll Angle Estimation of Rotating Vehicle in a Weak GPS Signal Environment
Article

High-Fidelity Depth Upsampling Using the Self-Learning Framework

1
The Ground Autonomy Laboratory, Agency for Defense Development, Daejeon 34186, Korea
2
Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA 02139, USA
3
Department of Electrical Engineering, KAIST, Daejeon 34141, Korea
*
Author to whom correspondence should be addressed.
This paper is an extended version of our paper published in: Shim, I.; Shin, S.; Bok, Y.; Joo, K.; Choi, D.-G.; Lee, J.-Y.; Park, J.; Oh, J.-H.; Kweon, I.S. Vision System and Depth Processing for DRC-HUBO+, In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May 2016.
Sensors 2019, 19(1), 81; https://doi.org/10.3390/s19010081
Received: 2 October 2018 / Revised: 11 December 2018 / Accepted: 15 December 2018 / Published: 27 December 2018
(This article belongs to the Section Physical Sensors)
This paper presents a depth upsampling method that produces a high-fidelity dense depth map using a high-resolution RGB image and LiDAR sensor data. Our proposed method explicitly handles depth outliers and computes a depth upsampling with confidence information. Our key idea is the self-learning framework, which automatically learns to estimate the reliability of the upsampled depth map without human-labeled annotation. Thereby, our proposed method can produce a clear and high-fidelity dense depth map that preserves the shape of object structures well, which can be favored by subsequent algorithms for follow-up tasks. We qualitatively and quantitatively evaluate our proposed method by comparing other competing methods on the well-known Middlebury 2014 and KITTIbenchmark datasets. We demonstrate that our method generates accurate depth maps with smaller errors favorable against other methods while preserving a larger number of valid points, as we also show that our approach can be seamlessly applied to improve the quality of depth maps from other depth generation algorithms such as stereo matching and further discuss potential applications and limitations. Compared to previous work, our proposed method has similar depth errors on average, while retaining at least 3% more valid depth points. View Full-Text
Keywords: depth upsampling; depth filtering; LiDAR; self-learning; self-supervised learning depth upsampling; depth filtering; LiDAR; self-learning; self-supervised learning
Show Figures

Figure 1

MDPI and ACS Style

Shim, I.; Oh, T.-H.; Kweon, I.S. High-Fidelity Depth Upsampling Using the Self-Learning Framework. Sensors 2019, 19, 81. https://doi.org/10.3390/s19010081

AMA Style

Shim I, Oh T-H, Kweon IS. High-Fidelity Depth Upsampling Using the Self-Learning Framework. Sensors. 2019; 19(1):81. https://doi.org/10.3390/s19010081

Chicago/Turabian Style

Shim, Inwook, Tae-Hyun Oh, and In S. Kweon 2019. "High-Fidelity Depth Upsampling Using the Self-Learning Framework" Sensors 19, no. 1: 81. https://doi.org/10.3390/s19010081

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop