Texture Synthesis Repair of RealSense D435i Depth Images with Object-Oriented RGB Image Segmentation
1
Aerospace Information Research Institute, Chinese Academy of Sciences, No.9 Dengzhuang South Road, Haidian District, Beijing 100094, China
2
Aerospace Information Research Institute, University of Chinese Academy of Sciences, No.19(A) Yuquan Road, Shijingshan District, Beijing 100049, China
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(23), 6725; https://doi.org/10.3390/s20236725
Received: 30 September 2020
/
Revised: 13 November 2020
/
Accepted: 19 November 2020
/
Published: 24 November 2020
(This article belongs to the Special Issue Mobile Multi-Sensors in Positioning, Navigation, and Mapping Applications)
A depth camera is a kind of sensor that can directly collect distance information between an object and the camera. The RealSense D435i is a low-cost depth camera that is currently in widespread use. When collecting data, an RGB image and a depth image are acquired simultaneously. The quality of the RGB image is good, whereas the depth image typically has many holes. In a lot of applications using depth images, these holes can lead to serious problems. In this study, a repair method of depth images was proposed. The depth image is repaired using the texture synthesis algorithm with the RGB image, which is segmented through a multi-scale object-oriented method. The object difference parameter is added to the process of selecting the best sample block. In contrast with previous methods, the experimental results show that the proposed method avoids the error filling of holes, the edge of the filled holes is consistent with the edge of RGB images, and the repair accuracy is better. The root mean square error, peak signal-to-noise ratio, and structural similarity index measure from the repaired depth images and ground-truth image were better than those obtained by two other methods. We believe that the repair of the depth image can improve the effects of depth image applications.
View Full-Text
Keywords:
depth image; texture synthesis; object-oriented image segmentation; sample block; hole filling
▼
Show Figures
Figure 1
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
MDPI and ACS Style
Zhang, L.; Xia, H.; Qiao, Y. Texture Synthesis Repair of RealSense D435i Depth Images with Object-Oriented RGB Image Segmentation. Sensors 2020, 20, 6725. https://doi.org/10.3390/s20236725
AMA Style
Zhang L, Xia H, Qiao Y. Texture Synthesis Repair of RealSense D435i Depth Images with Object-Oriented RGB Image Segmentation. Sensors. 2020; 20(23):6725. https://doi.org/10.3390/s20236725
Chicago/Turabian StyleZhang, Longyu, Hao Xia, and Yanyou Qiao. 2020. "Texture Synthesis Repair of RealSense D435i Depth Images with Object-Oriented RGB Image Segmentation" Sensors 20, no. 23: 6725. https://doi.org/10.3390/s20236725
Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
