HoPE: Horizontal Plane Extractor for Cluttered 3D Scenes
Abstract
:1. Introduction
- It is hard to distinguish points of a plane from outliers belonging to the objects atop the plane or proximal planes of similar height. Fitting such points to a global model as done by RANSAC [9,10,11], Hough Transform (HT) [12,13] and Expectation-Maximization (EM) methods [14] commonly leads to producing sloped planes as depicted in Figure 1b, which is counter-factual regarding extracting horizontal planes and also complicates the computation of robotic tasks involving the surface’s pose, such as retrieving the objects upon the surface, determining where to step on during stair climbing or orienting the end-effector for picking or placing objects.
- The results may be under- or over-segmented as depicted in Figure 1c. These phenomena are common for bottom-up methods such as Region Growing (RG) [15,16], of which a set of thresholds fails to find a balance between separating and merging the patches simultaneously. In addition, they give no clear instruction for relating quite a few thresholds with the output expected, such that the user can only determine that experimentally through an exhaustive search.
- The detection can be computational expensive. Virtually, due to the existence of outliers and the difficulty of choosing thresholds, it is hard to reach the optimal without using time-consuming stabilizing methods.
- It is hard to preserve the identities (IDs) of the plane patches extracted among successive sequences as the robot moving around and changing its viewpoints. Robotic tasks usually refer a particular plane at a time, and it should not be confused with others. Nevertheless, such dynamic motion and the occlusion of objects within the scene damp the geometric characters of plane patches crucial for retaining the temporal consistency of the IDs. Conventional techniques such as SLAM [19] and Iterative Closest Point (ICP) [20,21] can address the problem, however, they are ponderous for matching plane patches, because we merely consider preserving the identities of planes instead of points.
- Simplify the procedure of horizontal plane extraction with a sensor orientation guided transformation of 3D point clouds, providing approaches for fast yet robust clustering, refinement and identification which take full advantage of the inner structure of transformed point clouds.
- Minimize the number of thresholds used in a reasonable way, enabling the user to have a full control of the results in terms of the accuracy and computing time expected.
- An open-source horizontal plane extractor compatible with Point Cloud Library (PCL) [24] and Robot Operating System (ROS). It is available at https://github.com/DrawZeroPoint/hope.
2. Proposed Methodology
2.1. Input Data
2.2. Point Cloud Preprocessing
2.3. Z Clustering
2.4. Refinement with PCA
2.5. Nearest Neighbor Plane Matching
3. Experimental Results and Evaluations
3.1. TUM RGB-D Dataset
3.2. Indoor LiDAR-RGBD Scan Dataset
3.3. Synthetic Scene
4. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Ecins, A.; Fermüller, C.; Aloimonos, Y. Cluttered scene segmentation using the symmetry constraint. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA 2016), Stockholm, Sweden, 16–21 May 2016; pp. 2271–2278. [Google Scholar]
- Cho, H.; Yeon, S.; Choi, H.; Doh, N. Detection and Compensation of Degeneracy Cases for IMU-Kinect Integrated Continuous SLAM with Plane Features. Sensors 2018, 18, 935. [Google Scholar] [CrossRef] [PubMed]
- Trevor, A.J.B.; Rogers, J.G.; Christensen, H.I. Planar surface SLAM with 3D and 2D sensors. In Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA 2012), St. Paul, MN, USA, 14–18 May 2012; pp. 3041–3048. [Google Scholar]
- Farid, R. Region-Growing Planar Segmentation for Robot Action Planning. In AI 2015: Advances in Artificial Intelligence; Pfahringer, B., Renz, J., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 179–191. [Google Scholar]
- Zhang, T.; Caron, S.; Nakamura, Y. Supervoxel Plane Segmentation and Multi-Contact Motion Generation for Humanoid Stair Climbing. Int. J. Hum. Robot. 2017, 14, 1650022. [Google Scholar] [CrossRef]
- Dos Santos, G.A.M.; Ferrão, V.T.; Vinhal, C.D.N.; da Cruz Junior, G. Fast algorithm for real-time ground extraction from unorganized stereo point clouds. Pattern Recogn. Lett. 2016, 84, 192–198. [Google Scholar] [CrossRef]
- Herghelegiu, P.; Burlacu, A.; Caraiman, S. Robust ground plane detection and tracking in stereo sequences using camera orientation. In Proceedings of the 2016 20th International Conference on System Theory, Control and Computing (ICSTCC 2016), Sinaia, Romania, 13–15 October 2016; pp. 514–519. [Google Scholar]
- Teng, Z.; Xiao, J. Surface-Based Detection and 6-DoF Pose Estimation of 3-D Objects in Cluttered Scenes. IEEE Trans. Robot. 2016, 32, 1347–1361. [Google Scholar] [CrossRef]
- Fischler, M.A.; Bolles, R.C. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun. ACM 1981, 24, 381–395. [Google Scholar] [CrossRef]
- Gallo, O.; Manduchi, R.; Rafii, A. CC-RANSAC: Fitting planes in the presence of multiple surfaces in range data. Pattern Recogn. Lett. 2011, 32, 403–410. [Google Scholar] [CrossRef] [Green Version]
- Qian, X.; Ye, C. NCC-RANSAC: A Fast Plane Extraction Method for 3-D Range Data Segmentation. IEEE Trans. Cybernet. 2014, 44, 2771–2783. [Google Scholar] [CrossRef] [PubMed]
- Vera, E.; Lucio, D.; Fernandes, L.A.; Velho, L. Hough Transform for real-time plane detection in depth images. Pattern Recogn. Lett. 2018, 103, 8–15. [Google Scholar] [CrossRef]
- Limberger, F.A.; Oliveira, M.M. Real-time detection of planar regions in unorganized point clouds. Pattern Recogn. 2015, 48, 2043–2053. [Google Scholar] [CrossRef] [Green Version]
- Thrun, S.; Martin, C.; Liu, Y.; Hahnel, D.; Emery-Montemerlo, R.; Chakrabarti, D.; Burgard, W. A real-time expectation-maximization algorithm for acquiring multiplanar maps of indoor environments with mobile robots. IEEE Trans. Robot. Autom. 2004, 20, 433–443. [Google Scholar] [CrossRef]
- Rabbani, T.; van den Heuvel, F.; Vosselman, G. Segmentation of point clouds using smoothness constraint. Int. Arch. Photogr. Remote Sens. Spat. Inf. Sci. 2006, 36, 248–253. [Google Scholar]
- Xiao, J.; Zhang, J.; Adler, B.; Zhang, H.; Zhang, J. Three-dimensional point cloud plane segmentation in both structured and unstructured environments. Robot. Auton. Syst. 2013, 61, 1641–1652. [Google Scholar] [CrossRef]
- Georgiev, K.; Creed, R.T.; Lakaemper, R. Fast plane extraction in 3D range data based on line segments. In Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2011), San Francisco, CA, USA, 25–30 September 2011; pp. 3808–3815. [Google Scholar]
- Pang, C.; Zhong, X.; Hu, H.; Tian, J.; Peng, X.; Zeng, J. Adaptive Obstacle Detection for Mobile Robots in Urban Environments Using Downward-Looking 2D LiDAR. Sensors 2018, 18, 1749. [Google Scholar] [CrossRef] [PubMed]
- Zhang, L.; Chen, D.; Liu, W. Point-plane SLAM based on line-based plane segmentation approach. In Proceedings of the 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO 2016), Qingdao, China, 3–7 December 2016; pp. 1287–1292. [Google Scholar]
- Dubé, R.; Gawel, A.; Sommer, H.; Nieto, J.; Siegwart, R.; Cadena, C. An online multi-robot SLAM system for 3D LiDARs. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017), Vancouver, BC, Canada, 24–28 Septeber 2017; pp. 1004–1011. [Google Scholar]
- Dubé, R.; Gollub, M.G.; Sommer, H.; Gilitschenski, I.; Siegwart, R.; Cadena, C.; Nieto, J. Incremental-Segment-Based Localization in 3-D Point Clouds. IEEE Robot. Autom. Lett. 2018, 3, 1832–1839. [Google Scholar] [CrossRef]
- Luan, S.; Chen, C.; Zhang, B.; Han, J.; Liu, J. Gabor convolutional networks. IEEE Trans. Image Process. 2018, 27, 4357–4366. [Google Scholar] [CrossRef] [PubMed]
- Zhang, B.; Gu, J.; Chen, C.; Han, J.; Su, X.; Cao, X.; Liu, J. One-two-one networks for compression artifacts reduction in remote sensing. ISPRS J. Photogramm. Remote Sens. 2018. [Google Scholar] [CrossRef]
- Rusu, R.B.; Cousins, S. 3D is here: Point Cloud Library (PCL). In Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA 2011), Shanghai, China, 9–13 May 2011; pp. 1–4. [Google Scholar]
- Foote, T. tf: The transform library. In Proceedings of the 2013 IEEE Conference on Technologies for Practical Robot Applications (TePRA 2013), Woburn, MA, USA, 22–23 April 2013; pp. 1–6. [Google Scholar]
- Rusu, R.B. Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments. Künstliche Intell. 2010, 24, 345–348. [Google Scholar] [CrossRef] [Green Version]
- Muja, M.; Lowe, D.G. Fast approximate nearest neighbors with automatic algorithm configuration. VISAPP 2009, 2, 2. [Google Scholar]
- Sturm, J.; Engelhard, N.; Endres, F.; Burgard, W.; Cremers, D. A benchmark for the evaluation of RGB-D SLAM systems. In Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2012), Vilamoura, Portugal, 7–12 October 2012; pp. 573–580. [Google Scholar]
- Everingham, M.; Van Gool, L.; Williams, C.K.; Winn, J.; Zisserman, A. The Pascal Visual Object Classes (VOC) Challenge. Int. J. Comput. Vis. 2010, 88, 303–338. [Google Scholar] [CrossRef]
- Park, J.; Zhou, Q.Y.; Koltun, V. Colored Point Cloud Registration Revisited. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV 2017), Venice, Italy, 22–29 October 2017; pp. 143–152. [Google Scholar]
Method | Parameter Except and |
---|---|
RANSAC | Max iteration: 500 |
Distance threshold: | |
Region Growing | Number of neighbors (K): 20 |
Smooth threshold: 8.0 | |
Curvature threshold: 1.0 | |
Ours | - |
Subset | Parameters Used for All Subsets | Accuracy (%) |
---|---|---|
freburg1_360 | 83.28 | |
freburg1_desk | 82.99 | |
freburg1_rpy | 79.66 | |
freburg1_xyz | 84.33 |
Scene | Point Number | RANSAC | RG | Ours |
---|---|---|---|---|
apartment | ||||
bedroom | ||||
boardroom | ||||
lobby | ||||
loft |
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Dong, Z.; Gao, Y.; Zhang, J.; Yan, Y.; Wang, X.; Chen, F. HoPE: Horizontal Plane Extractor for Cluttered 3D Scenes. Sensors 2018, 18, 3214. https://doi.org/10.3390/s18103214
Dong Z, Gao Y, Zhang J, Yan Y, Wang X, Chen F. HoPE: Horizontal Plane Extractor for Cluttered 3D Scenes. Sensors. 2018; 18(10):3214. https://doi.org/10.3390/s18103214
Chicago/Turabian StyleDong, Zhipeng, Yi Gao, Jinfeng Zhang, Yunhui Yan, Xin Wang, and Fei Chen. 2018. "HoPE: Horizontal Plane Extractor for Cluttered 3D Scenes" Sensors 18, no. 10: 3214. https://doi.org/10.3390/s18103214