Reconstruction of Indoor Navigation Elements for Point Cloud of Buildings with Occlusions and Openings by Wall Segment Restoration from Indoor Context Labeling
Abstract
:1. Introduction
- A method of wall segment restoration using the outline and occupancy information from the horizontal projection of point cloud.
- A method of room segmentation that simulates the work of a laser scanner that not only divides separate rooms, but also distinguishes the connecting transition space between rooms.
- Identify the doors in the room and optimize the connections between the rooms by means of the planimetric projection statistical graph and the given rules.
2. Related Work
- For boundary extraction, the outlines of point cloud horizontal projection can be affected by the divergence data at the openings. The wall segments obtained from the wall projection are usually not able to provide the complete boundary outline information due to occlusions. Therefore, a single method usually does not provide good boundary due to the occlusions and local divergence data.
- For room segmentation, multiple building elements on multiple floors can be effectively segmented based on trajectory lines and viewpoints, but this method usually fails when faced with data without trajectory lines and viewpoints. The existing segmentation methods that do not rely on trajectory or viewpoint information suffer from under-segmentation or over-segmentation. The connecting transition space between rooms, as an important element of the indoor space, is rarely discussed in the literature.
- For door extraction, in cluttered indoor environments, the current method usually uses viewpoint information to project the occlusions in the room onto the wall to restore walls. In the absence of viewpoint information, walls that are heavily influenced by occlusions are usually unable to accurately extract door information. Secondly, there are some rooms with closed doors during data collection. Furthermore, it is still a challenge to detect closed doors in the point cloud without color information.
3. Methodology
3.1. Overview
- Building components extraction. Extracting the possible planar data from the point cloud as alternative building components using the random sample consensus (RANSAC) algorithm [36]. Then the alternative building components are classified and integrated according to their space characteristics to obtain the walls, floors and ceilings;
- Wall segment restoration. Project the point cloud of the wall surface onto the XOY plane to form the wall segment raster. Then the connectivity of disjoint wall segments is judged according to the relationship between building components. Finally, the energy minimization function is constructed to determine the best connectivity;
- Navigation element information extraction. Diving building components into navigation elements such as rooms, corridors and connecting transition space based on their geometric features and topological relationships. Then the openings on the wall surface that satisfy the threshold value are extracted as doors. Finally, the navigation elements are optimized according to the relationship between doors and connecting transition space;
- Navigation element reconstruction and navigation network generation. First, reconstruct the navigation element 3D model using the building components and the extracted navigation element information. Then, the indoor navigation network is generated and represented based on the connectivity and local geometric features between the navigation elements.
3.2. Building Components Extraction and Wall Segment Restoration
Algorithm 1. Wall segment topology analysis |
Input: Wall segment raster: Wall Guide line (The outline of a free-space evidence raster): G Wall segment search radius: rw Connection search radius: rc Ceiling: C |
Output: Topological relations of wall segment: Tboundary
|
3.3. Indoor Room Segmentation
3.4. Door Extraction
3.5. Navigation Network Generation and Model Reconstruction
4. Experiments and Results
4.1. Experimental Data and Evaluation Criteria
4.2. Parameter Settings
4.3. Experimental Results
5. Discussion
5.1. Model Accuracy Evaluation
5.2. Room Sgementation Evaluation
5.3. Component Evaluation
5.4. Limitations
6. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Otero, R.; Lagüela, S.; Garrido, I.; Arias, P. Mobile indoor mapping technologies: A review. Autom. Constr. 2020, 120, 103399. [Google Scholar] [CrossRef]
- Xiong, X.; Adan, A.; Akinci, B.; Huber, D. Automatic creation of semantically rich 3D building models from laser scanner data. Autom. Constr. 2013, 31, 325–337. [Google Scholar]
- Bi, S.; Yuan, C.; Liu, C.; Cheng, J.; Wang, W.; Cai, Y. A Survey of Low-Cost 3D Laser Scanning Technology. Appl. Sci. 2021, 11, 3938. [Google Scholar] [CrossRef]
- Liu, J.; Xu, D.; Hyyppa, J.; Liang, Y. A Survey of Applications With Combined BIM and 3D Laser Scanning in the Life Cycle of Buildings. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2021, 14, 5627–5637. [Google Scholar] [CrossRef]
- Wei, S.; Liu, M.; Zhao, J.; Huang, S. A Survey of Methods for Detecting Indoor Navigation Elements from Point Clouds. Geomat. Inf. Sci. Wuhan Univ. 2018, 43, 2003–2011. [Google Scholar]
- Giorgini, M.; Aleotti, J.; Monica, R. Floorplan generation of indoor environments from large-scale terrestrial laser scanner data. IEEE Geosci. Remote Sens. Lett. 2018, 16, 796–800. [Google Scholar]
- Ochmann, S.; Vock, R.; Wessel, R.; Klein, R. Automatic reconstruction of parametric building models from indoor point clouds. Comput. Graph. 2016, 54, 94–103. [Google Scholar]
- Mura, C.; Mattausch, O.; Villanueva, A.J.; Gobbetti, E.; Pajarola, R. Automatic room detection and reconstruction in cluttered indoor environments with complex room layouts. Comput. Graph. 2014, 44, 20–32. [Google Scholar]
- Liu, M.; Wei, S.; Huang, S.; Tang, N. Indoor Navigation Elements Extraction of Room Fineness Using Refining Space Separator Method. Geomat. Inf. Sci. Wuhan Univ. 2021, 46, 221–229. [Google Scholar]
- Yang, J.; Kang, Z.; Zeng, L.; Akwensi, P.H.; Sester, M. Semantics-guided reconstruction of indoor navigation elements from 3D colorized points. ISPRS J. Photogramm. Remote Sens. 2021, 173, 238–261. [Google Scholar]
- Lehtola, V.V.; Nikoohemat, S.; Nüchter, A. Indoor 3D: Overview on scanning and reconstruction methods. In Handbook of Big Geospatial Data; Springer: Berlin/Heidelberg, Germany, 2021; pp. 55–97. [Google Scholar]
- Shaobo, Z.; Zhichen, Y.; Yongsheng, Y.; Chao, S.; Quanyi, H. Study on Evacuation Modeling of Airport Based on Social Force Model. J. Syst. Simul. 2018, 30, 3648. [Google Scholar]
- Coughlan, J.M.; Yuille, A.L. Manhattan world: Compass direction from a single image by bayesian inference. In Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999; pp. 941–947. [Google Scholar]
- Schindler, G.; Dellaert, F. Atlanta world: An expectation maximization framework for simultaneous low-level edge grouping and camera calibration in complex man-made environments. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, Washington, DC, USA, 27 June–2 July 2004; p. I-I. [Google Scholar]
- CityGML. Available online: https://www.ogc.org/standards/citygml (accessed on 8 July 2022).
- Industry Foundation Classes (IFC). Available online: http://technical.buildingsmart.org/standards/ifc/ (accessed on 8 July 2022).
- Pintore, G.; Mura, C.; Ganovelli, F.; Fuentes-Perez, L.; Pajarola, R.; Gobbetti, E. State-of-the-art in Automatic 3D Reconstruction of Structured Indoor Environments. In Computer Graphics Forum; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2020; pp. 667–699. [Google Scholar]
- Ambruş, R.; Claici, S.; Wendt, A. Automatic room segmentation from unstructured 3-D data of indoor environments. IEEE Robot. Autom. Lett. 2017, 2, 749–756. [Google Scholar]
- Dehbi, Y.; Leonhardt, J.; Oehrlein, J.; Haunert, J.-H. Optimal scan planning with enforced network connectivity for the acquisition of three-dimensional indoor models. ISPRS J. Photogramm. Remote Sens. 2021, 180, 103–116. [Google Scholar]
- Yang, F.; Zhou, G.; Su, F.; Zuo, X.; Tang, L.; Liang, Y.; Zhu, H.; Li, L. Automatic indoor reconstruction from point clouds in multi-room environments with curved walls. Sensors 2019, 19, 3798. [Google Scholar]
- Nikoohemat, S.; Diakité, A.; Zlatanova, S.; Vosselman, G. Indoor 3D modeling and flexible space subdivision from point clouds. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2019, 4, 285–292. [Google Scholar]
- Bassier, M.; Vergauwen, M. Unsupervised reconstruction of Building Information Modeling wall objects from point cloud data. Autom. Constr. 2020, 120, 103338. [Google Scholar] [CrossRef]
- Lim, G.; Oh, Y.; Kim, D.; Jun, C.; Kang, J.; Doh, N. Modeling of architectural components for large-scale indoor spaces from point cloud measurements. IEEE Robot. Autom. Lett. 2020, 5, 3830–3837. [Google Scholar]
- Elseicy, A.; Nikoohemat, S.; Peter, M.; Elberink, S.O. Space subdivision of indoor mobile laser scanning data based on the scanner trajectory. Remote Sens. 2018, 10, 1815. [Google Scholar]
- Bormann, R.; Jordan, F.; Li, W.; Hampp, J.; Hägele, M. Room segmentation: Survey, implementation, and analysis. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May 2016; pp. 1019–1026. [Google Scholar]
- Li, L.; Su, F.; Yang, F.; Zhu, H.; Li, D.; Zuo, X.; Li, F.; Liu, Y.; Ying, S. Reconstruction of three-dimensional (3D) indoor interiors with multiple stories via comprehensive segmentation. Remote Sens. 2018, 10, 1281. [Google Scholar]
- Michailidis, G.-T.; Pajarola, R. Bayesian graph-cut optimization for wall surfaces reconstruction in indoor environments. Vis. Comput. 2017, 33, 1347–1355. [Google Scholar]
- Li, T.; Han, D.; Chen, Y.; Zhang, R.; Zhang, Y.; Hedgpeth, T. IndoorWaze: A Crowdsourcing-Based Context-Aware Indoor Navigation System. IEEE Trans. Wirel. Commun. 2020, 19, 5461–5472. [Google Scholar] [CrossRef]
- Zhou, M.; Lin, Y.; Zhao, N.; Jiang, Q.; Yang, X.; Tian, Z. Indoor WLAN Intelligent Target Intrusion Sensing Using Ray-Aided Generative Adversarial Network. IEEE Trans. Emerg. Top. Comput. Intell. 2020, 4, 61–73. [Google Scholar] [CrossRef]
- Litao, H.; Lijuan, Z.; Cheng, G.; Aiguo, Z. An indoor navigation network considering walking habits and its generation algorithm. Acta Geod. Cartogr. Sin. 2022, 51, 729. [Google Scholar]
- Vasquez-Espinoza, L.; Castillo-Cara, M.; Orozco-Barbosa, L. On the relevance of the metadata used in the semantic segmentation of indoor image spaces. Expert Syst. Appl. 2021, 184, 115486. [Google Scholar]
- Pham, T.T.; Reid, I.; Latif, Y.; Gould, S. Hierarchical Higher-Order Regression Forest Fields: An Application to 3D Indoor Scene Labelling. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Washington DC, USA, 11–18 December 2015; pp. 2246–2254. [Google Scholar]
- Cao, J.; Leng, H.; Lischinski, D.; Cohen-Or, D.; Tu, C.; Li, Y. Shapeconv: Shape-aware convolutional layer for indoor RGB-D semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October 2021; pp. 7088–7097. [Google Scholar]
- Menini, D.; Kumar, S.; Oswald, M.R.; Sandström, E.; Sminchisescu, C.; Van Gool, L. A real-time online learning framework for joint 3d reconstruction and semantic segmentation of indoor scenes. IEEE Robot. Autom. Lett. 2021, 7, 1332–1339. [Google Scholar]
- Diakité, A.A.; Zlatanova, S.; Alattas, A.F.M.; Li, K.J. Towards Indoorgml 2.0: Updates and Case Study Illustrations. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2020, 43, 334–337. [Google Scholar] [CrossRef]
- Schnabel, R.; Wahl, R.; Klein, R. Efficient RANSAC for point-cloud shape detection. In Computer Graphics Forum; Blackwell Publishing Ltd.: Oxford, UK, 2007; pp. 214–226. [Google Scholar]
- Cui, Y.; Li, Q.; Dong, Z. Structural 3D reconstruction of indoor space for 5G signal simulation with mobile laser scanning point clouds. Remote Sens. 2019, 11, 2262. [Google Scholar] [CrossRef]
- Yang, F.; Li, L.; Su, F.; Li, D.; Zhu, H.; Ying, S.; Zuo, X.; Tang, L. Semantic decomposition and recognition of indoor spaces with structural constraints for 3D indoor modelling. Autom. Constr. 2019, 106, 102913. [Google Scholar] [CrossRef]
- Wang, R.; Xie, L.; Chen, D. Modeling indoor spaces using decomposition and reconstruction of structural elements. Photogramm. Eng. Remote Sens. 2017, 83, 827–841. [Google Scholar] [CrossRef]
- Nikoohemat, S.; Diakité, A.A.; Zlatanova, S.; Vosselman, G. Indoor 3D reconstruction from point clouds for optimal routing in complex buildings to support disaster management. Autom. Constr. 2020, 113, 103109. [Google Scholar]
- Previtali, M.; Díaz-Vilariño, L.; Scaioni, M. Indoor Building Reconstruction from Occluded Point Clouds Using Graph-Cut and Ray-Tracing. Appl. Sci. 2018, 8, 1529. [Google Scholar] [CrossRef]
- Boykov, Y.; Veksler, O.; Zabih, R. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 2001, 23, 1222–1239. [Google Scholar] [CrossRef]
- Shi, W.; Ahmed, W.; Li, N.; Fan, W.; Xiang, H.; Wang, M. Semantic Geometric Modelling of Unstructured Indoor Point Cloud. ISPRS Int. J. Geo-Inf. 2018, 8, 9. [Google Scholar] [CrossRef]
- Pomerleau, F.; Liu, M.; Colas, F.; Siegwart, R. Challenging data sets for point cloud registration algorithms. Int. J. Robot. Res. 2012, 31, 1705–1711. [Google Scholar] [CrossRef]
- Khoshelham, K.; Tran, H.; Acharya, D.; Díaz Vilariño, L.; Kang, Z.; Dalyot, S. The Isprs Benchmark on Indoor Modelling–Preliminary Results. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2020, 42, 207–211. [Google Scholar] [CrossRef]
- ASL Datasets Repository. Available online: https://projects.asl.ethz.ch/datasets/ (accessed on 8 July 2022).
- ISPRS Benchmark on Indoor Modelling. Available online: https://www2.isprs.org/commissions/comm4/wg5/dataset/ (accessed on 8 July 2022).
- University of Zurich Dataset. Available online: https://www.ifi.uzh.ch/en/vmml/research/datasets.html (accessed on 8 July 2022).
Parameter | Description | Units | |
---|---|---|---|
Building component extraction | Angle between alternative building components and alternative wall plane | Degree | |
Angle between alternative wall surfaces | Degree | ||
Distance between alternative wall surfaces | Meter | ||
Height difference between the top of alternative wall surfaces | Meter | ||
Wall segment restoration | Radius of mutual search of boundary wall segments and guide lines | Meter | |
Search radius of connection relations of indoor wall segments | Meter | ||
The balance parameter between the data term and the smoothing term | - | ||
Indoor space decomposition | Number of emitted rays | - | |
Percentage of real wall segments in the boundary of the virtual enclosed area | - | ||
The percentage of the marked area in the free-space evidence raster | - | ||
Door extraction | The maximum limit of door height | Meter | |
The bottom limit of door height | Meter | ||
The maximum limit of door width | Meter | ||
The minimum limit of door width | Meter | ||
Filling threshold for column pixels on the wall surface | - | ||
Filling threshold for row pixels on the wall surface | - |
Data | Indoor Space Decomposition | Navigation Network | Model (Hide Ceiling) |
---|---|---|---|
Synth | |||
Office1 | |||
Office2 | |||
Room | |||
TUB1 | |||
TUB2 | |||
UOM |
Data | TUB1 | Office1 | Room |
---|---|---|---|
Proposed segmentation method | |||
Other segmentation method | |||
Ambru, et al. | Morphology-based segmentation | Yang et al. |
Data | Real Room Number | Detected Real Room Number | Correctness on 2D Floor Plane | Completeness on 2D Floor Plane |
---|---|---|---|---|
TUB1 | 10 | 11 | 0.987 | 0.992 |
TUB2 | 10 | 9 | 0.981 | 0.952 |
UOM | 7 | 7 | 0.983 | 0.936 |
Data | Door Number | Detected Door Number | Correctly Detected Door Number | Correctness on Door | Completeness on Door |
---|---|---|---|---|---|
TUB1 | 23 | 22 | 20 | 0.91 | 0.87 |
TUB2 | 28 | 29 | 27 | 0.93 | 0.96 |
UOM | 14 | 13 | 12 | 0.92 | 0.86 |
Data | Real Room Number | Detected Real Room Number | Detected Wall Surface Number | |
---|---|---|---|---|
Synth | 3 | 3 | 25 | |
Office1 | 7 | 7 | 106 | |
Office2 | 6 | 6 | 36 | |
Room | 6 | 6 | 29 | |
Data | Door Number | Detected Door Number | Correctnesson Door | Completenesson Door |
Synth | 3 | 3 | 1 | 1 |
Office1 | 18 | 18 | 0.88 | 0.88 |
Office2 | 5 | 5 | 1 | 1 |
Room | 4 | 5 | 0.80 | 1 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, G.; Wei, S.; Zhong, S.; Huang, S.; Zhong, R. Reconstruction of Indoor Navigation Elements for Point Cloud of Buildings with Occlusions and Openings by Wall Segment Restoration from Indoor Context Labeling. Remote Sens. 2022, 14, 4275. https://doi.org/10.3390/rs14174275
Liu G, Wei S, Zhong S, Huang S, Zhong R. Reconstruction of Indoor Navigation Elements for Point Cloud of Buildings with Occlusions and Openings by Wall Segment Restoration from Indoor Context Labeling. Remote Sensing. 2022; 14(17):4275. https://doi.org/10.3390/rs14174275
Chicago/Turabian StyleLiu, Guangzu, Shuangfeng Wei, Shaobo Zhong, Shuai Huang, and Ruofei Zhong. 2022. "Reconstruction of Indoor Navigation Elements for Point Cloud of Buildings with Occlusions and Openings by Wall Segment Restoration from Indoor Context Labeling" Remote Sensing 14, no. 17: 4275. https://doi.org/10.3390/rs14174275
APA StyleLiu, G., Wei, S., Zhong, S., Huang, S., & Zhong, R. (2022). Reconstruction of Indoor Navigation Elements for Point Cloud of Buildings with Occlusions and Openings by Wall Segment Restoration from Indoor Context Labeling. Remote Sensing, 14(17), 4275. https://doi.org/10.3390/rs14174275