# Pose Normalization of Indoor Mapping Datasets Partially Compliant with the Manhattan World Assumption

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Rotation around the Vertical Axis

#### 2.2. Orientation of the Vertical Axis

#### 2.3. Unambiguousness of the Rotation around the Vertical Axis

#### 2.4. Evaluation Method

#### 2.5. Used Materials

## 3. Results

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Volk, R.; Stengel, J.; Schultmann, F. Building Information Modeling (BIM) for Existing Buildings—Literature Review and Future Needs. Autom. Constr.
**2014**, 38, 109–127. [Google Scholar] [CrossRef] [Green Version] - Jung, W.; Lee, G. The Status of BIM Adoption on Six Continents. Int. J. Civil, Struct. Constr. Arch. Eng.
**2015**, 9, 406–410. [Google Scholar] - Borrmann, A.; König, M.; Koch, C.; Beetz, J. (Eds.) Building Information Modeling: Why? What? How? In Building Information Modeling; Springer: Berlin/Heidelberg, Germany, 2018; pp. 1–24. [Google Scholar]
- Arayici, Y.; Onyenobi, T.; Egbu, C. Building Information Modeling (BIM) for Facilities Management (FM): The Mediacity Case Study Approach. Int. J. 3-D Inf. Model.
**2012**, 1, 55–73. [Google Scholar] - Becker, R.; Falk, V.; Hoenen, S.; Loges, S.; Stumm, S.; Blankenbach, J.; Brell-Cokcan, S.; Hildebrandt, L.; Vallée, D. BIM—Towards the Entire Lifecycle. Int. J. Sustain. Dev. Plan.
**2018**, 13, 84–95. [Google Scholar] [CrossRef] - Mirarchi, C.; Pavan, A.; De Marco, F.; Wang, X.; Song, Y. Supporting Facility Management Processes through End-Users’ Integration and Coordinated BIM-GIS Technologies. ISPRS Int. J. Geo-Inf.
**2018**, 7, 191. [Google Scholar] [CrossRef] [Green Version] - Gao, X.; Pishdad-Bozorgi, P. BIM-Enabled Facilities Operation and Maintenance: A Review. Adv. Eng. Inform.
**2019**, 39, 227–247. [Google Scholar] [CrossRef] - Pătrăucean, V.; Armeni, I.; Nahangi, M.; Yeung, J.; Brilakis, I.; Haas, C. State of Research in Automatic As-Built Modeling. Adv. Eng. Inform.
**2015**, 29, 162–171. [Google Scholar] [CrossRef] [Green Version] - Becker, R.; Lublasser, E.; Martens, J.; Wollenberg, R.; Zhang, H.; Brell-Cokcan, S.; Blankenbach, J. Enabling BIM for Property Management of Existing Buildings Based on Automated As-Is Capturing. In Proceedings of the 36th International Symposium on Automation and Robotics in Construction (ISARC 2019), Banff, AB, Canada, 21–24 May 2019; pp. 201–208. [Google Scholar]
- Lehtola, V.V.; Kaartinen, H.; Nüchter, A.; Kaijaluoto, R.; Kukko, A.; Litkey, P.; Honkavaara, E.; Rosnell, T.; Vaaja, M.T.; Virtanen, J.P.; et al. Comparison of the Selected State-of-the-Art 3D Indoor Scanning and Point Cloud Generation Methods. Remote Sens.
**2017**, 9, 796. [Google Scholar] [CrossRef] [Green Version] - Chen, Y.; Tang, J.; Jiang, C.; Zhu, L.; Lehtomäki, M.; Kaartinen, H.; Kaijaluoto, R.; Wang, Y.; Hyyppä, J.; Hyyppä, H.; et al. The Accuracy Comparison of Three Simultaneous Localization and Mapping (SLAM)-Based Indoor Mapping Technologies. Sensors
**2018**, 18, 3228. [Google Scholar] [CrossRef] [Green Version] - Nocerino, E.; Menna, F.; Remondino, F.; Toschi, I.; Rodríguez-Gonzálvez, P. Investigation of Indoor and Outdoor Performance of Two Portable Mobile Mapping Systems. Proc. SPIE
**2017**, 10332, 125–139. [Google Scholar] - Masiero, A.; Fissore, F.; Guarnieri, A.; Pirotti, F.; Visintini, D.; Vettore, A. Performance Evaluation of Two Indoor Mapping Systems: Low-Cost UWB-Aided Photogrammetry and Backpack Laser Scanning. Appl. Sci.
**2018**, 8, 416. [Google Scholar] [CrossRef] [Green Version] - Soudarissanane, S.; Lindenbergh, R.; Menenti, M.; Teunissen, P. Scanning geometry: Influencing factor on the quality of terrestrial laser scanning points. ISPRS J. Photogramm. Remote Sens.
**2011**, 66, 389–399. [Google Scholar] [CrossRef] - Weinmann, M. Reconstruction and Analysis of 3D Scenes—From Irregularly Distributed 3D Points to Object Classes; Springer: Cham, Switzerland, 2016. [Google Scholar]
- Nüchter, A.; Borrmann, D.; Koch, P.; Kühn, M.; May, S. A Man-Portable, IMU-Free Mobile Mapping System. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2015**, II-3/W5, 17–23. [Google Scholar] [CrossRef] [Green Version] - Blaser, S.; Cavegn, S.; Nebiker, S. Development of a Portable High Performance Mobile Mapping System Using the Robot Operating System. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2018**, IV-1, 13–20. [Google Scholar] [CrossRef] [Green Version] - Wang, C.; Hou, S.; Wen, C.; Gong, Z.; Li, Q.; Sun, X.; Li, J. Semantic Line Framework-Based Indoor Building Modeling Using Backpacked Laser Scanning Point Cloud. ISPRS J. Photogramm. Remote Sens.
**2018**, 143, 150–166. [Google Scholar] [CrossRef] - Karam, S.; Vosselman, G.; Peter, M.; Hosseinyalamdary, S.; Lehtola, V. Design, Calibration, and Evaluation of a Backpack Indoor Mobile Mapping System. Remote Sens.
**2019**, 11, 905. [Google Scholar] [CrossRef] [Green Version] - Hillemann, M.; Weinmann, M.; Mueller, M.S.; Jutzi, B. Automatic Extrinsic Self-Calibration of Mobile Mapping Systems Based on Geometric 3D Features. Remote Sens.
**2019**, 11, 1955. [Google Scholar] [CrossRef] [Green Version] - Bassier, M.; Vergauwen, M.; Poux, F. Point Cloud vs. Mesh Features for Building Interior Classification. Remote Sens.
**2020**, 12, 2224. [Google Scholar] [CrossRef] - Weinmann, M.; Jäger, M.A.; Wursthorn, S.; Jutzi, B.; Weinmann, M.; Hübner, P. 3D Indoor Mapping with the Microsoft HoloLens: Qualitative and Quantitative Evaluation by Means of Geometric Features. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2020**, V-1-2020, 165–172. [Google Scholar] [CrossRef] - Ma, Z.; Liu, S. A Review of 3D Reconstruction Techniques in Civil Engineering and their Applications. Adv. Eng. Inform.
**2018**, 37, 163–174. [Google Scholar] [CrossRef] - Kang, Z.; Yang, J.; Yang, Z.; Cheng, S. A Review of Techniques for 3D Reconstruction of Indoor Environments. ISPRS Int. J. Geo-Inf.
**2020**, 9, 330. [Google Scholar] [CrossRef] - Pintore, G.; Mura, C.; Ganovelli, F.; Fuentes-Perez, L.; Pajarola, R.; Gobbetti, E. State-of-the-Art in Automatic 3D Reconstruction of Structured Indoor Environments. In Proceedings of the Eurographics 2020, Norrkoping, Sweden, 25–29 May 2020; Volume 39, pp. 667–699. [Google Scholar]
- Ochmann, S.; Vock, R.; Klein, R. Automatic Reconstruction of Fully Volumetric 3D Building Models from Point Clouds. ISPRS J. Photogramm. Remote Sens.
**2019**, 151, 251–262. [Google Scholar] [CrossRef] [Green Version] - Yang, F.; Zhou, G.; Su, F.; Zuo, X.; Tang, L.; Liang, Y.; Zhu, H.; Li, L. Automatic Indoor Reconstruction from Point Clouds in Multi-Room Environments with Curved Walls. Sensors
**2019**, 19, 3798. [Google Scholar] [CrossRef] [Green Version] - Nikoohemat, S.; Diakité, A.A.; Zlatanova, S.; Vosselman, G. Indoor 3D Reconstruction from Point Clouds for Optimal Routing in Complex Buildings to Support Disaster Management. Autom. Constr.
**2020**, 113, 103109. [Google Scholar] [CrossRef] - Tran, H.; Khoshelham, K. Procedural Reconstruction of 3D Indoor Models from Lidar Data Using Reversible Jump Markov Chain Monte Carlo. Remote Sens.
**2020**, 12, 838. [Google Scholar] [CrossRef] [Green Version] - Wu, K.; Shi, W.; Ahmed, W. Structural Elements Detection and Reconstruction (SEDR): A Hybrid Approach for Modeling Complex Indoor Structures. ISPRS Int. J. Geo-Inf.
**2020**, 9, 760. [Google Scholar] [CrossRef] - Hübner, P.; Weinmann, M.; Wursthorn, S.; Hinz, S. Automatic Voxel-based 3D Indoor Reconstruction and Room Partitioning from Triangle Meshes. ISPRS J. Photogramm. Remote Sens.
**2021**, 181, 254–278. [Google Scholar] [CrossRef] - Furukawa, Y.; Curless, B.; Seitz, S.M.; Szeliski, R. Reconstructing Building Interiors from Images. In Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 27 September–4 October 2009; pp. 80–87. [Google Scholar]
- Gankhuyag, U.; Han, J.H. Automatic 2D Floorplan CAD Generation from 3D Point Clouds. Appl. Sci.
**2020**, 10, 2817. [Google Scholar] [CrossRef] [Green Version] - Otero, R.; Frías, E.; Lagüela, S.; Arias, P. Automatic gbXML Modeling from LiDAR Data for Energy Studies. Remote Sens.
**2020**, 12, 2679. [Google Scholar] [CrossRef] - Shi, P.; Ye, Q.; Zeng, L. A Novel Indoor Structure Extraction Based on Dense Point Cloud. ISPRS Int. J. Geo-Inf.
**2020**, 9, 660. [Google Scholar] [CrossRef] - Coughlan, J.; Yuille, A. Manhattan World: Compass Direction from a Single Image by Bayesian Inference. In Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999; Volume 2, pp. 941–947. [Google Scholar]
- Coughlan, J.M.; Yuille, A.L. Manhattan World: Orientation and Outlier Detection by Bayesian Inference. Neural Comput.
**2003**, 15, 1063–1088. [Google Scholar] [CrossRef] - Schindler, G.; Dellaert, F. Atlanta World: An Expectation Maximization Framework for Simultaneous Low-Level Edge Grouping and Camera Calibration in Complex Man-Made Environments. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 27 June–2 July 2004; pp. 1–7. [Google Scholar]
- Straub, J.; Rosman, G.; Freifeld, O.; Leonard, J.J.; Fisher III, J.W. A Mixture of Manhattan Frames: Beyond the Manhattan World. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 24–27 June 2014; pp. 3770–3777. [Google Scholar]
- Kim, P.; Coltin, B.; Kim, H. Visual Odometry with Drift-Free Rotation Estimation Using Indoor Scene Regularities. In Proceedings of the British Machine Vision Conference (BMVC), London, UK, 4–7 September 2017; pp. 62.1–62.12. [Google Scholar]
- Straub, J.; Freifeld, O.; Rosman, G.; Leonard, J.J.; Fisher, J.W., III. The Manhattan Frame Model—Manhattan World Inference in the Space of Surface Normals. IEEE Trans. Pattern Anal. Mach. Intell.
**2018**, 40, 235–249. [Google Scholar] [CrossRef] [Green Version] - Faber, A.; Förstner, W. Detection of Dominant Orthogonal Road Structures in Small Scale Imagery. Int. Arch. Photogramm. Remote Sens.
**2000**, XXXIII, 274–281. [Google Scholar] - Saurer, O.; Fraundorfer, F.; Pollefeys, M. Homography Based Visual Odometry with Known Vertical Direction and Weak Manhattan World Assumption. In Proceedings of the IROS Workshop on Visual Control of Mobile Robots (ViCoMoR), Algarve, Portugal, 11 October 2012; pp. 1–6. [Google Scholar]
- Straub, J.; Bhandari, N.; Leonard, J.J.; Fisher, J.W., III. Real-Time Manhattan World Rotation Estimation in 3D. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 28 September–2 October 2015; pp. 1913–1920. [Google Scholar]
- Peasley, B.; Birchfield, S.; Cunningham, A.; Dellaert, F. Accurate On-Line 3D Occupancy Grids using Manhattan World Constraints. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Algarve, Portugal, 7–12 October 2012; pp. 5283–5290. [Google Scholar]
- Yazdanpour, M.; Fan, G.; Sheng, W. Online Reconstruction of Indoor Scenes With Local Manhattan Frame Growing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16–17 June 2019; pp. 964–970. [Google Scholar]
- Li, Y.; Brasch, N.; Wang, Y.; Navab, N.; Tombari, F. Structure-SLAM: Low-Drift Monocular SLAM in Indoor Environments. IEEE Robot. Autom. Lett.
**2020**, 5, 6583–6590. [Google Scholar] [CrossRef] - Liu, J.; Meng, Z. Visual SLAM with Drift-Free Rotation Estimation in Manhattan World. IEEE Robot. Autom. Lett.
**2020**, 5, 6512–6519. [Google Scholar] [CrossRef] - Martens, J.; Blankenbach, J. An Evaluation of Pose-Normalization Algorithms for Point Clouds Introducing a Novel Histogram-Based Approach. Adv. Eng. Inform.
**2020**, 46, 101132. [Google Scholar] [CrossRef] - Fichtner, F.W.; Diakité, A.A.; Zlatanova, S.; Voûte, R. Semantic Enrichment of Octree Structured Point Clouds for Multi-Story 3D Pathfinding. Trans. GIS
**2017**, 22, 233–248. [Google Scholar] [CrossRef] - Gorte, B.; Zlatanova, S.; Fadli, F. Navigation in Indoor Voxel Models. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2019**, IV-2/W5, 279–283. [Google Scholar] [CrossRef] [Green Version] - Coudron, I.; Puttemans, S.; Goedemé, T.; Vandewalle, P. Semantic Extraction of Permanent Structures for the Reconstruction of Building Interiors from Point Clouds. Sensors
**2020**, 20, 6916. [Google Scholar] [CrossRef] - Hübner, P.; Weinmann, M.; Wursthorn, S. Voxel-Based Indoor Reconstruction from HoloLens Triangle Meshes. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2020**, V-4-2020, 79–86. [Google Scholar] [CrossRef] - Martens, J.; Blankenbach, J. An Automated Approach for Point Cloud Alignment Based on Density Histograms. In Proceedings of the 26th International Workshop on Intelligent Computing in Engineering, Leuven, Belgium, 30 June–3 July 2019; pp. 1–11. [Google Scholar]
- Xu, Y.; Tong, X.; Stilla, U. Voxel-Based Representation of 3D Point Clouds: Methods, Applications, and its Potential Use in the Construction Industry. Autom. Constr.
**2021**, 126, 103675. [Google Scholar] [CrossRef] - Wijmans, E.; Furukawa, Y. Exploiting 2D Floorplan for Building-Scale Panorama RGBD Alignment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 308–316. [Google Scholar]
- Chen, S.; Nan, L.; Xia, R.; Zhao, J.; Wonka, P. PLADE: A Plane-Based Descriptor for Point Cloud Registration with Small Overlap. IEEE Trans. Geosci. Remote Sens.
**2020**, 58, 2530–2540. [Google Scholar] [CrossRef] - Huang, R.; Xu, Y.; Hoegner, L.; Stilla, U. Efficient Estimation of 3D Shifts Between Point Clouds Using Low-Frequency Components of Phase Correlation. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2020**, V-2-2020, 227–234. [Google Scholar] [CrossRef] - Bassier, M.; Vincke, S.; Mattheuwsen, L.; de Lima Hernandez, R.; Derdaele, J.; Vergauwen, M. Percentage of Completion of In-Situ Cast Concrete Walls using Point Cloud Data and BIM. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2019**, XLII-5/W2, 21–28. [Google Scholar] [CrossRef] [Green Version] - Koeva, M.; Nikoohemat, S.; Oude Elberink, S.; Morales, J.; Lemmen, C.; Zevenbergen, J. Towards 3D Indoor Cadastre Based on Change Detection from Point Clouds. Remote Sens.
**2019**, 6, 206. [Google Scholar] [CrossRef] [Green Version] - Maalek, R.; Lichti, D.D.; Ruwanpura, J.Y. Automatic Recognition of Common Structural Elements from Point Clouds for Automated Progress Monitoring and Dimensional Quality Control in Reinforced Concrete Construction. Remote Sens.
**2019**, 11, 1102. [Google Scholar] [CrossRef] [Green Version] - Kazhdan, M. An Approximate and Efficient Method for Optimal Rotation Alignment of 3D Models. IEEE Trans. Pattern Anal. Mach. Intell.
**2007**, 7, 1221–1229. [Google Scholar] [CrossRef] - Papadakis, P.; Pratikakis, I.; Perantonis, S.; Theoharis, T. Efficient 3D Shape Matching and Retrieval Using a Concrete Radialized Spherical Projection Representation. Pattern Recognit.
**2007**, 40, 2437–2452. [Google Scholar] [CrossRef] [Green Version] - Chaouch, M.; Verroust-Blondet, A. A Novel Method for Alignment of 3D Models. In Proceedings of the IEEE International Conference on Shape Modeling and Applications, Stony Brook, NY, USA, 4–6 June 2008; pp. 187–195. [Google Scholar]
- Fu, H.; Cohen-Or, D.; Dror, G.; Sheffer, A. Upright Orientation of Man-Made Objects. In Proceedings of the ACM SIGGRAPH, Los Angeles, CA, USA, 11–15 August 2008; p. 42. [Google Scholar]
- Lian, Z.; Rosin, P.L.; Sun, X. A Rectilinearity Measurement for 3D Meshes. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval (MIR ’08), Vancouver, BC, Canada, 30–31 October 2008; pp. 395–402. [Google Scholar]
- Chaouch, M.; Verroust-Blondet, A. Alignment of 3D Models. Graph. Model.
**2009**, 71, 63–76. [Google Scholar] [CrossRef] [Green Version] - Lian, Z.; Rosin, P.L.; Sun, X. Rectilinearity of 3D Meshes. Int. J. Comput. Vis.
**2010**, 89, 130–151. [Google Scholar] [CrossRef] - Sfikas, K.; Theoharis, T.; Pratikakis, I. ROSy+: 3D Object Pose Normalization based on PCA and Reflective Object Symmetry with Application in 3D Object Retrieval. Int. J. Comput. Vis.
**2011**, 91, 262–279. [Google Scholar] [CrossRef] - Zhang, D.; Lu, G. Content-Based Shape Retrieval Using Different Shape Descriptors: A Comparative Study. In Proceedings of the IEEE International Conference on Multimedia and Expo, Tokyo, Japan, 22–25 August 2001; pp. 317–320. [Google Scholar]
- Tangelder, J.W.H.; Veltkamp, R.C. A Survey of Content Based 3D Shape Retrieval Methods. Multimed. Tools Appl.
**2007**, 39, 441–471. [Google Scholar] [CrossRef] - Jolliffe, I.T.; Cadima, J. Principal Component Analysis: A Review and Recent Developments. Philos. Trans. Ser. A Math. Phys. Eng. Sci.
**2016**, 374, 20150202. [Google Scholar] [CrossRef] [PubMed] - Okorn, B.; Xiong, X.; Akinci, B.; Huber, D. Toward Automated Modeling of Floor Plans. In Proceedings of the International Symposium 3D Data Processing, Visualization and Transmission 3DPVT, Paris, France, 17–20 May 2010; pp. 1–8. [Google Scholar]
- Khoshelham, K.; Díaz-Vilariño, L. 3D Modeling of Interior Spaces: Learning the Language of Indoor Architecture. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2014**, XL-5, 321–326. [Google Scholar] [CrossRef] [Green Version] - Díaz-Vilariño, L.; Khoshelham, K.; Martínez-Sánchez, J.; Arias, P. 3D Modeling of Building Indoor Spaces and Closed Doors from Imagery and Point Clouds. Sensors
**2015**, 15, 3491–3512. [Google Scholar] [CrossRef] [Green Version] - Czerniawski, T.; Nahangi, M.; Walbridge, S.; Haas, C. Automated Removal of Planar Clutter from 3D Point Clouds for Improving Industrial Object Recognition. In Proceedings of the 33rd International Symposium in Automation and Robotics in Construction ISARC, Auburn, AL, USA, 18–21 July 2016; pp. 357–365. [Google Scholar]
- Horn, B.K.P. Extended Gaussian Images. Proc. IEEE
**1984**, 72, 1671–1686. [Google Scholar] [CrossRef] [Green Version] - Wang, Y.; Hao, W.; Ning, X.; Zhao, M.; Zhang, J.; Shi, Z. Automatic Segmentation of Urban Point Clouds Based on the Gaussian Map. Photogramm. Rec.
**2013**, 28, 342–361. [Google Scholar] [CrossRef] - Shui, W.; Liu, J.; Ren, P.; Maddock, S.; Zhou, M. Automatic Planar Shape Segmentation from Indoor Point Clouds. In Proceedings of the ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry, Zhuhai, China, 3–4 December 2016; Volume 1, pp. 363–372. [Google Scholar]
- Zhao, B.; Hua, X.; Yu, K.; Xuan, W.; Chen, X.; Tao, W. Indoor Point Cloud Segmentation Using Iterative Gaussian Mapping and Improved Model Fitting. IEEE Trans. Geosci. Remote Sens.
**2020**, 58, 7890–7907. [Google Scholar] [CrossRef] - Limberger, F.A.; Oliveira, M.M. Real-Time Detection of Planar Regions in Unorganized Point Clouds. Pattern Recognit.
**2015**, 48, 2043–2053. [Google Scholar] [CrossRef] [Green Version] - MacQueen, J. Some Methods for Classification and Analysis of Multivariate Observations. In Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA, 21 June–18 July 1967; Volume 5.1, pp. 281–297. [Google Scholar]
- Lloyd, S. Least Squares Quantization in PCM. IEEE Trans. Inf. Theory
**1982**, 28, 129–137. [Google Scholar] [CrossRef] - Ester, M.; Kriegel, H.P.; Sander, J.; Xu, X. A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD’96), Portland, OR, USA, 2–4 August 1996; pp. 226–231. [Google Scholar]
- Mitra, N.J.; Nguyen, A. Estimating Surface Normals in Noisy Point Cloud Data. In Proceedings of the Nineteenth Annual Symposium on Computational Geometry (SCG ’03), San Diego, CA, USA, 8–10 June 2003; pp. 322–328. [Google Scholar]
- Boulch, A.; Marlet, R. Fast and Robust Normal Estimation for Point Clouds with Sharp Features. In Proceedings of the Eurographics Symposium on Geometry Processing, Tallinn, Estonia, 16–18 July 2012; Volume 31, pp. 1765–1774. [Google Scholar]
- Yu, Z.; Wang, T.; Guo, T.; Li, H.; Dong, J. Robust Point Cloud Normal Estimation via Neighborhood Reconstruction. Adv. Mech. Eng.
**2019**, 11, 1–19. [Google Scholar] [CrossRef] - Sanchez, J.; Denis, F.; Coeurjolly, D.; Dupont, F.; Trassoudaine, L.; Checchin, P. Robust Normal Vector Estimation in 3D Point Clouds through Iterative Principal Component Analysis. ISPRS J. Photogramm. Remote Sens.
**2020**, 163, 18–35. [Google Scholar] [CrossRef] [Green Version] - Ochmann, S.; Klein, R. Automatic Normal Orientation in Point Clouds of Building Interiors. In Proceedings of the Computer Graphics International Conference, Calgary, AB, Canada, 17–20 June 2019; pp. 556–563. [Google Scholar]
- Chang, A.; Dai, A.; Funkhouser, T.; Halber, M.; Nießner, M.; Savva, M.; Song, S.; Zeng, A.; Zhang, Y. Matterport3D: Learning from RGB-D Data in Indoor Environments. In Proceedings of the International Conference on 3D Vision (3DV), Qingdao, China, 10–12 October 2017; pp. 667–676. [Google Scholar]
- Brent, R.P. An Algorithm with Guaranteed Convergence for Finding a Zero of a Function. In Algorithms for Minimization without Derivatives; Prentice-Hall Inc.: Englewood Cliffs, NJ, USA, 1973; Chapter 4. [Google Scholar]
- Khoshelham, K.; Tran, H.; Acharya, D. Indoor Mapping Eyewear: Geometric Evaluation of Spatial Mapping Capability of HoloLens. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2019**, XLII-2/W13, 805–810. [Google Scholar] [CrossRef] [Green Version] - Hübner, P.; Landgraf, S.; Weinmann, M.; Wursthorn, S. Evaluation of the Microsoft HoloLens for the Mapping of Indoor Building Environments. In Proceedings of the Dreiländertagung der DGPF, der OVG und der SGPF, Wien, Austria, 20–22 February 2019; Volume 28, pp. 44–53. [Google Scholar]
- Hübner, P.; Clintworth, K.; Liu, Q.; Weinmann, M.; Wursthorn, S. Evaluation of HoloLens Tracking and Depth Sensing for Indoor Mapping Applications. Sensors
**2020**, 20, 1021. [Google Scholar] [CrossRef] [Green Version] - Khoshelham, K.; Díaz Vilariño, L.; Peter, M.; Kang, Z.; Acharya, D. The ISPRS Benchmark on Indoor Modeling. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2017**, XLII-2/W7, 367–372. [Google Scholar] [CrossRef] [Green Version] - Khoshelham, K.; Tran, H.; Acharya, D.; Díaz Vilariño, L.; Kang, Z.; Dalyot, S. The ISPRS Benchmark on Indoor Modeling—Preliminary Results. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2020**, 43, 207–211. [Google Scholar] [CrossRef] - Girardeau-Montaut, D. CloudCompare: 3D Point Cloud and Mesh Processing Software. Available online: http://www.cloudcompare.org (accessed on 12 April 2021).
- Ahmed, S.; Liwicki, M.; Weber, M.; Dengel, A. Improved Automatic Analysis of Architectural Floor Plans. In Proceedings of the International Conference on Document Analysis and Recognition, Beijing, China, 18–21 September 2011; pp. 864–869. [Google Scholar]
- Roth, K.; Hageny, E.; Gillmann, C. Shape Analysis and Visualization in Building Floor Plans. In Proceedings of the Leipzig Symposium on Visualization in Applications (LEVIA’20), Leipzig, Germany, 15–16 October 2020; pp. 1–9. [Google Scholar]

**Figure 2.**Exemplary triangle mesh of a building with multiple Manhattan World systems (dataset “mJXqzFtmKg4” from Matterport3D [90]). The green bounding box on the top-down-view on the right-hand side illustrates the alignment along the dominant Manhattan World structure, considered as the ground truth pose, while the red bounding box illustrates the pose rotated by 30$\xb0$ around the vertical axis, as exemplarily used in Section 2.1.

**Figure 3.**The normal vectors ${\overrightarrow{n}}_{i}$ of the triangle mesh shown in Figure 2 visualized as an extended Gaussian image (thinned out by a factor of 25 for the sake of visibility). The normal vectors ${\overrightarrow{n}}_{i}^{h}$ that are horizontal within the range of ±45$\xb0$ are visualized in black, while the others are visualized in gray. The coordinate axes are visualized in red for $\overrightarrow{x}$, green for $\overrightarrow{y}$, and blue for the vertical axis $\overrightarrow{z}$.

**Figure 4.**Visualization of a one-dimensional 360$\xb0$ grid corresponding to Figure 2. The grid cells contain the summarized weights ${w}_{i}$ of the contained angles ${\gamma}_{i}$ with value colorization ranging from blue for low values over green and yellow to red for large values.

**Figure 5.**Visualization of a one-dimensional 90$\xb0$ grid corresponding to Figure 2. The grid cells contain the summarized weights ${w}_{i}$ of the contained angles ${\tilde{\gamma}}_{i}$ with value colorization ranging from blue for low values over green and yellow to red for large values.

**Figure 6.**The horizontal faces of the triangle mesh presented in Figure 2 corresponding to the horizontal normal vectors ${\overrightarrow{n}}_{i}^{h}$. The faces corresponding to the two peaks shown in Figure 5 are depicted in red. (

**a**) Faces corresponding to the largest peak at 60$\xb0$ in Figure 5 determining the dominating Manhattan World structure. (

**b**) Faces corresponding to the minor peak at 15$\xb0$ in Figure 5.

**Figure 7.**Exemplary triangle mesh of a building with a partially slanted ceiling (dataset “Attic” from [31]). The green line visualizes the reference orientation of the vertical axis considered as the ground truth, while the red line visualizes the vertical axis rotated by $-25$$\xb0$ around the horizontal $\overrightarrow{x}$ axis and by 15$\xb0$ around the horizontal $\overrightarrow{y}$ axis, as exemplarily used in Section 2.2.

**Figure 9.**Azimuth/inclination grid of a 1$\xb0$ resolution over the whole surface of the unit sphere corresponding to Figure 7. The grid cells contain the summarized weights ${w}_{i}$ of the contained normal vectors ${\overrightarrow{n}}_{i}$ at polar angles $({\phi}_{i},{\theta}_{i})$ with value colorization ranging from blue for low values over green and yellow to red for large values.

**Figure 10.**Transformation of $(\phi ,\theta )$ positions on the whole unit sphere to $(\tilde{\phi},\tilde{\theta})$ positions on one-eighth of the unit sphere by Equations (13) and (14). (

**a**) Generally, points corresponding to opposing normal vectors are transformed to the same point. (

**b**) In case the vertical axis $\overrightarrow{z}$ is the angle bisector between the directions of two normal vectors (same angle $\delta $ to the $\overrightarrow{z}$ axis), these are transformed to the same point even if they are not opposed. This needs to be dealt with by means of a cluster analysis per $(\tilde{\phi},\tilde{\theta})$ grid cell.

**Figure 11.**Transformed azimuth/inclination grid of a 1$\xb0$ resolution corresponding to Figure 7. The grid cells contain the summarized weights ${w}_{i}$ of the contained vertical normal vectors ${\overrightarrow{n}}_{i}^{v}$ at polar angles $({\tilde{\phi}}_{i},{\tilde{\theta}}_{i})$ with value colorization ranging from blue for low values over green and yellow to red for large values. The larger peak corresponds to the floor and the horizontal part of the ceiling, while the minor peak corresponds to one of the slanted ceiling surfaces.

**Figure 12.**The Microsoft HoloLens triangle meshes published in [31] and used for evaluation in this paper. The red box indicates the aligned ground truth pose.

**Figure 14.**Detailed visualization of the dataset “Case Study 6” from the ISPRS Indoor Modeling Benchmark dataset [96], also depicted in Figure 13f. The depicted axes represent the pose w.r.t. the local coordinate system as resulting from the proposed approach. The vertical axis is visualized in blue, while the two horizontal axes aligned with the dominant Manhattan World structure of the building are depicted in red and green, respectively. Note that, despite the large amount of uneven terrain, vegetation, and building structure deviating from the Manhattan World assumption, the building is aligned w.r.t. the three rooms with the Manhattan World structure.

**Figure 15.**The triangle meshes of the Matterport3D dataset [90] used for evaluation. The red box indicates the aligned ground truth pose.

**Figure 17.**Resulting vertical alignments of the triangle mesh “Attic” from Figure 12c for the two peaks in the histogram of ${\delta}_{v}$ values depicted in Figure 16. The green bounding box corresponds to the peak at ${\delta}_{v}\approx 0\xb0$ (i.e., the ground truth pose), while the red bounding box corresponds to the minor peak at ${\delta}_{v}\approx 30\xb0$.

**Figure 20.**Resulting horizontal alignments of the triangle mesh “mJXqzFtmKg4” from Figure 15f for the two peaks in the histogram of ${\delta}_{h}$ values depicted in Figure 18. The green bounding box corresponds to the peak at ${\delta}_{h}\approx 0\xb0$ (i.e., the ground truth pose), while the red bounding box corresponds to the minor peak at ${\delta}_{h}\approx 45\xb0$.

**Figure 21.**Resulting horizontal alignments of the triangle mesh “PuKPg4mmafe” from Figure 15h for the two peaks in the histogram of ${\delta}_{h}$ values depicted in Figure 19. The green bounding box corresponds to the peak at ${\delta}_{v}\approx 0\xb0$ (i.e., the ground truth pose), while the red bounding box corresponds to the peak at ${\delta}_{v}\approx 23\xb0$.

**Figure 22.**Detailed view of the triangle mesh “PuKPg4mmafe” from the Matterport3D dataset also depicted in Figure 15h and Figure 21. Note that in the case of the larger part of the building structure determining the Manhattan World system visualized by the red bounding box in Figure 21, large parts of the wall surfaces are missing as wall openings or constituted by curtains or other structures with inhomogeneous normal direction. The smaller part of the building structure on the right-hand side, which determines the Manhattan World system visualized by the green bounding box in Figure 21, however, has largely closed, smooth wall surfaces.

**Figure 23.**The green bounding box represents the horizontal alignment of the triangle mesh “ULsKaCPVFJR” from Figure 15i as it is published in [90] and used as the ground truth pose for the evaluation results presented in Table 1. The red bounding box, on the other hand, represents the horizontal alignment resulting from our presented approach.

**Figure 24.**The green bounding box represents the horizontal alignment of the triangle mesh “ur6pFq6Qu1A” from Figure 15j as it is published in [90] and used as the ground truth pose for the evaluation results presented in Table 1. The red bounding box, on the other hand, represents the horizontal alignment resulting from our presented approach.

**Table 1.**Evaluation results for the datasets presented in Figure 12, Figure 13 and Figure 15. The presented values represent 50 randomly chosen orientations per dataset within the range of $[-180\xb0,180\xb0)$ for rotations around the vertical axis and $[-30\xb0,30\xb0]$ for rotations around the horizontal axes. The reported numbers of points for the point clouds of the ISPRS Indoor Modeling Benchmark refer to point clouds downsampled to a resolution of 2 cm as used in this evaluation. The values marked in red are discussed in more detail in Section 4.

Source | Type | Dataset | Number of Points/ Triangles | Mean ${\mathit{\delta}}_{\mathit{v}}$ [°] | Std.Dev. ${\mathit{\delta}}_{\mathit{v}}$ [°] | Mean ${\mathit{\delta}}_{\mathit{h}}$ [°] | Std.Dev. ${\mathit{\delta}}_{\mathit{h}}$ [°] | Mean Time [s] | Std.Dev. Time [s] |
---|---|---|---|---|---|---|---|---|---|

HoloLens[31] | TriangleMesh | Office | 958,820 | 0.28 | 0.25 | 0.33 | 0.07 | 0.68 | 0.10 |

Basement | 695,041 | 0.45 | 0.06 | 0.10 | 0.08 | 0.50 | 0.04 | ||

Attic | 147,146 | 3.54 | 23.86 | 0.26 | 0.42 | 0.13 | 0.02 | ||

Residential House | 252,820 | 0.16 | 0.05 | 0.71 | 0.42 | 0.30 | 0.04 | ||

ISPRSIndoorModelingBenchmark [95,96]
| PointCloud | Case Study 1 | 5,014,452 | 0.01 | 0.05 | 0.03 | 0.16 | 4.41 | 0.19 |

Case Study 2 | 8,202,319 | 0.01 | 0.02 | 0.01 | 0.13 | 7.40 | 0.26 | ||

Case Study 3 | 5,906,718 | 0.02 | 0.01 | 0.04 | 0.17 | 5.68 | 0.29 | ||

Case Study 4 | 4,846,736 | 0.01 | 0.26 | 0.03 | 0.44 | 4.19 | 0.27 | ||

Case Study 5 | 4,409,794 | 0.02 | 0.07 | 0.02 | 0.06 | 3.96 | 0.23 | ||

Case Study 6 | 11,760,325 | 0.02 | 0.02 | 0.06 | 0.77 | 8.65 | 0.53 | ||

Matterport3D[90] | TriangleMesh | 2azQ1b91cZZ | 9,549,830 | 0.03 | 0.02 | 0.44 | 0.06 | 8.24 | 0.39 |

759xd9YjKW5 | 6,208,440 | 0.05 | 0.01 | 0.18 | 0.05 | 5.48 | 0.35 | ||

ac26ZMwG7aT | 10,811,581 | 0.05 | 0.09 | 0.52 | 0.06 | 9.84 | 0.49 | ||

fzynW3qQPVF | 9,105,979 | 0.09 | 0.02 | 0.05 | 0.06 | 10.75 | 0.60 | ||

gTV8FGcVJC9 | 14,436,867 | 0.05 | 0.05 | 0.11 | 0.07 | 12.29 | 0.96 | ||

mJXqzFtmKg4 | 8,237,802 | 0.07 | 0.33 | 2.73 | 14.29 | 6.90 | 0.54 | ||

p5wJjkQkbXX | 10,678,539 | 0.07 | 0.02 | 0.40 | 0.03 | 10.35 | 0.68 | ||

PuKPg4mmafe | 1,968,102 | 0.05 | 0.01 | 15.28 | 20.07 | 1.83 | 0.11 | ||

ULsKaCPVFJR | 6,612,194 | 0.05 | 0.01 | 44.41 | 0.04 | 5.51 | 0.47 | ||

ur6pFq6Qu1A | 9,277,187 | 0.02 | 0.01 | 12.85 | 0.05 | 9.42 | 0.42 | ||

VFuaQ6m2Qom | 9,453,891 | 0.03 | 0.02 | 0.13 | 0.06 | 8.53 | 0.37 | ||

Vt2qJdWjCF2 | 6,429,106 | 0.10 | 0.01 | 0.05 | 0.09 | 6.40 | 0.38 | ||

x8F5xyUWy9e | 2,862,858 | 0.07 | 0.01 | 0.21 | 0.08 | 2.66 | 0.16 | ||

ZMojNkEp431 | 4,690,777 | 0.06 | 0.05 | 0.18 | 0.08 | 4.31 | 0.27 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hübner, P.; Weinmann, M.; Wursthorn, S.; Hinz, S.
Pose Normalization of Indoor Mapping Datasets Partially Compliant with the Manhattan World Assumption. *Remote Sens.* **2021**, *13*, 4765.
https://doi.org/10.3390/rs13234765

**AMA Style**

Hübner P, Weinmann M, Wursthorn S, Hinz S.
Pose Normalization of Indoor Mapping Datasets Partially Compliant with the Manhattan World Assumption. *Remote Sensing*. 2021; 13(23):4765.
https://doi.org/10.3390/rs13234765

**Chicago/Turabian Style**

Hübner, Patrick, Martin Weinmann, Sven Wursthorn, and Stefan Hinz.
2021. "Pose Normalization of Indoor Mapping Datasets Partially Compliant with the Manhattan World Assumption" *Remote Sensing* 13, no. 23: 4765.
https://doi.org/10.3390/rs13234765