# Polylidar3D-Fast Polygon Extraction from 3D Data

^{*}

## Abstract

**:**

## 1. Introduction

- An efficient and versatile open source [15] framework for concave (multi)polygon extraction for 3D data. Input can be unorganized/organized 3D point clouds or user-provided meshes.
- A fast open source [16] dominant plane normal estimation procedure using a Gaussian Accumulator that can also be used as a stand-alone algorithm.
- Improved half-edge triangulation efficiency for organized point clouds; CPU multi-threaded and GPU accelerated mesh smoothing [16].
- Planar segmentation and polygon extraction performed in tandem using task-based parallelism to reduce latency for time-critical applications.

## 2. Background

#### 2.1. Planar Segmentation

- Point sets are naturally bounded (i.e., have finite extent). Bounded surfaces better correspond with most real-world flat surfaces.
- Holes inside a plane may be represented implicitly by the absence of points. This representation can also indicate obstacles embedded in a flat surface.
- Best-fit geometric planes can also be computed after segmentation using least-squares, principal component analysis (PCA), or RANSAC based methods [20].
- Merging similar planes can be rapidly performed by combining their points sets.

#### 2.2. Polygonal Shape Extraction

- Significantly reduced memory requirements, on the order of square root (perimeter vs. area).
- Faster computation of geometric values of interest, e.g., centroid, area, perimeter.
- Ability to dilate, erode, and simplify polygons through computational geometry routines.
- Holes inside a polygon account for gaps or obstacles on flat surfaces.

#### 2.3. 3D Data Denoising

#### 2.4. Dominant Plane Normal Estimation

## 3. Preliminaries

## 4. Mesh Creation

#### 4.1. Unorganized 3D Point Clouds

#### 4.2. Organized 3D Point Clouds

Algorithm 1: Extract Triangles from OPC. |

Algorithm 2: Extract Half-Edges from OPC. |

#### 4.3. User Provided Meshes

- Every vertex connects to a single edge-connected set of triangles.
- Every edge is shared by one or two triangles.

- Store only the first pair of edges found and ignore any others.
- Select the pair of edges that are most similar. Similarity between edges is defined by comparing angular distance of their owning triangle normals.
- Ignore all of them by labelling all as boundary edges.

## 5. Mesh Smoothing

#### 5.1. Laplacian Filter

- Neighboring vertices do not need to be found through lookup over $\mathcal{T}$, $\mathcal{HE}$, or an adjacency list.
- Neighborhood size can be increased by adjusting filter kernel-size. Increasing the kernel size is critical for extremely dense and noisy point clouds.
- Parallelization is trivial, similar to image filters, with all necessary neighborhood data for a vertex located close in memory.

#### 5.2. Bilateral Filter

## 6. Dominant Plane Normal Estimation

#### 6.1. Gaussian Accumulator

#### 6.1.1. Refined Icosahedron

#### 6.1.2. Gaussian Accumulator Properties

#### 6.1.3. Integrating the Gaussian Accumulator With Search

Algorithm 3: Find Cell Index. |

#### 6.2. Peak Detection

#### Application To Polylidar3D

## 7. Planar Segmentation and Polygon Extraction

#### 7.1. Planar Segmentation

Algorithm 4: Group Assignment. |

`ExtractPlanarSegment`uses the seed triangle to create edge-connected triangular subsets from $\mathcal{T}$ which have the same group label in $\mathcal{G}$ and meet user-provided planarity constraints (Line 7). If a user-specified minimum number of triangles is met then this set, ${\mathcal{T}}_{i}^{g}$, is added to ${\mathcal{T}}^{g}$. A dynamic task is then created to perform polygon extraction for this segment (Line 10). This procedure call is non-blocking; the region growing task continues to extract any remaining spatially connected planar segments before terminating. This means planar segmentation and polygon extraction may occur in parallel if multi-core is enabled.

Algorithm 5: Region Growing Task. |

#### 7.2. Polygon Extraction

- Data Structure Initialization
- Extract Exterior Hull/Shell
- Extract Interior Holes

#### 7.3. Algorithm Parallelization

`OpenMP`for data parallelism which is carried out in “hot” loops that are iteration independent, e.g., triangle group assignment in Algorithm 4 as well as computing triangle normals. We use the

`MARL`library to handle task scheduling and synchronization primitives [52]. Note that region growing of a single dominant plane normal is still a serial process as is the polygon extraction process of a single planar triangular segment. Therefore if only one dominant plane normal exists than task-based parallelism will provide minimal speed up. However group assignment is still fully parallelized. Benefits of parallelism are further explored in Section 9.4 experiments where speedup is calculated as number of threads and number of dominant plane normals vary.

## 8. Post Processing

- Polygon is simplified with parameter $\alpha $ meters
- Polygon is buffered outward by parameter ${\beta}_{pos}$ meters
- Polygon is buffered inward by parameter ${\beta}_{neg}$ meters
- Polygon is removed if its area is less than $\gamma $ meters
- Interior holes are removed if area is less than $\delta $ meters

## 9. Results

#### 9.1. Dominant Plane Normal Estimation

`scikit-image`image processing library [58]. Agglomerative hierarchical clustering (AHC) of any detected peaks is implemented in Python with the

`scipy`library [59]. The unwrapped 2D image of the icosahedron does not depend on the number of integrated normals but only the refinement level of the GA. Generated images are rather small (e.g., 90 × 34 pixels for a level four GA) resulting in very fast peak detection and clustering, e.g., it takes approximately 1 ms to detect peaks and perform AHC on a level four refined GA. FastGA results from additional datasets are shown below.

#### 9.2. Unorganized 3D Point Clouds

#### 9.2.1. Rooftop Detection

#### 9.2.2. Ground and Obstacle Detection

#### 9.2.3. Qualitative Results

#### 9.2.4. Execution Timings

#### 9.3. Organized 3D Point Clouds

#### 9.3.1. RGBD Cameras

#### 9.3.2. SynPEB Benchmark

#### 9.3.3. Organized Point Cloud Mesh Smoothing

#### 9.4. User-Defined Meshes

#### Parallelization Analysis

## 10. Discussion

#### 10.1. Point Cloud Characteristics and Parameter Selection

#### 10.2. Algorithmic Complexity

#### 10.3. Future Work

## 11. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

MDPI | Multidisciplinary Digital Publishing Institute |

LiDAR | Light Detection and Ranging |

RGBD | Red, Green, Blue, Depth (Image) |

OPC | Organized Point Cloud |

AHC | Agglomerative Hierarchical Clustering |

OGC | Open Geospatial Consortium |

GA | Gaussian Accumulator |

CPU | Central Processing Unit |

GPU | Graphics Processing Unit |

RAM | Random Access Memory |

GPGPU | General Purpose GPU |

SDK | Software Development Kit |

## References

- Pathak, K.; Birk, A.; Vaskevicius, N.; Pfingsthorn, M.; Schwertfeger, S.; Poppinga, J. Online three-dimensional SLAM by registration of large planar surface segments and closed-form pose-graph relaxation. J. Field Robot.
**2010**, 27, 52–84. [Google Scholar] [CrossRef] - Malihi, S.; Valadan Zoej, M.J.; Hahn, M.; Mokhtarzade, M.; Arefi, H. 3D Building Reconstruction Using Dense Photogrammetric Point Cloud. In Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Prague, Czech Republic, 12–19 July 2016; pp. 71–74. [Google Scholar] [CrossRef] [Green Version]
- Lerma, J.L.; Navarro, S.; Cabrelles, M.; Villaverde, V. Terrestrial laser scanning and close range photogrammetry for 3D archaeological documentation: The Upper Palaeolithic Cave of Parpalló as a case study. J. Archaeol. Sci.
**2010**, 37, 499–507. [Google Scholar] [CrossRef] - Balsa-Barreiro, J.; Fritsch, D. Generation of visually aesthetic and detailed 3D models of historical cities by using laser scanning and digital photogrammetry. Digit. Appl. Archaeol. Cult. Herit.
**2018**, 8, 57–64. [Google Scholar] [CrossRef] - Rusinkiewicz, S.; Levoy, M. Efficient variants of the ICP algorithm. In Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, Quebec City, QC, Canada, 28 May–1 June 2001; pp. 145–152. [Google Scholar] [CrossRef] [Green Version]
- Feng, C.; Taguchi, Y.; Kamat, V.R. Fast plane extraction in organized point clouds using agglomerative hierarchical clustering. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014; pp. 6218–6225. [Google Scholar] [CrossRef]
- Pham, T.T.; Eich, M.; Reid, I.; Wyeth, G. Geometrically consistent plane extraction for dense indoor 3D maps segmentation. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 4199–4204. [Google Scholar] [CrossRef]
- Schaefer, A.; Vertens, J.; Buscher, D.; Burgard, W. A Maximum Likelihood Approach to Extract Finite Planes from 3-D Laser Scans. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019; pp. 72–78. [Google Scholar] [CrossRef]
- Lee, T.k.; Lim, S.; Lee, S.; An, S.; Oh, S.y. Indoor mapping using planes extracted from noisy RGB-D sensors. In Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, 7–12 October 2012; pp. 1727–1733. [Google Scholar] [CrossRef]
- Cao, R.; Zhang, Y.; Liu, X.; Zhao, Z. Roof plane extraction from airborne lidar point clouds. Int. J. Remote Sens.
**2017**, 38, 3684–3703. [Google Scholar] [CrossRef] - Biswas, J.; Veloso, M. Planar polygon extraction and merging from depth images. In Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, 7–12 October 2012; pp. 3859–3864. [Google Scholar] [CrossRef] [Green Version]
- Holz, D.; Behnke, S. Fast Range Image Segmentation and Smoothing Using Approximate Surface Reconstruction and Region Growing. In Intelligent Autonomous Systems 12; Advances in Intelligent Systems and Computing; Lee, S., Ed.; Springer: Berlin/Heidelberg, Germany, 2013; Volume 194, pp. 61–73. [Google Scholar] [CrossRef]
- Edelsbrunner, H.; Kirkpatrick, D.; Seidel, R. On the shape of a set of points in the plane. IEEE Trans. Inf. Theory
**1983**, 29, 551–559. [Google Scholar] [CrossRef] [Green Version] - Castagno, J.; Atkins, E. Polylidar-Polygons From Triangular Meshes. IEEE Robot. Autom. Lett.
**2020**, 5, 4634–4641. [Google Scholar] [CrossRef] - Github-Polylidar. 2020. Available online: https://github.com/JeremyBYU/polylidar (accessed on 5 July 2020).
- Github-Fast Gaussian Sphere Accumulator. 2020. Available online: https://github.com/JeremyBYU/FastGaussianAccumulator (accessed on 5 July 2020).
- Github-Polylidard3D and KITTI. 2020. Available online: https://github.com/JeremyBYU/polylidar-kitti (accessed on 5 July 2020).
- Github-Polylidar3D with RealSense. 2020. Available online: https://github.com/JeremyBYU/polylidar-realsense (accessed on 5 July 2020).
- Github-Polylidard3D and SynPEB. 2020. Available online: https://github.com/JeremyBYU/polylidar-plane-benchmark (accessed on 5 July 2020).
- Kaiser, A.; Ybanez Zepeda, J.A.; Boubekeur, T. A Survey of Simple Geometric Primitives Detection Methods for Captured 3D Data. Comput. Graph. Forum
**2019**, 38, 167–196. [Google Scholar] [CrossRef] - Trevor, A.J.; Gedikli, S.; Rusu, R.B.; Christensen, H.I. Efficient organized point cloud segmentation with connected components. In Proceedings of the Semantic Perception Mapping and Exploration (SPME), Karlsruhe, Germany, 5 May 2013. [Google Scholar]
- Salas-Moreno, R.F.; Glocken, B.; Kelly, P.H.J.; Davison, A.J. Dense planar SLAM. In Proceedings of the 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany, 10–12 September 2014; pp. 157–164. [Google Scholar] [CrossRef]
- Oesau, S.; Lafarge, F.; Alliez, P. Planar Shape Detection and Regularization in Tandem: Planar Shape Detection and Regularization in Tandem. Comput. Graph. Forum
**2016**, 35, 203–215. [Google Scholar] [CrossRef] [Green Version] - Poppinga, J.; Vaskevicius, N.; Birk, A.; Pathak, K. Fast plane detection and polygonalization in noisy 3D range images. In Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France, 22–26 September 2008; pp. 3378–3383. [Google Scholar] [CrossRef]
- Furieri, A. Spatialite. 2017. Available online: https://www.gaia-gis.it/fossil/libspatialite/index (accessed on 14 January 2020).
- OSGeo. PostGIS. 2019. Available online: https://postgis.net/docs/ST_ConcaveHull.html (accessed on 14 January 2020).
- Github-Benchmark Concave Hull. 2020. Available online: https://github.com/JeremyBYU/concavehull-evaluation (accessed on 5 January 2020).
- Taubin, G. Curve and surface smoothing without shrinkage. In Proceedings of the of IEEE International Conference on Computer Vision, Cambridge, MA, USA, 20–23 June 1995; pp. 852–857. [Google Scholar] [CrossRef]
- Zheng, Y.; Fu, H.; Au, O.K.C.; Tai, C.L. Bilateral normal filtering for mesh denoising. IEEE Trans. Vis. Comput. Graph.
**2011**, 17, 1521–1530. [Google Scholar] [CrossRef] - Sun, X.; Rosin, P.L.; Martin, R.; Langbein, F. Fast and Effective Feature-Preserving Mesh Denoising. IEEE Trans. Vis. Comput. Graph.
**2007**, 13, 925–938. [Google Scholar] [CrossRef] - Borrmann, D.; Elseberg, J.; Lingemann, K.; Nüchter, A. The 3D Hough Transform for plane detection in point clouds: A review and a new accumulator design. 3D Res.
**2011**, 2, 3. [Google Scholar] [CrossRef] - Limberger, F.A.; Oliveira, M.M. Real-time detection of planar regions in unorganized point clouds. Pattern Recognit.
**2015**, 48, 2043–2053. [Google Scholar] [CrossRef] [Green Version] - Cohen, T.; Weiler, M.; Kicanaoglu, B.; Welling, M. Gauge Equivariant Convolutional Networks and the Icosahedral CNN. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA, 10–15 June 2019; Volume 97, pp. 1321–1330. [Google Scholar]
- Toony, Z.; Laurendeau, D.; Gagné, C. Describing 3D Geometric Primitives Using the Gaussian Sphere and the Gaussian Accumulator. 3D Res.
**2015**, 6, 42. [Google Scholar] [CrossRef] - Paris, R. Modified half-edge data structure and its applications to 3D mesh generation for complex tube networks. Electron. Theses Diss.
**2013**. [Google Scholar] [CrossRef] - Herring, J.R. OpenGIS Implementation Specification for Geographic Information-Simple Feature Access-Part 1: Common Architecture; Open Geospatial Consortium: Wayland, MA, USA, 2006; p. 95. [Google Scholar]
- de Berg, M.; Cheong, O.; van Kreveld, M.; Overmars, M. Delaunay Triangulations: Height Interpolation. In Computational Geometry: Algorithms and Applications; Springer: Berlin/Heidelberg, Germany, 2008; pp. 191–218. [Google Scholar]
- Github-Delaunator. 2020. Available online: https://github.com/mapbox/delaunator (accessed on 5 January 2020).
- Richard Shewchuk, J. Adaptive Precision Floating-Point Arithmetic and Fast Robust Geometric Predicates. Discret. Comput. Geom.
**1997**, 18, 305–363. [Google Scholar] [CrossRef] [Green Version] - Kazhdan, M.; Hoppe, H. Screened poisson surface reconstruction. ACM Trans. Graph.
**2013**, 32, 29:1–29:13. [Google Scholar] [CrossRef] [Green Version] - Bernardini, F.; Mittleman, J.; Rushmeier, H.; Silva, C.; Taubin, G. The ball-pivoting algorithm for surface reconstruction. IEEE Trans. Vis. Comput. Graph.
**1999**, 5, 349–359. [Google Scholar] [CrossRef] - Zhou, Q.Y.; Koltun, V. Dense Scene Reconstruction with Points of Interest. ACM Trans. Graph.
**2013**, 32. [Google Scholar] [CrossRef] - Zhou, Q.Y.; Park, J.; Koltun, V. Open3D: A modern library for 3D data processing. arXiv
**2018**, arXiv:1801.09847. [Google Scholar] - Github-Organized Point Filters. 2020. Available online: https://github.com/JeremyBYU/OrganizedPointFilters (accessed on 5 July 2020).
- Wenninger, M.J. Spherical Models; Courier Corporation: North Chelmsford, MA, USA, 1999; Volume 3. [Google Scholar]
- Mokbel, M.F.; Aref, W.G. Space-Filling Curves. In Encyclopedia of GIS; Shekhar, S., Xiong, H., Eds.; Springer: Boston, MA, USA, 2008; pp. 1068–1072. [Google Scholar] [CrossRef]
- Google. S2 Geometry. 2020. Available online: https://s2geometry.io/devguide/s2cell_hierarchy#s2cellid-numbering (accessed on 5 July 2020).
- Van Sandt, P.; Chronis, Y.; Patel, J.M. Efficiently Searching In-Memory Sorted Arrays: Revenge of the Interpolation Search? In Proceedings of the 2019 International Conference on Management of Data; SIGMOD’19; Association for Computing Machinery: Amsterdam, The Netherlands, 2019; pp. 36–53. [Google Scholar] [CrossRef]
- Khuong, P.V.; Morin, P. Array Layouts for Comparison-Based Searching. J. Exp. Algorithmics
**2017**, 22, 1.3:1–1.3:39. [Google Scholar] [CrossRef] - Dagum, L.; Menon, R. OpenMP: An industry standard API for shared-memory programming. IEEE Comput. Sci. Eng.
**1998**, 5, 46–55. [Google Scholar] [CrossRef] [Green Version] - Huang, T.W.; Lin, C.X.; Guo, G.; Wong, M. Cpp-Taskflow: Fast Task-Based Parallel Programming Using Modern C++. In Proceedings of the 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS), Rio de Janeiro, Brazil, 20–24 May 2019. [Google Scholar] [CrossRef]
- Github-A Hybrid Thread/Fiber Task Scheduler. 2020. Available online: https://github.com/google/marl (accessed on 5 July 2020).
- Whyatt, J.; Wade, P. The Douglas-Peucker line simplification algorithm. Bull.-Soc. Univ. Cartogr.
**1988**, 22, 17–25. [Google Scholar] - Flato, E.; Halperin, D. Robust and Efficient Construction of Planar Minkowski Sums; University of Tel-Aviv: Tel Aviv-Yafo, Israel, 2000. [Google Scholar]
- Github-Shapely: Manipulation and Analysis of Geometric Objects. Available online: https://github.com/Toblerity/Shapely (accessed on 5 July 2020).
- Blanco, J.L.; Rai, P.K. Nanoflann: A C++ Header-Only Fork of FLANN, a Library for Nearest Neighbor (NN) with KD-Trees. 2014. Available online: https://github.com/jlblancoc/nanoflann (accessed on 5 July 2020).
- Github-Google Benchmark. 2020. Available online: https://github.com/google/benchmark (accessed on 5 July 2020).
- Van der Walt, S.; Schönberger, J.L.; Nunez-Iglesias, J.; Boulogne, F.; Warner, J.D.; Yager, N.; Gouillart, E.; Yu, T.; the scikit-image contributors. scikit-image: Image processing in Python. PeerJ Inc.
**2014**, 2, e453. [Google Scholar] [CrossRef] [PubMed] - Virtanen, P.; Gommers, R.; Oliphant, T.E.; Haberland, M.; Reddy, T.; Cournapeau, D.; Burovski, E.; Peterson, P.; Weckesser, W.; Bright, J.; et al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nat. Methods
**2020**, 17, 261–272. [Google Scholar] [CrossRef] [Green Version] - Open Geo Data. 2020. Available online: https://www.opengeodata.nrw.de/produkte/geobasis/hm/3dm_l_las/3dm_l_las/ (accessed on 5 July 2020).
- Open Geo Data. 2020. Available online: https://www.opengeodata.nrw.de/produkte/geobasis/lbi/dop/dop_jp2_f10_paketiert/ (accessed on 5 July 2020).
- Data licence Germany - Zero - Version 2.0. Available online: https://www.govdata.de/dl-de/zero-2-0 (accessed on 5 July 2020).
- Geiger, A.; Lenz, P.; Stiller, C.; Urtasun, R. Vision meets Robotics: The KITTI Dataset. Int. J. Robot. Res. (IJRR)
**2013**, 32, 1231–1237. [Google Scholar] [CrossRef] [Green Version] - Jeremy Castagno. Polylidar3D Kitti Videos. 2020. Available online: https://drive.google.com/drive/folders/18R0alYprRYgwz5_MyzcdOQzf44496DOz?usp=sharing (accessed on 30 July 2020).
- Ahn, M.S.; Chae, H.; Noh, D.; Nam, H.; Hong, D. Analysis and Noise Modeling of the Intel RealSense D435 for Mobile Robots. In Proceedings of the 2019 16th International Conference on Ubiquitous Robots (UR), Jeju, Korea, 24–27 June 2019; pp. 707–711. [Google Scholar]
- Github-Intel RealSense SDK. 2020. Available online: https://github.com/IntelRealSense/librealsense/blob/master/doc/post-processing-filters.md (accessed on 5 July 2020).
- Hoover, A.; Jean-Baptiste, G.; Jiang, X.; Flynn, P.; Bunke, H.; Goldgof, D.; Bowyer, K.; Eggert, D.; Fitzgibbon, A.; Fisher, R. An experimental comparison of range image segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell.
**1996**, 18, 673–689. [Google Scholar] [CrossRef] [Green Version] - Torr, P.; Zisserman, A. MLESAC: A New Robust Estimator with Application to Estimating Image Geometry. Comput. Vis. Image Underst.
**2000**, 78, 138–156. [Google Scholar] [CrossRef] [Green Version] - Amdahl, G.M. Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities. In Proceedings of the Spring Joint Computer Conference, Association for Computing Machinery, AFIPS ’67 (Spring), New York, NY, USA, 18–20 April 1967; pp. 483–485. [Google Scholar] [CrossRef]
- Balsa-Barreiro, J.; Avariento, J.P.; Lerma, J.L. Airborne light detection and ranging (LiDAR) point density analysis. Sci. Res. Essays
**2012**, 7, 3010–3019. [Google Scholar] [CrossRef] - Graham, L. Random Points: How Dense Are You, Anyway? LiDAR News Mag.
**2014**, 4, 5. [Google Scholar] - Bergelt, R.; Khan, O.; Hardt, W. Improving the intrinsic calibration of a Velodyne LiDAR sensor. In Proceedings of the 2017 IEEE SENSORS, Glasgow, UK, 29 October–1 November 2017; pp. 1–3. [Google Scholar] [CrossRef]
- Github-Intel RealSense Post Processing. 2019. Available online: https://github.com/IntelRealSense/librealsense/blob/master/doc/post-processing-filters.md (accessed on 5 January 2020).
- Pham, T.T.; Do, T.T.; Sünderhauf, N.; Reid, I. SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 3213–3220. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Overview of Polylidar3D. Input data can be 2D point sets, unorganized/organized 3D point clouds, or user-provided meshes. Polylidar3D’s front-end transforms input data to a half-edge triangulation structure. The back-end is responsible for mesh smoothing, dominant plane normal estimation, planar segmentation, and polygon extraction. Polylidar3D outputs both planes (sets of spatially connected triangles) and corresponding polygonal representations. An example output of color-coded extracted planes from organized point clouds is shown (top right). An example of extracted polygons from a user-provided mesh is shown (bottom right). The green line represents the concave hull; orange lines show interior holes representing obstacles.

**Figure 2.**Example polygons that can be generated from plane segmented point clouds. (

**a**) 2D point set representation of a floor diagram with interior offices; (

**b**) Convex polygon; (

**c**) non-convex polygon; (

**d**) and non-convex polygon with holes. The exterior hull (green) and interior holes (orange) are indicated.

**Figure 3.**Examples of integrating normals from two identically “noisy” planes (red arrows, Gaussian distributed) into a UV Sphere (

**a**,

**b**) and Ball Sphere (

**c**,

**d**). Note the anisotropic property of sphere cells caused by unequal area and shape; (

**e**) The UV Sphere histogram is unable to detect the peak at the pole; (

**f**) The Ball sphere is able to detect both peaks, but the north pole cell is significantly larger leading to an incorrectly higher value than the equator cell.

**Figure 4.**Example conversion of an unorganized 3D point cloud to a 3D triangular mesh. (

**a**) Synthetic point cloud of a rooftop scene generated from an overhead laser scanner. A single wall is captured because the scanner is slightly angled; (

**b**) The point cloud is projected to the xy plane and triangulated, generating the dual 3D mesh. Only planes aligned with the xy plane can be captured.

**Figure 5.**Example conversion of a 7 × 7 Organized Point Cloud (OPC) to a half-edge triangular mesh. Points are represented by circles; red indicates an invalid value (e.g., 0 depth measurement). (

**a**) Implicit mesh of OPC with right cut triangles, ${\mathcal{T}}_{FC}$. A unique global id GID = 2 · (u · (N − 1) + v) + k is shown inside each triangle; (

**b**) Indexing scheme to define GIDs for triangles in ${\mathcal{T}}_{FC}$; (

**c**) Final mesh $\mathcal{T}$ with triangles created if and only if all vertices are valid. Unique indices into $\mathcal{T}$ are marked. ${\mathcal{T}}_{map}$ maps between GIDs in ${\mathcal{T}}_{FC}$ to $\mathcal{T}$.

**Figure 6.**Example non-manifold meshes that exhibit condition (1) violations. Red vertices specifically show examples where the vertex triangle set is not fully edge-connected. (

**a**–

**c**) will extract any missing triangles as holes from the blue triangular segment; (

**d**) The missing triangles in this mesh also cause condition (1) violations but will not be captured as holes. They cause the green and blue portion of the mesh to not be edge-connected for region growing in Section 7.1. They will be extracted as two separate segments with no holes in their respective interiors

**Figure 7.**Examples of non-manifold meshes where an edge is shared by more than two triangles. This common edge is shared by green and orange triangles. The green triangles form a two-manifold mesh with the blue triangles while the orange triangle(s) do not. The orange triangles in (

**a**,

**b**) have sufficiently different normals such the green triangles half-edges can be easily linked. However all triangles in (

**c**) have nearly equal normals making this impossible.

**Figure 8.**Visualization of a triangle’s neighborhood during bilateral filtering of an organized point cloud. Each 2 × 2 point group forms two triangles creating a mesh. Each triangle’s neighbors are defined by the kernel size. For example a kernel size of five includes all blue and red triangles.

**Figure 9.**Approximation of the unit sphere with an icosahedron. The level 0 icosahedron is shown on the left with increasing refinements to the right. Triangle cells become buckets of a histogram on S2.

**Figure 10.**Space filling curve (SFC) of a level 4 icosahedron generated using the S2 Geometry Library. Each cell’s surface normal is mapped to an integer creating a linear ordering for a curve. The curve is colored according to this mapping and traverses each cell.

**Figure 11.**Linear prediction model for a level four Gaussian Accumulator (GA). The 5120 GA cells are sorted in an array (Cells) by their corresponding spatial index s2id; (

**a**) Plot relating cell s2id and index position in the Cells array with a regressed line (green) to the data; (

**b**) Zoomed-in view showing model error (red line) indicating difference between predicted and actual index for each s2id.

**Figure 12.**(

**a**) Example basement scene mesh; (

**b**) Mesh triangle normals are integrated into the Gaussian Accumulator and colorized showing peaks for the floor and walls; (

**c**) Overview of unwrapping a refined icosahedron into a 2D image. Five overlapping charts are stitched together to create a grid. Padding between charts is accomplished by copying adjoining chart neighbors using the unwrapping process and its illustration from [33]; (

**d**) Unwrapped Gaussian Accumulator creating a 2D image used for peak detection. White boxes indicate detected peaks. Duplicate peaks are merged (1 & 5) with agglomerative hierarchical clustering.

**Figure 13.**(

**a**) An example mesh to demonstrate planar segmentation and polygon extraction using two dominant plane normals, represented by the floor and wall; (

**b**) Every triangle is inspected for filtering and clustered through group assignment. Blue and red triangles meet triangle edge length constraints and are within an angular tolerance of the floor and wall surface normals, respectively; (

**c**) Region growing is performed in parallel for the blue and red triangles. The top chair surface and floor are distinct planar segments; (

**d**) Polygonal representations for each planar segment are shown. The green line represents the concave hull; the orange line depicts any interior holes. Note that small segments and small interior holes are filtered.

**Figure 14.**(

**a**) Example extracted planar triangular segment ${{\mathcal{T}}_{i}}^{g}$. Note the four holes in the mesh; (

**b**) Projection of a triangle segment to a geometric plane. Only border edges (purple) are actually needed for projection; (

**c**) A polygon is extracted from border edges with a concave hull (green) and multiple interior holes (orange).

**Figure 15.**An example of polygon post processing. The shaded blue polygon is the original polygon extracted (see Figure 14c). All dashed lines indicate a new polygon generated from a step of post processing. (

**a**) The polygon is simplified; (

**b**) The polygon is applied a positive buffer. Two small holes have been “filled” in; (

**c**) The polygon is applied a negative buffer; only two holes remain.

**Figure 16.**Polygon extraction of rooftops from unorganized 3D point clouds generated by airborne LiDAR. Each figure (

**a**–

**e**) shows the satellite image overlaid with the extracted polygons (green) representing flat surfaces with interior holes (orange) representing obstacles. A colorized point cloud is also overlaid ranging from dark purple to bright yellow denoting a normalized low to high elevation. LiDAR data and satellite images are provided from [60,61] respectively, which are from ref. [62].

**Figure 17.**Polygon extraction from KITTI unorganized 3D point cloud data acquired on a moving vehicle [63]. Four scenes (

**a**–

**d**) are shown, each with two subimages. The top subimage shows polygons projected into the color image while the bottom image shows 3D point cloud and surface polygon(s) from a bird’s eye view. Obstacles on the ground such as the light and signal in (

**a**) are extracted as (orange) holes.

**Figure 18.**Real-time polygon extraction from an RGBD camera. Four scenes are shown from the basement (

**a**) and main floor (

**b**) datasets. Color and depth frames are shown side by side for each scene. Polygons are projected onto the color image. Green denotes the exterior hull, and orange denotes any interior holes in the polygon.

**Figure 19.**Example SynPEB scene with the highest noise level. Point cloud, generated mesh, and mesh smoothed through Laplacian and bilateral filtering are shown in (

**a**–

**c**), respectively. Planes and polygons generated by Polylidar3D are shown in (

**d**,

**e**). Red, green, and yellow blocks in (

**d**) represent missed, spurious, and oversegmented planes.

**Figure 20.**Polygon extraction from meshes of an indoor home environment. (

**a**,

**b**) show results on a basement mesh which has a smoother approximation and less noise; (

**c**,

**d**) show results on a significantly larger, denser, and nosier mesh of the main floor.

**Figure 21.**Results of parallel speedup and execution timing of plane/polygon extraction with up to eight threads. Both basement (

**a**) and main floor meshes (

**b**) are analyzed. Solid lines indicate parallel speedup and link to the left y-axis while the dashed lines indicate execution time and link to the right y-axis. The color indicates number of dominant plane (DP) normals extracted.

Level | # Vertices | # Triangles | Separation |
---|---|---|---|

0 | 12 | 20 | ${41.8}^{\circ}$ |

1 | 42 | 80 | ${18.0}^{\circ}$ |

2 | 162 | 320 | ${6.9}^{\circ}$ |

3 | 642 | 1280 | ${3.1}^{\circ}$ |

4 | 2562 | 5120 | ${1.5}^{\circ}$ |

Algorithm | Mean (ms) | Std (ms) |
---|---|---|

a Synthetic: 100,000 Random Normals | ||

K-D tree | 20.0 | 0.1 |

FastGA (ours) | 9.1 | 0.1 |

b Real World: 60,620 Normals | ||

K-D tree | 9.7 | 0.2 |

FastGA (ours) | 4.4 | 0.1 |

Algorithm | Parameters |
---|---|

Laplacian Filter | $\lambda $ = 1.0, iterations = 2 |

Bilateral Filter | ${\sigma}_{l}$ = 0.1, ${\sigma}_{a}$ = 0.1, iterations = 2 |

Plane/Poly Extr. | $tr{i}_{min}$ = 200, $an{g}_{min}$ = 0.94, ${l}_{max}$ = 0.9, $vertice{s}_{min}^{hole}$ = 8, $pt{p}_{max}=0.20$ |

Poly. Filtering | $\alpha =0.1$, ${\beta}_{neg}$ = 0.1, ${\beta}_{pos}$ = 0.00, $\gamma $ = 16, $\delta $ = 0.5 |

Algorithm | Parameters |
---|---|

Plane/Poly Extr. | $tr{i}_{min}$ = 3500, $an{g}_{min}$ = 0.97, ${l}_{max}$ = 1.25, $vertice{s}_{min}^{hole}$ = 6 |

Poly. Filtering | $\alpha =0.2$, ${\beta}_{neg}$ = 0.3, ${\beta}_{pos}$ = 0.02, $\gamma $ = 30, $\delta $ = 0.5 |

Point Outlier Removal | Mesh Creation | Plane/Poly Ext. | Polygon Filtering | Total |
---|---|---|---|---|

5.1 | 4.1 | 0.7 | 6.8 | 16.7 |

Algorithm | Parameters |
---|---|

Decimation | magnitude = 2 |

Temporal | $\alpha $ = 0.3, $\delta $ = 60.0, persistence = 2 |

Spatial | $\alpha $ = 0.35, $\delta $ = 8.0, magnitude = 2, hole fill = 1 |

Threshold | max distance = 2.5 m |

Algorithm | Parameters |
---|---|

Laplacian Filter | $\lambda $ = 1.0, kernel size = 3, iterations = 2 |

Bilateral Filter | ${\sigma}_{l}$ = 0.1, ${\sigma}_{a}$ = 0.15, kernel size = 3, iterations = 2 |

FastGA | level = 3, ${v}_{min}=50,{d}_{peak}=0.28,sampl{e}_{pct}=12\%$ |

Plane/Poly Extr. | $tr{i}_{min}$ = 500, $an{g}_{min}$ = 0.96, ${l}_{max}$ = 0.05, $pt{p}_{max}=0.1$, $vertice{s}_{min}^{hole}$ = 10 |

Poly. Filtering | $\alpha =0.02$, ${\beta}_{neg}$ = 0.02, ${\beta}_{pos}$ = 0.005, $\gamma $ = 0.1, $\delta $ = 0.1 |

Scene | RS Filters | Mesh | Laplacian | Bilateral | FastGA | Plane/Poly Ext. | Poly. Filt. | Total |
---|---|---|---|---|---|---|---|---|

Basement | 2.4 | 0.4 | 0.4 | 0.5 | 1.2 | 1.7 | 4.8 | 11.4 |

Main Floor | 2.4 | 0.4 | 0.4 | 0.5 | 1.3 | 1.6 | 5.1 | 11.7 |

Algorithm | Parameters |
---|---|

Laplacian Filter | $\lambda $ = 1.0, kernel size = 5, iterations = varies (predicted) |

Bilateral Filter | ${\sigma}_{l}$ = 0.1, ${\sigma}_{a}$ = 0.1, kernel size = 3, iterations = 2 |

FastGA | level = 5, ${v}_{min}=2,{d}_{peak}=0.1,sampl{e}_{pct}=12\%$ |

Plane/Poly Extr. | $tr{i}_{min}$ = 1000, $an{g}_{min}$ = 0.95, ${l}_{max}$ = 0.1, $pt{p}_{max}=0.07$, $vertice{s}_{min}^{hole}$ = 10 |

Method | f [%] | k [%] | RMSE [mm] | ${\mathbf{n}}_{\mathbf{o}}$ | ${\mathbf{n}}_{\mathbf{u}}$ | ${\mathbf{n}}_{\mathbf{m}}$ | ${\mathbf{n}}_{\mathbf{s}}$ | time |
---|---|---|---|---|---|---|---|---|

PEAC [6] | 29.1 | 60.4 | 28.6 | 0.7 | 1.0 | 26.7 | 7.4 | 33 ms |

MSAC [68] | 7.3 | 35.6 | 34.3 | 0.3 | 1.0 | 36.3 | 10.9 | 1.1 s |

PPE [8] | 73.6 | 77.9 | 14.5 | 1.5 | 1.1 | 7.1 | 16.5 | 1.6 hr |

Polylidar3D (proposed) | 47.3 | 78.3 | 7.2 | 0.1 | 0.3 | 22.8 | 4.9 | 34 ms |

Point Cloud | Mesh Creation | Laplacian | Bilateral | FastGA | Plane/Poly Ext. | Total | f [%] |
---|---|---|---|---|---|---|---|

500 × 500 | 9.3 | 1.1 | 3.0 | 6.6 | 14.9 | 33.9 | 47.3 |

250 × 250 | 2.0 | 0.5 | 0.7 | 2.5 | 4.1 | 9.8 | 44.6 |

Data & Size | Ours | Open3D | ||
---|---|---|---|---|

CPU-S | CPU-M | GPU | CPU-S | |

SynPEB, 500 × 500 | 7.0; 35.0 | 2.0; 9.2 | 0.8; 0.9 | 205.9; 240.6 |

RGBD, 120 × 212 | 0.7; 3.5 | 0.2; 0.9 | 0.1; 0.2 | 14.3; 17.5 |

Data & Size | CPU-S | CPU-M | GPU |
---|---|---|---|

SynPEB, 500 × 500 | 73.0; 354.0 | 19.4; 90.9 | 3.2; 4.4 |

RGBD, 120 × 212 | 7.2; 35.0 | 1.9; 9.2 | 0.5; 0.5 |

Algorithm | Parameters |
---|---|

FastGA | level = 4, ${v}_{min}=15,{d}_{peak}=0.1,sampl{e}_{pct}=12\%$ |

Plane/Poly Extr. | $tr{i}_{min}$ = 80, $an{g}_{min}$ = 0.95, ${l}_{max}$ = 0.1, $pt{p}_{max}=0.08$, $vertice{s}_{min}^{hole}$ = 6 |

Poly. Filtering | $\alpha =0.01$, ${\beta}_{neg}$ = 0.025, ${\beta}_{pos}$ = 0.0, $\gamma $ = 0.07, $\delta $ = 0.05 |

Algorithm | Parameters |
---|---|

FastGA | level = 4, ${v}_{min}=15,{d}_{peak}=0.1,sampl{e}_{pct}=12\%$ |

Plane/Poly Extr. | $tr{i}_{min}$ = 1000, $an{g}_{min}$ = 0.95, ${l}_{max}$ = 0.1, $pt{p}_{max}=0.08$, $vertice{s}_{min}^{hole}$ = 6 |

Poly. Filtering | $\alpha =0.02$, ${\beta}_{neg}$ = 0.05, ${\beta}_{pos}$ = 0.02, $\gamma $ = 0.25, $\delta $ = 0.1 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Castagno, J.; Atkins, E.
Polylidar3D-Fast Polygon Extraction from 3D Data. *Sensors* **2020**, *20*, 4819.
https://doi.org/10.3390/s20174819

**AMA Style**

Castagno J, Atkins E.
Polylidar3D-Fast Polygon Extraction from 3D Data. *Sensors*. 2020; 20(17):4819.
https://doi.org/10.3390/s20174819

**Chicago/Turabian Style**

Castagno, Jeremy, and Ella Atkins.
2020. "Polylidar3D-Fast Polygon Extraction from 3D Data" *Sensors* 20, no. 17: 4819.
https://doi.org/10.3390/s20174819