1. Introduction
Buildings play a vital role in urban planning and management, urban development [
1,
2,
3,
4], and so on. The extraction of building planar features has essential applications in urban construction [
5,
6,
7,
8,
9]. The building planar includes both rooftops and facades. Building planar data are obtained primarily using three methods: aerial photogrammetry, depth cameras, and light detection and ranging (LiDAR) scanners. These methods can provide dense point clouds of the rooftops and facades of the building. Aerial photogrammetry obtains detailed textural patterns of building planar features [
10,
11,
12,
13,
14,
15], and depth cameras acquire the distances from objects to the camera [
16,
17,
18,
19]. The LiDAR scanner provides high-density and high-precision geometric structure information of objects from different positions and provides high-quality point cloud data to reconstruct three-dimensional building models [
20,
21,
22].
The planar features of buildings can be extracted using the neighborhood information of points [
23]. The Hough Transform and RANSAC algorithms are generally used to segment planes in buildings [
24,
25]. The Hough Transform is used primarily to detect geometric shapes with given characteristics (such as lines, planes, and circles). Borrmann et al. [
26] proposed a novel design where the accumulator has the same patch size in each accumulator ball space cell. The method detects planar features, such as the underlying structures of the environment. Hulik et al. [
27] presented an optimization method of the Hough Transform to detect building planes, which has the same detection effect as the RANSAC algorithm when producing a more stable output result at the same speed. However, the size of the parameter space cannot be dynamically adjusted at runtime.
The RANSAC algorithm can detect planes or cylinders based on estimating the mathematical model parameters. Awwad et al. [
28] extracted plane points through the RANSAC algorithm and solved the pseudo plane problem combining the normal vector of points. Firstly, the normal vector of each point was calculated, and then the initial plane was formed by point cloud clustering according to the angle threshold of the normal vector. Finally, the plane is optimized by the RANSAC algorithm. This algorithm can achieve a better segmentation effect. Xu et al. [
29] segmented a building roof plane using the weighted RANSAC algorithm. They designed the weight function based on differences in the error distributions between the correct and incorrect plane hypotheses. However, the weighting definition requires reliable estimation of the normal vector for the surface where the point is located, which is less effective for small buildings or low point densities relative to the roof size. Li et al. [
30] proposed an improved RANSAC method to solve the spurious plane problems for plane segmentation in buildings based on the normal distribution transformation cells for building plane segmentation. Compared with the traditional RANSAC method, the improved approach better identifies planes. However, the algorithm requires experience to select the optimal cell size under different conditions.
Other methods have been proposed to extract building planes. Sampath and Shan [
31] extracted the plane of a building roof from unstructured airborne LiDAR point clouds using the fuzzy k-means approach based on the normal surface. The coplanar segments were separated based on density clustering and connectivity. The solution framework of this method can reach the overall within-cluster precision of 2.5°. Awrangjeb et al. [
32] extracted roof feature points of buildings by combining LiDAR data with multi-spectral orthophoto imagery. However, the algorithm’s precision, especially for flat planes, decreased with the registration error. Zhou et al. [
33] extracted planar features using gradients to calculate plane parameters effectively. However, this method may be affected by the number of iterations, the maximum number of clusters, and the minimum number of cluster pixels. Arnaud et al. [
34] proposed a dynamic plane segmentation algorithm by creating point clusters with the same assumed plane parameters. Using this algorithm to estimate a room’s maximum size gave an average error of less than 10%.
Beyond the above methods, the region growing algorithm is also a classical approach to detect planes, circles, and other parameterized shapes [
35]. The algorithm’s basic idea is to gather adjacent points from regions with similar properties. Jochem et al. [
36] used the region growing algorithm to segment roof planes from buildings by combining raster data with airborne LiDAR point clouds. Compared with other methods, this method has higher completeness (94.4%) and correctness (88.4%) in 3D plane detection. However, this method also has some problems concerning the effective transmission of large datasets to different computing nodes. Kwak et al. [
37] extracted the planar points using the region growing algorithm. Firstly, the seed points are randomly selected to determine the initial growth surface by fitting the neighborhood around the seed points, and then the distance between the point and the best fitting plane is taken as the regional growth condition to segment the plane.
The region growing algorithm alone can segment building planes. However, it may not be effective to segment the coplanar points of several joint planes. Thus, most research combined the region growing algorithm with other approaches to detect building planes. Deschaud and Goulette [
38] proposed a feature extraction algorithm based on the combination of region growing algorithm and Hough Transform, which used normal vector and voxel growth to detect planes in unorganized point clouds. This method is a robust normal estimation based on noisy data points, and the optimal local seed plane is determined by calculating the local plane fraction of each point. Dong et al. [
39] segmented building rooftops from airborne LiDAR point clouds by introducing the region growing algorithm and an adaptive RANSAC algorithm. This method extracted relatively small rooftop patches in the presence of outliers. However, the algorithm’s efficiency is relatively low when more buildings are segmented simultaneously with more outliers. Vo et al. [
40] proposed the region growing algorithm based on octrees to realize building point clouds’ plane segmentation. The voxel models simplified the initial data to calculate the local surface properties, avoiding expensive searches for adjacent points. In addition, the algorithm still needs to set the voxel size based on the dataset and the desired feature detection level.
Aiming at the problem that coplanar points where two planes intersect that cannot be separated by region growing algorithm, this paper proposes an effective segmentation method for building planes that combines the region growing algorithm with the distance algorithm based on boundary points to improve the segmentation precision of building planes. This method improves the region growing algorithm when extracting building coplanar points to obtain the optimal building plane segmentation.
3. Results
The density of the cottage’s data obtained is roughly uniform, with a distance of about 0.010 m from point to point. The pantry’s data density obtained by the depth camera is uneven, and the distance is between 0.010 m and 0.040 m from some points. We evaluated the plane segmentation results using the proposed method when the distance thresholds from the uncoarse extraction plane points to each plane boundary point is set to 0.015 m, 0.020 m, 0.025 m, 0.030 m, and 0.035 m, respectively. The proposed method under different distance thresholds was tested on two datasets of indoor scenes that are cottage and pantry (
Figure 5 and
Figure 6). Furthermore, the RANSAC algorithm, the region growing algorithm, and the RANSAC-RG method were used to test point cloud data from cottage and pantry to verify the effectiveness of the proposed method.
The cottage and pantry were obtained based on the room detection datasets (Full-3D) [
48] and the Stanford Large-Scale 3D Indoor Spaces Dataset by Matterport Camera [
49], respectively. The cottage data were scanned by using the Faro Focus 3D laser range scanner. The scanner has a full 360° × 305° field of view and a high density of 91,352 points/m
2. We removed its roof points and delete some discrete points because of missing some of the roof points of the cottage.
The RANSAC algorithm randomly selects some points from a dataset to fit the plane and calculates the distance from other points to the fitting plane by continuous iterative method. If the distance from the point to the fitting plane is within the set distance threshold, the point is regarded as the interior. Otherwise, it is regarded as the exterior, and the model with the most interior points is the optimal plane model [
50,
51]. The RANSAC algorithm fitting plane points is shown in
Figure 7. The region growing algorithm can also detect plane points. However, it will not determine the intersection points of two planes (
Figure 8b). The proposed method can segment the building plane by combining the region growing algorithm with the distance algorithm based on the boundary points (
Figure 8c).
In addition, the RANSAC-RG method combining the RANSAC algorithm with region growing algorithm was to evaluate the plane segmentation results since the RANSAC algorithm can cause the plane to become over-segmented. This is especially true when two objects on the same plane cannot be separated from each other in the algorithm. The region growing algorithm can separated different objects from the same plane. The RANSAC-RG method first uses the RANSAC algorithm to segment the building plane. Then, the subplane obtained by the RANSAC algorithm is optimally segmented by the region growing algorithm. However, the angle value between the normal vector of the current seed point and the normal vector of its adjacent points is too small to identify different objects when segmenting different objects from the same plane by region growing algorithm. The edge points of the plane will be lost when identifying successfully different objects from the same plane. Therefore, the distance algorithm based on boundary points is used to optimize the segmentation results. Finally, different planes are merged if they have the same normal vector, and their shortest distance is within 0.010 m (0.025 m) for the cottage (pantry) data.
The RANSAC (region growing, RANSAC-RG) algorithm was used to perform plane segmentation of a cottage and pantry, as shown in
Figure 9 and
Figure 10 (
Figure 11,
Figure 12,
Figure 13 and
Figure 14). Finally, the proposed method under the optimal distance threshold combing the region growing algorithm with the distance algorithm based on boundary points was used to perform the cottage’s and pantry’s plane segmentation (
Figure 15 and
Figure 16). The planes were labeled with different colors.
The segmentation accuracy was evaluated by comparing the number of points manually identified as a plane with the proposed method. The performance of these algorithms can be evaluated based on the classification precision. This paper evaluates the segmentation accuracy of several large planes of cottage and pantry using the proposed method under different distance thresholds from the uncoarse extraction plane points to each plane boundary point. In addition, the plane segmentation accuracy of the RANSAC (region growing and RANSAC-RG) algorithm was evaluated to verify the effectiveness of the proposed method under the optimal distance threshold. Correct refers to the ratio of the number of correct classification points assigned to a plane by an algorithm to the number of ground truth data of a plane (manually identified). Error refers to the ratio of the number of error classification points assigned to a plane by an algorithm to the number of the ground truth data of a plane (manually identified).
In addition, the measures of precision, recall, and F1 score are effective means for evaluating object classification. The average values of precision, recall, and F1 scores of several large planes are used to evaluate the segmentation results of cottage and pantry. Precision represents the ratio of the number of correctly predicted positive samples to the total number of predicted positive samples. Recall represents the ratio of the number of correctly predicted positive samples to the number of all samples in the actual class. The F1 score is the weighted average of the precision and recall. The three measurements are calculated using the equations as follows [
33]:
where TP denotes the number of points correctly assigned to a plane that is a part of that plane in the reference, and FP denotes the number of points incorrectly unassigned to a plane that is a part of that plane in the reference. FN denotes the number of points wrongly assigned to a plane that is not a part of that plane in the reference.
4. Discussion
The segmentation accuracy of several large planes of cottage and pantry were evaluated using the proposed method under different distance thresholds from the uncoarse extraction plane points to each plane boundary point. With the increase in the distance threshold, the correct rate of the plane segmentation was improved on the whole, while the error rate increased (
Table 1 and
Table 2). However, the correct rate of the plane segmentation does not always increase with the increase in the distance threshold. The correct rate of the cottage and pantry were the highest when the distance thresholds were set to 0.025 m (planes 4 and 8) (
Table 1) and 0.030 m (plane 04, plane 06, and plane 09) (
Table 2). The possible reason for this is that some points that do not belong to the plane are assigned to the plane because the distance threshold is set too large from the uncoarse extraction plane point to the boundary point. As the distance threshold increased, the precision of the cottage and pantry increased, while the recall rate decreased (
Table 3). The maximum value of the F1 score was achieved using the proposed method for the cottage and pantry at a distance threshold of 0.025 m and 0.030 m, respectively (
Table 3). Since the F1 score is the weighted average of the precision and recall, the distance threshold corresponding to the maximum of the F1 score value was optimal in this paper. Therefore, the optimal distance thresholds from the uncoarse extraction plane points to each plane boundary point of the cottage and pantry were 0.025 m and 0.030 m, respectively.
In addition, we compared the RANSAC (region growing and RANSAC-RG) algorithm with the proposed method under the optimal distance threshold based on the segmentation accuracy of several large planes of the cottage and pantry. Although the RANSAC algorithm can segment the building plane, it may over-segment and mistakenly divide the plane. This is especially true when two objects on the same plane cannot be separated from each other in the algorithm, such as the cottage’s door (plane 5) and cottage’s wall (plane 8) (
Table 4) (
Figure 9b). The RANSAC algorithm divides several planes into the same on (plane 03 and plane 07 should be two different planes) (
Table 5) (
Figure 13b). Although the correct rate of the cottage’s plane segmentation was 100.00% (plane 1 and plane 2) (
Table 4) using the RANSAC algorithm, the error rate of the plane segmentation reached 64.17% (plane 5) and 164.97% (plane 8) (
Table 4). The error rate of the pantry’s plane segmentation using the RANSAC algorithm reached 252.84% (plane 03) and 93.32% (plane 06) (
Table 5), respectively.
The region growing algorithm uses the normal vector and curvature information to segment plane points, and the highest correct rate of the cottage’s and pantry’s plane segmentation was 97.73% (plane 4) (
Table 4) and 96.47% (plane 08) (
Table 5), respectively. However, this algorithm cannot effectively segment the building plane’s point cloud, especially the intersection point of two planes (
Figure 10 and
Figure 14). When the error rate of plane segmentation is 0.00%, the correct rate of the cottage’s and pantry’s plane segmentation is 95.16% (plane 6) (
Table 4) and 90.61% (plane 06) (
Table 5), respectively, which is the result of lacking the intersection point of the two planes.
The RANSAC-RG method combining the RANSAC algorithm and region growing algorithm can separate different objects on the same plane, such as the cottage’s door (plane 5) and cottage’s wall (plane 8) (
Table 4) (
Figure 11b). In addition, the error rate of cottage’s plane segmentation by the method was only 1.70% (plane 8) (
Table 4), while the result of the RANSAC algorithm reached 164.97% (plane 8) (
Table 4). The RANSAC algorithm segmented several planes that belong to different planes into the same plane, and then some smaller clusters connecting the two planes segmented using the region algorithm was lost, making one plane incomplete (plane 03) (
Table 5). The correct rate of the plane segmentation of the method is generally higher than that of the region growing algorithm.
Compared with the RANSAC (region growing, and RANSAC-RG) algorithm, the cottage’s and pantry’s plane segmentation’s highest correct rate reached 99.93% (plane 4) (
Table 4) and 98.55% (plane 08) (
Table 5), respectively, using the proposed method. The proposed method can segment the plane points of buildings and significantly improve the segmentation effect, especially for coplanar points at the intersection of two planes (
Figure 12 and
Figure 16).
In the coarse extraction process, some non-smooth points in the plane will be removed. The segmentation accuracy of some planes is not high if there are too many non-smooth points inside a plane when using the region growing algorithm (the RANSAC-RG method and the proposed method) (plane 1) (
Table 4). However, the RANSAC algorithm’s segmentation accuracy may not be affected (
Figure 9). In addition, if there is no point connection in the middle part of a plane, the plane will be judged as two planes based on the region growing criterion by the region growing algorithm (the RANSAC-RG method and the proposed method) (plane 09) (
Table 5) (
Figure 14,
Figure 15 and
Figure 16b). Some points in the point cloud data of the pantry obtained from the depth camera cannot be roughly assigned on the same plane due to the limitations of the matching image accuracy. Thus, the segmentation of some planes is not ideal when using the region growing algorithm (RANSAC-RG method and the proposed method); however, the RANSAC algorithm can still segment certain planes (plane 05) (
Table 5) (
Figure 12b).
Although the RANSAC-RG method can optimize the segmentation results of the RANSAC (region growing) algorithm and has little difference in segmentation effect (especially for cottage data) with the proposed method, the algorithm still loses coplanar points at some intersection of the two planes (especially for pantry data) (
Figure 15b). This is because more edge points of the subplane segmented optimally by the region growing algorithm can be lost due to the uneven distribution of the edge points, and there will also be some edge points lost even if a distance algorithm is used to optimize plane segmentation.
The proposed method, like the RANSAC algorithm, the region growing algorithm, and the RANSAC-RG method, still has misjudgments in recognizing coplanar points. The highest error rate of the cottage’s and pantry’s plane segmentation was 2.30% and 2.44%, respectively, using the proposed method.
From the overall analysis of several large planes, compared with the region growing algorithms, the RANSAC-RG method, and the proposed method, the precision value of cottage’s plane segmentation using the RANSAC algorithm reached 97.38% (
Table 6). This may be because the RANSAC algorithm can cause excessive segmentation, and it can contain more correctly classified numbers of plane points at the same time. The recall values of cottage’s (pantry’s) plane segmentation using the proposed method, the RANSAC-RG method and the region growing algorithms have a higher segmentation accuracy than the RANSAC algorithm. It can be seen that the proposed method, the RANSAC-RG method, and the region growing algorithm can segment planes more effectively compared with the RANSAC algorithm. The F1 score values of the cottage’s and pantry’s plane segmentation using the proposed method reached 97.56% and 95.75%, respectively (
Table 6).
From the above analysis, the region growing algorithm cannot separate the coplanar points of two planes, and the RANSAC algorithm causes the plane to become over-segmented. Most research combined the region growing algorithm with other approaches to detect building planes. Our idea is how to accurately extract the coplanar points of two planes that cannot be separated by region growing algorithm. The traditional way to get the coplanar points of two planes is to calculate the distance from the uncoarse extraction plane points to the coarse extraction planes. Because the boundary information can express the object’s shape feature, our idea is that the coplanar points where two planes intersect were obtained from the distance threshold from the uncoarse extraction plane points to the coarse extraction plane’s boundary points. Our method is to optimize the region growing algorithm in extracting the coplanar points at the intersection of two planes. The algorithm can accurately segment the intersection planes formed from different angles because the region growing algorithm uses normal vector and curvature to segment the coarse extracted plane points, and the distance algorithm can obtain the coplanar points of the intersection of two planes.