The Point Cloud Semantic Segmentation Method for the Ming and Qing Dynasties’ Official-Style Architecture Roof Considering the Construction Regulations

Abstract

Point cloud semantic segmentation has played an important role in the scan-to-BIM process of the Ming and Qing Dynasties’ official-style architecture roof. To overcome the complexity of roof components’ shape and the scale differences between different roof component types, a point cloud semantic segmentation method for the MQDOAs roof considering the construction regulations is proposed in this paper. This method is composed of two stages. In the first stage, the features from the construction rules of MQDOAs, including the normalized symmetrical distance (NSD), relative height (RH) and local height difference (LHD), are extracted alongside the regular geometric features. To lower the influence of scale differences, a multi-scale feature connection strategy is also applied to construct the feature classification vector. In the second stage, RF method is applied to classify the point cloud. To verify the efficiency of the proposed method, we took the Hall of Complete Harmony as the study case. The experiments showed that our method achieved segmentation result in overall classification accuracy and reached 96.8%.

1. Introduction

The Ming and Qing Dynasties’ official-style architectures (MQDOAs) are considered the last peak of Chinese architectural history and were the important components of the ancient Chinese architectural system [1,2]. Suffering from weathering, fires and rotting, a mass of MQDOAs with wooden structural frame has disappeared. To preserve historical buildings, using the 3D point cloud, which provides precise geometric coordinates (X, Y, Z) in the form of millions of points, to record the shape of the cultural heritage has become one of the most efficient methods [3,4,5]. The captured original point cloud lacked structured information, such as semantics and hierarchy between parts, which disturbed the usage of point cloud in other application fields [6,7,8]. Hence, how to segment the point cloud into the sub dataset with semantic meaning has become a research hotspot.

Traditionally, the segmentation of the 3D point cloud is manually processed by using a digital workstation in a practice project. Apart from being time consuming, manual intervention for subdividing the datasets brings a certain degree of subjectivity [9]. Developing automatic or semi-automatic procedures for point cloud segmentation has become the consensus. Nowadays, automated point cloud segmentation methods have been proposed by researchers and achieve good performance [10,11,12].

However, it was still a challenge to segment the MQDOAs point cloud into the correct categories. On the one hand, the types of components composed of the MQDOAs were large, and the shapes of the components varied greatly. Only relying on the geometric features which are used in other architectural heritages made it difficult to meet the segmentation requirements [13]. On the other hand, the scale differences between different types of components also damaged the segmentation accuracy. The MQDOAs roof acted as a peculiarly characteristic and ornate part, and was the most complicated. Hence, the research on how to segment the point cloud of the MQDOAs roof into the categories had great significance.

1.1. Related Works

Traditional point cloud segmentation is the process of dividing the point cloud into multiple homogeneous regions which have the same properties [14]. Based on the different design mechanisms, Nguyen, A. and Le B. [15] divided the point cloud segmentation techniques into five categories: edge based, region based, attributes based, graph based and model based. Xie Y. et al. [16] classified the point cloud segmentation approaches into four major groups: edge based, model fitting, region growing, and unsupervised clustering based. Although there was a minor difference between these two classification criteria, most point cloud segmentation methods mainly relied on the purely mathematical model, especially before the appearance of the supervised learning methods.

In the field of culture heritage, plenty of traditional point cloud segmentation methods have been proposed. Wang Y. and Shi H. [17] proposed a multi-primitive segmentation method for the segmentation of column, planar and sphere based on the local sample. Hough transform (HT) was used to detect planes, cylinders and spheres [18]. To construct the BIM of the Milan Cathedral, Fassi F. et al. [19] extracted the plane and cylinder based on the region growing. Traditional point cloud segmentation methods performed well in man-made structures with regular geometrical shapes and allowed fast running time. However, in the process of segmenting the MQDOAs, some limitations were not ignored. On the one hand, the number of components consisting of MQDOAs was very large, and the shape of some components was irregular. It was difficult to choose the proper geometric model to fit objects. On the other hand, due to the absence of the supervised prior knowledge, the segmented point cloud had no strong semantic information. This semantic information was necessary in the scan-to-BIM process. To overcome these limitations, the researchers paid more attention to supervised learning methods for the point cloud semantic segmentation.

The supervised learning method can build an accurate predictive model for the segmentation of point cloud by learning the features from the given training data. In nature, they were a type of reasoning techniques. Therefore, besides obtaining the semantic label of each point, the supervised learning methods were more robust to noise, uneven density, and occlusions in the point cloud data compared with the traditional point cloud segmentation methods. Nowadays, machine learning (ML) techniques and deep learning (DL) techniques are the popular supervised learning point cloud segmentation methods [20,21,22,23]. From the perspective of AI, DL belongs to ML. The main difference between ML and DL techniques is whether or not the features are handcrafted [16].

Based on individual or contextual features, the ML techniques can be divided into individual point cloud semantic segmentation methods and statistical context methods [24]. The process of individual point cloud semantic segmentation methods mainly contains four stages—neighborhood selection, feature extraction, feature selection and semantic segmentation—as was summarized in Ref. [25]. Many mature methods which can complete specific functions in each stage have been proposed. Neighborhood selection decides the radius of the features calculation. Because the dimensions of different components from architectural heritages are different, with a fixed radius, it is difficult to meet the requirement of calculating the different components’ features. To obtain the best scale, Grilli E. et al. [26] calculated the features at different scales. According to the importance of the selected features based on the random forest (RF), the optimal radii for the different features were determined. To overcome the scale limitation, another strategy was proposed in [27]. In this method, the full-resolution point cloud firstly was subsampled into a low resolution level, and the subsampled point cloud was segmented into the categories at large scale; subsequently, the classified results were transferred to higher resolution data through the nearest neighborhood algorithm; at last, the higher resolution point cloud was further segmented into the new classes. In the stage of feature extraction and selection, plenty of works demonstrated that proper features can promote segmentation accuracy [28,29]. Because the different types of historical architectures have different appearances, it is necessary to design proper segmentation features, aiming at the different historical architectures. Nowadays, the most popular features mainly contain covariance features and other geometric features in the field of architecture heritage. However, whether these features are applicable to the Chinese ancient architectures should be explored. Moreover, considering that the construction of MQDOAs followed certain rules, the features from the construction regulation should be used in the segmentation process. In the stage of semantic segmentation, various classifiers have been proposed [30,31,32]. The experimental results showed that the RF classifier can obtain good performance in the robustness and accuracy of segmentation [33].

The statistical context methods mainly rely on the contextual features of points. Generally, this type of method is used in the post processing of individual point cloud semantic segmentation methods and can smooth the classification results. The conditional random fields are the most widely used context model [12]. Although the segmentation accuracy is increased, the time consumption of the statistical context methods is too large in practice.

Among the DL approaches, PointNet and its later improvement PointNet++ [34,35] were considered as pioneer works. Compared with the regular supervise machine learning, the DL methods do not need to design the features. A good review related on cloud segmentation based on DL can be seen in Ref. [36]. In the field of the architecture heritage, P. R. et al. [37] proposed a DL framework for cultural heritage based on the DGCNN. Y. Ji et al. [38] modified the DGCNN for the segmentation of the MQDOAs roof. The segmentation accuracy of the modified DGCNN performed better than PointNet, DGCNN and LDGCNN, and reached 87.14%.

Although the architectural heritage classification methods based on deep learning performed efficiently, the DL approaches significantly rely on the training datasets. Nowadays, the published datasets, such as ModelNet 40 [39], KITTI [40], Sydney Urban Objects [41], Semantic3D [42], S3DIS [43] and ArCH [37], were mainly collected from urban environments. There are still no published datasets focusing on MQDOAs with an adequate level of detail. This limited the usage of deep learning in the point cloud segmentation of the MQDOAs roof.

1.2. The Motivation and Contribution

In our previous work [29], the roof was extracted from the entire point cloud of MQDOAs. To further support the scan to BIM process, the 3D point cloud of MQDOAs roof was segmented into the sub-class with semantic information in this paper. Due to the absence of a point cloud dataset of the MQDOAs roof with an adequate level of detail, a standard machine learning method based on RF was applied. Considering that the construction of the Ming and the Qing Dynasties’ official-style architecture followed certain rules, which can be searched for in YingzaoFashi (Building Standards) [44] of the Song Dynasty or Gongchengzuofazeli (Structural Regulations) [45] published by Qing, we propose a point cloud segmentation method. The main contributions of this paper are listed as follows:

(1)

In addition to regular geometric features, other features from the construction regulations of MQDOAs roof were extracted and applied in this method.

(2)

A multi-scale feature vector strategy was proposed to lower the influence of the scale.

The research in this paper is organized as follows: (1) Section 2 gives a brief introduction of the study case; (2) in Section 3, the proposed method is given; (3) the experimental results are shown to demonstrate the efficiency of the approach in Section 4; (4) finally, a conclusion is conducted in Section 5.

2. Study Case

2.1. Hall of Complete Harmony

The Hall of Complete Harmony (中和殿) with a single-eave pyramidal roof is located between the Hall of Supreme Harmony (太和殿) and the Hall of Preserved Harmony (保和殿) on the central axis of the Forbidden City (Figure 1a). It is one of the most famous examples of the Ming and Qing Dynasties’ official-style architecture (Figure 1b), and is known as one of Three Great Halls of the Forbidden City. Built in 1420, the Hall of Complete Harmony was square in plane, and covered an area of approximately 580 ${\mathrm{m}}^2$, with 3 rooms wide and 3 rooms deep, respectively. The maximum height from the top of the baoding to the ground was 19 m.

The roof of the Hall of Complete Harmony has four slopes. Each slope is covered with yellow glazed tiles, including round tiles (筒瓦) and flat tiles (板瓦) which are distributed in the form of staggered parallel rows. The drip tiles (滴水), eave tiles (瓦当), lianyan (连檐) and rafters are located at the roof eaves, respectively. The number of vertical ridges which intersect at a copper tire gilded baoding (宝顶) is equal to the number of the slopes. Different types of zoushou (走兽) are distributed at the ends of the ridges. Four taoshou are at the four roof corners. Figure 2 shows the various components constituting the roof.

2.2. 3D Survey

To meet the requirement of the maintenance and other protection, a digital documentation project of the Hall of Complete Harmony was carried out by the Imperial Palace administration in 2010. In the process, the point clouds from 96 scan stations were respectively captured with the help of the terrestrial laser scanning (TLS) system Leica HDS6000. To register the data from various angles, a local coordinate system was defined as a benchmark to unify all the independent coordinates into one coordinate system. Relying on the commercial software package Leica Cyclone 6.0, the registration of point clouds from different scan stations was completed. The process is listed as follows:

(1)

Selected the fitted spherical centers of target balls which were located in the overlapping area between different scan stations as the registration features.

(2)

Calculated the transformation parameters based on the feature points.

(3)

Transformed the coordinates of target points to the reference coordinates system.

After merging scans, the average error of the target balls used as registration features was less than $2 \mathrm{mm}$ and the root mean square (RMS) error of overlap points between different scan stations was about $2.25 \mathrm{cm}$. Although there was some noise around the eave tiles, the overall quality of the point cloud was still very high and could be used for the semantic segmentation directly. The captured point cloud of the Hall of Complete Harmony was shown in Figure 3. The point cloud density of the Hall of Complete Harmony was 43,416 $\mathrm{points}/\mathrm{m}$². To support the process of the Scan-to-BIM, we divided the roof components into nine classes: zoushou, ridge, flat tile, baoding, taoshou, lianyan, rafter, drip tile and round tile.

3. Our Proposed Methods

As is shown in Figure 4, the semantic segmentation process mainly contained two parts:

(1)

A multi-scale classification features vector, which contained the geometric features at the different scales and the features from the construction regulations, was constructed (see Section 3.1).

(2)

Random forest was applied to fit a prediction model by the selected training data with semantic labels and predicted the test data (see Section 3.2).

Figure 4. The semantic segmentation process in this paper.

Figure 4. The semantic segmentation process in this paper.

3.1. Multi-Scale Classification Features Vector Construction

3.1.1. Single-Scale Feature Generation Based on Points

The construction of the Ming and the Qing Dynasties’ official-style architecture followed certain rules which resulted in the distribution of roof components of buildings in the Ming and Qing Dynasties also obeying some arrangement laws. Hence, in addition to the regular geometric features, which were widely used in other works [17,19,31], the features that came from the construction regulation are also designed in this section.

(1)

The regular geometric features

The covariance features were the popular geometric features in the segmentation of point clouds and mainly contained the linearity, planarity, sphericity, normal change rate, etc. They were the shape descriptors obtained as a combination of the eigenvalues extracted from the covariance matrix. Assuming that $C_P$ represents the neighborhood covariance matrix of point $p$, $C_P$ can be calculated by Equation (1)

$$C_P=\frac{1}{\left|N_K\right|}{\displaystyle \sum }_{p_i\subseteq N_k}\left(p_i-\overline{p}\right){\left(p_i-\overline{p}\right)}^T$$

(1)

where $\overline{p}=\frac{1}{\left|N_K\right|}{{\displaystyle \sum }}_{p_i\subseteq N_k}{p}_i$, and $N_k$ represents the number of the points. A set of positive eigenvalues ${\lambda }_i\left({\lambda }_1>{\lambda }_2>{\lambda }_3\right)$ is computed; then, the covariance features can be calculated according to the positive eigenvalues (or normalized eigenvalues).

Besides the covariance features, other geometric features, such as the verticality and roughness, are often applied in the point cloud classification process in the field of the architectural heritage. Plenty of works have demonstrated that proper features can promote the accuracy of the results [20,28]. However, selecting the proper features is a difficult task. On the one hand, the selected geometric features can express the shape of classification objects; on the other hand, these features should have differences.

To select the proper features for the semantic segmentation of the MQDOAs roof, a feature selection process was conducted as follows:

• At first, we analyzed the shape of each type of roof component;
• We tested the various geometric features that appeared in the previous work based on the former analysis results;
• At last, we selected the proper features from the experimental results through the visual inspection.

Through trial and error, linearity, planarity, sphericity, normal change rate, verticality and roughness were extracted as the semantic segmentation features in this paper. The used features are shown in

Table 1. The used regular geometric features.

Features	Feature Description	Formulas
Linearity	It can distinguish the ridge.	${\lambda }_l=\frac{{\lambda }_1 - {\lambda }_2}{{\lambda }_1}$	(2)
Planarity	It can facilitate the identification of lianyan and rafter.	${\lambda }_p=\frac{{\lambda }_2 - {\lambda }_3}{{\lambda }_1}$	(3)
Sphericity	It reflected the shape of baoding, round tile and zoushou.	${\lambda }_s=\frac{\sqrt{{\lambda }_3}}{\sqrt{{\lambda }_1}}$	(4)
Normal change rate	It was used for classifying the ridge, round tile and flat tile.	${\lambda }_c=\frac{{\lambda }_3}{{\lambda }_1 + {\lambda }_2 + {\lambda }_3}$	(5)
Verticality	It was essential to distinguish ridge, rafter and lianyan.	${\lambda }_v=1-n_z$	(6)
Roughness	It highlighted the surface of taoshou.	${\lambda }_r=\frac{\left|Ax + By + Cz + D\right|}{\sqrt{A^2 + B^2 + C^2}}$	(7)

and Figure 5.

(2)

Features from construction regulations

Based on the construction regulations, the components were arranged symmetrically along the symmetry axis of buildings (Figure 6a) and the topological relationship of the different types of components was relatively fixed (Figure 6b). Although the coordinates can describe the arrangement law of the roof components, the points of different component categories were staggered in the 3D space. Only relying on the coordinates $\left(\mathrm{X}, \mathrm{Y}, \mathrm{Z}\right)$ was difficult to meet the classification requirement. For example, zoushou must be on the ridges. However, the points belonged to ridge may be higher than the points of zoushou in some areas.

Considering that the shape of each section line along (or perpendicular to) the main direction of the buildings was similar (Figure 6c) and the arrangements of the roof components along the section line were also the same (Figure 6d), the relative locations, including the normalized symmetrical distance (NSD), which is the normalized distance from a point to the symmetry axis of the MQDOAs and the relative height (RH), were extracted from the section lines as the classification features.

As shown in Figure 6c, $L_i$ is a section line along the main direction of the building and $p_{in}\left(\begin{array}{ccc}{x}_{in}& y_{in}& z_{in}\end{array}\right)$ is a point that belongs to $L_i$. The feature $NS{D}_{in}$ of this point is calculated as follows:

$$NS{D}_{in}=\frac{2d_{in}}{l_i}$$

(8)

where $l_i$ is the length of $L_i$ on the $\mathrm{XOY}$ plane, $d_{in}$ is the distance from a point to the symmetry axis of the MQDOAs. Obviously, the NSD is the same, and the point class is the same. Moreover, the NSD of roof components located at the outermost contour of the roof is nearly 1 for each section line. On the $\mathrm{XOY}$ plane, the staggered phenomenon of points is lower.

Supposed that $p_{il}\left(\begin{array}{ccc}{x}_{il}& y_{il}& z_{il}\end{array}\right)$ and $p_{ih}\left(\begin{array}{ccc}{x}_{ih}& y_{ih}& z_{ih}\end{array}\right)$ are the lowest and highest points on $L_i$, separately; and $p_{im}\left(\begin{array}{ccc}{x}_{im}& y_{im}& z_{im}\end{array}\right)$ represents the lowest point located in the middle area of this section line. The feature $R{H}_{in}$ of the point $p_{in}\left(\begin{array}{ccc}{x}_{in}& y_{in}& z_{in}\end{array}\right)$ is described in (7).

$$R{H}_{in}=\{\begin{array}{cc}{z}_{in}-z_{im}& if z_{in}\ge z_{im}\\ \left(z_{in}-z_{il}\right)/\left(z_{maxm}-z_{il}\right)& if z_{in}<z_{im}\end{array}$$

(9)

In this equation, $z_{maxm}$ represents the highest point of the whole roof. Based on this equation, the staggered phenomenon of points from different roof components can be efficiently lowered at the Z axis. In particular, we have the following:

(1)

Along the Z direction, the main difference between $L_i$ and $L_j$ is the height caused by the $\mathrm{ab}$ section (Figure 6c). On the section line $L_i$, if $h_{ab}$ is added to the height from $p_{il}$ to $p_{im}$, the normalized height of points which are lower than $p_{im}$ will become almost the same as that of the points which are lower than $p_{jm}$ on $L_j$. Hence, when a point is lower than $p_{im}$, we can normalize to the highest section line.

(2)

When the point $p_{in}$ is higher than $p_{im}$, this point should belong to the ridge or zoushou. This is applicable to the other section line.

The arrangement of the flat tiles and round tiles also show a strong regularity in a local area along the section line. As shown in Figure 7, $\mathrm{P}1$ and $\mathrm{P}2$ are the points on the flat tile and the round tile, separately. Obviously, the points that belong to the flat tile are at the bottom of the local area between the two round tiles, and the height of these points is almost the same. Hence, the local height difference (LHD) between a point and the lowest point within a local area of the current point along the section line was applied in this paper. The size of the local area can be defined as being double the width of the round tile. The LHD of the $n_{th}$ point on the $i_{th}$ section line is expressed as follows:

$$LH{D}_{in}=Z_{in}-Z_{inl}$$

(10)

where $Z_{inl}$ is the height of the lowest point within a local area of the current point on the section line. Figure 8 shows the extracted features’ results along the section line. Notably, the section line perpendicular to the main direction of the building is extracted and the features are also calculated in this method.

3.1.2. Multi-Scale Feature Vector Construction

To lower the influence of scale, we applied a multi-scale feature vector for the semantic segmentation of the MQDOAs roof. The process of constructing the multi-scale feature vector is listed as follows:

(1)

We took different neighborhood size $\mathrm{s}$ to calculate the geometric feature vector of the current segmented point.

(2)

We connected geometric feature vectors at the selected scales and the features from the construction regulation into a long classification feature vector to form the multi-scale classification feature vector of the current segmented point.

The computational time grows with the increase in the search radii. Hence, it was impossible to calculate the geometric features at all scales. To determine the selected scales set $\mathrm{S}$, a simple strategy is given here. This strategy is described as follows:

(1)

Set the initial scale $\mathrm{s}0$.

(2)

Calculate the single-scale geometric feature vector at the current scale.

(3)

Input the calculated single-scale geometric feature vector into the classifier and compare the classification accuracy of the target data. If the classification accuracy at the current scale is higher than that at the previous scale, preserve the current scale into $\mathrm{S}$ and proceed to step (4); otherwise, output the selected scales set $\mathrm{S}$.

(4)

Increase the current scale according to the specified interval and turn to step 2.

Notably, from the perspective of machine learning, the validation data were used for scales selection. However, the point cloud of other Ming and Qing Dynasties’ official-style architectures with a detailed roof was hard to acquire, thus the data to be segmented for testing were used in this paper.

3.2. Data Training and Classifying Based on Random Forest

3.2.1. Random Forest

Random forest (RF) was first proposed by Leo Breiman and Adele Cutler [45] and it has been widely used in point cloud classification field of the culture heritage [26]. This classifier makes use of the multiple trees to train and predict samples. In the training stage, it inputs both the class labels and the extracted features to fit a prediction model and predict the other points without class labels.

As was described in [28], the RF has three advantages:

(1)

RF is considered a highly accurate and robust method because of the number of decision trees participating in the process.

(2)

RF can avoid the over-fitting problem, as it takes the average of all the predictions, which cancel out the biases.

(3)

RF offers a useful feature selection indicator (the relative importance or contribution of each feature in the prediction), which can help us estimate the optimal feature vector.

In this paper, we selected the Scikit Learn Python library (version 0.24.1) [46] to carry out the classification program. Considering several parameters (such as n_estimators, oob_score and max_depth, etc.) which were involved in this module, the default parameters were applied so that the generalization can be further increased.

3.2.2. Feature Selection

In addition to the geometric features (including linearity, planarity, sphericity, verticality, normal change rate and roughness), the features NSD, RH and LHD that were selected, X, Y and Z were also employed as the segmentation features in this paper. As described in Section 3.1, we constructed the multi-scale classification features vector.

3.2.3. Supervised Classification

After the feature selection, the training samples with the selected feature set were utilized to train the RF classifier, and the classifier was used to predict the labels of unlabeled points.

4. Experiments

4.1. Test Data and Evaluation Criteria

To evaluate the performance of the proposed method, the 3D survey data of the Hall of Complete Harmony was applied. Before the experiment, the roof was extracted from the 3D survey data and each point was labeled into the corresponding categories proposed in Section 2.1 by manual processing. The annotated class is shown in Figure 9.

We divided the roof into two parts: one part was regarded as the training data and the other part was used for the test data. The training set should contain all types of roof components. Considering the symmetric structure of the roof, the train data were mainly selected along the main direction of the building as is shown in Figure 10. Because taoshou is only located at the corners of the roof, some points located at the corners must be selected; otherwise, the taoshou will be incorrectly segmented. The point cloud statistics of the training set and test set are shown in

Table 2. Experimental data.

	Zoushou	Ridge	Flat Tile	Baoding	Taoshou	Lianyan	Rafter	Drip Tile	Round Tile
Train data	25,812	253,541	453,616	80,591	19,414	20,286	19,005	24,514	1,105,952
Test data	48,243	1,561,284	2,731,818	93,813	23,208	164,130	173,129	179,836	6,759,117

.

For evaluation, we employed four commonly used measures: overall accuracy (OA), precision, recall and $\mathrm{F}1\_\mathrm{score}$. They are computed as follows:

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\left(\mathrm{TP}+\mathrm{FP}\right)}$$

(11)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\left(\mathrm{TP}+\mathrm{FN}\right)}$$

(12)

$$\mathrm{F}1\_\mathrm{score}=2\times \frac{\mathrm{Recall}\times \mathrm{Precision}}{\mathrm{Precision}+\mathrm{Recall}}$$

(13)

$$\mathrm{OA}=\frac{\mathrm{TP}+\mathrm{FN}}{\mathrm{TP}+\mathrm{FP}+\mathrm{TN}+\mathrm{FN}}$$

(14)

where TP = true positive (sum of the values in the diagonal position), FP = false positive (sum of the values in the column without the main diagonal one), TN = true negative, and FN = false negative (sum of the values in the row without the main diagonal one).

4.2. Experimental Results and Analysis

The MQDOAs roof is composed of all types of tiles. Among them, the thickness of the tiles is about $2.5 \mathrm{cm}$. So, we took $2 \mathrm{cm}$ as the initial scale. Then, we increased the scale from $5 \mathrm{cm}$ to $0.2 \mathrm{m}$ gradually. The classification accuracy was calculated under different scales. When the scale was $0.2 \mathrm{m}$, the classification accuracy was lower than that under the scale of $0.1 \mathrm{m}$. Hence, in the experiment, the geometric features were calculated within $\mathrm{s}1=0.02 \mathrm{m}$, $\mathrm{s}2=0.05 \mathrm{m}, \mathrm{s}3=0.1 \mathrm{m}$ and $\mathrm{s}4=0.2 \mathrm{m}$, respectively. To verify the advantage of the proposed method, we connected the geometric feature vectors based on the different scale combinations (including $\mathrm{s}0$, $\mathrm{s}0+\mathrm{s}1$, $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2$, $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2+\mathrm{s}3$) and the proposed features to form a long dimension feature vector as the input features for the point cloud classification of the MQDOAs roof.

Moreover, two optimal feature vectors were also selected according to the importance rank of features based on the RF. The first optimal feature vector came from a pure multi-scale geometric feature vector without the proposed features based on the scale combination of $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2+\mathrm{s}3$ and a multi-scale feature vector, including the proposed features based on the scale combination of $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2+\mathrm{s}3$ in our experiment. The first optimal feature vector included Z, roughness (0.2), normal change rate (0.2), verticality (0.2), normal change rate (0.1), verticality (0.1), normal change rate (0.05), planarity (0.05) and sphericity (0.05); the second optimal feature vector contained Z, roughness (0.2), sphericity (0.2), verticality (0.2), verticality (0.1), normal change rate (0.05), planarity (0.05), LHD1 and LHD2 (which were extracted from the section line along and perpendicular to the main direction of building).

A comparative test was carried out on the combination of different scales. The classification results are shown in

Table 3. The experimental results based on different scales only relying on the regular geometric features.

Scales Combination	Metrics	Zoushou	Ridge	Flat Tile	Baoding	Taoshou	Lianyan	Rafter	Drip Tile	Round Tile	Average Value
s0	precision	0.55	0.75	0.58	0.86	0.1	0.75	0.78	0.69	0.83	0.654
	recall	0.47	0.39	0.75	0.97	0.05	0.74	0.69	0.62	0.82	0.611
	F1	0.5	0.52	0.66	0.91	0.07	0.74	0.73	0.65	0.82	0.622
	OA									0.741
s1	precision	0.68	0.85	0.81	0.92	0.11	0.81	0.77	0.75	0.89	0.732
	recall	0.6	0.57	0.93	0.93	0.05	0.81	0.74	0.71	0.91	0.694
	F1	0.64	0.68	0.86	0.93	0.07	0.81	0.75	0.73	0.9	0.708
	OA									0.858
s2	precision	0.77	0.94	0.78	0.93	0.26	0.76	0.81	0.78	0.9	0.770
	recall	0.88	0.83	0.85	0.94	0.14	0.74	0.72	0.77	0.9	0.752
	F1	0.82	0.88	0.81	0.93	0.18	0.75	0.76	0.78	0.9	0.757
	OA									0.871
s3	precision	0.77	0.95	0.7	0.98	0.35	0.85	0.86	0.83	0.89	0.798
	recall	0.83	0.9	0.75	0.95	0.19	0.82	0.78	0.78	0.88	0.764
	F1	0.8	0.92	0.73	0.96	0.25	0.83	0.82	0.8	0.88	0.777
	OA									0.846
s0 + s1	precision	0.58	0.86	0.83	0.93	0.2	0.82	0.79	0.71	0.92	0.738
	recall	0.55	0.73	0.93	0.92	0.11	0.79	0.66	0.75	0.92	0.707
	F1	0.56	0.79	0.88	0.92	0.14	0.81	0.72	0.73	0.92	0.719
	OA									0.884
s0 + s1 + s2	precision	0.77	0.96	0.87	0.96	0.45	0.85	0.84	0.81	0.95	0.829
	recall	0.89	0.89	0.95	0.91	0.26	0.8	0.7	0.85	0.94	0.799
	F1	0.82	0.92	0.9	0.93	0.33	0.82	0.76	0.83	0.95	0.807
	OA									0.925
s0 + s1 + s2 + s3	precision	0.87	0.97	0.88	0.98	0.55	0.92	0.93	0.87	0.97	0.882
	recall	0.94	0.96	0.95	0.94	0.3	0.89	0.82	0.9	0.95	0.850
	F1	0.9	0.96	0.91	0.96	0.39	0.9	0.87	0.89	0.96	0.860
	OA									0.944
Optimal feature vector	precision	0.75	0.97	0.85	0.97	0.4	0.88	0.88	0.84	0.96	0.833
	recall	0.95	0.95	0.92	0.96	0.66	0.87	0.81	0.86	0.93	0.879
	F1	0.83	0.96	0.89	0.97	0.5	0.87	0.85	0.85	0.95	0.852
	OA									0.929

and

Table 4. The experimental results based on different scales and features.

Scales Combination	Metrics	Zoushou	Ridge	Flat Tile	Baoding	Taoshou	Lianyan	Rafter	Drip Tile	Round Tile	Average Value
s0	precision	0.45	0.94	0.91	0.98	0.65	0.89	0.95	0.81	0.96	0.838
	recall	0.91	0.87	0.95	0.98	0.53	0.92	0.88	0.81	0.95	0.867
	F1	0.6	0.9	0.93	0.98	0.59	0.91	0.91	0.81	0.95	0.842
	OA									0.935
s1	precision	0.47	0.96	0.92	0.98	0.75	0.93	0.93	0.86	0.97	0.863
	recall	0.93	0.9	0.97	0.98	0.51	0.93	0.92	0.86	0.96	0.884
	F1	0.62	0.93	0.94	0.98	0.61	0.93	0.92	0.86	0.96	0.861
	OA									0.949
s2	precision	0.72	0.98	0.93	0.98	0.85	0.92	0.95	0.88	0.97	0.909
	recall	0.93	0.95	0.96	0.98	0.64	0.91	0.92	0.89	0.97	0.906
	F1	0.81	0.96	0.95	0.98	0.73	0.91	0.93	0.88	0.97	0.902
	OA									0.96
s3	precision	0.63	0.97	0.91	0.98	0.79	0.92	0.95	0.88	0.97	0.889
	recall	0.94	0.96	0.95	0.98	0.59	0.93	0.91	0.89	0.96	0.901
	F1	0.875	0.97	0.93	0.98	0.68	0.93	0.93	0.89	0.97	0.892
	OA									0.953
s0 + s1	precision	0.75	0.98	0.94	0.99	0.82	0.94	0.95	0.91	0.98	0.918
	recall	0.96	0.98	0.96	0.99	0.63	0.91	0.92	0.91	0.97	0.914
	F1	0.84	0.98	0.95	0.99	0.71	0.93	0.94	0.91	0.98	0.914
	OA									0.966
s0 + s1 + s2	precision	0.74	0.98	0.93	0.98	0.82	0.96	0.96	0.93	0.98	0.920
	recall	0.97	0.98	0.97	0.99	0.75	0.94	0.93	0.92	0.97	0.936
	F1	0.84	0.98	0.95	0.98	0.78	0.95	0.94	0.92	0.98	0.924
	OA									0.968
s0 + s1 + s2 + s3	precision	0.79	0.98	0.93	0.98	0.81	0.95	0.96	0.92	0.98	0.922
	recall	0.97	0.98	0.97	0.98	0.71	0.94	0.92	0.93	0.97	0.930
	F1	0.87	0.98	0.95	0.98	0.75	0.95	0.94	0.92	0.97	0.923
	OA									0.966
Optimal feature vector	precision	0.55	0.97	0.93	0.98	0.49	0.92	0.86	0.85	0.98	0.837
	recall	0.96	0.96	0.96	0.96	0.82	0.89	0.84	0.85	0.97	0.912
	F1	0.7	0.97	0.95	0.97	0.61	0.9	0.85	0.85	0.97	0.863
	OA									0.96

.

The segmentation result of the Hall of Complete Harmony’s roof is shown in Figure 11. Table 4 shows that the highest OA reached 96.8% when the proposed features were added at the s0 + s1 + s2 scales. Besides the OA, a rigid evaluation index, the average value of F1_score, was also applied to lower the impact of the quantitative variation of points within different categories in performance evaluation. As is shown in Figure 12, the lowest average value of F1_score was 62.2% only relying on the regular geometric features at the s0 scale, and the highest average value of F1_score (92.4%) still occurred when the proposed features were added at the s0 + s1 + s2 scales. Obviously, when the proposed features were added, the average F1_score was higher than that only relying on the regular geometric features at the same scale or the scale combination. On the other hand, the multi-scale feature connection strategy also improved the average F1_score.

The experimental results showed that the segmentation accuracy obtained after adding our proposed features and applying the multi-scale long feature vectors was significantly better than that of relying on the covariance features and single scale. Both the proposed features and multi-scale feature connection strategy can improve the point cloud classification accuracy.

4.3. Impact of Proposed Features

As was shown in Figure 13, at the same scale or scale combinations, the classification accuracies increased when the proposed features were input into the RF classifier. The max difference of classification accuracy between the regular geometric feature vector and the combination of regular geometric and proposed features accrued at $\mathrm{s}0$ scale and reached 19.4%. The minimum difference of classification accuracy accrued at s0 + s1 + s2 + s3 scale combination and reached 2.2%. In the case of using single-scale features, the classification accuracy only relying on the geometric features ranged from 74.1% to 87.1%, while the fluctuation of classification accuracy was less than 3% when the proposed features were added. This showed that the features from the construction regulations not only increased the classification accuracy, but also could lower the influence of the scales on the classification accuracy.

In addition to the overall accuracy, the proposed features also obtained good performance on the classification accuracy of each class. As shown in Figure 14, except the class zoushou, the $\mathrm{F}1\_\mathrm{score}$ of the other classes was higher than that without the proposed features. Although the $\mathrm{F}1\_\mathrm{score}$ of zoushou was lower, the highest difference was less than 5% at the same scale. This showed that the proposed features had good performance on the classification of the roof point cloud.

4.4. Influence of Different Scale Combinations

As shown in Figure 14, the highest classification accuracy based on the single-scale geometric feature vector was 87.1% and the lowest classification accuracy based on the multi-scales geometric features was 88.4%. Similarly, when the proposed features were added, the lowest classification accuracy of multi-scale long feature vectors was also higher than the highest classification accuracy of single-scale features vectors. Hence, compared with the single-scale feature vectors, the multi-scale long feature vectors (such as $\mathrm{s}0+\mathrm{s}1$, $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2$, $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2+\mathrm{s}3$) performed better when the same features were applied.

The classification accuracies based on two sets of optimal feature vector were 92.9% and 96%. They were still lower than 94.4% and 96.6%. In nature, the optimal features are generated from the combination of multi-scale features (s0 + s1 + s2 + s3). Some unimportant features may be ignored. However, these unimportant features play an important role in the class which contains few points [47]. This optimal-features strategy may reduce the segmentation accuracy of these classes. As shown in Figure 15, the difference of $\mathrm{F}1\_\mathrm{score}$ of taoshou between the optimal features the segmentation accuracy increased. Although the optimal-features strategy can reduce the time consumption, the multi-scale was essential for the fine point cloud segmentation of the MQDOAs roof.

For the combinations of scales, the strategy of connecting single-scale feature vectors into multi-scale feature vectors was not that the more numerous the scales, the more beneficial it was to improve the classification accuracy. As can be seen from Figure 13, the combination of three scales ($\mathrm{s}0+\mathrm{s}1+\mathrm{s}2$) had the highest segmentation accuracy in the comparison test rather than the combination of four scales $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2+\mathrm{s}3$.

The best experimental result of the multi-scale feature vector including the proposed features occurred at the $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2$ scales, and the scales combination was consistent with our proposed scale combination strategy. In fact, the classification accuracy increased from the scale $\mathrm{s}0$ to $\mathrm{s}3$. When the scale was equal to $0.2 \mathrm{m}$, the segmentation accuracy declined. The combination of scales $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2$ can be regarded as the optimal scale combination. However, the experimental result of multi-scale feature vector without the proposed features showed a different conclusion. Although the segmentation accuracy also declined only relying on the pure geometric features when the scale was equal to $0.2 \mathrm{m}$, the optimal scales combination was $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2+\mathrm{s}3$, rather than $\mathrm{s}0+\mathrm{s}1+\mathrm{s}2$. These experimental results showed the proposed scale selection strategy can be close to the optimal result and may not achieve the optimal result in some cases.

5. Conclusions

Considering the complexity of MQDOAs roof and the scale differences between different component types, the proposed method applied a multi-scale features vector for the point cloud segmentation of MQDOAs roof. The highlights of this work are listed as follows:

(1)

The features, including RH, NSD and LHD, were selected for the point cloud sematic segmentation of the Ming and Qing Dynasties’ official-style architecture roof, and the corresponding feature extraction methods were proposed.

(2)

For the fine segmentation of the MQDOAs roof, the multi-scale feature vector was essential, and the scale connection strategy was given in this paper.

(3)

The experimental results showed that our proposed method can achieve good performance and has robustness after the proposed features are added and the multi-scale strategy is applied.

Although our proposed method was not tested on the additional point cloud data of other MQDOAs roofs, due to the absence of data, the algorithm can achieve the presented results for other MQDOAs roofs, theoretically. This is mainly because the components which form the roofs are almost the same, and the distribution of components is also similar.

Additionally, it is worth noting that the classification of the Ming and Qing Dynasties’ official-style architecture was not the primary goal of our work. We hoped to reconstruct a historical building information model (HBIM) of the Ming and Qing Dynasties’ official-style architecture, automatically. In previous work [48], the method reconstructing the decorative components based on a template library was given. The future work will include (1) partial occlusion component information extraction (such as flat tiles); (2) the parameterization of components; (3) the HBIM reconstruction mechanism; and (4) using various strategies to ensure that the point cloud can be processed in memory.