The Point Cloud Semantic Segmentation Method for the Ming and Qing Dynasties’ Ofﬁcial-Style Architecture Roof Considering the Construction Regulations

: Point cloud semantic segmentation has played an important role in the scan-to-BIM process of the Ming and Qing Dynasties’ ofﬁcial-style architecture roof. To overcome the complexity of roof components’ shape and the scale differences between different roof component types, a point cloud semantic segmentation method for the MQDOAs roof considering the construction regulations is proposed in this paper. This method is composed of two stages. In the ﬁrst stage, the features from the construction rules of MQDOAs, including the normalized symmetrical distance (NSD), relative height (RH) and local height difference (LHD), are extracted alongside the regular geometric features. To lower the inﬂuence of scale differences, a multi-scale feature connection strategy is also applied to construct the feature classiﬁcation vector. In the second stage, RF method is applied to classify the point cloud. To verify the efﬁciency of the proposed method, we took the Hall of Complete Harmony as the study case. The experiments showed that our method achieved segmentation result in overall classiﬁcation accuracy and reached 96.8%.


Introduction
The Ming and Qing Dynasties' official-style architectures (MQDOAs) are considered the last peak of Chinese architectural history and were the important components of the ancient Chinese architectural system [1,2]. Suffering from weathering, fires and rotting, a mass of MQDOAs with wooden structural frame has disappeared. To preserve historical buildings, using the 3D point cloud, which provides precise geometric coordinates (X, Y, Z) in the form of millions of points, to record the shape of the cultural heritage has become one of the most efficient methods [3][4][5]. The captured original point cloud lacked structured information, such as semantics and hierarchy between parts, which disturbed the usage of point cloud in other application fields [6][7][8]. Hence, how to segment the point cloud into the sub dataset with semantic meaning has become a research hotspot.
Traditionally, the segmentation of the 3D point cloud is manually processed by using a digital workstation in a practice project. Apart from being time consuming, manual intervention for subdividing the datasets brings a certain degree of subjectivity [9]. Developing automatic or semi-automatic procedures for point cloud segmentation has become the consensus. Nowadays, automated point cloud segmentation methods have been proposed by researchers and achieve good performance [10][11][12].
However, it was still a challenge to segment the MQDOAs point cloud into the correct categories. On the one hand, the types of components composed of the MQDOAs were large, and the shapes of the components varied greatly. Only relying on the geometric features which are used in other architectural heritages made it difficult to meet the segmentation requirements [13]. On the other hand, the scale differences between different types of components also damaged the segmentation accuracy. The MQDOAs roof acted selection, plenty of works demonstrated that proper features can promote segmentation accuracy [28,29]. Because the different types of historical architectures have different appearances, it is necessary to design proper segmentation features, aiming at the different historical architectures. Nowadays, the most popular features mainly contain covariance features and other geometric features in the field of architecture heritage. However, whether these features are applicable to the Chinese ancient architectures should be explored. Moreover, considering that the construction of MQDOAs followed certain rules, the features from the construction regulation should be used in the segmentation process. In the stage of semantic segmentation, various classifiers have been proposed [30][31][32]. The experimental results showed that the RF classifier can obtain good performance in the robustness and accuracy of segmentation [33].
The statistical context methods mainly rely on the contextual features of points. Generally, this type of method is used in the post processing of individual point cloud semantic segmentation methods and can smooth the classification results. The conditional random fields are the most widely used context model [12]. Although the segmentation accuracy is increased, the time consumption of the statistical context methods is too large in practice.
Among the DL approaches, PointNet and its later improvement PointNet++ [34,35] were considered as pioneer works. Compared with the regular supervise machine learning, the DL methods do not need to design the features. A good review related on cloud segmentation based on DL can be seen in Ref. [36]. In the field of the architecture heritage, P. R. et al. [37] proposed a DL framework for cultural heritage based on the DGCNN. Y. Ji et al. [38] modified the DGCNN for the segmentation of the MQDOAs roof. The segmentation accuracy of the modified DGCNN performed better than PointNet, DGCNN and LDGCNN, and reached 87.14%.
Although the architectural heritage classification methods based on deep learning performed efficiently, the DL approaches significantly rely on the training datasets. Nowadays, the published datasets, such as ModelNet 40 [39], KITTI [40], Sydney Urban Objects [41], Semantic3D [42], S3DIS [43] and ArCH [37], were mainly collected from urban environments. There are still no published datasets focusing on MQDOAs with an adequate level of detail. This limited the usage of deep learning in the point cloud segmentation of the MQDOAs roof.

The Motivation and Contribution
In our previous work [29], the roof was extracted from the entire point cloud of MQDOAs. To further support the scan to BIM process, the 3D point cloud of MQDOAs roof was segmented into the sub-class with semantic information in this paper. Due to the absence of a point cloud dataset of the MQDOAs roof with an adequate level of detail, a standard machine learning method based on RF was applied. Considering that the construction of the Ming and the Qing Dynasties' official-style architecture followed certain rules, which can be searched for in YingzaoFashi (Building Standards) [44] of the Song Dynasty or Gongchengzuofazeli (Structural Regulations) [45] published by Qing, we propose a point cloud segmentation method. The main contributions of this paper are listed as follows: (1) In addition to regular geometric features, other features from the construction regulations of MQDOAs roof were extracted and applied in this method. (2) A multi-scale feature vector strategy was proposed to lower the influence of the scale.
The research in this paper is organized as follows: (1) Section 2 gives a brief introduction of the study case; (2) in Section 3, the proposed method is given; (3) the experimental results are shown to demonstrate the efficiency of the approach in Section 4; (4) finally, a conclusion is conducted in Section 5.

Hall of Complete Harmony
The Hall of Complete Harmony (中和殿) with a single-eave pyramidal roof is located between the Hall of Supreme Harmony (太和殿) and the Hall of Preserved Harmony (保和殿) on the central axis of the Forbidden City (Figure 1a). It is one of the most famous examples of the Ming and Qing Dynasties' official-style architecture (Figure 1b), and is known as one of Three Great Halls of the Forbidden City. Built in 1420, the Hall of Complete Harmony was square in plane, and covered an area of approximately 580 m 2 , with 3 rooms wide and 3 rooms deep, respectively. The maximum height from the top of the baoding to the ground was 19 m. results are shown to demonstrate the efficiency of the approach in Section 4; (4) finally, a conclusion is conducted in Section 5.

Hall of Complete Harmony
The Hall of Complete Harmony (中和殿) with a single-eave pyramidal roof is located between the Hall of Supreme Harmony (太和殿) and the Hall of Preserved Harmony (保 和殿) on the central axis of the Forbidden City (Figure 1a). It is one of the most famous examples of the Ming and Qing Dynasties' official-style architecture (Figure 1b), and is known as one of Three Great Halls of the Forbidden City. Built in 1420, the Hall of Complete Harmony was square in plane, and covered an area of approximately 580 m , with 3 rooms wide and 3 rooms deep, respectively. The maximum height from the top of the baoding to the ground was 19 m. The roof of the Hall of Complete Harmony has four slopes. Each slope is covered with yellow glazed tiles, including round tiles (筒瓦) and flat tiles (板瓦) which are distributed in the form of staggered parallel rows. The drip tiles (滴水), eave tiles (瓦当), lianyan (连檐) and rafters are located at the roof eaves, respectively. The number of vertical ridges which intersect at a copper tire gilded baoding (宝顶) is equal to the number of the slopes. Different types of zoushou (走兽) are distributed at the ends of the ridges. Four taoshou are at the four roof corners. Figure 2 shows the various components constituting the roof. The roof of the Hall of Complete Harmony has four slopes. Each slope is covered with yellow glazed tiles, including round tiles (筒瓦) and flat tiles (板瓦) which are distributed in the form of staggered parallel rows. The drip tiles (滴水), eave tiles (瓦当), lianyan (连檐) and rafters are located at the roof eaves, respectively. The number of vertical ridges which intersect at a copper tire gilded baoding (宝顶) is equal to the number of the slopes. Different types of zoushou (走兽) are distributed at the ends of the ridges. Four taoshou are at the four roof corners. Figure 2 shows the various components constituting the roof. results are shown to demonstrate the efficiency of the approach in Section 4; (4) finally, a conclusion is conducted in Section 5.

Hall of Complete Harmony
The Hall of Complete Harmony (中和殿) with a single-eave pyramidal roof is located between the Hall of Supreme Harmony (太和殿) and the Hall of Preserved Harmony (保 和殿) on the central axis of the Forbidden City (Figure 1a). It is one of the most famous examples of the Ming and Qing Dynasties' official-style architecture (Figure 1b), and is known as one of Three Great Halls of the Forbidden City. Built in 1420, the Hall of Complete Harmony was square in plane, and covered an area of approximately 580 m , with 3 rooms wide and 3 rooms deep, respectively. The maximum height from the top of the baoding to the ground was 19 m. The roof of the Hall of Complete Harmony has four slopes. Each slope is covered with yellow glazed tiles, including round tiles (筒瓦) and flat tiles (板瓦) which are distributed in the form of staggered parallel rows. The drip tiles (滴水), eave tiles (瓦当), lianyan (连檐) and rafters are located at the roof eaves, respectively. The number of vertical ridges which intersect at a copper tire gilded baoding (宝顶) is equal to the number of the slopes. Different types of zoushou (走兽) are distributed at the ends of the ridges. Four taoshou are at the four roof corners. Figure 2 shows the various components constituting the roof.

3D Survey
To meet the requirement of the maintenance and other protection, a digital documentation project of the Hall of Complete Harmony was carried out by the Imperial Palace ISPRS Int. J. Geo-Inf. 2022, 11, 214 5 of 20 administration in 2010. In the process, the point clouds from 96 scan stations were respectively captured with the help of the terrestrial laser scanning (TLS) system Leica HDS6000. To register the data from various angles, a local coordinate system was defined as a benchmark to unify all the independent coordinates into one coordinate system. Relying on the commercial software package Leica Cyclone 6.0, the registration of point clouds from different scan stations was completed. The process is listed as follows: (1) Selected the fitted spherical centers of target balls which were located in the overlapping area between different scan stations as the registration features. (2) Calculated the transformation parameters based on the feature points.
(3) Transformed the coordinates of target points to the reference coordinates system.
After merging scans, the average error of the target balls used as registration features was less than 2 mm and the root mean square (RMS) error of overlap points between different scan stations was about 2.25 cm. Although there was some noise around the eave tiles, the overall quality of the point cloud was still very high and could be used for the semantic segmentation directly. The captured point cloud of the Hall of Complete Harmony was shown in Figure 3. The point cloud density of the Hall of Complete Harmony was 43,416 points/m 2 . To support the process of the Scan-to-BIM, we divided the roof components into nine classes: zoushou, ridge, flat tile, baoding, taoshou, lianyan, rafter, drip tile and round tile.

3D Survey
To meet the requirement of the maintenance and other protection, a digital documentation project of the Hall of Complete Harmony was carried out by the Imperial Palace administration in 2010. In the process, the point clouds from 96 scan stations were respectively captured with the help of the terrestrial laser scanning (TLS) system Leica HDS6000. To register the data from various angles, a local coordinate system was defined as a benchmark to unify all the independent coordinates into one coordinate system. Relying on the commercial software package Leica Cyclone 6.0, the registration of point clouds from different scan stations was completed. The process is listed as follows: (1) Selected the fitted spherical centers of target balls which were located in the overlapping area between different scan stations as the registration features. (2) Calculated the transformation parameters based on the feature points.
(3) Transformed the coordinates of target points to the reference coordinates system.
After merging scans, the average error of the target balls used as registration features was less than 2 mm and the root mean square (RMS) error of overlap points between different scan stations was about 2.25 cm. Although there was some noise around the eave tiles, the overall quality of the point cloud was still very high and could be used for the semantic segmentation directly. The captured point cloud of the Hall of Complete Harmony was shown in Figure 3. The point cloud density of the Hall of Complete Harmony was 43,416 points/m 2 . To support the process of the Scan-to-BIM, we divided the roof components into nine classes: zoushou, ridge, flat tile, baoding, taoshou, lianyan, rafter, drip tile and round tile.

Our Proposed Methods
As is shown in Figure 4, the semantic segmentation process mainly contained two parts:

Our Proposed Methods
As is shown in Figure 4, the semantic segmentation process mainly contained two parts: (1) A multi-scale classification features vector, which contained the geometric features at the different scales and the features from the construction regulations, was constructed (see Section 3.1). (2) Random forest was applied to fit a prediction model by the selected training data with semantic labels and predicted the test data (see Section 3.2).
(1) A multi-scale classification features vector, which contained the geometric features at the different scales and the features from the construction regulations, was constructed (see Section 3.1). (2) Random forest was applied to fit a prediction model by the selected training data with semantic labels and predicted the test data (see Section 3.2).

Figure 4.
The semantic segmentation process in this paper.

Single-Scale Feature Generation Based on Points
The construction of the Ming and the Qing Dynasties' official-style architecture followed certain rules which resulted in the distribution of roof components of buildings in the Ming and Qing Dynasties also obeying some arrangement laws. Hence, in addition to the regular geometric features, which were widely used in other works [17,19,31], the features that came from the construction regulation are also designed in this section.
(1) The regular geometric features The covariance features were the popular geometric features in the segmentation of point clouds and mainly contained the linearity, planarity, sphericity, normal change rate, etc. They were the shape descriptors obtained as a combination of the eigenvalues extracted from the covariance matrix. Assuming that represents the neighborhood covariance matrix of point , can be calculated by Equation (1) where ̅ = | | ∑ ⊆ , and represents the number of the points. A set of positive eigenvalues ( > > ) is computed; then, the covariance features can be calculated according to the positive eigenvalues (or normalized eigenvalues).
Besides the covariance features, other geometric features, such as the verticality and roughness, are often applied in the point cloud classification process in the field of the

Single-Scale Feature Generation Based on Points
The construction of the Ming and the Qing Dynasties' official-style architecture followed certain rules which resulted in the distribution of roof components of buildings in the Ming and Qing Dynasties also obeying some arrangement laws. Hence, in addition to the regular geometric features, which were widely used in other works [17,19,31], the features that came from the construction regulation are also designed in this section.
(1) The regular geometric features The covariance features were the popular geometric features in the segmentation of point clouds and mainly contained the linearity, planarity, sphericity, normal change rate, etc. They were the shape descriptors obtained as a combination of the eigenvalues extracted from the covariance matrix. Assuming that C P represents the neighborhood covariance matrix of point p, C P can be calculated by Equation (1) where p = 1 |N K | ∑ p i ⊆N k p i , and N k represents the number of the points. A set of positive eigenvalues λ i (λ 1 > λ 2 > λ 3 ) is computed; then, the covariance features can be calculated according to the positive eigenvalues (or normalized eigenvalues).
Besides the covariance features, other geometric features, such as the verticality and roughness, are often applied in the point cloud classification process in the field of the architectural heritage. Plenty of works have demonstrated that proper features can promote the accuracy of the results [20,28]. However, selecting the proper features is a difficult task. On the one hand, the selected geometric features can express the shape of classification objects; on the other hand, these features should have differences.
To select the proper features for the semantic segmentation of the MQDOAs roof, a feature selection process was conducted as follows: • At first, we analyzed the shape of each type of roof component; • We tested the various geometric features that appeared in the previous work based on the former analysis results; • At last, we selected the proper features from the experimental results through the visual inspection.
Through trial and error, linearity, planarity, sphericity, normal change rate, verticality and roughness were extracted as the semantic segmentation features in this paper. The used features are shown in Table 1 and Figure 5.
(2) Features from construction regulations Table 1. The used regular geometric features.

Feature Description Formulas
Linearity It can distinguish the ridge.
Planarity It can facilitate the identification of lianyan and rafter.
Sphericity It reflected the shape of baoding, round tile and zoushou.
Normal change rate It was used for classifying the ridge, round tile and flat tile.
It was essential to distinguish ridge, rafter and lianyan.

Roughness
It highlighted the surface of taoshou.  [20,28]. However, selecting the proper features is a difficult task. On the one hand, the selected geometric features can express the shape of classification objects; on the other hand, these features should have differences.
To select the proper features for the semantic segmentation of the MQDOAs roof, a feature selection process was conducted as follows: (1) At first, we analyzed the shape of each type of roof component; (2) We tested the various geometric features that appeared in the previous work based on the former analysis results; (3) At last, we selected the proper features from the experimental results through the visual inspection.
Through trial and error, linearity, planarity, sphericity, normal change rate, verticality and roughness were extracted as the semantic segmentation features in this paper. The used features are shown in Table 1 and Figure 5.   Based on the construction regulations, the components were arranged symmetrically along the symmetry axis of buildings ( Figure 6a) and the topological relationship of the different types of components was relatively fixed (Figure 6b). Although the coordinates can describe the arrangement law of the roof components, the points of different component categories were staggered in the 3D space. Only relying on the coordinates (X, Y, Z) was difficult to meet the classification requirement. For example, zoushou must be on the ridges. However, the points belonged to ridge may be higher than the points of zoushou in some areas.
where is the length of on the XOY plane, is the distance from a point to the symmetry axis of the MQDOAs. Obviously, the NSD is the same, and the point class is the same. Moreover, the NSD of roof components located at the outermost contour of the roof is nearly 1 for each section line. On the XOY plane, the staggered phenomenon of points is lower.
round tile drip tile rafter lianyan taoshou baoding flat tile ridges zoushou  Considering that the shape of each section line along (or perpendicular to) the main direction of the buildings was similar (Figure 6c) and the arrangements of the roof components along the section line were also the same (Figure 6d), the relative locations, including the normalized symmetrical distance (NSD), which is the normalized distance from a point to the symmetry axis of the MQDOAs and the relative height (RH), were extracted from the section lines as the classification features.
As shown in Figure 6c, L i is a section line along the main direction of the building and p in x in y in z in is a point that belongs to L i . The feature NSD in of this point is calculated as follows: where l i is the length of L i on the XOY plane, d in is the distance from a point to the symmetry axis of the MQDOAs. Obviously, the NSD is the same, and the point class is the same. Moreover, the NSD of roof components located at the outermost contour of the roof is nearly 1 for each section line. On the XOY plane, the staggered phenomenon of points is lower. Supposed that p il x il y il z il and p ih x ih y ih z ih are the lowest and highest points on L i , separately; and p im x im y im z im represents the lowest point located in the middle area of this section line. The feature RH in of the point p in x in y in z in is described in (7).
In this equation, z maxm represents the highest point of the whole roof. Based on this equation, the staggered phenomenon of points from different roof components can be efficiently lowered at the Z axis. In particular, we have the following: (1) Along the Z direction, the main difference between L i and L j is the height caused by the ab section (Figure 6c). On the section line L i , if h ab is added to the height from p il to p im , the normalized height of points which are lower than p im will become almost the same as that of the points which are lower than p jm on L j . Hence, when a point is lower than p im , we can normalize to the highest section line. (2) When the point p in is higher than p im , this point should belong to the ridge or zoushou.
This is applicable to the other section line.
The arrangement of the flat tiles and round tiles also show a strong regularity in a local area along the section line. As shown in Figure 7, P1 and P2 are the points on the flat tile and the round tile, separately. Obviously, the points that belong to the flat tile are at the bottom of the local area between the two round tiles, and the height of these points is almost the same. Hence, the local height difference (LHD) between a point and the lowest point within a local area of the current point along the section line was applied in this paper. The size of the local area can be defined as being double the width of the round tile. The LHD of the n th point on the i th section line is expressed as follows: where Z inl is the height of the lowest point within a local area of the current point on the section line. Figure 8 shows the extracted features' results along the section line. Notably, the section line perpendicular to the main direction of the building is extracted and the features are also calculated in this method.
In this equation, represents the highest point of the whole roof. Based on this equation, the staggered phenomenon of points from different roof components can be efficiently lowered at the Z axis. In particular, we have the following: (1) Along the Z direction, the main difference between and is the height caused by the ab section (Figure 6c). On the section line , if ℎ is added to the height from to , the normalized height of points which are lower than will become almost the same as that of the points which are lower than on . Hence, when a point is lower than , we can normalize to the highest section line. (2) When the point is higher than , this point should belong to the ridge or zoushou. This is applicable to the other section line.
The arrangement of the flat tiles and round tiles also show a strong regularity in a local area along the section line. As shown in Figure 7, P1 and P2 are the points on the flat tile and the round tile, separately. Obviously, the points that belong to the flat tile are at the bottom of the local area between the two round tiles, and the height of these points is almost the same. Hence, the local height difference (LHD) between a point and the lowest point within a local area of the current point along the section line was applied in this paper. The size of the local area can be defined as being double the width of the round tile. The LHD of the point on the section line is expressed as follows: where is the height of the lowest point within a local area of the current point on the section line. Figure 8 shows the extracted features' results along the section line. Notably, the section line perpendicular to the main direction of the building is extracted and the features are also calculated in this method.
In this equation, represents the highest point of the whole roof. Based on this equation, the staggered phenomenon of points from different roof components can be efficiently lowered at the Z axis. In particular, we have the following: (1) Along the Z direction, the main difference between and is the height caused by the ab section (Figure 6c). On the section line , if ℎ is added to the height from to , the normalized height of points which are lower than will become almost the same as that of the points which are lower than on . Hence, when a point is lower than , we can normalize to the highest section line. (2) When the point is higher than , this point should belong to the ridge or zoushou. This is applicable to the other section line.
The arrangement of the flat tiles and round tiles also show a strong regularity in a local area along the section line. As shown in Figure 7, P1 and P2 are the points on the flat tile and the round tile, separately. Obviously, the points that belong to the flat tile are at the bottom of the local area between the two round tiles, and the height of these points is almost the same. Hence, the local height difference (LHD) between a point and the lowest point within a local area of the current point along the section line was applied in this paper. The size of the local area can be defined as being double the width of the round tile. The LHD of the point on the section line is expressed as follows: where is the height of the lowest point within a local area of the current point on the section line. Figure 8 shows the extracted features' results along the section line. Notably, the section line perpendicular to the main direction of the building is extracted and the features are also calculated in this method.

Multi-Scale Feature Vector Construction
To lower the influence of scale, we applied a multi-scale feature vector for the semantic segmentation of the MQDOAs roof. The process of constructing the multi-scale feature vector is listed as follows: (1) We took different neighborhood size s to calculate the geometric feature vector of the current segmented point. (2) We connected geometric feature vectors at the selected scales and the features from the construction regulation into a long classification feature vector to form the multi-scale classification feature vector of the current segmented point.
The computational time grows with the increase in the search radii. Hence, it was impossible to calculate the geometric features at all scales. To determine the selected scales set S, a simple strategy is given here. This strategy is described as follows: (1) Set the initial scale s0.
(2) Calculate the single-scale geometric feature vector at the current scale.
(3) Input the calculated single-scale geometric feature vector into the classifier and compare the classification accuracy of the target data. If the classification accuracy at the current scale is higher than that at the previous scale, preserve the current scale into S and proceed to step (4); otherwise, output the selected scales set S. (4) Increase the current scale according to the specified interval and turn to step 2.
Notably, from the perspective of machine learning, the validation data were used for scales selection. However, the point cloud of other Ming and Qing Dynasties' official-style architectures with a detailed roof was hard to acquire, thus the data to be segmented for testing were used in this paper.

Random Forest
Random forest (RF) was first proposed by Leo Breiman and Adele Cutler [45] and it has been widely used in point cloud classification field of the culture heritage [26]. This classifier makes use of the multiple trees to train and predict samples. In the training stage, it inputs both the class labels and the extracted features to fit a prediction model and predict the other points without class labels.
As was described in [28], the RF has three advantages: (1) RF is considered a highly accurate and robust method because of the number of decision trees participating in the process. (2) RF can avoid the over-fitting problem, as it takes the average of all the predictions, which cancel out the biases. (3) RF offers a useful feature selection indicator (the relative importance or contribution of each feature in the prediction), which can help us estimate the optimal feature vector.
In this paper, we selected the Scikit Learn Python library (version 0.24.1) [46] to carry out the classification program. Considering several parameters (such as n_estimators, oob_score and max_depth, etc.) which were involved in this module, the default parameters were applied so that the generalization can be further increased.

Feature Selection
In addition to the geometric features (including linearity, planarity, sphericity, verticality, normal change rate and roughness), the features NSD, RH and LHD that were selected, X, Y and Z were also employed as the segmentation features in this paper. As described in Section 3.1, we constructed the multi-scale classification features vector.

Supervised Classification
After the feature selection, the training samples with the selected feature set were utilized to train the RF classifier, and the classifier was used to predict the labels of unlabeled points.

Test Data and Evaluation Criteria
To evaluate the performance of the proposed method, the 3D survey data of the Hall of Complete Harmony was applied. Before the experiment, the roof was extracted from the 3D survey data and each point was labeled into the corresponding categories proposed in Section 2.1 by manual processing. The annotated class is shown in Figure 9.

Test Data and Evaluation Criteria
To evaluate the performance of the proposed method, the 3D survey data of the H of Complete Harmony was applied. Before the experiment, the roof was extracted fro the 3D survey data and each point was labeled into the corresponding categories p posed in Section 2.1 by manual processing. The annotated class is shown in Figure 9.
We divided the roof into two parts: one part was regarded as the training data a the other part was used for the test data. The training set should contain all types of ro components. Considering the symmetric structure of the roof, the train data were main selected along the main direction of the building as is shown in Figure 10. Because taosh is only located at the corners of the roof, some points located at the corners must be lected; otherwise, the taoshou will be incorrectly segmented. The point cloud statistics the training set and test set are shown in Table 2. round tile drip tile rafter lianyan taoshou baoding flat tile ridges zoushou

Test data
Train data Figure 10. The selected test data and train data. We divided the roof into two parts: one part was regarded as the training data and the other part was used for the test data. The training set should contain all types of roof components. Considering the symmetric structure of the roof, the train data were mainly selected along the main direction of the building as is shown in Figure 10. Because taoshou is only located at the corners of the roof, some points located at the corners must be selected; otherwise, the taoshou will be incorrectly segmented. The point cloud statistics of the training set and test set are shown in Table 2.

Supervised Classification
After the feature selection, the training samples with the selected feature set were utilized to train the RF classifier, and the classifier was used to predict the labels of unlabeled points.

Test Data and Evaluation Criteria
To evaluate the performance of the proposed method, the 3D survey data of the Hall of Complete Harmony was applied. Before the experiment, the roof was extracted from the 3D survey data and each point was labeled into the corresponding categories proposed in Section 2.1 by manual processing. The annotated class is shown in Figure 9.
We divided the roof into two parts: one part was regarded as the training data and the other part was used for the test data. The training set should contain all types of roof components. Considering the symmetric structure of the roof, the train data were mainly selected along the main direction of the building as is shown in Figure 10. Because taoshou is only located at the corners of the roof, some points located at the corners must be selected; otherwise, the taoshou will be incorrectly segmented. The point cloud statistics of the training set and test set are shown in Table 2. round tile drip tile rafter lianyan taoshou baoding flat tile ridges zoushou   For evaluation, we employed four commonly used measures: overall accuracy (OA), precision, recall and F1 _score. They are computed as follows:

Experimental Results and Analysis
The MQDOAs roof is composed of all types of tiles. Among them, the thickness of the tiles is about 2.5 cm. So, we took 2 cm as the initial scale. Then, we increased the scale from 5 cm to 0.2 m gradually. The classification accuracy was calculated under different scales. When the scale was 0.2 m, the classification accuracy was lower than that under the scale of 0.1 m. Hence, in the experiment, the geometric features were calculated within s1 = 0.02 m, s2 = 0.05 m, s3 = 0.1 m and s4 = 0.2 m, respectively. To verify the advantage of the proposed method, we connected the geometric feature vectors based on the different scale combinations (including s0, s0 + s1, s0 + s1 + s2, s0 + s1 + s2 + s3) and the proposed features to form a long dimension feature vector as the input features for the point cloud classification of the MQDOAs roof.
Moreover, two optimal feature vectors were also selected according to the importance rank of features based on the RF. The first optimal feature vector came from a pure multiscale geometric feature vector without the proposed features based on the scale combination of s0 + s1 + s2 + s3 and a multi-scale feature vector, including the proposed features based on the scale combination of s0 + s1 + s2 + s3 in our experiment. The first optimal feature vector included Z, roughness (0.2), normal change rate (0.2), verticality (0.2), normal change rate (0.1), verticality (0.1), normal change rate (0.05), planarity (0.05) and sphericity (0.05); the second optimal feature vector contained Z, roughness (0.2), sphericity (0.2), verticality (0.2), verticality (0.1), normal change rate (0.05), planarity (0.05), LHD1 and LHD2 (which were extracted from the section line along and perpendicular to the main direction of building).
A comparative test was carried out on the combination of different scales. The classification results are shown in Tables 3 and 4. The segmentation result of the Hall of Complete Harmony's roof is shown in Figure 11. Table 4 shows that the highest OA reached 96.8% when the proposed features were added at the s0 + s1 + s2 scales. Besides the OA, a rigid evaluation index, the average value of F1_score, was also applied to lower the impact of the quantitative variation of points within different categories in performance evaluation. As is shown in Figure 12, the lowest average value of F1_score was 62.2% only relying on the regular geometric features at the s0 scale, and the highest average value of F1_score (92.4%) still occurred when the proposed features were added at the s0 + s1 + s2 scales. Obviously, when the proposed features were added, the average F1_score was higher than that only relying on the regular geometric features at the same scale or the scale combination. On the other hand, the multi-scale feature connection strategy also improved the average F1_score.
The experimental results showed that the segmentation accuracy obtained after adding our proposed features and applying the multi-scale long feature vectors was significantly better than that of relying on the covariance features and single scale. Both the proposed features and multi-scale feature connection strategy can improve the point cloud classification accuracy. The segmentation result of the Hall of Complete Harmony's roof is shown in Figure  11. Table 4 shows that the highest OA reached 96.8% when the proposed features were added at the s0 + s1 + s2 scales. Besides the OA, a rigid evaluation index, the average value of F1_score, was also applied to lower the impact of the quantitative variation of points within different categories in performance evaluation. As is shown in Figure 12, the lowest average value of F1_score was 62.2% only relying on the regular geometric features at the s0 scale, and the highest average value of F1_score (92.4%) still occurred when the proposed features were added at the s0 + s1 + s2 scales. Obviously, when the proposed features were added, the average F1_score was higher than that only relying on the regular geometric features at the same scale or the scale combination. On the other hand, the multi-scale feature connection strategy also improved the average F1_score. Figure 11. The segmentation results based on the feature vector based on the scales combination of s0 + s1 + s2. Figure 11. The segmentation results based on the feature vector based on the scales combination of s0 + s1 + s2. The experimental results showed that the segmentation accuracy obtained after adding our proposed features and applying the multi-scale long feature vectors was significantly better than that of relying on the covariance features and single scale. Both the proposed features and multi-scale feature connection strategy can improve the point cloud classification accuracy.

Impact of Proposed Features
As was shown in Figure 13, at the same scale or scale combinations, the classification accuracies increased when the proposed features were input into the RF classifier. The max difference of classification accuracy between the regular geometric feature vector and the combination of regular geometric and proposed features accrued at s0 scale and reached 19.4%. The minimum difference of classification accuracy accrued at s0 + s1 + s2 + s3 scale combination and reached 2.2%. In the case of using single-scale features, the classification accuracy only relying on the geometric features ranged from 74.1% to 87.1%, while the fluctuation of classification accuracy was less than 3% when the proposed features were added. This showed that the features from the construction regulations not only increased the classification accuracy, but also could lower the influence of the scales on the classification accuracy.

Impact of Proposed Features
As was shown in Figure 13, at the same scale or scale combinations, the classification accuracies increased when the proposed features were input into the RF classifier. The max difference of classification accuracy between the regular geometric feature vector and the combination of regular geometric and proposed features accrued at s0 scale and reached 19.4%. The minimum difference of classification accuracy accrued at s0 + s1 + s2 + s3 scale combination and reached 2.2%. In the case of using single-scale features, the classification accuracy only relying on the geometric features ranged from 74.1% to 87.1%, while the fluctuation of classification accuracy was less than 3% when the proposed features were added. This showed that the features from the construction regulations not only increased the classification accuracy, but also could lower the influence of the scales on the classification accuracy. The experimental results showed that the segmentation accuracy obtained after adding our proposed features and applying the multi-scale long feature vectors was significantly better than that of relying on the covariance features and single scale. Both the proposed features and multi-scale feature connection strategy can improve the point cloud classification accuracy.

Impact of Proposed Features
As was shown in Figure 13, at the same scale or scale combinations, the classification accuracies increased when the proposed features were input into the RF classifier. The max difference of classification accuracy between the regular geometric feature vector and the combination of regular geometric and proposed features accrued at s0 scale and reached 19.4%. The minimum difference of classification accuracy accrued at s0 + s1 + s2 + s3 scale combination and reached 2.2%. In the case of using single-scale features, the classification accuracy only relying on the geometric features ranged from 74.1% to 87.1%, while the fluctuation of classification accuracy was less than 3% when the proposed features were added. This showed that the features from the construction regulations not only increased the classification accuracy, but also could lower the influence of the scales on the classification accuracy.   In addition to the overall accuracy, the proposed features also obtained good performance on the classification accuracy of each class. As shown in Figure 14, except the class zoushou, the F1 _score of the other classes was higher than that without the proposed features. Although the F1 _score of zoushou was lower, the highest difference was less than 5% at the same scale. This showed that the proposed features had good performance on the classification of the roof point cloud. In addition to the overall accuracy, the proposed features also obtained good performance on the classification accuracy of each class. As shown in Figure 14, except the class zoushou, the F1_score of the other classes was higher than that without the proposed features. Although the F1_score of zoushou was lower, the highest difference was less than 5% at the same scale. This showed that the proposed features had good performance on the classification of the roof point cloud.

Influence of Different Scale Combinations
As shown in Figure 14, the highest classification accuracy based on the single-scale geometric feature vector was 87.1% and the lowest classification accuracy based on the multi-scales geometric features was 88.4%. Similarly, when the proposed features were added, the lowest classification accuracy of multi-scale long feature vectors was also higher than the highest classification accuracy of single-scale features vectors. Hence, compared with the single-scale feature vectors, the multi-scale long feature vectors (such as s0 + s1 , s0 + s1 + s2 , s0 + s1 + s2 + s3 ) performed better when the same features were applied.
The classification accuracies based on two sets of optimal feature vector were 92.9% and 96%. They were still lower than 94.4% and 96.6%. In nature, the optimal features are generated from the combination of multi-scale features (s0 + s1 + s2 + s3). Some unimportant features may be ignored. However, these unimportant features play an important role in the class which contains few points [47]. This optimal-features strategy may reduce the segmentation accuracy of these classes. As shown in Figure 15, the difference of F1_score of taoshou between the optimal features the segmentation accuracy increased. Although the optimal-features strategy can reduce the time consumption, the multi-scale was essential for the fine point cloud segmentation of the MQDOAs roof.

Influence of Different Scale Combinations
As shown in Figure 14, the highest classification accuracy based on the single-scale geometric feature vector was 87.1% and the lowest classification accuracy based on the multi-scales geometric features was 88.4%. Similarly, when the proposed features were added, the lowest classification accuracy of multi-scale long feature vectors was also higher than the highest classification accuracy of single-scale features vectors. Hence, compared with the single-scale feature vectors, the multi-scale long feature vectors (such as s0 + s1, s0 + s1 + s2, s0 + s1 + s2 + s3) performed better when the same features were applied.
The classification accuracies based on two sets of optimal feature vector were 92.9% and 96%. They were still lower than 94.4% and 96.6%. In nature, the optimal features are generated from the combination of multi-scale features (s0 + s1 + s2 + s3). Some unimportant features may be ignored. However, these unimportant features play an important role in the class which contains few points [47]. This optimal-features strategy may reduce the segmentation accuracy of these classes. As shown in Figure 15, the difference of F1 _score of taoshou between the optimal features the segmentation accuracy increased. Although the optimal-features strategy can reduce the time consumption, the multi-scale was essential for the fine point cloud segmentation of the MQDOAs roof.
For the combinations of scales, the strategy of connecting single-scale feature vectors into multi-scale feature vectors was not that the more numerous the scales, the more beneficial it was to improve the classification accuracy. As can be seen from Figure 13, the combination of three scales (s0 + s1 + s2) had the highest segmentation accuracy in the comparison test rather than the combination of four scales s0 + s1 + s2 + s3. For the combinations of scales, the strategy of connecting single-scale feature vectors into multi-scale feature vectors was not that the more numerous the scales, the more beneficial it was to improve the classification accuracy. As can be seen from Figure 13, the combination of three scales (s0 + s1 + s2) had the highest segmentation accuracy in the comparison test rather than the combination of four scales s0 + s1 + s2 + s3.
The best experimental result of the multi-scale feature vector including the proposed features occurred at the s0 + s1 + s2 scales, and the scales combination was consistent with our proposed scale combination strategy. In fact, the classification accuracy increased from the scale s0 to s3. When the scale was equal to 0.2 m, the segmentation accuracy declined. The combination of scales s0 + s1 + s2 can be regarded as the optimal scale combination. However, the experimental result of multi-scale feature vector without the proposed features showed a different conclusion. Although the segmentation accuracy also declined only relying on the pure geometric features when the scale was equal to 0.2 m, the optimal scales combination was s0 + s1 + s2 + s3, rather than s0 + s1 + s2. These experimental results showed the proposed scale selection strategy can be close to the optimal result and may not achieve the optimal result in some cases.

Conclusions
Considering the complexity of MQDOAs roof and the scale differences between different component types, the proposed method applied a multi-scale features vector for the point cloud segmentation of MQDOAs roof. The highlights of this work are listed as follows: (1) The features, including RH, NSD and LHD, were selected for the point cloud sematic segmentation of the Ming and Qing Dynasties' official-style architecture roof, and the corresponding feature extraction methods were proposed. (2) For the fine segmentation of the MQDOAs roof, the multi-scale feature vector was essential, and the scale connection strategy was given in this paper. (3) The experimental results showed that our proposed method can achieve good performance and has robustness after the proposed features are added and the multiscale strategy is applied.
Although our proposed method was not tested on the additional point cloud data of other MQDOAs roofs, due to the absence of data, the algorithm can achieve the presented results for other MQDOAs roofs, theoretically. This is mainly because the components The best experimental result of the multi-scale feature vector including the proposed features occurred at the s0 + s1 + s2 scales, and the scales combination was consistent with our proposed scale combination strategy. In fact, the classification accuracy increased from the scale s0 to s3. When the scale was equal to 0.2 m, the segmentation accuracy declined. The combination of scales s0 + s1 + s2 can be regarded as the optimal scale combination. However, the experimental result of multi-scale feature vector without the proposed features showed a different conclusion. Although the segmentation accuracy also declined only relying on the pure geometric features when the scale was equal to 0.2 m, the optimal scales combination was s0 + s1 + s2 + s3, rather than s0 + s1 + s2. These experimental results showed the proposed scale selection strategy can be close to the optimal result and may not achieve the optimal result in some cases.

Conclusions
Considering the complexity of MQDOAs roof and the scale differences between different component types, the proposed method applied a multi-scale features vector for the point cloud segmentation of MQDOAs roof. The highlights of this work are listed as follows: (1) The features, including RH, NSD and LHD, were selected for the point cloud sematic segmentation of the Ming and Qing Dynasties' official-style architecture roof, and the corresponding feature extraction methods were proposed. (2) For the fine segmentation of the MQDOAs roof, the multi-scale feature vector was essential, and the scale connection strategy was given in this paper. (3) The experimental results showed that our proposed method can achieve good performance and has robustness after the proposed features are added and the multi-scale strategy is applied.
Although our proposed method was not tested on the additional point cloud data of other MQDOAs roofs, due to the absence of data, the algorithm can achieve the presented results for other MQDOAs roofs, theoretically. This is mainly because the components which form the roofs are almost the same, and the distribution of components is also similar.
Additionally, it is worth noting that the classification of the Ming and Qing Dynasties' official-style architecture was not the primary goal of our work. We hoped to reconstruct a historical building information model (HBIM) of the Ming and Qing Dynasties' official-style architecture, automatically. In previous work [48], the method reconstructing the decorative components based on a template library was given. The future work will include (1) partial occlusion component information extraction (such as flat tiles); (2) the parameterization of components; (3) the HBIM reconstruction mechanism; and (4) using various strategies to ensure that the point cloud can be processed in memory.