The production of as-built Building Information Modeling (BIM) models is becoming increasingly widespread in the Architectural, Engineering and Construction (AEC) industry. These models reflect the state of the building up to as-built conditions and are used for quality control, quantity take-offs, maintenance, and project planning [1
]. The geometry of an as-built BIM is obtained by creating a model from scratch based on measurements on site or updating an existing as-design model. The emphasis of this work is on the reverse engineering of as-built BIMs without a prior model since few buildings have a model. Currently, the objects involved are created manually by creating geometry representations based on point cloud data captured of the built structure. It is a cumbersome procedure which requires significant time investment and trained personnel. We specifically target the geometry of wall objects as they form the basis of the structure and can be reliably identified in point cloud data. This work is the fully integrated extension of the prototype presented in previous work [3
]. The method was expanded to operate on large scale datasets and includes additional functionalities to cope with the shortcomings of the initial prototype.
The reconstruction of wall geometry from point cloud data is typically considered as an unsupervised process. However, the interpretation of this data is sensitive to noise, clutter and the complexity of the structure. Also, the amount of data introduces significant computational challenges [4
]. The acquisition of point cloud data is typically performed by remote sensing techniques which are bound to Line-of-Sight (LoS). The resulting datasets are therefore inherently occluded in inaccessible areas and isolated spaces such as the areas above ceilings. Reasoning algorithms make assumptions about these zones which are prone to misinterpretation. This issue is often mitigated by integrating the sensor’s position and orientation. However, as it is our goal to operate on sensor independent point cloud data, we consider the inputs as unstructured. Also, we limit the scope of the procedure to the clustering of wall observations since many other classes can be automatically derived from their geometry e.g., doors, windows, floors and ceilings. The clustering is performed on object level based on both simplistic and complex BIM wall representations. We deal with wall observations which are not parallel to the main wall faces and are located in highly cluttered and noisy environments. Also, our approach can deal with complex wall configurations such as near-coplanar wall faces of different walls by simultaneously computing the wall labels for every segment. Similarly, we process the entire structure simultaneously since our method is capable of dealing with multi-storey observations.
The following paragraphs are part of this research. The background and related work are discussed in Section 2
. In Section 3
, the methodology is explained. The test design and experimental results are presented in Section 4
. Finally, the conclusions are discussed in Section 6
2. Background and Related Work
The reconstruction of BIM entities from point cloud data is a popular subject. Several procedures are discussed that target the reverse engineering of wall geometry from geometric inputs. The proposed unsupervised methods typically consist of the following set of consecutive algorithms [5
]. First, a preprocessing of the geometric inputs is performed to increase the computational efficiency. In 2D methods, the point cloud is typically restructured as a number of raster images, each representing a section of the data or projected information [6
]. In 3D methods, the point cloud is often represented in a voxel octree which significantly enhance nearest neighborhood searches [8
] (Figure 1
a). Following the preprocessing, the data is segmented into groups based on the associativity of the point characteristics. Algorithms such as RANSAC asses properties e.g., normals or colors and combine the points into regions with similar attributes. Alternative methods use region growing, Probabilistic Graphical Models, and so on [9
]. Once a set of regions is established, the geometry of the regions are often replaced by primitives to reduce the data. Typically, lines are employed in 2D procedures and planes or other primitives are used in 3D methods [8
]. The resulting segments are then classified into a number of preset outputs. In building reconstruction, the classes typically consists of a buildings’ common components such as floors, ceilings, walls and so on. A set of local and contextual features (Figure 1
b) are extracted from each observation after which its class is determined by feeding the features into a decision function. The functions involved can be based on heuristics or machine learning [14
]. Several researchers combined man made rules in a decision function to classify 3D geometry inputs [16
]. While being very efficient, heuristics are typically case specific as the model weights and parameters are intuitively determined. More flexible methods employ machine learning procedures to deal with the wide variety of object parameters, classes, and observations. Xiong et al. [15
] extract features from presegmented meshes and classify them using stacked learning. Markov random fields (MRF) and conditional random fields (CRF) are also proposed for the classification of building geometry [20
]. A popular approach is the use of convolutional neural networks (CNN) that recombine sets of neurons in different layers in order to process a set of input vectors to a known set of outputs [23
]. Especially for 2D inputs, inference accuracies of over 90% are reported for the detection of building objects on databases such as CMP [24
], eTRIMS [25
], and ECP [26
]. The resulting labeled wall segments serve as input to the wall clustering that is the emphasis of this work. A similar reasoning, as in segmentation and classification, is employed to identify a set of clusters that each contain all the relevant observations of a wall that conform to the BIM model (Figure 1
c). Finally, these groups serve as input for the application specific reconstruction algorithms. Depending on the application, different geometry definitions are considered for the walls. For instance, walls can be considered as volumetric wall entities (BIM) or surface based geometry where each wall face is a separate entity (CityGML) [27
]. Some applications even consider walls as a child entity of room geometry [28
The paradigm of clustering observations according to their wall representations is a common step in BIM reconstruction procedures. Similar to segmentation and classification, the methods involved range from heuristics to region growing, model based methods, graphs, and machine learning techniques [5
]. The majority of methods consider simplistic box definitions for the wall geometry representation i.e., walls are considered an aggregate of coplanar and parallel segments.
For methods that are only concerned with the reconstruction of wall faces, the clustering problem is typically reduced to the detection of the predominant plane [29
]. For instance, Oesau et al. [17
] rely on the collinearity of the segments of a wall face. They implement a Hough transform to retrieve all the points on a wall face in a 2D slice of the point cloud. Adan et al. [30
] present a similar method that represents the point cloud in a 2D histogram along its zenith. Similar results are presented by Previtali et al. [31
], Budroni et al. [32
], Valero et al. [33
], and Nikoohemat et al. [16
]. 3D methods are also considered. For instance, Thomson et al. [34
] implement a Point Cloud Library (PCL) based RANSAC method that extracts the main planar faces in a point cloud structure. They subdivide the output clusters based on the euclidean distance between the boundaries of the segments in order to separate coplanar walls. Overall, these methods are promising in scenes that have limited wall detailing and where complex wall configurations are not an issue.
The researchers that do actually cluster both sides of walls typically do so conform a box or rectangle. A promising method is presented by Mura et al. Mura et al. [29
] that project their detected wall segments onto a 2D plane and cluster them based on coplanarity and collinearity. They first compute a directional clustering which yields the main orientation of the individual walls. For each direction, a 1D mean-shift clustering [35
] is performed which identifies all possible offsets of parallel wall segments of that orientation. Finally, they compute a representative line for each wall cluster in 2D that is then used in the reconstruction step. Ochmann et al. [36
] on the other hand use heuristics to cluster parallel and coplanar wall segments. Each potential pair of wall surface lines fulfilling certain constraints is considered as the two opposite surfaces of a wall separating adjacent rooms. The segments are then clustered by evaluating the overlap and the orientation of the candidates. Aside from heuristics, model based methods are also used. Recent variants of the RANSAC algorithm [37
] are extended to not just fit simplistic models such as lines or planes but match entire walls. For instance, Oesau et al. [17
] employ a two-line hypothesis created by RANSAC to detect corner configurations for their walls. Similar to the single faced clustering, the majority does not consider the grouping of more complex wall geometry or highly associative walls.
There are currently few methods that cluster wall segments into more complex walls. These walls represent their actual geometry which includes wall detailing such as observations of niches, protrusions, and openings. They consist of segments with varying normals and sizes in complex configurations. However, there are several methods that target different clustering tasks that might be extended to the clustering of wall geometry. For instance, Ikehata et al. [38
] exploit k-medoids and heuristic segment clustering to cluster room observations. They project the observations of the rooms in a raster image and use the 2D representation to initialize a selective seeding algorithm to determine to determine which room an observation belongs too. Armeni et al. [39
] consider the idea of employing connected component graphs. Prior to the clustering of their observations, they subdivide their graph by removing invalid edges give some validation criterion e.g., the distance between the boundaries. As a result, some parts of the graph become isolated with only segments connecting to segments within the part. These connected components have a significantly lower combination complexity and also are less prone to misclustering since not every possible edge is considered. Armeni et al. use this method in combination with region growing to cluster room segments. They observe the wall density signature along the length of the edge in order to detect whether or not the edge collides with a wall (peak). Connected components are also used to cluster of roof objects. He et al. [41
] considers connected component graphs as aggregates of roof structures to detect all the roof observations belonging to a single building. The concept of component graphs is a promising technique which we also use for the complex wall clustering. Both heuristics and machine learning algorithms can be used to dispatch invalid edges from the graph after which the remaining linked observations can be subjected to a subsequent clustering. For instance, Gilani et al. [42
] use connected component graphs to merge line segments of roof edges in aerial lidar applications. Alternatively, Mura et al. [43
] use of structural paths for room segment clustering. They propose region growing in an adjacency matrix based on pairwise relations between the observations. Given the connectivity of the graph structure, they iteratively asses the edges for one of k
predefined connections. As a result, their method computes sets of patches enclosing a target space based on a seed node. In our research, we opt for the use of machine learning techniques in connected component graphs to simultaneous assign the state of all the observed segments which is expected to yield a more balanced clustering than the one-sided stopping criteria of, for instance, region growing.
A powerful machine learning method for the simultaneous segmentation and clustering are conditional random fields (CRFs) [44
]. Given a graph which consists of a set of nodes and edges, a best fit configuration can be computed by decoding the network. It is considered an instance of supervised pattern recognition where a set of states is assigned to a fixed number of observations. Several methods exist do compute decoding or inference of a CRF, most of which are approximate since the dependencies of most graphs are intractable. For instance, there are variants of the Iterated Conditional Mode (ICM) algorithm, graph cuts [28
] and loopy belief propagation [6
]. As this technique requires the exact number of states or walls to be known prior to the feature extraction, it is paramount to perform a proper seeding in the graph. This can be challenging since not all walls have stereotypical seeds. Also, the estimation of the model parameters when factorizing the graph is sensitive to outliers and requires robust features and training data. Despite these shortcomings, we believe CRF is a promising approach since allows the integration of both unary and pairwise relations which aids the clustering in complex wall configurations.
The objective of clustering wall geometry is retrieving a sufficient number of wall observations for each wall so BIM Wall objects can be modeled from them. The target entities are parametric volumetric BIM objects and thus the walls should be clustered according to this representation. Depending on the model, different types of clustering are required. For instance, CityGML [27
] has a surface based wall representation and Oesau et al. [17
] focus on room based reconstruction. We believe our method can also be extended towards other clustering tasks since the simultaneous assignment of wall or room labels allows for better informed decisions compared to one-sided decision functions such as region growing.
The paradigm of wall clustering is affected by a number of parameters. The following two parameters should be taken into consideration when clustering wall observations. First, the success of every method heavily relies on the correspondence of the model parameters to the observations of a real structure. For instance, observations of simplistic walls will be clustered properly by most methods since the abstract box definition applies to the majority of the observations. As the wall definitions become more complex to match the real situation, the parameters become increasingly diverse and more difficult to predict up to the point that human operators can also no longer appropriately cluster the data. It is stated that one-sided methods can only reliably cluster abstract walls while simultaneous assignment methods like ours are also suited to deal with complex wall geometry. Secondly, each process is subject to noise and outliers. In the subsequent processes of acquisition, segmentation, classification, and clustering, there is an accumulation of errors that will affect the final grouping of the observations. Each process should therefore be flexible enough to deal with the outliers from previous methods. This is more challenging for heuristic methods since random noise is hard to define with strict rules. Weighted decision functions can be more easily extended to accommodate these occurrences.
Several improvements can be considered for our workflow. For instance, the presented method is data driven as is the state-of-the-art in wall clustering. Currently, solely geometric features are computed for the seed estimation and subsequent clustering. There is the opportunity to integrate color or sensor information to enhance the clustering results. However, both offer only limited added value towards the retrieval of highly occluded multi-room walls. Another method is to learn not from manually labeled wall segments but from actual BIM models. Given the individual surfaces of the walls of a BIM model, geometric features, and model parameters can be learned from preexisting BIMs. However, not every BIM is modeled using the same convention, and could result in confusing parameters sets preferring single-storey walls over multi-storey walls depending on the training data set.