Geometry and Topology Reconstruction of BIM Wall Objects from Photogrammetric Meshes and Laser Point Clouds

Abstract

As the foundation for digitalization, building information modeling (BIM) technology has been widely used in the field of architecture, engineering, construction, and facility management (AEC/FM). Unmanned aerial vehicle (UAV) oblique photogrammetry and laser scanning have become increasingly popular data acquisition techniques for surveying buildings and providing original data for BIM modeling. However, the geometric and topological reconstruction of solid walls, which are among the most important architectural structures in BIM, is still a challenging undertaking. Due to noise and missing data in 3D point clouds, current research mostly focuses on segmenting wall planar surfaces from unstructured 3D point clouds and fitting the plane parameters without considering the thickness or 3D shape of the wall. Point clouds acquired only from the indoor space are insufficient for modeling exterior walls. It is also important to maintain the topological relationships between wall objects to meet the needs of complex BIM modeling. Therefore, in this study, a geometry and topology modeling method is proposed for solid walls in BIM based on photogrammetric meshes and laser point clouds. The method uses a kinetic space-partitioning algorithm to generate the building footprint and indoor floor plan. It classifies interior and exterior wall segments and infers parallel line segments to extract wall centerlines. The topological relationships are reconstructed and maintained to build wall objects with consistency. Experimental results on two datasets, including both photogrammetric meshes and indoor laser point clouds, exhibit more than 90% completeness and correctness, as well as centimeter-level accuracy of the wall surfaces.

1. Introduction

With the arrival of Industry 4.0 [1], the architecture, engineering, construction, and facility management (AEC/FM) industry has begun to transform to informatization, digitalization and intelligence. Using building information modeling (BIM) technology to promote productivity has increasingly attracted the attention of industries and enterprises [2,3]. In recent years, unmanned aerial vehicle (UAV) oblique photogrammetry and laser scanning technology have been continuously developed and widely used for BIM modeling in the field of AEC/FM [4]. UAVs have the characteristics of a low cost and high efficiency. Terrestrial laser scanning (TLS) has the characteristics of a noncontact approach, high sampling rate, high accuracy, high resolution, and panoramic scanning; such characteristics can significantly save time and reduce costs [5]. Many applications in the field of remote sensing can be solved by incorporating methodologies from the fields of photogrammetry and laser scanning [6,7].

BIM is a rich object-based parametric model. BIM modeling usually comprises three parts: geometric modeling, semantic modeling, and topological relationship modeling [3,8,9]. However, the current research on BIM modeling from laser point clouds is highly dependent on human interaction, which is time-consuming and laborious [10]. The laser point clouds obtained are usually unstructured and lack semantic information. They contain noise and suffer from shadowing from nearby objects. Due to the incomplete point cloud data caused by occlusion, the high density and magnanimity of point clouds, the complex structure of buildings, the staggered space, and the diversity of functions, significant challenges to the automatic extraction of geometric elements of building point clouds may occur [11]. Solid walls are one of the most important building structures in BIM [12]. Most existing studies focus on the plane segmentation and fitting of the plane parameters of the wall surface from the unstructured 3D point cloud, ignoring the occupied space surrounded by the boundary surface (usually shown as a pair of parallel walls) and the topological relationship between the wall objects [13]. In addition, most current studies usually use only a single data source [5,14,15,16,17,18], for example, indoor 3D point clouds [17,18], which cannot provide sufficient information to estimate the thickness of exterior walls and thus cannot meet the needs of BIM modeling for complex buildings. Both laser point clouds and pictures are essential data sources for BIM wall reconstruction, and their individual benefits should be merged to increase information availability.

In this study, a BIM solid wall modeling method based on oblique photogrammetric meshes and laser point cloud is proposed. The contributions of this study are two-fold: (1) A kinetic space partition algorithm to generate building footprint and indoor floor plan. It classifies interior and exterior wall segments and infers parallel line segments to extract wall centerlines; (2) A topology reconstruction method that reconstructs and maintains the topological relationship of detected wall elements is used to build BIM wall objects with topological consistency.

2. Related Works

In the AEC/FM field, the scan-to-BIM method using laser scanning technology has become increasingly popular. However, laser scanning suffers from shadowing from nearby objects. Moreover, relying only on indoor 3D point clouds cannot provide sufficient information to estimate the thickness of exterior walls. UAVs are relatively inexpensive, allowing for rapid surveys over infrastructure projects where required [6]. They can scan inaccessible objects not visible from the ground, such as roof tops. Therefore, a full and complete dataset can now be achieved due to their mobile nature. UAVs acquire data from the air using a vertical or oblique angle of view and suffer much less from shadowing from nearby objects. UAVs are perfect to be utilized in the BIM modeling workflow because they are ideal for scanning the external features of buildings due to their speed and maneuverability. The photogrammetric data can then be merged with laser scan data collected for the building internals to produce complete data of the whole building, inside and outside. Freimuth and Konig [14] used UAVs for the automated creation of images and point cloud data of particular construction objects. Alshawabkeh et al. [19] integrated a laser scanner and photogrammetry for heritage BIM enhancement. It has become a tendency to fuse multisource data for scanning to BIM modeling.

2.1. BIM Elements Detection

The current research on the reconstruction of BIM models based on 3D point clouds mainly focuses on the classification of building scenes and the extraction of structural elements, both of which usually rely on the assumption of good sampling of building components. Sanchez and Zakhor [20] used point cloud data and detected building interior planes. The method first uses the principal component analysis (PCA) algorithm to divide the interior points into ceilings, floors, walls and other small building structures and uses the random sampling consistency (RANSAC) to fit the detected plane primitives.

Romero-Jarén and Arranz [21] proposed an automatic segmentation and classification method for point clouds. It generates the surfaces of building elements, including floors, ceilings, walls, columns, etc. Coudron et al. [22] proposed using depth learning to enhance scene semantics and extract the plane structure from the point cloud. These methods mainly fit plane surface models. When there is obvious clutter and occlusion in the indoor environment, the extracted plane will have redundancy, fragmentation, and topology errors [23]. Therefore, many scholars use the cellar nature of indoor space (i.e., room) as knowledge to help reconstruct the building structure and space and reconstruct the wall surface model through space partitioning and room reconstruction [23,24,25]. These methods usually first extract wall lines or planes, build a 2D/3D cell complex, use the graph-cut optimization algorithm to solve the energy optimization function, semantically label the cell complex, build a 3D model of the room, and reconstruct the wall surface geometry.

2.2. Geometry and Topology Reconstruction

The problem of solid wall modeling is not only the geometry fitting of the wall surface. The wall solid also needs to be inferred from the extracted boundary surface (usually represented as a parallel wall). Therefore, Zhou and Li [26] proposed a line-feature method based on constrained least squares to achieve the reconstruction of interior walls. This method builds a solid-wall model by fitting parallel surfaces in the point cloud. Fotsing et al. [27] proposed a voxel-based solid-wall detection method from point clouds, which uses voxel morphological transformation to identify the space occupied by the wall. Tran et al. [28] proposed an indoor 3D modeling method based on shape grammar, which uses simple elements and applies the grammar rules controlled by the generation process iteratively to derive the 3D model of a complex environment. This method can reconstruct the indoor navigation space and its topological relationship and can generate parametric 3D models with a high geometric accuracy, rich semantic content, and BIM standards. Jung et al. [18] proposed extracting the projected parallel line of the wall in a two-dimensional Cartesian coordinate system using the constrained least squares method, dividing the solid wall into an interior wall and an exterior wall, and building the BIM wall model. However, this method does not further build the topological relationship of the solid walls [29]. From the existing research, BIM solid wall modeling includes the following general steps: point cloud segmentation and semantic classification, geometric element fitting, and the parameter extraction of wall solids. Bassier and Vergauwen [17] proposed an unsupervised wall-modeling method when the input point cloud has been segmented according to the semantics of a solid wall. This method proposes four connection types: the intersection, orthogonal, blended, and direct connection to restore the complete wall structure. Their research [30] further proposed a connection evaluation framework for building the topological relationship between walls.

Solid wall modeling methods can be classified as room-based methods and wall-based methods [17]. The simple method of inferring wall solid object parameters based on wall geometry extraction will encounter difficulty in environments with a large number of occlusions. The wall-based method does not consider architectural space and can only build wall objects. Due to the occlusion of indoor objects and missing data in point clouds, it is difficult to ensure the topological consistency of the building model. The room-based method considers the cellar characteristics of the room as a priori knowledge and can meet the modeling requirements in the case of missing point cloud data on the wall. At the same time, the method based on space partitioning to generate cell complexes can ensure topological consistency. Many scholars have studied methods based on space partitioning and optimizing cell complexes [9,25,31,32]. Some scholars limit the building space to the 2.5D hypothesis [32] or adopt the stricter Manhattan world assumption [33]. Other scholars deal with 3D modeling problems by putting forward implicit assumptions on input data as prior knowledge. Ochmann et al. [25] proposed a global optimization method to construct a wall entity by using wall elements as the adjacent area between adjacent reconstruction rooms while maintaining a reasonable connection between all wall elements. However, this method requires that each room contains at least one scanning station. Wang et al. [31] first used the wall line to divide the two-dimensional space, marked the dissected cell as indoor and outdoor labels, and determined walls by searching the adjacent edge of the remaining cell. Tang et al. [34] used deep learning to classify indoor 3D point clouds, determined the semantics of indoor space based on morphological methods, determined the value of cells in cell complexes using classification semantics, and extracted the parameter information of wall objects to achieve the construction of a BIM model. Because the definition of wall entities surrounded by boundary surfaces was not strictly considered, the extraction of solid wall parameters was inaccurate.

Above all, the as-built world can be captured by UAVs and laser scanning, which can then be used as part of the scan-to-BIM workflow. Extracting wall object parameter information from oblique photographic models and laser point clouds can meet the needs of solid wall modeling. The existing wall surface modeling method based on cell complexes can be expanded to meet the wall solid modeling requirements under the condition of noise and missing data. The topology between wall objects is not taken into account in most current studies, which highlights the need for additional research on topological reconstruction between wall objects.

3. Methods

The input of this study is a photogrammetric mesh and 3D laser point cloud, which are pre-aligned accurately. There are two methods for the coordinate registration of the UAV photogrammetric model and 3D laser point cloud: (1) using control points to improve the accuracy of UAV photogrammetry and laser scanning point clouds, respectively, and ensuring that the two are in the same coordinate system; and (2) using control points to improve the accuracy of the UAV image and selecting marker points from the UAV image as control points to improve the accuracy of the indoor laser point cloud. The basic idea of both methods is to transfer external spatial reference information into indoor space. Obtaining real-world scene data with a high precision is necessary for BIM modeling in the ACE field. The measurement coordinate system in this study adopts the Transverse Mercator projection [35], and the central longitude is the central longitude of the research area. The outdoor UAV photogrammetric model and the indoor laser scanning point cloud are in the same coordinate system. The input laser point cloud completely covered the ceiling area of the building interior, which is easy to meet for most laser scanners.

The flowchart of the proposed method is shown in Figure 1. It contains six steps: (1) line feature extraction, (2) regularization and fusion, (3) kinetic space partitioning and optimization, (4) wall centerline extraction, (5) wall refinement and topology reconstruction, and (6) solid wall reconstruction.

3.1. Line Feature Extraction

First, the contour-based line extraction method [36,37] is used to extract line features from the oblique photogrammetric surface mesh and 3D laser point cloud. For the photogrammetric mesh model, the relative ground elevation h_s of the cutting plane is given along the Z axis direction. The intersection of the triangle in the mesh and the cutting plane is used to obtain the intersection line, and the linear segment features of the building outline are clustered using the region growth algorithm.

For the input 3D point cloud data, along the Z axis, the given slice thickness is ${\mathsf{\sigma }}_s$. The input point cloud is segmented by a set of parallel planes, and the layered point cloud is projected onto the corresponding reference plane to form a contour point cloud slice. For a layered point cloud, its middle position is defined as the projection reference plane of the layer, and all data points in the layer are projected to the reference plane to form a point cloud slice P′. Then, line features are extracted from the segmented slice point cloud P′. The efficiency of the direct point-based calculation is low, while rasterization can not only improve the efficiency of the calculation but also ensure sufficient accuracy when the pixel size is sufficiently small. Common image edge-detection algorithms (such as the LSD algorithm [32]) use gradient information to calculate the edge information. However, for the rasterized point cloud slice, the centerline of the slice cannot be accurately extracted using edge gradient information, which will lead to a reduction in the accuracy of the extracted line feature. Therefore, in this study, a line feature detection algorithm based on eigenvector calculations (E-LSD) is proposed. This algorithm can accurately extract slice line features by replacing the gradient direction with the eigenvector direction. This method uses a small pixel size to ensure the geometric accuracy of the extracted line segment (usually the pixel size is set to 0.01–0.02 m). Algorithm 1 is a line segment detection algorithm based on the eigenvector proposed in this paper. The algorithm adopts the same region growing and line fitting strategy based on rectangle estimation as LSD algorithm [32], and the angle threshold ${\mathsf{\theta }}_{\mathrm{s}}$ used in the region growing algorithm is usually an empirical value. Due to the existence of measure errors of direction vector, the angle threshold usually adopts 10 to 30 degrees.

Algorithm 1. E-LSD

input: point cloud $P_{\tau }$.

output: list of rectangles.

${\mathrm{M}}_{\mathrm{occ}}\leftarrow \varnothing$; // 3D occupancy probability grid

P′$\leftarrow \varnothing$; // 2D slice point cloud

${\mathrm{P}}^{\prime }=\mathrm{ProjectTo}2\mathrm{D}\left({\mathrm{P}}_{\mathsf{\tau }}\right);$ // set the z value to 0 for each point

${\mathrm{M}}_{\mathrm{occ}}$= 2D_OPG(${\mathrm{P}}^{\prime }$); // calculate 2D coordinates of occupied and free pixels in OPG

Status(allpixels in ${\mathrm{M}}_{\mathrm{occ}}$) = NotUsed;

DensityAndLinearity(allpixels);

for each ($\mathrm{p}\in \mathrm{allpixels}$)

if Status(p) = NotUsed then

region = RegionGrow($\mathrm{p},{\mathsf{\theta }}_{\mathrm{s}},\mathrm{Status}$);

Rect = RectApprox(region);

end if

end for

3.2. Regularization and Fusion

Due to noise and missing data in the point cloud, the line features extracted from the oblique photographic mesh and point cloud may contain a significant number of redundant and broken lines. In addition, the extracted line features do not take into account the constraint that the surface of a wall typically appears as a pair of parallel faces. Therefore, in this paper, the line feature regularization method [38] is used to jointly regularize the two types of line features; then, the two types of line features are separately fused. With the combination of the line features of the building outline and line features of the building interior, parallel constraints are first added, and then the extracted redundant lines and broken lines are fused to reduce the data amount of extracting line features. The polar coordinates are used to represent linear segments; taking the x-axis as the polar axis and denoting the quadrant angle of each line segment as ${\mathrm{x}}_{\mathrm{i}}^{ }$, the relative angle between Line Segment ${\mathrm{L}}_{\mathrm{i}}^{ }$ and Line Segment ${\mathrm{L}}_{\mathrm{j}}$ in Neighbor ${\mathrm{M}}_{\mathrm{i}}$ is ${\mathsf{\alpha }}_{\mathrm{ij}}$. Let ${\mathrm{I}}_{\mathrm{i}}^{ }$ denote the quadrant angle of Line Segment ${\mathrm{L}}_{\mathrm{i}}^{ }$ and let ${\mathrm{I}}_{\mathrm{i}}^{\prime }$ denote the quadrant angle after reorientation; then, the quantity to be added to the initial orientation is $\mathsf{\Delta }{\mathrm{x}}_{\mathrm{i}}={\mathrm{I}}_{\mathrm{i}}^{ }-{\mathrm{I}}_{\mathrm{i}}^{\prime }, \mathsf{\Delta }{\mathrm{x}}_{\mathrm{i}}\in \left[-{\mathsf{\theta }}_{\max },{\mathsf{\theta }}_{\max }\right]$, and the object function for line-segment reorientation is:

$$U\left(\mathsf{\Delta }x\right)=\left(1-\lambda \right)D\left(x\right)+\lambda V \left(x\right)$$

(1)

$$D\left(\mathsf{\Delta }x\right)=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(\frac{x_i^{ }-x_i^{\prime }}{{\theta }_{max}}\right)}^2$$

(2)

$$V\left(\mathsf{\Delta }x\right)={\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{j>i}\frac{\left|{\theta }_{ij}^{ }-\left(x_i^{ }-x_i^{ }{}^{\prime }-x_j^{ }+x_j^{\prime }\right)\right|}{4{\theta }_{max}}$$

(3)

when ${\mathsf{\alpha }}_{\mathrm{ij}}$ approaches 0° or 180°, ${\mathsf{\theta }}_{\mathrm{ij}}^{ }={\mathsf{\alpha }}_{\mathrm{ij}}$. Approximately parallel line segments are corrected to become strictly parallel through $\mathrm{V}\left(\mathrm{x}\right)$. By Parameter $\mathsf{\lambda }\in \left[0,1\right]$ is a weight coefficient balancing the contributions of $\mathrm{D}\left(\mathrm{x}\right)$ and $\mathrm{V}\left(\mathrm{x}\right)$, and its default value is 0.8. The $\mathrm{U}\left(\mathrm{x}\right)$ objective function is minimized by solving the convex quadratic programming.

Continuing with the realignment of line segments, the expression of the straight line is still represented by a polar coordinate. With $\mathsf{\Delta }{\mathrm{x}}_{\mathrm{i}}\in \left[-{\mathrm{d}}_{\max },{\mathrm{d}}_{\max }\right]$ denoting the translation to be added to Line Segment i along its orthogonal vector, the formulations of $\mathrm{D}\left(\mathrm{x}\right)$ and $\mathrm{V}\left(\mathrm{x}\right)$ in Equation (1) are as follows:

$$D\left(\mathsf{\Delta }x\right)=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(\frac{x_i^{ }-x_i^{\prime }}{d_{max}}\right)}^2$$

(4)

$$V\left(\mathsf{\Delta }x\right)={\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{j>i}\frac{\left|d_{ij}^{ }-\left(x_i^{ }-x_i^{ }{}^{\prime }-x_j^{ }+x_j^{\prime }\right)\right|}{4d_{max}}$$

(5)

After reorientation and realignment, a new line-segment collection $L^{\prime }=\left\{L_1^{\prime },L_2^{\prime },\dots ,L_n^{\prime }\right\}$ is obtained. An example for line segment reorientation, realignment and line segment fusion is shown in Figure 2.

3.3. Kinetic Partitioning and Optimization

Architecture space partitioning and optimization mainly includes three parts: (1) voxel-based semantic classification of indoor space, (2) kinetic space partitioning, and (3) graph cut-based optimization.

The voxel-based semantic segmentation of indoor space first initializes the discrete 3D occupancy probability grid (OPG) and sets the value of each voxel as free. Then, the voxels containing the points are marked as occupied values. The last step is to mark the unknown area. The algorithm initializes the boundary (the maximum value of the point cloud elevation range), sets each voxel as unknown, and traverses the voxels layer by layer according to the z-axis direction. When the voxel value of the current layer in the occupancy probability grid is the same as that of the previous layer, the value is marked as unknown; if the voxel value of the current layer is different from that of the previous layer, the traversal stops. This marks all unknown areas. As shown in Figure 3, the returned occupancy probability grid contains S_voxel = {Free = 0, Occupied = 1, Unknown = −1}. Then, the 3D room segmentation method [11] utilizes the volumetric representation and sphere packing of indoor space to separate free indoor space. Similarly, the occupancy probability grid for photogrammetric mesh contains S_voxel = {Inside = 0, Occupied = 1, Outside = −1}. The Algorithm 2 is used to refine the classification of inside space. The function 3D_OPG calculates the evidence that each voxel is occupied, free, or of unknown status. The function VDB_EDT is an implementation of a distance transform algorithm; ${\mathrm{M}}_{\mathrm{edt}}$ represents the distance from each grid voxel to the closest obstacle. The SpherePacking function packs the indoor space with a set of compact spheres to form rooms as separated connected components. Each connected component is treated as an initial seed room region and the wavefront growing algorithm is used to mark unlabeled voxels to obtain the final classification for inside space or indoor free space.

Algorithm 2 Semantic Labeling of Inside Space or Indoor Free Space

input: P: photogrammetric mesh or indoor laser point cloud; ${\mathrm{s}}_{\mathit{voxel}}$: the voxel size of the grid map; ${\mathsf{\delta }}_{\mathrm{overlap}}$: overlap ratio threshold for two spheres; $r$: minimum room volume threshold

output: ${\mathrm{M}}_{\mathrm{label}}$: voxel-based grid with semantic labels

${\mathrm{M}}_{\mathrm{opg}}\leftarrow \varnothing$; // 3D occupancy probability grid

${\mathrm{M}}_{\mathrm{edt}}\leftarrow \varnothing$; // grid for Euclidean distance transform (EDT)

${\mathrm{M}}_{\mathrm{seed}}\leftarrow \varnothing$; //initial seed regions

${\mathrm{M}}_{\mathrm{label}}\leftarrow \varnothing$; // inside space or indoor free space

(1) ${\mathrm{M}}_{\mathrm{opg}}=$ 3D_OPG$\left(\mathrm{P},{\mathrm{s}}_{\mathrm{voxel}}\right)$; (2) ${\mathrm{M}}_{\mathrm{edt}}=$ VDB_EDT$\left({\mathrm{M}}_{\mathrm{occupied}}\right)$; // EDT (3) $\mathrm{R}=$ SpherePacking$\left({\mathrm{M}}_{\mathrm{edt}},{ \mathsf{\delta }}_{\mathrm{overlap}}\right)$; (4) ${\mathrm{M}}_{\mathrm{seed}}$ = InitialSeedRoom$\left({\mathrm{M}}_{\mathrm{edt}}, \mathrm{R}, \mathrm{r}\right)$; (5) ${\mathrm{M}}_{\mathrm{label}}=$ WavefrontGrowing$\left({\mathrm{M}}_{\mathrm{seed}}\right)$;

After the regularization and fusion steps are performed, the building outline segment and interior line segments are regularized jointly, and the two regularized segment sets are then fused separately. The building outline line and interior line segments are used as inputs for the kinetic space partitioning algorithm. A comparison between kinetic space partitioning and the traditional space partitioning algorithm has previously been discussed in the papers [39,40]. The algorithm based on kinetic space partitioning has obvious advantages over the traditional two-dimensional space partitioning method. The kinetic space partition algorithm is faster, and the constructed cell complex is simpler.

The kinetic space partitioning algorithm achieves the partitioning of the building topological space, thus transforming building space modeling into a problem of labeling the cell complex (Figure 4). The cell complex is expressed by the graph G = 〈V, E〉. With the source point s and sink point t, the node V corresponds to the polygon element of the cell complex, the undirected edge E corresponds to the adjacency relationship of the cells, and each cell is marked as free and non-free. Cut C of Graph G can divide Node V into two disjoint subsets A and B. The cost of Cut $\mathrm{C}=\left\{\mathrm{A}, \mathrm{B}\right\}$ is defined as the sum of the weights of all edges on the connecting boundary of two subsets A and B. The minimum cut is the cut with the lowest cost among all cuts of Graph G. The energy function is defined as:

$$E=W^{\left(n\right)}+W^{\left(t\right)}$$

(6)

where ${\mathrm{W}}^{\left(\mathrm{n}\right)}$ is the weight between vertex nodes and ${\mathrm{W}}^{\left(\mathrm{t}\right)}$ is the weight between a vertex and Terminals s and t.

$$W^{\left(n\right)}={{\displaystyle \sum }}_{\left\{p,q\in V\right\}}{w}_{〈p,q〉}={{\displaystyle \sum }}_{\left\{p,q\in V\right\}}{e}^{-\frac{{\left(c_p-c_q\right)}^2}{2{\delta }^2}}$$

(7)

where $\mathrm{p}$ and $\mathrm{q}$ are adjacent cells, ${\mathrm{w}}_{〈\mathrm{p},\mathrm{q}〉}$ is the weight between $\mathrm{p}$ and $\mathrm{q}$, and ${\mathrm{c}}_{\mathrm{p}}$ is the value of Cell $\mathrm{p}$. Therefore, Equation (7) can be interpreted as a discontinuity penalty between Cells $\mathrm{p}$ and $\mathrm{q}$. If $\mathrm{p}$ and $\mathrm{q}$ are more similar, then ${\mathrm{w}}_{〈\mathrm{p},\mathrm{q}〉}$ is larger. In contrast, if the difference between $\mathrm{p}$ and $\mathrm{q}$ is greater, then ${\mathrm{w}}_{〈\mathrm{p},\mathrm{q}〉}$ is smaller and close to 0. The two cells with large value gaps have very small weights, so they are the cells that need to be divided. After completing the cell-complex marking, the cell units are merged with the same label to obtain Polygon Q1 of the building outline and the indoor floor plan Q2.

3.4. Wall Centerline Extraction

In this step, the building footprint Q1 and indoor floor plan Q2 are taken as inputs. We search the wall line segments in Q2 that are parallel to the line segments in Q1 with distances less than ε. The wall line segments in Q2 are classified as exterior walls and interior walls. The wall line segments in Q2 are broken when the line contains both exterior and interior parts. All the wall lines in Q2 adjacent to the outside of Q1 are extracted as exterior wall line segments, and the rest are used as interior wall line segments. To extract the wall centerline, the parallel line pairs are extracted, and the interior wall centerlines are then extracted. The algorithm calculates the middle points of all wall line segments, and constructs Delaunay triangulation (DT) for the middle point set. For each line segment i, the nearest neighbor of each line segment is obtained by searching the neighbor points in DT, and then the nearest neighborhood M_i is constructed. We denote the angle between Line Segment i and Line Segment j as $\mathsf{\theta }$. If the angle is less than ${\mathsf{\theta }}_{max}$ and the distance between Line Segment i and Line Segment j is less than the threshold $\mathsf{\epsilon }$, the two line segments are parallel. For a straight wall, the centerline of the interior wall can be calculated using the median of the vertical intercept in the straight-line equation. The thickness of each wall is estimated by calculating the orthogonal distance between two parallel lines (Figure 5). If the outline of Q1 has no adjacent line segments in Q2, the most common thickness among the exterior walls is used for it.

For the case in which only an indoor laser point cloud is available, the user inputs the thickness of the exterior walls for modeling. Because the thickness of the exterior wall components cannot be determined from single exterior wall surface line segments, the exterior wall thickness ${\mathrm{t}}_1$ is assumed to be the most common exterior wall thickness. A dilate operator is performed for Q2 to obtain building footprint Q1. The wall line segments in Q2 are classified as exterior walls and interior walls. The centerlines of the exterior walls and interior walls are extracted by searching the parallel line pairs and calculating the median of the vertical intercept of the straight-line equation.

3.5. Wall Refinement and Topology Reconstruction

Because this method depends on the unit characteristics of the interior space, the same wall may be divided into different sections because it adjoins multiple rooms. The cases where the wall centerline needs to be merged are divided into three cases: adjacent and collinear, vertical, and intersecting. As shown in Figure 6, the walls with “T”, pseudo “T”, and the cross shapes in space are merged according to the primary and secondary principles to obtain the final wall object centerline parameters. The specific method is as follows: the center points of all wall centerline segments are obtained, and the DT is constructed.

The centerline i of each wall segment is traversed, the nearest neighbor of each wall centerline is obtained by searching the neighbor points in the DT, and the nearest neighborhood M_i is constructed. For each line segment i, the angle between line segment i and its nearest neighbor j is calculated. If the angle is less than 5° and the distance between Line Segment i and Line Segment j is less than the threshold ${\mathsf{\sigma }}_1$, the two segments are merged. The neighborhood M_i is updated in this way until all segments have been processed.

The center points of all updated wall centerline segments are again obtained to build the DT. For each line segment i, the angle between line segment i and its nearest neighbor j is calculated. If the angle is 90° and the distance between the end points of Line Segment i and Line Segment j is less than the threshold ${\mathsf{\sigma }}_1$, then the two line segments are perpendicular. If the angle is less than 45° and the distance between the end points of Line Segment i and Line Segment j is less than the threshold ${\mathsf{\sigma }}_1$, then the two line segments are considered to intersect. The neighborhood M_i is updated until all segments have been processed.

3.6. Solid Wall Reconstruction

After topological reconstruction, the topological map of the wall centerline is still maintained. Wall reconstruction depends on the elevation, wall type, and height parameters (Figure 7). The wall elevation is detected based on the Z value of the floor. A histogram method [32] is used to obtain the Z value of floor, and the average height from floor to ceiling. The elevations of all walls are sorted, and the level set H is extracted by Gaussian clustering [41]. The wall type is characterized in accordance with the wall thickness, material, and other parameters. To build the BIM solid wall object, each wall centerline is traversed, and the corresponding wall type and level parameters are selected as the input to the parametric BIM wall generation function. The topological connections are built between the walls, and between the walls and rooms. Moreover, the required reference information is added. The room boundary surface parameters are adjusted, the associated wall objects are considered, and the room object is rebuilt accordingly.

4. Results

To verify the effectiveness of the proposed method, two real-world datasets were used for the experiment (

Table 1. Detailed descriptions of the experimental datasets.

Test Site	Type	Size (m)	Points	Triangles	Altitude
Dataset A	Mesh	84.3 × 75.2 × 28.3	2.63 × 105	7.03 × 105	13.5 m
	Point Cloud	76.4 × 68.0 × 8.6	9.09 × 107	-
Dataset B	Mesh	62.8 × 18.5 × 22.3	1.17 × 105	4.63 × 105	13.4 m
	Point Cloud	56.8 × 17.6 × 4.5	2.81 × 107	-

). A DJI Phantom 4 RTK was used to capture UAV images of the building of interest from the air using an oblique angle of view. The control points were uniformly arranged in the survey area to ensure measurement accuracy. A Trimble TX8 scanner was used to obtain the laser scanning point cloud in an indoor environment, and targets are used to assist the registration of adjacent stations. A total station was used to capture the targets’ coordinates in the indoor environment. To meet the needs of coordinate registration between the UAV photogrammetric meshes and laser scanning point clouds, a control point layout was designed for UAV photogrammetry and laser scanning in the same coordinate system. Because multiple floors can usually be separated into single floors for processing and the floors are similar, only one floor of laser point cloud data was used in the experiment (Figure 8). For both datasets, the flight altitude is approximately 50 m, the forward overlap rate is 80%, the side overlap rate is 70%, and the flight speed is 4 m/s. The ground resolution of the image is 0.02 m. The Trimble TX8 captures data at one million points per second with a typical scan. The original point clouds are subsampled with a minimal distance of 5 mm between two points. For Dataset A, 11 photo-control points (5 as checkpoints) were set up and 6 target coordinates were captured. For Dataset B, nine photo-control points (four as checkpoints) were set up and six target coordinates were captured. For the photo-control point, CGS2000 coordinate system and Gauss three-dimensional zone projection were adopted; the central longitude is 121°, and the elevation system is the 1985 National Elevation Datum. The reference background solid walls of the datasets were manually obtained. The maximum-overlap metric was used to find the correspondences between the reference and the results. The reference model’s solid walls were modeled as cellular spaces to guarantee one-to-one correspondences, which is in line with the specification of the IndoorGML standard [42]. The completeness, correctness, and quality index are used to evaluate the accuracy of wall object detection:

$$Completeness=\frac{TP}{TP+FN}$$

(8)

$$Correctness=\frac{TP}{TP+FP}$$

(9)

$$Quality=\frac{TP}{TP+FN+FP}$$

(10)

Here, the number of true positive ($\mathrm{TP}$) refers to the number of solid walls detected in both the reference data and the reconstruction results. The number of false positive ($\mathrm{FP}$) is the number of detected solid walls not found in the reference data. The number of false negatives ($\mathrm{FN}$) is the number of undetected ground-truth solid walls. The accuracy of the wall surfaces is evaluated using ${\mathrm{M}}_{\mathrm{Acc}}$ [43,44], which is defined as:

$$M_{Acc}=Med\Vert {\pi }_j^T{p}_i\Vert , if \Vert {\pi }_j^T{p}_i\Vert \le r$$

(11)

where $\Vert {\mathsf{\pi }}_{\mathrm{j}}^{\mathrm{T}}{\mathrm{p}}_{\mathrm{i}}\Vert$ is the vertical distance between Vertex ${\mathrm{p}}_{\mathrm{i}}$ in the source model and Plane π in the reference model; and $\mathrm{r}$ is the truncation threshold that is used to prevent the influence of an incomplete or inaccurate source model. To determine how far off the closest surfaces in the reference model are from the sample points representing the source surfaces, the $\mathrm{Med}$ function is used to compute the median Euclidean distance between them. Relatively high completeness and low correctness scores mean that the reconstructed models contain most of the elements that are present in the corresponding reference models but that they also include a considerable number of incorrect facets.

The algorithm in this study is implemented by using C++, Computational Geometry Algorithm Library (CGAL) [45] and CloudCompare [46]. The BIM object generation plug-in program is developed based on C# and Revit API. All experiments use an Intel Core i7-10750H CPU (2.60 Hz) and 32 GB RAM.

Table 2. Input parameters for the experiment.

Parameters	Description	Dataset A	Dataset B
Line Feature Extraction
${\theta }_s$	Angle threshold. (Mesh/Point Cloud)	22.5°	22.5°
$h_s$	Elevation relative to ground. (Mesh)	2 m	2 m
${ \mathrm{s}}_{\mathrm{f}}$	Pixel size of image. (Point Cloud)	0.015 m	0.015 m
$t$	Thickness of slice. (Point Cloud)	0.1 m	0.1 m
$\mathsf{\tau }$	Number of points in voxel. (Point Cloud)	5	5
Regularization and Fusion
${\mathsf{\theta }}_{max}$	Parallel line angle threshold. (default 5°).	3°	3°
${\mathrm{d}}_{max}$	Parallel line fusion distance.	0.025 m	0.025 m
$\mathsf{\Delta }\mathrm{L}$	Broken lines distance threshold.	0.1 m	0.1 m
Kinetic Partition and Optimization
${\mathrm{s}}_{\mathrm{voxel}}$	Voxel size of a grid map. (Mesh)	0.25 m	0.25 m
$s_{\mathrm{voxel}}^{\prime }$	Voxel size of a grid map. (Point Cloud)	0.05 m	0.05 m
${\delta }_{overlap}$	The overlap ratio between two spheres.	0.8	0.8
r	Truncation threshold	0.1 m	0.1 m
Wall Centerline Extraction
$\mathsf{\epsilon }$	Parallel line distance	0.25 m	0.25 m
${\mathrm{t}}_1$	User input wall thickness	0.3 m	0.3 m
Wall Refinement and Topology Reconstruction
${\mathsf{\sigma }}_1$	Search radius for neighbors	0.25 m	0.25 m

shows the input parameters for the experiment.

The line features of the UAV oblique photogrammetric mesh and laser scanning point cloud are extracted first. In Dataset A, the angle threshold is set to 22.5°, and the relative ground elevation is 2 m for mesh line feature extraction. To extract line features from the 3D point cloud, an image pixel size of 0.015 m and a slice thickness of 0.1 m were applied. Then, the extracted two types of the line-segment features are combined and regularized together with a parallel line fusion distance of 0.025 m and an angle threshold of 3°. The line features extracted by the mesh and indoor line features are fused separately to reduce broken lines and redundant parallel line segments with a distance threshold of 0.1 m. The extracted regular line-segment feature results are shown in Figure 9a.

To extract line features from photogrammetric mesh on Dataset B, an angle threshold of 22.5° and a relative ground elevation of 2 m were used. An image pixel size of 0.015 m and a slice thickness of 0.1 m were applied for line feature extraction from the 3D point cloud. Then, the extracted two types of line features are combined and regularized together. In the experiment, the parallel line fusion distance is set to 0.025 m, and the parallel line angle threshold is 3°. The line features extracted from the photogrammetric mesh and indoor point cloud are fused, and the distance threshold is 0.1 m. The extracted regular line feature results on Dataset B are shown in Figure 9b.

The voxel-based semantic segmentation of the indoor space was applied for both the photogrammetric mesh and indoor point cloud. On Dataset A, the voxel size is set to 0.25 m for the oblique photogrammetric mesh and 0.05 m for the indoor point cloud. For the input indoor laser scanning point cloud, the voxel setting of semantic markers is usually smaller than the wall thickness. However, too small of a voxel size will reduce the program execution efficiency.

The kinetic space partitioning algorithm is used to carry out spatial partitioning of the regularized building outline features, and the results are shown in Figure 10a. The graph cut optimization algorithm is used to classify the cell complex, and the cells with the same semantic labels are merged to obtain the building footprint polygon. Similarly, the space partition algorithm is used to decompose the indoor space with regularized indoor line segments extracted from indoor point clouds. The resulting cell complex is shown in Figure 10c. The graph cut optimization algorithm is used to obtain the optimum floor plan, as shown in Figure 10d.

In Dataset B, the voxel size is set to 0.25 m for the oblique photogrammetric mesh and 0.05 m for the indoor point cloud. The resulting cell complex of the space partition result and building footprint are shown in Figure 11a,b. For the indoor laser scanning point cloud, the generated cell complex is shown in Figure 11c. The graph cut optimization algorithm classified the cell complex, and the resulting room floor plan is shown in Figure 11d.

To extract wall centerlines, a search radius of 0.025 m is applied for Dataset A. It searches the line segments of the exterior walls from the indoor floor plan, as shown in the blue contour line in Figure 12b. The parallel lines were classified as exterior walls and interior walls. The extracted wall centerlines are shown in green in Figure 12b. The wall thickness parameters are then extracted from parallel line segments. The most common exterior wall thickness is used for single line segments, shown in pink. The histogram method is used to obtain the average height from the floor to the ceiling. Two peaks in the histogram correspond to the level and height of the wall. The building footprint generated from Dataset B overlapped with the indoor floor plan in Figure 13c, and the exterior wall centerlines extracted from Dataset B are colored blue in Figure 13b.

In Dataset A, a total of 51 exterior wall lines and 44 interior wall lines are in the background. The correctness of the exterior wall extraction is 98.0%, and the completeness is 96.2%. As seen in

Table 3. Results of solid wall modeling.

Objects	Indexes	Dataset A	Dataset A (Indoor Point Cloud Only)	Dataset B	Dataset B (Indoor Point Cloud Only)
Exterior wall/interior wall	Background	51/44	51/44	10/18	10/18
Exterior wall	TP/FP/FN	50/1/2	18/25/33	10/0/0	8/6/2
	Correctness	98.0%	41.9%	100%	57.1%
	Completeness	96.2%	35.3%	100%	80.0%
	Quality	94.3%	23.7%	100%	50.0%
Interior wall	TP/FP/FN	43/3/1	26/3/18	18/0/0	15/0/3
	Correctness	93.5%	89.7%	100%	100.0%
	Completeness	97.7%	59.1%	100%	83.3%
	Quality	91.5%	55.3%	100%	83.3%
Wall surface	${\mathrm{M}}_{Acc}$@10 cm	2.704 cm	-	2.023 cm	-
Levels	Number	3	-	1	-
Wall Types	Number	5	-	3	-

, the correctness and completeness in the experiment using both photogrammetric mesh and laser point clouds are significantly improved when compared with the experiment using a single laser point cloud data source. The wall surface accuracy (10 cm cutoff threshold) is 2.705 cm. The level set contains three levels, the wall thickness parameters were clustered, and five different wall types were built. The results of the experimental solid wall modeling are shown in Figure 14.

In Dataset B, a total of 10 exterior wall lines and 18 interior wall lines are in the background. Table 3 shows that the correctness of exterior wall extraction is 100%, and the completeness is 100%. The wall surface accuracy (10 cm cutoff threshold) is 2.023 cm (Figure 15). The level set contains only one level, and three wall types were generated. The external program for automatically building the BIM model is developed depending on the Revit API, and the BIM object is generated according to the wall parameters.

5. Discussion

In this study, the construction of BIM solid walls by the combination of UAV oblique photogrammetric models and laser point clouds is discussed. When only indoor laser point clouds are used, the buffer method can be used to determine the exterior wall of the building. However, it classifies many interior walls into exterior walls, resulting in reduced accuracy.

The proposed method has significant advantages in reconstructing buildings from point clouds with heavy occlusions and missing data. To illustrate this, a comparison of the reconstruction result of Bassier’s method [17] (Figure 16a) and the proposed method (Figure 16b) is presented. Bassier’s method is a wall-based method; it cannot ensure watertight modeling of room spaces. Our method has two basic demands for the input data sources. The kinetic space partitioning algorithm typically requires at least one observation from every wall to create proper cells. It can cope with photogrammetric mesh or point cloud data with holes distributed on the wall. However, if the holes are too big to ensure that at least one wall line segment can be extracted, the proposed method may fail. The other demand is that the roof of the photogrammetric mesh and the ceiling of the indoor point cloud must perfectly cover the region to be modeled. If holes exist in the roof or ceiling region (Figure 16c), the initial labeling information for the cell complex will be incorrect.

In the experiment, some rooms did not obtain the required data because they were not licensed. Therefore, this method compromises and uses the extracted wall thickness as the default value for these walls. BIM has strict requirements to achieve a high accuracy of the model, and the UAV photogrammetric mesh may not fulfill the requirement of BIM modeling in some cases. However, the proposed method provides a framework for BIM modeling by fusing different data sources. Moreover, the modeling of doors and windows are not further considered in this study. After building the BIM wall objects, the structural elements and topological relationships of doors and windows can be reconstructed.

6. Conclusions

In this study, a BIM solid wall modeling method based on both photogrammetric meshes and laser point clouds is proposed. This method extracts the centerline of a solid wall based on a kinetic space partitioning algorithm and morphological operation and constructs the geometric and topological associations of wall objects through wall fusion and topological reconstruction. A solid wall object is constructed by using the wall centerline, level, height and wall thickness. Experimental results exhibit more than 90% completeness and correctness for solid walls. The proposed method also achieves centimeter-level accuracy for the wall surface. The method can quickly construct the geometric and topological relationships of BIM solid walls and meet the needs of building mapping and building 3D reconstruction.

However, the proposed fusion method for photogrammetric meshes and laser point clouds is still a loosely coupled method. It would be ideal to have a faster and more compact technique. Further work will also address the modeling of the geometric and topological relationships of doors and windows.