Evaluation of Automatic Building Detection Approaches Combining High Resolution Images and LiDAR Data

Abstract

In this paper, two main approaches for automatic building detection and localization using high spatial resolution imagery and LiDAR data are compared and evaluated: thresholding-based and object-based classification. The thresholding-based approach is founded on the establishment of two threshold values: one refers to the minimum height to be considered as building, defined using the LiDAR data, and the other refers to the presence of vegetation, which is defined according to the spectral response. The other approach follows the standard scheme of object-based image classification: segmentation, feature extraction and selection, and classification, here performed using decision trees. In addition, the effect of the inclusion in the building detection process of contextual relations with the shadows is evaluated. Quality assessment is performed at two different levels: area and object. Area-level evaluates the building delineation performance, whereas object-level assesses the accuracy in the spatial location of individual buildings. The results obtained show a high efficiency of the evaluated methods for building detection techniques, in particular the thresholding-based approach, when the parameters are properly adjusted and adapted to the type of urban landscape considered.

1. Introduction

The evolution and dynamism of the landscape and land uses is particularly noticeable in urban and peri-urban environments, due to the construction of new infrastructures, houses and buildings. Maintenance of urban cartographic databases requires a reliable and systematic methodology for building detection. Traditionally, this specific task has been carried out by photointerpretation, which is expensive and time consuming. The abundance of high resolution multi-spectral imagery and the increasing availability of LiDAR data have influenced combining these data sources and the development of digital image processing methods for automatic building detection and delineation. Several approaches have been reported in a large number of studies, yielding different degrees of accuracy. Main problematic aspects related to the early building detection methodologies were deeply analyzed by Mayer [1], who quoted several relevant factors that need to be considered when designing an object detection model. After the examination of a large number of methodologies, the main problems pointed out by this author were: a lack of a deep analysis of the contextual relations, an inadequate treatment of data variability, and an absence of a critical statistical evaluation of the results obtained. Dependent on the data source employed, building detection techniques can be classified in three groups: (i) using airborne or satellite imagery; (ii) using three-dimensional information; and (iii) combining both data sources.

Aerial images have traditionally been used to extract buildings for mapping applications. With the successive launching of high spatial resolution commercial satellites, high-resolution satellite imagery has become a cost-effective alternative to aerial photography for several applications [2]. Automatic building detection by means of high resolution imagery presents difficulties due to geometric reasons, buildings can constitute complex structures that create abrupt height discontinuities, occlusions, shadows [3], and radiometric reasons, as different material combinations can create a variety of intensity values in the spectral bands employed [4]. Two main groups of building detection techniques using high resolution imagery can be considered [3]: low-level and high-level vision techniques. Low-level vision techniques are mainly based on edge detection and extraction from images, followed by processes of definition of rules and hypothesis in order to identify the buildings. High-level vision techniques try to imitate the human cognition process and decision making skills which are based on the analysis of the information. Pattern and object recognition, and image classification are common high-level vision techniques.

Since buildings grouped in high-density urban or industrial areas usually have regular patterns and directionality, rather than a random distribution, low-level vision techniques have been initially considered as more adequate and straightforward to face the problem of building detection [5]. However, many of the low-level vision techniques are strongly restricted, making a priori assumptions such as that buildings have rectangular shape, flat roofs or specific spectral responses, or that they are arranged following a particular orientation. Previous research defined a series of rules that buildings should accomplish [6]. Similar approaches were used by [3,7,8,9], who detected edges and analyzed their mutual relationships to define building existence hypothesis. Some authors proposed the use of transforms between image representation spaces, such as Fourier [5] or Hough [10,11]. In general, low-level vision techniques have the advantage of presenting a relatively simple design and low computational cost, but they lack robustness due to their inherent methodological restrictions.

A group of interim approaches could be placed between low and high-level vision techniques, such as those based on the employment of active contour models. Thus, Mayunga et al. [12] present a semiautomatic system that requires the manual definition of a seed point inside each building. Afterwards, Theng [13] fully automates the system by adding an automatic corner detector that initializes the active contour model. Ahmady et al. [14] and Ge et al. [15] initialize the contours using regularly distributed circumferences that progressively merge after some repetition. Ok [16] applies the active contours technique specifically to simplify and merge the detected edges.

Most building detection high-level vision techniques are based on image classification. Although some methodologies have been proposed to detect buildings using pixel-based classifications [17,18,19,20], in the majority of the studies the buildings are considered as objects, and automatic segmentation methods based on image homogeneity are used to create the image-objects. Regarding the classification process, image-objects are mainly characterized using descriptive features based on the spectral response, the image texture, or the shape of the objects [2,21,22,23,24,25,26,27,28], or even using features derived from the wavelet transform [29,30]. Some authors establish contextual relationships between objects and sub-objects by using multi-scale segmentation techniques [31,32]. Others include ancillary data such as road maps [21]. In many cases, standard morphological filtering techniques, or even the Hough transform, are applied at the end of the process to generalize and smooth the shape of those objects classified as buildings [2,33,34,35]. Other different high-level vision techniques have been used to detect buildings, but generally offer less satisfactory results than those based on classification, being usually focused on solving uncommon or particular cases. Thus, L’Homme et al. [36] propose a methodologically simple method based on the extraction of the central positions of the buildings using the variance of the Grey Level Co-occurrence Matrix (GLCM). In the same sense, the methodology presented by [37,38,39,40] assumes that buildings are regularly arranged following a unique orientation, having higher intensity values than the background.

Building detection techniques based on images are generally improved by adding some contextual information, such as shadows. According to [38], the height is a common feature of buildings, and the projected shadows allow defining a context for building detection. In general, authors condition the existence of a building to the proximity or contiguity to shadows, following the direction defined by the solar incidence angle [2,6,21,25,37,38,39,40,41]. In spite of the fact that shadows may have different origins and properties—which makes their detection more difficult [42]—it is frequently easier to extract them than to directly analyze the buildings that cast shadows [6]. In practice, most of the authors use thresholding methods with very positive results, using the panchromatic band [6,26,43,44] or the intensity channel [25]. Some attempts have been made to automatically define the optimum thresholding value. Thus, Scanlan et al. [42] established the threshold value dividing the image in tiles and comparing the mean value from each tile with the median value of the whole image. Tsai [45] recursively divided the image up to each tile presents a bi-modal histogram of grey levels. Other more complex approaches were applied, based on pixel classification [31], differential morphological profile method [37,38,39,40], color spaces transformation [46] or digital models analysis [35].

The problem of automated building detection using only three-dimensional data sources has been the focus of a large number of researches. Two main data sources have usually been employed: photogrammetric restitution and, more recently, airborne laser scanning (ALS). In general, the methodologies proposed are less complex than when only images are used, and very often they are not statistically evaluated, since they constitute a transitional step for the representation of virtual 3D environments. The methodological approaches are dependent on the type of data sources used. Thus, most of the studies that use digital surface models (DSM) obtained from photogrammetric techniques are based on thresholding the normalized DSM (nDSM) to differentiate between buildings and vegetation [47,48,49,50,51,52]. However, when the source of data is ALS, automatic segmentation techniques are normally used, being the resultant objects classified according to their size, shape, height or rugosity [53,54,55,56,57]. Many of the errors produced using this approach are due to the misclassification of vegetation as buildings, and are partially corrected by the application of morphological filters and height or size thresholds. A similar approach is used by [58,59,60]. Other methodologies can be found in [61,62,63,64].

A different group of building detection and location techniques are those based on the combination of imagery and three-dimensional data. The use of two complementary sources of data usually improves the results. Most of these studies are based on two main approaches: object-based classification and thresholding-based detection. In the first approach, image-objects are created using automatic segmentation techniques. Then, the objects are characterized by means of spectral, shape and height features [65,66,67,68,69,70,71,72]. Approaches based on thresholding consist of the application of a threshold value to the nDSM to discriminate buildings and vegetation, combined with a threshold applied to the normalized difference vegetation index (NDVI) image in order to mask the vegetation. Then, buildings can be accepted or rejected according to different conditions, such as size and shape [73,74,75,76], spectral values [77] or texture features [78,79,80]. In addition to these two main approaches, some other specific methods have been reported. Cheng et al. [81] propose a single threshold applied to the nDSM, and then a correction of the shape of the detected building based on alignments extracted from the imagery. Paparoditis [82] offered a low-level vision solution that effectively integrates bi-dimensional and three-dimensional information by detecting and analyzing edges in both data sources. Guo and Yasuoka [83] used height data to initialize active contours, and spectral information to extract the buildings.

In this paper, two adapted versions of the two main approaches to detect and localize buildings based on images and 3D LiDAR data are compared and evaluated. The main objectives of this study are: to critically compare the performance of two building detection approaches, those based on thresholding and those based on object-based classification; and to evaluate the effect of the inclusion of some contextual relations with the shadows on building detection. The evaluation tests are carried out over different urban scenarios, and not only detection, but also location of buildings is considered.

2. Data and Study Area

Three study areas were defined, all in the province of Valencia (Spain), corresponding to the municipalities of Moncada, Oliva and Sagunto (see locations in Figure 1). Moncada is characterized by an urban center surrounded by large areas of suburban neighborhoods occupied by detached and semi-detached houses, together with some industrial areas. Oliva is a coastal touristic town presenting a residential strip with high apartment buildings, detached and semi-detached houses, and traditional village houses. The Sagunto area is divided in two main dense urban centers and several large industrial areas.

Figure 1. Location of the province of Valencia in Spain (left) and distribution of the three study areas (right).

Figure 1. Location of the province of Valencia in Spain (left) and distribution of the three study areas (right).

Table 1 shows some technical specifications of the data used in the tests. QuickBird satellite images were used for Moncada, and aerial orthophotographs acquired as part of the Spanish Programme of Aerial Orthophotography (PNOA) were used for Oliva and Sagunto. LiDAR data were collected using different flights and sensors, and they have a variety of nominal density values ranging from 0.5 to 2 points/m².

Table 1. Data technical details.

Study area	Imagery	LiDAR
Moncada	QuickBird Acquired: February 2004 Spatial resolution: 0.6 m/pixel Spectral bands: IR, R, G, B	Sensor: Optech ALTM 2033 Acquired: December 2003 Density: 1 point/m2
Oliva	Aerial orthophoto Acquired: June 2006 Spatial resolution: 0.5 m/pixel Spectral bands: IR, R, G	Sensor: Optech ALTM 3025 Acquired: September 2009 Density: 2 points/m2
Sagunto	Aerial orthophoto Acquired: June 2006 Spatial resolution: 0.5 m/pixel Spectral bands: IR, R, G	Sensor: RIEGL LMS-Q680 Acquired: August 2009 Density: 0.5 points/m2

Table 1. Data technical details.

Study area	Imagery	LiDAR
Moncada	QuickBird Acquired: February 2004 Spatial resolution: 0.6 m/pixel Spectral bands: IR, R, G, B	Sensor: Optech ALTM 2033 Acquired: December 2003 Density: 1 point/m2
Oliva	Aerial orthophoto Acquired: June 2006 Spatial resolution: 0.5 m/pixel Spectral bands: IR, R, G	Sensor: Optech ALTM 3025 Acquired: September 2009 Density: 2 points/m2
Sagunto	Aerial orthophoto Acquired: June 2006 Spatial resolution: 0.5 m/pixel Spectral bands: IR, R, G	Sensor: RIEGL LMS-Q680 Acquired: August 2009 Density: 0.5 points/m2

A digital terrain model (DTM) was created from the LiDAR data by eliminating points belonging to aboveground objects, such as vegetation or buildings. An iterative algorithm consisting of selecting minimum height points in a series of progressively smaller windows was used. First, a coarse DTM is computed using large window size. Afterwards it is refined adding new minimum height points chosen by using smaller windows. These are compared to the initial DTM and the use of a height threshold allows for removing the non-ground points in each iteration. The methodology used is fully described in [84]. In addition, a DSM was calculated considering the highest points. The difference between DSM and DTM produces the nDSM, which describes the height of the objects aboveground.

3. Methodology

This section describes two adapted approaches for building detection and location when imagery and three-dimensional information is available: thresholding-based and object-based classification. In addition, a shadow detection methodology followed by the definition of contextual relationships is described. This method is used in order to improve the building detection process. At the end of this section, the evaluation metrics employed are described. Quality assessment has been performed at two different levels: area and object. Area-level evaluates the building delineation performance; object-level evaluation assesses accuracy in the spatial location of individual buildings.

3.1. Thresholding-Based Approach

The adapted thresholding-based building detection approach is founded on the establishment of two threshold values: one refers to the minimum height to be considered as a building, applied over the nDSM, and the other refers to the presence of vegetation, which is defined as a minimum value of NDVI obtained from the image bands (Figure 2). The NDVI threshold value can be determined using the trial and error method, or in a semi-automatic manner by collecting samples of the classes: vegetation and non-vegetation. Both classes are parameterized using the mean and standard deviation values, considering a normal distribution of their histograms. The threshold value is defined as the point where the normal curves of both classes are intersected. The binary images produced in the thresholding step are smoothed by means of morphological opening and closing filters, and small objects are eliminated in order to remove noise. As a last step, intersecting the vegetation and height binary images, a mask of detected buildings is produced. Working with the thresholding-based approach, several different variations have been presented in the literature as a last step for building definition, including spectral or texture analysis techniques. For comparison purposes, only the size is considered as final condition for building definition.

Figure 2. Scheme of the thresholding-based building detection approach.

Figure 2. Scheme of the thresholding-based building detection approach.

3.2. Object-Based Classification Approach

The object-based classification approach follows the standard scheme of object-based image classification (Figure 3): segmentation, feature extraction and selection, and classification. In order to create meaningful image objects for building detection purposes, a region growing segmentation method is performed over the nDSM. As a building can present different spectral responses, the addition of spectral information for segmentation would produce an over-segmentation of the image, making the classification difficult, so they were not included for segmentation.

Figure 3. Scheme of the object-based classification building detection approach.

Figure 3. Scheme of the object-based classification building detection approach.

The produced objects were characterized for classification using four different groups of descriptive features: spectral, texture, shape and height; computed by means of the object-based image analysis software, FETEX 2.0 [85]. Spectral features provide information about the intensity values of the objects in the different spectral bands. Texture features provide information about the spatial distribution of the intensity values in the object analyzed. Histogram-based features (kurtosis and skewness) and texture features from the grey level co-occurrence matrix (GLCM) [86] were extracted, as well as the mean and the standard deviation of the edgeness factor [87], representing the density of edges in the neighborhood of a pixel. Shape features provide information about the dimensions of the objects and their contour complexity. These have been described by means of area, perimeter, compactness, shape index and fractal dimension features. Height features provide three-dimensional information about the parcel: the average, maximum and standard deviation values of the height, derived from the nDSM, were computed. Due to the large number of parameters defined (27), some features may be redundant in terms of efficient description of the objects, and the inclusion of these parameters can introduce noise in the classification. Therefore, as a first step, the study of the relationships between the features and their contribution to the classification accuracy is essential. A pre-selection was made to discard the use of correlated variables, by analyzing the correlation values and by grouping the variables using principal components analysis, helping to understand the links between variables in terms of redundancies. Then, stepwise linear discriminant analysis was used to determine the significance of the remaining features for the particular classification problem. This was done independently for each study area.

Classification was performed using decision trees built using C5.0 algorithm [88] and the boosting multi-classifier method [89]. This algorithm searches the features that best separate one class from the others by dividing data using mutually exclusive conditions, until the new generated subgroups are homogeneous, i.e., all the elements in a subgroup belong to the same class, or a stopping condition is satisfied. Different classes were defined in each study area, considering their respective variety in the typologies of buildings. Then, contiguous objects classified as buildings were merged and very small objects eliminated.

3.3. Shadow Detection and Contextual Relationships Definition

A common characteristic of urban areas in images is the presence of shadows, which depend on the height of the buildings and on the illumination conditions. This fact, that initially is a limitation for spectral characterization of buildings, could become an additional source of information if the shadows are properly detected and related to their respective buildings. Many authors assume a bimodal behavior of the histogram of images containing shadows, and as a result they try to automatically find the valley (lowest histogram frequency values) that divides the shadowed areas from the illuminated areas. The amount of shadows in a high resolution image depends on the azimuth and solar elevation angles, and the sensor observation angle [37], but also on the proportion of objects able to project these shadows. However, even when some shadows are present in the image, the histogram may not be clearly bimodal, making their detection quite difficult by only analyzing this frequency graph. Consequently, we used a semi-automatic methodology that requires manual sample selection. It consists of the definition of the threshold value on the first principal component of the original bands, by selecting samples of shadowed and illuminated areas. Both intensity values are then modeled by Gaussian curves, and the threshold value is determined as the intersection of both curves. The binary image obtained is cleaned and smoothed by means of morphological filters. Shadows are used to contextualize the adjacent buildings in the direction of the solar incidence angle on each image. This is used as a restrictive condition, so that every object detected as building but with no shadow projected in the solar incidence direction, will not be considered as building, i.e., it will be excluded.

3.4. Quality Assessment

The evaluation of the described approaches for building detection was carried out on two levels: area or pixel level; and object level. Reference buildings were manually delineated by direct photointerpretation of the images. Due to the different acquisition dates of LiDAR data and imagery, the selection of the reference areas for evaluation purposes was done ensuring that those areas were not affected by changes.

In urban areas, diverse types of buildings with different uses and characteristics coexist. In order to evaluate the performance of the building detection approaches on different building scenarios, three building types were defined for the study areas of Moncada and Sagunto: urban, suburban and industrial; and two for the study area of Oliva: urban and suburban. Urban buildings (Figure 4, left) are located at city and town centers and they are designed for residential and commercial purposes. These are diverse structures located in blocks and surrounded by urban elements like roads, artificial green areas or car parks. Suburban areas (Figure 4, center) are normally composed of abundant vegetation zones and detached and semi-detached buildings to accommodate single families. Industrial buildings (Figure 4, right) are characterized for presenting large dimensions, and they are aimed to manufacture, transform, repair, store and distribute products.

Figure 4. Examples of the building types defined: urban (left), suburban (center), and industrial (right). All the images belong to the study area of Sagunto.

Figure 4. Examples of the building types defined: urban (left), suburban (center), and industrial (right). All the images belong to the study area of Sagunto.

The evaluation at area level has been performed using a series of statistical parameters defined by McGlone and Shufelt [90] that have been repeatedly referred to in the literature [8,10,19,25,27,38,40,69,91,92]. Detected and reference buildings are spatially compared, and areas are categorized in four cases (see Figure 5): true positive (TP), true negative (TN), false positive (FP) and false negative (FN). TP represents those areas containing both detected and reference buildings. TN represents areas without reference or detected buildings. FP represents areas containing detected buildings but without reference buildings. FN represents undetected building areas.

Figure 5. Cases for evaluation of building detection at area level.

Figure 5. Cases for evaluation of building detection at area level.

Using these cases, the following area level quality metrics are defined: The branching factor (Equation (1)) is a measure of the degree to which a system over-detects as buildings non-built areas. The more accurate the detection, the closer the value is to zero. The miss factor (Equation (2)) indicates the omission error committed on the detection of built areas. These quality metrics are closely related to the boundary delineation performance of the building extraction system. The building detection percentage (Equation (3)) represents the percentage of reference built areas that is automatically detected. The quality percentage (Equation (4)) measures the absolute quality of the detection model by combining aspects of both boundary delineation accuracy and building detection rate to summarize the system performance.

$$Branching factor=\frac{FP}{TP}$$

(1)

$$Miss factor=\frac{FN}{TP}$$

(2)

$$Detection percentage=100\cdot \frac{TP}{TP+FN}$$

(3)

$$Quality percentage=100\cdot \frac{TP}{TP+FP+FN}$$

(4)

The object level assessment approach estimates the spatial correspondence between reference and detected buildings, and has been employed in several studies [17,26,54,70,93]. Building detection methodologies with low performance at pixel-level and high performance at object-level are inappropriate for automatic building delineation. However, these procedures may be suitable for the identification and spatial location of buildings, considering a subsequent manual delineation by means of photointerpretation techniques. Two metrics were considered for object-level evaluation: correctness and completeness, which reveal the errors of commission and omission, respectively. The correctness value indicates the percentage of the detected building objects that are at least partially overlapped with the reference buildings. Completeness value refers to the percentage of reference buildings overlapping the detected buildings.

4. Results and Discussion

4.1. Evaluation of Building Detection Methods

The results of the statistical assessment of the building detection using the thresholding-based approach are shown in Table 2. In general, the results show a high performance for the different building types analyzed.

Detection results obtained for type industrial building show very low values of branching and miss factors, meaning that buildings are precisely delineated. Quality percentage values are higher than 90% in both study areas. At object-level, completeness values show that all reference buildings are overlapped by detected buildings. On the other hand, correctness values are lower, showing that several small objects have been erroneously detected as buildings. Their very small size barely affects the area-level metrics, but it acts as noise in the object-level parameters. Figure 6(a) shows an image detail illustrating the detection results for industrial type buildings. Generally, these buildings appear properly defined and delimited; meanwhile little objects are erroneously detected as buildings due to the presence of characteristic elements of industrial areas, such as trucks or pallets, during the LiDAR data acquisition process.

Table 2. Evaluation results for the thresholding-based approach.

		Industrial		Urban			Suburban
		Moncada	Sagunto	Moncada	Oliva	Sagunto	Moncada	Oliva	Sagunto
Area-level	Branching factor	0.06	0.09	0.12	0.08	0.11	0.37	0.11	0.15
	Miss factor	0.02	0.02	0.04	0.03	0.02	0.14	0.12	0.16
	Detection percentage	97.7	98.2	96.5	97.2	98.1	87.8	89.1	86.3
	Quality percentage	92	90.3	86.6	90.6	88.3	66.1	81.1	76.6
Object-level	Correctness	72.4	73.3	74.3	71.9	87.2	89.8	100	99.2
	Completeness	100	100	100	100	100	98.4	99.3	95

Table 2. Evaluation results for the thresholding-based approach.

		Industrial		Urban			Suburban
		Moncada	Sagunto	Moncada	Oliva	Sagunto	Moncada	Oliva	Sagunto
Area-level	Branching factor	0.06	0.09	0.12	0.08	0.11	0.37	0.11	0.15
	Miss factor	0.02	0.02	0.04	0.03	0.02	0.14	0.12	0.16
	Detection percentage	97.7	98.2	96.5	97.2	98.1	87.8	89.1	86.3
	Quality percentage	92	90.3	86.6	90.6	88.3	66.1	81.1	76.6
Object-level	Correctness	72.4	73.3	74.3	71.9	87.2	89.8	100	99.2
	Completeness	100	100	100	100	100	98.4	99.3	95

Figure 6. Detection examples of industrial buildings in Sagunto (a), urban buildings in Oliva (b) and suburban buildings in Moncada (c) using the thresholding-based approach.

Figure 6. Detection examples of industrial buildings in Sagunto (a), urban buildings in Oliva (b) and suburban buildings in Moncada (c) using the thresholding-based approach.

Regarding the urban building type, the branching factor values are slightly higher than those obtained for industrial buildings, and the miss factor values are very close to zero. As a result, the quality percentage presents high values for the three studied areas. At object-level, the quality assessment shows similar results to the industrial building type, obtaining values for correctness markedly lower than for completeness. Figure 6(b) also shows some small objects erroneously detected as buildings due to the slight positional differences between LiDAR and image data.

Detection of suburban buildings presents more difficulties, which is reflected on the branching and miss factor results obtained for this urban type. In general, quality percentages vary from 66% to 78%, comparable to results reported by other authors in similar scenarios using LiDAR data and multispectral imagery [69]. Correctness and completeness values are more balanced than those obtained for industrial and urban building types. As shown in the example of Figure 6(c), most of the buildings are individually detected, but their shape is strongly conditioned by the effect of tree occlusions.

Table 3 shows the overall classification accuracy values for the three study areas, obtained by the cross-validation method. Since the main objective is to evaluate building detection, only two classes were considered: buildings and non-built areas.

Table 3. Overall classification accuracies obtained in the three study areas.

Study area	Overall accuracy
Moncada	97%
Oliva	95%
Sagunto	98%

Table 3. Overall classification accuracies obtained in the three study areas.

Study area	Overall accuracy
Moncada	97%
Oliva	95%
Sagunto	98%

Table 4 shows the results of the statistical assessment of building detection using the object-based classification approach. Area-level assessment results obtained for industrial and urban types present low branching and miss factors, and quality percentage values ranging from 80% to 90%. This suggests that this method is suitable for building delineation. At object-level, the completeness factor is very high, but the correctness quality metric values range from 50% to 60%. These results reveal high commission errors due to the incorrect detection of many small objects, especially vegetation with low infrared response, and other urban elements, as shown in Figure 7(a,b). However, at area-level there is a poor performance of the object-based classification method for the suburban building type. Although the results are adequate attending to the miss factor, the branching factor values are significantly higher. This means that this method produces an important over-detection of small buildings. At object-level, assessment results are more balanced than those obtained for industrial and urban buildings.

Table 4. Results of building detection applying the object-based classification method.

		Industrial		Urban			Suburban
		Moncada	Sagunto	Moncada	Oliva	Sagunto	Moncada	Oliva	Sagunto
Area-level	Branching factor	0.08	0.11	0.17	0.06	0.15	0.70	0.20	0.49
	Miss factor	0.04	0.04	0.05	0.10	0.03	0.17	0.16	0.17
	Detection percentage	96.5	96.1	95.1	90.9	97.5	85.6	86.3	85.5
	Quality percentage	89.6	86.8	81.7	86	85.5	53.6	73.9	60.4
Object-level	Correctness	59.5	50	59.5	51.6	53.8	86.6	93.1	84.1
	Completeness	100	96.4	96.3	100	100	93	99.3	95.7

Table 4. Results of building detection applying the object-based classification method.

		Industrial		Urban			Suburban
		Moncada	Sagunto	Moncada	Oliva	Sagunto	Moncada	Oliva	Sagunto
Area-level	Branching factor	0.08	0.11	0.17	0.06	0.15	0.70	0.20	0.49
	Miss factor	0.04	0.04	0.05	0.10	0.03	0.17	0.16	0.17
	Detection percentage	96.5	96.1	95.1	90.9	97.5	85.6	86.3	85.5
	Quality percentage	89.6	86.8	81.7	86	85.5	53.6	73.9	60.4
Object-level	Correctness	59.5	50	59.5	51.6	53.8	86.6	93.1	84.1
	Completeness	100	96.4	96.3	100	100	93	99.3	95.7

Figure 7. Detection examples of industrial buildings in Sagunto (a), urban buildings in Oliva (b) and suburban buildings in Moncada (c) using the object-based classification approach.

Figure 7. Detection examples of industrial buildings in Sagunto (a), urban buildings in Oliva (b) and suburban buildings in Moncada (c) using the object-based classification approach.

The comparison of the performance of object-based classification and thresholding-based approaches is presented in Table 5, showing the mean and the standard deviation values of the accuracy metrics, grouped by building types. At area-level, branching and miss factor mean values are similar for industrial and urban buildings. However, the mean quality percentage values are slightly higher when using the thresholding-based approach. The branching factor mean and standard deviation values obtained with the object-based classification approach for suburban buildings doubles those obtained using the thresholding-based approach. Detected objects, using object-based classification approach, frequently contain several actual buildings due to the difficulty of independently framing suburban buildings in the segmentation step. The quality percentage is also significantly higher in the case of the thresholding-based approach. Object-level metrics show a similar completeness value for both methods, but higher correctness mean value for the method based on thresholding; also presenting a lower standard deviation value. The better mean value performance indicates that the thresholding-based approach is more suitable for building detection in all the analyzed scenarios, meanwhile the lower standard deviation value suggests a better robustness for this approach.

Table 5. Comparison of mean (µ) and standard deviation (σ) of building detection metrics for the thresholding-based (Thresh) and object-based classification (Object).

		Industrial				Urban				Suburban
		Thresh.		Object		Thresh.		Object		Thresh.		Object
		µ	σ	µ	σ	µ	σ	µ	σ	µ	σ	µ	σ
Area-level	Branching factor	0.08	0.02	0.10	0.02	0.10	0.02	0.13	0.06	0.21	0.13	0.46	0.30
	Miss factor	0.02	0	0.04	0	0.03	0.01	0.06	0.04	0.14	0.06	0.17	0.08
	Detection percentage	98	0.4	96.3	0.3	97.2	0.8	94.5	3.3	87.7	5.2	85.8	6.7
	Quality percentage	91.2	1.2	88.2	2	88.5	2	84.4	2.4	74.6	11.1	62.6	16.1
Object-level	Correctness	72.9	0.6	54.8	6.7	77.8	8.2	55.0	4.1	96.3	5.6	87.9	21.1
	Completeness	100	0	98.2	2.5	100	0	98.8	2.1	97.6	2.3	96.0	3.9

Table 5. Comparison of mean (µ) and standard deviation (σ) of building detection metrics for the thresholding-based (Thresh) and object-based classification (Object).

		Industrial				Urban				Suburban
		Thresh.		Object		Thresh.		Object		Thresh.		Object
		µ	σ	µ	σ	µ	σ	µ	σ	µ	σ	µ	σ
Area-level	Branching factor	0.08	0.02	0.10	0.02	0.10	0.02	0.13	0.06	0.21	0.13	0.46	0.30
	Miss factor	0.02	0	0.04	0	0.03	0.01	0.06	0.04	0.14	0.06	0.17	0.08
	Detection percentage	98	0.4	96.3	0.3	97.2	0.8	94.5	3.3	87.7	5.2	85.8	6.7
	Quality percentage	91.2	1.2	88.2	2	88.5	2	84.4	2.4	74.6	11.1	62.6	16.1
Object-level	Correctness	72.9	0.6	54.8	6.7	77.8	8.2	55.0	4.1	96.3	5.6	87.9	21.1
	Completeness	100	0	98.2	2.5	100	0	98.8	2.1	97.6	2.3	96.0	3.9

Table 6. Comparison of mean (µ) and standard deviation (σ) of building detection assessment metrics with and without using shadow contextual information for the thresholding-based approach.

		Industrial				Urban				Suburban
		Without shadows		With shadows		Without shadows		With shadows		Without shadows		With shadows
		µ	σ	µ	σ	µ	σ	µ	σ	µ	σ	µ	σ
Area-level	Branching factor	0.08	0.02	0.07	0.01	0.10	0.02	0.10	0.02	0.21	0.13	0.20	0.14
	Miss factor	0.02	0	0.02	0	0.03	0.01	0.03	0.01	0.14	0.06	0.30	0.13
	Detection percentage	98	0.4	98	0.4	97.2	0.8	96.8	0.3	87.7	5.2	77.7	8.3
	Quality percentage	91.2	1.2	91.4	0.9	88.5	2	88.4	1.4	74.6	11.1	66.8	2.1
Object-level	Correctness	72.9	0.6	76.8	2.5	77.8	8.2	83.4	11.2	96.3	5.6	97.7	3.2
	Completeness	100	0	100	0	100	0	99.1	1.5	97.6	2.3	82.5	11

Table 6. Comparison of mean (µ) and standard deviation (σ) of building detection assessment metrics with and without using shadow contextual information for the thresholding-based approach.

		Industrial				Urban				Suburban
		Without shadows		With shadows		Without shadows		With shadows		Without shadows		With shadows
		µ	σ	µ	σ	µ	σ	µ	σ	µ	σ	µ	σ
Area-level	Branching factor	0.08	0.02	0.07	0.01	0.10	0.02	0.10	0.02	0.21	0.13	0.20	0.14
	Miss factor	0.02	0	0.02	0	0.03	0.01	0.03	0.01	0.14	0.06	0.30	0.13
	Detection percentage	98	0.4	98	0.4	97.2	0.8	96.8	0.3	87.7	5.2	77.7	8.3
	Quality percentage	91.2	1.2	91.4	0.9	88.5	2	88.4	1.4	74.6	11.1	66.8	2.1
Object-level	Correctness	72.9	0.6	76.8	2.5	77.8	8.2	83.4	11.2	96.3	5.6	97.7	3.2
	Completeness	100	0	100	0	100	0	99.1	1.5	97.6	2.3	82.5	11

Table 7. Comparison of mean (µ) and standard deviation (σ) of building detection assessment metrics with and without using shadow contextual information for the object-oriented classification approach.

		Industrial				Urban				Suburban
		Without shadows		With shadows		Without shadows		With shadows		Without shadows		With shadows
		µ	σ	µ	σ	µ	σ	µ	σ	µ	σ	µ	σ
Area-level	Branching factor	0.10	0.02	0.09	0.01	0.13	0.06	0.12	0.06	0.46	0.30	0.45	0.31
	Miss factor	0.04	0	0.04	0.00	0.06	0.04	0.06	0.04	0.17	0.08	0.20	0.08
	Detection percentage	96.3	0.3	96.3	0.3	94.5	3.3	94.5	3.3	85.8	6.7	83.2	7.1
	Quality percentage	88.2	2	88.7	1.3	84.4	2.4	84.6	2.5	62.6	16.1	61.5	16.2
Object-level	Correctness	54.8	6.7	57.3	3.2	55.0	4.1	59.3	1.0	87.9	21.1	92.7	21.0
	Completeness	98.2	2.5	98.2	2.5	98.8	2.1	98.8	2.1	96.0	3.9	91.7	3.6

Table 7. Comparison of mean (µ) and standard deviation (σ) of building detection assessment metrics with and without using shadow contextual information for the object-oriented classification approach.

		Industrial				Urban				Suburban
		Without shadows		With shadows		Without shadows		With shadows		Without shadows		With shadows
		µ	σ	µ	σ	µ	σ	µ	σ	µ	σ	µ	σ
Area-level	Branching factor	0.10	0.02	0.09	0.01	0.13	0.06	0.12	0.06	0.46	0.30	0.45	0.31
	Miss factor	0.04	0	0.04	0.00	0.06	0.04	0.06	0.04	0.17	0.08	0.20	0.08
	Detection percentage	96.3	0.3	96.3	0.3	94.5	3.3	94.5	3.3	85.8	6.7	83.2	7.1
	Quality percentage	88.2	2	88.7	1.3	84.4	2.4	84.6	2.5	62.6	16.1	61.5	16.2
Object-level	Correctness	54.8	6.7	57.3	3.2	55.0	4.1	59.3	1.0	87.9	21.1	92.7	21.0
	Completeness	98.2	2.5	98.2	2.5	98.8	2.1	98.8	2.1	96.0	3.9	91.7	3.6

4.2. Assessment of Contextual Information Derived from Shadows

The use of contextual information derived from shadows in the building detection process has a restrictive effect, especially remarkable in the case of suburban buildings. The presence of small buildings and trees makes the accurate detection of shadows difficult and, consequently, actual buildings are erroneously removed after being detected. As a result of this, the miss factor values generally increase detection and quality percentages are reduced as are the completeness percentages, which is more significant for the thresholding-based approach (Table 6 and Table 7) (Figure 8, center). Industrial and urban buildings barely show variations in the area level metrics when shadow contextual relationships are included, due to the typically large size of these types of buildings. The object-level correctness values usually increase for all building types, due to the removal of small objects that are incorrectly detected as buildings, and act as noise (see Figure 8, left and right). Therefore, the use of shadows involves a trade-off between the reduction of FP in urban and industrial areas, and the increase of FN in suburban areas. In general, the results show that, even when shadows have been frequently and successfully employed to improve building detection in images, their positive effect in building detection becomes restricted when three-dimensional information is also used.

Figure 8. Detection examples of urban buildings in Sagunto (left) and suburban buildings in Moncada (center) and Oliva (right), showing differences due to the use of shadow contextual information.

Figure 8. Detection examples of urban buildings in Sagunto (left) and suburban buildings in Moncada (center) and Oliva (right), showing differences due to the use of shadow contextual information.

5. Conclusions

This paper presents a quality assessment of two main approaches for building detection based on the use of high resolution imagery and low density airborne laser scanner data: thresholding-based and object-based classification. Methodological parameters of both approaches have been adapted to the study areas, in this case three different scenarios: urban, suburban and industrial buildings. The evaluation results have been obtained according to two criteria: area level and object level.

In general, results show that the thresholding-based building detection approach performs better in the different scenarios analyzed, especially in peri-urban areas. This method produces a more accurate building delineation and object detection than the object-based classification method. Moreover, object-based classification method presents a higher procedural complexity, due to the larger number of parameters needed for definition and steps to be carried out: type of data used in the segmentation, definition of classes, selection of training samples, extraction and selection of descriptive features, selection and application of the classification algorithm. The thresholding-based approach only requires the definition of two threshold values, the minimum height from LiDAR and the maximum NDVI from the spectral data. The building type is as a key factor in building detection performance. Thus, urban and industrial areas show better accuracies in detection metrics than suburban areas, due to the small size of suburban constructions, combined with the prominent presence of trees in suburban classes, which makes building detection more difficult. The contextual relations between buildings and shadows improve the object-level detection, removing small objects erroneously detected as buildings that negatively affect the correctness index. However, at area-level, their use generates building under-detection, especially in suburban areas.

The results obtained show a high efficiency of the evaluated methods for building detection techniques, in particular the thresholding-based approach when the parameters are properly adjusted and adapted to the type of urban landscape under consideration. These methodologies can be applied to the detection of new buildings for updating urban databases, as well as for the inclusion of this information in urban object classification and updating of geospatial databases.