Automatic Detection of Low-Rise Gable-Roof Building from Single Submeter SAR Images Based on Local Multilevel Segmentation

Abstract

Low-rise gable-roof buildings are a typical building type in shantytowns and rural areas of China. They exhibit fractured and complex features in synthetic aperture radar (SAR) images with submeter resolution. To automatically detect these buildings with their whole and accurate outlines in a single very high resolution (VHR) SAR image for mapping and monitoring with high accuracy, their dominant features, i.e., two adjacent parallelogram-like roof patches, are radiometrically and geometrically analyzed. Then, a method based on multilevel segmentation and multi-feature fusion is proposed. As the parallelogram-like patches usually exhibit long strip patterns, the building candidates are first located using long edge extraction. Then, a transition region (TR)-based multilevel segmentation with geometric and radiometric constraints is used to extract more accurate edge and roof patch features. Finally, individual buildings are identified based on the primitive combination and the local contrast. The effectiveness of the proposed approach is demonstrated by processing a complex 0.1 m resolution Chinese airborne SAR scene and a TerraSAR-X staring spotlight SAR scene with 0.23 m resolution in azimuth and 1.02 m resolution in range. Building roofs are extracted accurately and a detection rate of ~86% is achieved on a complex SAR scene.

1. Introduction

Synthetic aperture radar (SAR) sensors with high resolution, either airborne or spaceborne, show high potential in applications of urban remote sensing such as damage assessment [1,2,3], urban environment monitoring [4], and three-dimensional (3-D) reconstruction [5]. Compared with low- or medium-resolution SAR images that are usually utilized to monitor large-scale objects, very high resolution (VHR) SAR images at meter-level and submeter-level provide more details of the targets of interest in terms of both geometric structures and radiometric features. To ensure a reliable, consistent, and fast extraction of the information from complex SAR urban scenes, robust automatic information extraction methods are essential.

As one of the most important components in urban areas, buildings attract more attention for urban studies. Many techniques for building detection from VHR SAR images have been developed. These methods can be divided into two categories according to the types of experimental data. The first category relies on the availability of ancillary or multiple image data, e.g., multidimensional data [5], multitemporal data [6], InSAR [7], and PolSAR [8], which implies that the area under investigation is imaged more than once with different viewing configurations (changed incidence and/or aspect angle). Because of the complexity of data acquisition and usage, these methods obviously have limitations in some application scenarios such as emergency response or locations with restricted data [9]. The second category consists of methods based on a single SAR image. The related methods can be grouped into model-based methods [10,11], feature-based methods [12,13], and ontological semantic-based methods [9] with the common foundation of building signature extraction [14].

In meter-level SAR images, buildings are mainly represented by strip and block signatures, which are generally composed of layovers, double bounce lines and shadows. The related signature extraction methods include five types. (1) Region-based approaches, which first generate regions that are homogeneous with respect to the extracted features, such as gray statistics and texture features, and then produce individual primitive according to the radiometric, geometric, and topologic characteristics [14]. Related approaches involve Markov random fields [15], mean-shift [16]. (2) Edge-based approaches, such as the POL_CFAR edge detector [8], ROEWA detector [17]. (3) Bright line-based approaches, which consider layovers as line features with a certain width and further extract them via line detectors under multiscales [5,12]. (4) Range profile-based methods. Chen et al. [18] proposed a 1-D detector, referred to as the “range detector,” to detect the footprints of the illuminated wall of cuboid buildings. (5) Combined methods, which merge two or more kinds of the aforementioned methods [19].

Nowadays, spaceborne SAR images with submeter-level resolution can be obtained. In 2013, a new TerraSAR-X mode named staring spotlight (ST) [20], whose azimuth resolution improved to 0.24 m, was introduced for various applications such as earthquake damage mapping [1] and archaeological remote sensing [21]. This kind of submeter-level SAR data source provides new opportunities for building detection and extraction as more details can be observed in such images in terms of both geometric structures and radiometric features, making it possible to focus an analysis on individual buildings with small sizes.

As finer scattering mechanisms become visible in submeter data, the dominant building signatures, which can be used as cues for building extraction, have changed with the improvement of SAR image resolution. Bright spots, strips, and blocks can all be dominant signatures in submeter-level SAR images [8]. The methods suited for meter-level SAR images may be extended and tuned for submeter-level data by introducing new types of primitives, composed objects, and rules [12]. Soergel et al. [22] used principles from perceptual grouping to detect building features such as long, thin roof edge lines, groups of salient point scatters, and symmetric configurations from SAR images with decimeter resolution. Lu et al. [8] proposed a new building signature extraction method based on symmetric analysis. Symmetry axes of elongated building layover and shadow are first delineated, and local radiometric features are then extracted from both the range direction and the direction normal to the symmetry axes.

The availability of richer information from submeter-level SAR image has a positive effect on building detection and extraction, but gives rise to an increase in complexity and the heterogeneity of building signatures. The appearances of buildings strongly vary due to differences in structural elements (such as windows, balconies, or roof details), the size and material of the façade, and the building height. The dominant signatures are quite different for diverse building types (e.g., isolated small buildings, large buildings, and residential buildings), so it is necessary to propose more specific and targeted building detection methods.

In this paper, based on the new characteristics of submeter-level SAR images, the authors propose a framework for the more accurate detection of low-rise, gable-roof buildings from a single submeter-level resolution SAR image. The low-rise gable-roof building is one of the most typical building types in rural areas, shantytowns, and industrial parks of China. The detection and extraction of this type of building can be used for urban reconstruction and damage assessment. Previous studies related to gable-roof building detection can be found in [9,12,23]. Gui et al [9] proposed an ontological semantic-based method to extract buildings with different orientations and different structures. The results show a high detection rate, but the building outlines are inaccurate. Methods in [12,23] are oriented to meter-level SAR images, where both the flat- and gable-roof buildings are imaged as narrow strips. These two methods are not robust for gable-roof building detection in submeter-level SAR images due to the difference in building appearances. Therefore, it is essential to improve the traditional methods or propose a new method.

The advantages of our framework are as follows. (1) New types of feature primitives (e.g., building roof ridge and two adjacent parallelogram-like roof patches) on submeter-level SAR images are utilized to identify individual buildings. (2) Since building signature features are closely related, the authors adopt a transition region (TR)-based segmentation to simultaneously extract edge and regional features, which can unify the primitive extraction process. (3) In view of the fractured nature of the linear and regional features and the instability of feature extraction in VHR images, a multilevel threshold defining strategy is used in the TR-based segmentation to analyze the image with multiple levels to obtain more informative and robust features. Linear and regional features are fused to optimize final results.

The remainder of this paper is organized as follows. The backscattering model analyses of gable-roof buildings are described in Section 2. Section 3 introduces the processing chains of the proposed framework. The experimental results are provided in Section 4. Finally, the discussion and conclusions are presented in Section 5 and Section 6, respectively.

2. Gable-Roof Building Signature Analysis

Figure 1 shows a schematic view of the backscattering model of a typical low-rise gable-roof building. The typical layover and double-bounce, which are the dominant features in meter-level resolution SAR images, are not discernable and merged into the echoes of building roof in submeter resolution SAR images. The key features of this kind of building turn into parallelogram-like roof patches, roof edges, and shadowing effects. These features are interpreted in detail following radiometric and geometric analyses.

Radiometric analysis: The building signatures are divided into three parts (A1, A2, and A3 in Figure 1c), with the mean intensity of A1 > A2 > A3. Part A1 indicates the near-sensor roof patch, including scattering areas a, b, and c in Figure 1d. Roof scattering is affected by wall scattering and ground scattering, and those contributions are mixed (i.e., layover, a + b). A2 indicates the far-sensor roof patch, which is only characterized by scattering from the roof (denoted by d), and A3 is the shadow area. The intensity and spatial relationships of A1, A2, and A3 are used to identify building objects and exclude false alarms in subsequent building detection.

Geometric analysis: As A1 and A2 usually exhibit parallelogram-like long strip patterns, the long edge lines of the strips (illustrated with green, red and yellow lines in Figure 1c) can indicate the building orientation. Long edge lines are more likely to be obtained than the short edges because edges are likely to split into several fractions due to the effects of noise and complex surface scenes. Therefore, in this paper, the long edge lines of the two parallelogram-like patches are used as the cues for building detection.

According to the sensor viewing direction and the distance between roof edges and the sensor, the long edge lines are defined into three types: near-sensor edge, far-sensor edge, and roof ridge. As shown in Figure 1c, the green, red, and yellow lines indicate the position of near-sensor edge, roof ridge, and far-sensor edge, respectively. Figure 2 shows the spatial composing relationship of edge lines and roof patches. Adjacent feature primitives are merged to compose more structured features. The intensity and spatial relationships of all these features are used to identify building objects in subsequent building detection.

3. Method

Figure 3 shows the flow chart of our approach. The processing chain consists of four main steps: edge extraction, building candidate area extraction, parallelogram-like roof patch extraction using TR-based multilevel segmentation, and selection of roof candidates and individual building identification. To mitigate speckle noise in the SAR image, the original image is filtered with a non-local algorithm proposed by Deledalle et al. [25], which has a strong denoising effect and simultaneously preserving fine structures.

3.1. Edge Extraction and Classification

The edge detection is performed on the filtered image using a gradient computation method, gradient by ratio (GR), proposed by Dellinger et al. [26]. A ratio map is obtained from the GR method, which is then thresholded and binary linear strips are obtained (see Figure 4c). A local adaptive threshold, rather than a global threshold, is used to preserve more detailed edge information and to improve the detection rate. The threshold is derived from the gradient histogram of a local sub-window (30 × 30) with the highest 12% of the response pixels remaining. Several morphological processing steps, e.g., dilation, erosion, skeleton extraction, and removal of branch points, are used to optimize the binary image. Afterward, the Douglas–Peucker algorithm [27], which reduces the number of points in a curve that is approximated by a series of binary pixels, is applied to generate segment primitives. Segment primitives meeting a certain proximity and collinearity are reshaped by least-square-fit processing, which produces a new vector representation of the segments. Finally, edge lines are obtained through an iterative calculation of the global connection of segment primitives. Edge lines beyond a certain length are reserved as roof edge candidates and short edges are eliminated. In this study, a length threshold equal to 20 is used. The corresponding result is shown in Figure 4d.

The types of the roof edge candidates are determined by a two-rectangle classifier shown in Figure 5. The two rectangular windows on both sides of an edge line are constructed for intensity computation. The window length is the same as the length of the edge line, and the width r is set to five times the resolution of the image in this paper. The classification rules composed by the mathematical relationship are shown in Figure 5b. Three parameters, $M_{rec1}$, $M_{rec2}$, and $T_{shadow}$, are introduced to indicate the mean intensity of pixels in rec1 and rec2 and the shadow area of SAR image, respectively. $M_{rec1}$ and $M_{rec2}$ are obtained by the classifier, and $T_{shadow}$ is obtained by analyzing the intensity of sample pixels belonging to the shadow regions in the SAR image [16].

3.2. Building Candidate Area Extraction

Given an edge candidate $L_i$ with two endpoints $P_1(x_1,y_1)$ and $P_2(x_2,y_2)$, a rectangular region of interest labeled as $C_i$ is obtained as indicated in Figure 6. The two endpoints are extended by a distance d, which is selected by analyzing the size of an individual gable-roof building in the SAR image. The region of interest should be large enough to contain a whole building, but too large of a region increases the computational cost. The post processing is based on $C_i$.

3.3. Parallelogram-Like Roof Patch Extraction

3.3.1. Feature Primitive Extraction and Optimizing

A TR-based image segmentation method is used to extract the two parallelogram-like roof patches. TR-based image segmentation is a recently developed single-object segmentation technique [28,29] that first extracts the image transition region between the object and the background, and then takes the mean of the pixels in the transition region as the segmentation threshold for image thresholding. In this paper, instead of using local complexity and local variance as employed by Li et al. [28], the local gradient, which was used for edge extraction in Section 3.1, is also used as a descriptor to extract the transition region. For each edge candidate $L_i$, the corresponding binary linear strip from the GR is extracted as transition region $R_i$. Example images are shown along each step for better illustration purposes. Selecting an edge candidate from Figure 4d, the extracted area of a building candidate is shown in Figure 7a. Figure 7b shows the corresponding binary linear strip, which is used to generate the edge line. The binary linear strip is extracted from Figure 4c, and in this section, it is used as a transition region for image segmentation.

Given a suitable threshold $t$ computed from the transition region, such as the mean intensity of the pixels in transition region, a binary image is obtained after the image thresholding. The binary image is optimized by some morphological processing such as dilation and erosion. Then, the regions that overlap with the transition region are reserved as optimized building roof candidates $Cand{i}_i$. The example image is shown in Figure 7c. The boundary pixels of $Cand{i}_i$ are obtained and used to generate line segments using the Douglas–Peucker algorithm. The line segments parallel to $L_i$ are extracted as new roof edge candidates $L_{ij},j=1,\mathrm{...}N$, and $N$ is the total number of parallel lines. The type of $L_{ij}$ is determined using the two-rectangle classifier. Afterwards, the spatial composing relationship shown in Figure 2 is used to eliminate the false candidates. For example, if $L_i$ is a near-sensor edge, then $L_{ij}$ must be a roof ridge, or vice versa. The example image is shown in Figure 7d. Finally, the linear feature primitives ($L_i$, $L_{ij}$) and regional feature primitive ($Cand{i}_i$) are fused and optimized using the following steps to obtain a parallelogram-like roof patch.

• Shape-constrained feature optimizing. For each pair of parallel edges, $Cand{i}_i$ is reduced to the region between the two parallel lines, and a parallelogram P is matched by extending the edges. The edge length is set to the midline length of $Cand{i}_i$. The cost function $F(P)$ can be written as

  $$F(P)=1-\frac{f(Cand{i}_i\cap P)}{f(P)}$$

  (1)

  where function $f$ is an area computing function. The parallelogram P with the minimum $F(P)$ is reserved. The corresponding example image is shown in Figure 7e.
• Homogeneity-constrained feature optimizing. The fitted parallelogram P is fused with the mean shift segmentation in this step. First, the segmentation patches that overlap with P are extracted as $Cand{i}_i$, and the parallelogram matching function $F(P)$ is then computed again to obtain the final P. This processing step is not strictly necessary for the correct operation of the proposed technique but increases the accuracy of roof patch extraction. The corresponding example image is shown in Figure 7f.

3.3.2. Multilevel Thresholds

Gable-roof buildings are exhibited as two adjacent parallelogram-like roof patches with different intensity. The near-sensor roof patch, which has a higher intensity than the far-sensor one, is extracted first. Then, a multilevel quantization technique is used in the TR-based segmentation to extract the far-sensor roof patch. The multilevel quantization technique is formed by a multilevel threshold defining strategy.

In this study, the multilevel threshold is defined as

$$t_m=T_{\max }-\frac{T_{\max }-T_{shadow}}{M}\times (m-1),m\in [1,\mathrm{...}M]$$

(2)

where $T_{\max }$ is the upper limit of $t_m$, M is the total number of levels，and $T_{shadow}$ is the mean intensity of shadow areas. These three parameters are set in the following way: (1) $T_{\max }$ is selected according to the mean intensity of the transition region, which is a typical threshold in the TR-based segmentation. We set $T_{\max }=k{t}_0$, where k is a scaling factor and $t_0$ is the mean intensity. k is used to enhance the ability to separate the near-sensor roof patch from the far-sensor roof patch. If there is a distinct difference in intensity between the two kinds of roof patches, a small k (e.g., k = 1) can be effective. Otherwise, a larger value is needed. In our experiments, k = 1.2 is used. (2) The multilevel quantization technique is superior to the single-level technique because it can capture the detailed variation of intensities around edges and is resistant to speckle noise. A large M can capture more robust linear and regional features but increases the computational cost. In our experiments, a similar performance has been obtained using M = 6, 8, 10, 12. A middle value M = 8 is suggested. (3) $T_{shadow}$ is used to separate the object from the shadow, it determines the lower bound of the threshold. In this study, shadow areas are extracted according to the approach proposed in [16]; then, $T_{shadow}$ can be obtained.

Based on the multilevel image segmentation, the two roof patches are concatenated at multiple levels. The structure and intensity information of the near-sensor roof patch can be used to restrain the extraction of the far-sensor one. The example image is shown in Figure 7g. A building roof candidate containing both the near-sensor roof patch and the far-sensor roof patch is extracted under a level-5 threshold. The near-sensor roof patch is removed and the rest region (Figure 7h) is used to construct the far-sensor roof patch in a way similar to that of the near-sensor one. The final roof edges are shown in Figure 7i.

3.4. Individual Building Identification

First, the spatial relationship between the shadow and the roof patches is used to eliminate false alarms. The roof patches are reserved on the condition that there is an adjacent shadow area on the far-sensor side. Then, a contrast restriction is used to identify individual building. For each quadrangle constructed by the two extracted roof patches, the local contrast value $C_r$ is calculated from the filtered image by the following equation:

$$C_r=[\frac{1}{M_{in}}{\displaystyle \sum _{i\in A_{in}}{x}_i}].[\frac{1}{M_{out}}{\displaystyle \sum _{i\in A_{out}}(1-x_i)}]$$

(3)

where $A_{in}$ and $A_{out}$ are the inner regions of the quadrangle and the surrounding outer thick border, respectively; the thickness of $A_{out}$ is set to half of the roof width; $M_{in}$ and $M_{out}$ are the number of pixels contained in the regions $A_{in}$ and $A_{out}$, respectively; and $x_i$ represents the pixel intensity value normalized between 0 and 1. $C_r$ is a measure of the contrast between the pixels contained in the quadrangle and their surroundings. Higher differences between the mean intensities of the two regions produce higher $C_r$ values. This measure was proposed by Simonetto et al. [30] and used by Ferro et al. [12] for local contrast computation.

4. Materials and Experimental Results

The proposed method was verified on a TerraSAR-X staring spotlight SAR scene and a Chinese VHR airborne SAR scene. The imaging parameters are listed in

Table 1. Imaging parameters of experimental data.

Parameter	TerraSAR-X Staring Spotlight SAR	Chinese Airborne SAR
range resolution(m)	1.02	0.1
azimuth resolution(m)	0.24	0.1
center incidence angle(deg)	33	58
polarization type	HH	HH

. The TerraSAR-X staring spotlight image used in this paper was taken over the southern outskirt of Suzhou, Jiangsu province in China, in August 2014. The Chinese X-band airborne VHR SAR image was acquired over a rural area near Xi’an City in the Shanxi province of China in 2013. The building orientations are different in the two scenes.

4.1. Test on TerraSAR-X Staring Spotlight SAR Scene

As shown in Figure 8a, the TerraSAR-X staring spotlight SAR scene (555 × 410) contains 23 low-rise gable-roof buildings. The roof signatures are prominent, whereas the layover and double-bounce phenomena are not discernable, making the traditional building detection methods based on layover and double-bounce not applicable in this scenario. The contrast of intensity between the near-sensor roof patch and the far-sensor roof patch is more clearly presented in a mean shift segmentation result (Figure 8b). Near-sensor roof patches correspond to bright segments, whereas far-sensor roof patches correspond to gray segments.

To locate the building candidate area, edges are extracted as shown in Figure 8c. The results are fractured while the long edges are mostly near the long strip structures of the roofs. Then, TR-based multilevel segmentation is implemented to extract parallelogram-like roof patches. After feature optimization and false alarm elimination, individual buildings are identified in Figure 8d. All 23 gable-roof buildings are detected with accurate roof patch extraction.

To validate the performance of our algorithm, the marker-controlled watershed segmentation-based method [17] is also performed on the TerraSAR-X staring spotlight SAR scene. The OS-CFAR detector is used to extract bright pixels of buildings. Since the intensity of the pixels in far-sensor roof patches is relatively low, these pixels are lost. The result is shown in Figure 8e. All 23 gable-roof buildings are partly detected, with far-sensor roof patches missing.

4.2. Test on Chinese Airborne SAR Scene

A more complex scene imaged by a Chinese airborne SAR is shown in Figure 9. Thirty-six gable-roof buildings are included in this scene. Building signatures are severely affected by the surrounding environment, such as adjacent buildings, trees, grass, and other human-made objects. The processing procedure is the same as what was used for the TerraSAR-X image. The final experimental result is shown in Figure 10. Some statistics for quantitative evaluation were manually calculated and are summarized in

Table 2. Building detection results of Figure 10.

Total Buildings	Detected Buildings	Partially Detected Buildings	Merged Buildings	False Alarms	Detection Rate
36	31	2	4	0	86%

* Note: the number of detected buildings includes “partially detected buildings” and “merged buildings.” A building is defined as partially detected if it is detected for less than 90% of its pixels.

.

The detection rate is defined as the ratio of correctly identified buildings to the total number of gable-roof buildings in the SAR image. A building is defined as partially detected if it is detected for less than 90% of its pixels. The merged buildings indicate that two adjacent buildings are identified as one building. It can be seen that our method performs well, with 86% of buildings detected, including some buildings with fractured roof features, indicating that our method is robust and resistant to the effect of the surrounding environment. Let us consider, for example, Building B23 in Figure 11a, which has a heterogeneous roof patch. The mean shift segmentation result is shown in Figure 11b, and it is fractured. The transition region used for the extraction of roof edges and roof patches is shown in Figure 11c, and the extracted building roof candidates and parallel lines are shown in Figure 11d,e. The building roof candidates are malformed and the edge lines are partial extracted. Figure 11f is the final result. The whole roof edges are accurately obtained, which indicates that the proposed method has the ability to optimize fractured roof features.

However, there are some merged, partially extracted or undetected buildings. The merged buildings are caused by the overlap between adjacent buildings with the same orientation (Buildings B12 and B13, B24 and B25, B26 and B27, B31 and B32). Buildings B3 and B4 are partially detected as their signatures are disturbed on the left side. In the process of roof patch extraction, the near-sensor roof patch is extracted first, and its length is used as a prior knowledge to extract the far-sensor roof patch. Therefore, if the near-sensor roof patch is partially extracted, some pixels in the far-sensor one will be undetected too, thus leading to an incomplete and inaccuracy extraction of roof edges in the process of parallelogram construction. Meanwhile, the direction error of the parallel lines used to generate parallelogram affects the final result too (Building B5). Moreover, if the buildings are shadowed or their edges are too blurry to be extracted, they will be undetected (e.g., Buildings B14, B15, and B16).

5. Discussion

As shown by our results, the proposed method obtains accurate roof extraction from a TerraSAR-X staring spotlight mode SAR image and shows an 86% detection rate on a complex airborne SAR scene. The number of false alarms is limited due to the strict constraints of the radiometric and geometric characteristics of the roof signatures. Other objects such as trees, roads, and other human-made structures do not show similar radar signatures to those of gable-roof buildings. Figure 8d shows that, compared with the marker-controlled watershed segmentation-based method that oriented to meter-level SAR images, the proposed method has a distinct advantage on the extraction of building outlines.

The proposed method is feature-based, and the basis is the exploitation of the very high resolution. Meter-level SAR images from other beam modes with less spatial resolution, such as TerraSAR-X spotlight images, are not the application scenarios of our method as a low number of features are visible in such images. Except for the resolution limitation, the proposed method is sensitive to viewing configurations of the SAR sensor, which may lead to a missing of gable-roof building features. As shown in Figure 1, our method is based on the angle relationship of the incidence angle $\theta$ smaller than ${90}^{°}-\alpha$ ($\alpha$ is the roof inclination). If $\theta >{90}^{°}-\alpha$, only the near-sensor roof patch can be imaged. With a decreasing number of features, the gable-roof and flat-roof buildings can be detected in a unified way, such as in [12].

In the process of parallelogram-like roof patch extraction, each parallelogram is constructed based on a pair of parallel edges. As a future refinement, the other two edge lines (corresponding to the two short edges of a parallelogram) will be located and used for parallelogram optimizing based on perceptual grouping methods. The goal of this step is to provide a more accurate result of building outlines, which can be used as a starting point for further calculations, e.g., the estimation of building widths and lengths. Moreover, by exploiting the reconstruction of the shadow areas, height retrieval techniques can be also applied to estimate building heights.

6. Conclusion

This paper addresses the problem of automatic detection of low-rise gable-roof buildings in a single submeter-level SAR image. Unlike most related methods, which focus on the usage of building layover, double-bounce, and shadow signatures, the new dominant features, i.e., two adjacent parallelogram-like roof patches and roof ridge are utilized for building detection in a single submeter-level SAR image. Features are extracted in a coarse to fine manner, using a TR-based multilevel segmentation technique. The proposed method obtains accurate roof extraction from a TerraSAR-X staring spotlight SAR scene and shows an 86% detection rate on a complex airborne SAR scene. The false alarms are limited due to the strict constraints of the radiometric and geometric characteristics of roof signatures. As future developments, the building roof will be further optimized and used for the retrieval of building widths, lengths, and heights.