Urban Building Footprints Extraction and Heights Estimation from High-Resolution Spaceborne Remote Sensing Imagery Using a CNN-Transformer Network

Highlights

What are the main findings?

• A hybrid CNN–Transformer network (SECT-Net) is proposed to extract building footprints and shadows from high-resolution optical imagery.
• A shadow-geometry-based strategy enables building height estimation using only single-date optical imagery and satellite acquisition parameters, eliminating the need for stereo data, LiDAR, or DSM labels.

What are the implications of the main findings?

• The proposed framework significantly reduces data dependency and acquisition costs, providing a practical solution for large-scale 2D–3D urban mapping.
• The method can be extended to multi-temporal imagery for building change detection and disaster assessment, supporting remote sensing-based urban analysis and management.

Abstract

High-resolution multispectral remote sensing imagery for accurate building footprint extraction and reliable height estimation often faces numerous challenges in complex urban environments with diverse building structures, heterogeneous shadow patterns, and widespread occlusions. This study proposes a sparse edge-aware convolutional-transformer neural network model, SECT-Net, for precise extraction of building footprints and their cast shadows from high-resolution Jilin-1 multispectral images covering Shanghai, China. A shadow-based height estimation workflow was then developed to characterize building heights from their shadow lengths. Results show that the SECT-Net achieves high performance in building footprint extraction, with an IoU of 77.96%, an F1 score of 87.62%, and an overall accuracy of 97.16%, respectively. Building heights for more than 750,000 buildings were estimated across the entire Shanghai, with R² = 0.74 and RMSE = 5.66 m. A slight systematic underestimation of building heights is attributed to occlusions of high-rise buildings and the disruption of vegetation in residential communities in dense urban central areas. This study demonstrates that the proposed SECT-Net can accurately and precisely extract building footprints from high-resolution spaceborne remotely sensed images. The estimated building heights provide a reliable underpinning for urban morphology analysis and building monitoring in urban planning and scientific management.

1. Introduction

Rapid urbanization has resulted in more than half of the global population residing in urban areas, and nearly two-thirds of future population growth is expected to occur in cities by the mid-21st century [1]. The growth of urban populations drives intensive land use and the vertical expansion of buildings, resulting in highly dense, compact urban environments. As the primary setting for human activities, the built environment critically shapes urban morphology, socio-economic structures, thermal landscapes, and ecological conditions [2]. Although two-dimensional (2D) urban features, such as building footprints, impervious surfaces, and urban boundaries, have attracted extensive interest, the vertical dimension of urban structures has received relatively limited attention [3]. Three-dimensional (3D) urban structures significantly influence ventilation corridors, pollution dispersion [4], surface runoff, urban heat islands [5,6], population distribution, transportation patterns, and carbon emissions [7,8,9]. Therefore, building height, as a key 3D parameter, is essential for population estimation, energy and carbon assessment, urban monitoring, disaster evaluation, 3D modeling, and urban land planning.

Building footprints are fundamental data for modeling urban spatial structures and are widely applied in urban expansion monitoring, disaster management, and building statistics. Traditional extraction methods primarily rely on high-resolution remote sensing image processing and classification rules, such as thresholding, edge detection, and morphological operations [10,11,12]. These approaches typically use spectral, geometric, and spatial distribution characteristics in those built-up scenes with clear structures. However, they often fail to accurately distinguish buildings from other highly reflective objects in complex urban environments with diverse building types and severe occlusions, thereby reducing robustness and adaptability. Object-based image analysis (OBIA) has emerged as an effective alternative by segmenting high-resolution remote sensing images into meaningful objects and classifying them using multidimensional features, such as spectral, textural, and shape attributes [13,14]. Some methods that combine traditional image segmentation with machine learning classifiers (e.g., SVM, Random Forest, and AdaBoost) have significantly enhanced the accuracy and interpretability of building recognition [15,16]. However, these approaches still suffer from complex feature engineering, parameter sensitivity, and limited scalability and generalization in large-scale applications [17].

In recent years, deep learning (DL) techniques have become the dominant paradigm for automatic extraction of building footprints from remote sensing imagery. Pixel-wise semantic segmentation models based on convolutional neural networks (CNNs), such as U-Net, SegNet, and the DeepLab series, have been widely applied due to their ability to capture local spatial details and delineate connected building regions [18,19]. Instance segmentation frameworks, including Mask R-CNN and YOLO-based methods, further enable accurate building localization and boundary extraction, particularly in dense urban scenes [20,21]. More recently, Transformer-based architectures, such as Swin Transformer and SegFormer, have demonstrated superior performance by modeling long-range dependencies and fusing multi-scale contextual information, thereby enhancing building perception in complex urban environments [22]. Some studies further incorporate attention mechanisms, edge-aware modules, and deep supervision to improve sensitivity to building boundaries, morphological details, and spatial context [23,24,25].

Despite these advances, both CNNs and Transformers exhibit inherent limitations. CNN-based models are effective in capturing fine-grained local features and preserving boundary details but suffer from limited receptive fields and insufficient global context modeling. In contrast, Transformers excel at capturing long-range dependencies but often struggle to preserve precise spatial details and sharp object boundaries due to their patch-based representations and global attention mechanisms. In addition, their computational complexity may further limit their efficiency in high-resolution remote sensing tasks [26].

To address these challenges, hybrid CNN-Transformer architectures have recently emerged as a promising solution. Representative hybrid architectures, such as TransUNet [27] and CoAtNet [28], have demonstrated the effectiveness of integrating convolutional operations with Transformer-based global modeling in dense prediction tasks. In the field of remote sensing, UNetFormer [29], CMTFNet [30] and CTMFNet [31] further adapt this paradigm to complex scenarios characterized by large spatial extents and multi-scale object distributions, achieving improved segmentation performance in urban scene understanding.

Along with methodological advances, several large-scale building footprint data products have been released, including GABLE [32], 90_cities_BRA [33], and the East Asian Buildings dataset [34], which provide extensive spatial coverage at national and continental scales. Despite their wide applicability, most existing footprint products rely primarily on CNN-based architectures such as DeepLabv3+, BE-Net, and OCRNet. Due to limited receptive fields and insufficient modeling of global context, these methods often produce irregular or fragmented footprints, fail to fully extract large buildings, ignore small structures, or generate internal holes within building polygons [30]. These issues restrict their reliability for fine-grained urban analysis and 3D modeling applications.

Building height estimation represents another critical yet challenging component of 3D urban mapping. Conventional field-based surveying methods are highly accurate but labor-intensive, costly, and impractical for large-scale applications [35]. Remote sensing-based approaches, including stereo imagery, LiDAR, and interferometric SAR (InSAR), provide effective alternatives. LiDAR can acquire high-precision 3D point clouds but is constrained by high acquisition costs and limited spatial coverage [36,37]. InSAR enables large-area height estimation but suffers from phase unwrapping errors and instability in dense urban areas [38,39]. Shadow-based methods estimate building height using geometric relationships among shadow length, solar illumination parameters, and satellite viewing geometry, offering a cost-effective solution with single-temporal high-resolution optical imagery [40,41,42]. However, its accuracy might be affected by blurred shadow boundaries, overlapping shadows, or uneven terrain.

Several building-height products have also been developed, such as GHSL-2023 [43], WSF 3D [44], and CNBH-10m [45], which provide gridded height information at regional or global scales. Nevertheless, their relatively coarse spatial resolution makes it difficult to associate height values with individual buildings. Only a limited number of building footprint datasets include height attributes. For example, GABLE adopts a DL-based height estimation framework supervised by stereo-derived DSM data [32]. However, the availability and temporal update frequency of such high-quality DSM products remain limited, particularly in rapidly developing cities. 3D-GloBFP employs an XGBoost-based regression framework but requires extensive multi-source data, including remote sensing imagery, socioeconomic indicators, DEM, and DSM [46], resulting in high data dependence and computational complexity. Other products, such as CMAB, provide rich building attributes, including height, function, and age, but are limited to partial urban areas and lack continuous spatial coverage [47]. Moreover, many recent methods rely on high-quality reference height datasets for supervised learning, whose temporal representativeness directly constrains the timeliness of derived building height products, and updating such datasets is often costly and time-consuming.

To overcome these limitations, this study develops a building-centric representation framework with reduced data dependency for large-scale urban analysis using high-resolution optical imagery. Within this framework, building footprint extraction and shadow detection are formulated as essential intermediate tasks that provide critical geometric cues for subsequent height inference. A hybrid CNN-Transformer network is designed to jointly model local spatial details and global contextual dependencies, enabling reliable extraction of building footprints and shadows from single-source optical imagery. The extracted shadow information is then integrated into a geometry-driven formulation for building height estimation based on satellite imaging parameters. Compared with existing approaches that rely on stereo imagery, LiDAR, or supervised learning with DSM labels, the proposed framework requires only high-resolution optical imagery and satellite acquisition geometry, thereby significantly reducing data dependence and acquisition costs. The framework is demonstrated on Shanghai as a case study, showing its potential for automated generation of building footprints and heights. The proposed framework can be further extended to multi-temporal imagery for building change detection and disaster impact assessment, offering substantial potential for urban management and remote sensing-based 3D urban analysis.

2. Materials and Methods

2.1. Study Area and Dataset

2.1.1. Study Area

The study area, Shanghai, is located on the eastern coast of China (Figure 1). As one of the most densely populated and highly urbanized megacities in China, Shanghai serves as a national hub for the economy, finance, and shipping. The terrain in Shanghai is extremely flat, with slopes generally below 2° and negligible elevation variation, which minimizes terrain-induced effects in shadow-based height estimation. The central urban area is characterized by dense and diverse building types, including high-rise commercial, residential, and industrial structures, with significant height variability. In contrast, suburban districts (e.g., Qingpu, Fengxian, and Chongming) consist of low-density, irregularly distributed buildings mixed with vegetation, water bodies, and farmlands. Such a pronounced urban–rural gradient provides a representative landscape for multi-scale building extraction and height estimation.

2.1.2. Remotely Sensed Data

The dual temporal remote sensing data jointly used in this study were acquired from the Jilin-1 high-resolution optical satellite constellation, operated by Chang Guang Satellite Technology Co., Ltd. (Changchun, China). The multispectral data with a spatial resolution of 1 m provide detailed information on building boundaries and texture. The first dataset, acquired on 15 May 2024, with minimal cloud cover, was used for building footprint extraction. The second dataset, acquired on 16 February 2024, under lower solar elevation conditions, provides elongated building shadows suitable for shadow detection and height estimation. To ensure spatial consistency, the two temporal datasets were geometrically co-registered at a pixel level, enabling accurate alignment between building footprints and their corresponding shadows.

A total of 500 image patches, each with a size of 512 × 512 pixels, were selected across Shanghai. A stratified sampling strategy was adopted to meet spatial heterogeneity. Homogeneous regions in rural areas were sampled sparsely, while complex urban areas were sampled more densely. This ensures samples’ representativeness and model generalization.

2.1.3. Label Generation for Building Footprints and Shadows

Ground-truth labels for building footprints and shadows were manually annotated based on the Jilin-1 imagery. For building footprints, only clear rooftop boundaries with regular geometric structures were labeled, while temporary structures and ambiguous objects were excluded. For shadow labels, only shadows cast by buildings were annotated, based on their spatial relationship with buildings and solar illumination direction. Shadows from trees, vehicles, and other objects were excluded. Ambiguous cases, such as dark rooftops, were carefully distinguished using spatial context and illumination direction. Buildings partially occluded by vegetation were labeled according to visible rooftop boundaries. To ensure label quality, all annotations were independently reviewed by a second annotator, and inconsistencies were corrected through manual verification.

2.1.4. Reference Data for Building Height Validation

Building height validation was conducted using two types of reference data. First, a 1 m resolution digital surface model (DSM) derived from Gaofen-7 (GF-7) stereo imagery was used, covering approximately 70% of Shanghai. The GF-7 data provide reliable elevation information due to their high spatial and vertical accuracy. Second, high-resolution UAV LiDAR data were used for local validation. The LiDAR-derived DSMs (6 cm and 10 cm resolution) were used to obtain accurate height measurements for 110 buildings. Building heights were derived using a quantile-based approach to reduce noise and edge effects. Specifically, rooftop elevation was estimated using the 95th percentile of DSM values within each building footprint, while ground elevation was estimated using the 5th percentile within a 3 m buffer outside the building. The height was calculated as the difference between these two values.

2.2. Methodology

2.2.1. Network Architecture for Extracting Building Footprints and Shadows

A sparse edge-aware convolution-transformer neural network (SECT-Net) is proposed to extract building footprints and shadows. It is built upon the UNetFormer architecture [29], which combines a lightweight CNN encoder and a Transformer-based decoder with a global-local attention mechanism for efficient multi-scale contextual modeling.

To adapt the framework for building footprints extraction from high-resolution Jilin-1 remote sensing images, three modifications are conducted: (1) Multi-scale edge supervision module (MESM): During multi-scale feature extraction in the encoder, MESM explicitly supervises edge information at different resolutions for enhancing the network’s ability to capture fine building contour details; (2) Dual-path CNN-Transformer block (DP-CTB): Inspired by the sparse token transformers [48], the original global-local Transformer module is replaced with a dual-path CNN-Transformer block with sparse global attention. This design preserves the ability to capture global contextual dependencies while alleviating redundant global interactions through sparse token sampling; (3) Multi-scale auxiliary supervision (MAS): Auxiliary prediction heads are attached to intermediate layers of the decoder to provide additional supervision signals for multi-level features.

The overall architecture of SECT-Net comprises a ResNet50-based encoder and a DP-CTB-based decoder (Figure 2). The encoder begins with a convolutional stem (Conv Stem) and sequentially stacks four residual blocks (ResBlocks) to extract multi-scale features at 1/4, 1/8, 1/16, and 1/32 of the input resolution. During the encoding stage, the MESM is applied to each scale of the feature map to explicitly enhance the learning of the building’s edge information. In the decoding phase, multi-scale features are fused via a weighted integration mechanism and subsequently passed through a series of DP-CTB modules for progressive feature aggregation and semantic reconstruction. Finally, the feature refinement head [29] refines the fused features to produce predictions of building footprints and shadows.

(1)

Multi-scale Edge Supervision Module (MESM)

Edge information is a critical fine-grained feature in building extraction. To enhance boundary modeling, the proposed MESM introduces explicit edge supervision at multiple encoder stages and injects edge features into the decoder using scale-specific strategies.

Specifically, the encoder outputs multi-scale feature maps $F_1\in {\mathbb{R}}^{C_1\times H_1\times W_1}$ and $F_2\in {\mathbb{R}}^{C_2\times H_2\times W_2}$, corresponding to the shallow and intermediate layers. For each scale, a lightweight edge head $E{H}_i$ is designed to generate corresponding edge predictions:

$$E_i=E{H}_i\left(F_i\right), i=\mathrm{1,2},$$

(1)

where $E_i\in {\mathbb{R}}^{1\times H_i\times W_i}$ denotes the predicted edge map at the $i$-th scale. $E{H}_i$ is implemented as a lightweight convolutional module composed of a 3 × 3 convolution, batch normalization, and ReLU activation, followed by a 1 × 1 convolution for channel projection.

To effectively utilize edge information, different integration strategies are adopted for different scales. For the intermediate stage, the predicted edge map $E_2$ is concatenated with $F_2$ along the channel dimension to form an enhanced feature representation:

$$F_2^{aug}=Concat\left(F_2, \sigma \left(E_2\right)\right),$$

(2)

where $Concat(·)$ denotes concatenation along the channel dimension. $\sigma (·)$ is the sigmoid activation.

For the shallow stage, the finer edge prediction $E_1$ is used to guide feature refinement via a spatial gating mechanism:

$$F_1^{aug}=F_1\odot \left({1+G}_1\right),$$

(3)

where $\odot$ denotes element-wise multiplication, and $G_1$ is a learnable gating map derived from $E_1$ via a 1 × 1 convolution. This mechanism enables the network to focus on fine-grained boundary details during reconstruction.

During training, edge ground truths are generated by rasterizing building footprint vectors and extracting inner boundaries, producing single-channel masks for supervision.

(2)

Dual-path CNN-Transformer Block (DP-CTB)

To balance global structural consistency and local detail accuracy, a DP-CTB is proposed (Figure 3a). This module consists of a local convolutional branch and a sparse global attention branch.

Given an input feature map $X{\in \mathbb{R}}^{C\times H\times W}$, the local branch aims to capture fine-grained spatial and channel interactions using depth-wise separable convolution followed by channel recalibration via a squeeze-and-excitation mechanism, yielding a local feature representation $X_{local}$.

To facilitate global context modeling, the global branch first projects the input into a reduced feature space via a 1 × 1 convolution. A Spatial-Channel Token Sampler (Figure 3b) is then employed to select informative tokens from both spatial and channel dimensions.

For spatial token sampling, a three-route strategy is adopted to enhance diversity and robustness: block-aware sampling ensures uniform spatial coverage, boundary-aware sampling emphasizes high-gradient regions (e.g., object contours) via gradient magnitude estimation, and region-aware sampling focuses on semantically salient areas. The sampled tokens are aggregated as:

$$T_s=Merge\left(T_{block},T_{boundary},T_{region}\right),$$

(4)

where $Merge(·)$ denotes concatenation followed by duplicate removal.

In parallel, channel tokens are selected based on channel importance derived from globally pooled features and further aggregated to produce channel-wise modulation.

In practice, the number of spatial and channel tokens is fixed to $K_s$ = 196 and $K_c$ = 32, respectively. For spatial sampling, the proportions of block-aware, boundary-aware, and region-aware tokens are set to 0.3, 0.3, and 0.4, respectively.

Based on the sampled tokens, position-enhanced sparse attention is performed:

$$Attn(Q,K,V)=Softmax\left({\displaystyle \frac{Q{K}^T+Q{P}_K^T+P_Q{K}^T}{\sqrt{d}}}\right)V,$$

(5)

where $Q,K,V$ denote query, key, and value matrices, and $P_Q$, $P_K$ represent learnable positional embeddings.

The attended tokens are projected back to the spatial domain via cross-attention, producing a spatial global feature map $X_S\in {\mathbb{R}}^{C_r\times H\times W}$. In parallel, channel attention yields a channel-enhanced feature map $X_C\in {\mathbb{R}}^{C_r\times H\times W}$.

The spatial and channel global features are concatenated and projected:

$$X_{global}=Conv\left(\left[X_S,X_C\right]\right).$$

(6)

Finally, the outputs of the local and global branches are adaptively fused:

$$X_{out}=\alpha X_{local}+\left(1-\alpha \right)X_{global},$$

(7)

where $\alpha \in [0, 1]$ is a learnable balance factor.

The DP-CTB effectively integrates fine-grained local details with long-range structural dependencies through a structure-aware sparse modeling strategy. Instead of performing dense global interactions, the proposed token sampling mechanism selectively focuses on informative spatial regions (e.g., boundaries and salient areas), thereby reducing redundant computations while preserving representative global context.

(3)

Multi-scale Auxiliary Supervision (MAS)

To enhance semantic consistency and facilitate optimization, a multi-scale auxiliary supervision module is introduced into the multi-stage decoder features.

Specifically, the decoder generates three intermediate feature maps at different stages:

$$h_4\in {\mathbb{R}}^{C\times H_4\times W_4}, h_3\in {\mathbb{R}}^{C\times H_3\times W_3}, h_2\in {\mathbb{R}}^{C\times H_2\times W_2},$$

(8)

here, $h_4$ originates from the deepest decoder stage and contains the richest semantic information, while $h_2$ comes from a shallower stage and preserves more local details.

For each feature map, a lightweight auxiliary head $A{H}_i(·)$ is designed to generate a probability map at the corresponding scale:

$$A_i=A{H}_i\left(h_i\right) i=2, 3, 4 .$$

(9)

where $A_i\in {\mathbb{R}}^{1\times H_i\times W_i}$ denotes the predicted probability map at the i-th scale. Each $A{H}_i$ consists of a convolutional block with a 3 × 3 convolution, followed by batch normalization and ReLU activation, a dropout layer, and a 1 × 1 convolution for channel projection.

During training, these auxiliary predictions are up-sampled to the original input resolution to compute the loss and provide supervision. This design facilitates gradient propagation to shallow layers, accelerates model convergence, and enhances the semantic representation capability across multiple scales.

(4)

Loss Function

In the training phase, the proposed network is supervised by a composite loss function that consists of three components: a principal loss $L_{\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{i}\mathrm{p}\mathrm{a}\mathrm{l}}$, an auxiliary loss $L_{\mathrm{a}\mathrm{u}\mathrm{x}}$, and an edge loss $L_{\mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}}$. The overall loss can be expressed as:

$$L_{total}=L_{principal}+{\lambda }_{\mathrm{a}\mathrm{u}\mathrm{x}}{L}_{\mathrm{a}\mathrm{u}\mathrm{x}}+{\lambda }_{\mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}}{L}_{\mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}},$$

(10)

where ${\lambda }_{\mathrm{a}\mathrm{u}\mathrm{x}}$ and ${\lambda }_{\mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}}$ are the weight coefficients of the auxiliary and edge losses, respectively. In our experiments, ${\lambda }_{\mathrm{a}\mathrm{u}\mathrm{x}}$ is set to 0.4 and ${\lambda }_{\mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}}$ is set to 0.2 by default.

The principal loss supervises the final prediction of building footprints and shadows. It is formulated as a combination of the soft Dice loss $L_{\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{e}}$ and the cross-entropy loss $L_{\mathrm{c}\mathrm{e}}$:

$$L_{principal}=L_{\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{e}}+L_{\mathrm{c}\mathrm{e}},$$

(11)

where $L_{\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{e}}$ denotes the soft Dice loss computed on predicted probabilities, and $L_{\mathrm{c}\mathrm{e}}$ denotes the pixel-wise cross-entropy loss.

The auxiliary loss is designed to guide intermediate feature representations within the decoder. It is applied to the outputs of the auxiliary heads, which process features from multiple decoder stages to produce auxiliary predictions. Following the same formulation as the principal loss, each auxiliary prediction is supervised using a combination of Dice loss $L_{\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{e}}$ and the cross-entropy loss $L_{\mathrm{c}\mathrm{e}}$:

$$L_{\mathrm{a}\mathrm{u}\mathrm{x}}=\sum _{i\in \{\mathrm{2,3},4\}}{\alpha }_i\left(L_{dice}^i+L_{ce}^i\right),$$

(12)

where ${\alpha }_i$ denotes the weight assigned to the auxiliary prediction at stage $i$. $L_{dice}^i$ and $L_{ce}^i$ denote the losses computed for the $i$-th auxiliary prediction. In our implementation, the auxiliary losses are weighted as $\left({\alpha }_4,{\alpha }_3,{\alpha }_2\right)=\left(\mathrm{0.1,0.2,0.4}\right)$, where shallower features are given larger weights to emphasize fine-grained details.

During training, both edge predictions are supervised at the original image resolution. To address the severe class imbalance between boundary and non-boundary pixels, a dynamically weighted binary cross-entropy loss is adopted. In addition, a Dice loss is introduced to enforce structural consistency of the predicted edges. The final edge loss is defined as a weighted combination of the two scales:

$$L_{edge}=0.4L_{edge}^1+0.6L_{edge}^2.$$

(13)

2.2.2. Building Height Estimation Based on Shadow Lengths

Shadow-based height estimation from single-view high-resolution imagery is widely used. The building height is calculated from the extracted shadow length projected onto the ground and several key imaging-geometry parameters, such as the solar elevation angle, solar azimuth, sensor elevation angle, and sensor azimuth (Figure 4).

For the case where the sun and the sensor are on opposite sides of the building, the sensor can capture the complete ground shadow BC of the building (Figure 4a). The building height H is calculated by:

$$H=L_s\times \tan \beta .$$

(14)

When the sun and sensor are on the same side of the building, part of the shadow (segment BE in Figure 4b) may be occluded by the building itself and thus cannot be fully observed. In this case, the shadow length measured in the image is denoted as $L_s$, which corresponds to the projection along the sensor’s line of sight (EC) and is derived from the detected building rooftop area and the shadow region (Figure 4b). Building height H is then calculated using the following geometric correction formula [49]:

$$H=(L_s·\tan \beta ·\tan \alpha ·\sin \gamma )/(\tan \alpha ·\sin \gamma -\tan \beta ·\sin \theta ),$$

(15)

where α denotes the sensor elevation angle, β the solar elevation angle, θ the sensor azimuth, γ the solar azimuth. $L_s$ denotes the measured shadow length in the image, which corresponds to the complete ground shadow BC in the opposite-side case (Figure 4a) and the observable shadow segment EC due to partial occlusion of the ground shadow (Figure 4b).

The Jilin-1 images used for shadow-based height estimation were acquired on 16 February 2024. The solar elevation and azimuth angles at acquisition time were 40.8° and 149.2°, respectively. Satellite’s elevation and azimuth angles were 81.1° and 148.8°, respectively. Both the sun and the sensor were positioned on the same side of the buildings. Shadow occlusion detection was first conducted for each extracted building using its footprint and the shadow geometry (Figure 5). Building footprints often fall within shadows cast by neighboring buildings. When a large proportion of the footprint area is covered by neighboring shadows, the building’s own shadow region becomes difficult to reliably identify, leading to unstable shadow-length measurements. In this study, an overlap ratio threshold of 70% was adopted to filter out severely occluded samples. Buildings exceeding this threshold were excluded from further analysis.

According to the solar azimuth angles, projection sampling points were uniformly generated along the building boundary (N = 50). Only points located on the shadow-facing side were retained, resulting in approximately 20–30 valid sampling points per building. For each valid sampling point, pixel-level shadow tracing along the solar illumination direction continues until a non-shadow pixel is encountered. As a result, multiple candidate shadow-length samples were generated for each building. To reduce the effects of roof self-occlusion and overshadowing by nearby buildings, all candidate shadow-length samples were subjected to a $3\sigma$ outlier filtering procedure [50]. Specifically, the mean $\mu$ and standard deviation $\sigma$ were first computed for a set of shadow-length $L_i$ and samples were removed when satisfying:

$$\left|L_i-\mu \right|>3\sigma .$$

(16)

For each building, the effective shadow length was defined as the mean of all high-quality shadow-length samples that passed the quality control procedure. This representative shadow length was subsequently used to estimate the building height by Equation (15).

2.2.3. Comparison Methods and Evaluation Matrices

In our experiments, several representative deep learning methods, including U-Net [51], DeepLabV3+ [52], Swin-UNet [53], SegFormer [54], and UNetFormer [29], are compared with the proposed SECT-Net. These methods cover both CNN-based and recent Transformer-based segmentation architectures to ensure a comprehensive evaluation. To ensure a fair comparison, all models were trained on the same dataset with identical training/validation/test splits (8:1:1) at the patch level, where each sample corresponds to a 512 × 512 non-overlapping image tile, ensuring no spatial overlap between samples. All models were trained for 100 epochs using the AdamW optimizer with a batch size of 8. For CNN-based models (e.g., U-Net and DeepLabV3+) and CNN-Transformer hybrid models (e.g., UNetFormer and SECT-Net), a ResNet50 backbone was adopted with a unified learning rate of 6 × 10⁻⁴. For Transformer-based models (e.g., Swin-UNet and SegFormer), we followed their commonly recommended training configurations to ensure stable and competitive performance. In terms of architecture, Swin-UNet employs a hierarchical Swin Transformer as the encoder, while SegFormer adopts the MiT-B2 backbone, both providing model capacities broadly comparable to ResNet50. To mitigate overfitting and improve model performance, data augmentation techniques, including random vertical and horizontal flipping and random rotation, are applied to the training dataset.

Three widely used pixel-level evaluation metrics, namely intersection-over-union (IoU), F1-score, and overall accuracy (OA), were employed to quantitatively evaluate the segmentation performance of building footprints and shadows. The metrics are computed from pixel-wise classification results on the test set and averaged over all test images. They are calculated as follows:

$$IoU=TP/(TP+FP+FN),$$

(17)

$$F1 score=(2\times TP)/(2\times TP+FP+FN),$$

(18)

$$OA={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}},$$

(19)

where TP (true positive) denotes the number of pixels correctly predicted as the target class (i.e., building footprint or shadow, depending on the evaluated task). TN (true negative) denotes the number of pixels correctly predicted as non-target class. FP (false positive) represents the number of non-target pixels incorrectly predicted as the target class. FN (false negative) represents the number of target pixels incorrectly predicted as the non-target class.

For building height estimation, both Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) are used for the evaluation:

$$RMSE=\sqrt{({\displaystyle \sum _{i=1}^N}{({\widehat{H}}_i-H_i)}^2)/N},$$

(20)

$$MAE={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left|{\widehat{H}}_i-H_i\right|,$$

(21)

where N is the total number of buildings used for height evaluation; ${\widehat{H}}_i$ and $H_i$ represent the predicted and reference building heights for the i-th sample, respectively.

3. Results

3.1. Performance Assessment of Models for Mapping Building Footprints and Shadows

The evaluation metrics in

Table 1. Quantitative comparison of building footprint and shadow extraction on the test set.

Method	${\mathit{I}\mathit{o}\mathit{U}}_{\mathbf{f}\mathbf{o}\mathbf{o}\mathbf{t}\mathbf{p}\mathbf{r}\mathbf{i}\mathbf{n}\mathbf{t}}$	${\mathit{F}1}_{\mathbf{f}\mathbf{o}\mathbf{o}\mathbf{t}\mathbf{p}\mathbf{r}\mathbf{i}\mathbf{n}\mathbf{t}}$	${\mathit{O}\mathit{A}}_{\mathbf{f}\mathbf{o}\mathbf{o}\mathbf{t}\mathbf{p}\mathbf{r}\mathbf{i}\mathbf{n}\mathbf{t}}$	${\mathit{I}\mathit{o}\mathit{U}}_{\mathbf{s}\mathbf{h}\mathbf{a}\mathbf{d}\mathbf{o}\mathbf{w}}$	${\mathit{F}1}_{\mathbf{s}\mathbf{h}\mathbf{a}\mathbf{d}\mathbf{o}\mathbf{w}}$	${\mathit{O}\mathit{A}}_{\mathbf{s}\mathbf{h}\mathbf{a}\mathbf{d}\mathbf{o}\mathbf{w}}$
U-Net	63.27	74.07	96.12	57.49	69.86	97.60
DeepLabv3+	62.52	73.83	95.90	63.65	77.79	96.69
Swin-UNet	74.39	85.32	96.72	74.65	85.49	97.72
SegFormer	73.62	84.81	96.47	71.70	83.52	97.45
UNetFormer	75.72	86.15	96.91	74.13	85.14	97.67
SECT-Net	77.96	87.62	97.16	75.01	85.72	97.75

present the classification performance of the proposed SECT-Net and several representative deep learning approaches for building footprint and shadow extraction.

Overall, SECT-Net demonstrates competitive and consistently strong performance across most evaluation metrics. For building footprint extraction, SECT-Net achieves the highest scores in terms of IoU (77.96%), F1-score (87.62%), and OA (97.16%). For shadow extraction, SECT-Net also attains strong performance, with IoU, F1-score, and OA values of 75.01%, 85.72%, and 97.75%, respectively.

To further assess whether the observed performance differences are statistically significant, we conducted a statistical significance analysis using a one-sided Wilcoxon signed-rank test. The one-sided test was adopted based on an a priori hypothesis that SECT-Net improves performance over baseline methods.

Specifically, the test was performed on the paired distributions of per-image Overall Accuracy (OA) calculated for each individual image in the test set (N = 50). The statistical comparison was conducted on per-image samples rather than aggregated mean values. The results are summarized in

Table 2. Statistical significance test results based on per-image Overall Accuracy (OA).

Comparison	Building Footprint OA (p-Value)	Building Shadow OA (p-Value)
SECT-Net vs. U-Net	<0.001	0.0886
SECT-Net vs. DeepLabv3+	<0.001	<0.001
SECT-Net vs. Swin-UNet	0.0018	0.9999
SECT-Net vs. SegFormer	<0.001	<0.001
SECT-Net vs. UNetFormer	<0.001	<0.001

Statistical significance was evaluated using a one-sided Wilcoxon signed-rank test. Differences were considered significant at p < 0.05.

.

For building footprint extraction, SECT-Net demonstrates statistically significant improvements (p < 0.05) over all baseline methods. In contrast, for shadow extraction, statistically significant improvements are observed compared with DeepLabv3+, SegFormer, and UNetFormer, while the differences with U-Net and Swin-UNet are not statistically significant (p ≥ 0.05).

This indicates that, although SECT-Net achieves slightly higher average performance in shadow extraction, the improvement is not consistently observed across all test samples. The variability in shadow characteristics, such as low contrast and irregular boundaries, may lead to unstable performance differences between models, thereby reducing statistical significance.

As illustrated in Figure 6, the proposed SECT-Net produces building segmentation results with more coherent structures and improved completeness, effectively reducing fragmented regions and internal holes. Traditional CNN-based methods, such as U-Net and DeepLabv3+, are more susceptible to local appearance variations and image noise due to their limited receptive fields. As a result, they often produce irregular boundaries and suffer from fragmented predictions or internal holes. Transformer-based approaches, including Swin-UNet and SegFormer, improve the overall structural consistency by modeling global context. However, they may still suffer from boundary ambiguity or incomplete delineation in regions with complex building layouts or low contrast. UNetFormer shows competitive performance in scenes with clear foreground–background contrast, but its predictions degrade in challenging areas with blurred boundaries or closely spaced buildings. In contrast, by jointly integrating multi-scale edge supervision and global-local contextual modeling, SECT-Net achieves more accurate and structurally consistent building delineation, particularly in complex urban scenes.

Building shadows exhibit distinctive tonal characteristics in high-resolution imagery, allowing generally consistent shadow locations and spatial distributions across methods (Figure 7). However, notable differences remain in shadow completeness and boundary continuity. CNN-based methods, such as U-Net and DeepLabv3+, tend to produce fragmented shadow regions with noticeable internal holes. They are also prone to misclassifying dark objects (e.g., dark rooftops or the shadow of overpasses) as building shadows. Transformer-based approaches, including Swin-UNet and SegFormer, significantly alleviate the issues of internal holes and speckle noise by incorporating global context modeling. Nevertheless, they still exhibit limitations in precise boundary localization, often leading to over-smoothed predictions or undesired merging of adjacent shadow regions. UNetFormer achieves relatively competitive performance in regions with clear shadow–background contrast. However, its predictions become less reliable in challenging scenarios, such as closely spaced buildings or blurred shadow boundaries, where local ambiguities dominate. In contrast, the proposed SECT-Net achieves consistent improvements by effectively integrating multi-scale edge supervision with global-local contextual modeling. It produces more complete shadow regions with clearer boundaries, while suppressing misclassification and reducing shadow adhesion effects.

3.2. Spatial Distribution of Building Footprints and Heights

Using the proposed SECT-Net, we produced complete and spatially consistent building footprints across the entire Shanghai. As shown in Figure 8, the extracted buildings exhibit diverse morphological characteristics under heterogeneous urban contexts. Specifically, typical patterns include mixed-use urban areas integrating residential, commercial, and service functions (Figure 8a), large-scale industrial complexes with regular geometries (Figure 8b), low-density villa-type residential areas (Figure 8c), rural residential areas with scattered homesteads (Figure 8d), coastal port and logistics zones with large structured facilities (Figure 8e), and urban villages characterized by highly compact and irregular building layouts (Figure 8f).

Within the study area, the heights of 755,996 buildings were derived using the shadow-based height estimation method (Figure 9d) and aligned with a local LoD-1 3D building model (Figure 9f). In terms of height statistics, 571,440 buildings (76%) are shorter than 10 m, indicating that Shanghai is predominantly composed of low-rise residential structures. Buildings with a height of 10 to 30 m account for 19% of the total (143,204). Only 323 high-rise buildings exceed 100 m in height (i.e., <1% of the total), and they are mainly concentrated in central urban areas such as Huangpu, Jing’an, and Hongkou, as well as in the Lujiazui area of Pudong. The spatial distribution of high-rise buildings exhibits a clear core-periphery pattern, with dense concentrations in central business districts, and only a few are scattered in the suburbs.

Accuracy assessment was conducted for 559,614 buildings using reference heights extracted from the GF-7-derived DSM. The residual analysis (Figure 10a) indicates that 73% of height errors fall within the interval of [−5.0 m, 5.0 m]. Overall, a mean bias of −2.29 m indicates a slight underestimation. From the scatterplot of estimated vs. reference heights, the vast majority of them lie near the 1:1 line, indicating better agreement between predictions and reference values (Figure 10b). Regression analysis yields an R² of 0.74, an RMSE of 5.66 m, and an MAE of 3.96 m. However, the fitted trend line consistently lies below the 1:1 line, further confirming a systematic underestimation bias.

To provide a more comprehensive evaluation, we conducted a stratified error analysis across five height intervals (

Table 3. Stratified accuracy assessment of building height estimation across different height ranges.

Height Range	RMSE (m)	MAE (m)	N
<10 m	3.487	2.753	343,184
10–30 m	6.697	5.389	186,722
30–50 m	9.692	7.305	19,450
50–100 m	15.499	11.246	9936
>100 m	60.865	41.961	322

). The results show a clear increase in error with building height. For low-rise buildings (<10 m), the estimation is most accurate (RMSE = 3.49 m, MAE = 2.75 m). Errors increase gradually for 10–30 m (RMSE = 6.70 m) and more noticeably for 30–100 m (RMSE = 9.69–15.50 m). For high-rise buildings (>100 m), the error rises sharply (RMSE = 60.87 m, MAE = 41.96 m), indicating significant performance degradation. Overall, the method performs reliably for low- and medium-height buildings, but its accuracy decreases substantially for very tall structures.

By removing samples heavily affected by building occlusion or vegetation-induced shadow contamination (accounting for approximately 6% of the total samples), the accuracy of height estimation substantially improves, with R² increasing from 0.74 to 0.80 and RMSE decreasing from 5.66 m to 4.46 m (Figure 10c). These results provide quantitative evidence that occlusion and vegetation interference are major contributors to the observed systematic underestimation.

In addition to the GF-7 DSMs, UAV LiDAR-derived building heights were employed to assess local-scale estimation accuracy. The estimated heights of 110 buildings were compared with those derived from UAV LiDAR heights (Figure 10d). The results reveal a moderate positive correlation between the estimated and reference heights, yielding an R² of 0.49, an RMSE of 6.64 m and an MAE of 5.44 m. Most samples are concentrated in the low- to medium-height range (<60 m), where the estimated heights generally align with the reference values with moderate dispersion. Nevertheless, the regression slope of 0.894 and a negative intercept indicate a persistent tendency to underestimation, particularly for three taller buildings, consistent with the bias features observed in the GF-7 DSM-based validation.

To further examine the impact of the selected filtering threshold on height estimation, a sensitivity analysis was conducted based on different shadow overlap ratios (

Table 4. Sensitivity analysis of occlusion filtering threshold.

Threshold	Building Count	R2	RMSE (m)	MAE (m)
50%	555,494	0.738	5.646	3.960
60%	557,762	0.737	5.658	3.962
70%	559,515	0.736	5.662	3.964
80%	561,194	0.733	5.701	3.975
90%	562,809	0.729	5.751	3.987

).

Although the 50% threshold yields the lowest RMSE (5.646 m), the improvement compared with the 70% threshold (5.662 m) is marginal (only 0.016 m), while more than 4000 additional buildings would be excluded. In contrast, higher thresholds, such as 80% and 90%, retain more severely occluded samples and lead to noticeable accuracy degradation. Considering both estimation accuracy and sample completeness, the 70% threshold provides the best trade-off and was therefore adopted in this study.

4. Discussion

4.1. Ablation Test of SECT-Net

To evaluate the contribution of each component in the proposed SECT-Net, ablation experiments were conducted on the Jilin-1 dataset for both building footprint and shadow extraction. The baseline model was implemented using the UNetFormer architecture with a ResNet-50 backbone and original specification. Quantitative results for both tasks are reported in

Table 5. Comparison of ablation experiment results of the SECT-Net.

Method	${\mathit{I}\mathit{o}\mathit{U}}_{\mathbf{b}\mathbf{u}\mathbf{i}\mathbf{l}\mathbf{d}\mathbf{i}\mathbf{n}\mathbf{g}}$	${\mathit{F}1}_{\mathbf{b}\mathbf{u}\mathbf{i}\mathbf{l}\mathbf{d}\mathbf{i}\mathbf{n}\mathbf{g}}$	${\mathit{O}\mathit{A}}_{\mathbf{b}\mathbf{u}\mathbf{i}\mathbf{l}\mathbf{d}\mathbf{i}\mathbf{n}\mathbf{g}}$	${\mathit{I}\mathit{o}\mathit{U}}_{\mathit{s}\mathit{h}\mathit{a}\mathit{d}\mathit{o}\mathit{w}}$	${\mathit{F}1}_{\mathit{s}\mathit{h}\mathit{a}\mathit{d}\mathit{o}\mathit{w}}$	${\mathit{O}\mathit{A}}_{\mathit{s}\mathit{h}\mathit{a}\mathit{d}\mathit{o}\mathit{w}}$
baseline	75.72	86.15	96.91	74.13	85.14	97.67
+MESM	77.05	86.90	97.08	74.93	85.67	97.77
+DP-CTB	77.40	87.34	97.12	74.66	85.49	97.74
+MAS	76.84	86.90	97.03	74.55	85.42	97.72
+MESM + DP-CTB + MAS	77.96	87.62	97.16	75.01	85.72	97.75

.

As shown in Table 5, the introduction of MESM improves the IoU of building footprints from 75.72% to 77.05% and shadow IoU from 74.13% to 74.93%, demonstrating its effectiveness in enhancing boundary localization for both object types. By leveraging multi-scale feature aggregation and explicit edge supervision, MESM produces sharper and more complete contours. However, due to its strong sensitivity to local gradient variations, MESM is easily affected by texture and tone changes within building roofs, leading to internal holes and occasional false positives in both footprint and shadow predictions.

When DP-CTB is incorporated, the IoU increases to 77.40% for buildings and 74.66% for shadows. As shown in Figure 11, DP-CTB effectively captures long-range contextual dependencies, thereby suppressing noise and improving regional consistency in both tasks. Compared with MESM, DP-CTB demonstrates greater ability to suppress speckle noise and reduce internal holes, yielding more complete and smoother segmentation results. To verify the effectiveness of the MAS module, we conducted comparative experiments by integrating MAS alone into the baseline model. As shown in Table 5, the MAS module further improves the IoU to 76.84% (building) and 74.55% (shadow). MAS provides multi-scale semantic supervision, mitigating gradient vanishing and enhancing feature interaction between deep and shallow layers. This improves model consistency and training stability, and reduces speckle noise, while qualitative results show better alignment between predictions and ground truth (Figure 11).

Furthermore, when MESM is jointly applied with DP-CTB and MAS, the model achieves its best performance with an IoU reaching 77.96% for buildings and 75.01% for shadows. This indicates that the three modules act in a complementary manner rather than as independent components. Specifically, MESM enhances boundary awareness, DP-CTB captures long-range contextual dependencies, and MAS improves multi-scale feature consistency during optimization. Their joint use promotes effective feature interaction. Edge-enhanced features support global context modeling, while global representations help suppress local noise, resulting in more accurate and stable segmentation.

To further validate the selection of key hyperparameters in the proposed DP-CTB, a sensitivity analysis was conducted on the token sampling strategy, including the number of spatial tokens ($K_s$), the number of channel tokens ($K_c$), and the sampling ratios of block-aware, boundary-aware, and region-aware tokens. The quantitative results are presented in

Table 6. Sensitivity analysis of token sampling strategy.

${\mathit{K}}_{\mathit{s}}$	${\mathit{K}}_{\mathit{c}}$	${\mathit{r}\mathit{a}\mathit{t}\mathit{i}\mathit{o}}_{\mathbf{b}\mathbf{l}\mathbf{o}\mathbf{c}\mathbf{k}}$	${\mathit{r}\mathit{a}\mathit{t}\mathit{i}\mathit{o}}_{\mathbf{b}\mathbf{o}\mathbf{u}\mathbf{n}\mathbf{d}\mathbf{a}\mathbf{r}\mathbf{y}}$	${\mathit{r}\mathit{a}\mathit{t}\mathit{i}\mathit{o}}_{\mathbf{r}\mathbf{e}\mathbf{g}\mathbf{i}\mathbf{o}\mathbf{n}}$	${\mathit{I}\mathit{o}\mathit{U}}_{\mathbf{b}\mathbf{u}\mathbf{i}\mathbf{l}\mathbf{d}\mathbf{i}\mathbf{n}\mathbf{g}}$	${\mathit{I}\mathit{o}\mathit{U}}_{\mathbf{s}\mathbf{h}\mathbf{a}\mathbf{d}\mathbf{o}\mathbf{w}}$
64	32	0.3	0.3	0.4	76.15	74.11
128	32	0.3	0.3	0.4	76.88	74.57
196	32	0.3	0.3	0.4	77.40	74.66
256	32	0.3	0.3	0.4	76.51	74.15
196	8	0.3	0.3	0.4	76.35	73.89
196	16	0.3	0.3	0.4	76.65	74.44
196	32	0.3	0.3	0.4	77.40	74.66
196	64	0.3	0.3	0.4	77.06	74.71
196	32	1.0	0.0	0.0	76.74	74.42
196	32	0.0	1.0	0.0	76.71	74.25
196	32	0.0	0.0	1.0	76.84	74.57
196	32	0.3	0.3	0.4	77.40	74.66
196	32	0.3	0.2	0.5	77.05	74.59
196	32	0.5	0.2	0.3	77.00	74.73
196	32	0.4	0.4	0.2	77.03	74.60

.

For $K_s$, performance peaks at 196 and degrades beyond that, indicating that excessive spatial tokens may introduce redundant information and unnecessary computational overhead. For $K_c$ = 32, further increasing the number of channel tokens only brings marginal improvement in shadow extraction while slightly reducing building segmentation accuracy, suggesting limited benefit from overly dense channel interactions. For sampling ratios, single-strategy sampling consistently underperforms mixed strategies, confirming that block, boundary, and region cues are complementary. The ratio 0.3/0.3/0.4 achieves the best trade-off across both tasks.

4.2. Comparison of Different Building Footprint Products

A quantitative comparison was conducted between our produced building footprint product and three existing large-scale datasets—GABLE, 90_cities_BRA, and East Asian Buildings—on the Shanghai test dataset (

Table 7. Evaluation metrics of building footprint products on the test set.

Dataset	Pixel-Level			Boundary-Level	Instance-Level
	IoU	F1	OA	BF1	${\mathit{P}}_{0.5}^{\mathit{I}\mathit{o}\mathit{U}}$	${\mathit{R}}_{0.5}^{\mathit{I}\mathit{o}\mathit{U}}$	${\mathit{F}1}_{0.5}^{\mathit{I}\mathit{o}\mathit{U}}$
GABLE [32]	45.61	62.65	92.20	67.23	17.69	35.89	23.70
90_cities_BRA [33]	51.89	68.33	93.02	78.82	26.93	39.79	32.12
East Asian Buildings [34]	41.85	59.01	91.05	71.49	15.45	30.85	20.59
Ours	71.58	83.44	96.09	88.35	68.50	54.35	60.61

). In addition to pixel-wise metrics (IoU, F1-score, and OA), we further incorporate boundary- and instance-sensitive metrics to evaluate the building footprint products comprehensively. Specifically, the boundary F1-score (BF1) [55] is used to assess contour accuracy by measuring the correspondence between predicted and ground-truth boundaries within a predefined tolerance distance. In this study, a tolerance of 5 pixels is adopted following Guo et al. [23]. For instance-level evaluation, object-level precision, recall, and F1-score were computed on vectorized building polygons. A predicted polygon was regarded as a true positive if its IoU with a ground-truth polygon exceeded 0.5 under one-to-one matching [56].

Overall, our method achieves substantially higher performance across pixel-level, boundary-level, and instance-level metrics. In particular, it attains the highest IoU (71.58%), F1-score (83.44%), OA (96.09%), and BF1 (88.35%), indicating superior spatial consistency and contour completeness. At the instance level, our method also yields markedly higher object-level precision, recall, and F1-score under an IoU threshold of 0.5, demonstrating a stronger ability to preserve complete building objects in the vectorized footprint product.

The visual investigation reveals characteristic structural errors in the existing datasets when applied to complex urban environments in Shanghai (Figure 12). For the GABLE dataset, noticeable geometric inconsistencies are observed, including overlapping, misalignments, and discontinuities in the vectorized building outlines. These issues are likely related to its instance-based extraction framework with cascade contour refinement, which might be less stable in highly dense urban environments [32]. The 90_cities_BRA dataset contains numerous fragmented and isolated patches, which highlight the limitations of the CNN-based DeepLabv3+ architecture in forming coherent object-level representations [33]. The East Asian Buildings dataset tends to over-regularize building outlines via boundary enhancement, resulting in inaccurate rooftop boundaries for irregular or multi-section buildings [34]. These results indicate that our proposed method maintains robust performance across heterogeneous urban environments, providing a reliable basis for subsequent building height estimation and three-dimensional urban modeling.

In contrast, our method demonstrates a stronger ability to distinguish buildings from non-building objects such as vehicles, temporary structures, and paved surfaces, while effectively avoiding contour breakage and excessive geometric regularization. Compared with existing data products, our resulting building footprints exhibit markedly improved visual consistency, structural integrity, and object discrimination accuracy. By leveraging a DL-based roof segmentation model and a robust contour vectorization strategy, our model accurately identifies more buildings in large-scale urban areas. These improvements stem from the model’s enhanced capacity to learn building-related textures, materials, and structural patterns. This highlights its superior generalization performance in structurally complex urban environments.

4.3. Comparison of Different Building Height Products

As shown in

Table 8. Comparison of building height products under different data and methodological settings.

Product	Data Source	Method	Product Type	RMSE
CNBH-10 m [45]	Sentinel-1, PALSAR, Sentinel-2, LUOJIA 1–01, World population, SRTM, WSF 2019, Baidu building height	Random Forest	10 m Raster	6.1 m
GABLE [32]	Beijing-3, Gaofen-7, WorldView, WSF 2019	Semantic Flow Field-driven DSM Estimation network	Vector	3.7 m
3D-GloBFP [46]	Sentinel-1, Sentinel-2, DEM, DSM, World population, Nighttime light, Baidu building height	XGBoost	Vector	1.9–14.6 m
CMAB [47]	GES imagery, Amap road, POI, Baidu building height	XGBoost	Vector	7.6 m
He et al. [35]	Sentinel-1, Sentinel-2, SRTM, PALSAR, VIIRS, WSF 2019, ALOS AW3D 30, GEDI	Random Forest	30 m Raster	4.7–10.1 m
Ours	Jilin-1	Shadow-based algorithm	Vector	4.5–5.7 m

, the shadow-based method using a single-temporal Jilin-1 image achieves an RMSE of 5.7 m. Although this value falls within the range reported by several existing large-scale building height products, it should be noted that some of these products are generated at coarser spatial resolutions (e.g., 10 m or 30 m), and their accuracies are not strictly quantified. Nevertheless, the comparison provides a general indication that the proposed method can achieve competitive performance while relying on significantly simpler data inputs. These existing products are typically generated using data-driven frameworks that integrate heterogeneous multi-source datasets. In recent years, monocular depth estimation has emerged as a promising alternative for building height retrieval [57,58]. For example, GABLE [32] employs a DL-based DSM estimation network trained on stereo-derived reference heights. Although such learning-based approaches can achieve superior accuracy, their performance depends heavily on the availability of high-quality reference height datasets and on complex training pipelines.

Data-driven approaches typically require multiple heterogeneous datasets and reference height data to implement regression. Monocular depth estimation methods primarily rely on highly accurate reference-height data for supervised training and often require transfer learning to generalize to regions without pixel-level height labels [59,60]. In contrast, our proposed shadow-based method exploits explicit geometric relationships between building shadows and solar illumination, enabling height estimation with minimal data requirements. This advantage makes it particularly suitable for large-scale, data-scarce, or time-sensitive urban applications, where multi-view reconstruction or heavily supervised learning-based methods are difficult to deploy or maintain.

4.4. Uncertainty Analysis and Limitations

Despite the promising performance of the proposed framework, several sources of uncertainty may affect the accuracy of building height estimation. These uncertainties mainly arise from errors in shadow extraction and footprint delineation, as well as the propagation of these errors through the geometry-based estimation process. In addition, inconsistencies may be introduced by the use of multi-temporal imagery.

First, uncertainties are primarily introduced by errors in shadow extraction and footprint delineation, and their propagation through the geometry-based estimation process. Occlusion, vegetation interference, and low contrast may affect the accuracy of the extracted shadow length $L_s$, while footprint boundary inaccuracies may further bias its measurement. According to Equation (15), the height estimation formula can be equivalently rewritten as $H=K\cdot L_s$, where $K$ is a constant determined by the solar and sensor geometry for a given image. This implies a linear error propagation relationship $\Delta H=K\cdot \Delta L_s$. For the Jilin-1 imagery used in this study, $K\approx 1.003$, indicating negligible geometric error amplification. Therefore, the overall uncertainty is mainly dominated by upstream extraction errors rather than the geometric transformation itself.

Second, the use of multi-temporal imagery for footprint and shadow extraction may introduce occasional mismatches. Although geometric co-registration is applied, discrepancies may still occur due to local urban dynamics, such as temporary structures or construction activities. As a result, shadows extracted from one acquisition may not perfectly correspond to building footprints from another, leading to localized shadow-footprint mismatches. However, given the relatively short temporal interval between the two acquisitions, such inconsistencies are generally limited and only affect a small number of cases.

These uncertainties are further reflected in a systematic bias observed in the height estimation results. Although the shadow-based height estimation generally shows strong agreement with the DSM reference heights, a consistent underestimation is observed (Figure 10). This bias is mainly attributed to occlusion effects in dense urban areas, as widely reported in previous shadow-based studies [49,61]. The complex and irregular geometries of high-rise buildings make accurate footprint delineation challenging. In addition, the close spacing of buildings in dense urban areas leads to severe shadow occlusion, resulting in shortened observable shadow lengths (Figure 13a–c). For relatively low-rise buildings, vegetation cover may further obscure shadow boundaries, introducing additional negative bias (Figure 13d–f).

In particular, the limitation becomes more pronounced for high-rise buildings (>100 m), where the RMSE reaches 60.87 m. Although such buildings account for only a small proportion of the total samples (322 buildings), their large errors substantially affect the stratified accuracy results. This indicates that single-temporal shadow-based methods are less reliable for extremely high-rise cases, especially in dense urban cores.

Future work will focus on improving height estimation for these challenging samples by incorporating multi-temporal imagery acquired under different solar elevation conditions. Images with higher solar elevation angles (e.g., May) produce shorter shadows, which can reduce occlusion and shadow projection errors, while winter images provide longer and clearer shadows for geometric estimation. Combining multi-temporal observations and multi-view results may further improve the robustness and reliability of high-rise building height retrieval.

5. Conclusions

In complex urban environments, accurately characterizing building outlines, shadows, and heights remains challenging due to spatial complexity and the occlusion effect. This study systematically investigated the automated extraction of city-scale urban building footprints and height information from high-resolution spaceborne remote sensing imagery. We proposed a hybrid architecture, SECT-Net, that integrates the strengths of both CNNs and Transformers. The experimental results demonstrate that SECT-Net achieves competitive and generally improved performance compared with several representative deep learning segmentation networks, thereby improving the stability and reliability of building height inversion.

Our study demonstrates the feasibility of high-resolution multispectral images acquired by the Jilin-1 satellite for building mapping and height estimation of Shanghai, China. Compared with several existing building datasets, our extracted building footprints exhibit better geometric integrity, fewer fragmented areas, and higher confidence in complex urban settings. With the high-quality building footprints, a shadow-based height estimation framework was developed to extract heights for over 750,000 buildings across the entire city. When evaluated against two reference datasets (GF-7 DSM and UAV LiDAR DSM), the estimated building heights achieved an RMSE ranging from 4.5 to 6.6 m, demonstrating favorable accuracy.

Our proposed systematic workflow demonstrates promising performance in extracting building footprints and estimating building heights in complex urban environments. However, the present study was only validated in Shanghai, which is characterized by relatively flat terrain and regular urban morphology. The generalization of the proposed method needs to be further investigated for other regions with diverse terrain conditions (e.g., mountainous areas) and varying urban structures. Satellite remote sensing is on the way toward acquiring high-resolution imagery, which will support the precise and rapid extraction of large-scale buildings. Future studies will focus on extensive cross-region validation and methodological improvement. In addition, we will explore joint inversion of occlusion-aware shadow modeling, multi-source heterogeneous data fusion, and multi-view imaging for high-precision 3D city model reconstruction. This study generally provides a potential pathway for LoD-1 3D city modeling to supports fine-scale urban management of smart city.