Local Climate Zone Mapping by Integrating Hyperspectral and Multispectral Data with a Spectral–Spatial Fusion Network

Highlights

What are the main findings?

• Hyperspectral–multispectral fusion effectively enhances scene-level Local Climate Zone (LCZ) mapping in complex urban environments.
• The LCZ-HMSSNet improves the discrimination of structurally similar classes by integrated data fusion and spatial–spectral feature separation.

What are the implications of the main findings?

• Hyperspectral–multispectral feature fusion provides a practical direction for improving LCZ mapping beyond scene-based multispectral approaches.
• The proposed framework serves as a reference for high-accuracy LCZ classification with limited labeled training data.

Abstract

Local Climate Zone (LCZ) classification provides a standardized framework for characterizing urban morphology and its climatic implications. However, most existing remote sensing-based LCZ mapping methods rely on pixel-level classification and multispectral data alone, which limits their ability to capture urban scene heterogeneity and to distinguish structurally similar LCZ classes. In this paper, we propose LCZ-HMSSNet, a deep learning framework for scene-level LCZ classification that integrates PRISMA hyperspectral images with Sentinel-2 multispectral data. The proposed approach exploits both the spectral richness of hyperspectral data and the spatial context provided by multispectral observations, and incorporates a spatial–spectral feature separation mechanism to enhance the discriminability of the fused representations. Experiments conducted across six representative European cities evaluate the proposed method from multiple perspectives, including comparisons with different classification models, data contribution analysis, and structural ablation studies. The results demonstrate that the proposed method consistently outperforms MSI-only and existing LCZ classification approaches, achieving an overall accuracy (OA) of 0.988 and a Kappa of 0.985. In addition, the small-sample experiments indicate the robustness and potential of the proposed model, providing a practical reference for future LCZ mapping in data-scarce scenarios.

1. Introduction

Urbanization, a major driver of global development, has profoundly transformed urban environments over the past decades. As natural pervious surfaces have been gradually replaced by various forms of artificial impervious cover, the consequent alterations in land cover and surface energy balance have substantially altered urban structure and landscape patterns. These changes have, in turn, contributed to urban climate effects, particularly the Urban Heat Island (UHI) phenomenon [1].

The Local Climate Zone (LCZ) classification system proposed by Stewart and Oke [2] provides a standardized framework for describing internal structure and surface characteristics of urban areas, effectively linking urban morphology with their thermal and climatic properties. As shown in Figure 1, the LCZ consists of 10 urban LCZ types and 7 natural LCZ types, defined primarily by differences in urban structure, land cover, surface materials, and human activity. To date, the LCZ framework has been widely adopted as a standard approach for monitoring and analyzing the UHI [3,4,5].

Figure 1. Illustration of the LCZ classification scheme, showing representative LCZ classes (top) and corresponding remote-sensing image examples (bottom). LCZ 1–10 denote urban built LCZ types, while LCZ A–G represent natural land cover classes. The subfigures were modified from [2]. The high-resolution example images are sourced from Google Earth imagery.

LCZ mapping is generally conducted using two main approaches: Geographic Information System (GIS)-based methods and Remote Sensing (RS)-based methods. Although GIS-based approaches provide accurate and detailed mapping of urban LCZ types, they are limited by data consistency and availability, as their performance depends on the quality and completeness of geographic datasets, hindering large-scale uniform mapping [6]. In contrast, RS-based methods offer a more consistent and globally scalable solution by using satellite imagery, enabling large-area LCZ mapping with improved spatial and temporal coverage.

Most existing RS-based LCZ mapping approaches rely on pixel-level classification, largely influenced by the framework established by the World Urban Database and Portal Tool (WUDAPT) [7]. The WUDAPT framework generates LCZ maps at a spatial resolution of 100 m using Landsat 8 images and a Random Forest (RF) classifier. Approximately 100 cities around the world have been mapped within the WUDAPT framework. However, the overall accuracy (OA) of most LCZ maps remains around 70%, with considerable variation across cities [8,9]. This relatively low accuracy is primarily attributable to the pixel-based classification strategy, which overlooks the spatial–contextual information essential for urban structure characterization.

With the growing application of deep learning (DL) techniques, scene-level LCZ classification has emerged as a promising alternative to traditional pixel-based approaches [6]. Rather than labeling each pixel or object separately, this approach processes image patches as integrated units, acknowledging that an LCZ usually encompasses diverse surface materials and land cover types. Convolutional neural networks (CNNs) are now the predominant DL architecture for LCZ mapping, given that their hierarchical convolutional structure effectively captures spatial and contextual information within image patches. Moreover, compared to traditional machine learning classifiers such as RF [8,10,11], CNN-based models have demonstrated higher accuracy and enhanced generalization capabilities across different cities and datasets [11,12]. Among the various CNN architectures, ResNet has shown particularly robust performance in recognizing complex LCZ patterns and consequently is frequently adopted as a baseline model in recent studies [11]. Building on this foundation, many studies have sought to further enhance CNN-based LCZ classification by incorporating attention mechanisms designed to more effectively capture the structural and contextual characteristics of urban scenes. For example, MSMLA-Net [13] integrated multi-scale features and a Convolutional Block Attention Module (CBAM) into a ResNet backbone to jointly capture spectral and spatial information, while LCZNet [14] combined Inception modules, residual blocks, and Squeeze-and-Excitation (SE) units to improve feature representation and discriminability.

It should be noted that LCZ classification targets non-homogeneous urban regions characterized by mixtures of buildings and land-cover types at the neighborhood scale, rather than fine-grained building-level details. In this context, Sentinel-2 multispectral images (MSI) with a spatial resolution of 10 m have been widely adopted in LCZ studies and are considered sufficient for capturing the spatial contextual information required for LCZ classification [6]. The release of the So2Sat LCZ42 dataset [15] has established a global benchmark for DL-based LCZ classification, by providing spatially diverse samples derived from Sentinel-1 and Sentinel-2 images. Based on this dataset, several models have been developed using Sentinel-2 data, including Sen2LCZ-Net [16], MCFUNet-LCZ [17], LCZ-MFKNet [18], and MSCA-MSLCZNet [19]. These studies demonstrate the effectiveness of Sentinel-2 MSI in capturing scene-level spatial information for LCZ mapping.

Recent LCZ studies have also explored other data modalities and task formulations to better capture urban complexity. For example, Li et al. [20] proposed a deep mixed-scene decomposition framework that reformulates LCZ mapping as a mixed-scene unmixing problem using high-resolution remote sensing imagery and auxiliary semantic cues, enabling more flexible modeling of heterogeneous urban blocks. In addition, van Capel et al. [21] investigated LCZ classification using spatio-temporal thermal satellite imagery, demonstrating that temporal thermal dynamics can provide complementary information for distinguishing structurally similar LCZ types. These studies highlight the growing interest in extending LCZ classification beyond conventional single-source optical imagery. Nevertheless, within the domain of optical remote sensing, most existing LCZ models still rely primarily on MSI as the main data source. However, MSI is characterized by a limited number of discrete spectral bands, which constrains its ability to discriminate urban surface materials. In contrast, hyperspectral images (HSI) provide dozens to hundreds of contiguous spectral bands, enabling the capture of fine spectral characteristics of ground features. This provides a unique advantage in discriminating surface materials with similar composition but subtle differences in spectral response (e.g., different roof types or road surfaces). Recent studies have demonstrated the potential of HSI, as its rich spectral information allows for a more detailed characterization of urban materials in LCZ mapping relative to MSI. Zhang et al. [22] integrated Zhuhai-1 hyperspectral data, ALOS DEM, and Luojia-1 nighttime light data for LCZ mapping, highlighting the role of hyperspectral information in capturing urban surface diversity. Similarly, Vavassori et al. [23] initially evaluated PRISMA (PRecursore IperSpettrale della Missione Applicativa) hyperspectral and Sentinel-2 multispectral data for LCZ mapping, confirming the potential of PRISMA data. Subsequent studies showed that combining PRISMA with Sentinel-2 enhances LCZ classification performance [24]. However, most of these studies have been limited to pixel-based approaches, and scene-level LCZ classification using HSI remains largely underexplored. Beyond LCZ mapping, recent hyperspectral image classification studies have explored foundation models for large-scale representation learning (e.g., HyperSIGMA [25], enhanced multiscale spectral–spatial feature fusion networks [26], and cross-scene learning frameworks with self-training strategies (e.g., DHSNet [27]), demonstrating strong performance in generic HSI classification tasks. A key challenge in this transition is that the high spectral richness of HSI often comes at the cost of spatial resolution, which is critical for interpreting urban structures at the scene level [28]. In contrast, MSI provides higher spatial resolution and richer spatial contextual information. Therefore, integrating HSI with MSI offers a natural and effective way to jointly exploit spectral discriminability and spatial structural information for robust scene-level LCZ mapping.

To address the limited spectral discriminability of MSI-based approaches and the spatial resolution constraints of pixel-level hyperspectral LCZ mapping, this study proposes LCZ-HMSSNet (Hyperspectral–Multispectral Spectral–Spatial Network for Local Climate Zone Classification), a scene-level LCZ classification framework that integrates hyperspectral and multispectral RS data.

Our main contributions can be summarized as follows.

(1)

This study presents a hyperspectral–multispectral fusion framework for scene-level LCZ classification that integrates PRISMA hyperspectral images with Sentinel-2 multispectral data. By jointly exploiting the complementary spectral and spatial characteristics of the two data sources, the proposed approach provides an effective solution for representing the heterogeneous composition of LCZ scenes.

(2)

A spatial–spectral feature separation (SSFS) strategy is introduced to enhance the discriminability of fused representations. This design explicitly models complementary information, facilitating the recognition of structurally similar but morphologically distinct LCZ categories.

(3)

Comprehensive experiments conducted on multi-city European datasets validate the effectiveness of the proposed framework. Compared with representative CNN-based baselines and single-source inputs, the proposed LCZ-HMSSNet achieves consistent improvements in Overall Accuracy (OA), Average Accuracy (AA), and Cohen’s Kappa. More importantly, clear performance advantages are observed for challenging and minority LCZ classes, where consistent improvements in per-class F1-score are observed for challenging and minority LCZ classes.

2. Proposed Dataset

2.1. Data Source

2.1.1. Sentinel-2

Sentinel-2 is part of the European Copernicus programme operated by the European Space Agency (ESA). It provides wide-swath, high-resolution multispectral imaging capabilities that support a range of operational land applications, including the monitoring of vegetation, soil, and water cover, as well as the observation of inland waterways and coastal areas. The satellite carries an optical instrument payload that acquires imagery across 13 spectral bands, including four bands with a ground sampling distance (GSD) of 10 m, six bands with 20 m GSD, and three bands with 60 m GSD [29]. Sentinel-2 provides one of the highest spatial resolutions among freely available satellite sensors, making it well suited for large-scale LCZ mapping. In this study, only the 10 m and 20 m bands are used, and the 20 m bands are resampled to 10 m GSD using the nearest-neighbor method.

2.1.2. PRISMA

The PRISMA hyperspectral satellite is operated by the Italian Space Agency (ASI) and was launched in 2019. PRISMA is a small-class satellite flying in a frozen Sun-synchronous low Earth orbit at an altitude of approximately 615 km. It operates on a circular orbit with a nominal revisit cycle of about 29 days, while a potential revisit time shorter than 7 days can be achieved through off-nadir acquisitions for on-demand observations of specific targets. PRISMA provides hyperspectral observations across 240 spectral bands covering the wavelength range from the visible and near-infrared (VNIR) to the shortwave infrared (SWIR), spanning 400 to 2500 nm, with a spectral sampling interval of less than 12 nm [30,31]. The satellite adopts a push-broom imaging mode with an across-track swath width of approximately 30 km, corresponding to about 1000 pixels and a field of view (FOV) of 2.77°. Each PRISMA acquisition includes one panchromatic (PAN) band with a GSD of 5 m and 239 hyperspectral bands with a GSD of 30 m, covering a standard scene size of approximately 30 km × 30 km. The PAN channel spans the visible spectral range from 400 to 700 nm and is projected onto the same swath as the hyperspectral data, covering approximately 6000 pixels across track. The PAN data are appended to the hyperspectral cubes and are primarily used to support spatial enhancement of the hyperspectral images. In this study, the PRISMA Level-2D product was employed, which provides orthorectified surface reflectance data and serves as the basis for subsequent hyperspectral–multispectral data fusion.

2.2. Construction of the Proposed LCZ-HSMS Dataset

LCZ datasets are essential for training DL models capable of capturing complex spectral–spatial patterns in urban environments. However, existing datasets rarely provide multisource information that includes both MSI and HSI for comprehensive LCZ classification. To address this limitation and to enable an in-depth evaluation of multisource fusion strategies for LCZ mapping, we developed a new dataset named LCZ-HSMS (Local Climate Zone dataset with Hyperspectral and Multispectral data).

As illustrated in Figure 2, the construction of the LCZ-HSMS dataset involves a series of preprocessing steps designed to ensure spatial alignment between HSI and MSI. For PRISMA data, bands with a low signal-to-noise ratio (SNR $< 100$) or affected by strong atmospheric absorption were excluded, resulting in 193 effective bands, including 61 bands in the VNIR region and 132 bands in the SWIR region. Subsequently, spatial enhancement was performed using the PanNet (PNN) pan-sharpening (PAN) algorithm [32], which fuses hyperspectral data with a high-resolution panchromatic band to generate hyperspectral imagery at 5 m spatial resolution. The pan-sharpened PRISMA data were then co-registered with temporally adjacent Sentinel-2 images using the GeFolki Python package (last commit: 19 January 2019) [33]. During this step, the processed PRISMA data were resampled to a spatial resolution of 10 m, achieving sub-pixel geometric alignment with Sentinel-2 images. In principle, directly fusing hyperspectral and multispectral data within the network architecture could reduce spectral distortion. However, such strategies would require the network to simultaneously address cross-resolution alignment and discriminative feature learning, which is particularly challenging for hyperspectral data due to its band-dependent noise characteristics and limited spatial detail [34,35]. In contrast, PNN-based pan-sharpening explicitly decouples spatial enhancement from semantic feature learning, thereby significantly reducing the complexity of the fusion task [36]. Moreover, the scene-level LCZ mapping relies on robust spectral–spatial statistics at the patch level rather than precise pixel-wise reconstruction [6]. Within this context, moderate spatial enhancement with preserved global spectral trends is sufficient and more suitable for stable and generalizable LCZ classification. Finally, to reduce computational complexity and alleviate the curse of dimensionality inherent in hyperspectral data, Principal Component Analysis (PCA) [37] was applied to the combined hyperspectral images of all cities, ensuring a consistent spectral subspace across study areas. This unified PCA transformation was adopted to mitigate inter-city spectral variability caused by differences in atmospheric conditions, illumination, and acquisition effects, which are common in HSI data. The first 15 principal components, which collectively explain more than 99% of the total spectral variance, were retained for subsequent LCZ classification.

Figure 2. Construction process of the LCZ-HSMS dataset.

The LCZ-HSMS dataset was primarily constructed using the geographic information provided by the So2Sat LCZ42 dataset [15], developed by the Technical University of Munich (TUM), with a reference period around 2018. PRISMA hyperspectral scenes used in this study were mainly acquired around 2020. Considering the minor temporal mismatch between the So2Sat LCZ42 reference data and the PRISMA acquisitions, a manual screening procedure was applied to remove patches corresponding to areas with evident land-cover or LCZ changes, in order to ensure label consistency. In addition, since PRISMA scenes mainly cover the central urban cores of European cities and the diversity of LCZ types in these regions is relatively limited, the dataset focuses on the dominant built-up and natural LCZ classes commonly observed in European metropolitan areas. Compared with the So2Sat LCZ42 dataset, several LCZ classes (i.e., LCZ 7, 9, 10, C, E, and F) were excluded due to their scarce or fragmented spatial representation within the PRISMA image footprints. In addition, natural LCZ classes (A, B, D, and G) were manually refined and complemented through visual inspection of Sentinel-2 true-color composites and auxiliary datasets, including CLCPlus [38] and Urban Atlas [39], to ensure accurate delineation of vegetated and water areas. Following label refinement, all MSI and HSI data were geometrically aligned and divided into non-overlapping patches of $32\times 32$ pixels, corresponding to an area of $320\times 320$ m at 10 m spatial resolution. This patch size was selected as a trade-off between capturing sufficient neighborhood-scale spatial context and maintaining computational efficiency, and is consistent with common practice in scene-level LCZ classification studies as well as the widely used So2Sat LCZ42 benchmark [8,13,17]. Based on this process, co-located Sentinel-2 and PRISMA patches were extracted for six representative European cities, namely Milan, Rome, Paris, Berlin, Lisbon, and Madrid (Figure 3). All processed data were stored in HDF5 format comprising three datasets: sen2, prisma, and label, to support efficient input/output operations during deep network training. The resulting LCZ-HSMS dataset therefore provides a well-aligned multisource benchmark that captures the dominant morphological and natural structures of European urban environments, enabling a rigorous evaluation of spectral–spatial fusion models for LCZ mapping. The class imbalance observed in Table 1 is partly attributed to the limited spatial coverage and acquisition constraints of PRISMA data, which prevent uniform sampling of all LCZ types across cities. In addition, LCZ distributions naturally vary due to differences in urban structure, climate, and land-use patterns.

Figure 3. Location of six experiment sites and Sentinel-2 RGB composite bands.

Table 1. Distribution of scene-level LCZ samples across the six study cities used in this study. Built-up zones correspond to urban LCZ classes (LCZ 1–8), while natural zones correspond to LCZ classes A, B, D, F, and G. LCZ classes with insufficient samples were excluded.

Type	LCZ Class	Berlin	Madrid	Milan	Paris	Rome	Lisbon	Total
Built-up zones	LCZ 1—Compact high-rise	0	15	0	56	0	0	71
	LCZ 2—Compact mid-rise	1533	669	762	2769	1718	227	7678
	LCZ 3—Compact low-rise	0	26	117	0	0	113	256
	LCZ 4—Open high-rise	188	147	0	293	0	0	628
	LCZ 5—Open mid-rise	1336	338	199	197	1316	90	3476
	LCZ 6—Open low-rise	508	430	97	454	308	23	1820
	LCZ 8—Large low-rise	555	492	570	288	427	71	2403
Natural zones	LCZ A—Dense trees	448	0	0	571	5	42	1066
	LCZ B—Scattered trees	35	7	222	10	104	19	397
	LCZ D—Low plants	62	26	179	0	115	1	383
	LCZ F—Bare soil or sand	0	489	205	0	0	0	694
	LCZ G—Water	199	4	152	0	0	370	725
Total		4864	2643	2503	4938	4293	956	20,197

Note: LCZ classes 7, 9, 10, C, and E were excluded due to insufficient sample availability.

3. Methodology

In scene-level LCZ mapping, each LCZ patch may contain multiple land cover types with different distributions and materials. It is important to fully explore the spatial structure and spectral features in the images and recognize the subtle differences between the different categories for LCZ mapping. Therefore, the LCZ-HMSSNet method proposed in this article focuses on the extraction and fusion of the spatial and spectral features of a MSI and HSI for LCZ mapping. The overall framework of the proposed LCZ-HMSSNet structure is shown in Figure 4. It consists of two main modules: the HSI–MSI Feature Fusion module (HMFF) and the spatial–spectral feature separation module (SSFS).

Figure 4. LCZ-HMSSNet Overall Architecture.

3.1. Dual-Branch HSI–MSI Feature Fusion

In LCZ scene-level classification, the use of HSI spectral richness and MSI spatial detail together is highly beneficial for capturing the heterogeneous characteristics of urban environments. To this end, the proposed HMFF structure processes the HSI and MSI inputs through two parallel feature extraction branches with locally enhanced self-attention (LESA) module and fuses their representations using a cross-attention-based interaction mechanism, as illustrated in Figure 4. This design allows the network to fully exploit the complementary spectral and spatial properties of the two modalities before high-level semantic feature learning.

3.1.1. Structure of the HMFF

Both the HSI and MSI inputs are processed through two parallel extraction paths that share the same architectural design.

Denote the input of modality 1 and 2 by $I_1\in {\mathbb{R}}^{H\times W\times C_1}$ and $I_2\in {\mathbb{R}}^{H\times W\times C_2}$, respectively. The feature extraction pipelines of both modalities take the form of a shared four-stage structure, as shown in Figure 4. Each input is first processed by a single 3D convolution layer with a kernel size of $3\times 3\times 3$, followed by Batch Normalization and ReLU activation, to jointly encode local spectral–spatial variations. The resulting intermediate tensor is then given by:

$$F_i^{3D}=\mathrm{Conv}3\mathrm{D}\left(I_i\right)$$

(1)

The spectral dimension of $F_1^{3D}$ and $F_2^{3D}$ is flattened into the channel dimension to form 2D feature maps, which are then processed by a single 2D convolution layer with a $3\times 3$ kernel to refine the spatial representations:

$$F_i^{2D}=\mathrm{Conv}2\mathrm{D}\left(\mathrm{Flatten}\left(F_i^{3D}\right)\right)$$

(2)

To incorporate both local and long-range spatial dependencies, a LESA module is subsequently applied. The resulting modality-specific feature representations are

$$F_i=\mathrm{LESA}\left(F_i^{2D}\right)$$

(3)

where $F_i\in {\mathbb{R}}^{H\times W\times 64}$.

Then, the outputs of the two paths are passed into a Cross-Guided Attention (CGA) module. The CGA block operates on the pair of modality-specific feature maps $F_i$ and generates mutually enhanced representations by applying bidirectional gating between the two branches. The output of this stage can be expressed as

$${\tilde{F}}_i=\mathrm{CGA}\left(F_i\right)$$

(4)

where ${\tilde{F}}_i\in {\mathbb{R}}^{H\times W\times 64}$ denotes the CGA-refined feature map of branch i.

3.1.2. Locally Enhanced Self-Attention Module

In our structure, a lightweight LESA module is introduced after the convolution blocks, as shown in Figure 5. The module refines convolutional features by incorporating limited spatial interactions while avoiding the computational overhead typically associated with full self-attention [40]. By focusing on local contextual information, LESA produces spatially coherent representations that provide more stable features for subsequent cross-modal fusion.

Figure 5. Locality-Enhanced Self-Attention (LESA) Module.

Given a feature sequence $X\in {\mathbb{R}}^{B\times N\times C}$ obtained from the preceding convolutional layers, where B denotes the batch size, N the number of tokens, and C the feature dimension, we first reshape the token sequence and apply a single $3\times 3$ depthwise convolution (DWConv) to inject locality priors:

$$X_{\mathrm{loc}}=\mathrm{DWConv}\left(\mathrm{Reshape}\left(X\right)\right)$$

(5)

where $\mathrm{Reshape}(·)$ converts the flattened token sequence $X\in {\mathbb{R}}^{B\times N\times C}$ back into a 2D spatial feature map of shape $B\times C\times H\times W$ ($N=H\times W$), thereby restoring the original spatial topology.

By restoring the spatial structure, the subsequent DWConv can explicitly model local neighborhood interactions. Since DWConv operates independently on each channel, it enhances local structures such as edges, textures, and small-scale spatial variations while preserving channel-wise independence. This design prevents the attention mechanism from relying solely on global interactions and helps stabilize the learning of spatial correlations, especially under limited training samples.

The enriched features are subsequently projected into query (Q), key (K), and value (V) representations, which correspond to the standard components of the self-attention mechanism:

$$Q=X_{\mathrm{loc}}{W}_q,K=X_{\mathrm{loc}}{W}_k,V=X_{\mathrm{loc}}{W}_v,$$

(6)

where $W_q$, $W_k$, and $W_v$ denote learnable linear mappings. This formulation keeps the overall complexity linear in the number of tokens while ensuring that the enhanced local structures are explicitly encoded into the attention space.

The multi-head attention (MHA) [41] is then computed as

$$\mathrm{Attn}(Q,K,V)=\mathrm{softmax}\left({\displaystyle \frac{Q{K}^{\top }}{\sqrt{d}}}\right)V$$

(7)

where $\mathrm{Attn}(·)$ denotes the scaled dot-product attention operation, and d denotes the dimensionality of the query and key vectors. This formulation enables each token to selectively aggregate information from spatially relevant locations. Thanks to the locality-aware preprocessing, the resulting attention maps favor spatially consistent neighborhoods rather than distributing weights uniformly across the entire feature map.

Finally, a linear projection is applied to form the output representation:

$$X_{\mathrm{out}}=\mathrm{Attn}(Q,K,V)W_o$$

(8)

where $W_o$ denotes a learnable output projection matrix. This projection adapts the aggregated responses back to the original feature dimension.

Unlike center-based transformer designs such as the center transformer in [42], which explicitly emphasize a predefined central pixel and its surrounding regions through hierarchical region sampling, LESA does not assume the existence of a representative center pixel. Instead, LESA is designed for scene-level LCZ classification and enhances spatial coherence across the entire patch by injecting locality priors via depthwise convolution before self-attention. This design is better suited for modeling heterogeneous urban scenes, where LCZ categories are characterized by patch-level spectral–spatial patterns rather than single-pixel semantics. Overall, LESA contributes a distinct refinement to the convolutional features. By combining lightweight local enhancement with MHA, it produces smoother and more discriminative spatial responses, thereby offering stable intermediate representations that benefit the subsequent cross-branch interaction and global fusion modules.

3.1.3. Cross-Guided Attention Module

After obtaining the modality-specific feature maps, a CGA module is employed. It is designed to facilitate information exchange between hyperspectral and multispectral representations, $F_1$ and $F_2$, where $F_1,F_2\in {\mathbb{R}}^{H\times W\times C}$.

The module selectively weights the features using a bilateral channel attention mechanism. This process begins with the generation of two separate attention weight maps, $W_{1\to 2}$ and $W_{2\to 1}$, through dedicated attention sub-networks $A(·)$. The sub-network is implemented using a $1\times 1$ convolutional bottleneck structure, followed by ReLU and Sigmoid activation $\sigma (·)$. The attention mapping operation is formally defined as:

$$A\left(F\right)=\sigma \left({\mathrm{Conv}}_{1\times 1}^{\left(2\right)}\left(\mathrm{ReLU}\left({\mathrm{Conv}}_{1\times 1}^{\left(1\right)}\left(F\right)\right)\right)\right)$$

(9)

where ${\mathrm{Conv}}_{1\times 1}^{\left(1\right)}$ maps the channel dimension from C to $C/4$ and ${\mathrm{Conv}}_{1\times 1}^{\left(2\right)}$ restores it to C. The cross-guided attention weights are then calculated by:

$$W_{1\to 2}=A_1\left(F_1\right)\mathrm{and}{W}_{2\to 1}=A_2\left(F_2\right)$$

(10)

These weight maps (where $W\in {\mathbb{R}}^{H\times W\times C}$) are used for the enhancement of residual features through element-wise multiplication (⊙), resulting in enhanced features $F_1^{\prime }$ and $F_2^{\prime }$:

$$\begin{array}{cc}\hfill F_1^{\prime }& =F_1\odot W_{2\to 1}+F_1\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill F_2^{\prime }& =F_2\odot W_{1\to 2}+F_2\hfill \end{array}$$

(12)

Subsequently, the enhanced features $F_1^{\prime }$ and $F_2^{\prime }$ are concatenated along the channel dimension (the last dimension):

$$F_{\mathrm{Cat}}=\mathrm{Cat}(F_1^{\prime },F_2^{\prime })$$

(13)

where $\mathrm{Cat}(·)$ denotes the concatenation operation along the channel dimension.

Finally, $F_{\mathrm{Cat}}$ is processed through a fusion block, with a $3\times 3$ Conv2D layer, Batch Normalization (BN), and a ReLU activation, to generate the final fused feature $F_{\mathrm{Fused}}$:

$$F_{\mathrm{Fused}}=\mathrm{ReLU}\left(\mathrm{BN}\left({\mathrm{Conv}}_{3\times 3}\left(F_{\mathrm{Cat}}\right)\right)\right)$$

(14)

Overall, CGA establishes a bidirectional guidance mechanism in which each modality adaptively enhances the other through cross-attention and feature gating. This enables the network to exploit complementary spectral–spatial information from both branches, providing a more discriminative joint representation for subsequent fusion and classification.

3.2. Spatial–Spectral Feature Separation Module

When performing scene-level LCZ classification, it is important to account for the inherent differences between spatial structures and spectral characteristics across urban classes. Although the fused HSI–MSI features provide a comprehensive representation by jointly encoding spatial and spectral information, directly exploiting such fused features may not fully highlight attribute-specific discriminative cues.

Inspired by recent studies in hyperspectral image classification [43], we introduce a spatial–spectral feature separation (SSFS) module with a dual-branch design to enhance the discriminability of the fused representations. Specifically, the SSFS module decomposes the fused features into spatial tokens and spectral tokens, allowing each branch to focus on complementary discriminative attributes. This separation facilitates more effective and fine-grained LCZ classification, as illustrated in Figure 6.

Figure 6. Spectral–Spatial Feature Separation Module.

Assume the feature tensor is denoted as $X\in {\mathbb{R}}^{b\times n\times c}$, where b is the batch size, n is the number of flattened pixels, and c represents the feature channels. To extract complementary structural information, the module generates two token sequences by reshaping X along different dimensions: a spatial-oriented sequence $X_s$ and a channel-oriented sequence $X_c$. For each token sequence, we introduce a learnable attention matrix and a value-projection matrix. For the spatial token path, denote these matrices as $W_A^{\left(s\right)}$ and $W_V^{\left(s\right)}$.

We first compute the token attention weight:

$$A_s=\mathrm{softmax}\left(X_s{W}_A^{\left(s\right)T}\right),$$

(15)

where $\mathrm{softmax}(·)$ denotes the softmax normalization function applied along the token dimension. The projected value representation:

$$F_s=X_s{W}_V^{\left(s\right)}.$$

(16)

Then, the spatial token output is obtained through

$$T_s=A_s{F}_s^T.$$

(17)

Similarly, the channel token path adopts its own learnable matrices $W_A^{\left(c\right)}$ and $W_V^{\left(c\right)}$, generating

$$A_c=\mathrm{softmax}\left(W_A^{\left(c\right)}{X}_c\right),F_c=W_V^{\left(c\right)}{X}_c,$$

(18)

and

$$T_c=A_c{F}_c.$$

(19)

The outputs from the two token paths are passed through lightweight linear projection layers and then merged using a shallow convolutional refinement block. This process generates the final discriminative token representation used for classification.

4. Results and Discussion

4.1. Experimental Setup

The experiments were conducted using PRISMA hyperspectral data and Sentinel-2 multispectral images. The PRISMA data were reduced using PCA, and the first 15 principal components (PCs), which collectively explain more than 99% of the total spectral variance, were retained. This threshold was selected to preserve the dominant spectral information relevant for LCZ discrimination while reducing spectral redundancy and noise in the remaining components. Sentinel-2 contributed ten spectral bands at a spatial resolution of 10 m, and all samples were extracted as $32\times 32$ patches. The same fixed PCA projection matrix was used consistently across all experiments, including the Leave-One-City-Out (LOCO) evaluation, to ensure reproducibility. The influence of PCA dimensionality on classification performance is not further investigated in this work.

Unless otherwise specified, these patches were randomly split into training, validation, and test sets with a ratio of 60%, 20%, and 20%, respectively. To account for randomness, all experiments were repeated five times with different random seeds, and the mean and standard deviation of the results are reported. While this patch-level split is a standard baseline, it does not strictly enforce spatial disjointness between sets. Therefore, a strictly spatially disjoint evaluation based on the LOCO protocol is conducted separately in Section 4.6 to assess cross-city generalization performance.

The model was implemented in PyTorch (version 2.5.1)and trained using the NAdam optimizer with a learning rate of $1\times {10}^{-3}$ and a batch size of 64. Training was performed for up to 100 epochs with early stopping (patience of 10). Batch Normalization and ReLU activations were employed throughout the convolutional layers. No data augmentation was applied in order to preserve the original spectral characteristics of the MSI and HSI observations. Regarding the class imbalance across LCZ categories (Table 1), the model was trained using a standard cross-entropy loss without explicit class-weighting or re-sampling strategies. This design choice ensures a consistent training protocol across all experiments while enabling a fair evaluation of LCZ-HMSSNet under a realistic data distribution. Model performance was evaluated using several standard metrics commonly adopted in LCZ classification, including Overall Accuracy (OA), Average Accuracy (AA), Cohen’s Kappa coefficient. In addition, to provide a more comprehensive class-wise evaluation and to assess potential bias toward majority classes, per-class Recall, Precision, and F1-score are reported.

4.2. Comparison of the Different Classification Models

Table 2 summarizes the quantitative comparison among the four competing models, revealing a clear performance hierarchy across all metrics. The proposed LCZ-HMSSNet achieved the highest OA, AA, and Kappa, reaching 0.988, 0.978, and 0.985, respectively. This performance significantly outperforms the baseline models (ResNet18 [44], UNet [45], and Sen2LCZ-Net [16]). Among the competing approaches, Sen2LCZ-Net (Sen2) is trained using Sentinel-2 multispectral data only, while all other models are trained with the joint Sentinel-2 and PRISMA inputs. As a result, Sen2LCZ-Net (Sen2) exhibits the lowest overall performance across the global metrics, which can be primarily attributed to the absence of hyperspectral information rather than limitations of the model architecture itself. ResNet18 shows the weakest performance, particularly for structurally complex built-up classes (e.g., LCZ 3–5), where large intra-class variability and limited representative samples make discrimination particularly challenging. Furthermore, the class-wise performance analysis based on per-class Recall, Precision, and F1-score highlights the strong capability of LCZ-HMSSNet in resolving challenging LCZ classes. Substantial performance improvements are observed for typically challenging classes, such as LCZ 3, LCZ 4, LCZ 5, and LCZ 6, which are characterized by high intra-class variability and pronounced inter-class similarity. For example, in terms of F1-score, LCZ 3 is improved from 0.888 (UNet) and 0.854 (Sen2LCZ-Net) to 0.937, while LCZ 4 increases from 0.895 (UNet) and 0.896 (Sen2LCZ-Net) to 0.964. Similar trends can be observed for LCZ 5 and LCZ 6, where the proposed method consistently achieves higher Recall and F1-scores than all baseline models, indicating a reduced rate of both missed detections and misclassifications. Beyond these structurally challenging classes, the robustness of LCZ-HMSSNet is further evidenced by its performance on minority LCZ classes under severe class imbalance. For instance, LCZ 1 (71 samples), LCZ 3 (256 samples), LCZ B (397 samples), and LCZ D (383 samples) represent minority classes in the constructed dataset. Despite the limited number of training samples, LCZ-HMSSNet achieves competitive Recall, Precision, and F1-scores for these minority categories, particularly for LCZ 1 and LCZ 3, demonstrating its ability to learn discriminative spectral–spatial representations without relying solely on sample abundance. Moreover, the proposed LCZ-HMSSNet shows robust and stable performance for several compact and homogeneous classes, particularly LCZ 2, LCZ 8, and natural land-cover categories such as LCZ A, LCZ F, and LCZ G, where it consistently matches or surpasses all baseline methods in terms of Recall, Precision, and F1-score. In summary, LCZ-HMSSNet achieves the most balanced performance across all LCZ classes and results in substantial improvements in both global and per-class metrics. This comprehensive performance confirms that LCZ-HMSSNet effectively helps reduce the common issues of inter-class confusion and intra-class variability often encountered in complex urban environments, thereby ensuring the generation of highly accurate and reliable LCZ maps.

Table 2. Classification Performance Comparison of Different Model Performance.

	Metric/Class	ResNet18	UNet	Sen2LCZ-Net (Sen2)	Sen2LCZ-Net	Proposed
Overall Metrics	OA	0.945 ± 0.013	0.966 ± 0.003	0.931 ± 0.028	0.973 ± 0.007	0.988 ± 0.003
	AA	0.932 ± 0.023	0.951 ± 0.013	0.829 ± 0.075	0.937 ± 0.016	0.978 ± 0.010
	Kappa	0.930 ± 0.016	0.957 ± 0.003	0.911 ± 0.035	0.965 ± 0.009	0.985 ± 0.003
Per-Class Recall	LCZ 1	0.814 ± 0.130	0.843 ± 0.078	0.321 ± 0.410	0.732 ± 0.147	0.929 ± 0.124
	LCZ 2	0.967 ± 0.009	0.978 ± 0.004	0.969 ± 0.009	0.985 ± 0.005	0.993 ± 0.002
	LCZ 3	0.835 ± 0.145	0.883 ± 0.070	0.467 ± 0.353	0.815 ± 0.118	0.920 ± 0.025
	LCZ 4	0.814 ± 0.049	0.881 ± 0.031	0.758 ± 0.079	0.877 ± 0.050	0.952 ± 0.021
	LCZ 5	0.860 ± 0.050	0.931 ± 0.012	0.895 ± 0.049	0.954 ± 0.029	0.976 ± 0.011
	LCZ 6	0.957 ± 0.015	0.966 ± 0.017	0.905 ± 0.072	0.973 ± 0.010	0.986 ± 0.004
	LCZ 8	0.972 ± 0.006	0.983 ± 0.006	0.954 ± 0.030	0.990 ± 0.007	0.996 ± 0.001
	LCZ A	0.997 ± 0.004	0.995 ± 0.010	0.981 ± 0.016	0.990 ± 0.012	0.997 ± 0.003
	LCZ B	0.978 ± 0.016	0.969 ± 0.021	0.792 ± 0.083	0.944 ± 0.063	0.995 ± 0.008
	LCZ D	0.994 ± 0.006	0.990 ± 0.013	0.904 ± 0.089	0.987 ± 0.005	0.997 ± 0.006
	LCZ F	0.998 ± 0.005	0.998 ± 0.005	1.000 ± 0.000	0.997 ± 0.005	1.000 ± 0.000
	LCZ G	0.999 ± 0.003	0.998 ± 0.005	0.996 ± 0.006	0.997 ± 0.003	1.000 ± 0.000
Per-Class Precision	LCZ 1	0.874 ± 0.174	0.969 ± 0.043	0.464 ± 0.539	0.930 ± 0.090	0.978 ± 0.038
	LCZ 2	0.949 ± 0.011	0.974 ± 0.006	0.953 ± 0.024	0.981 ± 0.011	0.991 ± 0.004
	LCZ 3	0.911 ± 0.081	0.901 ± 0.074	0.647 ± 0.443	0.907 ± 0.044	0.955 ± 0.021
	LCZ 4	0.877 ± 0.045	0.911 ± 0.048	0.829 ± 0.115	0.919 ± 0.048	0.976 ± 0.020
	LCZ 5	0.916 ± 0.018	0.945 ± 0.004	0.874 ± 0.061	0.960 ± 0.006	0.980 ± 0.003
	LCZ 6	0.915 ± 0.036	0.957 ± 0.005	0.905 ± 0.025	0.965 ± 0.013	0.984 ± 0.006
	LCZ 8	0.962 ± 0.007	0.972 ± 0.010	0.968 ± 0.013	0.975 ± 0.014	0.990 ± 0.007
	LCZ A	0.996 ± 0.006	0.988 ± 0.008	0.952 ± 0.018	0.996 ± 0.009	1.000 ± 0.000
	LCZ B	0.927 ± 0.018	0.970 ± 0.032	0.856 ± 0.119	0.956 ± 0.059	0.982 ± 0.008
	LCZ D	0.982 ± 0.013	0.988 ± 0.014	0.992 ± 0.016	0.970 ± 0.032	0.997 ± 0.006
	LCZ F	0.994 ± 0.006	0.992 ± 0.013	0.945 ± 0.027	0.992 ± 0.010	0.997 ± 0.006
	LCZ G	1.000 ± 0.000	1.000 ± 0.000	0.999 ± 0.003	0.999 ± 0.003	1.000 ± 0.000
Per-Class F1-score	LCZ 1	0.830 ± 0.110	0.900 ± 0.046	0.364 ± 0.434	0.807 ± 0.060	0.949 ± 0.062
	LCZ 2	0.958 ± 0.008	0.976 ± 0.002	0.961 ± 0.016	0.983 ± 0.006	0.992 ± 0.003
	LCZ 3	0.862 ± 0.089	0.888 ± 0.042	0.535 ± 0.380	0.854 ± 0.063	0.937 ± 0.017
	LCZ 4	0.844 ± 0.046	0.895 ± 0.023	0.789 ± 0.080	0.896 ± 0.029	0.964 ± 0.011
	LCZ 5	0.887 ± 0.032	0.938 ± 0.005	0.884 ± 0.055	0.956 ± 0.017	0.978 ± 0.007
	LCZ 6	0.935 ± 0.016	0.961 ± 0.007	0.904 ± 0.036	0.969 ± 0.007	0.985 ± 0.003
	LCZ 8	0.967 ± 0.006	0.977 ± 0.008	0.961 ± 0.014	0.982 ± 0.010	0.993 ± 0.003
	LCZ A	0.997 ± 0.003	0.991 ± 0.008	0.966 ± 0.016	0.993 ± 0.006	0.998 ± 0.001
	LCZ B	0.951 ± 0.010	0.970 ± 0.023	0.813 ± 0.025	0.948 ± 0.030	0.989 ± 0.004
	LCZ D	0.988 ± 0.009	0.989 ± 0.012	0.944 ± 0.049	0.978 ± 0.016	0.997 ± 0.006
	LCZ F	0.996 ± 0.002	0.995 ± 0.007	0.972 ± 0.014	0.995 ± 0.004	0.998 ± 0.003
	LCZ G	0.999 ± 0.001	0.999 ± 0.003	0.997 ± 0.002	0.998 ± 0.001	1.000 ± 0.000

Figure 7 provides a visual comparison of LCZ maps for three representative cities: Milan, Berlin, and Madrid. Each row displays the true-color Sentinel-2 image, the Global LCZ Map product [46] (reference year: 2018), the results from the baseline Sen2LCZ-Net (trained with Sentinel-2 and PRISMA), and the predictions generated by our proposed LCZ-HMSSNet. It should be noted that the Global LCZ Map is derived from pixel-based classification, for which previous studies report overall accuracies typically above 70% and class-wise F1 scores ranging from approximately 50% to 80%, depending on the city. In contrast, the proposed approach adopts a scene-level LCZ classification strategy based on a fixed multi-city training set. Given this difference in classification granularity, a direct quantitative comparison is not meaningful, and the analysis is therefore limited to a qualitative visual comparison. The spatial patterns produced by the proposed method are largely consistent with the Global LCZ Map regarding general urban morphology and dominant LCZ zones. However, noticeable discrepancies arise in certain natural classes, particularly in the Madrid area. As shown in Figure 7l, the distribution of natural classes (e.g., LCZ D and F) significantly differs from that in the Global LCZ product (Figure 7j). These discrepancies may result from temporal inconsistencies between the datasets, including seasonal variations in the remote sensing images used to generate the maps, as well as differences in the reference years of the Global LCZ Map (2018) and the PRISMA/Sentinel-2 data used in this study (around 2020). Vegetation cover and spectral responses are known to vary across different seasons and years, which can influence the visual comparison results.

Figure 7. Visual comparison of LCZ classification results for three cities: Milan (top row), Berlin (middle row), and Madrid (bottom row). (a,e,i) True-color Sentinel-2 image; (b,f,j) Global LCZ Map (reference year: 2018); (c,g,k) classification results from Sen2LCZ-Net; (d,h,l) classification results from the proposed LCZ-HMSSNet. The light blue rectangle in (i–l) marks a construction zone in the southeast of Madrid, where transitional built-up areas tend to be misclassified.

A major limitation evident in the Global LCZ Map is the presence of excessive salt-and-pepper noise and widespread fragmentation. This manifests as numerous discontinuous patches of LCZ classes. This fragmented distribution poorly reflects the true spatial coherence and structural integrity of the real-world urban-rural continuum, consequently undermining the spatial interpretability of the map and complicating downstream urban analysis. In contrast, both the Sen2LCZ-Net and LCZ-HMSSNet produce more coherent and smoother LCZ delineations. This improvement is attributed to the inherent advantages of scene-level contextual modeling, which effectively mitigates intra-class variability in LCZ mapping by leveraging broader spatial context to enforce coherence within semantically consistent urban segments. Moreover, LCZ-HMSSNet demonstrates a superior capacity to accurately capture the structural complexity and spatial extent of the built-up LCZ categories (LCZ 1–8) compared to Sen2LCZ-Net. This improvement indicates that the combination of HSI and MSI observations, along with the spectral–spatial separation mechanisms in the proposed architecture, significantly enhances the model’s ability to distinguish subtle and crucial differences among built-up classes. However, the proposed method also exhibits limitations. A notable example can be observed in the Madrid scene (Figure 7l, lower-right corner). In this area, developing construction zones with initial building structures or exposed ground surfaces are often misclassified, as the model tends to label these regions as structurally similar built-up classes, particularly LCZ 6 (Open low-rise). This misclassification may be due to the spectral and spatial similarities between early-stage construction sites and established low-rise residential areas. Given that the model relies primarily on observed spectral and spatial features, distinguishing temporary from permanent structures remains challenging, especially when contextual information is limited.

4.3. Data Contribution Analysis

To assess the individual and joint contributions of MSI and HSI observations, we compare three input settings: Sentinel-2 only, PRISMA only, and the fusion of Sentinel-2 and PRISMA. Table 3 quantitatively compares the LCZ classification performance obtained from Sentinel-2, PRISMA, and their fusion (Sentinel-2+PRISMA). The overall metrics show a clear and consistent trend across the three data sources. PRISMA achieves higher OA, Kappa, and AA than Sentinel-2, demonstrating that hyperspectral observations provide stronger spectral discriminability for LCZ mapping. Specifically, OA improves from 0.959 to 0.979, AA from 0.923 to 0.974, and Kappa from 0.948 to 0.973. The fusion of Sentinel-2 and PRISMA further improves all metrics, reaching an OA of 0.988, an AA of 0.978, and a Kappa of 0.985, which confirms that the two modalities provide complementary information.

Table 3. Classification Performance Comparison of Three Data Sources.

	Metric/Class	Sentinel-2	PRISMA	Sentinel-2 + PRISMA
Overall Metrics	OA	0.959 ± 0.003	0.979 ± 0.003	0.988 ± 0.003
	AA	0.923 ± 0.016	0.974 ± 0.008	0.978 ± 0.010
	Kappa	0.948 ± 0.004	0.973 ± 0.003	0.985 ± 0.003
Per-Class Recall	LCZ 1	0.714 ± 0.378	0.952 ± 0.082	0.929 ± 0.124
	LCZ 2	0.979 ± 0.003	0.988 ± 0.005	0.993 ± 0.002
	LCZ 3	0.761 ± 0.207	0.913 ± 0.065	0.920 ± 0.025
	LCZ 4	0.847 ± 0.092	0.945 ± 0.027	0.952 ± 0.021
	LCZ 5	0.917 ± 0.018	0.950 ± 0.009	0.976 ± 0.011
	LCZ 6	0.968 ± 0.013	0.975 ± 0.007	0.986 ± 0.004
	LCZ 8	0.970 ± 0.015	0.989 ± 0.007	0.996 ± 0.001
	LCZ A	0.983 ± 0.016	0.998 ± 0.003	0.997 ± 0.003
	LCZ B	0.963 ± 0.016	0.991 ± 0.016	0.995 ± 0.008
	LCZ D	0.979 ± 0.018	0.997 ± 0.006	0.997 ± 0.006
	LCZ F	1.000 ± 0.000	1.000 ± 0.000	1.000 ± 0.000
	LCZ G	0.996 ± 0.007	0.994 ± 0.006	1.000 ± 0.000
Per-class Precision	LCZ 1	0.865 ± 0.132	0.958 ± 0.072	0.978 ± 0.038
	LCZ 2	0.967 ± 0.006	0.981 ± 0.002	0.991 ± 0.004
	LCZ 3	0.924 ± 0.093	0.976 ± 0.024	0.955 ± 0.021
	LCZ 4	0.888 ± 0.112	0.930 ± 0.009	0.976 ± 0.020
	LCZ 5	0.933 ± 0.026	0.971 ± 0.011	0.980 ± 0.003
	LCZ 6	0.957 ± 0.022	0.970 ± 0.018	0.984 ± 0.006
	LCZ 8	0.976 ± 0.007	0.981 ± 0.009	0.990 ± 0.007
	LCZ A	0.992 ± 0.007	0.999 ± 0.003	1.000 ± 0.000
	LCZ B	0.919 ± 0.060	0.986 ± 0.014	0.982 ± 0.008
	LCZ D	0.979 ± 0.010	1.000 ± 0.000	0.997 ± 0.006
	LCZ F	0.983 ± 0.015	1.000 ± 0.000	0.997 ± 0.006
	LCZ G	1.000 ± 0.000	1.000 ± 0.000	1.000 ± 0.000
Per-class F1-score	LCZ 1	0.717 ± 0.236	0.952 ± 0.042	0.949 ± 0.062
	LCZ 2	0.973 ± 0.002	0.984 ± 0.002	0.992 ± 0.003
	LCZ 3	0.816 ± 0.116	0.943 ± 0.046	0.937 ± 0.017
	LCZ 4	0.859 ± 0.012	0.937 ± 0.018	0.964 ± 0.011
	LCZ 5	0.925 ± 0.008	0.960 ± 0.008	0.978 ± 0.007
	LCZ 6	0.962 ± 0.005	0.972 ± 0.011	0.985 ± 0.003
	LCZ 8	0.973 ± 0.005	0.985 ± 0.005	0.993 ± 0.003
	LCZ A	0.988 ± 0.009	0.998 ± 0.001	0.998 ± 0.001
	LCZ B	0.940 ± 0.032	0.988 ± 0.014	0.989 ± 0.004
	LCZ D	0.979 ± 0.014	0.998 ± 0.003	0.997 ± 0.006
	LCZ F	0.992 ± 0.008	1.000 ± 0.000	0.998 ± 0.003
	LCZ G	0.998 ± 0.003	0.997 ± 0.003	1.000 ± 0.000

The class-wise results based on per-class Recall, Precision, and F1-score provide additional insights into these improvements. Several built-up classes exhibit large gains when PRISMA is used instead of Sentinel-2. For instance, for LCZ 1 and LCZ 3, which represent minority classes with limited training samples, the result based on PRISMA achieves markedly higher Recall and F1-scores than Sentinel-2, indicating a reduced number of missed detections. Specifically, the Recall of LCZ 1 increases from 0.714 to 0.952, and LCZ 3 improves from 0.761 to 0.920, highlighting the advantage of hyperspectral measurements in capturing subtle material differences among urban surfaces. Similar improvements can also be observed for minority natural classes such as LCZ B and LCZ D, where PRISMA consistently achieves higher Recall and F1-scores than Sentinel-2. These results suggest that hyperspectral data are particularly beneficial for under-represented classes, as the rich and continuous spectral information helps compensate for limited sample availability. Natural land-cover classes with distinctive and stable spectral signatures, such as LCZ A, LCZ F, and LCZ G, achieve consistently high accuracy for both Sentinel-2 and PRISMA inputs. The fusion results further enhance the overall robustness, providing more stable and balanced Recall, Precision, and F1-scores across both built-up and natural LCZ types. Overall, compared with single-source data, the fused input provides more stable predictions across different LCZ classes by exploiting the spectral richness of PRISMA and the spatial structure captured by Sentinel-2.

These quantitative findings are consistent with the t-SNE visualizations of the learned feature spaces, as shown in Figure 8. The PRISMA features form compact and well-separated clusters, particularly for spectrally well-defined LCZ types such as LCZ 1, LCZ 3, and the natural classes. These tight and isolated clusters correspond directly to the substantial accuracy improvements observed in Table 3 when replacing Sentinel-2 with PRISMA. In contrast, Sentinel-2 features exhibit more scattered and overlapping distributions, especially among dense built, which explains their lower per-class accuracy. This strong alignment between feature-space separability and numerical performance confirms that hyperspectral information enhances material discrimination, while Sentinel-2 contributes complementary spatial structure, and that their combination yields the most discriminative LCZ feature representation. Overall, these results demonstrate that hyperspectral data substantially improves LCZ discrimination compared with multispectral data alone, and that integrating both sources provides a more comprehensive and robust representation for urban scene classification.

Figure 8. Visualization of Different Data Sources features by t-SNE. (a) PRISMA (b) Sentinel-2.

4.4. Ablation Study of the Structure

In this network, each module contributes to the overall performance through its specific functionality. To assess the influence of these components, we conduct ablation experiments. The dual-branch structure only with convolutional layers serves as the baseline model. The detailed results are presented in Table 4.

Table 4. Ablation study for each module of LCZ-HMSSNet.

DBHM	LESA	CGA	SSF	OA	AA	Kappa
✓				0.971	0.965	0.964
✓			✓	0.981	0.970	0.978
✓	✓		✓	0.986	0.984	0.983
✓	✓	✓	✓	0.988	0.978	0.985

The DBHM provides the baseline accuracy by leveraging its dual-branch architecture for HSI and MSI processing, 3D convolutional spectral extraction, and 2D spatial encoding. This configuration achieves an OA of 0.971, an AA of 0.965, and a Kappa of 0.964, demonstrating that the designed multisource dual-branch structure already provides a solid baseline for LCZ recognition.

Incorporating the SSF module leads to a clear improvement across all metrics, with the OA, AA, and Kappa increasing to 0.981, 0.970, and 0.978, respectively. This improvement indicates that decoupling the fused HSI–MSI features into distinct spectral and spatial components helps reduce feature redundancy and allows the classifier to better exploit complementary information. The resulting representations exhibit clearer inter-class separation and more stable behavior across heterogeneous LCZ types, thereby producing more balanced and reliable results. Introducing the LESA module further enhances the accuracy, reaching an OA of 0.986, an AA of 0.984, and a Kappa of 0.983. Compared with the purely convolutional dual-branch baseline, LESA improves the capture of fine-grained spatial structures by incorporating local convolutional cues into the self-attention mechanism. This design appears to enhance both local detail representation and long-range contextual modeling, which is beneficial for distinguishing LCZ categories with subtle morphological variations. Finally, integrating the CGA module into the network achieves the best overall performance, with an OA of 0.988, an AA of 0.978, and a Kappa of 0.985. This result suggests the benefit of bidirectional cross-modal interaction, which allows each branch to refine its representations using complementary information from the other modality. By exchanging complementary spatial and spectral information, CGA produces more discriminative and modality-consistent feature representations, resulting in the highest performance among the ablation settings.

4.5. LCZ Map for Six Cities

Figure 9 presents the LCZ maps generated for the six study cities. The proposed method produces spatially coherent LCZ distributions, with clear delineation of compact and open built-up forms, vegetated zones, bare surfaces, and water bodies. The resulting spatial patterns are consistent with the characteristic urban morphology of each city.

Figure 9. LCZ maps generated by the proposed method for six representative European cities. (a) Berlin, (b) Lisbon, (c) Madrid, (d) Paris, (e) Milan, and (f) Rome.

In Berlin (a), LCZ 2 dominates the central districts and gradually transitions to LCZ 5 and LCZ 6 toward the periphery, while the Spree–Havel river system and surrounding forests are mapped as LCZ G and LCZ A/B. Lisbon (b) shows dense LCZ 2/3 structures in the historical core, LCZ 6 across the suburban hills, and well-defined representations of the Tagus River and major green corridors. Madrid (c) exhibits a compact LCZ 2 core that gradually expands into LCZ 5/6 neighborhoods, with extensive LCZ D/F surfaces characterizing the semi-arid surroundings; minor confusion occurs in developing construction areas where mixed structural signatures are present. Paris (d) displays its well-known concentric configuration, with LCZ 2 in the core and LCZ 5/6/8 across the metropolitan ring, while the Seine River and major urban parks form continuous LCZ G and LCZ A/B patches. Milan (e) shows a monocentric pattern, with LCZ 2 in the historical center gradually transitioning to LCZ 5 and LCZ 6, and green corridors and canal systems preserved as LCZ A/B and LCZ G. Rome (f) presents compact LCZ 2/3 structures in the historic core and more dispersed LCZ 5/6 forms in the outer districts, while vegetated regions and the Tiber River consistently captured as LCZ A/B and LCZ G.

Across all six cities, the predicted LCZ patterns show good agreement with known urban morphology and exhibit a high degree of spatial coherence. The proposed framework is trained using data from multiple cities and is designed to capture general spectral–spatial characteristics of LCZ classes. In principle, the trained model may be applied to a new city by following the same preprocessing and inference pipeline. However, since the PCA transformation for hyperspectral data is derived from the training cities, its transferability to cities with substantially different spectral characteristics has not been explicitly validated in this study. A systematic investigation of cross-city generalization is therefore left for future work.

4.6. Leave-One-City-Out Cross-City Evaluation

To assess the generalization capability of the proposed LCZ classification models, a leave-one-city-out (LOCO) cross-city evaluation was conducted. In this setting, the model is trained using data from all but one city and then tested on the held-out city, ensuring that the training and test samples are strictly disjoint at the city level. This evaluation provides a more challenging and realistic assessment of cross-city generalization performance compared with random patch-level splitting.

Regarding data preprocessing, PCA was applied at the preprocessing stage using the complete HSI from all cities, and the same fixed PCA projection matrix was used consistently across all LOCO folds. This strategy ensures a consistent spectral subspace across different cities and experimental settings, which is particularly important given the spectral variability caused by atmospheric conditions and acquisition differences in hyperspectral remote sensing images.

The results, summarized in Table 5, indicate that both the Sen2LCZ-Net and the proposed LCZ-HMSSNet maintain relatively stable performance across different test sites when using Sentinel-2 multispectral (MSI) data. However, the integration of PRISMA hyperspectral (HSI) data leads to a noticeable decline in cross-city generalization. While HSI provides rich spectral information that benefits intra-city classification, it appears to introduce significant spectral domain shifts in cross-city scenarios [47].

Table 5. LOCO cross-city classification performance (OA and Kappa) for representative test cities.

City	Sen2LCZ-Net				LCZ-HMSSNet
	Sentinel-2		S2 + PRISMA		Sentinel-2		S2 + PRISMA
	OA	Kappa	OA	Kappa	OA	Kappa	OA	Kappa
Berlin	0.675	0.590	0.485	0.374	0.653	0.571	0.505	0.351
Milan	0.720	0.658	0.530	0.400	0.687	0.605	0.542	0.460
Lisbon	0.800	0.741	0.538	0.422	0.754	0.685	0.569	0.456

The observed performance drop suggests that the narrow hyperspectral bands are highly sensitive to city-dependent factors, such as local atmospheric conditions, seasonal vegetation differences, and regional building material variations [48]. Unlike the broader bands of Sentinel-2 which provide more “stable” and transferable representations, the high-dimensional HSI features may emphasize locally dominant characteristics that do not generalize well to unseen urban environments. Consequently, while the LCZ-HMSSNet shows improvements over the baseline in most fusion cases, the results highlight that mitigating hyperspectral-induced domain shifts remains a critical challenge for large-scale LCZ mapping. Future work should explore domain adaptation or spectral alignment techniques to enhance the robustness of HSI-MSI fusion models.

4.7. Performance Under Limited Samples

Given that annotating LCZ samples is labor-intensive and real-world datasets are often limited in size, we evaluate the data efficiency of the proposed LCZ-HMSSNet. To this end, we conduct experiments in which the number of available training samples is progressively increased, while the validation and test sets remain fixed. This setup reflects practical LCZ mapping scenarios, where labeled data are costly to obtain and unevenly distributed across classes and cities. All reported accuracies are obtained by averaging the results over ten independent training runs to ensure statistical reliability.

Figure 10 compares the performance of LCZ-HMSSNet with UNet, ResNet18, and Sen2LCZ-Net across different training sample sizes. In each experiment, we randomly select 5 to 40 samples per class, with an increment of 5, to form the training set. Although all models benefit from the use of additional samples and exhibit gradual improvements in OA and Kappa, their sensitivities to limited supervision differ substantially.

Figure 10. Performance Comparison Under Different Training Sample Sizes.

When the training set contains only 5 samples per class, all methods show comparable performance, indicating that the extremely limited supervision does not allow any model to develop a clear advantage. LCZ-HMSSNet consistently achieves the highest accuracy starting from the 10-sample setting and exhibits the most stable growth as more samples become available. Even with only 10 samples per class, it already surpasses the baselines by a clear margin, and the performance gap becomes more pronounced as the training set expands. This robustness arises from the network’s ability to exploit complementary hyperspectral–multispectral information along with its locality-enhanced attention mechanisms, which strengthen spectral–spatial feature abstraction under weak supervision. In comparison, UNet and Sen2LCZ-Net improve steadily but remain less competitive in scarce-data conditions, while ResNet18 displays early performance saturation, indicating limited generalization capacity.

Overall, these findings demonstrate that LCZ-HMSSNet maintains strong performance under limited-sample conditions and scales effectively as more data become available, underscoring its suitability for LCZ mapping tasks where labeled samples are inherently scarce.

5. Conclusions

This paper investigated the potential of HSI and MSI data fusion for scene-level LCZ classification. We introduced LCZ-HMSSNet, a deep learning framework that employs two complementary dual-branch structures, including a parallel HSI–MSI feature extraction architecture and a spatial–spectral feature separation module. By moving beyond conventional pixel-based strategies, the proposed approach effectively addresses the inherent heterogeneity of LCZ scenes and provides a more semantically meaningful representation of urban structures. Experimental results across multiple European cities demonstrate that integrating HSI and MSI information consistently results in notable performance improvements over MSI-only methods and existing LCZ classification approaches. The resulting LCZ maps exhibit enhanced spatial coherence and more reliable discrimination of urban structure types, particularly in morphologically complex built-up areas. These results indicate that hyperspectral information provides valuable complementary information for LCZ mapping when appropriately combined with multispectral spatial context. Moreover, the proposed method shows strong robustness under limited training-sample conditions, indicating its practical applicability in real-world scenarios where annotated LCZ samples are scarce and unevenly distributed. This property is especially relevant for large-scale applications and for newly investigated cities lacking comprehensive reference data. Overall, this study demonstrates the practical relevance of HSI–MSI fusion for urban applications and confirms the feasibility of scene-level LCZ classification using PRISMA and Sentinel-2 data.

Despite these promising results, the proposed framework is subject to certain limitations, including its reliance on the availability of accurately co-registered hyperspectral and multispectral data and the increased computational cost associated with high-dimensional spectral inputs and attention-based modules. Future research will explore extending the proposed framework to multi-temporal observations and larger geographic regions, while also investigating lightweight and adaptive learning strategies. In addition, we will study the integration of weakly supervised and self-supervised learning to further improve the scalability and generalization capability of the framework.