SPFMamba: A Mamba-Based Network with Semantic Prompt and Frequency-Adaptive Fusion for Remote Sensing Image Semantic Segmentation

Highlights

What are the main findings?

• SPFMamba builds upon Mamba and combines semantic prompt global–local modeling with frequency-adaptive feature fusion to segment high-resolution remote sensing imagery.
• The proposed SPGLM block enhances semantic consistency and spatial continuity through prompt-guided global scanning and multidirectional local perception, while the HFAFM strengthens high-frequency details to improve boundary delineation and small-scale object representation.

What are the implications of the main findings?

• Experiments on three public datasets show that SPFMamba achieves strong segmentation performance with only 17.85 M parameters, demonstrating its effectiveness for complex land-cover segmentation.
• This study offers an efficient Mamba-based global–local modeling and semantic-guided detail fusion paradigm that can be extended to fine-grained remote sensing image interpretation tasks.

Abstract

Modern remote sensing images (RSIs) provide increasingly fine spatial detail, making pronounced scale variations and complex spatial distributions of land-cover classes more apparent and thereby increasing the difficulty of semantic segmentation. Recent remote sensing semantic segmentation methods therefore have increasingly adopted Transformer- and Mamba-based models to improve global contextual modeling. However, these models often suffer from intraclass inconsistency and interclass feature confusion, leading to fragmented object structures and imprecise boundary delineation. Accordingly, we develop SPFMamba, an architecture built around Mamba that combines semantic prompting with frequency-adaptive fusion, thereby improving global context modeling and fine-detail representation. During feature reconstruction, we propose a semantic prompt global–local Mamba (SPGLM) block to jointly model global semantic information and local spatial cues. Its parallel semantic prompt global and multidirectional local perception branches promote semantically coherent and spatially continuous feature distributions, thereby effectively preserving the structural integrity of ground objects. To further alleviate cross-level semantic discrepancies and strengthen the representation of small-scale targets, we design a high-frequency adaptive fusion module (HFAFM). It first refines high-frequency responses in shallow layers to retain small-scale object structures and boundary cues. Subsequently, deep semantic priors guide local cross-attention, allowing low-level spatial cues to be selectively integrated with deeper semantic representations. Evaluations across ISPRS Vaihingen, ISPRS Potsdam, and OpenEarthMap datasets show that SPFMamba delivers favorable segmentation accuracy with only 17.85 M parameters, while showing improved preservation of object structures and fine spatial details in qualitative comparisons.

1. Introduction

Recent progress in satellite and unmanned aerial vehicle imaging has made high-resolution remote sensing imagery (HRSIs) increasingly accessible, thereby facilitating Earth observation and scene analysis. Semantic segmentation constitutes a core task in RSI interpretation [1,2], focusing on fine-grained pixel-level classification [3]. This technique has been widely used in various practical scenarios, including military scene analysis [4], environmental protection [5], natural disaster monitoring [6], and land-use analysis [7]. Earlier machine-learning approaches [8], represented by random forests [9,10], support vector machines [11], and conditional random fields [12], generally exhibit limited effectiveness in semantic segmentation of HRSIs.

Deep neural networks have substantially advanced HRSI semantic segmentation by automatically learning multilevel and discriminative visual representations from input imagery [13,14]. Currently, deep learning methods based on convolutional neural networks (CNNs) and Transformers are the most widely used approaches [15,16]. Fully convolutional networks (FCNs) [17] pioneered the application of CNNs to image segmentation, achieving remarkable performance. However, the loss of spatial details during downsampling often results in coarse segmentation results. Subsequently, U-Net [18] employed a symmetric encoder–decoder structure, in which skip connections compensate for missing spatial details and improve segmentation accuracy. Nevertheless, traditional skip connections are still limited in bridging semantic gaps and scale discrepancies across multi-level features. In response, various feature fusion strategies have been proposed to improve representation capability. For instance, the attention feature fusion (AFF) module [19] introduces a unified attention mechanism to effectively mitigate semantic and scale inconsistencies during cross-level feature aggregation. Gradient decoupling guided network (GDGNet) [20] designs the global–local context fusion (GLCF) module comprising two parallel branches, through which low- and high-level features are adaptively combined to enhance feature representation.

Unlike natural images, RSIs contain large-scale objects with significant intraclass texture variations. Relying solely on local contextual information for pixel-wise classification leads to fragmented object structures. Different land-cover classes may exhibit high appearance similarity in local regions, while fine details of small-scale objects and boundary regions are often degraded, as shown in Figure 1, further increasing the difficulty of fine-grained segmentation. To address the difficulty of precisely delineating boundaries between different classes, Adaptive Patch Reshaping (APR) [21] jointly partitions the input image and its corresponding label patches and reassigns the resulting subregions to appropriate classes according to an information-theoretic criterion. From the perspective of model-level feature learning, effectively extracting global and local contextual information remains crucial for accurate pixel-level class representation. Transformer-based methods [22,23] have been widely adopted, leveraging self-attention operations to effectively capture global interactions [24] and significantly enhance segmentation performance. However, excessive reliance on global interactions may induce interference among similar features across different objects, thereby reducing feature discriminability and causing feature confusion [25], resulting in incomplete object structures and blurred boundaries. Furthermore, Transformer-based methods often incur high computational costs. Existing CNN- and Transformer-based approaches still face challenges in complex RSI semantic segmentation tasks.

By building on state space models (SSMs), Mamba provides an efficient mechanism for capturing long-range dependencies while keeping the computational cost proportional to the sequence length. This efficiency makes it well suited to semantic segmentation of RSIs. Vision Mamba [26] pioneered the application of bidirectional Mamba blocks in vision tasks, while VMamba [27] introduced the visual state-space (VSS) blocks and 2-D selective scanning (SS2D) modules to facilitate contextual information interaction and modeling through multidirectional scanning paths. Recent studies have extended Mamba-based architectures to a broad range of remote sensing applications, ranging from panchromatic sharpening [28], semantic segmentation [29,30,31], change detection [32], image fusion [33] and image classification [34,35]. For image fusion tasks, representative methods such as RFN-Nest [36] and SDNet [37] enhance the information preservation and detail representation of fused images by integrating complementary information from multiple source images. More recently, spatial–frequency joint enhancement mechanisms have been introduced into Mamba-based image fusion. SFMFusion [33] incorporates spatial–frequency enhancement into the Mamba block and employs a dynamic fusion Mamba block for feature fusion. However, current Mamba-based segmentation methods still encounter certain limitations. Most existing approaches directly employ VSS blocks containing SS2D modules for feature extraction, failing to adequately address multiscale local interactions and the inherent constraints of causal scanning sequences in complex HRSI scenes.

In response to these limitations, we present SPFMamba, a segmentation architecture centered on Mamba that leverages semantic prompting to guide global context modeling and adaptively integrates enhanced high-frequency details with deep semantic information. SPFMamba comprises two primary modules: the semantic prompt global–local Mamba (SPGLM) block and the high-frequency adaptive fusion module (HFAFM). Specifically, SPGLM introduces semantic prompts into the selective scanning process to facilitate global context modeling and employs strip convolutions to capture multidirectional local information, thereby strengthening both global dependency modeling and local feature extraction. The HFAFM leverages a high-frequency extraction mechanism and pixel-wise cross-attention to adaptively fuse shallow details guided by deep semantic features, thereby strengthening small-scale object representation and boundary delineation. The main contributions are threefold:

(1)

SPFMamba is developed as a segmentation architecture centered on Mamba for HRSIs segmentation. It extracts features with high intraclass consistency and interclass discriminability while maintaining a relatively small model size, thereby producing more complete object structures and clearer segmentation boundaries.

(2)

The SPGLM block is proposed, combining learnable prompt-guided global modeling with multidirectional local perception to mitigate the inherent causal constraints of sequential scanning and jointly capture global semantic information and local spatial details.

(3)

The HFAFM is introduced to enhance shallow high-frequency details in the frequency domain and achieve deep semantic-guided adaptive fusion via cross-attention, thereby reducing cross-level semantic inconsistency and alleviating the loss of small-object details.

2. Related Work

2.1. CNN- and Transformer-Based Approaches for Semantic Segmentation

After FCNs [17] were introduced, CNN-based architectures became the dominant framework for semantic segmentation. Nevertheless, their decoder design tends to neglect spatial information in 2-D images, often producing severely fragmented segmentation results. To mitigate this problem, U-Net [18] adopted a symmetric encoder–decoder architecture. Shallow spatial cues are transferred to the decoder and combined with high-level semantic representations through long skip connections. This design helps produce feature maps with both fine spatial resolution and strong semantic information, which are essential for accurate semantic segmentation. Nevertheless, such feature connection mechanisms are usually unaware of whether the fusion is appropriately tailored to the representation of specific objects.

To enhance feature fusion capabilities, attention mechanisms have been widely incorporated to facilitate adaptive weighted fusion. Selective kernel network (SKNet) [38] proposes a nonlinear fusion strategy based on the channel attention mechanism. However, it primarily focuses on soft feature selection within a single feature level, failing to address feature fusion across layers in skip connections. To this end, the AFF module [19] presents a general attention-based fusion framework designed to alleviate semantic and scale inconsistencies during cross-layer feature aggregation. GDGNet [20] leverages feature weighting and attention mechanisms to develop a GLCF module, enabling low-level spatial cues to be adaptively combined with high-level semantic representations.

As the spatial resolution of RSIs increases, objects exhibit strong local similarities, rendering global contextual information essential for accurate segmentation. By introducing transformers into networks [39,40,41,42] for global context extraction, researchers have addressed the inherent limitations associated with restricted local receptive fields. UNetFormer [43] constructs its decoder with a global–local transformer block (GLTB) where parallel global and local branches are used to integrate multiscale contextual information and spatial details. CST-UNet [44] employs two parallel encoders based on CNNs and Swin Transformer to extract texture details and global semantics. Multilevel context-aware segmentation network (MCSNet) [45] adopts the CSWin Transformer as its backbone, utilizing its cross-shaped window attention mechanism to model global context. CGGLNet [46] introduces a category-guided supervised transformer module (CGSTM) to estimate potential pixel-class information, thereby enhancing intraclass aggregation and interclass separability. Transformer architectures are effective at modeling global context; however, their application to HRSIs incurs considerable computational overhead and may introduce feature aliasing. Moreover, during cross-layer feature fusion, deep semantic features can overwhelm shallow spatial details, leading to imbalanced feature representations and making small objects more difficult to segment accurately.

In the present study, an HFAFM is proposed to enhance discriminative feature learning for small objects. By introducing high-frequency enhancement mechanisms into both spatial and channel branches, the proposed module strengthens edge and texture information in shallow features and further leverages a pixel-wise cross-attention mechanism to achieve semantic-guided detail completion. Consequently, this approach helps preserve shallow detail cues and enhances the representation of subtle structures in small objects, thereby mitigating spatial-detail degradation caused by downsampling while maintaining semantic consistency in RSIs.

2.2. Mamba-Based Semantic Segmentation Methods

Inspired by Mamba’s capacity for linear-complexity global dependency modeling, researchers have recently introduced Mamba into the vision domain as an efficient alternative to conventional convolutional and Transformer-based architectures. Specifically, Vision Mamba [26] and VMamba [27] achieve efficient visual representation learning by incorporating a bidirectional SSM and an SS2D module, respectively.

Following the emergence of vision Mamba models, researchers began applying them to the semantic segmentation of RSIs. Existing Mamba-based methods are generally divided into two categories. The first category mainly adopts Mamba-based encoder architectures. RS3Mamba [47] incorporates VSS blocks into a complementary branch, providing global contextual cues for the convolutional main branch. PPMamba [29] constructs its encoder–decoder architecture using ResBlocks and PPMamba modules, enabling each local region to selectively incorporate richer global contextual information. WCMamba [48] enhances feature representation by integrating wavelet transforms and proposes a pyramid wavelet convolution Mamba module (PWCMamba) to improve multiscale feature modeling and global–local context integration. The second group utilizes Mamba-based decoders. UMFormer [49] presents a heterogeneous encoder–decoder design, in which convolutional layers form the encoder, a Transformer operates at the intermediate stage, and Mamba modules constitute the decoder. Its global–local vision state-space (GL-VSS) block supplements global representations with local contextual cues. GLVMamba [50] enhances global and local context aggregation in a Mamba decoder by incorporating a global–local VSS (GLVSS) block. PyramidMamba [30] incorporates a pyramid fusion Mamba (PFM) module into the decoder, reducing semantic redundancy in feature fusion.

In this paper, SPGLM is proposed to efficiently bolster the global context interaction capabilities of the network. Specifically, SPGLM introduces a semantic prompt-guided selective scanning mechanism in which learnable prompts are embedded into Mamba’s state-space formulation to effectively mitigate inherent causal limitations and information-forgetting issues in conventional SSMs. Additionally, SPGLM integrates an asymmetric convolution-based local branch, facilitating collaborative modeling of elongated object geometries and multiscale spatial details in complex scenes while maintaining linear computational complexity. Consequently, this approach facilitates more consistent segmentation predictions for semantically identical pixels or regions distributed across different spatial locations from a global perspective.

3. Methodology

This section presents the architecture and key components of SPFMamba. Section 3.1 outlines the overall design of SPFMamba. Section 3.2 and Section 3.3 detail the SPGLM block and HFAFM, respectively. Section 3.4 presents the loss function.

3.1. Network Architecture

Figure 2 depicts the overall design of SPFMamba. The network follows an encoder–decoder layout with skip connections. A lightweight ResNet-18 was adopted as the backbone, with four ResBlock stages progressively encoding multiscale spatial representations. During decoding, the SPGLM block was introduced to strengthen global–local contextual interaction in the feature reconstruction process. Within the intermediate skip connections, a selective feature fusion module (SFM) was incorporated to perform a selective weighted sum operation, aggregating features derived from ResBlocks with those generated by the SPGLM block. Subsequently, the HFAFM was introduced at the final decoder skip connection, where deep semantic information produced by the SPGLM block was integrated with early spatial cues extracted by the first ResBlock to better capture fine-grained structures in urban scenes. The reconstructed feature maps are finally processed by a segmentation head, comprising bilinear interpolation and ReLU activation to recover the input resolution and generate the final segmentation results. The weighted sum operation of the SFM is formulated as follows:

$$W_1,W_2=\mathrm{Softmax}([\sigma ({\mathrm{Conv}}_{1\times 1}\left(\mathrm{GAP}\right(F_1))),\sigma ({\mathrm{Conv}}_{1\times 1}(\mathrm{GAP}(F_{SPGLM})))])$$

(1)

$$F_{SFM}=W_1\odot F_1+W_2\odot F_{SPGLM}$$

(2)

where $\odot$ denotes element-wise multiplication; F₁ denotes the channel-aligned feature maps extracted from the ResNet backbone; F_SPGLM denotes the feature maps obtained by 2× upsampling the output of the SPGLM block; ${\mathrm{Conv}}_{1\times 1}(\cdot )$ is a 1 × 1 convolution, $\mathrm{GAP}(\cdot )$ indicates global average pooling.

3.2. Semantic Prompt Global–Local Mamba Block

With the continuous improvement of RSI spatial resolution, land-cover objects exhibit increasingly rich, fine-grained texture details, offering neural networks greater potential for learning discriminative representations. However, this also increases the risk of feature overlap among different categories, thereby exacerbating interclass confusion. Moreover, in complex scenarios—such as roads occluded by shadows cast by high-rise buildings—relying solely on local information often fails to provide sufficient context for accurate object recognition. By leveraging scene-level contextual cues, the network can characterize global spatial organization, such as road network continuity and holistic building contours, thereby effectively reducing misclassifications. Therefore, simultaneously modeling long-range global dependencies and multiscale local features is essential to obtain representations characterized by strong intraclass consistency and enhanced interclass separability.

As illustrated in Figure 3b, the SPGLM block is designed to address the above challenges. In the global branch, the semantic-guided selective scanning 2D (SGSS2D) module injects semantic prompts into the output matrix of the SSM, facilitating semantic-aware adaptive retrieval. These learnable prompts can represent specific sets of pixels sharing similar semantic attributes, which effectively compensates for the loss of contextual details caused by sequential scanning, thereby establishing consistent feature associations across long-range spatial distances. In the local branch, asymmetric convolutions are employed to extract directional geometric details, thereby capturing fine-grained local representations of multiscale targets. By integrating global context and local structural details, SPGLM significantly enhances semantic segmentation performance for RSIs. The following sections detail the key components of the SPGLM block.

Compared with the original VSS block architecture (Figure 3a) [27], the proposed SPGLM is structurally optimized to leverage the advantage of SSMs in long-range dependency modeling with linear computational complexity [51]. The input feature $x\in {ℝ}^{B\times H\times W\times C}$ was processed by two parallel global and local branches, as shown in Figure 3b, where B, H, W and C denote the batch size, feature-map height, feature-map width, and channel dimension, respectively. The channel dimension C was set to 512, 256, and 128 at the three decoder stages, respectively. In the global branch, the input feature first underwent layer normalization (LN) and linear projection to expand the channel dimension to 2C. The projected representation was then refined by a depthwise separable convolution followed by SiLU activation, and the resulting feature was passed to the SGSS2D module. Inspired by MambaIRv2 [51], the SGSS2D enhances the original SS2D by modifying the output matrix $C_{SSM}\in {ℝ}^{L\times d}$ of the SSM (where L = HW denotes the flattened image sequence length and d represents the number of hidden states). Specifically, learnable semantic prompts were injected into matrix C_SSM to mitigate the lack of adaptive semantic awareness in the original SS2D when processing complex remote sensing scenes. This design facilitates the global querying of relevant pixels sharing similar semantic attributes across the image.

As shown in Figure 3c, we first adopt a low-rank decoupling strategy to construct a prompt pool $\mathcal{P}\in {ℝ}^{T\times d}$ comprising T learnable semantic prompts:

$$\mathcal{P}=MN,M\in {ℝ}^{T\times r},N\in {ℝ}^{r\times d},r\le \min \left(T,d\right)$$

(3)

where $N$ denotes the semantic basis matrix used to construct prompts. M is the coefficient matrix that combines the basis features within a specific block to generate instance-relevant prompts. $r$ denotes the internal rank. This low-rank parameterization lowers the parameter overhead while improving the representation capacity of the prompts.

To achieve pixel-wise adaptive prompt selection, given the input feature $x^{\prime }\in {ℝ}^{L\times C}$ was projected from C to T channels via a linear layer. A LogSoftmax function was then applied to estimate the sampling probability of each prompt in $\mathcal{P}$ with respect to the current input $x_i^{\prime }(i=1,2,\mathrm{\dots },L)$. Gumbel-Softmax trick [52] was then utilized to generate a routing matrix $R\in {ℝ}^{L\times T}$. Subsequently, instance-specific prompts were obtained through matrix multiplication $\mathrm{P}=R\mathcal{P}$, where $\mathrm{P}\in R^{L\times d}$. Finally, the generated instance-specific prompt P was incorporated into the output matrix C_SSM via residual addition (Figure 3d). The corresponding formulation is expressed as:

$$\overline{A},\overline{B}=\exp (\Delta A),\Delta B$$

(4)

$$h_t=\overline{A}{h}_{t-1}+\overline{B}{x}_t$$

(5)

$$y_t=(C_{SSM}+P)h_t+D{x}_t$$

(6)

$$y=\left[y_1,y_2,\mathrm{\dots },y_L\right]$$

(7)

where $\Delta$ denotes the timescale parameter used to discretize the continuous system parameters $\Delta A$ and $\Delta B$ into their discrete counterparts $\overline{A}$ and $\overline{B}$; x_t and y_t denote the input feature and output feature at time step t; $h_t\in {ℝ}^{\mathrm{d}}$ signifies the hidden state at time step t; C_SSM is the output matrix; $\mathrm{P}\in R^{L\times d}$ indicates the generated instance-specific semantic prompt; D represents the feed-forward projection parameters; and L denotes the flattened image sequence length.

By incorporating prompt learning into the four-directional scanning paths, the SGSS2D module enables each query pixel to attend to regions sharing similar semantic attributes across the image via prompts. This mechanism facilitates adaptive aggregation of global contextual information and promotes deep feature interaction, thereby significantly enhancing intraclass consistency in land-cover representation. Subsequently, the one-dimensional sequence features extracted along different scanning directions were reshaped and reorganized into a 2-D representation aligned with the input spatial resolution. This process constructs a semantically guided global receptive field, facilitating effective global context modeling. The overall formulation is expressed as:

$$X_{global}=\mathrm{LN}(\mathrm{SGSS}2\mathrm{D}(\mathrm{SiLU}(\mathrm{DW}\mathrm{Conv}(\mathrm{Linear}(\mathrm{LN}(X))))))$$

(8)

where X_global denotes the output feature of the global branch; $\mathrm{Linear}(\cdot )$ denotes linear projection; $\mathrm{LN}(\cdot )$ represents layer normalization; $\mathrm{SiLU}(\cdot )$ signifies the activation function; $\mathrm{DW}\mathrm{Conv}(\cdot )$ indicates depthwise separable convolution; and $\mathrm{SGSS}2\mathrm{D}(\cdot )$ refers to the SGSS2D module.

In the local branch, parallel horizontal and vertical strip convolutions were adopted to model the diverse geometric patterns and prominent directional properties of RSI objects. This design also mitigates the weakness of SSMs in fine-grained local feature representation due to their limited inductive bias [53,54]. This design employs orthogonal convolutional paths to effectively capture directional geometric patterns, thereby enhancing multiscale spatial detail representation. The process is formulated as follows:

$$X_{local}={\mathrm{CBR}}_{3\times 3}({\mathrm{CBR}}_{3\times 1}({\mathrm{CBR}}_{1\times 1}(X))+{\mathrm{CBR}}_{1\times 3}({\mathrm{CBR}}_{1\times 1}(X)))$$

(9)

where ${\mathrm{CBR}}_{\mathrm{a}\times \mathrm{b}}(\cdot )$ represents a composite operator consisting sequentially of an $\mathrm{a}\times \mathrm{b}$ convolution, batch normalization and ReLU activation.

Finally, the global semantic features X_global and the local geometric detail features X_local were fused via channel-wise concatenation. A convolutional layer is subsequently applied to facilitate feature interaction and channel alignment, further enriching the semantic representation and yielding fine-grained global–local contextual features. The fusion process is defined as follows:

$$X_{SPGLM}=X+{\mathrm{CBR}}_{3\times 3}(\mathrm{Concat}(X_{global},X_{local}))$$

(10)

where X_SPGLM denotes the output feature of the SPGLM block; $\mathrm{Concat}(\cdot )$ represents channel concatenation.

3.3. High-Frequency Adaptive Fusion Module

RSIs contain abundant fine-grained details. However, small-scale objects (e.g., vehicles) often occupy only a few pixels, which makes their characteristics vulnerable to distortion or suppression from background clutter during downsampling. To mitigate this issue, skip connections are widely adopted to integrate multilevel features. Features at different depths exhibit significant representational disparities: shallow features maintain detailed spatial information but lack strong semantic representation, while deep features offer improved semantic discriminability at the cost of fine spatial structures. To effectively integrate shallow spatial features with deep semantic content, existing approaches often utilize element-wise addition or channel concatenation. These fusion strategies usually rely on fixed weighting schemes and cannot adaptively adjust feature contributions according to image content, limiting the complementary interaction between spatial details and semantic information. To address these limitations, we introduced HFAFM, illustrated in Figure 4. This module explicitly enhances high-frequency details and introduces pixel-level local cross-attention to achieve cross-layer alignment and feature fusion guided by deep semantic information. By integrating shallow spatial information with deep semantics, HFAFM improves the discriminability and consistency of small-scale objects and structural features.

Specifically, the HFAFM comprises two submodules: the multiscale high-frequency enhancement feature module (MHFEM) and the local-window cross-attention adaptive fusion module (LCFM). As illustrated in Figure 4a, the features output by the first ResBlock were fed into the MHFEM for small-object detail enhancement. MHFEM employs channel and spatial branches to selectively refine fine-grained feature responses. Since small-scale objects occupy only a limited proportion of the original feature maps, directly computing weights from global features is susceptible to interference from low-frequency homogeneous backgrounds. Therefore, we employ a high-pass filter M that suppresses low-frequency components while preserving high-frequency information [55,56,57], as defined below:

$$M(i,j)=\left\{\begin{array}{c}0,i<\alpha H,j<\alpha W\\ 1,\mathrm{otherwise}\end{array},\alpha \in \right.\left[0,1\right]$$

(11)

where $\alpha$ is set to 0.3 and controls the suppression range of low-frequency components, thereby reducing low-frequency content while retaining high-frequency information associated with object boundaries and textures. H and W denote the height and width of the image, respectively.

In the channel branch, to enhance channels containing critical detail information, a two-dimensional discrete cosine transform (DCT) first mapped the input feature F_res to frequency-domain coefficients. The high-pass filter M was then applied, followed by the inverse DCT (IDCT) to restore the feature to the spatial domain, yielding a preliminary enhanced feature F_{high_c}. Considering that different channels contribute unequally to detail representation, the channel-wise statistics F_avg and F_max were derived from F_{high_c} through average- and max-pooling operations with an output size of k × k, where k is set to 16. These statistics were further fused to generate channel attention weights, which were multiplied with F_res to obtain $F_{channel}$. The process is described below:

$$F_{\mathrm{avg}}={\displaystyle \sum _{h=1}^k{\displaystyle \sum _{w=1}^k[\mathrm{ReLU}(\mathrm{Avg}{\mathrm{Pool}}_k(F_{high\_c})){]}_{h,w}}}$$

(12)

$$F_{\max }={\displaystyle \sum _{h=1}^k{\displaystyle \sum _{w=1}^k[\mathrm{ReLU}(\mathrm{Max}{\mathrm{Pool}}_k(F_{high\_c})){]}_{h,w}}}$$

(13)

$$F_{channel}=F_{\mathrm{res}}\odot \sigma ({\mathrm{CBR}}_{1\times 1}({\mathrm{CBR}}_{1\times 1}(F_{\mathrm{avg}})+{\mathrm{CBR}}_{1\times 1}(F_{\max })))$$

(14)

where $\mathrm{Max}{\mathrm{Pool}}_{\mathrm{k}}(\cdot )$ and $\mathrm{Avg}{\mathrm{Pool}}_{\mathrm{k}}(\cdot )$ denote max pooling and average pooling with an output size of k × k, respectively.

Within the spatial branch, high-pass filters were applied in parallel to the input feature map and its downsampled feature to capture multiscale structural information of small-scale targets. The frequency-domain features at the two scales were fused and subsequently mapped to the spatial domain via IDCT to yield a spatial attention mask M_spatial. Multiplying this mask by the original feature map enhances feature responses in high-frequency salient regions, enhancing the edge and texture representations of small objects. This procedure is formulated as follows:

$$M_{spatial}= \mathrm{IDCT}(\mathrm{DCT}(F_{res})\odot M+\mathrm{U}{\mathrm{p}}_2\times (\mathrm{DCT}({\mathrm{Down}}_2\times (F_{res}))\odot M))$$

(15)

$$F_{spatial}=F_{res}\odot M_{spatial}$$

(16)

where $\mathrm{DCT}(\cdot )$ and $\mathrm{IDCT}(\cdot )$ represent the discrete cosine transform and its inverse, respectively. ${\mathrm{Down}}_{2\times }(\cdot )$ and $\mathrm{U}{\mathrm{p}}_{2\times }(\cdot )$ indicate the 2× downsampling and upsampling implemented by bilinear interpolation.

To adaptively combine the spatial and channel representations, a gated fusion mechanism was introduced at the final stage. Specifically, F_spatial and F_channel were concatenated to dynamically generate an input-dependent gating weight g, facilitating the weighted aggregation of both feature streams:

$$g=\mathrm{sigmoid}\left({\mathrm{CB}}_{1\times 1}\right(\mathrm{Concat}(F_{spatial},F_{channel})))$$

(17)

$$F_{HFEM}=\mathrm{Re}\mathrm{LU}(\mathrm{GN}({\mathrm{Conv}}_{3\times 3}(g\odot F_{spatial}+(1-g)\odot F_{channel})))$$

(18)

where $\mathrm{Concat}(\cdot )$ denotes channel concatenation; $\mathrm{sigmoid}(\cdot )$ represents the activation function; ${\mathrm{Conv}}_{3\times 3}(\cdot )$ denotes a $3\times 3$ convolutional layer; $\mathrm{GN}(\cdot )$ indicates group normalization; and $\mathrm{Re}\mathrm{LU}(\cdot )$ denotes activation function.

To enhance semantic consistency between deep and shallow features while facilitating cross-level alignment, the LCFM performs weighted fusion of the enhanced shallow high-frequency details F_MHFEM under the guidance of the deep semantic feature F_x, thereby improving the representation of small-scale targets, as illustrated in Figure 4b. Specifically, the deep representation F_x was first upsampled to ensure that its spatial resolution is consistent with that of F_MHFEM. Subsequently, the upsampled F_x was projected into the query matrix Q, while F_MHFEM was projected into the key and value matrices K and V. To balance computational efficiency and local modeling capability, we divided the feature map into disjoint local windows along the spatial dimensions. The input feature is $x\in {ℝ}^{B\times H\times W\times C}$, and the window size is P = p₁ × p₂, where $p_1=p_2=16$. After a reshaping operation, the feature map is rearranged into a series of local windows, resulting in a feature representation of $x\in {ℝ}^{B\times {\displaystyle \frac{H}{p_1}}\times {\displaystyle \frac{W}{p_2}}\times P\times C}$. In each local window, the similarity between Q and K is computed, and the resulting attention weights are normalized using the Softmax function to aggregate V. This process enables the deep feature representation to adaptively aggregate semantically relevant shallow detail information from V within each window, thereby achieving pixel-wise adaptive feature interaction. The attention operation is defined as:

$$Attention(Q,K,V)=\mathrm{Softmax}({\displaystyle \frac{Q{K}^T}{\sqrt{d}}})V$$

(19)

where $d$ denotes the channel dimension and $\sqrt{d}$ is the scaling factor for normalization.

Finally, the attention output F_attention was merged with the upsampled F_x via a residual connection. The combined representation was refined by a 3 × 3 convolution for spatial integration. A pixel-wise weight map generated by a 1 × 1 convolution was further used to adaptively recalibrate the fused features. This design effectively preserves shallow edge and texture cues, improving the model’s performance in capturing small-scale objects and structural details in complex remote sensing scenes.

3.4. Loss Function

A multi-head supervision strategy was adopted to enhance the training stability of SPFMamba. This strategy consists of a semantic segmentation head and an auxiliary boundary head. The overall loss function is composed of a main loss and an auxiliary loss.

To mitigate class imbalance and preserve regional consistency, the main loss is defined below:

$$L_{ce}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N{\displaystyle \sum _{k=1}^K{y}_k^{\left(n\right)}}}\log {\widehat{y}}_k^{\left(n\right)}$$

(20)

$$L_{dice}=1-{\displaystyle \frac{2}{N}}{\displaystyle \sum _{n=1}^N{\displaystyle \sum _{k=1}^K\frac{y_k^{\left(n\right)}{\widehat{y}}_k^{\left(n\right)}}{y_k^{\left(n\right)}+{\widehat{y}}_k^{\left(n\right)}}}}$$

(21)

$$L_{main}=L_{ce}+L_{dice}$$

(22)

where N gives the total pixel count; K specifies the number of classes; $y^{\left(n\right)}$ denotes the one-hot encoded ground-truth semantic label; and ${\widehat{y}}_k^{\left(n\right)}$ represents the predicted probability that pixel n belongs to class k, where $n\in \left\{1,\mathrm{\dots },N\right\}$.

Considering that multiscale features contain rich structural cues, we introduced an auxiliary boundary branch to explicitly enhance boundary awareness. Specifically, features from different decoder stages, namely h₂, h₃, and h₄ were resampled to a common spatial size, concatenated and then processed by a convolutional layer to obtain a boundary-aware representation. The auxiliary head then transforms this representation into a boundary estimate y_edge. Meanwhile, the ground-truth boundary label g_edge was constructed from the ground-truth segmentation mask using a Laplacian operator followed by binarization. Finally, a binary Dice loss was adopted to measure the consistency of the predicted boundary with ground-truth boundary. The overall loss is:

$$L=L_{main}+\lambda L_{aux}(y_{edge},g_{edge})$$

(23)

where $\mathsf{\lambda }$ denotes the trade-off hyperparameter between the two loss terms, empirically determined as 0.4 through ablation experiments.

4. Experiments

4.1. Datasets

ISPRS Vaihingen: The Vaihingen dataset comprises 33 true orthophoto (TOP) images with a ground sampling distance (GSD) of 9 cm. Each TOP image consists of three spectral bands, namely near-infrared, red, and green. In addition, digital surface model (DSM) and normalized digital surface model (nDSM) are provided for this dataset. The dataset contains six land-cover categories, including five foreground classes (impervious surfaces, buildings, low vegetation, trees, and cars) and one background class (clutter). Following the commonly used split in previous studies [46], 16 tiles were allocated to training and the rest to testing, after which they were cropped into 1024 × 1024-pixel patches.

ISPRS Potsdam: The Potsdam dataset contains 38 TOP tiles, each measuring 6000 × 6000 pixels at a GSD of 5 cm. The semantic categories are identical to those in the Vaihingen dataset. In addition to DSM and nDSM data, each tile provides four spectral bands, namely red, green, blue, and near-infrared. Only the RGB bands were used as input in our experiments. Of the 38 tiles, 23 were used for training, tile 7_10 was excluded because of annotation errors, and the remaining tiles were reserved for testing. All retained tiles were cropped into 1024 × 1024-pixel patches.

OpenEarthMap: The OpenEarthMap dataset comprises 5000 images collected from 97 regions across 44 countries on 6 continents. The GSD ranges from 0.25 m to 0.5 m. It includes eight land-cover types: bareland, rangeland, developed space, road, tree, water, agriculture land, and building. Since the official test set is not publicly released, the validation set was adopted for quantitative evaluation in our experiments. Before being fed into the network, all images were resized to a fixed size of 1024 × 1024 pixels.

4.2. Implementation Details

Experiments were carried out in PyTorch 2.2.1 on a server with a single NVIDIA 3090 Ti GPU (NVIDIA Corporation, Santa Clara, CA, USA). The model was optimized using AdamW, and the encoder was initialized with pretrained weights. Specifically, the learning rates were initialized as 6 × 10⁻⁵ for the encoder and 6 × 10⁻⁴ for the remaining learnable parameters, while the weight decay coefficient was fixed at 0.01. In addition, a cosine annealing strategy was adopted to update the learning rate during the training process. During network training, the images were augmented through random cropping, flipping, and multiscale resizing. The model was trained for 105 epochs with a batch size of 4.

4.3. Evaluation Metrics

The proposed model was evaluated from two aspects, namely model performance and model complexity. The complexity metrics include floating-point operations (FLOPs) and the number of parameters, all computed with an input size of 3 × 1024 × 1024. The performance metrics include overall accuracy (OA), mean F1 score (mF1), and mean intersection over union (mIoU).

5. Results

5.1. Ablation Studies

5.1.1. Effectiveness of Each Module in SPFMamba

Ablation experiments were conducted on two datasets under identical training settings to quantify the contributions of individual modules and their combinations. The quantitative results are shown in

Table 1. The ablation experimental results of each module on the Vaihingen dataset.

Dataset	Method	mF1 (%)	OA (%)	mIoU (%)	Params (M)	FLOPs (G)
Vaihingen	Baseline	90.20	90.84	82.39	12.21	46.91
	Baseline + SPGLM	91.01	91.52	83.70	17.74	75.58
	Baseline + HFAFM	90.70	91.10	83.18	12.31	56.52
	Baseline + SPGLM + HFAFM	91.33	91.67	84.26	17.85	85.18
Potsdam	Baseline	92.44	90.97	86.17	12.21	46.91
	Baseline + SPGLM	92.83	91.33	86.83	17.74	75.58
	Baseline + HFAFM	92.64	91.13	86.51	12.31	56.52
	Baseline + SPGLM + HFAFM	93.12	91.78	87.34	17.85	85.18

Boldface denotes the best result in each column.

.

The baseline model is built upon a U-Net architecture with ResNet-18 as the backbone. The decoder employs the original VSS block, while the SFM is used for feature fusion between adjacent layers. Subsequently, the original VSS blocks were replaced with SPGLM to improve the joint capture of global context and local details across multiple feature scales, yielding Baseline+SPGLM. Across both datasets, this model improves mF1, OA, and mIoU by at least 0.39%, 0.36%, and 0.66%, respectively. Although the introduction of SPGLM increases the parameter count by approximately 5.53 M and the computational cost by about 28.67 GFLOPs, its superior feature modeling capability contributes to noticeable performance gains in complex scenes. Introducing the HFAFM into the baseline yields the Baseline+HFAFM, which increases mF1, OA, and mIoU by at least 0.20%, 0.16%, and 0.34%, respectively. Notably, the HFAFM introduces only 0.10 M additional parameters and has little impact on overall model complexity, while still providing stable performance improvements, demonstrating its effectiveness in feature fusion.

Finally, SPGLM and the HFAFM were jointly integrated to form the complete SPFMamba (Baseline+SPGLM+HFAFM). The results indicate that these modules exhibit good complementarity in feature extraction and fusion. Compared with the baseline, SPFMamba achieves improvements on both datasets, with minimum gains of 0.68%, 0.81%, and 1.17% in mF1, OA, and mIoU, respectively. These ablation studies validate the efficacy of the proposed global–local context modeling and feature fusion strategy.

5.1.2. Setting of the Number of SPGLM Blocks in the Decoder

To examine how the number of SPGLM blocks affects performance, we evaluated three configurations: [4,3,3], [8,6,6], and [16,12,12], as shown in

Table 2. Ablation experiment on SPGLM block count setting on the Vaihingen dataset.

Block Configuration	mF1 (%)	OA (%)	mIoU (%)	Params (M)	FLOPs (G)
[4,3,3]	90.59	91.16	83.01	14.76	69.68
[8,6,6]	91.33	91.67	84.26	17.85	85.18
[16,12,12]	90.93	91.33	83.56	24.02	116.20

Boldface denotes the best result in each column.

. The [8,6,6] configuration achieves the best segmentation performance, indicating that this configuration provides a favorable balance between model depth and representation capability. Parameter count and FLOPs increase with deeper configurations. Compared with the [4,3,3] configuration, the [8,6,6] setting benefits from increased decoder depth and enhanced feature representation. However, when the depth is increased to [16,12,12], the model performance instead decreases, with mF1, OA, and mIoU decreasing by 0.40%, 0.34%, and 0.70%, respectively. This indicates that excessive model complexity may introduce optimization difficulties and make the model more prone to overfitting, thereby leading to performance degradation. Therefore, the [8,6,6] configuration is adopted as the default setting.

5.1.3. Ablation Study on the Semantic Prompt Mechanism and Its Hyperparameters

To evaluate the contribution of the semantic prompt, we removed the semantic prompt mechanism introduced in Equation (6) while keeping the SPGLM structure and all other network components unchanged, as shown in

Table 3. Ablation study on the semantic prompt mechanism in the SPGLM block on the Vaihingen dataset.

Semantic Prompt	mF1 (%)	OA (%)	mIoU (%)
×	90.89	91.28	83.52
√	91.33	91.67	84.26

“×” and “√” indicate that the semantic prompt mechanism is disabled and enabled, respectively.

. Compared with the variant without semantic prompts, the proposed model improves mF1, OA, and mIoU by 0.44%, 0.39%, and 0.74%, respectively. This indicates that the semantic prompt enhances the semantic awareness of SSM readout and facilitates the aggregation of long-range contextual information.

The proposed SPGLM module incorporates a prompt-learning mechanism to aggregate global similar-pixel context information and integrate it into the SSM, mitigating the causal limitation of the scanning mechanism. The mechanism comprises two key hyperparameters: prompt-pool size $T$ and semantic-decoupling internal rank $r$. Specifically, $T$ determines the number of learnable semantic prototypes in the prompt pool, while $r$ controls the dimensionality of the low-rank prompt embedding. Considering the significant intraclass variability of remote sensing objects, $T$ is set larger than the number of classes in the dataset, with the aim of using richer semantic primitives to characterize the diverse appearances of the same category across different textures, illumination conditions, and contextual environments.

To analyze how $T$ and $r$ affect model performance and computational complexity, ablation experiments are conducted on the Vaihingen dataset with three representative configurations: $(T,r)$= (32,16), (64,32), (128,64).

Table 4. Ablation experiment on the hyperparameters of SPGLM block on the Vaihingen dataset.

$\mathit{T}$	$\mathit{r}$	mF1 (%)	OA (%)	mIoU (%)	Params (M)	FLOPs (G)
32	16	90.97	91.40	83.63	17.79	85.05
64	32	91.33	91.67	84.26	17.85	85.18
128	64	90.69	91.20	83.18	18.04	85.46

Boldface denotes the best result in each column.

shows that when $T$ and $r$ are set to relatively small values, the limited prompt pool capacity and low-dimensional prompt representation constrain the model’s ability to capture complex semantic variations in remote sensing scenes, leading to inferior segmentation performance. When $(T,r)=(64,32)$, the model obtains the optimal performance, suggesting that a moderate number of prompts and an appropriate inner rank can provide a favorable trade-off between semantic richness and feature discriminability. When $(T,r)$ is increased to (128, 64), both the parameter count and FLOPs increase, whereas the model performance shows no further improvement and even deteriorates. This phenomenon may be attributed to the feature redundancy introduced by an excessively large prompt pool and higher-dimensional prompt representations, as well as the potential risk of overfitting. Considering both segmentation accuracy and computational efficiency, $T=64$ and $r=32$ are adopted as the default configuration of the SPGLM block.

5.1.4. Ablation Study on the Kernel Size in the Local Branch

To investigate the effect of kernel size on local feature aggregation in Equation (9), we replaced the original 3 × 3 convolution with 5 × 5 and 7 × 7 convolutions, while keeping all other network structures and training settings unchanged.

Table 5. Ablation study on the convolution kernel size in Equation (9) on the Vaihingen dataset.

k × k	mF1 (%)	OA (%)	mIoU (%)
3 × 3	91.33	91.67	84.26
5 × 5	91.25	91.66	84.11
7 × 7	91.09	91.52	83.85

Boldface denotes the best result in each column.

shows that the 3 × 3 kernel yields the strongest overall results, with an mF1 of 91.33%, an OA of 91.67%, and an mIoU of 84.26%. Compared with the 5 × 5 and 7 × 7 kernels, the 3 × 3 kernel improves mIoU by 0.15% and 0.41%, respectively. Because the strip convolutions and the SGSS2D branch already capture directional local context and long-range dependencies, larger kernels provide limited additional information and may oversmooth fine-grained structures. Therefore, the 3 × 3 configuration is adopted as the default setting.

5.1.5. Hyperparameter Selection for the Loss Function

The overall loss function consists of two components, with $\mathsf{\lambda }$ controlling the contribution of the auxiliary loss. To determine an appropriate value, $\mathsf{\lambda }$ was evaluated under five settings: 0.2, 0.4, 0.6, 0.8, and 1.0.

Table 6. Ablation study on the loss weight $\mathsf{\lambda }$ on the Vaihingen dataset.

$\mathsf{\lambda }$	mF1 (%)	OA (%)	mIoU (%)
0.2	91.27	91.59	84.15
0.4	91.33	91.67	84.26
0.6	91.30	91.59	84.20
0.8	91.29	91.60	84.18
1.0	91.26	91.59	84.12

Boldface denotes the best result in each column.

shows the experimental results. When $\mathsf{\lambda }$ changes from 0.2 to 0.4, the segmentation performance is enhanced, reaching its optimum at $\mathsf{\lambda }=0.4$. This suggests that appropriately increasing the weight of the auxiliary boundary loss imposes more effective structural constraints, improving boundary awareness and overall segmentation accuracy. However, when $\mathsf{\lambda }$ exceeds 0.4, all performance metrics decline to varying degrees. This may be attributed to the excessively large weight assigned to the auxiliary loss, which causes the model to overemphasize boundary constraints, thereby impairing the learning of the primary segmentation task and ultimately degrading the overall segmentation performance. Therefore, $\mathsf{\lambda }=0.4$ is selected as the optimal value.

5.2. Comparative Experimental Results

The proposed SPFMamba was compared with several advanced segmentation methods, including multiscale attention-based ABCNet [58], edge information embedding-based EIGNet [59], frequency-domain enhancement-based AFENet [60], Transformer-based methods DC-Swin [61], UNetFormer [43], CGGLNet [46], SFFNet [62], and the Mamba-based methods PyramidMamba [30], RS3Mamba [47], and PPMamba [29].

5.2.1. Results on Vaihingen

Following the same protocol as in previous experiments, the accuracy results for the clutter/background class are not reported.

Table 7. Comparison results on the ISPRS Vaihingen dataset.

Method	Per-Class F1 Score(%)/Per-Class IoU Score (%)					mF1 (%)	OA (%)	mIoU (%)
	Imp.Surf.	Building	Low Veg	Tree	Car
ABCNet [58]	92.52/86.08	95.23/90.89	83.65/71.89	89.84/81.55	86.22/75.78	89.49	90.47	81.24
DC-Swin [61]	92.84/86.65	96.01/92.33	84.16/72.66	89.28/80.64	84.62/73.33	89.38	90.79	81.12
UNetFormer [43]	92.64/86.30	95.55/91.49	84.05/72.48	89.77/81.44	87.39/77.60	89.88	90.72	81.86
EIGNet [59]	92.50/86.05	95.86/92.05	85.12/74.09	90.62/82.85	87.52/77.81	90.32	91.00	82.57
CGGLNet [46]	93.07/87.04	96.13/92.56	85.34/74.42	90.38/82.45	90.30/82.32	91.05	91.41	83.76
SFFNet [62]	93.15/87.17	95.92/92.16	84.80/73.60	90.39/82.47	90.20/82.14	90.89	91.24	83.51
RS3Mamba [47]	92.73/86.45	95.37/91.16	85.04/73.98	90.09/81.97	86.13/75.64	89.87	90.91	81.84
PyramidMamba [30]	92.75/86.49	96.00/92.31	84.90/73.76	90.23/82.20	87.78/78.23	90.33	91.14	82.60
PPMamba [29]	93.11/87.12	95.94/92.20	85.02/73.94	90.50/82.65	88.68/79.66	90.65	91.32	83.11
AFENet [60]	93.13/87.14	95.68/91.71	85.17/74.17	90.44/82.55	89.50/80.99	90.78	91.26	83.31
Ours	93.59/87.96	96.39/93.03	85.38/74.48	90.57/82.76	90.74/83.04	91.33	91.67	84.26

Best results in bold, second best underlined.

indicates that SPFMamba attains the highest overall performance, surpassing the second-ranked method by 0.28% in mF1, 0.50% in mIoU, and 0.26% in OA. It also achieves the top F1 and IoU scores for nearly all categories except tree. Notably, for the small-scale car class, SPFMamba achieves the best F1 of 90.74% and IoU of 83.04%, outperforming the second-best method by 0.44% and 0.72%.

As shown in Figure 5, ABCNet enhances contextual modeling by introducing an attention mechanism, but it still suffers from incomplete object boundaries and local misclassification in complex regions. Transformer-based methods, such as UNetFormer, SFFNet, and DC-Swin, enhance global dependency modeling and improve structural representation, while DC-Swin is also advantageous in preserving intra-object integrity. However, these methods still show limitations in fine-grained segmentation of complex topological regions, often leading to confusion between adjacent objects, imprecise boundary localization, and local merging or misclassification. EIGNet improves boundary smoothness, but remains susceptible to false edges, resulting in incomplete segmentation. CGGLNet incorporates category information guidance and performs well in extracting large-scale buildings, but it still has limitations in complex local regions. In particular, category confusion and boundary adhesion remain evident in areas where vehicles are adjacent to surrounding objects and where shadow interference is strong. RS3Mamba and PPMamba employ SS2D for global context modeling, but they still exhibit missed detections and misclassifications when identifying small vehicles in low vegetation areas. AFENet enhances high-frequency details and supplements low-frequency context through adaptive frequency separation to improve boundary localization in certain vegetation regions, but building segmentation under shadow interference still suffers from insufficient connectivity and local fragmentation. Overall, SPFMamba outperforms the other methods, especially in the complex regions highlighted by the pink boxes. For large-scale buildings, SPFMamba maintains intra-class semantic consistency and avoids fragmentation or holes, thereby producing more complete segmentation results. For spatially adjacent targets, such as low vegetation patches on impervious surfaces or buildings under shadow interference, SPFMamba accurately identifies the separation regions between adjacent objects, effectively mitigating boundary merging in complex adjacent areas and producing clear and independent object contours. For small targets, SPFMamba demonstrates stronger structural preservation and finer detail discrimination, reducing missed detections and misclassifications and enabling more accurate extraction of small targets. These observations indicate that SPFMamba can produce more complete and precise segmentation results for objects of varying scales in complex scenes.

5.2.2. Results on Potsdam

As reported in

Table 8. Comparison results on the ISPRS Potsdam dataset.

Method	Per-Class F1 Score (%)/Per-Class IoU Score (%)					mF1 (%)	OA (%)	mIoU (%)
	Imp.Surf.	Building	Low Veg	Tree	Car
ABCNet [58]	92.67/86.35	96.56/93.36	87.16/77.24	88.64/79.60	96.01/92.33	92.21	90.56	85.78
DC-Swin [61]	93.14/87.16	96.88/93.94	87.41/77.64	88.40/79.22	95.38/91.16	92.24	90.90	85.83
UNetFormer [43]	92.76/86.50	96.58/93.39	86.95/76.92	88.29/79.04	96.03/92.36	92.12	90.58	85.64
EIGNet [59]	92.82/86.61	96.58/93.38	87.34/77.52	88.77/79.80	96.58/93.39	92.42	90.71	86.14
CGGLNet [46]	93.43/87.66	97.31/94.75	87.90/78.42	89.20/80.51	96.64/93.47	92.90	91.43	86.97
SFFNet [62]	93.50/87.80	97.22/94.58	87.92/78.45	89.23/80.56	96.56/93.34	92.89	91.43	86.95
RS3Mamba [47]	93.64/88.04	97.17/94.50	87.77/78.21	89.09/80.32	96.33/92.93	92.80	91.27	86.80
PyramidMamba [30]	93.54/87.87	97.06/94.30	88.00/78.57	89.11/80.37	96.50/93.24	92.84	91.47	86.87
PPMamba [29]	93.66/88.07	97.14/94.44	88.15/78.80	88.99/80.17	96.21/92.70	92.83	91.50	86.84
AFENet [60]	93.77/88.28	97.10/94.36	87.88/78.38	89.23/80.55	96.62/93.46	92.92	91.53	87.00
Ours	93.89/88.49	97.31/94.76	88.22/78.92	89.43/80.89	96.72/93.65	93.12	91.78	87.34

Best results in bold, second best underlined.

, SPFMamba was further evaluated on the larger ISPRS Potsdam dataset. Following the evaluation practice commonly used in previous studies, the accuracy results associated with clutter/background are not reported, since this category is sparsely represented in the dataset. SPFMamba ranks first in mF1, mIoU, and OA, exceeding the runner-up by 0.20%, 0.34%, and 0.25%. It also achieves the best performance in F1 and IoU across all classes. For the car class, the F1 and IoU reach 96.72% and 93.65%, respectively, outperforming representative Mamba-based methods such as RS3Mamba and PPMamba.

To facilitate intuitive comparison, visual results on several challenging test scenes are presented in Figure 6. Specifically, in the first and third scenes, the buildings marked by pink boxes exhibit local textures highly similar to impervious surfaces and background, rendering them susceptible to misclassification. In the second scene, small cars are easily obscured by complex background noise, often leading to false positives and missed detections. Compared with other methods, SPFMamba maintains high segmentation accuracy in these challenging regions, exhibiting more complete object structures and clearer boundary localization, especially within the pink-boxed areas. This indicates that the introduced SPGLM block, by leveraging semantic prompts and efficient sequence modeling, captures global contextual information more effectively, thereby enhancing intraclass feature consistency and improving interclass discriminability. The HFAFM strengthens edge and texture responses in the frequency domain for high-frequency details, such as small vehicles and building boundaries, while effectively suppressing interference from low-frequency background noise. Consequently, under the guidance of deep semantic features, the model preserves spatial details more effectively, leading to superior performance in both object completeness and boundary localization accuracy. Overall, SPFMamba not only achieves more complete segmentation of multiscale targets on the Potsdam dataset, but also demonstrates stronger boundary localization capability and finer local detail representation in complex backgrounds, further validating its effectiveness and robustness for semantic segmentation of remote sensing images.

5.2.3. Results on OpenEarthMap

To assess its generalization across geographically diverse regions, SPFMamba was further tested on the OpenEarthMap dataset, which contains geographically diverse scenes worldwide. As reported in

Table 9. Comparison results on the OpenEarthMap dataset.

Method	Per-Class F1 Score (%)/Per-Class IoU Score (%)								mF1 (%)	OA (%)	mIoU (%)
	Bareland	Rangeland	Developed	Road	Tree	Water	Agriculture	Building
ABCNet [58]	56.53/39.40	70.04/53.89	66.88/50.24	74.13/58.90	80.32/67.12	76.15/61.48	84.21/72.72	84.81/73.63	74.13	75.95	59.67
DC-Swin [61]	52.52/35.61	69.00/52.67	68.01/51.53	74.02/58.75	82.65/70.43	85.93/75.33	85.15/74.15	86.94/76.90	75.53	77.70	61.92
UnetFormer [43]	55.12/38.04	70.23/54.12	69.53/53.30	76.06/61.37	82.31/69.94	83.85/72.19	85.81/75.15	86.85/76.76	76.22	78.08	63.47
EIGNet [59]	55.80/38.69	71.67/55.85	69.35/53.08	76.51/61.96	81.91/69.36	84.14/72.62	86.92/76.86	87.33/77.52	76.70	78.54	63.24
CGGLNet [46]	57.27/40.12	72.24/56.54	71.51/55.66	77.22/62.89	82.79/70.63	85.11/74.08	85.88/75.25	86.88/76.80	77.36	78.98	64.00
SFFNet [62]	55.57/38.48	72.77/57.20	71.84/56.05	79.17/65.52	82.05/69.57	88.68/79.66	86.30/75.90	88.77/79.81	78.14	79.73	65.27
RS3Mamba [47]	49.65/33.02	71.27/55.36	70.03/53.89	75.90/61.16	82.47/70.18	83.34/71.44	86.93/76.88	87.15/77.23	75.84	78.55	62.39
PyramidMamba [30]	60.05/42.91	71.42/55.55	69.86/53.68	75.73/60.94	82.54/70.27	85.82/75.17	86.87/76.79	86.81/76.70	77.38	78.65	64.00
PPMamba [29]	60.09/42.95	73.63/58.26	71.80/56.01	78.30/64.34	83.04/70.99	86.16/75.68	87.55/77.85	88.16/78.83	78.59	79.95	65.61
AFENet [60]	64.08/47.15	72.80/57.24	71.74/55.93	78.09/64.06	82.96/70.89	88.27/79.00	87.01/77.00	87.83/78.30	79.10	79.90	66.19
Ours	58.93/41.77	74.41/59.24	73.09/57.59	80.32/67.11	83.37/71.48	88.21/78.91	87.78/78.22	88.61/79.56	79.34	80.74	66.74

Best results in bold, second best underlined.

, SPFMamba ranks first among all compared methods, reaching 79.34% mF1, 80.74% OA, and 66.74% mIoU. Compared with the respective second-best methods for individual metrics, SPFMamba improves mF1, OA, and mIoU by 0.24%, 0.79%, and 0.55%, respectively. SPFMamba achieves the highest F1 scores and IoU values in most classes. In particular, for the structurally complex road class, SPFMamba obtains F1 and IoU values of 80.32% and 67.11%, improving upon the runner-up SFFNet by 1.15% and 1.59% for these two metrics.

As illustrated in Figure 7, in the first and third scenes, roads among densely arranged buildings exhibit pronounced intraclass heterogeneity due to variations in illumination and shadow, surface materials, and road geometry. In the second scene, the water surface is partially obscured by aquatic vegetation, which often confounds models relying primarily on local information. Benefiting from the effective global-context modeling of SPGLM, SPFMamba mitigates local visual ambiguity, preserves the topological continuity of roads at complex intersections, and accurately delineates water bodies by exploiting water-body morphology and contextual cues from adjacent land-cover patterns. The fourth scene presents another challenging case, where water and bareland exhibit high spectral and textural similarity, causing many competing methods to produce fragmented predictions or imprecise boundary localization. In contrast, by effectively integrating multilevel semantic information with fine-grained detail features, SPFMamba achieves more complete segmentation with more continuous object contours and fewer fragmented regions. Overall, SPFMamba provides improved object integrity and more precise boundary delineation in ambiguous regions of large-scale, complex scenes.

6. Discussion

6.1. Model Complexity Analysis

The parameter counts and FLOPs of different methods are presented in

Table 10. Computational complexity analysis on the ISPRS Vaihingen dataset.

Type	Method	Params (M) ↓	FLOPs (G) ↓	mIoU (%) ↑
CNN-Based	ABCNet [58]	13.37	62.50	81.24
	EIGNet [59]	34.48	319.91	82.57
	AFENet [60]	20.24	102.36	83.31
Transformer-Based	DC-Swin [61]	66.95	277.20	81.12
	UNetFormer [43]	11.72	46.97	81.86
	CGGLNet [46]	36.88	608.33	83.76
	SFFNet [62]	34.18	206.70	83.51
Mamba-Based	RS3Mamba [47]	43.32	253.20	81.84
	PyramidMamba [30]	28.76	74.79	82.60
	PPMamba [29]	21.70	189.00	83.11
	Ours	17.85	85.18	84.26

Bold: best result; ↓: lower is better; ↑: higher is better.

. Most CNN-based methods contain fewer parameters than Transformer-based models, but their segmentation performance is generally lower, indicating that local convolutions alone may be inadequate for capturing non-local dependencies and scene-level contextual cues in HRSIs. In contrast, SPFMamba achieves better segmentation performance while maintaining a relatively compact model size and low computational cost. Compared with CGGLNet, SPFMamba uses only 17.85 M parameters and 85.18 G FLOPs, corresponding to reductions of 19.03 M parameters and 523.15 G FLOPs, while increasing mIoU from 83.76% to 84.26%. Compared with other Mamba-based methods, SPFMamba uses substantially fewer parameters and FLOPs, yet outperforms RS3Mamba and PPMamba in mIoU by 2.42% and 1.15%, respectively. This demonstrates the superior accuracy–efficiency balance of SPFMamba.

6.2. Heatmap Visualization of SPGLM

We adopt Gradient-weighted Class Activation Mapping (Grad-CAM) [63] to compare the activation patterns at the input and output of the SPGLM block, with the aim of analyzing the response regions of different target categories and investigating the role of SPGLM in global and local context modeling. Figure 8 compares the activation maps derived from the original features, the VSS block, and SPGLM. The original features show noticeable confusion and noise across the response regions of different categories. After the VSS block, the spatial integrity of the feature responses is improved, although some irrelevant activations still remain. By contrast, the feature responses after SPGLM are not only more concentrated within the target regions but also provide more complete coverage of the target objects. The improvement is more evident for buildings and trees of varying sizes, where SPGLM produces more consistent responses across scales than the VSS block. These results indicate that SPGLM establishes stronger contextual dependencies across spatial scales, thereby yielding more distinctive feature representations for different object categories in complex scenes.

7. Conclusions

In this work, we develop SPFMamba for HRSI semantic segmentation to address pronounced scale variations among objects and the loss of fine details. The proposed SPGLM enhances the discriminative representation of multiscale objects by jointly modeling global and local contextual information. The HFAFM leverages frequency-domain priors to strengthen high-frequency detail responses and improve the representation of small objects and boundary regions through shallow–deep feature interaction. SPFMamba achieved mIoU of 84.26%, 87.34%, and 66.74% on the ISPRS Vaihingen, ISPRS Potsdam, and OpenEarthMap datasets. Moreover, SPFMamba contains only 17.85 M parameters while maintaining competitive segmentation performance with reduced computational overhead. Future work will investigate multimodal fusion strategies, such as incorporating DSM or synthetic aperture radar (SAR) data, to improve robustness and discriminative capability in scenarios with shadow occlusion, complex structures, and spectral confusion.