Boundary-Aware Multi-Scale Feature Enhancement Based Few-Shot Hyperspectral Image Semantic Segmentation

Highlights

What are the main findings?

• A few-shot hyperspectral image semantic segmentation framework based on edge-aware multi-scale enhancement, integrating orthogonal-direction convolutions, slender feature enhancement, cross-stage feature fusion, and deep supervision mechanisms was proposed.
• The framework effectively addresses challenges of hyperspectral image segmentation under few-shot conditions and the poor accuracy of slender geographic objects, significantly improving feature utilization and model generalization ability.

What are the implications of the main findings?

• This study provides a new solution for hyperspectral image analysis under few-shot conditions.

Abstract

To address the issues of model overfitting under scarce samples and poor segmentation performance on slender objects in the task of semantic segmentation of remote sensing hyperspectral images, this paper proposes a hyperspectral image semantic segmentation framework that integrates edge awareness and multi-scale feature enhancement under extremely few-shot conditions. This architecture effectively integrates orthogonal-direction convolutions, elongated feature enhancement, multi-scale feature fusion, and deep supervision mechanisms, solving challenges such as difficulty in extracting features of slender objects, model overfitting under few-sample conditions, and insufficient generalization ability. The experimental results on multiple public datasets show that the proposed algorithm achieves excellent segmentation performance with just one small-sized sample per labeled category, surpassing existing popular algorithms and thereby confirming the algorithm’s effectiveness and superiority. On the PaviaU dataset, the overall accuracy (OA) and mean intersection over union (mIoU) improved by approximately 9.7% and 15.5% compared to the second-best model; especially for the segmentation of the key elongated feature ‘road’, the intersection over union reached 94.75%, highlighting the effectiveness of the proposed mechanism. This paper provides a novel and efficient solution for fine interpretation of hyperspectral images under few-sample conditions.

1. Introduction

The hyperspectral imaging technology captures fine spectral and spatial information of surface materials through continuous narrow-band images, realizes accurate identification of surface properties, and is widely used in environmental monitoring, precision agriculture and geological exploration. Hyperspectral image processing involves a large number of complex tasks [1]. With the rapid development in high-resolution remote sensing satellite technology, semantic segmentation of hyperspectral images has become one of the core tasks of intelligent remote sensing interpretation [2].

The hyperspectral image segmentation methods have evolved from manual feature design to deep learning. Early methods were mainly based on spectral features such as spectral angle mappers (SAMs), hybrid models, Support Vector Machines (SVMs), random forests, and Spectral Difference Matching Dimensionality Reduction Classification [3]. However, due to the challenges of hyperspectral images, such as high-dimensional characteristics, strong spatial–spectral coupling, and scarcity of labeled samples, traditional segmentation methods often have difficulty simultaneously ensuring feature discriminability and model efficiency, and struggle to handle high-dimensional data and complex spatial structures [4]. With the development in deep learning, convolutional neural networks (CNNs) and fully convolutional networks (FCNs) are widely used in hyperspectral image segmentation, and spatial–spectral features are automatically learned and extracted through convolutional neural networks [5,6]. For example, the Spectral–Spatial Residual Network (SSRN) [7] proposed by Zhong et al. extracts multimodal features through 3D convolution, which significantly improves the segmentation accuracy. For example, Roy, S. K. et al. proposed a 3D convolution-based hybrid spectral network (HybridSN) [8] to extract spectral and spatial features simultaneously through three-dimensional convolution, which significantly improved the classification accuracy. Li et al. designed a dual attention mechanism (DA) [9] to adaptively weight feature responses in channel and spatial dimensions. Zhao et al. propose spectral spatial angle mapping enhancement to reduce the dimensionality and the distinction between the target and background [10]. However, these methods have high complexity in the calculation of various types of convolutions when processing high-dimensional features of hyperspectral images, and require a large amount of data annotation. The acquisition of high-quality pixel-level annotated samples is costly, resulting in a severe shortage of training samples, and there is a significant disparity in quantity between background and target classes in semantic segmentation annotations. Traditional deep learning methods are prone to overfitting under small sample conditions. In addition, hyperspectral images have lower resolution compared to high-resolution panchromatic images. Therefore, it is difficult to extract the geometric features of elongated objects with severe pixel mixing (such as roads and rivers), and the segmentation performance of popular deep learning models is relatively poor. For example, U-Net and its variants, with their unique symmetric encoder–decoder structure and jump connections, have achieved remarkable results in medical image segmentation and remote sensing image analysis [11,12]. However, there are still obvious limitations to the direct application of standard U-Net to hyperspectral image segmentation. The standard convolutional operation is not sensitive to the boundary of the ground object, resulting in the blurring of the boundary of the segmentation result. It is difficult for a single receptive field to effectively capture scale changes from small to large features. The rich spectral and spatial texture information in the hyperspectral data is not maximized.

The attention mechanism has become a key technology to improve the performance of deep learning models by simulating the selective attention of the human visual system [13]. In remote sensing image segmentation, the attention module can help the model focus on key feature areas and bands. For example, CBAM (Convolutional Block Attention Module) [14] effectively improves feature representation ability by applying channel and spatial attention sequentially. In addition, multi-scale feature fusion is crucial for dealing with scale changes in features. SFFGL [15], which is a global spatialwise FCN, introduces a spatial attention mechanism (PAM) between the encoder and decoder, and effectively exploits the global spatial information from the whole HSIs. The pyramid pooling module (PSPNet) [16] and Hollow Space Pyramid Pooling (ASPP) [17] enhance the model’s ability to identify multi-scale targets by capturing different ranges of contextual information. Recent studies such as MSAFNet [18] have verified the effectiveness of multi-scale and attention integration by optimizing water extraction through continuous attention fusion modules. The attention multi-scale network (AMSNet) [19] also demonstrates the enhancement effect of multi-scale fusion attention in remote sensing image segmentation. The SSA-SiamNet [20] method can emphasize informative channels and locations and suppress less informative ones to refine the spectral–spatial features adaptively. Channel-Spatial Progressive Attention (CSPA) [21] is proposed to enhance spatial–spectral representations, which facilitates more effective generation of memory prompts and enhances the utilization and retrieval of historical information.

Few-shot learning solves the problem of insufficient annotation data through meta-learning [22], graph convolutional network [23], data augmentation [24] and transfer learning [25]. Prototype networks [26] learn a prototype (feature mean) for each class and classify by measuring the distance between a query sample and the prototypes. Their advantages are that the network is simple and efficient, with low computational complexity, but they are sensitive to outlier samples, the prototypes are easily affected by noise, and they ignore spatial context information, making insufficient use of the spatial structure in hyperspectral images. Graph convolutional networks represent images as graph structures and use node relationships to model long-range dependencies. Dual Semantic Graph Convolutional Network (DSGCN) [23] combines a superpixel-level graph convolutional network and CNN branches to extract spatial semantics and spectral change features at the same time, effectively utilizing the spatial topology of hyperspectral images, but with high computational complexity, large memory consumption, and sensitivity to the initial parameters of graph construction.

The previous models have poor segmentation performance for slender remote sensing features such as roads and rivers, and the scale changes in small to large features are not well characterized. This paper aims to propose a new architecture for semantic segmentation of low-shot hyperspectral images, construct a collaborative mechanism of multiple new modules, and provide an effective solution for improving the segmentation performance of slender features. In addition, the model avoids overfitting under limited data, realizes feature reuse, maximizes the feature utilization rate of limited samples, and enhances the generalization ability of the model through orthogonal-direction and multi-scale convolutions.

The contributions of this paper are as follows:

(1) Construct an edge-aware feature learning mechanism for hyperspectral images, and integrate an orthogonal-direction convolutional design to capture slender features.

(2) Construct a multi-scale pyramid feature extraction and a multi-stage encoder–decoder edge multi-scale feature enhancement and fusion mechanism.

(3) A multi-task collaborative learning mechanism is proposed, which optimizes the multi-task joint learning of segmentation, edge detection, and slender classification, and efficiently completes the fine ground boundary segmentation task in hyperspectral images.

(4) Combining the in-depth supervision mechanism with the semantic segmentation module, multi-level supervision improves training stability and effectively utilizes limited samples.

2. Methods

2.1. Overall Architecture

This paper proposes a novel Edge-aware Multi-Modal Feature Enhancement Network (EMMNet) for semantic segmentation of hyperspectral images under few-shot conditions. The overall network architecture is shown in Figure 1, adopting an encoder–bottleneck–decoder structure, with each encoder and decoder layer deeply integrating edge-aware and multi-modal feature enhancement mechanisms. The outputs include the primary segmentation result, boundary prediction, and elongated object features, which are then jointly optimized through multi-task learning. A deep supervision mechanism ensures the effectiveness and stability of training. The input consists of hyperspectral image (HSI) cube patches.

Figure 1. Overall architecture of EMMNet; the numbers indicate channel × height × width; MSBP represents multi-scale boundary pyramid module; EMRB represents Enhanced Multi-Branch Residual Module.

The overall forward propagation path of the framework is:

$$Y={\mathcal{F}}_{\mathrm{EMMNet}}(X)={\mathcal{F}}_{\mathrm{classifier}} \circ \mathcal{D}\circ \mathcal{B}\circ \mathcal{E}(X)$$

(1)

where $X\in {ℝ}^{p\times p\times D}$ is the hyperspectral image patch, p is the size of the patch, D is the number of channels, $\mathcal{E}$ are the encoder layers, $\mathcal{B}$ are the bottleneck layers, $\mathcal{D}$ are the decoder layers, and $\mathcal{F}$ is the boundary-aware classifier. $Y\in {ℝ}^{H\times W\times N_{class}}$ is the output segmentation map. The initial boundary-aware layer in the network is a multi-branch residual module, which is described in the following sections.

2.2. Dataset

The entire hyperspectral image is $I_{\mathrm{original}}\in {ℝ}^{H\times W\times B}$, with H, W, and B representing the height, width, and number of spectral bands of the entire image, respectively. Principal Component Analysis (PCA) is used to reduce the high-dimensional spectral features to D dimensions, retaining the main spectral information:

$$I_{\mathrm{pca}}=\mathrm{PCA}(I_{\mathrm{original}} ), I_{\mathrm{std}} ={\displaystyle \frac{I-\mu }{\sigma }}$$

(2)

where μ and σ are the mean and standard deviation, respectively.

From the normalized hyperspectral image $I_{\mathrm{std}}\in {ℝ}^{H\times W\times D}$, extract image patches X. At the edges of the image, if the size of the remaining part is smaller than the patch size, zero-pad it and label corresponding pixels as the background class. To balance the number of training samples for each class, class-balanced sampling is used, randomly selecting patches of each class from the entire image.

$$\mathcal{S}={\displaystyle \underset{c=1}{\overset{C}{\cup }}}\{(P_c,L_c)\mid c\in {\mathcal{R}}_c \}$$

(3)

where C is the number of classes, ${\mathcal{R}}_c$ is the randomly sampled index set of class c, $P_c$ is all samples of class c, and $L_c$ is all sample labels of class c. Each class samples $N_c$ samples.

2.3. Boundary-Aware Convolution Module

The Boundary-Aware Convolution (BAC) module serves as a basic module, integrated into multi-branch residual modules, and is applied to initial convolution layers, encoding layers, decoding layers, bottleneck layers, and other modules. It is specifically designed to enhance the boundary segmentation performance of features at various scales. Its structure is shown in Figure 2:

Figure 2. Edge-aware convolution module.

The edge is detected by

$$B=\sigma (Con{v}_{1\times 1} (\mathrm{ReLU}(\mathrm{BN}(Con{v}_{3\times 3} (X)))))$$

(4)

The input features are enhanced as

$$X_{boundary}=\mathrm{Concat}(X,B)$$

(5)

$$Y=\mathrm{ReLU}(\mathrm{BN}({\mathrm{Conv}}_{3\times 3} (X_{boundary} )))$$

(6)

where $\sigma$ is the sigmoid function and BN is the batch normalization operation. $Con{v}_{k\times k}$ is the convolution operation.

2.4. Enhanced Multi-Branch Residual Block (EMRB)

The multi-branch residual module focuses on extracting basic edge and slender object features. Different branches use convolution kernels of different shapes to extract structural information in horizontal and vertical directions, and integrate an edge-aware module to further extract boundary information. The structure is shown in Figure 3:

Figure 3. Enhanced multi-branch residual module.

Let the input feature map be $X\in {ℝ}^{B\times D \times H\times W}$, and the branch feature extraction is expressed as

$$\begin{array}{l}Fhor=\sigma (BN({\mathcal{W}}_{hor}\cdot X)), {\mathcal{W}}_{hor}\in {ℝ}^{C_{branch}\times D\times 1\times 3}\\ Fver=\sigma (BN({\mathcal{W}}_{ver} \cdot X)), {\mathcal{W}}_{ver} \in {ℝ}^{C_{branch}\times D\times 3\times 1}\end{array}$$

(7)

The features of the two branches are fused into:

$$F_{concat}=Concat(F_{hor},F_{ver})\in {ℝ}^{B\times 2C_{branch}\times H\times W}$$

(8)

$$F_{fused}=\sigma (BN({\mathcal{W}}_{fusion} \cdot F_{concat} )),{\mathcal{W}}_{fusion} \in {ℝ}^{C_{out }\times 2C_{branch}\times 1\times 1}$$

(9)

Perform a residual connection between the fused features and the original input:

$$F_{residual}=F_{fused} +{\mathcal{W}}_{shortcut} \cdot X$$

(10)

where ${\mathcal{W}}_{shortcut}$ is an identity mapping or a 1 × 1 convolution.

Finally, it is enhanced by the boundary awareness layer as

$$Y,B=\mathrm{BoundaryAwareConv}(F_{residual} )$$

(11)

2.5. Encoding Layer

The encoder uses a multi-level (for example, four-level) downsampling structure, sequentially $\mathrm{enc}1\to \mathrm{enc}2\to \mathrm{enc}3\to \mathrm{enc}4$, with each level containing multiple-branch residual modules, EMRB modules.

$\mathrm{enc}1$ can be expressed as

$$F_{\mathrm{enc}1}={\mathcal{F}}_{\mathrm{EMRB}} \circ MaxPool2d(F_{\mathrm{initial}} )$$

(12)

where ${\mathcal{F}}_{\mathrm{EMRB}}$ is a multi-branch residual EMRB module, $F_{\mathrm{initial}} =\sigma (BN(W_{init} \ast X_{input} ))$, $\circ$ denotes function composition.

$\mathrm{enc}2$ can be expressed as

$$F_{enc2}={\mathcal{F}}_{EMRB }\circ MaxPool2d(F_{enc1 })$$

(13)

The remaining stage encoder is represented as

$$F_{enc}^l={\mathcal{F}}_{EMRB}^l\circ \mathcal{P}down(F_{enc}^{l-1} )$$

(14)

where l is the layer index, and $\mathcal{P}down$ is the downsampling operation.

2.6. Bottleneck Layer

The bottleneck layer is responsible for capturing global contextual information while preserving edge details.

$$F_{bottleneck}={\mathcal{F}}_{MSBP} \circ {\mathcal{F}}_{EMRB}^{bottleneck} (F_{enc4} )$$

(15)

where ${\mathcal{F}}_{MSBP}$ is a multi-scale boundary pyramid module.

The multi-scale boundary pyramid module extracts elongated features at different scales, automatically learns the contribution weights of different scales, and enhances semantic information while preserving spatial details, as shown in Figure 4.

Figure 4. Multi-scale boundary pyramid module.

First, perform multi-scale feature extraction. For each scale $s\in \left\{1,2,4\right\}$:

$$F_s =\mathrm{Upsample}(\mathrm{ReLU}(\mathrm{BN}(Con{v}_{1\times 1 }(AdaptiveAvgPoo{l}_s (X)))))$$

(16)

The boundary is weighted by

$$F_s^{\prime }=F_s \odot (1+\mathrm{Upsample}(B))$$

(17)

where B is the boundary map output by the EMRB module.

Features are reconstructed as

$$Y=\mathrm{ReLU}(\mathrm{BN}(Con{v}_{3\times 3} (Concat(F_1^{\prime },F_2^{\prime },F_4^{\prime }))))+X$$

(18)

2.7. Decoding Layer

2.7.1. Mathematical Modeling of the Decoding Layer

The number of decoder layers corresponds to the number of encoder layers. For example, with 4 stages, they follow in order $dec4\to dec3\to dec2\to dec1$, and the process is shown in Figure 5.

Figure 5. Decoder framework.

The mathematical modeling of each module corresponding to Figure 5 is:

$$F_{up4}={\mathcal{U}}_{transpose} (F_{bottleneck} )$$

(19)

$$F_{fusion4}={\mathcal{F}}_{cross4}(F_{enc4},F_{up4 })$$

(20)

$$F_{dec4 }={\mathcal{F}}_{EMRB}^{dec4} (Concat[F_{fusion4},{\mathcal{U}}_{bilinear} (F_{enc4 }^{adj})])$$

(21)

The decoding layers for the other stages are

$$F_{dec}^l={\mathcal{F}}_{EMRB}^{dec,l} \circ Concat[{\mathcal{F}}_{cross}^l (F_{enc}^l,U_{transpose} (F_{dec}^{l+1} )),{\mathcal{U}}_{bilinear} (F_{enc}^{l,adj} )]$$

(22)

2.7.2. Boundary-Aware Cross-Stage Fusion Module

The output of each decoder layer is fused with the corresponding layer’s encoder features through the Boundary-aware Cross-Stage Fusion Module and then output to the next decoder layer. The structure of the Boundary-aware Cross-Stage Fusion is shown in Figure 6:

Figure 6. Boundary-aware cross-stage fusion module.

Features are aligned as

$$E_{aligned}=\mathrm{ReLU}(\mathrm{BN}(Con{v}_{1\times 1 }(E)))$$

(23)

The boundary is weighted as

$$E_{weighted}=E_{aligned} \odot (1+B)$$

(24)

The features are fused into

$$Y=\mathrm{ReLU}(\mathrm{BN}(Con{v}_{3\times 3} (Concat(E_{weighted} ,D))))$$

(25)

2.8. Boundary-Aware Classifier

The output of the decoder serves as the input to the boundary-aware classifier. It is specifically optimized for elongated objects during the training phase and uses a dual-path classification architecture, as shown in Figure 7, to address differences in segmentation performance under class imbalance.

Figure 7. The boundary-aware classifier.

The boundary features are extracted as

$$F_{boundary}=\mathrm{ReLU}(\mathrm{BN}(Con{v}_{3\times 3} (X)))$$

(26)

The boundary map is enhanced to

$$F_{map}=\mathrm{ReLU}(\mathrm{BN}(Con{v}_{3\times 3} (B)))$$

(27)

$$F_{boundary}=F_{boundary}+F_{map}$$

(28)

The output of the main classifier is

$$Y_{main}=Con{v}_{1\times 1} (\mathrm{ReLU}(\mathrm{BN}(Con{v}_{3\times 3} (Concat(X,F_{boundary} )))))$$

(29)

The output of the boundary is predicted as

$$Y_{boundary }=\sigma (Con{v}_{1\times 1} (F_{boundary} ))$$

(30)

2.9. Slender Object Classifier

The slender object classifier is optimized only for specific classes of slender objects, with ground truth constructed and loss calculated separately for these classes. The input to the slender object classifier is the slender object features output by the boundary-aware classifier.

$$Y_{elong}=Con{v}_{1\times 1} (F_{boundary})$$

(31)

The input to the slender object classifier is the boundary_features extracted by the preceding boundary-aware classifier module, rather than the output of a separate, dedicated slender geometric feature extractor. In high-resolution remote sensing imagery, slender objects (such as roads, field boundaries, and narrow rivers) are predominantly defined by their strong, elongated boundaries. Their geometric “slenderness” is inherently a spatial manifestation of boundary characteristics over a continuous region. Therefore, features that are highly responsive to boundaries (i.e., boundary_features) naturally contain the most discriminative information for identifying slender objects.

2.10. Deep Supervision Layer

The deep supervision layer addresses the gradient vanishing problem by providing multi-scale supervision to each encoder layer.

The outputs of supervision at each stage are

$$P_{ds}^l={\mathcal{U}}_{bilinear} ({\mathcal{W}}_{ds}^l \cdot F_{enc}^l ,{\mathrm{size}}_{input} )$$

(32)

where $l\in \{1,2,3,4\}$ represents the supervision stage.

The weighted fusion of the outputs at each stage is obtained:

$$P_{ds}^{fused} ={\displaystyle \sum _{l=1}^4{w}_l \cdot P_{ds}^l}$$

(33)

The final output is enhanced to

$$P_{final}^{enhanced}=P_{final}+\alpha \cdot P_{ds}^{fused}$$

(34)

where $\alpha$ is the fusion coefficient, taking an empirical value of 0.2.

2.11. Loss Function

The multi-task loss function is

$${\mathcal{L}}_{total} ={\mathcal{L}}_{main}+{\lambda }_{ds}{\mathcal{L}}_{ds}+{\lambda }_{elong}{\mathcal{L}}_{elong }+{\lambda }_{reg}{\mathcal{L}}_{reg}$$

(35)

where ${\mathcal{L}}_{main} =\mathrm{WCE}(P_{main}, Y_{gt})$ is the main classification loss, ${\mathcal{L}}_{elong} =\mathrm{WCE}(P_{elong}, Y_{gt} [\mathrm{elongated}])$ is the slender category loss, ${\mathcal{L}}_{ds} =\mathrm{WCE}(P_{ds}, Y_{gt})$ is the deep supervision loss, and ${\mathcal{L}}_{reg} =\parallel W{\parallel }_2^2 +\parallel \lambda -0.5 {\parallel }_2^2$ is the regularization loss. $\mathrm{WCE}$ is the weighted cross-entropy loss function.

The training category weights are calculated based on the distribution of each category:

$${\omega }_c ={\displaystyle \frac{media{n}_{c\in C} (fre{q}_{c })}{fre{q}_{c }+\mathsf{ϵ}}}$$

(36)

The weights are calculated based on the class distribution of the entire image, and median filtering is used to avoid the impact of extreme class weights. $\mathsf{ϵ}$ is an extremely small value to prevent division-by-zero errors.

2.12. Model Inference Process

To handle large-sized hyperspectral images, a sliding window prediction strategy is adopted. Let the input image size be H × W, the sliding window size be S × S, and the stride be d. At the image boundaries, if the remaining area is smaller than the window size, zero padding is added at the end.

The model’s inference for a single patch is

$$\widehat{Y}=\mathcal{M}(X_{in})\in {ℝ}^{1\times N\times S\times S}$$

(37)

Probability normalization and prediction are

$$P_{prob}=soft\max (\widehat{Y})$$

(38)

The category is predicted as

$$P=\underset{k}{\arg \max }{P}_{prob}$$

(39)

The confidence level is calculated as

$$S=\underset{k}{\max }{P}_{prob}$$

(40)

For overlapping areas, use the results of the patches with higher confidence to avoid edge artifacts. The inference images use the same image preprocessing methods as in the training process.

3. Experiments and Results

3.1. Dataset and Parameter Settings

(1) The PaviaU Dataset: Acquired in 2003 over the city of Pavia, Italy, by Germany’s airborne Reflective Optics Spectrographic Imaging System (ROSIS-03). The scene size is 610 × 340 pixels, consisting of 103 spectral bands after removing bands affected by water vapor. This image is divided into nine classes: Asphalt, Meadows, Gravel, Trees, Painted Metal Sheets, Bare Soil, Bitumen, Self-Blocking Bricks, and Shadows. The pseudo-color image of this dataset and the ground truth maps for the nine classes are shown in Figure 8:

Figure 8. The pseudo-color image (R = band 50, G = band 30, and B = band 10) and the ground truth of the PaviaU dataset.

(2) The Salinas Dataset: Captured using the AVIRIS sensor with 224 bands in the Salinas Valley, California. This dataset has a spatial size of 512 × 217 pixels, a spatial resolution of 3.7 m, and 204 bands after removing water absorption bands. It contains 16 land cover classes, namely Broccoli_green_weeds_1, Broccoli_green_weeds_2, Fallow, Fallow_rough_plow, Fallow_smooth, Stubble, Celery, Grapes_untrained, Soil_vineyard_develop, Corn_senesced_green_weeds, Lettuce_romaine_4wk, Lettuce_romaine_5wk, Lettuce_romaine_6wk, Lettuce_romaine_7wk, Vineyard_untrained, and Vineyard_vertical_trellis. Notably, it features delicate vegetables such as spinach, onions, and green peppers—with small spectral differences, posing an extreme challenge for narrow-band fine classification algorithms. The pseudo-color image of this dataset and the ground truth maps for the 16 classes are shown in Figure 9:

Figure 9. The pseudo-color image (R = band 57, G = band 27, and B = band 17) and the ground truth of the Salinas dataset.

(3) The WHU-Hi-LongKou Dataset: Captured using the Headwall Nano-Hyperspec sensor mounted on a DJI Matrice 600 Pro UAV platform over Longkou Town, Hubei province, China. This dataset has a spatial size of 550 × 400 pixels, a spatial resolution of about 0.463 m, and contains 270 bands ranging from 400 to 1000 nm. It encompasses nine land cover classes, namely Corn, Cotton, Sesame, Broad-leaf soybean, Narrow-leaf soybean, Rice, Water, Roads and houses, and Mixed weed. The dataset features a simple agricultural scene dominated by six major crop species, providing a benchmark for crop classification studies with high spatial resolution. The pseudo-color image of this dataset and the ground truth maps for the nine classes are shown in Figure 10:

Figure 10. The pseudo-color image (R = band 112, G = band 67, and B = band 22) and the ground truth of the Salinas dataset.

During training, the AdamW optimizer is used with a cosine annealing schedule. In this paper, the patch size is set to [128, 128], with 1 patch per class. For example, the PaviaU dataset has a total of nine samples, and the Salinas dataset has a total of 16 samples. The batch size is 2. No data augmentation is applied in order to examine the model’s own feature representation ability and generalization capability.

This paper uses PCA to reduce the spectral dimensions to D, here taking D = 30, so that the spectral dimensions have sufficient features while reducing the computational complexity of the data. The experimental study was conducted on an NVIDIA GeForce RTX 5060 GPU, training the model for 80 epochs using the AdamW optimizer with an initial learning rate of 1 × 10⁻⁴1e−4 and a weight decay of 1 × 10⁻⁴. The training set was constructed by randomly sampling just one 128 × 128 patch per land cover class, while the validation set comprised two independently sampled patches per class.

3.2. Evaluation Metrics

After obtaining the patch’s prediction map, compare it with the true labels $Y\in {\mathbb{Z}}^{S\times S}$ to calculate metrics. Overall accuracy (OA) and mean intersection over union (mIoU) are usually used.

OA is defined as

$$OA={\displaystyle \frac{{\displaystyle {\sum }_{c}T{P}_c} }{N}}$$

(41)

where N is the total number of pixels in the patch or the entire image. $T{P}_c$ (true positive) is the number correctly identified for class c.

mIoU is defined as

$$mIoU={\displaystyle \frac{1}{C}} {\displaystyle {\sum }_c\frac{ T{P}_c}{T{P}_c+F{P}_c+F{N}_c}}$$

(42)

where $F{P}_c$ (False positive) refers to the number of pixels incorrectly predicted as belonging to this class, and $F{N}_c$ (false negative) refers to the number of pixels that truly belong to this class but are predicted as other classes.

The empirical value of the loss function weight coefficient is: ${\lambda }_{ds} =0.2, {\lambda }_{elong} =0.3, {\lambda }_{reg} =0.2$.

3.3. Comparison Methods

This paper selects currently popular semantic segmentation frameworks for comparison to validate the performance of the algorithm proposed in this paper, mainly including U-Net, A2FPN (Attention Aggregation Feature Pyramid Network), ABCNet (attentive bilateral contextual network), MAResUNet (multi-stage attention ResU-Net), MSNet (multispectral semantic segmentation network), PSPN (pyramid scene parsing network), and 3D convolution-based hybrid spectral network (HybridSN [8]). U-Net [27] is a classic convolutional neural network that uses a basic encoder–decoder architecture, specifically designed for image segmentation tasks. A2FPN [28] is a novel framework for high-resolution remote sensing image semantic segmentation, enhancing multi-scale feature learning through attention-guided feature aggregation. ABCNet [29] is a new attentive bilateral contextual network that includes a spatial path and a context path for efficient semantic segmentation of high-resolution remote sensing images. MAResUNet [30] is a multi-stage attention ResU-Net for fine-resolution remote sensing image semantic segmentation, which reconstructs the skip connections in the original U-Net. MSNet [31] is a novel deep neural network structure for multispectral feature scene semantic segmentation. In this framework, ResNet-50 is used for feature extraction during the encoding stage, and cascading upsampling is adopted in the decoding stage to restore feature map resolution, while feature pyramid structures are utilized to progressively integrate multi-scale image features and spectral features during upsampling, finally obtaining the semantic segmentation result. PSPN [32] introduces a pyramid pooling module and a pyramid scene parsing network, leveraging contextual aggregation from different regions to exploit the capability of global contextual information.

3.4. Results and Analysis

This section compares the performance of different algorithms through visual and quantitative results. Figure 11 shows the visual segmentation results on the PaviaU dataset. It can be seen that the algorithm proposed in this paper achieves more accurate segmentation on elongated objects such as roads (class 1) and Self-Blocking Bricks (class 8). The visual results indicate that our algorithm performs excellently in edge preservation and small object recognition, as the building contours and road boundaries are clearer, verifying the effectiveness of the edge-aware convolution and elongated target enhancement mechanisms.

Figure 11. Visualization of the segmentation results on the PaviaU dataset: (a) original image RGB composite; (b) ground truth label map; (c) U-Net; (d) A2FPN; (e) ABCNet; (f) MAResUNet; (g) MSNet; (h) PSPN; (i) HybridSN; (j) ours.

Figure 12 shows the visual segmentation results of the Salinas dataset. It can be seen that when handling regions with uniform types within blocks and few labeled samples, our algorithm still maintains good segmentation performance. A2FPN, ABCNet, MAResUNet, MSNet, PSPN, HybridSN and others are not fully trained with few samples, leading to poor generalization performance. The basic model UNet shows relatively severe mixing between the boundaries of the two classes.

Figure 12. Visualization of the segmentation results on the Salinas dataset.

Figure 13 shows the visual segmentation results of the WHU dataset. The results in Figure 13 are consistent with Figure 12.

Figure 13. Visualization of the segmentation results on the WHU dataset.

The quantitative performance of different methods on the PaviaU dataset is shown in Table 1. The bold numbers in the table indicate the results achieved by our algorithm. The experimental results indicate that our algorithm outperforms the comparison methods in all metrics, demonstrating that the edge-aware multi-scale enhancement mechanism effectively enhances inter-class discriminability.

Table 1. Performance comparison of different methods on the PaviaU dataset.

Method	OA	mIoU	Class1 (Road)	Params (Million)	FLOPs (G)
U-Net	0.7830	0.7400	0.5346	31.4	20.59
A2FPN	0.7581	0.6003	0.6234	12.16	5.23
ABCNet	0.7934	0.6799	0.7201	13.82	1.99
MAResUNet	0.8209	0.5898	0.7429	26.36	5.12
MSNet	0.8322	0.7473	0.6816	97.08	8.73
PSPN	0.7706	0.6404	0.5017	46.59	22.30
HybridSN	0.4559	0.4054	0.2322	1.74	3.21
Ours	0.9289	0.9031	0.9475	33.49	30.24

The quantitative performance of the different methods on the Salinas dataset is shown in Table 2. The bold numbers in the table indicate the results achieved by our algorithm. Consistent with the visualization results, this demonstrates that our algorithm maintains good generalization performance on multi-type datasets.

Table 2. Performance comparison of the different methods on the Salinas dataset.

Method	OA	mIoU
U-Net	0.9342	0.9160
A2FPN	0.8111	0.7581
ABCNet	0.8519	0.6096
MAResUNet	0.5678	0.6700
MSNet	0.6368	0.3763
PSPN	0.8485	0.6601
HybridSN	0.5954	0.3462
Ours	0.9987	0.9989

The quantitative performance of the different methods on the WHU dataset is shown in Table 3. The bold numbers in the table indicate the results achieved by our algorithm. The performance metrics of this dataset are similar to those of the Salinas dataset.

Table 3. Performance comparison of different methods on the WHU dataset.

Method	OA	mIoU
U-Net	0.9433	0.8747
A2FPN	0.9126	0.7527
ABCNet	0.8225	0.4468
MAResUNet	0.6412	0.3575
MSNet	0.9004	0.7352
PSPN	0.8216	0.6328
HybridSN	0.5914	0.3034
Ours	0.9862	0.9776

3.5. Ablation Experiment

To verify the effectiveness of each module, we designed ablation experiments, starting from the basic U-Net and gradually adding modules such as multi-branch residual modules, multi-scale boundary pyramids, Boundary-aware Cross-Stage Fusion, deep supervision, and the slender object classifier to demonstrate segmentation performance. The results are shown in Table 4. The bold numbers in the table indicate the results achieved by our algorithm.

Table 4. Ablation study analysis (PaviaU dataset).

Configuration	OA	mIoU
Baseline U-Net	0.7830	0.7400
+ Multi-branch Residual Module	0.8532	0.8205
+ Multi-scale Boundary Pyramid	0.8645	0.8377
+ Boundary-aware Cross-Stage Fusion	0.8866	0.8683
+ Deep Supervision	0.9066	0.8883
+ Slender Classifier	0.9130	0.8911
Full Model-Without Boundary B	0.8426	0.8075
Full Model	0.9289	0.9031

The ablation experiments show that each component brings performance improvements, with the multi-branch residual module and the Boundary-aware Cross-Stage Fusion contributing the most, bringing mIoU increases of 10.88% and 3.65%, respectively. After removing boundary guidance, the overall class performance also declines. This demonstrates that boundary guidance is not a redundant design, but a key to solving critical challenges.

Visualization of feature heatmaps of key modules is shown in Figure 14. Each module effectively enhances boundary features.

Figure 14. Visualization of feature heatmaps of key modules.

3.6. Parameter Sensitivity Experiment

This section conducts a parameter sensitivity analysis on the weighting coefficients in the multi-task loss function, as shown in Table 5, Table 6 and Table 7. The bold numbers in the tables indicate the highest values of OA. The results indicate that the slender object loss (λ_elong) is the most sensitive: when its value increases from 0.0 to 0.3, the OA improves significantly (by about 9%), which shows that this module is crucial for enhancing the recognition of slender objects (such as roads); however, beyond 0.3, the gains decrease or even slightly decline, indicating the need for balance.

Table 5. Loss function weight coefficient sensitivity analysis (PaviaU dataset, OA, ${\lambda }_{reg}=0.2$).

${\mathit{\lambda }}_{\mathit{d}\mathit{s}}\backslash {\mathit{\lambda }}_{\mathit{e}\mathit{l}\mathit{o}\mathit{n}\mathit{g}}$	0.0	0.1	0.2	0.3	0.4	0.5
0.0	0.7830	0.8122	0.8451	0.8322	0.8201	0.8055
0.1	0.8105	0.8722	0.9010	0.9156	0.9058	0.8912
0.2	0.8324	0.8933	0.9178	0.9289	0.9210	0.9104
0.3	0.8518	0.9055	0.9251	0.9207	0.9115	0.8999
0.4	0.8401	0.8987	0.9165	0.9101	0.9005	0.8883
0.5	0.8367	0.8902	0.9091	0.9033	0.8947	0.8820

Table 6. Loss function weight coefficient sensitivity analysis (PaviaU dataset, OA, λ_elong = 0.3).

${\mathit{\lambda }}_{\mathit{d}\mathit{s}}\backslash {\mathit{\lambda }}_{\mathit{e}\mathit{l}\mathit{o}\mathit{n}\mathit{g}}$	0.0	0.1	0.2	0.3	0.4	0.5
0.0	0.8322	0.8502	0.8322	0.8200	0.8066	0.7955
0.1	0.8864	0.9104	0.9010	0.8891	0.8760	0.8622
0.2	0.9104	0.9251	0.9289	0.9210	0.9098	0.8976
0.3	0.9051	0.9233	0.9251	0.9165	0.9057	0.8920
0.4	0.8987	0.9160	0.9165	0.9072	0.8960	0.8833
0.5	0.8902	0.9072	0.9033	0.8950	0.8844	0.8728

Table 7. Loss function weight coefficient sensitivity analysis (PaviaU dataset, OA, ${\lambda }_{ds}=0.2$).

${\mathit{\lambda }}_{\mathit{d}\mathit{s}}\backslash {\mathit{\lambda }}_{\mathit{e}\mathit{l}\mathit{o}\mathit{n}\mathit{g}}$	0.0	0.1	0.2	0.3	0.4	0.5
0.0	0.8324	0.8455	0.8324	0.8205	0.8077	0.795
0.1	0.8722	0.9010	0.8933	0.8822	0.8701	0.8578
0.2	0.9010	0.9251	0.9178	0.9108	0.8993	0.8865
0.3	0.9156	0.9289	0.9289	0.9210	0.9098	0.8976
0.4	0.9058	0.9210	0.9212	0.9149	0.9044	0.8910
0.5	0.8912	0.9104	0.9104	0.9033	0.8947	0.8820

The deep supervision loss (${\lambda }_{ds}$) has a clear contribution: increasing ${\lambda }_{ds}$ (from 0.0 to 0.2) can consistently improve performance, especially under few-sample conditions, verifying that the depth supervision mechanism can effectively utilize limited samples and enhance gradient flow.

The regularization loss (${\lambda }_{reg}$) provides robustness: within a moderate range (0.1–0.3), model performance remains high and changes gradually, suggesting that appropriate regularization effectively prevents overfitting under very few samples.

This paper also compares the impact of data augmentation on accuracy and designs the following augmentation methods:

(1) Spatial augmentation: random horizontal/vertical flipping and random rotation.

(2) Spectral augmentation: adding Gaussian noise and random band mixing.

(3) Random erasing: simulating occlusion to enhance model robustness.

Using the above methods, the number of training samples is increased to 10 per class.

The results of the data augmentation experiments are OA = 0.9620, mIoU = 0.9475: data augmentation significantly improves model performance. Compared with the baseline method, OA increased by 3.5% and mIoU increased by 4.9% after augmentation.

4. Discussion

4.1. Analysis of Innovations

The innovations of the integrated framework constructed in this paper include:

(1) Integration of an edge-aware module, cross-modal enhancement, and a deep supervision effective image segmentation framework;

(2) Slender feature enhancement mechanism: Combining attention mechanisms to adaptively strengthen linear features of geographic objects, improving the recognition accuracy of targets such as roads and rivers;

(3) Through sample-balanced sampling and a dual-path fine classifier, the model’s generalization ability under conditions of scarce samples is significantly improved.

4.2. Limitations and Future Work

The multi-branch structure and deep supervision mechanism increase the model’s number of parameters and computational overhead, and future research could explore lightweight designs. In the future, the algorithm’s effectiveness will be validated on more hyperspectral datasets and in practical scenarios.

5. Conclusions

This paper proposes a few-shot hyperspectral image semantic segmentation framework based on edge-aware multi-scale enhancement, integrating orthogonal-direction convolutions, slender feature enhancement, cross-stage feature fusion, and deep supervision mechanisms. It effectively addresses challenges of hyperspectral image segmentation under few-shot conditions and the poor accuracy of slender geographic objects, significantly improving feature utilization and model generalization ability. The experimental results show that the segmentation accuracy and intersection-over-union metrics on public datasets are significantly better than popular methods, especially for slender object categories. This study provides a new solution for hyperspectral image analysis under few-shot conditions.