Adaptive Spectral Correlation Learning Neural Network for Hyperspectral Image Classification

Abstract

Hyperspectral imagery (HSI), with its rich spectral information across continuous wavelength bands, has become indispensable for fine-grained land cover classification in remote sensing applications. Although some existing deep neural networks have exploited the rich spectral information contained in HSIs for land cover classification by designing some adaptive learning modules, these modules were usually designed as additional submodules rather than basic structural units for building backbones, and they failed to adaptively model the spectral correlations between adjacent spectral bands and nonadjacent bands from a local and global perspective. To address these issues, a new adaptive spectral-correlation learning neural network (ASLNN) is proposed for HSI classification. Taking advantage of the group convolutional and ConvLSTM3D layers, a new adaptive spectral correlation learning block (ASBlock) is designed as a basic network unit to construct the backbone of a spatial–spectral feature extraction model for learning the spectral information, extracting the spectral-enhanced deep spatial–spectral features. Then, a 3D Gabor filter is utilized to extract heterogeneous spatial–spectral features, and a simple but effective gated asymmetric fusion block (GAFBlock) is further built to align and integrate these two heterogeneous features, thereby achieving competitive classification performance for HSIs. Experimental results from four common hyperspectral data sets validate the effectiveness of the proposed method. Specifically, when 10, 10, 10 and 25 samples from each class are selected for training, ASLNN achieves the highest overall accuracy (OA) of 81.12%, 85.88%, 80.62%, and 97.97% on the four data sets, outperforming other methods with increases of more than 1.70%, 3.21%, 3.78%, and 2.70% in OA, respectively.

1. Introduction

Hyperspectral images (HSIs) [1] have been used in many applications, such as military reconnaissance [2], precision agriculture [3], wetland dynamic monitoring [4], and so on [5,6,7]. Particularly, among HSI analysis and processing technologies, classification has become a very important and challenging information acquisition technology.

The filtering-based algorithm is a more useful way to extract spatial–spectral features by manually designing filters and directly interacting with HSIs [8]. Common filter-based feature extraction algorithms comprise the 3D Gabor filter [8,9], morphological profiles [10], scattering wavelet transform [11], scale-invariant feature transform [12], and local binary patterns [13], which can provide low-level and interpretable spatial–spectral features for HSI classification. Then, a suitable classifier can be used to classify land covers with the extracted spatial–spectral features from HSIs, where typical methods include multinomial logistic regression [14], composite kernel support vector machine (SVM-CK) [15], and sparse representation [16]. However, these methods need to be designed manually by experienced researchers for specific tasks.

As deep neural network (DNN)-based classification methods present advantages such as strong self-learning ability and excellent model generalization, various DNNs have been applied in remote sensing field, including convolutional neural networks (CNNs) [17,18], graph convolutional networks (GCNs) [19,20], and recurrent neural networks (RNNs) [21], obtaining better performance than other classification methods. RNNs have been widely used to analyze and process sequential data because of their unique directional feedback structure, where long short-term memory (LSTM) [22] and convolutional LSTM [23] are two special RNNs designed to solve learning problems related with the interdependence of long distance sequential data. According to [24], the convolutional LSTM has been further renamed as ConvLSTM2D for convenience. Thanks to the ability to model the long-term correlation of sequential data, there are two common ways to use LSTM or ConvLSTM2D for the spatial–spectral extraction and classification of HSIs. On the one hand, LSTM or ConvLSTM2D can be utilized alone as a basic unit to build the RNNs for HSI classification, such as SSLSTMs [25], bidirectional-ConvLSTM2D [26], the spatial–spectral ConvLSTM2D neural network (SSCL2DNN) [24], and the two-branch multidirectional spatial–spectral LSTM attention network [27]. On the other hand, LSTM or ConvLSTM2D can be integrated with CNNs or attention mechanisms to build a hybrid network, such as capsule network+ConvLSTM2D [28], tensor attention-driven ConvLSTM2D neural network [29], and nonlocal-dependent learning fully convolutional network [30]. To better preserve the intrinsic structure information of HSIs, Hu et al. [24] further proposed to extend the ConvLSTM2D to its 3D version (namely ConvLSTM3D) to directly extract the spatial–spectral features for HSI classification. Inspired by this, some ConvLSTM3D-based models have been gradually developed for HSI classification, such as SSCL3DNN [24], ConvLSTM3D+3D CNN (SSCRN) [31], ConvLSTM3D+attention mechanism (Dual-Channel A³CLNN) [32], and the regularized spatial–spectral global learning (RSSGL) framework [33]. However, the network structure designs and working mechanisms of these methods lack accurate interpretation, and there is a lack of obvious correlation between the extracted deep semantic features and the classification results of HSIs. To improve the interpretability of deep semantic features and make up for their lack of detailed information, some researchers have tried to integrate the traditional features into the training process of DNNs, where the complementary information and interaction of two different modality features can dynamically optimize the training direction of the whole networks for various applications, such as action recognition, object detection, lane detection, locomotion mode recognition, HSI change detection, and HSI classification [6,34], reducing the overfitting problems of the whole DNNs and improving their performance to a certain extent.

In particular, the special imaging mechanism with which HSIs are collected introduces rich and highly correlated spectral information, motivating the design of different spectral information learning submodules to be plugged into DNNs for spatial–spectral feature extraction. Particularly, attention mechanism is one of the most commonly used and effective ways. Li et al. [32] designed a spectral attention module by using the ConvLSTM2D as a basic unit to assist the dual-channel A³CLNN with adaptive learning of long-term spectral correlation, improving the classification accuracy of HSIs. An improved complex-valued deformable ConvLSTM2D was extended, improving the abilities of learning scale information and spectral correlations of the whole model [6]. Sun et al. [35] built a large kernel spectral–spatial attention network to learn the long-range 3D properties of HSIs. A dense spectral convolution module was designed for exploring the intrinsic similarity between spectral bands for HSI super-resolution [36]. In addition, transformer has gained widespread attention and success in the field of natural language processing due to its ability to handle global long-term dependencies in sequence data [37]. Subsequently, considering the sequence properties presented by the rich spectral information of HSIs, some researchers have attempted to introduce it into the task of HSI classification, achieving promising results. Xu et al. [38] proposed a double branch convolution–transformer network, where a convolution-spectral projection unit and a convolutional multihead self-attention network are applied for exploring the spectral correlation among spectral bands and local–global features for HSI classification. A center-masked transformer with a regularized center masked pretraining task was built to learn the dependencies between central land cover and its neighboring objects without labels during the pretraining process [39]. Shi et al. [40] developed a parallel dual-branch multiscale transformer, containing the spectral convolution, channel shrink soft split, and token-to-token modules for multiscale spatial–spectral feature extraction, which was followed by a pooled activation fusion module for feature fusion and the classification of HSIs. A spatial–spectral wavelet transformer was introduced, which unifies the downsampling with wavelet transforms for a lossless compression of features, preserving data integrity and improving the interaction between structural and shape information for HSI classification [41]. Aiming at the information loss of transformer during its propagation, a memory-augmented spectral–spatial transformer was constructed with a memory tokenizer and memory-augmented transformer encoder to effectively mix the spectral and spatial information for HSI classification. To fully eliminate the influence of multimodal heterogeneity, Hu et al. [42] innovatively viewed the global information as an intermediate agent to propose a new cross-memory quaternion transformer network, effectively improving the classification accuracy of land covers. Although the above spectral learning submodules of models can adaptively model the spectral correlations to obtain the spectral-enhanced spatial–spectral features for HSI classification, there are still some problems needed to further be solved. First, these modules usually only explored the long-range dependencies between different spectral bands, which, however, ignored the local correlations between the adjacent spectral bands, limiting the further improvements of classification performance of HSIs. Second, these modules are usually considered as the additional submodules to assist the feature extraction process of DNNs, which are rarely designed as the basic network units to build the new backbones for HSI classification.

Following our previous work in [6,24], to solve the above-mentioned problems, a novel adaptive spectral correlation learning neural network (ASLNN) is proposed for the spatial–spectral feature extraction and classification of HSIs. By considering the strong spectral correlations that exist between the adjacent spectral bands and between the nonadjacent spectral bands, a new adaptive adaptive spectral correlation learning block (ASBlock) is first designed by utilizing the group convolutional (GConv) layer and the ConvLSTM3D layer, jointly learning the short- and long-term spectral correlations. Then, by taking the designed ASBlock as a basic network unit, a full convolution-based spatial–spectral feature extraction network is constructed, adaptively modeling the unique spectral characteristics of different land covers and extracting enhanced deep spatial–spectral features to accurately discriminate between them. Furthermore, in order to improve the ability of deep semantic features to describe details and upgrade their interpretability, inspired by the conclusions in [6,8,43], the 3D Gabor filter is utilized as the heterogeneous feature extractor, and a simple but effective gated asymmetric fusion block (GAFBlock) is built to align and integrate these two modalities of spatial–spectral features, achieving competitive classification accuracy for HSIs. Experimental results on four common HSI data sets show the superiority of the proposed ASLNN model. The main contributions of this work can be summarized as follows.

(1) To adaptively learn the spectral information of HSIs, a new ASBlock is designed by integrating the GConv and ConvLSTM3D layers, which can be used to construct the backbones of other DNNs for joint learning of the short- and long-range correlations.

(2) Based on the designed ASBlock, a convolutional network can be constructed as the backbone for adaptively extracting the adaptive-spectral-enhanced spatial–spectral features.

(3) A GAFBlock is built to align and integrate the heterogeneous spatial–spectral features, and with the designed ASBlock-based backbone, a novel ASLNN model is further proposed to extract the adaptive spectral-enhanced spatial–spectral features, and it gives consideration to the interpretability of spatial–spectral feature extraction and the ability to describe detailed information design for the better classification of HSIs.

The remainder of this article is organized as follows. In Section 2, the framework and optimization process of the proposed ASLNN are described in detail. Section 3 reports the experimental settings, results, and comparative analysis, which is followed by the conclusions in Section 4.

2. Proposed ASLNN Model

2.1. Architecture Overview

Figure 1 presents the framework of the proposed ASLNN model for the spatial–spectral extraction and classification of HSIs. Due to the fact that the spectral correlations exist between adjacent spectral bands and between nonadjacent spectral bands, a universal plug-and-play ASBlock is constructed in Section 2.2 for extracting enhanced spatial–spectral features from HSIs. Then, an effective GAFBlock is further developed to integrate deep and heterogeneous features by adaptively learning their respective attribute information, improving the spatial–spectral extraction and classification performance of the whole model, as shown in Section 2.3. Finally, Section 2.4 outlines the structure and the training process of our ASLNN model in detail.

2.2. ASBlock

Although there have been plenty of deep models for the spectral-enhanced spatial–spectral feature extraction and classification of HSIs, they often failed to jointly consider the local and global spectral correlation information, limiting their classification performance.

Inspired by the ResNeSt that is designed to model the complementary of both feature-map attention and multipath representation [44], an ASBlock is constructed to adaptively learn the spectral correlations (from a local and global perspective) for the better classification of HSIs. Its inner structure is shown in Figure 2, which contains a spectral group split (SGS) subblock, a short-term spectral correlation learning (STC) subblock, a long-term spectral correlation learning (LTC) subblock, and a group spatial–spectral attention learning (GSS) subblock.

2.2.1. SGS Subblock

Let the input of our ASBlock be denoted as $X^l\in {\mathbb{R}}^{s_l\times s_l\times K_l\times C_l}$, where $s_l$, $K_l$, and $C_l$ are the spatial size, the spectral number, and the channel number, respectively. Firstly, the input $X^l$ is fed into the SGS subblock, where it is equally divided into N parts along the spectral dimension, forming a spectral group $\left\{X_i^l\in {\mathbb{R}}^{s\times s\times d\times C_l}\right\}$ as the input of other subblocks. $i=1,2,\dots ,N_s$, $N_s=\lfloor {\displaystyle \frac{K_l}{d}}\rfloor$. $\lfloor ·\rfloor$ means the round down operation, and d is the number of the spectral bands in each spectral group.

2.2.2. STC Subblock

Then, by considering the fact that there is the spectral correlation between the adjacent spectral bands, the spectral group built in the SGS subblock is fed into the STC subblock, where a $1\times 1\times 1$ GConv block and a $3\times 3\times 3$ GConv block are executed in turn to learn the short-term spectral information, where the GConv block contains a 3D GConv layer, a batch normalization (BN) layer, and a Rectified Linear Unit (ReLU) function, extracting the short-term spectral-enhanced spatial–spectral features from HSIs. The above calculation process is defined as

$$\begin{array}{c}\hfill {X_i^l}_S=f_{GC3}\left(f_{GC1}\left(X_i^l\right)\right),\end{array}$$

(1)

where $\left\{{X_i^l}_S\in {\mathbb{R}}^{s_l\times s_l\times d\times C_{l+1}}\right\}$ means the short-term spectral-enhanced spatial–spectral feature. $f_{GC1}(·)$ and $f_{GC3}(·)$ are the $1\times 1\times 1$ and $3\times 3\times 3$ 3D GConv blocks, respectively.

2.2.3. LTC Subblock

Furthermore, as spectral correlation also exists between nonadjacent spectral bands, a LTC subblock is further built by utilizing the ability of learning long-term dependencies of the ConvLSTM3D layer, thus adaptively modeling the long-range spectral correlation for spatial–spectral feature extraction and classification. Particularly, the short-term spectral enhanced spatial–spectral feature $\left\{{X_i^l}_S\right\}$ is first converted into a sequence with length $N_s$. And then this sequence is fed to a ConvLSTM3D layer to extract the long-term spectral-enhanced spatial–spectral feature from HSIs, as indicated as below

$$\begin{array}{c}\hfill \left\{{X_i^l}_L\right\}=f_{CL3D1}\left(\left\{{X_i^l}_S\right\}\right),\end{array}$$

(2)

where $\left\{{X_i^l}_L\in {\mathbb{R}}^{s_l\times s_l\times d\times C_{l+1}}\right\}$ means the long-term spectral-enhanced spatial–spectral feature. $f_{CL3D1}(·)$ means the $1\times 1\times 1$ ConvLSTM3D layer.

2.2.4. GSS Subblock

Then, to jointly learn and integrate the above short- and long-term spectral correlations, a GSS subblock is further developed to learn the adaptive spectral correlations from HSIs, as shown in Figure 3, which consists of a group self-spectral attention module and a self-intrinsic attention module, as shown in Figure 3a and Figure 3b, respectively.

The spatial–spectral feature $\left\{{X_i^l}_L\right\}$ is first fed into the group self-spectral attention module, where the group spectral attention weights are learned from the input along with its spectral dimension, adaptively modeling the short- and long-term spectral correlations to extract more discriminative spatial–spectral features. Due to the fact that different land covers present various structure characteristics, a self-intrinsic attention module is further built from the input $\left\{{X_i^l}_L\right\}$ to learn the intrinsic structure attention weight, thus jointly modeling the structure- and adaptively spectral correlation-enhanced spatial–spectral features for HSI classification. In particular, in order to avoid the gradient vanishing (or explosion) problem to some extent, a skip connection is considered [44], where the input spatial–spectral feature $\left\{{X_i^l}_L\right\}$ is reused as the output. The forward propagation of the GSS subblock can be written as

$$\begin{array}{cc}\hfill T& =f_{CBR1}\left(f_{concat}\left(\left\{{X_i^l}_L\right\}\right)\right)\hfill \\ \hfill \left\{{H_i^l}_L\right\}& =\left\{{X_i^l}_L\right\}\otimes f_{spe}\left(T\right)\hfill \\ \hfill \left\{{\widehat{H_i^l}}_L\right\}& =\left\{{H_i^l}_L\right\}\otimes f_{int}\left(T\right)\hfill \\ \hfill F^l& =f_{concat}(\left\{{\widehat{H_i^l}}_L\right\}+\left\{{X_i^l}_L\right\}),\hfill \end{array}$$

(3)

where $F^l\in {\mathbb{R}}^{s_L\times s_l\times K_l\times C_{l+1}}$ means the output of the GSS subblock, and $f_{concat}(·)$ denotes the concatenation operation. $f_{CBR1}(·)$ is the $1\times 1\times 1$ 3D Conv layer with the BN layer and the ReLU function. ⊗ means the element-wise multiplication operation. $f_{spe}(·)$ and $f_{int}(·)$ are, respectively, the group self-spectral attention module and the self-intrinsic attention module, which can be defined as

$$\begin{array}{cc}\hfill f_{spe}\left(T\right)& =G_{\sigma }\left(f_{C1}(softmax\left(f_{CBR1}\left(T\right)\right)\odot f_{CBR1}\left(T\right))\right)\hfill \\ \hfill f_{int}\left(T\right)& =\sigma (f_{C1}(softmax\left(f_{GAP}\left(f_{CBR1}\left(T\right)\right)\right)\hfill \\ \hfill & \odot f_{CBR1}\left(T\right)\left)\right),\hfill \end{array}$$

(4)

where $\sigma (·)$, $G_{\sigma }(·)$, and $softmax(·)$ denote the sigmoid, group sigmoid, and softmax functions, respectively. ⊙ is the matrix multiplication operation. $f_{GAP}(·)$ is the global average pooling (GAP) layer. $f_{C1}(·)$ is the $1\times 1\times 1$ 3D Conv layer without the BN layer and the ReLU function.

Finally, the above output $F^l$ is fed into another $1\times 1\times 1$ 3D GConv layer, and the input $X^l$ is further reused to obtain the final spatial–spectral feature $X^{l+1}\in {\mathbb{R}}^{s_L\times s_l\times K_l\times C_{l+1}}$ of the whole ASBlock

$$\begin{array}{c}\hfill X^{l+1}=f_{CBR1}\left(F^l\right)+X^l.\end{array}$$

(5)

Therefore, by taking the ASBlock in Figure 2 as a basic unit to build the adaptive convolutional backbone, the adaptive spectral-enhanced deep spatial–spectral features can be extracted for HSI classification.

2.3. GAFBlock

Some work has verified that combining deep and heterogeneous features can solve the problems of the lack of details and interpretability of deep features to some degree [6,43]. However, the parameters and time complexities are relatively high, limiting the applications of these algorithms in practice. As such, drawing on the gate structure of ConvLSTM-based models and the attention mechanism, a simple but effective GAFBlock is constructed to integrate these two modalities of spatial–spectral features with low complexity, as shown in Figure 4, obtaining more expressive spatial–spectral features for HSI classification.

First, following previous research [6,8,43], the 3D Gabor filter is applied as the heterogeneous spatial–spectral feature extractor, whose detailed calculation process and parameter settings for spatial–spectral feature extraction are the same as those in [6,43] to simplify the model complexity and experimental analysis. A 3D Gabor filter is a specialized linear filter designed to capture spatial–spectral features in volumetric data, such as HSIs. It extends the 2D Gabor filter by incorporating a third dimension (spectral) to analyze joint spatial–spectral variations. The filter combines a sinusoidal plane wave with a 3D Gaussian envelope, enabling multiscale and multi-orientation feature extraction. For a pixel at spatial–spectral coordinates ($\alpha$,$\beta$,$\gamma$), the circular 3D Gabor filter is defined as

$${\Psi }_{f,\phi ,\theta }(\alpha ,\beta ,\gamma )={\displaystyle \frac{1}{{\left(2\pi \right)}^{\frac{2}{3}}{\sigma }^3}}exp\left(j2\pi (\alpha x+\beta y+\gamma z)\right)exp\left(-{\displaystyle \frac{{\alpha }^2+{\beta }^2+{\gamma }^2}{2{\sigma }^2}}\right)$$

(6)

where $x=fsin\phi cos\theta$, $y=fsin\phi sin\theta$, and $z=fcos\phi$. Variable f is the central frequency (scale factor), $\phi$ and $\theta$ represent the orientation angles in the 3D frequency domain, and $\left(\sigma \right)$ is width of the Gaussian envelope in the 3D frequency domain. Therefore, heterogeneous spatial–spectral features can be extracted by using $N_g$ 3D Gabor wavelets with different frequencies and orientations, which can be written as $F^{\left(heterogeneous\right)}$. Hence, the two modalities of spatial–spectral features (i.e., deep feature $F^{\left(deep\right)}$, Gabor feature $F^{\left(heterogeneous\right)}$) can be obtained from HSI data.

After that, due to the fact that the heterogeneous features contain more detailed information of land covers, a $1\times 1\times 1$ 3D Conv layer and a softmax function are applied to obtain the attention weight for learning of the detailed structure information of HSIs, and another $1\times 1\times 1$ 3D Conv layer with a ReLU function is also executed to align the dimension of heterogeneous features with that of the deep features, which can be written as

$$\begin{array}{cc}\hfill F^{\left(heterogeneous\right)}& =f_{C1}\left(F^{\left(heterogeneous\right)}\right),\hfill \\ \hfill {\alpha }^{\left(det\right)}& =softmax\left(f_{C1}\left(F^{\left(heterogeneous\right)}\right)\right)\hfill \end{array}$$

(7)

where ${\alpha }^{\left(det\right)}$ is the attention weight for the learning of detailed information of heterogeneous features.

As for the deep feature $F^{\left(deep\right)}$, it is a channel attention structure containing a GAP layer, a $1\times 1\times 1$ 3D Conv layer with a ReLU function, and a $1\times 1\times 1$ 3D Conv layer with a softmax function to model the most important channel information, whose forward propagation can be expressed as

$$\begin{array}{cc}\hfill {\alpha }^{\left(cha\right)}& =softmax\left(f_{C1}\left(ReLU\left(f_{C1}\left(f_{GAP}\left(F^{\left(deep\right)}\right)\right)\right)\right)\right),\hfill \end{array}$$

(8)

where ${\alpha }^{\left(cha\right)}$ is the attention weight for learning the channel information of deep features.

Then, by projecting these two modalities of features into a common space, their complementary evidences can be learned. Moreover, with the gate structure and the skip connection, these two modalities of features are adaptively integrated, jointly modeling the detailed and semantic information to extract the discriminative spatial–spectral features from HSIs, as written below

$$\begin{array}{cc}\hfill F^{\left(fuse\right)}& =\sigma (F^{\left(heterogeneous\right)}\otimes {\alpha }^{\left(cha\right)}+F^{\left(deep\right)}\otimes {\alpha }^{\left(det\right)})+F^{\left(deep\right)},\hfill \end{array}$$

(9)

where $F^{\left(fuse\right)}$ is the fused spatial–spectral feature. $\sigma (·)$ is the sigmoid function as the gate structure.

2.4. ASLNN Model and Its Training Process

Based on the above ASBlock and GAFBlock, a novel ASLNN model is proposed, as shown in Figure 1, whose backbone contains a $3\times 3\times 3$ 3D Conv layer with a ReLU function, l ASBlocks, and a GAFBlock to explore the adaptive spectral correlations and integrate the advantages of details of heterogeneous features, extracting discriminative spatial–spectral features for HSI classification.

Suppose that the dimension of original HSI data is expressed as $W\times H\times D$, where W, H, and D are the width, the height, and the spectral number, respectively. Similar to the preprocessing process in most works, a 3D volume, containing the local spatial region with size $s\times s$ and the spectral band with number D, is extracted as the spatial–spectral information of each pixel x, which can be denoted as $X\in {\mathbb{R}}^{s\times s\times D}$. Then, same as the designs in [6,8,43], by considering the characteristics of heterogeneous features extracted by the 3D Gabor filter, its orientations ($\theta$ and $\varphi$) take the values of $\left\{0,{\displaystyle \frac{\pi }{n}},{\displaystyle \frac{2\pi }{n}},\dots ,{\displaystyle \frac{\left(n-1\right)\pi }{n}}\right\}$, $n=4$, and the number ($N_g$) of the 3D Gabor filters is 13 for each frequency (f) after removing some redundant filters. To simplify the experimental analysis, the value of d in each spectral group is fixed to 13 (with zeros appended if not divisible), where the number of the total spectral bands is set as K and the spectral group number is ($N_s$). Hence, the spatial–spectral information of pixel x is expressed as $X\in {\mathbb{R}}^{s\times s\times K}$, and $N_s=\lfloor {\displaystyle \frac{K}{d}}\rfloor$.

Based on the above data preprocessing framework, the input data X are first fed into a $3\times 3\times 3$ 3D Conv layer with a ReLU function to extract the low-level features. Then, to adaptively explore spectral correlations of HSIs, l ASBlocks are stacked to jointly model the short- and long-range spectral correlations, generating high-level deep spatial–spectral feature $F^{\left(deep\right)}\in {\mathbb{R}}^{s\times s\times K\times C_l}$ from the lth ASBlock. Meanwhile, X is convolved with $N_g$ 3D Gabor wavelets to extract the heterogeneous feature $F^{\left(heterogeneous\right)}\in {\mathbb{R}}^{s\times s\times K\times N_g}$. Then, these two modalities of spatial–spectral features (i.e., $F^{\left(deep\right)}$ and $F^{\left(heterogeneous\right)}$) are fed into the GAFBlock to adaptively explore and integrate their complementary information, extracting the fused spatial–spectral feature $F^{\left(fuse\right)}\in {\mathbb{R}}^{s\times s\times K\times C_l}$ from Equation (9) for HSI classification. Particularly, to simplify the parameter analysis process of the whole model, the channel number of the first $3\times 3\times 3$ 3D Conv layer is fixed to 32, l is 1, and the channel numbers in ASBlocks and GAFBlock are fixed to 64.

Finally, inspired by [6,45], the spatial–spectral feature $F^{\left(fuse\right)}$ is fed into a GAP layer and an orthogonal softmax layer (OSL) where the classification network contains fewer trainable parameters, and the cross-entropy is used as the loss function $\mathcal{L}$ of ASLNN, as expressed below

$$\begin{array}{cc}\hfill Y^P& =f_{OSL}\left(f_{GAP}\left(F^{\left(fuse\right)}\right)\right)\hfill \\ \hfill \mathcal{L}& =-\sum (Y^T·log\left(Y^P\right)),\hfill \end{array}$$

(10)

where $f_{OSL}(·)$ is the OSL. $Y^P$ and $Y^T$ are the prediction label of the whole ASLNN model and the ground-truth label of the input data, respectively.

By optimizing the equation above (10) with the adaptive momentum (ADAM) algorithm and the learning rate $lr$, our ASLNN can be trained in an end-to-end manner, thereby generating the classification results of land covers. Additional analysis on parameter settings will be reported in Section 3.

3. Experimental Results

In this section, to quantitatively and qualitatively evaluate the performance of our ASLNN, some spatial–spectral classification methods, e.g., APDCLNN [6], 3DG-CNN [43], SVM-CK [15], SSCL3DNN [24], 3DOC-SSAN [46], D²S²BoT [47], DBCTNet [38], HybridSN [48], and ISODATA [49], are selected as the comparison methods. Four public HSI data sets are applied for performance analysis, e.g., WHU-Hi-LongKou, Indian Pines, 2013 Houston Data, and MUUFL Gulfport. The overall accuracy (OA), average accuracy (AA), and Kappa coefficient ($\kappa$) are the adopted quantitative metrics. To avoid the deviation caused by a random selection of samples, 10 Monte Carlo runs are executed, reporting the average value of each metric in the following experimental results. Our ASLNN is built with the Tensorflow platform (i.e., Tensorflow-GPU 2.5.0, python 3.8.0) and trained on a desktop with an Intel Core i7-12700F processor and an Nvidia GeForce RTX 3080ti GPU.

3.1. Hyperspectral Data

In order to measure the classification performance of the proposed ASLNN model with the above comparison methods, the WHU-Hi-LongKou, Indian Pines, Houston Data, and MUUFL Gulfport are used as the HSI data sets, whose detailed information can be found in Figure 5 and Figure 6.

(1) WHU-Hi-LongKou: The WHU-Hi-LongKou data set, depicted in Figure 5, was procured on a DJI Matrice 600 Pro, conducting aerial surveys over Longkou Town in China in 2018. The data set contains 270 spectral bands with a spectral resolution of 0.4 to 1 μm and 550 × 400 pixels [50,51]. The scene covers nine specific categories, comprising six diverse crop species, totaling 204,542 samples.

(2) Indian Pines: Using the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor, the Indian Pines data set was acquired in 1992 in northwestern Indiana, USA, which consists of $145\times 145$ pixels with a spatial resolution of 20 m per pixel (mpp), and 200 bands in the wavelength range from 0.40 to 2.50 μm after removing some unusable spectral bands [52]. In addition, after ignoring the background pixels, 10,249 samples from 16 classes are used for experimental analysis.

(3) Houston Data: The Houston data set was collected in 2012 with the Compact Airborne Spectrographic Imager (CASI) sensor over the University of Houston campus and its surrounding area, and it is the official data set in the 2013 IEEE Geoscience and Remote Sensing Society (GRSS) Data Fusion contest, which is available online from the official Website: http://dase.grss-ieee.org/, accessed on 22 May 2025. Its spatial size is $349\times 1905$ pixels with a spatial resolution of 2.5 m, and there are 144 spectral bands in the wavelength range from 0.38 to 1.05 μm maintained by removing some noise-corrupted spectral bands [52]. Moreover, there are 15,029 samples from 15 labeled classes for experimental research.

(4) MUUFL Gulfport: It was collected in 2010 by using the ITRES CASI-1500 sensor over the University of Southern Mississippis Gulf park Campus, Long Beach, Mississippi, USA [53], which is available online from the official website: https://github.com/GatorSense/MUUFLGulfport, accessed on 22 May 2025. This data set contains $325\times 337$ pixels with a spatial resolution of 1 m where $325\times 220$ pixels are applied for analysis and 72 spectral bands with a wavelength range from 0.375 to 1.050 μm. After removing some noise bands and regardless of background pixels, 64 bands and 53,687 labeled samples with 11 urban land-cover classes are maintained for experimental research.

3.2. Experimental Settings

The parameter settings of the selected compared methods are used according to [6,15,24,38,43,46,47,48,49]. In addition, 10 samples are randomly selected from each class of the Indian Pines, Houston, and MUUFL Gulfport data sets to build the training set, while the other samples are used for testing. The training and testing sets of the WHU-Hi-LongKou data set have been officially divided. In our research, we use 25 training labels for each class, while the other samples are used for testing.

According to the structure design of the proposed ASLNN model, there are some key parameters needed to be tuned. Particularly, inspired by the work in [6,8,43], the value of the frequency f for 3D Gabor filter is fixed to 0.50, 0.25, and 0.125 for the four HSI data sets, respectively. In addition, in order to simplify the parameter analysis process and compare fairly, the kernel size and the channel number in the whole model are fixed, as shown in Section 2. Therefore, only two key parameters need to be analyzed in detail, such as the spatial size ($s\times s$), the number ($N_s$) of the spectral groups or the number (K) of the spectral bands. Compared with other methods, the complexity of parameter analysis of the proposed model is greatly reduced.

For the spatial size s, the effect of its value on the classification performance of the whole model is further studied, where s is searched from $\left\{3,5,7\right\}$. Specifically, due to the limitations of memory, the local spatial window with a larger size fails to be supported. According to the experimental results in Figure 7, the proposed ASLNN model produces quasi-optimal classification accuracy when s is set to 7 for the four HSI data sets.

3.3. Classification Performance

Based on the experimental settings,

Table 1. Classification results of different methods for the Indian Pines data set.

Class	ISODATA	SVM-CK	HybridSN	SSCL3DNN	3DOC-SSAN	3DG-CNN	APDCLNN	D2S2BoT	DBCTNet	ASLNN
1	78.15	97.78	98.61	100.00	99.44	98.06	99.72	56.16	100.00	97.22
2	33.10	59.56	45.90	53.30	72.08	69.68	68.43	78.44	86.31	66.29
3	32.68	64.95	54.22	50.37	65.78	55.98	66.96	60.47	58.17	83.78
4	72.24	80.44	83.57	88.63	89.30	84.41	95.29	89.63	100.00	85.90
5	77.75	87.53	78.27	76.58	84.57	90.61	73.68	97.57	91.96	88.58
6	74.02	94.39	93.79	90.17	94.06	92.26	83.58	94.40	94.72	96.25
7	55.87	98.89	100.00	100.00	100.00	99.44	100.00	60.52	100.00	100.00
8	84.36	93.59	98.89	100.00	98.01	95.64	99.98	99.78	100.00	92.09
9	84.36	100.00	100.00	100.00	100.00	100.00	100.00	17.04	100.00	100.00
10	52.78	74.07	68.62	75.36	77.38	67.06	75.97	77.81	71.10	75.57
11	20.02	59.56	59.80	59.66	62.05	59.21	73.49	89.11	58.89	69.12
12	30.70	57.63	60.62	64.97	63.19	71.34	75.15	46.23	42.19	80.44
13	69.23	97.74	99.28	99.79	97.38	98.31	96.56	89.62	99.28	95.38
14	52.70	92.10	83.10	88.29	85.73	89.10	84.38	96.25	98.08	99.60
15	43.61	78.62	77.18	87.66	89.18	82.39	90.03	74.41	97.87	93.61
16	68.47	98.31	96.14	100.00	100.00	99.40	98.67	48.08	100.00	98.67
OA	41.08	73.18	68.78	71.28	76.10	73.87	77.82	79.42	77.54	81.12
AA	60.87	83.45	81.12	83.42	86.13	84.60	86.37	73.47	87.45	88.99
$\kappa$	38.92	69.79	65.01	67.83	73.19	71.08	75.02	76.94	74.59	78.76

,

Table 2. Classification results of different methods for the Houston data set.

Class	ISODATA	SVM-CK	HybridSN	SSCL3DNN	3DOC-SSAN	3DG-CNN	APDCLNN	D2S2BoT	DBCTNet	ASLNN
1	85.41	90.81	89.52	85.98	91.37	88.76	90.07	87.80	75.58	92.67
2	68.56	88.04	84.57	81.27	81.78	86.75	88.70	82.25	87.54	94.87
3	86.27	88.96	87.46	95.68	92.73	98.20	92.99	97.15	99.27	92.88
4	71.24	86.97	83.54	84.80	85.24	88.44	89.74	91.65	85.65	92.33
5	67.85	99.12	97.89	95.05	96.42	99.86	94.87	95.20	97.88	99.51
6	69.19	80.38	65.92	73.38	88.47	73.41	90.96	83.23	91.22	91.03
7	57.63	65.84	72.04	73.86	68.90	69.37	83.39	81.01	86.64	77.05
8	49.59	53.99	55.51	60.19	65.02	52.72	58.02	90.85	42.38	62.53
9	50.64	76.99	62.15	70.21	69.58	63.82	78.16	75.53	84.94	80.13
10	17.09	76.95	45.52	68.82	79.47	73.06	71.25	62.87	91.53	82.87
11	59.10	69.00	60.24	61.57	65.36	69.67	67.36	96.77	57.63	74.43
12	21.89	72.27	60.78	61.91	84.43	73.47	68.42	69.38	43.90	81.28
13	11.30	72.79	55.23	57.60	90.50	76.80	86.14	66.04	85.40	88.67
14	78.84	91.99	96.24	98.85	96.39	97.18	96.32	100.00	90.19	98.41
15	80.13	95.94	86.14	84.48	95.54	98.61	94.41	95.08	98.61	99.34
OA	54.49	79.57	72.98	75.97	81.18	79.07	81.31	82.67	78.64	85.88
AA	57.66	80.67	73.72	76.91	83.41	80.68	83.39	84.99	81.36	86.58
$\kappa$	52.31	77.92	70.85	74.04	79.67	77.38	79.83	81.28	76.91	84.83

,

Table 3. Classification results of different methods for the MUUFL Gulfport data set.

Class	ISODATA	SVM-CK	HybridSN	SSCL3DNN	3DOC-SSAN	3DG-CNN	APDCLNN	D2S2BoT	DBCTNet	ASLNN
1	44.98	82.52	82.64	86.40	67.50	78.30	83.32	97.99	85.05	81.42
2	57.95	73.71	75.91	62.73	60.24	71.32	62.38	71.23	65.32	81.17
3	48.55	68.09	42.27	45.39	22.37	57.00	56.13	65.13	65.11	67.95
4	53.25	83.62	77.61	84.39	69.50	74.00	75.47	72.97	65.85	74.28
5	48.53	79.08	55.76	47.84	43.37	69.22	67.30	88.12	68.11	89.15
6	68.20	91.16	90.44	99.91	92.08	80.66	95.09	39.59	98.46	99.91
7	64.93	74.46	69.51	73.37	50.60	70.05	77.14	44.80	93.47	96.32
8	34.86	62.58	62.41	72.99	39.98	66.42	71.09	95.06	54.22	86.35
9	13.30	59.99	34.02	30.01	33.67	35.29	34.81	21.03	47.92	66.47
10	31.79	73.99	53.41	48.15	62.25	54.45	52.14	6.15	23.69	73.41
11	74.20	86.56	94.48	94.40	95.47	89.38	94.13	62.40	96.52	96.52
OA	50.27	76.41	69.29	70.87	53.65	70.90	73.03	76.82	73.94	80.62
AA	47.71	75.98	67.13	67.78	55.19	67.82	69.91	60.41	69.43	80.52
$\kappa$	40.55	69.93	60.78	62.60	43.14	63.21	65.54	70.63	66.75	74.15

and

Table 4. Classification results of different methods for the WHU-Hi-LongKou data set.

Class	ISODATA	SVM-CK	HybridSN	SSCL3DNN	3DOC-SSAN	3DG-CNN	APDCLNN	D2S2BoT	DBCTNet	ASLNN
1	81.24	99.34	97.21	90.02	94.72	93.77	98.80	98.74	95.06	99.62
2	83.30	95.48	97.19	88.84	97.89	74.18	96.98	74.48	93.84	97.89
3	77.20	99.90	94.04	94.81	98.40	88.58	85.16	84.20	99.56	98.23
4	49.54	90.23	77.77	91.89	82.53	93.70	89.08	99.93	84.22	95.31
5	60.95	92.54	87.63	93.40	86.08	89.77	95.61	84.22	89.66	97.91
6	81.73	90.45	99.62	95.12	97.45	96.78	97.43	95.59	93.46	99.39
7	77.98	97.44	99.89	99.57	99.95	96.13	99.52	99.89	98.72	99.75
8	81.06	87.69	70.06	92.80	72.33	82.36	89.68	88.35	90.96	95.22
9	61.53	93.74	90.08	96.38	96.07	88.73	97.92	61.34	85.36	96.94
OA	71.24	94.55	90.85	94.38	92.09	93.21	95.27	95.48	92.34	97.97
AA	72.92	94.09	90.39	93.65	91.71	89.33	94.47	87.42	92.32	97.81
$\kappa$	68.96	92.94	88.23	92.68	89.78	91.16	93.85	94.14	90.09	97.35

report the classification results (i.e., class-specific accuracy, OA, AA, and $\kappa$) of the proposed ASLNN model and other comparison methods under the Indian Pines, Houston Data, MUUFL Gulfport, and WHU-Hi-LongKou data sets, respectively.

Compared with other supervised algorithms, ISODATA failed to achieve comparable results. There might be several reasons why ISODATA fails to achieve comparable results to other supervised algorithms. First, as an unsupervised algorithm, it cannot utilize labeled training samples for model development, which inherently limits its ability to match the classification performance of supervised methods that leverage explicit class annotations. Second, the data set used in this paper contains a large number of continuous spectral bands, leading to a high-dimensional and information-redundant data space. This poses significant challenges for distance-based algorithms like ISODATA, as traditional distance metrics become ineffective, inter-class discrimination becomes ambiguous, and the reliability of clustering results decreases. Finally, real-world ground objects exhibit inherent spectral variability (e.g., intra-class spectral differences caused by lighting conditions, humidity, or phenological stages). However, ISODATA relies on linear distance metrics and assumes simple clustering structures [49], making it unable to model the complex nonlinear distributions of such spectral variations, resulting in suboptimal clustering performance.

By integrating the multilayer residual convolution and spectral–spatial bottleneck transformer, D²S²BoT was proposed to jointly model the local features and the long-range spectral correlation, as well as adaptively fuse global spectral and spatial dependencies, thus yielding the second-best classification results on the four data sets. As a strong baseline under the case of small-sample classification, SVM-CK was designed by incorporating the composite kernel into the SVM classifier for jointly using the spatial–spectral information [15], which can process high-dimensional small-sample data, and it is insensitive to the “curse of dimensionality”, generating the third-best classification results on the MUUFL Gulfport data set. Particularly, according to Figure 6c–f, compared with the other three HSI data sets, the MUUFL Gulfport data set presents the characteristics of category imbalance and large differences in the number of samples between different classes, thus obtaining better classification accuracy than APDCLNN and DBCTNet. Since the complementary information between different modality features is not considered, other DNNs (i.e., HybridSN, SSCL3DNN, 3DOC-SSAN, 3DG-CNN) yield the relatively poor classification performance for the four HSI data sets. Differently, on the basis of the spectral group and 3D Gabor filter, our ASLNN model adaptively learns the spectral correlations by jointly considering the relationships between the adjacent spectral bands and the nonadjacent spectral bands, and it integrates the fine-grained complementarity of deep and heterogeneous spatial–spectral features, thus achieving the best classification performance for the four HSI data sets under the premise of low complexity. All of these aspects verify the effectiveness and superiority of the proposed ASLNN model for HSI classification purposes.

More precisely, D²S²BoT and DBCTNet are designed by transformer to capture the local feature maps and simulate the long-range correlation of HSI pixels across both spectral and spatial dimensions. However, they only consider the long-range spectral correlation, and they ignore the local relationship between the adjacent spectral bands, limiting their classification performance. Compared with them, our ASLNN model jointly learns the short-range and long-range spectral correlations from both local and global perspectives, and it effectively integrates the heterogeneous spatial–spectral features, achieving an average improvement of 1.70%, 3.21%, 3.80% and 2.49% compared to D²S²BoT and DBCTNet on the OA value of four HSI data sets, respectively. Although APDCLNN utilizes the modified ConvLSTM2D layer and the heterogeneous feature fusion module to enhance the ability of spatial–spectral feature extraction for improving the classification accuracy of HSIs, its complexity is very high due to a large number of operations and complicated feature extraction and fusion processes. Compared with APDCLNN, our ASLNN model effectively integrates the heterogeneous spatial–spectral features with low storage and computing requirements, thus obtaining 3.30%, 4.57%, 7.59%, and 2.70% improvements in OA for the four HSI data sets, respectively. As for 3DG-CNN, the 3D Gabor filter is applied to build a 3D Gabor-modulated kernel to replace the random initialization kernel to improve the ability of representation and robustness of the extracted spatial–spectral features, which, however, overlooks the influence of heterogeneous features on the classification performance. Compared with it, the gains in OA generated by our ASLNN model are 7.25%, 6.81%, 9.72%, and 4.76% for the four HSI data sets, respectively. The proposed ASLNN model can obtain 7.94%, 6.30%, 4.21%, and 3.42% gains in OA for the four HSI data sets, respectively, when compared with SVM-CK. In addition, compared with other deep models (i.e., HybridSN, SSCL3DNN, and 3DOC-SSAN), our ASLNN model can achieve similar performance gains. Based on the above experimental results and analysis, effective learning of the spectral correlations and fusion of the heterogeneous spatial–spectral features are the keys to improving the classification accuracy of land covers. Therefore, the proposed ASLNN model can extract more discriminative and robust spatial–spectral features and obtain the highest classification accuracy for HSIs, illustrating its rationality and effectiveness.

Moreover, more intuitively, Figure 8, Figure 9, Figure 10 and Figure 11 illustrate their classification maps for the four HSI data sets, respectively. It can be observed that the classification maps yielded by our ASLNN model are the closest to the ground-truth maps of the four HSI data sets, where the classification qualities of class 3, class 6, class 7, and class 12 in Figure 8, class 1, class 2, class 4, and class 15 in Figure 9, class 2, class 5, class 6, class 9, and class 11 in Figure 10, and class 1, class 2, class 5, and class 8 in Figure 11 are significantly improved. These are consistent with the conclusions in Table 1, Table 2, Table 3 and Table 4. These experimental results further prove the effectiveness of our proposed ASLNN model.

3.4. Sensitivity Comparison of Different Training Samples

To further compare and analyze the sensitivity of different training samples on the classification performance of all supervised considered methods, Figure 12 reports the experimental results under different training sizes, where 10, 20, 30, and 40 samples are randomly selected from each class to build the training set for the Indian Pines, Houston, and MUUFL Gulfport data sets, respectively, while the other samples are applied for testing. Particularly, as for class 7 and class 9 in the Indian Pines data set, their numbers of the training samples are fixed to 10. For the WHU-Hi-LongKou data set, we use four kinds of ground truth setups with 25, 50, 100, and 150 training labeled samples (also provided by the official) for each class, respectively. We can observe that as the number of training samples increases from 10 to 40 (or 25 to 150), the classification performance of all methods presents a trend of first rising and then stabilizing. In particular, the proposed ASLNN model achieves the best classification performance in all cases for the four HSI data sets, which also demonstrates its advantages.

3.5. Ablation Study

Aiming at adaptively learning the inherent attribute information of HSIs, an ASBlock and a GAFBlock are designed to construct an ASLNN model to adaptively learn the adaptive spectral correlations and integrate the details and interpretability from the heterogeneous features, improving the classification accuracy of land covers in HSIs. To illustrate and measure the contributions of the designed ASBlock and GAFBlock to the classification results of our ASLNN, a detailed ablation study is conducted, where the proposed ASLNN models with and without each component are abbreviated as proposed (with) and proposed (without) for convenience, respectively.

3.5.1. Effectiveness of the ASBlock

Inspired by the ResNeSt block in [44], to adaptively model the correlations between the adjacent spectral bands and between the nonadjacent spectral bands of HSIs, a novel ASBlock is built by utilizing the GConv and ConvLSTM3D layers. To demonstrate its contribution for HSI classification of the whole model, our ASLNN model is compared with its variant (i.e., the proposed (without ASBlock)) where the ASBlock is replaced with the ResNeSt block, whose experimental results are given in

Table 5. Effectiveness research of the ASBlock.

Models	Indian Pines		Houston Data
	OA	$\mathit{\kappa }$	OA	$\mathit{\kappa }$
proposed (without)	71.86	68.38	72.92	70.84
proposed (with)	81.12	78.76	85.88	84.83
Models	MUUFL Gulfport		WHU-Hi-LongKou
	OA	$\mathit{\kappa }$	OA	$\mathit{\kappa }$
proposed (without)	73.65	66.91	94.96	93.51
proposed (with)	80.62	74.15	97.97	97.35

. It can be observed that compared with this variant, the ASBlock can bring 9.26%, 12.96%, 6.97%, and 3.01% gains in OA to the proposed ASLNN model for the four HSI data sets, respectively, verifying its feasibility and effectiveness in learning spectral correlations and extracting spatial–spectral features with strong expressive ability for HSI classification.

3.5.2. Structure Analysis of the GAFBlock

In order to adaptively align and fuse the heterogeneous spatial–spectral features (i.e., deep features and heterogeneous features) with low complexity, a simple but effective GAFBlock is constructed, and the experiments are carried out to compare our ASLNN with its variant (i.e., the proposed (without GAFBlock)) where the GAFBlock is replaced with the concatenation operation for HSI classification. From

Table 6. Structure analysis of the GAFBlock.

Models	Indian Pines		Houston Data
	OA	$\mathit{\kappa }$	OA	$\mathit{\kappa }$
proposed (without)	79.77	76.82	82.76	81.24
proposed (with)	81.12	78.76	85.88	84.83
Models	MUUFL Gulfport		WHU-Hi-LongKou
	OA	$\mathit{\kappa }$	OA	$\mathit{\kappa }$
proposed (without)	75.87	68.61	95.61	94.12
proposed (with)	80.62	74.15	97.97	97.35

, compared with the concatenation operation, with the GAFBlock, ASLNN obtains 1.35%, 3.12%, 4.75%, and 2.36% gains in OA for the four HSI data sets, respectively, which shows the effectiveness of the designed GAFBlock on fusing the heterogeneous spatial–spectral features. Particularly, compared with the MAFB submodule in [6], the newly designed GAFBlock contains fewer network parameters and has lower computational complexity, which reduce the adverse influence of overfitting problem on the proposed ASLNN model, thus improving its optimization efficiency and classification performance for HSIs.

3.6. Computational Costs

As presented in

Table 7. Computing performance comparison results of different methods for the WHU-Hi-LongKou data set.

Class	SVM-CK	HybridSN	SSCL3DNN	3DOC-SSAN	3DG-CNN	APDCLNN	D2S2BoT	DBCTNet	ASLNN
Training time (s)	684.37	1874.25	2111.42	1421.68	1774.97	2965.44	2158.94	3422.78	3756.27
Test time (s)	78.46	36.79	35.48	18.56	28.15	81.47	68.48	89.74	93.14
Memory (GB)	1.36	2.19	2.87	3.44	3.16	6.98	4.58	5.87	11.39
OA (%)	94.55	90.85	94.38	92.09	93.21	95.27	95.48	92.34	97.97

,

Table 8. Computing performance comparison results of different methods for the Indian Pines data set.

Class	SVM-CK	HybridSN	SSCL3DNN	3DOC-SSAN	3DG-CNN	APDCLNN	D2S2BoT	DBCTNet	ASLNN
Training time (s)	77.25	1239.48	1227.81	1106.59	1189.80	1278.36	1211.53	1369.74	1539.16
Test time (s)	2.42	13.49	11.89	1.13	9.788	18.32	16.31	7.88	20.09
Memory (GB)	0.91	1.78	1.54	1.12	1.85	2.24	1.08	1.34	9.08
OA (%)	73.18	68.78	71.28	76.10	73.87	77.82	79.42	77.54	81.12

,

Table 9. Computing performance comparison results of different methods for the Houston data set.

Class	SVM-CK	HybridSN	SSCL3DNN	3DOC-SSAN	3DG-CNN	APDCLNN	D2S2BoT	DBCTNet	ASLNN
Training time (s)	141.34	1360.75	138.96	1421.31	1358.07	1392.81	1256.47	2343.62	2674.33
Test time (s)	3.89	7.34	5.68	2.07	15.74	48.54	43.11	17.67	27.52
Memory (GB)	2.05	3.28	3.17	2.76	3.44	4.73	3.91	1.91	8.76
OA (%)	79.57	72.98	75.97	81.18	79.07	81.31	82.67	78.64	85.88

and

Table 10. Computing performance comparison results of different methods for the MUUFL Gulfport data set.

Class	SVM-CK	HybridSN	SSCL3DNN	3DOC-SSAN	3DG-CNN	APDCLNN	D2S2BoT	DBCTNet	ASLNN
Training time (s)	144.81	1134.12	1260.17	976.88	1428.71	1783.32	1677.26	1282.57	2396.31
Test time (s)	5.92	5.75	6.12	2.08	15.33	59.64	56.41	21.96	74.52
Memory (GB)	1.29	1.98	2.14	1.63	2.44	2.31	1.76	1.98	2.53
OA (%)	76.41	69.29	70.87	53.65	70.90	73.03	76.82	73.94	80.62

, we report the average training time, average inference time, and memory usage of all compared methods (supervised) on an NVIDIA RTX 3080 GPU. The results show that while our algorithm incurs relatively higher time costs due to its inherent complexity (especially the serial sequence processing design of ConvLSTM3D), it outperforms other methods in terms of classification accuracy.

3.7. Limitations and Future Directions

First, the model exhibits high computational complexity with its intricate architecture leading to significant computational overhead and prolonged processing times. Second, real-time deployment on drones or embedded devices remains currently infeasible due to memory requirements exceeding 2 GB and high inference latency (over 2000 ms inference). Third, because of various factors such as sensor types as well as daily and seasonal variations, there are usually significant differences in data distribution and category distribution between test data and training data in actual scenarios, limiting the application effectiveness of the proposed model in practical scenarios. Moving forward, our future work will focus on addressing these challenges through lightweight optimization techniques, such as model quantization, pruning, and knowledge distillation [54]. In addition, in order to improve the generalization of the proposed model in practical scenarios, domain adaptive hyperspectral or multisource remote sensing image classification methods are also the next focus of research, such as cross-domain few-shot classification [55], cross-domain end-to-end classification of HSIs, or multisource remote sensing data [56].

4. Conclusions

In this article, a novel ASLNN model has been proposed for the spatial–spectral feature extraction and classification of HSIs. Firstly, by integrating the GConv layer and the ConvLSTM3D layer, an effective ASBlock is designed to jointly capture the short- and long-term spectral correlations, which can be used as a plug-and-play feature extraction module to be inserted into any deep learning-based models or used as a fundamental network unit to construct a new backbone for adaptively modeling the local and global correlations, thus generating enhanced deep spatial–spectral features for HSI classification. After that, due to the rich detail and interpretability of heterogeneous features, the 3D Gabor filter is applied to extract heterogeneous spatial–spectral features, and a GAFBlock is further designed to fuse the heterogeneous spatial–spectral features with low requirements in terms of parameter and computational complexity. In addition, some network parameters are designed as fixed values, simplifying the whole model. Extensive comparison and ablation experiments on four commonly-used HSI data sets (i.e., Indian Pines, Houston, MUUFL Gulfport, and WHU-Hi-LongKou data sets) have been conducted to evaluate the proposed ASLNN model. Specifically, when 10, 10, 10 and 25 samples from each class are selected for training, ASLNN achieves the highest overall accuracy (OA) of 81.12%, 85.88%, 80.62%, and 97.97% on the four data sets, outperforming other methods with increases of more than 1.70%, 3.21%, 3.78%, and 2.70% in OA, respectively.