Self-Supervised Deep Multi-Level Representation Learning Fusion-Based Maximum Entropy Subspace Clustering for Hyperspectral Band Selection

Abstract

As one of the most important techniques for hyperspectral image dimensionality reduction, band selection has received considerable attention, whereas self-representation subspace clustering-based band selection algorithms have received quite a lot of attention with good effect. However, many of them lack the self-supervision of representations and ignore the multi-level spectral–spatial information of HSI and the connectivity of subspaces. To this end, this paper proposes a novel self-supervised multi-level representation learning fusion-based maximum entropy subspace clustering (MLRLFMESC) method for hyperspectral band selection. Firstly, to learn multi-level spectral–spatial information, self-representation subspace clustering is embedded between the encoder layers of the deep-stacked convolutional autoencoder and its corresponding decoder layers, respectively, as multiple fully connected layers to achieve multi-level representation learning (MLRL). A new auxiliary task is constructed for multi-level representation learning and multi-level self-supervised training to improve its capability of representation. Then, a fusion model is designed to fuse the multi-level spectral–spatial information to obtain a more distinctive coefficient matrix for self-expression, where the maximum entropy regularization (MER) method is employed to promote connectivity and the uniform dense distribution of band elements in each subspace. Finally, subspace clustering is conducted to obtain the final band subset. Experiments have been conducted on three hyperspectral datasets, and the corresponding results show that the proposed MLRLFMESC algorithm significantly outperforms several other band selection methods in classification performance.

1. Introduction

A hyperspectral image (HSI) is a unified image of interest targets captured by a specific wavelength of an optical sensor, consisting of hundreds of continuous bands with a fine resolution [1,2]. These bands are rich in available spectral and spatial information, allowing the identification of subtle differences between features. In recent years, due to the higher and higher spectral resolution of HSI, it has been successfully used in numerous applications, including military monitoring [3], food safety [4], medical diagnosis [5], etc. However, the strong inter-band correlation and high-dimensional hyperspectral data lead to a large amount of information redundancy, heavy computational and storage burdens, and dimensional catastrophe problems, which pose a great deal of difficulty and challenges to the development of HSI processing methods. Hyperspectral dimensionality reduction is of great significance as a form of preprocessing to address the above challenges.

The most commonly used methods for HSI dimensionality reduction to reduce spectral band redundancy include band selection (BS) [6,7] and feature extraction (FE) [8,9], where BS-based methods are based on the criteria of selecting the most representative band subset directly from the original HSI data without any transformation, and the obtained sub-bands are informative, distinguishable, and beneficial for subsequent tasks. Compared with FE, BS can reduce data dimensionality while preserving the physical meaning, inherent properties, and spectral characteristics of the original data, which is beneficial for interpreting the selected subset of bands in subsequent analysis and has been widely used in practical applications [10,11]. According to whether labeled information is used or not, the existing BS methods include three types as follows: supervised [12,13,14], semi-supervised [15,16,17], and unsupervised [18,19,20,21]. Owing to the fact that labeled HSI data are difficult to obtain, this paper focuses on unsupervised BS methods with the advantage of being more flexible, feasible, and effective in practice.

As a popular technique for unsupervised band selection, self-representation is implemented using the self-expression properties of data and different regularization constraints, where the most representative algorithms include fast and robust self-representation (FRSR) [22], robust dual graph self-representation (RDGSR) [23], and self-marginalized graph self-representation (MGSR) [24]. In addition, self-representation-based subspace clustering (SSC) has achieved a large number of successful results in unsupervised BS [25,26,27], where under the representation-based framework, the clustering structure of the spectral band can be learned in the low-dimensional subspace with the robustness of noise and outliers, and can effectively cluster high-dimensional data. However, in practical applications, the HSI data to be processed typically have great spectral variability and are located in nonlinear subspace, where traditional SSC-based BS methods with linear characteristics are not applicable for the nonlinear relationships of HSI, failing to achieve satisfactory performance.

Recently, deep neural networks have demonstrated superiority in handling high-dimensional data because of their remarkable ability to extract complex nonlinear relationships of features in an end-to-end learnable way. Deep learning models for HSI band selection have achieved tremendous success [28,29,30,31]. Although these methods are able to learn the complex nonlinear structure information of HSI, they ignore the correlation between bands, resulting in a large number of band redundancies. In view of this issue, deep learning-based clustering methods are introduced into band selection [32,33,34,35], considering the spatial information inherent in band images. However, they also have certain limitations. On the one hand, the representation learning of these models is embedded in the deep convolutional autoencoder, leading to a lack of an effective self-supervised representation ability; on the other hand, without considering the subspace clustering representation of the low-level and high-level information of the input HSI, these models ignore the meaningful multi-scale information embedded in different layers of deep convolutional autoencoders, imposing a waste of information that is conducive to clustering. In addition, due to the ignorance of the connectivity within the subspace, the existing models are blocked from further improving the clustering performance.

To solve the above problems, a self-supervised deep multi-level representation learning fusion-applying maximum entropy subspace clustering (MLRLFMESC) algorithm is proposed for BS in this paper, with the main contributions as follows:

(1)

Considering the multi-level spectral–spatial information of hyperspectral data, self-representation-based subspace clustering, comprising multiple fully connected layers, is respectively inserted between the encoder layers of the deep stacked convolutional autoencoder and its corresponding decoder layers, respectively, to realize multi-level representation learning (MLRL), which can fully extract low-level and high-level information and obtain more informative and discriminative multi-level representations.

(2)

Self-supervised information is provided to further enhance the representation capability of the MLRL, and a new auxiliary task is constructed for MLRL to perform multi-level self-supervised learning (MLSL). Furthermore, a fusion module is designed to fuse the multi-level spectral–spatial information extracted by the proposed MLRL to obtain a more informative subspace representation matrix.

(3)

To enhance the connectivity within the same subspace, the MER method is applied to ensure that the elements within the same subspace are uniformly and densely distributed, which is beneficial for subsequent spectral clustering.

The remainder of this paper is organized as follows. Section 2 gives a detailed description of the proposed MLRLFMESC algorithm. Section 3 presents the experiments and corresponding analysis of the proposed algorithm with other state-of-the-art BS methods. Finally, the conclusions and discussions are drawn in Section 4.

2. Proposed Method

This section describes in detail the proposed MLRLFMESC method for BS. The flowchart of this method is shown in Figure 1. The main steps are as follows: firstly, considering the low-level and high-level spatial information of hyperspectral data, the proposed method inserts multiple fully connected layers between the encoder layers and their corresponding decoder layers to achieve multi-level representation learning (MLRL), thus generating multiple sets of self-expression coefficient matrices at different levels of the encoder layers to obtain more informative and discriminative subspace clustering representations. Secondly, a new auxiliary task is constructed based on the MLRL, which provides multi-level self-supervised information to further enhance the representation ability of the model, termed multi-level self-supervision learning (MLSL). Finally, a fusion module is designed to integrate the multi-scale information extracted from different layers of representation learning to obtain a more differentiated self-expression coefficient matrix, where maximum entropy regularization (MER) is introduced to ensure that elements of the same subspace are evenly and densely distributed, thereby enhancing connectivity within each subspace and facilitating subsequent spectral clustering to determine the most representative band subset.

2.1. Multi-Level Representation Learning (MLRL)

The proposed MLRLFMESC method exploits a deep stacked convolutional autoencoder (SCAE) constructed by the structure of a symmetrical encoder–decoder as the core network for feature extraction to sufficiently extract the spectral–spatial information of HSI data. The original HSI data cube $\mathrm{X}={\left\{x_i\left|x_i\in {ℝ}^{W\times H}\right.\right\}}_{i=1}^B$ contains W × H spatial dimensions and B spectral band dimensions. To ensure that the input of HSI samples X can be constructed by the deep SCAE, the definition of the loss of function is as follows:

$${\mathcal{L}}_{SCAE}=\frac{1}{2}{‖\mathrm{X}-\hat{\mathrm{X}}‖}_F^2,$$

(1)

where the Frobenius norm is expressed as ${‖·‖}_F$, and the reconstructed HSI samples are denoted as $\hat{\mathrm{X}}$.

Self-representation-based subspace clustering assumes that all data points belong to a combination of linear or affinity subspaces, which is typically represented as a self-representative model. Let the spectral bands ${\left\{x_i\left|x_i\in {ℝ}^{W\times H}\right.\right\}}_{i=1}^B$ come from the union of n different subspaces ${{\displaystyle \cup }}_{j=1}^n\left\{S_j\right\}$ with the dimension ${\left\{d_j\right\}}_{j=1}^n$ in ${ℝ}^B$. In order to extract spatial information, considering the nonlinear relationships of HSI, the self-representative model is embedded in the latent space of the deep SCAE to implement self-representation properties and obtain representations of subspace clustering; then, the potential clusters are recovered with spectral clustering.

Inspired by the fact that the encoder with different layers can learn more complicated feature representations of the input HSI data, a self-representation model, comprising multiple fully connected layers, is inserted between the encoder layers of the deep stacked convolutional autoencoder and its corresponding decoder layers, respectively, to achieve MLRL; thus, the multi-level spectral–spatial information can be extracted. In order to capture the shared information between encoders and generate unique information for each layer, the consistency matrix ${\mathrm{C}}_C\in {ℝ}^{B\times B}$ and the discrimination matrix ${\left\{{\mathrm{D}}^l\right\}}_{l=1}^L\in {ℝ}^{B\times B}$ are defined, respectively. In view of the above-mentioned consideration, MLRL can be performed by the following loss function:

$${\mathcal{L}}_{exp}={\displaystyle \sum _{l=1}^L{‖{\mathrm{Z}}^l-{\mathrm{Z}}^l\left({\mathrm{C}}_C+{\mathrm{D}}^l\right)‖}_F^2} \mathrm{s}.\mathrm{t}. \mathrm{diag}\left({\mathrm{C}}_C+{\mathrm{D}}^l\right)=0,l\in \left\{1,\dots ,L\right\},$$

(2)

where $Z^l={\left[z_1^l,z_2^l,\dots ,z_B^l\right]}^T\in {ℝ}^{m\times B}$ represents the latent representation matrix and m is the dimension of the deep spatial feature.

The loss of self-expression ${\mathcal{L}}_{exp}$ is used to promote the learning of self-expression feature representations at different encoder levels. As for the discrimination matrix ${\left\{{\mathrm{D}}^l\right\}}_{l=1}^L$, the Frobenius norm is employed; thus, the connectivity of subspace representations related to each fully connected layer can be ensured. Meanwhile, to generate the sparse representation of the HSI, the l₁-norm is used in the consistency matrix ${\mathrm{C}}_C$. Accordingly, the regular terms added to the model are shown as follows:

$${\mathcal{L}}_{C_C}={‖{\mathrm{C}}_C‖}_1,$$

(3)

$${\mathcal{L}}_D={\displaystyle \sum _{l=1}^L{‖{\mathrm{D}}^l‖}_F^2},$$

(4)

The multi-level spectral–spatial information of HSI is obtained via MLRL to facilitate the feature learning process, thereby obtaining multiple sets of information representations accordingly.

2.2. Multi-Level Self-Supervised Learning (MLSL)

Aiming at the further improvement of the representation ability of the proposed model, MLSL is used as a self-supervised method to better learn self-expression feature representations by constructing auxiliary tasks for MLRL.

To perform MLSL, the auxiliary tasks are constructed as follows: Firstly, positive and negative sample pairs are formulated for the inputs and outputs of the MLRL. For a given input ${\mathrm{Z}}_i^l$ and its corresponding set of outputs ${\left\{{\mathrm{G}}_j^l={\mathrm{Z}}^l\left({\mathrm{C}}_C{}_j+{\mathrm{D}}_j^l\right)\right\}}_{j=1}^B$ at layer l of MLRL, ${\mathrm{Z}}_i^l$ and ${\mathrm{G}}_j^l$ are matched as a positive pair, while ${\mathrm{Z}}_i^l$ and ${\left\{{\mathrm{G}}_j^l\right\}}_{j=1,j\ne i}^B$ are treated as a negative pair. Subsequently, the MLSL is implemented by formulating a self-supervised loss function, expressed as follows:

$${\mathcal{L}}_S=-{\displaystyle \sum _{l=1}^L\log \frac{\exp \left({\left({\overrightarrow{\mathrm{Z}}}_i^l\right)}^T{\overrightarrow{\mathrm{G}}}_i^l/\sigma \right)}{{\displaystyle {\sum }_{j=1}^B\exp \left({\left({\overrightarrow{\mathrm{Z}}}_i^l\right)}^T{\overrightarrow{\mathrm{G}}}_j^l/\sigma \right)}}},$$

(5)

where σ is a temperature parameter that controls the distribution of the concentration level. ${\overrightarrow{\mathrm{Z}}}_i^l$ and ${\overrightarrow{\mathrm{G}}}_i^l$ are normalizations of ${\mathrm{Z}}_i^l$ and ${\mathrm{G}}_i^l$, respectively.

An l₂-normalization layer is used to satisfy $‖{\overrightarrow{\mathrm{Z}}}_i^l‖=1$ and $‖{\overrightarrow{\mathrm{G}}}_i^l‖=1$. The classifier of B-way softmax is exploited to classify ${\mathrm{Z}}_i^l$ as ${\mathrm{G}}_i^l$ in the loss function of MLRL.

2.3. Fusion Module with Maximum Entropy Regularization (MER)

Considering that the coefficient matrices of the information representations obtained from MLRL have multiple information of the input HSI data, it is preferable to fuse these matrices into a more discriminative and informative coefficient subspace representation matrix as the input of the subspace clustering.

The matrices ${\mathrm{C}}_C$ and ${\left\{{\mathrm{D}}^l\right\}}_{l=1}^L$ learned using MLRL are fused through the fusion module. Stacking ${\mathrm{C}}_C$ and ${\left\{{\mathrm{D}}^l\right\}}_{l=1}^L$ along the channel dimension can help acquire the stacked matrix ${\mathrm{C}}_S$. Then, the channels of ${\mathrm{C}}_S$ are merged using a convolutional kernel k to realize the fusion. Finally, a more informative subspace representation matrix ${\mathrm{C}}_F$ is obtained via channel fusion learning, expressed as follows:

$${\mathrm{C}}_F\in {ℝ}^{B\times B}=k\otimes {\mathrm{C}}_S,$$

(6)

where $\otimes$ is the convolution operation.

By using an appropriate kernel size k, ${\mathrm{C}}_F$ is able to capture more local information on ${\mathrm{C}}_C$ and each ${\left\{{\mathrm{D}}^l\right\}}_{l=1}^L$ with the block diagonal structure.

Entropy is a measure of uncertain information contained within a random variable, where for a discrete random variable X, the entropy can be calculated as $H\left(\mathrm{X}\right)=-{\displaystyle {\sum }_{y\in \mathrm{Y}}p\left(x\right)}\log p\left(x\right)$ with p(X) as the probability distribution function of X. The similarity between hyperspectral data samples i and j in the subspace representation matrix ${\mathrm{C}}_F$ can be expressed as $C_{F_{i,j}}$, and the MER method is applied to the subspace representation matrix. According to the fact that $\max H\left(\mathrm{C}\right)=\min -H\left(\mathrm{C}\right)$, the following loss function for ${\mathrm{C}}_F$ can be obtained as follows:

$${\mathcal{L}}_{C_F}={\displaystyle \sum _{i=1}^B{\displaystyle \sum _{j=1}^B{C}_{F_{i,j}}}\ln C_{F_{i,j}}} \mathrm{s}.\mathrm{t}. C_{F_{i,j}}\ge 0,$$

(7)

where $C_{F_{i,j}}=0$ satisfies $C_{F_{i,j}}\ln C_{F_{i,j}}=0$. The MER forces the equal strength of connections between elements from the same subspace. Simultaneously, it ensures a uniform dense distribution of elements belonging to the same subspace.

2.4. Implementation Details

The final loss function of the proposed MLRLFMESC method is expressed as follows:

$$\mathcal{L}\left(\theta ,\widehat{\theta },{\mathrm{C}}_F\right)={\mathcal{L}}_{SCAE}+{\lambda }_1{\mathcal{L}}_{\exp }+{\lambda }_2{\mathcal{L}}_{C_C}+{\lambda }_3{\mathcal{L}}_D+{\lambda }_4{\mathcal{L}}_S+{\lambda }_5{\mathcal{L}}_{C_F},$$

(8)

where ${\lambda }_1,{\lambda }_2,{\lambda }_3,{\lambda }_4,{\lambda }_5$ are parameters that balance the trade-off between the earlier-mentioned different losses. $\theta$ and $\widehat{\theta }$ are the parameters updated by the standard backpropagation of the network trained with the Adam gradient.

Once the network has been trained to obtain the matrix ${\mathrm{C}}_F$, a symmetric affinity matrix can be created for spectral clustering,

$$\mathrm{A}=\frac{1}{2}\left(\left|{\mathrm{C}}_F\right|+{\left|{\mathrm{C}}_F\right|}^T\right),$$

(9)

Matrix A shows the pairwise relationship between the bands. Given the above, the spectral clustering algorithms can be utilized to recover the underlying subspaces and cluster the samples into respective subspaces to obtain the clustering results. The clustering centers can be obtained via the average of the spectrum in each cluster. Then, the distances between the cluster center and each band in the same cluster can be calculated to find the closest band as the selected representative band of the cluster, and the final subset of bands can be obtained.

3. Experiments and Results

3.1. Hyperspectral Datasets

Comparative experiments are conducted on three publicly available HSIs with different scenarios to prove the effectiveness of the proposed algorithm, including the Indian Pines (IP) dataset, the Pavia University (PU) dataset, and the Salinas (SA) dataset. Detailed descriptions of the three hyperspectral datasets are given in

Table 1. Descriptions of three HIS datasets.

Descriptions	IP	PU	SA
Size	145 × 145	610 × 340	512 × 227
Bands	200	103	204
Classes	16	9	16
Samples	10,249	42,776	54,129
Resolution	20 m	1.3 m	3.7 m
Sensors	AVIRIS	ROSIS	AVIRIS
Wavelength	0.4–2.5 μm	0.43–0.86 μm	0.36–2.5 μm

.

3.1.1. Indian Pines (IP) Dataset

The IP dataset was captured using the AVIRIS sensor in Northwestern Indiana on 12 June 1992, containing 220 spectral bands with wavelengths of 0.4 to 2.5 μm and containing 145 × 145 pixels with a spatial resolution of 20 m. After the removal of 20 water absorption bands, 200 bands with 16 classes of crops are left for experiments. Figure 2 shows the IP dataset’s pseudo-color map as well as the true image feature class distribution.

3.1.2. Pavia University (PU) Dataset

The PU dataset was collected via the ROSIS sensor for the city of Pavia in North Italy during a flying activity in 2003, consisting of 115 bands with wavelengths of 0.43 to 0.86 μm and containing 610 × 340 pixels with a spatial resolution of 1.3 m. After the removal of 12 noise bands, there are 9 types of objects available in the remaining 103 bands (the types of objects are shown in the label of Figure 3). The pseudo-color map and real image feature class distribution of the PU dataset are shown in Figure 3.

3.1.3. Salinas (SA) Dataset

The SA dataset was gathered using the AVIRIS sensor in Salinas Valley, California, in 1998, which comprises 224 bands with wavelengths of 0.36 to 2.5 µm and contains 512 × 217 pixels with spatial resolution of 3.7 m. After deleting 20 water absorption bands, the remaining 204 bands containing 16 classes are utilized for experiments. The pseudo-color map and real image feature class distribution of the SA dataset are shown in Figure 4.

3.2. Experimental Setup

In order to prove the effectiveness of the proposed BS algorithm, five existing BS methods are used for comparison. Considering search-based, clustering-based, and ranking-based, as well as the comparison between traditional methods and deep learning methods, the selected comparable algorithms are UBS [18], E-FDPC [19], ISSC [25], ASPS_MN [5] and DSC [32], as these algorithms have open-source implementations provided by their respective authors. Performance analysis is conducted on a subset of bands obtained from various BS approaches using the same SVM classifier in [36] with an open-source code.

To quantitatively assess the quality of the selected band subsets, indicators of classification accuracy are used in this section, including overall accuracy (OA), average accuracy (AA), and the kappa coefficient (Kappa). For a fair comparative purpose, identical training and testing data subsets within each round are utilized when evaluating different BS algorithms. Specifically, 10% of samples from each class are randomly chosen as the training set, with the remaining samples allocated to the testing set. The experimental results are averaged via ten independent runs to reduce the randomness.

The deep SCAE network of the proposed method is composed of three symmetric layers of the stacked encoder and decoder with the following parameter settings: the stacked encoder consists of 10, 20, and 30 filters with corresponding kernel sizes of 5 × 5, 3 × 3, and 3 × 3, respectively. The network learning rate is $1.0\times {10}^{-4}.$ The trade-off parameters are ${\lambda }_1=1.0\times {10}^{-1}$, ${\lambda }_2=1.0\times {10}^{-2}$, ${\lambda }_3=1.0\times {10}^{-2}$, ${\lambda }_4=1.0\times {10}^{-2},$ and ${\lambda }_5=1.0\times {10}^{-2}$. The kernel size k is 3 × 3.

3.3. Randomness Validation by Random Selection of Training and Testing Sets

As for classification results, the training and testing sets have a significant influence on the classification performance. Therefore, before the comparison of various algorithms, this section gives the average and variance of OA across 10 distinct runs, allowing the assessment of disparities between experiment runs where each run involves alterations to the training and testing sets.

Figure 5a–c illustrates the box plots depicting the OA results for the six algorithms utilizing 35 bands across the three HSI datasets in 10 separate runs. The experiments entail the repeated random selection of training and testing datasets. To ensure an equitable comparison, the training and testing sets for all six algorithms within the same round remain identical. Sub-figures demonstrate that the proposed MLRLFMESC obtains the optimal mean OA with a perfect criteria bias, especially for IP and PU datasets, as shown in Figure 5a–c.

3.4. Ablation Study of the Proposed MLRLFMESC Method

To separately verify the effectiveness of maximum entropy regularization and self-supervised learning in the proposed MLRLFMESC method, ablation studies are conducted in this section. As can be shown in Equation (8) in Section 2, the proposed method adopts partial loss functions from the DSC method, namely ${\mathcal{L}}_{SCAE}$, ${\mathcal{L}}_{\exp }$, ${\mathcal{L}}_{C_c}$, and ${\mathcal{L}}_D$, collectively referred to as ${\mathcal{L}}_{DSC}$ in this paper. Building upon this foundation, the proposed method introduces the following two innovative techniques: the multi-level self-supervised learning model (MLSL) and maximum entropy regularization (MER). In MLSL, the ${\mathcal{L}}_S$ is introduced to obtain improved self-supervised features. In MER, the ${\mathcal{L}}_{C_F}$ is introduced to promote connectivity within the same subspace while ensuring a uniform and dense distribution of elements within the subspace, which is beneficial for subsequent spectral clustering. As a result, the ablation study should be implemented with/without the ${\mathcal{L}}_S$ and the ${\mathcal{L}}_{C_F}$ loss in

Table 2. Ablation study for maximum entropy regularization and self-supervised learning.

Ablation Components			Number of Selected Bands (OA Values)
${\mathcal{L}}_{\mathit{D}\mathit{S}\mathit{C}}({\mathcal{L}}_{\mathit{S}\mathit{C}\mathit{A}\mathit{E}},{\mathcal{L}}_{\mathbf{exp}},{\mathcal{L}}_{{\mathit{C}}_{\mathit{c}}},{\mathcal{L}}_{\mathit{D}})$	${\mathcal{L}}_{{\mathit{C}}_{\mathit{F}}}$	${\mathcal{L}}_{\mathit{S}}$	5	10	15	20	25
√	×	×	70.67 ± 1.43	79.16 ± 1.28	81.20 ± 0.65	82.98 ± 0.42	83.88 ± 0.52
√	√	×	73.02 ± 1.40	81.19 ± 1.64	82.77 ± 1.95	84.41 ± 2.54	85.09 ± 1.33
√	×	√	73.16 ± 1.48	80.94 ± 0.93	83.44 ± 0.08	84.72 ± 0.63	84.95 ± 0.86
√	√	√	74.15 ± 1.59	82.90 ± 1.58	85.05 ± 1.25	85.77 ± 1.61	86.00 ± 1.79

.

The following conclusions are given from Table 2: Firstly, when adding either the MER into the model (as shown in the second line in Table with the form of ${\mathcal{L}}_{C_F}$) or the multi-level self-supervised learning model (as shown in the third line in the Table with the form of ${\mathcal{L}}_S$), the OA performance can be better than the DSC method. The classification performance of the proposed MLRLFMESC with both MER and MLSL (as shown in the fourth line in Table with the form of ${\mathcal{L}}_S$ and ${\mathcal{L}}_{C_F}$) has the best OA performance with bold fonts under the selection with a different number of selected band subsets. These experiments demonstrate the effectiveness of maximum entropy regularization and self-supervised learning in the proposed MLRLFMESC approach.

3.5. Classification Results Analysis for Different BS Algorithms

In this section, comparative experiments are conducted on three publicly available HSIs with different scenarios to prove the effectiveness of the proposed algorithm, including the Indian Pines (IP) dataset, the Pavia University (PU) dataset, and the Salinas (SA) dataset. However, limited by the number of pages and considering the reproducibility of conclusions, only the results of the IP dataset are shown in this section, with the results of the PU and SA datasets in Appendix A and Appendix B.

3.5.1. BS Results with Different Number of Selected Bands

To evaluate the classification accuracy of the proposed MLRLFMESC algorithm while comparing it with the existing BS methods, the quantity n of selected bands in different BS methods is varied in the region of [5,35] with a step of five. The reason why we chose 35 as the maximum is that when using virtual dimension (VD) analysis, which is a widely used technique for selecting the number of bands, the VD is typically less than 35.

In Figure 6, the suggested MLRLFMESC technique clearly outperforms the other five BS approaches in terms of classification accuracy for OA, AA, and Kappa. In Figure 6a of the IP dataset, the proposed MLRLFMESC has the highest and most stable OA with a significant improvement over the other five comparable BS methods, especially when n = 5, 30, 35, and the proposed MLRLFMESC has better accuracies of 3.42%, 3.56%, and 2.83% compared to the suboptimal approach. The AA accuracy is shown in Figure 6b, and the MLRLFMESC technique exhibits the “Hughes” phenomena that grows and then drops as the number of bands increases. In terms of the Kappa given in Figure 6c, the MLRLFMESC approach has a growing advantage over the suboptimal approach as the number of chosen bands grows, notably when n = 30, 35, MLRLFMESC has 3.56% and 2.83% greater accuracy than the suboptimal approach, respectively. The other five BS approaches exhibit some “Hughes” phenomena in the OA, AA, and Kappa curves, indicating the necessity for band selection.

3.5.2. Classification Performance Analysis by Band Subsets Using Various BS Algorithms

To further assess the efficacy of the proposed MLRLFMESC approach for the analysis, Figure 7 shows the index distribution of the 30 bands selected from the IP dataset via various band selection approaches. It is usually regarded as a bad strategy if the selected bands are scattered across a relatively short range.

As can be seen in Figure 7, most band selection approaches pick bands that span a broad range of all bands.

Figure 8 presents the labeled image, the classification result maps for the various band selection methods, and all bands, which may be used to visually analyze the classification performance of various band selection methods on the IP dataset. There is some visual deviance, especially regarding the fact that the proposed algorithm has the best performance of the corn-min class with the orange color in the lower left corner and the better performance of the wood class with the brown color in the lower right corner. Further quantitative analysis is conducted for a fair comparison.

In order for a quantitative comparison,

Table 3. Comparison of classification results from various BS approaches on the IP dataset.

Algorithms	UBS	E-FDPC	ISSC	ASPS_MN	DSC	MLRLFMESC	All Bands
OA	82.98 ± 1.37	83.85 ± 1.53	81.52 ± 1.15	83.39 ± 1.27	83.72 ± 1.51	86.95 ± 1.09	88.43 ± 1.24
AA	81.86 ± 2.67	83.48 ± 1.73	81.55 ± 3.80	82.65 ± 2.18	82.11 ± 3.97	86.43 ± 2.87	86.69 ± 3.23
Kappa	80.06 ± 1.58	81.09 ± 1.67	78.35 ± 1.36	80.53 ± 1.49	80.91 ± 1.80	84.68 ± 1.27	86.42 ± 1.47
1	76.30 ± 25.74	85.47 ± 15.58	85.26 ± 34.62	81.20 ± 24.44	78.16 ± 36.84	86.28 ± 23.90	91.89 ± 8.11
2	76.49 ± 4.96	77.40 ± 5.65	76.78 ± 4.17	75.85 ± 3.22	78.58 ± 3.31	82.45 ± 3.23	84.83 ± 1.95
3	73.75 ± 7.15	77.19 ± 6.32	71.97 ± 4.14	75.46 ± 7.01	71.73 ± 10.16	80.19 ± 4.80	83.54 ± 2.35
4	61.55 ± 10.41	61.71 ± 13.52	60.35 ± 7.00	60.51 ± 8.64	56.46 ± 10.96	68.03 ± 13.12	66.93 ± 3.82
5	79.61 ± 8.40	87.84 ± 5.70	81.81 ± 8.37	86.96 ± 6.40	88.57 ± 7.68	89.94 ± 7.35	92.48 ± 3.06
6	95.61 ± 3.33	93.65 ± 2.78	91.36 ± 5.34	94.05 ± 2.75	95.58 ± 2.79	96.47 ± 3.31	96.10 ± 1.42
7	92.80 ± 18.75	86.72 ± 27.27	93.18 ± 20.00	93.27 ± 14.29	87.56 ± 25.00	93.24 ± 14.29	87.73 ± 5.13
8	96.98 ± 2.64	98.29 ± 2.92	94.76 ± 3.51	97.42 ± 1.77	97.43 ± 3.95	98.53 ± 2.12	97.19 ± 1.83
9	63.60 ± 38.96	77.04 ± 34.44	76.29 ± 50.00	63.45 ± 40.00	67.75 ± 44.64	75.43 ± 34.34	71.39 ± 18.61
10	73.09 ± 4.83	73.26 ± 6.68	69.52 ± 2.84	74.68 ± 2.33	74.67 ± 4.87	77.00 ± 4.61	80.28 ± 1.91
11	86.69 ± 3.09	87.79 ± 4.41	85.33 ± 2.83	86.28 ± 2.25	87.46 ± 3.62	89.38 ± 3.08	91.93 ± 1.18
12	78.99 ± 9.31	78.34 ± 8.03	75.92 ± 11.73	81.29 ± 9.21	75.12 ± 7.63	83.69 ± 9.58	81.97 ± 3.31
13	98.10 ± 3.66	96.52 ± 6.93	94.52 ± 5.75	95.62 ± 4.92	96.97 ± 8.07	97.91 ± 4.49	97.45 ± 2.55
14	96.86 ± 1.77	96.15 ± 1.83	96.48 ± 1.64	96.43 ± 1.72	96.89 ± 1.75	97.11 ± 1.33	97.50 ± 0.94
15	63.32 ± 18.33	63.06 ± 15.95	57.77 ± 10.63	64.37 ± 16.38	66.52 ± 13.70	71.87 ± 10.09	68.07 ± 6.64
16	95.95 ± 7.69	95.27 ± 5.25	93.49 ± 9.20	95.53 ± 12.20	94.26 ± 12.20	95.34 ± 9.30	97.77 ± 2.23

displays the average classification results of 10 runs with OA, AA, and Kappa, using various band selection methods (UBS, E-FDPC, ISSC, ASPS_MN, DSC, and the proposed MLRLFMESC, respectively) and all bands, where the best classification results are blacked out and the second-best results are underlined.

It can be easily seen from Table 3 that the proposed method outperforms all previous band selection methods except all bands in terms of OA, AA, and Kappa, as well as classification accuracy in most classes. The proposed MLRLFMESC approach provides the highest classification accuracy in classes 8, 12, and 15 when compared to existing approaches.

3.6. Time Consuming for Different BS Algorithms

In this section, a comparison of computing times across different band selection algorithms is conducted to discern their respective computational complexity.

Table 4. Comparison of time consumption from various BS approaches on the IP dataset.

Algorithms	MLRLFMESC	DSC	ASPS_MN	ISSC	E-FDPC	UBS
Time (s)	26.07	39.81	0.54	0.49	1.54	N/A

lists the average computation time for ten runs of various BS approaches on the IP dataset, where the ISSC has the lowest running time with bold font. Since MLRLFMESC and DSC use deep neural networks, the running time is longer compared to traditional BS methods. However, the runtime of MLRLFMESC is significantly less than that of DSC, efficiently obtaining the desired bands within an acceptable timeframe while achieving a superior classification performance.

4. Conclusions and Discussion

This paper presents a novel MLRLFMESC framework for unsupervised hyperspectral band selection. Self-representation subspace clustering is applied in deep SCAE to enable the learning of hyperspectral nonlinear spectra–spatial relationships in a trainable deep network.

(1)

From the results in Section 3, it can be seen that the proposed MLSL model retains good band subsets with multi-level spectral–spatial information and multi-level discriminative information representations.

(2)

A fusion module is employed to fuse the multi-level discriminative information representations, where the MER method is applied to enhance the objectiveness of the bands in each subspace while ensuring the uniform and dense distribution of bands in the same subspace, which was shown to be successful in the ablation study.

(3)

Comparable experiments indicate that the proposed MLRLFMESC approach performs better than the other five state-of-the-art BS methods on three real HSI datasets for classification performance.

Although this work, and even other existing BS methods, have been extensively studied and achieved excellent performance in classification tasks, there has been limited research on integrating BS algorithms into tasks such as hyperspectral image target detection, target tracking, unmixing, etc. An important research direction in the future is to apply band selection algorithms to different task requirements while considering high performance. Furthermore, this paper does not pay attention to the problem of the uneven distribution of samples in classification-oriented tasks, resulting in the better performance of large categories and the poor performance of small categories. Therefore, the expansion of small category samples and the design or improvement of targeted models are the direction of focus in further research.