Multi-Stage Convolutional Broad Learning with Block Diagonal Constraint for Hyperspectral Image Classification

Abstract

By combining the broad learning and a convolutional neural network (CNN), a block-diagonal constrained multi-stage convolutional broad learning (MSCBL-BD) method is proposed for hyperspectral image (HSI) classification. Firstly, as the linear sparse feature extracted by the conventional broad learning method cannot fully characterize the complex spatial-spectral features of HSIs, we replace the linear sparse features in the mapped feature (MF) with the features extracted by the CNN to achieve more complex nonlinear mapping. Then, in the multi-layer mapping process of the CNN, information loss occurs to a certain degree. To this end, the multi-stage convolutional features (MSCFs) extracted by the CNN are expanded to obtain the multi-stage broad features (MSBFs). MSCFs and MSBFs are further spliced to obtain multi-stage convolutional broad features (MSCBFs). Additionally, in order to enhance the mutual independence between MSCBFs, a block diagonal constraint is introduced, and MSCBFs are mapped by a block diagonal matrix, so that each feature is represented linearly only by features of the same stage. Finally, the output layer weights of MSCBL-BD and the desired block-diagonal matrix are solved by the alternating direction method of multipliers. Experimental results on three popular HSI datasets demonstrate the superiority of MSCBL-BD.

1. Introduction

With the rapid development of remote sensing technology, the spectral and spatial resolutions of hyperspectral images (HSIs) are increasing. HSIs can be used to identify subtle differences among ground objects, demonstrating strong discriminative ability [1,2], which has been applied in many fields such as crop monitoring [3], environmental analysis and prediction [4], climate detection [5], and forest surveys [6]. Pixel-wise classification is one of the common tasks in these applications. The commonly used classification methods include support vector machine (SVM) [7,8], and k-nearest neighbor [9]. However, due to the large number of bands and the strong correlation between adjacent bands, band redundancy exists in HSI data [10,11]. Direct classification with the original HSI will result in lower classification accuracy. Therefore, before HSI classification, performing feature extraction [12] (mapping the original data to another feature space through the mapping matrix) or feature selection (directly selecting several bands from the original band according to certain criteria or strategies) [13] can effectively improve the classification accuracy. Many works have focused on feature learning, such as manifold learning-based feature extraction [14,15] and some extended versions [16,17], metric learning-based dimensionality reduction [18,19], filtering-based feature learning [20,21] and multi-feature-based feature fusion [22,23].

The recent proposed broad learning system (BLS) [24,25] can be viewed as a three-layer forward neural network consisting of an input layer, an intermediate layer, and an output layer. The intermediate layer includes two parts: the mapped feature (MF) part, which is obtained by mapping the input with randomly generated or sparse-autoencoder-optimized weights; the enhancement node (EN) part achieves width expansion by mapping MF with randomly generated weights. The BLS has strong function approximation ability with a very flexible structure [26]. For example, Liu et al. [24] extended it to radial basis function network, which was effective and efficient in classification. By combining BLS and the Takagi–Sugeno fuzzy subsystem, a fuzzy BLS method was proposed [27], which can achieve better performance in terms of regression and classification tasks than the commonly used neuro-fuzzy and non-fuzzy methods. Chen et al. [26] provided a mathematical proof of the general approximation of BLS and various structural changes of BLS. Kong et al. [28] proposed a semi-supervised BLS by using the class probability framework and further applied it to the HSI classification task.

In recent years, deep learning [29] has been widely used in HSI classification because of its strong nonlinear mapping ability and end-to-end working mode, which can automatically learn the features with strong robustness [30,31] and the underlying regularities of samples [32]. For example, Chen et al. [33] used a multilayer autoencoder for HSI classification for the first time. Subsequently, Chen et al. [34] proposed an HSI classification method based on the spectral-spatial deep belief network (SS-DBN). Based on the autoencoder and deep belief network (DBN), many improved methods have been proposed, such as spatial updated deep autoencoder [35], diversified DBN model [36], and group belief network [37]. Boththe deep autoencoder and DBN belong to the fully connected neural networks, which have a large amount of parameters, and the input data generally require a 1D vector. However, the original HSI data are in the form of a 3D tensor. Vectorizing the original 3D HSI data into a 1D form not only destroys the inherent structure of HSI data but also leads to an increase in dimensionality. As an important deep learning model, the convolutional neural network (CNN) can greatly reduce the number of parameters and reduce the difficulty of network training by utilizing the local connection and weight-sharing mechanism. To this end, many CNN-based HSI classification methods have been proposed. Romero et al. [38] proposed an unsupervised layer-wise pre-trained CNN to extract the multi-layer sparse features from HSIs. Makantasis et al. [39] first applied kernel principal component analysis to the original HSI data to obtain its low-dimension tensor representation, and then used it as the input of CNN to realize spectral-spatial feature extraction and the classification of HSIs. Chen et al. [40] extracted the spectral, spatial, and spectral-spatial features of HSIs using 1D, 2D, and 3D CNNs, respectively. Zhao and Du [41] proposed the spectral-spatial feature-based classification method to extract the spectral and spatial features from HSIs by using balanced local discriminant embedding and CNN, respectively.

The training of a CNN requires a large number of labeled samples. However, the acquisition of labeled HSI samples is expensive, and only a subset with a quite small sample size can be used for training. Therefore, training CNNs with a small number of labeled HSI samples is the primary problem to be solved. To this end, numerous works have been conducted. For example, Ghamisi et al. [42] proposed a self-improving CNN by combining fractional-order Darwinian particle swarm optimization and CNN to address the problems of dimensionality and limited labeled samples. Santara et al. [43] designed an end-to-end CNN called band-adaptive spectral-spatial feature learning neural network (BASS-Net), which consists of three parts named band selection and segmentation, parallel network, and classification. Compared with the conventional CNN, BASS-Net combines the feature selection and parallel connection of the network, which greatly reduces the amount of parameters of the network. Therefore, BASS-Net can achieve better performance than CNN in situations with limited labeled samples. Chan et al. proposed a simplified CNN structure named principal component analysis network (PCANet) [44], which has attracted attention due to its simple training process and not needing a large number of iterative processes. Furthermore, Pan et al. [45] used kernel PCA to replace PCA for convolutional kernel learning to solve the problem of the insufficient nonlinear mapping ability in PCANet. By combining the rolling guidance filter and vertex component analysis network, Pan et al. [46] proposed a simplified deep-learning model for HSI classification. In addition to the above work, sample expansion is another commonly used solution to address the issue of insufficient labeled samples. For example, Li et al. [47] proposed a CNN with pixel-pair features (CNN-PPF), constructing a new training set with many more labeled samples than the original training set by comparing the labels between every two samples.

In summary, CNN has strong feature representation ability, but when labeled samples are limited, this ability is limited to some extent. The structure of BLS is simple and flexible, but the used linear mapped features cannot fully express HSI data. To this end, we constructed a novel multi-stage convolutional broad learning with a block-diagonal constraint (MSCBL-BD) method for HSI classification by taking full advantage of CNN and BLS. The main contributions of our work are summarized as follows: (1) By combing CNN and BLS, their advantages can be simultaneously utilized. When labeled samples are limited, the training set cannot characterize the complete distribution of HSIs. Although features extracted by CNN can provide strong discriminative ability, they may overfit to the training set. The concatenation of convolutional features and broad features can be seen as the combination of fine- and coarse-designed features, which have stronger generalization ability than either of them alone. (2) After the multi-layer mapping of the CNN, some information of the original HSI is inevitably lost. Therefore, multi-stage convolutional features are utilized and expanded stage-by-stage to mitigate the information loss to a certain extent. (3) Due to the use of width expansion and multi-stage features, the similarity between the features of different stages may be improved accordingly. This results in the redundancy of features. Therefore, we use a block-diagonal matrix to impose constraints on the multi-stage convolutional broad features to enhance the independence between the convolutional broad features of different stages, which is helpful to seek diversity in features and learn a more accurate HSI classification model.

The rest of this paper is organized as follows: We elaborate the proposed MSCBL-BD for HSI classification in Section 2. Experiments on three popular hyperspectral datasets are described in Section 3, followed by the discussion of the proposed method in Section 4. The conclusions are provided in Section 5.

2. MSCBL-BD for HSI Classification

2.1. Structure of MSCBL-BD

The structure of MSCBL-BD is shown in Figure 1, which mainly includes the following parts: (1) obtaining the low-dimensional neighboring region representation, and using PCA to perform band reduction on the original HSI and constructing the low-dimensional neighboring region representation; (2) extracting the multi-stage CBFs. Firstly, use the limited labeled samples to pre-train a three-stage CNN and extract the multi-stage convolutional features (CFs) of the HSI. Secondly, perform channel-wise global average pooling on the features of the first two stages. Thirdly, perform stage-wise width expansion on the CFs to obtain the broad features (BFs), then further combine the CFs and BFs to obtain the convolutional broad features (CBFs). (3) Imposing the block-diagonal constraints on the multi-stage CBFs. Map the CBFs of all three stages through a block-diagonal matrix to obtain the block-diagonal-constrained CBFs, which ensures that each CBF is only linearly represented by features of the same stage. The optimal solutions of the output layer weight and block-diagonal matrix can be found by the alternating direction method of multipliers (ADMM).

2.2. CNN Pre-Training

The original hyperspectral image is presented in the shape of a 3D cube. If vectorization is performed directly on the HSI, it not only leads to an increase in dimensions, but also destroys the inherent structure of HSI data. The neighboring region representation is a common spectral-spatial representation method in HSIs [48], which is constructed by selecting several pixels around the target pixel. Figure 2 shows a schematic diagram of selecting the surrounding 24 pixels to form a $5\times 5\times B$-size neighboring region representation, where B denotes the number of bands in the original HSI. According to this representation, not only the band information of the target pixel but also the information of neighboring pixels can be obtained. Furthermore, if the original high-dimensional neighboring region representation is directly used as the input of the CNN, redundant information will exists in the band and the number of network parameters will increase dramatically, thereby affecting the final performance of the CNN. A common method is utilizing a dimensionality reduction technique, such as principal component analysis (PCA), to reduce the number of bands in the neighboring region representation of an HSI. Define a low-dimensional neighboring region representation $\mathit{\chi }\in {\mathbb{R}}^{d_1\times d_2\times d_3}$ as the input data of the CNN, where $d_1$, $d_2$, and $d_3$ are the width, height, and number of bands, respectively.

Due to the large amount of parameters of the fully connected layer, the CNN used in our work only includes the convolutional, pooling, nonlinea, and SoftMax layers. The input is connected to the convolutional layer by the convolutional kernel to obtain the output feature maps. The calculation formula is:

$${\mathit{F}}^{\mathrm{C}}={\mathit{I}}^{\mathrm{C}}\ast {\mathit{K}}^{\mathrm{C}}+{\mathit{b}}^{\mathrm{C}}$$

(1)

where ${\mathit{b}}^{\mathrm{C}}$ is the bias, ${\mathit{F}}^{\mathrm{C}}$ represents the output features of the convolution layer, and ${\mathit{I}}^{\mathrm{C}}$ is the input of the convolutional layer. For the first layer, the input data is the neighboring region representation of the HSI, which is denoted as ${\mathit{I}}_1^{\mathrm{C}}=\mathit{\chi }$. ${\mathit{K}}^{\mathrm{C}}$ indicates the convolutional kernel of the convolutional layer and * represents the convolutional operation. In general, a pooling layer is added after a convolutional layer, which aims to quickly reduce the dimensions and enhance the invariance of the extracted features. The features ${\mathit{F}}^{\mathrm{P}}$ obtained by the pooling layer are:

$${\mathit{F}}^{\mathrm{P}}=\mathrm{down}\left({\mathit{I}}^{\mathrm{P}}\right)$$

(2)

where $\mathrm{down}(·)$ indicates a max-pooling operation and ${\mathit{I}}^{\mathrm{P}}$ is the input data of the pooling layer, which is also the output of the previous convolutional layer. To achieve nonlinear mapping, a convolutional layer or a pooling layer is typically connected to a nonlinear layer. Here, the activation function of the nonlinear layer is the sigmoid function:

$${\mathit{F}}^{\mathrm{N}}=1/1+exp(-{\mathit{I}}^{\mathrm{N}})$$

(3)

where ${\mathit{I}}^{\mathrm{N}}$ is the input data of the nonlinear layer, which is also the output of the previous pooling layer; ${\mathit{F}}^{\mathrm{N}}$ denotes the output nonlinear features of the nonlinear layer. The SoftMax function is generally used as the last layer of the CNN. The number of neurons in the SoftMax layer is equal to the number of classes. The SoftMax loss function is defined as [49]:

$$J({\mathit{W}}^{\mathrm{S}},{\mathit{b}}^{\mathrm{S}})=-\frac{1}{N}\sum _{i=1}^N\sum _{j=1}^{C}1\{y_i=j\}log\frac{e^{{\mathit{W}}_j^{\mathrm{S}}{\mathit{I}}_i^{\mathrm{S}}+{\mathit{b}}_j^{\mathrm{S}}}}{{\sum }_{l=1}^C{e}^{{\mathit{W}}_l^{\mathrm{S}}{\mathit{I}}_i^{\mathrm{S}}+{\mathit{b}}_l^{\mathrm{S}}}}$$

(4)

where N denotes the number of training samples; C is the number of classes; $y_i$ is the label of the ith sample; ${\mathit{W}}^{\mathrm{S}}$ and ${\mathit{b}}^{\mathrm{S}}$ are the weight and bias of the SoftMax layer, respectively; ${\mathit{I}}^{\mathrm{S}}$ is the input data of the SoftMax layer, which is obtained by the previous multiple convolutional, pooling, and nonlinear calculation procedures; and $1\{·\}$ is the indicator function, so that $1\left\{atruestatement\right\}=1,and1\left\{afalsestatement\right\}=0$. The CNN training process consists of two parts: forward and backward calculations. In the forward calculation process, the output of each layer is calculated according to the current parameters. In the backward calculation process, the weight and bias of each layer are updated by minimizing the loss function. The batch stochastic gradient descent algorithm is used for weight and bias update.

Here, one convolutional layer plus one pooling layer plus two nonlinear layers plus one $1\times 1$ convolutional layer is defined as one stage, and the pre-trained CNN can obtain the multi-stage convolutional feature:

$${\mathit{F}}_i^{\mathrm{stage}}=f\left(\mathit{\chi }|{\mathit{\theta }}_i\right)$$

(5)

where ${\mathit{F}}_i^{\mathrm{stage}}$ represents the ith stage feature, $i=1,\cdots ,s$; and s is the total number of stages, ${\mathit{\theta }}_i=\{{\mathit{K}}_1^{\mathrm{C}},{\mathit{b}}_1^{\mathrm{C}},\cdots ,{\mathit{K}}_i^{\mathrm{C}};{\mathit{b}}_i^{\mathrm{C}}\}$ denotes the learnable parameter set from the input to the ith stage of the CNN, including convolutional kernels and biases; and $f\left(·\right)$ denotes the nonlinear mapping procedure of th CNN under ${\mathit{\theta }}_i$.

2.3. MSCBL-BD

The conventional BLS can be regarded as a three-layer neural network, including an input layer, an intermediate layer (composed of an MF and an EN), and an output layer. The MF is obtained by mapping the input with the weights fine-tuned by the linear autoencoder. It is worth noting that the linear features obtained by linear mapping cannot fully express the complex spectral-spatial features of an HSI, thus affecting the final classification performance. The EN is obtained by mapping the MF with random weights to achieve the width expansion of the MF. The output layer is connected to both the MF and the EN. By minimizing the error between the output vector and the ground truth label vector, the objective function of BLS is constructed by:

$$\underset{{\mathit{W}}_{\mathrm{B}}^m}{min}{∥\left[{\mathit{Z}}_{\mathrm{B}}|{\mathit{H}}_{\mathrm{B}}\right]{\mathit{W}}_{\mathrm{B}}^m-\mathit{Y}∥}_2^2+{\lambda }_1{∥{\mathit{W}}_{\mathrm{B}}^m∥}_2^2$$

(6)

where the first term is the empirical risk, which aims to calculate the error between the model output vector and the real label vector; $\mathit{Y}$ denote the labels of the samples; ${∥·∥}_2$ is the $l_2$-norm; the second term is the structural risk, which is used to improve the generalization ability of the model; ${\mathit{Z}}_{\mathrm{B}}$ and ${\mathit{H}}_{\mathrm{B}}$, respectively, represent the features of the MF and EN; ${\mathit{W}}_{\mathrm{B}}^m$ indicates the connection weights of the output layer; and ${\lambda }_1$ is the coefficient of structural risk term. Equation (6) can be solved by ridge regression theory.

Since the linear sparse feature extracted by BLS cannot fully characterize the complex spectral-spatial features of an HSI, the CFs are used to replace the linear sparse features in the MF to achieve more complex nonlinear mapping. Furthermore, in order to reduce the information lost by the CNN in the multi-layer mapping process, the multi-stage features extracted by the CNN are utilized here and separately expanded in width. Given the multi-stage features $\left[{\mathit{F}}_1^{\mathrm{stage}}|\cdots |{\mathit{F}}_{s-1}^{\mathrm{stage}}|{\mathit{F}}_s^{\mathrm{stage}}\right]$, the channel-wise global average pooling is first performed on the CFs of the first two stages:

$${\mathit{P}}^{\mathrm{stage}}=\mathrm{dow}{\mathrm{n}}^{\mathrm{G}}\left({\mathit{F}}^{\mathrm{stage}}\right)$$

(7)

where $\mathrm{dow}{\mathrm{n}}^{\mathrm{G}}(·)$ represents the channel-wise global average pooling, which can summarize the global information of the feature map in each channel, to some extent. After the pooling operation, several 1D vectors ${\mathit{P}}^{\mathrm{stage}}=\left[{\mathit{P}}_1^{\mathrm{stage}}|\cdots |{\mathit{P}}_{s-1}^{\mathrm{stage}}\right]$ are obtained and, together with the CFs of the last stage, constitute the multi-stage CFs (MSCFs) ${\mathit{F}}^{\mathrm{stage}}=\left[{\mathit{P}}_1^{\mathrm{stage}}|\cdots |{\mathit{P}}_{s-1}^{\mathrm{stage}}|{\mathit{F}}_s^{\mathrm{stage}}\right]$. Furthermore, we expand the MSCFs with random weights to obtain the multi-stage BFs (MSBFs) ${\mathit{H}}_i^{\mathrm{stage}}$:

$${\mathit{H}}_i^{\mathrm{stage}}=\varphi ({\mathit{P}}_i^{\mathrm{stage}}{\mathit{W}}_i^{\mathrm{E}}+{\mathit{b}}_i^{\mathrm{E}})$$

(8)

where $\varphi (·)$ is the nonlinear function, such as the tansig. Splice the MSCFs and the MSBFs to obtain the MSCBFs $\mathit{A}$ and rewrite them as:

$$\begin{array}{c}\mathit{A}=\left[\left[{\mathit{P}}_1^{\mathrm{stage}}|{\mathit{H}}_1^{\mathrm{stage}}\right]|\cdots |\left[{\mathit{P}}_s^{\mathrm{stage}}|{\mathit{H}}_s^{\mathrm{stage}}\right]\right]\\ =\left[{\mathit{A}}_1^{\mathrm{stage}}|\cdots |{\mathit{A}}_s^{\mathrm{stage}}\right]\end{array}$$

(9)

Enhancing the linear independence among features can help to produce a more accurate classification model [25,26], so we introduce the block-diagonal constraint [50]. In the commonly used block-diagonal representation method, each sample is represented only by samples with the same class, which can enhance the mutual independence among different classes of samples [51,52]. Here, we use a block matrix $\mathit{D}=diag\left({\mathit{d}}_{11},\cdots ,{\mathit{d}}_{ss}\right)$ to map the MSCBFs into a subspace in which each feature is only represented by those of the same stage. MSCBL-BD optimizes the following objective function:

$$\begin{array}{c}\underset{{\mathit{W}}^m,\mathit{D}}{min}{∥\left(\mathit{A}\mathit{D}\right){\mathit{W}}^m-\mathit{Y}∥}_2^2+{\lambda }_1{∥{\mathit{W}}^m∥}_2^2\hfill \\ s.t.\mathit{A}=\mathit{A}\mathit{D},\mathit{D}=diag\left({\mathit{d}}_{11},\cdots ,{\mathit{d}}_{ss}\right)\hfill \end{array}$$

(10)

where ${\mathit{W}}^m$ denotes the output layer weights of MSCBL-BD. Further consider the error term $\mathit{E}$ and rewrite Equation (10) as:

$$\begin{array}{c}\underset{{\mathit{W}}^m,\mathit{D}}{min}{∥\left(\mathit{A}\mathit{D}\right){\mathit{W}}^m-\mathit{Y}∥}_2^2+{\lambda }_1{∥{\mathit{W}}^m∥}_2^2\hfill \\ s.t.\mathit{A}=\mathit{A}\mathit{D}+\mathit{E},\mathit{D}=diag\left({\mathit{d}}_{11},\cdots ,{\mathit{d}}_{ss}\right)\hfill \end{array}$$

(11)

Since the absolute block diagonal structure is difficult to learn, the work of [50] was consulted. Assuming that the components on the non-block diagonal are as small as possible, the incoherent extra stage is boosted, and the coherent intra-stage representation is further enhanced at the same time [50]. Two terms are constructed to achieve the above objectives: (1) ${∥\mathit{P}\odot \mathit{D}∥}_F^2$ is used to minimize the elements on the non-block diagonal, $\mathit{P}={\mathbf{1}}_D{\mathbf{1}}_D^{\mathrm{T}}-\tilde{\mathit{Y}}$, $\tilde{\mathit{Y}}=\left[\begin{array}{ccc}{\mathbf{1}}_{d_1}{\mathbf{1}}_{d_1}^{\mathrm{T}}& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathbf{1}}_{d_s}{\mathbf{1}}_{d_s}^{\mathrm{T}}\end{array}\right]$, where ⊙ is the Hadamard product, ${∥·∥}_F$ denotes the Frobenius norm, ${\mathbf{1}}_D$ is a D-dimensional vector whose elements are all 1; (2) construct a sparse term ${∥\mathit{Q}\odot \mathit{D}∥}_0$ to enhance the coherent intra-stage representation, where ${∥·∥}_0$ represents the $l_0$-norm, which can calculate the number of non-zero elements in a matrix, $Q_{ij}={∥{\mathit{x}}_i-{\mathit{x}}_j∥}_2^2$. Here, sparsity here is used to calculate the number of elements that are equal to 0. By minimizing the sparse term, we can make as many elements on the non-diagonal as possible trend to 0. Since the optimization of $l_0$-norm is NP-hard, a relaxed term ${∥\mathit{Q}\odot \mathit{D}∥}_1$ is used here, where ${∥·∥}_1$ denotes the $l_1$-norm. Then, Equation (11) can be rewritten as:

$$\begin{array}{c}\underset{{\mathit{W}}^m,\mathit{D},\mathit{E}}{min}\frac{1}{2}{∥\left(\mathit{A}\mathit{D}\right){\mathit{W}}^m-\mathit{Y}∥}_2^2+\frac{{\lambda }_1}{2}{∥{\mathit{W}}^m∥}_2^2+\frac{{\lambda }_2}{2}{∥\mathit{P}\odot \mathit{D}∥}_F^2\hfill \\ +{\lambda }_3{∥\mathit{Q}\odot \mathit{D}∥}_1+{\lambda }_4{∥\mathit{E}∥}_{2,1}+{\lambda }_5{∥\mathit{D}∥}_{\ast }\hfill \\ s.t.\mathit{A}=\mathit{A}\mathit{D}+\mathit{E}\hfill \end{array}$$

(12)

where ${\lambda }_2\sim {\lambda }_5$ are the balancing coefficients, ${∥\mathit{D}∥}_{\ast }$ is used to explore the potential correlation patterns [53], and ${∥·∥}_{\ast }$ denotes the nuclear norm. Due to the difficulty in solving the $l_1$-norm and the nuclear norm problem, auxiliary variables $\mathit{M}$ and $\mathit{N}$ are introduced, and Equation (12) can be rewritten as:

$$\begin{array}{c}\underset{{\mathit{W}}^m,\mathit{D},\mathit{E},\mathit{N},\mathit{M}}{min}\frac{1}{2}{∥\left(\mathit{A}\mathit{D}\right){\mathit{W}}^m-\mathit{Y}∥}_2^2+\frac{{\lambda }_1}{2}{∥{\mathit{W}}^m∥}_2^2\hfill \\ +\frac{{\lambda }_2}{2}{∥\mathit{P}\odot \mathit{D}∥}_F^2+{\lambda }_3{∥\mathit{Q}\odot \mathit{M}∥}_1+{\lambda }_4{∥\mathit{E}∥}_{2,1}+{\lambda }_5{∥\mathit{N}∥}_{\ast }\hfill \\ s.t.\mathit{A}=\mathit{A}\mathit{D}+\mathit{E},\mathit{M}=\mathit{D},\mathit{N}=\mathit{D}\hfill \end{array}$$

(13)

Equation (13) can be solved by ADMM [50], and the augmented Lagrangian expression is:

$$\begin{array}{c}L\left({\mathit{W}}^m,\mathit{D},\mathit{E},\mathit{N},\mathit{M},{\mathit{C}}_1,{\mathit{C}}_2,{\mathit{C}}_3\right)\hfill \\ =\frac{1}{2}{∥\left(\mathit{A}\mathit{D}\right){\mathit{W}}^m-\mathit{Y}∥}_F^2+\frac{{\lambda }_1}{2}{∥{\mathit{W}}^m∥}_F^2+\frac{{\lambda }_2}{2}{∥\mathit{P}\odot \mathit{D}∥}_F^2\hfill \\ +{\lambda }_3{∥\mathit{Q}\odot \mathit{M}∥}_1+{\lambda }_4{∥\mathit{E}∥}_{2,1}+{\lambda }_5{∥\mathit{N}∥}_{\ast }\hfill \\ +〈{\mathit{C}}_1,\mathit{A}-\mathit{A}\mathit{D}-\mathit{E}〉+〈{\mathit{C}}_2,\mathit{M}-\mathit{D}〉+〈{\mathit{C}}_3,\mathit{N}-\mathit{D}〉\hfill \\ +\frac{\mu }{2}\left({∥\mathit{A}-\mathit{A}\mathit{D}-\mathit{E}∥}_F^2+{∥\mathit{M}-\mathit{D}∥}_F^2+{∥\mathit{N}-\mathit{D}∥}_F^2\right)\hfill \end{array}$$

(14)

where ${\mathit{C}}_1$, ${\mathit{C}}_2$, and ${\mathit{C}}_3$ are Lagrangian multipliers, $〈{\mathit{C}}_1,\mathit{A}-\mathit{A}\mathit{D}-\mathit{E}〉=tr\left({\mathit{C}}_1^{\mathrm{T}}\left(\mathit{A}-\mathit{A}\mathit{D}-\mathit{E}\right)\right)$, and $\mu >0$ is the penalty parameter. Each variable is updated alternately to find the desired solution and the detailed calculation process of each variable is provided below.

(1) Update ${\mathit{W}}^m$. Fixing the other variables, the update process for ${\mathit{W}}^m$ is equivalent to solving the following objective function:

$$L=\underset{{\mathit{W}}^m}{min}\frac{1}{2}{∥\left(\mathit{A}\mathit{D}\right){\mathit{W}}^m-\mathit{Y}∥}_F^2+\frac{{\lambda }_1}{2}{∥{\mathit{W}}^m∥}_F^2$$

(15)

Calculate the derivative of Equation (15) with respect to ${\mathit{W}}^m$ and make it zero, then we can obtain the closed solutions:

$${\mathit{W}}^{m,(t+1)}={\left({\mathit{D}}^{\mathrm{T},\left(t\right)}{\mathit{A}}^{\mathrm{T},\left(t\right)}\mathit{A}{\mathit{D}}^{\left(t\right)}+{\lambda }_1\mathit{I}\right)}^{-1}{\mathit{D}}^{\mathrm{T},\left(t\right)}{\mathit{A}}^{\mathrm{T}}\mathit{Y}$$

(16)

(2) Update $\mathit{D}$. When the remaining variables are fixed, the expression of Equation (14) about $\mathit{D}$ is:

$$\begin{array}{c}L={min}_{\mathit{D}}\frac{1}{2}{∥\left(\mathit{A}\mathit{D}\right){\mathit{W}}^{m,(t+1)}-\mathit{Y}∥}_F^2+\frac{{\lambda }_2}{2}{∥{\mathit{P}}^{\left(t\right)}\odot \mathit{D}∥}_F^2+\langle {\mathit{C}}_1^{\left(t\right)},\mathit{A}-\mathit{A}\mathit{D}-{\mathit{E}}^{\left(t\right)}\rangle \hfill \\ +\langle {\mathit{C}}_2^{\left(t\right)},{\mathit{M}}^{\left(t\right)}-\mathit{D}\rangle +{\mathit{C}}_3^{\left(t\right)},{\mathit{N}}^{\left(t\right)}-\mathit{D}\rangle +\frac{{\mu }^{\left(t\right)}}{2}\left({∥\mathit{A}-\mathit{A}\mathit{D}-{\mathit{E}}^{\left(t\right)}∥}_F^2\right.\hfill \\ \left.+{∥{\mathit{M}}^{\left(t\right)}-\mathit{D}∥}_F^2+{∥{\mathit{N}}^{\left(t\right)}-\mathit{D}∥}_F^2\right)\hfill \\ =\frac{1}{2}{∥\left(\mathit{A},\mathit{D}\right){\mathit{W}}^{m,(t+1)}-\mathit{Y}∥}_F^2+\frac{{\lambda }_2}{2}{∥{\mathit{P}}^{\left(t\right)}\odot \mathit{D}∥}_F^2\hfill \\ +\frac{{\mu }^{\left(t\right)}}{2}\left({∥\mathit{A}-\mathit{A}\mathit{D}-{\mathit{E}}^{\left(t\right)}+\frac{{\mathit{C}}_1^{\left(t\right)}}{{\mu }^{\left(t\right)}}∥}_F^2\right.\hfill \\ \left.+{∥{\mathit{M}}^{\left(t\right)}-\mathit{D}+\frac{{\mathit{C}}_2^{\left(t\right)}}{{\mu }^{\left(t\right)}}∥}_F^2+{∥{\mathit{N}}^{\left(t\right)}-\mathit{D}+\frac{{\mathit{C}}_3^{\left(t\right)}}{{\mu }^{\left(t\right)}}∥}_F^2\right)\hfill \\ =\frac{1}{2}{∥\left(\mathit{A},\mathit{D}\right){\mathit{W}}^{m,(t+1)}-\mathit{Y}∥}_F^2+\frac{{\lambda }_2}{2}{∥\mathit{D}-\mathit{R}∥}_F^2\hfill \\ +\frac{{\mu }^{\left(t\right)}}{2}\left({∥\mathit{A}-\mathit{A}\mathit{D}-{\mathit{E}}^{\left(t\right)}+\frac{{\mathit{C}}_1^{\left(t\right)}}{{\mu }^{\left(t\right)}}∥}_F^2\right.\hfill \\ \left.+{∥{\mathit{M}}^{\left(t\right)}-\mathit{D}+\frac{C_2^{\left(t\right)}}{{\mu }^{\left(t\right)}}∥}_F^2+{∥{\mathit{N}}^{\left(t\right)}-\mathit{D}+\frac{{\mathit{C}}_3^{\left(t\right)}}{{\mu }^{\left(t\right)}}∥}_F^2\right)\hfill \end{array}$$

(17)

where $\mathit{R}=\tilde{\mathit{Y}}\odot {\mathit{D}}^{\left(t\right)}$. Calculate the derivative of Equation (17) with respect to $\mathit{D}$ and make it zero; thus, the optimal solution of $\mathit{D}$ is:

$$\begin{array}{c}{\mathit{D}}^{(t+1)}=\left[\frac{1}{{\mu }^{\left(t\right)}}{\mathit{A}}^{\mathrm{T}}\mathit{A}{\mathit{W}}^{m,\left(t+1\right)}{\mathit{W}}^{m,\left(t+1\right),\mathrm{T}}\right.+\left(\frac{{\lambda }_2}{{\mu }^{\left(t\right)}}+2\right)\mathit{I}\hfill \\ {\left.+{\mathit{A}}^{\mathrm{T}}\mathit{A}\right]}^{-1}\left(\frac{1}{{\mu }^{\left(t\right)}}{\mathit{A}}^{\mathrm{T}}\mathit{Y}{\mathit{W}}^{m,\left(t+1\right),\mathrm{T}}+\frac{{\lambda }_2}{{\mu }^{\left(t\right)}}\mathit{R}\right.+{\mathit{A}}^{\mathrm{T}}{\mathit{S}}_1\hfill \\ \left.+{\mathit{S}}_2+{\mathit{S}}_3\right)\hfill \end{array}$$

(18)

where ${\mathit{S}}_1=\mathit{A}-\mathit{E}+\left({\mathit{C}}_1^{\left(t\right)}/{\mu }^{\left(t\right)}\right)$, ${\mathit{S}}_2={\mathit{N}}^{(t+1)}+\left({\mathit{C}}_2^t/{\mu }^{\left(t\right)}\right)$, and ${\mathit{S}}_3=M^{\left(t\right)}+\left({\mathit{C}}_3^{\left(t\right)}/{\mu }^{\left(t\right)}\right)$.

(3) Update $\mathit{N}$. Fix the other variables and abbreviate Equation (14) as an expression only for variable $\mathit{N}$:

$$\begin{array}{c}{N}^{\left(t+1\right)}=\underset{\mathit{N}}{min}{\lambda }_5{∥\mathit{N}∥}_{\ast }+〈C_3^{\left(t\right)},\mathit{N}-\mathit{D}〉\hfill \\ +\frac{{\mu }^{\left(t\right)}}{2}{∥\mathit{N}-{\mathit{D}}^{\left(t+1\right)}∥}_F^2\hfill \\ ={\lambda }_5{∥\mathit{N}∥}_{\ast }+\frac{{\mu }^{\left(t\right)}}{2}{∥\mathit{N}-\left({\mathit{D}}^{\left(t+1\right)}-\frac{{\mathit{C}}_3^{\left(t\right)}}{{\mu }^{\left(t\right)}}\right)∥}_F^2\hfill \end{array}$$

(19)

According to [54], the solution can be obtained by using the singular value threshold operation:

$${\mathit{N}}^{\left(t+1\right)}=\mathit{U}{h}_{{\lambda }_5/{\mu }^{\left(t\right)}}\left(\mathsf{\Sigma }\right){\mathit{V}}^{\mathrm{T}}$$

(20)

where $\mathit{U}\mathsf{\Sigma }{\mathit{V}}^{\mathrm{T}}$ is the singular value decomposition of ${\mathit{D}}^{\left(t+1\right)}-\frac{{\mathit{C}}_3^{\left(t\right)}}{{\mu }^{\left(t\right)}}$, and $h_{{\lambda }_5/{\mu }^{\left(t\right)}}\left(·\right)$ is the soft threshold operation, which is defined as:

$$h_{\lambda }\left(x\right)=\left\{\begin{array}{c}x-\lambda ,\mathrm{if}x>\lambda \hfill \\ x+\lambda ,\mathrm{if}x<-\lambda \hfill \\ 0,\mathrm{otherwise}.\hfill \end{array}\right.$$

(21)

where $\lambda$ is a threshold value.

(4) Update $\mathit{M}$. The update process of variable $\mathit{M}$ is equivalent to solving the following problem:

$$\begin{array}{c}L=\underset{M}{min}{\lambda }_3{∥\mathit{Q}\odot \mathit{M}∥}_1+〈{\mathit{C}}_2^{\left(t\right)},\mathit{M}-{\mathit{D}}^{\left(t+1\right)}〉\hfill \\ +\frac{{\mu }^{\left(t\right)}}{2}{∥\mathit{M}-{\mathit{D}}^{\left(t+1\right)}∥}_F^2\hfill \\ ={\lambda }_3{∥\mathit{Q}\odot \mathit{M}∥}_1+\frac{{\mu }^{\left(t\right)}}{2}{∥\mathit{M}-\left({\mathit{D}}^{\left(t+1\right)}-\frac{{\mathit{C}}_2^{\left(t\right)}}{{\mu }^{\left(t\right)}}\right)∥}_F^2\hfill \end{array}$$

(22)

which can be updated by the point multiplication mechanism, and the optimal solution is calculated as follows:

$$\begin{array}{c}{\mathit{M}}_{i,j}^{\left(t+1\right)}=\underset{{\mathit{M}}_{i,j}}{arg\hspace{0.17em}min}{\lambda }_3{\mathit{Q}}_{i,j}\left|{\mathit{M}}_{i,j}\right|+\frac{{\mu }^{\left(t\right)}}{2}{\left({\mathit{Q}}_{i,j}-{\mathit{K}}_{i,j}\right)}^2\hfill \\ =h_{{\lambda }_3{\mathit{Q}}_{i,j}/{\mu }^{\left(t\right)}}\left({\mathit{K}}_{i,j}\right)\hfill \end{array}$$

(23)

where ${\mathit{K}}_{i,j}={\mathit{D}}_{i,j}^{\left(t+1\right)}-\left({\left({\mathit{C}}_3^{\left(t\right)}\right)}_{i,j}/u^{\left(t\right)}\right)$.

(5) Update $\mathit{E}$. When fixing the other variables, we rewrite the objective function of Equation (14) on variable $\mathit{E}$ and obtain:

$$\begin{array}{c}L=\underset{\mathit{E}}{min}{\lambda }_4{∥\mathit{E}∥}_{2,1}+〈{\mathit{C}}_1^{\left(t\right)},\mathit{A}-\mathit{A}{\mathit{D}}^{\left(t+1\right)}-\mathit{E}〉\hfill \\ +\frac{{\mu }^{\left(t\right)}}{2}{∥\mathit{A}-\mathit{A}{\mathit{D}}^{\left(t+1\right)}-\mathit{E}∥}_F^2\hfill \\ ={\lambda }_4{∥\mathit{E}∥}_{2,1}+\frac{{\mu }^{\left(t\right)}}{2}{∥\mathit{E}-\left(\mathit{A}-\mathit{A}{\mathit{D}}^{\left(t+1\right)}+\frac{C_1^{\left(t\right)}}{{\mu }^{\left(t\right)}}\right)∥}_F^2\hfill \end{array}$$

(24)

According to [50], let $\mathit{G}=\mathit{A}-\mathit{A}{\mathit{D}}^{\left(t+1\right)}+\frac{{\mathit{C}}_1^{\left(t\right)}}{{\mu }^{\left(t\right)}}$ and

$${\mathit{E}}^{\left(t+1\right)}\left(i,:\right)=\left\{\begin{array}{c}\frac{{∥{\mathit{G}}^i∥}_2-\frac{{\lambda }_4}{u^{\left(t\right)}}}{{∥{\mathit{G}}^i∥}_2}{\mathit{G}}^i,\mathrm{if}{∥{\mathit{G}}^i∥}_2>\frac{{\lambda }_4}{u^{\left(t\right)}}\hfill \\ 0,\mathrm{if}{∥{\mathit{G}}^i∥}_2\le \frac{{\lambda }_4}{u^{\left(t\right)}}\hfill \end{array}\right.$$

(25)

The above steps are alternated until a predetermined maximum number of iterations is reached, thereby obtaining the desired output layer weight ${\mathit{W}}^m$ and the block diagonal matrix $\mathit{D}$, and further calculating the prediction label vector ${\mathit{Y}}^{\prime }$:

$${\mathit{Y}}^{\prime }=\mathit{A}\mathit{D}{\mathit{W}}^m$$

(26)

Consequently, sparse signal recovery problems can be solved by a dozen different methods, such as orthogonal matching pursuit, K-SVD, ADMM, etc. Among them, ADMM was designed for general decomposition methods and decentralized algorithms in optimization problems. Furthermore, many state-of-the-art algorithms for $l_1$-norm-involved problems can be derived by ADMM. In addition, the problem in Equation (13) can be seen as a special case of the classical ADMM problem. Therefore, ADMM is used here to solve the problem in Equation (13).

3. Experiments and Analysis

To verify the validity of the proposed method, we selected three popular real HSI datasets: Indian Pines, Pavia University, and Salinas. The ground-truth maps and sample information of the three HSI datasets are shown in Figure 3. The spatial resolutions of the three datasets are $145\times 145$, $610\times 340$, and $512\times 217$, respectively. All experiments were performed on the MATLAB 2016a platform;the used computer was configured as: CPU Intel I7-4790, 16 G memory, and GPU GTX980. In order to eliminate the randomness, all experiments were repeated 10 times to obtain the mean values. A total of 200 labeled samples per class were randomly selected for training and the others were used for testing. Four evaluation indicators were selected to evaluate the experimental results, which were the classification accuracy of each class of ground object, the average classification accuracy (AA), the overall classification accuracy (OA) and the Kappa coefficient.

3.1. Parameter Setting

Except for the number of feature maps of the last convolutional layer (the numbers of feature maps are 9, 9, and 16 for the Indian Pines, Pavia University, and Salinas datasets, respectively), the network structures of the CNNs for three HSI datasets were the same. The detailed structure of the CNN is shown in

Table 1. Configuration of the CNN.

Layer	Input			Number of Filters	Width of Filters	Height of Filters	Step Size	Output
	Width	Height	Channel					Width	Height	Channel	Dimension
I1								17	17	15	4335
C1	17	17	15	30	4	4	1	14	14	30	5880
P1	14	14	30		2	2	2	7	7	30	1470
N1	7	7	30	/	/	/	/	7	7	30	1470
C2	7	7	30	30	1	1	1	7	7	30	1470
N2	7	7	30	30	1	1	1	7	7	30	1470
C3	7	7	30	30	4	4	1	4	4	30	480
P3	4	4	30	/	2	2	2	2	2	30	120
N3	2	2	30	/	/	/	/	2	2	30	120
C4	2	2	30	30	1	1	1	2	2	30	120
N4	2	2	30	/	/	/	/	2	2	30	120
C5	2	2	30	9/16	2	2	1	1	1	9/16	9/16
Softmax	1	1	9/16	/	/	/	/	1	1	9/16	9/16

. It can be seen that: (1) there were total 5 convolutional layers, i.e., C1∼C5. The convolutional kernel sizes of C2 and C4 were $1\times 1$, which is helpful for obtaining a deeper network structure at the cost of a small increase in network parameters. The convolutional kernel sizes of C1 and C3 were $4\times 4$, the convolutional kernel size of C5 was $2\times 2$. (2) There were two max-pooling layers, i.e., P1 and P2, both step sizes of which were 2. (3) A total of 4 nonlinear layers denoted as N1∼N4 were included. The activation functions used in all of the nonlinear layers were the Sigmoid function.

The CNN training process consists of two steps: forward calculation and back propagation. The former aims to calculate the classification result based on the current network parameters, and the latter updates the network parameters. Here, we used a stochastic gradient descent algorithm to update the CNN with a batch size of 100, a learning rate of 0.1, and an iteration of 1000. The CNN training process is based on the Matconvnet toolbox [55].

The parameter settings of MSCBL-BD on the three datasets are shown in

Table 2. Parameter settings of MSCBL-BD.

	Number of Iterations	Number of Nodes in EN	${\mathit{\lambda }}_1\sim {\mathit{\lambda }}_5$
Indian Pines		500	$\{0.1,10,1,5,1\}$
Pavia University	110	400	$\{0.01,5,0.01,10,0.1\}$
Salinas		500	$\{0.1,5,0.1,1,1\}$

, where ${\lambda }_1\sim {\lambda }_5$ are tuned to achieve the best performance via fivefold cross-validations from $\left\{0.01,0.1,1,5,10\right\}$.

3.2. Comparative Experiments

In order to verify the validity of the proposed MSCBL-BD, six benchmark methods and two special cases of MSCBL-BD (CBL and MSCBL) were selected for comparative experiments:

(1)

Traditional classification methods: SVM [8], whose optimal super-parameters are selected through the fivefold cross-validation method;

(2)

Deep learning methods: SS-DBN [34], CNN-PPF [47], and CNN [38]. To set the parameters of SS-DBN and CNN-PPF, we referred to the corresponding articles. The configuration of CNN is shown in Table 1.

(3)

Special cases of MSCBL-BD: CBL (the CBFs of the last stage are connected to the output layer and not constrained with the block diagonal matrix); MSCBL (the MSCBFs are connected to the output layer and not constrained with the block-diagonal matrix).

Table 3. Comparison of the classification performance on the Indian Pines dataset.

	SVM [8]	BLS [25]	SS-DBN [34]	CNN-PPF [47]	CNN [39]	CBL	MSCBL	MSCBL-BD
A1 (%)	75.49	82.54	77.20	92.23	89.98	92.84	94.14	96.58
A2 (%)	73.78	82.29	82.76	96.69	97.75	98.43	99.06	99.32
A3 (%)	95.26	95.26	94.42	99.86	98.80	99.40	99.82	100
A4 (%)	98.45	99.10	98.30	99.43	99.21	99.68	99.85	99.96
A5 (%)	99.71	99.71	99.86	99.86	99.93	100	100	100
A6 (%)	78.21	86.68	87.23	95.10	94.69	96.22	97.20	99.02
A7 (%)	66.18	69.65	76.02	89.48	89.59	92.28	94.12	95.96
A8 (%)	83.77	93.39	90.59	97.00	98.19	98.93	99.42	99.87
A9 (%)	98.27	98.91	94.93	99.89	99.02	99.54	99.81	99.81
AA (%)	85.46	89.73	89.04	96.62	96.35	97.48	98.16	98.95
OA (%)	79.80	84.26	84.61	94.51	94.10	95.78	96.80	98.01
Kappa	0.7611	0.8142	0.8178	0.9344	0.9296	0.9495	0.9617	0.9762

,

Table 4. Comparison of the classification performance on the Pavia University dataset.

	SVM [8]	BLS [25]	SS-DBN [34]	CNN-PPF [47]	CNN [39]	CBL	MSCBL	MSCBL-BD
A1 (%)	83.35	76.32	81.46	97.40	96.68	97.21	97.78	98.25
A2 (%)	86.12	90.13	92.96	97.40	98.93	99.10	99.27	99.39
A3 (%)	83.23	82.59	88.37	92.29	97.91	98.40	98.78	98.79
A4 (%)	94.66	95.49	95.55	97.63	98.54	98.62	98.80	98.80
A5 (%)	99.55	99.62	99.91	99.93	99.89	99.97	100	99.99
A6 (%)	88.87	91.25	88.92	97.89	99.64	99.96	99.96	99.92
A7 (%)	90.48	94.67	90.35	97.33	99.11	99.58	99.74	99.83
A8 (%)	84.39	84.54	85.93	93.97	98.31	98.27	98.73	98.96
A9 (%)	99.90	99.65	99.49	99.57	99.69	99.80	99.84	99.85
AA (%)	90.06	90.47	91.44	97.05	98.74	98.99	99.21	99.31
OA (%)	87.07	88.21	90.29	97.05	98.58	98.82	99.06	99.21
Kappa	0.8301	0.8445	0.8708	0.9626	0.9809	0.9842	0.9873	0.9893

and

Table 5. Comparison of the classification performance on the Salinas dataset.

	SVM [8]	BLS [25]	SS-DBN [34]	CNN-PPF [47]	CNN [39]	CBL	MSCBL	MSCBL-BD
A1 (%)	99.14	99.60	98.94	99.86	99.97	99.98	100	100
A2 (%)	99.45	99.65	98.95	99.59	99.95	99.98	100	100
A3 (%)	99.46	99.46	79.60	99.83	99.46	99.85	99.93	99.90
A4 (%)	99.58	99.43	99.58	99.68	99.51	99.92	99.91	99.93
A5 (%)	98.79	99.28	99.71	98.41	98.94	99.40	99.53	99.76
A6 (%)	99.79	99.81	99.91	99.78	100	100	100	100
A7 (%)	99.67	99.57	98.90	99.85	99.87	99.95	99.98	99.99
A8 (%)	84.40	83.18	87.04	83.28	85.61	91.05	92.41	94.24
A9 (%)	99.37	99.75	97.93	97.44	99.31	99.47	99.55	99.63
A10 (%)	94.65	95.76	95.79	95.84	98.98	99.44	99.64	99.72
A11 (%)	98.87	98.66	98.64	99.75	99.60	99.88	99.94	99.90
A12 (%)	99.95	100	99.73	100	99.90	99.95	99.94	99.98
A13 (%)	99.52	99.16	99.50	99.38	100	100	100	100
A14 (%)	97.91	98.12	99.95	99.52	99.79	99.96	100	100
A15 (%)	69.35	73.82	85.92	81.18	95.12	95.02	95.90	97.17
A16 (%)	99.02	98.94	99.75	98.51	99.84	99.89	99.91	99.97
AA (%)	96.19	96.51	96.24	96.99	98.49	98.98	99.16	99.39
OA (%)	91.67	92.18	93.76	92.98	95.94	97.22	97.67	98.27
Kappa	0.9067	0.9125	0.9302	0.9215	0.9546	0.9689	0.9740	0.9807

show the classification performance comparisons of the different methods on the three HSI datasets and the best values are shown in bold.

4. Discussion

From the experimental results, we can see that: (1) On all of three HSI datasets, for three evaluation indexes (AA, OA, and Kappa coefficients), MSCBL-BD, MSCBL, and CBL achieved higher performance than the other methods. Taking the Indian Pines dataset as an example, CBL outperformed BLS by 11.52% in terms of OA. The main reasons are two-fold: the features learned by BLS are linear sparse features and only the spectral information is utilized. Therefore, the features of the HSI cannot be fully represented. In addition, CBL, MSCBL, and MSCBL-BD outperformed CNN by 1.68%, 2.7%, and 3.91% on the OA, respectively, which verifies the superiority of width expansion. (2) On the three HSI datasets, MSCBL outperformed CBL by 1.02%, 0.24%, and 0.45%, respectively, because MSCBL utilizes the MSCFs and performs feature expansion for each stage, which makes the features learned by MSCBL more discriminative. Furthermore, MSCBL-BD outperformed MSCBL by 1.21%, 0.15%, and 0.6% on the three datasets, respectively, because the MSCBFs are mapped through a block-diagonal matrix, so that each obtained feature is linearly represented only by the features of its own stage, thereby enhancing the linear independence of the MSCBFs of each stage. This can help with learning a more accurate HSI classification model, which in turn improves the final classification accuracy. (3) Both SS-DBN and CNN are deep spectral-spatial classification methods. Compared with SS-DBN, CNN yielded higher OAs on all HSI datasets. This is because SS-DBN takes features by stacking the spectral vector and the vectorized neighboring region representation as the input of the DBN, which not only leads to an increase in the dimension of the input data, but also destroys the inherent structure of the data. In addition, in the case of a small number of labeled samples, as a kind of fully-connected neural network, over-fitting may occur in the training process of SS-DBN. Figure 4, Figure 5 and Figure 6 show the classification maps achieved by different methods. It can be seen that the classification maps obtained by MSCBL-BD on all of three HSI datasets are smoother and more detailed. Taking the Indian Pines dataset as an example, the benchmark methods misclassify more Soybean-clean, Soybean-notill, and Corn-notill as Soybean-mintill; misclassify more Grass-trees into Woods; and misclassify more Corn-notill into Corn-minitill.

5. Conclusions

A novel BLS based method (MSCBL-BD) for HSI classification was proposed in this paper. By replacing the linear sparse features with convolutional features, the nonlinear mapping ability of the model is improved and the complex spectral-spatial features of HSI can be better represented. Therefore, the CBL has higher classification accuracy than BLS. Furthermore, in order to reduce the information loss occurring in the multi-layer mapping process of CNN, the MSCBFs are utilized. Moreover, in order to train a more accurate HSI classification model, the block-diagonal constraint is introduced to the MSCBL. The MSCBFs are mapped by a block-diagonal matrix, and the obtained features are only linearly represented by those of their own stages. Therefore, the linear independence of the MSCBFs is enhanced and the classification accuracy is improved ultimately. The experimental results demonstrated the superiority of MSCBL-BD compared to some competitive methods. In our future work, we will increase the robustness of our method, so that satisfactory performance can be achieved when bands are missing.