Spatial-Spectral Graph Regularized Kernel Sparse Representation for Hyperspectral Image Classification

Abstract

This paper presents a spatial-spectral method for hyperspectral image classification in the regularization framework of kernel sparse representation. First, two spatial-spectral constraint terms are appended to the sparse recovery model of kernel sparse representation. The first one is a graph-based spatially-smooth constraint which is utilized to describe the contextual information of hyperspectral images. The second one is a spatial location constraint, which is exploited to incorporate the prior knowledge of the location information of training pixels. Then, an efficient alternating direction method of multipliers is developed to solve the corresponding minimization problem. At last, the recovered sparse coefficient vectors are used to determine the labels of test pixels. Experimental results carried out on three real hyperspectral images point out the effectiveness of the proposed method.

1. Introduction

Hyperspectral imaging sensors capture digital images in hundreds of narrow and contiguous spectral bands spanning the visible to infrared spectrum. Each pixel of a hyperspectral image can be represented by a vector whose entries correspond to various spectral-band responses. Different materials usually reflect electromagnetic energy differently at specific wavelengths. The wealth of spectral information of hyperspectral images promotes the development of many application domains, such as military [1], agriculture [2], and mineralogy [3]. Among the information processing procedures of these applications, classification is an important one, where pixels are assigned to one of the available classes according to a set of given training pixels. Various classifiers have been developed for hyperspectral image classification and have shown good performance. Some popular classifiers include support vector machine (SVM) [4], multinomial logistic regression [5], and sparse representation classification (SRC) [6].

Hyperspectral images usually have large homogeneous regions within which the neighboring pixels consist of the same type of materials and have similar spectral characteristics. Previous works have highlighted that hyperspectral image classification should not only focus on analyzing spectral features, but also incorporate the information of spatially-adjacent pixels. For instance, the authors in [7] incorporate the spectral and spatial information into SVM by taking advantage of the composite kernel technique. In [5], the spatial information is incorporated into multinomial logistic regression by some spatial feature extraction techniques. In [6], a joint sparsity model is introduced to SRC, where the neighborhood pixels are assumed to have the same labels. Very recently, some deep models (e.g., stacked autoencoder [8] and convolutional neural network [9,10,11,12]) have been proposed for the spatial-spectral classification of hyperspectral images and have shown good performance in terms of accuracy and flexibility.

Among the aforementioned classifiers, SRC is an excellent one and can directly assign a class label to a test pixel without a training process. Although SRC can achieve good performance in hyperspectral image classification [6], it is hard to classify the data that is not linearly separable. To overcome this drawback, kernel SRC (KSRC) is proposed in [13] to capture the nonlinear similarity of features. In view of this, this paper considers using KSRC for the classification of hyperspectral images. However, KSRC is also a pixel-wise classifier, which does not consider the spectral similarity of spatially-adjacent pixels. In the literature, some methods have been proposed to incorporate the spatial information into KSRC. In [13], a joint sparsity model is introduced to KSRC, which can be seen as a kernel extension of [6]. In [14], the spatial-spectral information is incorporated into KSRC by taking advantage of the neighboring filtering kernel technique. In [15], a centralized quadratic constraint that describes the spectral similarity of spatially-adjacent pixels is appended to the sparse recovery model of KSRC.

This paper considers incorporating the spatial-spectral information of hyperspectral images in the regularization framework of KSRC. Different from the relevant methods, a spatial-spectral graph is designed to describe the contextual information of hyperspectral images, where the spectral similarity of spatially-adjacent pixels (i.e., nodes) is measured by the weight of the edge. In addition, the acquisition of labeled pixels in most hyperspectral scenes is often an expensive procedure. Thus, the number of training pixels that can be used for hyperspectral image classification is often very limited. To deal with this problem, we assume that the training pixels are labeled by experts and then exploit the wealth of these training pixels. In detail, a spatial location constraint is introduced to capture the prior knowledge of the location information of training pixels and is thereby appended to the sparse recovery model of KSRC. As for the final classification procedure, the sparse coefficient vectors obtained by solving the corresponding regularization problem are used to determine the labels of test pixels.

The remainder of this paper is organized as follows. Section 2 briefly reviews the formulations of KSRC. In Section 3, we present the formulations of the proposed SSGL method and the corresponding optimization algorithm. The effectiveness of the proposed method is demonstrated in Section 4 by conducting experiments on three real hyperspectral images. Finally, Section 5 discusses and concludes this paper.

2. KSRC

This section briefly introduces the notations of kernel sparse representation for hyperspectral image classification. As with the assumption of SRC, KSRC also assumes that the features belonging to the same class approximately lie in the same low-dimensional subspace [16,17]. Suppose the given hyperspectral scene includes C classes, and that there exists a feature mapping function $\varphi$ which maps an L-band test pixel $\mathbf{x}\in {\mathbb{R}}^L$ and a J-sized training dictionary $\mathbf{A}=[{\mathbf{a}}_1,\dots ,{\mathbf{a}}_J]\in {\mathbb{R}}^{L\times J}$ to the high-dimensional feature space: $\mathbf{x}\to \varphi \left(\mathbf{x}\right)$, $\mathbf{A}\to \mathsf{\Phi }\left(\mathbf{A}\right)=[\varphi \left({\mathbf{a}}_1\right),\dots ,\varphi \left({\mathbf{a}}_J\right)]$, where the columns of $\mathbf{A}$ are composed of J training pixels from all classes. Then, for an unknown test pixel $\mathbf{x}\in {\mathbb{R}}^L$, it can be represented as follows:

$$\varphi \left(\mathbf{x}\right)=\mathsf{\Phi }\left(\mathbf{A}\right)\mathbf{s},$$

(1)

where $\mathbf{s}\in {\mathbb{R}}^J$ denotes an unknown sparse coefficient vector.

Suppose the dictionary $\mathbf{A}$ is given; the unknown sparse coefficient vector $\mathbf{s}$ can be obtained by solving the following optimization problem:

$$\underset{\mathbf{s}}{min}\frac{1}{2}\left|\right|\varphi \left(\mathbf{x}\right)-{\mathsf{\Phi }\left(\mathbf{A}\right)\mathbf{s}\left|\right|}_2^2+{\lambda \left|\right|\mathbf{s}\left|\right|}_1,$$

(2)

where $\lambda >0$ is used to control the sparsity of $\mathbf{s}$. Then, the class label of $\mathbf{x}$ can be determined by the following classification rule:

$$\mathrm{class}\left(\mathbf{x}\right)=arg\underset{c=1,\dots ,C}{min}\left|\right|\varphi \left(\mathbf{x}\right)-\mathsf{\Phi }\left(\mathbf{A}\right){\delta }_c\left(\mathbf{s}\right){\left|\right|}_2,$$

(3)

where ${\delta }_c(·)$ is the characteristic function that selects coefficients related with the c-th class and makes the rest zero.

Since all $\varphi$ mappings used in KSRC occur in the form of inner products, the kernel function K can be defined for any two pixels ${\mathbf{x}}_i\in {\mathbb{R}}^L$ and ${\mathbf{x}}_j\in {\mathbb{R}}^L$ as follows:

$$K({\mathbf{x}}_i,{\mathbf{x}}_j)=〈\varphi \left({\mathbf{x}}_i\right),\varphi \left({\mathbf{x}}_j\right)〉.$$

(4)

Some commonly-used kernel functions are: linear kernel ($K({\mathbf{x}}_i,{\mathbf{x}}_j)=〈{\mathbf{x}}_i,{\mathbf{x}}_j〉$), polynomial kernel ($K({\mathbf{x}}_i,{\mathbf{x}}_j)={(〈{\mathbf{x}}_i,{\mathbf{x}}_j〉+1)}^d$, $d\in {\mathbb{Z}}^{+}$), and Gaussian radial basis function (RBF) kernel ($K({\mathbf{x}}_i,{\mathbf{x}}_j)=exp(-\gamma ||{\mathbf{x}}_i-{\mathbf{x}}_j{\left|\right|}_2^2)$, $\gamma \in {\mathbb{R}}^{+}$). In this paper, only the RBF kernel is considered. After introducing (4) into (2), the corresponding optimization problem can be rewritten as follows:

$$\underset{\mathbf{s}}{min}\frac{1}{2}{\mathbf{s}}^T\mathbf{Q}\mathbf{s}-{\mathbf{s}}^T\mathbf{p}+{\lambda \left|\right|\mathbf{s}\left|\right|}_1+\mathrm{Const},$$

(5)

where $\mathrm{Const}=(1/2)K(\mathbf{x},\mathbf{x})$ is a constant that can be dropped in the optimization problem (5), $\mathbf{Q}=\langle \mathsf{\Phi }\left(\mathbf{A}\right),\mathsf{\Phi }\left(\mathbf{A}\right)\rangle$ is a $J\times J$ positive semi-definite matrix with entry ${\mathbf{Q}}_{ij}=K({\mathbf{a}}_i,{\mathbf{a}}_j)$, and $\mathbf{p}={[K({\mathbf{a}}_1,\mathbf{x}),\dots ,K({\mathbf{a}}_J,\mathbf{x})]}^T\in {\mathbb{R}}^J$. Analogously, the classification rule (3) can be rewritten as follows:

$$\mathrm{class}\left(\mathbf{x}\right)=arg\underset{c=1,\dots ,C}{min}{\delta }_c^T\left(\mathbf{s}\right)\mathbf{Q}{\delta }_c\left(\mathbf{s}\right)-2{\delta }_c^T\left(\mathbf{s}\right)\mathbf{p}.$$

(6)

3. Proposed Approach

3.1. Proposed Model

Suppose the observed hyperspectral image $\mathbf{X}$ consists of I pixels ${\{{\mathbf{x}}_i\in {\mathbb{R}}^L\}}_{i=1}^I$; i.e., $\mathbf{X}=[{\mathbf{x}}_1,\dots ,{\mathbf{x}}_I]\in {\mathbb{R}}^{L\times J}$. Then, the corresponding sparse recovery model of (2) for the observed hyperspectral image $\mathbf{X}$ can be written as follows:

$$\underset{\mathbf{S}}{min}\frac{1}{2}\left|\right|\mathsf{\Phi }\left(\mathbf{X}\right)-{\mathsf{\Phi }\left(\mathbf{A}\right)\mathbf{S}\left|\right|}_F^2+{\lambda \left|\right|\mathbf{S}\left|\right|}_{1,1},$$

(7)

where $\left|\right|·{\left|\right|}_F$ is the Frobenius norm, ${\left|\right|\mathbf{S}\left|\right|}_{1,1}={\sum }_{j,i}\left|{\mathbf{S}}_{ji}\right|$, $\mathsf{\Phi }\left(\mathbf{X}\right)=[\varphi \left({\mathbf{x}}_1\right),\dots ,\varphi \left({\mathbf{x}}_I\right)]$ denotes the corresponding data of $\mathbf{X}$ under the mapping function $\varphi$ and $\mathbf{S}=[{\mathbf{s}}_1,\dots ,{\mathbf{s}}_I]\in {\mathbb{R}}^{J\times I}$ is the unknown sparse coefficient matrix, with ${\mathbf{s}}_i\in {\mathbb{R}}^J$ being the sparse coefficient vector of pixel ${\mathbf{x}}_i$. Similarly, the corresponding sparse recovery model of (5) can be written as follows:

$$\underset{\mathbf{S}}{min}\frac{1}{2}\mathrm{Tr}\left({\mathbf{S}}^T\mathbf{Q}\mathbf{S}\right)-\mathrm{Tr}\left({\mathbf{S}}^T\mathbf{P}\right)+{\lambda \left|\right|\mathbf{S}\left|\right|}_{1,1},$$

(8)

where the constant term is dropped and $\mathrm{Tr}(·)$ denotes the trace of a matrix, $\mathbf{P}=\langle \mathsf{\Phi }\left(\mathbf{A}\right),\mathsf{\Phi }\left(\mathbf{X}\right)\rangle =[{\mathbf{p}}_1,\dots ,{\mathbf{p}}_I]\in {\mathbb{R}}^{J\times I}$ with entry ${\mathbf{P}}_{ij}=K({\mathbf{a}}_i,{\mathbf{x}}_j)$. The corresponding classification rule of (6) can be written as follows:

$$\mathrm{class}\left({\mathbf{x}}_i\right)=arg\underset{c=1,\dots ,C}{min}{\delta }_c^T\left({\mathbf{s}}_i\right)\mathbf{Q}{\delta }_c\left({\mathbf{s}}_i\right)-2{\delta }_c^T\left({\mathbf{s}}_i\right){\mathbf{p}}_i,\forall i=1,2,\cdots ,I.$$

(9)

The matrix $\mathbf{X}$ is not just a set of pixels; it is essentially a three-dimensional image; since for any i in the complete set $\mathcal{I}=\{1,2,\cdots ,I\}$, ${\mathbf{s}}_i$ is the sparse coefficient vector of pixel ${\mathbf{x}}_i$. Then, the matrix $\mathbf{S}$ can also be seen as a three-dimensional image. That is to say, the spatial relationship between every two pixels ${\mathbf{x}}_i$ and ${\mathbf{x}}_j$ is also suitable for that of the sparse coefficient vectors ${\mathbf{s}}_i$ and ${\mathbf{s}}_j$. It is natural to incorporate the spatial information of $\mathbf{X}$ by enforcing $\mathbf{S}$. However, directly enforcing $\mathbf{S}$ is too strict; it does not consider the variation of training pixels within each class. In view of this, the summation of sparse coefficients that belong to each class is considered. Specifically, the spatial information is incorporated by enforcing $\mathbf{TS}$, where $\mathbf{T}\in {\mathbb{R}}^{C\times I}$ is defined as:

$${\mathbf{T}}_{cj}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\mathrm{class}\left({\mathbf{a}}_j\right)=c\hfill \\ 0\hfill & \mathrm{else}\hfill \end{array}\right.,\forall c=1,2,\cdots ,C;j=1,2,\cdots ,J.$$

(10)

Let us define a spatial-spectral graph as $\mathcal{G}=\{\mathbf{V},\mathbf{E},\mathbf{W}\}$, where $\mathbf{V}=[{\mathbf{v}}_1,\dots ,{\mathbf{v}}_I]$ and $\mathbf{E}$ are the sets of vertices and edges, respectively, and $\mathbf{W}\in {\mathbb{R}}^{I\times I}$ is a weight matrix on the graph. In this paper, every node ${\mathbf{v}}_i$ is connected to its eight spatially-adjacent nodes (see Figure 1), and the weight between ${\mathbf{v}}_i$ and ${\mathbf{v}}_i$ is defined as follows:

$${\mathbf{W}}_{ij}=\left\{\begin{array}{cc}exp(-\beta ||{\overline{\mathbf{x}}}_i-{\overline{\mathbf{x}}}_j{\left|\right|}_2)+{10}^{-6}\hfill & \mathrm{if}{\mathbf{v}}_i\mathrm{and}{\mathbf{v}}_i\mathrm{are}\mathrm{connected}\hfill \\ 0\hfill & \mathrm{else}\hfill \end{array}\right.,\forall i,j$$

(11)

where $\beta >0$ and ${\overline{\mathbf{x}}}_i$ and ${\overline{\mathbf{x}}}_j$ are the pixels of the first three principal components of the hyperspectral image $\mathbf{X}$. It is apparent that for any two similar nodes, a large weight will be given. In this paper, the graph-based spatially-smooth constraint term is defined as follows:

$$\mathcal{S}\left(\mathbf{TS}\right)=\mathrm{Tr}\left(\left(\mathbf{TS}\right)\mathbf{L}{\left(\mathbf{TS}\right)}^T\right),$$

(12)

where $\mathbf{L}=\mathbf{D}-\mathbf{W}$ is the graph Laplacian and $\mathbf{D}$ is a diagonal matrix whose entries are column sums of $\mathbf{W}$ (i.e., ${\mathbf{D}}_{ii}={\sum }_j{\mathbf{W}}_{ij}$). After appending (12) to the sparse recovery model (8), we can get the following optimization problem:

$$\underset{\mathbf{S}}{min}\frac{1}{2}\mathrm{Tr}\left({\mathbf{S}}^T\mathbf{Q}\mathbf{S}\right)-\mathrm{Tr}\left({\mathbf{S}}^T\mathbf{P}\right)+{\lambda \left|\right|\mathbf{S}\left|\right|}_{1,1}+\frac{\alpha }{2}\mathrm{Tr}\left(\left(\mathbf{TS}\right)\mathbf{L}{\left(\mathbf{TS}\right)}^T\right),$$

(13)

where $\alpha >0$. In order to effectively work with limited training pixels, the prior knowledge of the location information of training pixels is included by a spatial location constraint term. The final sparse recovery model can be written as follows:

$$\underset{\mathbf{S}}{min}\frac{1}{2}\mathrm{Tr}\left({\mathbf{S}}^T\mathbf{Q}\mathbf{S}\right)-\mathrm{Tr}\left({\mathbf{S}}^T\mathbf{P}\right)+{\lambda \left|\right|\mathbf{S}\left|\right|}_{1,1}+\frac{\alpha }{2}\mathrm{Tr}\left(\left(\mathbf{TS}\right)\mathbf{L}{\left(\mathbf{TS}\right)}^T\right)+{\iota }_0({\mathbf{TS}}_{\mathsf{\Lambda }}-\mathbf{T}),$$

(14)

where $\mathsf{\Lambda }\subset \mathcal{I}$ is a set with ${\mathbf{S}}_{\mathsf{\Lambda }}$ being the corresponding coefficient matrix of the labeled pixels $\mathbf{A}$ and ${\iota }_0(·)$ denotes the indicator function; i.e., ${\iota }_0(·)$ is zero if the given matrix is equal to zero, and infinity otherwise. By using the location information of training pixels in the model (14), these training pixels can be treated as anchor samples whose coefficients are fixed throughout the classification process, and the graph-based spatially-smooth constraint can spread their label information to their neighbors until a global stable state is achieved on the whole data set.

3.2. Optimization Algorithm

By introducing two variables $\mathbf{M}\in {\mathbb{R}}^{J\times I}$ and $\mathbf{N}\in {\mathbb{R}}^{C\times I}$, the proposed optimization model (14) can be rewritten as follows:

$$\begin{array}{c}\underset{\mathbf{S},\mathbf{M},\mathbf{N}}{min}f(\mathbf{S},\mathbf{M},\mathbf{N})\\ \mathrm{s}.\mathrm{t}.\mathbf{M}=\mathbf{S},\mathbf{N}=\mathbf{TS}.\end{array}$$

(15)

where

$$f(\mathbf{S},\mathbf{M},\mathbf{N})=\frac{1}{2}\mathrm{Tr}\left({\mathbf{S}}^T\mathbf{Q}\mathbf{S}\right)-\mathrm{Tr}\left({\mathbf{S}}^T\mathbf{P}\right)+{\lambda \left|\right|\mathbf{M}\left|\right|}_{1,1}+\frac{\alpha }{2}\mathrm{Tr}\left(\mathbf{N}\mathbf{L}{\mathbf{N}}^T\right)+{\iota }_0({\mathbf{N}}_{\mathsf{\Lambda }}-\mathbf{T}).$$

(16)

The optimization problem (15) accords with the framework of the alternating direction method of multipliers (ADMM) [18,19]. The augmented Lagrangian function of (15) can be written as follows:

$$\mathcal{L}(\mathbf{S},\mathbf{M},\mathbf{N},\mathbf{H},\mathbf{B})=f(\mathbf{S},\mathbf{M},\mathbf{N})+\frac{\mu }{2}\left|\right|\mathbf{S}-\mathbf{M}-{\mathbf{H}\left|\right|}_F^2+\frac{\mu }{2}\left|\right|\mathbf{TS}-\mathbf{N}-{\mathbf{B}\left|\right|}_F^2,$$

(17)

where $\mu >0$ is the penalty parameter and $\mathbf{H}\in {\mathbb{R}}^{J\times I}$ and $\mathbf{B}\in {\mathbb{R}}^{C\times I}$ are two auxiliary variables. The corresponding iteration procedure can be written as follows:

$$\left\{\begin{array}{ccc}{\mathbf{S}}^t& =& arg\underset{\mathbf{S}}{min}\mathcal{L}(\mathbf{S},{\mathbf{M}}^{t-1},{\mathbf{N}}^{t-1},{\mathbf{H}}^{t-1},{\mathbf{B}}^{t-1})\hfill \\ {\mathbf{M}}^t& =& arg\underset{\mathbf{M}}{min}\mathcal{L}({\mathbf{S}}^t,\mathbf{M},{\mathbf{N}}^{t-1},{\mathbf{H}}^{t-1},{\mathbf{B}}^{t-1})\hfill \\ {\mathbf{N}}^t& =& arg\underset{\mathbf{N}}{min}\mathcal{L}({\mathbf{S}}^t,{\mathbf{M}}^t,\mathbf{N},{\mathbf{H}}^{t-1},{\mathbf{B}}^{t-1})\hfill \\ {\mathbf{H}}^t& =& {\mathbf{H}}^{t-1}-({\mathbf{S}}^t-{\mathbf{M}}^t)\hfill \\ {\mathbf{B}}^t& =& {\mathbf{B}}^{t-1}-({\mathbf{TS}}^t-{\mathbf{N}}^t)\hfill \end{array}\right.$$

(18)

where $t>0$ is the iteration number. The first step of (18) is to solve the $\mathbf{S}$-subproblem, which can be derived as:

$${\mathbf{S}}^t={(\mathbf{Q}+\mu \mathbf{I}+\mu {\mathbf{T}}^T\mathbf{T})}^{-1}(\mathbf{P}+\mu ({\mathbf{M}}^{t-1}+{\mathbf{H}}^{t-1})+\mu {\mathbf{T}}^T({\mathbf{N}}^{t-1}+{\mathbf{B}}^{t-1})),$$

(19)

where $\mathbf{I}$ is the identity matrix. The second step of (18) is to solve the $\mathbf{M}$-subproblem, which is the well-known soft threshold [20]:

$${\mathbf{M}}^t=\mathrm{soft}({\mathbf{S}}^t-{\mathbf{H}}^{t-1},\lambda /\mu ),$$

(20)

where $\mathrm{soft}(·,\tau )$ denotes the component-wise application of the soft-threshold function $y=\mathrm{sign}\left(y\right)max\left\{\right|y|-\tau ,0\}$. The third step of (18) is to solve a linear system, which can be written as:

$$\underset{\mathbf{N}}{min}\frac{1}{2}{\parallel \mathbf{N}-({\mathbf{TS}}^t-{\mathbf{B}}^{t-1})\parallel }_F^2+\frac{\alpha }{2\mu }\mathrm{Tr}\left(\mathbf{N}\mathbf{L}{\mathbf{N}}^T\right)+{\iota }_0({\mathbf{N}}_{\mathsf{\Lambda }}-\mathbf{T}).$$

(21)

If removing the last term, the solution of this system can be written as:

$${\mathbf{N}}^t=({\mathbf{TS}}^t-{\mathbf{B}}^{t-1}){\left(\frac{\alpha }{\mu }\mathbf{L}+\mathbf{I}\right)}^{-1}.$$

(22)

Let $\overline{\mathsf{\Lambda }}$ be the complementary set of $\mathsf{\Lambda }$ under the complete set $\mathcal{I}$. Then, the optimization problem (21) can be rewritten as:

$$\underset{{\mathbf{N}}_{\overline{\mathsf{\Lambda }}}}{min}\frac{1}{2}{\parallel \left[{\mathbf{N}}_{\mathsf{\Lambda }}{\mathbf{N}}_{\overline{\mathsf{\Lambda }}}\right]-(\mathbf{T}\left[{\mathbf{S}}_{\mathsf{\Lambda }}^t{\mathbf{S}}_{\overline{\mathsf{\Lambda }}}^t\right]-\left[{\mathbf{B}}_{\mathsf{\Lambda }}^{t-1}{\mathbf{B}}_{\overline{\mathsf{\Lambda }}}^{t-1}\right])\parallel }_F^2+\frac{\alpha }{2\mu }\mathrm{Tr}\left(\left[{\mathbf{N}}_{\mathsf{\Lambda }}{\mathbf{N}}_{\overline{\mathsf{\Lambda }}}\right]\left[\begin{array}{cc}{\mathbf{L}}_{\mathsf{\Lambda }\mathsf{\Lambda }}& {\mathbf{L}}_{\mathsf{\Lambda }\overline{\mathsf{\Lambda }}}\\ {\mathbf{L}}_{\overline{\mathsf{\Lambda }}\mathsf{\Lambda }}& {\mathbf{L}}_{\overline{\mathsf{\Lambda }}\overline{\mathsf{\Lambda }}}\end{array}\right]{\left[{\mathbf{N}}_{\mathsf{\Lambda }}{\mathbf{N}}_{\overline{\mathsf{\Lambda }}}\right]}^T\right),$$

(23)

where ${\mathbf{L}}_{\mathsf{\Lambda }\overline{\mathsf{\Lambda }}}={\mathbf{L}}_{\overline{\mathsf{\Lambda }}\mathsf{\Lambda }}^T\in {\mathbb{R}}^{J\times (I-J)}$ and ${\mathbf{N}}_{\mathsf{\Lambda }}=\mathbf{T}$. The solution of (23) can be derived as:

$${\mathbf{N}}_{\overline{\mathsf{\Lambda }}}=\left({\mathbf{TS}}_{\overline{\mathsf{\Lambda }}}^t-{\mathbf{B}}_{\overline{\mathsf{\Lambda }}}^{t-1}-\frac{\alpha }{\mu }{\mathbf{N}}_{\mathsf{\Lambda }}{\mathbf{L}}_{\mathsf{\Lambda }\overline{\mathsf{\Lambda }}}\right){\left(\frac{\alpha }{\mu }{\mathbf{L}}_{\overline{\mathsf{\Lambda }}\overline{\mathsf{\Lambda }}}+\mathbf{I}\right)}^{-1}.$$

(24)

For both (22) and (24), one has to solve a large linear system. Fortunately, $\mathbf{L}$ is a sparse and diagonally dominant matrix, and one can take the Gauss–Seidel method to get approximate solutions to these two systems. The last two steps of (18) are designed to update the auxiliary variables. The procedure of the proposed algorithm is detailed as follows:

• Input: A training dictionary $\mathbf{A}\in {\mathbb{R}}^{L\times J}$ and a hyperspectral data matrix $\mathbf{X}\in {\mathbb{R}}^{L\times I}$.
• Choose $\beta$, and compute the weight matrix $\mathbf{W}$ according to (11).
• Select the $\gamma$ parameter for the RBF kernel and compute the matrices $\mathbf{Q}$ and $\mathbf{P}$.
• Set $t=1$; choose $\mu$, $\lambda$, $\alpha$, ${\mathbf{S}}^1$, ${\mathbf{M}}^1$, ${\mathbf{N}}^1$, ${\mathbf{H}}^1$, ${\mathbf{B}}^1$.
• Repeat
• Compute ${\mathbf{S}}^t$, ${\mathbf{M}}^t$, ${\mathbf{N}}^t$, ${\mathbf{H}}^t$, ${\mathbf{B}}^t$ using (18).
• $t=t+1$.
• Until some stopping criterion is satisfied.
• Output: The estimated label of ${\mathbf{x}}_i$ using (6), $i=1,\dots ,I$.

3.3. Analysis and Comparison

The proposed spatial-spectral graph regularization with location (SSGL) method is designed by appending two spatial-spectral constraint terms to the sparse recovery model of KSRC. One is a graph-based spatially-smooth constraint, and the other is a spatial location constraint. When using the two terms, the following two issues should be mentioned.

• The graph-based spatially-smooth constraint is proposed by measuring the spatial relationship between every two spatially-adjacent pixels. If using this term, the test pixels should be arranged in the form of images.
• The spatial location constraint is exploited by integrating the location information of anchor samples, which are assumed to be taken from the test area and labeled by experts.

As described in the introduction part, deep models attract a lot of attention and provide competitive classification results very recently. Although the proposed SSGL method and the deep learning methods belong to two distinct techniques, we show the difference between them considering a further improvement of this work. In

Table 1. Difference between the proposed spatial-spectral graph regularization with location information (SSGL) method and the deep learning methods.

	Deep Learning Methods	SSGL
No. of parameters	Several parameters needed to be trained.	Four model parameters: $\beta$, $\gamma$, $\lambda$ and $\alpha$. One algorithm parameter $\mu$.
No. of training samples	Abundant training samples needed.	Moderate (or even limited) training samples needed.
Computational cost	Expensive	Moderate
Flexibility	Robust to the spectral distortion.	Restricted to the test area.
Accuracy	High	Moderate

, we compare the proposed SSGL method and the deep learning methods in the respects of parameter, training sample, computational cost, flexibility, and accuracy.

4. Experimental Results

4.1. Datasets

To test the performance of the proposed method, three real hyperspectral images have been considered.

(1) Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Indian Pines: The first image in our experiments deals with the standard AVIRIS image taken over northwest Indiana’s Indian Pine test site in June 1992. There are 220 bands in the image, covering the wavelength range of 0.4–2.5 $\mathsf{\mu }$m. The spectral and spatial resolutions are 10 nm and 17 m, respectively. This image consists of 145 × 145 pixels and 16 ground truth classes ranging from 20–2468 pixels in size. The false color image and the ground truth map are shown in Figure 2. We removed 20 noisy bands covering the region of water absorption and worked with 200 spectral bands. In the experiments, about 5% of the labeled pixels were randomly chosen for training, and the rest were used for testing, as shown in

Table 2. Sixteen ground-truth classes in the AVIRIS Indian Pines dataset and the number of training and test pixels used in the experiments.

Class		Set
No.	Name	Train	Test
C01	Alfalfa	3	51
C02	Corn-no till	72	1362
C03	Corn-min till	42	792
C04	Corn	12	222
C05	Grass/pasture	25	472
C06	Grass/trees	38	709
C07	Grass/pasture-mowed	2	24
C08	Hay-windrowed	25	464
C09	Oats	2	18
C10	Soybeans-no till	49	919
C11	Soybeans-min till	124	2344
C12	Soybean-clean till	31	583
C13	Wheat	11	201
C14	Woods	65	1229
C15	Bldg-grass-tree drives	19	361
C16	Stone-steel towers	5	90
Total		525	9841

.

(2) Reflective Optics System Imaging Spectrometer (ROSIS) University of Pavia: This dataset is an urban image acquired by the ROSIS sensor, with spectral coverage ranging from 0.43–0.86 $\mathsf{\mu }$m. The ROSIS sensor has a spatial resolution of 1.3 m per pixel, with 115 spectral bands. This image, with a size of 610 × 340 pixels, contains 103 spectral bands after the removal of noisy bands. There are nine ground truth classes of interest. The false color image and the ground truth map are shown in Figure 3. In the experiments, we randomly chose 40 samples per class for training and used the rest for testing, as shown in

Table 3. Nine ground-truth classes in the ROSIS University of Pavia dataset and the number of training and test pixels used in the experiments.

Class		Set
No.	Name	Train	Test
C1	Asphalt	40	6812
C2	Meadow	40	18,646
C3	Gravel	40	2167
C4	Trees	40	3396
C5	Metal sheets	40	1338
C6	Bare soil	40	5064
C7	Bitumen	40	1316
C8	Bricks	40	3838
C9	Shadows	40	986
Total		360	43,563

.

(3) AVIRIS Kennedy Space Center: This image was collected by the AVIRIS sensor over the Kennedy Space Center, Florida, in March 1996. There are 224 bands in the image, covering the wavelength range of 0.4–2.5 $\mathsf{\mu }$m. The spectral and spatial resolutions are 10 nm and 18 m, respectively. This image acquired over an area of 512 × 614 pixels contains 176 spectral bands after removing water absorption and low signal-to-noise bands. Thirteen ground-truth classes ranging from 105–927 pixels in size that occur in this scene are defined for this image. The false color image and the ground truth map are shown in Figure 4. In the experiments, 5% of the labeled pixels were randomly chosen for training, and the rest were used for testing, as shown in

Table 4. Thirteen ground-truth classes in the AVIRIS Kennedy Space Center dataset and the number of training and test pixels used in the experiments.

Class		Set
No.	Name	Train	Test
C01	Scrub	39	722
C02	Willow swamp	13	230
C03	Cabbage palm hammock	13	243
C04	Cabbage palm/oak hammock	13	239
C05	Slash pine	9	152
C06	Oak/broadleaf hammock	12	217
C07	Hardwood swamp	6	99
C08	Graminoid marsh	22	409
C09	Spartina marsh	26	494
C10	Cattail marsh	21	383
C11	Salt marsh	21	398
C12	Mud flats	26	477
C13	Water	47	880
Total		268	4943

.

4.2. Model Development and Experimental Setup

In the experiments, we compared the proposed SSGL method with four widely-used spatial-spectral classification methods based on the KSRC classifier. The first one is the majority voting (MV) method that performs a pixel-wise KSRC followed by majority voting within superpixel regions [21,22], where the superpixel regions are obtained by the entropy rate superpixel (ERS) algorithm [23]. The second one is the kernel joint SRC (KJSRC) method using kernel joint sparse representation classification [13,24], where the superpixel-based adaptive neighborhoods are used [24]. The third one is the condition random fields (CRF) method that combines pixel-wise KSRC and superpixel segmentation by using condition random fields [25]. The last one is the composite kernel with extended multi-attribute profile features (CKEMAP) method that integrates a composite kernel [7] into KSRC for the incorporation of spatial-spectral information, where the spatial features are extracted by EMAP [26,27]. In order to show the influence of the proposed two terms, we included a degeneration version (i.e., SSG) of SSGL for comparison, where the location information constraint term was dropped. Moreover, the pixel-wise KSRC [17] that only uses spectral features was also included as a baseline classifier. The corresponding optimization problems of the compared methods were all solved by ADMM. For the pixel-wise KSRC, the parameter $\gamma$ for the RBF kernel was experimentally set to two for the AVIRIS Indian Pines dataset, 0.5 for the ROSIS University of Pavia dataset, and 0.125 for the AVIRIS Kennedy Space Center dataset, and the parameters $\lambda$ and $\mu$ were empirically set to ${10}^{-4}$ and ${10}^{-3}$, respectively. For further details, one can refer to the existing literature [14]. Considering that all compared methods are based on the KSRC classifier, their parameters brought by KSRC were experimentally set to be the same as those of the pixel-wise KSRC. For MV, the additional parameters brought by the ERS segmentation were carefully optimized by reference to [28]. For both KJSRC and CRF, the parameters brought by the ERS segmentation were fixed to be the same as those of MV, and the remaining parameters were obtained by cross-validation. For CKEMAP, the EMAP features were extracted from the first three principal components of the hyperspectral image, and the area and standard deviation attributes were considered to build EMAP, as reported in [29]. More specifically, for the area attribute the following values were selected for references: 49, 169, 361, 625, 961, 1369, 1849, and 2401. For the standard deviation, the EMAPs were built according to the following reference values with respect to the mean of the individual features: 2.5%, 5%, 7.5%, and 10%. As for the weight parameter and the kernel parameters in the composite kernel framework [7], they were obtained by cross-validation. For the proposed method, $\mu$ was set to ${10}^{-4}$, $\alpha$ was set to 1, and $\beta$ was experimentally set to 50 for the AVIRIS Indian Pines dataset and 100 for both the ROSIS University of Pavia dataset and the AVIRIS Kennedy Space Center dataset. As for SSG, the same parameters were used as SSGL.

Before the following experiments, the original data were scaled in the range [0, 1]. The classification accuracies are assessed with OA (overall accuracy), AA (average accuracy), and KA (kappa coefficient of agreement). The quantitative measures were obtained by averaging ten runs to avoid any bias induced by random sampling. All experiments were carried out in a 64-b quad-core CPU 2.40-GHz processor with 8 GB of memory.

4.3. Numerical and Visual Comparisons

Table 5. Classification accuracies for the AVIRIS Indian Pines dataset using different classification methods. The best results are highlighted in bold. AA: average accuracy; CKEMAP: composite kernel with extended multi-attribute profile features; CRF: condition random fields; KA: kappa coefficient of agreement; KSRC: kernel sparse representation classification; KJSRC: kernel joint SRC; MV: majority voting; OA: overall accuracy; SSG: spatial-spectral graph regularization with location information; SSGL: spatial-spectral graph regularization with location information.

Class Type	KSRC	MV	KJSRC	CRF	CKEMAP	SSG	SSGL
C01	56.67	72.35	81.57	80.78	89.61	68.43	91.18
C02	78.33	85.27	87.22	88.84	91.24	79.90	95.01
C03	64.31	82.15	92.29	89.31	96.74	76.98	96.48
C04	52.07	90.86	87.88	91.98	84.91	69.37	85.68
C05	89.03	90.97	90.97	88.88	92.65	90.15	93.09
C06	96.46	96.11	96.33	99.31	98.70	98.58	98.77
C07	74.58	19.17	19.17	56.25	96.67	88.33	97.08
C08	98.75	99.78	99.57	99.66	99.70	99.44	99.48
C09	57.78	30.00	40.00	20.00	97.78	51.67	93.33
C10	72.87	86.74	85.91	86.59	90.48	84.41	92.76
C11	82.43	97.42	93.95	97.11	96.92	93.78	97.42
C12	76.74	97.75	95.61	97.99	91.36	90.03	96.72
C13	98.76	100	99.00	99.60	99.50	99.45	99.55
C14	95.30	99.78	97.99	98.53	99.06	98.49	99.29
C15	53.80	69.56	86.26	90.61	95.51	73.16	90.42
C16	88.56	91.89	96.78	96.33	93.78	87.89	88.67
OA(%)	81.33	91.92	92.39	93.83	95.17	88.97	96.16
AA(%)	77.28	81.86	84.41	86.36	94.66	84.38	94.68
KA(%)	78.66	90.75	91.34	92.96	94.50	87.39	95.62
Time(s)	17.03	17.33	17.31	18.48	23.48	34.69	36.16

summarizes the class-specific and global classification accuracies for the AVIRIS Indian Pines dataset. The processing times in seconds are also included for reference. Figure 5 shows the corresponding classification maps. From Table 5, it can be seen that all of the spatial-spectral methods yielded higher classification accuracies when compared to the pixel-wise KSRC. SSGL gave the highest global and most of the best class-specific accuracies, and CKEMAP performed the second highest. For the proposed methods SSG and SSGL, the improvement of SSGL over SSG was significant. From Figure 5, it can be seen that the maps of the spatial-spectral methods contain more homogeneous regions when compared with the map of the pixel-wise KSRC. The misclassified regions of SSGL and CKEMAP were smaller than those of the others.

The class-specific and global classification accuracies for the ROSIS University of Pavia dataset are provided in

Table 6. Classification accuracies for the ROSIS University of Pavia dataset using different classification methods. The best results are highlighted in bold.

Class Type	KSRC	MV	KJSRC	CRF	CKEMAP	SSG	SSGL
C1	73.25	91.69	92.01	91.92	97.30	93.30	97.49
C2	83.24	93.23	95.25	94.77	94.94	97.01	98.48
C3	78.68	93.01	90.90	91.54	91.86	90.58	97.70
C4	91.65	86.39	89.32	91.58	95.58	94.44	94.90
C5	99.42	99.17	99.49	99.89	99.57	99.37	99.36
C6	84.77	98.20	99.26	99.53	96.85	98.70	98.90
C7	92.65	99.70	95.30	99.06	98.40	99.70	99.86
C8	77.99	98.00	70.73	94.72	95.52	96.11	99.00
C9	99.27	95.86	89.62	99.73	99.60	99.50	99.51
OA(%)	82.96	93.88	92.38	94.86	95.83	96.24	98.19
AA(%)	86.77	95.03	91.32	95.86	96.62	96.52	98.36
KA(%)	78.21	92.04	90.05	93.32	94.56	95.09	97.63
Time(s)	98.80	100.61	137.69	107.55	322.54	224.29	210.24

, as well as the processing times. The corresponding classification maps are shown in Figure 6. From the table, it is clear that SSGL performed best followed by SSG, and the performances of the spatial-spectral methods were better than those of the pixel-wise method. The numerical comparisons are confirmed by inspecting the classification maps.

Table 7. Classification accuracies for the AVIRIS Kennedy Space Center dataset using different classification methods. The best results are highlighted in bold.

Class Type	KSRC	MV	KJSRC	CRF	CKEMAP	SSG	SSGL
C01	95.69	100	100	100	99.03	99.18	99.04
C02	85.09	79.43	80.61	88.30	88.39	90.22	95.35
C03	90.62	97.94	97.94	98.31	97.08	98.02	98.27
C04	46.78	63.05	50.92	56.90	86.86	45.48	88.49
C05	61.64	82.76	82.63	82.30	87.63	80.13	92.57
C06	45.58	73.04	69.45	60.05	89.08	35.02	99.12
C07	83.54	92.83	100	98.99	89.29	99.09	100
C08	89.29	99.12	99.29	98.78	98.07	98.26	99.05
C09	96.01	98.42	98.42	98.42	98.54	98.34	98.34
C10	95.30	87.08	87.57	100	99.19	99.63	99.71
C11	94.30	86.83	90.65	98.94	97.74	96.71	97.14
C12	88.05	100	78.07	97.17	97.04	95.30	97.63
C13	99.84	100	100	100	100	100	100
OA(%)	88.45	93.01	90.70	94.35	96.63	92.15	98.01
AA(%)	82.44	89.27	87.35	90.63	94.46	87.34	97.29
KA(%)	87.12	92.19	89.60	93.69	96.25	91.22	97.78
Time(s)	111.68	116.40	576.78	153.00	626.01	192.24	198.11

shows the class-specific and global classification accuracies and the processing times for the AVIRIS Kennedy Space Center dataset. It can be seen that Table 7 reveals almost the same results as Table 5. The classification maps shown in Figure 7 confirm the numerical comparisons further.

In this set of experiments, it can be seen that SSGL performed best in all scenes. This demonstrates the superiority of the spatial location constraint as well as the graph-based spatially-smooth constraint. However, for SSG, it is more suitable to classify the ROSIS scene. This is because the spatial resolution of the ROSIS scene is higher than those of the two AVIRIS scenes, and thus the spectral similarity of the spatially-adjacent pixels of which is more suitable for SSG to measure.

4.4. Analysis of Parameters

Among the parameters of the proposed method, $\beta$ was set experimentally and differently for the three given datasets. In this set of experiments, we evaluated the influence of $\beta$ by varying it from 10–200. Figure 8 shows the classification accuracies for both SSGL and SSG when applied to the three given datasets. It can be seen that when $\beta$ is small, the classification performance of SSG increases monotonically as $\beta$ increases; and the results typically saturate for large $\beta$s. When $\beta$ is relatively large, the accuracies of SSGL decrease slightly. In all cases, we can choose $\beta$ in a wide optimal region. Thus, the choice of $\beta$ is robust.

4.5. Influence of Anchor Samples

In SSGL, the spatial location constraint is built on the prior knowledge of the location information of training samples. That is to say, all training samples are treated as anchor samples. In this set of experiments, we built the spatial location constraint term by using different percentages of training sample to show the influence of anchor samples. Figure 9 shows the classification accuracies for SSGL when applied to the three given datasets. From the figure, we can see that the accuracies increase monotonically as the percentage increases, but the accuracies reach the optimal values asymptotically when a moderate number of anchor samples are used. Thus, we can build the spatial location term by using a relatively small number of anchor samples.

4.6. Different Numbers of Training Samples

In this set of experiments, we tested the performance of the compared classification methods in an ill-posed scenario, where different numbers of training pixels are used. Specifically, for both the AVIRIS Indian Pines dataset and the AVIRIS Kennedy Space Center dataset, we randomly chose 1–20% of the labeled pixels per class for training and the remaining pixels for testing. For the classes with limited labeled pixels, at least two pixels were chosen for training. For the ROSIS University of Pavia dataset, we built training sets by randomly choosing 10, 20, 40, 60, 80, and 100 training pixels per class and used the rest as test sets. Figure 10 shows the mean value and standard deviation of OA as a function of the number of training pixels for the three given datasets. From the figures, it can be seen that the proposed SSGL outperformed the compared methods in almost all cases. As expected, the performance of the spatial-spectral methods was better than that of the pixel-wise KSRC, and the classification accuracies decreased when the number of training pixels was reduced. When compared with the AVIRIS scene, SSG is more suitable for the ROSIS scene.

5. Discussion and Conclusions

A regularized kernel sparse representation method for spatial-spectral classification of hyperspectral images has been presented in this paper, where the spatial-spectral information of hyperspectral images is incorporated by appending two spatial-spectral constraint terms to the sparse recovery model of KSRC. One is the graph-based spatially-smooth constraint, which is designed to measure the spectral similarity of spatially-adjacent pixels. The other is the spatial location constraint, which is used to exploit the wealth of training pixels and thereby to reduce the cost of acquiring a large number of labeled pixels. By introducing the two constraints, the spatial and label information of the hyperspectral pixels—especially that of the training pixels—can be spread to their neighbors, which is very useful for the discrimination of each class. Experiments conducted on three real hyperspectral images have demonstrated that the proposed method can yield accurate classification results. Although the results obtained by the proposed method are very encouraging in hyperspectral image classification, further enhancements such as more robust constraint terms and more reasonable regularization strategies should be pursued in future developments.