Hyperspectral Unmixing with Bandwise Generalized Bilinear Model

Abstract

Generalized bilinear model (GBM) has received extensive attention in the field of hyperspectral nonlinear unmixing. Traditional GBM unmixing methods are usually assumed to be degraded only by additive white Gaussian noise (AWGN), and the intensity of AWGN in each band of hyperspectral image (HSI) is assumed to be the same. However, the real HSIs are usually degraded by mixture of various kinds of noise, which include Gaussian noise, impulse noise, dead pixels or lines, stripes, and so on. Besides, the intensity of AWGN is usually different for each band of HSI. To address the above mentioned issues, we propose a novel nonlinear unmixing method based on the bandwise generalized bilinear model (NU-BGBM), which can be adapted to the presence of complex mixed noise in real HSI. Besides, the alternative direction method of multipliers (ADMM) is adopted to solve the proposed NU-BGBM. Finally, extensive experiments are conducted to demonstrate the effectiveness of the proposed NU-BGBM compared with some other state-of-the-art unmixing methods.

1. Introduction

Hyperspectral images (HSIs) are usually acquired in hundreds of narrow contiguous spectral bands by a specific kind of imaging sensor, e.g., the Airborne Visible/Infrared Imaging Spectrometer, Hyperspectral Digital Imagery Collection Experiment and Compact Airborne Spectrographic Imager [1,2,3]. Due to the high spectral resolution, it is inevitable to bring about the problem of “mixed pixels”, and different materials usually occupy a single hyperspectral pixel [4,5,6]. The existence of mixed pixels have a large impact on many applications, such as object detection [7], subpixel mapping [8], classification [9,10,11,12], and matching [13,14,15]. Thus, hyperspectral unmixing has been exploited to decompose mixed pixels into a group of pure materials (called endmembers) and their corresponding proportions (called abundances) [16,17].

The linear mixing model (LMM) is a simple and widely used unmixing model. Each pixel can be formulated as a linear combination of endmembers with additional noise. The idea is that each incoming light ray interacts with only one material before reaching the sensor [18,19]. However, the LMM is invalid when there are intimate mixtures, terrain relief, or volumetric scattering [20]. Nonlinear mixing models (NLMMs) provide an alternative to take the above mentioned problem into account, which can be generally divided into two categories [20,21]. The first category includes some flexible models based on signal processing, such as post-nonlinear model [22], neural network model [23] and kernel model [24]. The second category includes some physical based models, such as intimate mixture model [25], bilinear mixture model (BMM) [26,27,28,29,30,31,32,33] and multilinear mixing model [34,35,36]. Among them, the BMM only takes the second-order scattering into consideration, while the higher-order interactions of light are ignored [18]. The reason is that the interactions of orders larger than two, not only have minor contribution for improving the unmixing accuracy than that of second-order scattering, but also bring in tremendous computational costs [21]. Several representative models known as the family of BMM have been proposed. The Nascimento model (NM) [26] is an extended LMM with additional virtual endmembers, the Fan’s model (FM) is the truncated Taylor expansion of nonlinear mixing function [27], and the GBM [28] can be seen as the generalization of LMM and FM, which can efficiently deal with the assumptions in BMM. Different methods have been proposed for GBM unmixing of HSI, Halimi et al. developed a Bayesian algorithm to estimate the abundance and noise variance of the GBM [28]. Besides, they also proposed a pixel-wise unmixing method based on the gradient descent algorithm (GDA) [29]. Moreover, Yokoya et al. proposed the semi-nonnegative matrix factorization (semi-NMF) as a new optimization method for GBM based HSI unmixing [30]. Furthermore, Li et al. developed the bound projected optimal gradient method (BPOGM) for GBM unmixng, and it can achieve the optimal convergence rate of $O\left(\frac{1}{k^2}\right)$, where k denotes the number of iteration in BPOGM [31].

Most unmixing methods based on the GBM are implicitly developed for Gaussian noise, and the underlying assumption is that the intensities of Gaussian noise remain the same for different bands of HSI. However, there remain two challenges for many real applications of GBM based hyperspectral unmixing. One is that each band of HSI is degraded by different intensities of AWGN, and the other is the widely existed mixed noise in real HSI, including Gaussian noise, impulse noise, dead pixels or lines, stripes and so on [37,38,39,40]. Aggarwal et al. proposed a hyperspectral unmixing method in the presence of mixed noise using joint-sparsity and total variation, which can take several kinds of noise into consideration [41]. However, it still ignored the nonlinear mixing effect and the different intensities of AWGN in different bands of HSI. In this paper, to overcome the above mentioned problems, we propose a novel nonlinear unmixing method based on the bandwise generalized bilinear model (NU-BGBM). First, a bandwise generalized bilinear model (BGBM) is proposed to take the complex mixed noise in real HSI into consideration, i.e., each band of HSI is contaminated by different types of noise, and the AWGN intensities of across different bands are assumed to be different. Second, a novel method based on the BGBM is proposed under the maximum a posteriori framework. The diagonal of weight matrix is the reciprocal of the variance of AWGN in each band, which can take the different intensities of AWGN in different bands into consideration. The impulse noise, dead pixels or lines, and stripes usually contaminate a small part of the whole HSI, which usually have the underlying sparse property.

The main contributions of this work lie in that we propose a BGBM to be adapted for complex mixed noise in real HSI. Besides, we propose a novel NU-BGBM under the maximum a posteriori framework, which can take different types of noise into consideration. Moreover, we use the alternative direction method of multipliers (ADMM) [42] for solving the proposed NU-BGBM. Finally, we conduct extensive experiments using both simulated and real HSIs to demonstrate the effectiveness and advantages of the proposed NU-BGBM.

The remainder of this paper is organized as follows. In Section 2, we introduce the related GBM and the formulation of the proposed NU-BGBM, and develop the ADMM for solving the proposed NU-BGBM. In Section 3, we demonstrate the efficiency and advantages of the proposed NU-BGBM on both simulated datasets and real HSIs. Finally, we conclude this paper in Section 4.

2. Bandwise Generalized Bilinear Model and Algorithm

In this section, we will introduce the related GBM at first. Then, we will introduce the proposed nonlinear unmixing model BGBM, and formulate the proposed unmixing method NU-BGBM in detail. Finally, we will solve the proposed NU-BGBM under the ADMM scheme.

2.1. The Related GBM

The LMM assumes that each pixel $\mathbf{y}\in {\mathbf{R}}^{D\times 1}$ containing D bands in HSI is a linear combination of M endmembers $\mathbf{E}=[{\mathbf{e}}_1,\dots ,{\mathbf{e}}_M]\in {\mathbf{R}}^{D\times M}$ as follows:

$$\mathbf{y}=\mathbf{E}\mathbf{a}+\mathbf{n},$$

(1)

where the linear representation vector $\mathbf{a}={[a_1,\dots ,a_M]}^T\in {\mathbf{R}}^{M\times 1}$ denotes the abundance, and $\mathbf{n}$ represents the additive noise in HSI. The abundances usually have to meet two constraints, i.e., abundance non-negativity constraint (ANC) and abundance sum to one constraint (ASC) as follows:

$$\begin{array}{c}{a}_i\ge 0,\forall i=1,\dots ,M,\hfill \\ \sum _{i=1}^M{a}_i=1.\hfill \end{array}$$

(2)

However, the real HSI usually has strong signature variability [43], and the abundances may not meet the ASC constraint in practice, so we do not explicitly impose the ASC constraint for unmixing of HSI in this paper.

As noted in the introduction, the LMM may not always hold true in many situations. To take the multi-scattering radiation among different endmembers into consideration, BMM takes the second-order scattering into consideration, which is equivalent to add an additional second-order interaction term to LMM as follows:

$$\mathbf{y}=\mathbf{E}\mathbf{a}+\sum _{i=1}^{M-1}\sum _{j=i+1}^M{b}_{i,j}{\mathbf{e}}_i\odot {\mathbf{e}}_j+\mathbf{n},$$

(3)

where $b_{i,j}$ denotes the amount of nonlinearities between the ith and jth endmembers, and ⊙ is the Hadamard product operation. By imposing various constraints on the nonlinear coefficient $b_{i,j}$ in BMM, many different kinds of BMMs have been proposed, which include NM [26], FM [27], GBM [28,29,30,31], MGBM [44] and so on. Among them, the GBM can be seen as the generalization of LMM and FM, it sets the nonlinear coefficient $b_{i,j}={\gamma }_{i,j}{a}_i{a}_j$, and the constraints imposed on the GBM can be written as follows:

$$\begin{array}{c}{a}_i\ge 0,\forall i=1,\dots ,M,\hfill \\ {\gamma }_{i,j}=0,\forall i,j=1,\dots ,M,i\ge j,\hfill \\ 0\le {\gamma }_{i,j}\le 1,\forall i,j=1,\dots ,M,i<j.\hfill \end{array}$$

(4)

Mathematically, the GBM for HSI having P pixels can be expressed as follows:

$$\mathbf{Y}=\mathbf{E}\mathbf{A}+\mathbf{F}\mathbf{B}+\mathbf{N},$$

(5)

where $\mathbf{Y}\in {\mathbf{R}}^{D\times P}$ denotes the reshaped HSI matrix, $\mathbf{A}=[{\mathbf{a}}_1,\dots ,{\mathbf{a}}_P]\in {\mathbf{R}}^{M\times P}$ is the abundance matrix, $\mathbf{F}=[{\mathbf{e}}_1\odot {\mathbf{e}}_2,\dots ,{\mathbf{e}}_{M-1}\odot {\mathbf{e}}_M]\in {\mathbf{R}}^{D\times M(M-1)/2}$ denotes the bilinear endmember matrix, $\mathbf{B}=[{\gamma }_{1,2}{\mathbf{a}}_1\odot {\mathbf{a}}_2,\dots ,{\gamma }_{M-1,M}{\mathbf{a}}_{M-1}\odot {\mathbf{a}}_M]\in {\mathbf{R}}^{M(M-1)/2\times P}$ is the bilinear abundance matrix, and $\mathbf{N}\in {\mathbf{R}}^{D\times P}$ denotes the noise matrix. Therefore, the constraints imposed on the GBM having P pixels can be written as follows:

$$\begin{array}{c}\mathbf{A}\le \mathbf{0},\hfill \\ \mathbf{0}\le \mathbf{B}\le \mathbf{C},\hfill \end{array}$$

(6)

where ${\mathbf{C}}_{(i,j),k}={\mathbf{A}}_{i,k}{\mathbf{A}}_{j,k}(k\in \{1,\dots ,P\})$.

2.2. Formulation of the Proposed BGBM and the Corresponding Unmixing Method NU-BGBM

The traditional GBM unmixing methods are implicitly developed in the presence of additive white Gaussian noise (AWGN), and the intensity of AWGN is assumed to be the same for each band of HSI. However, the HSIs in real world are usually degraded by mixed noise, i.e., the HSI is usually polluted by not only the Gaussian noise, but also other types of noise, such as impulse noise, dead pixels or lines, and stripes [37,38,39,40]. Besides, the intensity of AWGN is usually different for different band of HSI. The noise in real HSI can be generally classified into two classes, i.e., dense noise and sparse noise. The dense noise means that most of the HSI is contaminated, which mainly includes Gaussian noise. While the sparse noise means that only a small part of the HSI is contaminated, which mainly includes impulse noise, dead pixels or lines, and stripes. Therefore, the proposed bandwise unmixing model BGBM can be written as follows:

$${\mathbf{Y}}_i={\left(\mathbf{E}\mathbf{A}\right)}_i+{\left(\mathbf{F}\mathbf{B}\right)}_i+{\mathbf{S}}_i+{\mathbf{N}}_i,i=1,\dots ,D,$$

(7)

where ${\mathbf{Y}}_i$, ${\left(\mathbf{E}\mathbf{A}\right)}_i$, ${\left(\mathbf{F}\mathbf{B}\right)}_i$, ${\mathbf{S}}_i$ and ${\mathbf{N}}_i$ represent the ith band of $\mathbf{Y}$, $\mathbf{E}\mathbf{A}$, $\mathbf{F}\mathbf{B}$, $\mathbf{S}$ and $\mathbf{N}$, respectively. $\mathbf{S}$ and $\mathbf{N}$ denote the sparse noise and dense Gaussian noise of HSI, respectively. We assume that the AWGN of each band is independent with different intensities of Gaussian noise, ${\mathbf{N}}_i\sim \mathcal{N}(\mathbf{0},{\sigma }_i^2{\mathbf{I}}_P)(i=1,\dots ,D)$, and ${\sigma }_i^2$ represent the variance of Gaussian noise in ith band, so we can obtain $p({\mathbf{Y}}_i|[{\left(\mathbf{E}\mathbf{A}\right)}_i+{\left(\mathbf{F}\mathbf{B}\right)}_i+{\mathbf{S}}_i])\sim \mathcal{N}({\left(\mathbf{E}\mathbf{A}\right)}_i+{\left(\mathbf{F}\mathbf{B}\right)}_i+{\mathbf{S}}_i,{\sigma }_i^2{\mathbf{I}}_P)(i=1,\dots ,M)$. Since the AWGN of each band is independent, so we can get

$$\begin{array}{cc}\hfill p(\mathbf{Y}|(\mathbf{E}\mathbf{A}+\mathbf{F}\mathbf{B}+\mathbf{S}))& =\prod _{i=1}^{D}p({\mathbf{Y}}_i|[{\left(\mathbf{E}\mathbf{A}\right)}_i+{\left(\mathbf{F}\mathbf{B}\right)}_i+{\mathbf{S}}_i])\hfill \\ & =cexp(-\sum _{i=1}^D\frac{1}{2{\sigma }_i^2}\parallel {\mathbf{Y}}_i-{\left(\mathbf{E}\mathbf{A}\right)}_i-{\left(\mathbf{F}\mathbf{B}\right)}_i-{\mathbf{S}}_i{\parallel }_2^2)\hfill \\ & =cexp(-\frac{1}{2}\parallel \mathbf{W}(\mathbf{Y}-\mathbf{E}\mathbf{A}-\mathbf{F}\mathbf{B}-\mathbf{S}){\parallel }_F^2),\hfill \end{array}$$

(8)

where c denotes a constant, ${\parallel .\parallel }_F$ represents the Frobenius norm. $\mathbf{W}\in {\mathbb{R}}^{D\times D}$ is the diagonal matrix, and ${\mathbf{W}}_{i,i}=\frac{1}{{\sigma }_i}(i=1,\cdots ,D)$, the diagonal matrix $\mathbf{W}$ can be seen as a weighting matrix, the larger the variance of Gaussian noise is, the smaller the weight of the band is, so the weighting matrix can take different intensities of AWGN in different bands of HSI into consideration. Then, the abundances, bilinear abundance and the sparse noise of BGBM can be estimated under the maximum a posteriori framework as follows:

$$\begin{array}{cc}\hfill [\widehat{\mathbf{A}},\widehat{\mathbf{B}},\widehat{\mathbf{S}}]& =arg\underset{\mathbf{A},\mathbf{B},\mathbf{S}}{max}p((\mathbf{E}\mathbf{A}+\mathbf{F}\mathbf{B}+\mathbf{S})|\mathbf{Y})\hfill \\ & =arg\underset{\mathbf{A},\mathbf{B},\mathbf{S}}{max}p(\mathbf{Y}|(\mathbf{E}\mathbf{A}+\mathbf{F}\mathbf{B}+\mathbf{S}))p(\mathbf{E}\mathbf{A}+\mathbf{F}\mathbf{B}+\mathbf{S})\hfill \\ & =arg\underset{\mathbf{A},\mathbf{B},\mathbf{S}}{min}\frac{1}{2}{\parallel \mathbf{W}(\mathbf{Y}-\mathbf{E}\mathbf{A}-\mathbf{F}\mathbf{B}-\mathbf{S})\parallel }_F^2-ln\left(p(\mathbf{E}\mathbf{A}+\mathbf{F}\mathbf{B}+\mathbf{S})\right),\hfill \end{array}$$

(9)

where $p(\mathbf{E}\mathbf{A}+\mathbf{F}\mathbf{B}+\mathbf{S})$ represents the prior distribution of $\mathbf{E}\mathbf{A}+\mathbf{F}\mathbf{B}+\mathbf{S}$, which can be regarded as prior knowledge or constraints enforced on the abundances, bilinear abundance and the sparse noise of BGBM.

For the abundances and bilinear abundance of BGBM, they should also satisfy the constraints in Equation (6). For the sparse noise of BGBM, it has the underlying property of sparsity. The sparsity of sparse noise in BGBM can be ideally represented by the ${\ell }_0$ norm. However, ${\ell }_0$ norm based optimization is usually NP-hard to solve, so we replace the ${\ell }_0$ norm with ${\ell }_1$ norm. Thus, the proposed nonlinear unmixing method based on the BGBM can be formulated as follows:

$$\begin{array}{cc}\hfill \underset{\mathbf{A},\mathbf{B},\mathbf{S}}{\min }& \frac{1}{2}{\parallel \mathbf{W}(\mathbf{Y}-\mathbf{E}\mathbf{A}-\mathbf{F}\mathbf{B}-\mathbf{S})\parallel }_F^2+\lambda {\parallel \mathbf{S}\parallel }_1,\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \mathbf{A}\ge \mathbf{0},\mathbf{0}\le \mathbf{B}\le \mathbf{C},\hfill \end{array}$$

(10)

where $\lambda$ is the regularization parameter to strike a balance between the reconstruction error and the sparse regularization term, and ${\parallel \mathbf{S}\parallel }_1={\sum }_{i=1}^D{\sum }_{j=1}^P|{\mathbf{S}}_{ij}|$.

2.3. Solving the Proposed NU-BGBM with ADMM

The ADMM has been widely used for solving the constrained optimization problems, and it has obtained desirable performance in many different kinds of applications as well as hyperspectral unmixing [20,44,45]. To get more details of ADMM, please refer to [42]. Here, we provide details of adopting ADMM to solve the proposed NU-BGBM.

The model in Equation (10) can be reformulated as follows:

$$\begin{array}{cc}\hfill \underset{\mathbf{A},\mathbf{B},\mathbf{S},{\mathbf{V}}_1,{\mathbf{V}}_2,{\mathbf{V}}_3}{\min }& \frac{1}{2}\parallel \mathbf{W}(\mathbf{Y}-\mathbf{E}\mathbf{A}-\mathbf{F}\mathbf{B}-{\mathbf{V}}_1){\parallel }_F^2+l_{{\mathcal{R}}_{+}}\left({\mathbf{V}}_2\right)+l_{{\mathcal{R}}_{\mathrm{bound}}}\left({\mathbf{V}}_3\right)+\lambda {\parallel \mathbf{S}\parallel }_1,\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathbf{V}}_1=\mathbf{S},\hfill \\ & {\mathbf{V}}_2=\mathbf{A},\hfill \\ & {\mathbf{V}}_3=\mathbf{B},\hfill \end{array}$$

(11)

where $l_{{\mathcal{R}}_{+}}\left(\mathbf{X}\right)={\sum }_{i,j}{l}_{{\mathcal{R}}_{+}}\left({\mathbf{X}}_{i,j}\right)$ denotes the indicator function for the nonnegative orthant ${\mathcal{R}}_{+}$, ${\mathbf{X}}_{i,j}$ is the $i,j$-th element of $\mathbf{X}$, and $l_{{\mathcal{R}}_{+}}\left({\mathbf{X}}_{i,j}\right)$ is zero when ${\mathbf{X}}_{i,j}$ belongs to the nonnegative orthant, and $+\infty$ otherwise. $l_{{\mathcal{R}}_{\mathrm{bound}}}\left(\mathbf{X}\right)={\sum }_{i,j}{l}_{{\mathcal{R}}_{\mathrm{bound}}}\left({\mathbf{X}}_{i,j}\right)$ is the indicator function for the interval $[\mathbf{0},\mathbf{C}]$, and $l_{{\mathcal{R}}_{\mathrm{bound}}}\left({\mathbf{X}}_{i,j}\right)$ is zero when ${\mathbf{X}}_{i,j}$ belongs to the interval $[0,{\mathbf{C}}_{i,j}]$, and $+\infty$ otherwise. ${\mathbf{V}}_1$, ${\mathbf{V}}_2$ and ${\mathbf{V}}_3$ denote three auxiliary variables.

Equation (11) can be reformulated using the compact form as follows:

$$\underset{\mathbf{V},\mathbf{Q}}{\min }g(\mathbf{V},\mathbf{Q})s.t.\mathbf{G}\mathbf{Q}+\mathbf{H}\mathbf{V}=\mathbf{Z},$$

(12)

where $g(\mathbf{V},\mathbf{Q})=\frac{1}{2}\parallel \mathbf{W}(\mathbf{Y}-\mathbf{E}\mathbf{A}-\mathbf{F}\mathbf{B}-{\mathbf{V}}_1){\parallel }_F^2+l_{{\mathcal{R}}_{+}}\left({\mathbf{V}}_2\right)+l_{{\mathcal{R}}_{\mathrm{bound}}}\left({\mathbf{V}}_3\right)+\lambda {\parallel \mathbf{S}\parallel }_1$, $\mathbf{G}=\left[\begin{array}{c}\begin{array}{ccc}\mathbf{I}& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& \mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{I}\end{array}\end{array}\right]$, $\mathbf{Q}=\left[\begin{array}{c}\mathbf{S}\\ \mathbf{A}\\ \mathbf{B}\end{array}\right]$, $\mathbf{H}=\left[\begin{array}{ccc}-\mathbf{I}& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& -\mathbf{I}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& -\mathbf{I}\end{array}\right]$, $\mathbf{V}=\left[\begin{array}{c}{\mathbf{V}}_1\\ {\mathbf{V}}_2\\ {\mathbf{V}}_3\end{array}\right]$, $\mathbf{Z}=\left[\begin{array}{c}\mathbf{0}\\ \mathbf{0}\\ \mathbf{0}\end{array}\right]$.

The augmented Lagrangian function can be written as follows:

$$\mathcal{L}(\mathbf{V},\mathbf{Q},\mathsf{\Lambda })=g(\mathbf{V},\mathbf{Q})+\frac{\mu }{2}{\parallel \mathbf{G}\mathbf{Q}+\mathbf{H}\mathbf{V}-\mathbf{Z}-\mathsf{\Lambda }\parallel }_F^2,$$

(13)

where $\mu >0$ denotes the penalty parameter, $\mathsf{\Lambda }/\mu$ is the Lagrange multipliers. Therefore, we can sequentially optimize $\mathcal{L}$ with respect to $\mathbf{V}$, $\mathbf{Q}$ and $\mathsf{\Lambda }$.

To update $\mathbf{A}$, we solve

$$\begin{array}{cc}\hfill {\mathbf{A}}^{k+1}=& arg\underset{\mathbf{A}}{min}\frac{1}{2}\parallel \mathbf{W}(\mathbf{Y}-\mathbf{E}\mathbf{A}-\mathbf{F}{\mathbf{B}}^k-{\mathbf{V}}_1^k){\parallel }_F^2+\frac{\mu }{2}{\parallel {\mathbf{V}}_2^k-\mathbf{A}-{\mathsf{\Lambda }}_2^k\parallel }_F^2\hfill \\ \hfill =& {[{\left(\mathbf{W}\mathbf{E}\right)}^T\left(\mathbf{W}\mathbf{E}\right)+\mu \mathbf{I}]}^{-1}[{\left(\mathbf{W}\mathbf{E}\right)}^T\mathbf{W}(\mathbf{Y}-\mathbf{F}{\mathbf{B}}^k-{\mathbf{V}}_1^k)+\mu ({\mathbf{V}}_2^k-{\mathsf{\Lambda }}_2^k)].\hfill \end{array}$$

(14)

To update $\mathbf{B}$, we solve

$$\begin{array}{cc}\hfill {\mathbf{B}}^{k+1}=& arg\underset{\mathbf{B}}{min}\frac{1}{2}\parallel \mathbf{W}(\mathbf{Y}-\mathbf{E}{\mathbf{A}}^{k+1}-\mathbf{F}\mathbf{B}-{\mathbf{V}}_1^k){\parallel }_F^2+\frac{\mu }{2}{\parallel {\mathbf{V}}_3^k-\mathbf{B}-{\mathsf{\Lambda }}_3^k\parallel }_F^2\hfill \\ \hfill =& {[{\left(\mathbf{W}\mathbf{F}\right)}^T\left(\mathbf{W}\mathbf{F}\right)+\mu \mathbf{I}]}^{-1}[{\left(\mathbf{W}\mathbf{F}\right)}^T\mathbf{W}(\mathbf{Y}-\mathbf{E}{\mathbf{A}}^{k+1}-{\mathbf{V}}_1^k)+\mu ({\mathbf{V}}_3^k-{\mathsf{\Lambda }}_3^k)].\hfill \end{array}$$

(15)

To update $\mathbf{S}$, we solve

$$\begin{array}{cc}\hfill {\mathbf{S}}^{k+1}=& arg\underset{\mathbf{S}}{min}{\lambda \parallel \mathbf{S}\parallel }_1+\frac{\mu }{2}{\parallel {\mathbf{V}}_1^k-\mathbf{S}-{\mathsf{\Lambda }}_1^k\parallel }_F^2\hfill \\ \hfill =& {\mathcal{S}}_{\lambda /\mu }({\mathbf{V}}_1^k-{\mathsf{\Lambda }}_1^k),\hfill \end{array}$$

(16)

where ${\mathcal{S}}_{\tau }\left[x\right]=\mathrm{sgn}\left(x\right)\max (|x|-\tau ,0)$ represents the element-wise soft shrinkage operator [46], and $\tau$ denotes the threshold.

To update ${\mathbf{V}}_1$, we solve

$$\begin{array}{cc}\hfill {\mathbf{V}}_1^{k+1}=& arg\underset{{\mathbf{V}}_1}{min}\frac{1}{2}\parallel \mathbf{W}(\mathbf{Y}-\mathbf{E}{\mathbf{A}}^{k+1}-\mathbf{F}{\mathbf{B}}^{k+1}-{\mathbf{V}}_1){\parallel }_F^2+\frac{\mu }{2}{\parallel {\mathbf{V}}_1-{\mathbf{S}}^{k+1}-{\mathsf{\Lambda }}_1^k\parallel }_F^2\hfill \\ \hfill =& {[{\mathbf{W}}^T\mathbf{W}+\mu \mathbf{I}]}^{-1}[{\mathbf{W}}^T\mathbf{W}(\mathbf{Y}-\mathbf{E}{\mathbf{A}}^{k+1}-\mathbf{F}{\mathbf{B}}^{k+1})+\mu ({\mathbf{S}}^{k+1}+{\mathsf{\Lambda }}_1^k)].\hfill \end{array}$$

(17)

To update ${\mathbf{V}}_2$, we solve

$$\begin{array}{cc}\hfill {\mathbf{V}}_2^{k+1}=& arg\underset{{\mathbf{V}}_2}{min}{l}_{{\mathcal{R}}_{+}}\left({\mathbf{V}}_2\right)+\frac{\mu }{2}{\parallel {\mathbf{V}}_2-{\mathbf{A}}^{k+1}-{\mathsf{\Lambda }}_2^k\parallel }_F^2\hfill \\ \hfill =& \max ({\mathbf{A}}^{k+1}+{\mathsf{\Lambda }}_2^k,\mathbf{0}).\hfill \end{array}$$

(18)

To update ${\mathbf{V}}_3$, we solve

$$\begin{array}{cc}\hfill {\mathbf{V}}_3^{k+1}=& arg\underset{{\mathbf{V}}_3}{min}{l}_{{\mathcal{R}}_{\mathrm{bound}}}\left({\mathbf{V}}_3\right)+\frac{\mu }{2}{\parallel {\mathbf{V}}_3-{\mathbf{B}}^{k+1}-{\mathsf{\Lambda }}_3^k\parallel }_F^2\hfill \\ \hfill =& \min (\max ({\mathbf{B}}^{k+1}+{\mathsf{\Lambda }}_3^k,\mathbf{0}),\mathbf{C}).\hfill \end{array}$$

(19)

The primal and dual residuals $r^{k+1}$ and $d^{k+1}$ can be written as follows:

$$r^{k+1}=\mathbf{G}{\mathbf{Q}}^{k+1}+\mathbf{H}{\mathbf{V}}^{k+1},$$

(20)

$$d^{k+1}=\mu {\mathbf{G}}^T\mathbf{H}({\mathbf{Q}}^{k+1}-{\mathbf{Q}}^k).$$

(21)

According to [42], the stopping criterion can be written as follows:

$$\begin{array}{c}\parallel r^{k+1}{\parallel }_F/\sqrt{(3M+D)P}\le \epsilon \mathrm{and}\hfill \\ \parallel d^{k+1}{\parallel }_F/\sqrt{(3M+D)P}\le \epsilon .\hfill \end{array}$$

(22)

According to [42], $\mu$ has a large impact on the convergence speed, we update $\mu$ to keep the ratio between the primal norms and dual residual norms within a given positive interval, and they both converge to zero. When adopting the ADMM to solve the proposed NU-BGBM, we need to know the weighting matrix $\mathbf{W}$ before hand, i.e., estimating the stand deviation of AWGN for different bands of HSI. In this paper, the hyperspectral signal identification by minimum error (HySime) [47] is used to estimate the noise of HSI. The HySime assumes that there are usually high correlation among the neighboring bands of HSI, which can be solved by the multiple regression theory. Therefore, the detailed procedure for solving the proposed NU-BGBM is summarized in Algorithm 1.

Algorithm 1: Solving the proposed NU-BGBM with ADMM.

3. Experiments

In this section, we conduct experiments on both synthetic datasets and real HSIs to demonstrate efficiency and advantages of the proposed NU-BGBM. We compare the unmixing performance of the proposed NU-BGBM with the other four methods, i.e., the fully constrained least squares (FCLS) algorithm [48], GDA [29], semi-NMF [30] and BPOGM [31]. The FCLS is based on the LMM, the other three compared methods are all based on the GBM.

For the four compared methods, we adopt the recommended default parameters in the original references. For the proposed NU-BGBM, there are mainly four parameters, i.e., the regularization parameter $\lambda$, the Lagrange multiplier regularization parameter $\mu$, the error tolerance $\epsilon$ and the maximum number of iteration N. We have found that the regularization parameter $\lambda$ has much larger impact on the unmixing performance than the other three parameters. Thus, throughout the experiments, we fix $\mu =0.01$, $\epsilon ={10}^{-6}$ and $N=1000$, and tune $\lambda \in$ {${10}^{-5}$, ${10}^{-4}$, ${10}^{-3}$, ${10}^{-2}$, ${10}^{-1}$, 1, ${10}^1$, ${10}^2$, ${10}^3$, ${10}^4$, ${10}^5$}. Besides, since the GDA would diverge using random initialization [31], so all the GBM based unmixing methods are initialized by FCLS [48] for fair of comparison.

3.1. Experimental Results with Synthetic Data

In this subsection, we will conduct experiments on synthetic data to demonstrate efficiency and advantages of the proposed NU-BGBM in the presence of mixed noise. The bilinear mixing effect often appears between soil and vegetation [30,31], and we select six spectra from the U. S. Geological Survey (USGS) spectral library available at: http://speclab.cr.usgs.gov/spectral-lib.html, i.e., one type of water (Water+Montmor), two types of soil (Stonewall and Rangeland), and three types of vegetation (spurge, oak and pine). We downsample the original selected six spectra to P = 200 bands, and the spectra of them are shown in Figure 1, which are chosen as the six endmembers in the simulated experiment. After obtaining the endmember matrix $\mathbf{E}$, the bilinear endmember matrix $\mathbf{F}$ can be naturally obtained according to BGBM. Then, we generate the abundance matrix $\mathbf{A}$ of six selected endmembers according to [49], and the code is available at: https://bitbucket.org/aicip/mvcnmf. The synthetic HSI has 64 × 64 pixels having no pure pixels, the HSI is divided into 8 × 8 blocks and all pixels in each block are filled up by one of endmembers randomly selected from the selected six endmembers. The spatial low-pass filter of size 9 × 9 has been applied to the HSI to create linear mixture, and all pixels with abundances greater than 80% have been replaced by a mixture of all endmembers with equally distributed abundances, which aims at removing the probable pure pixels in the resulted HSI, and the generated abundances of the six endmembers are shown in the first row in Figure 2. The nonlinearity coefficients are uniformly drawn in the interval [0, 1], and the bilinear abundance matrix $\mathbf{B}$ can be also obtained according to BGBM. Then, three kinds of noise will be added to the HSI to simulate the mixed noise and different intensities of AWGN in each band of HSI in practice:

• Gaussian noise: all bands of the HSI are contaminated by zero mean i.i.d. Gaussian noise, and the signal-to-noise ratio (SNR) of each band is a random number ranging from 10 dB to 50 dB.
• Impulse noise: only 11 bands (60–70) are contaminated by 30% impulse noise.
• Dead lines: only 11 bands (120–130) are contaminated by dead lines.

To simulate the mixed noise in real HSI as much as possible, we conduct experiments in the presence of one type of noise, two types of noise and three types of noise, which can be seen in

Table 1. Comparison of RMSEs ($\times {10}^{-2}$) for different methods in the presence of different types of noise.

Type of Noise	FCLS	GDA	Semi-NMF	BPOGM	NU-BGBM
Gaussian noise	7.103	6.053	5.520	5.157	0.990
Impulse noise	7.123	6.395	6.161	5.403	0.167
Dead lines	6.812	5.773	5.796	5.436	0.171
Gaussian noise & Impulse noise	8.411	7.781	7.651	7.065	1.004
Gaussian noise & Dead lines	8.084	7.185	7.197	6.996	1.003
Impulse noise & Dead lines	7.941	7.254	7.469	6.962	0.296
Gaussian noise & Impulse noise & Dead lines	9.010	8.395	8.609	8.216	1.021

. After adding noise to the synthetic HSI, we adopt the compared methods and the proposed NU-BGBM to unmix the synthetic HSIs using the given true endmembers. To evaluate the unmixing performances of different methods, we adopt the root mean square error (RMSE) [30,31] as the quantitative evaluation criterion. Generally speaking, smaller RMSE indicates better HSI unmixing performance.

Table 1 gives the RMSEs of the compared methods and the proposed NU-BGBM in the presence of different types of noise. From Table 1, we can clearly see that when the synthetic HSI is only contaminated by Gaussian noise with different intensities for each band of HSI, the proposed NU-BGBM can obtain better unmixing performance than the other compared methods. This is due to that only our NU-BGBM can take the different intensities of AWGN in different bands into account under the maximum a posteriori framework. When the HSI is only degraded by impulse noise or dead lines, the proposed NU-BGBM can obtain much better unmixing performance than the other four compared methods, the underlying reason is that only our NU-BGBM can take the underlying sparsity of sparse noise into account, and the four compared methods all adopt the vector ${\ell }_2$ norm or the matrix Frobenius norm, but the vector ${\ell }_2$ norm or the matrix Frobenius norm is usually sensitive to non-Gaussian noise, and it would seriously degrade the unmixing performance in the presence of non-Gaussian noise. In the presence of two types of noise and three types of noise, the proposed NU-BGBM can also obtain better unmixing performances than the other compared methods, which demonstrate that our proposed method based on BGBM can handle the mixed noise and different intensities of AWGN in different bands efficiently, and hence our method is more robust and applicable than the other compared methods in the presence of complex mixed noise. Besides, the FCLS generally has worse unmixing performance than the other methods, this is due to that the FCLS is based on LMM, and the other methods are based on GBM or BGBM, which demonstrates the importance of taking nonlinear mixing effect into account to improve the accuracy of HSI unmixing. Furthermore, Figure 2 shows the abundance maps estimated by the compared methods and the proposed method in the presence of three types of noise, i.e., the synthetic HSI is contaminated by Gaussian noise, impulse noise and dead lines simultaneously. It can be clearly observed from Figure 2 that the abundances of six endmembers estimated by the proposed NU-BGBM can approximate obviously better to the ground truth than the other compared methods, which also demonstrate that our proposed NU-BGBM is more robust for the mixed noise and different intensities of AWGN in different bands than the other compared methods.

3.2. Experimental Results with Real Data

The first real HSI was acquired by the Airborne Visible Infrared Imaging Spectrometer (AVIRIS) over the Cuprite mining site, in the state of Nevada, in 1997. This HSI has 224 spectral bands ranging from 0.4 $\mathsf{\mu }$m to 2.5 $\mathsf{\mu }$m, and the spectral resolution is about 10 nm. The region of interest, containing 50 × 50 pixels, is used to assess the unmixing performance of the proposed NU-BGBM and the other four compared methods. These bands seriously contaminated by water absorption and noise are removed, remaining D = 189 spectral bands. The false color image of the selected Cuprite HSI can be seen in Figure 3a. HySime [47] can be adopted to estimate the number of endmembers in the selected Cuprite HSI. Besides, according to [28], the selected Cuprite HSI mainly has three materials, i.e., alunite, kolinite and muscovite.

For the unmixing of real HSI, GBM and BGBM based HSI unmixing usually uses the two-stage methods. First, endmembers are extracted from the real HSI by endmember extraction methods, and the bilinear endmembers can be naturally obtained from the extracted endmembers. Then, the abundances and bilinear abundances can be estimated by different abundance estimation methods. In this experiment, we adopt the vertex component analysis (VCA) [50] to extract endmembers from the selected Cuprite HSI first, then we estimate the abundances by the proposed NU-BGBM and the other compared methods. The extracted endmembers of the selected Cuprite HSI can be seen in Figure 4. Since the truth abundances are unknown for the real HSI, and we cannot compute the RMSE, so we use the abundance maps to analyze the unmixing performance of different methods qualitatively, and also adopt the reconstruction error (RE) and spectral mean angle distance (SMAD) [30,31] to evaluate the unmixing performance of different methods quantitatively. Figure 5 shows the abundance maps estimated by the proposed NU-BGBM and the other four compared methods for the selected real Cuprite HSI. It can be observed from Figure 5 that the abundance maps, estimated by the proposed NU-BGBM and the other four compared methods for the selected real Cuprite HSI, are fairly in accordance with each other.

Table 2. Comparison of REs ($\times {10}^{-2}$) and SMADs ($\times {10}^{-2}$) of different methods using extracted endmembers for the selected Cuprite real HSI.

	FCLS	GDA	Semi-NMF	BPOGM	NU-BGBM
RE	2.106	1.980	1.481	1.117	1.046
SMAD	3.131	2.920	2.738	2.077	1.891

gives the REs and SMADs of the proposed NU-BGBM and the other four compared methods for the first real HSI. It can be clearly seen from Table 2 that the proposed NU-BGBM can obtain better RE and SMAD than the other four compared methods, which demonstrate that the proposed NU-BGBM is efficient for the unmixing of the selected real Cuprite HSI by taking the mixed noise into account. Besides, the unmixing method based on the BGBM can obtain better RE and SMAD than these unmixing methods based on the LMM and GBM, which indicate that the BGBM is more suited for the selected real Cuprite HSI than the LMM and GBM. Moreover, to demonstrate the necessity of taking the nonlinear mixing effect into consideration, the root-sum-square (RSS) [30,31] method is adopted to calculate the residual errors of HSI unmixing for each pixel. Figure 6 displays the RSS maps of FCLS for the selected real Cuprite HSI, and the difference RSS maps between FCLS and GDA, semi-NMF, BPOGM, and NU-BGBM respectively. From Figure 6, it can be observed that taking the nonlinear mixing effect into consideration can better approximate to the selected real Cuprite HSI, which demonstrate that the selected real Cuprite HSI can be better characterized by nonlinear mixtures of endmembers.

The second real HSI was acquired by the Meris spectrometer over the gulf of Lion, in the south east of France. This HSI has 13 spectral bands ranging from 400 nm to 800 nm, and the spatial resolution is 300 m. The region of interest, containing 280 × 330 pixels, is used to assess the unmixing performance of the proposed NU-BGBM and the other four compared methods. The false color image of the gulf of Lion HSI can be seen in Figure 3b. HySime [47] can be also adopted to estimate the number of endmembers for the gulf of Lion HSI. Besides, according to [24], the selected Cuprite HSI mainly has three materials, i.e., water, agricultural areas, and forests and semi natural areas. Moreover, the Corine Land Cover classification map of gulf of Lion HSI is shown in the first row of Figure 7, which can be used as potential visual ground truth to to interpret and evaluate the unmixing performance of different methods.

In this experiment, we also adopt the VCA [50] to extract endmembers from the gulf of Lion HSI first. However, when the number of extracted endmembers is three, one extracted endmember is not meaningful, which is due to that the corresponding abundance map is not spatially coherent. When the number of extracted endmembers is four, the three abundance maps of the meaningful endmembers are all spatially coherent, and the meaningless endmember is the outlier, which is shown in Figure 8. Figure 7 shows the abundance maps obtained by the proposed NU-BGBM and the compared methods for the gulf of Lion HSI. It can be seen from Figure 7 that the abundances of forests and semi natural areas obtained by different methods are fairly consistent with each other, and the abundances of water and agricultural areas of the proposed NU-BGBM approximate to the estimated visual ground truth better than these of the other four compared methods, which demonstrate the efficiency of the proposed method.

Table 3. Comparison of REs ($\times {10}^{-2}$) and SMADs ($\times {10}^{-2}$) of different methods using extracted endmembers for the gulf of lion real HSI.

	FCLS	GDA	Semi-NMF	BPOGM	NU-BGBM
RE	1.138	1.044	0.899	0.898	0.353
SMAD	3.932	3.660	3.585	3.581	2.643

gives the REs and SMADs of the proposed NU-BGBM and the other four compared methods for gulf of Lion HSI. It can be observed from Table 3 that the proposed NU-BGBM can obtain better RE and SMAD than the other four compared methods, which demonstrate that the efficiency and advantages of the proposed NU-BGBM by taking the mixed noise into account. Besides, the unmixing method based on the BGBM all get smaller RE and SMAD than these unmixing methods based on the LMM and GBM, which demonstrate that the BGBM can better characterize the gulf of Lion HSI than the LMM and BGBM. Moreover, Figure 9 displays the RSS maps of FCLS for the gulf of Lion HSI, and the difference RSS maps between FCLS and GDA, semi-NMF, BPOGM, and NU-BGBM respectively. From Figure 9, it can be also clearly seen that unmixing the real HSI based on the GBM and BGBM can better approximate to the gulf of Lion HSI, which indicates that the gulf of Lion HSI is composed of nonlinear mixtures of endmembers.

The third real HSI is the Urban data (available at: http://www.escience.cn/people/feiyunZHU/Dataset_GT.html) acquired by the Hyperspectral Digital Imagery Collection Experiment (HYDICE) over the Copperas Cove near Fort Hood. This HSI has 210 spectral bands ranging from 400 nm to 2500 nm, and the spectral resolution is 10 nm. There are 307 × 307 pixels, and the spatial resolution is 2 m. We have removed bands 1–4, 76, 87, 101–111, 136–153 and 198–210 due to water vapor and atmospheric effects, remaining D = 162 spectral bands. The false color image of the Urban data can be seen in Figure 3c. This HSI mainly has six materials, i.e., asphalt, grass, tree, roof, metal and dirt, which can be observed in Figure 10.

Figure 11 shows the abundance maps obtained by the proposed NU-BGBM and the compared methods for the Urban HSI. It can be observed from Figure 11 that the abundances of metal and dirt obtained by different methods are fairly consistent with each other, the abundances of tree and roof obtained by semi-NMF, BPOGM, and NU-BGBM are different from these of FCLS and GDA, the abundances of asphalt and grass obtained by NU-BGBM are different from these of FCLS, GDA, semi-NMF and BPOGM. Besides, Figure 12 displays the RSS maps of FCLS for the Urban HSI, and the difference RSS maps between FCLS and GDA, semi-NMF, BPOGM, and NU-BGBM respectively. It can be observed from Figure 12 that it helps to approximate to the Urban HSI by taking the nonlinear mixing effect into consideration. Moreover,

Table 4. Comparison of REs ($\times {10}^{-2}$) and SMADs ($\times {10}^{-2}$) of different methods using extracted endmembers for the Urban real HSI.

	FCLS	GDA	Semi-NMF	BPOGM	NU-BGBM
RE	4.120	4.057	1.837	1.721	1.443
SMAD	12.713	12.646	9.541	9.000	7.353

shows the REs and SMADs of the proposed NU-BGBM and the other four compared methods for the Urban HSI. It can be observed from Table 4 that the proposed NU-BGBM can obtain the best RE and SMAD, which demonstrate that the advantages of taking the mixed noise and different intensities of AWGN in different bands into account.

4. Conclusions

In this paper, we propose a novel nonlinear unmixing method based on the bandwise generalized bilinear model under the maximum a posteriori framework, which can take the complex mixed noise and different intensities of AWGN in different bands into consideration. Besides, we develop the ADMM to solve the proposed NU-BGBM. In the synthetic data experiments, the proposed NU-BGBM can handle the mixed noise and different intensities of AWGN much better than the other compared methods, thus our method is more robust and applicable than the other compared methods in the presence of complex mixed noise. In the real data experiments, the proposed unmixing method based on the BGBM all can get smaller RE and SMAD than these unmixing methods based on the LMM and GBM, which demonstrate that the BGBM can better characterize the selected real Cuprite HSI, gulf of Lion HSI and Urban HSI than the LMM and GBM. The proposed NU-BGBM can obtain better unmixing performance than the other four compared methods on both synthetic datasets and real HSIs, which demonstrate the effectiveness and advantages of the proposed model BGBM and the corresponding unmixing method NU-BGBM.

Although the proposed NU-BGBM can obtain desirable performance for hyperspectral unmixing, there is still room for improvement. Since the spectral signatures of neighboring pixels usually have high correlation, so we can take it into consideration to further improve the unmixing performance in out future work.