Hyperspectral Image Restoration under Complex Multi-Band Noises

Abstract

Hyperspectral images (HSIs) are always corrupted by complicated forms of noise during the acquisition process, such as Gaussian noise, impulse noise, stripes, deadlines and so on. Specifically, different bands of the practical HSIs generally contain different noises of evidently distinct type and extent. While current HSI restoration methods give less consideration to such band-noise-distinctness issues, this study elaborately constructs a new HSI restoration technique, aimed at more faithfully and comprehensively taking such noise characteristics into account. Particularly, through a two-level hierarchical Dirichlet process (HDP) to model the HSI noise structure, the noise of each band is depicted by a Dirichlet process Gaussian mixture model (DP-GMM), in which its complexity can be flexibly adapted in an automatic manner. Besides, the DP-GMM of each band comes from a higher level DP-GMM that relates the noise of different bands. The variational Bayes algorithm is also designed to solve this model, and closed-form updating equations for all involved parameters are deduced. The experiment indicates that, in terms of the mean peak signal-to-noise ratio (MPSNR), the proposed method is on average 1 dB higher compared with the existing state-of-the-art methods, as well as performing better in terms of the mean structural similarity index (MSSIM) and Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS).

1. Introduction

Hyperspectral images (HSIs) are collected by high spectral resolution sensors, and consist of hundreds of bands ranging from ultraviolet to infrared wavelengths. Due to the richness of spatial and spectral information, this type of image has been widely used in many remote sensing applications [1,2]. However, HSIs inevitably suffer from various noise contamination in practice [3], which severely influences the image quality and limits the performance of the subsequent HSI processing tasks, such as unmixing [4], classification [5] and so on. Therefore, as a necessary pre-processing step of certain application tasks, HSI restoration is an important research topic and has attracted much attention in recent decades.

Regarding each band of HSIs as a gray image, we can simply use the traditional image denoising method to reduce noise in a band-by-band fashion, such as the sparse constraint methods [6,7,8,9], local polynomial approximation (LPA) methods [10,11,12] and so on. However, such methods ignore the intrinsic characteristics of HSIs, i.e., the spatial and spectral correlation among different bands, thus often resulting in unsatisfactory performance. To alleviate this issue, many methods have been proposed in order to better encode the spatial and spectral correlation knowledge of HSIs, e.g., [13,14,15,16]. Besides, nonlocal similarity [17], anisotropic diffusion [18] and conditional random fields [19], have also been utilized for the HSI restoration task recently.

Inspired by the spectral correlation of HSIs, various methods based on low-rank matrix factorization (LRMF) have been proposed in the past few years. For example, Zhang et al. [20] used a rank-constrained RPCA [21] while Wu et al. [22] and Xie et al. [23] employed weighted nuclear norm minimization (WNNM) [24,25] to enhance the restoration quality. In addition, considering the local similarity within a patch and the non-local similarity across the patches in a group simultaneously, a novel group low-rank representation (GLRR) model was proposed in [26]. Besides this kind of LRMF approach, a series of methods regarding HSIs as a 3-D tensor have also been proposed in recent years. Along this line, we can categorize these methods as tensor decomposition-based methods [27,28,29,30,31] and wavlet-based methods [32,33,34,35].

The current HSI restoration methods generally assume similar complexity of noise structures across all bands of an HSI dataset. In fact, most of them only assume simple independent and identical Gaussian/Laplacian noise on all HSIs, which implies that the noises on all bands share a similar noise type and extent. To make the approach better fit real noises with more complexity, some HSI restoration methods have been constructed to consider more complex noise beyond simple Gaussian/Laplacian [36,37]. Additionally, He et al. [38] considered the noise as a mixture of Gaussian and Laplacian distribution while Cao et al. [39] modelled the noise using a more general mixture of exponential power (MoEP) distributions. However, these methods all assume the noise in each HSI band as an identical distribution. Recently, Chen et al. [40] proposed modelling the noise distribution of each band as a mixture of Gaussian (MoG) distributions with different parameters sharing with the unique conjugate prior. This method, however, still assumes the same complexity (i.e., the number of components) of MoGs in all HSI bands.

For most practical HSIs, however, the noise across different bands is always evidently distinct, both in terms of distribution type and corruption extent. As shown in Figure 1, the noises in the shown three bands of the Urban (http://lesun.weebly.com/hyperspectral-data-set.html) HSI dataset (Figure 1a) and the Terrain (https://www.agc.army.mil) HSI dataset (Figure 1b) exhibit very different patterns. Specifically, band 76 of Urban and band 2 of Terrain are mainly contaminated by deadlines and the noises resemble Laplacian distributions; band 104 of Urban and band 139 of Terrain are more corrupted by Gaussian noise, while band 139 of Urban and band 151 of Terrain are simultaneously contaminated by several kinds of noise, so their noise distributions are multimodal. The current HSI restoration methods thus lack consideration on such band-noise-distinctness issue, which tends to weaken their robustness in handling real noisy HSI.

To address the aforementioned noise fitting issue, in this paper we propose a new HSI restoration method, which can more sufficiently encode such noise characteristics underlying HSIs. The main contribution of this work can be summarized as follows:

• The noise in each HSI band is modelled by a Dirichlet process Gaussian mixture model (DP-GMM), in which the Gaussian components of MoG in each band are adaptively determined based on the specific noise complexity of this band. The distinctness of noise structures in different bands is thus able to be faithfully reflected by the model.
• By using the hierarchical Dirichlet process (HDP) technique, we correlate the noise of different bands through a sharing strategy, in which the noise parameters of each band share a high-level noise configuration of the entire HSI.
• A variational Bayes algorithm is readily designed to solve the model, and each of the involved parameters can be effectively updated in closed-form.

This paper is organized as follows. Section 2 provides some preliminaries on the utilized HDP technique. Section 3 presents the main model against the investigated problem, and the variational Bayes algorithm for solving it is given in Section 4. Experimental results on synthetic and real datasets are demonstrated in Section 5, and finally the paper is rounded up with a conclusion.

2. Preliminaries

In this section, we review some preliminary knowledge about the Dirichlet process and hierarchical Dirichlet process to set the stage for the presentation of our model later.

2.1. Dirichlet Process

The Dirichlet process (DP), developed firstly by Ferguson [41], is parameterized by a concentration parameter $\gamma$ and a base distribution H. Consider a measurable set $\Theta$ and any of its finite partitions $\{A_1,A_2,\dots ,A_r\}$; the Dirichlet process, denoted by $DP(\gamma ,H)$, is a unique distribution over probability distributions on $\Theta$ such that

$$\mathrm{if}G\sim DP(\gamma ,H),\mathrm{then}\left(G\left(A_1\right),G\left(A_2\right),\dots ,G\left(A_r\right)\right)\sim Dir(\gamma H\left(A_1\right),\gamma H\left(A_2\right),\dots ,\gamma H\left(A_r\right))$$

(1)

where $Dir(·)$ denotes the Dirichlet distribution.

Besides, Ferguson [41] proved two important properties of the DP. The first is that any random sample G drawn from $DP(\gamma ,H)$ is a discrete distribution over $\Theta$. The second property considers repeated draws ${\theta }_1,{\theta }_2,\dots ,{\theta }_n$ from this discrete distribution G, i.e.,

$${\theta }_i\sim G,i\in \{1,2,\dots ,n\},$$

where these ${\theta }_i$s will exhibit a clustering property, i.e., they share repeated values with positive probability. In the DP-GMM, ${\theta }_i$ represents the parameters (i.e., mean and variance) of the corresponding Gaussian component to which the i-th data point belongs. Thus, the clustering property of ${\theta }_i$s can adaptively determine the number of Gaussian components in MoG.

Sethuranman [42] provides an equivalent but more explicit representation of DP based on the stick-breaking construction. Considering two infinite collections of independent random variables, ${\pi }_i\sim Beta(1,\gamma )$ and ${\eta }_i^{*}\sim H$, for $i=1,2,\dots$, the stick-breaking representation of G is as follows:

$${\pi }_i={\pi }_i^{{}^{\prime }}\prod _{j=1}^{i-1}(1-{\pi }_j^{{}^{\prime }}),G=\sum _{i=1}^{\infty }{\pi }_i{\delta }_{{\eta }_i^{*}}$$

where ${\delta }_{{\eta }_i^{*}}$ represents the Dirac Delta function located at ${\eta }_i^{*}$. The stick-breaking construction depicts the DP from a different point of view, in which the clustering property of DP is realized by the mixing coefficient ${\pi }_k$. The parameter $\gamma$ controls the dispersion level of ${\pi }_k$, and we simulate the stick-breaking process under different $\gamma$ in Figure 2. It is easy to see that this stick-breaking process is able to determine the number of Gaussian components of MoG through adjusting the value of ${\pi }_k$.

2.2. Hierarchical Dirichlet Process

The hierarchical Dirichlet process (HDP) [43] is originally proposed as a hierarchical Bayesian model for natural language processing, which provides a flexible framework for sharing mixture components among groups of related data. A two-level HDP is a collection of DPs, which is also drawn from a higher-level DP with base distribution H, i.e.,

$$G\sim DP(\gamma ,H),G_j\sim DP({\alpha }_j,G),$$

(2)

where j is an index for the data group. It should be noted that all distributions $G_j$ possess different parameters while sharing the distribution G.

3. HSIs Restoration Model Based on DP-GMM

3.1. Notation Explanation

For convenience of formulating our model, we first introduce some necessary notations to be used in the following sections. We use light lowercase letters, bold lowercase letters and bold uppercase letters to denote scalars, vectors and matrices, respectively. Given a matrix $\mathit{X}$, we use ${\mathit{x}}_{·j}$ to denote its j-th column vector, ${\mathit{x}}_{i·}$ to denote its i-th row vector, and $x_{ij}$ to denote the element in its i-th row and j-th column. For probability distributions, $\mathcal{N}(\mathit{\mu },\mathbf{\Sigma })$ represents the multivariate Gaussian distribution with mean vector $\mathit{\mu }$ and covariance matrix $\mathbf{\Sigma }$, $\mathcal{N}(\mu ,{\xi }^{-1})$ represents the univariate Gaussian distribution with mean $\mu$ and precision $\xi$, $Beta(a,b)$ represents the Beta distribution with parameters a and b, $Gam(c,d)$ represents the Gamma distribution with parameters c and d, and $Multi\left(\mathit{\pi }\right)$ represents the multinomial distribution with parameter $\mathit{\pi }$.

3.2. Model Formulation

Now, we give the formulations of our proposed model. Let us present the observed HSI data contaminated by noises as $\mathit{Y}\in {\mathbb{R}}^{d\times B}$, where $d=hw$, h and w are the height and width of the investigated HSI, respectively, and B is the number of bands. By considering a generative model, we can decompose the observed data into

$$\mathit{Y}=\mathit{X}+\mathit{E},$$

(3)

where $\mathit{E}$ denotes the noise term and $\mathit{X}$ is the expected restorated HSI.

3.2.1. The DP-GMM Model

In this study, we model the noise of each band as a DP-GMM, which is an advanced version of the non-parametric GMM model capable of adaptively rectifying the number of Gaussian components of MoG based on the data [44,45]. The noise complexity of each band can be adjusted by varying the number of components of MoG. What is more, in order to simultaneously encode the correlation among different bands, we associate the corresponding DP-GMM of each band at a higher level and conduct a two-level HDP.

Specifically, we consider the noise located at the i-th row and j-th column of the noise term $\mathit{E}$. Following the idea of [40,46], we can model it as an element-wise Gaussian mixture distribution:

$$e_{ij}\sim \sum _{k=1}^K{\pi }_{jk}\mathcal{N}(0,{\xi }_k^{-1}),$$

(4)

which can be equivalently reformulated as a two-level generative process:

$$e_{ij}\sim \mathcal{N}(0,{\xi }_{z_{ij}}^{-1}),z_{ij}\sim Multi\left({\mathit{\pi }}_j\right),$$

(5)

where ${\mathit{\pi }}_j$ is a vector of length K, $1\le z_{ij}\le K$. Then, we place a hierarchical Dirichlet distribution prior on ${\mathit{\pi }}_j$ to associate the noise of different bands, i.e.,

$${\mathit{\pi }}_j\sim Dir\left({\alpha }_j\mathit{\beta }\right),\mathit{\beta }\sim Dir(\gamma /K,\gamma /K,\dots ,\gamma /K).$$

(6)

Besides, ${\xi }_k$ is also generated from its conjugate prior, i.e.,

$${\xi }_k\sim Gam(a_0,b_0).$$

(7)

To allow the model to be flexible, we further assume $K\to +\infty$. Teh et al. [47] proved that the limit of Equations (5)–(7) is the following HDP noise model:

$$e_{ij}\sim \mathcal{N}(0,{\psi }_{ij}^{-1}),{\psi }_{ij}\sim G_j,G_j\sim DP({\alpha }_j,G),G\sim DP(\gamma ,H),$$

(8)

where $H=Gam(a_0,b_0)$.

The noise model of Equation (8) is too metaphysical to understand, so we reformulate it equivalently but more explicitly with two coupled latent variables $z_{ij}$ and $c_{jt}$ in a stick-breaking constructive manner, i.e.,

$$\begin{array}{cccccccc}\hfill c_{jt}& \sim Multi\left(\mathit{\beta }\right),\hfill & \hfill {\beta }_k& ={\beta }_k^{{}^{\prime }}\prod _{l=1}^{k-1}(1-{\beta }_l^{{}^{\prime }}),\hfill & \hfill {\beta }_k^{{}^{\prime }}& \sim Beta(1,\gamma ),\hfill & \hfill {\xi }_k& \sim Gam(a_0,b_0),\hfill \\ \hfill z_{ij}& \sim Multi\left({\mathit{\pi }}_{\mathit{j}}\right),\hfill & \hfill {\pi }_{jt}& ={\pi }_{jt}^{{}^{\prime }}\prod _{s=1}^{t-1}(1-{\pi }_{js}^{{}^{\prime }}),\hfill & \hfill {\pi }_{jt}^{{}^{\prime }}& \sim Beta(1,{\alpha }_j),\hfill & \hfill e_{ij}& \sim \mathcal{N}(0,{\xi }_{c_{j,z_{ij}}}^{-1}).\hfill \end{array}$$

(9)

It is worth noting that the $\mathit{\xi }$ in Equation (9) and the $\mathit{\psi }$ in Equation (8) have the following relation:

$${\xi }_{c_{j,z_{ij}}}={\psi }_{ij}.$$

We can explain the hierarchical structure of the noise model (8) using Figure 3a intuitively. In the first level, we attempt to fit the noise of the entire HSI data by a DP-GMM. In the second level, we also fit the noise of each band by a DP-GMM, but each of its components is shared from the first level. This sharing strategy associates the noise of different bands, and the DP-GMM, which allows an MoG to have a limited but unbounded number of Gaussian components, adjusts the noise complexity adaptivity for each band.

Note that we adopt the truncation strategy [48] for the DP-GMMs in the variational inference of Section 4, which means that the number of Gaussian components of DP-GMMs in the first and second levels are fixed as K and T (K and T is large enough, $T\le K$) as shown in Figure 3a. Based on such a truncation strategy, $z_{ij}$ ($1\le z_{ij}\le T$) denotes the index of the Gaussian component from which $e_{ij}$ is generated in the DP-GMM of band j ($1\le j\le B$), and $c_{jt}$ ($1\le c_{jt}\le K$, $1\le t\le T$) denotes the index of the Gaussian component of DP-GMM in the first level, with which the t-th Gaussian component of DP-GMM of band j in the second level is shared ($c_{jt}$ is marked in Figure 3a).

Finally, the concentration parameters $\mathit{\alpha }$ and $\gamma$ in Equation (8) mainly affect the number of Gaussian components of the second level MoG in each band and the first level MoG for the entire HSIs dataset, respectively. In order to enhance the adaptability of the model complexity, we can place the Gamma distribution on them [49], i.e.,

$$\gamma \sim Gam(c_0,d_0),{\alpha }_j\sim Gam(e_0,f_0).$$

(10)

In the above formulations, $a_0,b_0,c_0,d_0,e_0,f_0$ are hyperparameters of prior distributions, all of which can be easily set as small values to make the prior non-informative.

3.2.2. LRMF Model

As the conventional HSI restoration model, we assume that the expected HSI $\mathit{X}$ in Equation (3) has a low-rank property. Based on such an assumption, we formulate $\mathit{X}\in {\mathbb{R}}^{d\times B}$ as the product of two smaller matrices $\mathit{U}\in R^{d\times l}$ and $\mathit{V}\in {\mathbb{R}}^{B\times l}$, i.e.,

$$\mathit{X}=\mathit{U}{\mathit{V}}^T=\sum _{r=1}^l{\mathit{u}}_{·r}{\mathit{v}}_{·r}^T,$$

(11)

where $l\le B$ is fixed pre-manually. Furthermore, $\mathit{U}$ and $\mathit{V}$ can be generated from a multivariate zero-mean Gaussian distribution by the following method:

$${\mathbf{u}}_{·r}\sim \mathcal{N}(\mathbf{0},{\lambda }_r^{-1}{\mathit{I}}_{\mathit{d}}),{\mathit{v}}_{·r}\sim \mathcal{N}(\mathbf{0},{\lambda }_r^{-1}{\mathit{I}}_{\mathit{B}}),$$

(12)

where ${\mathit{I}}_d$ denotes the $d\times d$ identity matrix. Furthermore, the corresponding conjugate prior on each precision variable ${\lambda }_r$ is:

$${\lambda }_r\sim Gam(p_0,q_0).$$

(13)

3.2.3. The Entire Graphical Model

Combining Equations (3), (9)–(13), we can construct a full Bayesian model for HSI restoration. In brief, we call our model DP-GMM. The corresponding graphical model of our method is shown in Figure 3b. The goal turns to infer the posteriors of all involved variables:

$$p(\mathit{U},\mathit{V},\mathit{\xi },\mathit{C},\mathit{Z},\mathit{\beta },\mathit{\pi },\mathit{\lambda },\mathit{\alpha },\gamma |\mathit{Y}).$$

(14)

4. Variational Inference

Directly computing the posterior of Equation (14) under an HDP mixture prior is intractable, and thus approximate inference methods need to be designed. Although the Markov chain Monte Carlo (MCMC) sampling method can provide very accurate approximation to the posterior [50], this method is limited for massive scale data due to its high computational cost and difficulty to detect convergence. As a more efficient and deterministic alternative to MCMC, we adopt a Variational Bayes (VB) method [51] to solve our model in this paper. Given the observed data $\mathit{Y}$, maximizing the marginal log-likelihood $p\left(\mathit{Y}\right)=\int p\left(\mathit{Y}\right|\mathit{\theta }\left)p\right(\mathit{\theta })d\mathit{\theta }$ directly is computationally intractable. VB attempts to seek a lower bound of the log-likelihood and maximize it instead, i.e.,

$$\begin{array}{cc}\hfill lnp\left(\mathit{Y}\right)& =\int q\left(\mathit{\theta }\right)ln\frac{p\left(\mathit{Y}\right|\mathit{\theta }\left)p\right(\mathit{\theta })}{q\left(\mathit{\theta }\right)}d\mathit{\theta }+KL\left(q\left(\mathit{\theta }\right)\right|\left|p\left(\mathit{\theta }\right|\mathit{Y})\right)\hfill \\ & \ge \int q\left(\mathit{\theta }\right)ln\frac{p\left(\mathit{Y}\right|\mathit{\theta }\left)p\right(\mathit{\theta })}{q\left(\mathit{\theta }\right)}d\mathit{\theta }\hfill \\ & \triangleq \mathcal{L}\left(q\right)\hfill \end{array}$$

(15)

where $KL(·||·)$ denotes the $Kullback-Leibler$ (KL) divergence of two distributions. Assuming $q\left(\mathit{\theta }\right)={\prod }_i{q}_i\left({\theta }_i\right)$, we can obtain the closed-form solution to $q_j\left({\theta }_j\right)$ with other factors fixed through optimizing $\mathcal{L}\left(q\right)$ with respect to $q_j\left({\theta }_j\right)$, i.e.,

$$q_j^{*}\left({\theta }_j\right)=\frac{exp\left\{E_{\mathit{\theta }\backslash {\theta }_j}[lnp(\mathit{\theta },\mathit{Y})]\right\}}{\int exp\left\{E_{\mathit{\theta }\backslash {\theta }_j}[lnp(\mathit{\theta },\mathit{Y})]\right\}d{\theta }_j},$$

(16)

where $E_x[·]$ denotes the expectation over the variable x, and $\mathit{\theta }\backslash {\theta }_j$ represents the set of $\mathit{\theta }$ with ${\theta }_j$ removed. The entire inference can be accomplished by alternatively computing Equation (16) with respect to all the involved variables.

Based on the stick-breaking construction and the truncation strategy [48], we can approximate the posterior distribution of Equation (14) with the following factorized form:

$$q(\mathit{U},\mathit{V},\mathit{\xi },\mathit{C},\mathit{Z},\mathit{\beta },\mathit{\pi },\mathit{\lambda },\mathit{\alpha },\gamma )={\prod }_{i=1}^{d}q\left(u_{i·}\right){\prod }_{j=1}^{B}q\left(v_{j·}\right){\prod }_{r=1}^{l}q\left({\lambda }_r\right){\prod }_{k=1}^{K}q\left({\xi }_k\right)q\left({\beta }_k^{{}^{\prime }}\right){\prod }_{j,t}^{B,T}q\left(c_{jt}\right)q\left({\pi }_{jt}^{{}^{\prime }}\right){\prod }_{i,j}^{d,B}q\left(z_{ij}\right){\prod }_{j=1}^{B}q\left({\alpha }_j\right)q\left(\gamma \right)$$

(17)

Then we can analytically infer all the factorized distributions involved in Equation (17) as below. The computational details are provided in the Supplementary Material.

Update Noise Component: The parameters involved in the noise components are $\mathit{\xi }$, $\mathit{C}$, $\mathit{Z}$, $\mathit{\pi }$, $\mathit{\beta }$. We use the stick-breaking procedure by setting a large enough value of K and T for truncated approximation, and then get the following updating equations on $\mathit{Z}$ and $\mathit{C}$:

$$q\left(c_{jt}\right)=Multi\left({\mathit{\phi }}_{\mathit{jt}}\right),q\left(z_{ij}\right)=Multi\left({\mathit{\rho }}_{ij}\right),$$

(18)

where

$$\begin{array}{c}{\phi }_{jtk}\propto exp\left\{\sum _{i=1}^d{\rho }_{ijt}\left(\frac{1}{2}E[ln{\xi }_k]-\frac{1}{2}ln2\pi -\frac{1}{2}E\left[{\xi }_k\right]E\left[{(y_{ij}-{\mathit{u}}_{i·}{\mathit{v}}_{j·}^T)}^2\right]\right)+E[ln{\beta }_k]\right\},\\ {\rho }_{ijt}\propto exp\left\{\sum _{k=1}^K{\phi }_{itk}\left(\frac{1}{2}E[ln{\xi }_k]-\frac{1}{2}ln2\pi -\frac{1}{2}E\left[{\xi }_k\right]E\left[{(y_{ij}-{\mathit{u}}_{i·}{\mathit{v}}_{j·}^T)}^2\right]\right)+E[ln{\pi }_{jt}]\right\},\\ E[ln{\beta }_k]=E[ln{\beta }_k^{{}^{\prime }}]+\sum _{l=1}^{k-1}E[ln(1-{\beta }_l^{{}^{\prime }})],E[ln{\pi }_{jt}]=E[ln{\pi }_{jt}^{{}^{\prime }}]+\sum _{s=1}^{t-1}E[ln(1-{\pi }_{js}^{{}^{\prime }})].\end{array}$$

As for $\mathit{\xi }$, based on its conjugate priors, we have

$$q\left({\xi }_k\right)=Gam(a_k,b_k),$$

(19)

where

$$a_k=\frac{1}{2}\sum _{i,j,t}{\rho }_{ijt}{\phi }_{jtk}+a_0,b_k=\frac{1}{2}\sum _{i,j,t}{\rho }_{ijt}{\phi }_{jtk}E\left[{(y_{ij}-{\mathit{u}}_{i·}{\mathit{v}}_{j·}^T)}^2\right]+b_0.$$

Similarly, for $\mathit{\pi }$, $\mathit{\beta }$, we have

$$q\left({\pi }_{jt}^{{}^{\prime }}\right)=Beta(r_{jt}^1,r_{jt}^2),q\left({\beta }_k^{{}^{\prime }}\right)=Beta(s_k^1,s_k^2),$$

(20)

where

$$r_{jt}^1=\sum _i{\rho }_{ijt}+1,r_{jt}^2=\sum _{i,s=t+1}^{d,T}{\rho }_{ijs}+{\alpha }_j,s_k^1=\sum _{j,t}{\phi }_{jtk}+1,s_k^2=\sum _{j,t,s=k+1}^{d,T,K}{\phi }_{jts}+\gamma .$$

Update the Concentration Parameters: According to the conjugate priors of $\mathit{\alpha }$ and $\gamma$ in Equation (10), we can obtain the following updating equations:

$$q\left({\alpha }_j\right)=Gam(e_j,f_j),q\left(\gamma \right)=Gam(c,d),$$

(21)

where

$$e_j=T+e_0,f_j=f_0-\sum _{t}E[ln(1-{\pi }_{jt}^{{}^{\prime }})],c=K+c_0,d=d_0-\sum _{k}E[ln(1-{\beta }_k^{{}^{\prime }})].$$

Update the LRMF Components: The parameters involved in the LRMF components are $\mathit{U}$, $\mathit{V}$, $\mathit{\lambda }$. For each row of $\mathit{U}$, using the factorization in Equation (11), we can obtain

$$q\left({\mathit{u}}_{i·}\right)=\mathcal{N}({\mathit{\mu }}_{i·}^u,{\mathbf{\Sigma }}_i^u),$$

(22)

with mean ${\mathit{\mu }}_{i·}^u$ and covariance matrix ${\mathbf{\Sigma }}_i^u$, given by

$${\mathbf{\Sigma }}_i^u={\left(\sum _{j,t,k}{\rho }_{ijt}{\phi }_{jtk}E\left[{\xi }_k\right]E\left[{\mathit{v}}_{j·}^T{\mathit{v}}_{j·}\right]+\mathbf{\Lambda }\right)}^{-1},{\mathbf{\mu }}_{i·}^u=\sum _{j,t,k}{\rho }_{ijt}{\phi }_{jtk}E\left[{\xi }_k\right]y_{ij}{\mathit{v}}_{j·}{\mathbf{\Sigma }}_i^u,$$

where $\mathbf{\Lambda }=diag(E\left[{\lambda }_1\right],E\left[{\lambda }_2\right],\dots ,E\left[{\lambda }_l\right])$. Similarly, for each column of $\mathit{V}$, we have

$$q\left({\mathit{v}}_{j·}\right)=\mathcal{N}({\mathit{\mu }}_{j·}^v,{\mathbf{\Sigma }}_j^v),$$

(23)

and the mean ${\mathit{\mu }}_{j·}^v$ and the covariance matrix ${\mathbf{\Sigma }}_j^v$ are calculated by

$${\mathbf{\Sigma }}_{\mathbf{j}}^{\mathbf{v}}={\left(\sum _{i,t,k}{\rho }_{ijt}{\phi }_{jtk}E\left[{\xi }_k\right]E\left[{\mathit{u}}_{i·}^T{\mathit{u}}_{i·}\right]+\mathbf{\Lambda }\right)}^{-1},{\mathit{\mu }}_{j·}^v=\sum _{i,t,k}{\rho }_{ijt}{\phi }_{jtk}E\left[{\xi }_k\right]y_{ij}{\mathit{u}}_{i·}{\mathbf{\Sigma }}_j^v.$$

For parameter $\mathit{\lambda }$, we have

$$q\left({\lambda }_r\right)=Gam(p_r,q_r),$$

(24)

where

$$p_r=\frac{d}{2}+\frac{B}{2}+p_0,q_r=\frac{1}{2}E\left[{\mathit{u}}_{·r}^T{\mathit{u}}_{·r}\right]+\frac{1}{2}E\left[{\mathit{v}}_{·r}^T{\mathit{v}}_{·r}\right]+q_0.$$

Settings of Hyperparameters: We set all the hyperparameters involved in our model in a non-informative manner to minimize their impacts on the inference of the posterior distribution [51]. Specifically, throughout our experiments, we easily set $a_0$, $b_0$, $c_0$, $d_0$, $e_0$, $f_0$, $p_0$, $q_0$ as ${10}^{-6}$. Our proposed method behaves stably throughout all the experiments with this simple initialization choice.

5. Experimental Results

To demonstrate the effectiveness of the proposed DP-GMM model, we perform both synthetic and real HSI data experiments, and evaluate the experimental results quantitatively and visually. In order to comprehensively evaluate the performance of the proposed method, we compared the results with eight popular HSI restoration methods: the traditional image denoising method SVD [52], LRMF-based methods LRMR [20], LRTV [38], WNNM [24] and WSNM [23], tensor-based methods TDL [30] and BM4D [35], and mixture noise based-method NMoG [33]. These methods represent the state-of-the-art for the HSI restoration task. The parameters for these compared methods were manually adjusted according to their default strategies. In addition, to facilitate the numerical calculation and visualization, all the bands of the HSI datasets are normalized into [0,1].

5.1. Simulated Data Experiments

Two HSI datasets were employed in this experiment: Washington DC Mall (http://lesun.weebly.com/hyperspectral-data-set.html) with size $1208\times 307\times 191$ and RemoteImage (http://peterwonka.net/Publications/code/LRTC_Package_Ji.zip) with size $205\times 246\times 96$. Some bands of these two HSI datasets are heavily contaminated by noise and cannot be regarded as clean ground truth, and thus have to be deleted [33,36]. Due to the computer’s memory limitation, we also cropped the subimage of the HSI and finally obtained two synthetic datasets with size $200\times 200\times 160$ and $200\times 200\times 89$, respectively. These two HSI datasets are considered as clean data in this simulated experiment.

To simulate noisy HSIs, we added six types of noises to the clean HSI datasets to test the performance of all the compared methods. The details of the added noise are listed as follows:

• Case 1: For different bands, the noise intensity was equal in this case. Furthermore, the same zero-mean Gaussian noise $\mathcal{N}(0,{\sigma }^2)$ with $\sigma =0.05$ was added to all the bands
• Case 2: Entries in all bands were corrupted by zero-mean Gaussian noise but with different intensity. Furthermore, the standard deviation of Gaussian noise that was added to each band of HSIs was uniformly selected from 0.01 to 0.1.
• Case 3: All the bands were corrupted by Gaussian noise as Case 2. Besides, 40 bands of the DC Mall dataset (20 bands of the RemoteImage dataset) were randomly chosen to add deadlines, and the number of deadlines in each band is from 5 to 15 with width 1 or 2.
• Case 4: All the bands were contaminated by Gaussian noise as Case 2. Besides, 40 bands of the DC Mall dataset (20 bands of the RemoteImage) were randomly selected to add stripes, and the number of stripes is from 15 to 40 with width 1 or 2.
• Case 5: All the bands were corrupted by Gaussian noise as Case 2. In addition, different percentages of impulse noises which were uniformly selected from 0 to 0.15 were added to each band.
• Case 6: Each band was contaminated by Gaussian and impulse noise as Case 5. Besides, 20 bands of the DC Mall dataset (10 bands of the RemoteImage dataset) were randomly selected to add deadlines as Case 3, and another 20 bands of the DC Mall dataset (10 bands of the RemoteImage dataset) were randomly selected to add stripes as Case 4.

To fairly compare the overall performance of different methods, we set the rank of the expected HSIs as 7 in the DC Mall and RemoteImage dataset for all the compared methods. Furthermore, three measurements—the mean peak signal-to-noise ratio (MPSNR), mean structural similarity index (MSSIM) [53] and Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS) [54]—are utilized to evaluate the restoration quality:

$$\begin{array}{cc}\hfill \mathrm{MPSNR}& =\frac{1}{B}\sum _i^B{\mathrm{PSNR}}_i,\hfill \\ \hfill \mathrm{MSSIM}& =\frac{1}{B}\sum _i^B{\mathrm{SSIM}}_i,\hfill \\ \hfill \mathrm{ERGAS}& =\sqrt{\frac{1}{B}\sum _i^B\frac{\mathrm{MSE}({\mathrm{Ref}}_i,{\mathrm{Res}}_i)}{{\mathrm{Mean}}^2\left({\mathrm{Ref}}_i\right)}},\hfill \end{array}$$

where ${\mathrm{PSNR}}_i$ and ${\mathrm{SSIM}}_i$ denote the PSNR and SSIM values for the i-th band, and ${\mathrm{Ref}}_i$ and ${\mathrm{Res}}_i$ denote the i-th band of the reference image and the restorated image, respectively.

Table 1. Quantitative Evaluation of different methods on the simulated DC Mall dataset in different cases of noise.

Noise Case	Evaluation Index	Noise	SVD	BM4D	TDL	WNNM	WSNM	LRMR	LRTV	NMoG	DP-GMM
Case 1	MPSNR (dB)	26.02	38.96	36.66	40.23	36.98	36.98	38.79	38.45	39.05	39.60
	MSSIM	0.7627	0.9833	0.9728	0.9888	0.9806	0.9801	0.9848	0.9836	0.9865	0.9875
	ERGAS	187.97	41.37	53.24	35.34	52.74	52.74	41.98	43.27	42.19	38.22
Case 2	MPSNR (dB)	23.37	35.79	34.25	26.99	36.38	36.28	36.38	36.88	37.70	38.59
	MSSIM	0.6527	0.9600	0.9535	0.7904	0.9750	0.9732	0.9733	0.9739	0.9818	0.9842
	ERGAS	280.83	67.75	70.75	191.56	55.74	56.23	55.93	59.94	49.99	43.01
Case 3	MPSNR (dB)	22.51	33.90	31.67	25.12	35.99	35.34	34.45	35.96	36.99	37.99
	MSSIM	0.6317	0.9486	0.9275	0.7465	0.9730	0.9632	0.9619	0.9695	0.9767	0.9827
	ERGAS	308.89	102.41	128.78	240.34	59.22	81.04	84.31	76.22	65.47	47.72
Case 4	MPSNR (dB)	22.50	33.80	31.56	24.66	34.95	35.26	34.64	35.77	37.09	37.55
	MSSIM	0.6211	0.9280	0.9010	0.7035	0.9547	0.9690	0.9540	0.9642	0.9753	0.9790
	ERGAS	341.01	133.14	163.27	288.97	122.49	67.43	86.43	85.27	61.36	56.19
Case 5	MPSNR (dB)	16.47	27.17	25.12	21.80	34.33	34.45	35.07	36.28	36.15	37.61
	MSSIM	0.4122	0.8299	0.7339	0.5907	0.9263	0.9716	0.9644	0.9686	0.9688	0.9795
	ERGAS	601.66	206.72	225.50	319.36	183.85	72.97	64.44	78.02	126.97	48.80
Case 6	MPSNR (dB)	16.07	26.86	24.54	20.59	33.04	33.79	33.76	34.60	35.88	36.98
	MSSIM	0.3946	0.8202	0.7154	0.5412	0.8975	0.9616	0.9544	0.9599	0.9675	0.9782
	ERGAS	625.80	209.85	240.57	368.62	240.41	104.08	83.47	112.62	469.93	54.68

and

Table 2. Quantitative Evaluation of different methods on the simulated RemoteImage dateset in different cases of noise.

Noise Case	Evaluation Index	Noise	SVD	BM4D	TDL	WNNM	WSNM	LRMR	LRTV	NMoG	DP-GMM
Case 1	MPSNR (dB)	26.02	37.02	34.50	37.97	34.61	35.58	37.04	36.22	36.83	37.37
	MSSIM	0.6954	0.9655	0.9267	0.9705	0.9570	0.9575	0.9679	0.9588	0.9700	0.9700
	ERGAS	124.05	37.17	50.33	33.01	50.06	43.66	36.44	40.02	38.40	35.76
Case 2	MPSNR (dB)	23.25	33.85	32.12	27.65	34.18	33.02	34.67	34.72	35.56	35.79
	MSSIM	0.5554	0.9272	0.8785	0.7636	0.9399	0.8956	0.9464	0.9406	0.9610	0.9605
	ERGAS	186.22	54.26	64.95	112.77	50.80	82.17	47.62	47.44	44.30	42.69
Case 3	MPSNR (dB)	21.86	30.67	28.78	24.63	33.56	30.52	32.46	33.86	34.40	35.93
	MSSIM	0.5233	0.8764	0.8102	0.6693	0.9280	0.8246	0.9220	0.9314	0.9505	0.9600
	ERGAS	238.56	108.86	158.23	195.47	71.79	160.74	78.50	67.54	59.01	41.9869
Case 4	MPSNR (dB)	22.40	33.28	30.03	25.44	33.54	32.77	33.47	33.72	34.54	35.11
	MSSIM	0.5278	0.9056	0.8239	0.6750	0.9326	0.9152	0.9363	0.9310	0.9534	0.9575
	ERGAS	213.52	81.86	106.85	162.25	61.23	85.16	59.78	66.09	55.48	47.51
Case 5	MPSNR (dB)	17.15	27.98	27.65	23.99	29.82	30.75	33.77	33.52	33.61	35.63
	MSSIM	0.3155	0.7833	0.7090	0.5614	0.7858	0.8423	0.9351	0.9253	0.9490	0.9562
	ERGAS	394.87	124.32	108.07	169.46	229.53	182.01	52.78	75.27	60.58	43.26
Case 6	MPSNR (dB)	16.76	27.49	26.37	22.14	28.24	29.80	32.74	32.41	33.85	35.25
	MSSIM	0.3031	0.7781	0.6717	0.4797	0.7437	0.8153	0.9268	0.9151	0.9497	0.9550
	ERGAS	416.01	141.00	142.33	223.21	268.10	215.28	64.88	104.31	62.96	46.65

present the restoration results by all the compared methods in terms of the aforementioned three measurements. The best results for each quality measurement are highlighted in bold. It is clear that our proposed method provides the highest values in both MPSNR and MSSIM, and lowest ERGAS in most of the cases, which validates the superiority of the proposed method over the other methods. For Case 1 with the simple independent and identically distributed (i.i.d.) Gaussian noise, most of the methods have comparable performance, which confirms the effectiveness of these methods to some extent, especially for TDL. When the noise becomes more complex, e.g., the intensity of noise varies by bands, more stripes, more deadlines and more impulse noises are added, the advantages of our proposed method become more evident from Case 2 to Case 6. This advantage substantiates the robustness of our proposed method against complex noise, owing to its adaptability to distinguish noise complexities in different HSI bands in our model.

Figure 4 and Figure 5 present some typical bands of the DC Mall and RemoteImage HSI dataset before and after restoration under Case 3, Case 4 and Case 6, respectively. By comparing the restoration results, it can be clearly seen that DP-GMM performs best, effectively removing all types of noise while preserving more detailed information. In Case 3, the restored HSIs of almost all of the compared methods contain some evident deadlines. Similarly, some stripes still exist in the restoration results of most of the compared methods in Case 4. Under the mixture noise of Case 6, LRTV, NMoG and DP-GMM all have relatively better results than the other methods. In conclusion, our proposed method can have stable and better performance in all cases, which verifies the effectiveness of the proposed method with better capability in fitting a much wider range of noises than current methods.

In order to further compare the performance of all the restoration methods, we also show the spectral signatures of some pixels before and after restoration; for example, Figure 6 shows the spectrum of pixels (55, 110) and (110, 184) for the DC Mall HSI dataset under Case 6, and Figure 7 shows similar results for the RemoteImage HSI dataset. It is easy to see that our proposed method provides the best spectral signature among all of the compared methods, which is closest to the original one. Besides, the NMoG method, which adopts non-i.i.d. MoG to model the noises of HSIs, also has comparable performance. These advantages of DP-GMM and NMoG indicate the necessity of giving more consideration to the complex noise structure of HSIs.

5.2. Real Data Experiments

In this section, two real-world test HSI datasets are used in our experiments: the Hyperspectral Digital Imagery Collection Experiment (HYDICE) Urban (http://lesun.weebly.com/hyperspectral-data-set.html) dataset and the Army Geospatial Center (AGC) Terrain (https://www.agc.army.mil) dataset.

HYDICE Urban Dataset: The original image is $307\times 307\times 210$ in size, and we feed the entire dataset into the denoising algorithms directly. Some bands of Urban are seriously polluted by the atmosphere, water absorption and other unknown noises, such as bands 104–108, 139–151 and 206–210. Additionally, some other bands are contaminated by stripes, deadlines and Gaussian noise, such as bands 1–2, 76 and 198. In this first real data experiment, the rank is set as 7 for all the competing methods.

Figure 8 shows bands 76 (top), 104 (middle) and 139 (bottom) of the restored HSIs of the Urban dataset, and these three bands are typically polluted to varying degrees by different types of noise as analyzed in Figure 1a. Band 76 is relatively clean with several deadlines, and most of the competing methods can remove these deadlines except TDL and BM4D. As for bands 104 and 139, however, some stripes still exist in the restored images for almost all the competing methods. This is because these bands are heavily corrupted and can provide little useful information, and these methods lack noise adaptivity to deal with these complex cases. In contrast, our proposed DP-GMM method is able to completely suppress the stripes and effectively preserves the detailed information. What is more, the images restored by our proposed method are smoother than those restored by other compared methods. These visual results show that our proposed DP-GMM method can remove more complicated noise embedded in HSIs than other competing methods.

Figure 9 shows the vertical mean profiles of bands 76, 104 and 139, which is a quantitative comparison of the restorated results shown in Figure 8. The horizontal axis in Figure 8 represents the column number, and the vertical axis represents the mean digital number value of each column. Band 76 is almost noise-free with only one deadline. There is thus no visible change between the mean profiles before and after the restoration. When the noise becomes more complicated, the curves fluctuate rapidly in the original images for bands 104 and 139. After the restoration processing, the fluctuations are more or less suppressed. Obviously, the DP-GMM methods perform best and obtain the flattest curve without fluctuations. These results are in accordance with the visual results of Figure 8.

AGC Terrain Dataset: The second real data experiment employed the Terrain dataset as the test dataset, and it has $500\times 307$ pixels and 210 bands. The full Terrain image is mainly corrupted by deadlines, atmosphere, water absorption and others, especially the bands 104–109, 138–152 and 204–210. We also set the rank as 7 in this experiment for all the methods.

We selected three typical bands 2, 139 and 151 to display the performance of all the compared methods in Figure 10. These three bands exhibit evident noise distinctness, as shown in Figure 1b, both in terms of distribution types and corruption extent. In band 2, there are some deadlines clustered together, as shown in the amplified area, and SVD, BM4D, WNNM, WSNM, LRMR, LRTV and our proposed DP-GMM can all remove these deadlines completely. As for bands 139 and 151, which contain many stripes and other mixed noise, it is easy to see that the DP-GMM removes more noise, especially stripes, compared with the other methods, owing to its noise adaptivity for different bands. These visual results show that our proposed DP-GMM method is more robust when dealing with the complicated real noise of HSI data.

Figure 11 shows the vertical mean profiles comparison before and after restoration for each competing method, corresponding to the visualization of Figure 10. The horizontal axis in this figure represents the column number, and the corresponding vertical axis represents the mean digital number value of each column. The curves obtained by different methods in band 2 appear to be slightly smoother to different degrees, as compared with that of the original images. However, as for bands 139 and 151, due to the interference of the mixed noise, especially stripes and deadlines, there are sharp fluctuations in the curve for the original images. After the restoration, the fluctuations are suppressed by all of the competing methods to some extent. Obviously, the curves obtained by our proposed method are smoother than others, which indicates the better restoration results of DP-GMM. These quantitative results are also in accordance with the visual results of Figure 10.

6. Conclusions

We have proposed a new HSI restoration method by constructing the noise model as a DP-GMM under the Bayesian framework. Most of the current methods do not give much consideration to the variation of the noise complexity among the different bands of the natural HSI. In contrast, our method uses a hierarchical structure to fit the noise so that the noise of different HSI bands can be flexibly encoded by MoGs with different mixture component numbers. The noise distinctness among HSI bands can thus be faithfully reflected in the model, potentially making it more robust in real HSI scenarios. In order to validate the effectiveness of the proposed methods, we conducted a series of experiments on synthetic and real HSI datasets. For the synthetic HSI datasets, our proposed method performed better than all the competing methods in terms of MPSNR, MSSIM and ERGAS. Specifically, in terms of the MPSNR, our method is on average over 1 dB higher than the current state-of-the-art methods: LRTV and NMoG. For the real HSI dataset, our proposed method can remove almost all of the noises, and obtain better visual restoration results compared with other methods, especially for the heavily contaminated bands.