A Two-Stage Method to Test the Robustness of the Generalized Approximate Message Passing Algorithm

Abstract

We propose a two-stage method to test the robustness of the generalized approximate message passing algorithm (GAMP). A pursuit process based on the marginal posterior probability is inserted in the standard GAMP algorithm to find the support of a sparse vector, and a revised GAMP process is used to estimate the amplitudes of the support. The numerical experiments with simulation and real world data confirm the robustness and performance of our proposed algorithm.

1. Introduction

Compressed sensing (CS) has been a research focus in recent years [1]. Given the support of an unknown K-sparse vector $\mathbf{x}\in {\mathbb{R}}^N$ as $S=\{j\text{:}j\in \{1,\cdots ,N\},x_j\ne 0\}$, we consider an under-determined noisy linear system

$$\mathbf{y}=\mathbf{Ax}+\mathbf{n}$$

(1)

where $\mathbf{y}\in {\mathbb{R}}^M$ is a compressed sampling observation, $\mathbf{n}\in {\mathbb{R}}^M$ is an additive Gaussian noise, $\mathbf{A}\in {\mathbb{R}}^{M\times N}$ is the sensing matrix. The object is to reconstruct $\mathbf{x}$ from $\mathbf{y}$ and $\mathbf{A}$. There are many reconstruction algorithms to solve the CS problem. The message passing based methods, including approximate message passing (AMP) algorithm [2] and generalized approximate message passing (GAMP) algorithm [3], have remained the most efficient ever since they were proposed. They work very well in the zero-mean Gaussian sensing matrix case, but become unstable and divergent in the more general sensing matrix case. For example, although an improvement to the non-zero mean Gaussian sensing matrix case has in a sense been proposed [4], the highly columns-correlated non-Gaussian sensing matrix case is still a problem. We propose a new GAMP-like algorithm, termed Bernoulli-Gaussian Pursuit GAMP (BGP-GAMP), to raise the robustness of standard GAMP algorithm. Our algorithm firstly utilizes the marginal posterior probability of $\mathbf{x}$ to sequentially find the support S, and then estimates the amplitudes on S by a tiny revised GAMP algorithm, termed Fixed Support GAMP (FS-GAMP) which has been proposed in our previous work [5]. A recent work has been proposed by Rodger [6], which is similar to our work, but gives another viewpoint from a neural network statistical model. The numerical experiments verify our idea and show a good performance and robustness.

The rest of this paper is organized as follows. Firstly, we review the standard GAMP, especially Bernoulli-Gaussian prior GAMP, algorithm in Section 2. Secondly, we propose the FsGAMP and BGP-GAMP algorithms in Section 3 and Section 4, respectively. Thirdly, experiments are conducted in Section 5. Finally, we draw a conclusion in the last section.

2. GAMP Algorithm Review

The GAMP algorithm can be classified to minimum mean error estimation form (MMSE-GAMP) and maximum posterior estimation form (MAP-GAMP) [3]. We restrict our attention to MMSE-GAMP, since the problems in the real world are noisy, such that the posterior function has a very complex shape with details which depend on the observation $\mathbf{y}$. However, MMSE is robust to such fluctuations.

Firstly, GAMP algorithm approximates the true marginal posterior $p\left(z_m\right|\mathbf{y})$ by the output channel marginal posterior

$$p_{{\mathsf{z}}_m|{\mathsf{p}}_m}\left(z\right|{\widehat{p}}_m,{\nu }_m^p)\triangleq \frac{p_{{\mathsf{y}}_m|{\mathsf{z}}_m}\left(y_m\right|z)\mathcal{N}(z;{\widehat{p}}_m,{\nu }_m^p)}{\int p_{{\mathsf{y}}_m|{\mathsf{z}}_m}\left(y_m\right|z)\mathcal{N}(z;{\widehat{p}}_m,{\nu }_m^p)\mathrm{d}z},$$

(2)

where ${\mathsf{z}}_m$ is the estimation of ${\left(\mathsf{Ax}\right)}_m$ condition on the observation ${\mathsf{y}}_m$, and $p_{{\mathsf{y}}_m|{\mathsf{z}}_m}\left(y_m\right|z)$ is the likelihood of the output channel ${\mathsf{y}}_m={\mathsf{z}}_m+{\mathsf{w}}_m$. It is worth noting that ${\mathsf{w}}_m$ means a noise, a random variable, and it corresponds to the estimated random variable ${\mathsf{z}}_m$, instead of the original noise ${\mathsf{n}}_m$ in Equation (1). Therefore, $w_m$ is the residual after estimation of ${\left(\mathbf{Ax}\right)}_m$. The iterative algorithm computes the conditional mean and variance of ${\mathsf{z}}_m$. We want to obtain the posterior mean and variance estimates

$$\mathbb{E}\left\{{\mathsf{z}}_m\right|{\mathsf{p}}_m={\widehat{p}}_m,{\nu }_m^p\}=\int z_m{p}_{{\mathsf{z}}_m|{\mathsf{p}}_m}\left(z_m\right|{\widehat{p}}_m,{\nu }_m^p)\mathrm{d}{z}_m$$

(3)

$$\mathrm{var}\left\{{\mathsf{z}}_m\right|{\mathsf{p}}_m={\widehat{p}}_m,{\nu }_m^p\}=\int {(z_m-\mathbb{E}\left\{{\mathsf{z}}_m\right|{\mathsf{p}}_m={\widehat{p}}_m,{\nu }_m^p\})}^2{p}_{{\mathsf{z}}_m|{\mathsf{p}}_m}\left(z_m\right|{\widehat{p}}_m,{\nu }_m^p)\mathrm{d}{z}_m.$$

(4)

Secondly, GAMP algorithm approximates the true marginal posterior $p\left(x_n\right|\mathbf{y})$ by the input channel marginal posterior

$$p_{{\mathsf{x}}_n|{\mathsf{r}}_n}\left(x\right|{\widehat{r}}_n,{\nu }_n^r)\triangleq \frac{p_{{\mathsf{x}}_n}\left(x\right)\mathcal{N}(x;{\widehat{r}}_n,{\nu }_n^r)}{\int p_{{\mathsf{x}}_n}\left(x\right)\mathcal{N}(x;{\widehat{r}}_n,{\nu }_n^r)\mathrm{d}x}$$

(5)

where $p_{{\mathsf{x}}_n}\left(x\right)$ is the prior of random variable ${\mathsf{x}}_n$ in the input channel ${\mathsf{r}}_n={\mathsf{x}}_n+{\mathsf{v}}_n$. We want to obtain the posterior mean and variance estimates

$$\mathbb{E}\left\{{\mathsf{x}}_n\right|{\mathsf{r}}_n={\widehat{r}}_n,{\nu }_n^r\}=\int x_n{p}_{{\mathsf{x}}_n|{\mathsf{r}}_n}\left(x_n\right|{\widehat{r}}_n,{\nu }_n^r)\mathrm{d}{x}_n$$

(6)

$$\mathrm{var}\left\{{\mathsf{x}}_n\right|{\mathsf{r}}_n={\widehat{r}}_n,{\nu }_n^r\}=\int {(x_n-\mathbb{E}\left\{{\mathsf{x}}_n\right|{\mathsf{r}}_n={\widehat{r}}_n,{\nu }_n^r\})}^2{p}_{{\mathsf{x}}_n|{\mathsf{r}}_n}\left(x_n\right|{\widehat{r}}_n,{\nu }_n^r)\mathrm{d}{x}_n.$$

(7)

GAMP algorithm is an intrinsic parallel computing process, since it decouples ${\mathsf{y}}_m={\sum }_{n=1}^N{a}_{mn}{\mathsf{x}}_n+{\mathsf{w}}_m$ to the input channel ${\mathsf{r}}_n={\mathsf{x}}_n+{\mathsf{v}}_n$ estimation and the output channel ${\mathsf{y}}_m={\mathsf{z}}_m+{\mathsf{w}}_m$ estimation. According to the factor graph theory, the messages are probability measures. They can be classified to two types: from the variable node to the factor node, and from the factor node to the variable node. For AMP and GAMP algorithms, all the messages are passed simultaneously between the factor nodes and variable nodes along the edges on the factor graph. These probability measures can be approximated as Gaussian density, based on the derivation of AMP and GAMP algorithms. Therefore, the mean and variance of every entry of $\mathbf{x}$ are i.i.d. Gaussian random variable. This intrinsic parallel computing implies a flaw: if $\mathbf{A}$ does not satisfy restricted isometry property (RIP) very well, For example $\mathbf{A}$ has highly correlated columns, GAMP will diverge. In real world problems, such as radar application, the sensing matrix $\mathbf{A}$ may be deterministic, not statistics. Therefore the RIP analysis can not be applied to this situation, researchers use simulation to find the behavior of this kind of reconstruction. We will show it in Section 5.

The steps of GAMP are listed in Algorithm 1, where ${\left|\mathbf{A}\right|}^2$ is a componentwise magnitude squared, the elementwise product and division are denoted as ⊙ and ⊘, respectively, notation $[·]$ means the componentwise form of a vector, e.g., $\mathbf{a}=\left[a_i\right]$, and notation $|·|$ means the cardinality of a set, or the number of non-zero elements of a sparse vector.

Algorithm 1: Generalized Approximate Message Passing (GAMP).

Input: $\mathbf{y},\mathbf{A}$  Description: $\mathbf{x}(t+1),{\nu }^{\mathbf{x}}(t+1)$ 1: $\widehat{\mathbf{x}}(t=1)\leftarrow \mathbf{0};{\nu }^{\mathbf{x}}(t=1)\leftarrow \mathbf{1};\widehat{\mathbf{s}}(t=0)\leftarrow \mathbf{0}$ 2: repeat 3:  ${\nu }^{\mathbf{p}}\left(t\right)\leftarrow {\left|\mathbf{A}\right|}^2{\nu }^{\mathbf{x}}\left(t\right)$ 4:  $\widehat{\mathbf{p}}\left(t\right)\leftarrow \mathbf{A}\widehat{\mathbf{x}}\left(t\right)-\widehat{\mathbf{s}}(t-1)\odot {\nu }^{\mathbf{p}}\left(t\right)$ 5:  $\widehat{\mathbf{z}}\left(t\right)\leftarrow \left[{\widehat{z}}_m\left(t\right)\right],\mathrm{where}{\widehat{z}}_m\left(t\right)\triangleq \mathbb{E}\left\{{\mathsf{z}}_m\right|{\mathsf{p}}_m={\widehat{p}}_m\left(t\right),{\nu }_m^p\left(t\right)\}$ 6:  ${\nu }^{\mathbf{z}}\left(t\right)\leftarrow \left[{\nu }_m^z\left(t\right)\right],\mathrm{where}{\nu }_m^z\left(t\right)\triangleq \mathrm{var}\left\{{\mathsf{z}}_m\right|{\mathsf{p}}_m={\widehat{p}}_m\left(t\right),{\nu }_m^p\left(t\right)\}$ 7:  $\widehat{\mathbf{s}}\left(t\right)\leftarrow (1\oslash {\nu }^{\mathbf{p}}\left(t\right))\odot (\widehat{\mathbf{z}}\left(t\right)-\widehat{\mathbf{p}}\left(t\right))$ 8:  ${\nu }^{\mathbf{s}}\left(t\right)\leftarrow (1\oslash {\nu }^{\mathbf{p}}\left(t\right))\odot (1-{\nu }^{\mathbf{z}\left(t\right)}\oslash {\nu }^{\mathbf{p}}\left(t\right))$ 9:  ${\nu }^{\mathbf{r}}\left(t\right)\leftarrow 1\oslash ({({\left|\mathbf{A}\right|}^2)}^T{\nu }^{\mathbf{s}}(t))$ 10:  $\widehat{\mathbf{r}}\left(t\right)\leftarrow \widehat{\mathbf{x}}\left(t\right)+{\nu }^{\mathbf{r}}\left(t\right)\odot \left({\mathbf{A}}^T\widehat{\mathbf{s}}\left(t\right)\right)$ 11:  $\widehat{\mathbf{x}}(t+1)\leftarrow \left[{\widehat{x}}_n(t+1)\right],\mathrm{where}{\widehat{x}}_n(t+1)\triangleq \mathbb{E}\left\{{\mathsf{x}}_n\right|{\mathsf{r}}_n={\widehat{r}}_n\left(t\right),{\nu }_n^r\left(t\right)\}$ 12:  ${\nu }^{\mathbf{x}}(t+1)\leftarrow \left[{\nu }_n^x(t+1)\right],\mathrm{where}{\nu }_n^x(t+1)\triangleq \mathrm{var}\left\{{\mathsf{x}}_n\right|{\mathsf{r}}_n={\widehat{r}}_n\left(t\right),{\nu }_n^r\left(t\right)\}$ 13: until Terminated

When the prior is a Bernoulli-Gaussian distribution [7]

$$p_{{\mathsf{x}}_n}\left(x\right)=(1-\lambda )\delta \left(x\right)+\lambda \mathcal{N}(x;\theta ,\varphi ),$$

(8)

where λ is the sparsity-rate $K/N$, and the mean θ and variance $\varphi$ are hyper-parameters which are set to zero and one respectively at the initiation and those parameters do not change in the algorithm running time. It is called BG-GAMP algorithm. One can obtain the input channel marginal posterior

$$p_{{\mathsf{x}}_n|{\mathsf{r}}_n}\left(x\right|{\widehat{r}}_n,{\nu }_n^r)=\left((1-\lambda )\delta \left(x_n\right)+\lambda \mathcal{N}(x_n;\theta ,\varphi )\right)\frac{\mathcal{N}(x_n;{\widehat{r}}_n,{\nu }_n^r)}{{\zeta }_n}$$

(9)

$$\begin{array}{cc}& =(1-{\pi }_n)\delta \left(x_n\right)+{\pi }_n\mathcal{N}(x_n;{\gamma }_n,{\mu }_n)\hfill \end{array}$$

(10)

with the normalization factor

$${\zeta }_n\triangleq \int p_{{\mathsf{x}}_n}\left(x\right)\mathcal{N}(x;{\widehat{r}}_n,{\nu }_n^r)\mathrm{d}x=(1-\lambda )\mathcal{N}(0;{\widehat{r}}_n,{\nu }_n^r)+\lambda \mathcal{N}(0;{\widehat{r}}_n-\theta ,{\nu }_n^r+\varphi )$$

(11)

and $({\widehat{r}}_n,{\nu }_n^r,\lambda ,\theta ,\varphi )$-dependent variables

$${\pi }_n\triangleq \frac{1}{1+{\left(\frac{\lambda \mathcal{N}({\widehat{r}}_n;\theta ,{\nu }_n^r+\varphi )}{(1-\lambda )\mathcal{N}(0;{\widehat{r}}_n,{\nu }_n^r)}\right)}^{-1}}$$

(12)

$${\gamma }_n\triangleq \frac{{\widehat{r}}_n/{\nu }_n^r+\theta /\varphi }{1/{\nu }_n^r+1/\varphi }$$

(13)

$${\mu }_n\triangleq \frac{1}{1/{\nu }_n^r+1/\varphi },$$

(14)

where ${\pi }_n$ is the input channel marginal posterior support probability $\mathrm{Pr}\{{\mathsf{x}}_n\ne 0|{\widehat{r}}_n,{\nu }_n^r\}$, according to Equation (10), and then $\mathrm{Pr}\{{\mathsf{x}}_n=0|{\widehat{r}}_n,{\nu }_n^r\}=1-{\pi }_n$. Substituting Equation (10) to Equations (6) and (7), we have

$${\widehat{x}}_n={\pi }_n{\gamma }_n+(1-{\pi }_n)\theta$$

(15)

$${\nu }_n^x={\pi }_n({\left|{\gamma }_n\right|}^2+{\mu }_n)+(1-{\pi }_n){\left|\theta \right|}^2-{\left|{\widehat{x}}_n\right|}^2.$$

(16)

The hypothesis of Bernoulli-Gaussian data $\mathbf{x}$ is sparse. Sparsity-rate λ has been set to $K/N$. Although the sparsity K is unknown in practice, we can choose a small K value and an error bound, and then we do the recovery algorithm. If the recovery result makes the final reconstruction error lower than the bound, this result can be accepted, otherwise we decrease the K value and do the recovery again until the bound has been achieved. This procedure is like the Least Angle Regression (LARS) algorithm which tests all the regularization hyper-parameters.

3. Fixed Support GAMP Algorithm

According to the belief propagation theory, if the number of cycles on a factor graph is low, then the robustness of message passing will be enhanced [8]. Therefore, we have an idea that the message passing can be restricted just on the support Λ if Λ has been determined in some way. That is the meaning of FS-GAMP. The steps of FS-GAMP are listed in Algorithm 2. It is very similar to the GAMP algorithm, but the major difference between FS-GAMP and GAMP is that the entries which do not belong to the support Λ are assigned to 0 directly.

Algorithm 2: Fixed Support GAMP (FS-GAMP).

Input: $\mathbf{y},\mathbf{A},\Lambda$  Description: $\mathbf{x}(t+1),{\nu }^{\mathbf{x}}(t+1)$ 1: ${\widehat{\mathbf{x}}}_{\Lambda }(t=1)\leftarrow {\mathbf{0}}^{|\Lambda |\times 1};{\nu }^{{\mathbf{x}}_{\Lambda }}(t=1)\leftarrow {\mathbf{1}}^{|\Lambda |\times 1};\widehat{\mathbf{s}}(t=0)\leftarrow \mathbf{0}$ 2: repeat 3:  ${\nu }^{\mathbf{p}}\left(t\right)\leftarrow {\left|{\mathbf{A}}_{:,\Lambda }\right|}^2{\nu }_{\Lambda }^{\mathbf{x}}\left(t\right)$ 4:  $\widehat{\mathbf{p}}\left(t\right)\leftarrow {\mathbf{A}}_{:,\Lambda }{\widehat{\mathbf{x}}}_{\Lambda }\left(t\right)-\widehat{\mathbf{s}}(t-1)\odot {\nu }^{\mathbf{p}}\left(t\right)$ 5:  {(line 5)–(line 8) of GAMP} 6:  ${\nu }_{\Lambda }^{\mathbf{r}}\left(t\right)\leftarrow 1\oslash ({({\left|{\mathbf{A}}_{:,\Lambda }\right|}^2)}^T{\nu }^{\mathbf{s}}\left(t\right))$ 7:  ${\widehat{\mathbf{r}}}_{\Lambda }\left(t\right)\leftarrow {\widehat{\mathbf{x}}}_{\Lambda }\left(t\right)+{\nu }_{\Lambda }^{\mathbf{r}}\left(t\right)\odot ({\mathbf{A}}_{:,\Lambda }^T\widehat{\mathbf{s}}\left(t\right))$ 8:  ${\widehat{\mathbf{x}}}_{\Lambda }(t+1)\leftarrow \left[{\widehat{x}}_i(t+1)\right],\mathrm{where}i\in \Lambda ,{\widehat{x}}_i(t+1)\triangleq \mathbb{E}\left\{{\mathsf{x}}_i\right|{\mathsf{r}}_i={\widehat{r}}_i\left(t\right),{\nu }_i^r\left(t\right)\}$ 9:  ${\nu }_{\Lambda }^{\mathbf{x}}(t+1)\leftarrow \left[{\nu }_i^x(t+1)\right],\mathrm{where}i\in \Lambda ,{\nu }_i^x(t+1)\triangleq \mathrm{var}\left\{{\mathsf{x}}_i\right|{\mathsf{r}}_i={\widehat{r}}_i\left(t\right),{\nu }_i^r\left(t\right)\}$ 10: until

4. Bernoulli-Gaussian Pursuit GAMP Algorithm

The hypothesis of FS-GAMP algorithm is a known support Λ, but we do not know it. Now back to BG-GAMP, Equations (15) and (16) use the marginal posterior probability ${\pi }_n$ to weight the non-zero estimation part $({\gamma }_n,{\mu }_n)$ and the zero prior part θ. At each iteration of BG-GAMP, when ${\pi }_n$ is greater than zero-one switch threshold $\zeta =0.5$, the non-zero estimation part takes over the major contribution. In this research we suppose that the non-zero prior probability of every entry in $\mathbf{x}$ is $0.5$. That implies the maximal entropy assumption. Therefore, the zero-one switch threshold is also set to $0.5$ in accordance with the maximal entropy assumption. For a matrix which does not meet RIP conditions very well, we observe that the mean and the variance of one ${\pi }_n>\zeta$ entry shows no sign of convergence, while some entries’ ${\pi }_j<\zeta ,j\ne n$ already increase fast. When they are greater than ζ, the means and the variances of these entries also show no sign of convergence. This phenomenon spreads to more and more entries, causing all entries to be totally diverged.

This observation inspires us to maintain the current computing status until one ${\pi }_n>\zeta$ entry has converged, and then continue the next entry (or entries). In other words, we introduce a pursuit process of lines (4)–(6) in Algorithm 3 to sequentially find the support of $\mathbf{x}$, and then estimate the amplitudes on this support by FS-GAMP. BGP-GAMP means BG-GAMP with an intrinsic pursuit process. The steps of BGP-GAMP are listed in Algorithm 3. Note that the initial input parameters of FS-GAMP in line (7) are the output parameters of GAMP in line (3).

The computational complexity of GAMP part in BGP-GAMP algorithm is the same as the standard GAMP, that is $O\left(MN\right)$, at each iteration. The computational complexity of the FS-GAMP part in BGP-GAMP is $O\left(M\right|\Lambda \left|\right)$ at each iteration. Because $|\Lambda |\le K$, the computational complexity of BGP-GAMP at each iteration is less than $O\left(M\right(N+K\left)\right)$.

Algorithm 3: Bernoulli-Gaussian Pursuit GAMP (BGP-GAMP).

Input: $\mathbf{y},\mathbf{A}$  Description: $\mathbf{x}(t+1),{\nu }^{\mathbf{x}}(t+1)$ 1: $\widehat{\mathbf{x}}(t=1)\leftarrow \mathbf{0};{\nu }^{\mathbf{x}}(t=1)\leftarrow \mathbf{1};\widehat{\mathbf{s}}(t=0)\leftarrow \mathbf{0};\Lambda \leftarrow$ Ø 2: repeat 3:  {(line 3)–(line 12) of GAMP} 4:  $\pi (t+1)\leftarrow \left[{\pi }_n(t+1)\right]$, where ${\pi }_n(t+1)$ obtained from (12) 5:  $J\leftarrow \{j:{\pi }_j(t+1)>0.5\}$ 6:  $\Lambda \leftarrow \Lambda \cup J$ 7:  {FS-GAMP} 8: until Terminated.

5. Experiments

We investigate the performance of BGP-GAMP by three experiments. Four benchmark algorithms are adopted to compare with BGP-GAMP. Orthogonal matching pursuit (OMP) [9] and basis pursuit (BP) by interior point method to solve linear programming [10] are the most robust ${\ell }_0$ and ${\ell }_1$ optimization algorithms, other algorithms (old or new) are weaker than them in robustness. AMP and GAMP are not only the root cause of the robust problem, but also the basis of our method. The implementation of OMP comes from (http://www.cmc.edu/pages/faculty/DNeedell), BP comes from l1magic (http://users.ece.gatech.edu/justin/l1magic), AMP comes from Kamilov’s work (http://people.epfl.ch/ulugbek.kamilov) and GAMP comes from gamplab (http://eeweb.poly.edu/srangan/index.html).

We do $Q=100$ trials in each experiment, and use the relative mean square error (RMSE) $\xi =1/Q\times {\sum }_{l=1}^Q\parallel {\widehat{\mathbf{x}}}^l-{\mathbf{x}}^l{\parallel }_2/{\parallel {\mathbf{x}}^l\parallel }_2$ as the performance metric. Given $N/M=3,M/K=5$, we set $N=256$, and then $M=86,K=18$. Choose K locations at random as the support of $\mathbf{x}$, and the amplitude of each non-zero entry $x_i$ is independently drawn from $\mathcal{N}(0,1)$. We set the signal-to-noise ratio (SNR) to 25 dB.

The first experiment, termed γ test, investigates non-zero mean Gaussian sensing matrix case. We create two types of sensing matrix: type 1 means $A(:,j)=\gamma /N+{\mathbf{a}}_j/\sqrt{N}$; type 2 means $A(:,j)=\gamma /N+{\mathbf{a}}_j/{\parallel {\mathbf{a}}_j\parallel }_2$, where each entry of ${\mathbf{a}}_j$ is i.i.d. drawn from $\mathcal{N}(0,1)$. Set the range of γ to $[0,1,1.8,1.9,1.95,2,3,4,5,6,7,8,9,10,11,12]$. Figure 1a shows that GAMP violently diverges at $\gamma =2$, AMP fast diverges at $\gamma =5$. The severely sharp divergence of GAMP and AMP comes from the phase transition property of compressed sensing. Although BGP-GAMP also begins to diverge at $\gamma =5$, the deterioration speed is slower than AMP. Furthermore, the RMSE of BGP-GAMP is always less than that of AMP. When $\gamma <4$, the RMSE of BGP-GAMP is a little less than that of OMP, and when $\gamma <6$, the RMSE of BGP-GAMP is obviously less than that of BP. Figure 1b shows that AMP violently diverges at $\gamma =1.95$, GAMP violently diverges at $\gamma =3$, but BGP-GAMP just begins to diverge at $\gamma =8$. The RMSE of BGP-GAMP is always less than that of BP, and less than that of OMP until $\gamma =8$. Moreover, Figure 1a,b shows that BGP-GAMP outperforms OMP slightly in performance over a parameter range $\gamma <2$. In summary, our proposed algorithm obtains better robustness compared with the standard AMP/GAMP algorithms, at the same time our proposed algorithm achieves performance advantage over a suitable parameter range compared with the standard OMP and BP algorithms.

The second experiment, termed α test, investigates the highly correlated columns sensing matrix $\mathbf{A}$ case. The elements of $\mathbf{A}$ are neither normal nor i.i.d distributed. Create two types of sensing matrix: type 1 means $\mathbf{A}=(1/N)\mathbf{P}\mathbf{Q},p_{mr},q_{rn}\sim N(0,1)$, where $\mathbf{P}\in {\mathbb{R}}^{M\times R},\mathbf{Q}\in {\mathbb{R}}^{R\times N}$, and $R\triangleq \alpha N$; type 2 means $\mathbf{A}=\mathbf{P}\mathbf{Q}$ like type 1, but normalizes each column of $\mathbf{A}$. We know that $\mathbf{A}$ is low rank for $\alpha <M/N$. Set the range of α to $[0.20.3\cdots 1.0]$. Since GAMP totally diverges at every α with type 1 and type 2 cases, it is not visible in Figure 2a,b. AMP also totally diverges at every α with type 2 case, therefore it is also not visible in Figure 2b. The two figures show that BGP-GAMP remains stable even in a low rank scenario $\alpha \approx 0.4$, and its performance is obviously better than AMP and BP, and a little better than OMP.

It is worth noting that the threshold $0.1$ in Figure 1 and Figure 2 is the relative mean square error, not absolute mean square error, which means the average error amplitude is just $1/100$ of the original signal amplitude. These figures show, for instance, that the standard AMP algorithm and BP algorithm are approaching this threshold. Therefore, it is not a poor performance. We do not consider MAP-GAMP since MAP estimation is not general as MMSE estimation, please see [11] Section 3.6.3 for more discussion.

The last experiment uses real world data, namely, a segment of music by Mozart. The data has 81,000 sample points, and it is segmented to 54 blocks, each block has $N=1500$ sample points. The compressed sampling rate is set to $1/3$, therefore, the measurement number is $M=500$. We create the sensing matrix by DCT transform: $\mathbf{A}(i,:)=(1/N){\left(\mathrm{DCT}(\Phi (i,:))\right)}^T$, where every element of Φ is drawn from $\mathcal{N}(0,1)$, and then multiplied a scale factor $1/\sqrt{M}$ to it. Figure 3a shows the modulus of DCT coefficients of the original signal, and Figure 3b shows the modulus of DCT coefficients of the recovered signal. We can see that the major energy has been recovered from the compressed measurements.

6. Conclusions

In this paper we propose a new compressed sensing algorithm, the two-stage method. This algorithm pursues the support of a sparse vector sequentially, an meanwhile estimates the amplitudes of the sparse vector by the FS-GAMP process. Numerical experiments with simulation and real world data confirm that our new algorithm performs very well in correctness and robustness. In future work we plan to analyze the variation rate of the marginal posterior probability π to obtain better results.