Reference Information Based Remote Sensing Image Reconstruction with Generalized Nonconvex Low-Rank Approximation

Abstract

Because of the contradiction between the spatial and temporal resolution of remote sensing images (RSI) and quality loss in the process of acquisition, it is of great significance to reconstruct RSI in remote sensing applications. Recent studies have demonstrated that reference image-based reconstruction methods have great potential for higher reconstruction performance, while lacking accuracy and quality of reconstruction. For this application, a new compressed sensing objective function incorporating a reference image as prior information is developed. We resort to the reference prior information inherent in interior and exterior data simultaneously to build a new generalized nonconvex low-rank approximation framework for RSI reconstruction. Specifically, the innovation of this paper consists of the following three respects: (1) we propose a nonconvex low-rank approximation for reconstructing RSI; (2) we inject reference prior information to overcome over smoothed edges and texture detail losses; (3) on this basis, we combine conjugate gradient algorithms and a single-value threshold (SVT) simultaneously to solve the proposed algorithm. The performance of the algorithm is evaluated both qualitatively and quantitatively. Experimental results demonstrate that the proposed algorithm improves several dBs in terms of peak signal to noise ratio (PSNR) and preserves image details significantly compared to most of the current approaches without reference images as priors. In addition, the generalized nonconvex low-rank approximation of our approach is naturally robust to noise, and therefore, the proposed algorithm can handle low resolution with noisy inputs in a more unified framework.

1. Introduction

The multispectral sensors of a satellite system sometimes fail or degrade after the system is deployed, and degraded sensors provide blurred and noisy images. For example, one band of the multispectral sensor on board the MODIS satellite degraded after the launch, while the rest of the bands remained unaffected. Damage to images can also be caused by defocusing, atmospheric turbulence and some other factors besides the damage of sensors. Owing to sensor failure and poor observation conditions, remote sensing images are easily subjected to information loss, which hinders our effective analysis of the Earth. Digital imaging processing techniques are commonly used to improve image quality and increase application potential.

In a typical multispectral satellite imaging system, multiple images from different sensors in the same area are available. When one of those images in a multiple image set is degraded, another image in the set can be used as a prior image for reconstruction. However, some image reconstruction approaches have used multiple reference input images. Reconstruction approaches utilize multiple reference input images, which are used as the prior to improve the reconstruction performance in terms of spatial resolution and noise reduction. Li et al. [1] proposed a model utilizing spectral and temporal information as the complementary information for reconstructing the missing information in remote sensing images. Lizhe et al. and Hao et al. [2,3] developed a compressed sensing object function that uses a reference image as a prior. The sparsity constraints in the transform domain come from the target image, and the gradient priors in the spatial domain come from the auxiliary reference image. Peng et al. [4] proposed an algorithm based on the total variation approach using an auxiliary image. These methods have seen the potential for higher reconstruction performance when aided references are present, but the accuracy and quality of reconstruction are lacking.

Since the image prior knowledge plays a critical role in the performance of image reconstruction algorithms, designing effective regularization terms to reflect the image priors is the core of remote sensing images (RSI) reconstruction. To sum up, most of these reconstruction methods are based on some specific prior knowledge of the images, and they use the spectral or spatial information of the remote sensing images to recover the current pixel or to obtain high resolution images. Low-rank-inducing regularization terms have recently received considerable attention in sparse representation. The principle of low-rank approximation is that similar patches are grouped to share a similar underlying structure and form a low-rank matrix appropriately [5,6], and it can be solved by using the efficient singular value decomposition as the optimization tool. Using the low-rank framework, Dong et al. [7] proposed a nonlocal low-rank algorithm, called the spatially-adaptive iterative singular-value thresholding method. Making use of similar information from another relative band, it is easy to extend these nonconvex penalty functions on singular values of a matrix to improve low-rank matrix recovery performance.

In the proposed algorithm, instead of using the ${\ell }_0$ norm as the minimization constraint of compressive sensing, the generalized nonconvex low-rank approximation is exploited as the basic unit of sparse representation. The model integrates the wavelet texture feature as structural and textural reference information to establish a novel compressive sensing algorithm for remote sensing image reconstruction (GNLR-RI). Compared to traditional ${\ell }_0$ norm-based compressive sensing, the proposed GNLR-RI algorithm makes three contributions: (1) to overcome over-smoothing and texture detail loss of sparse representation, the wavelet coefficient of the reference image is injected as texture reference constraint information; (2) nonlocal similar patches from a reference image are extracted in the low-rank approximation step; (3) the spectral reference information and similarity within bands are jointly used. Therefore, the proposed algorithm can get accurate reconstruction results and improve calculation speed. The solution of the proposed algorithm is derived by the conjugate gradient algorithm, single value threshold (SVT), simultaneously calculating the low-rank matrix of similar image patches and then estimating the reconstructed image.

The remainder of this paper is organized as follows. A brief review is given on the related work of remote sensing image reconstruction with reference images and other generalized reconstruction methods, and auxiliary information is added in Section 2. Section 3 introduces the generalized nonconvex low-rank approximation model, which contains eight surrogate functions. The proposed model of introducing the generalized nonconvex low-rank approximation algorithm and the solution of the proposed model are presented in Section 4. Experiments are illustrated in Section 5. Finally, Section 6 summarizes our conclusions.

2. Related Work

2.1. RSI Reconstruction with a Reference

Reference images are used to restore missing information of pixel blocks, possibly caused by thick clouds or contrails, invalid detectors, optical dust, etc. For example, toward solving problems with the Aqua Moderate Resolution Imaging Spectroradiometer (MODIS) Band 6 and the Scan Line Corrector (SLC) failure on Landsat ETM+, Li et al. [1] introduced spectral and temporal pixels for reconstructing the missing information in remote sensing images by introducing information from auxiliary images, modulating the injected gray level and regularizing with a synthesis model and an analysis model on the basis of sparse representation. Their experiments showed that the synthesis model is suitable for using both spectral and temporal complementation, while spectral complementation is the best way for the analysis model.

For remote sensing reconstruction or restoration, some work has emerged that uses reference images to improve reconstruction performance. Lizhe et al. [2] proposed a compressed sensing-based model that used an auxiliary image from another sensor or time to reconstruct remote sensing images. Their method assumed that directions of edges and texture are similar in both the reference image and the reconstruction image, and therefore, they built a gradient similarity-constrained cost item to regularize the reconstruction image. Hao et al. [3] proposed a reconstruction model of a remote sensing image using priors of the auxiliary image from another sensor or time. The model employed priors from both the transform domain and spatial domain. Furthermore, the sparsity priors in the transform domain come from the target image, and the gradient priors in the spatial domain come from the reference image. The algorithm is based on the Bregman split method to optimize the hybrid regularization. Peng et al. [8,9] proposed a total variation approach with an auxiliary image to recover a multispectral image. The reference image provides texture and edge similarities to the degraded image by the regulating strength and direction of smoothness in an anisotropic way, and the amount of prior information is adaptively estimated with normalized local mutual information. Similar techniques are implemented in the partial differential equations for denoising of remote sensing images. These methods have the potential for higher reconstruction performance when aided references are present.

2.2. Generalized RSI Reconstruction

For ensuring image quality, additional prior information from images is needed. There are generalized remote sensing image reconstruction methods to improve image quality, to increase application potential and to implement digital image processing. The compressed sensing (CS) theory has gained much attention as a fundamental and newly-developed methodology in the information field. The CS theory states that the image that has a sparse representation in a certain domain can be recovered from a reduced set of measurements largely below Nyquist sampling rates [10,11,12]. However, Fowler [13] proposed a reconstruction strategy, compressive-projection principal component analysis (CPPCA), which recovers the hyperspectral image dataset using principal component analysis (PCA). Specifically, the CPPCA recovers both the coefficients associated with the PCA transform and an approximation to the PCA transform basis itself. Ly et al. [14] extended the concept of partitioned reconstruction to the spectral dimension of a hyperspectral dataset and incorporated into CPPCA the paradigm of segmented PCA, in which a dataset is segmented into multiple spectral partitions, and PCA is applied in each segment independently. Since multiple predictions drawn for a pixel vector of interest were made from spatially-neighboring pixel vectors within an initial non-predicted reconstruction, Chen et al. [15] proposed a two-phase hypothesis-generation procedure based on the partitioning and merging of spectral bands according to the correlation coefficients between bands to fine-tune the hypotheses.

The transforms that allow the image to have a sparse representation are named sparsifying transforms. One of the most commonly-used sparsifying transforms is finite difference [16], which is more often known as total variation (TV) regularization for image recovery. The popularity of TV regularization lies in its desirable properties, such as convexity, simplicity and its ability to preserve edges. For example, Xiaohua et al. [17] proposed a novel Inverse Synthetic Aperture Radar (ISAR) imaging framework combining a local sparsity constraint and a nonlocal total variation (NLTV). The sparsity is a prior form in which the number of strong scattering points is smaller than that of pixels in the image plane. It plays the role of the classification of the strong scattering points from the clutter background. Xiyan et al. [18] considered the fusion problem as the colorization of each pixel in the panchromatic image and, thus, introduced a term concerning the gradient of the panchromatic image in the function of total variation to preserve edges. Peng et al. [9] indicated that the distributions of pixel intensity of the multimodal image of different CCD sensors differ greatly form each other, and the directions of their edges are very similar. Then, the edge information and these similar structures are used as the important priors or constraints in the total variation image restoration.

On the other hand, similar patches are grouped such that patches in each group share a similar underlying structure to form a low-rank approximation. The low rank induces penalties related to simultaneous sparse coding (SSC), group sparsity and structured sparsity by employing the low-rank characteristic of nonlocal patches in sparse representation. Hongyan et al. [19] introduced a new hyperspectral imagery restoration method based on the low-rank matrix recovery, which suggested that a clean hyperspectral imagery patch can be regarded as a low-rank matrix. Wei et al. [20] proposed a method that integrates the nuclear norm, TV regularization and the ${\ell }_1$ norm into a unified framework. The nuclear norm is used to exploit the spectral low-rank property, and the TV regularization is adopted to explore the spatial piecewise smooth structure of the hyperspectral imagery. Usually, nonconvex approaches, like ${\ell }_p$ ($0<p<1$) or the nuclear ${\ell }_1$ norm minimization will guarantee a better recovery by directly attacking the ${\ell }_0$ minimization problem [21,22].

An important research area is the sparse representation method with dictionary learning (DL), which builds an adaptive basis from particular image instances for sparse approximations [23]. Qiegen et al. [16] propose a novel gradient-based dictionary learning method for image recovery that effectively integrates the popular total variation (TV) and dictionary learning technique into the same framework. Specifically, they train dictionaries from the horizontal and vertical gradients of the image and then reconstruct the desired image using the sparse representations of both derivatives. The dictionary learning strategy alleviates the drawback of a fixed basis (finite difference, wavelet, etc.) that a given basis might not be universally optimal for all images [24], but at the cost of non-convexity and non-linearity [25]. Most existing DL methods adopt a two-step iterative scheme, where the sparse approximations are found with the dictionary fixed, and the dictionary is subsequently optimized based on the current sparse coefficients [26].

Exact reconstruction of sparse reconstruction can be achieved by nonlinear algorithms, such as orthogonal matching pursuit (OMP) [27], the iterative shrinkage algorithm [28], the Bregman split algorithm [29] and the alternative direction multiplier method [30]. Recent advances have shown that replacing the ${\ell }_0$ norm with a nonconvex surrogate function can obtain better sparse representation recovery performance. Weisheng et al. [6] proposed a spatially-adaptive iterative singular value thresholding (SAIST) method by revealing the assumption that the basis is orthogonal, and the SSC problem can be rewritten in a low-rank view in some special cases. Shuyuan et al. [31] proposed a multi-task learning and K-SVD-based super resolution image restoration method that learned a redundant dictionary from some example image patches.

2.3. Reconstruction Model Based on Reference Images

As in psychological research of the visual cortex, energy distribution in the wavelet coefficient can be identified as the unique features of texture characterization, so we have extra texture features of the reference image and target image. Let $\alpha (i,j)$ denote the wavelet coefficient, $F_{\ell 1\_energy}$ denote the ${\ell }_1$ norm-based energy, $F_{sd}$ denote the standard deviation and $F_{aad}$ and $F_{entropy}$ denote the average absolute deviation and entropy.

$$F_{l1\_energy}\left(\alpha \right)=\frac{1}{MN}\sum _{i=1}^M\sum _{j=1}^N\left|\alpha (i,j)\right|$$

(1)

$$F_{sd}\left(\alpha \right)={\left(\frac{1}{MN}\sum _{i=1}^M\sum _{j=1}^N{\left(\alpha (i,j)-\overline{\alpha }\right)}^2\right)}^{1/2}$$

(2)

$$F_{aad}\left(\alpha \right)=\frac{1}{MN}\sum _{i=1}^M\sum _{j=1}^N\left|\alpha (i,j)-\overline{\alpha }\right|$$

(3)

$$F_{entropy}\left(\alpha \right)=-\frac{1}{MN}\sum _{i=1}^M\sum _{j=1}^N{\left|\alpha (i,j)\right|}^{2}log{\left|\alpha \left(i,j\right)\right|}^2$$

(4)

where $\alpha (i,j)$ is the wavelet coefficient and $\overline{\alpha }$ is the mean value of the wavelet coefficient. Research shows that the combination of these characteristics results in a better performance. Therefore, in this study, the empirical combination with the ${\ell }_1$ energy norm $F_{\ell 1\_energy}$, the standard deviation $F_{sd}$, the average absolute deviation $F_{aad}$ and the entropy $F_{entropy}$ of every wavelet coefficient sub-patch were computed to form the texture feature vector F. As for the reference image, we obtain the texture feature vector $F_{ref}$ in the same way and compute the metric distance to objectively measure the similarity of these two texture feature vectors. The literature shows that not only the texture feature, but also the similarity measurement influence the accuracy of texture feature extraction. Two feature vectors from target images and reference images are as follows:

$$F_k=\left[F_{l1\_energy}\left({\alpha }_k\right),F_{sd}\left({\alpha }_k\right),F_{add}\left({\alpha }_k\right),F_{entropy}\left({\alpha }_k\right)\right]$$

(5)

$$F_{ref\_k}=\left[F_{l1\_en}\left({\alpha }_{ref\_k}\right),F_{sd}\left({\alpha }_{ref\_k}\right),F_{add}\left({\alpha }_{ref\_k}\right),F_{en}\left({\alpha }_{ref\_k}\right)\right]$$

(6)

where $F_k$ denotes the wavelet coefficient sub-blocks of the target image and $F_{ref\_k}$ denotes the wavelet coefficient sub-blocks of the reference image. As for the relationship of the $kth$ sub-block of in the reference image and target image, the Canberra distance is adopted to calculate their similarity:

$$w_k=Canb\left(F_k,F_{ref\_k}\right)=\sum _{h=1}^u\frac{\left|F_{kh}-F_{ref\_kh}\right|}{\left|F_{kh}\right|+\left|F_{ref\_kh}\right|}$$

(7)

In the formulation above, h represents the divergence of these two feature vectors, and the denominator normalizes the divergence. Therefore, the similarity measurement can avoid the scale effects, and the constrained item is formed as follows:

$$\begin{array}{c}\hfill \sum _{k=1}^U{w}_k{∥{\alpha }_k-{\alpha }_{ref\_k}∥}_2^2<\xi \end{array}$$

(8)

where ξ is a small constant, $U=M\times N/\left(m\times n\right)$ is the number of the sub-blocks of the wavelet coefficients and $w_k$ denotes the distance between the wavelet coefficient k-th block of the texture features in the target image and the reference image. More similar texture feature vectors correspond to blocks of wavelet coefficients that are more alike.

The reconstruction object formula of the compressive sensing with the reference information model can be presented as:

$$\alpha =argmin{∥\alpha ∥}_1,s.t\left\{\begin{array}{c}{∥y-\varphi \psi \alpha ∥}_2^2<\delta \hfill \\ {\displaystyle \sum _k^U}{w}_k{∥{\alpha }_k-{\alpha }_{ref\_k}∥}_F^2<\xi \hfill \end{array}\right.$$

(9)

where ${∥y-\varphi \psi \alpha ∥}_2^2$ is the ${\ell }_2$ data fidelity term and ${∥\alpha ∥}_1$ represents the regularization term denoting prior knowledge. There are many optimization methods to solve the above ${\ell }_1$ minimization problem, such as the iterative shrinkage algorithm [28], the Bregman split [29] and the alternative direction multiplier method [30]. However, recent advances have shown that replacing the l1 norm with the nonconvex surrogate function can obtain better CS recovery performance. For example, the methods proposed by Fazel [32], Gasso [33], Chartrand [34] and Trzasko [35] have proved that in certain situations, the nonconvex surrogate function is able to recover the sparsity coefficient more efficient.

3. Generalized Nonconvex Low-Rank Approximation Model

A nonconvex low-rank model for CS recovery exploits the nonlocal structured sparsity via low-rank approximation for image reconstruction. Under the assumption that each exemplar patch of $x_i\in C^n$ (size $\sqrt{n}\times \sqrt{n}$ at position i) is able to find plenty of similar patches in its neighborhood area, a large number of k-nearest-neighbor searches have been implemented for each exemplar patch in a local window, namely:

$$H_i=\left\{i_j∥x_i-x_{i_j}∥<T\right\}$$

(10)

where T is a pre-defined threshold value and $H_i$ denotes the assemblage of positions relating to the patches similar to $x_i$. Under the assumption that these image patches have similar structures, the data-formed data matrix $x_i$ has a low-rank property. A data matrix $X_i=\left[x_{i_0},x_{i_1},\dots ,x_{i_{m-1}}\right],X_i\in C^{n\times m}$ is acquired after the search, which is decomposed into $X_i=L_i+W_i$, where $L_i$ denotes the low-rank matrix and $W_i$ denotes the Gaussian noise. Then, the low-rank problem can be solved as:

$$L_i=\underset{L_i}{argmin}rank\left(L_i\right),s.t.{∥X_i-L_i∥}_F^2\le {\sigma }_{\omega }^2$$

(11)

where ${∥·∥}_F^2$ denotes the Frobenius norm and ${\sigma }_{\omega }^2$ denotes the variance of additive Gaussian noise. The rank minimization almost is an NP-hard problem; hence, we use the nuclear norm ${∥·∥}_{*}$ (sum of singular values) to replace a series of convex surrogate functions of the rank to obtain an approximated solution. Using the nuclear norm, the rank minimization problem can be efficiently solved by the technique of singular value thresholding (SVT).

In this paper, we consider a smooth, but non-convex surrogate of the rank rather than the nuclear norm. Specifically, according to [32], the rank minimization problem with regard to $L_i$ can be approximately solved by minimizing the following function:

$$L_i=\underset{L_i}{argmin}S\left(L_i,\epsilon \right),s.t.{∥X_i-L_i∥}_F^2\le {\sigma }_{\omega }^2$$

(12)

where ${∥•∥}_F^2$ denotes the Frobenius norm, ${\sigma }_{\omega }^2$ denotes the variance of additive Gaussian noise, $S\left(L_i,\epsilon \right)\text{=}Gdet\left({\left(L_i{L}_i^T\right)}^{1/2}+\epsilon I\right)\text{=}Gdet\left(U{\Sigma }^{1/2}{U}^{-1}+\epsilon I\right)\text{=}Gdet\left({\Sigma }^{1/2}+\epsilon I\right)$, ε is a small constant value, Σ denotes the eigenvalue matrix of $L_i{L_i}^T$, i.e., $L_i{L_i}^T=U\Sigma U^{-1}$, and ${\Sigma }^{1/2}$ is a diagonal matrix whose diagonal elements are the singular values of the matrix $L_i$. By taking a proper parameter λ, Equation (12) can be transformed into:

$$L_i=\underset{L_i}{argmin}{∥X_i-L_i∥}_F^2+\lambda S\left(L_i,\epsilon \right)$$

(13)

For each exemplar image patch, we can approximate the matrix $X_i$ with a low-rank matrix $L_i$ by solving Equation (13). Many nonconvex surrogate functions have been proposed to achieve a better approximation to the ${\ell }_0$ norm [36], including the logarithm function (Log) [37], the ${\ell }_p$ norm (Lp) [38], Geman (Geman) [39], Laplace (Lap) [35] and the exponential type penalty (Etp) [40]. In addition, there are several discontinuous functions, such as the smoothly-clipped absolute deviation (Scad) [41], Capped ${\ell }_1$ (Cappedl1) [42] and the minimax concave penalty (Mcp) [43]. The definitions and super-gradients of these surrogate functions have similar monotonous trends, as displayed in Table 1 and Figure 1.

Table 1. Generalized nonconvex surrogate functions.

Nonconvex Surrogate Functions	Formulation ${\mathit{G}}_{\mathit{\theta }}\mathbf{\left(}\mathit{x}\mathbf{\right)}\mathbf{,}\mathit{x}\mathbf{\ge }\mathbf{0}\mathbf{,}\mathit{\theta }\mathbf{>}\mathbf{0}$	Super-Gradient $\mathbf{\nabla }{\mathit{G}}_{\mathit{\theta }}\mathbf{\left(}\mathit{x}\mathbf{\right)}$
Log	$\frac{\theta }{log\left(\gamma +1\right)}log\left(\gamma x+1\right)$	$\frac{\gamma \theta }{\left(\gamma x+1\right)log\left(\gamma +1\right)}$
Lp	$\theta x^p,0<p<1$	$\left\{\begin{array}{c}\infty ,x=0,\hfill \\ \theta p{x}^{p-1},x>0.\hfill \end{array}\right.$
Geman	$\frac{\theta x}{x+\gamma }$	$\frac{\theta \gamma }{{\left(x+\gamma \right)}^2}$
Lap	$\theta \left(1-exp\left(-\frac{x}{\gamma }\right)\right)$	$\frac{\theta }{\gamma }exp\left(-\frac{x}{\gamma }\right)$
Etp	$\frac{\theta }{1-exp(-\gamma )}\left(1-exp\left(-\gamma x\right)\right)$	$\frac{\theta \gamma }{1-exp(-\gamma )}exp\left(-\gamma x\right)$
Scad	$\left\{\begin{array}{c}\theta x,x<\theta ,\hfill \\ \frac{-x^2+2\gamma \theta x-{\theta }^2}{2\left(\gamma -1\right)}\theta ,x\le \gamma \theta ,\hfill \\ \frac{{\theta }^2\left(\gamma +1\right)}{2},x>\gamma \theta .\hfill \end{array}\right.$	$\left\{\begin{array}{c}\theta ,\mathrm{if}x<\theta ,\hfill \\ \frac{\gamma \theta -x}{\gamma -1},\lambda <x\le \gamma \theta ,\hfill \\ 0,\mathrm{if}x>\gamma \theta .\hfill \end{array}\right.$
Cappedl1	$\left\{\begin{array}{c}\theta x,x<\gamma ,\hfill \\ \theta \gamma ,x\ge \gamma .\hfill \end{array}\right.$	$\left\{\begin{array}{c}\theta ,\mathrm{if}x<\gamma ,\hfill \\ \left[0,\theta \right],\mathrm{if}x=\gamma ,\hfill \\ 0,\mathrm{if}x>\gamma .\hfill \end{array}\right.$
Mcp	$\left\{\begin{array}{c}\theta x-\frac{x^2}{2\gamma },\mathrm{if}x<\gamma \theta ,\hfill \\ \frac{1}{2}\gamma {\theta }^2,\mathrm{if}x\ge \gamma \theta .\hfill \end{array}\right.$	$\left\{\begin{array}{c}\theta -\frac{x}{\gamma },\mathrm{if}x<\gamma \theta ,\hfill \\ 0,\mathrm{if}x\ge \gamma \lambda .\hfill \end{array}\right.$

Figure 1. Some popular nonconvex surrogate functions of the ${\ell }_1$ norm (left) and their super-gradients (right). (a) Log penalty; (b) ${\ell }_p$ norm (Lp) penalty; (c) Geman penalty; (d) Laplace (Lap) penalty; (e) Exponential type penalty (Etp) penalty; (f) Smoothly-clipped absolute deviation (Scad) penalty; (g) Capped ${\ell }_1$ (Cappedl1) penalty; (h) Minimax concave penalty (Mcp) penalty.

4. Problem Definition

In multispectral remote sensing applications, the target images’ and the reference images’ intensity distributions are different, but their edge directions and texture information are often similar. One of the reference images can be used as a prior image in the reconstruction process. This section gives the generalized nonconvex low-rank approximation algorithm for CS recovery exploiting the nonlocal structured sparsity via low-rank approximation for remote sensing image reconstruction. Let $x\in C^N$ denote the target images, where x is sparsely expressed as $x=\psi \alpha$ with a sparse signal α, and $y\in C^M$ denotes the observed data; the measurement matrix $\varphi \in C^{M\times N}(M<N)$ maps x to y. The remote sensing image reconstruction problem is to reconstruct x. Figure 2 shows the relationship between high resolution target images and low resolution observed images. The different low and high resolution images contain different time series or different bands. In some cases, we use one or two different time series or different bands as the reference image and the other as the unknown target image.

Figure 2. Relationship between the high resolution image and the low resolution image.

In this section, a compressed sensing model with reference images integrating low-rank regularization and wavelet textural constraints is proposed and solved with the conjugate gradient algorithm and single value threshold (SVT). Figure 3 shows the framework of the proposed GNLR-RI method. The reconstruction object formula of the generalized nonconvex low-rank approximation with reference information model can be presented as:

$$\left(\widehat{\alpha },{\widehat{L}}_i\right)=\underset{\alpha ,L_i}{argmin}\eta \sum _i\left\{{∥{\tilde{R}}_i\psi \alpha -L_i∥}_F^2+\lambda S\left(L_i,\epsilon \right)\right\},s.t.\left\{\begin{array}{c}{∥y-\varphi \psi \alpha ∥}_2^2<\delta \hfill \\ {\displaystyle \sum _j^U}{w}_j{∥{\alpha }_j-{\alpha }_{ref\_j}∥}_F^2<\xi \hfill \end{array}\right.$$

(14)

where ${\tilde{R}}_{i}x=\left[R_{i_0}x,R_{i_1}x,\dots ,R_{i_{m-1}}x\right]=X_i$ denotes the patch matrix similar to the patch $x_i$. The variable R denotes the location of the similarity patches; ${\displaystyle \sum _i}{R}_i\psi \alpha$ represents the patch average result. The minimization of the objective constraint function turns into non-constraint optimization.

$$\begin{array}{c}\left(\widehat{\alpha },{\widehat{L}}_i\right)=\underset{\alpha ,L_i}{argmin}{∥y-\varphi \psi \alpha ∥}_2^2+{\displaystyle \sum _j^U}{w}_j{∥{\alpha }_j-{\alpha }_{ref\_j}∥}_F^2+\eta {\displaystyle \sum _i}\left\{{∥{\tilde{R}}_i\psi \alpha -L_i∥}_F^2+\lambda S\left(L_i,\epsilon \right)\right\}\hfill \end{array}$$

(15)

Figure 3. Framework of the proposed GNLR-RI method.

4.1. Solving the Proposed Model

As it is difficult to get a closed solution from Equation (15) directly, the optimization process is divided into three steps.

$${\alpha }^{k+1}=\underset{\alpha }{argmin}{∥y-\varphi \psi \alpha ∥}_2^2+\sum _j^U{w}_j{∥{\alpha }_j-{\alpha }_{ref\_j}∥}_F^2$$

(16)

$${\widehat{L}}_i=\underset{L_i}{argmin}\sum _i\left\{{∥{\tilde{R}}_i\psi \alpha -L_i∥}_F^2+\lambda S\left(L_i,\epsilon \right)\right\}$$

(17)

$${\alpha }^{k+1}=\underset{\alpha ,L_i}{argmin}{∥y-\varphi \psi \alpha ∥}_2^2+\eta \sum _i{∥{\tilde{R}}_i\psi \alpha -L_i∥}_F^2$$

(18)

For the first step, the reference image wavelet information is ${\ell }_2$ norm regularization; thus, the conjugate gradient algorithm is used to solve it. By setting the derivatives of the objective formula in Equation (16) with respect to the sparse coefficient α to zero, we can obtain:

$$-2{\psi }^T{\varphi }^T\left(y-\varphi \psi {\alpha }^{k+1}\right)+2\sum _j^U{w}_j\left({\alpha }_j-{\alpha }_{ref\_j}\right)=0$$

(19)

$${\psi }^T{\varphi }^T\varphi \psi {\alpha }^{k+1}={\psi }^T{\varphi }^{T}y+\sum _j^U{w}_j\left({\alpha }_j-{\alpha }_{ref\_j}\right)$$

(20)

and where $\alpha ={\displaystyle \sum _j^U}{w}_j{\alpha }_j$, we can employ the conjugate gradient algorithm to solve Equation (20).

For the second step, the solution of $L_i$ can be obtained by solving the following minimization problem:

$$L_i=\underset{L_i}{argmin}\eta {∥{\tilde{R}}_{i}x-L_i∥}_F^2+\lambda S\left(L_i,\epsilon \right)$$

(21)

where ${\tilde{R}}_{i}x=\left[R_{i_0}x,R_{i_1}x,\dots ,R_{i_{m-1}}x\right]=X_i$ denotes the patch matrix similar to the patch $x_i$. By substituting Equation (11) into Equation (21), $L_i$ can be rewritten as:

$$\underset{L_i}{min}{∥X_i-L_i∥}_F^2+\frac{\lambda }{\eta }\prod _{j=1}^{n_0}G\left({\sigma }_j\left(L_i\right)+\epsilon \right)$$

(22)

where ${\sigma }_j$ is the j-th singular value of $L_i$. Let $f\left(\sigma \right)={\displaystyle \prod _{j=1}^{n_0}}G({\sigma }_j+\epsilon )$, which can be solved by using a local minimization method. With the first order Taylor expansion, $f\left(\sigma \right)$ can be approximated as:

$$f\left(\sigma \right)=f\left({\sigma }^{\left(k\right)}\right)+⟨\nabla f\left({\sigma }^{\left(k\right)}\right),\sigma -{\sigma }^{\left(k\right)}⟩$$

(23)

where ${\sigma }^{\left(k\right)}$ denotes the value of σ in the k-th iteration. This can be worked out by solving the following equation iteratively,

$$\begin{array}{c}{L}_i^{\left(k+1\right)}=\underset{L_i}{argmin}{∥X_i-L_i∥}_F^2+\frac{\lambda }{\eta }{\displaystyle \sum _{l=1}^{n_0}}\left(\frac{{\displaystyle \prod _{j=1}^{n_0}}\left({\sigma }_j^{\left(k\right)}+\epsilon \right)}{{\sigma }_l^{\left(k\right)}+\epsilon }\nabla G\left({\displaystyle \prod _{j=1}^{n_0}}\left({\sigma }_j^{\left(k\right)}+\epsilon \right)\right){\sigma }_l\right)\hfill \end{array}$$

(24)

After the constants in Equation (24) are ignored, Equation (22) is rewritten into:

$$L_i^{\left(k+1\right)}=\underset{L_i}{argmin}\frac{1}{2}{∥X_i-L_i∥}_F^2+\tau \mathrm{\Psi }\left(L_i,{\omega }^{\left(k\right)}\right)$$

(25)

where $\tau =\lambda /2\eta$ and $\mathrm{\Psi }\left(L_i,{\omega }^{\left(k\right)}\right)={\sum }_{l=1}^{n0}{\sigma }_l{\omega }_l^{\left(k\right)}$. ${\omega }_l^{\left(k\right)}=\nabla G\left({\displaystyle \prod _{j=1}^{n_0}}\left({\sigma }_j^{\left(k\right)}+\epsilon \right)\right){\displaystyle \prod _{j=1}^{n_0}}\left({\sigma }_j^{\left(k\right)}+\epsilon \right)/\left({\sigma }_l^{\left(k\right)}+\epsilon \right)$ denotes the weighted nuclear norm. According to the proximal operator of the weighted nuclear norm, the solution in the $(k+1)th$ iteration can be obtained as:

$$L_i^{\left(k+1\right)}=U{\left(\tilde{\Sigma }-\tau diag\left({\omega }^{\left(k\right)}\right)\right)}_{+}{V}^T$$

(26)

where $U\tilde{\Sigma }{V}^T$ denotes the SVT of $X_i$, ${\left(x\right)}_{+}=max\left\{x,0\right\}$.

For the third step, after the solution of all $L_i$ is obtained, the following minimization problem can be solved to reconstruct the wavelet coefficient matrix by using the conjunction gradient algorithm (CG):

$$-2{\psi }^T{\varphi }^T\left(y-\varphi \psi {\alpha }^{k+1}\right)+2\eta \sum _i{\psi }^T{\tilde{R}}_i^T\left({\tilde{R}}_i\psi {\alpha }^{k+1}-L_i\right)=0$$

(27)

By setting the derivatives of the objective formula in Equation (27) with respect to the sparse coefficient α to zero, we can obtain:

$$\left({\psi }^T{\varphi }^T\varphi \psi +\sum _i{\psi }^T{\tilde{R}}_i^T{\tilde{R}}_i\psi \right){\alpha }^{k+1}={\psi }^T{\varphi }^{T}y+\sum _i{\psi }^T{\tilde{R}}_i^T{L}_i$$

(28)

Noting that the value of ${\displaystyle \sum _i}{\tilde{R}}_i^T{\tilde{R}}_i$ represents the number of overlapping patches and ${\displaystyle \sum _i}{\tilde{R}}_i{L}_i$ represents the patch average result, we can employ the conjugate gradient algorithm to solve Equation (28).

Now, this image reconstruction algorithm is summarized here, is called GNLR-RI and represents the eight nonconvex surrogate functions mentioned above. The detailed description of the proposed method is listed in Algorithm 1. The difference between the NLR-CS-baseline and the GNLR-RI mainly lies in the approximation scheme of the ${\ell }_1$ norm and the representation of texture features. Therefore, during the reconstruction process, the low-rank operation strongly encourages the similarity of the patches between the target image and the reference image.

Algorithm 1: GNLR-RI.

Input: Sparse coefficient of reference image ${\alpha }_{ref}$

Initialization: Set target wavelet coefficient α to zero; ${\omega }_i={\left[1,1,\dots ,1\right]}^T,{\Lambda }^{\left(1\right)},{\mu }^{\left(1\right)}=0\lambda ,\eta ,p,\tau =\lambda /\phantom{\lambda 2\eta }2\eta ,\beta ,K,J$ and α

While convergence criterion not met, do

1. Compute the texture feature vectors F, $F_{ref}$ of target wavelet coefficient α and reference wavelet coefficient ${\alpha }_{ref}$ using Equations (5) and (6).

2. Add the constrained term ${\displaystyle \sum _{k=1}^U}{w}_k{∥{\alpha }_k-{\alpha }_{ref\_k}∥}_2^2$ to the compressed sensing target objective function.

3. Add the low-rank approximation based on the reference image term.

4. Solve the optimization problem Equation (20) via the conjunction gradient algorithm.

5. Iteration: For $\left(k>K_0\right)$, do

1) Form a matrix $X_i$ making up similar patches of $x^{\left(k\right)}$ and set $L_i^{\left(0\right)}=X_i$

2) For $j=1,2,\dots ,J$, do

a) If $\left(k>K_0\right)$, update the weights $${\omega }_l^{\left(k\right)}=\nabla G\left(\prod _{j=1}^{n_0}\left({\sigma }_j^{\left(k\right)}+\epsilon \right)\right)\prod _{j=1}^{n_0}\left({\sigma }_j^{\left(k\right)}+\epsilon \right)/\phantom{\prod _{j=1}^{n_0}\left({\sigma }_j^{\left(k\right)}+\epsilon \right)\left({\sigma }_l^{\left(k\right)}+\epsilon \right)}\left({\sigma }_l^{\left(k\right)}+\epsilon \right)$$

b) Compute $L_i$ via Equation (26) and output ${\alpha }_i={\alpha }_i^{\left(j\right)}$ when $j=J$.

End for

End for

6. Solve the optimization problem Equation (28) via the conjunction gradient algorithm.

Output: $\widehat{X}=X_{.}^{\left(K\right)}$

5. Experiment Results

During the experiments, single-channel and multichannel satellite images from MODIS, Landsat 7, Landsat 8 and Google Earth are used as the simulated data to test the performance of our proposed reconstruction framework. (1) Landsat 7 and Landsat 8 provide PAN images at 15-m spatial resolution and 30-m spatial resolution. (2) MODIS provides images at 500-m resolution. (3) Google Earth provides three-channel color tested images. For the Landsat 7 and Landsat 8 image data, Band 4 with 30-m resolution is the near-infrared band with a better spectrum characteristic in which a body of water has a clear outline, and Band 8 is panchromatic with 15-m resolution; thus, Band 4 is chosen as the target image, and Band 8 is chosen as the reference image, which is down-sampled into 30-m spatial resolution to group similar blocks injected into the reconstructed image to compensate the smoothed areas nonlinearly. For the MODIS data, we use Band 4 as the target image and Band 3 as the reference image to evaluate the proposed algorithm.

Furthermore, our proposed algorithm is compared to the well-known reconstruction-based models NLR-CS-baseline [7], orthogonal matching pursuit (OMP) and compressive sampling matching pursuit (CoSaMP) [24] by evaluating the experimental results quantitatively and visually. NLR-CS-baseline uses the standard nuclear norm, which is the $l_1$ norm minimization. CoSaMp and OMP reconstructs signal by using a very small number of points.

Furthermore, our proposed algorithm is compared to the well-known reconstruction-based models NLR-CS-baseline [7], orthogonal matching pursuit (OMP), compressive sampling matching pursuit (CoSaMP) [27], multi-hypothesis predictions combined with block-based compressed sensing with smoothed projected Landweber reconstruction (MH-BCS-SPL) [44], recovery via collaborative sparsity (RCoS) [45] and adaptively-learned sparsifying basis via L0 minimization (ALSB) [46] by evaluating the experimental results quantitatively and visually. NLR-CS-baseline uses the standard nuclear norm, which is the L1 norm minimization. CoSaMp and OMP reconstruct the signal by using a very small number of points. The multi-hypothesis prediction is used to generate a residual in the domain of the compressed-sensing random projections in MH-BCS-SPL. RcoS compels local 2D sparsity and nonlocal 3D sparsity simultaneously in an adaptive hybrid space-transform domain to utilize the intrinsic sparsity of natural images and to confine the CS solution space. ALSB enforces the intrinsic sparsity of natural images substantially by sparsely representing overlapped image patches using the adaptively-learned sparsifying basis in the form of the L0 norm.

5.1. Evaluation of the Low Rank Penalty Function and Different Nonconvex Surrogate Functions

First, we justify the necessity of adding the low-rank approximation method. For convenience, the objective function Equation (15) recovering the $L1$ norm without reference information is denoted as L1-WRI. The reconstruction method introduced the reference information constraint with $L1$ norm Equation (9) denoted as L1-RI. The objective function Equation (15) that employed the nonconvex surrogate logarithm function is denoted as GNLR-RI-Log. Figure 4 demonstrates the results of these three methods. The proposed algorithm that combines these and introduces low-rank approximation provides better performance.

Figure 4. The reconstruction image of three methods. (a) L1-without reference information (WRI); (b) L1-RI; (c) GNLR-RI-Log.

Second, in this section, the reference images of Landsat 7 and Landsat 8 are down-sampled to the same scale of the target images, and the MODIS images are the same scale with 256 × 256, where 256 is the length and width of the input image. Common indices, such as the peak signal to noise ratio (PSNR) and root mean square error (RMSE) are adopted to give a quantitative assessment. Table 2 shows PSNR (dB)/RMSE with different generalized nonconvex surrogate functions. The reconstructed images with different nonconvex functions are displayed in Figure 5. It can be observed that continuous nonconvex surrogate functions, such as Log, Lp, Geman, Lap and Etp, have the same performance as piecewise nonconvex surrogate functions, such as Scad, Cappedl1 and Mcp.

Table 2. PSNR (dB)/RMSE with different generalized nonconvex surrogate functions.

Functions	Landsat 8	MODIS	Landsat 7
Log	32.655/3.263	27.843/3.274	25.616/11.477
Lp	32.804/5.463	26.068/3.266	25.569/11.259
Geman	32.655/5.434	26.194/3.253	25.964/10.922
Lap	32.714/5.387	27.193/3.312	25.580/10.843
Etp	32.803/5.466	27.310/3.282	25.681/11.499
Scad	32.667/5.435	26.223/3.293	25.958/11.319
Cappedl1	32.695/5.478	27.081/3.305	25.757/11.271
Mcp	32.752/5.499	26.251/3.268	25.863/11.018

Figure 5. Reconstructed images with different nonconvex surrogate functions. (a) GNLR-Log; (b) GNLR-Lp; (c) GNLR-Geman; (d) GNLR-Lap; (e) GNLR-Etp; (f) GNLR-Scad; (g) GNLR-Cappedl1; (h) GNLR-Mcp.

In this subsection, we focus on usual nonconvex penalties proposed for recovering sparsity in Figure 1. As all of the penalty functions share common properties, concave and monotonically increasing on $\left[0,\infty \right)$, thus their super-gradients are nonnegative and monotonically decreasing. Our proposed general solver is based on this key observation. One of the nonconvex penalties that circumvents Lasso weak points is the SCAD penalty, which has the unbiasedness properties. Among all other penalty functions that lead to sparsity, a popular one is the Lp pseudonorm when $0<p<1$. The main interest of this penalty resides in its quasi-smooth approximation of the L0 sparsity measure as p tends toward the null value. It can provide sparser solutions than Lasso. For the log penalty, we shift the coefficients by a small quantity to avoid an infinite value when the parameter vanishes.

5.2. Performance Comparison for Single-Channel Compressed Sensing

In this subsection, one of the nonconvex surrogate logarithm functions (GNLR-RI-Log) is selected to test the performance of the single-channel experiment. The correlation coefficient (CC) and structural similarity (SSIM) are added to give a quantitative assessment of the reconstruction results. Table 3 shows the performance indexes PSNR (dB), RMSE, CC and SSIM of the Log function with different images and patch sizes. Figure 6 demonstrates image recovery quality intuitively of different images. It can be observed that the proposed method, GNLR-RI-Log, can reconstruct the images more precisely and obtain good group similarity by extracting structural information from the reference image. Obviously, the results illustrate that we should select an appropriate patch size in the procedure of processed images to derive optimal reconstruction quality.

Table 3. The performance indexes PSNR (dB), RMSE, correlation coefficient (CC) and structural similarity (SSIM) of the Log function with different images and patch sizes.

Images	Patch Sizes	PSNR	RMSE	CC	SSIM
(1)	2 × 2	24.006	17.930	0.958	0.898
(1)	4 × 4	24.128	17.780	0.958	0.897
(1)	6 × 6	24.212	17.777	0.958	0.896
(1)	8 × 8	24.287	17.937	0.957	0.896
(2)	2 × 2	30.469	8.908	0.980	0.844
(2)	4 × 4	30.615	8.709	0.980	0.832
(2)	6 × 6	30.552	8.816	0.979	0.828
(2)	8 × 8	30.806	8.823	0.979	0.832
(3)	2 × 2	24.404	11.264	0.925	0.802
(3)	4 × 4	24.485	11.470	0.921	0.782
(3)	6 × 6	24.936	11.295	0.922	0.790
(3)	8 × 8	23.919	11.822	0.914	0.773
(4)	2 × 2	24.128	17.780	0.958	0.897
(4)	4 × 4	24.212	17.777	0.958	0.896
(4)	6 × 6	24.287	17.937	0.957	0.896
(4)	8 × 8	30.469	8.908	0.980	0.844

Figure 6. The reconstruction image of Landsat 7 and Landsat 8. (a–d) Reconstruction Image (1–4). (a) Reconstruction Image 1; (b) Reconstruction Image 2; (c) Reconstruction Image 3; (d) Reconstruction Image 4.

5.3. Multichannel Reconstruction with References

For the multichannel, Bands 4/5/6 of Landsat 8 data and Bands 1/2/3 of Google Earth data are set as simulated images to be reconstructed in the multichannel reconstruction subsection. Reference images are derived from related Band 8 of Landsat 8 and related gray one of Google Earth. Commonly-used metrics for multichannel images, such as spectral angle mapper (SAM), RASE, relative dimensionless global error (ERGAS) and Q4, are evaluated to check the availability of quantitative remote sensing, since PSNR is not the only index that reflects the reconstruction effect. Note that the ideal result is one for Q4, while it is zero for SAM, RASE and ERGAS. Table 4 shows that GNLR-RI-Log has more advantages with Google Earth data than Landsat 8 data. It can be observed that for Landsat 8 data, our method GNLR-RI-Log works better than NLR-CS-baseline, OMP, CoSaMp and RCoS in terms of PSNR, while it performs worse than MH-BCS-SPL and ALSB. Except ERGAS, for the other three indexes, SAM, RASM and Q4, GNLR-RI-Log obtains comparable and better results than most of the competing methods. On the other hand, for the Google Earth data, GNLR-RI-Log produces more moderate results and performs best among the competing methods in terms of PSNR, SAM, RASM, ERGAS and Q4.

Table 4. Reconstruction evaluation of Landsat 8 Bands 4/5/6 with Band 8 as the reference and Google Earth Bands 1/2/3 with the gray one as the reference. SAM, spectral angle mapper; ERGAS, relative dimensionless global error; OMP, orthogonal matching pursuit; CoSaMP, compressive sampling matching pursuit; CS, compressed sensing; MH-BCS-SPL, multi-hypothesis predictions combined with block-based compressed sensing with smoothed projected Landweber reconstruction; RCoS, recovery via collaborative sparsity; ALSB, adaptively-learned sparsifying basis.

	Landsat 8 Bands 4/5/6					Google Earth Bands 1/2/3
	PSNR	SAM	RASM	ERGAS	Q4	PSNR	SAM	RASM	ERGAS	Q4
OMP	17.644	0.092	0.162	0.041	0.986	19.741	0.018	0.059	0.015	0.944
CoSaMP	17.327	0.074	0.146	0.037	0.989	19.462	0.017	0.054	0.014	0.954
NLR-CS-baseline	17.294	0.195	0.434	0.109	0.865	30.055	0.010	0.062	0.016	0.930
MH-BCS-SPL	27.568	0.162	0.165	0.041	0.985	31.963	0.010	0.071	0.018	0.917
RCoS	18.412	0.140	0.511	0.128	0.820	23.495	0.025	0.219	0.054	0.607
ALSB	28.753	0.138	0.144	0.036	0.989	32.821	0.028	0.064	0.016	0.932
reference	band 8					gray
GNLR-RI-Log	22.459	0.023	0.223	0.762	0.951	41.967	0.009	0.018	0.005	0.995

5.4. Parameter Evaluation

Similar to the detail-preserving regularity scheme, this subsection evaluates the sensitivity of the proposed method to parameter settings by varying one parameter at a time while keeping the rest fixed at their nominal values. In the reference information constrained reconstruction algorithm, there are two free parameters, λ and η, in the reconstructed object formula Equation (17). Bands 4/8 of Landsat 8 and Bands 2/3 and 3/4 of MODIS data were tested in this subsection. PSNR, RMSE, CC and SSIM values versus the parameters λ and η are plotted in Figure 7. It is obvious that more fine-tuning of the parameters may lead to better results, but the results with the parameter settings are consistently promising. By manually changing the parameters λ and η, some experiments are used to analyzing their variation tendency in Table 5.

Figure 7. Results of the proposed method with different values of parameters λ and η tested in Bands 4/3 of MODIS. (a) PSNR versus λ and η; (b) RMSE versus λ and η; (c) CC versus λ and η; (d) SSIM versus λ and η.

Table 5. PSNR (dB) with different parameters in different images.

λ	η	Landsat 8		MODIS
		Band 4	Band 8	Band 2	Band 3
0.25	0.07	24.668	26.674	24.229	27.311
0.45	0.45	25.834	27.050	23.763	26.969
0.67	0.33	24.843	26.702	23.798	27.427
0.87	0.25	24.474	26.902	24.027	27.604

To make a fair comparison among the competing methods, we have carefully tuned their parameters to achieve the best performance. The parameters of the other competing methods are designed as follow: for OMP and CoSaMp, default parameters (if required as input arguments) are used; for the NLR-CS-baseline algorithm, the main parameters are set as follows: patch size $6\times 6$ and similar patches $m=45$ are selected for each exemplar patch. For the MH-BCS-SPL method, we use an empirical regularization parameter value λ, and the initial search window is set to $w=1$. We have also carefully tuned the parameters of the RCoS and ALSB algorithms for the purpose of hopefully achieving the best possible performance.

5.5. Performances with Varied Noise Levels

In this subsection, we discuss the impact of noise on the reconstruction performance of the proposed algorithm with a 0.1 sampling rate, which means observation data in the compressive sensing. One of the nonconvex surrogate logarithm functions (GNLR-RI-Log) is selected to compare to the NLR-CS-baseline, CoSaMP and OMP algorithms. Bands 4/8 of Landsat 8 and Bands 3/4 of MODIS images were tested in the experiments with added Gaussian noise with a mean of zero and variance of (0,4,8,12). As can be seen in Table 6, the proposed GNLR-RI-Log method performs much better than the NLR-CS-baseline method in terms of noise added. Figure 8 shows the reconstruction results for a sample rate of 0.1 and a noise level of $\sigma =6$. It can be observed that our GNLR-RI-Log method can reconstruct the images more precisely than NLR-CS-baseline, CoSaMP and OMP, compensate for the over-smoothness and obtain good group similarities compared to the other algorithms by extracting structural information from reference images.

Table 6. The performance indexes PSNR (dB) with different reconstruction methods and noise levels. Landsat 8 Band 4 (reference: Band 8) and MODIS Band 6 (reference: Band 7).

	GNLR-RI-Log	NLR-CS-baseline	CoSaMP	OMP	GNLR-RI-Log	NLR-CS-baseline	CoSaMP	OMP
$\sigma =0$	32.863	23.502	28.754	28.021	31.225	22.025	24.803	24.274
$\sigma =4$	32.786	23.499	28.202	27.664	31.279	22.023	24.610	24.122
$\sigma =8$	32.615	23.496	26.798	26.879	30.994	22.022	23.974	23.717
$\sigma =12$	32.639	23.483	25.103	25.651	30.874	22.015	23.0007	23.086

Figure 8. The reconstruction results for a sample rate of 0.1 and a noise level of $\sigma =8$. (a–d) Reconstructing Landsat 8 images with GNLR-RI-Log, NLR-CS-baseline, CoSaMp and OMP; (e–h) Reconstructing MODIS images with GNLR-RI-Log, NLR-CS-baseline, CoSaMp and OMP. (a) GNLR-RI-Log; (b) NLR-CS-baseline; (c) CoSaMP; (d) OMP; (e) GNLR-RI-Log; (f) NLR-CS-baseline; (g) CoSaMP; (h) OMP.

5.6. Computational Complexity

A good remote sensing reconstruction model is expected to be not only effective, but also computationally efficient. The single-channel image is processed on an Intel(R) Core(TM) i7-6700 CPU @ 3.41 GHz with our MATLAB implementation (MATLAB 2015 with 64 bit). The computational cost performances of some representative methods are shown in Table 7, where we use a $256\times 256$ single-channel image as the input. As can be seen in Table 7 where we abbreviate iteration to iter., in terms of execution time, reconstruction with GNLR-RI-Log is, as expected, faster than RCoS and ALSB due to low-rank approximation. On the other hand, the execution times of RCoS and ALSB are much slower than GNLR-RI-Log and MH-MS-BCS-SPL due to RCoS and ALSB learning the adaptive sparsifying basis from a fraction of all patches. OMP and CoSamp are the fastest reconstruction methods with their runtimes of 0.106 and 0.031 s per iteration, respectively. In summary, GNLR-RI-Log achieves favorable performance in terms of both reconstruction accuracy and efficiency.

Table 7. Reconstruction time for a $256\times 256$ single-channel image.

Methods	OMP	CoSaMP	NLR-CS-baseline	MH-BCS-SPL	RCoS	ALSB	GNLR-RI-Log
Runtime (s)	0.106/iter.	0.031/iter.	3.56/iter.	1.39/iter.	21.08/iter.	17.11/iter.	0.22/iter.

6. Conclusions

This paper proposes a novel image reconstruction scheme that introduces wavelet coefficients of reference images as reference information based on compressed sensing and generalized nonconvex low-rank approximation. Nonlocal low-rank regularization enables us to exploit the similarity of patches and the nonconvexity of metric rank minimization. In addition, the single value threshold and conjugate gradient algorithms jointly offer a principled and computationally-efficient solution to image reconstruction. This approach has the following characteristics: (1) the wavelet coefficient acted as the texture feature constraint and extracted structural information from reference images to reconstruct remote sensing images; (2) high-resolution images can be achieved via simultaneously using compressed sensing and generalized nonlocal low-rank approximation. Some experiments are made to evaluate the proposed method, and the results show that the proposed method can achieve higher resolution than the state-of-the-art approaches. However, the proposed method has limited performance for the regions and bands where the spatial correlation is not high. In the future, we will further investigate it and use some change detection approaches to improve the performance of our algorithm.