A Sparse Analysis-Based Single Image Super-Resolution

Abstract

In the current study, we were inspired by sparse analysis signal representation theory to propose a novel single-image super-resolution method termed “sparse analysis-based super resolution” (SASR). This study presents and demonstrates mapping between low and high resolution images using a coupled sparse analysis operator learning method to reconstruct high resolution (HR) images. We further show that the proposed method selects more informative high and low resolution (LR) learning patches based on image texture complexity to train high and low resolution operators more efficiently. The coupled high and low resolution operators are used for high resolution image reconstruction at a low computational complexity cost. The experimental results for quantitative criteria peak signal to noise ratio (PSNR), root mean square error (RMSE), structural similarity index (SSIM) and elapsed time, human observation as a qualitative measure, and computational complexity verify the improvements offered by the proposed SASR algorithm.

1. Introduction

1.1. Background

Super resolution (SR) is a remarkable topic related to digital-image processing research. A hardware upgrade is the traditional solution to enhancing resolution, but is expensive in almost all cases. As a software-based technology, SR is an inexpensive and effective image processing technique. Many SR techniques have been proposed, including single-image super-resolution (SISR), which reconstructs a single low resolution (LR) image into a high resolution (HR) output image.

Conventional SISR techniques can be categorized into three classes. The first are interpolation-based techniques [1,2], which are the simplest solutions to the SISR problem. Bi-cubic and bilinear algorithms are the most well-known techniques in this category. Although their ringing and blocking effects are remarkable, simple structure and low complexity implementation are their main advantages. The second category comprises reconstruction-based techniques [3,4,5,6]. They deal with SR as an inverse problem. Their cost function has a general term for adapting HR and LR images and a regularization scheme to ensure better matching between the HR and LR images. Suresh et al. [6] used an adaptive edge detector as a regularization term in order to preserve edges and sharp image details. In a similar approach, Chen et al. [7] used a Tikhonov regularization term. Although most existing efforts in this category have worked well for small magnification factors, their performance decreases at larger magnification factors.

The third category of SISR algorithms is learning-based techniques [8,9,10,11,12,13]. These use a training dataset that includes HR–LR image pairs, and are based on a construction scheme for HR–LR databases called a dictionary. The extracted HR–LR image patches that have common features feed the dictionary. In HR image reconstruction, the LR patch closest to the input LR patch is found in the LR learned database. Its corresponding HR patch is then extracted and the resulting HR patches are merged to reconstruct the desired HR output image. These methods offer improved performance at a wide range of magnification factors and better quality in some image classes.

Training an effective HR–LR dictionary is a crucial stage in such algorithms. The sparse decomposition model has been used to propose SR algorithms based on dictionary learning techniques [10,11,12,13]. The sparse synthesis model assumes that each HR–LR image patch is composed of a few atoms from a learned HR–LR dictionary. The alternative sparse analysis model has been rarely examined for use in the SR field. In this study, we propose sparse analysis HR–LR operator learning and present a new approach to SISR.

1.2. Prior Work

Many SISR techniques have been proposed based on sparse coding and dictionary learning [10,11,12,13,14,15,16]. Yang et al. [11] proposed the first algorithm, which is known as SCSR. Their method was based on the sparse-synthesis model, which relies on the same sparse representation vector for each HR image patch and its corresponding LR patch in the HR–LR dictionary. HR–LR dictionary learning and HR image reconstruction were two main stages of these algorithms. In the first stage, the K-SVD dictionary learning method [17] has been modified for the HR–LR dictionary pair leaning by many authors. K-SVD is a dictionary learning algorithm based on K-means clustering and SVD matrix decomposition. It stands for K-means in addition to singular value decomposition [17].

Yang et al. proposed joint dictionary learning [12] and coupled dictionary learning [10] for the SCSR technique. Zeyde et al. [13] modified the SCSR algorithm to reduce the computational complexity and simplify HR dictionary learning using a single-image scale-up technique (SISU). Zhang et al. [14] proposed a dual dictionary-learning SR algorithm consisting of primary dictionary learning and residual dictionary learning to restore the main frequency detail and the residual frequency detail, respectively.

Chen et al. [7] took into account the differences in the structure of the patches and proposed the classified SR technique for smooth and non-smooth regions. They used bi-cubic interpolation for the smooth regions, and SR was implemented using the dictionary learning method for the non-smooth regions. Dong et al. [18] proposed the non-locally centralized sparse representation (NCSR) model to suppress the sparse coding noise in image restoration algorithms, such as SR [18]. Timofte et al. [19] combined sparse learned dictionaries with the neighbor embedding method and proposed the anchored neighborhood regression (ANR)method. Zhang et al. [20] proposed the adaptive mixed samples ridge regression (AMSRR) to effectively optimize learned dictionaries. Lu et al. [21] utilized the sparse domain selection method and achieved accurate and stable HR image recovery. Naderahmadian et al. modified the dictionary update step in online dictionary learning [22]. Other algorithms that have been proposed include classified dictionaries [23], sub-dictionary learning [24], and solo dictionary learning [25].

These SR algorithms are based on the assumption that each patch of the image can be restored as a linear combination of a few columns from a trained over-complete dictionary. Recently, a different model, sparse analysis, has been proposed [26] as a powerful alternative model for signal processing problems. Extensive research has focused on the high competency of the sparse analysis model [27,28,29]. Ning et al. [30] proposed an SR algorithm using iterative refinement and the analysis sparsity model in the Lab color space. Hawe et al. [31] proposed an algorithm for learning an analysis operator based on lp-norm minimization on the set of full rank matrices and implementation of the conjugate gradient method on manifolds. They considered the importance of suitable sparse analysis operators on output image quality. Nevertheless, it seems that the sparse analysis model has more potential for SR problems and requires further attention.

1.3. Contribution

Although synthesis-based sparse HR–LR dictionary learning methods have been studied extensively in SR algorithms, the analysis-based sparse model is an alternative model. We propose a coupled HR–LR sparse analysis operator that exploits mapping between HR–LR image patches. In addition to the use of patch-ordering total variation measures, we propose a novel sparse analysis operator learning method in the patch selection phase. In this way, the HR–LR patch pairs are trained based on image texture complexity, resulting in more-efficient HR–LR sparse analysis operators being trained.

We show that the proposed approach improves the quality of reconstructed HR images, as well as reducing the complexity of dictionary learning. Our experimental results show that the proposed method is comparable to state-of-the-art methods, both qualitatively and quantitatively. The main contributions of this paper are:

• a novel SR technique is proposed for mapping between HR–LR patches based on HR–LR sparse analysis operators;
• a new sparse operator learning method is proposed in the patch selection stage that considers image texture complexity;
• the computational complexity of the algorithm is less than in previous approaches;
• the proposed SASR results perform better in comparison with state-of-the-art and classic interpolation-based methods (bicubic SR, SCSR [12], SISU [13], ANR [19], GR [19], NE + LS [19] and GOAL [31], AMSRR [20], and SDS [21]).

1.4. Organization

The remainder of the paper is organized as follows. The proposed sparse analysis SR method is described in detail in Section 2. Section 3 presents experiments on the performance of the proposed SASR method in comparison with the bi-cubic SR, SCSR [12], SISU [13], ANR [19], GR [19], NE + LS [19], GOAL [31], AMSRR [20], and SDS [21] methods. The paper is concluded in Section 4.

2. Proposed Sparse Analysis-Based SR Algorithm (SASR)

This work aims at establishing a SR algorithm based on sparse analysis theory. We initially modify the sparse analysis operator learning algorithm by selection of an informative patch from learning images and then propose a novel SR algorithm based on coupled sparse analysis operators.

2.1. Sparse Analysis Model

In the conventional sparse synthesis model, image patch $x\in R^m$ is said to be “k-sparse” if it can be represented by a linear combination of k atoms from the over-complete dictionary $D^{m\times n}$ as

$$x=D{\alpha }_{s}s.t:{\Vert {\alpha }_s\Vert }_0^0=k.$$

(1)

where ${\propto }_s$ is the sparse synthesis representation of $x$.

In the sparse analysis model, as an alternative model to the sparse synthesis model, an image patch is said to be l-cosparse if it produces a sparse vector when it is analyzed with a sparse analysis operator ${\mathsf{\Omega }}^{n\times m}$:

$${\alpha }_A=\mathsf{\Omega }xs.t:{\Vert {\alpha }_A\Vert }_0^0=m-l.$$

(2)

While $D$ is square and invertible, the two models are equivalent. In contrast, for a long time, the over-complete case of $D$ and $\mathsf{\Omega }$ were confused and were even considered to be equivalent. Elad et al. [26] considered the algebraic similarity between the two models, and showed that there is a surprising difference between them for over-complete dictionaries and operators.

Based on the assumption that an image patch can be restored as a linear combination of atoms from an over-complete dictionary, many algorithms have been proposed that aim to solve inverse problems regularized with the sparse-synthesis model. Given the observation that $y=Mx$, where $M\in R^{m\times d}$ is a linear observation matrix, the optimization formulation of these two inverse problems can be represented as

$$\hat{x}=\underset{x}{\arg \min }={\Vert {\alpha }_s\Vert }_0^0{s.t:\Vert MD{\alpha }_s-y\Vert }_2^2\le \epsilon ,$$

(3)

and

$$\hat{x}=\underset{x}{\arg \min }={\Vert \mathsf{\Omega }x\Vert }_0^0{s.t:\Vert Mx-y\Vert }_2^2\le \epsilon .$$

(4)

Elad et al. [26] showed that if $\mathsf{\Omega }={\mathrm{D}}^{†}$ (the pseudo-inverse of $D$), the solutions of both the synthesis and analysis models are substantially different in the general over-complete case. In addition, they showed that the two formulations refer to very different models, and the analysis is much richer and may lead to better performance in various applications.

2.2. Image SR Using Coupled Sparse Analysis Operators

Sparsity has been used as an effective regulator for the ill-posed SISR problem. The core of traditional sparse representation-based SR is employment of the same sparse synthesis vector for corresponding HR and LR patches over the HR and LR learned dictionaries. Whereas sparse analysis has been successfully applied to many inverse problems in signal processing, we applied it to the SR problem. In our setting, we work with two coupled sparse analysis operators: ${\mathsf{\Omega }}_h$ for high resolution patches, and ${\mathsf{\Omega }}_l$ for low resolution ones. Because the basic assumption is the same sparse analysis vector for both HR and LR patches with respect to their analysis operators, the sparse analysis vector for LR patches is used directly to recover the corresponding high resolution patches for the HR analysis operator. Learning the HR–LR patch pair selection is the first step in the operator learning phase, and learning patch pair selection plays an important role in such methods. In this way, all patches extracted from each image are ordered based on structural similarity using a greedy algorithm. Next, the total variation measure is used to evaluate texture complexity and select a number of informative patches in each learning image. Figure 1 summarizes the proposed SASR algorithm.

2.2.1. Coupled Sparse Analysis Operator Learning

(a) Patch Selection Based on Ordering

Let Y be an image of size $N_1\times N_2$ and $\left\{y_k\right\}$ indicate the kth overlapping patch having size $\sqrt{n}\times \sqrt{n}$. In the first stage, we extract all possible patches from the image. Assuming that each patch is a point in space $R^n$, the goal is to reorder the patches so as to minimize the total variance (TV) measure:

$${\Vert \left\{y_k\right\}\Vert }_{TV}={\displaystyle \sum _{j=2}^{N_1\times N_2}\Vert y_j-y_{j-1}}{\Vert }_2^2.$$

(5)

The smoothness of the ordered patches ${\left\{y_k\right\}}^o$ can be measured using the TV. Minimizing the $\Vert {\left\{y_k\right\}}^o{\Vert }_{TV}$ results in the ordering of all extracted patches $\left\{y_k\right\}$ based on a structural similarity measure. We chose a simple approximate solution, which was to start from a random point and then continue from each patch $y_i$ to its nearest neighbor $y_j$ with a Euclidean distance similarity measure. TV minimization of patches using the proposed patch ordering is summarized in Algorithm 1.

Algorithm 1: Proposed patch ordering based on structural similarity

Task: Reorder the image patches $\left\{y_k\right\}$ Parameters: We are given $\sqrt{n}\times \sqrt{n}$ image patches $\left\{y_k\right\}$ and distance function ω. Let be the set of indices of all overlapping $\sqrt{n}\times \sqrt{n}$ patches extracted from the image. Initialization: Choose random index $i\in A$. Set $r=1$, $\mathsf{\Omega }\left(r\right)=\left\{i\right\}, \mathsf{\Lambda }\left(1\right)=\left\{i\right\}$. Main iteration: For $j\in A$ Find $y_j$ as the nearest neighbor to $y_i$ If $j\in A$ and $j\notin \mathsf{\Lambda }$ Set $\mathsf{\Omega }\left(r+1\right)=\left\{j\right\}.$ Otherwise: Find $y_j$ as the nearest neighbor to $y_i$ such that $\in A$ $\mathrm{and} j\notin \mathsf{\Lambda }$. Set $\mathsf{\Omega }\left(r+1\right)=\left\{j\right\}$. Output: Set Ω holds the proposed patch-based patch ordering.

We then calculate the total variation of $\left\{y_k\right\}$ and ${\left\{y_k\right\}}^o$ using Equation (5). After calculating the TV measure for each image in the database, we assign the number of patches for each image proportional to its TV to the total TV of images as

$$m_i=\frac{T{V}_i}{{\displaystyle \sum _{j=1}^{M}T{V}_j}}\times M,$$

(6)

where M is the total number of training patches and $m_i$ is the number of patches extracted from the ith image. After this step, we uniformly choose $m_i$ patches from each ordered image patch. Figure 2 shows examples of training images, and the results for the calculated TV measure, sampling step, and number of patches extracted for each image are shown in

Table 1. Total variance (TV) measure, sampling step, and number of extracted patches for sample images in Figure 2 (total number of training patches is 50,000).

Image	Total Variation	Sampling Step	Number of Extracted Patches
Penguin	940	13	5044
Boats	1248	10	6696
Old woman	1672	7	8974
Ladybird	1047	12	5621
Girls	4409	3	23,662

.

(b) Coupled Sparse Analysis Operator Learning

In this section, we introduce the proposed sparse analysis operator learning method, which trains the coupled ${\mathsf{\Omega }}_l$ and ${\mathsf{\Omega }}_h$ operators for the HR and LR patches. Sample set $K$ contains training image patch pairs $P={\left\{x_k,y_k\right\}}_{k=1}^K$, where $X={\left\{x_k\right\}}_{k=1}^K$ are the selected high resolution patches and $Y={\left\{y_k\right\}}_{k=1}^K$ are the corresponding low resolution patches. At this stage, the main goal is to learn coupled sparse analysis operators $\{{\mathsf{\Omega }}_h,{\mathsf{\Omega }}_l\}$, so that the sparse analysis vector of each high resolution patch is the same as the sparse analysis vector of the corresponding low resolution patch:

$${\alpha }_k={\mathsf{\Omega }}_h{x}_k\begin{array}{ccc}& \mathrm{and}& \end{array}{\alpha }_k={\mathsf{\Omega }}_l{y}_k.\begin{array}{cc}& \forall \left\{x_k,y_k\right\}\in P\end{array},k=1,\dots K.$$

(7)

The analysis K-SVD algorithm is applied to train the low resolution sparse analysis operator ${\mathsf{\Omega }}_l$. In this operator training algorithm, Robinstein et al. [25] supposed that given training set $Y={\left\{y_k\right\}}_{k=1}^K\in R^{n\times K}$ as K clean vectorized patches of the LR images, $Z={\left\{z_k\right\}}_{k=1}^K\in R^{n\times K}$ is a noisy version of the patches contaminated with additive zero-mean white Gaussian noise. They proposed the analysis K-SVD algorithm to find $\mathsf{\Omega }\in R^{h\times n},$ which forces the sparse analysis vector $\mathsf{\Omega }{y}_k$ to be sparse for each $y_i$. They formulated an optimization task for the learning process as:

$$f\left({\mathsf{\Omega }}_l,y_k\right)=\underset{{\mathsf{\Omega }}_l,z_k}{\min \frac{1}{2}}{\Vert {\mathsf{\Omega }}_l{y}_k\Vert }_0^0+\lambda {\Vert y_k-z_k\Vert }_2^2.$$

(8)

Considering the constraint that each HR patch and its corresponding LR patch has the same sparse analysis vector with respect to the HR and LR sparse analysis operators, the LR training patches are analyzed by ${\mathsf{\Omega }}_l$ to obtain the sparse analysis vectors by ${\propto }_k={\mathsf{\Omega }}_l{y}_k$. Therefore, for high resolution sparse analysis operator ${\mathsf{\Omega }}_h$, we should have:

$${\alpha }_k={\mathsf{\Omega }}_h{x}_k,$$

(9)

The solution of the problem is given by the following pseudo-inverse expression:

$${\mathsf{\Omega }}_h=A{X}^{†}=A{X}^T{\left(X{X}^T\right)}^{-1},$$

(10)

where matrix $X$ is constructed with the selected high resolution training patches as its columns, and A contains sparse analysis vector ${\propto }_k$ for paired low resolution patches over ${\mathsf{\Omega }}_l$ as its columns.

2.2.2. HR Image Reconstruction

Given LR image $Y\in R^{N_1\times N_2}$, we establish an SR sparse analysis method for reconstructing HR image $X\in R^{f{N}_1\times f{N}_2}$, where $f$ is the magnification scale. First, we extract all LR patches by raster scanning $Y=\left\{y_k\right\}\in R^{\sqrt{n}\times \sqrt{n}}$, where $n$ is the size of the LR patches. Because sparse analysis vectors for high and low resolution patch pairs must have the same corresponding operators, we have:

$${\alpha }_k\approx {\mathsf{\Omega }}_l{x}_k\approx {\mathsf{\Omega }}_h{y}_k.$$

(11)

In order to restore $x_k$ from $y_k$, the LR sparse analysis operator is first applied to the LR patch to obtain vector $\alpha$. Next, $\alpha$ is applied to the HR sparse analysis operator to obtain HR image patch $x_k$ using ${\mathsf{\Omega }}_h{}^{\mathfrak{t}}\alpha$. After recovering all the HR image patches, a complete HR image $X^{\prime }$ can be constructed by merging all these HR image patches by averaging the overlapping regions between adjacent patches. Because of the noise, and according to Equation (11), the reconstructed HR image may not exactly satisfy the observation model between the HR and LR images ($Y=MX$), where $M$ is a linear observation model for downsampling and blurring. In order to enhance the output HR image, the regression output is refined as follows:

$$X^{*}=\underset{X}{\arg \min }{\Vert Y-MX\Vert }^2+c{\Vert X-X^0\Vert }^2$$

(12)

where $X^0$ is the initial reconstructed HR image, the first term represents the reconstruction error and the second term recovers an image close to $X^0$. Using gradient descent, optimal solution $X^{*}$ can be obtained as the final recovered HR image [12] as

$$X_{t+1}=X_t+\tau \left\{M^T\left(Y-M{X}_t\right)+\left(X_t-X^0\right)\right\}.$$

(13)

where $X_t$ is the estimate of the high-resolution image after the t-th iteration and $\tau$ is the step size of the gradient descent.

3. Experimental Results

3.1. Test Setup

To evaluate the efficiency of the proposed SASR algorithm, it was compared with the following classic and state-of-the-art methods: bicubic interpolation, SCSR [12], SISU [13], ANR [19], GR [19], NE + LS [19] GOAL [31], AMSRR [20], and SDS [21]. All the source codes of the competing methods are publicly available on their authors’ webpages. For fair comparison, care was taken to ensure that the parameters used in the methods were similar. Because the first step in all learning-based SR algorithms is to select a dataset, all algorithms were tested on the same dataset. The HR and LR sparse analysis operators were trained using the dataset proposed by Yang et al. [12] (http://www.ifp.illinois.edu/~jyang29/). These HR images were down-sampled by bicubic interpolation to generate HR–LR image pairs for the training phase. To test the proposed algorithm, we took two standard datasets, SET14 (number of images = 14) and SET5 (number if images = 5), as testing images, which are two commonly used benchmark datasets in super-resolution literature. These images contain a nice mixture of details, flat regions, shading and texture. All testing was performed at a 3× magnification factor to validate the effectiveness of the proposed method. For patch selection in both the training and testing phases, our algorithm used 3 × 3 HR–LR patches with a two-pixel overlap.

All of the experiments were performed using MATLAB R2014a on an Intel (R) Core (TM) i5-M 460 @2.53 GHz machine with 4 GB of RAM.

The performance of the proposed algorithm was evaluated for root mean square error (RMSE), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and running time between the luminance channels of the original and restored images. PSNR is defined as

$$PSNR\left(X,\hat{X}\right)=10{\log }_{10}\frac{{255}^2}{MSE\left(X,\hat{X}\right)}$$

(14)

where X is the original HR image of size $M\times N$, $\widehat{X}$ is reconstructed HR image, and $MSE(X,\widehat{X}$) denotes the mean square error between $X \mathrm{and} \widehat{X}$, which is defined as

$$MSE(X,\hat{X})=\frac{1}{MN}{{\displaystyle \sum _{i=1}^M{\displaystyle \sum _{j=1}^N\left({\stackrel{}{X}}_{i,j}-{\hat{X}}_{i,j}\right)}}}^2$$

(15)

RMSE is the square root of the MSE of Equation (14). Although, these two criteria have been very often used by SR researchers, they do not represent the human visual system very well. Therefore, another form of measure, such as structural similarity measure (SSIM), has also been used in SR algorithms. SSIM measure is defined as:

$$SSIM=\frac{\left(2{\mu }_{\stackrel{}{X}}{\mu }_{\hat{X}}+C_1\right)\left(2{\sigma }_{X\hat{X}}+C_2\right)}{\left({\mu }_{\stackrel{}{X}}{}^2+{\mu }_{\hat{X}}{}^2+C_1\right)\left({\sigma }_X{}^2+{\sigma }_{\hat{X}}{}^2+C_2\right)}$$

(16)

where $C_1$ and $C_2$ are constants, ${\mu }_X$ and ${\mu }_{\widehat{X}}$ are the mean of $X$ and $\widehat{X}$, respectively, and ${\sigma }_X$ and ${\sigma }_{\widehat{X}}$ are the standard deviation of original and reconstructed images, respectively.

Because the human visual system is more sensitive to changes in the luminance channel, an LR color image first was transformed to YCbCr color space, then all SR methods were only applied to the luminance channel. The chromatic channels were reconstructed by bicubic interpolation, as is commonly performed in SR algorithms.

3.2. Results

We quantitatively and qualitatively evaluated the performance of the proposed method in comparison with bicubic interpolation, the SCSR [12], SISU [13], ANR [19], GR [19], NE + LS [19], GOAL [29], AMSRR [20], and SDS [21] methods. The sparse representation-based methods (SCSR [12] SISU [13]) and regression-based methods (ANR [19], GR [19], and NE + LS [19]) were used as for baseline methods. Three methods (AMSRR [20], SDS [21], and GOAL [29]) were used as for the state-of-the-art methods.

Table 2. PSNR (dB) of the reconstructed HR images with a magnification factor of m = 3.

	Lena	Barbara	Zebra	Butterfly	Cat	Sails	Coastguard	Parrot	Pool	Face	Average
Bicubic	29.80	26.91	21.56	24.58	23.33	25.72	26.79	30.69	33.62	33.88	27.68
SRSC [12]	30.87	27.40	23.13	25.48	23.77	26.38	26.80	30.94	34.85	34.31	28.39
SISU [13]	31.04	27.86	23.16	25.60	23.89	26.44	27.30	31.20	35.28	34.85	28.66
GR [19]	30.56	27.54	23.14	25.61	23.97	26.33	27.15	31.00	34.61	34.72	28.46
ANR [19]	31.11	27.83	23.28	25.77	23.93	26.52	27.23	31.20	35.32	34.94	28.71
NE + LS [19]	31.08	27.73	23.14	25.65	23.86	26.40	27.24	30.14	35.38	34.80	28.54
GOAL [29]	31.22	27.79	23.18	25.83	24.10	26.55	27.27	31.30	35.47	34.89	28.76
AMSRR [20]	31.47	27.85	23.27	25.82	24.12	26.56	27.11	30.91	35.39	34.65	28.71
SDS [21]	31.60	27.80	23.21	25.72	23.98	26.51	27.09	30.88	34.78	34.66	28.62
Proposed SASR	31.68	27.87	23.24	25.92	24.19	26.47	27.13	31.16	34.96	34.69	28.83

,

Table 3. RMSE of the reconstructed HR images with a magnification factor of m = 3.

	Lena	Barbara	Zebra	Butterfly	Cat	Sails	Coastguard	Parrot	Pool	Face	Average
Bicubic	8.24	11.49	19.98	15.04	17.37	13.19	11.66	7.44	5.31	5.15	11.48
SRSC [12]	7.28	10.86	17.77	13.55	16.51	12.23	11.66	7.23	4.61	4.90	10.66
SISU [13]	7.15	10.31	17.71	13.37	16.28	12.14	11.00	7.01	4.38	4.60	10.39
GR [19]	7.55	10.69	17.75	13.35	16.14	12.29	11.19	7.17	4.73	4.67	10.55
ANR [19]	7.09	10.34	17.46	13.11	16.21	12.02	11.08	7.87	4.36	4.56	10.41
NE + LS [19]	7.11	10.46	17.75	13.30	16.33	12.20	11.07	7.02	4.33	4.63	10.42
GOAL [29]	7.00	10.39	17.67	13.02	15.89	11.98	11.27	6.93	4.29	4.58	10.30
AMSRR [20]	6.95	10.59	17.51	13.00	15.86	12.20	11.07	7.27	4.32	4.70	10.34
SDS [21]	6.71	10.52	17.58	13.11	16.20	12.22	11.16	7.30	4.50	4.71	10.40
Proposed SASR	6.64	9.27	17.54	12.88	15.72	12.09	11.21	7.04	4.55	4.69	10.16

and

Table 4. SSIM of the reconstructed HR images with a magnification factor of m = 3.

	Lena	Barbara	Zebra	Butterfly	Cat	Sails	Coastguard	Parrot	Pool	Face	Average
Bicubic	0.8422	0.7737	0.6914	0.8412	0.7023	0.6603	0.6064	0.8612	0.9540	0.8477	0.7780
SRSC [12]	0.8581	0.7976	0.7426	0.8716	0.7491	0.7060	0.6194	0.8607	0.9597	0.8490	0.8014
SISU [13]	0.8699	0.8144	0.7501	0.8749	0.7499	0.7101	0.6422	0.8751	0.9655	0.8678	0.8120
GR [19]	0.8592	0.7964	0.7549	0.8761	0.7629	0.7109	0.6428	0.8726	0.9585	0.8689	0.8103
ANR [19]	0.8720	0.8124	0.7557	0.8793	0.7564	0.7160	0.6424	0.8790	0.9652	0.8706	0.8149
NE + LS [19]	0.8701	0.8087	0.7478	0.8748	0.7485	0.7160	0.6397	0.8761	0.9656	0.8665	0.8114
GOAL [29]	0.8745	0.8098	0.7510	0.8807	0.7627	0.7141	0.6342	0.8782	0.9651	0.8695	0.8139
AMSRR [20]	0.8701	0.8291	0.7546	0.8808	0.7659	0.7165	0.6427	0.8725	0.9656	0.8611	0.8159
SDS [21]	0.8698	0.8292	0.7541	0.8794	0.7564	0.7156	0.6423	0.8720	0.9556	0.8668	0.8141
Proposed SASR	0.8807	0.8296	0.7470	0.8743	0.7669	0.7093	0.6588	0.8731	0.9598	0.8660	0.8166

show the quantitative comparison for the PSNR, RMSE, and SSIM measures. These tables reveal that the proposed SASR method can possess better performance compared with some state-of-the-art methods, and gained average improvements of 0.07 dB in PSNR, 0.14 in RMSE, and 0.0007 in the SSIM criteria.

Figure 3 and Figure 4 show the effectiveness of the proposed method. Figure 3 and Figure 4 present the image super-resolution results of all methods for the “Butterfly” and “Face” images at 3× magnification. For more precise comparison, specific zoomed scenes are provided. We observed that the bicubic and GR [19] methods tended to generate blocking and ringing artifacts. Although the SCSR [12], SISU [13], ANR [19], NE + LS [19], GOAL [29], AMSRR [20], and SDS [21] methods improved the visual details and edges, it can be seen that the proposed SASR algorithm had more acceptable performance when reconstructing sharp edges, and showed more detail compared to the other algorithms.

Although the objective quality of a few HR images obtained by the proposed method had a slight advantage over the other methods, the average quantitative evaluations of the proposed method are competitive to those of the others.

According to

Table 5. Average elapsed time of the reconstructed HR images with a magnification factor of m = 3.

Benchmark	Proposed SASR	SCSR [12]	SISU [13]	GR [19]	ANR [19]	NE + LS [19]	AMSRR [20]	GOAL [29]	SDS [21]
Set 5	1.10	92.11	2.50	0.53	0.79	14.34	40.50	12.34	3.15
Set 14	1.19	89.29	2.55	0.48	0.71	15.93	38.70	12.93	3.06

, the elapsed time of our algorithm is comparable with GR [19] and ANR [19], while its performance was much faster than other compared algorithms. Although in terms of elapsed time our proposed algorithm was not the fastest method, it achieved satisfactory results. According to Table 5, in terms of elapsed time, the proposed method was faster than SCSR [12], SISU [13], NE + LS [19], and AMSRR [20], while it was comparable with GR [19].

To compare the computational cost of the proposed and SCSR [12] methods, an image patch size of $5\times 5$ was used for LR patches, and HR patches were obtained by implementing four feature extraction filters in order to extract high frequency details. Accordingly, after vectorization, LR patches with dimensions of $25\times 1$ and HR patches with dimensions of $100\times 1$ were obtained, leading to an LR dictionary with dimensions of $25\times 1024$ and an HR dictionary with dimensions of $100\times 1024$. In the SISU method proposed by Zeyde et al. [13], the HR patch dimension decreased from $100\times 1$ to $30\times 1$ after implementing a dimension reduction algorithm. The ANR [19], GR [19], and NE + LS [19] methods followed Zeyde et al. [13].

GOAL [29] is based on a sparse analysis model and uses a geometric conjugate gradient method on the oblique manifold to solve the optimization task. The performance of the analysis approach can be further increased by learning the particular operator with regard to the specific problem or employing a specialized training set [31]. However, in the proposed SASR method, selected training patches were obtained from the patches having a size of $3\times 3$, and they were ordered by the proposed simple algorithm, leading to LR and HR dictionaries with dimensions of $9\times 1024$. Therefore, the structure of the proposed SASR method was simpler in the learning phase. Moreover, because the patch extraction method should be the same at the learning and reconstruction phases, this difference can also be observed in the HR image reconstruction stage. In the reconstruction phase, other methods have used greedy algorithms, such as matching pursuit (MP) and orthogonal matching pursuit (OMP), to determine the sparse representation vectors for each low resolution image patch. In the proposed SASR method, sparse analysis vectors for LR image patches were obtained through two simple operations: calculation of the pseudo-inverse matrix and multiplication. Total evaluation for the quantitative criteria (PSNR, RMSE, SSIM), human observation as a qualitative measure, and elapsed time for computational complexity comparison verify the improvement of the proposed SASR algorithm.

4. Conclusions

We proposed a single-image super-resolution algorithm based on recent sparse analysis signal representation with coupled HR–LR sparse analysis operator learning. We showed that conventional sparse synthesis SR methods can be reformulated into sparse analysis operators. The proposed method, SASR, uses a new patch extraction method from the learning dataset with patch ordering and total variation measures. The proposed approach uses a new sparse matrix for analysis of the image, in contrast with other sparse super-resolution approaches, which are all synthesis-based approaches.

The experimental results show that the performance of the proposed algorithm is comparable to that of state-of the-art algorithms in terms of quantitative metrics and visual quality. Although in terms of elapsed time our proposed algorithm is not the fastest method, it achieves satisfactory results.

Although the performance improvement was revealed in the average results of empirical evaluation of our proposed method, there is a high potential for extending the theory of this approach in SR applications. In addition, the proposed method has the potential to be applied to de-noising and de-blurring problems. Further study will focus on optimizing the sparse analysis operator training process to improve reconstruction efficiency.