Weights-Based Image Demosaicking Using Posteriori Gradients and the Correlation of R–B Channels in High Frequency

Abstract

In this paper, we propose a weights-based image demosaicking algorithm which is based on the Bayer pattern color filter array (CFA). When reconstructing the missing G components, the proposed algorithm uses weights based on posteriori gradients to mitigate color artifacts and distortions. Furthermore, the proposed algorithm makes full use of the correlation of R–B channels in high frequency when interpolating R/B values at B/R positions. Experimental results show that the proposed algorithm is superior to previous similar algorithms in composite peak signal-to-noise ratio (CPSNR) and subjective visual effect. The biggest advantage of the proposed algorithm is the use of posteriori gradients and the correlation of R–B channels in high frequency.

1. Introduction

Simplifying processes and considering equipment costs, typical digital cameras usually use a single image sensor (e.g., CCD or CMOS) to capture the color image information. In a single-sensor digital camera with color filter array (CFA), only one of the red (R), blue (B), and green (G) elements can be sampled at each pixel location, while the rest elements need to be reconstructed by interpolation. The process of interpolation is called demosaicking [1], which is essential for digital cameras to generate the full-color images. Bayer pattern [2] CFA is the most common CFA, as shown in Figure 1. Half of the pixels in the Bayer pattern CFA are G pixels, for the reason that the G components contain more image details than the R and B components.

To generate the full-color images, certain original demosaicking algorithms have been proposed and applied to digital cameras. One of the original demosaicking algorithms was bilinear interpolation algorithm [3], which can process images at a high speed because of its low computational complexity. The algorithm in [3] performs well in smooth regions, but serious color artifacts appear when processing image edges.

To generate clearer full-color images, certain improved algorithms utilizing the correlation between adjacent pixels have been proposed in the literature [4,5,6,7]. More adjacent pixels are used compared to that of the work in [3]. Malvar et al. [4] used a linear filter to recover the missing components. The linear filter uses nine or eleven adjacent pixels. Luo et al. [5] lowered the computational complexity of the work in [4] by reducing the number of filter coefficients. Wang et al. [6] proposed an improved linear interpolation algorithm by correcting the bilinear interpolation [3] with correction values. Zhou et al. [7] further applied the correlation among the three primary color channels (R, G, B) on the basis of the bilinear interpolation algorithm [3]. However, the algorithms in [4,5,6,7] also cause color artifacts and distortions when processing image edges.

The algorithms in [3,4,5,6,7] work well only when the color differences are constant in small areas, but they all fail in edge regions where the color differences change rapidly. To further improve the quality of reconstructed images and mitigate the color artifacts and distortions in image edges, some other types of interpolation algorithms have been proposed and developed [8,9,10,11]. The algorithms in [8,9,10,11] first identify edge directions with edge indicators, and then the missing components are estimated along the edges rather than across the edges. Edge indicators are the key to the performance of image reconstruction. Adams and Hamilton [8] proposed an adaptive color plan (ACP) interpolation algorithm which applies chrominance information (R or B) and luminance information (G) to calculate the edge indicators. Based on the work in ACP [8], Lee et al. [9] used additional diagonal information to calculate the edge indicators more accurately. Menon et al. [10] proposed demosaicing with directional filtering (DWDF) and a posteriori decision. The algorithm in [10] determines the interpolation direction by making a hard direction decision. Furthermore, Menon et al. [10] adopted the posteriori idea when recovering the missing G components, which gave great inspiration to our work. Chen and Chang [11] proposed an improved edge detecting method for image demosaicking to mitigate the color artifacts and distortions.

To further mitigate the color artifacts and distortions in image edges, some demosaicking algorithms, which use multidirectional weights, have been proposed in the literature [12,13,14,15,16,17,18]. Zhang and Wu [12] proposed a weights-based algorithm using a directional linear minimum mean square-error estimation (DLMMSE) technique. Firstly, the missing components are estimated adaptively by DLMMSE in the horizontal and vertical directions, and then the directional estimations are combined with weights to generate the missing pixel values. Shi et al. [13] proposed a region-adaptive demosaicking (RAD) algorithm exploiting the weights of multidirectional information and the high execution speed of bilinear interpolation [3]. The input image is divided into two kinds of regions: edge and smooth. Different methods are applied to different regions. However, the way of calculating weights in [13] is inappropriate, which causes color artifacts and distortions. Chung and Chan [14] proposed using integrated gradients (IG) extracted directly from color differences ($G-R$ or $G-B$) to estimate the missing components. Pekkucuksen and Altunbasak [15] proposed a gradient-based threshold-free (GBTF) color filter array interpolation algorithm. The algorithm in [15] combines estimations from every direction without setting thresholds. Zhang et al. [16] proposed an effective image reconstruction scheme using local directional interpolation and nonlocal adaptive thresholding (LDI-NAT). The missing components are first estimated with multidirectional weights, and then the estimations are enhanced by a non-local adaptive threshold filter. Chen et al. [17] proposed an improved weights-based interpolation algorithm based on a voting strategy. Firstly, the interpolation directions are determined by the voting strategy, and then the missing components are estimated along the determined interpolation direction with a weights-based method. Wu et al. [18] proposed using directional predictors and classifiers based on polynomial interpolation to recover the missing components in mosaic images. The algorithm in [18] works well at the cost of increasing computational complexity.

In recent years, some interpolation algorithms have been proposed and greatly developed [19,20,21,22,23,24,25]. Kiku et al. [19] proposed a residual interpolation (RI) algorithm by introducing guide filter [20] into the GBTF. In [21], Kiku et al. applied the minimized Laplace energy to RI and proposed a minimized Laplacian residual interpolation (MLRI) algorithm. Yu et al. [22] further improved the MLRI by using horizontal and vertical weights. To design a high-performance algorithm, Wang and Jeon [23] combined multidirectional interpolation with guide filter. Monno et al. [24] proposed an adaptive residual interpolation (ARI) algorithm, which adaptively combines the two RI-based algorithms (RI and MLRI) and selects an appropriate iteration number at each pixel. The algorithm in [24] can achieve better performance in image reconstruction compared to the two original RI-based algorithms (RI and MLRI). Thomas and Farup [25] developed a diffusion-based demosaicing algorithm in different versions and tested it on different periodic and random mosaics. In particular, an ordinary least squares regression analysis was performed on test images to evaluate the effectiveness of the algorithm in [25], which inspired our work. The algorithms in [19,20,21,22,23,24,25] outperform the conventional color difference interpolation algorithms in [8,9,10,11,12,13,14,15,16,17,18].

The proposed algorithm, which adopts the posteriori idea and exploits the great correlation of R–B channels in high frequency, is based on the work in [8,16]. We introduce the posteriori idea into ACP [8] when recovering the missing G components. The correlation of R/B channels in high frequency is used for R/B interpolation at B/R positions. Compared with other algorithms in related literature [8,10,12,13,14,16], the proposed algorithm has better performance to reconstruct the full-color images.

Note that the presence of noise in data not only deteriorates the visual quality of captured images, but also often causes serious demosaicking artifacts [26]. In this paper, we do not consider the presence of noise, nor the texture classification [27]. In the case of noisy images, a denoising procedure [28] would be an unavoidable step.

The remainder of this paper is organized as follows. Section 2 starts with a brief review of ACP [8] and the process of R/B interpolation at B/R positions in LDI-NAT [16]. Section 3 introduces the proposed algorithm. Experimental results and performance analyses are provided in Section 4. Finally, conclusions and remarks on possible future work are given in Section 5.

2. Related Work

2.1. The Outline of G Interpolation in ACP

The process of G interpolation in ACP [8] is shown in Figure 2. Firstly, directional filters are used to estimate the G components at R/B positions in the horizontal and vertical directions. The directional filter applied is $[-0.25,0.5,0.5,0.5,-0.25]$, which is a finite impulse response (FIR) filter. In the Bayer-sampled image as shown in Figure 1, the G estimations in the horizontal and vertical directions, $G^{\mathrm{h}}$ and $G^{\mathrm{v}}$, are calculated as follows [8]:

$$G^{\mathrm{h}}(i,j)=\frac{1}{2}[G(i,j-1)+G(i,j+1)]+\frac{1}{4}[2R(i,j)-R(i,j-2)-R(i,j+2)],$$

(1)

$$G^{\mathrm{v}}(i,j)=\frac{1}{2}[G(i-1,j)+G(i+1,j)]+\frac{1}{4}[2R(i,j)-R(i-2,j)-R(i+2,j)],$$

(2)

where $i$ and $j$ indicate the row and the column of the positions, respectively. After the G values are estimated in the horizontal and vertical directions, respectively, the following two equations are used to calculate the gradients [8]:

$$D^{\mathrm{h}}(i,j)=\left|R(i,j-1)-R(i,j+1)\right|+\left|2G(i,j)-G(i,j-1)-G(i,j+1)\right|,$$

(3)

$$D^{\mathrm{v}}(i,j)=\left|R(i-1,j)-R(i+1,j)\right|+\left|2G(i,j)-G(i-1,j)-G(i+1,j)\right|,$$

(4)

where $D^{\mathrm{h}}$ and $D^{\mathrm{v}}$ indicate the gradients in the horizontal and vertical directions, respectively. Finally, select $G^{\mathrm{h}}$ or $G^{\mathrm{v}}$ as the estimated G value $G(i,j)$ by comparing the $D^{\mathrm{h}}$ and $D^{\mathrm{v}}$ values. The details are as follows [8]:

$$G(i,j)=\{\begin{array}{l}{G}^{\mathrm{v}}(i,j),\mathrm{if}{D}^{\mathrm{h}}(i,j)>D^{\mathrm{v}}(i,j)\\ G^{\mathrm{h}}(i,j),\mathrm{if}{D}^{\mathrm{h}}(i,j)\le D^{\mathrm{v}}(i,j)\end{array}.$$

(5)

2.2. The Process of R/B Interpolation at B/R Positions in LDI-NAT

Figure 3 shows the process of B interpolation at R positions in LDI-NAT [16]. The steps of interpolation are briefly described below. Note that all the G values have been recovered and are available. Firstly, the color differences are defined as $G-B$ in the northwest (NW), northeast (NE), southwest (SW), and southeast (SE) directions. Then, the gradients of color differences in the NW, NE, SW, and SE directions are calculated with the diagonal information of the three primary channels (R, G, B). Next, the reciprocals of gradients are defined as the weights in the NW, NE, SW, and SE directions, respectively. The final color difference estimations can be generated by combining the weights with the color differences. Finally, the G values are added to the final color difference estimations to recover the B components. The R interpolation at B positions is the same as above.

3. The Proposed Algorithm

3.1. The Outline of the Proposed Algorithm

Since the algorithm in ACP recovers the missing components along the edges rather than across the edges, it performs better than the bilinear algorithm [3]. However, it exists a problem that the process of interpolation in ACP is always carried out in one direction, horizontal or vertical, while the actual image edge direction is not necessarily horizontal or vertical, which will lead to color aliasing.

Combining the estimations of different directions with weights is a solution to the problem mentioned above. The effective combination of weights and directional estimations is the key. Therefore, it is essential for the interpolation algorithm to obtain weights effectively. However, we find that the gradients calculated by the method in ACP [8] cannot be used as weights directly, or it will cause serious color artifacts and distortions. To match the weights with the directional estimations, a novel weights-based demosaicking algorithm that applies the posterior idea and the great correlation of R–B channels in high frequency when calculating the gradients and weights at each pixel is proposed.

The outline of the proposed algorithm is shown in Figure 4. Firstly, the proposed algorithm recovers the G components by adopting the posteriori idea. Then, the R/B components at G positions are recovered by the same method as in [8]. Finally, the correlation of R–B channels in high frequency is used to recover the missing R/B components at B/R positions. The novel part of the proposed algorithm is marked by a red dotted line in Figure 4.

3.2. The G Interpolation at R/B Positions

It is vital to recover the missing G components because the human eyes are more sensitive to the green color and the number of G pixels are twice that of the R or B pixels. The quality of G interpolation directly affects the final interpolation effect. Therefore, the missing G components are interpolated first. The complete process of G interpolation at R positions (the process of G interpolation at B positions is the same) is shown in Figure 5.

Firstly, the directional estimations $G^{\mathrm{h}}$ and $G^{\mathrm{v}}$ are calculated by the same method as in ACP [8], as shown in Equation (1) and Equation (2), respectively. The 5 × 5 image block is adopted, as shown in Figure 1. Then, the directional gradients need to be calculated appropriately. This is a crucial step because weights are obtained based on the gradients. Note that it is the most important factor for a successful interpolation algorithm to obtain the weight suitable for the estimation of each direction. The gradients calculated by Equation (3) and Equation (4) are independent of the directional estimations $G^{\mathrm{h}}$ and $G^{\mathrm{v}}$, attributing the poor matching between the weights and direction estimations. To make the weights match the estimations better, the correlation between the gradients and estimations needs to be increased. For this reason, the posteriori idea is introduced into the calculation of gradients. To be more specific, we replace the original G values with $G^{\mathrm{h}}$ and $G^{\mathrm{v}}$ when calculating the gradients. Equation (3) and Equation (4) are modified as follows, respectively:

$$D^{\mathrm{h}}(i,j)=\left|R(i,j-1)-R(i,j+1)\right|+\left|2G^{\mathrm{h}}(i,j)-G^{\mathrm{h}}(i,j-1)-G^{\mathrm{h}}(i,j+1)\right|,$$

(6)

$$D^{\mathrm{v}}(i,j)=\left|R(i-1,j)-R(i+1,j)\right|+\left|2G^{\mathrm{v}}(i,j)-G^{\mathrm{v}}(i-1,j)-G^{\mathrm{v}}(i+1,j)\right|.$$

(7)

The posteriori gradients in the horizontal and vertical directions, $D^{\mathrm{h}}$ and $D^{\mathrm{v}}$, are available now. Next, the weights in the horizontal and vertical directions are calculated. We have the following considerations: (i) Where the gradient is large, the color changes rapidly, and the weight in this direction should be small; where the gradient is small, the color changes slowly and the weight should be large. That is to say, the bigger the gradient is, the smaller the weight in this direction will be, and vice versa. (ii) The neighboring gradient values can be involved to the calculation of weights. Based on the two considerations mentioned above, the weights in the horizontal and vertical directions, $W^{\mathrm{h}}$ and $W^{\mathrm{v}}$, are defined as:

$$W^{\mathrm{h}}(i,j)=\frac{1}{{\left({\displaystyle \sum _{a=i-2}^{i+2}{\displaystyle \sum _{b=j-2}^{j+2}{D}^{\mathrm{h}}(a,b)}}\right)}^2},$$

(8)

$$W^{\mathrm{v}}(i,j)=\frac{1}{{\left({\displaystyle \sum _{a=i-2}^{i+2}{\displaystyle \sum _{b=j-2}^{j+2}{D}^{\mathrm{v}}(a,b)}}\right)}^2},$$

(9)

where $a$ and $b$ indicate the row and the column in the 5 × 5 image block, respectively. Finally, average the horizontal estimation $G^{\mathrm{h}}$ and vertical estimation $G^{\mathrm{v}}$ according to different weights. The G interpolation equation is given as follows:

$$G(i,j)=\frac{G^{\mathrm{h}}(i,j)\times W^{\mathrm{h}}(i,j)+G^{\mathrm{v}}(i,j)\times W^{\mathrm{v}}(i,j)}{W^{\mathrm{h}}(i,j)+W^{\mathrm{v}}(i,j)}.$$

(10)

All the G values can be recovered by applying the steps above to the R/B positions.

3.3. The R/B Interpolation at G Positions

After the G components are recovered completely, the R/B components at G positions need to be interpolated. To keep the computational complexity low, we use the same method as in ACP [8]. Considering a 5 × 5 image block centered on the G pixel, the interpolation equations are given as follows [8]:

$$R(i,j)=\frac{1}{2}[R(i,j-1)+R(i,j+1)]+\frac{1}{4}[2G(i,j)-G(i,j-2)-G(i,j+2)],$$

(11)

$$B(i,j)=\frac{1}{2}[B(i-1,j)+B(i+1,j)]+\frac{1}{4}[2G(i,j)-G(i-2,j)-G(i+2,j)].$$

(12)

3.4. The R/B Interpolation at B/R Positions

In this section, a novel method for recovering R/B components at B/R positions is introduced, which is shown in Figure 6. Considering a 5 × 5 image block as shown in Figure 1, the interpolation steps at $R_{13}$ pixel are as follows. Note that all the G values and the R/B values at the G positions are available. Firstly, the color differences are redefined in the proposed algorithm. Conventional algorithms [15,16,19,21,22] define the color differences as $G-R$ or $G-B$, but we calculate the color differences by $R-B$ or $B-R$. Because the correlation between the B components and the R components in high frequency is more than the G components. Menon et al. proved this in [10]. Furthermore, the color differences in the proposed algorithm are calculated along the north (N), south (S), west (W), and east (E) directions rather than the four diagonal directions (NW, NE, SW, and SE), which is different from LDI-NAT [16]. For the reason that a pixel is closer to adjacent pixels in its N, S, W, and E directions than it is in its NW, NE, SW, and SE directions, the correlation in N, S, W, and E directions is greater than in its NW, NE, SW, and SE directions. Therefore, the color differences in the N, S, W, and E directions, $d_{\mathrm{RB}}^{\mathrm{N}}$, $d_{\mathrm{RB}}^{\mathrm{S}}$, $d_{\mathrm{RB}}^{\mathrm{W}}$, and $d_{\mathrm{RB}}^{\mathrm{E}}$, are defined as:

$$d_{\mathrm{RB}}^{\mathrm{N}}=B_8-R_8,$$

(13)

$$d_{\mathrm{RB}}^{\mathrm{S}}=B_{18}-R_{18},$$

(14)

$$d_{\mathrm{RB}}^{\mathrm{W}}=B_{12}-R_{12},$$

(15)

$$d_{\mathrm{RB}}^{\mathrm{E}}=B_{14}-R_{14}.$$

(16)

Then, the gradients are calculated along the N, S, W, and E directions. We have the following considerations: (i) Both the center pixels and the neighboring pixels need to be used to calculate the gradients. (ii) The central pixels contribute more than the neighboring pixels. (iii) The additional information from the B channel needs to be used for more accurate estimates because of the great correlation existing in R–B channels. Based on the considerations mentioned above, the gradients are defined as:

$${\nabla }_{\mathrm{RB}}^{\mathrm{N}}=\left|B_8-B_{18}\right|+\left|R_3-R_{13}\right|+\frac{1}{2}\left|G_2-G_{12}\right|+\frac{1}{2}\left|G_4-G_{14}\right|,$$

(17)

$${\nabla }_{\mathrm{RB}}^{\mathrm{S}}=\left|B_8-B_{18}\right|+\left|R_{23}-R_{13}\right|+\frac{1}{2}\left|G_{22}-G_{12}\right|+\frac{1}{2}\left|G_{24}-G_{14}\right|,$$

(18)

$${\nabla }_{\mathrm{RB}}^{\mathrm{W}}=\left|B_{12}-B_{14}\right|+\left|R_{11}-R_{13}\right|+\frac{1}{2}\left|G_{16}-G_{18}\right|+\frac{1}{2}\left|G_6-G_8\right|,$$

(19)

$${\nabla }_{\mathrm{RB}}^{\mathrm{E}}=\left|B_{12}-B_{14}\right|+\left|R_{15}-R_{13}\right|+\frac{1}{2}\left|G_{10}-G_8\right|+\frac{1}{2}\left|G_{20}-G_{18}\right|,$$

(20)

where ${\nabla }_{\mathrm{RB}}^{\mathrm{N}}$, ${\nabla }_{\mathrm{RB}}^{\mathrm{S}}$, ${\nabla }_{\mathrm{RB}}^{\mathrm{W}}$, and ${\nabla }_{\mathrm{RB}}^{\mathrm{E}}$ indicate the gradients in the N, S, W, and E directions, respectively. Next, the weights ${\theta }^X$ are defined as the reciprocals of gradients, which can be expressed as:

$${\theta }^X=\frac{1}{{\nabla }_{\mathrm{RB}}^X},X\in (\mathrm{N},\mathrm{S},\mathrm{W},\mathrm{E}),$$

(21)

where ${\theta }^X$ indicates the weights in the four directions (N, S, W, and E) at the $R_{13}$ pixel.

The color differences and weights in the N, S, W, and E directions are available after the above steps. The final color difference estimation, $d_{\mathrm{RB}}$, can be calculated by combining the color differences with the weights, which can be expressed as:

$$d_{\mathrm{RB}}=\frac{{\displaystyle \sum {\theta }^X\times d_{\mathrm{RB}}^X}}{{\displaystyle \sum {\theta }^X}},X\in (\mathrm{N},\mathrm{S},\mathrm{W},\mathrm{E}).$$

(22)

Finally, the missing B component at the $R_{13}$ pixel can be recovered by adding the final color difference estimation $d_{\mathrm{RB}}$ to the $R_{13}$ pixel value:

$$B_{13}=R_{13}+d_{\mathrm{RB}}.$$

(23)

The steps to interpolate the R components at the B pixels are the same as above.

4. Experimental Results and Performance Analyses

To evaluate the effectiveness of the proposed algorithm in image reconstruction, we used MATLAB R2017b with an Intel Core i7-5500U and 2.40 GHz CPU computer to test the Kodak dataset [29] that contains 24 color images. The 24 color images of the Kodak dataset are shown in Figure 7. Each image is either 768 × 512 or 512 × 768 in size. Note that though all the images in Figure 7 are compressed for display, our experiments still used the original Kodak images.

To evaluate the quality of reconstructed images, we used the composite peak signal-to-noise ratio (CPSNR), which is a commonly used evaluation standard. We compared the proposed algorithm with ACP [8], DWDF [10], DLMMSE [12], RAD [13], IG [14], and LDI-NAT [16]. Compared to the two original algorithms (ACP [8] and LDI-NAT [16]), the proposed algorithm has two improvements. Therefore, the comparisons in the case of with/without each improvement were also needed. The original G interpolation and the original R/B interpolation at the B/R positions (OAO), the improved G interpolation and the original R/B interpolation at B/R positions (IAO), and the original G interpolation and the improved R/B interpolation at B/R positions (OAI) were involved in the evaluation. The results of CPSNR are shown in

Table 1. The CPSNR (dB) of the Kodak images.

Images	ACP [8]	DWDF [10]	DLMMSE [12]	RAD [13]	IG [14]	LDI-NAT [16]	OAO	IAO	OAI	Proposed
Figure 7a	34.02	35.27	34.98	34.36	35.44	35.82	34.07	36.21	34.65	36.51
Figure 7b	38.93	40.22	38.57	38.83	37.21	40.19	38.94	40.38	39.78	41.24
Figure 7c	39.26	41.66	40.12	39.23	41.51	41.48	40.45	41.81	41.14	41.85
Figure 7d	38.76	39.87	37.90	38.29	38.90	40.17	39.04	40.44	39.66	40.83
Figure 7e	34.17	36.14	36.98	34.69	36.27	37.00	34.97	37.02	35.37	37.08
Figure 7f	33.86	37.56	36.15	33.99	37.14	36.27	35.20	37.39	35.98	37.63
Figure 7g	41.53	41.14	40.66	40.86	40.88	41.45	40.30	41.27	41.05	41.94
Figure 7h	32.10	33.81	33.29	31.78	33.85	33.82	32.41	33.54	33.00	33.69
Figure 7i	41.93	41.31	41.93	40.52	41.47	40.75	39.16	40.54	39.64	42.08
Figure 7j	41.19	41.06	40.86	40.87	41.38	41.16	39.45	41.44	40.23	41.95
Figure 7k	35.56	37.98	36.05	35.83	38.00	37.98	36.26	38.39	37.07	38.75
Figure 7l	40.55	42.22	39.98	38.99	41.57	41.08	40.36	42.11	41.42	42.55
Figure 7m	30.41	31.64	31.37	31.20	32.09	32.03	30.49	32.62	30.96	32.70
Figure 7n	35.62	36.34	37.10	35.56	36.75	37.14	35.84	36.98	36.13	37.00
Figure 7o	36.87	38.82	35.95	36.22	38.43	38.86	37.50	39.34	38.20	39.42
Figure 7p	39.21	41.48	40.89	38.64	40.79	39.24	38.50	41.02	39.31	41.04
Figure 7q	37.85	39.70	39.68	38.50	41.14	39.92	38.54	40.05	39.13	40.06
Figure 7r	34.47	34.71	34.20	35.14	35.55	35.29	33.86	35.78	34.22	35.81
Figure 7s	37.71	38.13	38.17	37.51	38.58	38.15	36.78	38.71	37.41	38.82
Figure 7t	37.12	37.95	37.81	37.10	38.99	39.28	37.86	39.27	38.43	39.66
Figure 7u	34.62	36.43	34.69	35.00	37.02	37.08	35.57	37.46	36.03	37.71
Figure 7v	35.71	36.65	36.12	36.04	37.89	37.54	36.27	37.67	36.79	38.08
Figure 7w	39.86	40.87	40.52	39.32	42.62	42.33	41.45	41.59	40.98	41.61
Figure 7x	31.54	30.06	32.39	32.52	34.21	33.16	31.98	33.75	32.58	33.81
Average	36.79	37.96	37.56	36.71	38.24	38.20	36.89	38.53	37.47	38.84

. The best result of each image was marked in bold. Two decimal digits were retained.

Additionally, inspired by the work in [25], an ordinary least squares regression analysis that can identify possible errors was performed. To be specific, we performed CPSNR fitting between the proposed algorithm and the other nine algorithms, respectively, and computed the p values ($p$). Experimental results show that all the p values are less than ${10}^{-4}$. The smaller the p value is, the more convincing the comparisons between the proposed algorithm and the other nine algorithms are.

Furthermore, the structural similarity index (SSIM), as an index to measure the similarity between the reconstructed image and the original image, was also widely applied to evaluate the effectiveness of interpolation algorithms. However, according to the work in [25], the correlation between the CPSNR and SSIM is high. Therefore, there is no reason to calculate the SSIM if the CPSNR is available. Meanwhile, it is beneficial to avoid large tables.

Combining the results in Table 1 and the p values, we have the following analyses. In the results of CPSNR from Table 1, the proposed algorithm had different degrees of improvement of CPSNR in 18 out of 24 images compared to the other nine algorithms. Especially in Figure 7b, the CPSNR of the proposed algorithm was at least 0.86 dB higher than that of the other nine algorithms. Furthermore, the average CPSNR of IAO, OAI, and the proposed algorithm reached 1.64 dB, 0.58 dB, and 1.95 dB higher than that of OAO ($p<{10}^{-4}$), respectively. These results demonstrate that the two improvements have significant effects, especially the combination of the two. The proposed algorithm did not generate the best results in all the 24 images, but the average CPSNR of the proposed algorithm was the highest among the ten algorithms, reaching 0.60 dB and 0.64 dB higher than that of IG [14] and LDI-NAT [16] ($p<{10}^{-4}$), respectively. Furthermore, what is worth mentioning again is that all the p values were less than ${10}^{-4}$, which demonstrates that the comparisons are convincing. Therefore, the proposed algorithm actually has better performance to reconstruct the full-color images compared to the other nine algorithms.

In addition to objective evaluations, visual effect is also an important standard to evaluate the interpolation effectiveness. To show a visual comparison of reconstructed images processed by different algorithms, we reconstructed an image by the ten different algorithms mentioned above. The position in Figure 7i that has been marked by a yellow box in Figure 8 contains many edge regions that are prone to color artifacts and distortions. Therefore, Figure 7i was selected as the test image. Furthermore, we magnified the position of the yellow box in Figure 8 to show the comparisons of visual effect more clearly. The pixels contained in the yellow box are 60 × 60 in size, starting at (385,390). The reconstructed images of the enlarged region of the yellow box processed by the ten different algorithms are shown in Figure 9.

The image processed by the proposed algorithm has slight color artifacts and distortions at the edges of the number part, as shown in Figure 9k, but the color artifacts and distortions are more serious visually in the images processed by the other nine algorithms, as shown in Figure 9b–j. The image processed by the proposed algorithm also preserves more details. The visual comparisons of reconstructed images demonstrate that the proposed algorithm has a better effect in image interpolation when compared to the other nine algorithms, especially in reconstruction of details.

5. Conclusions and Remarks on Possible Future Work

In this paper, we propose a weights-based demosaicking algorithm that applies the posterior idea and the great correlation of R–B channels in high frequency. To reconstruct the G components at the R/B positions, the G values in the horizontal and vertical directions are first estimated. Then, the gradients are calculated by a novel posteriori method in the horizontal and vertical directions. The weights are obtained based on the posterior gradients in the horizontal and vertical directions. Finally, the weights and G estimations are combined to generate the full G image. After interpolating the R/B components at the G positions by the same method as in ACP [8], another novel weights-based method is applied to recover the missing R/B components at the B/R positions. Firstly, the color differences are calculated in the N, S, W, and E directions by using the great correlation of R–B channels in high frequency. Then, additional B/R channel information is added to the calculation of gradients. The weights are obtained based on the gradients. Finally, the weights and color differences are combined to generate the final color difference estimations that are added to the B/R pixel values to recover the missing R/B components at the B/R positions. Experimental results show that our algorithm performs well not only in the objective experimental measurement but also in the subjective visual effect.

However, the R/B interpolation at the G position is still processed in one direction, which will negatively affect the interpolation effect. Therefore, we will focus on how to improve the R/B interpolation effect at the G positions in the future work.