Parameter Estimation of Poisson–Gaussian Signal-Dependent Noise from Single Image of CMOS/CCD Image Sensor Using Local Binary Cyclic Jumping

Since signal-dependent noise in a local weak texture region of a noisy image is approximated as additive noise, the corresponding noise parameters can be estimated from a given set of weakly textured image blocks. As a result, the meticulous selection of weakly textured image blocks plays a decisive role to estimate the noise parameters accurately. The existing methods consider the finite directions of the texture of image blocks or directly use the average value of an image block to select the weakly textured image block, which can result in errors. To overcome the drawbacks of the existing methods, this paper proposes a novel noise parameter estimation method using local binary cyclic jumping to aid in the selection of these weakly textured image blocks. The texture intensity of the image block is first defined by the cumulative average of the LBCJ information in the eight neighborhoods around the pixel, and, subsequently, the threshold is set for selecting weakly textured image blocks through texture intensity distribution of the image blocks and inverse binomial cumulative function. The experimental results reveal that the proposed method outperforms the existing alternative algorithms by 23% and 22% for the evaluative measures of MSE (a) and MSE (b), respectively.

Past studies have engineered various noise parameter estimations methods to adopt the Poisson-Gaussian signal-dependent noise model for CMOS image sensors and achieved satisfactory results. One kind is based on deep learning, which depends on the ability of the convolutional neural network in memorizing training data [20][21][22][23][24][25]. The other is based on calculation from single image. The latter is broadly classified into two categories viz. methods based on variance stabilization transformation (VST) and methods based on fitting sample pairs.

Related Work
Parameter estimation methods based on VST transform signal-dependent noise to additive white Gaussian noise (AWGN) in the transform domain. Pyatykh and Hesser [12] devised a VST-based parameter estimation method using an optimization procedure, where the input noisy image was initially transformed into a corrupted AWGN and later analyzed and estimated using principal component analysis (PCA) in the transform domain. Mäkitalo and Foi [13] developed an algorithm for estimating noise in these image sensors by coalescing an iterative VST and resultant estimated noise of the AWGN. However, the premise of VST asserts that the variance of the image data be negligible, thereby, rendering these methods unsuitable for images with large noise variances. The VST-based noise level parameter estimation algorithm needs to use the real noise model parameters when performing the VST, and estimating the noise model parameters is exactly the purpose of the algorithm. This iterative relationship will lead to the unstable estimation effect of such methods at the actual running time.
Parameter estimation methods based on fitting sample pairs focus on accurately identifying weakly textured image blocks in a noisy image. This kind of estimation method can be divided into four categories according to the definition of the texture intensity of weakly textured image blocks, they are methods based on variance or standard deviation, methods based on gradients, methods based on image grey entropy, and methods based on histograms.
The method based on variance or standard deviation was performed by wavelet analysis [10,15,16]. It separates the noise in the image block into the high-frequency area, while retaining the image information containing the image texture in the low-frequency area, then calculate the standard deviation or variance of the low-frequency area of the image block, measure the texture intensity of the image block with the size of its standard deviation or variance, and set the threshold to select relatively flat image blocks. However, while separating the image noise, part of the texture and edge information also comes to the high frequency area, which leads to inaccurate texture intensity calculation. At the same time, the low frequency part of losing part of the texture information is less than the real value when calculated as the pixel intensity estimate, while the high frequency part containing the texture will be larger than the real noise variance estimate, which eventually leads to the noise variance estimate is larger at the same pixel intensity level.
To this effect, an image gradient matrix [14,17] is harnessed to evaluate the image texture intensity. The authors compute the texture intensity of the image block as the sum of the eigenvalues of the image block gradient covariance matrix, which in turn contributes to the selection of weakly textured image blocks. However, the proposed method only considers the image pixel changes in the horizontal and vertical direction of the image block when calculating the texture strength, as opposed to all directions, and the resulting texture strength still varies from the real texture strength.
In [18], a histogram of local mean values is employed to select image blocks with continuous pixel intensity as weakly textured image blocks. However, since image blocks with different texture intensities can still contain similar average pixel values, the selected weak texture image blocks may contain blocks with higher texture strength, thus making the estimated noise variance high, and this method may cause the pixel mean of the selected image blocks all concentrated in a certain range, so that the estimated noise level estimates of the noise variance in other pixel intensity regions can deviate greatly from the real values.
In [19], the local grey entropy of the image blocks was used to select the weakly textured image blocks. The drawback of the method is that the grey entropy of the image is expressed as the bit average of the set of image gray scale levels, reflecting the average amount of information, but the number of pixels inside the image with different textures may also be the same, so the image grey entropy is not completely linearly correlated with its texture strength. The key to the noise parameter estimation algorithm based on fitting sample pairs is the selection of weakly textured image blocks. However, the existing image block texture intensity definition methods cannot fully characterize the texture information of image blocks. To improve the overall accuracy of selecting these weakly textured image blocks and estimating the noise parameters of a noisy image thereafter, we propose a novel methodology based on local binary cyclic jumping as applied to a Poisson-Gaussian signaldependent noise model. First, N×N image blocks are extracted from a noisy image. Next, the texture strength of the image blocks is defined and computed using local binary cyclic jumping. Then, the image blocks with weak texture strengths are nominated to elicit the set of sample pairs of pixel intensities and noise variances, and the traditional ordinary least squares is applied to fit the sample pairs, this iterative process will be terminated until the estimated noise variance remains unchanged and optimal noise parameters are obtained.

Poisson-Gaussian Signal-Dependent Noise Model
The signal-dependent noise of a CMOS image sensor can be modelled as: where i and j represent the row and column of a pixel in the image, respectively; I(i, j) denotes a pixel value of the noisy image at (i, j); η(i, j) denotes a value of the noise at (i, j); and x(i, j) denotes a pixel value of the noise-free image at (i, j). η p (x(i, j)) represents the signal-dependent Poisson noise component of the photon noise resulting from fluctuations in the number of detected photons. Pixel values affected by the Poisson noise component pursue a Poisson distribution, whose variance and mean values are given by 1 a (x(i, j)), as shown below: where a is the photon noise parameter as per the quantum efficiency of the sensor. A large value of a is indicative of higher number of photons required to elicit a response from the sensor, thereby proportionally magnifying the impact of the photon noise on the image. Furthermore, η N (i, j) is the signal-independent zero-mean Gaussian noise component that characterizes the thermal and electrical noise contributed by circuits: where b is the variance of the Gaussian noise. A large value of b is indicative of greater impact from thermal and electrical noise on the image. Since the mean and variance of the Poisson distribution are equal, it can be obtained by formula (2): , it can be deduced that: Therefore, the overall noise variance σ 2 (i, j) of a pixel at location (i, j) then can be given as: The goal of the proposed noise parameter estimation model is to obtain the values of a and b from a noisy image generated by the CMOS image sensor.

Proposed Noise Parameter Estimation Model
The proposed algorithm comprises five principal modules, which are extraction of image blocks, defining and estimating texture strength using LBCJ, selection of weakly textured image blocks based on texture strength, estimating pixel intensity and variance sample pairs, and fitting sample pairs to elicit noise parameters. 1 : Extraction of Image Blocks The first module of the algorithm is to extract a certain number of image blocks with the same size from the noise image, which is to prepare for the later selection of weak texture image blocks as sample data. For a noisy image of size R × S, N × N-sized image blocks are extracted by sequentially making the central pixel traverse every pixel from the left to right column and top to bottom row. The size of N will have a certain impact on the accuracy of the estimation, and it will also affect the running time of the algorithm. If N is too small, it will increase the complexity and running time of the algorithm; if N is too large, it will affect the accuracy of the algorithm. According to [19], we set the size parameter N of the image block to 15 pixels in this research, to ensure a good estimation effect while taking into account faster algorithm execution efficiency. Then B_n = (R − N + 1) × (S − N + 1) image blocks were extracted from the supplied noisy image, where B_n is the total number of image blocks. 2 : Defining and Estimating Texture Strength using Local Binary Cyclic Jumping After extracting a certain number of image blocks from the noise image, these image blocks need to be selected, and some relatively flat image blocks with weak texture information are selected as image sample data for subsequent estimation. At this time, a judgment is required to measure the intensity of the image block texture. Therefore, a method based on local binary cyclic jumping is designed to define the texture intensity of image blocks.
For an image block I k (k ∈ [1, B_n]) of size N × N, as shown in Figure 1a, where m and n denote the row and column of a pixel, respectively, the central pixel is represented by I k (m, n) for m ∈ [2, N − 1], n ∈ [2, N − 1], and its eight-neighbor connected domain is shown in Figure 1b. The individual pixel values in eight-neighbor connected domain are shown in Figure 1c. Their corresponding binary forms are labelled L 0 -L 7 , as shown in Figure 1e.
Sensors 2021, 21, x FOR PEER REVIEW the corresponding pixel is set to 1, else 0, as shown in Figure 1d,e. The pixel inte the homogeneous region are close and according to [16,17], the maximal intensi ence of two pixels in the homogeneous region is set to 15. Since the absolute value the threshold is set to half of the maximum difference, which is 7.5. The binary the corresponding pixel can then be written as:  Next, a local texture binary cyclic jumping is performed. For this, the absolute difference between the central pixel and every adjacent pixel is computed, as shown in Figure 1c,d. If this difference is greater than the predefined threshold value, the binary value of the corresponding pixel is set to 1, else 0, as shown in Figure 1d,e. The pixel intensities in the homogeneous region are close and according to [16,17], the maximal intensity difference of two pixels in the homogeneous region is set to 15. Since the absolute value is taken, the threshold is set to half of the maximum difference, which is 7.5. The binary value of the corresponding pixel can then be written as: Likewise, after computing the binary values of pixels in the eight-neighbor connected domain, a local binary cyclic jumping is formed by arranging L 0 -L 7 in a circular loop, as shown in Figure 1f. Then the cycle jumping number of the eight-bit binary circle sequence is calculated for the central pixel I k (m, n) by (8), which reflects texture intensity information of the central pixel: where J_n{ L 0 ∼ L 7 } represents the number of cyclic transitions of binary values in the binary sequence composed of L 0 ∼ L 7 , since it is a cyclic transitions, the number of transitions is independent of starting position. The texture strength, T k , of the image block I k is defined as average of the texture intensity information t k (m, n) around all central pixels 3 : Selection of Weakly Textured Image Blocks Based on Texture Strength After calculating the texture intensity of each image block, the weak texture image block needs to be selected according to the texture intensity. The key is to determine the threshold of the weakly textured image blocks. Therefore, a method based on texture intensity distribution of image blocks is designed to calculate the threshold of the weakly textured image blocks.
First, we derive the statistical distribution of texture intensity of a flat image block affected by noise: assume that the image block is a flat area before being contaminated by signal dependent noise, according to literature [18] and (1), for large photon counts, in a noisy flat patch, I k can be approximated as: where x k (m, n) is the noise-free image data, η k (m, n) is the independent zero-mean Gaussian with the standard deviation σ( f ), σ 2 ( f ) is the noise variance, and f is the intensity of the flat patch, The following deduction can be then acquired: In the same way, the noise η k (m + u, n + v) in the image block I k which is adjacent to I k (m, n) also obeys the Gaussian distribution, with a mean value of 0 and a variance of σ 2 ( f ), as shown in the formula (12).
As I k is a noisy flat patch, the x k on it is approximately same, and (13) is, hence, obtained. The pixel level of a noise-free image will change after being affected by noise. Assuming that the flat area with pixel level f is affected by the overall noise standard deviation σ( f ), there is no texture intensity jump between I k (m, n) and its neighboring pixel I k (m + u, n + v), we denote this probability as P 0 . Otherwise, the probability is recorded as P 1 . Then, the following deduction can be then acquired: where er f (•) is the Gauss error function. Therefore, 7] are independent identically distributed variables and follow the 0-1 distribution.
From Figure 1, we can infer that a cyclic jump occurs when the sum of any adjacent two-bits in an eight-bit binary sequence equals 1 and the corresponding probability, P L , is computed as: Conversely, a cyclic jump is skipped when this sum does not equal 1 and the corresponding probability is, then,1 − 2P 0 P 1 . Therefore, t k (m, n) follows a binomial distribution i.e., t k (m, n) ∼ B(8, P L ), and, resultantly, τ is the threshold of the weakly textured image blocks and can be expressed as a function of the given noise level, as shown in (16), where ζ is the confidence level, set to 1-10 −6 as in [18]. F −1 represents an inverse binomial cumulative function. The image blocks where the texture strengths are less than τ, are defined as weakly textured image blocks.

: Estimating Pixel Intensity and Variance Sample Pairs
After selecting the weak texture image block, we need to estimate pixel intensity and variance sample pairs to prepare for the subsequent fitting of the sample pair to effective noise parameters.
Estimated pixel values of the weakly textured image blocks are expressed as an average of all pixel values within the blocks, given byx wk for wk ∈ [1, B_weak_n]. B_weak_n is the number of weakly textured image blocks: where wk is the index of weakly textured image blocks, B_weak_n is the total number of weakly textured image blocks. In this study, the characteristics of natural images are used to estimate the noise level. Due to the redundancy of natural images, the data of natural images only span lowdimensional subspaces, so the noise variance of weak texture image blocks can be estimated based on principal component analysis (PCA), thereby producing better estimation results.
Consider the variance of the data projected to a certain direction u, the definition of the minimum variance direction u min is as follows: u min = argmin u Var u T I wk (18)  The noise varianceσ 2 wk of a weakly textured image block I wm is computed as: where u T min I wm is the Euclidean norm of the vector u T min I wm , and u min is the minimum variance direction vector calculated using PCA and defined as the eigenvector associated with the minimum eigenvalue of the covariance matrix given in (20). (20) where Cp is the covariance matrix. Thus, the sample pairs of pixel intensity and noise variance estimated from the weakly textured image blocks is x wk ,σ 2 wk , wk ∈ [1, B_weak_n]. 5 : Fitting Sample Pairs to Elicit Noise Parameters The ordinary least squares method was used to fit the sample pairs and obtain the measurable parametersâ andb. Here, we define three parameters viz. X = [x 1 ,x 2 , L,x m , L,x B_weak_n ] T , X 1 = [X, 1], a matrix of size B_weak_n × 2, where the first column vector is X and the second is 1, and Q = σ 2 1 ,σ 2 1 , · · · ,σ 2 m , · · · ,σ 2 B_weak_n T . The measurable parametersâ andb are then fitted using a least squares method given as: Modules 3 -5 are conducted iteratively to acquire optimal noise parameters as in [18]: (a) An initial noise variance is estimated using all blocks in the supplied noisy image; (b) weakly textured image blocks are selected from step 3 by using the initial noise variance; (c) the current image noise variance is estimated from the weakly textured image blocks in modules 4 -5 and (6); and (d) module 3 is revisited to initiate a new iteration of obtaining another set of weakly textured image blocks based on the current noise variance. The process is complete when the estimated noise variance remains unchanged.

Experimental Results
The experiments were performed in MATLAB 2016a on a computer with 3.30 GHz Intel Pentium G3260 CPU and 4 GB random access memory.
A test set of 24 true standard Kodak PCD0992 noise-free images [26], as shown in Figure 2, was adopted for the experiment. A signal-dependent noise was added to them in compliance with Equations (1)- (6). 16  Because almost all denoising methods are based on gray image [27], we changed the three-dimensional RGB full-color image into one-dimensional gray image to process. The graying formula is as follows: where R, G, and B, respectively, represent three channels of the RGB image. Intel Pentium G3260 CPU and 4 GB random access memory. A test set of 24 true standard Kodak PCD0992 noise-free images [26], as shown i Figure 2, was adopted for the experiment. A signal-dependent noise was added to them in compliance with Equations (1)-(6). 16 pairs of noise parameters were chosen and ex pressed as a pairwise combination of a = {0.005, 0.010, 0.015, 0.020} and b = {0.0016, 0.003 0.0064, 0.0100}. Because almost all denoising methods are based on gray image [27], we changed th three-dimensional RGB full-color image into one-dimensional gray image to process. Th graying formula is as follows:  The proposed method was validated against four existing state-of-the-art noise parameter estimation methods based on: (a) image gradient matrix [18]; (b) image histogram [19]; (c) image local gray entropy [20]; and (d) CNN [28], which has achieved superlative results.
In order to fairly verify whether the proposed technique improved the accuracy of estimating noise parameters a and b, the operations of all steps of the algorithm, besides selecting the weakly textured image blocks, remained consistent across the board.
The mean square error (MSE) was used to measure the accuracy of the noise estimation result. The smaller the mean square error, the better the estimation result. MSE values of the 24 test images given in Figure 2 where (a, b) is a set of preconfigured noise parameter values, (â i_tI ,b i_tI ) is the estimated noise parameter values, i_tI is the index of the test image, and M is the total number of test images (i.e., 24 for this study). Figure 3 enumerates the comparative MSE results of the various noise parameter estimation methods cited in this study with linear least squares fitting. From Figure 3, it is evident that the proposed estimation method obtains lower and smoother MSE values compared to those of other methods, which is attributed to the conscientious selection of weakly textured image blocks using LBCJ. Considering that LBCJ captures the change in the pixel values in the eight-neighbor connected domain of the central pixel in multiple directions, it interprets the textural alterations, and thereby selects weakly textured image blocks more accurately, which in turn warrants a more precise parameter estimation. timation methods cited in this study with linear least squares fitting. From Figure 3, it is evident that the proposed estimation method obtains lower and smoother MSE values compared to those of other methods, which is attributed to the conscientious selection of weakly textured image blocks using LBCJ. Considering that LBCJ captures the change in the pixel values in the eight-neighbor connected domain of the central pixel in multiple directions, it interprets the textural alterations, and thereby selects weakly textured image blocks more accurately, which in turn warrants a more precise parameter estimation.
(a) (b)  Figure 4 shows the visual comparison of weakly textured image block selection results under different methods. The proposed method was compared with the existing methods based on local grey entropy, image histogram and image gradient matrix, which are all the methods based on the selection of weakly textured image block. The noise parameter was set to a = 0.005, b = 0.0016. It can be seen from Figure 4 that each method can obtain a good selection result of weakly textured image blocks. In areas where the texture changes frequently, such as the cloud in the upper right corner, the proposed method can select the weakly textured image blocks more accurately.

Computational Complexity
The running time and memory consumption of the algorithm we proposed is related to the size of N. In order to compare the running time and memory consumption of our algorithm and other methods, we further tested our method in MATLAB 2016a on a computer with 3.30 GHz Intel Pentium G3260 CPU and 4 GB random access memory. For fair comparison, the competing methods were also tested in the same environment. We se-

Computational Complexity
The running time and memory consumption of the algorithm we proposed is related to the size of N. In order to compare the running time and memory consumption of our algorithm and other methods, we further tested our method in MATLAB 2016a on a computer with 3.30 GHz Intel Pentium G3260 CPU and 4 GB random access memory. For fair comparison, the competing methods were also tested in the same environment. We selected one of the Kodak pictures and added 16 sets of noise to it respectively, the results are compiled in Tables 1 and 2. With the comparison of running time, as can be seen, our method was faster than the other three competing methods. In addition, when the noise intensity increased significantly, the running time of the other three methods increased significantly, while the running time of our method was relatively kept within a relatively small change interval. With the comparison of memory consumption, we can see that the memory consumption of our proposed algorithm was smaller than that of method based on image gradient matrix and method based on local grey entropy, which was similar to that of method based on image histogram. In general, our algorithm has faster running speed and smaller memory consumption.

Conclusions
This study proposed a new methodology of determining the noise parameters of a Poisson-Gaussian signal-dependent noise using local binary cyclic jumping. Through calculating the LBCJ information in the eight neighborhoods around the pixel to define the texture intensity and using the binomial cumulative function to determine the selection threshold, the weakly textured image blocks can be selected more accurately. The experimental results have shown that the proposed algorithm solicits lower MSE values and exhibits a superior performance over those of the existing algorithms. Therefore, it can be considered that the proposed algorithm can help improve the image quality of the CMOS image sensors and other digital imaging systems.