No-Reference Image Blur Assessment Based on Response Function of Singular Values

Shanqing Zhang 1, Pengcheng Li 1, Xianghua Xu 1, Li Li 1,* and Ching-Chun Chang 2 ID 1 School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; sqzhang@hdu.edu.cn (S.Z.); lpc199402@163.com (P.L.); xu@hdu.edu.cn (X.X.) 2 Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK; c.chang.2@warwick.ac.uk * Correspondence: lili2008@hdu.edu.cn; Tel.: +86-136-6663-2945

Blur is a major factor affecting the image quality. In the past few years, a series of NR assessment methods have been proposed. Blur is usually created by low pass filtering, which widens the edge of the image. Marziliano et al. [14] presented a method by detecting the width of the image edges in the spatial domain. First of all, the edges are extracted using Sobel edge detection operators. Then, the blur degree of the whole image is estimated by calculating the mean edge width. In the literature [15], Ferzli et al. presented a metric based on the just noticeable blur (JNB). The image local contrast and edge width are calculated as the inputs to a probability summation model, which produces the blur score. Narvekar et al. [16] introduced an approach based on JNB. The probability of detecting a blur is first estimated using a probabilistic model, then the blur score is produced by computing

•
We designed a response function of the singular values in the DCT domain, which is effective to characterize the image blur. • We combine the spatial information of the blurred image and the spectral information of the gradient map, in order to reduce the impact of the image content.

•
We assign SIFT-dependent weights to the image blocks, according to the characteristics of the human visual system (HVS).
The rest of this paper is organized as follows. Section 2 describes our methodology. Section 3 discusses the experimental results. Finally, Section 4 concludes with a summary of our paper.

The Proposed Methods
We propose a no-reference blur image assessment method based on the response function of the singular values in the DCT domain. The flowchart of our method is shown in Figure 1. It includes two phases; in the first phase, we calculate four components, as follows: (1) the RFSV for every block of the gradient map; (2) the variance for every block of the gray image; (3) the DCT domain entropy for every block of the gradient map; and (4) the block weight for every block of the saliency map. The sum of the RFSV is used to evaluate the blur degree. The sum of variances and the DCT domain entropy are used to normalize the sum of the RFSV. The block weight of the saliency map is designed to adapt to the characteristics of the HVS. In the second phase, we combine these four components to generate the final blur score. We assign SIFT-dependent weights to the image blocks, according to the characteristics of the human visual system (HVS).
The rest of this paper is organized as follows. Section 2 describes our methodology. Section 3 discusses the experimental results. Finally, Section 4 concludes with a summary of our paper.

The Proposed Methods
We propose a no-reference blur image assessment method based on the response function of the singular values in the DCT domain. The flowchart of our method is shown in Figure 1. It includes two phases; in the first phase, we calculate four components, as follows: (1) the RFSV for every block of the gradient map; (2) the variance for every block of the gray image; (3) the DCT domain entropy for every block of the gradient map; and (4) the block weight for every block of the saliency map. The sum of the RFSV is used to evaluate the blur degree. The sum of variances and the DCT domain entropy are used to normalize the sum of the RFSV. The block weight of the saliency map is designed to adapt to the characteristics of the HVS. In the second phase, we combine these four components to generate the final blur score.

Computing Gray Image and Gradient Map
To begin with, a blur image is converted to a grayscale image ( ) In order to acquire the shape information of the grayscale image, the gradient map is first calculated as follows:

Computing Gray Image and Gradient Map
To begin with, a blur image is converted to a grayscale image I(x, y), where x ∈ {1, 2, 3, . . . , M}, y ∈ {1, 2, 3, . . . , N}. In order to acquire the shape information of the grayscale image, the gradient map is first calculated as follows: where " * " denotes the convolution and "T" denotes the transpose. Then, the grayscale image and the gradient map are divided into blocks of the same size. The block size is set to m × n. The grayscale image block and the gradient block sets are denoted by B I ij and B G ij , respectively, where i ∈ {1, 2, 3, . . . R}, j ∈ {1, 2, 3, . . . K}, R = M/m , K = N/n , and • is the floor operator.

Computing RFSV for Every Block
First of all, each gradient block in B G ij is converted into DCT domain, and denoted by the following: where D 11 is the DC coefficient of a block, and the rest variables are the AC coefficients that reflect the image's edges and shapes. By replacing the DC coefficient with 0, we obtain the following Next, we estimate the blur degree from the DCT domain information of a block image. In order to analyze the change between the DCT coefficients, the horizontal and vertical direction difference values are calculated and are denoted, respectively, by the following: where x ∈ {1, . . . , m} and y ∈ {1, . . . , n − 1}, and where x ∈ {1, . . . , m − 1} and y ∈ {1, . . . , n}. F → i,j (:) ∈ R r×1 and F ↑ i,j (:) ∈ R r×1 can be represented collectively as a difference matrix F ∈ R r×2 , as follows: where (:) denotes a conversion from a matrix to a column, and r is the length of the column. In order to illustrate the validity of the differential matrix to estimate the blur degree of the image, we plot (in Figure 2) the value of the difference matrix in the horizontal direction and vertical direction of a block under different blur degrees. One can observe from Figure 2a,b that the distribution of the scattered points is very scattered, which is due to the weak dependencies between F → i,j and F ↑ i,j . The dependencies increase with a stronger Gaussian blur, resulting an approximately elliptical structure in Figure 2c, which concentrates around the zero values. The dependencies can be distinguished from the singular values of the difference matrices. The singular values are computed by singular value decomposition (SVD). where U ∈ R r×r and V ∈ R 2×2 are the unitary matrices. The columns of U and V are the left and right singular vectors. S ∈ R r×2 is a diagonal matrix with the ordered singular values s 1 and s 2 (s 1 > s 2 ) on the diagonal. In order to demonstrate the validity of both singular values to capture the dependencies of the differential matrix elements, we select an image from the categorical subjective image quality (CSIQ) database [4]. Then, the singular values of each block are computed using Equation (8). In our experiment, we randomly select thirty blocks from the total blocks. The distribution of the singular values s 1 and the singular values s 2 of thirty blocks with different standard deviations of the Gaussian blur are shown in Figure 3. It is shown in Figure 3 that the singular values, s 1 and s 2 , of a block decrease with the increasing blur strength. If an image block is sharp, the singular values, s 1 and s 2 , tend to be large. Inspired by the Harris corner detection, we design a response function of singular values (RFSV), as follows: In Equation (9), E ij is a RFSV of an image block. In our experiment, we set α = 0.01. Intuitively, if a block is blurred, the singular values, s 1 and s 2 , tend to be small, and RFSV of the block will also be small. We can say that the RFSV is positively related to s 1 and s 2 . Therefore, the sum of RFSV can estimate the degree of the blur within an image. An experiment is designed to prove this, we randomly select 10 undistorted images from the CSIQ database. The relation between the sum of the RFSV and the Gaussian blur standard deviation is shown in Figure 4. It is observed in Figure 4, that the sum of the RFSV of an image decreases significantly with the increasing standard deviation of the Gaussian blur.  is a diagonal matrix with the ordered singular values s1 and s2 (s1 > s2) on the diagonal. In order to demonstrate the validity of both singular values to capture the dependencies of the differential matrix elements, we select an image from the categorical subjective image quality (CSIQ) database [4]. Then, the singular values of each block are computed using Equation (8). In our experiment, we randomly select thirty blocks from the total blocks. The distribution of the singular values s1 and the singular values s2 of thirty blocks with different standard deviations of the Gaussian blur are shown in Figure 3.
It is shown in Figure 3 that the singular values, s1 and s2, of a block decrease with the increasing blur strength. If an image block is sharp, the singular values, s1 and s2, tend to be large. Inspired by the Harris corner detection, we design a response function of singular values (RFSV), as follows: In Equation (9), ij E is a RFSV of an image block. In our experiment, we set 0.01 Intuitively, if a block is blurred, the singular values, s1 and s2, tend to be small, and RFSV of the block will also be small. We can say that the RFSV is positively related to s1 and s2. Therefore, the sum of RFSV can estimate the degree of the blur within an image. An experiment is designed to prove this, we randomly select 10 undistorted images from the CSIQ database. The relation between the sum of the RFSV and the Gaussian blur standard deviation is shown in Figure 4. It is observed in Figure 4, that the sum of the RFSV of an image decreases significantly with the increasing standard deviation of the Gaussian blur.
is a diagonal matrix with the ordered singular values s1 and s2 (s1 > s2) on the diagonal. In order to demonstrate the validity of both singular values to capture the dependencies of the differential matrix elements, we select an image from the categorical subjective image quality (CSIQ) database [4]. Then, the singular values of each block are computed using Equation (8). In our experiment, we randomly select thirty blocks from the total blocks. The distribution of the singular values s1 and the singular values s2 of thirty blocks with different standard deviations of the Gaussian blur are shown in Figure 3.
It is shown in Figure 3 that the singular values, s1 and s2, of a block decrease with the increasing blur strength. If an image block is sharp, the singular values, s1 and s2, tend to be large. Inspired by the Harris corner detection, we design a response function of singular values (RFSV), as follows: In Equation (9), ij E is a RFSV of an image block. In our experiment, we set Intuitively, if a block is blurred, the singular values, s1 and s2, tend to be small, and RFSV of the block will also be small. We can say that the RFSV is positively related to s1 and s2. Therefore, the sum of RFSV can estimate the degree of the blur within an image. An experiment is designed to prove this, we randomly select 10 undistorted images from the CSIQ database. The relation between the sum of the RFSV and the Gaussian blur standard deviation is shown in Figure 4. It is observed in Figure 4, that the sum of the RFSV of an image decreases significantly with the increasing standard deviation of the Gaussian blur.

Computing Variance and the DCT Domain Entropy for Every Block
The sum of the RFSV can be used to measure the blur degree of an image. However, from Figure  4, we discover that the sum of the RFSV of the different blur images are different, although they are at the same Gaussian blur degree. This is because the different blur images have different contents. In order to obtain similar blur scores of the images with the similar blur degree, we should minimize the impact of the image content.
Li et al. [21] proposed to minimize the impact of the image content by using the sum of block variances, because of the fact that the different images have different variances. Zhang et al. [22] indicated that the use of variance is not enough to eliminate the effects of the image content completely, because some of the images with the almost equal variance may have different contents. Therefore, they proposed to add entropy on the basis of variance. However, we found that the images with different contents might have the same entropy. As shown in Figure 5, the entropies of the two images are both 7.23. Because the spatial information of the image (e.g., variance and entropy) are not enough to eliminate the effects of the image content completely, we should also consider the spectral information of the image. The DCT domain entropy was employed to assess the image quality in the literature [26,27]. Different from the image entropy, the DCT domain entropy represents the probability distribution of the local DCT coefficients. We combine the variance and DCT domain entropy to normalize the sum

Computing Variance and the DCT Domain Entropy for Every Block
The sum of the RFSV can be used to measure the blur degree of an image. However, from Figure 4, we discover that the sum of the RFSV of the different blur images are different, although they are at the same Gaussian blur degree. This is because the different blur images have different contents. In order to obtain similar blur scores of the images with the similar blur degree, we should minimize the impact of the image content.
Li et al. [21] proposed to minimize the impact of the image content by using the sum of block variances, because of the fact that the different images have different variances. Zhang et al. [22] indicated that the use of variance is not enough to eliminate the effects of the image content completely, because some of the images with the almost equal variance may have different contents. Therefore, they proposed to add entropy on the basis of variance. However, we found that the images with different contents might have the same entropy. As shown in Figure 5, the entropies of the two images are both 7.23. Because the spatial information of the image (e.g., variance and entropy) are not enough to eliminate the effects of the image content completely, we should also consider the spectral information of the image.

Computing Variance and the DCT Domain Entropy for Every Block
The sum of the RFSV can be used to measure the blur degree of an image. However, from Figure  4, we discover that the sum of the RFSV of the different blur images are different, although they are at the same Gaussian blur degree. This is because the different blur images have different contents. In order to obtain similar blur scores of the images with the similar blur degree, we should minimize the impact of the image content.
Li et al. [21] proposed to minimize the impact of the image content by using the sum of block variances, because of the fact that the different images have different variances. Zhang et al. [22] indicated that the use of variance is not enough to eliminate the effects of the image content completely, because some of the images with the almost equal variance may have different contents. Therefore, they proposed to add entropy on the basis of variance. However, we found that the images with different contents might have the same entropy. As shown in Figure 5, the entropies of the two images are both 7.23. Because the spatial information of the image (e.g., variance and entropy) are not enough to eliminate the effects of the image content completely, we should also consider the spectral information of the image. The DCT domain entropy was employed to assess the image quality in the literature [26,27]. Different from the image entropy, the DCT domain entropy represents the probability distribution of the local DCT coefficients. We combine the variance and DCT domain entropy to normalize the sum The DCT domain entropy was employed to assess the image quality in the literature [26,27]. Different from the image entropy, the DCT domain entropy represents the probability distribution of the local DCT coefficients. We combine the variance and DCT domain entropy to normalize the sum of the RFSV, and as a result, the blur scores generated by our method are less affected by the image content. The variance and DCT domain entropy are defined by the following: where σ 2 ij denotes the variance of a block B I ij , and µ denotes the mean value of the corresponding block. Each DCT coefficient is normalized by the following: and the DCT domain entropy of a local block L ij is computed by the following:

Computing Block Weight
In practice, the visually salient regions are often used to estimate the sharpness of an image. For example, in Figure 6a, the visually salient region is the image's foreground, rather than the blur background. If an image is sharp, the salient regions will also be sharp. Therefore, this characteristic is crucial to the final blur score. In the literature [12], the SIFT features were used to assess the image quality. In the literature [21], the SDSP model was used to generate a saliency map. In this paper, the saliency map is constructed using SIFT technology. Compared with the SDSP model, the SIFT technology has a lower time complexity and a higher accuracy. The saliency map in Figure 6b is detected from the blurred image in Figure 6a. It is observed that the SIFT points can accurately locate the visually salient regions. Therefore, the number of SIFT points is used to assign weights. We combine the visual saliency with the normalized sum of the RFSV to generate the blur score. The saliency map is divided into blocks of the same size. The saliency map blocks are denoted by B S ij , and the block weight denoted by w ij , as follows: where i ∈ {1, 2, 3, . . . , R}, j ∈ {1, 2, 3, . . . , K}, n ij is the SIFT number of the block B S ij , and β is a constant determined by experiments. The final blur score is denoted by the following: where r is the scale factor. of the RFSV, and as a result, the blur scores generated by our method are less affected by the image content. The variance and DCT domain entropy are defined by the following:

Computing Block Weight
In practice, the visually salient regions are often used to estimate the sharpness of an image. For example, in Figure 6a, the visually salient region is the image's foreground, rather than the blur background. If an image is sharp, the salient regions will also be sharp. Therefore, this characteristic is crucial to the final blur score. In the literature [12], the SIFT features were used to assess the image quality. In the literature [21], the SDSP model was used to generate a saliency map. In this paper, the saliency map is constructed using SIFT technology. Compared with the SDSP model, the SIFT technology has a lower time complexity and a higher accuracy. The saliency map in Figure 6b is detected from the blurred image in Figure 6a. It is observed that the SIFT points can accurately locate the visually salient regions. Therefore, the number of SIFT points is used to assign weights. We combine the visual saliency with the normalized sum of the RFSV to generate the blur score. The saliency map is divided into blocks of the same size.  (14) where r is the scale factor.

Experimental Settings
In this section, six public image quality databases are used to estimate the performance of our method, including the laboratory for the image as well as the Video Engineering (LIVE) [28], CSIQ [4], Tampere Image Database 2008 (TID2008) [29], Tampere Image Database 2013 (TID2013) [30], Camera Image Database (CID2013) [31], and Blurred Image Database (BID) [32] . Among them, LIVE, CSIQ, TID2008, and TID2013 are synthetic blur databases, while CID2013 and BID are real blur databases. The images in the synthetic blur databases are generated using Gaussian low-pass filtering, and thus they aim to simulate the pure blur. By contrast, the real blur databases contain images captured by cameras in complex environments; in other words, they are distorted by multiple distortions. The number of blurred images that are tested on the six databases are 145, 150, 100, 125, 474, and 586, respectively. For each image in LIVE and CSIQ, human assessment scores are provided by a difference mean opinion score (DMOS), and for each image in TID2008, CID2013, TID2013, and BID, the human assessment scores are provided by the mean opinion score (MOS).
We compare the performance of our method with seven existing methods, JNB [15], CPBD [16], S3 [17], LPC [18], MLV [19], BIBLE [21], and Ref. [22]. Four common criterions are used as a reference, including the Pearson Linear Correlation Coefficient (PLCC), root mean square error (RMSE), Spearman Rank Order Correlation Coefficient (SRCC), and the Kendall rank order correlation coefficient (KRCC). SRCC and KRCC are employed for measuring the monotonicity. The PLCC and RMSE are employed to assess the prediction accuracy. In order to calculate these values, a nonlinear fitting function is needed to describe the relation between the predicted scores and the human assessment scores. In this paper, we employ a four-parameter logistic function, as follows: where τ 1 , τ 2 , τ 3 , and τ 4 are the four fitting parameters. Typically, an excellent method produces high values of SRCC, KRCC, and PLCC, and low values of RMSE. In implementation, the block size is 6 × 6, the scale factor r is 0.1, the exponential of variance σ 2 ij is 1, the exponential of the DCT domain entropy c ij is 2, the parameter of response function E ij is set with α = 0.01, and the parameter of the weight function w ij is set with β = 20. Lowe's Matlab source code is applied to detect the SIFT points, and the detailed configurations can be found in the literature [33].

Image-Level Evaluation
We test the proposed method with some blurred images. Six images with different blur degrees from the CSIQ database are shown in Figure 7. The human assessment scores are provided by DMOS, and the predicted blur scores generated by the proposed method as well as the seven existing blind image blur methods, including JNB [15], CPBD [16], S3 [17], LPC [18], MLV [19], BIBLE [21], and Ref. [22].
It is shown in Figure 7 that six images have increasing blur degrees. From Table 1, one can see that the blur scores generated by our method decrease drastically with the increasing blur degree. What's more, our blur scores are consistent with the human assessment scores. By comparing with the seven existing methods, we find that LPC and Ref. [22] can generate scores that are consistent with the blur degree. However, the blur scores generated by the other metrics do not always make an accurate evaluation. The CPBD method generates incorrect scores between Figure 7d Figure 7. DMOS-difference mean opinion score; JNB-just noticeable blur; CPBD-cumulative probability of blur detection; S3spectral and spatial sharpness; LPC-local phase coherence; MLV-maximum local variation; BIBLE-blind image blur evaluation. In the next experiment, we test our method with six images of the LIVE database. It is shown in Figure 8 that they all have similar blur degrees. Because the DMOS of these images monotonically increase, a good blind blur method is expected to generate the monotonically increasing or decreasing scores. Moreover, these blur scores should be nearly identical. The blur scores generated by different methods are summarized in Table 2.   Figure 7. DMOS-difference mean opinion score; JNB-just noticeable blur; CPBD-cumulative probability of blur detection; S3-spectral and spatial sharpness; LPC-local phase coherence; MLV-maximum local variation; BIBLE-blind image blur evaluation. In the next experiment, we test our method with six images of the LIVE database. It is shown in Figure 8 that they all have similar blur degrees. Because the DMOS of these images monotonically increase, a good blind blur method is expected to generate the monotonically increasing or decreasing scores. Moreover, these blur scores should be nearly identical. The blur scores generated by different methods are summarized in Table 2. From Table 2, our method can generate nearly identical and monotonically decreasing scores. However, the blur scores generated by other methods do not have better monotonicity. Therefore, our method effectively distinguishes the images with similar blur degrees, which are highly consistent with DMOS.

Database-Level Evaluation
In order to illustrate the validity of the proposed method, we calculate the overall performance on the four public image databases. Figure 9 shows the scatter plots of the objective scores generated by the different methods versus the subjective scores given by the databases. We choose five recent methods for comparison, including S3, MLV, LPC, BIBLE, and Ref. [22]. From Table 2, our method can generate nearly identical and monotonically decreasing scores. However, the blur scores generated by other methods do not have better monotonicity. Therefore, our method effectively distinguishes the images with similar blur degrees, which are highly consistent with DMOS.

Database-Level Evaluation
In order to illustrate the validity of the proposed method, we calculate the overall performance on the four public image databases. Figure 9 shows the scatter plots of the objective scores generated by the different methods versus the subjective scores given by the databases. We choose five recent methods for comparison, including S3, MLV, LPC, BIBLE, and Ref. [22]. Ref. [22], LIVE Ref. [22], CSIQ Ref. [22], TID2008 Ref. [22], TID2013 In Figure 9, our method shows less biasness and the best correlation of all of the databases. LPC produces slightly worse results in the LIVE and CSIQ databases. In the other two databases, S3 produces slightly worse results. MLV, BIBLE, Ref. [22], and our method produce a somewhat similar fitting. Although, the fitting results produced by Ref. [22] are better than those of methods. A closer comparison indicates that our method achieves the best fitting results, because the scatter points are more densely and evenly clustered around the fitting line. For the four synthetic databases, our method achieves the best consistency and correlation.
We use four criteria to compare the performance of these methods, including PLCC, KRCC, SRCC, and RMSE. Some experimental results between the proposed method and the others on the four synthetic databases are summarized in Table 3. For each performance measure, we mark the top In Figure 9, our method shows less biasness and the best correlation of all of the databases. LPC produces slightly worse results in the LIVE and CSIQ databases. In the other two databases, S3 produces slightly worse results. MLV, BIBLE, Ref. [22], and our method produce a somewhat similar fitting. Although, the fitting results produced by Ref. [22] are better than those of methods. A closer comparison indicates that our method achieves the best fitting results, because the scatter points are more densely and evenly clustered around the fitting line. For the four synthetic databases, our method achieves the best consistency and correlation.
We use four criteria to compare the performance of these methods, including PLCC, KRCC, SRCC, and RMSE. Some experimental results between the proposed method and the others on the four synthetic databases are summarized in Table 3. For each performance measure, we mark the top two results in boldface. Meanwhile, taking into account the differences between the number of images in the different databases, we calculate the weighted average value of the four synthetic databases. It is shown in Table 3 that our method demonstrates the best performance among all of the methods.
In order to verify our method, we use three criteria to compare the performance of six methods, including SRCC, PLCC, and RMSE. The experimental results are summarized in Table 4. In the CID2013 database, our method achieves the best accuracy and the second best monotonicity. In the BID database, our method achieves the second best accuracy and monotonicity. The experimental results indicated that our method also performs well on the real blur images.

Impact of Block Sizes
In the proposed method, the block size is crucial to the final blur score. In order to determine the block size, we set the block size from 4 × 4 to 12 × 12. For each block size, we calculate the average results of the SRCC, PLCC, and RMSE, which are listed in Table 5. It is shown in table that the best results can be achieved when the setting block size is 6 × 6. Therefore, the block size is selected 6 × 6 in the experiment.

Conclusions
Blur is one of the most critical factors that affects image quality. Hence, it is of great significance to design an objective assessment algorithm for the blur degree of the digital images. In this paper, we propose an efficient NR image blur assessment method. The three main contribution of our method are as follows: (1) we observed that to some extent the blur introduces distortion to the DCT coefficients, and thus we design a response function of the singular values (RFSV) to characterize the change; (2) we combine the spatial (i.e., variance of the blurred image) and spectral (i.e., DCT domain entropy of the gradient map) information to minimize the impact of the image content; and (3) we use SIFT-dependent weights to normalize the sum of RFSV, and therefore, the produced score is close to that assessed by the HVS. Compared with the six state-of-the-art NR methods on the six blurred image databases, the blur scores produced by our method are closest to the human subjective scores. Moreover, the proposed method achieves a high accuracy and strong robustness. While the existing methods are effective for synthetic blur, they are limited in predicting the real blur. Our method is more effective for the real blur than most of the existing methods, and yet there is still room for improvement. In the future, we will analyze more useful features and develop advanced assessment algorithms for the real blur databases.
Author Contributions: S.Z. designed the algorithm, P.L. conducted the experiments and wrote the paper, X.X. and L.L. analyzed the results, and C.-C.C. provided helpful suggestions.

Conflicts of Interest:
The authors declare no conflicts of interest.