Review: A Survey on Objective Evaluation of Image Sharpness
Round 1
Reviewer 1 Report
Introduction:
1. You should start your discussion with all methods of image quality evaluation such as image noise, dynamic range, color accuracy, contrast etc.
2. Elaborate more on the increasing number of evaluation methods of a combination of methods as mentioned in the introduction
3. Please provide some specific metrics or details about high real-time performance.
4. Please discuss why one sharpness evaluation algorithm is unrealistic to handle all potential images.
Evaluation method and analysis
There are four categories on sharpness methods. What about other methods such as human visual, mathematical modelling, statistical and hybrid methods.
Since the latest method is DL, it may be beneficial to delve into more details on this method.
Author Response
Dear reviewer,
Thank you very much for allowing a resubmission of our manuscript, with an opportunity to address the reviewers’ comments. The manuscript has been carefully revised according to the reviewers’ valuable advice. Furthermore, we asked a native English-speaking colleague to check and correct the full text. The followings are our point-by-point response to the comments, and the revised manuscript is marked in the text using the "Track Changes" function.
- You should start your discussion with all methods of image quality evaluation such as image noise, dynamic range, color accuracy, contrast etc.
Response: Thank you for your careful work and valuable advice. We have revised the brief introduction of the sharpness evaluation metric in the first paragraph, starting with various methods of image quality evaluation, and briefly introduced the evaluation metrics such as image noise, image color, artifact, and sharpness. Modifications have been marked with the "Track Changes" function in the first paragraph of the introduction.
- Elaborate more on the increasing number of evaluation methods of a combination of methods as mentioned in the introduction.
Response: Thank you for your careful work and valuable advice. We have added elaboration to the combination of methods in the 3rd paragraph of the introduction, and the revisions are marked using "Track Changes" function.
- Please provide some specific metrics or details about high real-time performance.
Response: Thank you for your careful work and valuable advice. Through extensive literature reading and analysis, we found that many spatial domain evaluation methods possess high real-time performance because they directly calculate the quantitative relationships between image pixels with low computational complexity. For example, Marziliano et al. calculate the edge width in the vertical direction to measure sharpness based on the principle of edge diffusion of blurred images. It is simple for a computer to calculate the edge width of an image, so this method has high real-time performance. Bahrami et al. further obtains the quality score by calculating the maximum local variation (MLV) of image pixels. Calculating the maximum variation of a local pixel is equivalent to finding the local maximum of an image pixel, and the calculation is fast. Zhang et al. proposed an image-filtering evaluation method based on the Sobel operator and image entropy. One of the advantages of the Sobel operator is the fast operation speed. From the perspective of high real-time performance, many scholars use the Sobel operator for some algorithm research. These methods are described in detail later in sections 2.1.2, 2.1.3.
- Please discuss why one sharpness evaluation algorithm is unrealistic to handle all potential images.
Response: Thank you for your careful work and valuable advice. One sharpness evaluation method cannot take into account all kinds of distorted images. For example, the study done in Vu[1] illustrates that two images with the same spectral domain features but different contrast appear to have different sharpness. If the two images are evaluated with the spectral domain evaluation method, their quality result score is found to be similar, while the sharpness obtained with the spatial domain method is inconsistent.
- Vu, C. T.; Phan, T. D.; Chandler, D. M. S3: A spectral and spatial measure of local perceived sharpness in natural image. IEEE Transactions on image processing. 2012, 21, 934-945.
- There are four categories on sharpness methods. What about other methods such as human visual, mathematical modelling, statistical and hybrid methods.
Response: Thank you for your careful work and valuable advice. The evaluation method based on human vision is usually a subjective evaluation method, but in this paper we focus on objective evaluation methods. There are also studies on evaluation methods based on human vision combined with other methods, for example, with deep learning methods, and we believe that this type of evaluation methods can be classified as combination methods. Most of the mathematical modeling, statistical, and hybrid-based methods are built on features such as pixes or gray gradients of images, and these methods can be classified into spatial domain methods or combination methods according to the actual situation. In the related literature, such as section 2.1.3 literature [47], we also use the "Track Changes" function labeled.
- Since the latest method is DL, it may be beneficial to delve into more details on this method.
Response: Thank you for your careful work and valuable advice. We analyze more details of the deep learning-based methods in depth, and add some deep learning-based methods in Section 2.3.2.
Author Response File: 
Author Response.docx
Reviewer 2 Report
-Overall the review paper refers to an important issue. Some points below can help the readability of the article:
-The organization could be better. The scope of the paper has been concentrated on introducing the evaluation metrics of image sharpening. But, part of the review paper has been concentrated on some image-sharpening methods which are far from the aims of the paper.
-Several sharpness evaluation methods such as ones that use learning-based methodology (table 3), may use spatial or transform domain evaluation metrics in their final evaluation. I think it could be separated or at least brought into the correct category (spatial or spectral).
Author Response
Dear reviewer,
Thank you very much for allowing a resubmission of our manuscript, with an opportunity to address the reviewers’ comments. The manuscript has been carefully revised according to the reviewers’ valuable advice. Furthermore, we asked a native English-speaking colleague to check and correct the full text. The followings are our point-by-point response to the comments, and the revised manuscript is marked in the text using the "Track Changes" function.
-The organization could be better. The scope of the paper has been concentrated on introducing the evaluation metrics of image sharpening. But, part of the review paper has been concentrated on some image-sharpening methods which are far from the aims of the paper.
Response: Thank you for your careful work and valuable advice. We are sorry for introducing methods that are not fit the topic of the paper. And we have now replaced literature presentations that do not match the topic, sunch as literature [75], [76], [79] in section 2.3.2.
-Several sharpness evaluation methods such as ones that use learning-based methodology (table 3), may use spatial or transform domain evaluation metrics in their final evaluation. I think it could be separated or at least brought into the correct category (spatial or spectral).
Response: Thank you for your careful work and valuable advice. After a large literature analysis, we found that most learning-based evaluation methods are studied in combination with spatial domain or transform domain features. However, we still classify this part of the methods as learning-based methods because their research content is dominated by learning-based methods. There were some problems with our presentation of Table 3, which have now been corrected. The modifications are shown Section 2.3 and Table 3, and are marked using the "Track Changes" function.
Author Response File: Author Response.docx
Reviewer 3 Report
A very well organized, exhaustive survey paper.
One suggestion: Text in line 292-294 could be re written for a better understanding and clarity.
Author Response
Dear reviewer,
Thank you very much for allowing a resubmission of our manuscript, with an opportunity to address the reviewers’ comments. The manuscript has been carefully revised according to the reviewers’ valuable advice. Furthermore, we asked a native English-speaking colleague to check and correct the full text. The followings are our point-by-point response to the comments, and the revised manuscript is marked in the text using the "Track Changes" function.
One suggestion: Text in line 292-294 could be re written for a better understanding and clarity.
Response: Thank you for your careful work and valuable advice. We are sorry for not expressing this part clearly, and the modifications have been marked with the "Track Changes" function in Section 2.3.3.
Author Response File: Author Response.docx
Reviewer 4 Report
Comments: 1. What is the main question addressed by the research? A survey on objective evaluation of image sharpness. 2. Do you consider the topic original or relevant in the field? Does it address a specific gap in the field? Accepted. The gaps have been addressed accordingly. However, it can be improved. 3. What does it add to the subject area compared with other published material? Yes. 4. What specific improvements should the authors consider regarding the methodology? What further controls should be considered? Please check my comments. 5. Are the conclusions consistent with the evidence and arguments presented and do they address the main question posed? Yes. 6. Are the references appropriate? Yes. Further Comments: 1. Keywords: it is suggested to have 5 keywords. 2. Introduction: First paragraph is too short. it can be elaborated further. 3. Introduction: Para 3 & 4 are weird. Fix this. 4. Chapter 3.1: You can add searching keywords for your trend analysis 5. Please include from which database such as scopus, wos, google scholar. 6. All abbreviations must be explained (at least full name of each of them)
Author Response
Dear reviewer,
Thank you very much for allowing a resubmission of our manuscript, with an opportunity to address the reviewers’ comments. The manuscript has been carefully revised according to the reviewers’ valuable advice. Furthermore, we asked a native English-speaking colleague to check and correct the full text. The followings are our point-by-point response to the comments, and the revised manuscript is marked in the text using the "Track Changes" function.
- Keywords: it is suggested to have 5 keywords.
Response: Thank you for your careful work and valuable advice. The keywords are changed to: evaluation metric; image sharpness; no-reference; image quality; evaluation algorithm. Modifications have been marked with the "Track Changes" function in Keywords.
- Introduction: First paragraph is too short.
Response: Thank you for your careful work and valuable advice. We have revised the brief introduction of the sharpness evaluation metric in the first paragraph, starting with various methods of image quality evaluation, and briefly introduced the evaluation metrics such as image noise, image color, artifact, and sharpness. Modifications have been marked with the "Track Changes" function in the first paragraph of the introduction.
- Introduction: Para 3 & 4 are weird. Fix this.
Response: Thank you for your careful work and valuable advice. We reorganized the expressions in Para 3 & 4 , and deleted and revised some illogical contents. The modifications have been marked with the "Track Changes" function in Para 3 & 4 of the introduction.
- Chapter 3.1: You can add searching keywords for your trend analysis.
Response: Thank you for your careful work and valuable advice. We add the trend analysis related to search keywords. The modifications have been marked with the "Track Changes" function in Section 3.1.
- Please include from which database such as scopus, wos, google scholar.
Response: Thank you for your careful work and valuable advice. We indicate the source of statistical data related information, and modifications have been marked with the "Track Changes" function in Section 3.1.
- All abbreviations must be explained (at least full name of each of them).
Response: Thank you for your careful work and valuable advice. All abbreviations in the text have been corrected, and the modifications have been marked with the "Track Changes" function in Section 3.2.1.
Author Response File: Author Response.docx
