Statistical Assessment of Accuracy and Precision in Satellite Spatial Quality Measurement Using Natural Edge Targets from KOMPSAT-3A

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper proposes and validates a novel approach for assessing the spatial quality of high-resolution satellites in orbit by using natural edge targets extracted from urban imagery as an alternative to conventional ground-based artificial edge targets. Taking KOMPSAT-3A as a case study, the authors automatically identified 8,414 natural edge targets from 24 images covering 11 cities worldwide. They applied an improved ESMP algorithm to compute key spatial quality metrics, including relative edge response (RER), full width at half maximum (FWHM), MTF50, and MTFA. By statistically comparing these results with eight years of historical measurements from artificial targets, the study demonstrates that although the single-measurement precision of the natural target method, quantified by the coefficient of variation, is roughly twice that of the artificial target approach, the accuracy of their averaged estimates is comparable. More importantly, a single urban image can yield hundreds of natural edge samples, enabling statistically robust spatial quality assessment within a single day. This effectively overcomes the longstanding limitations of artificial targets, such as scarcity, high deployment costs, and maintenance challenges.

Nevertheless, several aspects could be further refined.

1. Certain technical terms are introduced without their full names on first use, which may hinder clarity for a broader readership；
2. Critical processing steps including Savitzky-Golay filtering, edge response function fitting, MTF computation, and noise suppression lack formal mathematical definitions or equations, reducing methodological transparency and reproducibility；
3. The paper does not describe how natural edge targets are automatically detected from imagery. Details about the underlying algorithmic pipeline, whether based on edge detectors, semantic segmentation, or other techniques, are missing；
4. The rationale for selecting the 11 cities remains unclear. It is uncertain whether factors such as architectural style, urban density, or climatic conditions were systematically considered to ensure representativeness；
5. The claim that the method performs less effectively in European and Asian cities is attributed to building characteristics, but this assertion lacks quantitative support such as measurements of building density or image texture complexity that would strengthen the argument；
6. Satellite imaging exhibits anisotropy, and edge orientation significantly influences MTF estimation. The paper does not mention whether natural edge orientations were normalized or corrected to account for this effect;
7. The Phoenix dataset was excluded as an outlier, yet no investigation into the cause of its anomalous behavior, such as atmospheric conditions, surface reflectance properties, or extraction errors, is provided. This simplistic handling weakens the perceived robustness of the proposed method;
8. Figure 5 does not clearly distinguish between accuracy and precision in its annotations, which could lead to misinterpretation;
9. The discussion would benefit from a more explicit comparison with prior studies that have explored natural targets for MTF assessment. Clarifying how this work advances beyond earlier efforts, particularly in terms of automation, scale, and statistical reliability, would better highlight its original contribution.

Comments on the Quality of English Language

Minor improvements could include defining all acronyms at first use and slightly simplifying a few complex sentences to enhance readability for a broader audience. No major language revisions are needed.

Author Response

I would like to express my gratitude for your kind comment.
I've attached a Word file with a response to your comment. Please read it and offer your advice.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This paper assesses the spatial quality of an on-orbit high-resolution Earth observation satellite using edges extracted from natural targets in urban imagery, instead of conventional ground-based artificial edge targets.

1. The abstract is somewhat long. It is recommended to condense it by removing less essential details and focusing primarily on the main contributions and key results of the study. This would make the abstract more concise and easier for readers to grasp the core findings.
2. The reliable extraction of NT edges is heavily dependent on specific urban architectures, leading to a dataset biased towards Western-style cities (e.g., US, Australia) while excluding major regions in Europe and Asia. This limits the global operational applicability of the proposed method for all-orbit missions.
3. While the paper uses a fixed EdgeLine=21 to ensure a sufficient sample size, it does not provide a rigorous sensitivity analysis on how this specific threshold impacts the bias of spatial quality estimators compared to the "ideal" 40-pixel convergence point.
4. Acquired images are often affected by the surrounding imaging environment. If the image quality is degraded due to environmental factors, it may be challenging to accurately extract edges. I recommend that the authors consider using recent image enhancement and restoration methods to improve the quality of the acquired images. At least, this could be discussed as a potential solution to address such cases.

[1] Image quality restoration framework for contrast enhancement of satellite remote sensing images

[2] IHDCP: Single Image Dehazing Using Inverted Haze Density Correction Prior (https://ieeexplore.ieee.org/document/11368686)

5. In Section IV (Results and Discussion), the manuscript should include more visual results. For example, edge detection outcomes under different scenes should be presented to provide a clearer and more intuitive demonstration of the method’s performance.
6. It is unclear whether other methods can achieve results similar to those reported in Table V. The authors are encouraged to provide comparative analysis to clarify how the proposed method performs relative to existing approaches.
7. The study primarily focuses on urban geometric structures but fails to quantitatively analyze how varying atmospheric conditions (e.g., aerosol optical depth) and solar illumination geometry across the 11 global cities affect the consistency of NT-based measurements.

Author Response

I would like to express my gratitude for your kind comment.
I've attached a Word file with a response to your comment. Please read it and offer your advice.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

After reading the response letter and the revised manuscript, some major concerns have not been addressed.

1. The abstract requires further polishing. In general, abstracts should avoid including citations. The authors are encouraged to focus on clearly and concisely presenting the core contributions and key findings of the paper itself, rather than referencing prior work.
2. The methodology appears highly dependent on specific urban morphologies. The researchers had to exclude cities in Europe, South America, and Asia because their urban structures (smaller footprints and high-rise buildings) yielded insufficient NT edges. This limits the global applicability of the proposed automated method, as it may only be reliable for cities with low-rise, large-footprint layouts like those in the western US or Australia.
3. The reviewers should also consider the performance of the proposed method under realistic low-quality conditions, such as hazy or noisy images. In real-world scenarios, captured images often contain noise and other degradation, and it is important to understand how robust the method is in such situations. The authors should discuss the expected performance of their approach on low-quality inputs and, if possible, include experiments or analysis on hazy/noisy images. They should also review recent image enhancement and restoration methods, such as IHDCP, SLP, etc that are designed to handle such degradations, to position their work within the broader context and to suggest potential solutions for robustness.
4. To ensure a sufficient number of samples, the study uses a fixed EdgeLine of 21 pixels, even though the spatial quality estimators only converge at approximately 40 pixels. This intentional truncation introduces a systematic bias compared to the true performance of the satellite. Although the authors mention the theoretical possibility of a look-up table for compensation, no such correction was implemented or validated in this study
5. While the study concludes that Natural Edge Targets (NT) provide accuracy comparable to artificial targets, the Coefficient of Variation (CV) for NT edges is approximately two times higher than that of artificial targets. This inherent lower precision suggests that while NT edges are useful for operational efficiency, they may not yet provide the same level of statistical confidence as dedicated ground targets for high-stakes calibration.

Author Response

Thank you for your review comments, and please read the attached file.

Author Response File: Author Response.docx