Evaluation Analysis of Landsat Level-1 and Level-2 Data Products Using In Situ Measurements

Round 1

Reviewer 1 Report

Reviewer comments on “remotesensing-861977”

Evaluation Analysis of Landsat Level-1 and Level-2 Data Products Using In-Situ Measurements by Pinto et al. is an interesting and well-written manuscript. This manuscript shows the validation of TOA and BOA (surface reflectance) from Landsat ETM+ and OLI data against in situ measurements collected for heterogeneous surfaces. This study showed good agreement between Landsat and in situ data at longer wavelengths compared to the shorter wavelengths (visible) and this could be due to error in the aerosol retrievals as claimed by the authors. I have the following serious issues which need to be addressed:

1. The authors claimed that low agreement for level 2 data between in situ and Landsat data is mainly due to error in aerosol retrievals but they have not prived it or have not shown any evidence. The authors should consider AOD (aerosol optical depth) data and validate L2 data for different levels of AOD. In other words, the authors need to perform a sensitivity analysis for AOD.
2. Figures 1 to 8:
   1. In situ measurements of BOA greater than TOA reflectance, why?
   2. For some sites, BOA is less than TOA only for blue and green wavelengths, why?
   3. For some sites, the difference between BOA and TOA is high (and low), why?

Theoretically, BOA should be less than TOA reflectance – If you look at Equation 1 (a & b) for Radiative Transfer Model (6SV) in Vermote et al. (2016, RSE) [Vermote, E.; Justice, C.; Claverie, M.; Franch, B. Preliminary analysis of the performance of the Landsat 8/OLI land surface reflectance product. Remote Sens. Environ. 2016, 185, 46–56] which shows that TOA reflectance is the combination of “atmospheric path reflectance” and “surface reflectance”. Simply, TOA = Atmospheric reflectance + surface reflectance and for surface reflectance = TOA reflectance – Atmospheric reflectance. This relationship shows that surface reflectance (BOA) should always be less than TOA reflectance. This relationship also satisfies the definition of atmospheric correction; i.e., removal of atmospheric contribution from TOA (TOA – atmospheric path reflectance) to get surface reflectance (BOA). Therefore, the theoretical explanation for (a), (b), and (c) is required.

3. It is quite easy to get a high correlation and a well-fitted line (slope) by using all wavelength data together or dark and bright surfaces data together. If you go for individual land type, then results might be different as I can see for validation of TOA and BOA. The authors should show separate validation for TOA and BOA for each site, and report correlation. I can see statistics for individual sites in Tables 2 to 5, but the correlation is missing. It would be interesting to see the correlation between each site.
4. It would be great if the authors explicitly mention the motivation of this study.
5. All the in situ data used should be made available for reviewers and readers and can be added to the supplementary data.

 

Author Response

Please, find attached a document containing the authors responses. Thank you.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper presents a radiometric performance evaluation of the Level-1 16 and Level-2 data products for the Landsat-7 ETM+ and Landsat-8 OLI sensors using in-situ measurements from eight test sites. Please consider following comments.

 

1. In-situ measurements were compared with L7 ETM+ and L8 OLI data. Please provide details about preprocessing of in-situ spectral data (e.g., spectral interpolation, if applied) and processes for convolving in-situ data into satellite data bands (e.g., applying spectral response function).

 

2. Both wavelength units, micrometers and nanometers, were used together, e.g., nanometers in figures 1-8, and micrometers in descriptions for the figures (e.g., lines 228, 265 and 307). Please unify wavelength units indicating identical spectra.

 

3. Is it correct that the wavelength of ASD measured spectra starts from 250 nm in line 220?

 

4. There are some typographical errors, e.g., lines 114, 126 and 443. Please correct typographical errors in overall manuscript.

Author Response

Please, find attached a document containing our responses. Thank you.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Second review of “remotesensing-861977”.

The authors were asked the following questions in the first review – they only added the theoretical explanation which is already well-known. But, they have not provided any evidence of their claim by adding calculations and solving the equation for BOA.

For example, for (1): the authors were asked to prove that the low agreement between BOA and TOA is due to AOD, and to prove it the authors must use different levels of AOD, instead, the author shave checked the impact of AOD over TOA.  Of course, if AOD is increased, then TOA will be increased due to more scattering but this does not confirm that the low agreement is due to AOD. Again, I would suggest to use AOD retrievals for the corresponding dates of BOA and TOA, sort the data with different AOD levels, and performed the analysis.

For (2): the authors have replied that the relationship between TOA and BOA is not linear. They also have mentioned general theoretical information about scattering due to molecules and aerosols. According to the authors, if there are high aerosol loadings in the atmosphere or atmosphere is clear, then the transmission will be decreased or 1, respectively. For high aerosol loadings, TOA will be greater than BOA, however, for clear sky, BOA = TOA. But, the authors have not explained when BOA will be greater than TOA – According to my experience and understanding, by solving Equation (1) (Vermote et al., 2016) for BOA, there are two possibilities which the authors have mentioned; i.e., either BOA < TOA during high aerosol loading, and BOA = TOA for the clear sky (transmission = 1). Therefore, it is recommended to solve the equation (1) for BOA and show that BOA can be greater than TOA and mentioned those specific conditions. This should explain why for BOA > TOA or BOA < TOA for specific sites for the shorter wavelength.

For (2), (3), and (4): the authors have provided general information – As mentioned above for (2), the authors need to explain by solving the equation – why for the same wavelength, BOA becomes less than or greater than TOA. For (4), the authors replied that due to more scattering in blue and green, TOA is greater than BOA, but they have not explained that for the same wavelengths (blue and green), for some sites, BOA is greater than TOA – as aerosol scattering is high for these wavelengths.

The following issues are valid and important that need to be explained and addressed in the revised versions.

Major concerns (also raised in the first review):

1. The authors claimed that low agreement for level 2 data between in situ and Landsat data is mainly due to error in aerosol retrievals but they have not prived it or have not shown any evidence. The authors should consider AOD (aerosol optical depth) data and validate L2 data for different levels of AOD. In other words, the authors need to perform a sensitivity analysis for AOD.
2. Theoretically, BOA should be less than TOA reflectance – If you look at Equation 1 (a & b) for Radiative Transfer Model (6SV) in Vermote et al. (2016, RSE) [Vermote, E.; Justice, C.; Claverie, M.; Franch, B. Preliminary analysis of the performance of the Landsat 8/OLI land surface reflectance product. Remote Sens. Environ. 2016, 185, 46–56] which shows that TOA reflectance is the combination of “atmospheric path reflectance” and “surface reflectance”. Simply, TOA = Atmospheric reflectance + surface reflectance and for surface reflectance = TOA reflectance – Atmospheric reflectance. This relationship shows that surface reflectance (BOA) should always be less than TOA reflectance. This relationship also satisfies the definition of atmospheric correction; i.e., removal of atmospheric contribution from TOA (TOA – atmospheric path reflectance) to get surface reflectance (BOA). Therefore, the theoretical explanation for (a), (b), and (c) is required.
3. In situ measurements of BOA greater than TOA reflectance, why?
4. For some sites, BOA < TOA and BOA > TOA only for blue and green wavelengths, why?
5. For some sites, the difference between BOA and TOA is high (and low), why?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The revised paper considered overall comments and suggestions, so I do not suggest further comments.

Author Response

Thank you for the suggestions and comments.

Round 3

Reviewer 1 Report

Again, the authors were failed to explain "why BOA is greater than TOA" at visible channels. Figure 1 added by the authors in the response file does not show the case which was asked (why BOA > TOA at visible channel). Figure 1 shows that BOA is less than TOA for both AOD values, and this is quite normal. However, the authors were asked why their data are showing BOA > TOA for the visible bands, but the authors were unable to explain it. Anyway, overall, the author's responses are not satisfactorily and did not explain the different behavior of BOA at visible bands for different regions.