Validation of AERONET-Estimated Upward Broadband Solar Fluxes at the Top-Of-The-Atmosphere with CERES Measurements

Round 1

Reviewer 1 Report

This study presented a thorough, statically meaningful comparison between AERONET TOA fluxes and CERES measurements. Since AERONET has become a major source for the validation of many atmospheric parameters, this topic is surely of the interest of the community. I don’t see any problem in the approaches used in this study, and the results are convincing. This paper is well organized. I suggest it be published with attention to following concerns:

line 191-200: I don’t quite get this. As I understand, the TOA albedo will depend on surface BRDF and the aerosol loadings in the atmosphere, which will be changing temporally. How the simulation take into account these changes? Or, do you have enough evidences show that this correction factor does not change much with surface/aerosol changes? In figure 3, 4, 5, the AOD scale is too large that one can barely see the changes in AOD, those plots should be re-created with proper scale.

 

Author Response

This study presented a thorough, statically meaningful comparison between AERONET TOA fluxes and CERES measurements. Since AERONET has become a major source for the validation of many atmospheric parameters, this topic is surely of the interest of the community. I don’t see any problem in the approaches used in this study, and the results are convincing. This paper is well organized. I suggest it be published with attention to following concerns:

line 191-200: I don’t quite get this. As I understand, the TOA albedo will depend on surface BRDF and the aerosol loadings in the atmosphere, which will be changing temporally. How the simulation take into account these changes? Or, do you have enough evidences show that this correction factor does not change much with surface/aerosol changes?

Reply: BRDF and aerosol load change with time, this is right. But as it is said line 199 of the original manuscript, the correction factor applied “simulates in average the TOA upward solar flux”. The important word is “in average”. It is not an instantaneous correction where each AERONET value would be recalculated at CERES SZA. For this reason and depending on the BRDF and aerosol load of each case, the instantaneous AERONET values corrected with the average correction factor might be higher, lower or similar to their corresponding CERES values.

In figure 3, 4, 5, the AOD scale is too large that one can barely see the changes in AOD, those plots should be re-created with proper scale.

Reply: New figures with a more appropriate AOD scale have been included in the revised manuscript. Only for the sites with AOD < 1: SEB, ALF, RIB, BRL, EGB and GSF.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments are attached in the pdf file

Comments for author File: Comments.pdf

Author Response

Sicard 2019 compare and validate the AERONET TOA flux with the help of CERES flux measurements at the TOA. By making use of AERONET L2 V3 data over the past 15 years from 8 different sites to respesent 4 different aerosol types: Mineral dust, Biomass Burning, Background Continental and Urban/Industrial. This study validated the AERONET TOA flux with CERES measurement and talks about the limitations on the AERONET TOA flux, which underestimate/overestimate over specific sites. Results, strongly suggest that there is a strong correlation between the co-located data points for TOA flux. And this study is first of it's kind, which makes this paper very interesting.

Minor edits

Line #046: onboard Terra to onboard NASA Terra satellite

Reply: Corrected.

Line #101: Spacing in this line should be corrected

Reply: Corrected.

Line #107: particular to particular,

Reply: Corrected.

Line #146: It is better to expand the location abbreviations plus, describe what each color represent (Aerosol types)

Reply: The color legend (as well as the equator line) disappeared during edition of the paper. Sorry about that. It has been added. The location abbreviations have been added in the caption for the sake of clarity of the figure.

Line #188: 60 min. difference to 60 min. A difference

Reply: This is not the beginning of the new sentence. The following modification has been made “…since a 60-min. difference at the hour …”.

Line #197: The two variables SZAaer and SZAcer, Are they the mean of all observation for a day?

Reply: They are the mean of all observations. One observation corresponds to a specific day and a specific time.

Line #223-226: What about the dust carried from Saharan desert over atlantic ocean and deposits over the amazon?, Does the Angstrom Exponent or Sphere fraction from AERONET data say something about the Saharan dust transport?

Reply: When strong Saharan mineral dust intrusions reach the Amazon, it is likely that the AE and sphericity may reflect such transport. But it is clearly out of the scope of the paper to investigate more deeply situations with more or less well determined aerosol mixing. The first comment of Reviewer 3 is about the representativeness of the sites selected and in relation with this comment. The following discussion has been added in the text in the first paragraph of Section 2.2. “It is important to note the inherent limitation of the aerosol type representativeness of the selected sites due to the selection process itself. In such conditions MD is probably more representative of North African mineral dust, BB of Brazilian biomass burning and BC of Canadian background continental.”.

Line #237: comma after According to [20]

Reply: Corrected.

Line #245: comma after aerosol size of [21]

Reply: Corrected.

Line #255: comma after In these cases

Reply: Corrected.

Line #276: Are the AOD and SA mentioned are mean for the data points in each PDF bins?

Reply: Yes, the following clarification has been added “(each point represents the mean in each PDF bins)”.

Line #301-302: In the abstract you mentioned AERONET version 3 data, but here you mention version 2, you might want to make it consistent.

Reply: I am talking of an improvement present in Version 3 but implemented in Version 2. The text has been modified as follows “One of the most important improvements in AERONET Version 3 algorithm (implemented in Version 2 already), regarding Version 1, …”.

Line #314-315: If we now look at the SA vs. flux difference, a general tendency is observed: AERONET fluxes are underestimated for large SA (> 0.2), i.e. bright surfaces This trend is not clearly supported by the data. You might want to rephrase it or remove it.

Reply: The linear regression line of SA vs. PDF is added in the following figure (red dash line) and in general large SA values tend to produce AERONET underestimations (negative Diff values).

Line #332: colon after to make because . Also these three reasons should have references

Reply: Corrected. The main reference is already given [24] Hsu et al., 2004.

Line #333-334: There is a clear advancement on the phase matrix calculation for non spherical particles over the last decade. So you can't say it as unknown, preferably say 'limited availability'.

Reply: Corrected as “the angular properties due to particle non-sphericity is also of limited availability”.

Line #353: 1 and it varies between 0.52 and 0.98 to `1: it varies between 0.52 and 0.98 I would encourage the author to improve the wordings used in the paper. But overall, this a really good study and publication worthy.

Reply: Corrected.

Author Response File: Author Response.docx

Reviewer 3 Report

The author in current manuscript presents his work on validating AERONET estimated upward broadband TOA fluxes using CERES measurements. 8 AERONET sites are selected for four typical type of aerosols. Spatial-temporal criteria are used to select CERES data for validation. It is found that the overestimations of AERONET correlate with large AOD, while the underestimations are linked to large surface albedo. The study is clearly presented and have good points. I recommend the publication of the manuscript with minor revision.

 

 

Major:

1. The author mentioned that the selection of the AERONET is global scale. However, for each type of aerosols, the two selected sites are usually in the same area such as Amazon, Canada or North Africa. The author could add discussions about the limits of the representative of validation.

 

2. In the all sites comparison, the author argue that the underestimation by AERONET is linked to the surface albedo (Sede Boker), while in the site by site analysis, we don't see any correlation for Sede Boker between the underestimation and the surface albedo. The bias for this site is actually very small. If the underestimation is caused by high albedo, the individual site should be strongly negatively biased ?

 

Minor:

1. Is the way of citing the references agrees with the journal's recommendation?

2. P3 L109: albedo table lookups or albedo lookup tables?

3. P3 L109: “based on” repeats for a second time.

4. P3 L116: after “editions”, better to add a comma.

5. P3 L122: “No Data Quality Summary”, these have to be capitalized?

6. eq. (1): could author explain how this equation comes and how it was used?

7. P4, L158: delete “that” after “too”?

8. P4, L160: I think it's not necessary to use “min.”, “min” will be fine.

9. P5, L181: please precise which RT model is used.

10: P8, L276: the AOD and SA are actualy mean AOD and mean SA?

11: P8, L286: “bias of -3.67” should be the intercept. It's confusing to use “bias” here.

Author Response

The author in current manuscript presents his work on validating AERONET estimated upward broadband TOA fluxes using CERES measurements. 8 AERONET sites are selected for four typical type of aerosols. Spatial-temporal criteria are used to select CERES data for validation. It is found that the overestimations of AERONET correlate with large AOD, while the underestimations are linked to large surface albedo. The study is clearly presented and have good points. I recommend the publication of the manuscript with minor revision.

Major:

The author mentioned that the selection of the AERONET is global scale. However, for each type of aerosols, the two selected sites are usually in the same area such as Amazon, Canada or North Africa. The author could add discussions about the limits of the representative of validation.

Reply: In order to be able to compare our results at the TOA to the results from Garcia et al. (2008) at the surface, we intended to use the same sites and to avoid as much as possible to introduce new sites. Another strong restriction for the site selection is the availability of AERONET V3L2.0 data for at least 7 years. For biomass burning for example, another strong source region is the sub-Saharan Africa. However, very few AERONET sun-photometers are running in this part of the world. The following discussion has been added in the text “It is important to note the inherent limitation of the aerosol type representativeness of the selected sites due to the selection process itself. In such conditions MD is probably more representative of North African mineral dust, BB of Brazilian biomass burning and BC of Canadian background continental.”.

In the all sites comparison, the author argue that the underestimation by AERONET is linked to the surface albedo (Sede Boker), while in the site by site analysis, we don't see any correlation for Sede Boker between the underestimation and the surface albedo. The bias for this site is actually very small. If the underestimation is caused by high albedo, the individual site should be strongly negatively biased ?

Reply: As one can see on Fig. 4b in Sede Boker SA is relatively high and constant and the PDF is both positive and negative, although slightly shifted towards the negative values (Diff = -5.87 Wm^-2). When Sede Boker points are added on the all sites comparison, the majority of points with large negative values are from Sede Boker, thus SA is high. In turn, the points with large positive values are associated to SA values which are the mean of high (Sede Boker) and not so high (BRL, GSF and BEI) instantaneous SA values. The conclusion is that large underestimations of AERONET TOA fluxes is observed at sites with large SA, almost exclusively, although such sites may also produce overestimation in certain conditions.

Minor:

Is the way of citing the references agrees with the journal's recommendation?

Reply: Yes.

P3 L109: albedo table lookups or albedo lookup tables?

Reply: “albedo lookup tables”. Corrected. Thanks.

P3 L109: “based on” repeats for a second time.

Reply: “based on” has been replaced by “derived from”.

P3 L116: after “editions”, better to add a comma.

Reply: Corrected.

P3 L122: “No Data Quality Summary”, these have to be capitalized?

Reply: It has been left as “No data quality summary”.

eq. (1): could author explain how this equation comes and how it was used?

Reply: The explanation is actually already given in the text. The overall uncertainty is the sum of:

the mean all-sky SW TOA flux bias (-0.20 Wm^-2) the RMS error (±0.91 Wm^-2) the relative RMS errors due to the difference between nadir-viewing and oblique-viewing [50 – 60º] in clear-sky conditions (±4.2 % of )

The sum of all three error sources is given by Eq. 1.

P4, L158: delete “that” after “too”?

Reply: Corrected.

P4, L160: I think it's not necessary to use “min.”, “min” will be fine.

Reply: Corrected here and throughout the text.

P5, L181: please precise which RT model is used.

Reply: The following information has been added “…, namely GAME (Global Atmospheric Model) described in [17,18],…”.

10: P8, L276: the AOD and SA are actualy mean AOD and mean SA?

Reply: Yes. Reviewer 1 had also the same doubt. The following clarification has been added at the end of the sentence“(each point represents the mean in each PDF bins)”.

11: P8, L286: “bias of -3.67” should be the intercept. It's confusing to use “bias” here.

Reply: It is true that -3.67 is found as the intercept of the best fit line on the y-axis. However the writing of the equation of the best fit line as y=a·x+b assumes a bias b defined as the intercept of the equation on the y-axis: en x=0, y should be theoretically 0, but it is not because there a systematic bias b=-3.67 exists. The term bias has been maintained in the revised manuscript.

Author Response File: Author Response.docx

Reviewer 4 Report

General comments:
It is recommended to clarify and add the explanation as followings before publication:
The manuscript discussed the validation effort of AERONET-estimated broadband solar fluxes at the TOA with reliable CERES measurement.

It is assumed that the purpose of the manuscript is "just to validate and provide results". If so, it is suggested to provide uncertainty analysis:

1) The authors described the results but there is no uncertainty analysis, therefore no conclusion. The reader can find the detailed description of the inter-comparison using only scatterplot with mean biases, slope and correlation coefficients. However, can not find the systematic analysis where the difference comes from.
2) It is not clear what is exact meaning of the conclusion, i.e., the agreement of -15%~15%. The AERONET-estimated broadband solar fluxes at the TOA can be used where, such as climate study and so on?
3) The uncertainty analysis (i.e. quantitative uncertainty analysis) needs to include the effects:
A. aerosol optical properties (at least AOD, SSA-aerosol absorption is the main property that can control the amount of downward/upward flux calculation)
B. surface properties (surface albedo and its dependence on direction-BRDF)
C. Downward flux at surface needs also to interpret the effects of surface albedo in analysis of the accuracy of upward flux.

If the main purpose of the manuscript is not just to validate and provide results of comparison, please provide clearly.

Minor comments:
The numbers indicate the line number through the manuscript.
1) 143: Table 1 includes the date range of AERONET, and the temporal collocation period between AERONET and CERES measurements needs to be considered. The manuscript explained the only longer AERONET observations, but it needs to consider longer temporal collocation between AERONET and CERES measurements.
2) 177: during the time difference such as 5 minutes and/or 60 minutes, do you consider the any change of aerosol optical properties and it effect to estimate upward flux at the TOA?
3) Figure 4: AODs in Figure 2 and in Figure 5 are different from each other. It means that MD in Figure 2 shows relatively small AOD and peak AOD compared with those of BB that show the large peak AOD and large seasonal variations. But in Figure 5, BB shows quite small AOD (due to the y-axis ticks I can find the approximate values of AOD), and MD shows large AOD. Of course there are difference between monthly mean values and instantaneous values. But if one of the purposes of the manuscript is to show the inter-comparison of flux (TOA) difference according to different aerosol types, it needs to consider the characteristics of aerosol types.
4) 453: …. pointing out the limitation of the "average" SZA correction performed…
What is the meaning of the limitation of the SZA correction? Underestimation or overestimation?

Author Response

It is recommended to clarify and add the explanation as followings before publication:
The manuscript discussed the validation effort of AERONET-estimated broadband solar fluxes at the TOA with reliable CERES measurement.

It is assumed that the purpose of the manuscript is "just to validate and provide results". If so, it is suggested to provide uncertainty analysis:

1) The authors described the results but there is no uncertainty analysis, therefore no conclusion. The reader can find the detailed description of the inter-comparison using only scatterplot with mean biases, slope and correlation coefficients. However, can not find the systematic analysis where the difference comes from.
2) It is not clear what is exact meaning of the conclusion, i.e., the agreement of -15%~15%. The AERONET-estimated broadband solar fluxes at the TOA can be used where, such as climate study and so on?
3) The uncertainty analysis (i.e. quantitative uncertainty analysis) needs to include the effects:
A. aerosol optical properties (at least AOD, SSA-aerosol absorption is the main property that can control the amount of downward/upward flux calculation)
B. surface properties (surface albedo and its dependence on direction-BRDF)
C. Downward flux at surface needs also to interpret the effects of surface albedo in analysis of the accuracy of upward flux.

If the main purpose of the manuscript is not just to validate and provide results of comparison, please provide clearly.

Reply: The author is aware of the potential limitations (few sites, few points per sites, etc.) of the study to be taken as a validation. This potential limitation is already mentioned in the introduction of the original manuscript: “The purpose of this article is to perform a validation, or at least to present a statistically representative estimation of the quality, of AERONET-estimated TOA upward solar fluxes by comparing them to satellite measurements.”. The author has kept the word “validation” in the revised manuscript, but, if the reviewer thinks it is not appropriate, the author is willing to change the wording (“comparison”, “estimation”, or similar). However, the author thinks that at least a statistically representative estimation of the quality of AERONET-estimated TOA upward solar fluxes has been achieved, and that including more in-land sites (coastal sites are problematic when observed from space) will likely not change the actual results.

As far as the uncertainty analysis is concerned, CERES fluxes are given with their uncertainty (see Eq. 1 and the detailed explanation given in the answer of comments #6 of Reviewer 3; and also vertical error bars in Fig. 3, 4 and 5). AERONET estimations are given without any type of error or uncertainty, and it is the magnitude we want to compare/evaluate in order to know how close it is from the truth. AERONET flux estimations are the output of an algorithm which is not modifiable, at least immediately. Here none of the traditional Monte Carlo or error propagation methods can be applied to know the sensitivity of some properties on the AERONET flux estimation. So, to the author’s understanding of what an uncertainty analysis is, such an analysis can not be performed for the AERONET flux estimations. Nonetheless the sensitivity of the difference CERES-AERONET with properties of interest such as AOD (point A of the reviewer) and SA (point B) can be addressed and are actually given in the discussion presented in the paper. To the author’s understanding this is not strictly speaking an uncertainty analysis but a sensitivity analysis. The sensitivity of SSA on the fluxes (point A), although it would help a lot the analysis of the results as mentioned by the reviewer, can not be included at this point. This is motivated by two reasons: 1) Garcia et al. (2008) who validated AERONET fluxes at the surface found no significant sensitivity of the fluxes with the single scattering albedo (SSA), and, more restrictively, 2) usually the AERONET SSA inversion corresponding to the time-space co-located AERONET-CERES measurements is not available (as an example, among the 93 points at Sede Boker, none of them is inverted in terms of SSA; for the 75 points at GSFC, no SSA inversion is available as well). The BRDF (point B) models used either in CERES retrievals or in AERONET inversions can not be obtained easily and the authors think that the effect of BRDF on both CERES retrievals and AERONET inversions is/should be the subject of other papers. Point C of the reviewer (effects of SA on downward flux at surface) is already addressed in Garcia et al. (2008; reference [4] in the paper) and frequently referenced in our work.

Minor comments:

The numbers indicate the line number through the manuscript.
1) 143: Table 1 includes the date range of AERONET, and the temporal collocation period between AERONET and CERES measurements needs to be considered. The manuscript explained the only longer AERONET observations, but it needs to consider longer temporal collocation between AERONET and CERES measurements.

Reply: Table 1 is dedicated to AERONET data only. CERES date ranges are included in the text in Section 2.1.2 (line 99-100 of the original manuscript): for CERES/Terra 01/03/2000 – 31/10/2018, for CERES/Aqua 03/07/2002 – 31/10/2018, and for CERES/S-NPP 27/01/2012 – 31/12/2018. The temporal co-location has been searched between both CERES and AERONET databases.

2) 177: during the time difference such as 5 minutes and/or 60 minutes, do you consider the any change of aerosol optical properties and it effect to estimate upward flux at the TOA?

Reply: For both 5 and 60 min time difference, the possible change in aerosol optical properties is not considered since a fundamental hypothesis to compare both magnitudes is that they are representative of the same aerosol/surface scene. The following sentence has been added in the Section 3.2 and in the conclusions: “Some of this variability is caused by the possible change in the aerosol optical properties due to the time difference (5 and 60 min) between both datasets.”.

3) Figure 4: AODs in Figure 2 and in Figure 5 are different from each other. It means that MD in Figure 2 shows relatively small AOD and peak AOD compared with those of BB that show the large peak AOD and large seasonal variations. But in Figure 5, BB shows quite small AOD (due to the y-axis ticks I can find the approximate values of AOD), and MD shows large AOD. Of course there are difference between monthly mean values and instantaneous values. But if one of the purposes of the manuscript is to show the inter-comparison of flux (TOA) difference according to different aerosol types, it needs to consider the characteristics of aerosol types.

Reply: The y-axis scale of the AOD plots of Figures 4 and 5 has been adjusted for the sites with AOD<1 for better visualization of the results (see also the answer to comment #2 of Reviewer 1). For MD, Fig. 2 and 4c/d show similar AOD values. The special case of the BB category (underlying the difference observed by the reviewer between monthly (Fig. 2) and instantaneous values (Fig. 5g/h)) is drawn to the reader’s attention in the first paragraph of Section 3.1 of the original manuscript (“An important note has to be made at this point. As it can be contrasted from Figure 2b and Table 3, the coincidences at the BB sites occur between April and August and are thus nearly outside of the BB season which is from August to December. In such conditions the coincidences at the BB sites are not going to be representative of BB but rather of Brazilian forest without nearly any BB influence.”). There is nothing we can do about this under-representativeness of the BB sites, at least in this part of the world, since it all relies on the orbit of the satellite.

4) 453: …. pointing out the limitation of the "average" SZA correction performed… What is the meaning of the limitation of the SZA correction? Underestimation or overestimation?

Reply: The word “average” is important here. The SZA correction is made on average, which means not on a point-by-point basis. As a consequence, the SZA difference is for some points overcorrected and for some others under-corrected. This produces in relative terms, i.e. no matter the sign, a larger variability: 2-6 % for the sites where the SZA correction is applied vs. < 1% for the sites where the SZA correction is not applied.

Author Response File: Author Response.docx

Round 2

Reviewer 4 Report

Thank you very much for the clear explanation. I can find the reasonable explanation with your reply. Only one thing is that the author need to add the more clear sentence regarding AERONET TOA flux estimation data.

Please include the explanation that AERONET TOA flux estimation data used in the paper are not given any type of error or uncertainty as it is the output of an algorithm which is not modifiable in Session 2.1.1 as the authors mentioned below.

------------------------------

You replied that "AERONET estimations are given without any type of error or uncertainty, and it is the magnitude we want to compare/evaluate in order to know how close it is from the truth. AERONET flux estimations are the output of an algorithm which is not modifiable, at least immediately."

 

Author Response

We thank again Reviewer #4 for his/her review.

Section 2.1.1 has been modified as suggested by the reviewer. Specifically, the following sentence has been added at the end of Sectioni 2.1.1: “AERONET flux estimates are the output of an algorithm and their uncertainty is not known as it has never been assessed by the AERONET team or collaborators.”.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.