Capability of Jason-2 Subwaveform Retrackers for Significant Wave Height in the Calm Semi-Enclosed Celebes Sea
Round 1
Reviewer 1 Report
Overall, the paper is much improved. However, see discussion below on Table 1 and Fig 11. The discussion in Sec 4.1 I think that this discussion needs to be pointed out in the paper with Table 1, so that readers are aware of the issues. The conclusion that you need >~uncontaminated trailing gates to get good results is very useful. Also, note the new comment on fitting lines to noisy data (I am sorry that I did not think of this earlier.)
l 109: “post[-]processing” is not the right word to describe WIW19. The “post-processing” part is that it uses a series of WF to determine the trailing edge length. It is really a “non-local adaptive” retracker whereas ALES is local in using only the current swh (is this true?) to set the length of the trailing edge. ALES could probably be improved by using a small window with average or median to set its window so it is less “distracted” by bad values, whereas WIW19 has incorporated this in the basic design. This is correctly described in l 268.
L 130, Table 1: I will repeat my criticism of the previous versions that allowing sigma0 up to 30 dB is guaranteed to get some bad data from AGC blooms into the dataset. This limit needs to be lowered to at most 24 dB (see below) to exclude the most contaminated SGDR values. This will make a lot of the pale blue points in Fig 6 disappear. Authors comment in l 263 indicates they know this is a problem. I have revised the limit here as I realized that the previous comment was for TOPEX which has sigma0 nearly 3 dB lower than the Jasons, so a limit of 20-21 dB would be appropriate. Examination of Fig 11 (an excellent addition) shows that a limit of 22 dB would get the main body of data and insure 15 UTE gates. It would seem to me that the conclusion from Figs 9, 10, Tables 4, 5 is that you definitely need >~15 UTE to get good results. The study in this section should be used to select the limit for using data. The UTE value needs to be translated to sigma0 so that “regular” users can apply it. Whether the limit used in the paper is 20 (probably cleaner) or 22 dB is up to the authors, but it would have a clear basis. Perhaps another paper studying the length of UTE and 3 vs 4 parameter fits is warranted.
I disagree with the authors’ response saying a reduced limit does not improve the results in Fig 6; the revised Fig 6 looks just as I would expect. Upping the limit to 22 dB will get a fair amount more data but avoid many of the UTEs < 15.
L 218-224: How are the lines in Fig 6, 8, Table 2,3 fit? Given the number and large distance of outliers a more robust method than least squares is needed. When two datasets each with significant errors are fit, a method that takes account of the errors in both is needed. As commented on Table 1, the fits involving SGDR are not really valid as clearly bad data have been included. Also, based on these figures the authors would be justified in more forcefully pointing out that ALES does not give reliable results for swh < 0.5m (actually no method really does, but ALES complete lack of returns for those values would seem to be a problem).
L 325: The suggestion that the fitting method results in discrepancies as large as those shown in Fig 10b needs additional support. The sentence should be reworded to be more tentative.
L 400 and following: I do not see this in the figures or understand the comment. Fig 13, 14 both show the shift in WW3 without nesting.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
GENERAL REMARKS:
This study gives a detailed comparison and validation of Jason-2 subwaveform retrackers for wave height in the Celebs Sea and its region. Results show a fair comparison and relationship between different retrackers and model being used.
Better performance given by the retrackers use versus just SGDR data in the coastal areas, gives encouragement that retrackers are important, and can contribute to altimeter enhancements, with a significant difference, and perform better when approaching the land areas. This offers useful insight to present existing altimetry data, in spite of still existing limitations.
There are a few minor corrections and comments listed below that authors could address and correct before their final submission of this manuscript.
DETAILED REMARKS:
#1:
Line 66:
“WIW19”
Please give a full definition of acronyms being used for the first time.
#2:
Line 68-69:
“Use of these retracking algorithms have been reported significantly improved coastal altimetry measurements.”
This sentence needs to be corrected to something like:
“Use of these retracking algorithms have been reported to significantly improve coastal altimetry measurements.”
#3
Line 71-73:
“In coastal areas where spatial scales are generally small, it is unrealistic to expect spot buoy measurements, along-track altimeter measurements and gridded wave model results are all compatible with each other.”
This sentence needs to be corrected to something like:
“In coastal areas where spatial scales are generally small, it is unrealistic to expect spot buoy measurements, along-track altimeter measurements and gridded wave model results to be all compatible with each other.”
#4
Line 155-157:
“On the other hand, [17] reported that agreement between 1/20° WW3 model and quality-controlled altimetry SWH data [2] is good even the Indonesian Seas are included.”
This sentence needs to be corrected to something like:
“On the other hand, [17] reported that agreement between 1/20° WW3 model and quality-controlled altimetry SWH data [2] is good even when the Indonesian Seas are included.”
#5
Line 329, page 13:
Missing in Figure 10: panel (b).
#6
Line 364-366:
“The good agreement with WW3 with altimetry SWH data in the calm Celebes Sea would encourage us further studies on wave dynamics and WW3 model representability in semi-enclosed seas.”
This sentence needs to be corrected to something like:
“The good agreement of WW3 with altimetry SWH data in the calm Celebes Sea would encourage us to further study on wave dynamics and WW3 model representability in semi-enclosed seas.”
#7
Conclusions section looks a bit short, might be extended to address major results from this study.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
The capability of Jason-2 subwaveform retrackers ALES and WIW19 are analyzed in this manuscript, and the Jason-2 SWH of these two retrackers are compared to SGDR which is the standard product of Jason-2, and the results of WW3. There are two main works in this manuscript. One is the comparisons of Jason-2 SWH between ALES, WIW19 and SGDR. The other is the comparisons between two subwaveform retrackers, SGDR and WW3.
- The comparison without accurate reference standards is meaningless. For Jason-2 SWH of ALES, WIW19 and SGDR, The qualitative results obtained in this study are well known, such as the SWH of subwaveform retrackers are better than that of SGDR. The comparisons between ALES, WIW19 and SGDR can not conclude the results which one is better between ALES and WIW19. The comparisons only show their differences.
- SWH of WW3 model is used in this study, but the accuracy of the WW3 model results has not been confirmed, so the comparisons are meaningless. The analyses between Line 151 -161 are not enough. The excellent agree between WW3 and altimeters in open ocean does not mean the good results of WW3 in the Celebes Sea. The (1/20)° WW3 results show good agreement between WW3 and altimeters, but the spatial resolution of WW3 used in this study is (1/12)°. The accuracy of (1/12)° WW3 results are not confirmed. The SWH data of buoy or other in situ measurements is needed.
- Supplement the descriptions of two subwaveform retrackers.
- The mean standard deviations of WW3 model in Line 182 is much smaller than that of ALES, WIW19 and SGDR?
- Why there is no SWH data less than 0.4m for ALES (Line 195) ?
- Line 358, why the improvements for WIW19 are insignificant? If the WW3 model results are accuracy, does it mean that WIW19 is worse than ALES?
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
- The results of the comparisons between ALES and WIW19 should be concluded clearly. The comparisons of different methods which are just to give their differences or some well-known results is no meaningless.
- Wave hindcasts of WW3 should not be used to check the consistency of these Jason-2 SWH data with wind fields (Line. 136). It should be used to evaluate the capability of Jason-2 subwaveform retrackers for SWH. So the accuracy of WW3 results should be proven firstly. The numerical simulation results of ocean wave by WW3 are influenced by the input of wind fields. In this study, WW3 simulation results need to be proven to be good accuracy. It can be compared to the reanalysis of ECMWF.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Main points
- I am missing how you define "coastal" measurements. What is your distance2coast of your investigated data points? I would not assume, that you have over 32000 points in a typical coastal scenario of about <10km (or so)
You explain that these "coastal" conditions are somehow apparent in the whole area, but you might need to prove this. I would assume that very close to the shoreline, the results are worse than in the center of the Celebes Sea
- Section 2 completely misses the source of your material, i.e. your retracked data. Where did you get the SGDR, which baseline version is it? Although this might be self-explanatory, it would be good practice. Where did you get the ALES processed data from? There are also different version of ALES algorithms out, that might show different performances. Added after reading the paper: Ah ok, now I see that Marcello Passaro kindly provided you the ALES data. Anyway, which version (from which year) of ALES is it?
- I do not believe that ALES is not capable of estimating low SWHs (or suppresses large SWHs). Its performance in low SWH scenarios might not be optimal (decreasing sampling resolution for very steep slopes of the leading edge) but there should be some values. Added after reading:
- "In addition, low SWH values in ALES dataset could have been inherently filtered out in this study." It is very unclear what the filtering actually does. You would need to demonstrate, why ALES should not be capable of estimating low SWHs
- There is new version of ALES out, named WHALES, as been evaluated in Schlembach2020. Why do you compare the WIW algorithm with an older version of ALES?
L19-20: "Coastal retracking algorithms, or so-called subwaveform retrackers, well suppress extraordinarily large SWH, but the ALES retracker fails to 20 observe SWH smaller than 0.5 m,"
- By this statement you say that all coastal retrackers are subwaveform retrackers, but this is not true, e.g. SAMOSA+ is also a coastal retracking but does not follow a subwaveform approach.
- I think one cannot say that coastal retracker suppress large SWH, why should they want to do that? Also I do not believe that ALES fails to observe small SWHs completely.
L26: typo Wavewath III
L30: intensity, more specifically its the power
L31: be more precise, which delay exactly corresponds to the distance? speaking about times is quite general
L33: tops -> crests
L39: Satellite altimeters are per se were originally designed to measure the sea level, not speaking about coast or open-ocean. The Brown-Hayne model is originally designed for open-ocean. The latter statement is more applicable.
L48: What does WIW stand for?
L49: What means "uniform reflectance". How does WIW measure it? Maybe you could elaborate a bit more about the algorithm details of WIW, since this is a main part of this work
L56: Explain the content of Section 2 first
L73: "at a single point" is not very clear, better write process each waveforms independently from each other (or so)
L75: this is very unclear. "spatially-consistently"?!
L76: What kind of "obvious errors"?
L77: I would assume that commonly-used quality control filters are briefly explained here
L105: typo SHW
L109: You should also plot the SWH distribution of the used WW3 model, although this subsection is dedicated to the intercomparison among three algorithms. Later on, there is no histogram of the WW3 values, so I would show them here.
L119: much more significant is lingustically more correct
L130: This is just a nice to have: A density plots would be nicer to get better idea of the frequency of occurence of the values, e.g. as in Abdalla2018, Fig. 4
L133: How come that you have so many less values for the scatter plots (around 33000) as compared to the histogram data (around 1.8 million)?
L168: This is too much of speculation. How could you know how the SGDR retracker actually performs so it estimates smaller SWHs? You would need to show an example to prove your assumption. Otherwise leave out this sentence.
L171: Again I am missing the definition about the unifrom reflectance, as already I already mentioned in L49. This seems to be crucial for the better performance of WIW, so you should elaborate it in this work a bit more. You should not expect the reader to study the other reference for understanding the main idea of the WIW algorithm.
L178: In the abstract your conclusion is that ALES is not capable of estimating low SWHs. In this sentence you say that low SWHs "could inherently" been filtered out. What is correct? The filtering seems to be a crucial part of this study, since you deduct many things from results of the filtering. You should elaborate on the methodology of filtering.
L207: Fails to observe small SSHs? I suppose you were talking about SWHs throughout the whole study?
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Reviewer 2 Report
This study inter-compares the performance of three altimeter waveform retracker algorithms for retrieving significant wave height in a semi-enclosed sea - the Celebes Sea - where the (low) sea state conditions are mostly dominated by wind seas and where sea slicks may occasionally corrupt altimeter measurements. Despite the significance of this topic for Remote Sensing readership, I think the current manuscript is not mature enough for a publication in this journal. My main critics concerns the following aspects:
1) The main result of this study concerns the better performance of the retracker algorithm developed by one of the co-author (Wang et al., 2019) for retrieving significant wave height, which is not very surprising since this algorithm makes uses of neighboring waveforms in order to determine an optimal subwaveform window size while the two other algorithms correspond to single waveform processing.
2) The authors do not provide any information on the performance of the model in the studied region so that one cannot conclude anything from the comparisons between altimeter records and model outputs. I understand that in-situ data and quality controlled altimeter measurements are lacking in this region, but the author could at least provide model skills for the global model and for similar nested regional implementation of the WW3 model. I also invite the author to consider triple collocation techniques (e.g. two altimeter missions and one model results) in order to get estimate of both model and altimeter accuracy.
3) The ALES retracker (Passaro et al., 2016) is used in this study while there has been, since then, improved versions of this algorithm, called ALES+ (Passaro et al. 2018) and WHALES (Schlembach et al., 2020). See also Schlembach et al (2020) for a detailed comparisons of current state of the art LRM and DD altimeter retrackers performance.
4) The analysis presented in this paper is extremely succinct (namely 3 histograms of the altimeter SWH, 6 scatter diagrams of altimeter vs. altimeter or model results, 2 tables with error metrics) and should be expanded to support the conclusion of the study (e.g. outlier and quality flag analysis, triple collocation analysis...)
5) The second main results of this study corresponds to the detection of swells in the Celebes Sea. This result is based on the comparisons between altimeter measurements and the results of two model simulations with and without wave forcing at their boundaries. Similar conclusions could have been reached by comparing directly the model results of the two model configurations. Clearer conclusions could be drawn by comparing the spatial distribution of the errors between the two model configurations for spectral parameters including wave directions and wave period.
References
Passaro, M., Rose, S.K., Andersen, O.B., Boergens, E., Calafat, F.M., Dettmering, D., Benveniste, J., 2018. ALES+: Adapting a homogenous ocean retracker for satellite altimetry to sea ice leads, coastal and inland waters. Remote Sensing of Environment 211, 456–471. https://doi.org/10.1016/j.rse.2018.02.074
Schlembach, F., Passaro, M., Quartly, G.D., Kurekin, A., Nencioli, F., Dodet, G., Piollé, J.-F., Ardhuin, F., Bidlot, J., Schwatke, C., Seitz, F., Cipollini, P., Donlon, C., 2020. Round Robin Assessment of Radar Altimeter Low Resolution Mode and Delay-Doppler Retracking Algorithms for Significant Wave Height. Remote Sensing 12, 1254. https://doi.org/10.3390/rs12081254
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Reviewer 3 Report
The paper cannot be published without error estimates, discussion on altimeter swh.
Specific comments:
P 3, l 84: Mention that inner nest of wave model is the area shown in Fig 1.
P 2, l 77: Some details of “quality control filters” must be given (not just Ref to [6]).
P 3, ll 87-96: Rearrange these sentences into a more coherent description of model input (ll 94-96), parametrization(s), assumptions, etc. The current arrangement does not seem optimum.
P 3. ll 99-100: Sentence says average over 50 20 Hz data to get 1.8 km separation. This is not right. 50 20 Hz pt gives 2.5 sec ~ 18 km, not 1.8 km. 1.8 km corresponds to about 5 20 Hz points. The values would likely be pretty noisy and are likely correlated because of tracker window update.
P 4, Fig 2: Why was 0.25 m selected as bin size? Show at least one set of along track results. Discuss noise level once averaging interval in previous comment is corrected/explained. Make font on N=# much bigger.
P 5, Fig 3; P 6, Table 1: Include direct comparison of ALES/SGDR.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Regarding the WHALES comment:
>> The reason is simply that I could not find WHAES data for 20 Hz Jason-2. Also, ALES has already been widely distributed so that there would be many users.
You could have asked Marcello Passaro to provide you with 20 Hz data from Jason-2. This would thus been a more fair comparison with his actual version of his retracker and provided an added value to your paper.
L42: "As schematically shown in Fig. 1a, observed 42 waveforms are generally so perturbed that so-called “retracking” processes are necessary to fit an 43 idealized model to the real waveform." Retracking is not just performed because the waveform is perturbed. It is the general process on extracting geophysical parameters from the power echo return signal.
L77: It is not clear to me, how this echogram is generated. Are the rows correspond to the individual 20-Hz measurements?
L108: "then fit the Brown model" correct grammar of sentence, please reformulate
Reviewer 3 Report
Full comments pasted below, but in them I state that the paper should not be published without the authors revisiting their data selection criteria. If, after researching this, they wish to proceed with the selection criteria as they are, the paper needs to explain the difference(s) from the standard criteria.
The paper is much improved, but I still note some difficulties with some of the explanations.
Minor grammar throughout: Fix “agreements”.
Fig 1: The explanation of the radargram is partly incomplete and partly misleading. The waveform in (a) would be not seen at the point indicated as there is not enough power in the leading edge gates. Waveform (a) would be what is seen just above the words “contaminated echoes”. A slick gives a nearly specular reflection that makes a bright parabola in the radargram as the altimeter approaches and recedes from the slick, as long as the target is in the first Fresnel zone of the target. When the slick is at an angle such that the signal is scattered away from the altimeter, the return is very low, rather than bright. Strong return is seen at in later gates along track as the range is greater than when it is at nadir.
Line 90: “slicks” usually refers to films, biological or hydrocarbon (e.g., oil) that dampen the capillary waves on the surface. Uncontaminated water that lacks waves is usually just called “smooth”. Unless it is known that there are “slicks”, I would use “smooth”.
Line 102: Lots of things can contaminate the trailing edge – rain, land, ships, not just “slicks”. Leave out “slick” in the sentence.
Lines 114-117: This discussion (and perhaps the entirety of Ref 6, which I have not read) appears to misrepresent the waveform information. The center of the radargram in Fig 1(b) contains basically no information about wave height other than that it is very low – the surface is smooth enough so that the altimeter sees basically a specular reflection. Away from the center of the “slick” such as at “contaminated echoes” one could fit a Brown model and get swh by removing the contaminated tail. Perhaps the color scale in the radargram is distorted because the slick is so bright at nadir. A color scale that makes the radargram more clearly show the slope/step up at the leading edge would help.
Table 1: The criteria in Table 1 that allow sigma0 up to 30 dB will have much contaminated data. Sigma0s greater than about 18-20 dB already indicate sigma0 “blooms” that give corrupted tracking/fitting. SGDR swh <~ 0.3 m is also very likely to be incorrect.
Figs 4, 5: Since the bin size in Fig 6 is 0.2 m, why not use that in Figs 4, 5. (I think that my question about using a bin of 0.25 m was misinterpreted.) The fact that most of the smaller swh values disappear upon averaging is not surprising and is indicative of how noisy the 20 Hz data are.
Lines 192-195, Fig 6: As stated in the text, it is likely that SGDR suffers from contamination for low swh; however, much of this is likely from the excessively large sigma0 range allowed in Table 1 as discussed above. The more traditional tighter limits must be evaluated before publication. (WIW19 may be able to tolerate the bad WF for higher sigma0 because it effectively eliminates them in the algorithm itself.)
Figs 6, 8, 9: The number 1 conclusion from these figures is that ALES has no real measurement capability for swh <~1 m (certainly 0.5 m). This observation makes much of the discussion moot. The figures do show that WIW19 does have significant skill (Line 232)in spite of the negative comments above about the explanation(s) of radargrams.
Lines 240-243: This is related to comment above about Lines 114-117, but the meaning/explanation of “length of slick-free trailing edge” does not fit with how a radargram looks. Perhaps expanding and numbering the gate scale on the radargram would help; but I think that the authors need to rethink the explanation/wording of this.
Fig 10-13: Use scales only 0-3 m so the data/symbols (particularly Fig 13) are more discernable.
