Evaluation of HF Radar Wave Measurements in Iberian Peninsula by Comparison with Satellite Altimetry and in Situ Wave Buoy Observations

Round 1

Reviewer 1 Report

I am confused with this paper. technically, it seems sound and relatively robust for the purpose of comparing wave data from three different technologies (buoys, coastal HF radars, satellite altimetry) in different coastal regions of the hiberian peninsula . it is rather well written (except for a few typos scattered here and there in the text), extensively documented (although neglecting some interesting wave results from HFR processing in other regions), and overall worth the publication; but, I am still lacking something here. what is the novelty of the proposed results, analysis approach and data? HFR systems have been providing currents and waves operationally for at least 10y-20y (US, Australia, Taiwan, Japan, even across Europe) to the point that JCOMM consider it a mature technology. same for wave buoys, with the trend now to include SA data into the equation. So, the main question is, what is the added value of this publication to the science community? it is a very similar paper to that in https://doi.org/10.1016/j.measurement.2018.06.060 for instance

it is not presenting new algorithms but instead uses poorly documented, closed source proprietary software for the task of inverting 2nd order Bragg to sea-state parameters (HFR-wise), with the assumption of spatial homogeneity in the wave field within the range cells which have extensively proven to be a wrong assumption. From a data analysis point of view, it uses extremely common (and abused) statistical descriptors without any indications on their reliability and neglect the use of circular statistics. Some of the results are not comparable - for instance Fig.6 a,b,c,d refer to populations with different samples (with a ratio of up 1:4 for 6a VS 6d). There is no description of the data QC (and a lot of wave data in Fig. 7d1 for instance make little sense), the theoretical limits of the HFR wave inversions (in terms of SWH) and the upper-lower frequency ranges that can be resolved. Specifically this figure refers to RC 15, which is way too offshore to provide significantly good 2nd order Bragg in a conventional seasonde HFR. 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

This paper compares the wave height measurement results of HF ground wave radar with Altimeter and wave buoy. The time span of comparison is as long as 36 months, with large amount of data, comprehensive analysis and detailed results, which has a good reference significance for the research direction.I think the paper should be published.

Author Response

Thank you very much for the comments.

The revised manuscript is uploaded.

Reviewer 3 Report

Evaluation of HF radar wave measurements in 2 Iberian Peninsula by comparison with satellite 3 altimetry and in situ wave buoy observations 4

Isabel Bué 1,2, Álvaro Semedo 3,2 and João Catalão 2

 The aim of the paper is to explore the robustness of wave measurements from SeaSonde CODAR HFR systems in the western Iberian Peninsula. The performances of the HF radars are evaluated through the comparison between High Frequency Radar (HFR) Significant Wave Heights (SWH)  observations with those performed by Satellite Altimeters (SA) and in situ buoys in the West Iberian Peninsula. Also Wave period and Direction form HF radars and in situ buoys are compared.

The time window taken into account for the analyses is 36 months. Wave parameters from 9 CODAR Seasonde HFR systems at two different operation frequency (4,96 and 12-13,5 MHz), 12 buoy observations and from Sentinel 3A and 3B (SRAL) are used in the study. All data were hourly time paired.

A first comparison between HFR and 4 buoys has been performed according to the proximity of the RCs with the buoys.

Due to weak collocation of buoys and HFR wave data, the comparison has moved between HFR and SA SRAL wave data. Buoys data are used for the SRAL SWH measurements validation.

The R, Bias, RMSE and SI statistics have been used in the comparison of buoys, HFR and SRAL datasets.

In general, an overestimation by the HFR systems is reported.

The HFR system showed to have the potential for wave data retrieving, with adaptations and improvements, acting in a complementary way respect to in situ sensors for coastal areas monitoring.

One major strength of this paper is the evaluation of the performances of the HF radar system over a long period (36 months) which highlighted the weakness of the HFR system in retrieving wave data to be used for marine safety, tsunami warning, hazard detection, Search and Rescue operations, or coastal zone management. Previous comparative studies, are limited to shorter periods, preventing a real assessment of the possible use of these data to shoreline and coastal zones monitoring activities. As noted, the low availability of wave data in some of the HF radar has contributed to its lower performance. As the authors report, the mitigation of the discrepancies mentioned in the paper can be carried out with new updates for the system software.  The HFR's ability to detect wave height extreme events during the Emma storm passage over the Iberian Peninsula was also an important aspect, showing the potential of these HFR sites to capture extreme wave events.

The paper is well structured and clear. Maybe some aspects could be treated shortly or moved to a supplementary material section, in order to reduce the number of pages making the paper lighter.  

Minor Revisions

Line 246: due to the buoys’ geographical (their to eliminate) distance to the respective HFR RC

Lines 262 -264: Which controls have been made to remove spurious data? How the data screening was carried out? It is evident that the number of invalid HFR wave data is high and in general it would be discussed.

Line 285: substitute Duo with Due

3.1 HFR versus buoy

3.1.1 SWH

It could be interesting and important to verify if statistics metrics present some seasonality, in order to highlight the better agreement, if there is, when the SHW values are >2m therefore in stormy season. Obviously if the number of data allows it.

Figure 6. The results are encouraging but it is important to stress that when the number of available data increase (Fig 6 d)) the performance of statistics metrics gets much worse. There Is only one sentence in this sense. 

3.1.2 Wave period

The comparison of wave period is very interesting but t is difficult to understand the physical meaning of an average between a peak and a mean period (Ta in the text). I believe that might make more sense to limit the comparison between Tc and Tm / Tp parameters.

3.1.3 Wave Direction

Fig. 10 d) It is impressive the absence of variation of Faro coastal wave directions in one month also with respect to the variation of HFR wave directions. Buoys data in this case have been controlled? It is possible the presence of some sensor’s problem?

Take into account the possibility to move lines from 470 to 479 and consequently Fig. 11 after lines 231 to complete the description of the HFR data availability.

3.2 Satellite altimeter versus buoy

Table 5 Again it is important to discuss the small number of pairs (SA-HFR) used to compute the statistics metrics. 

As more general comment, it is important to discuss the possible reasons why there are a lot of spurious and null values (in part associated to the second-order spectrum covered close to the first-order region when swh<2m, as reported by the authors) that it is important when looking into the performance of the HF radars system. The high numbers of null and spurious data can affect the statistics metrics and in general the utility of a system dedicated to an operative monitoring of the sea state. It could be interesting to control if spurious data have time dependence, studying the time variability of the invalid data.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 3 Report

I agree with the proporsed revisions but please update the references reported in your replies to the two reviewers with those of the paper. There is a mismatching between one reference in a reply and in the paper.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

The paper is written in a manner that makes it difficult to take the results are credible.  This is due to writing style, a demonstrated lack of understanding about some of the data, and huge gaps in information about the treatment of data. The statistical analysis and Figures are very useful for the evaluation, but given that the reader cannot tell if the data going into this analysis are appropriate, they are  not sufficiently convincing. Gaps in the explanation of some details about the figures compound the problems.  I greatly appreciate the detailed information in the statistics, scatter plots and QQ plots. Frankly, this aspect of the paper is far better than normal, and presents a great deal of useful information – if the presumed input to the statistics is reasonable. It is my hope that the problems mentioned above and detailed below are simply a matter of editing and adding detail. Therefore, I recommend that this paper be accepted with major revisions.

Major concerns:

• From where are the HFR data sets available? This information must be included for all the data sets used in this work.
• Does the equation on line 126 apply to all sea states? Wind seas and swell? As mentioned below, statements about the range of applicability and key caveats are important.
• Section 2.2.1: What are the error characteristics of the buoy observations?
• Section 2.2.2: The statement that only data >0 km from the coast are used implies that data within one footprint of the coast are used. This is a very bad idea, as these will be land contaminated and horribly skew the results. This suggests that the authors are unfamiliar with issues in using this type of data, and is part of the reason it is important to state the known issues with errors when writing about applications of data sets.  Yet, reading on further, I see that the authors refer to pairs of time series rather than pairs of observations.
• Should I be concerned that there appears to be almost no overlap between HFR and buoy observations in Fig. 2? This suggests that the intercomparison is either flawed by using data that are mismatched in space, or that the comparison is much more limited that implied earlier in the paper. Reading on, I see that only four pairs of buoy:HFR data are used in this analysis.  This is far too small a number for meaningful statistics.
• The caption for Table 2 does not make sense: ‘closest distance versus highest percentage of useful data’ implies a ratio or some information beyond a single number.
• Paragraph from lines 246 to 255: Why are the HFR data missing? This kind of information is critical to understanding conditions that are included or excluded from the analysis. It could be that the authors are excluding all poor matches by considering them as spurious.
• The large differences in departures for the HFR data from the buoy data in are concerning when the correlations presented are all very high. Please explain how such high correlations are achieved given such differences in variability. Is this just an unusually different month used as an example? The same questions apply to the examples and the results in Figure 7. How do the color bars in Figures 5 & 7 relate to data density?

Minor concerns:

• I urge the authors to use better terminology or to better explain their terminology.
  1. I recommend that “ground truth” be replaced with “comparison data” (without the quotes).
  2. Accuracy is neither defined nor a clear term in this context. These seem to be measures of uncertainty, such as a rms difference. I suggest that the word ‘accuracy’ not be used as the authors are using it.
  3. A sentence like “HFR individual range cells versus SA analysis presents 0.94-0.99 of correlation, still a negative but reduced bias and 0.53 m of accuracy” (in the abstract) uses a lot of good key words, but does not convey information in a useful way. In fact, it is extremely difficult to determine what is meant except the part about the bias. Please take the time to use more careful terminology and to convey the meaning in a much clearer fashion.   Note that data sets can have great correlations, but this only means that that their changes from there means are proportional. The slope (gain) could be far different from 1.0, thus a correlation alone is not sufficient as a comparison.
• Lines 35 and 36: This statement of the ‘Best way’ must be for an application other than that suggested by the authors, as they point out serious flaws. Or perhaps the metric for best is something like the longest record? Statements like this that are presented without context and without supporting details detract from the credibility of the statements in the paper. I do a lot outreach training people how to assess the veracity of statements, and such bold and unqualified statements are a strong indicator that the work should not be trusted. Please work through the paper and properly qualify all such statements. Note – as someone familiar with the observing system, I assure you that this statement is true only with several qualifications.
• The two examples (altimetry and SAR) are not very good examples when the emphasis is on extremes. While there are publications indicating otherwise, a simple check of the sampling for extremes shows that such observations are not great for extreme conditions, except perhaps for those rare occurrences where these satellites sample the extreme. Perhaps extremes are not the point of this paper, but this type of unqualified example causes concern about quality of the work.
• Lines 57 & 58: Wave spectra can also be measured by imaging from drones and from satellite instruments such as SWIM.
• Paragraph from lines 57 to 68: It would be worthwhile to mention that that sampling needs in both space and time are different for coastal regions than for the open ocean. Coastal applications tend to benefit from higher resolution and the greatly improved temporal sampling provided by HFR.
• Line 77: lower or weaker than what?
• Figure 1: the red dots are unlikely to be the study areas, as these don’t overlap with any buoy locations. Perhaps the authors mean HFR locations? It would help to show the areas covered by each HFR. Reading on, I found Fig. 2 to be very useful. It would be helpful to reference to that figure.
• Table 2: Variables in the table should be defined in the caption. Why are the variables that are not discussed included in the table?

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

This manuscript makes a useful contribution to the literature by combining the three data sets described in the title.  It makes the interesting point that the Sentinel-S3 instruments make significant wave height measurements beyond about 5 km from the coast, compared with traditional altimetry measurements that are not useful within about 10km from the coast.  This is a perfect background in which to evaluate the use of HFR as a complementary technology to improve temporal and spatial coverage.  The comparison with buoys is constrained by the location of the buoys with respect to the HF radars.  The comparison of buoy SWH with S3 SWH showed a high correlation (0.89 to 0.99) and, subsequently, the S3 SWH observations are compared with collocated HFR data.  This is good methodology.

The HFR data has low availability (Figures 3 and 4) compared with other SeaSonde stations.  It would be useful to seek out the reasons for data availability less than 90% over the 36 month study period.  As presented the data availability does not support the conclusion that HFR is a robust source of wave results for operational support.

The comparison of HFR SWH with Buoy SWH had correlation coefficients of 0.84, 0.87, 0.85 and 0.66 (Figure 5) over the 36 month study period, which does not support a strong conclusion of robustness for HFR as an operational instrument; even though the text claims these to be ‘reasonably good’ (line 330).  The comparison between HFR SWH and S3 SWH for the low frequency radars for RC4 (Table 7) shows correlations of 0.99, 0.94 and 0.95 which do support the conclusion that HFR SWH is validated.  The same is not true for the higher frequency radars (Table 8) where the best correlations are 0.70, 0.53 and 0.82.  It would be useful to see some explanation on why the SWH results at 12-13.5MHz are significantly less accurate than those at 4-5 MHz.  I was not able to reconcile the overall correlations in Figure 6 (0.84, 0.87, 0.85, 0.66) with the monthly values in the appendix where the number of months with correlations exceeding those in Figure 6 are 2, 3, 4 and 14 respectively.  Why are the monthly R values generally lower than the overall ones (except for ALFA & FARO)?  And why in line 353 does the text say that the individual monthly analysis can produce a better agreement?  The SWH HFR data for storm Emily are very good and well worth publishing.

The comparisons between Tp and Tc show that the HFR Tc results are not reliably accurate.  There is an opportunity here to use the SWH which uniquely defines the Pierson-Moskowitz spectrum to find Tc (or, indeed, Tp).  This would use the assumption of the P-M spectrum that has already been applied to find HFR SWH.  It might also shed some light on the right panels in Figure 7.  It would give some cohesion to the presentation in this manuscript.  (For example the statement about the nexus between SWH and wave period (line 384) becomes clear.) The Tp comparisons with Tc do not support the conclusion that the HFR Tc values are robust for operational application.

The wind direction results are derived from first-order HFR echoes and this results in better quality data than the SWH and Tc.  The ‘limitation on the frequency transmitting sector’ in line 432 (and repeated in line 435) should be carefully explained because only first-order spectral energy is used.  The bias towards directions perpendicular to the coast (line 617) needs explanation.

The manuscript is well laid out, but the writing is poor.  This manuscript was not ready to submit: it needed much more care in detail and removal of typographical errors.  There appears to be some misunderstanding about how the HFRs actually work. There are mistakes about HFR technology (e.g. the statements about side-lobes).  There is a predisposition to adding subjective comments where the data should speak for themselves (e.g.  line 580).   There are many minor errors :

Line 24 ‘good’ is subjective and not all results are in fact good.

Line 25 overcome

Line 32 coast-parallel: coast is not an adjective

Line 38 of > to

line 54 generalisation is not true, e.g. WaMoS is an X-band radar technique that gives wave directional spectra that are more robust and accurate than HFR.

Line 63 till > to;  waves measurements are to about 30km, not 100km

Line 69 ‘theory for measuring wave parameters’ was written by Barrick 1977, not Crombie

Line 75 of> for

Line 80 have > has

Line 83 exceed

Line 97 complementary to what?

Line 101 corresponding

Line 106 of > off

Line 109 depression? Do you mean ‘fronts’

Line 129 sensor

Line 136 MWP needs to be clearly defined.  You are using Tp from buoys and Tc from HFR; and later you are directly correlating them (apples and oranges).  At this point in the manuscript you need to explain this and state clearly what you mean by MWP.

Line 148  it would be helpful to define ‘deviation’.  [35] did not help me.

Line 166 use

Line 171 more

Line 172 Do you mean ‘current velocities and wave heights..’?  It makes no sense to talk about wave velocities.

Line 189 second-order

Line 193 it will vary from site to site depending

Line 195 I suggest that you bring together all the assumptions involved in the analysis e.g. in lines 206-214.

Line 212 this is where an early definition of MWP will be useful

Line 223 S3 is defined to be the special subset of SA.  Henceforth in this manuscript you should use S3 consistently and not swing between both SA and S3

Line 227 SA > S3 see line 223

Line 236 this is confusing. Do you mean ‘distance giving highest’?

Line 252 availability of wave data for comparing pairs

Figure 3 caption HFR SWH wave data

Line 273 from > with

Line 305 The R>0.89 correlation between: avoids subjective comment

Line 315 coastal zones > distances greater than 5km from the coast.  A plot of RMSE and R versus distance from coast would make this point clearer – it is a useful result.

Line 317 proven > presented

Line 321 buoy’s : where > were

Line 324 on 26

Line 330 avoid subjective judgement

Line 336 – 337 high N should improve the statistical metrics.

Line 371 another

Line 375 explain ‘frequency range’

Line 378 clarify MWP, Tp and Tc.

Figure 7 This is an unusual way to present comparisons between pairs of parameters.  Why not scatter plots of SWH(buoy) vs SWH(HFR) and MWP(buoy) vs MWTc(HFR)?

Line 400 capturing

Line 403 relevant > relative

Line 428-438 MWD from SeaSonde HFR are derived from 1^st order Bragg echoes.  This whole paragraph seems to imply that MWD HFR SeaSonde come from the full spectrum.  Directional biases are conflated with inaccuracies in the Doppler spectrum.  The limitation on the frequency transmitting sector needs to be explained in terms of the algorithm used for MWD.  This para needs to be re-thought and re-written.

Line 452 ‘not in total agreement’ is a subjective euphemism for ‘poor agreement’S3 values are being used to compare with each individual RC annulus.  Presumably S3 values closer than 5km from the coast are not used – this corresponds to the RC annulus coming close to the coast.  How far offshore are S3 values included?

Figure 11 It would be useful to know exactly what

Line 476 their low operating frequency

Line 479 delete ‘slight’; in > to

Line 490 readers might like to know the nature of these inconsistencies – remembering that you are evaluating the technologies for operational applications.

Line 493 evaluate how the correlation

Line 501 present less consistent; you do not need to make the subjective qualifier

Line 503 satisfactory correlation > a correlation of 0.61 – 0.70 was; the reader can then judge whether 0.61-0.70 is satisfactory.

Line 518 Even fewer studies

Line 522 potential to measure

Line 528 are validated > is carried out

Line 538 further from the coastline

Line 547 very nearshore > beyond 5km from the shore

Line 552 delete ‘goog’; near to > beyond 5km from

Line 560 ‘averaged’ is not the correct word.  MSH is derived by fitting a

Pierson-Moskowitz spectral form by a least-squares procedure.  This is not averaging.

Line 563 This assumption can cause errors, and

line 564 delete ‘Following this assumption’ and start the sentence with Coastal effects

line 566 explanations

line 571 studies

line 574-576 the cells are circular and return to the coast, so you cannot ‘ensure the deep water assumption’

line 580 had reached 0.66-0.87; your readers can decide whether this is reasonable

line 582 better?  The correlations in appendix A appear to be mostly lower than these (except for ALFA)

line 583 delete ‘the’ at the end of the line

line 588 conducting > leading

line 592-593 this is a sentence with no verb

line 602 I could not access [45] but why quote one result? Was it typical?

Line 606 MWP are derived from 2^nd order echoes and MWD are derived from 1^st order echoes, so do not lump them together like this.

Line 608 median correlation?  Put ‘parameters, correlation values of 0.52-0.65 were achieved.

Line 610 this says ‘agreement…….is hard’.  Please re-word this

Line 617 I am intrigued by the bias towards perpendicular to the coast and I look forward to your explanation of how that might happen

Line 622 conducting > leading

Line 633

Line 639 ‘inside the area of the respective RC’ see line 452 comment

Line 655 sidelobes – no, not for SeaSonde where the antennas are broad beam.   [20,46,51] do not refer to SeaSonde and are incorrectly cited here.

Lines 660-661 beam-forming HFRs do not have this problem.

Line 671-672 data availability here does not support ‘ensures continuous data return’

Line 680 evaluated

Line 683 what does ‘single-side’ mean?

Line 684 antenna side lobes – no

Line 688-689 sentence with no verb.  Suggest spectrum being influenced…being retrieved.

Line 689-690 ‘distance….. to the sea state’ is not a clear concept

Line 690 conduct > lead

Line 696 stable > potential

Line 697 best not to include tsunamis here because that is a current measurement from the 1^st order Bragg peaks, not a wave measurement from 2^nd order

Line 699 show

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The authors are to be complimented on the response and on the changes to the paper.  It is now much clearer to read. The comments below however point out serious shortcomings in terms of the writing and the methodology.  I can’t tell if these can be addressed with simple revisions to the writing, or more comparison needs to be done – that depends a lot on the goals of the paper, which seem to shift from location to location. While the comments are rather negative, I feel that the much of the work is well done and there are very good points made. I believe that if the authors are persistent in improving this paper it will be published and make a notable contribution. However, my best guess is that this paper will need major revisions to reach that state.

Major concerns:

• The abstract remains very challenging to read. It is hard to find the focus and key points, and the statistics are hard to work with for biases, as it is not clear which data set is being used as comparison data (truth).  This should be a minor fix from an editing perspective, but it is a major flaw in the paper.
• Reading the first and second paragraphs of the introduction, I am still unclear as to the topic of this paper. This topic is not stated until line 100 (page 2), which is far too late. Again, an easy fix.
• If the goal is as stated, then the random error would be better assessed in buoy to buoy and CODAR to CODAR comparisons. Biases require the comparison to between these or to satellite. Please be clear why this approach is not used to assess random errors.  It could be that it is done in the reference, but if so please be clearer about that. The buoy to buoy and CODAR to CODAR observations could be used to better assess the statistical significance of biases or trends relative to satellite data.

Minor concerns

• Line 49: If the application is observations off the coast of Portugal, then observations on a global scale are irrelevant. If the application seems to be more about coastal measurements, so this statement is distracting (off topic).