Developments in Scope and Availability of HF Radar Wave Measurements and Robust Evaluation of Their Accuracy

Abstract

HF radar systems form part of many operational coastal monitoring systems providing near-real-time surface currents for many useful applications. Although wave measurements have been possible with these systems for many years, they have not yet been adopted widely for operational monitoring because they have not been thought to be sufficiently accurate or reliable. However, the value of such data is beginning to be appreciated, and this is motivating more work on wave measurement with HF radar systems with many more papers on accuracy assessment and data availability appearing in the literature. In this paper, the wave measurement capability, limitations, and differences between different radar types are reviewed, and methods to assess accuracy are discussed and applied to phased array HF radar data obtained from the University of Plymouth WERA radars using the Seaview Software inversion method during April and November 2012 compared with directional buoy data. Good accuracy over a range of different wave parameters will be demonstrated. Newly available single-radar inversions are shown to be less accurate than dual-radar inversions, although they still provide useful data, and ways to improve performance are discussed. Swell and wind–sea components in the directional spectra are identified, and qualitative agreement with buoy peak parameters is demonstrated. Recommendations are given on statistical methods for the validation of wave parameters.

1. Introduction

Waves are routinely measured using wave buoys, and there are many of these around the world’s coasts, with a few further offshore. These provide very useful information for wave model and satellite algorithm development, calibration, and validation and also more locally for activities such as shipping, sailing, coastal recreational activities, coastal erosion, and sediment transport studies and to inform offshore engineering operations such as oil and gas and wind farms. Waves interact with and are modified by currents and bathymetry, and these interactions can become more complex closer to the coast, where waters are shallower and coastal topography can be very variable. Buoys are usually single-point measurements in particular areas of interest and cannot capture the likely spatial variability in the wave field. Waves are also measured from satellites, which are able to measure spatial variations in waves, but not usually with the time resolution needed to resolve developments associated with local storms or sea breezes. HF radar systems, which are normally located on the coast, measure the signal backscattered from ocean waves from near their location to tens of kilometres offshore in near-real-time. This signal carries information about surface currents because they change the phase speed of ocean waves, which induces a Doppler shift in the signal, which can be easily measured. HF radar systems measuring this property are now a routine part of many operational monitoring systems around the world [1,2,3,4]. The signal also carries information about the ocean wave directional spectrum, but this part of the signal is weaker and extracting quantitative measurements much more difficult. However, if this information can be obtained with sufficient accuracy and reliability, it will add significant value to the spatial measurement capabilities of these radars and provide many more applications for this technology.

The potential for wave measurement with HF radar was first established by Barrick and Weber [5,6,7]. They developed a first- and second-order theory of electromagnetic scattering from moving ocean waves, which led to non-linear integral equations relating the ocean wave directional spectrum to the radar power (known as Doppler) spectrum. The first-order signal provides surface current measurements. Wave measurements can be obtained by inverting the second-order equations. There are two immediate problems: how to ensure that the radar signal has sufficient signal-to-noise (at the locations where wave measurement is required) to clearly identify the second-order part of the Doppler spectrum; non-linear integral equations cannot be inverted accurately without imposing additional conditions. The first problem has two parts: radar design and operating parameters; noise, interference, and propagation conditions. Although the latter need to be considered in the radar design and parameter selection process, they will still influence performance and cannot be completely designed away. Most HF radars in recent years use FMCW modulation, either continuous (e.g., WERA [8], G-HFDR) or interrupted (FMICW, e.g., SeaSonde [9], Pisces [10]), which minimises the impact of interference compared to the pulsed radars used in earlier work. The signal from different ranges is determined using an FFT [11]. FMICW allows the transmitter and receiver to be co-located (in the continuous case, self-interference has to be avoided), and higher powers can be used to give longer ranges, although maximum power levels are subject to radio frequency licensing agreements. The operating frequency determines both the likely maximum range for both wave and current measurements (low radio frequency, longer range, and vice versa, with the maximum range at either frequency also reduced due to propagation losses in higher seas) and also wave measurement ability (low radio frequencies can measure higher waves and high frequencies lower waves) [12]. Other factors such as the physical footprint of the radar system can be more important than wave measurement potential when deciding on a radar system, particularly when the priority is for surface current measurement. So, many systems in use around the world are so-called direction-finding radars of the SeaSonde type. Direction finding is used to obtain the spatial distribution of surface currents, but cannot be used for wave measurement. The inversion process is even more complicated for these radars since the second-order equations are convolved with the broad-beam antenna pattern obtained from a monopole and two loop antennas mounted on a single pole. Wave measurement solutions assume that the wave spectrum is uniform around constant range rings, which can be difficult to justify in coastal waters. So-called phased array radars (WERA, G-HFDR, Pisces) use an array of antennas phased to provide data in particular directions, and thus, backscatter from patches or cells (often of the order of 1 km × 1 km) in the coverage area can be identified; assuming these are small enough so the directional spectrum is constant across the cell, the second-order equations apply without additional complexity.

The methods to obtain wave measurements can essentially be divided into three main strands:

• Empirical algorithms using buoy data and regression to obtain estimates for significant wave height and, sometimes, additional parameters, e.g., [13,14,15];
• Algorithms building on the work of Barrick [16] for the frequency spectrum and Lipa [17,18] for narrow-band swell, both of which were based on a linearisation of the equations to derive closed-form expressions [6];
• Numerical inversion methods using model fitting [9] (the method normally used with SeaSonde radars); regularisation [19,20]; singular-value decomposition [21]; Bayesian methods [22]; optimisation [23]; neural networks [24]; and constrained iteration [25,26].

This paper will look at the performance of the constrained iteration numerical inversion method, which is the method used in the software supplied by Seaview Sensing and is hereafter referred to as SV. This method has been tested at many different locations with different radars and operating frequencies [27]. Two recent developments and a paper [28] building on [16,17] (item 2 above) have motivated the new results presented in this paper. The original Seaview software package was written to analyse data from two radars looking at the same area of sea from different directions. Most early radars operated in this way, and two radars are needed to measure surface current vectors. Wyatt [27,29] showed that two radars are also required to accurately measure waves. This was less clear in Barrick’s original work [6], although it was pointed out in [30] that his expression was limited to the case of waves travelling perpendicular to the radar direction. As a result, attempts to use Barrick’s formulation usually involve a calibration to match local buoy wave heights, e.g., [31,32]. The SV software package has now been restructured to allow for inversion using one to N radars (and also bistatic configurations, although this has yet to be fully implemented and tested); so, a quantitative assessment of single-radar inversion can be obtained, and the software can also be used in the context of radar networks, albeit still being restricted to data with sufficient signal-to-noise. The second development is the ability to separate and identify wind sea and swell components in directional spectra [33]. The SV method makes no assumptions about the shape of the directional spectrum over a frequency band with an upper limit that depends primarily on the radio frequency. Beyond this, a Pierson–Moskowitz spectrum is assumed with a ${\mathrm{sech}}^2\left(\beta (\theta -{\theta }_w)\right)$ directional distribution [34], where the parameters: $\beta$ and wind (or short wave) direction, ${\theta }_w$, are determined from the radar data using a maximum likelihood method [35]. Bi- and multi-modal spectra, in frequency and or direction, can be measured. The work presented in this paper used the SV core inversion version $6.0\_\alpha 13$.

There is one empirically based parameter used in the SV method. This is used to parameterise high-order non-linearities not accounted for in the Barrick equations, but identified by [36,37]. This effect can be approximated by a wave-height-dependent scaling of the first-order amplitude. The scaling used takes the form $(1-M{H}_{\mathrm{s}}^2{k}_0^2/8)$ times the amplitude of first-order peak, where $k_0$ is the radar wavenumber and M is the empirically determined parameter in the range $[0,1]$. $M=0$ implies no scaling. For dual-radar inversions $M=0.3$ is used since this gave the best results when tested with the November data set discussed in this paper. The theory suggests that there should be a directional impact on this scaling that has not yet been taken into account. This is more apparent with single-radar inversions and is discussed in Section 3.2 below.

Al-Attabi et al. [28] (AlA) used Barrick’s method [16] for the wind–sea component combined with a directional wave model, $0.5\beta {\mathrm{sech}}^2\beta (\theta -\theta \left(f\right))$, where $\beta$ is the spreading parameter, which is assumed to depend on the wave frequency relative to the peak as determined empirically by [34]. They determined the peak direction at each frequency, $\theta \left(f\right)$, from second-order Doppler ratios assuming a ${cos}^s(\theta -\theta \left(f\right))/2$, where $s=2$ is the assumed constant spreading parameter. A similar model is used to estimate wind direction from the ratio of the first-order peaks. They obtained the swell component using [18]. To implement this approach, a swell–wind separation frequency is required. They used a wave age formulation, which depends on an externally provided wind speed. If the wind speed is not available, a fixed partition frequency of 0.12 Hz is used. The swell component obtained is removed if its contribution to the Doppler spectrum relative to the wind–wave contribution is less than a specified threshold, determined by trial and error. The methods were applied to dual-radar data where these were available, otherwise to single-radar data.

AlA was tested using data from the University of Plymouth WERA radars [38,39] during April 2012. The radars use a frequency of about $12.3$ MHz. Previous tests of the Seaview method have used data from the same system for November 2012 [27]. In this paper, both data sets were analysed. The April data were used to compare with [28]. The University of Plymouth radars are sited on the North coast of Cornwall, U.K., providing metocean measurements in a region of the Celtic Sea of great current interest for wave power and offshore floating wind power applications.

The aim of this paper was to provide more evidence of the good accuracy and availability of detailed wave measurements using phased array radars with the SV method to encourage greater use of this technology. In particular, the latest version of the software package, $6.0\_\alpha 13$, both for dual- and, for the first time, for single-radar systems was validated. Another aim was to promote the use of a range of statistics, not just, e.g., the correlation and/or root-mean-squared difference, and clarity in the way these are calculated, when assessing wave measurement performance and comparing with other methods.

In the next section, we present the data sets and discuss the statistical methods used to evaluate the performance of the wave measurement methods. Section 3 presents the results, followed by the discussion and conclusions.

2. Materials and Methods

2.1. Radar Data Quality and Availability

Seaview processing normally imposes a number of different quality controls, which have been established as a result of many different deployments with different radars and different operating frequencies [12]. In particular, a 15 dB second-order signal-to-noise ratio was required before inverting for wave data. We note that [28] used a very low signal-to-noise limit of 5 dB relative to their calculated noise level, which used [40] modified to only use that part of the Doppler spectrum with frequencies $\left|f\right|>1.5f_b$, where $f_b$ is the Bragg frequency and the range is limited to the side of the smaller Bragg peak. This modification was made to avoid spurious noise levels in high seas, but we determined that [40] produces sensible noise levels in a range of different conditions without the need for the modification. SV calculates the noise level using the 5th percentile level in the spectrum, and in most cases that have been tested, this gives a slightly lower noise level than [40] or its variation. However, for this comparison, we also imposed a SV 5 dB limit at the inversion stage and removed all other quality requirements at the initial validation stage apart from deleting buoy and radar wave measurements if the change in $H_{\mathrm{s}}$ over an hour exceeds the CEFAS WAVENET despiking condition of 3 m [41]. Additional quality requirements can be imposed at the validation stage, and the impact of these will be discussed later. The SV noise levels at the buoy location for the PEN (radar 0) and PER (radar 1) are shown in Figure 1. Note this only shows noise levels for cases that exceed the 5 dB level, but it can be seen that most of the time, they are in fact above 10 dB. There is a period at the beginning with low signal-to-noise levels at PER. PER is also further from the buoy (see the later maps to be referenced), so the signal-to-noise ratio is lower throughout. A few Doppler spectra for this period were examined; there was little or no second-order evident, and in some cases, significant interference was present. The Seaview results for the case when we used data from both sites were poor in this period, as we will see. These poor data can be removed by increasing the signal-to-noise requirement.

Table 1. Data availability for April period for buoy, dual-, and single-radar inversions. Percentage figures for the first two rows are with respect to the total possible during this period: 696 hourly, buoy data measured every half hour. In the other two rows, the percentages are with respect to the number of available data files.

	Buoy	Dual	PEN	PER
Radar spec			691/99%	692/99%
Data files	1392/100%	687/99%	687/99%	687/99%
Wave data	1353/97%	654/95%	678/99%	662/96%
CEFAS filter	1352/97%	653/95%	677/99%	644/94%

and

Table 2. Data availability for November period for buoy, dual-, and single-radar inversions. In this case, the maximum number of possible hourly radar measurements is 960.

	Buoy	Dual	PEN	PER
Radar spec			955/99%	958/99%
Data files	1920/100%	953/99%	953/99%	953/99%
Wave data	1899/99%	936/98%	949/99%	938/99%
CEFAS filter	1898/99%	929/98%	948/99%	922/97%

show data availability during both the April and the November periods. The data files refer to those that contain inversions for currents and wind direction. If they also contain wave data at the buoy location, they were included in the wave data count. It can be seen that radar wave data availability was lower in April probably due to issues with the PER data.

The figures in Table 1 and Table 2 are for data availability at the buoy position. The overall data availability is presented in Figure 2, which shows more than $99\%$ availability for currents over most of the region and $80\%$ for waves over the southern half of the region.

2.2. Buoy Data

Data from a Seawatch Mini II directional wave buoy deployed at 5.61W, 50.31N within the field of view of the radar were used in the comparison presented in this paper. The location of the buoy is shown on the wave maps, which are shown later in this paper. The data used in [28] were in the form of directional spectra, which were obtained from the original data using a statistical model. The wave parameters were derived from these data, and this may be the source of some uncertainty in their parameter estimates. The spectral coefficients, $S\left(f\right)$, $d\left(f\right)$, $a1\left(f\right)$, $b1\left(f\right)$, $a2\left(f\right)$, $b2\left(f\right)$, or skewness$\left(f\right)$, and kurtosis$\left(f\right)$, which should be more reliable, were not available, but, for the SV work reported here, they and integrated parameters were calculated from the original heave, pitch, and roll data using standard methods [42]. The directional spectra were determined from the Fourier coefficients using a maximum likelihood method [43,44,45].

2.3. Statistical Methods

A number of papers have looked into the statistical measures that give the best indication of comparability between wave measurement data sets. Many published papers on HF radar wave measurement claim ‘good agreement’ between radar and buoy data (often just significant wave height) without any baseline from which to judge what ‘good’ means. It is also the case that authors sometimes define their statistical measures differently from others or use them without any definition. For example, Refs. [28,46] defined the $RMSE$ between model (or radar) data, $R_i$, and observations (buoy data), $O_i$, as:

$$RMS{E}_M=\sqrt{\frac{{\sum }_{i=1}^N{(R_i-O_i)}^2}{N}}$$

and scatter index, $S{I}_M$, as

$$S{I}_M=\sqrt{\frac{{\sum }_{i=1}^N{|(R_i-\overline{R})-(O_i-\overline{O})|}^2}{{\sum }_{i=1}^N{O}_i^2}}$$

whereas [47] used

$$RMS{E}_B=\sqrt{\frac{{\sum }_{i=1}^N{|(R_i-\overline{R})-(O_i-\overline{O})|}^2}{N-1}}$$

and

$$S{I}_B=\frac{RMS{E}_B}{\overline{O}}.$$

In the results presented here, Seaview uses $RMS{E}_M$ and defines the scatter index as

$$SI=\frac{\sqrt{\frac{1}{N}{\sum }_{i=1}^N{|(R_i-\overline{R})-(O_i-\overline{O})|}^2}}{\overline{O}}=\sqrt{\frac{N-1}{N}}\times S{I}_B.$$

Regression lines are also used to assess accuracy, and AIA [28] included the slope as one of their measures using a regression through the origin.

In order to compare different methods or results from different locations, full details on the statistical methodology are needed.

The next question is which statistic is (or statistics are) needed to clearly identify differences in performance. This issue is addressed in [46,47]. Mentaschi et al. [46] identified problems with SI in the case of negative bias and suggested instead using the [48] index, HH, defined as

$$\mathrm{HH}=\sqrt{\frac{{\sum }_{i=1}^N{(R_i-O_i)}^2}{{\sum }_{i=1}^N{R}_i{O}_i}}.$$

Bryant et al. [47] proposed some performance scores to overcome the observation that individual statistics are sometimes misleading. These are the Willmott performance index, defined as

$$\mathrm{WPI}=1-\frac{{\sum }_{i=1}^N|R_i-O_i|}{{\sum }_{i=1}^N\left(\right|R_i-\overline{O}|+|O_i-\overline{O}\left|\right)}$$

and the one used by the Interactive Model Evaluation and Diagnostic System (IMEDS, [49]), which is calculated as follows.

$$X_{rms}=\sqrt{\frac{1}{N}\sum _{i=1}^N{O}_i^2}$$

$$E_{rms}=\sqrt{\frac{1}{N}\sum _{i=1}^N{(R_i-O_i)}^2}=RMS{E}_M$$

$$bias=\frac{1}{N}\sum _{i=1}^N(R_i-O_i)$$

$$p_{rms}=1-\frac{E_{rms}}{X_{rms}},p_b=1-\frac{\left|bias\right|}{X_{rms}},\mathrm{IMEDS}=\frac{p_{rms}+p_b}{2}$$

Both $\mathrm{WPI}$ and $\mathrm{IMEDS}$ vary between 0 and 1, with 1 being a perfect score. These indices are also presented later.

Another approach referred to in the literature [50,51,52] uses the mean and standard deviation of the proportional difference.

$$pd=\frac{O_i-R_i}{(O_i+R_i)/2}$$

An approach that has been used by Seaview is to calculate the statistics needed to plot a Taylor diagram [53]. These are the rms centred difference and radar standard deviation, both normalised, (${}_N$), by the observation standard deviation, defined as:

$$rms{c}_N=\frac{\sqrt{\frac{{\sum }_{i=1}^N{|(R_i-\overline{R})-(O_i-\overline{O})|}^2}{N}}}{\sqrt{\frac{{\sum }_{i=1}^N{(O_i-\overline{O})}^2}{N}}}$$

$$Sst{d}_N=\frac{\sqrt{\frac{{\sum }_{i=1}^N|{(R_i-\overline{R})}^2}{N}}}{\sqrt{\frac{{\sum }_{i=1}^N{(O_i-\overline{O})}^2}{N}}}$$

and the centred correlation coefficient, i.e., the correlation coefficient between $R_i-\overline{R}$ and $O_i-\overline{O}$. This has proven to be a useful indicator of performance. These and the other statistics were calculated using Python numpy.

All of the above can be used for non-directional statistics. For directional parameter comparisons, we have recommended the use of vector correlation [54], referred to as KVCORR, the phase difference, KPHASE (mean difference in angle), and the concentration, CONC, [55], which measures the tightness of the difference distribution, larger numbers implying better agreement. An alternative to [54] is [56], where the correlation coefficient, HVCORR, was derived using a complex regression approach. In this method, the squared magnitude of the correlation is the proportion of variance (of the buoy data in our case) explained by the regression equation, so being a direct analogue of the correlation coefficient for scalar variables. The phase difference, HPHASE, is also obtained from the complex correlation. This method was used in the AlA evaluation.

The relative merits of all the above approaches will be discussed in Section 5.

3. Results

Comparisons of various wave parameters and spectral data between SV and the buoy are presented here. Where available, the statistics are compared with those referred to in AlA.

3.1. Parameter Validations

In these comparisons, the frequency range used for both buoy and radar parameters was 0.029–0.275 Hz, this being the maximum range for the dual-radar inversions. Doppler returns corresponding to higher frequencies involve interactions between waves of similar sizes, which are, therefore, outside the scope of this inversion method. AlA used a maximum frequency of 0.35 Hz, but here, the higher frequencies use a modified Barrick’s weighting function [6] and are, thus, more empirically based (and way beyond the approximations made in Barrick’s analysis).

The comparison between SV and AlA for the April data is not perfect; the SV statistics are based on the newly processed buoy data, whereas the AlA statistics use the original buoy spectra.

3.1.1. Wave Height

SV normally provides significant wave height, $H_{\mathrm{s}}$, obtained from the zeroth moment of the frequency spectrum. AlA provided statistics for rms wave height $H_{\mathrm{rms}}$. In order to make a more-direct comparison, we used $H_{\mathrm{rms}}=H_{\mathrm{s}}/1.4$.

Figure 3 shows scatter plots of $H_{\mathrm{rms}}$, $H_{\mathrm{s}}$ for the full data set and a time series of $H_{\mathrm{s}}$ for the April period. Some of the statistics are included in the summary

Table 3. Comparative statistics for April dual-radar inversions for magnitude parameters, AlA values in parentheses.

Parameter	cc	Slope	${\mathit{SI}}_{\mathit{M}}$	$\mathbf{HH}$	Bias	WIP	IMEDS
$H_{\mathrm{rms}}$	0.92 (0.92)	0.98 (1.02)	0.20 (0.21)	0.20 (0.21)	−0.03 (0.03)	0.83	0.89
$f_{\mathrm{p}}$	0.6 (0.63)	0.96 (0.85)	0.27 (0.24)	0.28 (0.27)	0 (−0.0)	0.66	0.86
$f_1$	0.73 (0.55)	0.98 (0.55)	0.12 (0.15)	0.12 (0.24)	0 (−0.0)	0.68	0.94

. Note that the number of data included in the SV wave statistics for April is 635, which is 97.2% of the available data. This compares with a figure of 626 quoted in [28], Table 5, the difference being due to a few extra buoy measurement obtained by reprocessing that data set. A time difference of 15 min was used to match radar and buoy data, and data gaps were not always aligned. The Seaview statistics for the complete data set are given in

Table 4. Statistics for all dual-radar inversions for magnitude parameters.

Parameter	cc	Slope	SI	${\mathit{SI}}_{\mathit{M}}$	HH	Bias	WPI	IMEDS
$H_{\mathrm{s}}$	0.91	0.97	0.20	0.18	0.18	0	0.82	0.91
$T_{\mathrm{p}}$	0.54	0.93	0.30	0.28	0.29	0.27	0.68	0.84
$T_1$	0.80	0.98	0.11	0.11	0.11	0.08	0.73	0.94
$T_{\mathrm{e}}$	0.77	0.99	0.13	0.12	0.13	−0.02	0.74	0.94

. This includes 1557 measurements or 98.4% of the available wave data.

Variations in swell parameters across the field of view of the radar were used to remove spurious large-amplitude, but unrealistic swell features associated with noise or ship signals close to the first-order Bragg peaks [33]. The median direction, median, and standard deviation of the wavenumber of the largest partition at each cell were used. The swell $H_{\mathrm{s}}$ shown is that of the largest swell component found after this filtering, which requires the direction of a partition to be within 60${}^{\circ }$ of the median direction and its wavenumber to lie within twice the standard deviation from the median wavenumber. The wind wave partition is identified as the one with the highest wavenumber with the direction within 60${}^{\circ }$ of the radar-measured wind direction.

3.1.2. Period/Frequency

Seaview normally provides the period parameters: peak, $T_{\mathrm{p}}$, energy, $T_{\mathrm{e}}$, and first moment, $T_1$, where the latter two are defined in terms of spectral moments $m_N=\int E\left(f\right)f^{N}df$ as follows.

$$T_{\mathrm{e}}=\frac{m_{-1}}{m_0},T_1=\frac{m_0}{m_1}$$

AIA [28] presented the peak and mean frequency ($f_{\mathrm{p}},f_1$), but did not define the mean. For comparison, we assumed their mean is our $1/T_1$. The frequency plots for April are shown in Figure 4. The scatter plots are for the full data set. The $T_{\mathrm{e}}$ plots are limited to 15 s for clarity, although a few measurements greater than this can be seen on the time series plot. After time matching, there were only 2 buoy and 1 radar measurements greater than 15 s. These were included in the statistics. The inversion limit of 0.275 Hz reduced the accuracy of the radar-measured $T_1$ and $f_1$. $T_{\mathrm{e}}$ is a more reliable measure for the radar data since it was dominated by the lower-frequency part of the spectrum.

3.1.3. Directions

The direction statistics for the full data set and the time series for April are shown in Figure 5 and

Table 5. Comparative statistics for April dual-radar inversions for direction parameters, AlA values in ().

Parameter	Complex Correlation HVCORR	Phase Angle HPHASE	CONC
Peak Direction	0.64 (0.57)	−5.77 (19)	3.44
Mean Direction	0.75 (0.72)	−7.98 (15)	5.26
Wind Direction	0.8 (0.6)	0.25 (−4)	3.47

, the latter showing both the SV and AlA statistics. The peak direction plot also shows swell and wind–sea directions. The SV statistics for the complete data set are given in

Table 6. Statistics for all dual-radar inversions for direction parameters.

Parameter	KVCORR	KPHASE	HVCORR	HPHASE	CONC
Peak Direction	0.86	−4.48	0.66	−4.98	3.89
Mean Direction	0.9	−4.21	0.75	−5.46	5.24
Wind Direction	0.83	4.59	0.81	2.79	3.33

.

Wind direction is also included. The SV estimates this based on [35]. This gives a wind direction and directional spread factor. The latter, expressed as directional spreading in degrees, is compared with the average buoy directional spread factor over three frequencies about the Bragg frequency in Figure 6. The buoy data were noisier; the ranges were similar, but the details were different. The relationship between this measure of spread, $\sigma$, to the s used in [28] was ${\sigma }^2=\frac{1}{1+s}$. The mean values in April found were 47.48 for the buoy and 46.98 for the radar corresponding to s values of 1.41 and 1.44 with ranges of 0.44–4.65 and 0.55–2.53, respectively. AlA assumed a constant value of 2.

3.2. Single-Radar Inversions

Dual-radar systems with the primary mission of current measurement may not have much or any region of overlap suitable for dual-radar wave measurement. If accurate measurements could be made from single-radar data, many of these systems could add this to their portfolio of measurements. In addition, if one radar were to fail or become subject to a strong local noise source, the ability to obtain single-radar wave measurements would ensure better temporal continuity.

However a single-radar system does not have enough information to properly resolve amplitude and direction ambiguities, as discussed in [27]. The SV method can be applied to single-radar data, but the results demonstrate the expected limitations. One way to overcome this limitation is to assume spatial homogeneity over a wider area than the nominal spatial resolution of the radar and use data from different beams (in the case of a phased array radar) and/or ranges (or equivalently, more than one cell on a measurement grid) in the inversion process. This is the approach that is discussed in this paper. A similar requirement for spatial homogeneity is needed to invert data from broad-beam radars [9], in this case continuity around a fixed range in all directions. Another approach is to constrain the inversion using a wave model [57] or use data collected at the same cell, at the same time, with different radio frequencies [58].

Previous work on single-radar metocean measurement [29] looked at the wind direction measurement, which, even with a dual-radar, uses data from several cells. It was shown that the left–right ambiguity that arises with this measurement, if made along a single direction, could be resolved in some situations using neighbouring cells, but was not as robust as the dual-measurement. Preliminary work on applying the full inversion method to single-radar data was reported in [58]. It was concluded that the use of neighbouring cells is not sufficient to fully constrain the inversion. Although that work claimed to have used 25 neighbouring cells, it was later discovered that a bug in the software package limited this to 1 cell, so there was no basis for the conclusion that had been made. The latest version of the software corrects that bug. Using a cell and its immediate neighbours is not sufficient because their Doppler spectra are correlated through the range-processing and beam-forming operations. The results presented in this paper used, at each cell, it and the eight cells at four-times the cell resolution. Tests at smaller separations using more cells were carried out, but the results were not as good. Compared with dual-radar inversions, using nine cells increased the time taken, which, if applied at every cell, is not currently practical for operational use. The inversion was, therefore, limited to a subgrid of cells using the four-times cell resolution. Parallelising the code is planned and will remove this limitation.

For the single-radar inversions using the non-linear correction parameter $M=0.3$ led to the overestimation of the wave height for the Perranporth radar and underestimation for Pendeen. These were likely to be manifestations of the theoretical directionality in the missing non-linear term in Barrick’s equations. The dominant wave direction in the region of measurement is from west or southwest and, thus, roughly at most locations towards or away from the Perranporth radar (so mostly high non-linearity) and perpendicular to the Pendeen radar look directions (low non-linearity). This suggests that M should be increased for waves coming towards and away from the radar and decreased for the perpendicular case. For Perranporth, we set $M=0.7$, and at Pendeen, $M=0$ based on a few cases only. Accounting for directionality is likely to be particularly beneficial for single-radar inversions, but requires the implementation of the full non-linear term.

The same statistics used for dual-radar measurement in Table 4 and Table 6 are presented for the single-radar measurements in

Table 7. Statistics for single-radar inversions for magnitude parameters.

Site	Parameter	cc	Slope	$\mathit{SI}$	${\mathit{SI}}_{\mathit{M}}$	$\mathbf{HH}$	Bias	$\mathbf{WPI}$	$\mathbf{IMEDS}$
PEN	$H_{\mathrm{s}}$	0.87	0.88	0.26	0.23	0.27	0.25	0.73	0.82
	$T_{\mathrm{p}}$	0.46	0.81	0.31	0.30	0.36	1.4	0.63	0.77
	$T_1$	0.80	0.85	0.11	0.11	0.19	1.04	0.55	0.84
	$T_{\mathrm{e}}$	0.79	0.88	0.12	0.12	0.17	0.88	0.64	0.87
PER	$H_{\mathrm{s}}$	0.73	1.22	0.39	0.35	0.4	−0.67	0.57	0.64
	$T_{\mathrm{p}}$	0.46	0.9	0.37	0.35	0.38	0.65	0.6	0.79
	$T_1$	0.65	0.84	0.14	0.14	0.22	1.1	0.5	0.82
	$T_{\mathrm{e}}$	0.7	0.96	0.15	0.14	0.15	0.28	0.67	0.91

and

Table 8. Statistics for single-radar inversions for direction parameters.

Site	Parameter	KVCORR	KPHASE	HVCORR	HPHASE	CONC
PEN	Peak direction	0.82	−10.28	0.67	−14.73	3.07
	Mean direction	0.9	−9.9	0.82	−11.25	5.25
	Wind direction	0.79	−0.04	0.77	−3.19	2.78
PER	Peak direction	0.69	−0.11	0.34	20.79	1.93
	Mean direction	0.78	−4.66	0.45	8.15	2.63
	Wind direction	0.73	14.62	0.69	16.73	2.2

. The number of data files for the dual- and for the two single-radars was 1640. Those containing waves at the buoy location after applying the CEFAS condition were 1583 (96.5%), 1626 (99.1%), and 1567 (95.5%), respectively. Since the PEN radar is nearer to the buoy site, the number of files was larger. A higher proportion of PER wave measurements were removed by the CEFAS despiking. The numbers included in the tables, i.e., those within 15 min of a buoy measurement, are 1557, 1600, and 1540, respectively.

3.3. Spectra Comparisons

Figure 7 and Figure 8 show the variation of the frequency and mean direction spectra with time, showing good agreement between the dual-radar and buoy. Also shown are the results of the two single-radar inversions. It can be seen that, while many of the main features are identifiable in the single-radar data, the performance of the two cases was rather different. This is consistent with the expected limitations of single-radar inversions [27]. At the buoy location, the PEN radar is roughly aligned with waves propagating from/to 14°N and PER 84°N. In both cases, it is difficult to accurately measure waves perpendicular to these directions. So, looking at Figure 8, it can be seen that the events early in April and in late November were picked up better by the PEN radar, whereas PER measured the event in mid-April better. For most of the data, the peak direction was more aligned with the PER radar, and as can be seen in Figure 7, the frequency spectra amplitudes were in better agreement with the buoy. The single-radar frequency spectra extended to higher frequencies than the dual-radar case, but the results at these higher frequencies were noisy and not accurate in many cases. The PER frequency spectra seemed to be noisier than the PEN or the dual ones.

To look in more detail at the differences between the single- and dual-radar inverted spectra, Figure 9 shows the energy, direction, spread spectra, and directional wave spectra, the latter plotted on a log scale, to compare a wide dynamic range. The data were from a time during the Spring spectral peak seen in Figure 7. As was clear there, the single-radar spectrum at Pendeen was lower in amplitude than the buoy spectrum, although the shape and directionality were similar. The Perranporth spectrum showed a more-spread peak and included an additional high-frequency component, also evident in Figure 7. The underestimation in height for Pendeen was presumably due to the mean direction near the peak of the spectra, which is roughly to the east. Figure 10 is a case when the spectral peak was roughly to the south, and in this case, the Pendeen inversion was better than that for Perranporth. The neighbours used were not fully able to compensate for this directionality issue. The non-linear correction parameter was probably also playing a role in these differences.

4. Swell and Wind–Sea Partitions

The buoy data were not partitioned, but the peak direction and period can be used to have some idea as to the validity of the radar directional wave spectra partition process. Making the assumption that the wind wave component is always larger than lower ranked swell components, the peak component was taken as the swell or wind–wave component with the largest significant wave height. The period and direction of the centre of gravity for each component and for the peak component are plotted in Figure 11 together with the buoy values and radar measurements, which correspond to the peak frequency in the directional spectrum. The peak partitions, identified with a black circle around the colour-coded partition type, are mostly aligned with the buoy peaks with a few outliers, which could indicate bimodality with similarly sized, but not equal peaks in the buoy and radar data or occasional misidentification of the partition type. There were fewer peaks identified using partitions (585 (92%) in April; 859 (94%) in November) compared with the number of radar measurements of wave spectra at the buoy position (635 and 912, respectively). Gaps in the coverage at the buoy position indicate times where, for swell, partitioning failed to produce a spatially consistent swell component and/or, for wind, a partition sufficiently aligned with the wind.

Maps showing swell and wind–sea components separately are presented in Figure 12. Plotting these maps made it clear that the 5 dB second-order signal-to-noise criterion, whilst not having a huge impact on accuracy at the buoy location, introduced noise in the radar measurements at longer ranges. A 12 dB criterion was used for the mapping, as it was found to get rid of most of the long-range noise. During the period of these maps (and commonly at this location), swell is predominantly from the southwest, refracting towards the coast in the shallower waters. Wind–sea is similar in wave height coming from the north. Variations in spatial coverage with time are clear and are characteristic of HF radar measurements, particularly of waves due to the sensitivity of the second-order spectrum to noise and the interference of different kinds. Gaps in the coverage indicate locations where waves were not measurable (radar data failed the quality criteria) or for the same reasons referred to above.

5. Discussion

This paper was originally motivated by the paper of Al-Attabi [28], which extended the work of Barrick to provide a hybrid swell wind–sea method. That work is in the tradition of simplified inversion methods, which avoid the complexity of inverting the integral equations. One of our aims was to show that full integral inversion, as implemented in the new version, $6.0\_\alpha 13$, of the Seaview Sensing software, continues to provide more-accurate results, and the statistics in Table 3 and Table 5 demonstrate that. In the process of performing that comparison, it became clear that accurate specification of the statistics used to evaluate performance is needed to ensure the comparison is consistent. Previous work and a literature search revealed a number of different statistics used to evaluate wave measurement and model performance, so the scope of this paper was extended to see whether a particular statistic or set of statistics provided a more-reliable measure of performance so that this can be recommended for future use. This is discussed below. We also noted that noise levels are measured in different ways, and so, signal-to-noise criteria are not always equivalent.

The comparisons between radar and buoy wave measurements presented in Section 3 used a 5 dB second-order signal-to-noise threshold for the radar data, as discussed in Section 2.1. Additional quality criteria are normally applied by Seaview when displaying data or assessing performance. These are listed in

Table 9. Radar metocean quality measures.

Quality Measure	Description	Seaview Setting
Signal-to-noise ratio, sn	dB difference between 2nd-order peak and noise	≥15
Inversion residual, res	Measure of the convergence of the integration	<0.5
Wind, squalwind	1st-order peak wind sea, $k_0{H}_{\mathrm{s}}$	>0.08
Wave, squalwave	$k_0{H}_{\mathrm{s}}$	>0.18
Spectrum, squalspec	$k_0{H}_{\mathrm{s}}$	>0.32

together with their defaults. The inversion residual measures the difference between the measured Doppler spectra and the Doppler spectra obtained by integrating Barrick’s equations [6] using the wave measurement obtained at each inversion iteration. This number decreases as the iteration progresses until it is small enough that convergence is deemed to have been achieved. Non-convergent behaviour indicates that the radar data quality is poor and the inversion fails. The $k_0{H}_{\mathrm{s}}$ criteria are derived from the ratio of the first-order Bragg frequency to the peak frequency of a Pierson–Moskowitz spectrum corresponding to the radar-measured significant wave height ($PM{f}_{\mathrm{p}}=\frac{1}{0.729}\sqrt{(\frac{0.0213}{H_{\mathrm{s}}})}$ [59]). If that ratio is less than one, corresponding to $k_0{H}_{\mathrm{s}}<0.18$, the radio frequency used is too low to measure wind waves. The wave criterion uses a ratio threshold of $1.5$, so there is a better chance that the first-order Bragg peak is wind driven. The spectrum criterion is for a ratio of $2.0$ and was found to give more-accurate spectral parameters in earlier work [12].

The impact of increasing the second-order signal-to-noise, reducing the inversion residual, and applying the $k_0{H}_{\mathrm{s}}$ thresholds, separately and together, on the comparison statistics for $H_{\mathrm{s}}$ and mean direction is shown in Figure 13. The results are not really surprising. Tighter thresholds (roughly going down the lists in the figure) reduce the amount of data available for comparison (N) and, in most cases, increase accuracy, i.e., reduce or increase the statistics as appropriate, although the variations in the numbers are quite small. In the case of $H_{\mathrm{s}}$, the correlation coefficient and the WPI statistic gave less-clear evidence of increased accuracy. There is some evidence from pdmean and directional phases that the biases actually increase, although the numbers are quite small. Similar results were found for other parameters, as can be seen in Appendix A, although IMEDS was less consistent in these cases. Figure 14 shows the Taylor diagram [60] for the different quality measures. The best performance is obtained for cases where the radar variance is the same as the buoy variance (normalised standard deviation 1), the normalised RMS differences are as small as possible (RMSD on the plot), and the correlation coefficient is as high as possible. The black square indicates equality. The different quality thresholds’ cases are closely clustered, but the zoom on the right of the figure shows the improvement of accuracy with increasing application of the thresholds. The advantage of the Taylor diagram is that it combines information about the correlation, rms errors, and relative variance and provides a clear performance indication. In this case, a 15 dB signal-to-noise ratio, 0.5 residual, and $k_0{H}_{\mathrm{s}}>0.18$ gave the best result. However, as noted in Figure 13, this case significantly reduced the amount of available data without significantly increasing the accuracy, so a lower signal-to-noise threshold, perhaps 10 or 12 dB, may be preferred by users, in particular where long-range mapping is required, and a lower threshold than this introduces noise at longer ranges (Section 4) and, so, is not recommended.

Taylor diagrams for the period parameters are shown in Figure 15. With the exception of the peak period, which shows little variation, these show the same tendencies as $H_{\mathrm{s}}$, although slightly less clearly.

6. Conclusions

This paper is not the first one to present a validation of the SV wave inversion method as was noted in the Introduction. Here, we extended the scope of such validations to include swell and wind–sea partitions, single-radar inversions, and a comprehensive look at the impact of different quality measures. The standard quality measures that have been applied in earlier work were shown to still give the most-accurate results, but the analysis presented suggests that they can be relaxed a little to increase coverage without seriously impacting the accuracy.

The partitioning method to separate wind–sea from swell components in the directional spectrum was mostly successful with good qualitative agreement with the buoy peak periods and directions. However, it is not always possible to identify swell and wind–sea components satisfying the requirements outlined in Section 3.1.1, so there are fewer swell/wind–sea partitions identified than available directional spectra. Some modifications to the criteria will be investigated, but better radar data quality, or better signal processing strategies to remove interference, sidelobe contamination, etc., are likely to be needed to improve the partition processing.

The frequency and direction spectra shown in Figure 7 and Figure 8 showed that single-radar inversions do provide some useful information, but are not as accurate as dual-radar inversions. The wave height correction parameter may be the source of some of the differences. The development of a more-rigorous non-linear correction that includes the effect of directionality is needed. Increasing the speed of single-radar inversion, which uses many neighbours and is, thus, slower than dual-radar inversion, is expected to be achieved through parallelisation of the code.

A number of different statistical measures of accuracy were discussed. For amplitude parameters, Taylor diagrams, combining the relative standard deviation, proportional rms difference, and correlation coefficient, seem to give a more-reliable estimate of accuracy relative to wave buoy measurements, and we encourage its use for wave measurement validation, whatever the method used to obtain the wave measurements. The two direction statistics for the correlation and angle difference behaved similarly, but the direct relation between the correlation and explained variance in the Hanson [56] method is an advantage, so is recommended. Concentration [55] is also a useful indicator of accuracy. A strong recommendation is that authors provide or reference their formulae for the statistics used.

The motivation for many empirically based methods is to find simple solutions that can be easily applied and are possibly more robust than full inversion methods, which are seen to be too complicated. However, they do involve many assumptions in terms of spectral shape and directional distributions, amongst others. No such assumptions are made in the SV method, apart from the linearising model used for frequencies beyond the inversion range, and, although numerically complex, once installed, it can be run through a server with no manual intervention or as a simple command line. On an HP elitebook laptop, the dual-radar inversion at each cell took about 11 ms, and the whole process took about 40 s for a case with 2939 inverted cells. Near-real-time processing is achievable. In comparison with the AlA method, similar numbers of inversions were made during April, and the accuracy was higher.

This paper provided more evidence for more-widespread use of the wave measurement capability of phased array radar systems. There are differences between buoy and radar measurements, particularly evident in the directional spectra, e.g., Figure 9 and Figure 10, which can be attributed to a number of factors, e.g., limitations in the inversion method; poorer-quality radar data; a difference between spatio-temporal (radar) and temporal (buoy) measurements; and even errors in the buoy measurements. Real wave measurement differences arising due to the obvious differences in the measurement methods between the radar and buoy could be important, but since both estimate wave spectra, which assume either spatio-temporal or temporal stationarity, we expect such differences to be small. The main limitation in the inversion method referred to here is the approximation used for the non-linear term. The implementation of the full non-linear expression is pending. HF radar systems do require monitoring and are susceptible to deteriorations in performance due to antenna or other hardware problems, which need to be quickly resolved to maintain good-quality metocean measurements. The variability in spatial mapping coverage with dual-radar systems (e.g., see Figure 12), due to variations in the signal-to-noise as a result of hardware failure, external noise, interference, or environmental conditions, could be compensated for in part using single-radar inversions, albeit accepting a further reduction in accuracy until that process can be made more robust by assuming more spatial homogeneity (enabled by parallel processing) and by replacing the M approximation with a more-robust formulation. Depending on the application, the availability of the spatial mapping of wave parameters to complement widely used surface current mapping and the easier maintenance of radar systems on land may outweigh the reduction in precision.