**5. Hindcast Validation**

An extended 19-month hindcast model validation was performed with complete meteorological forcings. The details of the model setup can be found in Section 3. During the simulation period, RMS wind speed errors are less than 5 m/s with direction RMS errors order 50 degrees. For sea level atmospheric pressure, the RMS errors are near 2 mb. For the offshore temperature and salinity, a zero gradient boundary condition is used. Since the meteorological forcings are at 3 h intervals, the effect of the sea breeze may not be completely captured.

The results are presented in 15 day increments in Table 9 for water levels. There are fewer stations available with measured data for comparison than for the tidal calibration. In addition, there are gauge datum issues at several water level stations. Generally, the water level RMS errors do not exceed 15 cm and are consistent from month to month in almost all regions. At Point Reyes, there are issues with the data, which cause errors in the subtidal water level forcings for several months indicated as blanks.

As shown in Table 10, current amplitude RMS errors are consistent throughout the period and are generally less than 35 cm/s. The salinity response is summarized in Table 11. Generally, the model salinity was in agreement with the observations at most of the stations. However, it was overestimated in the northern portion of San Pablo Bay and throughout Suisun Bay. This is believed to be due to the fact that the river subtidal water levels were not included since no measured river stage data were available. As a result, the model results could not correctly reflect the freshwater runoff during the high flow months when substantial river subtidal levels were present. This in effect, limited the amount of freshwater entering the Bay through the Delta. The temperature response is summarized in Table 12 and exhibited a normal seasonal response, but in October 1980 there was some evidence of overheating by about 2 °C in Suisun Bay.

In addition to the validation in terms of RMS errors, the NOS skill assessment criteria [40,41] are also applied to the hindcast. We show in Table 13 the results at some of the major water level stations. Additional model skill assessment results for currents, salinity, and temperature are given in [22]. Generally, the skill assessment indicates that most water-level related statistical parameters pass the NOS skill assessment criteria for different scenarios, and that amplitudes and epochs of major harmonic constituents such as M2, S2, N2, K2, K1, O1, P1, and Q1 from the tide-only scenario simulation are very close to the observed values at almost all stations.

Most of CF (Central Frequency), NOF (Negative Outlier Frequency), POF (Positive Outlier Frequency), MDNO (Maximum Duration of Positive Outliers), and MDPO (Maximum Duration of Positive Outliers) either pass or are close to the criteria at the Bay current stations for not only the tide-only scenario but also the hindcast scenario, since tidal current dominates the signal in San Francisco Bay region. See Schmalz [22] for more complete definitions of the skill assessment parameters.

The tidal and hindcast simulations indicate that the SFBOFS runs robustly and that the results are in acceptable agreement with the measurements. The model package was therefore loaded into the COMF-HPC on NCEP's high performance computers to perform semi-operational nowcast/forecasts.

**Table 9.** Water Surface Elevation Hindcast Validation: April 1979–October 1980. For each box, the first column of values corresponds to the first 15 days of the month, with the second column denoting the remaining portion of the month. Within each column: Row 1 corresponds to the RMSE in cm. Row 2 corresponds to the Willmott Relative Error in percent. Row 3 corresponds to the model mean in cm relative to MLLW. Row 4 denoting the observed water level mean in cm with respect to MLLW. 



**Table 9.** *Cont.*







**Table 13.** Nineteen-month Hindcast Water Level Skill Assessment Results. Results are with respect to MLLWL. RMSE is root mean square error, SD is standard deviation of the error; e.g., model minus observation, and CF is central frequency of the errors with respect to a reference level of 0.15 m. 


**Table 12.***Cont.*

#### **6. Semi-Operational Nowcast/Forecast Simulation**

The SFBOFS runs four cycles each day. In each cycle, the model performs a six hour nowcast followed by a 48 h forecast. During the model preparation process, the COMF-HPC automatically searches for and obtains the necessary observed data and other model (e.g., NAM4 and RTOFS) generated data to obtain the required forcings.

#### *6.1. COMF-HPC Generated Input Forcings*

For the nowcast, the subtidal water levels along the open ocean boundary are determined using an adjustment of the Global RTOFS (G-RTOFS) latest hourly subtidal forecast guidance. The adjustment is determined by averaging the hourly subtidal anomalies at Point Reyes (NOAA gauge) over the previous six-hour nowcast period and ramping the forecast subtidal values to the adjustment. The astronomical tide is determined from the tidal constituent netCDF file and the application of the latest node factor and equilibrium argument values at six-minute intervals. The total open ocean boundary six-minute water level values are the sum of the adjusted subtidal levels and the predicted tidal values at each boundary grid point. Salinity and temperature along the open ocean boundary are obtained from the adjusted G-RTOFS forecast guidance. The adjustment is determined by averaging the salinity and temperature anomalies at San Francisco (NOAA gauge) over the previous six-hour nowcast period and ramping the nowcast values to the adjustment.

For the forecast, along the open ocean boundary, water levels are specified as a superposition of the tide predictions and the subtidal water level forecast. Note the nowcast adjustments are maintained for the forecast period for water level, salinity and temperature open boundary conditions.

For both nowcast and forecast, the most recent NAM4 results in NCEP's data tank are input into COMF-HPC to get the necessary input surface forcings.

The methodology to treat the Sacramento and San Joaquin River forcings in nowcast and forecast scenarios is different because no river stage subtidal signals are available in the forecast period. Even during the nowcast, stage data are not necessarily available. COMF-HPC uses the following approach to handle this.

Subtidal river stage data adjustment is performed on the boundary nodes of the two rivers. Real-time observed stage height data from USGS 11337190 Station are taken for the San Joaquin River nodes adjustment and the data from USGS 11455420 are used for the Sacramento River nodes. The real challenge though is how to determine the subtidal water level time series for the whole nowcast and forecast time window.

As shown in Figure 7, the green curves indicate the subtidal stage height time series, which is computed as the direct water level measurement minus the tidal prediction. The vertical black time line is the current run cycle time, for example 12Z. Since the cron job is launched after the cycle time (after the NAM4 and RTOFS forcings of the same cycle are obtained), the USGS river stage reading end time, *RT(end*), is always on the right side of the black time line. The reading start time *RT(I)*, however, can be on either side of the nowcast start time, *ZetaT(I)*.

The ultimate goal is to obtain the subtidal stage height for the whole nowcast and forecast period, the time between *ZetaT(I)* and *ZetaT(end)*. For the upper case in Figure 7, when *RT(I)* is later than *ZetaT(I)*, we assume that stage height between *ZetaT(I)* and *RT(I)* equals the height at *RT(I)*. For model stability, the subtidal stage height from *RT(I)* to *RT(end)* is decomposed into "mean" and "fluctuating" parts. The "mean" is indicated by the horizontal black line in Figure 7, and the "fluctuating" part is in green. The "fluctuating" part at *RT(end)* is ramped off lineally to zero in the next six hours. The "fluctuating" part in the rest of forecast time period is therefore taken as zero. In other words, the subtidal stage height in this period is the "mean" from *RT(I)* to *RT(end).*

**Figure 7.** Diagram on how measured river stage height is used in COMF.

As water temperatures are not available for the two USGS stations, the real-time temperature measurement data are obtained from Port Chicago, a NOAA Gauge Station with NOS\_ID of 9415144. When no real-time stage non-tidal data are available from the NCEP data tank, the climatological stage height and temperature data are automatically input into the model.

#### *6.2. Semi-Nowcast/Forecast Results*

The SFBOFS semi-operational nowcast and forecast model assessment period started from 10 March 2013 and continued to 10 June 2013. The results from these simulations were concatenated into continuous time series for analysis using the NOS skill assessment software [17]. The model ran robustly in the whole assessment period. Generally, the results of water level, current, temperature and salinity agree well with observations, and CF, NOF, POF, MDNO, MDPO, WOF and other statistical variables pass the criteria in both nowcast and forecast scenarios. Figure 8, as an example, shows the agreement of model results and observation of water level at three major stations. Refer to Peng and Zhang [42] for complete model skill assessment results at all stations for the water level, current, salinity, and water temperature.

**Figure 8.** The comparison of modeled *versus* observed sea levels at three stations in April 2013. The station locations can be found in Figure 1.

Semi-nowcast/forecast model performance is statistically shown in Figure 9. The Taylor diagrams [43] indicate that the water level results are better than the water temperature and salinity. Water level correlation coefficients at all stations are higher than 0.98, while the salinity correlation coefficient at S1 is only about 0.50 for both nowcast and forecast scenarios. The normalized modeled standard deviation at all stations is close to 1.0 for water level, but it is higher than 2.0 for some stations for salinity. Similar to the hindcast scenarios, as mentioned previously, the water level performs the best followed by water temperature and salinity. One should note that the RMSD value shown in these normalized Taylor diagrams needs to be multiplied by its corresponding measured standard deviation as listed in Tables 14–16 to get its real value.

**Figure 9.** Normalized Taylor diagrams of water level, surface temperature and surface salinity for nowcast and forecast scenarios. S1, S2, S3, *etc.* are station series numbers. Si of water level, temperature and salinity does not necessarily indicate the same station. The modeled standard deviation of each variable at each station is normalized by its corresponding observed standard deviation. The data are from October 2013.


**Table 14.** Observed water level standard deviations (m) at selected stations in nowcast and forecast scenarios**.**

**Table 15.** Observed temperature standard deviations (°C) at selected stations in nowcast and forecast scenarios.


**Table 16.** Observed salinity standard deviations (PSU) at selected stations in nowcast and forecast scenarios.


The semi-nowcast/forecast results can be found on the SFBOFS web page [44]. To serve the San Francisco Bay maritime community, the SFBOFS provides users with nowcast and forecast guidance for water levels, currents, water temperature, and salinity out to 48 h, four times per day. The SFBOFS model domain on the web is divided into two separate subdomains (the San Francisco Bay and the San Francisco Bay Entrance), allowing users to focus on their area of interest. Nowcast/forecast animations of each of the two subdomains as well as time series at over 50 locations are available for winds, water levels, currents, temperature, and salinity.

Figure 10 is a snapshot from nowcast salinity animation of the larger subdomain at 0600 PST of 2 December 2013. Figure 11 illustrates that model salinity results agree well with the measurement at locations where Sacramento and San Joaquin Rivers have noticeable effect on salinity distributions. Meanwhile the available measurement at Port Chicago indicates, as shown in Figure 12, that the water level nowcast is also in good agreement with observations. The satisfying model results for both water level and water salinity near the two rivers are largely due to the fact that river stage boundary conditions have been employed.

The SFBOFS webpage offers not only the latest model output graphics as shown in Figures 10–12, but also links where users can get access and download one-year historic output files (in NetCDF format) through CO-OPS's OPenDAP and THREDDS servers.

**Figure 10.** Nowcast salinity distribution for San Francisco Bay (12/02/13 0600 PST).

**Figure 11.** The nowcast/forecast water surface salinity *versus* measurement at stations where the Sacramento and San Joaquin Rivers have noticeable effects on salinity distrubution.

**Figure 11.** *Cont.*

**Figure 12.** The nowcast/forecast *versus* observed water levels at Port Chicago.

#### **7. Conclusion and Discussion**

This paper details how the SFBOFS was setup, tested and extensively validated in tidal and hindcast scenarios. The performance of the model package during the three-month semi-operational nowcast and forecast using the NOS COMF-HPC is discussed. FVCOM, the core of SFBOFS, ran robustly during the trial. Amplitudes and epochs of the M2 S2, N2, K2, K1, O1, P1, and Q1 constituents from the tide scenario simulation are very close to the observed values at all stations. NOS skill assessment and RMS errors of all variables indicate that most statistical parameters pass the assessment criteria for both hindcast and nowcast/forecast scenarios and model outputs have good agreements with the measurement. We have to note that OTIS Regional Tide Solutions harmonic analysis results were reduced by 5% on the open boundary. Though this ad hoc treatment ensures very good water level results, more work needs to be done to understand the dynamics behind the adjustment.

Modeled water level and salinity from Martinez to Mallard Island (see Figure 10 for locations) showed strong disagreement with measurement during hindcast period when flow river boundary conditions were employed for the Sacramento and San Joaquin Rivers. The model water level results after using stage river boundary for the two rivers were greatly improved. However, salinity disagreement still existed, though in very low occurrence, during the past ten months after the semi-operational nowcast/forecast trial period. As shown in Figure 13, the model predicted salinity at Port Chicago was in agreement with the observations from October 15–October 25. However, on October 26 and 27, the model salinity predictions at Port Chicago abruptly deviated from the observations. On the 10/27/18Z cycle, the model under predicted the salinity by up to 8 PSU in the nowcast time window. A comparison of the model river forcing water surface elevation to the USGS stage data at the two rivers showed no indication that the model stage was in error.

The location of the river boundaries is still within the tidal domain and either a stage or flow boundary condition is not entirely appropriate. In effect, the boundary location is not at the head of tide and is a tidal river with flow in both directions. In the case of a stage boundary condition, no unique stage discharge relationship exists. The stage is a function of both the discharge and the offshore subtidal water level. The imposition of the stage boundary condition yields accurate water level prediction, but is problematic for salinity, since the appropriate discharge cannot always be specified. For a flow boundary condition, since the boundary is not at the head of tide, tidal wave reflections will occur and will lead to inaccurate stage predictions in the lower delta and even at Port Chicago.

In addition, the model grid cannot represent the complex water channel system in the delta region. One can compare the real delta system in Figure 1 and the model grid in Figure 2. While previous work [15] has used a 20 m deep rectangular "false delta" to produce the appropriate tidal prism of the unresolved area, this approach was not used in the present SFBOFS, since it was felt that the entire delta region may need to be represented as discussed by MacWilliams *et al.* [45]. This further effort initially considered outside the scope of the SFBOFS is now being considered to improve the salinity prediction in the lower delta and to also potentially provide additional navigation guidance to the Ports of Stockton and Rio Vista. As an interim measure, a data assimilation scheme is being considered within the present SFBOFS, to correct the model salinity predictions from Martinez to Rio Vista on the Sacramento River and Antioch on the San Joaquin River at the start of each nowcast/forecast cycle. Future work will also consider the extension of the offshore boundary to include the Farallon Islands, which will allow for a more accurate specification of the offshore water level and current boundary conditions.

#### **Acknowledgments**

Richard Patchen, Chief Science Officer (retired) of the CSDL provided several insights on model grid development. Jiangtao Xu, CSDL, provided valuable assistance with the development of multiple grids and with the use of the SMS software. Philip Richardson, CSDL, assisted with the hindcast initial condition specification, validation data preparation, and the skill assessment. Special thanks to the two anonymous reviewers, who provided many insights and suggestions, which greatly improved the paper.
