# Comparison of NMC and Ensemble-Based Climatological Background-Error Covariances in an Operational Limited-Area Data Assimilation System

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Model

#### 2.2. Methods

#### 2.2.1. NMC Method

#### 2.2.2. Ensemble Method

**K**is the gain matrix, H is the observation operator and M is the operator corresponding to the forecast evolution during a period of 6 hours. From Equation (9), it can be seen that the differences in the forecasts for the ensemble method are the result of the different initial conditions (${\u03f5}_{a}$), and they are the result of the explicit observation perturbations (${\u03f5}_{o}$) and implicit background perturbations (${\u03f5}_{b}$). Additionally, in one of the ensemble experiments (EDA with perturbed LBCs),the LBC perturbation contributed to the forecast difference (although not explicitly written in Equation (9)). Comparing Equation (9) with Equation (4), it can be noted that much larger forecast ranges were present in the NMC method. For the estimation of the B matrix, the difference between two perturbed ensemble members is used (rather than the differences between the perturbed and unperturbed control members). As shown by [13], this result is estimating twice the background-error covariances and the result must be divided by a factor 2. It was also shown that the main conceptual difference between the NMC and the ensemble method is that in the analysis error equation, the NMC method replaced $\mathit{I}-\mathit{K}\mathit{H}$ by $-\mathit{K}\mathit{H}$

## 3. Results

#### 3.1. Diagnostic Comparison

#### 3.1.1. Geographical Distribution of the Standard Deviations

#### 3.1.2. Horizontal Spectral Densities

#### 3.1.3. Standard Deviation

#### 3.1.4. Horizontal and Vertical Correlations

#### 3.2. Impact on the Analysis and Forecast

#### 3.2.1. Impact on the Analysis

#### 3.2.2. Impact on The Forecast Quality

## 4. Summary and Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Kalnay, E. Atmospheric Modeling, Data Assimilation and Predictability; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Bannister, R. A review of operational methods of variational and ensemble-variational data assimilation. Q. J. R. Meteorol. Soc.
**2017**, 143, 607–633. [Google Scholar] [CrossRef] - Gustafsson, N.; Janjić, T.; Schraff, C.; Leuenberger, D.; Weissmann, M.; Reich, H.; Brousseau, P.; Montmerle, T.; Wattrelot, E.; Bučánek, A.; et al. Survey of data assimilation methods for convective-scale numerical weather prediction at operational centres. Q. J. R. Meteorol. Soc.
**2018**, 144, 1218–1256. [Google Scholar] [CrossRef] - Parrish, D.F.; Derber, J.C. The National Meteorological Center’s spectral statistical-interpolation analysis system. Mon. Weather Rev.
**1992**, 120, 1747–1763. [Google Scholar] [CrossRef] - Fisher, M. Background error covariance modelling. In Proceedings of the Seminar on Recent Development in Data Assimilation for Atmosphere and Ocean, Reading, UK, 8–12 September 2003; pp. 45–63. [Google Scholar]
- Errico, R.M.; Privé, N.C.; Gu, W. Use of an OSSE to evaluate background-error covariances estimated by the NMC method. Q. J. R. Meteorol. Soc.
**2015**, 141, 611–618. [Google Scholar] [CrossRef] - Sun, J.; Wang, H.; Tong, W.; Zhang, Y.; Lin, C.Y.; Xu, D. Comparison of the impacts of momentum control variables on high-resolution variational data assimilation and precipitation forecasting. Mon. Weather Rev.
**2016**, 144, 149–169. [Google Scholar] [CrossRef] - Xu, D.; Shen, F.; Min, J. Effect of Adding Hydrometeor Mixing Ratios Control Variables on Assimilating Radar Observations for the Analysis and Forecast of a Typhoon. Atmosphere
**2019**, 10, 415. [Google Scholar] [CrossRef] - Široká, M.; Fischer, C.; Cassé, V.; Brožkova, R.; Geleyn, J.F. The definition of mesoscale selective forecast error covariances for a limited area variational analysis. Meteorol. Atmos. Phys.
**2003**, 82, 227–244. [Google Scholar] [CrossRef] - Pereira, M.B.; Berre, L. The use of an ensemble approach to study the background error covariances in a global NWP model. Mon. Weather Rev.
**2006**, 134, 2466–2489. [Google Scholar] [CrossRef] - Buehner, M. Ensemble-derived stationary and flow-dependent background-error covariances: Evaluation in a quasi-operational NWP setting. Q. J. R. Meteorol. Soc.
**2005**, 131, 1013–1043. [Google Scholar] [CrossRef] - Ştefănescu, S.E.; Berre, L.; Pereira, M.B. The evolution of dispersion spectra and the evaluation of model differences in an ensemble estimation of error statistics for a limited-area analysis. Mon. Weather Rev.
**2006**, 134, 3456–3478. [Google Scholar] [CrossRef] - Berre, L.; Ecaterina Ştefănescu, S.; Belo Pereira, M. The representation of the analysis effect in three error simulation techniques. Tellus A
**2006**, 58, 196–209. [Google Scholar] [CrossRef] - Storto, A.; Randriamampianina, R. Ensemble variational assimilation for the representation of background error covariances in a high-latitude regional model. J. Geophys. Res. Atmos.
**2010**, 115. [Google Scholar] [CrossRef][Green Version] - Fischer, C.; Montmerle, T.; Berre, L.; Auger, L.; Ştefănescu, S.E. An overview of the variational assimilation in the ALADIN/France numerical weather-prediction system. Q. J. R. Meteorol. Soc.
**2005**, 131, 3477–3492. [Google Scholar] [CrossRef] - Brousseau, P.; Berre, L.; Bouttier, F.; Desroziers, G. Background-error covariances for a convective-scale data-assimilation system: AROME–France 3D-Var. Q. J. R. Meteorol. Soc.
**2011**, 137, 409–422. [Google Scholar] [CrossRef] - Berre, L.; Monteiro, M.; Pires, C. An impact study of updating background error covariances in the ALADIN-France data assimilation system. J. Geophys. Res. Atmos.
**2013**, 118, 11–075. [Google Scholar] [CrossRef] - Bučánek, A.; Brožková, R. Background error covariances for a BlendVar assimilation system. Tellus A Dyn. Meteorol. Oceanogr.
**2017**, 69, 1355718. [Google Scholar] [CrossRef] - Monteiro, M.; Berre, L. A diagnostic study of time variations of regionally averaged background error covariances. J. Geophys. Res. Atmos.
**2010**, 115. [Google Scholar] [CrossRef][Green Version] - Ménétrier, B.; Montmerle, T.; Berre, L.; Michel, Y. Estimation and diagnosis of heterogeneous flow-dependent background-error covariances at the convective scale using either large or small ensembles. Q. J. R. Meteorol. Soc.
**2014**, 140, 2050–2061. [Google Scholar] [CrossRef] - Buehner, M.; Gauthier, P.; Liu, Z. Evaluation of new estimates of background-and observation-error covariances for variational assimilation. Q. J. R. Meteorol. Soc.
**2005**, 131, 3373–3383. [Google Scholar] [CrossRef] - Hacker, J.; Draper, C.; Madaus, L. Challenges and Opportunities for Data Assimilation in Mountainous Environments. Atmosphere
**2018**, 9, 127. [Google Scholar] [CrossRef] - ALADIN International Team. The ALADIN project: Mesoscale modelling seen as a basic tool for weather forecasting and atmospheric research. WMO Bull.
**1997**, 46, 317–324. [Google Scholar] - Termonia, P.; Fischer, C.; Bazile, E.; Bouyssel, F.; Brožková, R.; Bénard, P.; Bochenek, B.; Degrauwe, D.; Derkova, M.; El Khatib, R.; et al. The ALADIN System and its Canonical Model Configurations AROME CY41T1 and ALARO CY40T1. Geosci. Model Dev. Discuss.
**2017**, 2017, 1–45. [Google Scholar] [CrossRef] - Gerard, L.; Piriou, J.M.; Brožková, R.; Geleyn, J.F.; Banciu, D. Cloud and precipitation parameterization in a meso-gamma-scale operational weather prediction model. Mon. Weather Rev.
**2009**, 137, 3960–3977. [Google Scholar] [CrossRef] - Geleyn, J. Use of a modified Richardson number for parameterizing the effect of shallow convection. J. Meteorol. Soc. Jpn. Ser. II
**1986**, 64, 141–149. [Google Scholar] [CrossRef] - Tudor, M.; Ivatek-Šahdan, S.; Stanešić, A.; Horvath, K.; Hrastinski, M.; Odak Plenković, I.; Bajić, A.; Kovačić, T. Changes in the ALADIN operational suite in Croatia in the period 2011-2015. Hrvatski Meteorološki Časopis
**2016**, 50, 71–89. [Google Scholar] - Stanešić, A. Assimilation system at DHMZ: Development and first verification results. Hrvatski Meteorološki Časopis
**2011**, 44, 3–17. [Google Scholar] - Donlon, C.J.; Martin, M.; Stark, J.; Roberts-Jones, J.; Fiedler, E.; Wimmer, W. The operational sea surface temperature and sea ice analysis (OSTIA) system. Remote Sens. Environ.
**2012**, 116, 140–158. [Google Scholar] [CrossRef] - Ivatek-Šahdan, S.; Stanešić, A.; Tudor, M.; Plenković, I.O.; Janeković, I. Impact of SST on heavy rainfall events on eastern Adriatic during SOP1 of HyMeX. Atmos. Res.
**2018**, 200, 36–59. [Google Scholar] [CrossRef] - Bouttier, F.; Courtier, P. Data Assimilation Concepts and Methods March 1999; Meteorological Training Course Lecture Series; ECMWF: Reading, UK, 2002; p. 59. [Google Scholar]
- Berre, L. Estimation of synoptic and mesoscale forecast error covariances in a limited-area model. Mon. Weather Rev.
**2000**, 128, 644–667. [Google Scholar] [CrossRef] - Buizza, R.; Leutbecher, M.; Isaksen, L. Potential use of an ensemble of analyses in the ECMWF Ensemble Prediction System. Q. J. R. Meteorol. Soc.
**2008**, 134, 2051–2066. [Google Scholar] [CrossRef] - Hoskins, B.J.; McIntyre, M.; Robertson, A.W. On the use and significance of isentropic potential vorticity maps. Q. J. R. Meteorol. Soc.
**1985**, 111, 877–946. [Google Scholar] [CrossRef] - Bölöni, G.; Horvath, K. Diagnosis and tuning of the background error statistics in a variational data assimilation system. Q. J. Hung. Meteorol. Serv.
**2010**, 114, 1–19. [Google Scholar] - Desroziers, G.; Berre, L.; Chapnik, B.; Poli, P. Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc.
**2005**, 131, 3385–3396. [Google Scholar] [CrossRef] - Roebber, P.J. Visualizing multiple measures of forecast quality. Weather Forecast.
**2009**, 24, 601–608. [Google Scholar] [CrossRef] - Ferro, C.A.; Stephenson, D.B. Extremal dependence indices: Improved verification measures for deterministic forecasts of rare binary events. Weather Forecast.
**2011**, 26, 699–713. [Google Scholar] [CrossRef] - Saltikoff, E.; Haase, G.; Delobbe, L.; Gaussiat, N.; Martet, M.; Idziorek, D.; Leijnse, H.; Novák, P.; Lukach, M.; Stephan, K. OPERA the Radar Project. Atmosphere
**2019**, 10, 320. [Google Scholar] [CrossRef]

**Figure 1.**Normalized standard deviation of the surface pressure for (

**a**) ENS, (

**c**) ENSLBC and (

**e**) NMC experiments. The normalized standard deviation of the specific humidity at level 34 (approximately 500 hPa) for (

**b**) ENS, (

**d**) ENSLBC and (

**f**) NMC experiment. The normalization was performed by dividing the standard deviations by the maximum horizontal value of the standard deviation. The maximum values of the standard deviations for the surface pressure were: ENS: 0.6 hPa; ENSLBC: 0.77 hPa; NMC: 6.76 hPa. The maximum values of the standard deviations for the specific humidity at level 34 were as follows: ENS: 0.26 g/kg; ENSLBC: 0.29 g/kg; NMC: 0.55 g/kg.

**Figure 2.**Horizontal correlation spectra of (

**a**) temperature, (

**b**) specific humidity, (

**c**) divergence and (

**d**) vorticity on level 34, which is located approximately near the 500 hPa level for the ENS (blue; full lines), ENSLBC (orange; dashed lines) and NMC (green; dotted lines) experiments.

**Figure 3.**Vertical profile of the horizontally averaged standard deviations for (

**a**) temperature, (

**b**) specific humidity, (

**c**) divergence and (

**d**) vorticity for the ENS (blue; full lines), ENSLBC (orange; dashed lines) and NMC (green; dotted lines) experiments.

**Figure 4.**Vertical profile of the horizontal correlation length scale for (

**a**) temperature and surface pressure, (

**b**) humidity, (

**c**) divergence and (

**d**) vorticity for the ENS (blue; full lines), ENSLBC (orange; dashed lines) and NMC (green; dotted lines) experiments.

**Figure 5.**Vertical profile of the horizontally averaged vertical correlations at level 34 (approximately 500 hPa) for (

**a**) temperature and (

**b**) vorticity for the ENS (blue; full lines), ENSLBC (orange; dashed lines) and NMC (green; dotted lines) experiments.

**Figure 6.**(

**a**) Vertical profile of the horizontally averaged standard deviations for temperature, (

**b**) Vertical profile of the horizontal correlation length scale for humidity, (

**c**) Vertical profile of the horizontally averaged vertical correlations at level 34 (approximately 500 hPa) for divergence, (

**d**) Horizontal correlation spectrum of vorticity and for NMC1224 (blue; full lines), NMC2436 (orange; dashed lines) and NMC (green; dotted lines) experiments.

**Figure 7.**Vertical cross section of the analysis increments due to single radiosonde temperature observations with an innovation of 1K at approximately 500 hPa normalized by its maximum value and plotted with contours with levels from −1 to 1 K with 0.25 K increments for (

**a**) temperature, (

**b**) specific humidity, (

**c**) zonal wind component, and (

**d**) meridional wind component. A zero contour line was omitted to maintain the clearness of the plot. The experiment where the ENSLBC B matrix was used was denoted with blue full lines for positive values and blue dotted lines for negative values, while the experiment where the NMC B matrix was used was denoted with red full lines for positive values and red dashed lines for negative values. The location of the observation was marked with a black dot.

**Figure 8.**Mean vertical analysis increments for the month of June 2017 and for (

**a**) temperature, (

**b**) specific humidity, (

**c**) divergence, (

**d**) vorticity. The analysis increments for the NMC experiment (solid red), and the ENSLBC experiment (dashed blue) were horizontally averaged by the model level and in time over all analyses performed in the data-assimilation cycle during the month of June 2017.

**Figure 9.**Vertical profile of the bias (dashed line) and root-mean-square error (RMSE) (full line) for (

**a**) temperature [K], (

**b**) geopotential height [m], (

**c**) specific humidity [g/kg] and (

**d**) wind speed [m/s] from the analysis and for the experiments: ENSLBC (blue) and NMC (red). The number of data used in the verification is shown with the gray dashed line on top of the x-axis. The statistical scores were computed against radiosonde observations at 00 UTC over all radiosonde stations in the domain and over June 2017 and they represent analysis fit to the assimilated observations.

**Figure 10.**Temporal evolution of the surface pressure tendency mean (dashed line) and root-mean-square (full line) averaged over the domain and over 11 forecasts as a function of the forecast lead time for the ENSLBC (blue) and NMC experiments (red).

**Figure 11.**Normalized mean root-mean-square difference between the ENSLBC and NMC experiment for (

**a**) mean sea level pressure, (

**b**) cloud cover during 24 hours of forecasting computed over all surface observations inside the domain (700 stations) for June 2017. The error bars show the 95% confidence intervals using the Student’s t-test (the negative values indicate that the ENSLBC experiment is better; the difference is significant if the error bars do not cross the zero line.

**Figure 12.**Vertical profile of the bias (dashed line) and standard deviation (STD) (full line) for (

**a**) temperature [K], (

**b**) geopotential height [m], (

**c**) specific humidity [g/kg] and (

**d**) wind speed [m/s] for the experiments: ENSLBC (blue) and NMC (red). The number of data used in the verification is shown with the gray dashed line on top of the x-axis. The verification scores were computed against radiosonde observations at 12 UTC (using 12- and 36-hour forecasts initialized at 00UTC) over all radiosonde stations in the domain and over June 2017.

**Figure 13.**Normalized root-mean-square difference between the ENSLBC and NMC experiments for (

**a**) temperature, (

**b**) specific humidity, (

**c**) wind speed and (

**d**) geopotential height at 500 hPa during 48 hours of forecasting computed against radiosonde observations for June 2017. The error bars show the 95% confidence intervals using the Student’s t-test (the negative values indicate that the ENSLBC experiment is better; the difference is significant if the error bars do not cross the zero line (black).

**Figure 14.**First row: A Wilson diagram for the 12-hour precipitation for June 2017 for the NMC experiment (red) and for the ENSLBC experiment (blue). On the y-axis, the hit rate is shown. On the x-axis, the false alarm ratio is shown. The orange contours denote the threat score and the black lines denote the frequency bias. The ideal score is in the upper left corner. The verification was performed for different thresholds of the 12-hour accumulated precipitation, and they are indicated on the graphs with different symbols. Second row: The symmetric extremal dependence index for the 12-hour precipitation for June 2017 and for the NMC experiment (red) and for the ENSLBC experiment (blue). The thresholds used are indicated in the plot with arrows and were the same as for the Wilson diagram [mm/12 h]: 0.1, 0.3, 1, 3, 10, and 30.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Stanesic, A.; Horvath, K.; Keresturi, E.
Comparison of NMC and Ensemble-Based Climatological Background-Error Covariances in an Operational Limited-Area Data Assimilation System. *Atmosphere* **2019**, *10*, 570.
https://doi.org/10.3390/atmos10100570

**AMA Style**

Stanesic A, Horvath K, Keresturi E.
Comparison of NMC and Ensemble-Based Climatological Background-Error Covariances in an Operational Limited-Area Data Assimilation System. *Atmosphere*. 2019; 10(10):570.
https://doi.org/10.3390/atmos10100570

**Chicago/Turabian Style**

Stanesic, Antonio, Kristian Horvath, and Endi Keresturi.
2019. "Comparison of NMC and Ensemble-Based Climatological Background-Error Covariances in an Operational Limited-Area Data Assimilation System" *Atmosphere* 10, no. 10: 570.
https://doi.org/10.3390/atmos10100570