Efficient Formulation and Implementation of Data Assimilation Methods

Abstract

This Special Issue presents efficient formulations and implementations of sequential and variational data assimilation methods. The methods address three important issues in the context of operational data assimilation: efficient implementation of localization methods, sampling methods for approaching posterior ensembles under non-linear model errors, and adjoint-free formulations of four dimensional variational methods.

1. Efficient Formulation and Implementation of Data Assimilation Methods

Data Assimilation is the process by which imperfect numerical forecasts and sparse observational networks are fused in order to estimate the state ${\mathbf{x}}^{*}\in {\mathbb{R}}^{n\times 1}$ of a system [1,2] which (approximately) evolves according to some model operator,

$$\begin{array}{c}\hfill {\displaystyle {\mathbf{x}}_p^{*}={\mathcal{M}}_{t_{p‒1}\to t_p}\left({\mathbf{x}}_{p‒1}^{*}\right),\mathrm{for}1\le p\le M,}\end{array}$$

(1)

where, for instance, $\mathcal{M}:{\mathbb{R}}^{n\times 1}\to {\mathbb{R}}^{n\times 1}$ is a numerical model which mimics the ocean and/or the atmosphere dynamics, n is the number of model components, M is the number of observations (which form the assimilation window), and p denotes time index at time $t_p$. Sequential and smoothing methods are commonly utilized in order to perform the estimation process [3,4,5]. In the context of sequential data assimilation, when Gaussian assumptions are done over background and observational errors, based on Bayes rule, the posterior mode of the error distribution can be computed as follows:

$$\begin{array}{c}\hfill {\displaystyle {\mathbf{x}}^a={\mathbf{x}}^b+\mathbf{A}·{\mathbf{H}}^T·{\mathbf{R}}^{‒1}·\left[\mathbf{y}‒\mathcal{H}\left({\mathbf{x}}^b\right)\right]\in {\mathbb{R}}^{n\times 1},}\end{array}$$

(2a)

where ${\mathbf{x}}^a\in {\mathbb{R}}^{n\times 1}$ is known as the analysis state. The analysis covariance matrix reads as

$$\begin{array}{c}\hfill {\displaystyle \mathbf{A}={\left[{\mathbf{B}}^{‒1}+{\mathbf{H}}^T·{\mathbf{R}}^{‒1}·\mathbf{H}\right]}^{‒1}\in {\mathbb{R}}^{n\times n},}\end{array}$$

(2b)

where m is the number of observed components from the model domain, $\mathcal{H}:{\mathbb{R}}^{n\times 1}\to {\mathbb{R}}^{m\times 1}$ is the observation operator, $\mathbf{B}\in {\mathbb{R}}^{n\times n}$ is the unknown background error covariance matrix, and $\mathbf{R}\in {\mathbb{R}}^{m\times m}$ stands for the data error covariance matrix. Likewise, ${\mathcal{H}}^{\prime }\left(\mathbf{x}\right)\approx {\mathbf{H}}^T\in {\mathbb{R}}^{n\times m}$ is a linearized observation operator (with the linearization performed about the background state). Typically, the moments of the prior distribution,

$$\begin{array}{c}\hfill {\displaystyle \mathbf{x}\sim \mathcal{N}\left({\mathbf{x}}^b,\mathbf{B}\right),}\end{array}$$

(3)

can be estimated based on an ensemble of model realizations [6]. However, since ensemble members come at high computational costs owing to current operational data assimilation settings (i.e., numerical grid resolutions), ensemble sizes are bounded by the hundreds, while their underlying error distributions range on the order of billions [7]. Consequently, sampling errors can impact the quality of the analysis state [8]. In practice, localization methods can be utilized in order to mitigate the impact of sampling errors during the assimilation steps [9]. However, the implementation of localization methods is not immediate, and further analyses are needed before they can be implemented. Besides, for highly non-linear observation operators, ensemble-based methods such as the ensemble Kalman filter (EnKF) can fail to obtain reasonable estimates of posterior moments. Another important issue is that prior errors might not follow a normal distribution (as is commonly assumed), and therefore non-Gaussian models should be chosen to describe prior error distributions. For instance, Gaussian mixture models (GMMs) are an option in this context. This Special Issue addresses all of these important concerns in the context of sequential and variational data assimilation:

• In the EnKF implementation based on a modified Cholesky decomposition (EnKF-MC) [10,11], the covariance matrix estimator proposed by Bickel and Levina in [12] and the conditional independence of model components regarding their spatial distances are exploited in order to obtain sparse Cholesky factors of the precision background error covariance matrix. This is done in order to reduce the computational cost of the analysis step, and to mitigate the impact of spurious correlations during the assimilation of observations. Given the relation between ${\mathbf{A}}^{‒1}$ and ${\mathbf{B}}^{‒1}$ in (2b) and by using the Bickel and Levina estimator, Nino-Ruiz proposes a “A Matrix-Free Posterior Ensemble Kalman Filter Implementation Based on a Modified Cholesky Decomposition” [13].
• Non-linear observation operators can be commonly found in the context of observations mapped from satellite radiances. Consequently, posterior kernels of error distributions are no longer Gaussian. Thus, alternatives to EnKF formulations are a must under such circumstances, and therefore, sampling methods based on Markov chain Monte Carlo (MCMC) methods can be exploited to successfully sample from posterior error distributions. In “Cluster Sampling Filters for Non-Gaussian Data Assimilation” [14], Attia et al. propose filters which account for non-Gaussian errors in prior and observations. Furthermore, the convergence of MCMC is sped up by using Verlet integrators. On the other hand, Nino-Ruiz et al. [15] proposes “A Robust Non-Gaussian Data Assimilation Method for Highly Non-Linear Models” wherein prior errors are modeled by fitting GMMs while gradient approximations of the three-dimensional variational cost function are exploited for accelerating its convergence towards posterior modes.
• The application to actual scenarios of operational data assimilation methods are widely discussed by Soldatenko et al. in [16] and by Kou et al. in [17].

We hope that you enjoy reading this Special Issue about efficient formulations and implementations of ensemble-based methods.