Multivariate Interpolation of Wind Field Based on Gaussian Process Regression

Abstract

The resolution of the products of numerical weather prediction is limited by the resolution of numerical models and computing resources, which can be improved accurately by a well-chosen interpolation algorithm. This paper is intended to improve the accuracy of spatial interpolation towards wind fields. A new composited multi-scale anisotropic kernel function for weather processes using two-dimensional space information is proposed. To fix the underfitting in this kernel caused by unilateral space information, multiple variables (wind direction, air temperature, and atmospheric pressure) are introduced, which generates a multivariate correction model based on the novel kernel function and Gaussian process regression. Focusing on different weather processes, two multivariate correction models are designed. The new models pave a new way to employ multi-scale local information, and extract the anisotropy and structure information. The experiments on 10 m wind fields for the weather processes without cyclones and for the weather processes with cyclones validate the efficiency. The mean RMSE of the multivariate correction model for the weather processes without cyclones is reduced by around 15% for the u wind component compared with that of a simple composited kernel. For the weather processes with cyclones, the mean RMSE of the novel model declines by around 55% compared to that of spline, and by about 95% compared to that of back propagation neural networks.

1. Introduction

The reanalysis and interpretation of products of numerical weather prediction (NWP) is essential in an NWP system, which is of great importance for the improvement in accuracy of prediction systems [1,2,3,4,5]. However, limited by the resolution of numerical models, computing power, and storage capacity, the resolution of such products are not adequate for more accurate inferences [6,7], such as extreme weather processes analyses. A well-defined interpolation algorithm can increase the resolution of these products more accurately.

Conventional surface fitting methods, such as bilinear and spline methods, are only applicable to specific cases [8]. They are deterministic without taking the randomness of natural weather events into account. Interpolations only rely on local information without taking the anisotropy, physical characteristics, spatial dependences, etc. into consideration [9].

To overcome these problems, several interpolation methods based on different models have been proposed: fuzzy inference [10], hybrid models such as artificial neural networks (ANNs) combined with Kalman filters (KFs) [11], ANNs conjuncted with Markov chains [12], kernel machines such as weighted least square support vector machines (WLS-SVMs) [13], Gaussian process regression (GPR) [14,15,16,17], and so forth. Hybrid models and kernel machines are the mainstay of these new approaches.

With appropriate hybrid components, hybrid models are capable of handling both stationary and nonstationary series. The model that is composed of an autoregressive integrated moving average (ARIMA) model and an ANN is more efficient than either an ANN or an ARIMA alone when forecasting wind speed [18]. Moreover, the KF-ANN model [11], the ARIMA-ANN model [19], and the ARIMA-Kalman model [19] all perform well in forecasting nonlinear wind speed. The increasing number of imported algorithms makes the hybrid models become more effective but more difficult to operate, especially in choosing hybrid components and fine tuning.

Compared with hybrid models, kernel machines (such as SVM and GPR) are easier to operate and can give rise to acceptable results. The proper kernel functions play an important role in kernel machines while predicting winds speed [20,21]. Excluding the choice of kernels, tuning skills required by kernel machines are not as difficult as that by hybrid models because of the single model framework. Hence, kernel machines are the more versatile and simpler models compared with hybrid models.

GPR is one of the most popular methods among various kernel machines. Firstly, the Gaussian process (GP) defines a particular distribution for any finite number of samples. The distribution functions, which are called the family of finite dimensional distributions, reasonably describe the randomness of each sample. For localized and instantaneous weather processes, the sampling environment of each sample cannot be identical. Sampling in the same position at different moments or at the same moment in different positions will result in different distributions for each sample. Secondly, owing to the instantaneity of weather processes [22,23], the influence radius and influence intensity of the wind genesis factors are different. By composing kernel functions, GPR is applied to multi-scale inference, which can extract different radii and intensities [24]. Thirdly, most weather processes with cyclones are non-stationary, whereas the kernel trick in GPR, which means mapping the features in low-dimensional space into high-dimensional space, can handle them. In addition, as a nonparametric algorithm with no demand of the professional tuning skills, it guarantees universality and portability.

Most GPR applications are proposed for time series [14,15,16,17] that handle the wind data sampled in one position at multiple moments. They confirm the efficiency of GPR, but they are not appropriate for interpolating the space series of wind fields sampled from different regions at the same time or sampled from the same time but from different regions. The reasons can be listed as follows. First, most methods for time series take time, or the series itself as the explanatory feature [11,18,19], which is one-dimensional. However, the spatial distributions of the wind fields are two-dimensional at least. Substantially, air temperature, atmospheric pressure, and spatial inhomogeneity influence wind genesis [25,26,27]. Furthermore, for each factor, the influencing radius is not the same [22,23]. A single kernel cannot capture this multi-scale information [28]. In addition, the non-stationary of the spatial series of the weather processes with cyclones is more obvious. However, the kernel for interpolating time series is mostly designed for stationary series [24]. Therefore, considering only a one-dimensional interpreting variable and a single scale without handling the anisotropy, and designing interpolation algorithms only for stationary series, models are blind to spatial series of weather processes. This is a challenge when accurately improving the resolution of products released by NWP.

Considering the relationship of the factors of wind genesis [29,30,31], introducing more information may be one way to meet this challenge. The SVM model, which takes wind speed and temperature or wind speed and wind direction as the input features, generates ideal predictions [32,33]. Grover et al. adopted wind direction, space distance, pressure, and temperature to infer long-range spatial dependencies by GPR [34]. All models work but are not applicable for determining weather processes with cyclones, such as typhoons, which are multi-range dependent and instantaneous. SVM could not capture the different distribution of each sample, which is natural in weather processes. Although GPR can fix this problem by importing random processes, Grover’s proposal can only be used to figure out long-range dependencies. Additionally, the single squared exponential kernel in their method is too smooth for weather processes. To handle multi-range dependencies of the weather processes with cyclones, a multi-scale kernel is required, and more features are needed to correct the underfitting parameters trained by the unilateral spatial information.

Taking all of the above into account, a two-dimensional multi-scale anisotropic kernel function based on GPR is proposed. Taking space coordinates (longitude and latitude) as explanatory variables, the kernel is designed to capture detailed features. Despite the one-sided spatial information, the first principle feature is also introduced as the correction feature to help correct the hyperparameters of the multi-scale anisotropic kernel function. It is generated from air temperature, air pressure and wind direction by principle component analysis (PCA). Different models are specified for weather processes without cyclones and weather processes with cyclones.

The rest of the paper is organized as follows. The basis of the GPR model and the kernel functions associated with weather processes are described in Section 2. Section 3 provides details about the multi-scale anisotropic kernel function for weather processes and the multivariate correction models for spatial interpolation based on GPR. The numerical experiment in Section 4 shows the efficiency of the new models. The dataset used here is ERA Interim [35], a 10 m wind field dataset released by the European Centre for Medium-Range Weather Forecasts (ECMWF). The last section provides conclusions.

2. GPR and Kernel Functions Associated with Weather Processes

2.1. GPR with the Kernel Trick

GPR is a nonparametric algorithm based on a Bayesian framework. The predictions of any finite number have a joint Gaussian distribution [24], which is implied by Equation (3).

Assume there is a training set $\mathcal{D}$ of n observations, $\mathcal{D}=\left\{\left({\mathbf{x}}_i,y_i\right)|i=1,2,\cdots ,n\right\}$, where $\mathbf{x}$ denotes the D-dimensional input vector, and y denotes the scalar observed label of $\mathbf{x}$. $\mathbf{X}$, $\mathbf{y}$ represent the collection of $\mathbf{x}$, y, respectively. The test set ${\mathcal{D}}^{\prime }$ of m test input ${\mathbf{x}}_{\ast }$ is denoted as ${\mathcal{D}}^{\prime }=\left\{\left({\mathbf{x}}_{\ast }i\right)|i=1,2,\cdots ,m\right\}$. ${\mathbf{X}}_{\ast }$ represent the collection of ${\mathbf{x}}_{\ast }$.

Given the training input $\mathbf{X}$ and the corresponding scalar labels $\mathbf{y}$, the optimal interpolation value $y_{\ast }$ at any test input ${\mathbf{x}}_{\ast }$ in ${\mathbf{X}}_{\ast }$ is

$$y_{\ast }=K_{\ast }{K}^{-1}\mathbf{y}.$$

(1)

The prediction $y_{\ast }$ follows a Gaussian distribution

$$p\left(y_{\ast }|{\mathbf{x}}_{\ast },\mathbf{X},\mathbf{y}\right)=\mathcal{N}\left(K_{\ast }{K}^{-1}\mathbf{y},K_{\ast \ast }-K_{\ast }{K}^{-1}{K}_{\ast }\right),$$

(2)

and the joint distribution of $\mathbf{y}$ and $y_{\ast }$ is

$$\begin{array}{c}\left[\begin{array}{c}\mathbf{y}\\ y_{\ast }\end{array}\right]\sim \mathcal{N}\left(0,\left[\begin{array}{cc}K& K_{\ast }\\ K_{\ast }& K_{\ast \ast }\end{array}\right]\right),\end{array}$$

(3)

where K, $K_{\ast }$, and $K_{\ast \ast }$ denote the covariance matrix generated by the kernel function $k\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)$, in which $\mathbf{x},{\mathbf{x}}^{\prime }$ are the input vectors either in the test set or in the training set. Specifically, K denotes the covariance matrix of the training set, $K_{\ast }$ denotes the covariance matrix between the training set and the test set, and $K_{\ast \ast }$ denotes the covariance matrix of the test set.

Projecting the input vectors $\mathbf{x}$ in the D-dimensional space into an N-dimensional feature space, GPR is capable of handling nonlinear problem, which is a common characteristic for weather processes. The idea of projecting is also known as kernel trick.

2.2. The Characteristics of Kernel Functions for Weather Processes

A kernel trick is employed in several kernel machines. An abundance of kernels can be chosen.

Different kernels suit different problems. Some are suitable for linear problems, e.g., the linear covariance function; some focus on periodic variation, e.g., the periodic covariance function; some are sensitive to the diversity in different directions, e.g., the Gabor kernel function; and some perform well while describing rough physical process, e.g., the Matérn kernel function.

The most popular kernel is the squared exponential (SE) covariance function, but it is overly smooth for depicting weather processes [24]. The function is expressed as follows:

$$k_{se}\left(\mathbf{r}\right)=exp\left(-\frac{{\mathbf{r}}^2}{2l^2}\right),$$

(4)

where $\mathbf{r}=\mathbf{x}-{\mathbf{x}}^{\prime }$, and l is the length-scale hyperparameter.

Fixing the over-smoothness of the SE, the Matérn kernel is suitable for approaching weather processes, especially when $\nu =\frac{1}{2}$, or $\nu =\frac{3}{2}$. The general expression of Matérn kernel is

$$k_{matern}\left(\mathbf{r}\right)=\frac{2^{1-\nu }}{\Gamma \left(\nu \right)}{\left(\frac{\sqrt{2\nu }\mathbf{r}}{l}\right)}^{\nu }{K}_{\nu }\left(\frac{\sqrt{2\nu }\mathbf{r}}{l}\right),$$

(5)

where $\nu$ and l are the hyperparameters, $K_{\nu }$ is a modified Bessel function, and $\Gamma$ denotes the Gamma function. When $\nu =\frac{1}{2}$, it is simplified as

$$k_{\nu =\frac{1}{2}}\left(\mathbf{r}\right)=exp\left(-\frac{\mathbf{r}}{l}\right).$$

(6)

The Gabor kernel is preferred in image processing for its capability of edge and texture extraction. It is a short-time Fourier transform (STFT) with a Gaussian window function. The variation in wind direction and wind speed between each wind belt is exactly what the Gabor kernel can capture. It is written as

$$k_{Gabor}\left(\mathbf{r}\right)=exp\left(-\frac{1}{2}{\mathbf{r}}^{\mathrm{T}}{\Lambda }^{-2}\mathbf{r}\cos \left(2\pi {\mathbf{r}}_p^{\mathrm{T}}\mathrm{I}\right)\right),$$

(7)

where $\Lambda =diag\left(l\right),{\mathbf{r}}_p=\left(\mathbf{x}-{\mathbf{x}}^{\prime }\right){\mathbf{p}}^{-1}$, $\mathbf{p}$ denotes the matrix of periodical length-scale hyperparamters, and l denotes the length-scale hyperparamter.

It has been proven that the summation, the product, and the scale of existing kernels are still kernels [24]. Namely,

$$\begin{array}{cc}k\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)& =k_1\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)+k_2\left(\mathbf{x},{\mathbf{x}}^{\prime }\right),\hfill \\ k\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)& =k_1\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)\ast k_2\left(\mathbf{x},{\mathbf{x}}^{\prime }\right),\hfill \\ k\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)& =\alpha k_1\left(\mathbf{x},{\mathbf{x}}^{\prime }\right),\hfill \end{array}$$

(8)

where $\alpha$ denotes a real number. Mapping a kernel to $u\left(\mathbf{x}\right)=\left(\cos \left(\mathbf{x}\right),\sin \left(\mathbf{x}\right)\right)$ periodic space also gives rise to a kernel that extracts periodic information.

A significant advantage of combining kernel functions is the capability of extracting multi-scale information. By combining the kernels mentioned above according to Equation (8), the new kernel will be capable of extracting multiple pieces of information: periodical information, the diversity in different directions, or rough and localized features. The length-scale hyperparameters of kernels imply the influencing radius of each specific feature.

3. The Multivariate Spatial Interpolation Algorithm

3.1. The Multi-Scale Anisotropic Kernel Function

Weather processes are divided into weather processes without cyclones and weather processes with cyclones. Their wind fields are illustrated in Figure 1. Weather processes with cyclones are recognized by the vortexes generated by cyclones. The weather processes without cyclones do not have apparent vortexes.

Considering the analogous distributions of winds between two close positions, the spatial position (longitude and latitude) is taken as the input feature. The closer the positions are, the more similar the winds predicted by the kernel function are.

Moreover, to deal with the anisotropy resulting from the uneven distribution of wind speed and wind direction alongside latitude and longitude, the automatic relevance determination (ARD) pattern is adopted. The anisotropy is implied by $\mathrm{diag}{\left(\mathit{l}\right)}^{-2}$ in each feature dimension, where $\mathit{l}$ denotes the length-scales. The feature with a larger length-scale has a lower influence on the predictions.

For weather processes without cyclones, similar directions of the prevailing winds in each wind belt can be regarded as periodic information. The diversity between wind belts is regarded as the edge, which can be detected by a kernel that is capable of edge extraction. Additionally, real weather processes require a kernel that maintains the roughness rather than filtering it.

The weather processes with cyclones have features similar to those of the weather processes without cyclones. From the typhoon eye to the outside of the vortex, wind fields in the typhoon region change violently but periodically. The vortex texture requires a kernel that is sensitive to the variation in different directions. Additionally, the wind fields of the weather processes with cyclones are rough because of the different changes in wind direction and wind speed alongside each direction.

Considering all of these, the multi-scale anisotropic kernel $k_v$ is proposed for weather processes, which is

$$k_v\left(·,·\right)=k_m\left(·,·\right)+k_p\left(·,·\right)+k_g\left(·,·\right)+k_{\epsilon }\left(·,·\right),$$

(9)

where $k_m,k_p,k_g$, and $k_{\epsilon }$ represent the Matérn kernel when $\nu =\frac{1}{2}$, the periodical Matérn kernel when $\nu =\frac{1}{2}$, the Gabor kernel, and the noise kernel, respectively. The Matérn kernel describes the roughness of weather processes. For weather processes without cyclones, the periodical Matérn kernel justifies the similarity of each belt. The edge between each wind belt is extracted by the Gabor kernel. For the weather processes with cyclones, the periodical Matérn kernel and Gabor kernel are adopted for the variation in the vortex.

Concretely,

$$\begin{array}{c}{k}_m\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)=exp\left(-\mathbf{r}\right),\\ k_p\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)=exp\left(-\mathbf{r}\right),\\ k_g\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)=exp\left(-\frac{1}{2}\mathbf{r}\cos \left(2\pi {\mathbf{r}}_p^{\mathrm{T}}\mathrm{I}\right)\right),\\ k_{\epsilon }\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)={\sigma }^2\delta \left(\mathbf{x},\mathbf{z}\right).\end{array}$$

(10)

For the Matérn $k_m$,

$$\mathbf{r}=\left(\mathbf{x}-{\mathbf{x}}^{\prime }\right){\mathbf{P}}^{-1}{\left(\mathbf{x}-{\mathbf{x}}^{\prime }\right)}^{\mathrm{T}}.$$

(11)

For the periodical Matérn $k_p$,

$$\begin{array}{cc}\mathbf{r}& =\sqrt{\left(u\left(\mathbf{x}\right)-u\left({\mathbf{x}}^{\prime }\right)\right){\mathbf{P}}^{-1}{\left(u\left(\mathbf{x}\right)-u\left({\mathbf{x}}^{\prime }\right)\right)}^{\mathrm{T}}},\hfill \\ u\left(\mathbf{x}\right)& ={\left[\sin 2\pi \frac{\mathbf{x}}{\mathbf{p}}\cos 2\pi \frac{\mathbf{x}}{\mathbf{p}}\right]}^{\mathrm{T}},\hfill \end{array}$$

(12)

where $\delta \left(·,·\right)$ denotes the Kronecker function.

For the Gabor kernel $k_g$,

$$\begin{array}{c}\mathbf{r}={\left(\mathbf{x}-{\mathbf{x}}^{\prime }\right)}^{\mathrm{T}}{\mathbf{P}}^{-1}\left(\mathbf{x}-{\mathbf{x}}^{\prime }\right),\\ {\mathbf{r}}_p=\frac{\mathbf{x}-{\mathbf{x}}^{\prime }}{\mathbf{p}},\end{array}$$

(13)

where $\mathbf{p}$ is the matrix of periodical length.

Inside,

$$\mathbf{P}=\mathrm{diag}{\left(\mathit{l}\right)}^2.$$

(14)

$\mathbf{p}$, $\mathbf{P}$, and ${\sigma }^2$ are unknown here, which are deduced by the logarithm of the marginal likelihood.

3.2. Multivariate Correction Models

Although spacial distributions of wind fields are informative, they are still inadequate to describe the entire fields precisely. Wind genesis is intricate. In addition to the uneven spatial dependence, the influence of Coriolis force, the underlying surface, etc. cannot be ignored [29,30,31,36,37]. Consequently, adopting spatial distribution features only partially determines the natural weather processes. Importing more features can fix that. Thus, the $k_v$ kernel is upgraded to the multivariate correction models $k_{vmp}$ and $k_{vms}$.

Temperature, atmospheric pressure and wind direction are taken as secondary features. However, in view of the potentially redundant information in these features [33], a tradeoff is required. It is proven that the principle components (PCs) corresponding to a large accumulated variance explained ratio of the squared feature vector retains a large possible variance and filters out redundant information in low variability [38].

Assume the secondary features are $\mathbf{X}$, the PCs are deduced by

$$\begin{array}{c}\Sigma \nu =\lambda \nu ,\\ \Sigma ={\mathbf{X}}^{\mathrm{T}}\mathbf{X},\end{array}$$

(15)

where $\Sigma$ denotes the covariance matrix of $\mathbf{X}$, $\nu$ denotes PCs, and $\lambda$ denotes eigenvalues. The accumulated variance explained ratio $r_k$ of top k PCs is

$$r_k=\frac{{\displaystyle \sum _{j=1}^k}{\lambda }_j}{{\displaystyle \sum _{j=1}^n}{\lambda }_j},1\le k\le n,$$

(16)

where the eigenvalues ${\lambda }_1\le {\lambda }_2\le \cdots \le {\lambda }_k\le \cdots \le {\lambda }_n$ correspond to PCs ${\nu }_1,{\nu }_2,\cdots ,{\nu }_k,\cdots ,{\nu }_n$; ${\lambda }_1,\cdots ,{\lambda }_k$ and ${\nu }_1,\cdots ,{\nu }_k$ denote the top k eigenvalues and top k PCs, respectively; and n is the total number of eigenvalues or PCs. The reconstructed new feature ${\mathbf{X}}^{\prime }$ is

$${\mathbf{X}}^{\prime }=\sum _{j=1}^k{\lambda }_j{\nu }_j.$$

(17)

The ratio $r_k$ indicates how much information of $\mathbf{X}$ is preserved in ${\mathbf{X}}^{\prime }$.

In this study, k is set to one because the $r_1$ of the dataset adopted in Section 4 is $75\%$ at least. However, the choice of k remains open for other datasets. The projected feature ${\mathbf{X}}^{\prime }$ of the secondary features $\mathbf{X}$ is introduced as the new correction feature. This helps correct the multi-scale anisotropic kernel function $k_v$.

Importing the correction feature, the multivariate correction models are

$$\begin{array}{ccc}{k}_{vms}\left(·,·\right)& =\hfill & \hfill k_v\left(·,·\right)+k_v^{\prime }\left(·,·\right),\end{array}$$

(18)

which is proposed for the series of the meridional region across wind belts without cyclones, and

$$\begin{array}{ccc}{k}_{vmp}\left(·,·\right)& =\hfill & \hfill k_v\left(·,·\right)\ast k_v^{\prime }\left(·,·\right),\end{array}$$

(19)

which is proposed for the non-stationary series of weather processes with cyclones. Among these, $k_v^{\prime }$ denotes the kernel trained by the correction feature ${\mathbf{X}}^{\prime }$. The correction feature ${\mathbf{X}}^{\prime }$ contains the auxiliary variability information of the secondary features.

4. A Case Study of Wind Fields 10 m in Height

The dataset of 10 m wind fields used in this study is from the ERA Interim dataset published by the ECMWF. It is subject to global atmospheric reanalysis and has been continuously updated in real time since 1979. The region of the data used in this study ranges from $0^{\circ }$ N to ${60}^{\circ }$ N and from ${120}^{\circ }$ E to ${180}^{\circ }$ E. The original resolution of the wind field dataset is $0.125\times {0.125}^{\circ }$. Here, the resolution of the training set is $0.25\times {0.25}^{\circ }$, which is sampled from the original dataset, and the rest are imposed for testing. Concretely, the secondary features are sea surface temperature (SST), mean sea level pressure (MSLP), and wind direction computed from the u wind component and the v wind component.

The efficiency of interpolation is evaluated by the root mean square error (RMSE), which is expressed as

$$\frac{{\displaystyle \sum _{i=1}^n}{\left(y_{\ast ,i}-y_{t,i}\right)}^2}{n},i=1,\cdots ,n,$$

(20)

where $y_{\ast ,i}$ denotes the i-th predicted u or v wind component, $y_{t,i}$ denotes the i-th u or v wind component in the original data, and n denotes the number of the interpolated points. The wind speed of the i-th interpolated point is deduced by

$$\sqrt{{\left(y_i^u\right)}^2+{\left(y_i^v\right)}^2},$$

(21)

where $y_i^u$ and $y_i^v$ denote the u and v wind component either in predictions or in the original data, respectively. The wind speed error of the i-th interpolated point is

$$\sqrt{{\left(y_{t,i}^u\right)}^2+{\left(y_{t,i}^v\right)}^2}-\sqrt{{\left(y_{\ast ,i}^u\right)}^2+{\left(y_{\ast ,i}^v\right)}^2},$$

(22)

where $y_{t,i}^u,y_{t,i}^v$ are the i-th u, v wind component from the original data, respectively; and $y_{\ast ,i}^u$ and $y_{\ast ,i}^v$ are the i-th predicted u and v wind components, respectively.

4.1. Weather Processes without Cyclones

The first experiment compares the efficiency of the $k_v$ and $k_{vms}$ models. Because a large region results in a huge covariance matrix that is computationally expensive to invert, the region is divided into subregions firstly and interpolations are then made on each of them. The grid size of each subregion is $7\times 241$, which is the number of interpolated points.

To obtain a steady estimation of the unknown hyperparameters mentioned in Section 3.1, each training series is processed 100 times. Two kinds of series are used, one is sampled from different regions at the same time, and the other is sampled from the same region at 20 different moments. After training, the hyperparameters of the series from different regions at the same time are used to interpolate the series at the same moment but from different regions. The hyperparameters of the series at different times in the same region are used to interpolate the series of the same region sampled at other times.

The mean RMSE after interpolating the series sampled from different regions with $k_v$ outperforms the result of the spline method, while the RMSE does not, which is shown in Figure 2a,b. Single multi-scale anisotropic kernel $k_v$ is underfitting while interpolating the space series.

Weather processes without cyclones change seasonally. Interpolating the regions sampled from forty different times (within 10 days of the sampling time in the training set) confirms that what $k_v$ extracts is not contradictory. As shown in Figure 2c,d, in most regions, the RMSE, after interpolating the series sampled from the same region but at different times with $k_v$, is always smaller than that after interpolation with the spline method.

When interpolating the space series sampled from the same time, the mean RMSE of all the interpolated regions with $k_{vms}$ is reduced by $10\%$ at least compared with that of the spline method, which is shown in Figure 3a,b. Additionally, the number of regions increases; the RMSE of these regions is smaller than that after using the spline method. This indicates that $k_{vms}$ has captured the multi-scale local spacial feature. Figure 3c,d shows the results after interpolating the space series sampled from the same region at different times. The mean RMSE of Figure 3c,d declines by $21.44\%$ for the u wind component and by $25.13\%$ for the v wind component compared with that of the spline method.

In Figure 4, the promotion of the $k_{vms}$ method, compared with the spline method, outperforms that of the $k_v$ method, which implies that the correction model $k_{vms}$ fixes the one-sided feature trained by single space information.

4.2. Weather Processes with Cylcones

Typhoons Fengshen, Kalmaegi, Fung-wong and Kammuri in 2014 are selected as examples of weather processes with cyclones. Their tracks are shown in Figure 5. Using kernel $k_{vmp}$, 26 regions of Fung-wong that are sampled from 26 different times are trained. The grid size of each region is $81\times 81$, which is the number of interpolated points. After training, the hyperparameters are used to interpolate the other typhoon regions sampled from Fengshen, Kalmaegi, and Kammuri.

Except for the baseline spline, the other baseline for comparison is a back propagation (BP) neural networks, which is composited of several fully connected layers. In this study, the entire work can be regarded as a nonlinear regression function. The BP framework used here is shown in Figure 6. All layers, except the output layers, are activated by ReLu activation functions. The output layer is activated by linear activation functions.

Figure 7 shows the interpolated wind speed and the wind vector field of typhoon Kamurri at 0000 UTC on 26 September 2014. Compared with the reference field shown in Figure 7a, the structural information and scale information are preserved. The field interpolated by BP exhibits an obvious sawtooth phenomenon at the interpolation position. BP adopts wind direction, SST, longitude, and latitude to fit wind fields. It is natural that the result is worse than $k_{vmp}$. There are no scale parameters in BP. What is generated by BP is only a picewise linear function that cannot approach the nonlinear features well. The same features employed in BP are also used to train GPR with a simple multi-scale anisotropic kernel $k_v$, the result is still under-performing compared with the $k_{vmp}$ model. This implies that the major interpreting variables (longitude and latitude) make more sense than the other features, while interpolating wind fields. Spline behaves well owing to the fact that it takes a small number of points around the target point to approach. Although the spline method does not take longer-dependencies into consideration, the result is still acceptable because the weather processes with cyclones dramatically change mainly due to the local environment. However, as shown in Figure 8, Figure 9 and Figure 10, the result of the spline method is still over-smooth in some positions.

Figure 8, Figure 9 and Figure 10 show the speed error after interpolating. Figure 8 shows the region of Kamurri at 1800 UTC on 28 September, Figure 9 gives the region of Fengshen at 0000 UTC on 8 September, and Figure 10 shows the region of Kalmaegi at 1800 UTC on 13 September. The error bar of the result of $k_{vmp}$ in Figure 8 centers around 0.4, which indicates that the mean RMSE of $k_{vmp}$ is smaller than that of the spline and BP methods. The speed error of $k_{vmp}$ shown in Figure 9 and Figure 10 also validates the improvement. Considering the instantaneous variation of wind speed and wind direction in the vortex, the approach function cannot be too smooth. However, the spline method is over-smooth because it involves two times continuous derivatives. For the Matérn kernel function in the $k_{vmp}$ model, the process is mean square continuous (if a sequence of points ${\mathbf{x}}_{\mathbf{1}},\cdots ,{\mathbf{x}}_{\mathbf{n}}$ and a fixed point ${\mathbf{x}}_{\ast }$ are assumed, then a process $f\left(\mathbf{x}\right)$ is mean square continuous at ${\mathbf{x}}_{\ast }$ if $\mathbb{E}\left[{\left|f\left({\mathbf{x}}_k\right)-f\left({\mathbf{x}}_{\ast }\right)\right|}^2\right]\to 0$ as $k\to \infty$), but not mean square differential (if a covariance function $k\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)$ is assumed, the mean square derivative in the i-th direction of a process $f\left(\mathbf{x}\right)$ is defined as $\frac{\partial f\left(\mathbf{x}\right)}{\partial x_i}=\frac{{\partial }^{2}k\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)}{\partial x_i\partial x_i^{\prime }}$, where $\mathbf{x},{\mathbf{x}}^{\prime }$ denotes two different input vectors, and $x_i,x_i^{\prime }$ denote the vector in the i direction) [24]. This leads to the reasonable roughness that the weather processes require because the smooth function is infinitely derivable in the domain of definition.

The improvement of the mean RMSE in all interpolated regions sampled from Fengshen, Kalmaegi, and Kammuri, after using the hyperparameters trained by Fung-wong, is shown in

Table 1. The improvement of RMSE after interpolating typhoon regions with $k_{vmp}$, spline, and BP methods.

Wind Component	${\mathit{k}}_{\mathit{vmp}}$	Spline	bp	$\frac{{\mathit{k}}_{\mathit{vmp}}-\mathbf{spline}}{\mathbf{spline}}$(%)	$\frac{{\mathit{k}}_{\mathit{vmp}}-\mathbf{bp}}{\mathbf{bp}}$(%)
u	0.0492	0.1176	0.9747	58.16	94.95
v	0.0422	0.0953	0.9822	55.72	95.71

.

There is always a tradeoff between accuracy and cost. Although the result of multivariate correction models (either $k_{vms}$ or $k_{vmp}$) outperforms that of the spline method, the interpolation time of these methods does not. The computing demand of the naive implementation in this experiment is nearly 100 times greater than that of the spline method. The complexity of the spline method is $\mathcal{O}\left(mn\right)$ [40], while the main cost of GPR increases as $\mathcal{O}\left(n^3\right)$ [24]. The inverse of the covariance matrix requires more computing resources. Works focusing on reducing the computing cost of the inverse of the covariance matrix have been published. Fine and Scheinberg proposed the low-rank incomplete Cholesky factorization, which take $\mathcal{O}\left(n{k}^2\right)$ if the number of decomposition pivots that are greater than the threshold is k [41]. Bae D.H. et al. optimize the memory distribution in a cloud computing system for a massive matrix and made excellent progress [42]. After optimization and parallelization, this method is expected to be operationalized in quickly processing global data generated by NWP systems.

5. Conclusions

In this study, to improve the resolution of products released by the NWP system, a two-dimensional multi-scale anisotropic kernel $k_v$ for the spatial interpolation of wind fields is proposed. To fix the partial information trained by space information, multivariate correction models, namely the $k_{vms}$ model for weather processes without cyclones and the $k_{vmp}$ model for weather processes with cyclones, were designed by importing a correction feature (the first principle component of temperature, pressure, and wind direction generated by PCA). The numerical examples of 10 m wind fields in ERA Interim reanalysis demonstrate improvement. Given correction feature, $k_{vms}$ outperforms the spline and simple $k_v$ models. For weather processes with cyclones, interpolating typhoon regions with $k_{vmp}$ generates the best result. The $k_{vmp}$ model works well compared with the spline and BP methods.

The interpolation time of the models remains undetermined. To construct models that can quickly improve the resolution of global data released by the NWP system, research will focus on reductions in the computation of this naive experiment. Possible approaches include optimizing the inverse algorithm of the massive matrix, GPR sparse inferences or paralleling. After optimization, these new models can be used in variational data assimilation to improve interpolation operators, and this will lead to more accurate forecasting.