Functional Location-Scale Model to Forecast Bivariate Pollution Episodes

Abstract

Predicting anomalous emission of pollutants into the atmosphere well in advance is crucial for industries emitting such elements, since it allows them to take corrective measures aimed to avoid such emissions and their consequences. In this work, we propose a functional location-scale model to predict in advance pollution episodes where two pollutants are involved. Functional generalized additive models (FGAMs) are used to estimate the means and variances of the model, as well as the correlation between both pollutants. The method not only forecasts the concentrations of both pollutants, it also estimates an uncertainty region where the concentrations of both pollutants should be located, given a specific level of uncertainty. The performance of the model was evaluated using real data of SO${}_2$ and NO${}_x$ emissions from a coal-fired power station, obtaining good results.

1. Introduction

Forecasting air quality and concentrations of pollutants in the atmosphere by means of statistical methods is an active area of research given the transcendence of the problem and the difficulty to find optimal solutions using deterministic mathematical models. Among the different methods that can be found in the literature to tackle this problem, models for time series analysis such as the integrated autoregressive moving average—ARIMA [1,2,3], multivariate regression [4,5,6,7], generalized linear or additive models (GAM) [8,9,10,11] and artificial neural networks (ANN) [12,13,14,15,16,17,18,19] are the most extended. Due to the increased access to continuous data over time, functional data analysis [20,21] was also proposed for air quality forecasting and outlier detection [22,23,24]. Parametric [25,26] and nonparametric [27,28,29] functional regression methods were tested. A functional framework allows considering the inherent correlation between observations, instead of considering them as independent realizations of an underlying stochastic process. Some functional approaches add related meteorological variables to the models [30,31,32,33,34], which can improve the result of the predictions and help to understand the process underlying the evolution of the pollutants.

Most of the documents in the literature propose solutions to predict the concentration of each pollutant individually, being much scarcer those focused on predicting more than one pollutant at a time. Vector autoregressive moving average (VARMA) [35,36] and vector autoregressive integrated moving average (VARIMA) [37] models were applied to reach this objective. In this work, we proposed a method for the simultaneous forecasting of pollution episodes when two pollutants, i.e., SO${}_2$ and NO${}_x$, are involved. Apart from transport, one of the main sources of these pollutants is public electricity and heating. Their negative effects on human health are well known, and goes for mild (i.e., eyes irritated, nose or headache) to severe (i.e., lung damage or reduced oxygenation of tissues). They also have negative effects on animals and plants, as well as in other substances, such as water and soils. In addition, NO${}_x$ is a precursor of the tropospheric ozone. High levels of ozone contributes to climate change, cause adverse impacts on health and can damage vegetation.

Pollution episodes (incidents) are abnormally large emissions of one or more pollutants in short periods of time. Although the improvement of the chemical processes and particle filter systems have significantly reduced the amount and intensity of the pollution episodes, they are still of particular interest for the industries, as they may be subject to sanctions, or for other reasons, such as public health deterioration or industry discredit. Therefore, pollution industries, such as coal-fired power plants, are very interested in determining in advance when these episodes of excessive contamination might occur. Specifically, this is the purpose of our work: forecasting pollution episodes of SO${}_2$ and NO${}_x$ early enough to allow corrective measures to be taken. Our approach uses a location-scale model [11,38,39] that treats the predictors, the concentrations of both pollutants over time, as functions, while the response is a scalar, the concentration of the pollutants some time in advance. The novelty of our approach is the combination of a biviariate location-scale model with functional additive models. This method combines the simplicity of the location and scale models with the capacity of functional data analysis to deal with data in the form of functions.

The document is structured as follows: In Section 2 we show the mathematical model proposed to solve the problem under analysis and the algorithm used to estimate a solution from the data. Section 3 is devoted to test the validity of the model using real data. Finally, a discussion of the results and the main conclusions of our work are exposed in Section 4.

2. Methodology

2.1. Mathematical Model

Let ${\{{\mathbf{X}}_i,{\mathbf{Y}}_i\}}_{i=1}^n$ be a set of observations of a stochastic process, $\mathbf{X}=\left(X^1\left(t\right),\dots ,X^p\left(t\right)\right)$, where $X^j\left(t\right)\in L_2\left[0,T\right],j=1,\dots ,p$, are predictor covariates and $\mathbf{Y}=(Y_1,Y_2)$, with $Y_j\in \mathbb{R}$, a response variable. In this context, the following bivariate location-scale model [40,41] is assumed

$$\left(\begin{array}{c}\hfill Y_1\\ \hfill Y_2\end{array}\right)=\left(\begin{array}{c}\hfill {\mu }_1\left(\mathbf{X}\right)\\ \hfill {\mu }_2\left(\mathbf{X}\right)\end{array}\right)+{\mathsf{\Sigma }}^{1/2}\left(\mathbf{X}\right)\left(\begin{array}{c}\hfill {\epsilon }_1\\ \hfill {\epsilon }_2\end{array}\right)$$

(1)

where ${\mathsf{\Sigma }}^{1/2}\left(\mathbf{X}\right)$ represents the Cholesky decomposition of the variance-covariance matrix $\mathsf{\Sigma }\left(\mathbf{X}\right)$

$$\mathsf{\Sigma }\left(\mathbf{X}\right)=\left(\begin{array}{cc}\hfill {\sigma }_1^2\left(\mathbf{X}\right)& {\sigma }_{12}\left(\mathbf{X}\right)\\ \hfill {\sigma }_{12}\left(\mathbf{X}\right)& {\sigma }_2^2\left(\mathbf{X}\right)\end{array}\right)$$

(2)

so that $\mathrm{Var}\left(\mathbf{Y}\right|\mathbf{X})=\mathsf{\Sigma }\left(\mathbf{X}\right)={\mathsf{\Sigma }}^{\mathbf{1}/\mathbf{2}}\left(\mathbf{X}\right){\left({\mathsf{\Sigma }}^{\mathbf{1}/\mathbf{2}}\left(\mathbf{X}\right)\right)}^{\mathbf{T}}$. To guarantee the model identification in (1), the bivariate residuals $({\epsilon }_1,{\epsilon }_2)$ are assumed to be independent of the covariates, with zero mean, unit variance, and zero correlation. Despite we do not assume any distribution for the error term, within the framework of functional data analysis this work might be addressed under the assumption of other structures for error distribution: generalized Gauss-Laplace distribution that relax the constrictive assumption of the normal distribution errors [42], generalized linear mixed models (GLMMs) [38] to estimate random effects and dependent (temporal or spatial) errors, and generalized additive models for location, scale and shape (GAMLSS) [43] to model the dynamically variable distribution, considering skewness and kurtosis.

We define the unconditionally probabilistic region for the errors $({\epsilon }_1,{\epsilon }_2)$ as

$${\epsilon }_{\tau }\left(k\right)=\{({\epsilon }_1,{\epsilon }_2)\in {\mathbb{R}}^2|f({\epsilon }_1,{\epsilon }_2)\ge k\}$$

f being the density function of the bivariate residuals $({\epsilon }_1,{\epsilon }_2)$ and k the $\tau -$quantile of $f({\epsilon }_1,{\epsilon }_2)$. Then, for a given $\mathbf{X}$, we define a conditional $\tau th$- uncertainty region for $(Y_1,Y_2)$ containing $\tau \%$ of the observations as:

$$R_{\tau }\left(\mathbf{X}\right)=\left(\begin{array}{c}\hfill {\mu }_1\left(\mathbf{X}\right)\\ \hfill {\mu }_2\left(\mathbf{X}\right)\end{array}\right)+{\mathsf{\Sigma }}^{1/2}\left(\mathbf{X}\right){\epsilon }_{\tau }$$

(3)

2.2. Estimation Algorithm

To implement an algorithm that allows applying the mathematical model exposed in the previous section, we propose using a functional additive models to estimate the means, variances and covariances in (1). Given a sample of size n, ${\{{\mathbf{X}}_i,(Y_{i1},Y_{i2})\}}_{i=1}^n$, where ${\mathbf{X}}_i=\left(X_i^1\left(t\right),\dots ,X_i^p\left(t\right)\right)$, the steps of the proposed estimation algorithm are the following:

Step 1: Perform a decomposition of each covariate $X^j\left(t\right)$ in basis functions of the form $X^j\left(t\right)\approx {\sum }_{k=1}^K{\xi }_k^j{\varphi }_k\left(t\right)$, where ${\varphi }_k$ ($k=1,\dots ,K$) are K basis functions (i.e., B-splines, wavelets), and ${\xi }_{il}$ are either the coefficients of an expansion in fixed basis or the principal component scores of the Karhunen-Loève expansion [44,45]. As a result, we obtain the transformed covariates

$${\tilde{\mathbf{X}}}_i=\left(\left({\xi }_{i1}^1,\dots ,{\xi }_{iK}^1\right);\left({\xi }_{i1}^2,\dots ,{\xi }_{iK}^2\right);\dots ;\left({\xi }_{i1}^p,\dots ,{\xi }_{iK}^p\right)\right)i=1,\dots ,n$$

Step 2: For $r=1,2$, fit an additive model to the sample ${\{{\tilde{\mathbf{X}}}_i,Y_{i1},Y_{i2}\}}_{i=1}^n$ and obtain an estimation of the means

$${\widehat{\mu }}_r\left({\mathbf{X}}_i\right)={\alpha }_r+\sum _{j=1}^p\sum _{k=1}^K{\widehat{f}}_{rk}^j\left({\xi }_{ik}^j\right)$$

(4)

and then estimate ${\sigma }_r^2\left(\mathbf{X}\right)$ from the sample ${\{{\tilde{\mathbf{X}}}_i,{(Y_{ir}-{\widehat{\mu }}_r\left({\mathbf{X}}_i\right))}^2\}}_{i=1}^n$ as

$${\widehat{\sigma }}_r^2\left({\mathbf{X}}_i\right)=exp\left({\widehat{\beta }}_r+\sum _{j=1}^p\sum _{k=1}^K{\widehat{g}}_{rk}^j\left({\xi }_{ik}^j\right)\right)$$

(5)

Then, compute the correlation $\rho \left(\mathbf{X}\right)$, which is related to the covariance by ${\sigma }_{12}\left(\mathbf{X}\right)={\sigma }_1\left(\mathbf{X}\right){\sigma }_2\left(\mathbf{X}\right)\rho \left(\mathbf{X}\right)$, using the sample ${\{{\mathbf{X}}_i,{\widehat{\delta }}_i\}}_{i=1}^n$, as follows:

$$\widehat{\rho }\left({\mathbf{X}}_i\right)=tanh\left(\widehat{\gamma }+\sum _{j=1}^p\sum _{k=1}^K{\widehat{m}}_k^j\left({\xi }_{ik}^j\right)\right)$$

being

$${\widehat{\delta }}_i=\frac{\left(Y_i^1-{\widehat{\mu }}_1\left({\mathbf{X}}_i\right)\right)\left(Y_i^2-{\widehat{\mu }}_2\left({\mathbf{X}}_i\right)\right)}{{\widehat{\sigma }}_1\left({\mathbf{X}}_i\right){\widehat{\sigma }}_2\left({\mathbf{X}}_i\right)}$$

where $f_{rk}^j$, $g_{rk}^j$ and $m_k^j$ are smooth and unknown functions, ${\alpha }_r$, ${\beta }_r$ and $\gamma$ are coefficients, p the number of predictors (covariates), and K the number of basis. Please note that the link functions $H_{\sigma }(·)=exp(·)$ and $H_{\rho }(·)=tanh(·)$ used in the variance and correlation structures, respectively, ensure that the restrictions on the parameter spaces (${\sigma }_r^2\left(\mathbf{X}\right)\ge 0$ and $0\le \rho \left(\mathbf{X}\right)\le 1)$ are maintained. Moreover, in order to guarantee the identification of the model, we assume that all the means of functions $f_j$, $g_j$ and $m_j$ are zero.

Step 3: Compute the standardized residuals

$$\left(\begin{array}{c}\hfill {\widehat{\epsilon }}_{i1}\\ \hfill {\widehat{\epsilon }}_{i2}\end{array}\right)={\widehat{\mathsf{\Sigma }}}^{-1/2}\left({\mathbf{X}}_i\right)\left(\begin{array}{c}\hfill Y_{i1}-{\widehat{\mu }}_1\left({\mathbf{X}}_i\right)\\ \hfill Y_{i2}-{\widehat{\mu }}_2\left({\mathbf{X}}_i\right)\end{array}\right)i=1,\dots ,n$$

(6)

where

$$\widehat{\mathsf{\Sigma }}\left({\mathbf{X}}_i\right)=\left(\begin{array}{cc}\hfill cc{\widehat{\sigma }}_1^2\left({\mathbf{X}}_i\right)& {\widehat{\sigma }}_{12}\left({\mathbf{X}}_i\right)\\ \hfill {\widehat{\sigma }}_{12}\left({\mathbf{X}}_i\right)& {\widehat{\sigma }}_2^2\left({\mathbf{X}}_i\right)\end{array}\right)$$

(7)

with ${\widehat{\sigma }}_{12}\left({\mathbf{X}}_i\right)={\widehat{\sigma }}_1\left({\mathbf{X}}_i\right){\widehat{\sigma }}_2\left({\mathbf{X}}_i\right)\widehat{\rho }\left({\mathbf{X}}_i\right)$.

Step 4: Obtain the kernel estimation of the bivariate density $\widehat{f}({\epsilon }_1,{\epsilon }_2)$ given by

$$\widehat{f}(({\epsilon }_1,{\epsilon }_2),\mathbf{H})=\frac{1}{n}\sum _{i=1}^n{K}_{\mathbf{H}}\left(\begin{array}{c}\hfill {\epsilon }_1-{\widehat{\epsilon }}_{i1}\\ \hfill {\epsilon }_2-{\widehat{\epsilon }}_{i2}\end{array}\right)$$

(8)

where $K(·)$ is the kernel which is a symmetric probability density function and $\mathbf{H}$ is a $2\times 2$ positive definite matrix. Then, obtain the ${\tau }^{th}$ unconditional bivariate uncertainty region on the residual scale as

$${\widehat{\epsilon }}_{\tau }=\left\{({\epsilon }_1,{\epsilon }_2)\right)\in {\mathbb{R}}^2|\widehat{f}({\epsilon }_1,{\epsilon }_2))\ge \widehat{k}\}$$

(9)

$\widehat{k}$ being the empirical $\tau$ quantile of the values $\widehat{f}({\epsilon }_{11},{\epsilon }_{12}),\dots ,\widehat{f}({\epsilon }_{n1},{\epsilon }_{n2})$. Finally, for a given $\mathbf{X}$, the conditional bivariate uncertainty region $R_{\tau }\left(\mathbf{X}\right)$ is estimated according to (3)

$${\widehat{R}}_{\tau }\left(\mathbf{X}\right)=\left(\begin{array}{c}\hfill {\widehat{\mu }}_1\left(\mathbf{X}\right)\\ \hfill {\widehat{\mu }}_2\left(\mathbf{X}\right)\end{array}\right)+{\widehat{\mathsf{\Sigma }}}^{1/2}\left(\mathbf{X}\right){\widehat{\epsilon }}_{\tau }$$

(10)

3. Case Study: Joint Forecasting of $\left({\mathrm{SO}}_2,{\mathrm{NO}}_x\right)$ Pollution Episodes

The mathematical model exposed in the previous section was applied to the forecasting of pollution episodes registered at a coal-fired power station located in the northwest of Spain. ${\mathrm{SO}}_2$ and ${\mathrm{NO}}_x$ are two of the main air pollutants generated by combustion processes, and both have harmful effects on human health. Moreover, it was proven that both pollutants are correlated [46], which is consistent with the model in (1). Fortunately, pollution episodes are not very frequent and the trend is that they will become scarcer as technology advances.

Let $t_0$ be the present time measured each five minutes, and $\mathrm{SO}\left(t_0\right)$ and $\mathrm{NO}\left(t_0\right)$ the concentrations obtained respectively by the series of bi-hourly SO${}_2$ and NO${}_x$ means at instant $t_0$. Being $t_h$ the prediction horizon time, the interest is to predict

$$(Y_1,Y_2)=\left(SO(t_0+t_h),NO(t_0+t_h)\right)$$

and provide an uncertainty region for these estimations given a specific value of $\tau$, using the predictive covariates

$$\mathbf{X}=\left(X^1\left(t\right),X^2\left(t\right),X^3\left(t\right),X^4\left(t\right)\right)=\left(\mathrm{SO}\left(t\right),\mathrm{NO}\left(t\right),{\mathrm{SO}}^{\prime }\left(t\right),{\mathrm{NO}}^{\prime }\left(t\right)\right)\mathrm{with}t\in [t_0-t_{lag},t_0]$$

where $({\mathrm{NO}}^{\prime }\left(t\right),{\mathrm{SO}}^{\prime }\left(t\right))$ represents the first derivatives of the functions that approximate the concentrations of both pollutants. These derivatives are obtained from the functional representation of the discrete data, according to Step 1 of the estimation algorithm. Please note that $t_{lag}$ represents the lagged time used in the predictors. In particular,, we are interested in predicting an hour in advance, according to the requirements of current Spanish legislation and, therefore, we will consider $t_h=12$ (60 min) from now on.

Most of the time, these concentrations times series are low, close to zero, and in order to obtain a reasonably large number of pollution incidents, we took as our sample a historical matrix ${\left\{({\mathbf{X}}_i,{\mathbf{Y}}_i)\right\}}_{i=1}^N$ with pollution data of approximately 12 years, which includes a considerable number of pollution episodes (see [9] for a detailed description of the historical matrix construction). In summary, in the historical matrix not all the data are used, but only part of them, following a quantile-weighted criterion. This means that the larger the concentration, the greater the number of observations of that concentration in the sample. Figure 1 shows a sample of the historical matrix, on top are the curves of both pollutants $\left(\mathrm{NO}\right(t),\mathrm{SO}(t\left)\right)$ measured in $t_{lag}=20$ discretization points and evaluated in 5 B-spline basis functions of order $p=4$. On the bottom, the first derivative of the B-spline curves $({\mathrm{NO}}^{\prime }\left(t\right),{\mathrm{SO}}^{\prime }\left(t\right))$ with order $p-1$ are represented.

In this paper, we tested four models using as predictors different combinations of the covariates $\left(X^1\left(t\right),X^2\left(t\right),X^3\left(t\right),X^4\left(t\right)\right)$ that include the concentrations of both pollutants and their first derivatives. In particular, we will consider models given by:

$$\left(\begin{array}{c}\hfill SO(t_0+12)\\ \hfill NO(t_0+12)\end{array}\right)=\left(\begin{array}{c}\hfill {\mu }_1\left({\mathbf{X}}^1\right)\\ \hfill {\mu }_2\left({\mathbf{X}}^2\right)\end{array}\right)+\left(\begin{array}{cc}\hfill {\sigma }_1^2\left({\mathbf{X}}^1\right)& {\sigma }_{12}({\mathbf{X}}^1,{\mathbf{X}}^2)\\ \hfill {\sigma }_{12}({\mathbf{X}}^1,{\mathbf{X}}^2)& {\sigma }_2^2\left({\mathbf{X}}^2\right)\end{array}\right)\left(\begin{array}{c}\hfill {\epsilon }_1\\ \hfill {\epsilon }_2\end{array}\right)$$

(11)

The four considered models, M${}_1$, M${}_2$, M${}_3$ and M${}_4$, are configured in

Table 1. Selected models from equation in (11). Cross X indicates the covariates included in each of the four considered models. The derivatives of the functions are indicated with a single quote.

	${\mathit{X}}^1$				${\mathit{X}}^2$
Model	$\mathrm{SO}\left(\mathit{t}\right)$	$\mathrm{NO}\left(\mathit{t}\right)$	${\mathrm{SO}}^{\prime }\left(\mathit{t}\right)$	${\mathrm{NO}}^{\prime }\left(\mathit{t}\right)$	$\mathrm{SO}\left(\mathit{t}\right)$	$\mathrm{NO}\left(\mathit{t}\right)$	${\mathrm{SO}}^{\prime }\left(\mathit{t}\right)$	${\mathrm{NO}}^{\prime }\left(\mathit{t}\right)$
M${}_1$	X					X
M${}_2$	X	X			X	X
M${}_3$	X		X			X		X
M${}_4$	X	X	X		X	X	X	X

where the cross X indicates the covariates included in each model.

To validate and compare the four proposed models, we randomly select from the full historical matrix a training set ${\mathbf{M}}^I={\left\{({\mathbf{X}}_i^I,{\mathbf{Y}}_i^I)\right\}}_{i=1}^{n_{train}}$ and a test set ${\mathbf{M}}^{II}={\left\{({\mathbf{X}}_i^{II},{\mathbf{Y}}_i^{II})\right\}}_{i=n_{train}+1}^N$.

The estimates ${\widehat{\mu }}_1$, ${\widehat{\mu }}_2$, $\widehat{\mathsf{\Sigma }}$ were obtained from the samples in the first matrix ${\mathbf{M}}_t^I$. The bivariate uncertainty regions for the values of the covariates on the second matrix ${\mathbf{M}}^{II}$ were obtained using (3).

The estimated coverage $\widehat{\tau }$ is given by

$$\widehat{\tau }=\frac{1}{n_{test}}\sum _{i=n_{train}+1}^{N}I\{{\mathbf{Y}}_i^{II}\in {\widehat{R}}_{\tau }\left({\mathbf{X}}_i^{II}\right)\};n_{test}=N-n_{train}$$

(12)

The performance of the proposed predictors was evaluated in two pollution incidents. A bivariate representation of these episodes is shown in Figure 2. The orientation of the points shows a clear correlation between both pollutants although the range of concentrations is quite different.

The nominal and the estimated coverages for different time lags and training sample sizes are shown in

Table 2. Nominal $\tau$ and estimated $\widehat{\tau }$ coverages for each of the four models under study. Two time lags, $t_{lag}=10$, $t_{lag}=20$, two sizes of the training sample $n_{train}=10,000$ and $n_{train}=4900$, and two numbers of principal components, K = 3 and K = 5, were considered. Results correspond to the test sample.

				$\widehat{\mathit{\tau }}$
$\mathbf{\tau }$	${\mathit{t}}_{\mathit{lag}}$	${\mathit{n}}_{\mathit{train}}^{•}$	K	M${}_1$	M${}_2$	M${}_3$	M${}_4$
0.50	10	20	3	0.43	0.47	0.45	0.51
			5	0.42	0.48	0.47	0.49
	10	49	3	0.51	0.52	0.52	0.52
			5	0.51	0.50	0.50	0.50
	20	20	3	0.45	0.49	0.44	0.49
			5	0.48	0.46	0.43	0.46
	20	49	3	0.50	0.54	0.51	0.50
			5	0.49	0.49	0.49	0.48
0.75	10	20	3	0.70	0.73	0.72	0.75
			5	0.69	0.74	0.73	0.74
	10	49	3	0.76	0.78	0.78	0.78
			5	0.77	0.76	0.76	0.76
	20	20	3	0.70	0.72	0.70	0.72
			5	0.70	0.71	0.69	0.69
	20	49	3	0.77	0.78	0.75	0.73
			5	0.75	0.74	0.72	0.72
0.90	10	20	3	0.88	0.87	0.87	0.89
			5	0.86	0.88	0.87	0.88
	10	49	3	0.91	0.90	0.90	0.90
			5	0.90	0.90	0.90	0.90
	20	20	3	0.87	0.87	0.85	0.86
			5	0.87	0.86	0.86	0.84
	20	49	3	0.90	0.89	0.90	0.86
			5	0.87	0.87	0.88	0.85
0.95	10	20	3	0.93	0.93	0.93	0.93
			5	0.93	0.93	0.93	0.93
	10	49	3	0.96	0.93	0.93	0.93
			5	0.95	0.94	0.94	0.94
	20	20	3	0.93	0.93	0.92	0.92
			5	0.92	0.92	0.93	0.90
	20	49	3	0.95	0.94	0.95	0.92
			5	0.92	0.92	0.93	0.90

. The coverages correspond to the bivariate solution, and were obtained for $n_{test}$ consecutive observations that might or might not correspond to pollution incidents.

RMSE values for each model are shown in

Table 3. RMSE values for two pollution episodes and the four models tested, considering curves with two different time lags, size of the training samples and number of principal components.

				Episode 1				Episode 2
Repsonse	${\mathit{t}}_{\mathit{lag}}$	${\mathit{n}}_{\mathit{train}}^{•}$	K	M${}_{\mathbf{1}}$	M${}_{\mathbf{2}}$	M${}_{\mathbf{3}}$	M${}_{\mathbf{4}}$	M${}_{\mathbf{1}}$	M${}_{\mathbf{2}}$	M${}_{\mathbf{3}}$	M${}_{\mathbf{4}}$
NO${}_x$	10	20	3	20.7	25.2	20.0	23.0	1.9	1.7	1.4	1.4
			5	19.6	18.6	20.6	16.4	0.9	0.8	0.8	0.6
		49	3	20.9	24.7	18.2	23.8	1.8	1.9	1.4	1.4
			5	19.2	17.5	19.3	16.7	0.9	0.9	0.8	0.7
	20	100	3	34.0	24.7	19.9	19.6	3.3	5.5	1.7	3.1
			5	40.9	23.3	46.5	29.2	1.5	2.5	1.0	1.8
		49	3	30.2	18.2	18.7	20.4	3.4	5.3	1.7	2.8
			5	36.2	27.6	39.8	35.6	1.5	2.5	1.0	1.8
SO${}_2$	10	100	3	505.0	841.0	407.5	837.3	544.8	531.9	419.8	419.1
			5	914.9	868.6	669.4	516.3	215.6	230.3	184.5	199.1
		49	3	686.0	685.3	515.8	518.3	481.4	484.4	338.0	361.1
			5	991.1	925.5	846.3	682.9	199.9	214.7	170.6	179.9
	20	100	3	1463.4	2172.7	825.4	1199.9	1154.5	1133.1	709.5	659.0
			5	1470.6	2531.2	1002.9	1428.5	525.5	482.1	352.0	341.3
		49	3	1.458.7	2485.4	768.4	698.4	1162.6	1125.8	644.8	628.7
			5	1787.6	2811.0	1111.2	951.5	548.1	492.0	352.7	359.3

Note: $n_{train}=100·n_{train}^{•}$.

, considering an expansion of the functions in three or five principal components, and lags $t_{lag}=10$ and $t_{lag}=20$. Please note that this table makes reference to the marginal distributions, and that the range of concentrations for each pollutant is very different, therefore the RMSE values are also different.

For the two episodes analyzed, Figure 3 shows the observed and the predicted values as well as the quantile for $\widehat{\tau }=0.95$, calculated for the test sample. The results correspond to curves observed in ten points ($t_{lag}=10$) and represented in a basis expansion in three functional principal components. These univariate confidence intervals were respectively constructed from (11) as ${\mu }_1\left({\mathbf{X}}^1\right)+{\sigma }_1\left({\mathbf{X}}^1\right){\epsilon }_1^{0.95}$ and ${\mu }_2\left({\mathbf{X}}^2\right)+{\sigma }_2\left({\mathbf{X}}^2\right){\epsilon }_2^{0.95}$, ${\epsilon }_1^{0.95}$ and ${\epsilon }_1^{0.95}$ being the 0.95 quantile of the distributions of errors ${\epsilon }_1$ and ${\epsilon }_2$, respectively.

Table 4. Maximum memory consumption (MB) and computation time (seconds) for the four models tested, $M_i$, following different strategies concerning the time lag, $t_{lag}$, the size of the training sample, $n_{train}$, and the number of basis functions, K.

			Memory (MB)				Runtime (seconds)
${\mathit{t}}_{\mathit{lag}}$	${\mathbf{n}}_{\mathbf{train}}$	K	M${}_{\mathbf{1}}$	M${}_{\mathbf{2}}$	M${}_{\mathbf{3}}$	M${}_{\mathbf{4}}$	M${}_{\mathbf{1}}$	M${}_{\mathbf{2}}$	M${}_{\mathbf{3}}$	M${}_{\mathbf{4}}$
10	20	3	652.70	1083.03	1319.57	2266.34	17.98	29.66	35.99	61.62
		5	1090.78	1855.39	2299.14	4075.88	35.55	55.29	63.15	124.7
	49	3	329.94	548.58	669.42	1153.38	10.53	17.42	21.38	37.01
		5	552.74	942.68	1171.67	2089.53	19.79	31.28	46.62	88.20
20	20	3	653.34	1084.15	1320.66	2268.16	18.11	29.63	35.97	61.21
		5	1091.90	1857.26	2300.99	4078.95	32.14	49.51	72.45	124.81
	49	3	330.10	548.84	669.69	1153.82	10.51	17.56	21.49	37.82
		5	553.01	943.13	1172.20	2090.36	17.80	36.44	39.03	76.11

Note: $n_{train}=100·n_{train}^{•}$.

shows the maximum consumed memory and the runtime (in seconds) for the four models tested and two different dimensions of the submatrices are executed in a Intel Core i7-2600K with 16 GB of RAM.

4. Discussion

We begin the discussion of the results analyzing Table 2 that show the estimated coverage for the bivariate prediction depending on the time lag, the size of the training sample, the number of principal components and the model used. It can be appreciated that the estimated coverages are generally lower than the theoretical coverages, although very close. Therefore, the mathematical models proposed show a good performance although there is a trend to underestimate the observed values. This effect can also be appreciated in Figure 3, where the mean tends to be under the observed values. Then, in order to be on the safe wide, it would be preferable to use the quantile $\tau =0.95$, which provides greater guarantee of predicting the highest values of the pollution episodes. Regarding the rest of the parameters, it is not possible to establish a combination of them that provides the best results. However, in general they were obtained for the lowest training size, $n_{train}=49,000$, and for models that includes one or two derivatives (models $M_3$ and $M_4$).

The prediction errors are shown in Table 1, where the best results (minimum RMSE) are marked in bold. As can be seen, they correspond to model $M_4$ for NO${}_x$ and model $M_3$ for SO${}_2$. In both cases, these models incorporate the derivatives of the original functions. Accordingly, we conclude that the derivatives contribute positively to improve the results, which reinforces the role of the functional approach. However, there is an asymmetry between both pollutants: using the concentrations of SO${}_2$ and their derivatives improves the results for NO${}_x$, but using the concentrations of NO${}_x$ and their derivatives is not an advantage in the estimation of SO${}_2$. When SO${}_2$ and NO${}_x$ concentrations of both episodes are plotted against time (Figure 4), a slight advance can be seen on the first pollutant compared to the second, which would explain this asymmetry.

With respect to the time lag, the minimum RMSE values were obtained for the shorter period of time $t_{lag}=10$, so it seems that using 20 observations to predict one our in advance introduced noise into the model instead of adding useful information. This result is in agreement with those obtained for the same data in previous studies of some of the authors, which indicated that only a few observations close to the time of prediction contribute to that prediction. Talking about the size of the training sample, simplifying the original data by removing small values of the concentrations improves the results in most of the cases, so this would be the advisable option.

When the effect of the number of principal components used as basis functions is analyzed, using $K=5$ is always favorable for episode 2, for both SO${}_2$ and NO${}_x$, but not for episode 1, for which the trend is opposite.

Although they are not shown in the article, so as not to overstretch it, a comparison of the estimated coverages using 3 or 5 principal components, or 5 B-splines basis functions, tell us that there are not substantial differences among them, so it seems that one or other base functions can be used interchangeably.

Finally, regarding memory consumption and runtime for model training, it is evident, from Table 4 that more complex models consume more resources and requires more computing time. For fixed values of the time lag ($t_{lag}$) and the size of the training sample ($n_{train}$), model $M_4$ is between 3 and 7 times more expensive than model $M_1$ in terms of memory consumption and runtime. Using time lags $t_{lag}=10$ or $t_{lag}=50$ has no effect in terms of computation requirements; and employing 5 principal components instead of 3 principal components implies an approximately double memory consumption and runtime.

To conclude, it is possible to establish that the functional location-scale model proposed were quite a good approach (in terms of coverage and prediction error) to forecast bivariate pollution episodes one hour in advance, as it is required by the Spanish legislation. The best results were obtained when the derivatives of functions adjusted to the observed data are included in the model, when the raw data are filtered and when the shorter period of time is used for the prediction. The size of the training data and the type and number of basis functions are, instead, parameters on which definitive conclusions could not be drawn.