Multi-Output Variational Gaussian Process for Daily Forecasting of Hydrological Resources ^†

Abstract

Water resource forecasting plays a crucial role in managing hydrological reservoirs, supporting operational decisions ranging from the economy to energy. In recent years, machine learning-based models, including sequential models such as Long Short-Term Memory (LSTM) networks, have been widely employed to address this task. Despite the significant interest in forecasting hydrological series, weather’s nonlinear and stochastic nature hampers the development of accurate prediction models. This work proposes a Variational Gaussian Process-based forecasting methodology for multiple outputs, termed MOVGP, that provides a probabilistic framework to capture the prediction uncertainty. The case study focuses on the Useful Volume and the Streamflow Contributions from 23 reservoirs in Colombia. The results demonstrate that MOVGP models outperform classical LSTM and linear models in predicting several horizons, with the added advantage of offering a predictive distribution.

1. Introduction

Hydrological forecasting plays a crucial role in planning and operation activities. On a short-term scale, it allows for management of water systems and resources, including irrigation, flood control, and hydropower generation. In the energy sector, hydrological forecasting supports the optimal scheduling of hydroelectric power generation. Accurately predicting hydroelectric plants’ water release and reservoir volume enables scheduling optimal thermal plant generation while minimizing fuel costs and improving the energy sector’s sustainability [1,2]. For example, Brazil’s National Electrical System Operator provides streamflow time series of hydropower plants that supports forecasting research [3]. In Colombia, hydropower plants contribute 97.37% of the renewable energy, including 87.54% from reservoirs [4]. Therefore, there is a great interest in developing accurate hydrological forecasting models to manage and exploit water resources sustainably and effectively.

Forecasting models can be short-, mid-, or long-term, where the former enable dispatch and optimization for power systems. Long-term forecasting supports reliability planning through system expansions and weather analysis at a large scale [5,6]. Hence, the design of forecasting models depends on the prediction horizon, with two primary approaches: physically-driven concept rules and data-driven models which learn from time-series samples. Models in the first category have demonstrated their capability to predict various flooding scenarios. However, physical modeling often requires extensive knowledge and expertise in hydrological parameters and various datasets, demanding intensive computation which results in unsuitability for short-term prediction [7]. Since data-driven models can learn complex behavior from the data, streamflow forecasting traditionally relies on this second approach. The most extensively used data-driven forecasting model is the Linear AutoRegression (LAR), due to its simplicity and interoperability [8]. However, LAR fails to adequately represent streamflow series because of the complex water resource patterns, such as varying time dependencies, randomness, and nonlinearity [3,6,9].

The most prominent models for dealing with nonlinear trends are neural networks (NNs), known for their flexibility and outperformance of other nonlinear models [10]. For instance, a hybrid model coupling extreme gradient boosting to NNs predicted monthly streamflow at Cuntan and Hankou stations on the Yangtze River, outperforming baseline support vector machines [11]. Additionally, recurrent architectures, such as Long Short-Time Memory (LSTM), have proven to improve the scores of classical NNs in daily streamflow forecasting, given their ability to capture seasonality and stochasticity [5,8]. However, the numerous neural network architectures available make researchers question which one will best fit a given problem, as no single model is universally applicable [7]. Further, the inherent noise present in hydrological time series influences forecasting accuracy [3,10].

Some operational tasks demand uncertainty quantification for the prediction due to the inherent noise. Gaussian Processes (GP) satisfy such a requirement by approximating a predictive distribution. GP-based forecasting has proven remarkable results in streamflow forecasting up to one day and month ahead [12,13]. In addition, a kernel function, combining squared exponential, periodic, and rational quadratic terms, allowed GP models to fit streamflow time series for the Jinsha River [9]. In another approach, a probabilistic LSTM coupled with a heteroscedastic GP produced prediction intervals without any post-processing to manage the daily streamflow time series uncertainty [14]. However, GP-based approaches pose two research gaps [15]. Firstly, the probabilistic couple approach still complicates the model calibration. Secondly, natural probabilistic modes such as GP have only been used to study scalar value signals.

This work develops a forecasting methodology using a GP-based probabilistic approach applied to hydrological resources, supporting multiple output predictions and reducing the model training complexity. The methodology, termed MOVGP, combines the advantages of Multi-Output and Variational GPs for taking advantage of relationships among time series, adapting the individual variability to handle large amounts of samples. The research compares the performance of the MOVGP against an LSTM neural network and a Linear AutoRegression (LAR) model in forecasting two multi-output hydrological time series, namely, Useful Volume and Streamflow Contributions of 23 reservoirs. It is worth noting that the considered time series correspond to actual reservoir data taken into account for hydropower generation in Colombia. Attained results prove the ability of MOVGP to be adapted to varying prediction horizons, generally outperforming contrasted models.

The paper agenda is as follows: Section 2 covers methodologies and theoretical bases used for developing and training Multi-Output Variational GP models; Section 3 validates the MOVGP training and tests the three models in terms of the Mean Square Error (MSE); Final remarks and future work conclude the work in Section 4.

2. Mathematical Framework

2.1. Gaussian Process Modeling Framework

A Gaussian Process (GP) is a collection of random variables related to the infinite-dimensional setting of a joint Gaussian distribution. Consider the dataset of N samples $\mathcal{D}=\{\mathbf{X},\mathbf{Y}\}$, where $\mathbf{X}\in {\mathbb{R}}^{L\times N}$ is the design matrix, with columns of vector inputs $\mathbf{x}\in {\mathbb{R}}^L$ of L features, and $\mathbf{Y}\in {\mathbb{R}}^{D\times N}$ is the target matrix, with columns of vector outputs $\mathbf{y}\in {\mathbb{R}}^D$ of D outputs for all N cases. GP framework conditions a subset of observations to create a map that models the relationship between $\mathbf{X}$ to $\mathbf{Y}$.

Then, the Single-Output GP (SOGP) attempts to represent a scalar-valued function $f:{\mathbb{R}}^L\to \mathbb{R}$, i.e., where $D=1$ with GP framework. This model is completely specified by a mean function $m:{\mathbb{R}}^L\to \mathbb{R}$ and covariance (kernel) function $k:{\mathbb{R}}^L\times {\mathbb{R}}^L\to \mathbb{R}$, with vector parameters ${\theta }_m$ and ${\theta }_k$ notated, respectively, as follows:

$$f\left(\mathbf{x}\right)\sim \mathcal{GP}(m(\mathbf{x}\mid {\theta }_m),k(\mathbf{x},{\mathbf{x}}^{\prime }\mid {\theta }_k))$$

(1)

In more realistic scenarios, single-output observation y presents Gaussian noise $ϵ\sim \mathcal{N}(0,{\sigma }_N^2)$ models as a noise-added version of the function f, such as $y\left(\mathbf{x}\right)=f\left(\mathbf{x}\right)+ϵ$. Let ${\mathbf{X}}_{*}\in {\mathbb{R}}^{L\times N^{\prime }}$ be a test matrix with $N^{\prime }$ test vector inputs ${\mathbf{x}}_{*}\in {\mathbb{R}}^L$, ${\mathbf{f}}_{*}\in {\mathbb{R}}^{N^{\prime }}$, in which $\mathbf{m}=\left[m({\mathbf{x}}_i\mid {\theta }_m)\right]\in {\mathbb{R}}^N$ denotes mean train vector, ${\mathbf{m}}_{*}=\left[m({\mathbf{x}}_{*i}\mid {\theta }_m)\right]\in {\mathbb{R}}^{N^{\prime }}$ denotes mean test vector, $\mathbf{K}=\left[k({\mathbf{x}}_i,{\mathbf{x}}_j\mid {\theta }_k)\right]\in {\mathbb{R}}^{N\times N}$ denotes covariance train matrix, ${\mathbf{K}}_{*}=\left[k({\mathbf{x}}_i,{\mathbf{x}}_{*j}\mid {\theta }_k)\right]\in {\mathbb{R}}^{N\times N^{\prime }}$ denotes covariance train–test matrix, and ${\mathbf{K}}_{**}=\left[k({\mathbf{x}}_{*i},{\mathbf{x}}_{*j}\mid {\theta }_k)\right]\in {\mathbb{R}}^{N^{\prime }\times N^{\prime }}$ denotes covariance test matrix. The joint Gaussian distribution of the observations vector $\mathbf{y}$ and test outputs vector ${\mathbf{f}}_{*}$, named previously, are specified as shown:

$$\begin{array}{c}\hfill \left[\begin{array}{c}\mathbf{y}\\ {\mathbf{f}}_{*}\end{array}\right]\sim \mathcal{N}\left(\left[\begin{array}{c}\mathbf{m}\\ {\mathbf{m}}_{*}\end{array}\right],\left[\begin{array}{cc}\mathbf{K}+{\sigma }_N^2{\mathbf{I}}_N& {\mathbf{K}}_{*}\\ {\mathbf{K}}_{*}^{\top }& {\mathbf{K}}_{**}\end{array}\right]\right)\end{array}$$

(2)

where ${\mathbf{I}}_N$ is the identity matrix of size N derived in the conditional distribution, calculated as follows:

$${\mathbf{f}}_{*}|{\mathbf{X}}_{*},\mathbf{X},\mathbf{y}\sim \mathcal{N}(\overline{{\mathbf{f}}_{*}},\mathrm{cov}\left({\mathbf{f}}_{*}\right))$$

(3)

with the following definitions:

$$\overline{{\mathbf{f}}_{*}}={\mathbf{m}}_{*}+{\mathbf{K}}_{*}^{\top }{[\mathbf{K}+{\sigma }_N^2{\mathbf{I}}_N]}^{-1}(\mathbf{y}-\mathbf{m})$$

(4)

$$\mathrm{cov}\left({\mathbf{f}}_{*}\right)={\mathbf{K}}_{**}-{\mathbf{K}}_{*}^{\top }{[\mathbf{K}+{\sigma }_N^2{\mathbf{I}}_N]}^{-1}{\mathbf{K}}_{*}$$

(5)

Notice, from Equations (4) and (5), that mapping construction is analytic and, therefore, does not employ an optimization process. Nevertheless, selection of parameters at ${\theta }_m$ and ${\theta }_k$ and observation noise variance ${\sigma }_N$ can be estimated using marginal likelihood from Equation (2), $p(\mathbf{y}\mid \mathbf{X})=\mathcal{N}(\mathbf{m},\mathbf{K}+{\sigma }_N^2{\mathbf{I}}_N)$, and minimizing negative log marginal likelihood:

$$\begin{array}{c}\hfill \underset{{\theta }_m,{\theta }_k,{\sigma }_N}{min}-ln\left(p(\mathbf{y}\mid \mathbf{X})\right)=\frac{1}{2}{(\mathbf{y}-\mathbf{m})}^{\top }{\mathbf{K}}_y^{-1}(\mathbf{y}-\mathbf{m})+\frac{1}{2}ln(\mid {\mathbf{K}}_y\mid )+\frac{N}{2}ln\left(2\pi \right)\end{array}$$

(6)

where ${\mathbf{K}}_y=\mathbf{K}+{\sigma }_N^2{\mathbf{I}}_N$ is the covariance matrix for the noisy observations. Thus, the optimization problem in Equation (6) can be efficiently solved via a gradient-based optimizer [16].

2.2. Multi-Output Gaussian Process (MOGP)

MOGP generalizes SOGP mapping for $D\ge 1$ outputs as $f^D:{\mathbb{R}}^L\to {\mathbb{R}}^D$ with GP framework, where $f^D$ is a vector-valued function. The MOGP model, similar to the SOGP model, is entirely defined by its mean vector function $m^D:{\mathbb{R}}^L\to {\mathbb{R}}^D$ and covariance matrix function $k^D:{\mathbb{R}}^L\times {\mathbb{R}}^L\to {\mathbb{R}}^{D\times D}$, each with vector parameters ${\theta }_m$ and ${\theta }_k$, respectively, expressed as follows:

$$f^D\left(\mathbf{x}\right)\sim \mathcal{GP}(m^D(\mathbf{x}\mid {\theta }_m),k^D(\mathbf{x},{\mathbf{x}}^{\prime }\mid {\theta }_k))$$

(7)

Let ${\Sigma }_N\in {\mathbb{R}}^{D\times D}$ be a diagonal matrix such that ${\Sigma }_N=\mathrm{diag}{\left\{{\sigma }_{Nd}^2\right\}}_{d=1}^D$, with ${\sigma }_{N,d}$ being the $d^{th}$ output observation noise variance and ${\mathbf{y}}^D\in {\mathbb{R}}^{ND}$ being a ravel version vector of target matrix $\mathbf{Y}$. Following the procedure established in Equation (3), deriving the MOGP posterior distribution takes place as follows:

$${\mathbf{f}}_{*}|{\mathbf{X}}_{*},\mathbf{X},\mathbf{Y}\sim \mathcal{N}(\overline{{\mathbf{f}}_{*}},\mathrm{cov}\left({\mathbf{f}}_{*}\right))$$

(8)

with the following definitions:

$${\overline{{\mathbf{f}}_{*}}}^D={\mathbf{m}}_{*}^D+{\mathbf{K}}_{*}^{D\top }{\left[{\mathbf{K}}_y^D\right]}^{-1}({\mathbf{y}}^D-{\mathbf{m}}^D)$$

(9)

$$\mathrm{cov}\left({\mathbf{f}}_{*}\right)={\mathbf{K}}_{**}^D-{\mathbf{K}}_{*}^{D\top }{\left[{\mathbf{K}}_y^D\right]}^{-1}{\mathbf{K}}_{*}^D$$

(10)

where ${\mathbf{K}}_y^D={\mathbf{K}}^D+{\Sigma }_N\otimes {\mathbf{I}}_N$, ⊗ represents the Kronecker product between matrices and upper index D denotes $D-\mathrm{dimension}$ version of SOGP quantities.

To deal with developing an admissible correlation between outputs, the Linear Model of Coregionalization takes place, expressing each output of MOGP as a linear combination of Q (known as latent dimension) independent SOGP as follows:

$$f^D\left(\mathbf{x}\right)=\sum _{q=1}^Q{\mathbf{a}}_q{g}_q\left(\mathbf{x}\right)$$

(11)

where ${\mathbf{a}}_q\in {\mathbb{R}}^D$ is the vector coefficients, with values $a_q^d$ associated with contributions of the q-th independent SOGP $g_q\left(\mathbf{x}\right)$ at the $d^{th}$ output with kernel function $k_q$. In this way, the covariance matrix of the MOGP model is given by the following [17,18]:

$$k^D(\mathbf{x},{\mathbf{x}}^{\prime })=\sum _{q=1}^Q{\mathbf{B}}_q{k}_q(\mathbf{x},{\mathbf{x}}^{\prime })$$

(12)

where ${\mathbf{B}}_q\in {\mathbb{R}}^{D\times D}={\mathbf{a}}_q{\mathbf{a}}_q^{\top }$ is a semi-definite positive matrix known as the coregionalization matrix.

2.3. Variational Gaussian Process (VGP)

The main challenge in implementing MOGP models lies in their complexity $\mathcal{O}\left(D^3{N}^3\right)$ and storage demand $\mathcal{O}\left(D^2{N}^2\right)$ becoming intractable for a dataset of a few thousand samples [19], because the need to invert the matrix ${\mathbf{K}}_y^D$ in Equations (9) and (10) is usually performed by Cholesky decomposition. To overcome the problem of computational complexity, a new set of $M\ll N$ trainable inducing points $\mathbf{Z}\in {\mathbb{R}}^{L\times M}$ and inducing variables $\mathbf{u}=f^D\left(\mathbf{Z}\right)\in {\mathbb{R}}^{DM}$ augment the output variables ${\mathbf{f}}^D=f^D\left(\mathbf{X}\right)\in {\mathbb{R}}^{DN}$. The marginal distribution for the output variables is expressed as $p({\mathbf{f}}^D\mid \mathbf{X})=\int p({\mathbf{f}}^D\mid \mathbf{X},\mathbf{u})p\left(\mathbf{u}\right)d\mathbf{u}$. The Variational Gaussian Process (VGP) allows approximating $p({\mathbf{f}}^D\mid \mathbf{X})$ with $q({\mathbf{f}}^D\mid \mathbf{X})$ by marginalizing out the set of inducing points [20]:

$$q({\mathbf{f}}^D\mid \mathbf{X}):=\int p({\mathbf{f}}^D\mid \mathbf{X},\mathbf{u})q\left(\mathbf{u}\right)d\mathbf{u}$$

(13)

Since the output distribution comes from a MOGP, $q\left(\mathbf{u}\right)$ is assumed as Gaussian $\mathcal{N}(\mathbf{u}\mid {\mathbf{m}}_{\mathbf{z}},\mathbf{S})$, with mean ${\mathbf{m}}_{\mathbf{z}}\in {\mathbb{R}}^{DM}$ and covariance $\mathbf{S}\in {\mathbb{R}}^{DM\times DM}$, so that the approximating distribution also becomes Gaussian:

$$q({\mathbf{f}}^D\mid \mathbf{X})=\mathcal{N}\left({\mathbf{f}}^D\mid \mathbf{A}{\mathbf{m}}_{\mathbf{z}},{\mathbf{K}}^D+\mathbf{A}(\mathbf{S}-{\mathbf{K}}_{M,M}){\mathbf{A}}^{\top }\right)$$

(14)

with $\mathbf{A}={\mathbf{K}}_M{\mathbf{K}}_{M,M}^{-1}$, ${\mathbf{K}}_M\in {\mathbb{R}}^{DN\times DM}$ as the kernel function in Equation (12) evaluated at all pairs of inducing–training points and ${\mathbf{K}}_{M,M}\in {\mathbb{R}}^{DM\times DM}$ the kernel function values between pairs of inducing points. Since optimizing the parameters of $q\left(\mathbf{u}\right)$ yields a stochastic framework, the cost function in Equation (6) turns into a tractable marginal likelihood bound for the multi-output case:

$$logp({\mathbf{y}}^D\mid \mathbf{X})\ge \sum _{n=1}^N{\mathbb{E}}_{q(f_n^D\mid {\mathbf{x}}_n)}[logp(y_n^D\mid f^D\left({\mathbf{x}}_n\right))]-\mathrm{KL}[q\left(\mathbf{u}\right)\Vert p\left(\mathbf{u}\right)]$$

(15)

in which $\mathrm{KL}\left[q\right(\mathbf{u})\Vert p(\mathbf{u}\left)\right]$ is the Kullback–Leibler (KL) divergence between $q\left(\mathbf{u}\right)$ and $p\left(\mathbf{u}\right)$. This approach offers a notable benefit by diminishing the complexity of the MOGP to $\mathcal{O}\left(DN{D}^2{M}^2\right)$ due to the inversion of the matrix ${\mathbf{K}}_{M,M}$, which is smaller than ${\mathbf{K}}_y^D$. As a result, the model can efficiently handle an increased number of samples N, providing an opportunity to gather more information from the dataset at a reduced cost.

3. Results and Discussions

The current section aims to thoroughly communicate the implications of the forecasting results on hydric time series using MOVGP. Firstly, we offer a detailed description of the considered dataset, including information on its sources, characteristics, and preprocessing steps. Further, the manuscript details the hyperparameter tuning experiments and describes the validation strategy. Finally, we examine the forecasting performance and highlight notable trends and observations.

3.1. Dataset Collection

We validate the MOVGP regressor on a hydrological time series forecasting task of the Useful Volume and Contributions from 23 Colombian reservoirs, daily recorded daily from 1 January 2010, to 28 February 2022, yielding 4442 daily measurements. Despite the volumetric nature of the raw data, the hydroelectric power plants report Useful Volumes and Streamflow Contributions as their equivalent in kilowatt-hour (kWh) units, since such a representation is more practical for daily operations. Figure 1 visually describes the statistics for each reservoir. Note that amplitudes vary from millions to billions of kWh among reservoirs, due to each generating at a different capacity. In the case of Useful Volumes, one finds some highly averaging time series with a few variations (see reservoirs A and L), but also cases of low means with a significant variation (as the reservoir B). Notice from Streamflow Contributions that the reservoir K boxplot does not appear, due to zero values reported for all time series. Some reservoirs also present outlier volume reductions (black crosses), contrasting with the outlier increments in streamflow. Two main factors produce the above nonstationary and non-Gaussian behavior: the Colombian weather conditions produce unusual rainy days and long dry seasons, and the operation decisions can impose water saving or generation at total capacity each day.

3.2. MOVGP Setup and Hyperparameter Tuning

Firstly, we define the validation task as predicting the H-th day in the future using the current hydrological measurement in all the reservoirs, yielding $L=23$ inputs. Prediction horizon H ranges from one to twenty-five days, exploring short- and medium-term hydrological forecasting performance. For proper validation, MOVGP and contrasting approaches were trained on the first 10 years and validated on the data from 28 February 2021 to 28 February 2022, corresponding to 365 testing samples. Figure 2 presents the testing time series for the Useful Volume of three reservoirs. Noting the varying scales that lead to potential bias during the training stage, a preprocessing step normalizes the dataset by centering the time series from each reservoir on zero and scaling it to unit standard deviation. Normalizing means and standard deviations result from the training subset statistics avoids test biasing.

Each type of time series, Useful Volume and Streamflow Contribution, considers an individual forecasting model. Hence, the experimental framework trains two independent MOVGPs with $D=23$ outputs. The proposed methodology considers a constant for the MOVGP mean function, $m^D(\mathbf{x}\mid {\theta }_m)={\theta }_m$, with ${\theta }_m\in {\mathbb{R}}^D$ as the single trainable vector parameter. The proposed methodology builds the MOVGP covariance function in Equation (12) from the widely used squared exponential, in Equation (16), allowing a smooth data mapping:

$$k_q(\mathbf{x},{\mathbf{x}}^{\prime }\mid {\theta }_q)=\exp \left(-\frac{1}{2}{(\mathbf{x}-{\mathbf{x}}^{\prime })}^{\top }{\Theta }_q^{-2}(\mathbf{x}-{\mathbf{x}}^{\prime })\right)$$

(16)

where the diagonal matrix ${\Theta }_q=\mathrm{diag}{\left\{{\Delta }_{ql}\right\}}_{l=1}^L\in {\mathbb{R}}^{L\times L}$ gathers the length scale factors ${\Delta }_{ql}\in {\mathbb{R}}^{+}$ from each input dimension. The trainable covariance parameters become ${\sigma }_{N,d}$ and ${\Delta }_{lq}$ from the Q independent SOGP within the MOVGP framework. Then, a 10-fold time series split model selection determines the optimal hyperparameter setting for the forecasting models by searching within the following grid: number of inducing variables $M\in \{4,8,16,32,64,128\}$ and latent space dimension $Q\in \{2,4,8,16,23,46,69,92,115\}$.

Figure 3 presents the 10-fold-averaged cross-validation mean squared error (MSE) along the grid search while fixing the model horizon to $H=1$ for both the Useful Volume and Streamflow Contributions. Hyperparameter tuning exhibits that, the larger the Q and M, the smaller the MOVGP error and the slower the improvement. Therefore, the forecast task on a very short horizon yields complex models that hardly overfit. However, the latent dimension influences the performance significantly more than the induced variables, agreeing with the model development: the latent dimension controls the embedding quality, while the induced variables reduce the computational burden without compromising the performance.

For each considered horizon $H\in \{1,2,3,5,7,9,10,15,20,25\}$, Figure 4 illustrates the hyperparameters which reach the best testing MSE. For the Useful Volume time series, shown in Figure 4a, the number of latent variables Q increases while the number of inducing points M decreases. A Pearson correlation coefficient of $-0.82$ between the optimal Q and M indicates that the model trades off its complexity between hyperparameters: increasing Q allows a more flexible model, whereas increasing M produces MOVGP models that retain more information about the time series. In turn, the optimal Q for the Streamflow Contribution remains at the highest evaluated value while M decreases for the last horizons (Figure 4b). Such a fact suggests that the latent space is large enough to decode the relationship between past Streamflow Contributions and the farthest horizon. Thus, a flexible model evades a large explicit memory to seize the relevant dynamics, and vice versa.

3.3. Performance Analysis

The performance analysis compares MOVGP against two widely considered hydrological forecasting models: a straightforward Linear AutoRegression (LAR) model as a baseline and a Long Short-Term Memory (LSTM) network with the hidden space dimension and the number of recurrent layers as hyperparameters. Specifically for the LSTM, the same model selection strategy—10-fold time series split—tunes the hyperparameters using the training subset. Figure 5 illustrates the MSE for the 10-fold cross-validation attained by the three contrasted models along the explored horizons for both time series. In general, error increases with the prediction horizon, due to the forecasting task becoming more complex for far away days. Nonetheless, the autoregressive model outperforms LSTM, closely followed by MOVGP, up to a 15-day horizon, suggesting linearly-captured time dependencies in the short term. In contrast, MOVGP reaches the lowest error for the longest horizons, followed by LSTM, evidencing nonlinear time relationships at medium-term which are profited by more elaborate models. The above results indicate that MOVGP was the most flexible model on average, exploiting the time-varying interactions, competing at short-term, and outperforming at medium-term horizons.

At the testing stage, trained LAR, LSTM, and MOVGP models forecast the last 365 days on the dataset for both time series at each horizon considered. Figure 6 depicts reservoir-wise testing MSE boxplots computed over the 10 prediction horizons for the Streamflow Contributions. Note that reservoir T makes the models perform the worst, whereas reservoir D becomes the least challenging. Moreover, the widespread error at reservoir L contrasts with the small dispersion at reservoir D. Therefore, varying boxplots advise the changing forecasting complexity over the horizons and reservoirs despite corresponding to the same hydrological time series. According to the grand averages in dashed lines, the MOVGP model obtains the best average performance, followed by the LAR. Thus, for the Contributions time series, MOGP offers a better explanation for nonlinearities in the data than LSTM.

Figure 7 offers time series plots for Useful Volume and Contributions with their respective one-day horizon predictions from the forecast models for three reservoirs of interest. Notice, in Figure 7a, that the MOVGP and Linear models reach a well-fit prediction and learn the smoothness of the reservoir data, but the MOVGP model presents the advantage of yielding a predictive distribution and, therefore, a confidence interval. In addition, the LSTM model shows abrupt changes that deviate from the actual behavior of the curve, producing a higher error. For Figure 7c, a narrow confidence interval describes the time series noise. Observe the presence of a peak at day 65 that is out of the confidence interval, possibly an outlier classified as an anomaly by predictive distribution offered by the MOVGP model.

In the case of the Streamflow Contributions, the three models closely follow the abrupt curve trend and lie within the confidence intervals for reservoir A in Figure 7b. Lastly, Figure 7d displays peaks in the Streamflow Contributions of reservoir T. Although no model catches the peak’s tendency, MOVGP explains them as outliers because of the predictive distribution. In this way, the MOVGP model is less influenced by anomalies, producing better generalization and, thus, outperforming the other models.

4. Concluding Remarks

This work proposed a forecasting methodology for multiple output prediction of Useful Volume and Streamflow Contributions of Colombian reservoirs using Variational Gaussian Processes. Since the coregionalization of MOVGPs imposes a unique latent process, generating multiple outputs, we devoted a single model for each hydrological variable to minimize overgeneralization issues. The proposed MOVGP was compared against LSTM-based and Linear AutoRegressive models using actual time series. The hyperparameter tuning stage proved that MOVGP suitably adapted to time complexity by optimizing the number of latent variables and inducing points to control model flexibility. The comparison in testing data, shown in Figure 5, revealed that the MOVGP outperformed the others in predicting long-term horizons, particularly when the linear model missed relationships between inputs and outputs. Therefore, MOVGP outperforms hydrological forecasting, providing prediction reliability and outlier detection through the predictive distribution.

For future work, we devise the following research directions. First, we will extend the methodology to support energy-related time series such as daily thermoelectric schedules. Secondly, we will develop deep learning models to learn complex patterns in hydrological time series.

Finally, to overcome the overgeneralization and linear coregionalization restriction issues, we will work on time-variant convolutional kernel integration.