Echo State Network and Sparrow Search: Echo State Network for Modeling the Monthly River Discharge of the Biggest River in Buzău County, Romania

Abstract

Artificial intelligence (AI) has become an instrument used in all domains with good results. The water resources management field is not an exception. Therefore, in this article, we propose two machine learning (ML) techniques—an echo state network (ESN) and sparrow search algorithm–echo state network (SSA-ESN)—for monthly modeling of the water discharge of one of the biggest rivers in Romania for three periods (S, S1, and S2). In both models, R² was over 0.989 on the test and training sets and the mean absolute error (MAE) varied between 4.4826 and 7.6038. The performance of the SSA-ESN was similar, but the ESN had the shortest run time. The influence of anomalies on the models’ quality was assessed by running the algorithms on a series without the aberrant values, which were detected by the seasonal hybrid extreme studentized deviate (S-H-ESD) test. The results indicate that removing the anomalies significantly improved both models’ performance, but the run time was increased.

1. Introduction

In recent decades, the demand for metaheuristic algorithms has surged alongside the growing complexity of various real-life problems. While traditional optimization methods struggle with the exponential growth in search space, metaheuristics offer efficient and effective solutions. Given that the constraints (such as convexity and nonlinearity, autocorrelation, normality, etc.) that should be fulfilled by variables when using classical optimization approaches are avoided when employing machine learning (ML) algorithms, these algorithms represent a change in perspective and a significant advancement in addressing optimization challenges [1,2].

In the case of water resources management and hydraulics, the advantage of sophisticated computational models aimed at enhancing streamflow forecast precision has the potential to yield substantial cost savings. Various methods have been used for this purpose. For example, Tanty and Desmukh [3] presented some applications of artificial neural networks in hydrology, and Li and Young [4] modeled a suspended sediment load by machine learning. Hayder et al. [5] used PSO–FNN in a study of river discharge in Malaysia, whereas Khan et al. [6] predicted the water level of a river in India. To predict the dynamic pressure distribution in hydraulic structures, Samadi et al. [7] investigated the performances of gene expression programming, artificial neural networks (ANNs), and classification and regression trees. Haghiabi et al. [8] compared the efficiency of a multilayer perceptron and adaptive neuro-fuzzy inference system in predicting triangular labyrinth weirs’ discharge coefficient and found that the first technique gave the best results. Other authors [9,10,11,12] studied ultrasound cavitation’s effect in water by AI techniques.

In the context of forecasting the water flow of a river in Pakistan, Adnan et al. [13] demonstrated the efficiency and reliability of ML techniques. They combined locally weighted learning (LWL) with additive regression (AR), dagging (DG), bagging (BG), random subspace (RS), and rotation forest (RF) to achieve accurate predictions. Van Thieu et al. [14] integrated extreme learning machines with three AI techniques to improve the streamflow forecast of the Nile River (at Aswan). Other studies have also confirmed the effectiveness of ML techniques in predicting rivers’ water discharge based solely on historical records and assessing flood susceptibility in different catchments [15,16,17,18,19,20,21,22,23].

ESN falls in to the recurrent neural network (RNN) category. It functions by inputting data, followed by triggering a nonlinear response in the neurons from the “reservoir” (hidden layer). ESN output consists of a linear combination (trained for optimizing the weights) of all neurons’ responses in the hidden layer. This network mitigates gradient explosion/vanishing by addressing RNN weight selection [24]. Given its ability to handle complex data, with high variability, robust generalization capabilities, and the use of inexpensive training procedure, ESN has been used in various domains [25], such as time-series analysis, forecast, and classification [26,27], signal processing [28], speech recognition [29,30], pattern classification [31,32], finance, bioinformatics, and engineering [33,34,35,36,37], etc. It was also proved that ESN has a low sensitivity to parameter variation [33]. Compared to other algorithms, such as BP and RNN, ESN offers superior prediction accuracy, faster computation, good generalization capabilities, and robust nonlinear mapping capabilities [36,37].

The advantages of ESN for time-series forecasting are: (1) the reservoirs’ weights are fixed during training after being randomly generated [27], (2) the memoryless readout network that allows linear regression for training the weights [28], (3) the short-term memory [36], and (4) the fact that computation of the derivatives is not necessary for training [38].

Introduced in 2020, SSA, based on sparrows’ foraging and anti-predation behavior, has quickly gained popularity due to its simplicity and efficiency in global optimum detection. Since its inception, SSA has been used in various research fields, such as civil [39], electrical [40,41], industrial [42], and mechanical engineering [43], image processing [44], networking [45], communication [46], environmental modeling [47,48], with remarkable results. Several extended versions of SSA, including hybridization, have also been proposed to prevent premature convergence and enhance diversity [49].

The Buzău River, studied in this article, is one of the main rivers in Romania and on which one of the biggest dams was built. Only a few studies on its water discharge have been performed. Modeling this data series is challenging due its statistical features [50,51,52]. In [50], the authors analyzed the minima, maxima, and standard deviation of the Buzău monthly river flow to assess the data homogeneity of the series. The output indicated the necessity of a deeper investigation in this direction. The Mann–Kendall test and the non-parametric Sen slope [51,52] at a significance level of 5% indicated an increasing trend in annual, monthly, fall, and winter series after the dam had been put into operation. However, the fit trend was linear and could not capture the series oscillations.

Another attempt to evaluate the dam’s impact on the water discharge was made in [53] using IHA indicators and software. The output showed the extent to which the indicators changed after 1984 with respect to the previous period. Two parametric regression models of the recorded flow at Nehoiu vs. Bâsca (the main affluent of Buzău upstream Nehoiu) were also proposed. Generalized regression neural network models trained on 1955–1983 and tested on 1984–2010 series were not satisfactory [54]. Three articles [55,56,57] provided promising results in modeling the studied series using AI methods. Within this framework, the primary purpose of this article is to build forecasting models for the monthly river discharge of the Buzău River for the periods 1955–2010, 1955–1983, and 1984–2010 using ESN and SSA-ESN. These algorithms were chosen because despite SSA and ESN’s confirmed performances in various applications, ESN was rarely employed for hydrological predictions. Since our scientific literature search did not find any applications of the hybrid SSA-ESN in this field, this study investigates the suitability of ESN and SSA-ESN for forecasting long series of hydrological records. It also proves the superiority of this approach against others, such as ARIMA, MLP, BPNN, ELM, ESN, LSTM, CNN-LSTM, CNN-LSTM, and SSA-BP on three series.

Another aspect rarely discussed in hydro-meteorological data modeling is the influence of aberrant values on the model’s quality. To our knowledge, such an analysis has not been performed for ESN or SSA-ESN models built on any kind of series. This paper provides a comparison between the output obtained after running these algorithms on raw series and datasets without outliers using three goodness-of-fit indicators. Moreover, comparisons of run time in both scenarios indicate an increase in run time when aberrant values are removed from the series and open the direction for a future investigation of this behavior of the algorithms.

Given its specific design, this study emphasizes that the river flow pattern changed after 1984, as reflected by the performances of the models proposed for the entire series and its subseries.

2. Data Series and Methodology

2.1. Study Area and Data Series

With a catchment of 5264 km² situated in the Curvature Carpathians, a 1043 m mean elevation, and a length of 302 km, the Buzău River is a tributary of the Siret, located in southeastern Romania (Figure 1).

In the mountainous sector, the river’s dynamic is significant, mainly when increased precipitations occur. The climate in Buzău County is temperate continental, varying with the altitude, relief orientation, and configuration from north to south. The most important tributaries of the river in the mountainous zone and sub-Carpathians, such as Bâsca Roziliei, Bâsca Chiojdului, Pănătău, Slănic, and Câlnău, play a crucial role in the region’s water system. Upstream of Nehoiu, the river collects over 80% of the annual water volume. Due to the building of the Siriu Dam, the Buzău River discharge was altered with respect to the period before January 1984 [58,59].

The analyzed series, denoted in the following by S, consists of the monthly water discharge recorded from the beginning of 1955 to the end of 2010. The series was provided by the National Institute of Hydrology and Water Administration, where the raw datasets were meticulously preprocessed to remove any measurement errors, ensuring a level of data accuracy.

Generally, various techniques can be used to remove inconsistencies and/or incoherencies in datasets. Homogenization is such a solution when a significant number of series recorded in the same region on various catchments are available [60,61,62,63]. When a singular data series is available, removing inconsistencies from the data series can be achieved by singular spectrum analysis [64,65,66], which is based on the reconstruction of a data series after a specific type of decomposition. Another possible approach involves other kinds of decompositions, such as empirical mode decomposition (EMD) and ensemble empirical mode decomposition (EEMD) [67,68,69,70]. A deeper investigation should be conducted in a future study.

The data series was complete. In an opposite situation, different approaches to missing data are possible: (1) to let the algorithm run (likely because it works with labeled values, in contrast with the classical methods for time-series modeling) and (2) yo replace the absent values with the series mean or median, values resulting from linear interpolation, or generated by AI algorithms called autoencoders.

Two subseries of S were also studied: S1—before January 1984 and S2—after December 1983. Figure 2 depicts the series and the values of some statistics. Min (m³/s) is the minimum, Max (m³/s) is the maximum, and CV (%) is the coefficient of variance. Skewness and Kurtosis are dimensionless. Excepting Min, all the basic statistics corresponding to S₂ are the lowest, indicating an attenuation of the river flow after 1984.

2.2. Methodology

To better understand the characteristics of the series S, apart from the basic statistics (displayed in Figure 2), the series was tested for stationarity against nonstationarity with the KPSS test [71], homoscedasticity vs. heteroskedasticity by the Fligner–Killeen test [72], and change point existence against nonexistence by the Pettitt test [73].

Anomalies (outliers) are records that significantly differ from the general behavior of a time series and cannot be considered measurement errors or noise [74]. Identifying anomalies in a time-series dataset is crucial because they may impact forecast model accuracy. Even a small number of outliers can considerably diminish the reliability and precision of the forecasts.

The procedures developed for detecting anomalies in the temporal domain take into account the local or global aspect. In the first case, the vicinities of each record are considered, whereas in the second one, the entire series is analyzed [75]. In this research, we utilized a G-H-ESD procedure for detecting multiple outliers, implemented in the R package AnomalyDetection [76], based on the generalized extreme studentized deviate proposed by Roesner [77]. The algorithm’s input is formed by the time series, its length (n), and the maximum number of anomalies, k ≤ 0.49 × n.

The stages of the algorithm are as follows [78].

(a)

Determine the seasonal component (S_t) using the seasonal and trend decomposition using LOESS (STL) [79,80]. STL consists of two loops: the inner one deriving the trend (T_t), S_t, and residual (R_t), and the outer one responsible for the algorithm’s robustness with respect to anomalies.

(b)

Compute the median of the given series.

(c)

Compute the residual by subtracting S_t and the median from the data series.

(d)

Detect the anomalies using ESD as follows.

• Compute

  $$C_j=\underset{1\le j\le n}{\max }\left|x_j-\overline{x}\right|/s,$$

  (1)

  for the extreme value $x_j$ detected, where $\overline{x}$ = average and s = standard deviation.
• Compare C_j with the critical value:

  $${\lambda }_j={\displaystyle \frac{(n-j)t_{p,n-j-1}}{\sqrt{(n-j-1+t_{p,n-j-1}^2)(n-j+1)}}},$$

  (2)

  where $t_{p,n-j-1}$ is the value of the Student statistics at the significance level p and $n-j-1$ degrees of freedom.
• If x_j is an anomaly, discard it and compute the critical values using the new data series.
• Repeat the previous steps j times, considering that the number of anomalies is equal to the highest j for which C_j > λ_j.

(e)

List the anomalies and the corresponding timestamp.

The series S, S1, S2, and those obtained after the anomalies removal, denoted by S_a, S1_a, S2_a, are modeled using ESN and SSA-ESN.

When implementing AI methods, it is customary to partition the analyzed series into two disjoint parts, the first to train the model and the other to validate it. The series partition can significantly influence the prediction accuracy. Hence, carefully considering the proportion between these two sets is necessary. A higher proportion of data in the training set can help the model recognize patterns within it, while a lower proportion might lead to an incomplete evaluation of the model’s effectiveness.

The series were standardized before modeling. The training sets were before January 2006 for S, before January 1984 for S1, and between January 1984 and December 2005 for S2. For comparison reasons, the test set was kept unchanged, i.e., January 2006–December 2010. The same sets were kept for modeling S_a, S1_a, S2_a.

Remember that the choice of the random seed plays a significant role in ML algorithms, influencing the weight initialization and the selection of data involved in the algorithm’s deployment. To ensure that results can be reproduced, it is necessary to fix the seed. To evaluate the models’ performance, the algorithms were executed varying the seeds and then choosing the best results.

The models’ quality was assessed employing MAE, MSE, and R² for the training and test sets. Smaller MAE and MSE or bigger R² values indicate better models.

The technical specifications of the hardware and software used in the evaluation process are of utmost importance. The hardware comprised an AMD Ryzen 9 5900X 12-core processor CPU (3.70 GHz, 12 cores, 24 threads) with 64 GB of RAM, and an NVIDIA GeForce RTX 3090 GPU. Matlab R2023a was used to implement the models.

2.2.1. ESN

ESN [38] represents a version of a recurrent neural network [81], designed for tasks that require sequential data processing. Its structure, displayed in Figure 3, is optimized for this purpose.

ESN is formed by the input, hidden (reservoir), and output layers. Consider an ESN with k, n, and m input, reserve, and output units. The input, reservoir, and output matrices are U, X, and Y, respectively. The matrix of the connection weights between the first and second layers ($W_{in}\in {\mathbf{R}}^{n\times k})$ and that of the connections inside the second layer ($W\in {\mathbf{R}}^{n\times k}$) are randomly generated and remain unchanged. To achieve predictive output is sufficient to train the connection weights between the last two layers included in the matrix ($W_{out}\in {\mathbf{R}}^{m\times (k+n)}$) [82,83].

At stage t, the reservoir layer’s state requires an update, which is described by Equation (3):

$$X\left(t+1\right)=f(W_{in}·U\left(t+1\right)+W·X\left(t\right)+W_{back}·Y(t\left)\right)$$

(3)

where f is the reservoir layer neurons’ activation function (usually sigmoid or tanh). $W_{back}$ is the feedback matrix, which is null when the user does not require the output feedback.

The output state equation of the ESN model is:

$$Y\left(t+1\right)=W_{out}z\left(t+1\right),$$

(4)

where $z(t+1)$ is the concatenation of $X\left(t+1\right)$ and $U\left(t+1\right)$ [24].

Due to the RNN’s auto-feedback property, $X\left(t\right)$ reflects the previous input traces, so ESN has a short-term memory that dynamically changes.

The parameters utilized for running the following are as follows:

• Number of samples 30;
• Number of neurons in the reservoir 1000;
• Learning rate 0.1;
• Regularization parameter 0.1 [21].

2.2.2. SSA-ESN

SSA [84] is a novel metaheuristic algorithm based on the simulation of the sparrow flock foraging: sparrows that find better food act as “discoverers,” while others become “followers.” Additionally, the algorithm selects a certain proportion of sparrows for scouting and early warning. Upon detecting danger, they immediately abandon food to prioritize safety. The following matrix presents sparrow positions:

$$Y=\left[\begin{array}{cccc}{y}_{1,1}& y_{1,2}& \cdots & y_{1,d}\\ y_{2,1}& x_{2,2}& \cdots & y_{2,d}\\ ⋮& ⋮& ⋮& ⋮\\ y_{n,1}& y_{n,2}& \cdots & y_{n,d}\end{array}\right],$$

(5)

n being the volume of the population and d the dimension of the variable. The fitness function of the sparrow flock is given by the following:

$$F\left(Y\right)=\left[\begin{array}{c}f\left(\right[\begin{array}{cccc}{y}_{1,1}& y_{1,2}& \cdots & y_{1,d}\end{array}\left)\right]\\ f\left(\right[\begin{array}{cccc}{y}_{2,1}& y_{2,2}& \cdots & y_{2,d}\end{array}\left)\right]\\ ⋮\hspace{1em}\hspace{1em}⋮\hspace{1em}\hspace{1em}⋮\hspace{1em}\hspace{1em}⋮\\ f\left(\right[\begin{array}{cccc}{y}_{n,1}& y_{n,2}& \cdots & y_{n,d}\end{array}\left)\right]\end{array}\right],$$

(6)

where the fitness score of each sparrow is found in a row of the matrix F and represents the energy reserve.

Discoverers with higher energy reserves are prioritized for food acquisition during the search process. Discoverers typically have a broader search range than followers, comprising 10% to 20% of the population. The update rule (at the moment t + 1) for the positions of discoverers is as follows:

$$Y_{i,k}^{t+1}=\left\{\begin{array}{c}{Y}_{i,k}^t·exp\left({\displaystyle \frac{-i}{\alpha ·{iter}_{max}}}\right),\mathrm{}\mathrm{i}\mathrm{f} R_2<ST\\ Y_{i,k}^t+Q·L,\mathrm{}\mathrm{i}\mathrm{f} R_2\ge ST\end{array}\right.,$$

(7)

$$\alpha \in \left(\mathrm{0,1}\right), R_2\in \left[\mathrm{0,1}\right],ST\in \left[\mathrm{0.5,1}\right].$$

In (7), ${iter}_{max}$ = the highest number of iterations, $Y_{i,k}$ = information on the position of the i-th individual in the flock in the dimension k, and t = the iteration moment. The numbers $\alpha$ and Q are random, the latter being generated from a Gaussian distribution. $R_2$ and $ST$ are the warning and safety values, respectively, and L is a row vector with d elements equal to 1.

Equation (8) describes the process of updating the followers’ (joiners’) positions:

$$Y_{i,k}^{t+1}=\left\{\begin{array}{c} Q·\mathrm{e}\mathrm{x}\mathrm{p}\left({\displaystyle \frac{Y_{worst}^t-Y_{i,k}^t}{k^2}}\right), \mathrm{i}\mathrm{f} k>n/2\\ Y_P^t+\left|Y_{i,k}^t-Y_P^{t+1}\right|·A^{+}·L, \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(8)

where $Y_p$ is the actual best position of a discoverer, $Y_{worst}$ is the actual global worst location, “A” is a row matrix where each element is either 1 or −1, “⋅” indicates element-wise multiplication, and:

$$A^{+}={A^T\left({AA}^T\right)}^{-1}.$$

(9)

When danger is detected, the sparrow flock acts according to the following rules in the case of danger:

$$Y_{i,k}^{t+1}=\left\{\begin{array}{c}{Y}_{best}^t·\beta ·\left|Y_{i,k}^t-Y_{best}^{t+1}\right|, if f_i>f_g\\ Y_{i,k}^t+M·\left({\displaystyle \frac{Y_{i,k}^t-Y_{worst}^t}{f_i-f_g+\beta }}\right), if f_i=f_g\end{array}\right.$$

(10)

where $Y_{best}$ is the actual global best location, β controls the step size, and M ∈ [−1,1]. $f_i$ denotes the i-th individual current fitness value, $f_g$ and $f_w$ are the current global and worst best, and $\beta$ is a very small number added to prevent division by zero.

The SSA steps for optimizing ESN reservoir parameters are as follows.

(1)

Data preprocessing: Normalize the input time-series data to eliminate scale differences, enhance model convergence speed, and improve prediction accuracy.

(2)

Parameter initialization: Set the SSA’s key parameters—the size of the sparrow population, scouting and warning rate, flight distance (R₂)—and the ESN’s basic parameter ranges (such as reservoir size, initial state, spectral radius, input weight).

(3)

ESN parameter optimization: Based on SSA’s randomly generated positions, calculate the population fitness according to the update formula and obtain the current global optimum and individual best values.

(4)

Iteration termination: If the criteria for stopping the iterations are satisfied, the iteration is stopped and the optimal result is listed. Otherwise, the algorithm is performed again from the third step for further iteration.

(5)

ESN network prediction: Select the best individual from SSA as the optimization solution for reservoir parameters. Utilize these optimal parameters for ESN model prediction.

The parameters used to run the hybrid ESN-SSA are the following:

• Number of parameters to be optimized = 3—learning rate, reservoir size, regularization coefficient.
• Lower bounds for the parameters—0.1, 100, and 0.1, respectively.
• Lower bounds for the parameters—2000, 1500, and 0.2, respectively.
• Sparrow population—10.
• Maximum number of iterations—50.
• Initial size of the reservoir—30.

3. Results and Discussion

The KPSS test did not reject the hypothesis of level stationarity, but did reject that of trend stationarity. The Fligner–Killeen test for homogeneity of variances rejected the series’ homoscedasticity (p-value = 0.001385). The Pettitt breakpoint test indicates the existence of a breakpoint in the time series. Moreover, 29 aberrant values were found by the statistical tests.

After running S-H-ESD, 29 anomalies were determined. Figure 4a presents the series and the anomalies (blue dots) and lists the moment of their apparition. Eighteen of them were recorded before January 1984, and most were in March–May. The series without anomalies is presented in Figure 4b.

Figure 5 contains the charts of the S, S1, and S2 (in blue) and the predicted series obtained by ESN (in red) for the test period January 2006–December 2010. It shows that the most significant differences between the raw series and the forecast are registered for the maxima and minima of the test set, meaning that the extreme values in the test set are not very well estimated.

Comparisons of Figure 5a–c indicate that the best evaluation of the minima is provided for S2, especially for the periods May 2007–June 2007, November 2007–December 2007, and February 2010–September 2010. No significant difference is noticed for the maxima. All models follow exactly the data series shape.

The quality of the ESN output was assessed by computing the mean absolute error (MAE), mean standard error (MSE), and R². Their values are contained in

Table 1. Performance of the ESN models.

Series		Training Set			Test Set
	Run Time (s)	MAE	MSE	R2 (%)	MAE	MSE	R2 (%)
S	0.99927	6.69	80.58	99.76	5.00	42.71	99.69
S1	0.89032	7.60	102.95	98.91	5.56	48.74	99.16
S2	0.86929	5.74	57.35	99.48	4.48	36.61	99.52

. The run time was under 1 s, proportional to the series length: the shortest for S2 and the longest for S.

In terms of MAE and MSE, the best model was that for S2, followed by that for S, whereas with respect to R², the ESN model for S is followed by that for S2. The ESN model for S1 was the worst for both training and test sets. In all cases, the MSE (MAE) for the test set was at least 1.56 (1.26) times lower than for the training test, indicating that the algorithm applies well to the test set what it learned from the training set. The values of R² were comparable on both sets (test and training).

The goodness-of-fit indicators for the hybrid SSA-ESN models are presented in

Table 2. Performance of the SSA-ESN models.

Series		Training Set			Test Set
	Run Time (s)	MAE	MSE	R2 (%)	MAE	MSE	R2 (%)
S	0.96	6.69	80.58	99.75	5.00	42.72	99.68
S1	0.86	7.60	102.94	98.93	5.56	48.73	99.17
S2	0.79	5.74	57.34	99.48	4.48	36.60	99.53

. The run time is lower in all situations compared to the ESN. The other indicators are almost the same as in Table 1, suggesting that the initial algorithm (ESN) returned optimized results.

The explanation for the S2 models being the best is that the training and test set belong to the same period (after January 1984). Model S is built on a training set that contains elements before and after building the dam, while the test set belongs to the last period. Hence, its accuracy is lower than that of the model for S. The worst model is that for S1, whose training set is before building the dam, and its test set is after that date. This result indicates a modification in the water flow regime after January 1984.

Table 3. Performance of the ESN models for the series without aberrant values.

Series		Training Set			Test Set
	Run Time (s)	MAE	MSE	R2 (%)	MAE	MSE	R2 (%)
S_a	2.27	5.10	40.33	99.68	4.40	33.28	99.72
S1_a	1.06	6.06	54.87	94.69	4.87	37.95	96.34
S2_a	0.94	4.98	39.33	99.99	4.47	32.21	99.99

and

Table 4. Performance of the SSA-ESN models for the series without aberrant values.

Series		Training Set			Test Set
	Run Time (s)	MAE	MSE	R2 (%)	MAE	MSE	R2 (%)
S_a	155.59	4.86	44.49	91.91	4.42	37.41	91.32
S1_a	62.99	5.48	54.64	92.84	4.69	37.97	93.83
S2_a	110.31	4.47	39.73	90.95	4.16	35.46	90.59

present the models’ goodness-of-fit indicators, providing a clear comparison of their performance.

Figure 6, on the other hand, offers a visual representation of the test set and the corresponding forecasts. Comparison of the ESN and SSA_ESN on the series S_a, S1_a, S2_a leads to the following conclusions.

• The run time of SSA_ESN is much higher than that of ESN. Indeed, for the hybrid algorithm (Table 4), it is 155.59 s (62.99 s and 110.31 s) for S_a, (S1_a and S2_a), whereas for the single one it is 2.27 s (1.06 s, and 0.94 s) for the same sets (Table 3).
• Since MAE from the hybrid algorithm belongs to the interval [4.47, 4.86] for the test and [4.16, 4.42] for the training set, compared to the intervals [4.98, 5.1] and [4.40, 4.87], respectively, it results that SSA-ESN performs better in terms of MAE.
• With respect to MSE, ESN is the best on the training set for S_a and S2_a (40.33 and 39.33—Table 3, compared to 44.49 and 39.73—Table 4), and on the test set for S_a, S1_a, and S2_a (33.28, 37.95, and 32.21, respectively—Table 3, compared to 37.42, 37.97, and 35.46, respectively—Table 4).
• Considering R², ESN is the best on all series after discarding the aberrant values compared to SSA-ESN.

Overall, the following conclusions can be drawn.

• The lowest run time was that of ESN on S2.
• The lowest MAEs were recorded for SSA-ESN on S2, with values of 4.16 on the training set and 4.47 on the test set.
• The lowest MSEs were obtained by running ESN on S2_a: 39.33 on the training set and 32.21 on the test set.
• The highest R² (over 99.99%) corresponds to ESN on S2_a.
• The removal of aberrant values significantly enhanced the performance of the ESN algorithm, demonstrating its adaptability.

We have to point out that in this article, the test set was 60 months in all cases, so the forecast period was kept at 60 months for comparison reasons. The analysis can be conducted for various training:test ratio, so for different forecast periods (longer or shorter than 60 months). Obviously, the algorithms’ performance will be different. Such a study will be conducted in the future.

Modeling hydrological data is challenging due to the high variability of river flow, reflected in the values of the basic statistics, stationarity, change point existence, and heteroskedasticity, as in the present case study. Under these conditions, one cannot suppose that elementary methods will yield good results in capturing all these features. Still, we fitted ARIMA models [56] to compare their performances with those of AI models. The results of the best-fit ARIMA model will be presented in the following.

Another issue is that a wide range of articles should report the use of ESNs in modeling hydrological data to assess the method performances on this kind of series correctly. Our search returned its use only for rainfall-runoff (in this case, the input is the rainfall, and the output is the runoff, so we cannot make comparisons). for rainfall–runoff (in this case, the input is the rainfall and the output is the runoff, so we cannot make comparisons). Moreover, other studies on modeling the Buzău River water flow have not been undertaken. Given the study’s novelty from this viewpoint, there are only a few articles on this topic that can be used for comparison [55,56,57].

It is important to indicate that a direct comparison of the goodness-of-fit parameters in this study with those in other papers analyzing different data series is not valid. This is because the algorithms must be executed in the same scenarios, including the same data series and training and test sets for AI models. Therefore, it is necessary to indicate that all the AI and ARIMA models, presented in extenso in [56,57], and compared below adhered to these criteria.

Comparison of the run time of the algorithms on S, S1, and S2 is given in

Table 5. Comparisons of the run time on S, S1, and S2 [56,57].

	MLP	BPNN	ELM	ESN	LSTM	CNN-LSTM	PSO-ELM	SSA-BP	SSA-ESN
S	5.11	1.32	0.70	1.00	4.33	10.18	84.35	475.43	0.96
S1	3.87	1.23	0.75	0.89	3.57	6.35	57.37	399.83	0.86
S2	2.11	1.16	0.65	0.87	3.59	5.86	51.52	435.16	0.79

, and the goodness-of-fit indicators are contained in

Table 6. Comparisons of AI methods’ goodness-of fit indicators [56,57] on S, S1, and S2.

Method	Series	Training Set			Test Set
		MAE	MSE	R2 (%)	MAE	MSE	R2 (%)
	S	12.04	221.17	34.13	10.90	234.79	26.80
ARIMA	S1	13.92	239.56	45.23	11.49	260.64	25.01
	S2	10.22	192.18	34.18	12.36	299.09	48.82
	S	10.16	206.05	35.76	9.75	146.23	9.69
MLP	S1	11.41	252.19	27.44	8.85	135.73	16.18
	S2	9.26	181.70	36.93	10.10	158.14	2.33
ESN	S	6.69	80.58	99.76	5.00	42.72	99.69
	S1	7.60	102.95	98.91	5.56	48.74	99.16
	S2	5.74	57.35	99.48	4.48	36.61	99.52
ELM	S	6.01	98.12	83.05	4.60	41.29	88.70
	S1	6.79	126.33	76.14	5.21	54.54	81.84
	S2	5.03	78.63	79.71	4.01	32.21	89.71
LSTM	S	6.79	87.69	99.39	4.92	41.48	99.83
	S1	10.51	213.22	98.99	7.64	98.74	99.74
	S2	5.72	60.07	99.92	4.49	35.65	99.97
BPNN	S	6.96	152.44	52.89	5.52	125.06	31.07
	S1	11.00	326.62	18.30	7.94	116.36	40.80
	S2	8.14	145.38	50.21	8.29	158.55	42.17
SSA-ESN	S	6.69	80.58	99.75	5.00	42.72	99.68
	S1	7.60	102.94	98.93	5.56	48.73	99.17
	S2	5.74	57.34	99.48	4.48	36.60	99.53
SSA-BP	S	5.73	91.26	83.97	4.29	32.50	92.97
	S1	7.00	105.40	92.76	5.20	44.62	96.12
	S2	7.71	132.45	53.11	8.09	168.60	19.76
CNN-LSTM	S	6.03	93.81	89.45	4.24	36.00	94.58
	S1	6.52	115.09	88.39	4.48	39.98	94.26
	S2	4.74	62.00	93.01	3.52	29.83	95.04
PSO-ELM	S	6.01	98.13	83.05	4.60	41.28	88.68
	S1	6.78	126.55	75.96	5.13	52.18	83.35
	S2	5.04	70.70	79.66	3.99	30.968	89.94

.

The time taken to run SSA-ESN (and ESN) was the second (third) smallest, after that for ELM. The time to run MLP was between 2.11 s (for S2) and 5.11 s (for S).

Among the hybrid methods involving SSA, SSA-BP (SSA-ESN) performed best on the training sets of S and S1 (S2) in terms of MAE (MSE). With respect to MAE and MSE, SSA-BP outperformed SSA-ESN on the test sets of S and S1. When considering all goodness-of-fit indicators, SSA-ESN was the best on S2. Among all hybrid methods, CNN-LSM obtained remarkable results on the test set in terms of MAE and MSE. Taking R² as the main criterion, BPNN was the worst model, followed by ELM. However, ELM best fit the S1 and S2 test sets with respect to MAE and S2 in terms of MAE and MSE. ESN and SSA-ESN were among the best models considering all criteria together. MLP and ARIMA gave the worst results among all methods.

The study’s main limitations are the following.

(1) The models do not capture the maxima.

According to [85], a complex method could be applied to address the first issue. The extreme events are memorized using a memory network, and the tail values are modeled by extreme value loss (EVL) [86]. Lastly, the results are combined to predict the raw data series.

(2) A significant increase in the time necessary to run the hybrid algorithm can be seen in Table 4 compared to ESN (Table 3). The same behavior was noticed when running SSA-ESN on daily data series (experiments will be presented in detail in another article): the ESN run time was 82.84 s for S, 6.15 s for S1, and 4.99 s for S2, and the SSA-ESN run time was 6505.60 s for S, 3805.99 for S1, and 3194.93 s for S2. Although SSA-ESN showed better performance in terms of MSE and R², the increase in run time may not justify the marginal improvements in accuracy unless the task requires high precision.

This issue should be addressed by optimizing the parameters in the SSA algorithm.

Two other important topics would be of interest:

(a) Data preprocessing to remove inaccuracies (when data do not come from sources that have already performed the preprocessing, as in our case). This can be achieved using techniques involving decomposing and recomposing the data series [64,65,66,67].

(b) Replacement of the missing values (which was not the case here) by linear interpolation, average or median values of the recorded series, or use of an autoencoders.

4. Conclusions

This was a comprehensive investigation into the capability of two AI algorithms (ESN and SSA-ESN) to provide reliable models for the monthly river discharge of the Buzău River in Romania over 55 years. The results fill a gap in modeling hydro-meteorological data series using ESN [86,87]. More research should be done to confirm the suitability of ESN and SSA-ESN in this research field.

It has been proved that both models are comparable with respect to the goodness-of-fit indicators. The MAE values for both ESN and SSA-ESN were between 5.74 (for S2) and 7.60 (for S1) on the training set. On the test set, MAEs were in the range 4.48 (for S2) to 5.56 (for S1) on the test set. All R² values were above 98.91%. MSE belongs to the interval [57.34, 102.94] on the training sets and to [36.60, 48.73] on the test sets, revealing that the algorithm performs better on the latter.

The removal of aberrant values considerably increases the algorithms’ run time from under 1 s to over 1 min (between 62.99 s and 155.59 s). The MSE values notably decreased on the training sets of all models, from the interval [57.34, 102.96] to the interval [39.73, 44.49]. Since the models did not fit the maxima well, future research will focus on better fitting for extremes and adapting the algorithm to reduce the run time.