3.1. Machine Learning and Deep Learning
To build an SAEF model, we selected four representative base learners (SVM, LSSVM, LSTM, BiLSTM) according to the following criteria: (1) representativeness—covering kernel-based and least-squares kernel methods (suitable for trend and low-to-moderate nonlinearity) and gated recurrent neural networks (suitable for strong nonlinearity and temporal dependence); (2) complementarity—the models are expected to complement each other in handling smooth trends, peak responses, and seasonal transitions; and (3) reproducibility and computational feasibility—priority was given to models that train stably under our data size and experimental setup. Brief model-specific rationales are provided in the following subsections. In the data preprocessing stage, we converted continuous daily runoff data into input-output pairs required for supervised learning based on the idea of a sliding time window [
43]. Specifically, we selected a sequence of length
days as the input feature to forecast the runoff value of the first day in the future, thereby constructing a correspondence between the input sequence
and the output target
. This study determined the optimal input structure of machine learning and deep learning models through trial and error:
Step 1: Construct ML and DL models for different input lengths to forecast the future 1-day runoff of each watershed.
Step 2: Use Kling–Gupta Efficiency (KGE) as a model performance evaluation index to comprehensively measure the forecasting effects of different input lengths and model structures in each river basin. KGE comprehensively considers correlation, mean deviation, and variance ratio, and can reflect the forecasting ability of the model more comprehensively than a single error index.
Step 3: Based on the performance of the KGE index, select the optimal input length and model combination and apply it to subsequent integrated forecasting.
This study selected four ML and DL models that are relatively mature and representative in the field of hydrological forecasting, namely SVM, LSSVM, LSTM, and BiLSTM. These models have shown good performance in capturing the nonlinear characteristics and temporal dynamics of hydrological processes.
To ensure fair comparison and maintain controlled experimental conditions across different machine learning models, this study adopts a consistent hyperparameter strategy for SVM, LSSVM, LSTM, and BiLSTM. It is important to note that the primary objective of this research is not to optimize individual model configurations, but rather to develop and evaluate the proposed SAEF framework. For SVM and LSSVM, key hyperparameters such as the penalty parameter , insensitive loss , and the regularization and kernel parameters in LSSVM were manually adjusted through multiple preliminary trials to obtain stable and reasonably representative performance for runoff prediction. Similarly, the architectural hyperparameters of LSTM and BiLSTM—particularly the number of hidden units—were also fixed across all scenarios. By using consistent hyperparameters once reasonable configurations were selected, we ensure that the observed performance differences primarily reflect the intrinsic properties of the models rather than discrepancies arising from heterogeneous tuning strategies. This controlled experimental design strengthens the fairness, interpretability, and scientific rigor of the comparative analyses and the subsequent ensemble weight optimization. In addition, all hydrological inputs were normalized to the min–max scaling method to ensure stable model training and eliminate the influence of differing variable magnitudes.
3.1.1. Support Vector Machines
SVM was originally designed for binary classification tasks. By minimizing estimation error and achieving linear separation of inputs in a mapped high-dimensional feature space, SVM exhibits core advantages of low overfitting risk and excellent generalization ability, while supporting both linear and nonlinear classification as well as regression tasks.
Considering the rationality of training samples and data complexity comprehensively, and based on the principle of risk minimization, given a dataset
(where
denotes input variables,
corresponds to the respective output variables, and
represents the number of variable dimensions), the decision function of the SVM model can be expressed as:
In this formula, all samples corresponding to non-zero coefficients are support vectors; denotes Lagrange multipliers; is a parameter value solved via a constrained optimization problem involving the insensitive coefficient ; is a threshold determined based on training samples; is a kernel function satisfying the Mercer condition (in the form of ); and represents a penalty factor that balances model complexity and fitting error.
Linear kernel, polynomial kernel, and radial basis function (RBF) are the three most widely used core kernel functions in SVM models. The linear kernel is suitable for linearly separable data, but monthly runoff time series exhibit significant complexity, making it inapplicable. The polynomial kernel suffers from numerous parameters and excessively high computational complexity; thus, it is generally not adopted. As a local kernel function, the RBF possesses extremely strong nonlinear mapping capability and demonstrates excellent performance in most sample scenarios, serving as the preferred choice when the appropriate kernel function is unknown. Consequently, this study employs the Gaussian radial basis kernel function, whose mathematical expression is given as follows:
In the formula, denotes the kernel function parameter with .
The parameters of the SVM model were manually tuned through several preliminary experiments, and the final selected configuration was kept fixed in all experiments to maintain comparability with LSSVM under the same controlled settings.
3.1.2. Long Short-Term Memory
As an improved derivative of Recurrent Neural Networks (RNN), LSTM models share the same three-layer basic structure (input layer, hidden layer, output layer). Their core advantage lies in the integration of self-recurrent units, which can effectively memorize long-term sequential information and alleviate the problems of gradient vanishing and gradient explosion. The functionality of these self-recurrent units is jointly regulated by the forget gate (), input gate (), and output gate ().
The core function of the forget gate (
) is to select the information to be discarded in the current unit during training. It receives the output
of the previous unit and the input
of the current unit, then nonlinearly maps them to an output vector
with values in the range [0, 1] via the Sigmoid function, which acts on the unit state
at the previous time step. Specifically, a value of 1 indicates “complete retention” of information, while 0 denotes “complete discard”. Its mathematical expression is given as follows:
where
denotes the output vector of the previous neuron;
is the input vector of the current neuron;
represents the sigmoid activation function;
and
are the weight matrix and bias vector of the network, respectively, which regulate the signal transmission intensity and baseline offset.
The input gate (
) is responsible for dynamically regulating the state update of the current unit. Firstly, it processes the input information via the Sigmoid function to select key features requiring update; subsequently, a candidate update vector
with values in the range (−1, 1) is generated through the tanh activation layer; finally, the state of the current unit is updated by fusing the outputs of the two aforementioned parts. Its mathematical expression is given as follows:
where
denotes the weight matrix of the candidate cell state;
is the bias vector of the candidate cell state;
represents the weight matrix of the input gate;
is the bias vector of the input layer;
is the potential update vector of the cell state, which stores new feature information to be integrated.
After the data is processed by the forget gate and input gate, the unit state at the previous time step
is updated to the current time step state
. During the update process, the previous state
is first multiplied by the forget gate output
to filter and retain information, followed by fusing the candidate update vector generated by the input gate
at time
to obtain the current unit state
. The specific equation is given as follows:
The output gate (
) generates the output
based on the current unit state
. Its working mechanism is as follows: first, the Sigmoid function is used to select the feature information to be output from the current unit state; subsequently, the unit state is normalized by the tanh function is multiplied by the output of the Sigmoid function, and finally, the output
at the current time step is obtained. The specific equation is given as follows:
where
denotes the weight matrix of the output gate;
is the bias vector of the output gate;
represents the output vector of the unit at the current time step, which transmits effective feature information to the next unit.
Although LSTM models typically benefit from larger datasets, the daily runoff series used in this study (1826 samples) provides a sufficient time span for training medium-sized recurrent architectures commonly applied in hydrological forecasting. In addition, all hydrological inputs were normalized using the min–max scaling method to stabilize gradient propagation during training. To ensure methodological fairness, the number of hidden units and the overall architectural configuration of the LSTM model were fixed across all basins and experiments. Furthermore, the 1826-day daily runoff record provides sufficient temporal continuity for both recurrent models to extract seasonal hydrological patterns, ensuring that their performance comparison is not undermined by data sparsity. This controlled design guarantees that the comparison with BiLSTM and kernel-based models reflects intrinsic model behavior rather than differences in hyperparameter tuning.
3.1.3. Least Squares Support Vector Machine
LSSVM is a significant improved variant of the conventional SVM. As an extended form of SVM, LSSVM not only has a sound theoretical framework but also optimizes the solution mechanism by converting the inequality constraints of SVM into equality constraints. Specifically, it replaces the complex quadratic programming problem in SVM with solving linear equations through constructing a loss function. Compared with the original SVM, LSSVM reduces computational complexity, significantly enhances training efficiency and prediction accuracy, and ensures reliable global optimality. Currently, this model has been widely applied in various fields, including time series forecasting. The core parameters of LSSVM are the bandwidth of the squared kernel function and the regularization parameter. The reasonable configuration of these two parameters is crucial to the generalization performance of the LSSVM model, and their values need to be determined in close combination with specific application scenarios and the characteristics of training samples. The detailed calculation process and mathematical expressions of the LSSVM model are as follows:
(1) Assume training sample data are given (where ,), with as the input variable and as the output variable. The modeling form of the LSSVM is expressed as:
In the formula, is the weight vector; is the bias term; is the kernel space mapping function.
(2) Based on the principle of structural risk minimization, the evaluation problem of the LSSVM model can be transformed into an optimization problem:
In the formula, is the regularization parameter (with ); is the slack variable.
Construct the Lagrangian function, and solve the aforementioned optimization problem via the method of Lagrange multipliers:
In the formula, denotes the Lagrangian function.
(3) When
,
and
can be eliminated. By incorporating the kernel function
that satisfies the Mercer condition, the solution process of the optimization problem can be simplified to a linear equation:
The RBF is selected as the kernel function of the LSSVM model, and in this case:
where
denotes the kernel width;
represents the squared kernel width.
(4) Finally, the solved LSSVM model is expressed as:
Similar to SVM, the regularization parameter and kernel parameter of LSSVM were manually examined through multiple trials, and the resulting configuration was fixed throughout the study. This ensures that SVM and LSSVM operate under consistent and comparable hyperparameter conditions.
3.1.4. Bidirectional Long Short-Term Memory
The BiLSTM neural network is an advanced variant of the traditional Bidirectional Recurrent Neural Network (BRNN), which replaces conventional RNN units with LSTM units. Composed of two LSTM components (forward and backward), BiLSTM can effectively capture comprehensive feature representations by integrating historical and future information of the sequence. The hidden layer of the model consists of two parts: the forward LSTM unit state and the backward LSTM unit state. After historical sequences are transmitted from the input layer to the hidden layer, forward and backward computations are performed, respectively. By learning the past and future features of the sequence, BiLSTM ultimately generates the output results. The BiLSTM model was implemented under the same data volume, input normalization approach, and hyperparameter configuration as the LSTM model, ensuring that both models operate under fully comparable conditions. By keeping the architectural settings fixed throughout all experiments, any observed performance differences between LSTM and BiLSTM can be attributed solely to their directional processing mechanisms rather than variations in model configuration. This controlled setup enables a scientifically valid and meaningful comparison of the two recurrent architectures on the medium-sized hydrological time series used in this study.
3.2. Arctic Puffin Optimization
3.2.1. Standard APO Algorithm
APO is a newly proposed swarm intelligence optimization algorithm inspired by the efficient hunting strategies exhibited by Arctic terns during flight and underwater foraging. The algorithm simulates the behavioral characteristics of terns in different environments and is designed to include multiple stages, such as population initialization, aerial exploration, underwater development, and behavioral conversion, with the aim of achieving a balance between global search and local development and improving the ability to solve complex optimization problems.
- (1)
Population initialization
In the initial stage of the APO algorithm, the distribution of the
Arctic puffin population is abstracted as a set of candidate solutions. Each “puffin” represents a solution vector, whose position is initialized as follows:
In this context, denotes the position vector of the th individual at the initial time, and denote the lower and upper bounds of each dimension in the search space, respectively, and is a random vector uniformly distributed over the interval .
- (2)
Flight phase
The aerial flight phase simulates the behavior of puffins searching for prey in the air and diving quickly to catch it, and is mainly used for global exploration. First, the Levy flight mechanism is used to perform long-distance random jumps from the current position, with the following update formula:
Among them, represents the updated position of the th individual after the th iteration, is the position of an individual randomly selected from the current population and different from , is the Levy flight step length generated according to the problem dimension , and is a random perturbation term that follows a standard normal distribution.
After completing its aerial search, the puffin quickly dives to catch its prey. This process is controlled by introducing a speed factor to scale the position. The updated formula is as follows:
where
is a random number in the
interval. This mechanism helps to break out of local optima and enhance global search capabilities by dynamically adjusting the movement amplitude of individuals.
- (3)
Underwater foraging stage
During the underwater foraging stage, puffins flexibly adjust their search strategies based on the distribution of food resources in the environment, with the aim of strengthening their local development capabilities. First, puffins use a cooperative encirclement strategy, with multiple individuals surrounding schools of fish to conduct group searches. The update formula is as follows:
where
is the collaboration coefficient used to adjust the intensity of group cooperative search,
is a random number in the
interval, and
,
, and
are different individual positions randomly selected from the current population.
If there is little food in the current search area, the puffin will enter enhanced search mode, expanding the search range through adaptive strides. The updated formula is:
where
is the maximum iteration count,
is the current iteration count, and
is a factor that converges dynamically during the iteration process. It is used to control the step size and prevent premature convergence.
In addition, when predators are detected, puffins quickly avoid danger by flying away from the dangerous area. The updated formula is:
where
is a random factor within the
interval, used to adjust the intensity of avoidance behavior.
- (4)
Behavior switching and population update
In order to achieve a balance between exploration and exploitation, APO integrates new solutions generated at each stage through a behavior switching mechanism, selects the optimal individuals based on fitness values, and updates the next generation population. Its update strategy is as follows:
Finally, select the
individuals with the highest fitness as the new generation population:
3.2.2. Improved Arctic Puffin Optimization
In meta-heuristic optimization algorithms, achieving an efficient balance between global exploration and local exploitation is a key factor in ensuring the algorithm’s excellent performance. Although APO demonstrates certain advantages in both global and local search, standard APO still has limitations. To address the limitations of standard APO, this paper proposes an IAPO Algorithm. IAPO uses two strategies to enhance its ability to overcome these limitations:
- (1)
Elite Opposition-Based Learning Method
The distribution characteristics of the initial population in the search space have a significant impact on the search efficiency and solution accuracy of intelligent optimization algorithms. However, from the execution process of the APO algorithm, the standard APO relies on a random method to generate the initial population, which to some extent suffers from uneven distribution and insufficient exploration, making the algorithm prone to getting stuck in local optima. To solve this limitation, the IAPO algorithm introduces an improved strategy of Elite Opposition-Based Learning to generate the initial population of
Arctic puffins. This method uses elite information to guide the population to construct directional reverse samples and selects the optimal individuals through a competition mechanism, thereby improving the distribution quality of the initial population in the solution space [
44].
Specifically, assume that the optimization variable is
dimensional, and initially generate
individuals, denoted as:
For each dimension
, define the minimum and maximum boundaries of that dimension in the current population as follows:
The corresponding elite reverse position can be calculated as:
where
represents the reverse value of the
th individual in the
th dimension. To prevent reverse position out-of-bounds, when the calculation result exceeds the boundary range, a random perturbation repair mechanism is introduced:
Subsequently, the original individual set is merged with its reverse sample to construct a candidate set containing individuals. These individuals are then sorted according to their fitness values, and the top individuals with the best performance are selected as the final initial population.
- (2)
Enhanced behavior conversion factor
The design of the behavior conversion factor largely determines the overall performance of the APO algorithm. In the original APO algorithm, the factor
is defined as:
where
is a random number between 0 and 1.
In the APO algorithm, based on the behavior conversion factor and the threshold parameter (where the original APO algorithm sets ), the algorithm can dynamically switch search strategies during iteration: when the algorithm tends to perform global exploration; when , it switches to local development mode.
Although the original behavior conversion factor
achieves a certain degree of balance between global exploration and local exploitation in the APO algorithm, its flexibility remains insufficient when dealing with complex optimization problems, manifesting as the global search phase potentially ending too early, leading the algorithm to become trapped in a local optimum. To overcome this limitation, this paper employs an improved behavioral transition factor
. This factor combines the nonlinear characteristics of the cosine function with the dynamic changes in the objective function’s fitness, enabling adaptive adjustment of the search strategy. This approach more effectively balances global exploration and local exploitation, thereby enhancing the algorithm’s performance in complex problems. The enhanced behavioral transition factor is defined as:
The improved conversion factor is more suited to the search requirements of complex optimization problems by introducing a nonlinear decay mechanism. This factor utilizes the smoothing characteristics of the cosine function to achieve a smooth transition between global exploration and local development, reducing the violent fluctuations caused by strategy switching during the search process. At the same time, it retains the random perturbation component to enhance the algorithm’s ability to escape from local optima, thereby improving overall optimization performance.